Accord and Toolbox

Pro-Res

PROmoting integrity in the use of RESearch results in evidence based policy: a focus on non-medical research)

PREAMBLE

We present here a draft statement of principles that lie behind seeking/using ethical evidence from non-medical research to inform policy. In talking about ethical evidence, we are both tackling the principle of evidence per se and the way this evidence is built throughout the whole research process from inception through to application or use. The short, clear, succinct and actionable statement we present here is designated the ‘Accord’. This is the baseline that we intend the further consultation process to be built on. Neither its title nor content is ‘fixed’ at this point. We aim to explore its potential with the appropriate constituencies and across the range of stakeholders. These include the producers of research, disseminators and intermediaries, influencers, policy advisers, decision-makers and implementers. The section following the Accord statement draws out the elements of the brief Accord statement in terms of slightly more detailed principles together with a rationale for this approach. The draft Accord is based on the work accomplished by the first phase of the PRO-RES Project and based on declared foundational assumptions about the values, principles and standards involved in ethical research conducted with integrity. The Accord will be presented on the PRO-RES website and linked to a ‘Toolbox’ to aid stakeholders in assessing the ethics and integrity of research evidence and supportive resources to help produce such evidence across the range of non-medical research activities.

We are aiming to develop a culture of ethical research based on continuous discursive engagement. By that we mean:

  • There needs to be an ethical discourse to be sure that researchers are aware of, and sensitive to, the ethical dimensions of their work. That awareness depends on engagement in ethical discourse as an integral aspect of engagement in research.
  • To bring about a cultural change in research activity, there has to be engagement of everyone responsible for the process, including researchers, stakeholders, peers and the users of research.
  • This engagement needs to be continuous. Ethical issues can arise at every stage of research: conception, development, proposal, process, conclusion. Dissemination and use. Ethical consideration cannot be a single-stage process.

The Accord

(on ethical evidence in non-medical research)

  • We commit to only use research that is undertaken ethically.
  • We recognise that an underpinning by high quality research and evidence, including policy appraisals and evaluations, is a pre-condition for evidence-based policy-/decision-making, and hence rational policy actions and outcomes.
  • We will seek to employ high quality evidence that has been gathered, collated and analysed using sound, robust and ethical methods.
  • We will attempt to ensure that the funding, management, conduct, dissemination and governance of research meets high standards of ethics and integrity.
  • As individuals and institutions involved in collecting and/or using evidence in policymaking, we aim to be transparent on how the high quality of that evidence is assured and will flag up any potential conflicts of interest.
  • We agree that the independence and integrity of individuals responsible for the gathering of research evidence and its use in policymaking must be respected and supported in ways that ensure the evidence they produce is neither biased nor misleading.

The following points explain the rationale behind the Accord and supply links to supportive resources that will help in seeking to promote ethics and integrity in the evidence produced in all non-medical research:

  • Under a commitment to evidence-based policy, all evidence should be based as far as possible on ethically sound research
  • There are many forms of research. They include not just formal research projects and programmes, but a range of actions relating to investigation, discovery, exploration, practice, and disciplinary development. Every kind of research needs to be done ethically.
  • Research should be beneficent (or at least non-maleficent) in its aims, its substantive focus, in the process of research, and its application.
  • Ethical issues can arise at every stage of research: conception, development, proposal, process, conclusion and dissemination. It follows that ethical consideration cannot be a single-stage process; it has to be continuous.
  • Researchers have to be aware of, and sensitive to, the ethical dimensions of their work. That awareness depends on engagement in ethical discourse as an integral aspect of engagement in research. Ethical conduct cannot adequately be guaranteed by a fixed number of pre-set rules.
  • All researchers should aim to develop a culture of ethical research, based on continuous discursive engagement. To achieve this, there has to be engagement of everyone responsible for the process, including researchers, stakeholders, peers and the users of research.
  • Research should not be based on pre-formed prejudicial ideologies or biased political or financial interests.
  • Conflicts of interest should ideally be avoided in the production of research evidence. If this is not possible, all conflicts of interest should be openly disclosed.
  • All sources of information used to formulate research evidence should be acknowledged.
  • In order to produce high quality evidence, research must be methodologically robust.
  • Only research that has also been conducted ethically and with integrity can be considered ‘high quality’.
  • All research should be funded, managed, conducted and disseminated ethically and with integrity.
  • The processes and institutions involved in theselection of evidence, including research,to inform policy should be independent, open and transparent.
  • The effectiveness and impact of all policies should be honestly and transparently assessed or evaluated using high quality research methods.

To achieve these ends:

  • The Accord is supported by foundational statements that clarify the values, virtues, principles and standards that are applicable to research and the production of evidence used in policymaking. Please check out the Foundational Statements
  • Clear and agreed definitions of terms and concepts are required so that all policymakers should be able to recognise, identify and distinguish the characteristics of high-quality evidence in their field. Please check out the Glossary of Terms and Concepts
  • Ethical research practice can often only be understood and explained in context. Illustrative case studies are made available – with both ethically positive and negative elements–not just success stories. So that users can be aided in their ethical decision making with the insights offered by complex cases. Please check out the Case Examples
  • A repository of resources must be made available to guide and support the interpretation and application of the Accord. Please check out the Framework/Resources

To help implement the Accord and the principles behind it users need to know:

How to conduct research ethically and with integrity (for researchers, managers and funders).

How to ensure research is conducted ethically and with integrity(for reviewers in research ethics appraisal).

How to supply evidence for effective policymaking (for researchers, managers, funders).

How to select good quality research(for science/policy advisors and policymakers).

How to evaluate the ethical impact of policies(for ALL above stakeholders).

A TOOLBOX FOR ASSESSING THE ETHICAL QUALITY OF RESEARCH EVIDENCE

WHO were the researchers and the research agency?

For the individual researcher:

  • What are the credentials of the researcher?
  • What is/was their competence; experience; track record?
  • Who do/did they work for?
  • Do they have any vested/conflicts of interest?
  • Do they adhere to any specific professional/ethical codes and/or guidelines?
  • How was the specific project that generated the evidence in question funded?

For the research agency:

  • What are the credentials of the research agency?
  • What is their competence; experience; track record?
  • Who do/did they work for?
  • What kind of research/data-gathering agency are they?
  • How is the agency funded/by whom?
  • How is the agency governed – how was it founded and with what purpose?
  • Does the agency commit to adhere to certain codes/guidelines – does it have a ‘mission statement’?
  • Does the agency have any vested/conflicts of interests?
  • How does it manage data protection regulations?
  • How was the specific project that generated the evidence in question funded?

We present here a draft statement of principles that lie behind seeking/using ethical evidence from non-medical research to inform policy. In talking about ethical evidence, we are both tackling the principle of evidence per se and the way this evidence is built throughout the whole research process from inception through to application or use. The short, clear, succinct and actionable statement we present here is designated the ‘Accord’. This is the baseline that we intend the further consultation process to be built on. Neither its title nor content is ‘fixed’ at this point. We aim to explore its potential with the appropriate constituencies and across the range of stakeholders. These include the producers of research, disseminators and intermediaries, influencers, policy advisers, decision-makers and implementers. The section following the Accord statement draws out the elements of the brief Accord statement in terms of slightly more detailed principles together with a rationale for this approach. The draft Accord is based on the work accomplished by the first phase of the PRO-RES Project and based on declared foundational assumptions about the values, principles and standards involved in ethical research conducted with integrity. The Accord will be presented on the PRO-RES website and linked to a ‘Toolbox’ to aid stakeholders in assessing the ethics and integrity of research evidence and supportive resources to help produce such evidence across the range of non-medical research activities.

HOW was the research/data-gathering and analysis conducted?

  • What exactly was done to gather and analyse the data?
  • What research plan or analytical ‘design’ was used?
  • What specific methods were employed both to gather data and to analyse it?
  • Was there an original protocol made available publicly? (If so, did the research deviate from this? If so, was this justified?)
  • What kinds of data were gathered? (Were there checks for validity, reliability, authenticity of sourcesetc.)
  • How were data managed and analysed?
  • Is there any evidence of bias? If so, where and what?
  • Were other stakeholders (community members, research participants, general public, etc.) involved in any part of the research or data-gathering? If so, why and how?
  • Were relevant personal identities protected and, if so, how?
  • Was the process transparent? If not, why were there limitations on transparency?

There is no implicit judgment of the ‘ethical quality’ of the variety of methods that can be employed. What matters is, again, the transparency of those conducting the research, and their offering of clear justifications/rationale for any methods used. Thus covert research, deception, community/societal engagement, social engineering etc. are not to be regarded as inherently unethical – the judgement of whether they are or not might depend upon the context in which they are used and, whether a policymaker/advisor considers evidence derived from a particular method is justifiable. Neither is there any implication that only primary research is of evidential value – all forms of secondary data analysis can be subjected to these questions: from meta-analyses of controlled experimental studies to simple frequency counts of questionnaire responses. The ‘validity’ of primary research data depends upon the rigour of the research design and its accurate execution; the validity of most forms of secondary data analysis depends upon access to/availability of raw source data. Even documentary or archival analyses are valid to be tested against accurate use of source materials.

WHOM/WHAT was the prime focus of the study?

  • Who or what were the subject/objects/participants of the study?
  • Could these ‘subjects’ or ‘objects’ have been considered vulnerable in any way – or made more vulnerable by the enquiries being conducted?
  • How was the welfare of the subject/objects/participants ensured?
  • How was the welfare of the researcher(s) (if appropriate) ensured?
  • Can any risks of harm be foreseen/anticipated and mitigated as a consequence of engaging in enquiries/research about/with the ‘objects’ of this study?

These elements concern the relationship between researcher and researched and how the researcher treats the researched. The subjects/objects/participants could have been humans, animals, organisms or parts of such, material objects, ecosystems, organisations, communities, societies etc. – or any combination of the aforementioned. Thus research by economists might be a study of banking ‘systems’ without references to bankers per se. Research enquiries related to public health might be concerned with the public and not individual members of that ‘public’. Researcher welfare issues are likely to arise out of their relationships with the subjects/objects of study – so researcher health and
welfare needs to be considered and any forms of reflective practice they adopt encouraged and disclosed. Once more these questions are not just related to primary research, nor simply to research with humans or live animals – they apply equally to any form of secondary research/data gathering and to material objects or places. Thus, for example, a volcanologist is unlikely to be able to cause undue
harm to the objects of their study, but is likely to put themselves at risk when engaging with the primary objects of their attention. On the other hand, if they adopted some physical engagements with volcanoes(bombs?) – the possibility of harm to other aspects of the ecosystem and communities has to be envisaged.

WHY was the research/enquiry/analysis conducted?

  • What was the purpose of the research enquiry?
  • For what reason was it conducted?
  • Who supplied the funding
  • How was it funded?
  • What were/are the funder’s intentions?
  • What were/are the researcher’s intentions?
  • What were/are the research agency’s intentions?
  • Were participant communities involved in determining the need for this research?
  • Were potential impacts evaluated and appropriate actions planned?

Motive and intent are key ethical issues. They go to why the research was conducted in the first place and what outcomes were hoped for and by whom. Impacts could be environmental, social, psychological, political etc. Hence the question of who commissioned and funded the research/enquiry is doubly important – details on the funding agency is key to full transparency.

WHEN and WHERE was the research/analysis conducted and/or policy advice provided?

  • In what context was the research/analysis/enquiry carried out?
  • What was the nature of the research site/setting?
  • When was the research/analysis conducted?

Most ethical judgements rely upon a full understanding the context in which the action under consideration occurred – the place and the time. This requires a comprehensive understanding of place and time: geographical, institutional, organizational etc. and diurnal, annual, chronological, historical and so on. Thus there are wide variations between a laboratory site, urban settings entailing risk and threats, libraries, and high- and low-resource countries. Laboratories can vary in licensing levels, while field sites vary in the kinds of permissions required. Historical archival research varies considerably in terms of ethical risk from the study of more contemporary documentation but engaging in historical enquiry may still entail risks to the present in terms of societal or communal stigmatisation and/or reputation. For example, knowledge of how and why a particular organisation was established may ‘taint’ its current reputation.

Was the research REVIEWED in advance for its scientific or analytic ‘quality’ and its adherence to ethics?

  • Is any form of pre-project review of the approach provided for within the institutional/sectoral set-up?
  • Was there independent review/appraisal by a competent body for the ethical issues raised by this research/enquiry?
  • Who reviewed the research methodologically/scientifically for quality issues prior to its implementation?
  • What regulatory approvals were granted for the research if any?
  • What additional permissions were necessary/granted for the research?

There are many stages/steps in terms of approval and/or appraisal processes to assess the quality of and risks (ethical etc.) for research projects. In some countries/institutions these processes are absent, but the increase of multinational, interdisciplinary approaches to research implicates researchers in ensuring some formal reviews are conducted. Reviewing standards and standard operating procedures are increasingly shared internationally and across institutions. In addition novel citizen science evaluation methods are emerging such as crowd reviewing. It may be difficult for all forms of research/analytical agency to secure independent assessment for the ethics of their work. Increasingly organisations do strive to establish their own in-house system with a degree of independence provided by some external memberships. There is no ‘best’ or single way of doing this, the importance again is for transparency – clarifying if any form of assessment of quality and ethics is done prior to the commencement of research and/or enquiry.

What were the OUTCOMES of the research?

  • How were the research/analysis findings reported, shared and/or disseminated? What policy advice was derived and given?
  • If parts/all of the analysis were not published, what was the reason for this?
  • How ‘selective’ were the reporting of findings?
  • Were the research findings implemented in practice – i.e. ‘applied’ or used?
  • What were the consequences of the findings being, or not being, implemented?
  • Were there any limitations on what could be accomplished with the findings – dissemination and/or application?
  • Could any form of ‘impact assessment’ be performed?
  • Was any evaluation of the outcomes conducted or planned for?

The research findings could be disseminated in a range of different ways – in academic publications, peer-reviewed scholarly publications, in-house technical reports, commissioned reports, independent white papers, official policy documents, policy briefings, participant feedback, social media, news media and so on. What was done with the ‘outcomes’ links back to the original ‘why?’ question, or what was hoped for/intended for the research. The researchers might not be in a position to directly apply the findings, but they might be better able to guide and assist those who can – i.e. the policymakers. A decision might be made to withhold publication of findings – justifications for such an action would have to be clear and strong.