Development and oversight of ethical health promotion quality assurance and evaluation activities involving human participants

This paper considers the role of ethics in the development of health promotion quality assurance and evaluation involving human participants and pays particular attention to guidelines issued by the Australian National Health and Medical Research Council (NHMRC). The imagined audience is primarily health promotion practitioners in local, regional and state health promotion services, although the comments are generally applicable to all workers in the population or public health field. The aim is to convince readers that consideration of ethical issues should be incorporated into all health promotion research, quality assurance and evaluation activities right from the beginning of planning the activity. Regrettably, ‘ethics’ is often equated principally with ‘ethics committee approval’, which is seen by many health workers, including those involved with health promotion quality assurance and evaluation, as an inconvenient, irrelevant and activity-delaying hurdle that must be cleared just when one is, most frustratingly, ready to get started. In contrast, I contend that when ethical thinking underpins the planning and conduct of research, evaluation and quality assurance, the activity itself becomes the better for it and ‘ethics approval’ can mostly be secured without much trouble. Some misunderstandings are also clarified.

This paper concerns the intersection of several concepts: health promotion, evaluation, quality assurance, research (particularly human research) and ethics, including two related but distinct concepts, namely ethical research and the independent ethical oversight of human research. Most readers will have commonsense understandings of these terms that are perfectly adequate for what are intended to be very practical comments. However, readers wanting introductory discussions of all except health promotion are referred to the opening sections (pp. 1–18) of the National Statement on Ethical Conduct in Human Research¹ (hereafter ‘National Statement’) and Ethical Considerations in Quality Assurance and Evaluation Activities.² In particular, readers’ attention is drawn to the discussions in the National Statement¹ of the four values and principles of the ethical conduct of research (research merit and integrity, justice, beneficence and respect; pp. 11–13), the risks and benefits of research (pp. 15–18) and the definitions in the Ethical Considerations in Quality Assurance and Evaluation Activities² (p. 2) of quality assurance (‘[a]n activity where the primary purpose is to monitor or improve the quality of a service…’) and evaluation (‘…the systematic collection and analysis of information to make judgements, usually about the effectiveness, efficiency and/or appropriateness of an activity’). As for health promotion, I will here regard it as the activities that commonly constitute the basis of papers reported in this Journal, the content and selection of which are, I assume, informed by the scope of the Journal (printed on the inside front cover of each edition), which refers to the ‘…knowledge base and evidence for health promotion action’. I particularly note the reference to action and so, while conscious that even basic epidemiological research can be done with health promoting action in mind, I will focus my comments on the quality assurance and evaluation of interventions.

The diversity of health promotion quality assurance and evaluation activities, and health promotion research generally, is exemplified by the wide range of:

• purposes for which health promotion quality assurance and evaluation are conducted (e.g. strategic planning; community needs assessment; identification of environmental and personal precursors to health, illness and behaviour; identification of service gaps; exploration of the experiences of participants in health promotion programs; process, output and outcome evaluation of programs; service development etc.)

• participants, including program users, community members and staff in health and other agencies

• scales of evaluation, from small-scale pilot studies to randomised controlled trials and complex community interventions

• methodologies and methods used, which consequently involve a vast array of sources and types of data.

Consequently, my remarks are intended to be generic rather than specific to any situation. The development of health promotion evaluation and quality assurance activities that are ethical requires an understanding of the relevant ethical principles and their routine (as in regular, not thoughtless) application to each specific situation.

Up to this point, the terms ‘research’, ‘quality assurance’ and ‘evaluation’ have been used, and it is important to emphasise that what any activity is called makes no difference to whether it requires the responsible people to consider its ethical dimensions. It may, however, affect the process within any particular organisation by which such activities are ethically reviewed. Similarly, putting aside conflicts of interest, who is conducting the activity, its purpose (specifically regarding quality assurance and evaluation, that they are ‘managerial activities’ usually intended to improve practice and outcomes) and how it is funded are irrelevant to whether it is ethical and whether it requires independent ethics review. And although most journals, including this one, now insist that papers reporting research, quality assurance and evaluation activities involving human participants have received approval from an appropriate ethics review process, the publication of findings is not in itself, in my view, a compelling ethical reason to obtain ethics approval.

The principal consideration from an ethical perspective is what being a participant in the research, quality assurance or evaluation involves and the risks to which the participant may be exposed by taking part. By this criterion, an activity that would generally be agreed to be ‘research’ being conducted by people who call themselves ‘researchers’ may raise few if any ethical concerns, whereas an activity called ‘quality assurance’ being conducted by people who never think of themselves as ‘researchers’ may have serious ethical issues: it all depends on what being a participant involves. Hopefully it is clear by now that, generally speaking, I regard quality assurance and evaluation activities involving humans as specific types of human research, or at least activities that have much in common with human research, and are bound by similar ethical principles and similar requirements for ethics review (for some erroneous comments about ethics review and health promotion quality assurance and evaluation, see Box 1).

The NHMRC is required by law to issue guidelines for the ethical conduct of research involving humans. The latest iteration of the guidelines, the National Statement, issued in 2007,¹ was developed jointly with the Australian Research Council and the Australian Vice-Chancellors Committee. Amendments and updates, which are available on the NHMRC’s website (https://www.nhmrc.gov.au/guidelines-publications/e72, accessed 21 May 2015), have been made since 2007.

Without going into details, the National Statement has sections covering the principles of ethical research conduct, the risks to participants and the benefits of research to both participants and the general public, participant consent for involvement in research, ethical considerations specific to particular types of research and particular groups of participants, and the required structures and processes for ethics review of research within institutions (health services, universities etc.). As suggested already, it is the requirements of this last section that health promotion workers have traditionally focused on, whereas in my view a careful reading of the whole document would lead to better quality assurance and evaluation and less frustrated workers.

Many health promotion workers will be aware of their organisation’s Human Research Ethics Committee (HREC) and its role in reviewing and approving research, but it is important to note that the National Statement allows organisations to establish different review and approval mechanisms for the different levels of risk to which participants may be exposed. Note that it is the level of potential risk to participants that determines the ethics review process within an organisation, not the potential public benefit, nor the name of the activity (‘research’ or ‘quality assurance’ or ‘evaluation’), nor who is conducting it, nor who is resourcing it, nor whether it will be published, although any of these may influence the types and level of risk to which participants may be exposed. As an aside, an organisation does not have to establish its own HREC or alternative processes. It can, if it chooses, accept the decisions of HRECs in other organisations to ensure that all human research conducted within it and by its staff meets the requirements of the National Statement.

In the National Statement, the risk potentially associated with a piece of research, quality assurance or evaluation is divided into three categories:

1. ‘Negligible risk’: any foreseeable risk is no more than inconvenience (e.g. filling in a form online, completing a street survey, giving up time)

2. ‘Low risk’: the only foreseeable risk is discomfort (e.g. minor side effects of medication, anxiety induced by an interview; note that ‘negligible risk’ and ‘low risk’ are distinct categories of risk, not a single ‘negligible and low risk’ category, as some people seem to think)

3. ‘More than low risk’: the risk, even if unlikely, is more than discomfort (note that this category is not called ‘high risk’, simply ‘more than low risk’).

All ‘more than low risk’ research, quality assurance and evaluation must be reviewed by an appropriately constituted ‘full’ HREC, but institutions can (not must) establish non-HREC procedures for ‘low risk’ and ‘negligible risk’ activities. Without being totally prescriptive, the National Statement provides guidelines for these processes that are generally less onerous in terms of paperwork and quicker in terms of processing, but no less rigorous from an ethical standpoint than review by an HREC. Should the members of a non-HREC committee or process decide that the proposal is ‘more than low risk’, they are obliged to refer it to the full HREC. The NHMRC has produced a two-page guide, including a useful flowchart, for the review of low and negligible risk activities.³ In the absence of an organisational policy on the review of negligible and low risk research, all research proposals must be submitted to an HREC; that is, an individual department cannot decide to set up its own processes for the review and approval of negligible and low risk research unless there is an organisation-wide policy legitimising this.

The National Statement also allows institutions to choose to ‘exempt’ some research from ethics review, but the conditions are very strict: the research must be negligible risk and it must involve only the use of existing collections of data that contain only non-identifiable data about the participants. But note that the decision to exempt research that meets these criteria from independent ethics review must be made by the organisation, not the researcher or his/her department, and the requirement that the research be conducted ethically still applies.

Finally, some comments on a couple of issues that lead to frustration among researchers. First, researchers sometimes get upset when the HREC returns an ethics application with criticisms of the scientific methods proposed for the conduct of the study: ‘Their job is to look at the ethics, not the methods,’ the researchers say. This complaint is easily dismissed in my view by considering that poor methods produce unreliable results that cannot be put to any practical use. Under such circumstances, it is impossible to claim that the public benefit justifies the risk, however small, because there is no public benefit. Methodologically poor studies are always unethical.

Second, researchers often become frustrated if they perceive the decisions of an HREC to be inconsistent either with previous decisions it has made or with the decisions of other HRECs. All committees are fallible and it is of course possible that an HREC will make an unjustifiably inconsistent decision, but, in my experience, most alleged inconsistencies are attributable to other factors. Most significantly, there is a common misunderstanding among health workers generally that ethical principles can be applied like a Maggie Beer recipe: follow all the steps carefully and whoever is doing it will produce similarly palatable results. But that is not the nature of ethical principles. Although there may be widespread agreement about a principle, its interpretation and how it is applied under particular circumstances are likely to differ somewhat among different people. Indeed, it is an underlying principle of the ethics review of research in Australia that although the NHMRC provides principles and processes for the ethics review of research, decisions about the ethical acceptability of individual research projects are made by committees (commonly HRECs) that are representative of the community, as well as having a required range of knowledge and experience. Hence, different HRECs may use the same ethical principles to come, after collective discussion, to different decisions. This is somewhat similar to the principles underpinning trial by jury: different juries may come to different decisions about the guilt or innocence of an accused despite the facts of the case and the law being similar for both juries. If ethical principles could be rigidly and uniformly applied without any reference to individual and community views, we could dispense with HRECs and develop an app.

Notwithstanding the previous two paragraphs, when HRECs reject applications, insist on certain conditions or seek further information, applicants can reasonably expect a written justification for the HREC’s decisions, including reference to the relevant sections of the National Statement.

The NHMRC has long recognised that practitioners, researchers, managers, organisations and HRECs can experience uncertainties and difficulties with the ethical dimensions of quality assurance activities.⁴ Is quality assurance research? Does quality assurance need independent ethics review, particularly because it is widely promoted as a routine and expected part of practice in the health services, rather than a more optional add-on, such as research, and because it often involves low risk activities? If it does need ethics review, does it need to go through the whole HREC rigmarole? An added complication is that the people who conduct quality assurance seldom think of themselves as ‘researchers’ and so it has not been first nature for them to consider ethical questions and procedures more traditionally associated with research when conducting quality assurance. These questions are particularly pertinent in health promotion, where the interventions and quality assurance methods used are often, but not always, less intrusive than in clinical practice.

In 2014, the NHMRC issued a new three-page statement concerning ‘ethical considerations’ of both quality assurance and evaluation.² The document recognised that ethics review of quality assurance and evaluation activities may not always be required, but emphasised that whatever an activity is called, participants must be ‘afforded appropriate protections and respect’; that the activity must be conducted ethically with consideration given to, for instance, risk, consent and privacy; and that organisations should develop policies for the oversight, and where necessary ethics review, of quality assurance and evaluation activities. The NHMRC provided seven ‘triggers’, the presence of any one of which should occasion formal ethics review according to the requirements in the National Statement:

1. Infringement of privacy

2. Use of data collected for a different purpose

3. Gathering information from participants that is beyond that which would normally be collected during the participants’ involvement with the service (to which could be added requiring the participant to do something that would not normally be part of the participants’ involvement with the service)

4. Use of non-standard protocols or equipment

5. Comparison of cohorts of participants

6. Randomisation of participants or the use of control groups or placebos

7. Identification of minority or vulnerable groups in the analyses (this is a common situation for health promotion workers because the identification of disadvantage and the promotion of equity are often priorities in their programs, and there is a constant risk that stigmatisation of disadvantaged and minority groups may occur as a result).

In my experience, many health promotion quality assurance and evaluation activities, and many of those reported in this Journal, would trigger at least one of these factors.

Although this paper has been somewhat prescriptive (in terms of process, if not outcomes) about what is required ethically in health promotion research, quality assurance and evaluation, I am conscious that there are and will continue to be many grey and even disputed areas. For example:

• There is ongoing concern that the systems of ethics review of human research that have been established in Australia and many other countries have, in many cases, been designed principally for research involving clinical interventions that use drugs and/or invasive investigative procedures and surgery, where there can be a serious threat to participants’ health. It has been claimed that such ethics review processes are inappropriate for social science research, which often involves low risk surveys and qualitative methods.^5,6 Much health promotion research, especially that pursuing quality assurance, uses such low risk methods. Although I have some sympathy with the criticisms raised, the reality is that the system established by the NHMRC is the required system in Australia for all human research, and even trenchant critics of current approaches accept the need for some formal system of ethics review and regulation of qualitative research.⁷

• There is ongoing debate about appropriate quality standards for the conduct^8,9 and reporting^10–12 of qualitative research, a group of methods commonly used in health promotion.

• Quality and ethical standards need to be developed for emerging research methods, such as online research (e.g. how does one seek consent?).

• What is the boundary between routine practice and quality assurance and evaluation? For example, if a health promotion service is developing a strategic plan and calls for public submissions on the content of the plan and/or interviews stakeholders about the plan, do these activities require independent ethics review?

It is not the purpose of this paper to provide answers to these dilemmas. Rather, I hope that it provides some information about considerations and procedures that will assist health promotion workers to consider the ethical dimensions and requirements of such issues when planning and conducting quality assurance and evaluation.

However, such dilemmas do beg the question, could Australia’s system of ethics review of quality assurance and evaluation (and more broadly ‘research’) in health promotion be made more appropriate for the types of activity involved? Tackling this would require attention to four variables: (1) the ethical principles considered relevant to the types of activity; (2) the characteristics of the system mandated for the ethics review of quality assurance and evaluation proposals; (3) the competencies of the people who review such proposals in accordance with the system in place; and (4) the ethical awareness of the people developing and conducting the activity. Space considerations limit the discussion of these factors to comments on the last two – the two that are most able to be influenced by health promotion workers themselves. In my experience, members of HRECs based in health services are extremely conscientious and hard working, but a minority would have a solid understanding of health promotion or the methods used in health promotion research, quality assurance and evaluation. Institutions could correct this when making appointments to HRECs and health promotion workers could approach their local HREC and/or its individual members with offers to conduct some form of awareness raising. A word of caution though: such awareness raising would need to be done by a person with a good understanding of relevant research methods and ethical principles, not a disgruntled health promotion worker. Second, again in my experience, health promotion practitioners and researchers have a very poor understanding of research ethics and the requirements of the National Statement. As noted at the beginning of this article, ethics is often seen as a bureaucratic hurdle rather than an aid to good research. Box 2 provides a procedural checklist to assist health promotion practitioners incorporate ethical principles into the design of their quality assurance and evaluation activities and secure independent ethics approval as simply as possible.

Health promotion workers are motivated to try to improve the health of individuals and communities and to promote equity. Quality assurance and evaluation, by seeking to improve policies, programs and outcomes, are central to this goal. However, quality assurance and evaluation, like all research, must be ethically justified; no matter how great the potential public benefit from the results of a particular project, there can never be any excuse for infringing the dignity and rights of each individual participant. The first priority of every health promotion worker involved in quality assurance or evaluation must be to ensure that the activity is conducted in an ethical manner. The main way to achieve this is to be well informed about the principles of human research ethics and to apply them from the very beginning of each quality assurance and evaluation project. Health promotion practitioners who are always thinking critically about the ethical issues involved in their quality assurance and evaluation activities are likely to have few problems securing formal ethics approval to proceed.