Who was studied
The researchers recruited 59 participants (33 women and 26 men), most of whom (53) were potential candidates for Phase I clinical trials; six were already taking part in an HIV vaccine trial. The participants ranged from 16 to 55 years old, and 92% (54) had completed high school. All of them had received information on vaccine trials through information sessions and, in most cases, written materials.
A study investigating “informed consent” in AIDS vaccine trials in Africa has found that participants may not understand the trial risks to the extent that standard measures would indicate.
The African-based HIV/AIDS Vaccines Ethics Group (HAVEG - part of the South African AIDS Vaccine Initiative) has been working to ensure that African vaccine trials are conducted ethically. One of the cornerstones of ethical research is “informed consent”: the participant’s right to freely choose whether to participate in a trial, based on full disclosure of the possible risks and benefits.
HAVEG has argued that “genuine consent means more than satisfying legal formalities (eg, signing consent forms)”, and that simple yes/no checklists may not be an adequate way to measure real understanding and genuine consent. This suspicion was confirmed by a recent study published in the December 15th edition of the Journal of Acquired Immune Deficiency Syndromes, in which HAVEG researchers used four different methods to measure how well participants understood various aspects of clinical trials. They found that assessments of “understanding” varied widely, depending on how they were measured, and that the simplest and most common measurements often overestimated the degree of understanding.
Assessment methods
Seven aspects, or “components” of vaccine trials were identified, through extensive consultations with clinical trial staff and community representatives, as being crucial for ethical involvement. These components were:
- trial aims,
- eligibility,
- risks of false-positive antibody tests if vaccinated,
- risks of a false sense of security against HIV infection,
- methodological aspects (e.g., randomisation, placebos, “blinding”)
- compensation for research-related injury, and
- the right to withdraw from the trial.
Four different assessment methods were then used to measure participants’ understanding of these components. The goal was to compare the apparent understanding levels according to the different methods, and gauge how well these reflected the “real” level of understanding. The 4 assessment methods were:
- Self-reporting: Participants were asked to rate their own understanding of each trial component as “little/no understanding” or “good enough”.
- Forced-choice checklist: Participants answered a questionnaire with three true-or-false statements about each component.
- Vignettes: Hypothetical scenarios were read to the participants (e.g., asking what a fictitious trial participant should do if she wanted to leave the trial). Participants then described what they thought would (or should) happen in those scenarios.
- Narratives: Participants were asked standardised questions, and answered them as though they were describing aspects of HIV vaccine trials to a friend.
In each case, answers were evaluated to determine whether the participant’s understanding was “good enough” to allow for true informed consent. Comparisons were made between the 4 different methods, for each of the seven components.
Results
Assessments of whether understanding levels were “good enough” varied, depending on which method was used. The self-report gave the highest scores – 86% of the participants rated their own understanding as “good enough” overall. The checklist assessment yielded 82%, narrative 69%, and vignette 67%.
Levels of understanding also varied for each individual component of the study, with the self-report and checklist generally showing the highest apparent levels of understanding. The following examples show potentially dangerous misunderstandings that would have been missed by simple checklists or self-reports:
- One participant who rated his/her understanding of placebos as “good enough” on the checklist revealed in the narrative that “I cannot differentiate between a placebo and the vaccine. But I think it (is) clear that both protect.”
- The antibodies that an AIDS vaccine can produce in an HIV-negative person can cause false positive results on subsequent HIV antibody tests. One participant rated his/her understanding of this concept as “good enough” on the checklist and self-report. However, when asked to explain a false-positive antibody test result in a narrative, the participant said “it means [she] was not too much into the vaccine trials, it means that she was not faithful … It means it (was) not working.”
To avert such dangerous misunderstandings, it is crucial to be able to truly and accurately measure trial participants’ real degree of understanding of trial risks and benefits. The results of the “Beyond the Checklist” study show that measurements – especially simple closed-ended measurements – of such understanding may very well be inaccurate.
The researchers state that their results “should caution us that closed-ended measures may overstate levels of understanding”, that this study “contributes initial information to a complex ethical issue”, and that “there is preliminary evidence that … extended discussion and interactions with trial staff are the best ways of improving understanding.”
Reference
Lindegger G et al. Beyond the checklist: assessing understanding for HIV vaccine trial participation in South Africa. J Acquir Immune Defic Syndr. 43:560-566, 2006.