Consent for Research in a Learning Healthcare System
I had the privilege of participating in an HHS meeting (unfortunately coinciding with MLK “I had a dream” festivities on the Mall at the same time) in which many people gave their views about informed consent, clinical trials assessing comparative clinical care strategies and the SUPPORT Trial.Ā Here is my testimony, which can also be found on the OHRP website.Ā The verbal presentation can be found at: www.youtube.com/watch?v=1Hunhqfz8C4
Below is the written statement:
I am pleased and honored to have the opportunity to address the Committee on each of the items listed in āSection III. Issues for Discussionā of the āNotice of a Department of Health and Human Services Public Meeting and Request for Comments on Matters Related to the Protection of Human Subjects and Research Studying Standard of Care Interventionsā published on p.Ā 38,343 of the Federal Register on June 26, 2013. I have a number of roles and associations that are relevant to the questions at hand. However, the views I express in this document do not represent the official positions of the organizations with which I associate or that fund my activitiesāthey represent my personal opinions only.
I have worked as a clinical investigator for more than 30 years. My research interests include clinical trials and outcomes research involving development of new interventions, but also the evaluation of interventions already in use and the delivery of health services. I lead a clinical and translational research enterprise at Duke University with hundreds of faculty investigators and more than 1000 employees, and thus have a definite interest in promoting research. Most recently, I was chosen to serve as the Principal Investigator of the Coordinating Center for the NIH Healthcare Systems Research Collaboratory, a major effort to develop our national capacity to answer important questions about medical practice. I am also the co-Chair of the Clinical Trials Transformation Initiative (CTTI), a Cooperative Agreement between the FDA and Duke University that includes more than 60 member organizations representing industry, academia, government, and advocacy groups.Ā Ā Ā
My career as a doctor began in 1978, and I have spent many years directing a busy coronary care unit and attending in a large outpatient clinical practice. During this time, I have witnessed many instances in which ignorance about efficacy and relative risks of interventions led to harm to individuals and populations, despite the best intentions of healthcare providers, patients, families, and the healthcare system as a whole. We simply did not know, for instance, that Type I anti-arrhythmic drugs caused sudden death rather than preventing it; that hormone replacement therapy in postmenopausal women caused cardiovascular events rather than preventing them; or that high-dose erythropoietin stimulating agents (ESAs) caused tumor growth and adverse vascular events rather than improving quality of life in cancer patients with anemia or in patients with renal failure. My own specialty, cardiovascular medicine, is the most evidence-intensive field of medicine, but our own research has shown that 85% of major recommendations in this field lack high-quality evidence to support them.
Erythropoietin stimulating agents deserve special mention in light of recent controversies about blood transfusions. At the time this class of drugs first emerged, it seemed self-evident that ānormalizingā hemoglobin levels would benefit patients, a concept supported by observational studies. Small early clinical trials (probably biased by selective reporting) also showed trends toward benefit with more aggressive use of high-dose ESAs. This practice of aggressive use was recommended in clinical practice guidelines well before adequately powered randomized trials were done, resulting in widespread clinical uptake. When randomized trials were finally done, there was no evidence for benefit with high-dose ESAs and considerable evidence for harm., Depending on pilot studies and trials with inadequate statistical power to guide treatment decisions is treacherous and potentially dangerous to patients.
Finally, as I have gotten older, I have encountered my own chronic health problems, and multiple members of my family have developed health issues that required āeducated guessesā together with their healthcare providers. Doctors, administrators, policymakers, patients, and family members must make decisions every day about the best course of actionādecisions that will affect individual patients, the clinic, the hospital, or the health system as a whole. Prior to the 1970s these decisions were, with few exceptions, informed by experience stored in a clinicianās memory or on paper, or by reasoning based on our current understanding of human biology.
However, in recent decades we have benefitted from three major advances in healthcare research methods relevant to this discussion: the advent of computerized storage and analysis of medical data, the maturation of the disciplines of biostatistics and epidemiology, and the widespread acceptance of the epidemiological tool of randomization. We have learned more about the specific ways in which human observation and memories are flawed, and that methods to account for bias and confounding are essential to avoiding faulty conclusions that can lead in turn to devastating consequences.Ā
We have known for some time that most beneficial interventions have a modest effect; and for effects with a risk reduction smaller than 40% or so, randomization is essential to avoid drawing incorrect conclusions. But despite this knowledge, we have been limited in our ability to aggregate high-quality data in ways that would allow us to efficiently conduct controlled trials and evaluate interventions, because healthcare transactions were not based on electronic standards. Thanks to continuing advances in technology and information systems, we now have the ability to gather needed information very rapidly. Unfortunately, the existing culture of healthcare, perhaps best exemplified by the divide between research and practice and by inhibitions about the use of randomized allocation of interventions, keeps us from advancing life-saving knowledge as effectively as we should.
Given the choice between a pure guess and expert opinion, I would take expert opinion, but given the choice between opinion and high-quality evidence, an expert informed by high-quality evidence is vastly superior. The question before us is how to unleash the power of the knowledge we can gain while respecting and protecting the interests of individual participants in the system.
My detailed views on the subject under discussion accord with those expressed in the Institute of Medicineās CERIC Committee document, which I have signed and endorsed. I will however provide some brief additional comments. I believe there are three key changes in the background that necessitate a reconsideration of the approach to consent comparing āstandard of careā interventions:
First, we are in the midst of a revolution in medical knowledge that could have a major effect in reducing death and disability both nationally and globally. This revolution is occurring thanks to our rapidly expanding ability to aggregate data that can truly inform which interventions are best for a given patient or patients. (By āinterventionsā I mean drugs, devices, behavioral therapies, and health services strategies at the level of individuals, clinics, hospitals, and health systems.)Ā When my career began, the concept of aggregating information on a large scale could be imagined but was not yet technically feasible. Now, we will soon have more than 300 million Americans with electronic health records, and tabulations of patient characteristics and outcomes are being made routinely at every level of the system.
Second, as a consequence of the availability of these vast amounts of data, there is a pressing need to reform rules governing research that made perfect sense in the āold worldā but are now hindering further progress. These issues are well described in the Hastings Center Report, already discussed in other documents submitted for this meeting. The bottom line, however, is that the current artificial separation of research and practice reduces the rate at which we are able to provide evidence that is critically needed to inform the myriad of decisions heretofore relegated to the best guesses of experts armed with inadequate methods. Furthermore, when health systems collect data systematically, current norms encourage the use of suboptimal methods. As documented in the Hastings Center Report, this likely leads to numerous bad decisions, because employing the best methods and disseminating the continuously aggregating knowledge would lead to labeling the learning activity as āresearch,ā thereby triggering a set of highly regulated activities that are costly and interrupt the flow of practice.
Finally, I believe that the best way to cut through the layers of bureaucracy that are holding us back is to do the empirical research that will provide perspective on the expectations of patients and research participants, and then to involve the public more broadly in the research enterprise. There is a substantial body of literature about what the vast majority of people would consider to be reasonable to inform them and gain their consent. I am confident that empirical research will lead to a new norm in which learning is much more continuous, the intensity of oversight is gauged to the level of risk, and systematic approaches to informing, including and gaining the consent of research participants become more sensible and streamlined, with a reduction in the length and complexity of current consent documents, which are both broadly unpopular and poorly suited to accomplishing their ostensible purpose.
Comments on Specific Questions Raised by OHRP
1. How should an IRB assess the risks of standard of care interventions provided to subjects in the research context? a. Under what circumstances should an IRB consider those to be risks that may result from the research?
A reasonable approach to this issue is to focus on the incremental risks beyond those of the standard of care. This concept has been well-articulated by others, especially in the document submitted by the American Association of Medical Colleges.
A significant and related issue is the requirement to discuss all reasonable alternatives and their risks and benefits when learning takes place under the rubric of research. Given that we know that most decisions in medicine are not informed by high-quality evidence, it is the difficulty of informing patients about the uncertainty inherent in clinical practice that leads to a situation in which the researcher has to āmake up for lost ground,ā not only by explaining the incremental risks and benefits of the study, but also by explaining the issues involved in routine practice.
B. Under what circumstances should an IRB refrain from considering those risks as unrelated to the research?
In many cases where research concerning alternatives within usual care is involved, the interventions being compared are not known to have a net difference in risk/benefit balance.Ā Arguments are often based on indirect comparisons, opinions, and non-systematic empirical tabulations and are typically accompanied by considerable bias as well as intellectual and financial conflicts of interest. When the proper evaluation has been done to establish that high-quality evidence does not provide a clear choice between alternative interventions, I believe that the IRB should consider these known or perceived risks not as being related to the research, but instead as part and parcel of risks inherent to clinical practice.
Others have pointed out that in cases when there is equipoise between two interventions, the net benefits and risks may be the same, but the specific benefits and risks may differāfor example, one may cause headaches and the other may cause myalgia. In such circumstances patients known to be at high risk for adverse outcomes from one of the treatments should be excluded, and when a reasonable person might choose one over the other, the patient should be informed.
C. What type of evidence should an IRB evaluate in identifying these risks?
An IRB should evaluate all available evidence, but the most persuasive form of evidence should be well-conducted clinical trials, or well-conducted registries or epidemiological studies that have been published in the peer-reviewed literature. Opinion uninformed by high-quality data should be considered, but it should also be clearly understood for what it is, and the amount of uncertainty should be considered on a continuum.Ā
The skills needed to inform these determinations are considerable, and I believe that the complexity of the task of evidence evaluation is one of many reasons to involve multiple centers and leverage the composite expertise of IRBs or central IRBs to debate these issues. The Internet is well-suited for enabling local IRBs to participate in joint sessions, an approach that is being piloted in the CTSA organizations funded by the NIH.
2. What factors should an IRB consider in determining that the research-related risks of standard of care interventions, provided to research subjects in the research context, are reasonably foreseeable and therefore required to be disclosed to subjects?
This question is difficult to answer conclusively because the evidence for the incremental risks of a given intervention exists on a continuum of severity and uncertainty. For this reason, the āreasonable personā yardstick may be the most appropriate measure to apply. For a properly constituted IRB, research risks should be disclosed when they exceed the threshold above which a reasonable person would want to be informed. As noted above, I believe these determinations are best made through discussions that provide perspective and a chance to air and consider differences of opinion before decisions are made. The widely documented variation among IRBs suggests that discussions across IRBs would lead to more consistent, higher-quality outcomes.
A. What criteria should be used by the IRB to evaluate whether the risks to subjects are reasonably foreseeable?
It may be useful here to distinguish āforeseeable risksā from hypotheses and study aims. Patients can understand that a study is needed because the best choice of intervention is not yet known. When describing a trial, the expressed purpose of the study should include the outcomes under evaluation, because it is within the realm of prospectively defined outcomes that the uncertainty can be reduced through pre-specified hypothesis testing. Simply because a trial is evaluating total mortality, for example, does not mean that increased mortality is an expected risk, but rather that it is a possible risk or in fact a possible benefit (remembering that in a comparison within usual care, a risk of one therapy is a benefit of the comparator) that is worth measuring and may be discovered as a result of the trial.Ā
3. How should randomization be considered in research studying one or more interventions within the standards of care? Should the randomization procedure itself be considered to present a risk to the subjects? Why or why not? If so, is the risk presented by randomization more than minimal risk?Ā Should an IRB be allowed to waive informed consent for research involving randomization of subjects to one or more standard of care interventions? Why or why not?
The act of randomization undeniably reduces autonomy, but in many cases there is no reasonable evidence that it increases risk. In fact, there is substantial evidence that participating in a randomized trial may reduce overall risk relative to standard practice. I believe that potential research participants should be informed of this body of knowledge.Ā
Overall, it is appropriate that OHRP is focused on randomization, as it is a necessary part of developing reliable evidence to support decisions. But randomization is also a complex concept that is poorly understood even by many healthcare professionals. Much can be learned through well-conducted observational studies, and there is universal agreement that randomized trials and observational studies are complementary. Yet, as recent work by the Observational Medical Outcomes PartnershipĀ makes clear, even with advanced, recently developed methods, we cannot sort out the confounding and bias when making treatment comparisons for typical effect sizes (i.e., smaller than 40%). In plain English: Without applying the tool of randomization to assess interventions with modest effects, we run a significant risk of drawing the wrong conclusions!
Randomization raises a number of issues in practical application, not least because it directly calls attention to the uncertainty inherent in medical practice. Randomization constitutes a tacit admission that we know so little about the relative effects of interventions that our best available option is to āflip a coinā to determine who gets which intervention. The need to explain these concepts as part of the informed consent process is far more time-consuming and causes much more discomfort for patients, families, and providers than does consent to use data to make observations with no intrusion into treatment selection.Ā Ā
The alternative, however, is what we have now: practitioners, clinics, and systems routinely recommending one intervention over another without explicitly recognizing the degree of uncertainty or the prevalent financial and professional conflicts that shade these recommendations. Under the rubrics of āoperations improvementā and ācompetitive intelligence,ā health systems often support such decisions with proprietary analyses of observational data that are performed using suboptimal methods and that lack explicit public accountability or airing of the methods and results. Yet the best systematic review performed to date found no evidence that participation in a randomized trial increases risk, and the trend in the evidence is toward reduction in risk. This finding is consistent with results from the SUPPORT study, in which babies included in the trial had a lower rate of mortality, regardless of randomized treatment assignment, when compared with allowing the physician and parents to choose the oxygen saturation level within the range of standard care.Ā
Cluster randomization adds a particular twist to this discussion, because changes are made at the level of a provider, clinic, unit, or system. This approach allows delivery of the intervention to proceed efficiently according to the same standard with each patient. Many cluster randomized trials have not required consent at all, because a determination has been made by an IRB that the intervention involved only minimal risk to patients and that obtaining consent of the sort typical of standard randomized controlled trials would be impractical. My personal belief is that empirical research will demonstrate that the vast majority of people will agree that standard consent is not needed for this type of research when its incremental risks are judged to be minimal by an IRB, but that people will want to be informed. However, this empirical research needs to be done.
A consequence of the move of physicians from private practice to an employment model is that they become employees of systems that make corporate decisions. However, while health system formularies are often restricted and services are provided or withheld at unit levels without explicit acknowledgement to patients or families of the options that could have been considered (and without public display of the findings that generated the policies), cluster randomization explicitly attempts to learn, in the most rigorous manner possible, which of several options is best. Cluster randomization mimics what happens in āreal-worldā healthcare, where intervention decisions are increasingly made at the level of a practice, unit, or hospital to provide consistency and reliable delivery.
In summary, I do not believe randomization itself imposes incremental risk as a general principle, although each case should be assessed on its merit.
4. How, and to what extent, does uncertainty about risk within the standard of care affect the answers to these questions? What if the risk significantly varies within the standard of care?
The only legitimate reason to undertake a research study involving humans applies in situations wherein enough uncertainty exists that reasonable people would agree that the best choice for those being studied is not known based on a rational evaluation of all available evidence. The level of risk almost always varies within the standard of care, because harms of interventions depend on patient characteristics and the proficiency and other characteristics of the healthcare providers implementing the intervention. In my opinion, the inflection point is reached when there is some proven level of risk versus a situation in which the possible risk is not proven and different opinions exist about the presence and extent of risk within the patient and provider communities. Additionally, if risk without countervailing benefit exists in an easily identifiable subpopulation, those patients should be excluded from participation to protect them from harm.
5. Under what circumstances do potential risks qualify as reasonably foreseeable risks? For example, is it sufficient that there be a documented belief in the medical community that a particular intervention within the standard of care increases the risk of harm, or is it necessary that there be published studies identifying the risk?
The criteria for explicitly identifying āforeseeable risksā in general should require credible evidence. The history of ābeliefs in the medical communityā is replete with errors due to bias both conscious and unconscious, and the complexities of detecting and interpreting treatment effects in the ranges typically achieved with medical interventions (10%ā30%).
In conclusion, the widespread adoption of electronic health records and the development of methods for leveraging novel sources of data could enable us to increase the number of important healthcare decisions informed by high-level evidence by a log order or more, thereby saving countless lives and preventing disabilities caused by suboptimal knowledge regarding the most effective practices. But in order to enable such dramatic progress, we must also develop sensible approaches to the oversight of research within a learning healthcare system, ones that encourage the adoption of the best methods and the timely provision of relevant information to participants in a healthcare system that meets their expressed needs. This effort must include use of randomization, together with its attendant explicit acknowledgement of uncertainty.Ā Consequently, we should develop streamlined approaches for encouraging learning with methods that will yield reliable results capable of benefiting both research participants andāas high-quality information accumulatesāfuture patients as well. Empirical research on how to inform and acquire consent in this new world could pave the way to a revolution in knowledge about how to provide the best healthcare possible.
Robert M. Califf, MD, MACC
Vice Chancellor for Clinical and Translational Research
Director, Duke Translational Medicine Institute
Donald F. Fortin Professor of Cardiology
Duke University School of Medicine