Uncertainty in Medicine: A Talk With Dr. Djulbegovic

In part 2 of this interview, Benjamin Djulbegovic, MD, PhD, discusses the uncertainty principle in clinical trials.  Dr. Djulbegovic is associate professor of medicine, Divisions of Blood and Marrow Transplantation, H. Lee Moffitt Cancer Center and Research Institute at the University of South Florida, Tampa.

Oncology News International: What are the ethical implications of the uncertainty principle?

Dr. Djulbegovic: An ethical issue arises in the assessment of the quality of the preexisting knowledge. In recent years, increasing evidence points to systematic review and meta-analysis, when possible, as the best method for research synthesis of available knowledge regarding a given treatment. The logical question then becomes: Is it ethical to perform a trial if it is not preceded by systematic review and meta-analysis?

A second question relates to the choice of the control group. We should bear in mind that any testing of medical treatments is an exercise in comparison. Basing trials on the uncertainty principle addresses this important issue in the design of a clinical trial-the choice of an adequate comparator for the intervention under investigation.

Studies violate the uncertainty principle when the intervention and the control or comparison group are known in advance to be nonequivalent in their effects on the outcomes of main interest. Even if a study is properly reported, extra caution might be needed in its interpretation if the choice of the comparator treatment was not based on uncertainty about the relative value of the treatments being assessed.

The main point here is that explicit acknowledgment of uncertainty prior to the trial is the only mechanism that we have to ensure a proper choice of an adequate control group. Studies can meet all other quality criteria regarding conduct and design and can still be biased if the control group was inferior or inadequate.

For example, we recently found that for every single quality criterion analyzed, industry-sponsored studies are equal to or better than publicly sponsored studies. Yet, industry-sponsored studies were more biased because they more often chose an inferior control group in their trials (as evidenced by increased percentage of placebo/no therapy comparators in the industry-sponsored studies).

I believe this is because more awareness and greater regulatory pressures are applied to ensuring the appropriate technical aspects of a trial’s conduct (proper randomization, power analysis, etc) while the explicit requirement to assess the relative uncertainty in a trial’s design has not been mandated so far.

Explicit acknowledgment of uncertainty is the only mechanism that exists today to ensure a proper choice of a control group. In my opinion, explicit acknowledgment of uncertainty should be mandatory and also should be included in the CONSORT statement, a detailed report on how the quality of the conduct and reporting of clinical trials can be further improved (Begg et al).

ONI: How much uncertainty is necessary for an ethical clinical trial?

Dr. Djulbegovic: Uncertainty can have many shapes and grades. Equipoise is another term used to represent uncertainty about the relative value of treatment alternatives. Equipoise refers to the point where there is no preference between treatments (ie, it is thought equally likely that treatment A or B will turn out to be superior).

In the words of Richard Lilford, "At this point, we may be said to be ‘agnostic’ or ‘resting’ on the fulcrum of a decision: we would take odds 1:1 in a bet. Equipoise is different from simply ‘not knowing’ or being ‘uncertain’-we can be uncertain but not in equipoise."

In decision-analytic terminology, equipoise is defined as the situation in which the competing treatments have equal expected utilities. This is also known as the "threshold," which describes a clinical situation of indifference between alternative treatment options. Bradford-Hill has argued that randomization is acceptable "only in our state of ignorance."

In information theory terminology, equipoise can also be said to be equivalent to entropy, which is a measure of choice. Entropy is maximized when the probabilities of all choices are equal. Thus, equipoise simply represents the point of maximum uncertainty.

The uncertainty might be in the mind of the patient, the clinician, or the community. The literature distinguishes three forms of equipoise: collective, individual, or community.

• Collective equipoise ("clinical equipoise"), a concept originally introduced by Freedman, represents a product of the professional consensus of a group of experts who honestly disagree about which treatment is better.
• Individual equipoise ("theoretical equipoise") represents an honest belief of individual physicians that one treatment is not preferred over another.
• Community equipoise is concerned with involvement of patients and lay-people in the design of randomized controlled trials based on the principle that medical knowledge is not confined to the profession of physicians and that physicians have ethical responsibility for shared decision-making.

In answering the key question "how much uncertainty can we accept prior to entering a patient into a clinical trial?" the normative theoretical ideal implies that a randomized clinical trial should be performed only if there is an equal split (50:50) between a choice of two alternative treatment interventions, whether from the clinical, theoretical, or community perspective.

However, it is not known if people actually require this level of uncertainty prior to beginning a randomized controlled trial.

ONI: Has there been research into how much uncertainty patients, clinicians, and ethicists are willing to accept prior to starting a clinical trial?

Dr. Djulbegovic: Specifically, attitudes of people toward equipoise have been evaluated in only one study to date. In this study, Johnson et al surveyed laypeople to determine to what degree collective equipoise can be disturbed before potential participants perceive a trial as unethical.

They found that half of the potential participants perceived a trial as unethical when equipoise was disturbed beyond 70:30. In other words, when 70% of experts favor one treatment, 50% of potential participants would prefer that treatment be purposely selected and administered rather than subjected to randomization for research purposes. When equipoise is disturbed beyond 80:20, less than 3% of potential participants would consider human trials morally justifiable.

We recently repeated this survey among Institutional Review Board (IRB) members of the University of South Florida and obtained very similar results to Johnson et al (unpublished data). Obviously, more work needs to be done in this area, including measurement of equipoise among researchers and patients.

This work also raised a burning question: "How do we, in fact, measure uncertainty related to patients’ entry into clinical trials?"

If our patients have a right to ask what is already known about the therapeutic alternative being tested in a trial, shouldn’t they also have a right to know what all investigators participating in the trial believe about the relative value of that treatment vs another?

Should all controlled trials then be preceded by a formal survey of all investigators, the results of which will be presented to patients at the time when they are asked to participate in a trial?

ONI: What are the implications of the uncertainty principle for clinical research?

Dr. Djulbegovic: If the uncertainty principle plays a role in the conduct of randomized controlled trials, we would expect to see the distribution of belief probabilities averaging about 50:50, split between preference for standard and innovative therapies. If this were not the case, patients might be expected to be rather cautious about entering into clinical trials of innovative therapies and would request those treatments that are expected to be superior.

However, if the uncertainty principle is applied, there is no a priori reason to be cautious about entering a randomized trial, because innovative therapies are just as likely as standard treatments to be beneficial or harmful on average. This is because investigators do not know in advance what they will discover. (Having hunches, beliefs, or hopes does not mean that investigators know in advance the outcomes of clinical trials.)

Indeed, if scientists knew what the results of a proposed study would be, then there would be no purpose in conducting experimental clinical trials. In addition, informed consent would have to reflect this knowledge.

I suspect that, under these circumstances, few patients would be willing to enroll in a clinical study and accept randomization. Thus, by preserving the uncertainty principle, we are, in fact, preserving the system of clinical trials in medicine. In the final analysis, the uncertainty principle is the most important principle for human experimentation and the one on which stands nearly the entire system of human experimentation.

ONI: Is the uncertainty principle at work in research today?

Dr. Djulbegovic: We hypothesized that if the uncertainty principle was at work in clinical trials, the results of clinical trials over the long run would be negative as often as they were positive.

Only a few trials have looked at the overall outcomes of clinical trials in a given area. Gilbert et al reviewed 46 randomized clinical trials that compared a variety of surgical and anesthetic treatments in various disorders; they found that 49% of the innovative therapies were successful when compared with standard treatments.

In the fields of cardiology, neurology, psychiatry, and respiratory medicine, Colditz et al examined 36 randomized clinical trials from 1980; they found a 61% chance that a patient who received the new therapy would fare better than a patient getting standard therapy.

In oncology, Machin et al examined the results of innovative cancer therapies in 32 Medical Research Council trials over a 30-year period. They found that 45% of trials favored innovative treatment when assessed at the 10% level of statistical significance.

In our own study of 132 randomized controlled trials in multiple myeloma, we observed that 44% of the trials favored standard treatments while 56% favored innovative treatments (P = .17). Therefore, it appears that we can expect that about 50% of our innovations are successful.

ONI: Is a 50% success rate in randomized clinical trials a good investment?

Dr. Djulbegovic: The 50% success rate associated with randomized clinical trials is a very good investment for society in clinical research, particularly if the intangible benefits of innovative treatments are also considered. This result should be put into perspective when one examines the number of clinical trials that are currently being performed.

There are about 60,000 clinical trials currently underway, with more than 200 published in the scientific literature every week (Martin et al, Sackett et al). In light of legal, scientific, and ethical requirements that no treatment can be approved without prior testing in humans, it is likely that the number of clinical trials will continue to explode.

However, I would argue that the magnitude of the expected payoff brought about by innovations tested in controlled clinical trials is directly linked to the uncertainty principle. Adherence to the uncertainty principle promotes patient confidence in the clinical trial system and ensures that the clinical trials we invest in address the source of uncertainty directly instead of hedging in favor of the innovative treatment in order to have a "successful" trial.

Thus, by endorsing a fundamental ethical principle, it is possible to further advance research. Not only is preserving the ethics of clinical research the best investment strategy available, but desirable dividends from the research invested in randomized controlled trials can be obtained only if this fundamental ethical principle is conserved.

In conclusion, I would say that the practical, scientific, and moral dilemmas of clinical research are best captured by the acknowledgment of the uncertainty about relative values of competing treatment alternatives. I believe that explicit assessment of uncertainty will help the process of resolving existing uncertainty about the desired benefits and unknown risks of inadequately tested therapies.

One of the goals of our recent writings on the subject is to put everything on the table as explicitly as possible, so that even those who may not agree with us have a clear picture of where we stand and where future work is needed.

References:

Begg CB, Cho, MD, Eastwood S, et al: Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA 276:637-639, 1996.

Bradford-Hill A et al: Clinical trials and the acceptance of uncertainty. Br Med J 294:1419, 1987.

Bradford-Hill A et al: Medical ethics and controlled trials. Br Med J 2:1043-1049, 1963.

Colditz GA et al: How study design affects outcomes in comparisons of therapy: I. Medical. Stat Med 8:441-454, 1989.

Djulbegovic B, Bennett CL, Adams JR, et al: Industry-sponsored research (reply to letters). Lancet 356:2193-2194, 2000.

Djulbegovic B, Clarke M, et al: Scientific and ethical issues in equivalence trials. JAMA 285:1206-1208, 2001.

Djulbegovic B et al: Diagnostic entropy as a function of therapeutic benefit/risk ratio. Med Hypotheses 45:503-509, 1995.

Djulbegovic B, Lacevic M, Cantor A, et al: The uncertainty principle and industry-sponsored research. Lancet 356:635-638, 2000.

Freedman B et al: Equipoise and the ethics of clinical research. N Engl J Med 317:141-145, 1987.

Gilbert JP, McPeek B, Mosteller F, et al: Statistics and ethics in surgery and anesthesia. Science 198:684-689, 1977.

Johnson N et al: At what level of collective equipoise does a clinical trial become ethical? J Med Ethics 17:30-34, 1991.

Lilford RJ et al: Equipoise and "the uncertainty principle" are not two mutually exclusive concepts. Br Med J Sep 25: 2000.

Lilford RJ, Djulbegovic B, et al: Equipoise is essential principle of human experimentation (letter). Br Med J 322:299-300, 2001.

Ludmerer KM, et al: Time to Heal. New York, Oxford Press, 1999.

Machin D et al: Thirty years of Medical Research Council randomized trials in solid tumors. Clin Oncology 9:100-114, 1997.

Martin JB, Kasper DL, et al: In whose best interest? Breaching the academic-industrial wall. N Engl J Med 343:1646-1649, 2000.

Mathe G, Brienza S, et al: From methodology to ethics and from ethics to methodology. Biomed & Pharmacother 42:143-153, 1988.

Matthews JR et al: Quantification and Quest for Medical Certainty. Princeton, NJ, Princeton University Press, 1995.

Sackett DL, Hoey J, et al: Why randomized controlled trials fail but needn’t: A new series is launched. CMAJ 162:1301-1302, 2000.

World Medical Association: World Medical Association Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects. JAMA 284:3043-3045, 2000.

Part 1 appeared in the November 2001 issue of Oncology News International.