Commentary (Zhong/Lynn): Some Elements of Prognosis in Terminal Cancer

The claim that physicians make substantial, systematically optimistic errors in prognostication undergirds the article by Lamont and Christakis. This claim seems to contrast with our experience in the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT).[1] In that project, we developed a well-calibrated, accurate prognostic model for estimating the survival time (up to 6 months) of hospitalized patients with each of nine serious illnesses.[2] The model was developed with 4,301 patients and tested on the ensuing 4,028 patients. Physicians estimated the likelihood of surviving 2 and 6 months.

As Lamont and Christakis correctly point out, including physicians’ survival estimates improved the predictive accuracy of the SUPPORT model.[2] However, the hypotheses suggested by Lamont and Christakis–that physicians’ survival estimates in advanced cancer patients are inaccurate overall, as well as biased in the direction of excessive optimism–have not been examined previously with SUPPORT data.

Analysis of SUPPORT Data

To test these empirical claims, we analyzed the SUPPORT data of patients with three oncologic diagnoses at advanced stages: colon (N = 406), lung (N = 764), and multiple-organ system failure with malignancy (N = 594).[2] We compared physician estimates of prognosis (between the second and sixth study day) to prognoses estimated by the SUPPORT prognostic model (on the third study day), as well as to actual survival.

For the 1,757 patients, 484 different physicians gave prognoses. Figure 1 presents the estimates for all three diagnoses in aggregate. The mean model estimates, mean physicians’ estimates, and actual survival are virtually identical at both 2 and 6 months. When we examined the three diagnoses separately, the mean of model estimates was always within 3% of actual survival. The mean of physician estimates was always within 7.5% of model estimates, and the mean of physician estimates was always within 9% of actual survival. The direction of differences varied across time and disease type, but the physicians and the models always erred in the same direction.

Lamont and Christakis defined serious errors as those in which the physician’s survival estimate was double or half of actual survival. This definition may be misleading with regard to very short survival durations because the difference between 2 and 4 days, or between 2 and 4 weeks, may not be viewed by patients or families as serious. The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments did not ask physicians to estimate the time of death. Thus, we are unable to replicate the definition of Lamont and Christakis. Instead, SUPPORT asked physicians to estimate each patient’s likelihood of surviving 2 and 6 months.

Since the SUPPORT model provides a continuous estimate for 6 months, Figure 2 shows the array of differences between the physicians’ estimates and the model’s estimates, both “optimistic errors” (physicians’ estimates greater than the model) and “pessimistic errors” (physicians’ estimates less than model). The rate of errors of various sizes does not seem surprising, and the weight of error is not evidently biased in either direction.

Clearly, these data are not consistent with a claim that physicians are systematically optimistic in their estimates. Overall, physicians are quite accurate, and when their estimates differ substantially from a statistical model, they are as likely to be optimistic as pessimistic.

Why are these results so different from those in the literature summarized by Lamont and Christakis? Perhaps physicians make more substantial errors, proportionately, when asked to prognosticate for persons who are all very close to death (as in hospice enrollment studies). It may be that physicians respond more accurately when asked to predict risk of death by certain time points, rather than timing of death. Perhaps it is easier to be precise with hospitalized patients or with the three clinical conditions examined here. The professional literature has barely started to characterize the phenomenon of prognostication.

SUPPORT Patients’Estimates of Survival

In SUPPORT, physicians, on average, were quite accurate; however, SUPPORT patients were not.[3] Patients with cancer generally overestimated their survival. Perhaps this arises from physicians’ systematic miscommunication. However, since SUPPORT patients had to have been diagnosed with metastatic disease for more than a month, and since information about survival is available in many places, the fact that patients did not seek out correct information may reflect psychological and social factors beyond physician communication.

Patients did not have to be accurate about prognosis in order to change their mind about the merits of comfort care. Although patients were certainly overly optimistic, once they acknowledged that they had a 10% chance of dying within 6 months, they were much more likely to prefer comfort care to life extension.[3] Perhaps the thresholds at which behavior shifts are important, and the accuracy below or above a threshold is not very important, as long as patients and families are not actually misled. Possibly, as long as the patients are capable of guiding their own life closure, any misunderstanding of their precise prognosis may not be as serious an error as it first appears.

Is Earlier Knowledge of Prognosis Beneficial?

The weight loss and inability to carry out activities of daily living that cancer patients usually experience in their last month or two of life[4] may be sufficient to warn most patients to address religious and family concerns at the end of life. Research has yet to examine whether attending to these issues months earlier would serve these patients or their families well.

Indeed, what is it that one might aim to improve by ensuring that the patient and family have an accurate picture of the patient’s situation? If a major problem is that medical experts offer largely worthless treatments at that point, more direct means might be applied to solve that problem. It may be harsh or unreasonable to expect dying patients and troubled families to attend to our statistics and to make sense of the frustratingly uncertain characterizations of the likely effects of each alternative treatment, including palliative care.

Certainly, we agree with Lamont and Christakis that physicians should be wary of claims that optimism and hope prolong survival, but physicians should also be wary of assuming that patients and families prefer to confront their fate early and often. They also should avoid making the assumption that they know what patterns of care best serve their patients and families without more research. Moreover, better prognostic information in the hands of patients and families may not alter the course of care.

Power of the “Usual Course of Care”

Two facts from SUPPORT make it likely that habitual patterns are much stronger than patient choice. First, our efforts to provide education and information to patients, families, and physicians were utterly ineffectual.[1] Second, the location of death correlated strikingly with the availability of hospital beds.[5] Together, these findings make it reasonably likely that the usual course of care reflects long-standing accommodations between physiology, human sentiment, and care system arrangements. The power of the “usual course of care” seems likely to outweigh the usual patient preferences, which are variable and lack enduring commitment, even by the patient.

Thus, practitioners should guard against optimism and error but should also aim for effective reform. Reformers should directly diminish the availability of ineffective treatments and oversupplied hospital beds or specialists and should enhance the availability of palliative care and comprehensive services from diagnosis through death and bereavement. Innovation and responsible evaluation are essential to guide effective and enduring reforms.

References:

1. The SUPPORT Principal Investigators: A controlled trial to improve care for seriously ill hospitalized patients: The study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT). JAMA 274: 591-1598, 1995.

2. Knaus WA, Harrell FE, Lynn J, et al: The SUPPORT prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Ann Intern Med 122:191-203, 1995.

3. Weeks JC, Cook EF, O’Day SJ, et al: Relationship between cancer patients’ predictions of prognosis and their treatment preferences. JAMA 279:1709-1714, 1998.

4. Morris JN, Suissa S, Sherwood S, et al: Last days: A study of the quality of life of terminally ill cancer patients. J Chronic Dis 39:47-62, 1986.

5. Pritchard RS, Fisher ES, Teno JM, et al: Influence of patient preferences and local health system characteristics on the place of death. J Am Geriatr Soc 46:1242-1250, 1998.