Choosing Measures to Evaluate the Quality of Cancer Care

ABSTRACT: This is the first in a series of reports on presentations from "Ensuring Quality Cancer Care," a symposium held in Chicago and sponsored by the Robert H. Lurie Comprehensive Cancer Center of Northwestern University and the VA Chicago Health Care System. The reports, which put the discussions into a broader context, have been prepared for ONI by researchers at Northwestern University, working under the direction of Dr. Charles L. Bennett.

In 1999, the Institute of Medicine released an important report entitled "Ensuring Quality Cancer Care" that emphasized the importance of providing patients with quality cancer care and establishing frameworks with which to measure the care that patients receive. The issues raised by this report are critical for determining the future of the health care system.

Katherine Kahn, MD, of the RAND Corporation, is a national leader in the area of quality of medical care, with recent efforts focusing on breast cancer care.[1] Dr. Kahn shared her findings at the "Ensuring Quality Cancer Care" symposium of the Robert H. Lurie Comprehensive Cancer Center of Northwestern University and the VA Chicago Health Care System.

Defining Quality of Care

To measure quality of cancer care, it is essential to consider the definition of quality of health care. The quality of health care can be loosely defined according to the framework of the structures providing health care, the processes involved in caring for patients, and the outcomes of the care that patients receive.

In evaluating structure, researchers often assess the organization of the provider, the patient’s access to the provider, and the comprehensiveness of the care provided by clinicians.

Interactions between patients and different health care providers within the system are the source of both objective and subjective data that can be used to evaluate health care processes.

One example of a process measure is the time from an abnormal Pap smear to a diagnostic resolution of the results from the smear. Subjective data could include the patient’s impressions of the clinician and the tone of their discussion before and after the Pap smear. Objective data might include the length of time between the laboratory’s interpretation of the test results and the communication of these results to the health care provider.

However, while process measures are generally useful for measuring quality of care, Dr. Kahn suggests that evaluating the appropriateness of the care along with process measures can lead to a much richer picture of the patient’s experience within the health care system.

Measuring outcomes of health care, the third component of evaluating quality of health care, involves measuring the effects of the health care process on health. Mortality is one fairly simple outcome to measure. For example, 5-year survival rates and the length of remission are common health outcomes used in evaluating cancer care.

However, measurement of outcomes may also encompass a wide variety of patient characteristics, including the clinical, financial, psychological, and social impact of the health care process on the patient. A quality-of-life (QOL) assessment of patients can include all of these different factors and gives patients an opportunity to provide qualitative feedback to their health care provider and institution.

Data from QOL assessments may affect the care that future patients receive. Results from these types of measurements can also be combined with other measures, such as the cost of care for the patient, provider, and institution, to analyze the cost-effectiveness of certain treatments for these different entities in the health care system.[1]

Challenges in Designing Studies

In her presentation, Dr. Kahn emphasized that it is important to consider how to evaluate the complex interactions among the various components of the health care system.

She stated that researchers encounter many challenges in designing a study to measure the quality of cancer care.

First, the study must determine if a problem in care exists. A baseline measurement of factors and continued surveillance of these factors will allow researchers to detect a deviation from the standard of care.

Second, once a problem is detected, the researchers are faced with the delicate task of attributing the problem to a combination of patient, provider, and hospital characteristics.

A misattribution of the problem to a health care provider could result in many patients needlessly switching their provider, creating added hassles for the patient and removing patients from a provider who provides quality care.

Finally, researchers must carefully consider the initiatives they suggest for quality improvement. The data from the study and resulting intervention must ultimately produce consistent long-term changes, rather than effects that are short lived.

Although quality of health care studies have tremendous potential for improving the ways in which health care is delivered, Dr. Kahn demonstrates that designing an effective quality of health care study demands careful attention.

Hospital Death Rates Study

In the mid-1980s, the Health Care Financing Administration (HCFA) released annual reports about hospital death rates for Medicare patients in an effort to identify hospitals that provided low quality of care. In response to this effort, Dr. Kahn collaborated with several other researchers and clinicians in a study that examined the variation in hospital death rates at a variety of acute-care hospitals in the United States.

Previously, other groups had found that while death rates varied by hospital, these differences were often attributable to differences in the severity of illness or level of comorbidities in the patient populations.[2]

Dr. Kahn and her collaborators evaluated whether hospitals with high age-, sex-, race-, and disease-specific death rates provided lower quality of care to patients than hospitals with lower death rates.[2]

They also evaluated the relative contributions of severity of illness and quality of care in statistical models that addressed an individual’s in-hospital probability of death.

Retrospective medical record review was used to find information on congestive heart failure and acute myocardial infarction, which comprised 17.5% of Medicare hospital deaths.

The results indicated that hospitals with high death rates do not provide a lower quality of care than hospitals with lower death rates: 25% of the difference in death rates among hospitals was attributed to the higher average severity of illness of patients with acute myocardial infarctions in higher-death-rate hospitals. However, differences in severity of illness were not correlated with the higher death rates of patients with congestive heart failure.

Higher death rates at certain hospitals may be related to other factors besides quality of care such as longer average inpatient stay.

From this study, the authors concluded that it is difficult to accurately assess differences in hospital care, even when using carefully constructed measures of quality of care.

Key Issues

Dr. Kahn reviewed some key issues that may influence the rates of certain quality measures. For example, in the case of screening rates for a type of cancer, including women with symptoms (diagnostic evaluations) could increase the rate of screening. Many outcome measures may also vary with patient and population characteristics such as age, co-morbidity, access to screening, treatment choices, eligibility for treatment, and family history of cancer.

A researcher would expect survival to be better among women who have had continuous screenings for breast cancer throughout their lives, compared with women who rarely had mammograms.

Patient surveys can be misleading. For example, if only the most functional breast cancer patients respond to a survey about QOL measures, then the population will appear to be healthier.[1]

Outcome measures and process measures may need to be adjusted for many of these factors after data collection. These factors should also be taken into account when selecting the study population and determining the methods to be used for data collection.

Some settings may not have sufficient numbers of cancer cases to yield an accurate estimate of a certain measure. For instance, it would be difficult to assess accurately the quality of cancer care offered by a physician who sees only one or two cancer cases per year.

One solution to this problem could be to include data from a number of years or to include data from similar sized practices and similar patient populations.

Case identification through medical record review at a particular hospital may result in a very different yield than cancer patient identification using a national tumor registry.

It is also important to evaluate measures chosen for a study for reliability and validity. If a QOL measure does not increase as a patient’s clinical condition improves, the QOL assessment may not accurately reflect the patient’s mental and physical health.[1]

It is preferable to use measurements that have predictive validity, internal consistency, and good face validity. However, the only way to determine if certain measures are reliable and valid is to evaluate the measures themselves through repeated studies.

A High Priority

Ensuring quality cancer care for patients has become a high priority for health care providers and policy makers. To determine how to improve cancer care, clinicians and researchers must first select measures with which to evaluate the quality of care that is currently provided to individuals.

More initiatives are needed to address the numerous challenges inherent in developing these measures and ultimately evaluating the many different aspects of care. By devoting additional resources and consideration to this area, it is likely that both the quantity and quality of life of cancer patients will be improved.

References:

1. Mandelblatt JS, Ganz PA, Kahn KL: Proposed agenda for the measurement of quality-of-care outcomes in oncology practice. J Clin Oncol 17:2614-2622, 1999.

2. Park RE, Brook RH, Kosecoff JK, et al: Explaining variations in hospital death rates: Randomness, severity of illness, quality of care. JAMA 264:484-490, 1990.