Historical and Methodological Perspectives on Cancer Outcomes Research

Outcomes research is the study of the net effects of the health care process on the health and well-being of individuals and populations. It encompasses a wide breadth of issues, including measurement of patient preferences and health status, broadly referred to as quality of life. Evaluation of health-related quality of life in research studies has been facilitated by the development of a number of measurement tools. In addition to general health tools, cancer-related tools are available, some of which include cancer site-specific or symptom-specific measures. Preference assessment, from the perspective of the patient or general population, is necessary to incorporate quality of life into economic analyses. Various techniques are available to assign preference values to outcomes; metrics such as quality-adjusted life-years (QALYs) are then used to combine quality and quantity of life into a usable value for economic analyses. In the future, quality of life and economic measurements should be incorporated into phase III trials, effectiveness trials, and observational studies.

Introduction

The most crude measure of health outcome is vital status-alive or dead. For many diseases, such as cancer, curative treatment is not always possible. Still, the outcomes of noncurative care may be very different. For instance, therapeutic strategies may be associated with similar survival but different toxicities; alternatively, one therapy may yield better survival but more severe side effects, while another treatment may offer poorer survival but better quality of life during the patient's remaining months or years. Thus, decisions about alternative therapies are often based on quality of life considerations, in addition to the likelihood of survival [1,2]. Even when cures are possible, the costs of treatment may exceed an individual's or society's willingness to pay.

Outcomes research is the study of the net effects of the health care process on the health and well-being of individuals and populations. As such, it encompasses a wide breadth of issues, including research on practice patterns, effectiveness, appropriateness of care, and measurement of patient preferences and health status. Thus, outcomes research can best be seen as a discipline that endeavors to ask which treatment works best, under which circumstances, for which individuals, and at what cost.

The growing interest in the costs and quality of health care during the past decade has contributed to the dramatic growth of the field of outcomes research [3]. Clinicians, consumers, employers, insurance companies, and government agencies are increasingly looking to the results of outcomes research to assist them in determining how to get the most value for their health care efforts. For comprehensive reviews of outcomes research, the interested reader is referred to several excellent sources [4-7].

This paper concentrates on the measurement of health status and preferences for oncology-related health outcomes, broadly referred to as "quality of life" (QOL). Following a summary of the historical evolution of QOL measurement in cancer research, we will de scribe the tools commonly used in the measurement of quality of life and the major methodological issues that investigators or consumers of QOL research should consider. Measurement of patient and population preferences for health outcomes and incorporation of these preferences into economic evaluations of cancer interventions are also highlighted. We conclude with recommendations for future cancer outcomes research.

Evolution of Cancer Outcome Measures

The most general disease outcome measure is death. For cancer, 5-year survival or the interval of disease-free survival has customarily been used to evaluate the success of treatment. Clinical events, such as severity of illness, tumor response, or stage shifts, have served as intermediate measures of outcome, principally because they are believed to be associated with differences in survival. For example, in the landmark Health Insurance Plan (HIP) mammography screening trial, downstaging of breast cancer was initially used to assess effectiveness [8,9]. Proxy measures of intermediate clinical outcomes have included events such as number of consultations, days in the intensive care unit, and need for blood transfusions [10].

While quality of life has been an implied medical outcome since the time of Hippocrates [3,11], the landmark paper by Karnofsky marked the first explicit effort to systematically assess the impact of cancer treatment on the patient's quality, and not quantity, of life. The Karnofsky performance index uses a scale from 0% to 100% for physician rating of functional status, with 100% reflecting ability to carry on normal activities and 0% representing death. The index was originally used in conjunction with subjective symptoms and tumor response to evaluate the use of nitrogen mustards as palliative cancer therapy [12]. Although the Karnofsky scale was a seminal contribution, it does not meet today's standards for validation and demonstration of instrument reliability (see "Measurement and Analysis Issues" section) .

The next major tools designed to assess the impact of cancer therapy on quality of life were not developed until the early 1980s. In the prototype, the Spitzer Quality of Life Index, QOL continued to be physician-rated [13]. Thus, physician or researcher assessment of symptoms, toxicities, and/or quality of life remained the standard in cancer outcomes research for almost 4 decades.

Increasing consumerism and patient participation in health care decisions, occurring in parallel with the growth in interest in outcomes of care in the late 1970s and 1980s, set the stage for the development of patient-based measures of general and cancer-related quality of life [3,14]. For example, in 1976, Ware and colleagues [15] presented results validating a patient self-reported measure of general health status from the Rand Health Insurance Experiment. Other patient-rated general health QOL measures developed in this period include the Sickness Impact Profile (SIP) [16,17], Psychosocial Adjustment to Illness Scale (PAIS) [18,19], and Nottingham Health Profile [20].

In parallel with the development of these patient-rated general health QOL measures, patient assessment tools for measuring the quality of life of cancer patients began to appear during the past 20 years; preliminary incorporation of such measures into cooperative group, randomized, controlled trials occurred in the late 1980s [3,21]. Examples of the instruments that have been used (and will be discussed below) include the LASA-P [22], Padilla Quality of Life Index [23], EORTC-QOL [24,25], CARES [26,27], FLIC [28], and, most recently, the Functional Assessment of Cancer Therapy (FACT) [29]. Cancer site-specific tools have also been developed in this same time period.

Despite this explosion of QOL measures, incorporation of QOL outcomes into medical research has been slow, and, when used, outcomes have often been poorly measured. For instance, in a recent review of QOL measurement across a variety of medical conditions, Gill and Feinstein [30] found that 159 different measures of quality of life were used in the 75 articles reviewed. Despite the large number of measures, fewer than half defined the target domains, only 17% included a patient rating of quality of life, and just 9% elicited patients' preferences for health outcomes. In phase III breast cancer randomized controlled trials, as another example, only 4% of trials published from 1985 to 1989 and 6% published from 1990 to 1994 included any QOL assessment; only one study included a measure of patient preference [31].

Conceptualization and Definition of QOL

The current construct of quality of life draws on earlier use of social indicators, such as housing and employment status, to measure the well-being of populations [32-34]. Breslow expanded the measurement of population well-being to include the WHO definition of health as "not only the absence of disease or of disability, but an overall state of physical, mental, and social well-being." [35] More recently, De Haes defined quality of life as "the subjective evaluation of life as a whole" [36]; Cella and Cherin expanded this definition to include patients' satisfaction with their level of functioning, compared with an ideal level [37]. Both of these latter definitions reflect the importance of the individual's subjective experience; the second highlights the role of personal values, or preferences, for health outcomes [11]. Further discussion of the conceptual underpinnings of quality of life is contained in several excellent sources [4,5,7,38-41].

The current consensus among outcomes researchers is that quality of life is a multidimensional concept [11,42-47]. Quality of life can be further specified as those domains of living that could be affected by treatment for cancer or other disease conditions. The term health-related quality of life (HRQOL) is used to refer to this latter component of overall quality of life [4]. Although the number and types of HRQOL domains suggested by different researchers vary, there is substantial overlap. Most would agree that HRQOL measurement should include assessment of at least three of the following domains: (1) somatic concerns, such as pain and symptoms; (2) functional ability; (3) family well-being; (4) emotional well-being; (5) spirituality; (6) treatment satisfaction, including financial impact of illness; (7) future orientation; (8) sexuality, intimacy, and body image; (9) social functioning; and (10) occupational functioning [4,11,46,48-50]. The precise number and types of domains should be sufficiently broad to capture the impact of the illness or treatment on the patient, but not impose an undue burden on the patient or researcher; if the impact is largely unknown, the greatest number of dimensions should be included.

A 1990 National Cancer Institute-sponsored workshop on the measurement of quality of life in cancer clinical trials recommended that QOL assessment be multidimensional, include general and cancer-specific tools, be patient self-reported, be measured at more than one point in time, and be evaluated controlling for relevant medical and sociodemographic patient characteristics [46]. Although the workshop recommended that patient ratings should be self-administered when possible, interviewer administration might be preferred for elderly, low-literacy, or moderately or severely ill groups.

Current Tools for HRQOL Measurement

This section will briefly review the currently available tools for measurement of HRQOL, including general health measures that can be applied to patients with different clinical conditions, generic cancer-specific instruments, cancer site-specific instruments, and symptom-specific measures. This discussion is not intended to be exhaustive. For a more in-depth discussion of these and other instruments, the reader is referred to several reviews [11,39,40,46,49,51-53]. Many of the instruments have been published and/or reviewed in one source document to facilitate researcher access [4,54-57].

General Tools-General health measures can be useful in cancer research because they have usually been validated in large and varied populations, and thus allow for comparison of cancer patients with other patient or population groups [11]. The Sickness Impact Profile (SIP) was one of the earliest general health status measures [16,17]. It is a 136-item tool that can be interviewer- or self-administered; it yields 12 dimension-specific scores-mental function (alertness and emotional behavior), ambulation, body care and movement, mobility, communication, eating, home management, recreation and pastimes, sleep and rest, social interactions, and work-and one global score of quality of life. The major advantages of the SIP are its comprehensiveness, applications to large populations and diverse disease states, ability to detect post-treatment changes in functioning, and known psychometric properties; the disadvantage is its length [4,11].

The original instruments developed for the Rand Health Experiment [15,58] and the subsequent Medical Outcomes Study (MOS) [59-61] have been modified to yield two "short forms" (SF-20 and SF-36). The MOS instruments use patient self-reports and include six domains: physical functioning, social functioning, role functioning, mental health, health perceptions, and pain. Each domain is scored separately and there is no global score; summary global scoring is currently being developed. These scales have excellent reliability and validity in healthy populations and groups with a variety of chronic diseases. They can be self- or interviewer-administered in person or over the telephone, and foreign language translations are also available.

The Nottingham Health Profile is a brief self-administered tool using a simple yes/no format measuring perceived health problems and their impact on function [20]. This instrument is most sensitive to severe impairment and may not detect differences in mildly impaired patients. The Psychological Adjustment to Illness Scale (PAIS and PAIS-SR) uses patient-ratings on 46 items covering seven domains (health care orientation, vocational environment, domestic environment, sexual relationships, extended family relationships, social environment, and psychological distress); provides dimension-specific and overall scores; and has been used with cancer patients [18,19].

Additional patient-rated general QOL measures include the Health Utilities Index (versions Mark I, II, III, and 15Q) [6,62-64], and the EuroQol [65-67]. The Health Utilities Indices (HUI) are multiattribute utility measures that include several domains: mobility and ability to get around, physical function, self-care, role activity, emotional well-being (memory, happiness, anxiety, and depression), pain and discomfort, social activity, and health problems (from physical deformity to sensory impairment). The EuroQol instrument, developed by a consortium of investigators from Western Europe, defines 14 health states using six domains (mobility, self-care, main activity, social relationships, pain, and mood), to yield a possible 216 health states. The instrument was originally designed to be self-administered; telephone versions are currently being evaluated. Both the HUI and the EuroQol include preference (utility) scale measures, in which individuals assign a preference "weight" or "utility" to their health state. The HUI uses a scale from 0 to 1, with 0 representing death and 1 perfect health; the EuroQol scale is from 0 to 100, with zero representing worst health and 100 perfect health (see below for a further discussion of preference/utility measurement).

Physician-rated general health instruments include the Quality of Well-Being Scale, the Disabilities/Distress Index, and the Index of Health-Related Quality of Life. The Quality of Well-Being Scale includes measures of well-being and prognosis, and was developed for use in health planning activities [68,69]. Well-being represents HRQOL and includes 23 broad symptom complexes combined with three attributes of functional status (impairment in mobility, physical activity, and role functioning) used to generate health states. These health states have associated utility scores (on a scale from 1 to 10) derived from the general population [70].

The Disability/Distress Index was originally developed in England to measure the performance of hospitals [71,72]. It is a physician-rated measure and includes two components: observed disability and subjective distress. The original 32 combinations of disability and distress have recently been expanded to include 176 health state combinations of disability, distress, and discomfort. The Index of Health-Related Quality of Life [52] is a similar measure containing 10 potential combinations of 107 descriptors [73,74].

Generic Cancer-Related Tools-Many instruments have been developed to measure QOL among cancer patients, reflecting the concern that generic health measures may not be sufficiently responsive to changes associated with a cancer diagnosis. For example, a recent report on the quality of life of prostate cancer patients found that general HRQOL measures did not fully capture important differences in quality of life between men with and without cancer, controlling for levels of sexual and urinary dysfunction [75]. Thus, cancer-specific tools aim at measuring symptoms and the effects of treatment on the day-to-day life of a cancer patient. Some of the most commonly utilized tools include two quality of life indexes, the Q-TWiST (quality-time without symptoms or toxicity), the CAncer Rehabilitation Evaluation System (CARES), the Functional Living Index-Cancer (FLIC), the European Organization for Research and Treatment of Cancer Quality of Life (EORTC-QOL), and the Functional Assessment of Cancer Therapy (FACT).

The Quality of Life Index developed by Spitzer et al represents one of the earliest cancer-specific instruments [13]. Five observer-rated domains are included as single items, along with a rating of global health. Unfortunately, the observer ratings have poor correlation with patient ratings; however, this tool might be useful when the patient is too ill to respond and a proxy is required. A similar tool was developed by Padilla and colleagues as a patient-rated instrument [23]. It has the advantage of being brief and includes three domains (physical, psychological, and symptoms) covered in 14 items.

In the Q-TWiST, physicians rate quality of life over specified periods of time, such as time on adjuvant chemotherapy or time with recurrence, and use these weights in survival analyses comparing outcomes from different treatment options [76-79].

CARES is a patient self-reported multidimensional instrument that includes five domains (physical, sexual, and psychosocial function, and medical and marital interactions) and a global rating of quality of life [27]. Earlier versions of CARES were called the Cancer Inventory of Problem Situations [26]. The long form of the current instrument contains 139 items; the short form has 59 items (CARES-SF) [80]. Both versions function well in diverse ambulatory populations with cancer, discriminate among patients with different cancer types and varying cancer stages [81,82], and have been translated into Spanish [83].

FLIC is a multidimensional instrument with 22 self-reported items, including items about cancer treatment-specific side effects [28,84]. The item scores are summed for a single global score. This instrument has been shown to have good psychometric properties with ambulatory patients; it functions less well for the more severely disabled [85].

The EORTC Core Quality of Life instrument, developed by a cooperative panel for inclusion in cancer clinical trials, contains a general 30-item multidimensional (physical, role, cognitive, emotional, social, and global function, and cancer-related symptoms) self-rating of cancer-related quality of life [24,25] and site-specific cancer modules [86]. The tool has the advantage of being brief, easy to administer, including many domains, and being available in several languages.

Another recently developed tool, FACT, was also designed for use in cancer clinical trials [29]. Like the EORTC-QOL, the FACT includes a 38-item general set of questions encompassing several dimensions and several nine-item cancer site-specific modules. The FACT also builds in a preference/utility approach. A Spanish language translation is currently being developed.

Cancer Site-Specific Tools-Evaluation of specific cancer site-related quality of life has expanded rapidly in the past decade. As mentioned above, several instruments, such as the EORTC-QOL and the FACT, include cancer site-specific measures. Others have been developed specifically to assess site-specific treatment toxicities and quality of life. Three measures with known psychometric properties are summarized here; others can be identified through reviews of the current literature.

Two breast cancer tools have been used in recent clinical trials-the Breast Cancer Chemotherapy Questionnaire and the Linear Analogue Self-Assessment (LASA) for Breast Cancer. The Breast Cancer Chemotherapy Questionnaire [87] uses 30 self-rated items to evaluate the toxicity of adjuvant chemotherapy in a manner similar to the symptom checklists described below. The LASA for Breast Cancer is a physician-rated measure of quality of life with 18 items taken from the SIP and 13 breast-cancer-specific items [88]. The LASA-P extends this measure by using a visual analogue scale for patient-rated quality of life [22].

The Performance Parameter for Head and Neck Cancer is an observer-rated tool developed to capture the speech, swallowing, and body image/social interaction difficulties particular to head and neck cancer patients [89].

Cancer Symptom-Specific Tools-This class of instruments has been developed to examine the impact of adverse effects of cancer treatments, including chemotherapy and radiotherapy, on patient quality of life. These tools are often used in cancer clinical trials. The most widely utilized is the Rotterdam Symptom Checklist, a 34-item self-report instrument of known psychometric properties measuring physical function and psychological distress [90].

Other symptom-based scales used in oncology research include the Symptom Distress Scale [91,92], the Memorial Pain Assessment Card [93], and the Morrow Assessment of Nausea and Emesis (MANE) [94].

Summary-Thus, a plethora of instruments is currently available for the evaluation of quality of life; more can be expected to be developed in the future as the interest in quality of life as a distinct health outcome evolves. The ultimate choice of a particular instrument depends on many factors, including the goals of the study, the resources available, the psychometric properties of the tool, the sample size needed, the language and culture of the target population, and prior use in the population of interest [11,46,49]. These and other methodological aspects of QOL measurement are presented in the next section.

Measurement and Analysis Issues

In selecting an instrument to measure quality of life, there are several methodological concerns that should be addressed in any research setting, including validity and reliability, use of translated tools, modification by sociodemographic variables and comorbidity, control for baseline status, timing of measurement, and use of proxy responses. A number of statistical issues should also be considered, such as handling of missing data and informative censoring, calculation of power and sample size, adjustment for multiple significance testing, analyses of repeated measures, scaling and scoring, and need for aggregation over individuals. These areas are addressed in the NCI Consensus Workshop on Quality of Life [46] and other reviews [11,49]. Each will be briefly highlighted below.

Methodological Concerns-Reliability has two components: the internal consistency of the scale items and the within-individual reproducibility over time. Likewise, validity includes two components: internal and external validity. The internal validity of a measure refers to the ability to measure what the investigator intends to measure, and the ability of the items of the tool and the interpretation of their score to "make sense" (face and construct validity) [47].

A related concern for internal validity is whether the instrument has the ability to "predict" the hypothesized outcome (predictive validity). For example, a QOL instrument should be able to discriminate between individuals with different quality of life, such as women with early-stage breast cancer and women with advanced metastatic disease, and be able to detect changes over time in the quality of life in an individual [82].

Another concern of internal validity is the responsiveness of the instrument to detect ceiling or floor effects (ie, when patients with the highest possible scores improve or those with the lowest possible scores deteriorate) [86,95,96]. External validity is concerned with how well the results from one group will apply to other groups.

Existing instruments should be accompanied by data about their reliability and validity; newly developed tools should include a full evaluation of their psychometric properties. There are several available reviews of the properties of existing instruments used in cancer research [11,46,49,54,57].

Selection of an instrument should also be guided by practical considerations, while maintaining reliability and validity. The total number of items should be sufficient to capture important domains but not overburden the data collectors and respondents; lengthy instruments are more likely to have missing data elements than shorter tools because they are more tedious and time consuming to complete [46].

Simple translation of an existing tool into another language is rarely adequate to ensure reliability and validity in the new setting. At a minimum, translation and back-translation (translation back into english or whatever was the original language) are suggested. However, even with a good translation, cultural differences may still adversely affect the instrument's properties and require extensive validation experience before implementation [47,52].

Sociodemographic variables, such as marital status, education, income, and race, have been noted to relate to both cancer survival [97-99] and cancer quality of life [100]. Comorbid medical conditions will also affect quality of life and survival outcomes. For example, Ware and colleagues [15] have noted that QOL response patterns vary across patient groups with different coexistent medical conditions, such as diabetes, arthritis, and depression [46]. Thus, it is suggested that QOL researchers measure sociodemographic and health factors; availability of such data will allow for control of potential confounding variables, as well as calculation of results for relevant subgroups of the study population.

The timing of QOL assessment is important. At a minimum, quality of life should be evaluated at baseline, prior to treatment, and at one follow-up point. Baseline data can be used to predict survival, to measure change over time controlling for initial status, and to describe the study patients so that others can evaluate the results in the context of their populations. It has been recommended that the baseline measurement occur prior to randomization, if possible, to avoid bias due to knowledge of treatment assignment and differences in time between randomization and initiation of treatment [46].

Timing of subsequent evaluations depends on the goals of the study, the expected response patterns for treatments, and the natural history of the disease. For example, if short-term treatment toxicities are of interest, quality of life should be assessed during or immediately after treatment has been administered; if long-term quality of life and survival are the endpoints of the trial, measurement should occur over the relevant periods to detect late-occurring treatment effects. The number of follow-up points should be adequate to capture all relevant outcomes, but not be so great as to lead to increases in missing data.

The place where the instrument is administered is also an important consideration. Clinic or office administration may be convenient for study personnel, but may yield biased estimates of quality of life due to anxiety or social desirability (eg, giving a response that patient's think the medical staff want to hear or would approve of).

Proxy responses generally have poor correlation with patient responses, whether the proxy is a physician or family member [46,101,102]. However, proxy data may be considered in situations when the patient is too ill to respond, when the patient is expected to decline in physical or cognitive function over the time of the trial, or when the patient is a child. In these situations, the availability of proxy responses may be preferable to having missing data or having a different respondent for different phases of the study. If it is anticipated that proxy data will be used, the researcher should attempt to measure both patient and proxy response over several points in time. If the responses are fairly well correlated, the proxy response can be used when the patient is no longer able to complete the QOL instrument. Baseline assessments should not be based on proxy responses; inability to complete baseline QOL measures should be considered as an exclusion criterion [46].

Statistical Issues-Statistical issues in QOL measurement, such as handling of missing data, sample size calculations and significance levels, repeated measures, and scoring, are also germane to understanding the proper use of QOL in clinical research [46]. This discussion is not meant to be comprehensive but to highlight common concerns; statistical consultation is recommended when designing QOL research projects.

Missing data can occur either as missing individual items or complete missing data for an evaluation point. Complete missing data can result from a variety of situations, including patient refusal, failure to locate the patient, or patient illness or death. The latter two scenarios are especially relevant for cancer researchers. In these situations, it is likely that respondents with complete data have a substantially different quality of life than the nonrespondents, leading to a biased estimation of the impact of treatment or other intervention on true quality of life.

If baseline data have been obtained and follow-up data are missing, potential bias can be evaluated by looking for baseline differences between patients with complete and patients with miss ing information. If no important differences are seen, a variety of methods of imputing missing data points can be used. For example, if specific items are missing on a completed questionnaire, then the instructions of the designer of the instrument can be followed. If entire instruments are missing, techniques that can be used to model outcomes include the Q-TWiST strategy, assignment of average scores for similar patients, or performing survival analyses on time-to-event data, using declines in QOL scores of a specified amount or death, whichever comes first, as the endpoint. In any case, the amount of and reasons for missing data should be included in the presentation of results.

When QOL assessment is to be incorporated into a clinical trial or other research protocol, the original estimation of sample size should consider the power to detect a specified difference in QOL score(s). If quality of life is one of many clinical outcomes of interest, the sample size will need to be adjusted upward to allow for appropriate multivariate analyses. Likewise, if dimension scores are used individually as variables (ie, as with the MOS) and not combined into a summary rating score, each dimension needs to be considered as a separate variable for the purpose of sample size calculation. Lastly, when targeting an initial sample size, the investigators should consider the numbers of patients anticipated to be lost to follow-up. This a priori attention to sample size estimation will allow the researcher to be more confident about having adequate power to detect clinically relevant differences in quality of life between patients in all treatment arms.

A related concern for the analysis of QOL outcomes, particularly when domain-specific scores are used, is the possibility of multiple statistical testing. In this setting, since one or more results may be statistically significant by chance alone using standard alpha levels (ie, P = .05), significance levels should be adjusted for the number of comparisons (eg, using the Bonferroni correction method). Under these conditions, large samples are likely to be necessary for adequate power. An alternative approach is to specify a small number of primary hypotheses for the main analyses, reducing the sample size needed. For example, the major hypothesis of a cancer chemotherapy trial might be that patients in arm X will have higher scores on a physical functioning scale than patients on arm Y at point Z in time.

If quality of life is measured at more than one point in time, as recommended above, the use of repeated measures must be considered in the analysis. For example, if quality of life at the end of the trial is the variable of interest, this measure is not independent of baseline quality of life. The general approach to this area of analysis will be dictated by the research hypotheses: Is the investigator concerned with the change in quality of life over time or with the quality of life at a particular point? Repeated measures (using ANOVA or ANCOVA) can be used to estimate within-individual and inter-individual variation, evaluate trends across multiple measurement times, and adjust for covariates. These approaches assume that data are missing in a random manner; as noted above, this is not always the case.

Scaling and scoring of QOL instruments require a variety of assumptions. In terms of scaling, individual levels of times may not constitute a natural hierarchy from least desirable to most desirable (ie, able to walk 1 mile, able to walk across a room; wheelchair bound). In overall scoring, domains may interact to produce different QOL results. For example, satisfaction with health may be low, although the scores for a physical function domain may be high [103].

Drawing on psychometric traditions in psychology, the standard approach to producing an aggregate QOL score is to sum the numerical rating for the items (usually rated on a Likert scale) across all domains. This method assumes that the individual (and society) values each domain equally. Thus, equal-interval scales and aggregate scores may not fully capture preferences for health outcomes. Preference-weighting methods have been developed to address this concern. Summary scores (either preference or nonpreference weighted) can also be used in combination with length of time in the given state to yield measures of "quality-adjusted life-years" (QALYs) (see below).

Preferences for Outcomes (Utility Measurement)

Having measured several domains or health states comprising health-related quality of life, it becomes necessary to evaluate how important a given dimension is to the individual (the individual's preference, or "utility") and how that preference changes over time or by age of the patient [104]. This section discusses the theoretical and historical underpinnings of utility measurement, summarizes techniques used to assign utilities to QOL outcome states, and highlights the incorporation of preferences into economic evaluations of oncologic interventions.

Theoretical Underpinnings-The theoretical constructs underlying the process of valuing health outcomes are derived from neoclassical theory of preference, vonNeumann-Morgenstern utility, and principles of risk. Briefly, the neoclassical theory of preference, which dates back to the turn of the century, demonstrated that one could derive a theory of "choice under certainty" by referring to individual preferences, and that preferences could be measured on a scale that assigned a number to an outcome. The normative choice would then be the outcome that had the highest value on the scale; this value was referred to as the "utility." In the 1940s, vonNeumann and Morgenstern extended these concepts to address how individuals make decisions under conditions of uncertainty ("game theory") [105]. These and other theoretical constructs provide the underpinnings for several of the preference assessment methods discussed below [106,107].

Preference assessment is necessary to incorporate quality of life into economic analyses; this use reflects its relevance to populations. In certain situations, conclusions about the value of an outcome for the population may be contrary to that for the individual [11,108]. For example, a woman may prefer to incur the risks and toxicities of bone marrow transplantation for a low probability of cure of advanced breast cancer; from the societal perspective, the cost effectiveness of this technology may exceed an acceptable threshold expenditure of cost for a given health gain. This discussion will consider the individual perspective, with the societal viewpoint highlighted when pertinent.

Preference or severity weights can be assigned using a variety of approaches, including researcher or expert judgment, published judgments, inferred social values, or direct patient, family, or population assessment. Patients are commonly used as the source of preference weights, assigning a value to their own state of health. Patients are actually experiencing the outcome under question, making them good judges of the value of the state. However, if the outcome is being evaluated from the societal perspective, it is sometimes recommended that the general population (without the disease) be the judge of the value of the outcome. Not surprisingly, individuals with and without the condition or disease will value that health state differently [103]. For example, a healthy person might rate quality of life as a paraplegic as a 3 on a 0 to 10 scale, while an individual with spina bifida might be quite happy and rate quality of life as a paraplegic as a 7.

A related issue is whether an individual can adequately assess a health state that he or she has not experienced [4]. In addition, for patients or general population groups, preferences for health states vary as a function of time. For instance, in the Beaver Dam Health Outcomes Study, preferences for health states changed as longitudinally followed respondents aged [109]. However, preference choices seem to be fairly consistent across different gender, race, health, and socioeconomic status groups [70,110-112].

A contemporary example of the use of an expert panel to incorporate societal preference weights into economic analyses is represented by the setting of priorities for health interventions considered for insurance coverage in Oregon [113].

In general, in preference/utility measurement, researchers should consider the perspective of those who would make the decision about alternative diagnostic or treatment strategies-in essence the consumers of the treatment or intervention. For cancer, this approach suggests that the values of outcomes should generally be determined by the patients themselves. Some suggest that in the case of costly treatments, such as bone marrow transplants, the perspective of the insurance industry or the general population should be considered. Others suggest that for most cost-effectiveness analyses, the societal perspective is preferable. The selection of the group whose preferences, or utilities, will be used to measure health outcomes may govern the strategy that appears to be most effective or most cost effective. Therefore, some analysts measure outcomes according to more than one perspective. This approach makes the policy and ethical issues more explicit, and the assessment less biased, than when only one perspective is used.

Techniques to Assign Values to Outcomes-Once the perspective, or judge of values, is defined, the next task involves the selection of a technique to assign values to the relevant outcome states. Several methods have been developed for this purpose, including rating scales, willingness to pay, standard gamble, and time tradeoffs. These are briefly mentioned here; in-depth discussions of these approaches can be found in a variety of sources [4,107,114,115], and in Dr. Weeks' paper in this supplement.

Rating scale techniques are used in several current health outcome index measures, such as the HUI, Quality of Well-Being (QWB), and EuroQol. The scale is an interval scale, usually anchored at death and perfect health. The bottom anchor of "worst health" has also been used [65-67], reflecting the fact that individuals may consider certain health states to be "worse than death." Including negative ratings below death also captures this construct. Individuals are asked to place their current health state on the scale; these ratings are then converted to a "utility" score that can be used to quality adjust the time spent in the health state (see below).

Willingness-to-pay methods assign a monetary value to a given outcome by asking questions such as, how much would you be willing to pay to avoid hair loss associated with chemotherapy or how much would you be willing to pay to reduce the likelihood of this outcome? Other approaches focus on a nonmonetary unit of measure to value outcomes, usually a unit on a scale from 1 to 10 or from 0 to 100.

The standard gamble technique draws on utility theory and asks individuals, given a particular health state, what risk of death (the gamble) they would accept in exchange for restoring perfect health.

The time tradeoff method of valuing outcomes was developed by Torrance and colleagues [114]. Using this approach, the subject is asked to evaluate the relative desirability of two outcomes, such as perfect health for 5 years compared to, say, living with a below-the-knee leg amputation for 20 years. This technique has been used to evaluate surgical treatments for prostate cancer by asking patients to tradeoff rates of survival and impotency associated with surgery or radiotherapy [116].

Two recently published studies of cancer patients serve as examples of the application of the aforementioned utility measurement techniques. In the first, using hypothetical scenarios to evaluate preferences for cancer treatment, and drawing on standard gamble and time tradeoff methods, Yellen and colleagues evaluated cancer treatment preferences as a function of patient age [117,118]. In that study, older and younger patients were equally likely to agree to aggressive cancer treatment, but older patients were found to be less willing than their younger counterparts to trade treatment toxicity for extended survival. McQuellen and colleagues extended these scenarios to elicit the preferences of breast cancer patients for toxicity versus life expectancy improvements associated with adjuvant chemotherapy [119].

The technique used to measure preferences for health outcomes should have face and construct validity and be reproducible, comparable across interventions, and easily administered. All of the aforementioned preference assessment techniques require fairly extensive interviews and a patient group that can comprehend the complexity of the task. Thus, the validity of rating health outcomes and states will depend on the degree to which the raters understand the states they are evaluating. It is often difficult to rate a health state directly; paired comparisons between states can also be used [120]. For example, rather than asking how the subject would rate life as a paraplegic, say on a scale from 1 to 10, one could ask how the subject would rate paraplegia compared to having one leg amputated. The time tradeoff technique described above is a type of paired comparison. Related to validity concerns, preference measurement may reflect factors other than health considerations, such as risk aversion and general life values [121]. Despite these caveats, the ultimate goal of QOL research is to measure outcomes that are important to individuals; at present, this is an inexact science.

Combining Quality and Quantity-In order to incorporate quality of life into economic analyses, such as cost-effectiveness or cost-utility analyses, it is necessary to combine the duration of life in different health states with the state's associated preference or utility weight. Ideally, cost-effectiveness analyses would be conducted in a uniform manner by all analysts, calculating the ratio of the differences in costs of the interventions being compared to the differences in their health outcomes, with outcomes represented as a combined standardized measure of quality and quantity of life.

The most common technique used to combine quality and quantity of life is quality-adjusted life-years (QALYs). (Other terms that are synonyms for QALYs include well-years, well life expectancy, health life years, and years of healthy life.) This approach assumes an equity principle, ie, that the resulting QALYs are equally valuable regardless of who gains them and at what stage of life they are gained. That is, a QALY gained for an infant is the same as a QALY gained by an elderly individual.

Other approaches to combining quality and quantity of life include using disability-adjusted life-years, healthy year equivalents, years of healthy life, full healthy life, saved young life equivalents, and health-adjusted person-years. For disability-adjusted life years (DALYs), the severity weight for given health states are estimated by a panel of experts [122]. This approach was developed by the World Bank to measure the global burden of disease and the effectiveness of health interventions, as indicated by reductions in disease burden. DALYs are calculated as the value of disability-free life-years that are lost as the result of premature death or disability. Healthy year equivalents (HYEs) use standard-gamble and/or time-tradeoff methods, and have been proposed as a more complete representation of the choices individuals make about their health states, since they involve valuation of both the relevant health states and the duration of life [123]. However, this two-stage technique requires more data collection than other approaches. The years of healthy life (YHL) is a related metric developed at the National Center for Health Statistics; this tool provides researchers with a catalogue of age-specific, utility-based measures of life expectancy for the US population [124].

For a discussion of the remaining currently proposed approaches, such as full healthy life, saved young life equivalents, and health-adjusted person-years, the reader is referred to other sources [4].

The uptake of quality-adjusted measures into economic evaluations of oncology outcomes has been slow; the majority of research to date has relied on physicians' estimations for preference weighting [125-130].

Recommendations for Future Research

Based on this review of the past and present state-of-the-art in measurement of quality of life in health outcomes research, we recommend, as much as is feasible, that the next phase of clinical oncology research development do the following:

1. Incorporate QOL measurement into phase III randomized controlled trials, effectiveness trials, and observational research studies.

2. Expand patient-rated measures of cancer-specific and cancer site-specific quality of life.

Develop practical measures of individuals' preferences for health outcomes.

Assess variations in quality of life and preferences by age, race, gender, culture, general health status, and/or social class.

3. Refine methods of incorporating quality of life and preferences into economic analyses of oncology interventions.

Evaluate the ability to use general health QOL measures to compare the resources used in cancer prevention, detection, diagnosis, and treatment with resources associated with interventions for other health conditions.

Compare general health QOL measures with cancer-specific and cancer site-specific measures.

4. Study and elucidate the ethical issues raised by the implications of quality of life for clinical decision making, equity, distribution of resources, empowerment, and self-determination.

References:

1. Selby P, Robertson B: Measurement of quality of life in patients with cancer. Cancer Surv 6(3):521-543, 1987.

2. Mazur DJ, Hickam DH: Patient preferences: survival vs quality-of-life considerations. J Gen Intern Med 8:374-377, 1993.

3. Strain JJ: The evolution of quality of life evaluations in cancer therapy. Oncology 4(5):22-27, May 1990.

4. Patrick DL, Erickson P: Health Status and Health Policy. Quality of Life in Health Care Evaluation and Resource Allocation. New York, Oxford University Press, 1993.

5. Baldwin S, Godfrey C, Propper C (eds): Quality of Life: Perspectives and Policies. London, Routledge, 1990.

6. Drummond MF, Stoddard GL, Torrance GW: Methods for the Economic Evaluation of Health Care Programmes. New York, Oxford University Press, 1987.

7. Fallowfield L: The Quality of Life: The Missing Measurement in Health Care. London, Souvenir Press, 1990.

8. Shapiro S, Strax P, Venet L, and Venet W: Periodic Screening for Breast Cancer: The Health Insurance Plan Project and Its Sequelae, 1963 - 1986. Baltimore, MD, The Johns Hopkins University Press, 1980.

9. Chu KC, Connor RJ: Analysis of the temporal patterns of benefits in the Health Insurance Plan of Greater New York trial by stage and age. Am J Epidemiol 133:1039-1049, 1991.

10. Horn SD, Sharkey PD, Bertram DA: Measuring severity of illness: Homogeneous case mix groups. Med Care 21:4-25, 1983.

11. Ganz PA: Quality of life measures in cancer chemotherapy, methodology and implications. PharmacoEcon 5:376-388, 1994.

12. Karnofsky DA, Abelmann WH, Craver LF, et al: The use of the nitrogen mustards in the palliative treatment of carcinoma. Cancer 1:634-656, 1948.

13. Spitzer WO, Dobson AJ, Hall J, et al: Measuring the quality of life of cancer patients. A concise QL-index for use by physicians. J Chronic Dis 34:585-597, 1981.

14. Heithoff KA, Lohr KN: Effectiveness and Outcomes in Health Care. Washington, DC, National Academy Press, 1990.

15. Ware JE, Johnston SA, Davies-Avery A, et al: Conceptualization and measurement of health for adults in the health insurance study. Santa Monica, CA, Rand Corporation, 1979.

16. Bergner M, Bobbitt RA, Kressel S, et al: The Sickness Impact Profile: Conceptual formulation and methodological development of a health status index. Int J Health Serv 6:393-415, 1976.

17. Bergner M, Bobbitt RA, Carter WB, et al: The Sickness Impact Profile: Development and final version of a health status measure. Med Care 19:787-805, 1981.

18. Derogatis LR, Lopez M: Psychosocial Adjustment to Illness Scale (PAIS & PAIS-SR): Scoring, Procedures & Administration Manual: I. Baltimore, Clinical Psychometric Research, 1983.

19. Derogatis LR: The Psychosocial Adjustment to Illness Scale (PAIS). J Psychosom Res 30:77-91, 1986.

20. Hunt S, McKenna SP, McEwen J, et al: The Nottingham Health Profile: Subjective health status and medical consultations. Soc Sci Med 15A:221-229, 1981.

21. Holland JC, Silberfarb P, Tross S, et al: Cancer and Leukemia Group B Experience, Brookline, MA, USA, in Ventafridda V et al (eds): Assessment of Quality of Life and Cancer Treatment: Proceedings of the International Workshop, November 1985, International Congress Series, vol 702, pp 89-101. Elsevier Science Publishers B.V. (Biomedical Division), 1986.

22. Priestman TJ, Baum M: Evaluation of quality of life in patients receiving treatment for advanced breast cancer. Lancet 1:899-901, 1976.

23. Padilla GV, Presant C, Grant MM, et al: Quality of life index for patients with cancer. Res Nurs Health 6:117-126, 1983.

24. Aaronson NK, Bullinger M, Ahmedzai S: A modular approach to quality-of-life assessment in cancer clinical trials. Recent Results Cancer Res 111:231-249, 1988.

25. Aaronson NK, Ahmedzai S, Bergman B, et al: The European Organization for Research and Treatment of Cancer Quality of Life Q-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85:365-376, 1993.

26. Schag CC, Heinrich RL, Ganz PA. Cancer Inventory of Problem Situations: An instrument for assessing cancer patients rehabilitation needs. J Psychosocial Oncol 1:11-24, 1983.

27. Schag CAC, Heinrich RL: Development of a comprehensive quality of life measurement tool: CARES. Oncology 4 (5):135-138, May 1990.

28. Schipper H, Clinch J, McMurray A, et al: Measuring the quality of life of cancer patients: The Functional Living Index-Cancer: Development and validation. J Clin Oncol 2:472-483, 1984.

29. Cella DF, Tulsky DS, Gray G, et al: The Functional Assessment of Cancer Therapy (FACT) scale: Development and validation of the general measure. J Clin Oncol 11:570-579, 1993.

30. Gill TM, Feinstein AR: A critical appraisal of the quality of quality-of-life measurements. JAMA 272:619-626, 1994.

31. Mandelblatt JS: Historical and methodological perspectives on cancer outcomes research. Presented at the 1995 Quality of Life Symposium: Economic and Quality of Life Outcomes in Oncology. St. Mary Medical Center, Long Beach, Calif, April 4, 1995.

32. Andrews FM, Withey SB: Social Indicators of Well Being: Americans' Perception of Life Quality. New York, Plenum, 1976.

33. Campbell A, Converse P, Rodgers WL: Quality of American Life: Perceptions, Evaluations and Satisfaction. New York, Russell Sage Foundation, 1976.

34. Campbell A: The Sense of Well-Being in America: Recent Patterns and Trends. New York, McGraw-Hill, 1981.

35. Breslow L: A quantitative approach to the World Health Organization definition of health: Physical, mental and social well-being. Int J Epidemiol 1:347-355, 1972.

36. De Haes JCJM: Quality of life: Conceptual and theoretical considerations, in Watson, et al (eds): Psychosoc Oncol, pp 61-70. Oxford, Pergamon press, 1988.

37. Cella DF, Cherin EA: Quality of life during and after cancer treatment. Compr Ther 14:69-75, 1988.

38. Walker SR, Rosser RN (eds): Quality of Life Assessment: Key Issues in the 1990s. London, Kluwer Academic Publishing, 1992.

39. Spilker B: Quality of Life Assessments in Clinical Trials. New York, Raven Press, 1990.

40. Spilker B, Molinek FR Jr, Johnson KA, et al: Quality of life bibliography and indexes. Med Care 28:DS1-DS77, 1990.

41. Patrick DL, Erickson P: What constitutes quality of life? Concepts and dimensions. Clin Nutr 7:53-63, 1988.

42. Hollandsworth JG Jr: Evaluating the impact of medical treatment on the quality of life: A five year update. Soc Sci Med 26:425-434, 1988.

43. Aaronson NK: Quality of life: What is it? How should it be measured? Oncology 2 (5):69-74, May 1988.

44. Aaronson NK: Quality of life assessment in clinical trials: Methodologic issues. Controlled Clin Trials 10:195S-208S, 1989.

45. De Haes JCJM, Van Knippenberg FCE: The quality of life of cancer patients: A review of the literature. Soc Sci Med 20:809-817, 1985.

46. Nayfield SG, Hailey BJ, McCabe M: Quality of Life Assessment in Cancer Clinical Trials. Report of the Workshop on Quality of Life Research in Cancer Clinical Trials, July 16-17, 1990. Bethesda, MD, U.S. Department of Health and Human Services, NCI, NIH, 1991.

47. Guyatt GH, Feeny DH, Patrick DL: Measuring health-related quality of life. Ann Intern Med 118:622-629, 1993.

48. Patrick DL, Deyo RA: Generic and disease-specific measures in assessing health status and quality of life. Med Care 27(suppl):S217-S232, 1989.

49. Cella DF, Tulsky DS: Measuring quality of life today: Methodological aspects. Oncology 4 (5):29-38, May 1990.

50. Feeny D, Guyatt G, Patrick D (eds): Proceedings of the International Conference on the Measurement of Quality of Life as an Outcome in Clinical Trials. Controlled Clinical Trials: Design Methods, and Analysis 12:1S-280S, 1991.

51. Moinpour CM, Hayden K, Thompson IM: Quality of life end points in cancer clinical trials: Review and recommendations. J Natl Cancer Inst 81:485-495, 1989.

52. Guyatt GH, Veldhuyzen Van Zanten SJ, Feeny DH, et al: Measuring quality of life in clinical trails: A taxonomy and review. Can Med Assoc J 140:1441-1448, 1989.

53. Lohr KN: Advances in health status assessment: Conference proceedings. Med Care 30:1-293, 1992.

54. Tchekmedyian NS, Cella DF: Quality of life in current oncology practice and research. Oncology 4 (5):21-232, May 1990.

55. Clearing House on Health Indexes: Bibliography of Health Indexes. National Center for Health Statistics, Hyattsville, MD.

56. BRS Information Technologies: Health and psychosocial instruments (HAPI). Oxford, New York, Oxford University Press, 1993.

57. Scott J, Erickson P: The On-Line Guide To Quality-Of-Life Assessment (OLGA). Oxford, New York, Oxford University Press, 1993.

58. Brook RH, Ware Jr JE, Davies-Avery A, et al: Overview of adult health status measures fielded in RAND's Health Insurance Study. Med Care 17(suppl):1-131, 1979.

59. Stewart AL, Hays RD, Ware JE Jr: The MOS short-form general health survey: Reliability and validity in a patient population. Med Care 26:724-735, 1988.

60. Stewart AL, Greenfield S, Hays RD, et al: Functional status and well-being of patients with chronic conditions: Results from the Medical Outcomes Study. JAMA 262:907-913, 1989.

61. Stewart AL, Ware JE: Measuring Functioning and Well-Being: The Medical Outcomes Study Approach. Durham, NC, Duke University Press, 1992.

62. Torrance GW, Thomas WH, Sackett DL: A utility maximization model for evaluation of health care programs. Health Serv Res 7:118-133, 1972.

63. Torrance GW, Zipursky A: Cost-effectiveness of antepartum prevention of Rh immunization. Clin-Perinatol 11:267-281, 1984.

64. Ontario Ministry of Health: An overview of the survey content of the Ontario Health Survey. Ontario Health Survey Bulletin, 1990.

65. EuroQol Group: EuroQol-A new facility for the measurement of health related quality of life. Health Pol 16:199-208, 1990.

66. Brooks RG, Jedteg S, Lindgren B, et al: EuroQol: Health-related quality of life measurement. Results of the Swedish questionnaire exercise. Health Pol 18:37-48, 1991.

67. Nord E: EuroQol: Health related quality of life measurement: Valuations of health states by the general public in Norway. Health Pol 18:25-36, 1991.

68. Fanshel S, Bush JW: A health-status Index and its application to health-services outcomes. Oper Res 18:1021-1066, 1970.

69. Patrick DL, Bush JW, Chen MM: Towards an operational definition of health. J Health Soc Behav 14:6-23, 1973.

70. Kaplan RM, Bush JW: Health-related quality of life measurement for evaluation research and policy analysis. Health Psychol 1:61-80, 1982.

71. Rosser RM, Watts VC: The measurement of hospital output. Int J Epidemiol 1:361-368, 1972.

72. Rosser RM: Issues of measurement in the design of health indicators: A review, in Culyer AJ (ed): Health Indicators: An International Study for the European Science Foundation, pp 34-81. New York, St. Martin's Press, 1983.

73. Rosser RM, Walter SR: A Health index and output measure, in Walter SR, Rosser RM (eds): Quality of life: Assessment and Application. Lancaster, England, MTP Press, 1988.

74. Rosser RM: From health indicators to quality-adjusted life years: Technical and eth ical issues, in Rosser RM, Hopkins A, Costain B (eds): Measuring the Outcomes of Medical Care, pp 1-17. London, Royal College of Physicians, 1990.

75. Litwin MS, Hays RD, Fink A, et al: Quality-of-life outcomes in men treated for localized prostate cancer. JAMA 273:129-135, 1995.

76. Gelber RD, Goldhirsch A: A new endpoint for the assessment of adjuvant therapy in postmenopausal women with operable breast cancer. J Clin Oncol 4:1771-1779, 1986.

77. Gelber RD, Goldhirsch A: Methods for assessing treatment efficacy in trials for adjuvant therapy for breast cancer. Recent Results Cancer Res 115:211-219, 1989.

78. Gelber RD, Goldhirsch A, Cavalli F: Quality-of-life-adjusted evaluation of adjuvant therapies for operable breast cancer. Ann Intern Med 114:621-628, 1991.

79. Goldhirsch A, Gelber RD, Simes RJ, et al: Costs and benefits of adjuvant therapy in breast cancer: A quality-adjusted survival analysis. J Clin Oncol 7:36-44, 1989.

80. Schag CAC, Ganz PA, Heinrich RL: CAncer Rehabilitation Evaluation System-Short Form (CARES-SF): A cancer specific rehabilitation and quality of life instrument. Cancer 68:1406-1413, 1991.

81. Ganz PA, Schag CAC, Lee JJ, et al: The CARES: A generic measure of health-related quality of life for patients with cancer. Qual Life Res 1:9-29, 1992.

82. Ganz PA, Coscarelli, Schag CA, et al: Breast conservation versus mastectomy: Is there a difference in psychological adjustment or quality of life in the year after surgery? Cancer 69:1729-1738, 1992.

83. Canales S, Ganz PA, Schag CAC: Translation and validation of a quality of life instrument for Hispanic American cancer patients: methodological considerations. Qual Life Res 4(1):3-11, 1995.

84. Morrow GR, Lindke J, Black P: Measurement of quality of life in patients: Psychometric analyses of the Functional Living Index-Cancer (FLIC). Qual Life Res 1:287-296, 1992.

85. Ganz PA, Haskell CA, Figlin RA, et al: Estimating the quality of life in a clinical trial of patients with metastatic lung cancer using the Karnofsky performance status and the Functional Living Index-Cancer. Cancer 61:849-856, 1988.

86. Sprangers MA, Cull A, Bjordal K, et al: The European Organization for Research and Treatment of Cancer approach to quality of life assessment: Guidelines for developing questionnaire modules. EORTC Study Group on Quality of Life. Qual Life Res 2:287-295, 1993.

87. Levine MN, Guyatt GH, Gent M, et al: Quality of life in stage II breast cancer: An instrument for clinical trials. J Clin Oncol 6:1798-1810, 1988.

88. Selby PJ, Chapman JAW, Etazadi-Amoli J, et al: The development of a method for assessing the quality of life of cancer patients. Br J Cancer 50:13-22, 1984.

89. List MA, Ritter-Sterr C, Lansky SB: A performance status scale for head and neck cancer patients. Cancer 66:564-569, 1990.

90. De Haes JCJM, Van Knippenberg FCE, Neijt JP: Measuring psychological and physical distress in cancer patients: Structure and application of the Rotterdam Symptom Checklist. Br J Cancer 62:1034-1038, 1990.

91. McCorkle R, Young K: Development of a symptom distress scale. Can Nurs 1:373-378, 1978.

92. McCorkle R, Benoliel JG: Symptom distress, current concerns and mood disturbance after diagnosis of life-threatening disease. Soc Sci Med 17:431-438, 1983.

93. Fishman B, Pasternak S, Wallenstein SL, et al: The Memorial Pain Assessment Card. Cancer 60:1151-1158, 1987.

94. Morrow GR: A patient report measure for the quantification of chemotherapy induced nausea and emesis: Psychometric properties of the Morrow Assessment of Nausea and Emesis (MANE). Br J Cancer 19:S72-S74, 1992.

95. Bindman AB, Keane D, Lurie N: Measuring health changes among severely ill patients: The floor phenomenon. Med Care 28:1142-1152, 1990.

96. Ganiats TG, Palinkas LA, Kaplan RM: Comparison of Quality of Well-Being scale and Functional Status Index in patients with atrial fibrillation. Med Care 30:958-964, 1992.

97. Bassett MT, Krieger N: Social class and black-white difference in breast cancer survival. Am J Publ Health 76:1400-1403, 1986.

98. Axtell IM, Myers MH: Contrasts in survival of black and white cancer patients. J Natl Cancer Inst 60:1209-1215, 1978.

99. Berg JW, Ross R, Latourette HB: Economic status and survival of cancer patients. Cancer 39:467-477, 1977.

100. Mor V: Cancer patients' quality of life over the disease course: Lessons from the real world. J Chronic Dis 40:535-544, 1987.

101. Brody DS: Physician recognition of behavioral, psychosocial, and social aspects of medical care. Arch Intern Med 140:1286-1289, 1980.

102. Slevin ML, Plant H, Lynch D, et al: Who should measure quality of life, the doctor or the patient? Br J Cancer 57:109-112, 1988.

103. Patrick DL, Peach H (eds): Disablement in the Community. Oxford, England, Oxford University Press, 1989.

104. Hays RD, Stewart AL, Sherbourne CD, et al: The 'states vs weights' dilemma in quality of life measurement. Qual Life Res 2:167-168, 1993.

105. vonNeumann J, Morgenstern O: Theory of Games and Economic Behavior. Princeton, NJ, Princeton University Press, 1944.

106. Torrance GW: Toward a utility theory foundation for health status index models. Health Serv Res 11:349-369, 1976.

107. Torrance GW: Utility approach to measuring health-related quality of life. J Chronic Dis 40:593-600, 1987.

108. Dean HE: Political and ethical implications of using quality of life as an outcome measure. Semin Oncol Nurs 6:303-308, 1990.

109. Fryback DG, Dasbach EJ, Klein BE, et al. The Beaver Dam Health Outcomes Study: Initial catalog of health-state quality factors. Medical Decision Making 13:89-102, 1993.

110. Sackett DL, Torrance GW: The utility of different health states as perceived by the general public. J Chronic Dis 31:697-704, 1978.

111. Wolfson AD, Sinclair AJ, Bambardier C, et al: Preference measurements for functional status in stroke patients: Inter-rater and inter-technique comparisons, in Kane RL, Kane RA (eds): Values and Long Term Care, pp 191-214. Lexington, MA, Lexington Books, 1982.

112. Balaban DJ, Sagi PC, Goldfarb NI, et al: Weights for scoring the quality of well-being instrument among rheumatoid arthritics: a comparison to general population weights. Med Care 24: 973-980, 1986.

113. Hadorn D: Setting health care priorities in Oregon: Cost-effectiveness meets rule of rescue. JAMA 265:2218-2225, 1991.

114. Torrance GW: Social preferences for health states: An empirical evaluation of three measurement techniques. Socio-Econ Plan Sci 10:129-136, 1976.

115. Torrance GW, Boyle MH, Horwood SP: Application of multi-attribute utility theory to measure social preferences for health states. Operations Res 30:1043-1069, 1982.

116. Singer PA, Tasch ES, Stocking C, et al: Sex or survival: Trade-offs between quality and quantity of life. J Clin Oncol 9:328-334, 1991.

117. Yellen SB, Cella, Leslie WT: Age and clinical decision making in oncology patients. J Natl Can Inst 86:1766-1770, 1994.

118. Weeks J: Preferences of older cancer patients: Can you judge a book by its cover? J Natl Can Inst 86:1743-1744, 1994.

119. McQuellen RP, Muss HB, Hoffman SL, et al: Patient preferences for treatment of metastatic breast cancer: A study of women with early-stage breast cancer. J Clin Oncol 13:858-868, 1995.

120. Thurstone L: The Measurement of Value. Chicago, University of Chicago Press, 1959.

121. Wilson IB, Cleary PD: Linking clinical variables with health-related quality of life. JAMA 273:59-65, 1995.

122. World Bank: World Development Report 1993: Investing in Health. New York, Oxford University Press, 1993.

123. Mehrez A, Gafni A: Quality-adjusted life years, utility theory, and healthy-years equivalent. Medical Decision Making 9:142-149, 1989.

124. Erickson P, Wilson R, Shannon I: Years of Healthy Life. Statistical Notes. Number 7, 1995. US Department of Health and Human Services, PHS, Centers for Disease Control and Prevention, National Center for Health Statistics, Hyattsville, Maryland. DHHS Publication No. (PHS) 95-1237 4-1485 (4/95).

125. Hillner BE, Smith TJ: Efficacy and cost effectiveness of adjuvant chemotherapy in women with node-negative breast cancer. N Engl J Med 324:160-168, 1991.

126. Hillner BE, Smith TJ, Desch CE: Efficacy and cost-effectiveness of autologous bone marrow transplantation in metastatic breast cancer. JAMA 267:2055-2061, 1992.

127. Smith TJ, Hillner BE, Desch CE: Efficacy and cost-effectiveness of cancer treatment: Rational allocation of resources based on decision analysis. J Natl Can Inst 85:1460-1474, 1993.

128. Desch CE, Hillner BE, Smith TJ, et al: Should the elderly receive chemotherapy for node-negative breast cancer. J Clin Oncol 11:777-782, 1993.

129. Smith TJ, Hillner BE: The efficacy and cost-effectiveness of adjuvant therapy of early breast cancer in premenopausal women. J Clin Oncol 11:771-776, 1993.

130. Schulman KA, Yabroff KR: Measuring the cost effectiveness of cancer care. Oncology 9(6):523-533, June 1995.