Economic Analysis During Phase III Clinical Trials: Who, What,When, Where, and Why?

With the recent surge in interest in health care reform and the growth of managed care organizations, the cost of care has become a major determinant of the types and intensity of therapy that patients receive. If data on the cost effectiveness of new treatments are not collected during phase III trials, diffusion of new technologies may be restricted by managed care companies and insurers. We can collect the data needed to evaluate the costs and cost effectiveness of new therapies, as well as quality of life data, in phase III trials. But to collect these data effectively, many methodologic questions must be answered. For example, who needs to be involved in an economic analysis of a medical treatment; what data should be collected, when, and from whom; what data collection sources should be used; and how should the data be collected and validated? To embark on economic studies of new therapies, it is imperative that these issues be addressed early, during the design phase of a trial, and a course of action determined.

Introduction

Medical care is expensive-cancer care alone accounted for $35 billion in 1990 [1]. In an era of health care reform, costs and cost-effectiveness estimates are taking on a larger role in the decision to use a new therapy. However, American efforts at quickly evaluating the cost effectiveness of new therapies lag behind the progress in Canada and Australia, where evidence of both clinical effectiveness and cost effectiveness is required before new drugs are allowed to be marketed [2,2a]. In the past 5 years, medical research journals have begun to publish articles on the costs of care of many therapies. However, few of these studies included early assessments of costs of care, especially during phase III trials. Because cost-effectiveness information is typically gathered after phase III trials and FDA approval, physicians have postponed asking the question, "Can we afford this new therapy?" Insurers, formulary committees, physicians, and health maintenance organizations (HMOs) must make policy decisions in the absence of objective data on cost effectiveness.

The cost effectiveness of a new therapy is only one of the concerns that need to be addressed when designing phase III trials. A therapy that is very effective and provides a higher quality of life or long-term survival requires more than a cost-effectiveness analysis. Alternative therapies, especially standard therapies, must also be studied. While studies on costs of care during clinical trials are appropriate to guide the development of more cost-effective therapeutic and supportive care interventions, such considerations should not diminish enthusiasm for investigation of new therapies that may prove very expensive and possibly not cost effective.

In the past year, the Cancer and Leukemia Group B (CALGB) and the Eastern Cooperative Oncology Group (ECOG) have formed task forces to study the economics of cancer therapies. These groups will provide oncologists with economic analyses as well as objective evidence of the clinical benefits and toxicities of new cancer agents. Current efforts are evaluating costs of care and cost-effectiveness of new technical procedures such as laparoscopic colectomy, supportive care such as hematopoietic growth factors, and alternative infusion schedules and chemotherapeutic agents (intermittent higher dose vs continuous infusion lower dose of 5-FU).

This article reviews the methodologic issues that should be considered during the planning phase of phase III clinical trials so that economic analyses can be incorporated. We do not intend to review the field of modeling of clinical trial results. The reader is referred to the review of Smith et al for information in this area [1], as well as to Drs. Smith and Hillner's article in this supplement.

We address our comments to the following basic questions: How are costs of care and cost effectiveness defined as they pertain to economic analysis? What sources of data should be used? Who should be involved in economic analysis? When should economic analysis be considered? How should the data be collected and for how long of a period of time? Where should these studies be conducted? What are the possible pitfalls in doing economic analyses of phase III clinical trials?

What Are "Costs of Care" and "Cost Effectiveness"?

The costs of care can be defined as the economic burden therapy creates in monetary terms. Costs can be broken into two main categories: direct medical costs and indirect costs (see Table 1). Indirect costs include all of the supportive costs to therapy such as lost wages, traveling time, and lost income due to death. Direct medical costs account for resources spent on items such as laboratory tests, antibiotics, radiologic and other diagnostic procedures, and in-home care.

Although indirect costs may account for up to 60% of a family's income, they are rarely included in the economic analysis of phase III trials [3]. Collecting data on indirect costs can be methodologically difficult or even impossible. In addition, many patients are elderly and do not work; hence, their indirect costs differ markedly from those of persons who miss work when receiving time-consuming therapies such as chemotherapy. However, in some cases it is important not to discount indirect costs when viewing the procedure as a whole. This may be especially important in clinical trials that involve patients or family members who lose time from work.

Other costs, including marginal costs, avoidable costs, variable costs, and fixed costs, are also important for economic analyses. The reader is referred to the review article by Bennett et al for in-depth definitions of these terms [3].

In evaluating costs of care in a clinical trial, results must be generalizable so that cost comparisons can be made among different types of hospitals, health care systems, and even countries. Broad evaluations based on reviews of medical charges have been reported in some studies but are likely to be inadequate for several reasons [4]:

1. Many patients are covered by prepaid plans. Their portion of the medical charge is not representative of the costs. Also, each plan might negotiate a different charge rate for a particular service. Consequently, collecting data from providers on the charges that they bill will produce inaccurate figures. Charges for care may differ significantly even for patients who receive similar care at the same hospital.

2. Many patients accrue very high charges, but hospitals actually collect only a fraction of the bills.

3. Charges for separate departments of a hospital are often set by hospital administrators in such a way that profits from some cost centers subsidize losses from other cost centers.

4. Charges represent local factors such as rent, wages, and costs of ancillary supportive care efforts, which are likely to vary significantly in different geographic settings.

A more accurate method of computing costs of care might be to collect data on "cost drivers." For example, room and board, pharmacy, diagnostics, blood bank, and professional services account for 90% of the costs of autologous stem cell transplantation [5]. To determine the differences between charges and costs, a standard such as the Medicare ratio of costs to charges (RCCs) can be used [6]. To estimate the reasonable and necessary costs of providing care, Medicare uses diagnosis-related groups, as well as detailed annual reports from hospitals. Medicare fees are then determined by use of costs-to-charges ratios rather than the full hospital charge (see Table 1).

The selection of a definition of costs is important for several reasons. In the context of a clinical trial, it may determine the source of data collection. If resources are to be measured, then careful inventories of all resources must be carried out. However, if charges are to be evaluated via the costs derived from ratios of costs to charges, then all bills for each patient must be collected.

After determining the costs of a therapy, its cost effectiveness (or "money for value") can be determined. Many definitions of cost effectiveness are used in analyses of clinical trials. A standard cost-effectiveness analysis takes into account the added years of life a person lives due to the treatment, and is expressed in terms such as dollars per life-years gained (see Table 1).

A cost-utility analysis incorporates factors associated with quality of life during treatment. A utility value can be determined by using patient or provider surveys that incorporate time trade- off questions (ie, how many years of perfect health would you be willing to trade off in order to live a bit longer but in your current state of health?). This value expresses the period of time living with the disease that is equivalent to living a shorter period of time with perfect health. Cost-utility estimates are expressed in dollars per quality-adjusted life years (QALYs) saved, and highlight the importance of improving both the quality of life and the quantity of life with new advances in therapy or supportive care (see Table 1).

Cost-benefit analysis is controversial and not used much in the United States but is worth mentioning. Cost-benefit analysis converts benefits into dollars. For instance, if a lawyer makes $150,000 per year and the cost of a therapy is $100,000 per life-year gained, the cost benefit is $50,000 per life-year. Cost benefit can be markedly different for varying socioeconomic groups. Although placing a dollar value on a year of life is controversial, physicians can use cost-benefit analyses to help determine the benefit of a therapy (see Table 1).

To determine cost effectiveness, cost utility, or cost benefit, two therapies are compared. In a clinical trial, one therapy might be a placebo and the other a new medication, or two existing therapies might be compared-for example, chemotherapy vs allogeneic bone marrow transplantation for acute nonlymphocytic leukemia [7]. Although many phase III clinical trials are based on studies of a new therapy vs a placebo, the relevant economic analysis needs to be based on a comparison of the new therapy vs the market leader (which is never a placebo).

Who Should Be Involved in Costs of Care Research?

Economic analysis of clinical trials is based on theoretical and methodologic research in several disciplines such as health services, economics, accounting, psychology, and clinical medicine (see Table 2). These analyses will improve if investigators can apply and adapt the principles of these fields to clinical trials. However, they can only improve if researchers with expertise in these areas provide input as the trial is being designed. Clinicians with expertise in health services are especially valuable additions. They know where the medical resources are being used, are familiar with patient and physician decision-making processes, and are able to provide needed input into protocol design, data collection methods, analysis, and interpretation of the results.

Accountants with backgrounds in health care are vital to determine the standard costs of procedures and products. Psychologists with expertise in quality of life research direct studies aimed at interpreting patient utility values. Economists with a background in health care economics can then use cost data along with utility figures to extrapolate cost effectiveness and cost utility. Finally, with the growth of medical information systems in hospitals, outpatient offices, and HMOs, the possibility of collecting data electronically in databases becomes feasible. Also important will be the ability to track resources across multiple providers (both inpatient and outpatient) and to integrate financial data, clinical data, laboratory results, pharmacy information, data from tumor registries, and pathology information. Therefore, information management personnel trained in the health care field will become an integral part of the cost-analysis team.

When Should Economic Analysis Be Considered?

Costs of care analyses are more relevant in some phase III clinical trials than in others (see Table 3). We outline a framework for identifying which types of studies are a priority for economic analysis. Economic data are of the highest priority in clinical trials in which this information is likely to be a major determinant of whether or not physicians use a particular agent [8]. Cost considerations provide support for choosing one therapy over another, adding to traditional considerations such as survival, disease-free survival, and response rate [9]. For costs of care to be a major determinant of physician practice patterns, two conditions must hold: (1) The quality-adjusted survival differences between the two alternatives are expected to be small, and (2) moderate to large differences are expected in costs of care.

Even when economic assessments of costs of care are not major factors in choosing a treatment strategy, economic data may contribute to the study design in other ways. The inclusion of cost analyses should be considered in the context of health care reform issues. The data may help policymakers answer the question, "Is a new therapy priced too high, given its likely clinical benefits profile?" Information about costs of care could help policymakers decide whether to encourage a pharmaceutical company to lower the price of a new agent or face significant obstacles to its use in general practice. For example, intravenous immunoglobulin is useful for preventing infection in patients with chronic lymphocytic leukemia (CLL). However, it was found to be prohibitively expensive, costing $6 million per life-year saved, and therefore policymakers view it unfavorably [8].

Economic data can also influence reimbursement decisions by insurers and managed care companies. For instance, ongoing studies of autologous bone marrow transplant (ABMT) programs as adjuvant therapy for breast cancer may indicate that ABMT is within the accepted range of increased cost or is more cost effective over time, compared with standard treatment, despite much higher initial treatment costs [10]. Finally, global decisions, such as those seen in the Oregon Medicaid program, are based on cost-effectiveness estimates and are used to determine which therapies are reimbursable. In this program, therapies and procedures with cost-effectiveness profiles less than a predetermined threshold level will be reimbursed, while those with cost-effectiveness profiles higher than the threshold level will not [11,12].

Questions About Data Collection: How, What, How Often, and Which Patients?

There are many possible sources for economic data collection, including medical records, patient bills, and cooperative group forms (see Table 4). Which data should be collected depends on many variables. These include the budget and resources of the study (including personnel), whether the treatment is a subset of a larger treatment, which resources are expected to change, and the time frame of the study. Concerns regarding the validity of the data must also be addressed.

Physician input is critical for determining which data should be collected. A hypothesis as to which resources will change with the new therapy has to be devised before a data collection plan can be formed. For example, supportive care therapies such as hematopoietic growth factors undoubtedly increase most pharmaceutical budgets, but cost offsets are likely to be found with respect to hospitalization time and laboratory tests [5]. Usually a subset of charges, such as those related to length of hospitalization, medication, x-rays, and blood tests, are representative of the total costs of inpatient procedures. These data can be collected from patient bills, hospital charts, or hospital billing systems.

When representative charges are not appropriate for the study, data collection managers can potentially gather data from patient charts. This method is effective for a study in which only a small number of resources may vary due to the new treatment. For example, consider a study whose goal is to compare the costs of catheter-infused fluorouracil vs intravenous fluorouracil. A data collector could extract from patient charts the number of catheter lines placed and the resources used for subsequent complications, such as infection or blood clot. However, to ensure seamless data collection, the data managers must not be overburdened. Additionally, if the study setting is a managed care organization, the possibility of using management information personnel to collect data needs to be evaluated. This is because these organizations control which physicians, hospitals, medications, and pharmacies are used, and all bills are paid centrally. Therefore, their financial databases serve as complete sources of information.

Special procedures to ensure data quality need to be derived. Coordination of retrieval of economic data from multiple providers and sources within a given hospital is complex. Extensive training sessions are needed prior to initiation of the study to ensure uniformity of data collection approaches. Research assistants must be instructed in hierarchical approaches to data collection when two alternative sources of resource utilization may exist and one source is preferred over the other. When missing data appears to be a problem, solutions must be devised that address alternative sources of data. Finally, coordination of the research assistant with the project director and the physician collaborator is important. Data sources often require sensitive inquiries to the medical records departments or financial management systems of individual hospitals. Close cooperation among the study investigators, research assistants, and organizations leaders allows these inquiries to be carried out more effectively.

The study designer must also consider which patients to include in the data collection. In some trials, all patients should be included in the economic analysis, so that the results can be generalized for other populations. However, in other trial settings, data collection on all patients may be too expensive or too difficult to measure, and only a subset of patients would need to be included in the economic analysis to achieve statistical significance.

Another important consideration is the length of time an economic analysis is designed to cover (see Table 5). Because medical costs are incurred over long periods of time, assessments have the potential to detect savings in future costs at the expense of higher short-term costs or vice versa. Having stated this, one must realize that long-term studies of costs of care pose difficulties in logistics, design, and analysis. Also, it is important to determine if the treatment patterns have remained consistent over time. For example, most hospitals embarking on new protocols pass through a learning curve period [13]. During this period, charges may be high because patients are hospitalized longer than in later periods when improvements in training and technology lead to shorter hospitalizations and fewer diagnostic tests. The length of this learning curve period may vary among hospitals; therefore, data on treatment patterns should be collected and included in the cost analysis.

Measurement intervals also need to be chosen in such a way as to balance concerns related to the efficiency of data collection with those related to logistics and funding (see Table 5). Staff may not be available for frequent assessments of costs of care, nor may there be enough financial support for these efforts. However, assessments of costs of care that are spaced too far apart are more likely to be incomplete and inaccurate. Because many phase III trials include a quality of life study, it may be possible to combine some of the cost data collection with the quality of life instruments.

Where Should the Data Be Collected?

For an economic analysis to be statistically significant, many patients from many locations should be included (see Table 6). Whenever possible, simple, prospective, and uniform data collection methods should be used. Because the impact of the treatment on costs of care is being evaluated, it is best to measure costs of care prospectively while the patient is undergoing therapy. This is when most of the resources that are being used can be readily identified. When multiple sites are involved, the project staff must coordinate with multiple financial departments at many hospitals and physicians' offices; however, marked differences in the financial systems may limit the ability to use these data in a systematic fashion.

Use of a retrospective financial paper trail, or billed charges, leads to several problems with data collection [14]. Timeliness of data retrieval and missing data are major problems with retrospective evaluations. Billing information and medical record reviews are the optimal sources of data [3]. Patients generally receive medical care from several physicians, hospitals, and home care agencies, and purchase pharmaceuticals at both hospital pharmacies and local pharmacies, thus complicating the assessment of how much care they receive. To reduce staff burden, therefore, data collection can focus primarily on resources that were used frequently during the clinical trial or that were especially expensive [5,14]. While, in theory, economic analyses must account for the use of all resources, in practice it is not feasible to abstract data on all resources.

Pitfalls of Economic Analyses of Phase III Clinical Trials

There are many pitfalls associated with economic analyses of phase Ill clinical trials that should be acknowledged (see Table 7). First, these analyses should not be considered simply as "add-ons" to a clinical trial [15]. Data collection and interpretation cost money. Funding for the economic analyses during a clinical trial needs to be obtained separately. When economic analyses are not funded, it may be prudent to cancel the entire clinical trial, especially if the major outcome measure is an economic one.

A second pitfall relates to timing. While useful insights may result from a clinical and economic analysis of a phase III licensing trial for a new drug, the policy implications are unclear if the drug does not receive FDA approval [15]. If a drug is cost effective but not available for marketing, then one must wonder if the economic analysis was worthwhile. On the other hand, performing economic analyses based on phase III licensing trials several years after a drug has received FDA approval for marketing raises similar questions of timeliness. Policymakers are unlikely to use late cost-effectiveness estimates, since practice patterns are likely to have changed or newer drugs or alternative therapies may have been developed in the meantime. Therefore, it is important to choose a therapy that is likely to be successfully marketed or to conduct economic analysis during nonlicensing phase III trials.

A third pitfall of economic analyses relates to overburdening of data collectors. Data managers for clinical trials spend a great deal of effort accounting for clinical assessments of disease status and toxicities, and ensuring that diagnostic tests and procedures are carried out according to the protocol schedule. Complex data collection forms that require extensive effort to track down financial information may divert attention from the clinical details and jeopardize the quality of the study. Careful consideration must be given to alternative data collection efforts or the use of additional personnel.

A fourth pitfall is associated with quality assurance methodologies. While audits of clinical trial results are mandatory, strategies for overseeing economic analyses have not been developed. Review of medical bills, patient logs, and other data sources will require expertise that, to date, has not been a part of oversight committees. If cost-effectiveness issues become part of the criteria required by FDA officials or health plans prior to approving a therapy for marketing or general use, consideration must be given to hiring accountants or health services researchers to review economic data.

An additional pitfall relates to patient and health care system confidentiality. Studies that require extensive review of patient bills and other financial information will generally require informed consent from patients and approval of the human subjects committee. Studies that evaluate economic outcomes among multiple health care systems (hospitals, providers, physicians, or cities) may suggest that one setting is less efficient than others while, in fact, variations in patient selection and referral patterns may account for most of the variation.

A final pitfall concerns economic analyses that are based on subset analyses at a small number of clinical sites. These results may differ from those that might be found if a larger number of sites were evaluated [5,16]. If practice patterns or referral patterns vary markedly among the various sites, economic analyses must be based on results from many, if not all, of the study sites. For example, one site in a multiple-site study of GM-CSF for autologous bone marrow transplantation included a new technology-peripheral stem cell support-for both the treatment and the placebo group, while the other sites did not [17]. Therefore, economic analyses based on results from the site using stem cells might differ from those at the other sites and would need to be reported separately.

Conclusions

In the age of managed care, the cost of care is increasingly becoming a major factor in determining whether new or expensive therapies come into use in clinical practice. To ensure that costly but cost-effective therapies are given proper consideration, data on cost effectiveness, including quality of life measures, must be collected during phase III trials. For cancer treatments, for example, CALGB and ECOG have organized task forces to evaluate the feasibility of including economic analyses in phase III trials [15].

Successful completion of an economic analysis during a phase III clinical trial requires careful consideration of many variables. Careful planning can alleviate methodologic issues related to study design, conduct, analysis, and interpretation. This will ensure the validity of cost-effectiveness analyses so that policymakers can apply the data to the efficient allocation of scarce medical resources.

References:

1. Smith TJ, Hillner BE, Desch CE: Efficacy and cost-effectiveness of cancer treatment: Rational allocation of resources based on decision analysis. J Natl Cancer Inst 85:1460-1474, 1993.

2. Detsky AS: Guidelines for economic analysis of pharmaceutical products: A draft document for Ontario and Canada. Pharmacoeconomics 3:354-361, 1993.

2a. Henry D: Economic analysis as an aid to subsidization decisions: The development of Australian guidelines for pharmaceuticals. Pharmacoeconomics 2:54-67, 1992.

3. Bennett CL, Gulati SC, Smith TJ, et al: A health policy perspective on high-dose cancer therapy: Cost, effectiveness, and cost-effectiveness, in Antman K, Armitage JO (eds): High Dose Cancer Therapy: Pharmacology, Hematopoietins, and Stem Cells, 2nd ed, pp 130-148. Philadelphia, Williams & Wilkens, 1995.

4. Finkler SA: The distinction between cost and charges. Ann Intern Med 96:102-109, 1982.

5. Gulati SC, Bennett CL: Granulocyte-macrophage colony-stimulating factor (GM-CSF) as adjunct therapy in relapsed Hodgkin disease. Ann Intern Med 116:177-182, 1992.

6. Eddy DM: Cost-effectiveness analysis: Is it up to the task? JAMA 267:3342-3348, 1992.

7. Welch HG, Larson EB: Cost effectiveness of bone marrow transplantation in acute nonlymphocytic leukemia. N Engl J Med 321:807-812, 1989.

8. Weeks J, Tiemey MR, Weinstein MC: Cost-effectiveness of prophylactic intravenous immune globulin in chronic lymphocytic leukemia. N Engl J Med 325:81-86, 1991.

9. Lobo PJ, Powles RL, Hanrahan A, et al: Acute myeloblastic leukemia-A model for assessing value for money for new treatment programs. Br Med J 302:323-326, 1991.

10. Hillner BE, Smith TH, Desch CE: Efficacy and cost-effectiveness of autologous bone marrow transplantation in metastatic breast cancer-Estimates using decision analysis while awaiting clinical trial results. JAMA 267:2055-2062, 1992.

11. Daniels N: Is the Oregon rationing plan fair? JAMA 265:2232-2235, 1991.

12. Hadorn DC: Setting health care priorities in Oregon: Cost-effectiveness meets the rule of rescue. JAMA 265:2218-2225, 1991.

13. Bennett CL, Armitage JL, Armitage GO, et al: Costs of care and outcomes for high-dose therapy and autologous transplantation for lymphoid malignancies: Results from the University of Nebraska 1987 to 1991. J Clin Oncol 13(4):969-973, 1995.

14. Bennett CL, Armitage JL, Buchner D, et al: Economic analysis in phase III clinical cancer trials. Cancer Invest 12:336-342, 1994.

15. Bennett CL, Smith TJ, George SL, et al: Free-riding and the prisoner's dilemma: Problems funding economic analyses of phase III cancer clinical trials. J Clin Oncol 13:2457-2463, 1995.

16. Glaspy JA, Bleecker G, Crawford J, et al: The impact of therapy with filgrastim (recombinant granulocyte-colony stimulating factor) on the health care costs associated with cancer chemotherapy. Eur J Cancer 29A(suppl 7):S23-S30, 1993.

17. Advani R, Chao NJ, Horning SJ, et al: GM-CSF as an adjunct to autologous hematopoietic stem cell transplantation for lymphoma. Ann Intern Med 116:183-189, 1992.