Why so many dead-end phase II trials?

America's clinical trial pipeline is drying up, with fewer safe and effective new cancer drugs reaching the market. Some experts contend that FDA's control of the drug approval process hinders the development of new products.

Current research, however, suggests that a "publish or perish" ethos—common among some junior researchers—is at least part of the problem, thus producing an overabundance of dead-end phase II trials that divert resources from more promising new compounds that could go from phase III to FDA approval.

A related question is whether perverse incentives encourage researchers to label their results "positive" without doing the rigorous comparative studies, resulting in trials that produce "encouraging" results, but ultimately go nowhere.

A research team, headed by Ian Tannock, MD, professor of medicine, Princess Margaret Hospital and the University of Toronto, Canada, sought to determine how many promising phase II trials actually lead to phase III testing. They reviewed 200 promising phase II trials presented at ASCO meetings in 1995-1996, and 2006, randomly selecting 20 abstracts with encouraging results for each of five cancer sites (breast, lung, GI, GU, and GYN) for each time period.

"For the trials presented in the 1995-1996 period, we searched for subsequent randomized trials in which one treatment arm was similar to that in the phase II study," Dr. Tannock said. For the studies presented in 2006, the researchers sent a questionnaire to the authors asking whether they recommended testing the drug in phase III, whether a phase III was planned, and whether they had the resources (budget, patients, and drugs) to conduct a randomized controlled trial.

Results question motives

Looking at the data from 1995-1996, the researchers found that 10 years after presenting phase II trials with promising results, only 13 of 100 regimens were evaluated in phase III testing.

Of the 100 investigators who presented a phase II trial in 2006, 42 returned the questionnaire. Of these, 36 confirmed that the results met the efficacy criteria, and 25 felt the regimen should be evaluated in phase III testing. Nevertheless the survey showed that only 10 of the investigators planned to undertake phase III trials, with just 8 stating that they had the necessary resources. The reasons for not initiating a randomized controlled trial included insufficient efficacy, lack of financial support, and limited access to subjects.

"It would appear," Dr. Tannock said, "that the proportion of phase II trials that go on to phase III testing would not be any different in the 2006 series." Dr. Tannock said that quite a few phase II trials are undertaken when there is very little likelihood, regardless of their results, that they will lead to a phase III trial.

Moreover, many trial-ending limitations, such as lack of financial support, were known when the researchers first planned the phase II study, implying that they never intended to move their regimens to phase III studies.

"Unfortunately," Dr. Tannock concluded, "the real purpose of many phase II trials may be to legitimize the use of nonapproved compounds, or to enhance the investigators' CVs."

A study led by Andrew J. Vickers, PhD, Memorial Sloan-Kettering Cancer Center, analyzed the "go/no go" decision-making process that determines whether phase II trials move on to definitive phase III testing (Clin Cancer Res 13:972-976, 2007). Using a prespecified protocol, the researchers conducted a systematic review of phase II trials published in the Journal of Clinical Oncology or Cancer, in the 3 years leading up to June 2005.

"We focused on design aspects of these trials and how they ultimately affect the phase II to phase III transition," Dr. Vickers told Oncology NEWS International.

When asked why so many phase II trials don't progress to phase III, Dr. Vickers responded, "There are a variety of explanations; however, we found that a large number of phase II studies require historical data to set an appropriate target response. And many of those trials are not setting the historical target appropriately, and that seems to be associated with outcome of the studies."

New thinking required

Dr. Vickers said that we're facing a relatively new problem: "A typical phase II study 15 years ago would have examined a cytotoxic agent in late-stage patients.There was no need to use historical data for determining the null hypothesis as we know that few patients would respond in the absence of treatment."

He explained that with the agents being tested today, researchers need to look at different endpoints, such as overall survival. "We're still thinking too much in terms of the old design; the methodology hasn't caught up with the new endpoints we are looking at," he said.

Dr. Vickers' study found that poorly chosen targets reduce the ability of phase II trials to determine which agents should be considered for randomized trials and which agents should be dropped from further evaluation.

"The cost in resources, both time and money, is hard to overestimate," Dr. Vickers said. Addressing the problem, his team devised a guideline to improve the design and reporting of phase II trials (see Table). Under the guideline, better use of historical data will improve the sensitivity and specificity of phase II testing, potentially leading to a better prospect of phase III success. "I believe that these are the first published guidelines on how to report phase II trials that require historical data to set an appropriate null hypothesis," Dr. Vickers said.

Some trials lost in the process

In an interview, nationally recognized expert Robert B. Catalano, PharmD, said: "There are just so many phase II trials that are simply not well thought out. In effect, they come up with these small, hypothesis-driven designs in order to minimize the number of patients subjected to inferior regimens."

Dr. Catalano, vice president of regulatory affairs for the Coalition of Cancer Cooperative Groups and an editorial advisor to ONI, pointed out another problem: "It takes just as much time and resources for the NCI-sponsored cooperative groups to develop a phase II trial as it does for a major phase III investigation. Moreover, since there are such limited resources, the cooperative groups have to painfully pick and choose which protocols they're going to develop." Many times, the review process takes so long that the science has actually passed these protocols by the time the trial is up and running, he said.

"I actually don't know why we do a lot of the phase II trials that never go on to phase III," Dr. Catalano said. Private drug sponsors, he said, will do selected phase II trials if they meet their R&D timelines. Then they'll make the go/no-go decision depending on the result, which is often a minimal hypothesis-driven response.

"But is that result worth the millions of dollars it takes to bring that agent into phase III? I would suspect not. The problem is, they're all looking for the next billion-dollar drug," Dr. Catalano said.

Setting the bar too low

Ramaswamy Govindan, MD, associate professor of medicine, Washington University School of Medicine, has been involved in the design of a number of phase II trials. He told ONI that too many phase II trials use response data that set the bar too low for determining whether an agent shows promising results.

"The issue is really identifying the appropriate endpoint, and how do you determine whether or not the agent is truly promising," Dr. Govindan told ONI.

In an era of limited resources, he said that it is vitally important to reevaluate how we select agents that move from phase II to III. Currently, most phase II studies looking at response rate as an endpoint conclude that the agent is promising. "Unfortunately, many trials are designed to be self-serving," Dr. Govindan said. He stressed that what is often construed as "encouraging results" in phase II ends up to be clinically insignificant in larger, randomized trials.

Perverse incentives at work

Bruce E. Hillner, MD, professor of medicine, Virginia Commonwealth University, an expert in the trends of clinical trials, has found that the United States accounts for approximately half of the world's pilot studies, whereas in Europe, investigators chiefly use their resources to perform comparative randomized controlled trials, yielding more telling data for evidence-based cancer care.

Financial and academic advancement incentives in the United States to enroll patients in phase II studies are well known, and although trial authors are expected to acknowledge potential conflicts of interest, compliance is lax.

A financial tie with the pharmaceutical industry is another factor driving the trend of poorly planned phase II trials. "In some cases, drug companies use promising results from small phase II trials to encourage the use of an already FDA-approved drug off-label for another indication, sparing itself the huge expense of a phase III trial," Dr. Hillner said.

To expand evidence-based oncology care, we need to recognize the flaws in the drug development system so they can be corrected. As Dr. Tannock pointed out, self-aggrandizement among ambitious young researchers is one trend driving the troubling preponderance of dead-end phase II trials.

There are also numerous political, cultural, and financial tensions that confound the drug development process. But as Dr. Hillner observed, "In many cases, researchers simply label their results 'encouraging' without comparing them to outcomes seen with currently available drugs. This failure to define a benchmark is the major reason so many of these studies are just short-term fireworks."