Who's Who from Genetics community demands federal government stop Duke Potti trials

Text of their letter. See next Blog entry for more info.

To: Harold Varmus, NCI

Cc: Otis Brawley ACS; Kim Lyerly, Duke University; John F. Potter, Department of Defense; The Cancer Letter; Don Wright, Office of Research Integrity

Re: Concerns about prediction models used in Duke clinical trials

Dear Dr. Varmus,

We understand that NCI is aware of three cancer clinical trials funded by the Department of Defense and Duke University, based at least in part on results reported in papers by Duke
oncologist and genomics researcher Anil Potti and Joseph Nevins (a list of articles is appended
below).

Drs. Potti, Nevins, and their colleagues have made claims about the ability of RNA expression patterns to predict responses to therapy in cancer patients, and these prediction
models are currently being used in Duke’s clinical trials to help physicians choose the treatments that cancer patients receive.

Recently, published and peer-reviewed re-analyses of the work done by Potti and Nevins revealed serious errors that questioned the validity of the prediction models upon which these ongoing clinical trials are based. This led to a temporary suspension of the trials and a Duke-led
review involving independent statistical experts. However, despite written statements from the external experts, who uniformly stated they were not given sufficient information to confirmthe validity of the models, the trials have been reinitiated.

We, the undersigned, who have followed this debate closely have concluded that the inability of independent experts to substantiate the above researchers’ claims using the researchers’ own data means that it is absolutely premature to use these prediction models to influence the therapeutic options open to cancer patients.

We strongly urge that the clinical trials in question (NCT00509366, NCT00545948,
NCT00636441) be suspended until a fully independent review is conducted of both the clinical trials and of the evidence and predictive models being used to make cancer treatment decisions.

For this to happen, sufficiently detailed data and annotation must be made available for review.

The data should be sufficiently documented for provenance to be assessed (as both gene and sample mislabeling have been documented in these data), and the computer code used to predict which drugs are suitable for particular patients must be made available to allow an independent
group of expert genomic data analysts to assess its validity and reproducibility using the data supplied.

Should the data and analysis presented by Potti, Nevins, and colleagues be validated, it would then, of course, be appropriate to reinitiate the trials. Until that time, however, we believe that the steps outlined in this letter are necessary given the potential of patients being assigned to improper treatment arms in the clinical trials in question and of the associated potential risk posed to these patients.

We are therefore requesting that NCI either intercede directly, or work with other entities with jurisdiction over these trials (e.g. ORI, DoD and Duke University) to ensure that the above requirements are met before these trials are allowed to continue enrolling patients.

Sincerely,
Anna E. Barón, Professor, Department of Biostatistics and Informatics, Colorado School of Public Health Director, Biostatistics and Bioinformatics Core, University of Colorado Cancer Center,

Karen Bandeen-Roche, Professor and Chair of Biostatistics, Johns Hopkins Bloomberg School of Public Health

Donald A Berry, Chairman of Department of Biostatistics, Head of Division of Quantitative Sciences, The University of Texas M.D. Anderson Cancer Center

Jennifer Bryan, Associate Professor, Department of Statistics and the Michael Smith Laboratories, University of British Columbia

Vincent J. Carey, Associate Professor of Medicine (Biostatistics), Harvard Medical School

Kathryn Chaloner, Professor and Head Department of Biostatistics, College of Public Health, Iowa State University

Mauro Delorenzi, Head Bioinformatics Core Facility (BCF), Swiss Institute of Bioinformatics (SIB)

Bradley Efron, Professor of Statistics, Stanford University

Robert C. Elston, Professor and Chair, Department of Epidemiology and Biostatistics, Case Western Reserve University School of Medicine

Debashis Ghosh, Professor of Statistics and Public Health Sciences, Penn State University

Judith D. Goldberg, Professor and Director of Biostatistics, New York University School of Medicine

Steve Goodman, Professor of Oncology, Johns Hopkins School of Medicine

Frank E Harrell Jr., Professor of Biostatistics, Vanderbilt University

Susan Galloway Hilsenbeck, Professor Lester and Sue Smith Breast Center and Dan L Duncan Cancer Center at Baylor College of Medicine

Wolfgang Huber, Research Group Leader, European Molecular Biology Laboratory (EMBL) Heidelberg, Germany

Rafael A. Irizarry, Professor of Biostatistics, Johns Hopkins Bloomberg School of Public Health

Christina Kendziorski, Associate Professor of Biostatistics and Medical Informatics, University of Wisconsin at Madison

Michael R. Kosorok, Professor and Chair of Biostatistics and Professor of Statistics and Operations Research, University of North Carolina at Chapel Hill

Thomas A. Louis, Professor of Biostatistics, Johns Hopkins Bloomberg School of Public Health

J. S. Marron, Amos Hawley Distinguished Professor of Statistics and Operations Research, University of North Carolina, Chapel Hill

Michael Newton, Professor of Biostatistics and Medical Informatics and of Statistics, University of Wisconsin at Madison

Michael Ochs, Associate Professor of Oncology Biostatistics, Johns Hopkins School of Medicine

John Quackenbush, Professor of Computational Biology and Bioinformatics, Department of Biostatistics, Harvard School of Public Health

Gary L. Rosner, Director, Research Program in Quantitative Sciences and Division of Oncology Biostatistics/Bioinformatics, The Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins

Ingo Ruczinski, Associate Professor of Biostatistics, Johns Hopkins Bloomberg School of Public Health

Steven Skates, Associate Professor of Medicine (Biostatistics), Harvard Medical School

Terence P Speed, Professor and Head, Bioinformatics Division, Walter and Eliza Hall Institute of Medical Research

John D. Storey, Associate Professor of Genomics, Princeton University

Zoltan Szallasi, Professor and Group Leader, Center for Biological Sequence Analysis, Department of Systems Biology, Technical University of Denmark

Robert Tibshirani, Associate Chairman and Professor of Health Research and Policy, and Statistics, Stanford

Scott Zeger, Vice Provost for Research and Professor of Biostatistics, Johns Hopkins University

Relevant articles:
A partial list of the papers in question includes Potti et al., Nature Medicine 2006, Potti et al.,
NEJM 2006, Hsu et al., JCO 2007, Bonnefoi et al., Lancet Oncology 2007, Dressman et al.,
JCO 2007, and Augustine et al, 2009. These papers have made important claims about the
ability of RNA expression patterns to explain particular aspects of cancer biology and to
provide tools for predicting prognosis and patient response to therapy. Some of these
approaches have been patented (US Patent Application 20090105167) and have been in use in
clinical trials since 2007 (NCT00509366, NCT00545948, NCT00636441).
The quality of the data that serve as the basis for the analysis described in these papers has
already been called into question both in the scientific literature (Coombes et al., Nat Med
2007, Baggerly et al, JCO 2008, Baggerly and Coombes, Ann App Stat 2009) and elsewhere
(the Cancer Letter, Oct 2, 9, and 23, 2009, Jan 29, May 14, and July 16, 2010). Some doubts
raised (Baggerly and Coombes, 2009) were so severe that Duke suspended clinical trials and
convened an independent panel to review the approaches (the Cancer Letter, Oct 2, 9, 23, 2009)
before letting the trials resume (the Cancer Letter, Jan 29, 2010). However, the panel’s report
(obtained from the NCI under FOIA) shows that even the panel was unable to confirm or refute
the validity of the claims based on publicly available data (the Cancer Letter, May 14, 2010).
Despite this, the trials were resumed and continue to use these models as an integral component
in deciding patient treatment.