From the New York Times magazine Sunday Dec. 3 p. 17:
Perhaps the most stinging reminder of these pitfalls comes from a timeless paper published by the statistician Richard Peto. In 1988, Peto and colleagues had finished an enormous randomized trial on 17,000 patients that proved the benefit of aspirin after a heart attack. The Lancet agreed to publish the data, but with a catch: The editors wanted to determine which patients had benefited the most. Older or younger subjects? Men or women?
Peto, a statistical rigorist, refused — such analyses would inevitably lead to artifactual conclusions — but the editors persisted, declining to advance the paper otherwise. Peto sent the paper back, but with a prank buried inside. The clinical subgroups were there, as requested — but he had inserted an additional one: “The patients were subdivided into 12 … groups according to their medieval astrological birth signs.” When the tongue-in-cheek zodiac subgroups were analyzed, Geminis and Libras were found to have no benefit from aspirin, but the drug “produced halving of risk if you were born under Capricorn.” Peto now insisted that the “astrological subgroups” also be included in the paper — in part to serve as a moral lesson for posterity. I’ve often thought of Peto’s paper as required reading for every medical student.
The larger point of the piece is an exploration of why doctors continue to pursue uses for drugs that have failed clinical trials. They look for any sub-group that may have been responsive to the drug. They are falling into the same statistical trap Dr. Peto tried to avoid. Here’s his bio from Wikipedia:
Sir Richard Peto FRS (born 14 May 1943) is Professor of Medical Statistics and Epidemiology at the University of Oxford, England.[2]
He attended Taunton’s School in Southampton and subsequently studied Natural Sciences at Queens’ College, Cambridge University. His career has included collaborations with Richard Doll beginning at the Medical Research Council Statistical Research Unit in London. He set up the Clinical Trial Service Unit (CTSU) in Oxford in 1975 and is currently co-director.
He was made a Fellow of the Royal Society in 1989 for his contributions to the development of meta-analysis. He is a leading expert on deaths related to tobacco use. He was knighted for his services to epidemiology and to cancer prevention in 1999, and he received an honorary Doctor of Medical Sciences degree from Yale University in 2011.
A second pointn is economic. At least some of these doctors are undoubtedly thinking of the high cost of the clinical trial. “We have to get something out of this expensive study,” is implicit. That, of course, is the fallacy of sunk costs. We can’t get back what was spent on the study. Those costs are sunk costs and should be irrelevant to decisions about the future.
