|
|
||||||||
J Thorac Cardiovasc Surg 2007;133:1414-1418
© 2007 The American Association for Thoracic Surgery
Statistics for the Rest of Us |
a Department of Cardiothoracic Surgery, Guys Hospital, London, United Kingdom
b Clinical Operational Research Unit, University College London, London, United Kingdom.
Received for publication October 31, 2006; revisions received January 31, 2007; accepted for publication February 12, 2007. * Address for reprints: Tom Treasure, Professor of Cardiothoracic Surgery, Guys Hospital, London, United Kingdom. (Email: tom.treasure{at}gmail.com).
| Introduction |
|---|
|
|
|---|
Authors may recognize their work in the examples we present, and so we may well cause offence. This is not our intention. Readers may be tempted to search our own work for examples of errors, and they will find themmore egregious than any of these. The alternative to citing real instances was for us to make them up or make them anonymous. However, then maybe some would say, "Surely no one would do that?" so they had to be real. We did not search systematically for examples; they come from articles we know through our research in mesothelioma. Finally, we have not shied away from using prominent examples, from leading authors, in leading journals. We needed to cite the best work or our concerns could be discounted as the result of a pedants trawl of the worlds literature. They are quoted with respect for the authors and editors, who we know have broad shoulders.
| Presumption of Efficacy |
|---|
|
|
|---|
The following statement (personal communication) from Eric Butchart, a pioneer of the pleuropneumonectomy procedure (also called extrapleural pneumonectomy or EPP), summarizes the situation: "We have recently analysed our experience with both pleuropneumonectomy and pleurectomy/decortication for mesothelioma. The very strong message from this analysis is that adjuvant therapy is essential in order to achieve any degree of long term survival with either surgical procedure."
The inference that no degree of long-term survival is achieved with surgery alone is embedded in the statement by Butchart. If there is a perceived improvement in survival, logically it should be attributed to the adjuvant chemotherapy, and yet authors of surgical case series tend to conclude that surgery confers a survival advantage.
This belief would seem to be widespread. In the Methods section of the first report of a randomized trial of pemetrexed (in which a difference in survival was found between two regimens of chemotherapy), there is the statement that "candidates for curative surgery" were excluded.4
The clear implication is that curative surgery exists for these patients and that somewhere surgical cure is being achieved, but no reference is cited.
When surgeons perform very radical surgery on many patients, there is deep felt belief, held by medics and the public alike, that the operation must be doing some good. This is a very human tenet, which we call the presumption of efficacy.
| Lead-time Shift |
|---|
|
|
|---|
The introduction to the most quoted surgical series affirms that the "natural history of malignant pleural mesothelioma includes a median survival of 412 months without intervention."6
There are three citations in support of this statement. They were published in 1980, 1984, and 1989, 10 to 20 years before the citation in 1999. Two of these observational series specifically comment on the delay and lateness in making a diagnosis.
Statements such as "recent series show that overall survival is much longer than previously thought when [mesothelioma] is diagnosed at an early stage"7
illustrate the potential for confusion between lengthening survival by therapy and by starting the clock sooner.
There are factors other than the vintage of the data. In 2001 we identified all patients in two London thoracic surgical centers in whom the diagnosis of mesothelioma was made. The two sets of 127 and 234 patients were similar in age, sex, and the mix of tumor types.8
Neither group was treated aggressively; surgery at most was talc pleurodesis, and none had EPP. The survival of the two groups was very different. There are various possible explanations including chance, lead-time shift, and differing histologic criteria, but these uncertainties only add to the reasons for exercising caution when using historical series in comparisons. Either or both of these series of cases diagnosed in the 5 years up to 2001 would be a more fair comparison than the survival of patients diagnosed in the 1970s and 1980s, but they might still be misleading. Comparisons against historical controls are inherently of limited reliability.
| Comparison Groups May Have a Different Case Mix |
|---|
|
|
|---|
To make the point, let us look at one definable factor on which contemporary surgical series and historical survival series are different. The retrospective case series included all patients with a diagnosis of mesothelioma. More than 40% of patients would have had sarcomatoid histology, for which survival time is much reduced. We cannot know exactly how many, so we cannot correct for it. Other than retrieve all the pathologic material and have it re-examined with contemporary criteria, ideally "blind," we can never know. These sarcomatoid tumors are increasingly excluded from EPP in current practice.
To compare current surgical figures with an all-case historical series would be like reporting your present day results for surgically treated nonsmall cell cancer and comparing 5-year survival with the natural history of all lung cancer, including small cell, from the 1970s before computed tomography and before positron emission tomography.
If the results of current practice are to be held up for inspection against any comparator, historical or otherwise, surely just as much care should be exercised in the analysis of the comparison group as in that of the surgical series. These data may not be a finding of the research reported, but they are an intrinsic part of the evidence presented, and yet even the summary statistics may be quoted blandly without adequate definition or caveat, as in the example cited: "The natural history of malignant pleural mesothelioma includes a median survival of 412 months."6
And yet it is done in the case of mesothelioma without qualification or citation in a recent "evidence based approach."9
There is a consequence worse than inadvertently misleading ourselves and our colleagues. Once such statements appear in reputable journals, it is reasonable for others to cite them; they are regarded as fact. In 2006, by which time the historical series quoted were 20 to 30 years out of date, they appeared on a public web page in similar terms and in the present tense: "survival averages four to 12 months."10
| In Multimodality Cancer Treatment We Cannot Laud a Single Component |
|---|
|
|
|---|
The series are reported as surgical seriesthe radical surgery is center stage. However, any survival gain cannot be attributed to radical surgery. If there is an advantage, it should be attributed to the full treatment package.
| "Intention-to-Treat" Analysis Is Required |
|---|
|
|
|---|
The most obvious breach of intention-to-treat analysis is to exclude from the survival curves the patients who died in the immediate postoperative period.6
It goes further than that. An extreme scenario in mesothelioma is as follows. Patients are selected for radical surgery, and in some (but not all) EPP is performed. Of those who survive surgery, some (but not all) are selected to have radiotherapy. Some of these (but not all) continue to survive, and some (but not all) of these are well enough to be given chemotherapy and complete the course. Patients who complete treatment have a median survival much longer than would have been expected in comparison with the natural history of the disease. This survival is claimed as evidence for the effectiveness of multimodality treatment.
The patient pool under consideration is reduced at each of multiple stages through both clinical selection and survival. The patients had to be worked up, undergo, survive, and recover from each of the first two components and then be worked up for and undergo the final component. All take time (maybe as much as 3 months each), each step may eliminate some patients, and each intervention takes a toll on patients. Those who arrive at the end are clinically and self-selectedly survivors.
There may be a place for per-protocol analysis in evaluation of the mechanics of some surgical techniques, but it can surely have no place in the evaluation of multimodality therapy as arduous and drawn out as that offered for mesothelioma. Intention-to-treat analysis is essential.
| Face Validity |
|---|
|
|
|---|
A fundamental principle of curing cancer by surgery has been eradication of all disease at operation. There may be an emerging role of reducing tumor bulk as an adjunct to other treatment, but for curative surgery the first principle is to clear all the cancer. It is for that reason that certain forms of cancers for which radical surgery used to be performed with intent to cure are no longer operated on. These include small-cell lung cancer, lymphoma, and many cases of breast cancer.
These cancers are characterized as already being systemic diseases at presentation, and the cancer is thus beyond the surgeons knife. That distinction, between the surgically curable and incurable cancers, has been negotiated since the early reports of radical surgery for mesothelioma. On the face of it, the creeping nature of mesothelioma and the arbitrariness of the anatomic clearance of the endothoracic fascia make it an unlikely candidate for surgical cure. The onus to counter that argument is on those who believe that EPP can cure.
Analysis of an operated series revealed three factors associated with longer postoperative survival: histologic type, mediastinal lymph node involvement, and surgical clearance, in that order of effect size.6
The smallest survival difference was associated with whether the resection margins were clear (R0) or not (R1).
That the difference between survival time for R1 and R0 (ie, the difference between the operation failing to eradicate all disease and the operation possibly eradicating all disease) is smaller than the differences attributed to nodal status and histology undermines the notion that this cancer lends itself to surgical cure.
| Factors for "Case Selection" Should Be Tested on a New Data Set |
|---|
|
|
|---|
Three factors were found in multivariable analysis to be associated with survival. These were used to classify patients into groups with 0, 1, 2, or 3 of these characteristics: epithelioid histology, no extrapleural nodes, and clear resection margins. There were significant differences among the four groups thus created. However, inasmuch as the factors were derived from a multivariable analysis and then used to categorize the same data, this is completely unsurprising. As a numerical argument it is circular. To prove the validity of these factors as predictors, they should be applied to a new series, not to the one from which they were derived. To be fair, this point is made by the authors6
but does not seem to have been heeded.
Importantly, even if validated as factors associated with survival after EPP, such analysis cannot be used to select those likely to "benefit most" from EPP because the notion of differential benefit is based entirely on the presumption of efficacy.
| Criteria for Case Selection Must Be Applicable Before Surgery |
|---|
|
|
|---|
If we are to have a set of criteria on which to select or counsel patients, they must be available before surgery and be amenable to validation, indeed have been validated, before they can be used.
| Association Is Not Causation |
|---|
|
|
|---|
The problem is that many factors are used by experienced and knowledgeable teams when they decide to operate on one patient but not another. These factors include all available clinical information but cannot all be accounted for: some are not consciously recognized, some are not made explicit, and some are unknown unknowns.
In Flores Cox model,11
perhaps it was not the operation that was the determinant but that a sage surgeon, supported by many investigations and many colleagues, deemed the patient a candidate for surgery. It would be indeed surprising if these choices did not produce a cohort of patients who did better than those not selected, that is, unless the authors are suggesting that the decision to operate is some sort of haphazard event. Hence association is the correct word.
| Hope Springs Eternal in the Human Breast |
|---|
|
|
|---|
| Conclusions |
|---|
|
|
|---|
One solution to all of the problems raised might be a randomized trial. A randomized clinical trial provides a contemporary matched comparison group with a similar case mix, avoids the pitfalls of retrospective multivariable analysis, and is analyzed on an intention-to-treat basis. There are many forms of evidence for what we as surgeons do,12,13
but once removed from rather obvious cause-and-effect, when benefits are small and treatments are complex, it is unlikely that a case series will give secure evidence.
We have to conclude that there is as yet no reliable evidence for the effectiveness of radical surgery for mesothelioma, and patients should be informed of that fact. Our belief is that in the context of mesothelioma, radical surgery should be offered only in a trial in which its effectiveness can be tested. Some worry that randomized trials have the effect of disadvantaging half the patients. Where they are wrong is in thinking that they know which half.
A total of 126 trials of cancer treatments have been reported to date in 36,567 children.14
Who did better, those on the new treatments or those in the control groups? The answer is that the trial results are evenly spread either side of zero. For each group of patients denied benefit, another was spared a less effective novel treatment.
We owe it to todays patients with mesothelioma to get it right. The epidemic is still rising in Europe and there will be many patients in the next 10 to 20 years. We owe it to the rest of the world where asbestos was less well controlled or remains uncontrolled. Equally important is this: we owe it to our own scientific integrity.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
T. Treasure and S. Senan The Need for Randomized Trials in Mesothelioma: Let's Walk the Talk Oncologist, March 1, 2011; 16(3): 261 - 263. [Full Text] [PDF] |
||||
![]() |
E. Teh, F. Fiorentino, C. Tan, and T. Treasure A systematic review of lung-sparing extirpative surgery for pleural mesothelioma J R Soc Med, February 1, 2011; 104(2): 69 - 80. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. E. Van Schil, P. Baas, R. Gaafar, A. P. Maat, M. Van de Pol, B. Hasan, H. M. Klomp, A. M. Abdelrahman, J. Welch, J. P. van Meerbeeck, et al. Trimodality therapy for malignant pleural mesothelioma: results from an EORTC phase II multicentre trial Eur. Respir. J., December 1, 2010; 36(6): 1362 - 1369. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Utley, F. Fiorentino, and T. Treasure Obtaining an upper estimate of the survival benefit associated with surgery for mesothelioma, Eur J Cardiothorac Surg, September 1, 2010; 38(3): 241 - 244. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Treasure Surgery for mesothelioma: MARS landing and future missions Eur J Cardiothorac Surg, March 1, 2010; 37(3): 509 - 510. [Full Text] [PDF] |
||||
![]() |
G. P Currie, S. J Watt, and N. A Maskell An overview of how asbestos exposure affects the lung BMJ, August 24, 2009; 339(aug24_1): b3209 - b3209. [Full Text] |
||||
![]() |
T. Treasure Commentary: The big question remains unanswered BMJ, March 10, 2009; 338(mar10_2): b759 - b759. [Full Text] |
||||
![]() |
T. Treasure Are randomised trials needed in the era of rapidly evolving technologies? Eur J Cardiothorac Surg, March 1, 2009; 35(3): 474 - 478. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Treasure Pulmonary metastasectomy for colorectal cancer: weak evidence and no randomised trials Eur J Cardiothorac Surg, February 1, 2008; 33(2): 300 - 302. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |