| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
J Thorac Cardiovasc Surg 2006;132:247-248
© 2006 The American Association for Thoracic Surgery
Statistics for the Rest of Us |
Johns Hopkins Center for Clinical Trials, NETT Coordinating Center, Baltimore, Md.
Received for publication December 21, 2005; accepted for publication March 3, 2006. * Address for reprints: Steven Piantadosi, MD, Johns Hopkins Center for Clinical Trials, NETT Coordinating Center, 615 N. Wolfe St, Room 5010, Baltimore, MD 21205-2011 (Email: spiantad{at}jhmi.edu).
The current work by Tiruvoipati and colleagues
1
measures substantive deficiencies in the reporting of randomized trials in the surgical literature. Among other things, the authors of that report indicate that lack of awareness of the CONSORT guidelines
2
contributes to the deficiencies they observed. We cannot know for sure whether the problems in the surgical literature are worse than in other contexts, but we think they are because of the general underapplication of rigorous experimental design methods for clinical questions in surgery.
3
The Tiruvoipati study
1
is a review of problems relatively near the end of the evidentiary pipeline. However, I believe it is helpful to reflect on the whole culture and process of therapeutic inference in surgery to understand how the literature might be improved and, in turn, how the literature might improve the science. This means examining investigators' attitudes and beliefs, the mindset of peer reviewers, the role of surgical journals, and the demands of readers.
With regard to attitudes and beliefs, surgeons have regularly been taught some things that dissuade them from relying on experimental methods. These include a great deal of respect for opinion and experience, the anticipation of large treatment effects, the reliability of incremental improvements in a surgical procedure, the difficulty and/or non-necessity of randomization, an underappreciation of selection and observer bias, and confidence in favorable risk/benefit ratios in properly selected patients. Such beliefs support the adequacy of relatively informal methods for evaluating treatments and are descendants of historical authoritarianism in medicine. Other contexts, like therapeutic development of drugs, have appropriately and successfully replaced the authoritarian perspective with an experimental one because the beliefs listed above are not routinely true and because of our frequent need to detect modest-sized but valuable treatment effects. Drug regulation has been instrumental in effecting this change.
Drug developers and other nonsurgeons can provide worthwhile alternative views to some of the attitudes and beliefs of surgeons regarding rigorous clinical trial methodology. An example is the value, validity, feasibility, and frequent necessity of randomization as a device to remove bias and increase reliability. Treatment masking also deserves more than a customary dismissal. Also, many surgical treatment effects, when present, are modest in size. Such mitigating, rather than curative, effect sizes require large rigorous trials to provide convincing evidence.
Much of what is taught to young investigators is carried over to those who peer review manuscripts, where scientific culture reinforces itself. It is not enough to know that trial reporting guidelines are relevant to a particular study and make them, in part, the currency of a review. What is more important is to see beyond reporting weaknesses and assess the true quality of the trial. Even more useful is the ability to know what research design is appropriate and possible and gauge the strength of evidence on that basis. Unavoidably, therapeutic questions in surgery confound 3 effects: (1) efficacy of the procedure, (2) prognosis through patient selection, and (3) practitioner skill and supportive care. Strength of evidence from a surgical study depends largely on the ability of the research design to separate those effects, especially the first and second. It can be intimidating as a reviewer to imagine rejecting a superficially well-done study because of critical design flaws.
Journal editors play a crucial role in quality improvement that goes beyond the grooming of manuscripts. Culling is their most powerful tool, and it has to be applied aggressively if a journal is to improve itself. Multiple journals that improve themselves will begin to elevate the scientific discipline. This gives the editorial process considerable leverage and importance. As a repository of the scientific culture that speaks from both past and present, it is vital to improve journals energetically. Journal editors lack direct influence in many matters (eg, determining which questions are addressed by research studies), but we should not underestimate the breadth of power they wield.
The problem of improving clinical trials in surgery comes full circle when considering how the demands of readers affects the science. Readers' expectations depend on knowledge, attitudes, and beliefs acquired in training and through experience, and they exercise those demands in two ways. First, the effect of a published clinical trial probably depends as much on the receptivity of the audience as on the quality of the science. The "word on the street" or lack of it depends on receptivity, as does the willingness of readers to incorporate published findings into both their clinical and research practices. Second, readers translate their demands into concrete decisions as peer reviewers. Thus the discipline is not necessarily hard wired for progress but instead relies on the influence of teachers, of which journal editors are one example. In this way the article by Tiruvoipati and colleagues
1
points to a direction for improvement.
| See related articles on pages 229, 233, 241, 243, 245, and 249.
|
 |
References |
|---|
 Top References |
|---|
Related Articles
This article has been cited by other articles:
|
|
 |
|
  M. Akhtar, E. M. Tuzcu, S. R. Kapadia, L. G. Svensson, R. K. Greenberg, E. E. Roselli, S. Halliburton, V. Kurra, P. Schoenhagen, and S. Sola Aortic root morphology in patients undergoing percutaneous aortic valve replacement: Evidence of aortic root remodeling J. Thorac. Cardiovasc. Surg., April 1, 2009; 137(4): 950 - 956. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. C. Bauer Randomized Trial Reporting in General Endocrine Journals: The Good, the Bad, and the Ugly J. Clin. Endocrinol. Metab., October 1, 2008; 93(10): 3733 - 3734. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
