| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
J Thorac Cardiovasc Surg 2005;129:1223-1225
© 2005 The American Association for Thoracic Surgery
Editorial |
Department of Surgery, Massachusetts General Hospital, Boston, Mass
Received for publication January 11, 2005; accepted for publication February 10, 2005. * Address for reprints: David F. Torchiana, MD, Department of Surgery, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114 (Email: dtorchiana{at}partners.org).
|
In this edition of the Journal, an article comparing outcomes in California and New York State demonstrates that the low-volume programs in California are the main reason for the disparity in cardiac surgical outcomes between the two states.1 The report presents results in a way that we are all used to seeing, risk adjusted with confidence limits, statistical significance, and the like, with primary and secondary conclusions that make sense. Although everything the authors assert may be right on the money, we need to use caution in interpreting this kind of report, because for the California outcomes it relies exclusively on administrative data, and there lies a slippery slope.
The first problem with using claims data in clinical studies is that it is starting with an inexact tool. Administrative data are generated by professional coders who work in hospital medical records departments. Coders are trained to review charts and identify the diseases and procedures that define the most resource-intensive diagnosis-related group (DRG) assignment for each hospitalization. By definition, DRGs encompass similar but not identical patient categories, so coding does not create exact lists of clinical conditions. Going back to International Classification of Diseases, Ninth Revision (ICD-9) codes helps, but it does not solve this fundamental problem. More importantly, diagnosis codes are not dated in administrative data and do not reliably distinguish between comorbidities and procedural complications. For example, diagnoses such as stroke, atrial fibrillation, and renal failure could either increase the risk of surgery as comorbid variables or represent postoperative sequelae. This makes for a reliable but not very useful risk-adjustment system: bad outcomes obviously do predict bad outcomes. In addition, hospitals spend variable amounts of time, energy, and money to make coding better, but their view is toward optimizing reimbursement, not perfecting clinical accuracy. 3M, the company that brought us Post-It Notes, takes coding optimization a step further with a product called DRG Assurance, which helps institutions to improve documentation of revenue-critical diagnoses and secondarily improves the intensity of coding. Coding practices therefore vary significantly among institutions. In the final analysis, as seen through the distorted prism of administrative data, a program like DRG Assurance that was designed to yield higher institutional payment changes coding, increases the recorded comorbidity level of an institution’s patients, increases the complication rate, and reduces risk-adjusted mortality. In reality, of course, the outcomes have not changed, just the coding.
If administrative data are inaccurate and labile, why does anyone use them for clinical research? The main reason is because these data are accessible to everyone and relatively inexpensive to use. In fairness, at times administrative data may be the only source of information available to look at a clinical question. In these circumstances, the accuracy caveats have to be prominently acknowledged and conclusions drawn with a conservative brush, lest we unwittingly validate a growing approach to quality measurement with which we are unlikely to be happy. That is the second big problem with administrative data—the increasing tendency to overreach in its use, especially by those around the fringes of health care. There has been a virtual explosion of claims-based health care report cards and measurement systems in the media and on the Internet. In the past, doctors viewed these efforts askance but paid them little attention, because they were being ignored by the public. As administrative data start to be applied to the design of tiered insurance products, endorsed by employers and payers as a means for employees to make "value" decisions, and become a favored approach to health reform among health benefits consultants pushing the concept of "transparency," we can no longer afford to maintain that indifference.2 Claims data are now commonly being used to create reports that rank institutions or providers, with risk adjustment and statistical power added in a rudimentary fashion to enhance credibility. To say that the results are imprecise would be an understatement. At the 2004 meeting of the American Association for Thoracic Surgery, Michael Mack’s group reported a comparison of various administrative outcome data sets versus accurate clinical data reported by their group to the Society of Thoracic Surgeons. The variation was striking, with as much as a 46% difference in case volume and as much as a 76% difference in risk-adjusted mortality.3
The study of cardiac surgery results has been a national pastime for the last three decades, and heart surgeons have spent a lot of time and resources looking at outcomes. Our specialty has generally excelled under the scrutiny. The early and decisive move to a national database by the Society of Thoracic Surgeons, the groundbreaking quality improvement work of the Northern New England consortium and widespread and generalizable improvements in cardiac surgery outcomes have been among the positive results. There have been many lesson learned, but two stand out. First, important questions require accurate clinical information, and administrative data sets don’t make the grade.4,5 Second, provider-specific profiling, even with accurate data, can be bad for the public,6 an equally important but separate topic that is outside the scope of this editorial.
Figure 1 is a graphic summary created by Gregg Meyer, former director of the Center for Quality Improvement and Patient Safety at the Agency for Healthcare Research and Quality, the federal agency that studies these sorts of questions. The figure proposes appropriate uses for different categories of data. Administrative data are cheap and ubiquitous and have value for research, hypothesis generation, and quality improvement. Use of administrative data for quality reports to drive pay-for-performance insurance products is a real problem, because the information can be so inaccurate, but may be unavoidable because the payers have these data and will use them anyway. We need to draw a line, though, when clinical quality measures are intended to be used by patients for making choices or by the government for shaping important policy decisions. It is then imperative that the data set have real clinical information as its foundation and be appropriately analyzed and presented. For cardiothoracic surgery, the solution is to make an audited, compulsory Society of Thoracic Surgeons data set a national standard. The only defense against misinformation is authoritative data.
|
References
Related Article
This article has been cited by other articles:
|
|
 |
|
  O. J. Benavidez, K. Gauvreau, P. D. Nido, E. Bacha, and K. J. Jenkins Complications and Risk Factors for Mortality During Congenital Heart Surgery Admissions Ann. Thorac. Surg., July 1, 2007; 84(1): 147 - 155. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Shahian, F. H. Edwards, V. A. Ferraris, C. K. Haan, J. B. Rich, S.-L. T. Normand, E. R. DeLong, S. M. O'Brien, C. M. Shewan, R. S. Dokholyan, et al. Quality Measurement in Adult Cardiac Surgery: Part 1--Conceptual Framework and Measure Selection Ann. Thorac. Surg., April 1, 2007; 83(4_Supplement): S3 - S12. [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian, T. Silverstein, A. F. Lovett, R. E. Wolf, and S.-L. T. Normand Comparison of Clinical and Administrative Data Sources for Hospital Coronary Artery Bypass Graft Surgery Report Cards Circulation, March 27, 2007; 115(12): 1518 - 1527. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C.-A. Mestres, J.-M. Revuelta, and A. C. Yankah The European Working Time Directive: quo vadis? A well-planned and organized assassination of surgery. Eur. J. Cardiothorac. Surg., October 1, 2006; 30(4): 571 - 573. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
