|
|
||||||||
J Thorac Cardiovasc Surg 2002;124:97-104
© 2002 The American Association for Thoracic Surgery
Surgery for Congenital Heart Disease (CHD) |
From the Department of Cardiology, Children's Hospital, Boston, Mass.
This work was supported by National Institutes of Health/National Heart, Lung, and Blood Institute grant K08HL2936-01 (Dr Jenkins) and by the Kobren Fund (Dr Gauvreau).
Received for publication March 15, 2001. Revisions requested Aug 17, 2001; revisions received Nov 2, 2001. Accepted for publication Dec 14, 2001. Address for reprints: Kathy J. Jenkins, MD, MPH, Children's Hospital, Department of Cardiology, 300 Longwood Ave, Boston, MA 02115 (E-mail: jenkins{at}cardio.tch.harvard.edu).
| Abstract |
|---|
|
|
|---|
| Introduction |
|---|
|
|
|---|
We have recently developed a consensus-based method to adjust for case-mix differences when comparing in-hospital mortality for groups of children undergoing surgery for congenital heart disease.
8 Our method can provide reasonable estimates of overall institutional performance with fairly modest data requirements. To explore the usefulness of this new research tool, we analyzed hospital discharge data from centers with large surgical volumes in 6 states using the Risk Adjustment in Congenital Heart Surgery (RACHS-1) method. Our analyses uncovered several distinct patterns of programmatic outcomes, which should assist in program-specific efforts to reduce mortality.
| Methods |
|---|
|
|
|---|
Analysis of unadjusted mortality rates
For each institution performing at least 100 cardiac surgical cases per year, an overall mortality rate was calculated for the entire caseload. Institutions were ranked in order of increasing mortality from 1 (lowest mortality) to 22 (highest mortality).
Application of the RACHS-1 method
The RACHS-1 method can be applied in 2 ways. The simpler approach groups procedures with similar expected short-term mortality rates into 6 predefined risk categories, in which category 1 has the lowest risk for death and category 6 the highest (Table 1), and examines mortality separately within each category. The more complex multivariate method incorporates 4 additional clinical factors: age stratified as 30 days or less, 31 days to 1 year, and 1 year or older; prematurity identified by the presence of appropriate ICD-9-CM diagnosis codes; presence of a major noncardiac structural anomaly in addition to the cardiac defect (eg, tracheoesophageal fistula, cleft lip, or palate); and presence of combinations of cardiac surgical procedures.
|
Application of the more complex method allows computation of an overall risk-adjusted rank for each institution, incorporating nearly the entire institutional caseload into a single measure of performance. To begin, expected mortality rates, adjusting for case mix, were calculated for each center. Risk categories 2, 3, 4, and 6 (no cases were placed in risk category 5, which contains very few procedures) were used as binary covariates in a logistic regression model predicting mortality, with category 1 as the reference group. The resulting model was then used to calculate the predicted probability of death for each individual case in the data set. The average predicted probability of death for all cases within a particular institution, which was calculated by summing the predicted probabilities for each case and dividing by the total number of cases, represents the expected mortality rate for that center, adjusting for case mix. The observed and expected mortality rates were compared by calculating the standardized mortality ratio (SMR) for each center, which was defined as the observed mortality rate divided by the expected mortality rate. If the observed mortality rate for an institution is higher than expected, meaning that the center performs worse than would be expected given its case mix, the SMR is greater than 1. If the institution's observed mortality rate is lower than expected, indicating better performance than would be anticipated, the SMR is less than 1. Institutions can then be ranked from lowest to highest on the basis of their individual SMR values, providing an assessment of relative performance for the overall caseload.
Similar methodology can be applied for the 4 additional clinical factors to improve the case-mix adjustment further. Binary covariates representing age 30 days or less, age 31 days to 1 year, prematurity, presence of a major noncardiac structural anomaly, and presence of combinations of cardiac surgical procedures were included in the logistic regression model already containing risk category. The resulting model was used to calculate the predicted probability of death for each patient in the data set; the average predicted probability of death for all patients within a particular institution is the expected mortality rate for that center, adjusting for risk category and the other clinical factors. Once again, observed and expected mortality rates for each institution were compared using SMR values, and institutions were ranked according to increasing SMR.
A 95% confidence interval was calculated for each SMR.
10 If the confidence interval contains the value 1, the difference in observed versus expected rates is not considered to be statistically significant.
| Results |
|---|
|
|
|---|
|
The caseload for each institution was then stratified by risk category. No cases were identified in category 5; this category was therefore excluded from further analyses. Within each of the remaining risk categories, mortality rates were computed for institutions performing 10 or more cases within that category. All 22 institutions performed at least 10 cases in risk categories 1, 2, and 3; 10 institutions performed 10 or more cases in category 4; and only 3 institutions performed 10 or more cases in category 6. Institutional mortality rates for each risk category are shown in Figure 1.
|
|
|
Expected mortality rates and SMRs adjusting for age, prematurity, presence of a major noncardiac structural anomaly, and combinations of cardiac procedures in addition to risk category are also shown in Table 2
to refine the case-mix adjustment further. The rankings displayed only minor changes.
The additional information provided by the more complex multivariate risk-adjustment method can be inferred from inspection of the expected institutional mortality rates in Table 2
. Although the mortality rate for the group as a whole is 4.0%, the expected mortality rates at each institution vary nearly 3-fold, from 2.4% to 7.0%.
| Discussion |
|---|
|
|
|---|
Despite the broad diversity of procedures performed at each of the centers, we were able to create risk-adjusted measures and rankings of overall performance for each institution. We applied the RACHS-1 methodology to examine risk category-specific mortality rates and to calculate SMRs, adjusting for risk category and other clinical factors. Although most of the descriptive information could be derived from the simpler method, which adjusted for risk category only, the incorporation of additional clinical factors also explained some of the differences in observed mortality, as evidenced by the changes in expected mortality rates when these additional variables were incorporated into the model.
In addition to the institutional performance rankings listed in Table 2
, important insights about mortality differences have come from analysis of the information within each risk category. All of the institutions in our analysis were able to perform procedures in risk category 1 with little, if any, mortality. For higher-risk procedures, several distinct patterns emerged. Some institutions displayed a consistent relative performance across risk categories. For institutions performing at better than average levels, focus should be placed on improving outcomes for higher-risk procedures, especially those in categories 4 and 6, in which overall mortality rates remain high. Institutions performing considerably worse than average for all risk categories may need to consider global, rather than incremental, programmatic changes. Several institutions showed worsening performance for higher-risk procedures. These patterns may suggest that institutions should consider targeted referral to centers with better performance or other measures to improve performance for higher-risk procedures. Some institutions displayed an interesting pattern of poorer performance for lower-risk than for higher-risk procedures. For programs with this pattern, careful intrainstitutional assessment should be made of possible systematic differences (eg, assignment of certain types of cases to specific surgeons and location of postoperative care) in the way that simpler cases are handled in comparison with more complex ones. In any case the finding of increasing or decreasing risk-adjusted performance as complexity increases should prompt focused inquiries within institutions about potential explanatory factors.
Although analyses such as these should prompt inquiry, definitive conclusions about quality of care at individual centers should not be made from this type of investigation. Data analyses should be considered exploratory, given the many possible competing explanations for higher-than-expected death rates at some institutions, including data-coding errors, case-mix differences insufficiently accounted for with the RACHS-1 method, and chance variability, especially for the analysis of a single time period. Nearly all of the differences demonstrated did not reach statistical significance in this single calendar year. Furthermore, inaccuracies in the data may be present because hospital discharge data are collected primarily to support billing and are verified by various statewide agencies charged with this purpose and not by investigators. We suspect that procedures might be more accurately coded than diagnoses as a part of the billing process, but this has not been tested. Similarly, despite its importance, in-hospital mortality is only one of the clinically important end points that should be of interest to program directors. Relative performance with respect to late mortality and functional and neurologic outcomes, as examples, would also be of obvious interest for efforts at quality improvement. Such analyses would require specific data collection and the development of risk-adjustment tools beyond RACHS-1. Better data sets would clearly improve the accuracy of overall assessments in general.
| Summary |
|---|
|
|
|---|
| Acknowledgments |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
T. Domi, D. S. Edgell, B. W. McCrindle, W. G. Williams, A. K. Chan, D. L. MacGregor, A. Kirton, and G. A. deVeber Frequency, Predictors, and Neurologic Outcomes of Vaso-occlusive Strokes Associated With Cardiac Surgery in Children Pediatrics, December 1, 2008; 122(6): 1292 - 1298. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. A. Bacha, L. A. Larrazabal, F. A. Pigula, K. Gauvreau, K. J. Jenkins, S. D. Colan, F. Fynn-Thompson, J. E. Mayer Jr., and P. J. del Nido Measurement of technical performance in surgery for congenital heart disease: The stage I Norwood procedure J. Thorac. Cardiovasc. Surg., October 1, 2008; 136(4): 993 - 997. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. F. Welke, B. S. Diggs, T. Karamlou, and R. M. Ungerleider The Relationship Between Hospital Surgical Case Volumes and Mortality Rates in Pediatric Cardiac Surgery: A National Sample, 1988-2005 Ann. Thorac. Surg., September 1, 2008; 86(3): 889 - 896. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. A. Larrazabal, K. J. Jenkins, K. Gauvreau, V. L. Vida, O. J. Benavidez, G. A. Gaitan, F. Garcia, and A. R. Castaneda Improvement in Congenital Heart Surgery in a Developing Country: The Guatemalan Experience Circulation, October 23, 2007; 116(17): 1882 - 1887. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Mildh, V. Pettila, H. Sairanen, and P. Rautiainen Predictive value of paediatric risk of mortality score and risk adjustment for congenital heart surgery score after paediatric open-heart surgery Interactive CardioVascular and Thoracic Surgery, October 1, 2007; 6(5): 628 - 631. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Miyaji, S. Kohira, T. Miyamoto, K. Nakashima, H. Sato, K. Ohara, and H. Yoshimura Pediatric cardiac surgery without homologous blood transfusion, using a miniaturized bypass system in infants with lower body weight J. Thorac. Cardiovasc. Surg., August 1, 2007; 134(2): 284 - 289. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. O. Al-Radi, F. E. Harrell Jr, C. A. Caldarone, B. W. McCrindle, J. P. Jacobs, M. G. Williams, G. S. Van Arsdell, and W. G. Williams Case complexity scores in congenital heart surgery: A comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system J. Thorac. Cardiovasc. Surg., April 1, 2007; 133(4): 865 - 875. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. H. Mildh, V. Pettila, H. I. Sairanen, and P. H. Rautiainen Cardiac Troponin T Levels for Risk Stratification in Pediatric Open Heart Surgery Ann. Thorac. Surg., November 1, 2006; 82(5): 1643 - 1648. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L. Hannan, M. A. Ybarra, J. W. Ojito, F. A. Alonso, A. F. Rossi, and R. P. Burke Complex Neonatal Single Ventricle Palliation Using Antegrade Cerebral Perfusion. Ann. Thorac. Surg., October 1, 2006; 82(4): 1278 - 1285. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. F. Welke, I. Shen, and R. M. Ungerleider Current assessment of mortality rates in congenital cardiac surgery. Ann. Thorac. Surg., July 1, 2006; 82(1): 164 - 170. [Abstract] [Full Text] [PDF] |
||||
![]() |
M Hatherill, S Salie, Z Waggie, J Lawrenson, J Hewitson, L Reynolds, and A Argent Hyperchloraemic metabolic acidosis following open cardiac surgery Arch. Dis. Child., December 1, 2005; 90(12): 1288 - 1292. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L. Hannan, M. A. Ybarra, J. A. White, J. W. Ojito, A. F. Rossi, and R. P. Burke Patterns of Lactate Values after Congenital Heart Surgery and Timing of Cardiopulmonary Support Ann. Thorac. Surg., October 1, 2005; 80(4): 1468 - 1474. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. A. Connor, K. Gauvreau, and K. J. Jenkins Factors Associated With Increased Resource Utilization for Congenital Heart Disease Pediatrics, September 1, 2005; 116(3): 689 - 695. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Riphagen, M. McDougall, S. M. Tibby, N. Alphonso, D. Anderson, C. Austin, A. Durward, and I. A. Murdoch "Early" Delayed Sternal Closure Following Pediatric Cardiac Surgery Ann. Thorac. Surg., August 1, 2005; 80(2): 678 - 684. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Khoshnood, C. De Vigan, V. Vodovar, J. Goujard, A. Lhomme, D. Bonnet, and F. Goffinet Trends in Prenatal Diagnosis, Pregnancy Termination, and Perinatal Mortality of Newborns With Congenital Heart Disease in France, 1983-2000: A Population-Based Evaluation Pediatrics, January 1, 2005; 115(1): 95 - 101. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Kang, T. Cole, V. Tsang, M. Elliott, and M. de Leval Risk stratification in paediatric open-heart surgery Eur. J. Cardiothorac. Surg., July 1, 2004; 26(1): 3 - 11. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. H. Shuhaiber Risk adjustment in analysis of surgery for congenital heart disease J. Thorac. Cardiovasc. Surg., May 1, 2003; 125(5): 1171 - 1171. [Full Text] [PDF] |
||||
![]() |
D. Spiegelhalter Ranking institutions J. Thorac. Cardiovasc. Surg., May 1, 2003; 125(5): 1171 - 1173. [Full Text] [PDF] |
||||
![]() |
D. A. Danford Relative Institutional Performance in Congenital Heart Surgery AAP Grand Rounds, October 1, 2002; 8(4): 45 - 46. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |