HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Osman O. Al-Radi Christopher A. Caldarone Brian W. McCrindle Jeffrey P. Jacobs Glen S. Van Arsdell William G. Williams Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Al-Radi, O. O. Articles by Williams, W. G. Search for Related Content PubMed PubMed Citation Articles by Al-Radi, O. O. Articles by Williams, W. G. Related Collections Congenital - cyanotic Related Article

J Thorac Cardiovasc Surg 2007;133:865-875
© 2007 The American Association for Thoracic Surgery

Surgery for Congenital Heart Disease

Case complexity scores in congenital heart surgery: A comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system

^a Hospital for Sick Children, University of Toronto, Toronto, Canada
^b Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, Tenn.
^c The Congenital Heart Institute of Florida, University of South Florida, Saint Petersburg, Fla.

Read at the Eighty-fifth Annual Meeting of The American Association for Thoracic Surgery, San Francisco, Calif, April 10-13, 2005.

Received for publication April 20, 2005; revisions received April 26, 2006; accepted for publication May 17, 2006.

^_* Address for reprints: William G. Williams, MD, 555 University Avenue, Room 1525, Toronto, ON, M5G 1X8, Canada. (Email: bill.williams{at}sickkids.ca).

	Abstract

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
Objective: The Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery system were developed by consensus to compare outcomes of congenital cardiac surgery. We compared the predictive value of the 2 systems.

Methods: Of all index congenital cardiac operations at our institution from 1982 to 2004 (n = 13,675), we were able to assign an Aristotle Basic Complexity score, a Risk Adjustment in Congenital Heart Surgery score, and both scores to 13,138 (96%), 11,533 (84%), and 11,438 (84%) operations, respectively. Models of in-hospital mortality and length of stay were generated for Aristotle Basic Complexity and Risk Adjustment in Congenital Heart Surgery using an identical data set in which both Aristotle Basic Complexity and Risk Adjustment in Congenital Heart Surgery scores were assigned. The likelihood ratio test for nested models and paired concordance statistics were used.

Results: After adjustment for year of operation, the odds ratios for Aristotle Basic Complexity score 3 versus 6, 9 versus 6, 12 versus 6, and 15 versus 6 were 0.29, 2.22, 7.62, and 26.54 (P < .0001). Similarly, odds ratios for Risk Adjustment in Congenital Heart Surgery categories 1 versus 2, 3 versus 2, 4 versus 2, and 5/6 versus 2 were 0.23, 1.98, 5.80, and 20.71 (P < .0001). Risk Adjustment in Congenital Heart Surgery added significant predictive value over Aristotle Basic Complexity (likelihood ratio ² = 162, P < .0001), whereas Aristotle Basic Complexity contributed much less predictive value over Risk Adjustment in Congenital Heart Surgery (likelihood ratio ² = 13.4, P = .009). Neither system fully adjusted for the child’s age. The Risk Adjustment in Congenital Heart Surgery scores were more concordant with length of stay compared with Aristotle Basic Complexity scores (P < .0001).

Conclusions: The predictive value of Risk Adjustment in Congenital Heart Surgery is higher than that of Aristotle Basic Complexity. The use of Aristotle Basic Complexity or Risk Adjustment in Congenital Heart Surgery as risk stratification and trending tools to monitor outcomes over time and to guide risk-adjusted comparisons may be valuable.

	Introduction

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 

Dr Al-Radi

Because each congenital heart defect is a rare condition, assessing the quality of care based on crude outcomes is problematic. Several groups have proposed systems of assessing quality of care by assigning surgical operations a risk score or grouping operations of similar risk into categories. Two such systems, namely the Aristotle Basic Complexity (ABC) score and the Risk Adjustment for Congenital Heart Surgery (RACHS-1), were developed by consensus of experts.^1,2 The methodologic details of each system are described in the respective references. Briefly, the Aristotle committee, consisting of experts from 50 centers in 23 countries, developed the ABC score. Potential for mortality, potential for morbidity, and technical difficulty for each operation contribute up to 5 points each to this continuous score (range 1.5 to 15). The score was used by its authors to group the procedures as follows: level 1, scores 1.5 to 5.9; level 2, scores 6 to 7.9; level 3, scores 8 to 9.9; and level 4, scores 10 to 15.^3-5 On the other hand, in the RACHS-1 system, which was developed between 1993 and 1995, congenital cardiac operations were stratified into 1 of 6 categories. The risk category of some procedures additionally varied depending on age.^6-7 RACHS-1 was validated in 2 independent populations and was found to have good predictive value.^8,9 However, no studies have compared the predictive value of the 2 systems in the same population.

We sought to assess the predictive value of ABC and RACHS-1 by comparing in-hospital mortality and length of stay as predicted by the respective system with the observed in-hospital mortality and length of stay at our institution.¹⁰

	Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
The outcomes of 13,675 index (first operation of an admission) congenital cardiac surgeries performed on children (age < 18 years) between July 1, 1982 and June 30, 2004 were available in the cardiovascular surgery database (CVSDB) at the Hospital for Sick Children. These index operations were performed on 10,860 children who had a total of 16,538 cardiac operations. More than 1 index operation was performed on 1937 (19% of 10,860) children. Only the index operations with both ABC score and RACHS-1 category assigned were included in the study (n = 11,438). CVSDB is a prospective clinical database. Data in CVSDB are maintained by a dedicated staff and are validated by monthly audits of operating room logs, surgeons’ office files, morbidity and mortality conferences, and a clinical nurse coordinator. Monthly output reports from CVSDB are sent to the faculty. An automated algorithm was used to assign ABC score and RACHS-1 values to the procedure codes in CVSDB. Because of the paucity of RACHS-1 category 5, it was combined with category 6.

For in-hospital mortality, logistic regression models were generated for ABC and RACHS-1 separately and then combined in 1 model. All 3 models included an identical set of operations. ABC score was modeled as a continuous variable with appropriate transformation using restricted cubic splines to account for nonlinear relationships.¹¹ Compared with traditional transformations (log, square root, or polynomials), cubic splines are better suited for biologic associations as they allow flexibility for nonlinear relationships.¹² The locations of change in the curvature are set at specific points known as knots. Five knots at quantiles of the predictors were used in our analyses. Because 1937 children had more than 1 index operation, Huber–White robust sandwich estimates of the variance covariance matrix were used to penalize for clustering by patient in all logistic models.^13-16 The predictive value of the models was assessed by the area under the receiver operator characteristics (ROC) curve, also known as the c-index, the model likelihood ratio (LR) ² statistic, and the adequacy index.¹⁷ To test for a difference in the predictive value of the 2 systems, we used the LR ² test for nested models to assess whether ABC adds predictive value to a model that includes RACHS-1 and whether RACHS-1 adds predictive value to a model that includes ABC. These analyses were done with and without adjustment for year of operation and the child’s age at operation. Such tests are more sensitive than tests comparing ROC areas (c-index).¹¹ However, the comparison between the ROC areas was also done and presented for the sake of completeness. The latter was obtained using bootstrap confidence intervals (CIs) from 1000 resamples. The adequacy index is the fraction of the total LR ² explained by a set of variables that could be explained by omitting the competing variable. The clinical utility of predictive models was assessed by the frequency of patients identified by the model with very low or very high risk of death. Models with higher frequency of extreme predictions are more likely to be clinically useful. Model calibration was assessed by bootstrap estimates of predicted mortality versus actual mortality.¹¹

For length of stay, a rank correlation U-statistic for paired censored data was used to estimate the fraction of pairs for which the prediction using RACHS-1 was more discriminating compared with ABC,¹⁴ and both were analyzed as continuous variables. Patients who died before discharge were censored in this analysis. A competing risk analysis without censoring death but rather treating it as a competing event was also conducted to produce cumulative incidence plots.^18,19

Mathematical representations of the logistic models are presented in an appendix (Appendix A). The R statistical package, Hmisc,¹⁴ Design,¹¹ and Cmprsk^18,19 libraries (www.r-project.org) were used for all analyses.

	Results

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
Of the 13,675 index operations in CVSDB, an ABC score could be assigned to 13,138 (96%) operations, a RACHS-1 category could be assigned to 11,533 (84%) operations, and both ABC and RACHS-1 could be assigned to 11,438 (84%) operations. Only operations that were assigned both ABC and RACHS-1 (n = 11,438) were used in all subsequent analyses. Patient demographics and crude outcomes are presented in Table 1. Exploratory plots of hospital death (observed and predicted) versus ABC score levels and RACHS-1 categories are shown in Figure 1, A and B, respectively. As illustrated in both plots, the probability of hospital death declined with time and markedly so during the 1990s. Improvement in survival was most significant for high-risk procedures, such as those with ABC score above 9.9 or RACHS-1 categories 4 and 5/6 (Figure 1).

View larger version (15K): [in this window] [in a new window]   Figure 1. Observed (gray) and predicted (black) probability of in-hospital death over calendar year of operation. A, For ABC levels (score range), 1 (1.5-5.9), 2 (6-7.9), 3 (8-9.9), and 4 (10-15). B, For RACHS-1 categories 1, 2, 3, 4, and 5/6.

Similarly, RACHS-1 category 2, which was the median category, was chosen as the reference category. The ORs adjusted for year of operation were 0.23, 1.98, 5.80, and 20.71 for RACHS-1 categories 1 versus 2, 3 versus 2, 4 versus 2, and 5/6 versus 2 (CIs: 0.14-0.36, 1.64-2.40, 4.64-7.26, and 15.52-27.64, respectively; P < .0001).

Is the Predictive Value of ABC Higher or Lower than That of RACHS-1?
The predictive values measured by the c-index and LR ² for ABC score and RACHS-1 with and without adjustment for the year of operation are shown in Table 2. Both the c-index and the LR ² are higher for RACHS-1 models. Using the LR ² test for nested models, ABC did not add predictive value to a model that includes RACHS-1 (LR ² = 6.2, df = 4, P = .18), whereas RACHS-1 added clinically and statistically significant predictive value to a model that includes ABC (LR ² = 182, df = 4, P < .0001). The difference between the c-index of ABC and RACHS-1 models was also significant (P = .018, c-index 0.698 vs 0.733, respectively).

The adequacy index (ie, the proportion of predictive LR ² value attributable to ABC score) and ABC score adjusted for year of operation were 72% and 80%, respectively. On the other hand, the adequacy index for RACHS-1 and RACHS-1 adjusted for year of operation was 99% and 98%, respectively. Therefore, ABC was sensitive to adjustment for the year of operation, whereas RACHS-1 was not. Figure 2 summarizes the comparison between the predictive value of ABC and that of RACHS-1, as well as the effect of adjustments for year of operation.

View larger version (14K): [in this window] [in a new window]   Figure 2. The predictive value of robust logistic models of ABC, RACHS-1, and both scores combined. Unadjusted models, models adjusted for the calendar year of operation, and models adjusted for calendar year of operation and the child’s age at operation are presented. The predictive value is represented by the model LR and adequacy index (% likelihood ratio 2 attributable to the nested model, which does include both ABC and RACHS-1; see Methods). P values shown are of an LR 2 test for nested models with 4 degrees of freedom.

View larger version (16K): [in this window] [in a new window]   Figure 3. The number of patients (Frequency) assigned to intervals of predicted risk by ABC, RACHS-1, or both with extreme (ie, very low [<1%] and very high [>15%]) scores predicted in-hospital mortality. Models with more frequent extreme predictions are more clinically useful. The cutoffs of 1% and 15% were approximately the 5th and 95th percentiles of predicted risk of in-hospital death.

Is ABC Score and/or RACHS-1 Associated With the Child’s Length of Stay in the Hospital?
Postoperative length of stay in the hospital was strongly associated with year of operation, ABC score, and RACHS-1. RACHS-1, however, was more concordant with length of stay compared with ABC (P < .0001). A competing risk analysis demonstrated that both ABC and RACHS-1 were predictive of hospital discharge and death when they were treated as competing risk events. The cumulative incidence plots for death and discharge from hospital for each ABC level or RACHS-1 category are presented in Figure 4, A and B, respectively. The risk of death increased with each increase in ABC score or RACHS-1. The mean length of stay decreased for each increase in ABC score or RACHS-1.

View larger version (20K): [in this window] [in a new window]   Figure 4. Cumulative incidence plots of death before discharge versus discharge alive over time from the day of operation (time 0). A, For ABC levels (score range), 1 (1.5-5.9), 2 (6-7.9), 3 (8-9.9), and 4 (10-15). B, For RACHS-1 categories 1, 2, 3, 4, and 5/6. The time point (x-axis point) corresponding to a cumulative incidence (y-axis point) of 0.5 on the dashed curve represents the mean time to discharge from hospital.

	Discussion

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
RACHS-1, as the name implies, was developed as a method of risk adjustment in congenital heart surgery.¹ Our analysis showed that RACHS-1 was strongly associated with in-hospital mortality and with length of stay. However, RACHS-1 is better characterized as a method of risk stratification than as adjustment. As Jenkins and colleagues¹ acknowledged, every child is different. We have shown that the child’s age at operation is an important prognostic factor that is only partially accounted for within a few RACHS-1 category codes. The predictive value of future versions of RACHS-1 may be improved by adding an adjustment for age to more diagnostic codes. The addition of other procedure-specific key predictive factors may further improve RACHS-1 predictive value.

The Aristotle committee intended to assess the "performance" of surgical care providers and hypothesized that performance = outcomes x complexity.² We did not assess this hypothesis; rather, we focused on assessing ABC as it correlates with short-term outcomes, namely in-hospital mortality and length of stay. There was a strong association between ABC and in-hospital mortality and length of stay; however, its predictive value was lower than that of RACHS-1. This was attributable to some extent to its failure to adjust for the child’s age. When we adjusted ABC by including age at operation in predictive models with ABC, the predictive value of such models improved to a level very close to that of RACHS-1.

Further, the predictions of both systems need to be adjusted for the year of operation to account for improvements in outcomes of congenital heart surgery that have occurred over the last 2 decades and that will continue to occur in the future.

ROC curves are used to compare the predictive value of risk models extensively in the medical literature. However, because the resultant c-index (area under the ROC curve) is a rank-based statistic, it fails to reward extreme predictions that are true and fails to penalize extreme predictions that are false. Therefore, the c-index is insensitive to potentially important differences; a small difference in the c-index may actually be of large clinical and statistical importance. For this study, we chose a more sensitive statistic known as the LR ², which is not rank based and appropriately rewards correct extreme predictions. In the case of our analysis, due to the large sample size, the 2 methodologies result in the same conclusions, which were both presented, namely a significant difference between ABC and RACHS-1. However, the magnitude of the difference is more prominent using the LR method versus the ROC c-index method.

Both ABC and RACHS-1 are, however, useful guides to assess the quality of surgical care providers over time. The graphical exploration of trends over time and a comparison of institutional outcomes within risk levels may be useful in detecting outliers and in generating hypotheses about differences between institutions or methods of care, as in Figures 1 and 4. Importantly, however, a comparison of institutions on the basis of in-hospital mortality is a very blunt measure of a much more complex scenario and unlikely to allow fair comparisons or meaningful information upon which to base changes in practice. Specific hypotheses to compare or improve quality of care must be tested with truly risk-adjusted models using more comprehensive data, in a way that would let the data speak for themselves.²⁰

	Limitations

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
More index operations were assigned an ABC score than were assigned a RACHS-1 category, 94% versus 86%. However, to adequately compare the 2 systems, only operations that were assigned both an ABC score and a RACHS-1 category were included in the analysis. When analyses were done using all possible procedures that were assigned either an ABC score or a RACHS-1 category, the overall conclusions were the same as those of the analyses presented here.

Because 19% of the children had more than 1 index operation, clustering of the outcome by patient was taken into account. This was achieved by penalizing the model estimates to account for clustering.

In-hospital mortality was the outcome available, and it was rigorously validated in our database. However, it does not completely represent the mortality associated with the early hazard phase described post-cardiac surgery.²¹ Notwithstanding the fact that the 2 systems (ABC and RACHS-1) were designed to predict short-term outcomes, the relationship between the scores and time-related survival, both in early and late hazard phases, will be of future interest.

	Conclusions

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
We have shown that both ABC and RACHS-1 have a strong association with in-hospital mortality and length of stay. The predictive value of RACHS-1 is higher than that of ABC. Adding patient- and procedure-specific variables may improve their predictive value. Neither system in isolation is adequate for risk-adjusted comparisons between providers of care or institutions. Their use as risk stratification and trending tools to monitor outcomes over time, with the intention to guide risk-adjusted comparisons, could be valuable.

	Appendix

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 
Logistic models 1, 2, and 3 were used to model ABC score, unadjusted, adjusted for year of operation, and adjusted for year of operation and the child’s age, respectively. In-hospital death was the outcome variable.

Model 1

(1)

where X = –6.006689 + 0.5913527 ABC – 0.01976131 (ABC-3)₊ ³ + 0.2401961(ABC-6)₊ ³ –0.2436142(ABC-6.5)₊ ³ + 0.02857559(ABC-9)₊ ³ – 0.005396189(ABC-10.3)₊ ³ and (x) = x if x > 0, 0 otherwise.

Model 2

(2)

where X = 63.52153 + 0.5333782ABC – 0.01307427(ABC-3)₊ ³ + 0.1365018(ABC-6)₊ ³ –0.1291932(ABC-6.5)₊ ³ – 0.0004472514(ABC-9)₊ ³ + 0.006212918(ABC-10.3)₊ ³ –0.03479037 Year + 0.000424993(Year-1983)₊ ³ – 0.002778462(Year-1988)₊ ³ + 0.003802972(Year-1992)₊ ³ – 0.001442604(Year-1997)₊ ³ – 6.898965 x 10^–6(Year-2003)₊ ³ and (x)_– = x if x > 0, 0 otherwise.

Model 3

(3)

where X = 37.21254 – 0.2004645ABC + 0.04727446(ABC-3)₊ ³ – 0.7220843(ABC-6)₊ ³ +0.7783477(ABC-6.5)₊ ³ – 0.1522020(ABC-9)₊ ³ + 0.04866409(ABC-10.3)₊ ³ –0.01958611Year – 0.000331136(Year-1983)₊ ³ + 4.973796 x 10^–5(Year-1988)₊ ³ + 0.0004777783(Year-1992)₊ ³ + 0.000103515(Year-1997)₊ ³ – 0.0002998952(Year-2003)₊ ³ –0.3045867Age + 0.002549708(Age-0.13)₊ ³ – 0.003324442(Age-3.05)₊ ³ +0.0007674379(Age-12.33)₊ ³ + 7.35287 x 10^–6 (Age-47.61)₊ ³ – 5.142649 x 10^–8 (Age-164.96)₊ ³ and (x)₊ = x if x > 0, 0 otherwise.

The model coefficients and predictive value statistics are shown in Table A1. Logistic models 4, 5, and 6 were used to model RACHS-1, unadjusted, adjusted for year of operation, and adjusted for year of operation and the child’s age, respectively. In-hospital death was the outcome variable:

(4)

where X = –4.577 + 1.413 {RACHS = 2} + 2.132 {RACHS = 3} + 3.184 {RACHS = 4} + 4.158 {RACHS = 5/6} and {c} = 1 if subject is in group c, 0 otherwise; (x)₊ = x if x > 0, 0 otherwise

Model 5

(5)

where X = –18.94 + 1.486 {RACHS = 2} + 2.171 {RACHS = 3} + 3.245 {RACHS = 4} + 4.517 {RACHS = 5/6} + 0.00735 Year + 0.0001281(Year-1983)₊ ³ – 0.001991(Year-1988)₊ ³ + 0.003059(Year-1992)₊ ³ –0.001056(Year-1997)₊ ³ – 0.0001395(Year-2003)₊ ³ and {c} = 1 if subject is in group c, 0 otherwise; (x)₊ = x if x > 0, 0 otherwise

Model 6

(6)

where X = –55.25 + 1.372 {RACHS = 2} + 1.902 {RACHS = 3} + 2.544 {RACHS = 4} + 3.538 {RACHS = 5/6} + 0.02628 Year – 0.0005129(Year-1983)₊ ³ + 0.0003196(Year-1988)₊ ³ + 0.0004602(Year-1992)₊ ³ + 6.697 x 10^–5(Year-1997)₊ ³ – 0.0003338(Year-2003)₊ ³ –0.2219 Age + 0.001799(Age-0.13)₊ ³ – 0.002338(Age-3.05)₊ ³ + 0.0005317(Age-12.33)₊ ³ +7.442 x 10^–6(Age-47.61)₊ ³ – 1.054 x 10^–7(Age-165)₊ ³ and {c} = 1 if subject is in group c, 0 otherwise; (x)₊ = x if x > 0, 0 otherwise.

The model coefficients and predictive value statistics are shown in Table A2.

	References

Top Abstract Introduction Materials and Methods Results Discussion Limitations Conclusions Appendix References

 

1. Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123:110-118.[Abstract/Free Full Text]
   
2. Lacour-Gayet F, Clarke D, Jacobs J, Comas J, Daebritz S, Daenen W, et al. The Aristotle score: a complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004;25:911-924.[Abstract/Free Full Text]
   
3. Lacour-Gayet F. Risk stratification theme for congenital heart surgery. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2002;5:148-152.[Medline]
   
4. Lacour-Gayet F, Clarke D, Jacobs J, Gaynor W, Hamilton L, Jacobs M, et al. The Aristotle score for congenital heart surgery. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2004;7:185-191.[Medline]
   
5. Lacour-Gayet F, Clarke DR. The Aristotle method: a new concept to evaluate quality of care based on complexity. Curr Opin Pediatr 2005;17:412-417.[Medline]
   
6. Jenkins KJ, Gauvreau K. Center-specific differences in mortality: preliminary analyses using the Risk Adjustment in Congenital Heart Surgery (RACHS-1) method. J Thorac Cardiovasc Surg 2002;124:97-104.[Abstract/Free Full Text]
   
7. Jenkins KJ. Risk adjustment for congenital heart surgery: the RACHS-1 method. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2004;7:180-184.[Medline]
   
8. Larsen SH, Pedersen J, Jacobsen J, Johnsen SP, Hansen OK, Hjortdal V. The RACHS-1 risk categories reflect mortality and length of stay in a Danish population of children operated for congenital heart disease. Eur J Cardiothorac Surg 2005;28:877-881.[Abstract/Free Full Text]
   
9. Boethig D, Jenkins KJ, Hecker H, Thies WR, Breymann T. The RACHS-1 risk categories reflect mortality and length of hospital stay in a large German pediatric cardiac surgery population. Eur J Cardiothorac Surg 2004;26:12-17.[Abstract/Free Full Text]
   
10. Iezzoni LI. Risk adjustment for measuring health care outcomes. 3rd ed.. Chicago: Health Administration Press; 2003.
    
11. Harrell Jr FE. Regression modeling strategies with applications to linear models, logistic regression, and survival analysis. New York: Springer; 2001.
    
12. Harrell Jr FE, Lee KL, Pollock BG. Regression models in clinical studies: determining relationships between predictors and response. J Natl Cancer Inst 1988;80:1198-1202.[Abstract/Free Full Text]
    
13. White H. Maximum likelihood estimation of misspecified models. Econometrica 1982;50:1-25.
    
14. Hmisc: A Package of Miscellaneous S Functions. 2006.
    
15. Proc Fifth Berkeley Symposium Math Stat. 1967.
    
16. Feng Z, McLerran D, Grizzle J. A comparison of statistical methods for clustered data analysis with Gaussian error. Stat Med 1996;15:1793-1806.[Medline]
    
17. Califf RM, Phillips III HR, Hindman MC, Mark DB, Lee KL, Behar VS, et al. Prognostic value of a coronary artery jeopardy score. J Am Coll Cardiol 1985;5:1055-1063.[Abstract]
    
18. Aalen OO. Nonparametric estimation of partial transition probabilities in multiple decrement models. Annals of Statistics 2006;61:534-545.
    
19. Gray RJ. A class of K-sample tests for comparing the cumulative incidence of a competing risk. Annals of Statistics 1988;16:1141-1154.
    
20. Blackstone EH. Let the data speak for themselves?. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2004;7:192-198.[Medline]
    
21. Monro JL. The next challenge-adapting to change. Eur J Cardiothorac Surg 2004;26:1063-1072.[Free Full Text]
    

This article has been cited by other articles:

				K. F. Welke, B. S. Diggs, T. Karamlou, and R. M. Ungerleider The Relationship Between Hospital Surgical Case Volumes and Mortality Rates in Pediatric Cardiac Surgery: A National Sample, 1988-2005 Ann. Thorac. Surg., September 1, 2008; 86(3): 889 - 896. [Abstract] [Full Text] [PDF]

				S. M. O'Brien, J. P. Jacobs, D. R. Clarke, B. Maruszewski, M. L. Jacobs, H. L. Walters III, C. I. Tchervenkov, K. F. Welke, Z. Tobota, G. Stellin, et al. Accuracy of the Aristotle Basic Complexity Score for Classifying the Mortality and Morbidity Potential of Congenital Heart Surgery Operations Ann. Thorac. Surg., December 1, 2007; 84(6): 2027 - 2037. [Abstract] [Full Text] [PDF]

				L. Mildh, V. Pettila, H. Sairanen, and P. Rautiainen Predictive value of paediatric risk of mortality score and risk adjustment for congenital heart surgery score after paediatric open-heart surgery Interactive CardioVascular and Thoracic Surgery, October 1, 2007; 6(5): 628 - 631. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Osman O. Al-Radi Christopher A. Caldarone Brian W. McCrindle Jeffrey P. Jacobs Glen S. Van Arsdell William G. Williams Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Al-Radi, O. O. Articles by Williams, W. G. Search for Related Content PubMed PubMed Citation Articles by Al-Radi, O. O. Articles by Williams, W. G. Related Collections Congenital - cyanotic Related Article