J Thorac Cardiovasc Surg 2005;129:1318-1321
© 2005 The American Association for Thoracic Surgery
Surgery for Acquired Cardiovascular Disease |
A validated simple model to predict coexistent coronary disease in patients undergoing mitral valve surgery
Eric Lim, MRCSa,*,
Ayyaz Ali, MRCSa,
Zain Khalpey, MRCSa,
Hutan Ashrafian, MBBSa,
Christopher Jackson, PhDc,
Ziad Ali, MRCSa,
Themis Chamageorgakis, FRCSa,
Francis Wells, FRCSb,
John Pepper, FRCSa,
Anthony DeSouza, FRCSa,
Neil Moat, FRCSa
a Department of Cardiothoracic Surgery, Royal Brompton Hospital, London, United Kingdom
b Department of Cardiothoracic Surgery, Papworth Hospital, Cambridge, United Kingdom
c Department of Epidemiology and Public Health, Imperial College School of Medicine and Technology, London, United Kingdom.
Received for publication September 16, 2004; revisions received October 16, 2004; accepted for publication October 28, 2004.
* Address for reprints: Eric Lim, Department of Cardiothoracic Surgery, Papworth Hospital, Cambridge CB3 8RE, United Kingdom. (Email: eric.lim{at}cvsnet.org).
 |
Abstract
|
|---|
OBJECTIVE: The primary limitation of the American Heart Association/American College of Cardiology guidelines is specificity. To improve the selection process, we proposed a simple additive model including age (1 point for every 5 years above 50), male sex (2 points), hypercholesterolemia (2 points), angina (3 points), and electrocardiographic evidence of ischemia (3 points). We recommend screening angiography at 3 or more points. This model was previously derived from 359 patients at Papworth Hospital.
METHODS: The validation cohort was a consecutive series of patients who underwent mitral valve surgery at the Royal Brompton Hospital. Preoperative coronary angiography reports were obtained, and coronary disease was defined as luminal narrowing of 50% in 2 or more views. Sensitivities and specificities were calculated for the American Heart Association/American College of Cardiology criteria, the simple additive model, and a logistic regression model. Receiver operating characteristic curves were used to validate accuracy and compare discrimination with logistic regression.
RESULTS: From 1998 through 2003, angiographic details were available for 342 (86%) of 396 patients who underwent mitral valve surgery. The sensitivity and specificity of the American Heart Association/American College of Cardiology guidelines were 100% and 5%, respectively; those of the simple additive model were 91% and 44%, respectively; and those of logistic regression were 93% and 41%, respectively. The receiver operating characteristic areas for the simple additive and logistic regression model were 0.78 (95% confidence interval, 0.73–0.84) and 0.80 (95% confidence interval, 0.74–0.85), respectively.
CONCLUSIONS: This is the third independent cohort to highlight the poor specificity of the American Heart Association/American College of Cardiology guidelines. Although high sensitivity is achieved, the cost is the majority of patients requiring screening angiography. Our validated simple model improved the specificity and selection; however, this was achieved at the expense of decreased sensitivity.
Screening for coexistent coronary disease remains an important aspect of the assessment of patients before mitral valve surgery. Coronary and mitral valve disease are inextricably linked as a causative agent for mitral valve disease (eg, ischemic mitral regurgitation), a coexistent agent that modifies operative strategy (eg, concomitant coronary disease in patients with degenerative mitral valve disease), and an adverse prognostic predictor of survival.1
The primary limitation of the widely used American Heart Association/American College of Cardiology (AHA/ACC) guidelines2 is specificity (previously estimated at 1%).3 The inability to accurately rule in coexistent coronary disease (the selection process) leads to almost universal recommendation for preoperative screening angiography.
A number of proposals have been made to improve selection criteria and increase the discriminatory ability achieved with sophisticated statistical models.4 Although mathematically accurate, they are difficult to implement at the bedside or consulting room. In 2003, we proposed a simple additive model derived from a cohort of patients in Cambridge, with the following 5 variables: age (1 point for every 5 years above 50), male sex (2 points), hypercholesterolemia (2 points), angina (3 points), and electrocardiographic evidence of ischemia (3 points). We recommend screening angiography at 3 or more points.
The aim of this study was to validate and assess the discriminatory value of our simple additive model on a different cohort of patients who underwent mitral valve surgery at the Royal Brompton Hospital in London, United Kingdom.
|
Methods
|
|---|
The validation cohort was identified from a database of a consecutive series of patients who underwent mitral valve surgery from 1998 through 2003 at the Royal Brompton Hospital in a 3-surgeon series (J.P., A.D.S., and N.M.). Reports from preoperative screening coronary angiography were obtained from patients listed for elective mitral valve surgery. Experienced cardiologists had reviewed the coronary angiograms, and the degree of luminal narrowing was obtained by means of visual estimation. The presence of significant coronary disease was defined as luminal narrowing of 50% in 2 or more views.
Risk factors for coronary artery disease were defined as follows: age (
35 years for men and 51 years for women), family history (first-degree relative with a myocardial infarction before the age of 50 years in men and 60 years in women), smoking, diabetes, and hypercholesterolemia (defined as receiving medication for hypercholesterolemia or serum cholesterol of 5.0 mmol/L [193 mg/dL]). Evaluated AHA/ACC indications were a history of angina or myocardial infarction or the presence of one or more risk factors for coronary artery disease. The presence of ischemic changes on echocardiography was defined as any resting ST-segment or T-wave abnormality. The cause of mitral regurgitation was determined by means of operative assessment in conjunction with histopathologic examination of valve specimens.
Statistical methodology for the development of our simple additive model (from logistic regression analysis) has been previously described.3 Sensitivities and specificities were calculated for the AHA/ACC criteria, our simple additive model, and a logistic regression model. Receiver operating characteristic (ROC) curves were used to validate the accuracy of our simple additive model and also to compare the discrimination of our model with that of a full logistic regression model derived by Lin and colleagues4 from the Cleveland Clinic (both models are detailed in the Appendix).
|
Results
|
|---|
From January 1, 1998, to January 1, 2003, a total of 396 patients underwent mitral valve surgery. Angiographic details were unavailable for 54 (14%) patients, 41 with and 13 without AHA/ACC-defined indications, leaving 342 patients. The validation cohort had a mean (SD) age of 65 (11) years, and a total of 182 (61%) had mitral valve repair, and 114 (39%) had mitral valve replacement (Table 1).
The sensitivities and specificities of the AHA/ACC guidelines were 100% and 5%, respectively; those of the simple additive model were 91% and 44%, respectively; and those of the logistic regression model were 93% and 41%, respectively (Tables 2 and 3).
View this table:
[in this window]
[in a new window]
|
TABLE 2. Sensitivity and specificity of AHA/ACC criteria, the simple additive model, and full logistic regression (Lin and colleagues4)
|
|
The area under the ROC curves for the simple additive and logistic regression models were 0.78 (95% confidence interval [CI], 0.73–0.84) and 0.80 (95% CI, 0.74–0.85), respectively (Figure 1).
|
Discussion
|
|---|
This is now the third study (in 3 independent patient populations) that highlight the limitations of the AHA/ACC selection criteria for recommendations of preoperative screening angiography in patients before mitral valve surgery.3,4 The intrinsic problem lies in specificities that ranged from 1% to 5%, the latter achieved in this study. Our results demonstrate that it is possible to increase the specificity of the screening guidelines to 44% with more sophisticated statistical modeling. The results from our simple additive model, derived from logistic regression analysis and designed to be able to be used at the bedside without the requirement of a calculator, were comparable with prediction with full logistic regression analysis.
Optimum patient selection is the main drive to improve the criteria for screening coronary angiography, a procedure that carries a 1.7% risk of major complication, including a 0.11% risk of death, a 0.05% risk of myocardial infarction, and a 0.07% risk of stroke.5 Although the absolute risk increment itself might seem low, the low specificity of current AHA/ACC screening guidelines results in the vast majority of patients meeting the criteria. Therefore substantial mortality and morbidity might be experienced on a population basis when large numbers are exposed to these "low" risks.
The effect on cost-benefit analysis with improved selection has been considered previously by Lin and colleagues.4 Before consideration of finances, we would be more concerned in determining the effect of missing patients with potentially treatable coronary disease, and there is (not surprisingly) little in the literature to help guide this decision-making process. It is conceivable that a potential range of outcomes would include neither harm nor postoperative angina (no flow-limiting disease), residual postoperative angina, or inability to wean from cardiopulmonary bypass after surgical intervention, resulting in death. What price could possibly be attached to the (as yet unquantifiable) risk of death as a result of missing patients with potentially treatable coronary disease?
|
Where Is the Balance?
|
|---|
As we attempt to improve the specificity of the current AHA/ACC guidelines (to reduce the morbidity of patients undergoing unnecessary coronary angiography), we will inevitably reduce the sensitivity of the selection process (miss patients with potentially treatable coronary disease).3 This is due to the statistical properties of the relationship between sensitivity and specificity; as one increases, the other decreases, and vice versa.6
The guidelines of the AHA/ACC do not miss any patients with coexistent coronary disease (100% sensitivity), but this is achieved by almost universal recommendations for screening angiography (low specificity) and unnecessarily exposing patients to the risks of coronary angiography; as such, achieving the balance in this situation is extremely difficult. If the aim is not to miss any patients with coexistent coronary disease, then the AHA/ACC guidelines are successful in this regard. However, similar results can also be achieved by using age over 35 years alone as the sole indication (sensitivity, 100% [95% CI, 100.0%-100.0%]; specificity, 0.38% [95% CI, 0.0%-1.0%]), and detailed guidelines might not be required.
A similar but extreme counterargument, for example, would be to increase the specificity by performing angiography only on patients with a history of myocardial infarction, where the sensitivity would be close to 100% (allowing for misdiagnosis of myocardial infarction). However, this would be undertaken with unacceptable loss to sensitivity (missing patients with coronary disease).
Therefore the balance does not rely on a test that would favor individual sensitivity or specificity but rather an approach that takes the balance of both into account. The crux of the argument lies in the individual surgeon’s opinion of the (unquantifiable) risks of untreated coronary disease weighted against the (known) risks of morality, morbidity, and cost associated with screening angiography in the majority (76% in this series) of patients who undergo mitral valve surgery without coexistent coronary disease.
|
Potential Limitations
|
|---|
In this study we have used the model of Lin and colleagues4 on an unselected cohort undergoing mitral valve surgery; its design and use was in patients with degenerative disease. However, when applied solely to the patients with degenerative disease in our cohort, the discriminating ability was similar (area under the ROC curve, 0.79 [95% CI, 0.71- 0.87]).
Of the 7 patients without indications in the simple model for coronary angiography, 4 had single-vessel and 3 had double-vessel disease. Of the 6 patients without indications in the logistic regression model, 4 had single-vessel, 1 had double-vessel, and 1 had triple-vessel disease. It is important to note, however, that neither statistical model takes into account disease severity but merely the presence of significant coronary disease.
|
Conclusion
|
|---|
The high sensitivity of the AHA/ACC guidelines for screening angiography to detect coexistent coronary disease in patients before mitral valve surgery is achieved at a cost of poor specificity and results in the majority of patients requiring screening angiography. Our validated simple model has the discriminating ability of a more complex logistic regression model with the advantage of being easily implemented at the bedside and improved specificity over the AHA/ACC recommendations. This (and usually in other statistical models) is achieved by sacrificing sensitivity and carries the risk of missing patients with coexistent coronary disease.
|
Criteria for the simple additive and logistic regression model
|
|---|
|
References
|
|---|
- Grigioni F, Enriquez-Sarano M, Zehr KJ, Bailey KR, Tajik AJ. Ischemic mitral regurgitation. long-term outcome and prognostic implications with quantitative Doppler assessment. Circulation 2001;103:1759-1764.[Abstract/Free Full Text]
- Bonow RO, Carabello B, de Leon Jr AC, Edmunds Jr LH, Fedderly BJ, Freed MD, et al. ACC/AHA guidelines for the management of patients with valvular heart disease. a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Management of Patients With Valvular Heart Disease). J Am Coll Cardiol 1998;32:1486-1588.[Free Full Text]
- Lim E, Ali ZA, Barlow CW, Jackson CH, Hosseinpour AR, Halstead JC, et al. A simple model to predict coronary disease in patients undergoing operation for mitral regurgitation. Ann Thorac Surg 2003;75:1820-1825.[Abstract/Free Full Text]
- Lin SS, Lauer MS, Asher CR, Cosgrove DM, Blackstone E, Thomas JD, et al. Prediction of coronary artery disease in patients undergoing operations for mitral valve degeneration. J Thorac Cardiovasc Surg 2001;121:894-901.[Abstract/Free Full Text]
- Scanlon PJ, Faxon DP, Audet A-M, Carabello B, Dehmer GJ, Eagle KA, et al. ACC/AHA guidelines for coronary angiography: executive summary and recommendations: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Coronary Angiography) Developed in collaboration with the Society for Cardiac Angiography and Interventions. Circulation 1999;99:2345-2357.[Free Full Text]
- Shapiro DE. The interpretation of diagnostic tests. Stat Methods Med Res 1999;8:113-134.[Abstract/Free Full Text]
Top
Abstract
Methods
