| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
J Thorac Cardiovasc Surg 2003;125:618-624
© 2003 The American Association for Thoracic Surgery
Surgery for Acquired Cardiovascular Disease |
From the Department of Community Research and Community Education, The Hope Heart Institute,a and the Center for Cost and Outcomes Researchb and the Pharmaceutical Outcomes Research and Policy Program,c University of Washington, Seattle, Wash.
Funded in part by the American Heart Association Grant-in-Aid Program and the National Institutes of Health.
Presented in part at the annual meeting of the International Society of Technology Assessment in Health Care, Philadelphia, Pa, June 2001.
Received for publication July 1, 2002. Revisions requested Aug 19, 2002; revisions received Aug 21, 2002. Accepted for publication Aug 26, 2002. Address for reprints: Darryl T. Gray, MD, ScD, University of Washington Box 358853, Center for Cost and Outcomes Research, 146 N Canal St, Suite 300, Seattle, WA 98103 (E-mail: tolvadtg{at}u.washington.edu).
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
1 revascularizes the distribution of the left anterior descending coronary artery through a thoracotomy, without using cardiopulmonary bypass. MIDCAB immobilizes the beating heart with devices such as the CTS-2 pressure stabilizer (Guidant Corporation, Indianapolis, Ind). MIDCAB is controversial 2 as an alternative to off-pump bypass through a sternotomy (OPCAB) with the Octopus stabilizer (Medtronic, Inc, Minneapolis, Minn) or to conventional procedures, including conventional on-pump coronary artery bypass grafting (C-CAB) and percutaneous transluminal coronary angioplasty (PTCA) with or without stenting. It typifies new technology for which financial as well as clinical implications must now be evaluated. 3 The medical literature increasingly includes economic analyses, 4 and guidelines for conducting and reporting such studies have long been available. 5-10 We originally envisioned assessing the cost-effectiveness of MIDCAB through a systematic review 11 of published comparative economic analyses. However, review of sample MIDCAB evaluations showed widespread variations in study design, procedures compared, and target populations, as well as significant methodologic weaknesses that precluded formal aggregation of study results. We therefore postulated that formal assessment of existing MIDCAB cost comparisons would find minimal adherence to published guidelines for performing such studies. |
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
Checklist design
We assessed the methodologic quality of articles meeting our inclusion criteria by applying a 1996 checklist published by Drummond and associates 9 to issues particularly relevant to MIDCAB evaluations. The checklist questions as applied herein are described as follows:
1. Was a well-defined question posed in answerable form?
Ideally the question should have specified the clinical strategies compared, time horizons, measures of clinical effectiveness, target patient populations, and cost-analysis perspective. A sample statement might run as follows: "We analyzed initial revascularization procedures performed for clinically significant stenosis restricted to the left anterior descending coronary artery distribution. From a societal medical care perspective, we assessed the cost-effectiveness of diagnostic catheterization followed by MIDCAB versus diagnostic catheterization followed by angioplasty and stenting in providing 2-year survival free from repeat revascularization procedures." Studies were considered adequate if the question was clearly defined even if it was limited in scope (eg, only considering inpatient outcomes).
2. Were comprehensive descriptions of the competing alternatives given?
Cost-effectiveness analyses are intrinsically comparative. Interventions can only be cost-effective relative to alternatives specified in the analysis. MIDCAB and OPCAB differ in chest incisions, stabilizers used, potentially treatable vessels, and results. Therefore studies that included both procedures had to distinguish clearly between them to receive full credit. Similarly, because device costs and potential outcomes of PTCA with stenting differ from those of PTCA alone, 12 studies comparing both procedures with MIDCAB had to consider these percutaneous approaches separately. We did not require specification of the rationale for excluding specific possible options (eg, considering OPCAB but not PTCA).
3. Was the effectiveness of the program or service established?
For at least partial credit, studies had to analyze competing procedures' success in achieving their clinical objective of revascularizing ischemic myocardium. This could have involved assessment of success through such measures as intraprocedural angiography, 13 intravascular ultrasonography, subsequent treadmill testing, and/or freedom from reintervention observed during specified postprocedural follow-up intervals. Although comparisons of patient-perceived outcomes (eg, quality-adjusted survival) were considered ideal, 10 they were not required. Assessments of perioperative mortality or other complications address procedure safety but were not considered adequate to measure procedure effectiveness.
Outcome and cost differences must be reasonably ascribable to treatment, rather than other factors. Although heretofore uncommon in procedure evaluations, 14 this assessment is best achieved by randomly allocating patients to the therapies being compared. 15 Nonrandomized comparisons may suffer from confounding by clinical indication, 16 in which clinical features influencing treatment choices also affect outcome independent of any actual treatment effect. For example, in concurrent intrainstitutional nonrandomized comparisons, eligible patients presumably underwent MIDCAB. Patients ineligible for MIDCAB (eg, because of multivessel disease) who underwent OPCAB or C-CAB differed from those undergoing MIDCAB in clinically important ways, thus confounding assessment of treatment effects. Conversely, results for patients who underwent stenting of arteries generally inaccessible by MIDCAB (eg, the circumflex coronary artery) would be of limited relevance in MIDCAB comparisons. The evolution of PTCA altered the indications for single-vessel C-CAB, 17 and similar selective referral could complicate MIDCAB studies.
To establish procedural effectiveness, nonrandomized studies of MIDCAB had to minimize such confounding. This could have been achieved by studying control patients treated in time frames or institutions without MIDCAB (while adjusting for resulting center effects, secular trends, etc) by restricting analyses to patients eligible for both MIDCAB and other procedures, for example, because of the presence of isolated left anterior descending coronary artery disease, or by other design approaches. Given the potential for confounding, reporting treatment group comparability on relevant parameters (eg, vessels treated, gender, age, and diabetes status) was considered essential. However, the mere absence of statistically significant differences in measured factors would not have proven treatment group similarity, 18 especially in studies with limited statistical power. This approach would also be questionable unless measured variables captured key patient characteristics (eg, target vessel distribution) that would have influenced original treatment choice as well as outcome. Alternatively, studies could have adjusted for residual confounding with logistic regression, propensity scoring, or other analytic approaches. 19
To answer questions of cost-effectiveness, studies also had to follow the intent-to-treat principle. 20 The primary analysis should have ascribed to the original treatment group all costs and outcomes of patients in whom any revascularization attempt was aborted or converted to another procedure. It was considered inappropriate to exclude such patients or to ascribe their results to the second procedure performed.
4. Were all the important and relevant costs and consequences for each alternative identified?
Studies were assessed on their capture of costs relevant to the (apparent) analytic perspective. For example, hospital perspective analyses should have estimated institutional costs, including stabilizer purchase prices and hospital overhead. If applicable, analyses should also have included cost implications of maintaining cardiopulmonary bypass standby capability during MIDCAB and of reusing disposable stabilizers. Whereas studies taking the hospital perspective would appropriately have excluded costs of care provided by nonsalaried physicians or physician assistants, they should have included costs of such personnel if they were hospital employees.
Because preoperative diagnostic catheterizations can be performed before or during the surgical admission, catheterization costs should have been consistently included or excluded for surgical patients. Because costs of diagnostic catheterization cannot be separated from those of angioplasty or stenting performed during the same procedure, comparisons of MIDCAB versus percutaneous revascularization should have included diagnostic catheterization costs for both strategies. Results and costs of assessing success of the revascularization with intraprocedural imaging, follow-up treadmill testing, and so on should have been consistently included or excluded for all study patients.
Comparisons of medical costs from a societal perspective would properly have included costs generated by cardiologists, cardiac surgeons, anesthesiologists, and radiologists, and costs of physician- or hospital-employed physician assistants if applicable. Operating room times that differed for MIDCAB, OPCAB, and C-CAB would have generated cost differences reflecting variable durations of anesthesiologist and surgeon involvement. Excluding costs similar for both procedures (eg, surgeon reimbursement) might not have affected societal perspective cost differences. However, if frequencies of repeated procedures or treated complications differed, then ignoring surgeon costs would have distorted treatment strategy costs. Although it is recommended, 10 analyses were not required to take broader societal perspectives that would have included patient out-of-pocket costs, nonmedical economic costs of lost productivity, and so on.
The study design should also have reasonably captured the range of clinical outcomes considered. Such follow-up would ideally have tracked outcomes for several months after the initial revascularization. Even studies that analyzed only short-term complications needed sufficient follow-up (eg, 30 days) to capture their frequency, outcomes, and costs.
5. Were costs and consequences measured accurately in appropriate units?
Currency is the best measure of costs and should have been expressed with stated foreign exchange rates if applicable. Resource use should also have been measured separately in appropriate units (eg, length of total hospital stay in days) to allow readers to consider how unit costs in their own institutions might affect study results. Similarly, outcomes such as clinical success should have been captured by proportions of patients with successful revascularization measured by intraprocedural imaging, freedom from repeat revascularization, and so on. Major complications (eg, recurrent bleeding necessitating reoperation) should have been distinguished from minor wound infections and other minor complications. Studies capturing subsequent revascularizations should have explicitly focused on repeat procedures involving the original anatomic site rather than new lesions. Studies also needed to identify one primary outcome measure (eg, a composite such as 1-year survival free of repeat revascularization) for use in calculating cost-effectiveness ratios (described under question 8).
6. Were the costs and consequences considered valued credibly, given the (apparent) cost perspective?
Studies reporting hospital costs needed to specify the basis for cost estimates (eg, true production costs or hospital-wide or department-specific cost-to-charge ratios). 21 Studies measuring societal perspective medical care costs should have combined hospital production costs or reimbursement with physician cost estimates reflecting Resource Based Relative Value Scales, 22 patient care time multiplied by compensation and practice expenses expressed per unit time, reimbursement (eg, Medicare fee schedules), or other measures. From no perspective would billed charges adequately represent costs, 23 because charges capture neither the true cost of providing care nor the reimbursement actually paid.
Costs for care provided in different calendar years should have been expressed in constant-year dollars (eg, 1998 dollars), based on the Consumer Price Index. 24 Measured outcomes should also have been captured comprehensively. For example, analyses that captured postdischarge outcomes and costs should have included admissions to other facilities (eg, through review of referring cardiologist office records).
7. Were costs and outcomes adjusted for differential timing?
Costs observed beyond 1 year of follow-up for individual patients should have been discounted to current value (eg, at 3%/year). Discounting is independent of inflation and reflects time preferences related to foregone potential interest or investment income associated with receiving a given dollar amount in a future year rather than today. For studies that followed up individual patients for less than 1 year, discounting was deemed not applicable. Discounting of future treatment effects 10 is still controversial and was not required.
8. Was an incremental analysis of costs and consequences of alternatives performed?
Ideally, studies would have measured both costs and clinical effectiveness. Those that found one treatment to be more expensive but more effective should have presented incremental cost-effectiveness ratios (ICERs), which identify net cost increases relative to net improvements in outcome. For example, studies finding that MIDCAB generated a greater 1-year survival free of repeated revascularization but at higher costs than PTCA with or without stenting would express these results as MIDCAB's mean incremental costs per added year of reintervention-free survival.
MIDCAB studies that considered inpatient survival and complications (rather than success rates) could only measure incremental costs per added uncomplicated live discharge. This ratio does not capture incremental costs associated with successful myocardial revascularization. Therefore, authors of such studies were given credit for just presenting absolute or relative differences in mean costs per patient.
One procedure could be superior to the alternative in either the cost or outcome dimension and also be at least as good, if not better, in the other dimension as well. Here the first procedure dominates the second, and ICERs cannot be calculated because no cost increase will improve outcome. As has been done elsewhere, 25 specifying the presence of dominance as justification for not calculating ICERs was considered ideal. However, we gave partial credit for merely identifying one procedure as being both less expensive and also better clinically than the alternative.
9. Was allowance made for uncertainty in the estimates of costs and consequences?
Authors should have addressed the uncertainty associated with any observed cost or outcome differences, and they received partial credit for performing any statistical analyses at all. Full credit required providing appropriate descriptions of central tendency (eg, means vs. medians) of cost with standard deviations or interquartile ranges and of confidence intervals around point estimates of outcomes frequencies (eg, percent mortality) or statistical analyses (with parametric versus nonparametric tests, log transformation, and so on) appropriate for sample sizes and for cost data with possibly skewed distributions. 26 Authors should also have distinguished differences that were clinically or economically important but statistically nonsignificant because of the small sample size from those that reflected clinically or economically unimportant differences. We did not require currently used approaches, such as generating confidence intervals around calculable ICERs. 27
As an adjunct or alternative to statistical analyses, we gave credit for performing sensitivity analyses 28 that estimated the impact on hypothetic study results of varying values for key parameters (eg, stabilizer costs) over reasonable ranges. New cardiovascular and other procedures have learning curves. 29 This complicates assessment of the clinical and economic outcomes of procedures that must be sufficiently mature to be reasonably evaluated but not yet so widely accepted as to preclude abandoning them if justifiable. Furthermore, costs may decrease with accumulating experience, even without improvements in outcome. 30 Although it was not required by the checklist, investigators could have addressed the learning curve by separately analyzing results from their early versus late MIDCAB experience and through sensitivity analyses modeling hypothetical impacts on study results of future changes in MIDCAB or other procedure results or costs reflecting evolving trends.
10. Did the presentation and discussion of results include issues of most concern to potential users?
Although no study can address all concerns of interest to all readers, discussion sections needed to be reasonably comprehensive to receive full credit. For example, discussions should have addressed all components of the study question as originally asked. Conclusions also had to be consistent with data-driven results; assertions of cost-effectiveness required support by data on clinical effectiveness as well as cost. Discussions should have properly summarized study strengths and weaknesses and considered the impact of study limitations on the strength of the conclusions. Studies that did not acknowledge their failure to address key issues were not considered adequate. Ideally, discussions should also have compared results with findings of reports published at least 2 years earlier, acknowledging relevant similarities and differences in study design, patient population, cost perspectives, and so on. Studies should also have considered the implications of their results for choices to be made between MIDCAB and alternatives by decision makers with similar perspectives while also at least considering the study results' broader implications for decision makers with other perspectives.
Checklist application and data analysis
Each reviewer independently answered the question of whether each article adequately addressed each checklist item as: yes, no, partially, or not applicable. After generating final ratings by resolving discrepancies through consensus, we assessed numbers of items adequately addressed by each study. Because we appraised all known studies meeting our inclusion criteria, there was no larger universe of studies to which tests of statistical inference performed on our own results would have applied.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
9 comparing both clinical effectiveness (success of revascularization) and costs of competing strategies. The other 9 studies were partial economic evaluations, generally with inpatient complications as the only clinical outcome measured. All 10 studies were nonrandomized and generally compared concurrent intrainstitutional series. Stated results generally favored PTCA, OPCAB, or MIDCAB over C-CAB (Table 1). However, most studies had multiple weaknesses (Table 2), reducing the validity of their conclusions. Although most studies received partial credit on many items, percentages of articles adequately addressing applicable individual checklist items ranged from 0% for item 4 (identifying relevant costs and consequences) to only 50% for items 2 (comprehensive description of alternatives) and 5 (appropriate units). No studies followed up patients beyond 1 year, so item 7 (discounting) was universally not applicable. Individual studies ranged from 0% to 67% in the percentage of applicable items adequately addressed, with a mean of just 24%. Only 2 studies adequately addressed more than 50% of applicable items.
|
|
Although most studies appeared to focus on hospital costs, none specifically stated their analytic perspective. Few studies specified their sources of cost data; others only measured billed hospital charges. Some studies apparently excluded certain major hospital costs (eg, MIDCAB stabilizer purchase prices) in a manner not consistent with any readily definable perspective. Costs of standby cardiopulmonary bypass or of stabilizer reuse were generally not addressed. No studies included costs of physician or physician assistant involvement. Although most studies included costs from multiple calendar years, only 1 expressed them in constant-year dollars. Even when it was possible, few studies calculated ICERs. Statistical analyses were limited, and no studies reported results of sensitivity analyses. Discussions were generally not comprehensive. Of 3 studies that reported the purported cost-effectiveness of MIDCAB, none included data sufficient to support their conclusions.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
5 We applied a well-known 10-item checklist for evaluating economic analyses 9 to published comparisons of costs of MIDCAB versus those of alternative procedures. We found minimal adherence to basic analytic principles, with only 2 of 10 studies adequately addressing more than 50% of applicable items. This finding mirrors results of general reviews of the radiology 31 and oncology 32 literature.
Our own study had several limitations. Completeness of our capture of relevant studies is unknown. However, the search strategy described here should have captured virtually all English-language peer-reviewed MIDCAB cost analyses. Appraisals such as ours are inherently subjective. Criteria for distinguishing among adequate, partially adequate, and inadequate handling of individual, often multidimensional checklist items were difficult to specify prospectively. However, these principles are laid out in Drummond and coworkers' checklist, 9 which was designed to promote objective assessment. Individual discrepancies were easily resolved by consensus.
Checklist items were variable in their importance in evaluating MIDCAB studies per se. We therefore could neither generate defensible weighted composite scores for individual studies nor specify score thresholds that would distinguish high-quality studies. We may also have underestimated true study quality if perceived space constraints discouraged authors from describing how they had actually addressed various checklist issues. However, economic analyses reporting guidelines 9,10 dictate that investigators should document the degree to which appropriate methodology supports their results. Finally, our unblinded evaluation revealed investigator and institutional identities and study results. However, neither of us had participated in any of the investigations cited, nor did either of us have previous clinical opinions regarding the procedures compared.
Despite these limitations, our analysis argues for greater involvement of experts in cost analyses in study design and execution, with increased awareness of relevant methodologic principles among investigators, manuscript reviewers, and editors. We hope that such efforts will enhance adherence to published guidelines in future MIDCAB studies. Although the impact of MIDCAB cost analyses may decline if OPCAB replaces MIDCAB as the dominant off-pump surgical procedure, 33 our recommendations for designing studies and reporting of their results apply equally to other surgical procedures. In the current environment, economic considerations may be expected to continue to influence medical resource allocation. Only with greater methodologic rigor can published economic analyses be expected to reasonably inform health care policy decisions in cardiovascular surgery and other fields.
|
Appendix: Articles reviewed |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
|
Acknowledgments |
|---|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Articles reviewedReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  C. Rao, O. Aziz, S. S. Panesar, C. Jones, S. Morris, A. Darzi, and T. Athanasiou Cost effectiveness analysis of minimally invasive internal thoracic artery bypass versus percutaneous revascularisation for isolated lesions of the left anterior descending artery BMJ, March 24, 2007; 334(7594): 621 - 621. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. S. Ranawat and A. S. Ranawat Minimally Invasive Total Joint Arthroplasty: Where Are We Going? J. Bone Joint Surg. Am., November 1, 2003; 85(11): 2070 - 2071. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
