|
|
||||||||
J Thorac Cardiovasc Surg 2001;121:409-417
© 2001 The American Association for Thoracic Surgery
Presidential Address |
From Sutter Memorial Hospital, Sacramento, Calif.
Received for publication July 12, 2000. Accepted for publication July 12, 2000. Address for reprints: Bradley J. Harlan, MD, 5301 F St, Sacramento, CA 95819 (E-mail: Bradharlan{at}aol.com).
| Introduction |
|---|
|
|
|---|
The Western Thoracic Surgical Association has had a long and proud record of supporting outcomes research and reporting. This has been part of several presidential addresses in the past. Members of this Association have served and are serving in important capacities in the STS database project.
When I began planning this address 2 years ago, the plan was for the California Coronary Artery Bypass Graft Mortality Reporting Program (CCMRP) to release results publicly in 1999. For many reasons the release has been repeatedly postponed and has not yet occurred. The impact of the release and the reaction to the release was what I thought would bring maximum timeliness to my address, but that was not to be.
The imminent release of statewide data for California, the experience with projects in New York, Pennsylvania, New Jersey, Northern New England, and Minnesota, and the proliferation of other statewide projects have stimulated me to review the history of these projects, assess where they are now, and look to the future, with both predictions and recommendations.
I have tried to gain access to the information in the statewide reports by using the Internet. I think the Internet is the medium most used by our patients to find out what we do and how well we do it, and this use will continue to grow in the future. Therefore, it is important to know the present state of Internet information.
| New York |
|---|
|
|
|---|
Today, everyone wanting information about the results of coronary artery bypass grafting (CABG) in New York, at least as of 1996, can go to the New York Department of Health Web site (www.health.state.ny.us) and click on "Info for Consumers." This will take them to a page where they can click on "heart disease" and then access 3 reports on CABG, covering 5 years up to 1996, including 90,130 patients.
The collection of statewide data in New York regarding hospital outcomes of cardiac surgery began in the 1970s under the auspices of the Cardiac Advisory Committee, which reported to the state department of health (Edward Hannan: personal communication). During the 1980s large variations in hospital mortality rates were observed. At the request of the then-commissioner, Dr David Axelrod, a patient-level, risk-adjusted data system was developed and pilot tested in 1988 for use in 1989. There was broad participation of cardiac surgeons in setting up this program. Of the 21 members of the Cardiac Advisory Committee, 20 were physicians and 9 were cardiac surgeons. Participation in the CSRS was mandatory to keep a certificate of need.
A form was completed at discharge by hospital cardiac surgery departments for every cardiac surgical patient in New York, including demographic data, dates of admission, surgery, and discharge, risk factors, complications, discharge status, surgeon, and hospital.
3 The CSRS used mathematical prediction models for risk adjustment, collecting data on clinical risk factors with known prognostic importance, such as low ejection fraction, left main coronary artery stenosis, unstable angina, congestive heart failure, and chronic obstructive pulmonary disease. A multivariate logistic-regression risk-adjustment model was used.
4
The initial plan was to publish the results in the literature, keeping the identity of the hospitals confidential. An article published in the Journal of the American Medical Association (JAMA) appeared in December 1990.
3 This article stated that 3 of 28 hospitals had better than expected mortality rates and 4 had worse than expected. Subsequent site visits to the "worse than expected" hospitals showed a high percentage of quality of care problems.
Dr Axelrod unilaterally decided to release the data to the media, with the hospitals identified, and on December 4, 1990, an article appeared in New York Newsday
5 entitled "Ranking Open-Heart Surgery: State Study Lists Best Hospitals." The story was widely covered by the print and broadcast media throughout the state. The era of public reporting of outcomes of cardiac surgery was born.
New York Newsday discovered that the state department of health also had data on individual surgeon performance. They asked for these data and were refused. Newsday then sued the department of health under the Freedom of Information Act. This was decided in favor of Newsday by the state supreme court.
On December 18, 1991, New York Newsday published the article,
6 "Heart Surgeons Rated: State Reveals Patient-Mortality Records." This article stated, "For the first time, the state Health Department has released, by name, adjusted mortality rates for each of 140 New York heart surgeons, a list that can be used to compare doctors' performance." The era of surgeon-specific public reporting of cardiac surgery outcomes was born.
The Newsday article listed all cardiac surgeons who did CABGs in 1989 and 1990 with their caseloads and their risk-adjusted mortality rates. The risk-adjusted mortality rates ranged from 0% to 21.7%. The article emphasized the higher mortality of the group of surgeons who did fewer than 50 cases a year, with Dr Hannan, the biostatistician of the report, stating that the average adjusted mortality rate of doctors averaging under 50 bypasses a year was 6.1%, nearly twice the 3.1% mortality rate of surgeons averaging more than 50 operations a year.
Many surgeons criticized the premise and methodology of the report, criticisms that have sparked a lively debate in the media and in the medical literature that has continued to the present day. The Newsday article stated that interviews with 20 cardiac surgeons from a dozen medical centers indicated the vast majority of surgeons were at odds with the report. The Newsday article quoted Dr Frank Spencer as saying, "This system may make a hell of a contribution if it proves valid one day. But right now, any statistician would blow it out of the water." The article stated that several surgeons warned that some surgeons were turning down difficult cases to protect their statistics. Thus, from the start, a number of areas of controversy were defined: the quality of data collection and analysis, the relationship between surgeon volume and mortality, and the effect on patient access to cardiac surgical care.
Consumer advocates hailed the data's publication. The New York Public Interest Research Group said, "It is a great victory for health consumers. If the state collects data and taxpayers pay for it, consumers should be able to look at it. Patients deserve to have as much information as they can when they make life-and-death decisions." The Center for Medical Consumers in Manhattan, said, "It is another piece of information that can be helpful in making a choice as to where to seek care."
The Cardiac Advisory Committee voted overwhelmingly to recommend to hospitals that they submit data to the Department of Health in a fashion that would make it impossible to identify specific physicians (Edward Hannan: personal communication). Following this recommendation, much discussion was held between the department of health and the Cardiac Advisory Committee and other physician leaders. In 1992, it was agreed that data on operative morality would be compiled for the most recent 3 years according to surgeon and would be attributed by name only to surgeons who performed at least 200 operations in a single hospital during that 3-year period. Surgeons below this level would be lumped together in the category "other."
The Cardiac Advisory Committee took an active role in initiating change in hospitals with higher than expected mortality. Site visits were conducted, processes reviewed, and specific recommendations for change were made. The surgical privileges for some low-volume surgeons were restricted by the hospitals and other low-volume surgeons with poor results elected to stop performing cardiac surgery. The public release of the results stimulated surgeons to review their care of certain subsets of patients and make improvements. A group in Albany
7 found a high mortality in patients with active ischemia and initiated changes such as preoperative insertion of intra-aortic balloon pumps and better pharmacologic management. This resulted in a dramatic drop in the mortality of these patients.
It is likely that some of the criticism of the data and data analysis was reasonable. But it is also likely that both the surgeons and the data collectors and analyzers shared in this responsibility. Until the dramatic events surrounding the public release of these data, this project had a low level of importance for many cardiac surgeons. This rapidly changed, sometimes to the point where under-coding was replaced by over-coding.
2 The prevalence of some risk factors, such as chronic obstructive pulmonary disease and unstable angina, increased markedly. Eventually, because of appropriate changes in definition, better data reporting, better auditing, and refinement of risk-adjustment algorithms, the fit between expected and actual mortality rates, which did not exist in 1989, became present in 1992.
8
By the middle of the 1990s, the argument was being made that the CSRS had resulted in improvement in results of CABG in New York. In a 1994 article, Hannan and colleagues
9 reported that from 1989 to 1992 there was a 21% decrease in actual mortality and a 41% decrease in risk-adjusted mortality. Hannan and colleagues
8 published an article in 1995 arguing that the elimination of low-volume surgeons was an important contribution to this improvement.
These studies intensified the debate over the relationship between volume and quality. A study of the Veterans Administration system, published in 1996,
10 concluded that a volume threshold related to quality could not be found. In 1996, the Ad Hoc Committee on Cardiac Surgery Credentialing of the STS concluded,
11 "Until conclusive data become available that link volume to outcome, volume should not be used as a criterion for credentialing of cardiac surgeons by hospitals, managed care groups, or others. Instead, each surgeon should be evaluated on his or her individual results."
The contention that the New York report was an important stimulus to quality improvement was debated by the publication of an article from Massachusetts in 1997,
12 showing a 42% decline in risk-adjusted mortality in Massachusetts from 1990 to 1994, where there was no statewide public reporting of CABG results. The highly publicized events occurring in New York, and the possibility that public reporting would come to Massachusetts, may have stimulated changes in Massachusetts that otherwise would not have occurred.
By 1995, at least the New York Times was convinced that the CSRS was improving results in New York. In June of 1995, the Times published an article
13 with the headline, "Heart Bypasses Are Safer, Study Shows." In September of 1995, the Times published an editorial
14 entitled, "Rating the Surgeons," which stated, "New York State's annual report card for cardiac surgeons is clearly having beneficial effects."
In 1992 the issue of access for high-risk patients was given major coverage when an article appeared in the New York Times,
15 entitled, "Faint Hearts." As fate would have it, a woman was turned down for surgery because she had a fresh, large myocardial infarction. Her daughter was a reporter for the New York Times. After great difficulty, the daughter eventually found a surgeon who would operate on her mother. The reporter concluded her article with the statement, "My mother beat the odds."
The controversy over access heightened in 1996 with the publication of an article from the Cleveland Clinic
16 analyzing their referrals for CABG from New York State between 1989 and 1993. The Cleveland Clinic found that their patients from New York had a higher percentage of reoperations, more severe symptoms, and a higher mortality than patients from Ohio, other states, and other countries. They concluded that public dissemination of outcome data may have been associated with increased referral of high-risk patients from New York to the Cleveland Clinic.
The Cleveland Clinic study has problems. These investigators chose the period of analysis to begin in January of 1989, although the first public release of data for New York hospitals did not occur until December of 1990, and the public release of surgeon-specific data, which should have been more likely to increase surgeons' unwillingness to operate on high-risk patients, did not occur until December of 1991. Thus, the increase in referrals to the Cleveland Clinic began well in advance of the public release of data in New York. Also, the increase in referrals was relatively modest, rising from an average of 61 patients a year to 96 patients a year. These 35 extra patients, in the year 1992, represented only 0.2% of the 16,000 patients undergoing CABG in New York that year.
Another article on the subject of access appeared in 1998, from the Duke Clinical Research Institute.
17 These investigators examined national Medicare data and found that the proportion of New Yorkers receiving CABG outside of New York actually declined from 1987 to 1992, that the risk profile increased, and that risk-adjusted mortality declined more in New York than in the nation as a whole.
Thus, New York set the stage for publicly released CABG results, defined the areas of controversy, profoundly affected the careers of some surgeons, and, as we shall see, affected the way similar projects evolved in other states
| Pennsylvania |
|---|
|
|
|---|
The Pennsylvania project began in the late 1980s (David B. Campbell: personal communication). Participation was mandatory. The goals (Table I) reflected the desire to provide large purchasers of health care, as well as individual patients, with information they could use in making health care decisions based on quality and value.
|
The Pennsylvania project is now in limbo. Data are not being harvested and no target date has been set for the release of the next report. It is entirely possible that the Pennsylvania project will not continue, as the Pennsylvania Health Care Cost Containment Council turns its interest to measuring quality related to large health care purchasers (David B. Campbell: personal communication).
| New Jersey |
|---|
|
|
|---|
In 1998 the New Jersey Department of Health adopted regulations requiring hospitals to assure that each of its cardiac surgeons perform a minimum of 100 procedures a year at that hospital by the year 2001. New cardiac surgeons will have 3 years to reach this level (Emmanuel Noggoh: personal communication). To my knowledge, these will be the first minimum volume criteria applied to any type of surgeon anywhere in the nation.
The New Jersey project began in the early 1990s. The model used by New Jersey has evolved over the years, stimulated by a data quality improvement group established in 1997, including additional cardiac surgeons (Alfred Casale: personal communication). During this time the definitions evolved to those of the STS and the report released this year has the same definitions as the STS, but on slightly different forms. Participation is mandatory.
The report released in 1999, covering 1996 to 1997, had both hospital and surgeon-specific data. The report released this year, covering 1998, including 8377 patients, contains only hospital data. The data are presented as risk-adjusted mortality with expected ranges, and it is noted whether a hospital is better or worse than the expected range. For 1998, for the 14 hospitals, 2 were better than expected and 2 were worse than expected.
| California |
|---|
|
|
|---|
CCMRP is a cooperative venture between a state agency, the Office of Statewide Health Planning and Development (OSHPD) (Web site: www.oshpd.state.ca.us), which is involved in health facility construction and financing, health manpower, quality of health care, and access to health care, and a private organization, the Pacific Business Group on Health (PBGH) (Web site: www.pbgh.org). PBGH is a business health coalition of 32 large purchasers representing more than 3 million employees, dependents, and retirees with $3.5 billion in annual health care expenditures. Members of PBGH include Bank of America, Transamerica Corp, Atlantic Richfield, Lockheed, Pacific Telesis Group, Charles Schwab, GTE, as well as the University of California and Stanford University. PBGH has been active for the past decade in the assessment of health care quality, studying the rates of cesarean sections in the early 1990s. Their Web site for reports, www.healthscope.org, lists 39 different report cards for hospitals, health maintenance organizations, and medical groups.
The goals of CCMRP were similar to the goals of the other statewide programs (Table II). To participate, hospitals agreed to submit to periodic audits and allow the results to be publicly released. A technical advisory panel was chosen, consisting of 11 physicians (Table III), 6 of whom are cardiac surgeons. These individuals have worked long and hard to assure the quality of this project, and every cardiac surgeon in California owes them a large debt of gratitude.
|
|
PBGH and OSHPD could not use the STS software and methods because the data collection software, risk-adjustment algorithms, and surgical results were proprietary and confidential. Surgeon-specific data were not collected because it was believed that analysis of these data is difficult because of low numbers. Also, PBGH and OSHPD believed that focusing on the hospital results emphasized the reality that outcome of CABG surgery reflects a team contribution, not simply the performance of a single individual. They also thought not having surgeon-specific data would remove some of the controversy that resulted from the other statewide projects and would encourage participation in this voluntary program.
We still await the public release of data giving results. The only data available now relate to volume of CABG cases by hospital (www.healthscope.org). These volume figures are available for 1997. Hospitals that have declined to participate in the CCMRP are indicated. Forty of the 120 hospitals in California performing CABGs have declined to participate. Three of the 10 highest-volume hospitals declined to participate.
The third unique feature of the California project is that there is a parallel data project, initiated under the leadership of Joe Carey in 1995, when the California Society of Thoracic Surgeons (CASTS) was formed. The CASTS uses the STS database. This project involves approximately 45 hospitals compared with the 80 in the CCMRP. The Duke Clinical Research Institute has analyzed the CASTS data by hospital, rather than the usual STS method of individual or group. The CASTS results are yet to be released. It will be interesting to compare the CCMRP data with the CASTS data when they are released.
| Confidential statewide and regional projects |
|---|
|
|
|---|
The Northern New England Cardiovascular Disease Study Group began in 1987 and first reported their blinded results in JAMA in 1991.
19 The study involved all cardiac surgeons in Maine, New Hampshire, and Vermont. The crude in-hospital mortality for isolated CABGs varied from 3.1% to 6.3% among centers and from 1.9% to 8.2% among surgeonsstatistically significant variability. The group published a follow-up article in 1996,
20 after having instituted a program of quality improvement involving training in continuous quality improvement techniques and site visits. This study group found a statistically significant 24% reduction in hospital mortality.
In 1993 the Minnesota Society of Thoracic Surgeons and the Minnesota Cardiac Surgery Database were organized under the leadership of Kit Arom and subsequently grew to an organization involving 46 cardiac surgeons and 14 institutions.
21 The data are analyzed with the use of the STS format and definitions. A quality improvement program has been established involving exchange of practice guidelines, promotion of standardization, and exchange site visits.
The Minnesota program will be incorporated into a 4-state study of quality assessment and quality improvement programs, also including Colorado, Iowa, and Alabama (T. Bruce Ferguson: personal communication). This study is the result of a 3-year, $1.5 million dollar grant from the Agency for Healthcare Research and Quality, with T. Bruce Ferguson and Frederick Grover as principal investigators. They propose to demonstrate that a continuous quality improvement program can be successfully implemented on a national level using voluntary data collection.
Other statewide projects are evolving in Washington (Richard Goss: personal communication), Michigan (Richard Prager: personal communication), and Massachusetts (David Shahian: personal communication). These projects plan to evolve from quality assessment to quality improvement.
| Effects |
|---|
|
|
|---|
Hospitals have clearly responded to these studies. A study of Pennsylvania hospitals showed 77% set up administrative mechanisms to monitor quality and 38% devoted a larger share of their financial resources to improve quality.
22 However, these studies do not seem to have had any effect on market share. A study of New York, published in the New England Journal of Medicine in 1996,
4 showed there was no change in the percentage of patients in the highest and lowest mortality hospitals between 1989 and 1993.
The impact of these reports on cardiologists is minimal. Hannan and colleagues
23 surveyed New York cardiologists and got a very poor 36% response rate. Of these, only 22% routinely discussed the report with their patients and only 38% said the information affected their referral to surgeons "very much" or "somewhat." Schneider and Epstein
24 surveyed Pennsylvania cardiologists, with a much better 64% response rate, and similarly found that only 10% believed mortality rates, as outlined in the guide, were "very important" in assessing the performance of a cardiac surgeon. Eighty-three percent reported the guide had little or no effect on their referral patterns.
I have already discussed the effect the New York program had on some low-volume surgeons. In 1999 Burack and colleagues
25 published a survey of New York cardiac surgeons, with a 69% response rate, and found most did not experience a change in practice. Despite all the efforts to improve reporting, approximately half picked the wrong definition of chronic obstructive pulmonary disease or the wrong statistical method. Almost two thirds refused to operate on at least one high-risk patient over the previous year.
The effects of statewide reporting on large purchasers are unclear. Pennsylvania found no difference in quality among health plans. A recent article in JAMA, by Erickson and colleagues,
26 concluded that patients in New York State with private managed care and Medicare managed care insurance were significantly less likely to use lower-mortality hospitals for CABG surgery than patients with private fee-for-service insurance. This report stimulated considerable comment in the media, with an article in the New York Times
27 entitled, "Those in Managed Care Have Cardiac Surgery at Higher-Risk Hospitals, Study Says," and an article in the Wall Street Journal
28 entitled, "Heart Patients in Managed-Care Plans in New York May Go to Lesser Hospitals."
What about the ultimate effects, the most important effects, those on patients? I think that statewide reporting, at least in New York, has clearly improved the quality of patient care and the outcomes of CABG. But what about the goal of educating patients and thereby making them wiser and more effective consumers? I think the evidence for that, at least so far, does not exist. Schneider and Epstein
29 surveyed a large group of patients in Pennsylvania who had undergone CABG. Only 12% were aware of the Consumer Guide before surgery and only 1.5% reported knowing the rating of their surgeon. This may have been caused by poor dissemination of the report, since I have mentioned how hard it is to find the report online. It remains to be seen whether more effective use of the Internet will improve our patients' knowledge about our results.
| Forces of the future |
|---|
|
|
|---|
The forces of consumerism will not abate. The public wants more information about health care and public officials will continue to respond to this desire. Large purchasers will continue to demand evidence that they are getting value for their dollar. And, in an ironic twist for purchasers, the public will demand to know that their insurance companies and health maintenance organizations are interested in quality and not just price.
The Internet will continue to transform health care as it transforms so much else in our lives. It will make access to information regarding outcomes far easier than it has ever been in the past, and a resourceful public will continue to use the Internet to inform itself. We already see this increasingly in our practices, as patients are far better informed than in the past.
These forces will increase the demand for health outcomes information in the future.
| Goals |
|---|
|
|
|---|
Universal participation is important to achieve the other goals. It is admirable that the voluntary programs in Northern New England and Minnesota have 100% participation. The 67% institution participation in California must be improved.
We should work toward universal use of the STS database. It is the largest and best database, now containing more than 800,000 cases of isolated CABG.
30 The Duke Clinical Research Institute, which manages the database, is probably the premier institute of its kind in the country. I believe that if the STS database had been available for use in California by PBGH and OSHPD in 1995, it would have been used. I believe it will be possible to persuade the powers that be in California to evolve toward use of the STS database in the future. The New Jersey program is evolving toward the STS. In New York, it will take considerable work, probably involving an increase in auditing of the STS database, but I think it is possible that the New York program could eventually adopt the STS database. Universal use of the STS database would provide a single national standard, encourage participation by surgeons and institutions because it is the most trusted, and create important cost saving by avoiding duplication.
Data collection and analysis should be simplified and made less expensive. Increased use of technology such as wireless devices and the Internet should make collection and transmission of data easier and more cost-effective. Lowering the cost of these projects would be an important factor in encouraging universal participation.
We should move toward universal public release of data. Making our information public enhances our credibility and, thereby, the trust of the public, health care purchasers, state health officials, and politicians. In an age when the public has access to statistics from airlines about delayed flights and lost luggage, they should have access to information about the quality of heart surgery.
The very availability of information about results stimulates quality improvement, but we should continue to expand our efforts in this area and learn from the experience in Northern New England, Minnesota, and the quality improvement projects of the STS.
We can be proud of what our specialty has accomplished in the past decade in this relatively new and exploding area of outcomes analysis and reporting. By continuing our hard work and broad involvement, we can help it evolve in a way that will benefit those we care the most aboutour patients.
| Acknowledgments |
|---|
| Footnotes |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
S. H. Johnson, P. F. Theurer, G. F. Bell, L. Maresca, T. Leyden, R. L. Prager, and Michigan Society of Thoracic and Cardiovascular Su A Statewide Quality Collaborative for Process Improvement: Internal Mammary Artery Utilization Ann. Thorac. Surg., October 1, 2010; 90(4): 1158 - 1164. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. J. Epstein Do Cardiac Surgery Report Cards Reduce Mortality? Assessing the Evidence Med Care Res Rev, August 1, 2006; 63(4): 403 - 426. [Abstract] [PDF] |
||||
![]() |
G. L. Grunkemeier and A. P. Furnary Mandatory Database Participation: Risky Business? Ann. Thorac. Surg., September 1, 2005; 80(3): 799 - 801. [Full Text] [PDF] |
||||
![]() |
F. L. Grover, J. C. Cleveland Jr, and L. W. Shroyer Quality Improvement in Cardiac Care Arch Surg, January 1, 2002; 137(1): 28 - 36. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. L. Holman, R. M. Allman, M. Sansom, C. I. Kiefe, E. D. Peterson, K. J. Anstrom, S. S. Sankey, S. G. Hubbard, R. G. Sherrill, and for the Alabama CABG Study Group Alabama Coronary Artery Bypass Grafting Project: Results of a Statewide Quality Improvement Initiative JAMA, June 20, 2001; 285(23): 3003 - 3010. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |