| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
J Thorac Cardiovasc Surg 2000;119:661-672
© 2000 The American Association for Thoracic Surgery
LEAD ARTICLE |
From the Great Ormond Street Hospital for Children NHS Trust,a London, the Department of Statistical Science, University College,b London, Department of Psychology, and The University of Manchester,c United Kingdom.
This study was supported by a research grant (PG94166) from the British Heart Foundation.
Address for reprints: M. R. de Leval, MD, Great Ormond Street Hospital for Children NHS Trust, Great Ormond Street, London WC1N 3JH, United Kingdom.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
Human factors data
Three sets of human factors data were collected.
The Surgical Team Assessment Record (STAR)
Self-assessment questionnaires investigating a range of organizational, team, situational, and personal factors were filled in at the completion of the operation by the surgeon, the first and second assistant, the anesthetist, the perfusionist, and the scrub nurse (Appendix B).
Case reports
The ASOs were observed by a human factors researcher who attended the operation from the induction of anesthesia to admission to the intensive care unit. A detailed description of the operation was written down as the procedure was taking place. This included information on individual and team performance, communication within each team and between different teams, as well as situational and organizational data. In addition, errors or failures that occurred during the procedure were carefully recorded and categorized as minor or major negative events. Minor events were failures that disrupted the "surgical flow" of the procedure but which, in isolation, were not expected to have serious consequences for the safety of the patient. Major events were failures that were likely to have serious consequences for the safety of the patient. Major and minor events were judged by the human factors researcher to be either "compensated for" or "uncompensated." An event was deemed to have been "compensated for" when the patient recovered or when the consequences of the event were negated by appropriate action.
Because some ASOs took place simultaneously in several institutions, 50 operations were not attended by a human factors researcher. Therefore 193 reports were available for analysis. They were reviewed at the completion of the study by 2 human factors experts (J.C. and J.T.R.) and by a surgeon (M.dL.). As expected, the accuracy and the depth of these reports increased with experience. So that the consistency and reliability of the data could be ensured, 10 cases considered to be part of this learning curve were discarded from the analysis (5 cases each for 2 human factors researchers). Furthermore, 1 of the 3 human factors observers who attended 10 ASOs never gained sufficient knowledge of the procedure to make informed judgments about problems experienced during the cases. His reports were also rejected in the final analysis. In all, 173 reports were thought to be sufficiently reliable.
Volume of cases
The number of ASOs performed during the course of this study was looked at as a human factor. Surgeons were divided into low (<15 cases) or high (
15 cases) volumes. Six surgeons were accordingly classified as high-volume surgeons and 14 surgeons were classified as low-volume surgeons.
Outcome events
Clinical outcomes (OC) were divided into 4 categories:
All records had a code number unknown to those who analyzed the data to maintain anonymity and to protect confidentiality.
Statistical analysis
The primary analyses were based on logistic regression analyses of 2 binary outcomes. The first modeled the probability of death and the second modeled the probability of death and/or near miss. The results of the logistic regression analyses were summarized in terms of odds ratios. For a continuous variable such as a time measurement, the odds ratio compared the odds of death for 2 individuals whose time measurements differ by 1 unit, typically 1 minute in this study. If individuals’ times differed by k minutes, then their odds of death would be related by a factor of (odds ratio)k. For example, if for bypass time an odds ratio was estimated as 1.02 for a difference of 1 minute, the odds ratio comparing 2 patients whose bypass times differed by 60 minutes would be 3.28.
Patient characteristics and procedural variables were first submitted to a univariate analysis. Patient-specific variables were then combined to develop a multivariable baseline model for death and a similar one for death and/or near miss so as to study the human factors variables after adjustment for patient characteristics.
Because of their close correlation with clinical outcomes, anesthetic time (induction of anesthesia and insertion of lines), crossclamping time, and bypass time were used to define "surrogate outcomes." These surrogate outcomes were defined as the linear combination of the intraoperative timings best related to the probability of negative outcomes after adjustment for patient-specific variables (coronary artery pattern [CAP], VSD, and sex). An indicator of a missing length of anesthesia time was also included to avoid significant data loss.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
|
|
|
).
|
|
On the basis of the results of the univariate analysis and good background knowledge, a multivariable baseline logistic regression model, including CAP, sex, and an indicator for VSD cases, was then generated (Table IV). Separate models were estimated for death and death and/or near miss. The following points about these models are noteworthy: First, there was no evidence that the balloon atrial septostomy variable added anything to either model; second, there were no deaths among patients with CAP 2, so these cases were grouped with CAPs 4, 5, and 6 for death and with CAP 1 in another analysis. There was no difference in the results for either grouping of CAP 2. No formal significance level was used in the choice of these models. They are simply used to provide comparable adjustments for CAP, VSD, and sex in subsequent analyses of negative outcomes.
|
|
Surgical Team Assessment Records (STAR)
None of the variables from the STAR form was significant at the 1% level on clinical outcomes when added to the baseline multiple regression model. The 1% level was used to make an adjustment for multiple comparisons. Adding the STAR variables to the regression models for surrogate outcomes of death or death and/or near miss, using a semiparametric analysis, also showed no significance at the 1% level.
Surgical volume
The inclusion of the variable high-volume (15 cases) versus low-volume (<15 cases) surgeons did not add significantly to the baseline model for death (P = .376) or death and/or near miss (P = .265). The odds of death for cases of a high-volume surgeon were estimated to be 0.62 (95% CI: 0.22, 1.78) times that for cases of a low-volume surgeon. The odds ratio for death and/or near miss was estimated to be 0.70 (95% CI: 0.38, 1.31). The addition of volume to the model of surrogate outcomes demonstrates a strong relationship with the classification of surgeons as high or low volume (P = .001). It must be noted, however, that less than 5% of the variation in the timings variable can be explained by the volume of surgery variable.
Minor and major events
The lists of minor and major events are shown in Appendix C. Tables VI and VII show the frequency of major and minor negative events per case and also the frequency of major and minor uncompensated negative events per case. The number of cases that had negative outcomes (ie, death and/or near miss) is shown in parentheses. For example, 38 cases had 1 major event. In 33 cases the event was compensated for, whereas in 5 cases the major event was left uncompensated. Table VIII summarizes the results of adding information on major and minor events separately to the baseline models. For each type of event, 4 analyses are recorded. The first looks only at the total number of major or minor events (per case) for both the death and death and/or near miss outcomes. The second analysis adds the number of major uncompensated events per case to the first model. The third and fourth analyses are the analogous models for the number of minor negative events per case and the number of uncompensated minor negative events.
|
|
|
shows that the number of major and minor negative events per case has a strong relationship to both outcomes (ie, death and death and/or near miss). The number of uncompensated major events per case has a major predictive relationship with death (after adjustment for the number of major events per case) (P = .003). For deaths and/or near misses, the number of major events remains the dominant predictor (P < .001), although there is some evidence for an additional risk associated with the lack of compensation (P = .026). The number of uncompensated minor events per case adds little to the information provided by the number of minor events. Hence, whether or not human compensation occurs for a minor event is not the key factor; rather, it is the overall number of minor events per case that is important (P < .001). Table IX presents the results of adding both the number of major events per case, the number of uncompensated major events per case, and the number of minor events per case jointly to the baseline models. The same general patterns found in the separate analyses are seen. The important distinction is that this table provides, after adjusting for the effects of major and major uncompensated events per case, suggestive evidence of a link between the number of minor events per case and death (P = .03), as well as strong evidence of a link between the number of minor events per case and death and/or near miss (P = .001).
|
). For each uncompensated major event, the estimated odds of death are increased by a factor of 13 x 0.44 = 5.7 and the estimated odds of a death and/or near miss are increased by a factor of 40. The risk associated with the occurrence of a minor event is less (1.4 and 1.4) per event, but minor events have a multiplicative effect. |
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
1 This has been particularly apparent in high technology fields such as operating rooms 2-4 or intensive care units. 5 At the same time, there is an increasing awareness that models and algorithms for risk adjustment based on patient-related and procedural variables investigate only a portion of factors determining outcomes. 6 The present study is an attempt to carry out a comprehensive and prospective analysis of human factors together with those variables classically included in outcome analysis. It opens the field for novel research and analytical methods for the study of complex relationships between variables that are not traditionally analyzed together.
Critique of the methodology
Multicenter study and confidentiality issue
The participation of all the surgeons of one country who perform a particular operation in such a sensitive study is a unique feature of this research. It displays openness to failure, which is most commendable in a climate of increasing exposure of the medical profession to litigation and legal proceedings against failure, knowing that in the United Kingdom there is no law that protects confidentiality of human factors research. One of the pitfalls of this multicenter study, however, is that a number of ASOs were performed simultaneously in different institutions. This has accounted for a significant loss of case reports.
Capturing human factors
Although questionnaires are routinely used in human factors research, the implementation of this method in our study was not successful at capturing the wide range of human factors that could have influenced outcomes. Knowledge of the outcome of the case when the STAR forms were filled in may have caused hindsight bias that contributed to their unreliability. Should similar questionnaires be used in the future, they should be filled in before the operations to target features that can potentially influence outcomes.
The case study reports, which were written on-line without knowledge of the outcome by human factors experts who attended the operations, were the most valuable source of human factors data. The training of these researchers and the assessment of their abilities remain an issue to be addressed for future research. The principal human factors researcher had been in post for 6 months before the start of the data collection. This was not possible to achieve with the other 2 researchers, hence the additional waste of data.
Video camera recordings could be a useful adjunct to the observers’ reports. Videotaped records of medical treatments were shown to capture performance deficiencies that were not revealed by retrospective self reports. 7
We believe that human factors should be included in risk factors analysis and that more research is needed to detect them, analyze them, and assess their impact on medical outcomes. To that end, health care organizations could benefit greatly from the training and appointment of human factors researchers who have acquired a specific training in medical fields.
Defining negative events
In a previous study, 8 we have used the concept of near misses as failure equivalents with the assumption that death and near misses were the result of the same underlying mechanisms. Near misses, as defined in the present study, are in reality severe temporary or permanent complications. Some recent information would suggest that the risk factors for death and the risk factors for complications are in fact quite different. 9,10 According to these studies, the risk factors for complications seem to be related more to patient variables than to structural hospital characteristics. The highest quality, lowest risk hospitals do not have a lower incidence of complications but a higher prevalence of "rescue" from complications. In other words, it is the failure to rescue that leads to a negative outcome. This fits with our personal observation. After a period of retraining after a cluster of operative deaths, 8 one of us (M.dL.) performed 120 consecutive neonatal ASOs with 3 deaths (2.5%). Despite the return of the mortality to low levels, the incidence of near misses has remained unchanged. The apparent difference is that near misses are now less likely to lead to death; in other words, the operating team is better at handling complications.
Major and minor events
The impact of major and minor human failures on outcomes over and above the traditional patient risk factors is the most important finding of this study. Major failures without compensation are likely to lead to death, but appropriate human defense mechanisms can prevent catastrophic consequences.
Uncompensated major events have a multiplicative effect. If each of them is appropriately compensated, however, death can be avoided whatever the number. Minor events are different. They are more subtle and insidious, and many of them are not even noticed by the operators and the team members. No conscious attempt to compensate for them is therefore made in many instances, thus the lack of correlation between compensation and outcomes for minor events. The main feature of minor events is their multiplicative effect. In isolation they have little impact, but their multiplication has a strong relationship to negative outcomes, whereas a single uncompensated major event is likely to lead to death.
Furthermore, after adjustment for the effects of the total number of major uncompensated events in a case, there is still a link between minor events and negative outcomes.
Major and minor events represent various types of human errors. Human error has become an important topic in many areas of applied psychology, particularly in safety research. It is now widely accepted that error tolerance, error detection, and error recovery are as important as error prevention. Error detection is the first step in error handling. A great number of minor events were undetected errors; hence very little was done to enhance error tolerance or error recovery. Compensation is a form of error recovery whereby a strategy to remedy the situation is implemented before negative consequences accrue. 11 There are parallels between the compensations of negative events by the operating room team and error recovery mechanisms that have been reported in other high technology systems. 12 Our observations suggest, for example, that the surgeon’s diagnostic skill, knowledge of the various surgical strategies to correct a problem, and communication with the rest of the team are important prerequisites of compensation. Uncompensated events correspond to the "failure to intervene" errors that have been observed in pilots who do not act promptly to correct their errors. 13
The present study demonstrates that patients’ risk factors and human factors can combine their effects to lead to a negative outcome. It would be a logical step to assume that human compensation actually addresses both mechanisms of failure. In that context, the words "compensation," "recovery," or "rescue" would take on a wider meaning and include human strategies deployed to prevent negative outcomes in the presence of poor patient risk factors and/or human errors.
Future research
This study has concentrated on human failure at the sharp end of the system. Many minor and major events, however, have their origins beyond the scope of the operating room. They arise from administrative, strategic, and other top-level decisions made by governments, hospital trusts, educational bodies, departmental managers, designers, and the like, which create latent conditions. It is in that context that adverse surgical events would have some similarities with an organizational accident or a system accident in which breakdowns arise from the combined effects of active and latent failures. 14 Investing in the understanding of the latent failures that underlie the active failures (major and minor negative events) is probably the most cost-efficient way, long term, to improve safety in health care.
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
|
Appendix: Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
The concept of risk management in complex organizations is widely known in industry and the military. The organization and management of a nuclear power plant or an aircraft carrier requires the complex interaction of many individuals in the performance of precise tasks. One of the limitations of this report is clearly described in the discussion of the paper. The attempt to examine human factors was compromised by the use of a postoperative questionnaire. This type of inquiry permits hindsight bias as described by the authors.
The concept of constant, on-line grading of events is best demonstrated by the evaluation protocols used in the United States Navy during carrier landings. Every landing is graded by the landing signals officer. A videotape is made of every landing. If a pilot disputes his grade, the videotape can be used as a part of the review—instant replay. A pilot who is consistently scoring lower than a B grade or its equivalent may be grounded, and if his scores do not improve with retraining, he may be asked to leave the air corps. Thus a pilot may be dropped from the air wing of a carrier despite the fact that he has always landed on the carrier and never damaged his plane.
In contrast, cardiac surgeons do not have a systematic protocol for examining successful outcomes. If we are pursuing perfection, then we ought to review each case, both early after the operation and later in follow-up, to determine which aspects of the management could be improved.
The concept of compensation for a negative event is well defined by Dr de Leval. In both commercial and military aviation, simulators exist for testing the response of a pilot to varying conditions. To my knowledge, there is no formal attempt at this kind of training in cardiac surgery. Landing a plane on a carrier or performing the arterial switch procedure requires a complex interaction between technology, knowledge, and eye-hand coordination. The price of a mistake in either area can be high.
I have several questions for the authors.
There were 243 ASOs, but human factors research was undertaken in only 173. Do you think that the failure to include 30% of the outcomes in the human factors analysis affects the data?
Second, are you planning to repeat the study or expand the study with a preoperative questionnaire and a refined approach to human factors analysis?
Third, would the inclusion of long-term data add to the weight of the conclusion since technical problems occurring during the surgical procedure might lead to a high incidence of late pulmonary stenosis or left ventricular dysfunction in survivors?
Finally, experienced heart surgeons are inclined to state that any CAP can be switched. Your data suggest that less commonly occurring CAPs are associated with higher risk. Should all low-volume centers be required to clearly define the CAP before performing surgery? If a low-volume center cannot clearly define the CAP as type I or II, should the patient then be referred to a higher-volume center?
Dr de Leval. Thank you, Dr Lamberti. Major and minor events were extracted from only 173 reports because of the learning period required for the human factors researchers to become familiar with the operation. In addition, 1 of the researchers never reached a sufficient level of understanding of the procedure and his reports were discarded.
The answer to the second question is, yes. Human factors questionnaires should be filled in before or during the procedure, but not afterward, to prevent hindsight bias.
The third question relates to long-term data. We are in the process of analyzing the 6-month results.
With regard to the last question, I believe that the ASO should be performed only in institutions capable of dealing with all variants of CAPs.
|
|
|
|
Footnotes |
|---|
Read at the Seventy-ninth Annual Meeting of The American Association for Thoracic Surgery, New Orleans, La, April 18-21, 1999.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: DiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  S. C. Stoica, D. Kalavrouziotis, B.-J. Martin, K. J. Buth, G. M. Hirsch, J. A. Sullivan, and R. J.F. Baskett Long-Term Results of Heart Operations Performed by Surgeons-in-Training Circulation, September 30, 2008; 118(14_suppl_1): S1 - S6. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Skinner, T. Hornung, and E. Rumball Transposition of the great arteries: from fetus to adult Heart, September 1, 2008; 94(9): 1227 - 1235. [Full Text] [PDF]
|
|
|
|
 |
|
 A. Bognar, P. Barach, J. K. Johnson, R. C. Duncan, D. Birnbach, D. Woods, J. L. Holl, and E. A. Bacha Errors and the Burden of Errors: Attitudes, Perceptions, and the Culture of Safety in Pediatric Cardiac Surgical Teams Ann. Thorac. Surg., April 1, 2008; 85(4): 1374 - 1381. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. E. Roselli, G. B. Pettersson, E. H. Blackstone, M. E. Brizzio, P. L. Houghtaling, R. Hauck, J. M. Burke, and B. W. Lytle Adverse events during reoperative cardiac surgery: Frequency, characterization, and rescue J. Thorac. Cardiovasc. Surg., February 1, 2008; 135(2): 316 - 323. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Lingard, G. Regehr, B. Orser, R. Reznick, G. R. Baker, D. Doran, S. Espin, J. Bohnen, and S. Whyte Evaluation of a Preoperative Checklist and Team Briefing Among Surgeons, Nurses, and Anesthesiologists to Reduce Failures in Communication Arch Surg, January 1, 2008; 143(1): 12 - 17. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Holzhey, S. Jacobs, T. Walther, M. Mochalski, F. W. Mohr, and V. Falk Cumulative sum failure analysis for eight surgeons performing minimally invasive direct coronary artery bypass J. Thorac. Cardiovasc. Surg., September 1, 2007; 134(3): 663 - 669. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. W. ElBardissi, D. A. Wiegmann, J. A. Dearani, R. C. Daly, and T. M. Sundt III Application of the Human Factors Analysis and Classification System Methodology to the Cardiovascular Surgery Operating Room Ann. Thorac. Surg., April 1, 2007; 83(4): 1412 - 1419. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. L. Tucker An Empirical Study of System Improvement by Frontline Employees in Hospital Units MSOM, January 1, 2007; 9(4): 492 - 505. [Abstract] [PDF]
|
|
|
|
 |
|
 D. R. Wong, T. J. Vander Salm, I. S. Ali, A. K. Agnihotri, R. M.J. Bohmer, and D. F. Torchiana Prospective assessment of intraoperative precursor events during cardiac surgery. Eur. J. Cardiothorac. Surg., April 1, 2006; 29(4): 447 - 455. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Svenmarker and M. Appelblad Reporting of perfusion-related incidents: pitfalls and limitations Perfusion, September 1, 2005; 20(5): 243 - 248. [PDF]
|
|
|
|
 |
|
 R. Amalberti, Y. Auroy, D. Berwick, and P. Barach Five System Barriers to Achieving Ultrasafe Health Care Ann Intern Med, May 3, 2005; 142(9): 756 - 764. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. M. Sundt, J. P. Brown, P. N. Uhlig, and STS Workforce on Patient Advocacy, Communications, Focus on Patient Safety: Good News for the Practicing Surgeon Ann. Thorac. Surg., January 1, 2005; 79(1): 11 - 15. [Full Text] [PDF]
|
|
|
|
 |
|
 M Ricci, A P Goldman, M R de Leval, G A Cohen, F Devaney, and J Carthey Pitfalls of adverse event reporting in paediatric cardiac intensive care Arch. Dis. Child., September 1, 2004; 89(9): 856 - 859. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M B Weinger, D C Gonzales, J Slagle, and M Syeed Video capture of clinical care to enhance patient safety Qual. Saf. Health Care, April 1, 2004; 13(2): 136 - 144. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Aberg and J. Hentschel Improved total quality by monitoring of a cardiothoracic unit. Medical, administrative and economic data followed for 9 years Interactive CardioVascular and Thoracic Surgery, March 1, 2004; 3(1): 33 - 40. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J Carthey The role of structured observational research in health care Qual. Saf. Health Care, December 1, 2003; 12(90002): ii13 - 16. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. W. Allen, K. Gauvreau, B. T. Bloom, and K. J. Jenkins Evidence-Based Referral Results in Significantly Reduced Mortality After Congenital Heart Surgery Pediatrics, July 1, 2003; 112(1): 24 - 28. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Miller, A. Elixhauser, and C. Zhan Patient Safety Events During Pediatric Hospitalizations Pediatrics, June 1, 2003; 111(6): 1358 - 1366. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. de Leval Beyond Flatland J. Thorac. Cardiovasc. Surg., January 1, 2003; 125(1): 12 - 19. [Full Text] [PDF]
|
|
|
|
 |
|
 V. A. Ferraris and S. P. Ferraris Risk Stratification and Comorbidity Card. Surg. Adult, January 1, 2003; 2(2003): 187 - 224. [Full Text]
|
|
|
|
|
|
P. Massoudy, A. Baltalarli, M. R. de Leval, A. Cook, U. Neudorf, G. Derrick, K. P. McCarthy, and R. H. Anderson Anatomic Variability in Coronary Arterial Distribution With Regard to the Arterial Switch Procedure Circulation, October 8, 2002; 106(15): 1980 - 1984. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. Falk Manual control and tracking--a human factor analysis relevant for beating heart surgery Ann. Thorac. Surg., August 1, 2002; 74(2): 624 - 628. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J Reason Combating omission errors through task analysis and good reminders Qual. Saf. Health Care, March 1, 2002; 11(1): 40 - 44. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
