| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
From the Department of Anesthesia Intensive Care, St. Olavs Hospital University Hospital of Trondheim, Trondheim, Norway.
Address correspondence to: Dr. Sigurd Fasting, Department of Anesthesia and Intensive Care, St. Olavs Hospital, University Hospital of Trondheim, N-7006 Trondheim, Norway. Phone: +47-73868108; Fax: +47-73868117; E-mail: sigurd.fasting{at}medisin.ntnu.no
| Abstract |
|---|
|
|
|---|
Methods: We prospectively recorded anesthesia-related data from all anesthetics for five years. The data included intraoperative adverse events, which were graded into four levels, according to severity. We selected four adverse events, representing important quality and safety aspects, for statistical process control analysis. These were: inadequate regional anesthesia, difficult emergence from general anesthesia, intubation difficulties and drug errors. We analyzed the underlying process using p-charts for statistical process control.
Results: In 65,170 anesthetics we recorded adverse events in 18.3%; mostly of lesser severity. Control charts were used to define statistically the predictable normal variation in problem rate, and then used as a basis for analysis of the selected problems with the following results:
Conclusion: By applying statistical process control methods to the analysis of adverse events, we have exemplified how this allows us to determine if a process is stable, whether an intervention is required, and if quality improvement efforts have the desired effect.
| Introduction |
|---|
|
|
|---|
Conclusions drawn from simple snap-shot measurements of the frequency of adverse events are, however, not useful unless the characteristics of the underlying process is understood.3 Statistical tools, such as process control methods, may be applied to make inferences about process performance.
We have established a system for mandatory routine reporting of adverse events during anesthesia.4 This investigation is intended to demonstrate the application of statistic process control methods to such data. These methods enable us to distinguish between natural variability and significant changes in the anesthetic process, and to continuously evaluate different aspects of anesthetic process quality. To exemplify this, we have analyzed four selected adverse events which reflect different aspects of quality and safety of anesthetic practice.
| Methods |
|---|
|
|
|---|
The data field for intraoperative adverse events includes a list of 22 common anesthetic problems (Appendix I, available as additional material at www.cja-jca.org), as well as a field for severity.4 Other data fields on the chart relate to the patient, the operation, type of anesthetic, and timing of events.
An intraoperative adverse event is defined as an event which requires one or more measures - either to prevent further complications, or to treat a situation that is currently or potentially serious, and which does not routinely occur during the conduct of anesthesia. The events are graded according to severity: grade 1 is a trivial problem, easily dealt with and not affecting the patients condition; grade 2 indicates moderate difficulty, with some effect on the patient, but of a minor severity; grade 3 is a serious event which is difficult to manage, or causing a serious deterioration in the patients state, and which may or may not have consequences for the patient postoperatively; grade 4 events imply a fatal outcome.
Our department at the Trondheim University Hospital (930 beds, annual admission rate 43,000 patients) delivers 17,500 anesthetics per year, and most types of surgery are performed. The anesthesiologist works in cooperation with a qualified nurse anesthetist, who has 18 months postgraduate education plus 12 months practical training in anesthesia. The adverse events are coded and graded jointly by the anesthesiologist and the nurse.
Data were retrieved from the department database for the years 19972001. All general and regional anesthetics (spinal, epidural, plexus) were included, except those involving patients under 16 yr and cardiac anesthesia cases. This represented a total of 65,170 cases. In 661 cases, an initial regional anesthetic was converted to a general anesthetic, mainly because of inadequate analgesia. Adverse events in these cases were ascribed to regional or general anesthesia depending on the type of event.
We analyzed the type, occurrence, and severity of intraoperative adverse events.
To evaluate different aspects of the anesthetic process, we selected four intraoperative adverse events, of relevance to anesthetic quality and safety. Their occurrences were analyzed using control chart analysis, a statistical process control method. The events chosen were: inadequate analgesia during brachial plexus block, emergence from general anesthesia, intubation problems, and medication errors.
Inadequate analgesia during brachial plexus block, insufficient for surgery, must be supplemented with large doses of iv analgesics or converted to general anesthesia. The failure rate for such blocks reflects one aspect of service quality in regional anesthesia.
Emergence from general anesthesia may be associated with life threatening problems with the airway or circulation during awakening. The rate of these problems reflects aspects of the quality of our work during the critical phase of awakening, re-establishment of reflexes and spontaneous ventilation. We addressed our problems during emergence in the first quarter of 1999 as their occurrence was found to be unacceptably high. We found that the problems most likely were due to residual drug effect, or misjudgment of the patients respiratory status before extubation. Educational department meetings focused on preventive strategies, including the use of opioids and neuromuscular blocking drugs. At that time, we changed from long- to intermediate-acting muscular relaxants.
Intubation problems can have catastrophic outcomes. We studied grade 2 and 3 problems, i.e., severe or moderately severe. These are caused in part by patient factors (anatomy), and in part by anesthetic factors, including failure to recognize anatomical signs of difficult intubation, choice of intubation techniques, and their correct use. The frequency of intubation problems thus reflects aspects of both patient safety and anesthetic quality.
Medication errors also reflect a quality aspect of our service. We have previously shown these to be rare in our department,5 and therefore we increased the sampling period for these events to three months. The chart exemplifies the difficulties related to charting rare events, and their use as quality indicators.
Statistical methods
We charted the occurrence of adverse events during a defined time interval (bi-monthly or tri-monthly), the rate being the number of adverse events per hundred anesthetics. The rate of adverse events is expected to vary from period to period. This demonstrates the natural variation caused by the random combination of many different causes, e.g., time of day, the patients physical condition, methods used, and working routines. It represents the systemic variation inherent as a regular part of the process and characterizes a stable process.6
Unnatural variations, on the other hand are observations that are very unlikely to occur within a stable process. They characterize the unstable process, and are usually presumed to represent special events or deviations from the regular process.
The frequency of adverse events during the defined time intervals were compared to the average of the entire 19972001 period in a control chart, used for statistical process control.69 The control chart used was a p-chart, where each data point expresses the proportion (percent) of cases with adverse events in a given interval. It also includes probability limits for natural process variation. These were calculated from the binomial-based standard deviation (SD) of proportions [SD = SQRT(p*(1-p)/n)], where p was the long-time average proportion of events, and n the number of cases in the interval. The upper/lower control limits for natural variability were set to ± 3 SD from the long-time average, in accordance with recommendations from the literature.7,9 We used normal distribution as an approximation to the binomial-probability distribution, which is acceptable when n is large and p is low.10 Consequently the likelihood for data points within these 3 SD limits to represent natural variability is high (P = 0.9973).
Observations beyond the 3 SD control limits are unlikely to occur within an unchanged process (P = 0.0027), and probably represent unnatural variation. Numerous supplementary tests for out of control based on other statistical calculations have been suggested, for example nine or more successive points on the same side of the average, or 12 of 14 consecutive points on the same side of the average.79
Whether the process is in statistical control (stable) or not in control (unstable) determines what kind of action is appropriate to improve the process. To improve a stable process the regular underlying factors included in the process must be changed. In an unstable process the new special causes must be identified and removed from the regular process.
Age was compared using t test. American Society of Anesthesiologists (ASA) physical status class using Wilcoxon Rank-sum test. We used a Chi-square test for categorical data. Values of P < 0.05 were considered statistically significant.
| Results |
|---|
|
|
|---|
|
|
Use of control charts (Figures 1
, 2a
, 2b
; Appendices IIIIV)
These control p-charts show the recorded variation in rate of the selected problems, and illustrate how charting of adverse events can be used as a technique for statistical process control.
|
|
|
Figure 2a
shows a control chart of the difficulties during emergence from general anesthesia. There were 1,123 cases among 45,087 general anesthetics (2.5%), and the variation ranged between 3.85% and 1.25%. There is a trend towards a decrease in difficult emergence during the years 19972001. The process is statistically unstable, with points outside or at the control limits both at the beginning and towards the end of the time period. In addition, 12 of 14 consecutive points are on the same side of the centre line both at the start and at the end of the study period, indicating an unstable process, influenced by special causes.79 We addressed our problems with emergence in early 1999. In Figure 2b
the data points are treated as two different processes, before and after the intervention. They present as two different stable processes, with a lower mean value in the second time period. If the rates of the two periods are compared, there was a decrease in adverse events after the intervention (613 events/20,540 cases) vs (510 events/24,547 cases; P < 0.001).
The chart of intubation difficulties is presented in Appendix III (available as additional material at www.cja-jca.org). There were 429 grade 2 and 3 problems within 28,081 intubations (1.5%), and the variation was between 0.52% and 2.20%. The process is statistically stable, as there were no points outside the control limits or any trends to either side of the long time mean.
The chart of drug/medication errors is presented in Appendix IV (available as additional material at www.cja-jca.org), with three-monthly time periods. There were 81 drug errors in 65,170 anesthetics (0.12%), with a variation from 0.03% to 0.25%. The process is stable, as no points are outside the control limits. Note that the lower control limits are not applicable, as the limit calculation returns negative numbers.
| Discussion |
|---|
|
|
|---|
Why is it important to record near misses?
Since anesthetic mortality and serious morbidity occur infrequently, they have limited value as indicators of quality in single institutions.1,1113 Alternatively, one may include a broader range of less serious but more frequent events.1,2,11,14 Such minor intraoperative anesthesia-related events have been said to be irrelevant, being merely surrogate outcomes for real postoperative morbidity,15 but have been shown to influence postanesthesia care.16 Previously, in our subgroup of serious non-fatal intraoperative events (grade 3),17 one third led to a change in the patients postoperative course, either by unplanned admittance to the intensive care unit, or postponement of surgery. In a study by Boëlle and coworkers, undesirable anesthesia and recovery room events were associated with development into critical events with an odds ratio of 3.44.8.18 Therefore, intraoperative adverse events may be used to analyze the safety and quality of the anesthetic process.
Adverse event rates
Most studies of mandatory case reporting are either more than ten years old,1,2 or group intraoperative and postoperative problems together.1921 Our method of recording problems has been described earlier.4 In contrast to studies where information is collected on a voluntary basis, our recording is mandatory and is completed for every anesthetic. Even an uneventful anesthetic is recorded. Minimal extra workload, relevant feedback, and a non-punitive atmosphere are important to improve compliance. However, underreporting is a recognized problem in such systems.22,23
We do not imply that the rates of adverse events in our study represent an acceptable level of quality; further analysis is needed to determine this. However, such rates can be used internally for quality comparisons. If an event occurs more commonly than is acceptable, the process can be analyzed and corrective measures taken, and the quality circle closed.24
Statistical process control - as a quality improvement tool
Statistical process control methods, first proposed by Walter Shewhart,25 have been used for many years for process improvement in the industry. They have also been applied in health care for describing and analyzing processes that affect quality of care in healthcare organizations,3,2628 as well as in anesthesia.18,24,2931
Variation is expected in any process. Different conditions, patients, staff, and methods all combine randomly and contribute to variation in performance, even when the process itself remains unchanged. Under such conditions, isolated observations provide insufficient information on which to base decision-making, as they may be the result of chance, rather than real deviation in process performance. Decision-making requires a series of observations, so that recognizable and predictable patterns can be appreciated. Statistical process control methods, and control charts can be used to accomplish this.69,3235
If a new data point on the chart has a higher value than the previous one, but both points are within the control limits, this reflects natural variation within a stable process. Were this increase to be acted upon as if the process had fundamentally changed, the analysis and action taken could be wrong (as the process probably is unchanged). The probability of measurement points occurring outside control limits, within an unchanged process is very small when using 3 SD limits. Such wide limits prevent too many false alarms, but may also conceal significant trends, as too few true alarms may go off.9
A special cause is most often intermittent, and will show up as a spike on the chart. However, a trend in the data points may indicate that a special cause is making the process unstable. A quality improvement initiative may result in such a trend. There are a separate set of within limit rules for instability, including rules for trends.79
In summary, the control chart shows whether a process is stable or subject to special cause variation. This determines if valid comparisons can be made, and indicates the correct approach to improve the process. It will also show if a quality improvement initiative has been effective. However, the control chart says little, by itself, about quality, as a process can be stable with minimal natural variation, and still reflect poor quality if the frequency of a quality associated adverse event is too high.
Examples of control chart application
The departments ability to supply well functioning brachial plexus blocks is illustrated in Figure I
. The process is statistically stable. However, the average failure rate is high compared to published studies.36 Clearly the department must take corrective action, but we have not solved this problem, as can be seen from the chart. The chart illustrates that even if the process is stable it still may reflect poor quality.
Figure 2a
presents the rate of difficulties during emergence from general anesthesia as an unstable process. A quality improvement initiative in 1999, may be the special cause that destabilized the process. Figure 2b
presents two different processes, before and after the intervention. There are clearly two stable processes; the lower mean value in the second time period indicates a successful intervention. The current rate of emergence problems must now be reevaluated for acceptability from a quality and safety viewpoint, thus closing the quality circle.
The chart presenting intubation difficulties shows a stable process, with an average rate of 1,5%. This is, for example, comparable to 1.8% difficult intubations in a study by Rose and Cohen,37 and such comparisons with the international literature will determine if our frequency is acceptable.
We found a low rate of drug errors. As a consequence of this, the lower control limit falls below zero, and therefore the p-chart methodology cannot detect decreases in the error rate if quality efforts are made. There are other methods for charting infrequent events,7,8 or alternatively a qualitative analysis may form a basis for preventive strategies.5
| Conclusion |
|---|
|
|
|---|
| Footnotes |
|---|
Revision received June 11, 2003. Accepted for publication February 19, 2003.
| References |
|---|
|
|
|---|
2 Cooper JB, Cullen DJ, Nemeskal R, et al. Effects of information feedback and pulse oximetry on the incidence of anesthesia complications. Anesthesiology 1987; 67: 68694.[Medline]
3 Levett JM, Carey RG. Measuring for improvement: from Toyota to thoracic surgery. Ann Thorac Surg 1999; 68: 3538.
4 Fasting S, Gisvold SE. Data recording of problems during anaesthesia: presentation of a well-functioning and simple system. Acta Anaesthesiol Scand 1996; 40: 117383.[Medline]
5 Fasting S, Gisvold SE. Adverse drug errors in anesthesia, and the impact of coloured syringe labels. Can J Anesth 2000; 47: 10607.[Abstract]
6 Benneyan JC. Statistical quality control methods in infection control and hospital epidemiology, part I: introduction and basic theory. Infect Control Hosp Epidemiol 1998; 19: 194214.[Medline]
7 Benneyan JC. Statistical quality control methods in infection control and hospital epidemiology, part II: chart use, statistical properties, and research issues. Infect Control Hosp Epidemiol 1998; 19: 26583.[Medline]
8 Lee K, McGreevey C. Using control charts to assess performance measurement data. Jt Comm J Qual Improv 2002; 28: 90101.[Medline]
9 Finison LJ, Finison KS, Bliersbach CM. The use of control charts to improve healthcare quality. J Healthc Qual 1993; 15: 923.[Medline]
10 Rosner B. Normal approximation to the binomial distribution In: Rosner B (Ed.). Fundamentals of Biostatistics. Pacific Grove: Duxbury; 2000: 13844.
11 Cooper JB. How to measure what happens. Can J Anaesth 1991; 38: 10323.[Medline]
12 Lee A, Lum ME. Measuring anaesthetic outcomes. Anaesth Intensive Care 1996; 24: 68593.[Medline]
13 Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ 2000; 320: 75963.
14 Spencer FC. Human error in hospitals and industrial accidents: current concepts. J Am Coll Surg 2000; 191: 4108.[Medline]
15 Orkin FK, Cohen MM, Duncan PG. The quest for meaningful outcomes (Editorial). Anesthesiology 1993; 78: 41722.[Medline]
16 Bothner U, Georgieff M, Schwilk B. The impact of minor perioperative anesthesia-related incidents, events, and complications on postanesthesia care unit utilization. Anesth Analg 1999; 89: 50613.
17 Fasting S, Gisvold SE. Serious intraoperative problems - a five-year review of 83,844 anesthetics. Can J Anesth 2002; 49: 54553.
18 Boelle PY, Garnerin P, Sicard JF, Clergue F, Bonnet F. Voluntary reporting system in anaesthesia: is there a link between undesirable and critical events? Qual Health Care 2000; 9: 2039.
19 Schwilk B, Muche R, Treiber H, Brinkmann A, Georgieff M, Bothner U. A cross-validated multifactorial index of perioperative risks in adults undergoing anaesthesia for non-cardiac surgery. Analysis of perioperative events in 26907 anaesthetic procedures. J Clin Monit Comput 1998; 14: 28394.[Medline]
20 Bothner U, Georgieff M, Schwilk B. Building a large-scale perioperative anaesthesia outcome-tracking database: methodology, implementation, and experiences from one provider within the German quality project. Br J Anaesth 2000; 85: 27180.
21 Schwilk B, Muche R, Bothner U, Goertz A, Friesdorf W, Georgieff M. Quality control in anesthesiology. Results of a prospective study following the recommendations of the German Society of Anesthesiology and Intensive Care (German). Anaesthesist 1995; 44: 2429.[Medline]
22 Sanborn KV, Castro J, Kuroda M, Thys DM. Detection of intraoperative incidents by electronic scanning of computerized anesthesia records. Comparison with voluntary reporting. Anesthesiology 1996; 85: 97787.[Medline]
23 Cullen DJ, Bates DW, Leape LL, Adverse Drug Event Prevention Study Group. Prevention of adverse drug events: a decade of progress in patient safety. J Clin Anesth 2000; 12: 60014.[Medline]
24 Gaitini LA, Vaida SJ, Ben-David B, et al. Using rate-based events to improve clinical practice. J Healthc Qual 2000; 22: 49.
25 Berwick DM. Controlling variation in health care. A consultation from Walter Shewhart. Med Care 1991; 29: 121225.[Medline]
26 Schumock GT, Seeger JD, Kong SX. Control charts to monitor rates of adverse drug reactions. Hosp Pharm 1995; 30: 1088; 10912; 10956.[Medline]
27 Clark DE, Cushing BM, Bredenberg CE. Monitoring hospital trauma mortality using statistical process control methods. J Am Coll Surg 1998; 186: 6305.[Medline]
28 Boggs PB, Hayati F, Washburne WF, Wheeler DA. Using statistical process control charts for the continual improvement of asthma care. Jt Comm J Qual Improv 1999; 25: 16381.[Medline]
29 Lagasse RS, Steinberg ES, Katz RI, Saubermann AJ. Defining quality of perioperative care by statistical process control of adverse outcomes. Anesthesiology 1995; 82: 11818.[Medline]
30 Bothner U, Georgieff M, Schwilk B. Validation of routine incidence reporting of one anaesthesia provider institution within a nation-wide quality of process assessment program. J Clin Monit Comput 1998; 14: 30511.[Medline]
31 Vitez TS, Macario A. Setting performance standards for an anesthesia department. J Clin Anesth 1998; 10: 16675.[Medline]
32 Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman, and clinical governance: Shewharts forgotten lessons. Lancet 2001; 357: 4637.[Medline]
33 Laffel G, Blumenthal D. The case for using industrial quality management science in health care organizations. JAMA 1989; 262: 286973.[Abstract]
34 Benneyan JC. Use and interpretation of statistical quality control charts. Int J Qual Health Care 1998; 10: 6973.[Medline]
35 Humble C. Caveats regarding the use of control charts. Infect Control Hosp Epidemiol 1998; 19: 8658.[Medline]
36 Neal JM, Hebl JR, Gerancher JC, Hogan QH. Brachial plexus anesthesia: essentials of our current understanding. Reg Anesth Pain Med 2002; 27: 40228.[Medline]
37 Rose DK, Cohen MM. The airway: problems and predictions in 18,500 patients. Can J Anaesth 1994; 41: 37283.
This article has been cited by other articles:
![]() |
W. S. Sandberg, E. H. Sandberg, A. R. Seim, S. Anupama, J. M. Ehrenfeld, S. F. Spring, and J. L. Walsh Real-Time Checking of Electronic Anesthesia Records for Documentation Errors and Automatically Text Messaging Clinicians Improves Quality of Documentation Anesth. Analg., January 1, 2008; 106(1): 192 - 201. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Thor, J. Lundberg, J. Ask, J. Olsson, C. Carli, K. P. Harenstam, and M. Brommels Application of statistical process control in healthcare improvement: systematic review Qual. Saf. Health Care, October 1, 2007; 16(5): 387 - 399. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. M. Davies Learning from others/Apprendre des autres Can J Anesth, October 1, 2003; 50(8): 755 - 760. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |