| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |




From the Department of Anesthesiology,
* Faculty of Medicine and the Faculty of Nursing, University of Montreal, and the
Department of Psychology, Université du Québec à Montréal, Montreal, Canada.
Address correspondence to: Dr Jean-François Hardy, Department of Anesthesiology, Faculty of Medicine, University of Montreal, C.P. 6128, succursale Centre-Ville, Montreal, Quebec, H3C 3J7, Canada. E-mail: jean-francois.hardy{at}umontreal.ca
| Abstract |
|---|
|
|
|---|
Source: We reviewed relevant major data banks - Medline, Dissertation Abstract, Psyclit and Cochrane - between 1980 and 2000 and bibliographies from primary sources. We used the following keywords for our search: quality improvement, anesthesia, quality, patient perceptions, consumer satisfaction, continuous quality improvement, outcome measures.
Principal findings: The review yielded 14 pertinent studies. Studies were divided into two groups (A & B), according to the quality of the psychometric evaluation (tests performed to verify the reliability and validity of an instrument). While all studies reported high levels of patient satisfaction with anesthesia services, many used methods of questionable value. None of the 14 studies controlled for any confounding variables, such as social desirability. Four studies had seriously biased their data collection and the majority of the studies lacked rigour in the development of the instrument used to measure patient satisfaction. Only one study presented a definition of the concept measured, and none provided a conceptual model of patients' satisfaction with anesthesia services.
Conclusion: The currently available studies of patient satisfaction are of questionable value. Only rigorous methods and reliable instruments will yield valid and clinically relevant findings of this important issue in anesthesiology.
THE major data banks contain a vast array of articles, published between 1980 and 2000, that examined patient satisfaction with medical care. Few of these publications, however, cover patient satisfaction with services provided by anesthesiologists. Despite the paucity of publications in the field, several authors believe that patient satisfaction is an attribute of the quality of care in anesthesia.13
Recently, a review on patient satisfaction and anesthesia care was published,4 focusing on the appropriateness of different methodologies used to measure patient satisfaction. In contrast, this present article seeks to provide a more critical review of the methodology, the development of the instrument used by the authors and psychometrics (when available) and the results obtained by selective studies treating this concept.
| Sources |
|---|
|
|
|---|
Data sources
We searched Medline, Current Contents, the Cochrane Data Base, and Dissertation Abstract for studies published between 1980 and March 2000. We searched the bibliographies of primary and review articles for other relevant studies. Our search was restricted to French and English publications.
We used the following key words to retrieve articles: quality improvement, anesthesia (including anaesthesia), quality, patient satisfaction, patient perceptions, consumer satisfaction, continuous quality improvement, outcome measures. We also combined some of the main keywords: quality and anesthesia, quality improvement and anesthesia, patient (and consumer) satisfaction and anesthesia, quality and patient satisfaction and anesthesia (including anaesthesia).
Study selection
We selected studies that emphasized patients' satisfaction with anesthesia services. Included were studies covering both ambulatory and hospitalized patients. Since relatively few studies focused mainly on patient satisfaction with anesthesia services, we did not use any selection criteria and included all publications in the present review. We excluded drug trials regarding patient satisfaction over pain management. These studies used relief of pain as the sole indicator of patient satisfaction.
Data extraction and synthesis
Data were extracted following three specific categories: source, population (characteristics of the sample), a summary of the main results vis-a-vis patient satisfaction and the instrument used. Studies were divided in two groups (A and B). Group A includes those studies in which no psychometric testing (tests performed to evaluate the reliability and validity of instruments) was done on the scales employed to measure patient satisfaction. Group B includes the studies that performed certain psychometric tests on the scale employed to measure patient satisfaction.
| Principal findings |
|---|
|
|
|---|
|
Main results on patient satisfaction and psychometrics
The majority of studies, whether from group A or B, reported high levels of patient satisfaction with anesthesia. However, when patients were given an opportunity to express themselves freely, they noted certain unsatisfactory elements in the care received from anesthesiologists.
The Group B studies, although using more rigorous methods than those in Group A, still obtained very high levels of satisfaction. Only one study11 (in Group B) defined patient satisfaction, and none of the six studies included a conceptual framework of their constructs. Furthermore, no study included other questionnaires to control for confounding variables, such as level of psychological distress or social desirability, even though these variables may affect a patient's ability to properly complete a questionnaire on satisfaction and to answer rationally.
The psychometric testing of the scales was performed in a variety of ways. Shevde & Panagopoulos,19 presented a factor analysis of the data to reveal the presence of four distinct factors (specific complications of anesthesia, characteristics of the anesthesiologist, anxiety about being hospitalized and pain). No other testing was performed on their scale. Pestey16 evaluated the reliability of her scale using a test-retest with short term surgery patients only (n=38). Pearson correlation coefficients were calculated for each individualized questions (Q2=0.76, Q3=0.88, Q4=0.76, Q5=0.79, Q7=0.66). Whitty et al.,15 developed a scale using patient focus groups, comments from anesthesiologists and a theory of patient satisfaction. Content validity was verified with patients, anesthesiologists, experts on questionnaire design, and a "fog index" (Microsoft Word software that screens a text and identifies words exceeding the level of grade 6). No other testing was performed on their scale. Tong et al.11 used a Kappa (interrater agreement) between nurses and anesthesiologists to assess the reliability of their two scales (a standardized check-off form for adverse events, a 24-hr postoperative telephone questionnaire). Values of k >0.9 were obtained for both scales. As to the reliability of the questions on satisfaction, these authors relied on the psychometric informations obtained in a previous study20 and did no testing of the data (data obtained using the questionnaire of Abramovitz and colleagues). Dexter and colleagues13 performed various tests on their new scale (Iowa Satisfaction with Anesthesia Scale [ISAS]) and obtained a value of 0.80 for internal consistency (Cronbach's Alpha), and correlations of R2 = 0.74 (after one hour) and R2 = 0.76 (after four days) for stability (test-retest). Content validity (evaluation of the extent to which the method of measurement includes all the major elements relevant to the construct being measured) of the ISAS was verified once, before the study began; the verification was performed by professionals. To establish convergent validity, the authors compared patients' overall scores to those predicted by their anesthesia provider (R2 = 0.23, P > 0.01). Fleisher et al.9 relied on the correlation between the question regarding quality of care and the one regarding satisfaction with pain management to verify the internal consistency of their instrument. A Cronbach's Alpha of 0.62 was obtained. No other testing was performed on the scale.
| Discussion |
|---|
|
|
|---|
Another possible explanation for these positive results is that most researchers have a bias towards measuring satisfaction. In other words, they do not allow elements of dissatisfaction to emerge from their surveys. Questionnaires are oriented more towards the researchers' desire for positive results. They do not offer open-ended questions where patients can advance opinions or express dissatisfaction concerning elements covered or not by the survey.
Only one study11 presented a clear definition of the concept of patient satisfaction. As for the remainder, since no definition was provided, it is not clear what these studies truly measured. For example, the ISAS scale13 appears to measure more patients' physical condition at a precise moment in time than the patients'satisfaction regarding anesthesia services. Further, none of the 14 studies presented a conceptual framework relative to patients' satisfaction with anesthesia services, thus leaving the reader in the dark as to what variables might explain the underlying concept.
Another important cause for concern is that no study attempted to control for any confounding variables. Respondents might not be answering the items of primary interest for the reasons we assume. There may be other motivations influencing their responses. One type of motivation that can be assessed fairly easily is social desirability. Social desirability is defined as the motivation of a person to present herself or himself in a way that society regards as positive.21 If respondents are influenced by social desirability, they may distort their answers and thus invalidate their evaluation of satisfaction. For instance, in the Dexter et al. study,13 90% of the respondents, in the first version of their instrument, answered that they totally agreed with the statement "I like my anesthetist". Also, in the study by Keep et al.10 one patient who had been awake during her esophagoscopy mentioned that she was satisfied with the anesthetic but would prefer to be asleep the next time! Did social desirability influence or help to explain the high levels of satisfaction obtained? Since none of the 14 studies reviewed controlled for social desirability, the validity of the results remains open to question. Perhaps the use of Marlow & Crowne's abridged social desirability scale,22 along with a patient satisfaction scale, would help to increase the validity of the results obtained by excluding respondents vulnerable to social desirability.
In at least four of the studies reviewed,7,8,13,17 patients may have been influenced by either the presence of the anesthesiologist or the suspicion that the anesthesiologist could identify them. For instance, if the anesthesiologist remains nearby while the patient is completing the questionnaire, as occurred in three studies10,13,17 the patients' right to anonymity and confidentiality is not protected. Patients cannot be expected to express their comments, suggestions or grievances if they have even the slightest doubt that their right to confidentiality and anonymity is not being respected.23
Few clinicians are aware of the importance of psychometric evaluation in the development of a new scale.4 For example, Dexter et al.13 tested for reliability after an interval of four days, mentioning that "the test-retest method of assessing reliability is sensitive to the time interval between testing. The longer the time interval, the lower is the reliability". Yet, for paper-and-pencil measures, experts recommend a waiting period of two weeks to a month between tests, the goal being to keep subjects from remembering their answers to the first test, which could lead to an overestimation of the test's reliability.21,24 Further, it is considered irrelevant to perform test-retest evaluation with instruments measuring patient satisfaction since this concept may vary in time and it is also influenced by the effect of memory which is an undesirable consequence in the measurement of this concept. It is more appropriate to focus on obtaining reliable results regarding the internal consistency of the scale, thus an elevated Alpha of Cronbach. Similarly, having a small developmental sample (less than 10 subjects per items) to evaluate internal consistency (Cronbach's Alpha) can produce inappropriately favourable results.21,25 When the ratio of subjects to items is relatively low and the sample size is small, correlations between items will be influenced by chance to a substantial degree. An Alpha of Cronbach of 0.80 using only 49 subjects, on a scale of 11 items,13 should therefore be questioned. Finally, even the employment of a very large sample12 (10,811 patients) did not compensate the evidence and consequences of a lack of rigour in the methods and the development of the instrument to measure patient satisfaction. Indeed, in the study by Myles et al.,12 the variable «patient satisfaction», considered as a major outcome, was measured by a question with only three choices of answer and results dichotomized between satisfied vs somewhat dissatisfied-dissatisfied. Such strategy is not very well considered,26 and it does not generate enough variance in the distribution of the data collected. In addition, services offered by anesthesiologists include many components. We doubt that such a complex issue can be properly evaluated by only one question. Thus there are serious doubts about the significance of results obtained with an unvalid and unreliable instrument. Further, even though the authors recognize the influence of social desirability on respondents, no measures were taken to evaluate the impact of this important confounding variable.27
The process that must be followed to obtain valid results is complex. It involves more than writing up a few questions and passing them along to the desired population. As well, a newly developed scale must be tested to verify what it really measures, in view of producing valid results. The initial testing of a scale is not sufficient to verify all of the attributes of both reliability and validity of the scale. Results obtained with a new scale must therefore be interpreted with caution. For the sake of psychometrics, one must "sacrifice" at least one sample; only then can one pretend to significant results on the concept measured.26
In a first trial, such psychometric measures as establishing content validity, internal consistency and construct validity (determines whether the instrument actually measures the theoretical construct it purports to measure) are considered sufficient for evaluating the properties of a scale, taking into account that a theoretical construct of the variables of the study is presented at first.26 Indeed, it is inappropriate to try to perform every psychometric test available on a new scale in a single study. Achieving the complete psychometric profile of a new scale takes time.26 Any new instrument must go through several specific phases and trim before it can be considered to be reliable and valid. Also, the control of major confounding variables is essential to reduce the "contamination" of the main variable (e.g. satisfaction) to be measured by an instrument.
The task of maintaining reliability and validity of instruments is never concluded. Measurement devices are nearly always subject to the effects of social context. Values and practice patterns change, individuals think differently about a number of matters, and terminologies go out of fashion.28 Scale development is a continuous and retroactive process. Finally, care should be taken when attempting to generalize results obtained with a new instrument of unestablished validity and reliability; otherwise, one risks making false assumptions concerning the population from which the sample was drawn.
| Conclusions |
|---|
|
|
|---|
As with all standardized data collection, creating and performing high quality surveys of patients' perceptions can be challenging. Valid and reliable patient survey data can enable practitioners to identify areas for improvement, and demonstrate to external reviewers the quality of care they provide to their patients.29 That is why the development of valid and reliable instruments to measure patient satisfaction must adhere to specific psychometric requirements and control for major confounding variables. The procedure is complex. Use of a non-validated and/or non-specific instrument will only produce unreliable and meaningless results. Generalization of results from a given patient population is hazardous, especially when the initial methods are questionable.
Finally, this review allowed emphasis on the absence of appropriate instruments to measure patient satisfaction with anesthesia services, the many biases present in the methodology employed, the lack of rigour regarding the psychometric testing of the instrument used and also the very high levels of satisfaction usually obtained by surveys on patient satisfaction with anesthesia services to be demistifyed. We proposed solutions to increase the validity of the measurement of patients' satisfaction regarding the services provided by anesthesiologists.
| Acknowledgments |
|---|
| Footnotes |
|---|
Accepted for publication October 22, 2000.
| References |
|---|
|
|
|---|
2 Bierstein K. Consumer satisfaction surveys in anesthesiology practice. American Society of Anesthesiologists Newsletter 1996; 60: 269.
3
Eagle CL, Davies JM. Current models of "quality" - an introduction for anaesthetists. Can J Anaesth 1993; 40: 85162.
4
Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: a review of current methodology. Anesth Analg 1998; 87: 108998.
5
Cook D. Systematic reviews: the case for rigorous methods and rigorous reporting. Can J Anaesth 1997; 44: 3503.
6 Brown DL, Warner ME, Schroeder DR, Offord KP. Effect of intraoperative anesthetic events on postoperative patient satisfaction. Mayo Clin Proc 1997; 72: 205.[Medline]
7 Preble LM, Perlstein L, Katsoff-Seidman, L, O'Connor TZ, Barash PG. The patient care evaluation system: patients' perceptions of anesthetic care. Connecticut Med 1993; 57: 3636.
8 Zvara DA, Nelson JM, Brooker RF, et al. The importance of the postoperative anesthetic visit: do repeated visits improve patient satisfaction or physician recognition? Anesth Analg 1996; 83: 7937.[Abstract]
9 Fleisher LA, Mark L, Lam J, et al. Disseminating information using an anesthesiology consultant report: impact on patient perceptions of quality of care. J Clin Anesth 1999; 11: 3805.[Medline]
10 Keep PJ, Jenkins JR. From the other end of the needle. The patient's experience of routine anaesthesia. Anaesthesia 1978; 33: 8302.[Medline]
11 Tong D, Chung F, Wong D. Predictive factors in global and anesthesia satisfaction in ambulatory surgery patients. Anesthesiology 1997; 87: 85664.[Medline]
12
Myles PS, Williams DL, Hendrata M, Anderson H, Weeks AM. Patient satisfaction after anaesthesia and surgery: results of a prospective survey of 10 811 patients. Br J Anaesth 2000; 84: 610.
13 Dexter F, Aker J, Wright WA. Development of a measure of patient satisfaction with monitored anesthesia care. The Iowa satisfaction with anesthesia scale. Anesthesiology 1997; 87: 86573.[Medline]
14 Penon C, Ecoffey C. Patients opinion on the quality of anaesthetic management. (French) An Fr Anesth Réanim 1995; 14: 3745.
15 Whitty PM, Shaw IH, Goodwin DR. Patient satisfaction with general anaesthesia. Too difficult to measure? Anaesthesia 1996; 51: 32732.[Medline]
16 Pestey ME. Patient satisfaction with anesthesia services delivered in the cost containment atmosphere of healthcare today. Unpublished master thesis. New Haven, Connecticut: Southern Connecticut State University, 1992.
17 Burrow BJ. The patient's view of anaesthesia in an Australian teaching hospital. Anaesth Intensive Care 1982; 10: 204.[Medline]
18 Dodds CP, Harding MI, More DG. Anaesthesia in an Australian private hospital: the consumer's view. Anaesth Intensive Care 1985; 13: 3259.[Medline]
19
Shevde K, Panagopoulos G. A survey of 800 patients' knowledge, attitudes, and concerns rearding anesthesia. Anesth Analg 1991; 73: 1908.
20 Abramovitz S, Coté AA, Berry E. Analyzing patient satisfaction: a multi-analytic approach. Quality Review Bulletin 1987; 13: 12230.
21 DeVellis RF. Scale development: Theory and Applications. London: Sage Publications, 1991.
22 Strahan R, Gerbasi KC. Short homogenous versions of the Marlow-Crowne social desirability scale. J Clin Psychol 1972; 28: 1913.
23 Strasser S, Davis RM. Measuring patient satisfaction for improved patient services. Ann Arbor, Michigan: Health Administration Press, 1991.
24 Nunnally JC, Bernstein IH. Psychometric Theory, 3rd ed. Montreal: McGraw Hill Inc., 1994.
25 Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to their Ddevelopment and Use, 2nd ed. Oxford: Oxford University Press, 1995.
26 Crocker L, Algina J. Introduction to Classical and Modern Test Theory. Fort Worth: Harcourt Brace Jovanovich College Publishers, 1986.
27 Le May S, Hardy J-F, Taillefer M-C, Dupuis G. Inappropriate methods for patient satisfaction (Letter). Br J Anaesth 2000; 84: 821.
28 Williams MA. Instrument development: always unfinished (Editorial). Res Nurs Health 1989; 12: iiiiv.
29 Epstein KR, Laine C, Farber NJ, Nelson EC, Davidoff F. Patients' perceptions of office medical practice: judging quality through the patients' eyes. Am J Med Qual 1996; 11: 7380.
This article has been cited by other articles:
![]() |
C. Salzwedel, C. Petersen, I. Blanc, U. Koch, A. E. Goetz, and M. Schuster The Effect of Detailed, Video-Assisted Anesthesia Risk Education on Patient Anxiety and the Duration of the Preanesthetic Interview: A Randomized Controlled Trial Anesth. Analg., January 1, 2008; 106(1): 202 - 209. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Saal, M. Nuebling, Y. Husemann, and T. Heidegger Effect of timing on the response to postal questionnaires concerning satisfaction with anaesthesia care Br. J. Anaesth., February 1, 2005; 94(2): 206 - 210. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. McGrath, H. Elgendy, F. Chung, D. Kamming, B. Curti, and S. King Thirty percent of patients have moderate to severe pain 24 hr after ambulatory surgery: a survey of 5,703 patients: [Trente pour cent des patients ont des douleurs moderees a severes, 24 h apres la chirurgie ambulatoire : une enquete aupres de 5 703 patients] Can J Anesth, November 1, 2004; 51(9): 886 - 891. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Heidegger, M. Nuebling, R. Germann, H. Borg, K. Fluckiger, T. Coi, and Y. Husemann Patient satisfaction with anesthesia care: information alone does not lead to improvement: [La satisfaction du patient face a l'anesthesie : l'information ne suffit pas a l'amelioration] Can J Anesth, October 1, 2004; 51(8): 801 - 805. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Kranke, A. M. Morin, N. Roewer, and L. H. Eberhart Patients' Global Evaluation of Analgesia and Safety of Injected Parecoxib for Postoperative Pain: A Quantitative Systematic Review Anesth. Analg., September 1, 2004; 99(3): 797 - 806. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. Souron, L. Delaunay, and P. Schifrine Intrathecal morphine provides better postoperative analgesia than psoas compartment block after primary hip arthroplasty: [L'administration intrathecale de morphine procure une meilleure analgesie postoperatoire que le bloc du plexus lombaire a la suite d'une arthroplastie primaire de la hanche] Can J Anesth, June 1, 2003; 50(6): 574 - 579. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Le May, J.-F. Hardy, F. Harel, M.-C. Taillefer, and G. Dupuis Patients' perceptions of cardiac anesthesia services: a pilot study : [Les perceptions de patients sur l'anesthesie cardiaque : une etude pilote] Can J Anesth, December 1, 2001; 48(11): 1127 - 1142. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Lepage, P. Drolet, M. Girard, Y. Grenier, and R. DeGagne Music Decreases Sedative Requirements During Spinal Anesthesia Anesth. Analg., October 1, 2001; 93(4): 912 - 916. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |