CJA
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
This Article
Right arrow Full Text (PDF)
Right arrow Submit a scholarly reply
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Devitt, J. H.
Right arrow Articles by Szalai, J. P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Devitt, J. H.
Right arrow Articles by Szalai, J. P.

Canadian Journal of Anesthesia, Vol 44, 924-928, Copyright © 1997 by Canadian Anesthesiologists' Society

ARTICLES

Testing the raters: inter-rater reliability of standardized anaesthesia simulator performance

JH Devitt, MM Kurrek, MM Cohen, K Fish, P Fish, PM Murphy and JP Szalai
Department of Anaesthesia, Sunnybrook Health Science Centre, Toronto, Ontario. j.hugh_devitt@mail.magic.ca

PURPOSE: Assessment of physician performance has been a subjective process. An anaesthesia simulator could be used for a more structured and standardized evaluation but its reliability for this purpose is not known. We sought to determine if observers witnessing the same event in an anaesthesia simulator would agree on their rating of anaesthetist performance. METHODS: The study had the approval of the research ethics board. Two one-hour clinical scenarios were developed, each containing five anaesthetic problems. For each problem, a rating scale defined the appropriate score (no response to the situation: score = 0; compensating intervention defined as physiological correction: score = 1; corrective treatment: defined as definitive therapy score = 2). Video tape recordings, for assessment of inter-rater reliability, were generated through role-playing with recording of the two scenarios three times each resulting in a total of 30 events to be evaluated. Two clinical anaesthetists, uninvolved in the development of the study and the clinical scenarios, reviewed and scored each of the 30 problems independently. The scores produced by the two observers were compared using the kappa statistic of agreement. RESULTS: The raters were in complete agreement on 29 of the 30 items. There was excellent inter-rater reliability (= 0.96, P < 0.001). CONCLUSIONS: The use of videotapes allowed the scenarios to be scored by reproducing the same event for each observer. There was excellent inter-rater agreement within the confines of the study. Rating of video recordings of anaesthetist performance in a simulation setting can be used for scoring of performance. The validity of the scenarios and the scoring system for assessing clinician performance have yet to be determined.


This article has been cited by other articles:


Home page
 PediatricsHome page
M. B. Brett-Fleegler, R. J. Vinci, D. L. Weiner, S. K. Harris, M.-C. Shih, and M. E. Kleinman
A Simulator-Based Tool That Assesses Pediatric Resident Resuscitation Competency
Pediatrics, March 1, 2008; 121(3): e597 - e603.
[Abstract] [Full Text] [PDF]

Home page
 Canadian J. AnesthesiaHome page
P. J. Morgan, J. Lam-McCulloch, J. Herold-McIlroy, and J. Tarshis
Simulation performance checklist generation using the Delphi technique: [Generation d'une liste de verification de la performance simulee a l'aide de la methode Delphi]
Can J Anesth, December 1, 2007; 54(12): 992 - 997.
[Abstract] [Full Text] [PDF]

Home page
 SEMIN CARDIOTHORAC VASC ANESTHHome page
C. L. Lake
Simulation in Cardiothoracic and Vascular Anesthesia Education: Tool or Toy?
Seminars in Cardiothoracic and Vascular Anesthesia, December 1, 2005; 9(4): 265 - 273.
[Abstract] [PDF]

Home page
 Canadian J. AnesthesiaHome page
A. K. Wong
Full scale computer simulators in anesthesia training and evaluation: [Des simulateurs informatises grandeur nature pour la formation et l'evaluation en anesthesie]
Can J Anesth, May 1, 2004; 51(5): 455 - 464.
[Abstract] [Full Text] [PDF]

Home page
 Br J AnaesthHome page
P. J. Morgan, D. Cleave-Hogg, S. DeSousa, and J. Tarshis
High-fidelity patient simulation: validation of performance checklists
Br. J. Anaesth., March 1, 2004; 92(3): 388 - 392.
[Abstract] [Full Text] [PDF]

Home page
 Br J AnaesthHome page
M. A. Olympio, R. Whelan, R. P. A. Ford, and I. C. M. Saunders
Failure of simulation training to change residents' management of oesophageal intubation{dagger}
Br. J. Anaesth., September 1, 2003; 91(3): 312 - 318.
[Abstract] [Full Text] [PDF]

Home page
 Br J AnaesthHome page
J. M. Weller, M. Bloch, S. Young, M. Maze, S. Oyesola, J. Wyner, D. Dob, K. Haire, J. Durbridge, T. Walker, et al.
Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists
Br. J. Anaesth., January 1, 2003; 90(1): 43 - 47.
[Abstract] [Full Text] [PDF]

Home page
 Anesth. Analg.Home page
M. A. Rosenblatt and K. J. Abrams
The Use of a Human Patient Simulator in the Evaluation of and Development of a Remedial Prescription for an Anesthesiologist with Lapsed Medical Skills
Anesth. Analg., January 1, 2002; 94(1): 149 - 153.
[Abstract] [Full Text] [PDF]

Home page
 Canadian J. AnesthesiaHome page
P. J. Morgan, D. M. Cleave-Hogg, C. B. Guest, and J. Herold
Validity and reliability of undergraduate performance assessments in an anesthesia simulator
Can J Anesth, March 1, 2001; 48(3): 225 - 233.
[Abstract] [Full Text] [PDF]


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Copyright © 1997 by the Canadian Anesthesiologists' Society.