This Article Full Text (PDF) Submit a scholarly reply Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Boker, A. M.A. Search for Related Content PubMed Articles by Boker, A. M.A.

NEW SCALE TO IMPROVE INTER-EXAMINER RELIABILITY DURING ORAL EXAM

Department of Anesthesia and Critical Care, King Abdulaziz University Hospital, Jeddah, Saudi Arabia

INTRODUCTION

In an attempt to improve validity and reliability of the oral examination process, the western region anesthesia residency training program introduced and tested a simple one-page scale to assess trainees’ performance during the oral component of the end-of-year promotion examinations.

METHODS

After obtaining the IRB approval, five faculty members and ten residents participated in the end-of-year promotion exam. Each of the two examiner teams (two or three examiners each) examined all residents in two Structured Oral Examination (SOE) scenarios. Each examiner assigned independently a score for each resident performance based on the predetermined scale. The scale ranged from 55 which represent extremely poor performance to 65 which corresponds to outstanding performance, with the grade of 60 signifying an average performance at the expected level of training. Each of the marks in the scale has clear description in three domains (knowledge, Organization, and Communications and expression) in addition to general description category. Correlation between each member of the examiner teams and between score of each candidate mark and their final written mark were determined. Pearson correlation (pair-wise deletion) was used for analysis.

RESULTS

For team 1, correlation between examiners was r = 0.69 (p<0.001), while for team 2, the range was r=0.76 to 0.8 (p<0.001). The correlation between the final oral exam result and the MCQ scores for the study cohort was r=81 (P=0.004).

DISCUSSION

Relatively high degrees of correlation between examiners as well as oral and MCQ were obtained with the incorporation of such simple structured scale to grade residents’ pperformance during SOE. The newly introduced scale can be used for both types of assessment, summative and formative. Further, it can be used for variety of specialty exams hence it has generally applicable framework to fit variety of post-graduates training settings.

REFERENCE:

Can J Anaesth. 2000 Sep; 47 (9):914–8[Abstract/Free Full Text]

This Article Full Text (PDF) Submit a scholarly reply Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Boker, A. M.A. Search for Related Content PubMed Articles by Boker, A. M.A.