Analysis of inter-examiner reliability of OSCE assessments

Karolin Hijazi, Rasha Abu Eid, Jennifer Foley

Research output: Contribution to journalConference articlepeer-review

Abstract

Aims: To assess inter-examiner reliability of OSCE assessments in a graduate-entry dental school.

Materials and Methods: Ethical approval was granted from the College Ethics Research Board. Data for 30 OSCE stations undertaken by the first three years of a dental course during the period 2010–2012 were co-marked by two independent examiners who were blinded to each other’s mark sheet. No communication was allowed between examiners. Individual OSCE stations covered the following themes: (i) Disinfection and Infection Control; (ii) General Health and Disease; (iii) Communication Skills and (iv) Patient Care. Total exam scores for each year were analysed by Wilcoxon signed-rank tests. Inter-examiner reliability for both mark sheet scores (0–20) and global scores (1–5) were analysed with Pearson-and Spearman’s correlation (IBM® SPSS® Statistics 19).

Results: Comparing the total examination scores for the two sets of examiners, the median of differences approached (p=0.113). For all stations, inter-examiner reliability demonstrated a significant correlation for both OSCE- and global scores(r=0.716 and r=0.655, both p=0.001, df=544). Inter-examiner reliability demonstrated a significant correlation for OSCE scores as follows: Disinfection and Infection Control (r=0.698, p=0.001, df=109), General Health and Disease(r=0.705, p=0.001, df=122), Communication Skills (r=0.607, p=0.001,df=109) and Patient Care (r=0.699, p=0.001, df=178). Global scores correlated closely for all OSCE stations (ranger=0.484–0.650, all p=0.001).

Conclusions: In general, a moderate correlation existed between examiners for all OSCE-type stations. Assessing the reliability between OSCE examiners is important for quality assurance purposes.
Original languageEnglish
Pages (from-to)e206
Number of pages1
JournalEuropean Journal of Dental Education
Volume17
Issue number2
Early online date11 Apr 2013
DOIs
Publication statusPublished - 1 May 2013
Event38th Annual Meeting of the Association for Dental Education in Europe: Evidence-based dentistry: from bench to clinic - Lyon, France
Duration: 29 Aug 20121 Sept 2012
https://adee.org/meetings/meetings-track

Fingerprint

Dive into the research topics of 'Analysis of inter-examiner reliability of OSCE assessments'. Together they form a unique fingerprint.

Cite this