Analysis of inter-examiner reliability of OSCE assessments

Karolin Hijazi, Rasha Abu Eid, Jennifer Foley

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

Abstract

Aims: To assess inter-examiner reliability of OSCE assessments in a graduate-entry dental school.

Materials and Methods: Ethical approval was granted from the College Ethics Research Board. Data for 30 OSCE stations undertaken by the first three years of a dental course during the period 2010–2012 were co-marked by two independent examiners who were blinded to each other’s mark sheet. No communication was allowed between examiners. Individual OSCE stations covered the following themes: (i) Disinfection and Infection Control; (ii) General Health and Disease; (iii) Communication Skills and (iv) Patient Care. Total exam scores for each year were analysed by Wilcoxon signed-rank tests. Inter-examiner reliability for both mark sheet scores (0–20) and global scores (1–5) were analysed with Pearson-and Spearman’s correlation (IBM® SPSS ® Statistics 19).

Results: Comparing the total examination scores for the two sets of examiners, the median of differences approached (p=0.113). For all stations, inter-examiner reliability demonstrated a significant correlation for both OSCE- and global scores (r=0.716 and r=0.655, both p=0.001, df=544). Inter-examiner reliability demonstrated a significant correlation for OSCE scores as follows: Disinfection and Infection Control (r=0.698, p=0.001, df=109), General Health and Disease(r=0.705, p=0.001, df=122), Communication Skills (r=0.607, p=0.001,df=109) and Patient Care (r=0.699, p=0.001, df=178). Global scores correlated closely for all OSCE stations (ranger=0.484–0.650, all p=0.001).

Conclusions: In general, a moderate correlation existed between examiners for all OSCE-type stations. Assessing the reliability between OSCE examiners is important for quality assurance purposes.
Original languageEnglish
Title of host publicationAssociation of Dental Education in Europe (ADEE) 38th Annual Meeting Proceedings
PublisherWiley
DOIs
Publication statusPublished - 2012
Event38th Annual Meeting of the Association for Dental Education in Europe: Evidence-based dentistry: from bench to clinic - Lyon, France
Duration: 29 Aug 20121 Sep 2012
https://adee.org/meetings/meetings-track

Conference

Conference38th Annual Meeting of the Association for Dental Education in Europe
Country/TerritoryFrance
CityLyon
Period29/08/121/09/12
Internet address

Fingerprint

Dive into the research topics of 'Analysis of inter-examiner reliability of OSCE assessments'. Together they form a unique fingerprint.

Cite this