Can borderline-regression method be used to standard set OSCEs in small cohorts?

Rosa Moreno Lopez* (Corresponding Author), David Hope

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Introduction

Absolute methods of standard setting (SS) are more suitable for OSCEs. However, previous studies have theorised that borderline regression method (BRM) is not reliable for small sample sizes.

Materials and methods

OSCE results for the 2017–2019 cohorts were analysed to compare BRM versus Modified Angoff. We reported on whether the stations in multiple cohorts were sufficiently equivalent to aggregate the results together to calculate which method of SS was more reliable. We finally used the bootstrapping method to compare the accuracy of BRM for small versus simulated larger cohorts.

Results

BRM was a valid method for SS OSCEs in this dataset when station quality was sufficiently high. However, a large gap between the Angoff and BRM in some of the OSCEs was observed, which could be explained by poor use of the grading scale. Model fit statistics were generally adequate even with low sample sizes. Using the bootstrap of datasets, the error rate was much higher for low-quality stations but was not an issue in high-quality ones.

Discussion

This study adds to the evidence that well-designed OSCEs can use BRM for small cohorts. However, there is a need for the institutions to properly assess their stations and their assessors, before embarking into using this method, to prevent from having to remove stations.

Conclusions

This analysis suggests that BRM is an acceptable replacement for Angoff SS in small cohorts, where there is a range of candidates undertaking the assessments and there are well-designed OSCES with well-trained examiners.
Original languageEnglish
JournalEuropean Journal of Dental Education
Early online date7 Jan 2022
DOIs
Publication statusE-pub ahead of print - 7 Jan 2022

Keywords

  • Standard setting
  • OSCE
  • borderline-regression
  • Modified Angoff

Fingerprint

Dive into the research topics of 'Can borderline-regression method be used to standard set OSCEs in small cohorts?'. Together they form a unique fingerprint.

Cite this