Selection into medicine: the predictive validity of an outcome-based procedure

Sanne Schreurs (Corresponding Author), Kitty B. Cleutjens, Arno M. M. Muijtjens, Jennifer Cleland, Mirjam G. A. Oude Egbrink

Research output: Contribution to journalArticle

3 Citations (Scopus)
3 Downloads (Pure)

Abstract

BACKGROUND: Medical schools must select students from a large pool of well-qualified applicants. A challenging issue set forward in the broader literature is that of which cognitive and (inter)personal qualities should be measured to predict diverse later performance. To address this gap, we designed a 'backward chaining' approach to selection, based on the competences of a 'good doctor'. Our aim was to examine if this outcome-based selection procedure was predictive of study success in a medical bachelor program.

METHODS: We designed a multi-tool selection procedure, blueprinted to the CanMEDS competency framework. The relationship between performance at selection and later study success across a three-year bachelor program was examined in three cohorts. Study results were compared between selection-positive and selection-negative (i.e. primarily rejected) students.

RESULTS: Selection-positive students outperformed their selection-negative counterparts throughout the entire bachelor program on assessments measuring cognitive (e.g. written exams), (inter)personal and combined outcomes (i.e. OSCEs). Of the 30 outcome variables, selection-positive students scored significantly higher in 11 cases. Fifteen other, non-significant between-group differences were also in favor of the selection-positives. An overall comparison using a sign test indicated a significant difference between both groups (p < 0.001), despite equal pre-university GPAs.

CONCLUSIONS: The use of an outcome-based selection approach seems to address some of the predictive validity limitations of commonly-used selection tools. Selection-positive students significantly outperformed their selection-negative counterparts across a range of cognitive, (inter)personal, and mixed outcomes throughout the entire three-year bachelor in medicine.

Original languageEnglish
Article number214
Number of pages10
JournalBMC Medical Education
Volume18
Issue number1
DOIs
Publication statusPublished - 17 Sep 2018

Fingerprint

medicine
bachelor
selection procedure
student
OSCE
applicant
performance
Group
university
school

Keywords

  • selection
  • backward chaining
  • outcome-based
  • predictive validity

Cite this

Schreurs, S., Cleutjens, K. B., Muijtjens, A. M. M., Cleland, J., & Oude Egbrink, M. G. A. (2018). Selection into medicine: the predictive validity of an outcome-based procedure. BMC Medical Education, 18(1), [214]. https://doi.org/10.1186/s12909-018-1316-x

Selection into medicine : the predictive validity of an outcome-based procedure. / Schreurs, Sanne (Corresponding Author); Cleutjens, Kitty B.; Muijtjens, Arno M. M.; Cleland, Jennifer; Oude Egbrink, Mirjam G. A.

In: BMC Medical Education, Vol. 18, No. 1, 214, 17.09.2018.

Research output: Contribution to journalArticle

Schreurs, S, Cleutjens, KB, Muijtjens, AMM, Cleland, J & Oude Egbrink, MGA 2018, 'Selection into medicine: the predictive validity of an outcome-based procedure', BMC Medical Education, vol. 18, no. 1, 214. https://doi.org/10.1186/s12909-018-1316-x
Schreurs S, Cleutjens KB, Muijtjens AMM, Cleland J, Oude Egbrink MGA. Selection into medicine: the predictive validity of an outcome-based procedure. BMC Medical Education. 2018 Sep 17;18(1). 214. https://doi.org/10.1186/s12909-018-1316-x
Schreurs, Sanne ; Cleutjens, Kitty B. ; Muijtjens, Arno M. M. ; Cleland, Jennifer ; Oude Egbrink, Mirjam G. A. / Selection into medicine : the predictive validity of an outcome-based procedure. In: BMC Medical Education. 2018 ; Vol. 18, No. 1.
@article{a849595b6577427f9e48d519cf14745f,
title = "Selection into medicine: the predictive validity of an outcome-based procedure",
abstract = "BACKGROUND: Medical schools must select students from a large pool of well-qualified applicants. A challenging issue set forward in the broader literature is that of which cognitive and (inter)personal qualities should be measured to predict diverse later performance. To address this gap, we designed a 'backward chaining' approach to selection, based on the competences of a 'good doctor'. Our aim was to examine if this outcome-based selection procedure was predictive of study success in a medical bachelor program.METHODS: We designed a multi-tool selection procedure, blueprinted to the CanMEDS competency framework. The relationship between performance at selection and later study success across a three-year bachelor program was examined in three cohorts. Study results were compared between selection-positive and selection-negative (i.e. primarily rejected) students.RESULTS: Selection-positive students outperformed their selection-negative counterparts throughout the entire bachelor program on assessments measuring cognitive (e.g. written exams), (inter)personal and combined outcomes (i.e. OSCEs). Of the 30 outcome variables, selection-positive students scored significantly higher in 11 cases. Fifteen other, non-significant between-group differences were also in favor of the selection-positives. An overall comparison using a sign test indicated a significant difference between both groups (p < 0.001), despite equal pre-university GPAs.CONCLUSIONS: The use of an outcome-based selection approach seems to address some of the predictive validity limitations of commonly-used selection tools. Selection-positive students significantly outperformed their selection-negative counterparts across a range of cognitive, (inter)personal, and mixed outcomes throughout the entire three-year bachelor in medicine.",
keywords = "selection, backward chaining, outcome-based, predictive validity",
author = "Sanne Schreurs and Cleutjens, {Kitty B.} and Muijtjens, {Arno M. M.} and Jennifer Cleland and {Oude Egbrink}, {Mirjam G. A.}",
note = "Acknowledgements: The authors would like to thank Dr. Kelly Dore for valuable advice and sharing Computer-based Assessment for Sampling Personal characteristics (CASPer) assignments in order to develop our Situational Judgement Test and Angela Verheyen and Guus Smeets for their essential support in gathering data.",
year = "2018",
month = "9",
day = "17",
doi = "10.1186/s12909-018-1316-x",
language = "English",
volume = "18",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Selection into medicine

T2 - the predictive validity of an outcome-based procedure

AU - Schreurs, Sanne

AU - Cleutjens, Kitty B.

AU - Muijtjens, Arno M. M.

AU - Cleland, Jennifer

AU - Oude Egbrink, Mirjam G. A.

N1 - Acknowledgements: The authors would like to thank Dr. Kelly Dore for valuable advice and sharing Computer-based Assessment for Sampling Personal characteristics (CASPer) assignments in order to develop our Situational Judgement Test and Angela Verheyen and Guus Smeets for their essential support in gathering data.

PY - 2018/9/17

Y1 - 2018/9/17

N2 - BACKGROUND: Medical schools must select students from a large pool of well-qualified applicants. A challenging issue set forward in the broader literature is that of which cognitive and (inter)personal qualities should be measured to predict diverse later performance. To address this gap, we designed a 'backward chaining' approach to selection, based on the competences of a 'good doctor'. Our aim was to examine if this outcome-based selection procedure was predictive of study success in a medical bachelor program.METHODS: We designed a multi-tool selection procedure, blueprinted to the CanMEDS competency framework. The relationship between performance at selection and later study success across a three-year bachelor program was examined in three cohorts. Study results were compared between selection-positive and selection-negative (i.e. primarily rejected) students.RESULTS: Selection-positive students outperformed their selection-negative counterparts throughout the entire bachelor program on assessments measuring cognitive (e.g. written exams), (inter)personal and combined outcomes (i.e. OSCEs). Of the 30 outcome variables, selection-positive students scored significantly higher in 11 cases. Fifteen other, non-significant between-group differences were also in favor of the selection-positives. An overall comparison using a sign test indicated a significant difference between both groups (p < 0.001), despite equal pre-university GPAs.CONCLUSIONS: The use of an outcome-based selection approach seems to address some of the predictive validity limitations of commonly-used selection tools. Selection-positive students significantly outperformed their selection-negative counterparts across a range of cognitive, (inter)personal, and mixed outcomes throughout the entire three-year bachelor in medicine.

AB - BACKGROUND: Medical schools must select students from a large pool of well-qualified applicants. A challenging issue set forward in the broader literature is that of which cognitive and (inter)personal qualities should be measured to predict diverse later performance. To address this gap, we designed a 'backward chaining' approach to selection, based on the competences of a 'good doctor'. Our aim was to examine if this outcome-based selection procedure was predictive of study success in a medical bachelor program.METHODS: We designed a multi-tool selection procedure, blueprinted to the CanMEDS competency framework. The relationship between performance at selection and later study success across a three-year bachelor program was examined in three cohorts. Study results were compared between selection-positive and selection-negative (i.e. primarily rejected) students.RESULTS: Selection-positive students outperformed their selection-negative counterparts throughout the entire bachelor program on assessments measuring cognitive (e.g. written exams), (inter)personal and combined outcomes (i.e. OSCEs). Of the 30 outcome variables, selection-positive students scored significantly higher in 11 cases. Fifteen other, non-significant between-group differences were also in favor of the selection-positives. An overall comparison using a sign test indicated a significant difference between both groups (p < 0.001), despite equal pre-university GPAs.CONCLUSIONS: The use of an outcome-based selection approach seems to address some of the predictive validity limitations of commonly-used selection tools. Selection-positive students significantly outperformed their selection-negative counterparts across a range of cognitive, (inter)personal, and mixed outcomes throughout the entire three-year bachelor in medicine.

KW - selection

KW - backward chaining

KW - outcome-based

KW - predictive validity

U2 - 10.1186/s12909-018-1316-x

DO - 10.1186/s12909-018-1316-x

M3 - Article

C2 - 30223816

VL - 18

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

IS - 1

M1 - 214

ER -