Does the UKCAT predict performance on exit from medical school? A national cohort study

R K MacKenzie, J A Cleland, D Ayansina, S Nicholson

Research output: Contribution to journalArticle

6 Citations (Scopus)
8 Downloads (Pure)

Abstract

Objectives
Most UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UK Clinical Aptitude Test (UKCAT: http://www.ukcat.ac.uk), and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.

Methods
This was an observational study of 6294 medical students from 30 UK medical programmes who took the UKCAT from 2006 to 2008, for whom selection data from the UK Foundation Programme (UKFPO), the next stage of UK medical education training, were available in 2013. We included candidate demographics, UKCAT (cognitive domains; total scores), UKFPO Educational Performance Measure (EPM) and national exit situational judgement test (SJT). Multilevel modelling was used to assess relationships between variables, adjusting for confounders.

Results
The UKCAT—as a total score and in terms of the subtest scores—has significant predictive validity for performance on the UKFPO EPM and SJT. UKFPO performance was also affected positively by female gender, maturity, white ethnicity and coming from a higher social class area at the time of application to medical school An inverse pattern was seen for a contextual measure of school, with those attending fee-paying schools performing significantly more weakly on the EPM decile, the EPM total and the total UKFPO score, but not the SJT, than those attending other types of school.

Conclusions
This large-scale study, the first to link 2 national databases—UKCAT and UKFPO, has shown that UKCAT is a predictor of medical school outcome. The data provide modest supportive evidence for the UKCAT's role in student selection. The conflicting relationships of socioeconomic contextual measures (area and school) with outcome adds to wider debates about the limitations of these measures, and indicates the need for further research.
Original languageEnglish
Article numbere011313
Number of pages11
JournalBMJ Open
Volume6
Issue number10
Early online date7 Oct 2016
DOIs
Publication statusPublished - 2016

Fingerprint

Medical Schools
Cohort Studies
School Admission Criteria
Aptitude Tests
Fees and Charges
Medical Education
Medical Students
Social Class
Observational Studies
Demography
Research

Keywords

  • Education & Training
  • Medical Education & Training

Cite this

Does the UKCAT predict performance on exit from medical school? A national cohort study. / MacKenzie, R K; Cleland, J A; Ayansina, D; Nicholson, S.

In: BMJ Open, Vol. 6, No. 10, e011313, 2016.

Research output: Contribution to journalArticle

@article{9b5b6d6e75c34fa09d2e0ad0ec68e6c4,
title = "Does the UKCAT predict performance on exit from medical school? A national cohort study",
abstract = "ObjectivesMost UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UK Clinical Aptitude Test (UKCAT: http://www.ukcat.ac.uk), and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.MethodsThis was an observational study of 6294 medical students from 30 UK medical programmes who took the UKCAT from 2006 to 2008, for whom selection data from the UK Foundation Programme (UKFPO), the next stage of UK medical education training, were available in 2013. We included candidate demographics, UKCAT (cognitive domains; total scores), UKFPO Educational Performance Measure (EPM) and national exit situational judgement test (SJT). Multilevel modelling was used to assess relationships between variables, adjusting for confounders.ResultsThe UKCAT—as a total score and in terms of the subtest scores—has significant predictive validity for performance on the UKFPO EPM and SJT. UKFPO performance was also affected positively by female gender, maturity, white ethnicity and coming from a higher social class area at the time of application to medical school An inverse pattern was seen for a contextual measure of school, with those attending fee-paying schools performing significantly more weakly on the EPM decile, the EPM total and the total UKFPO score, but not the SJT, than those attending other types of school.ConclusionsThis large-scale study, the first to link 2 national databases—UKCAT and UKFPO, has shown that UKCAT is a predictor of medical school outcome. The data provide modest supportive evidence for the UKCAT's role in student selection. The conflicting relationships of socioeconomic contextual measures (area and school) with outcome adds to wider debates about the limitations of these measures, and indicates the need for further research.",
keywords = "Education & Training, Medical Education & Training",
author = "MacKenzie, {R K} and Cleland, {J A} and D Ayansina and S Nicholson",
note = "The authors thank the UKCAT Research Group for funding this independent evaluation and thank Rachel Greatrix of the UKCAT Consortium for her support throughout this project, and their feedback on the draft paper. The authors also thank the UKFPO for data provision and their support throughout. They also thank Professor Amanda Lee and Katie Wilde for their input into the application for funding, and ongoing support.",
year = "2016",
doi = "10.1136/bmjopen-2016-011313",
language = "English",
volume = "6",
journal = "BMJ Open",
issn = "2044-6055",
publisher = "BMJ Publishing Group",
number = "10",

}

TY - JOUR

T1 - Does the UKCAT predict performance on exit from medical school? A national cohort study

AU - MacKenzie, R K

AU - Cleland, J A

AU - Ayansina, D

AU - Nicholson, S

N1 - The authors thank the UKCAT Research Group for funding this independent evaluation and thank Rachel Greatrix of the UKCAT Consortium for her support throughout this project, and their feedback on the draft paper. The authors also thank the UKFPO for data provision and their support throughout. They also thank Professor Amanda Lee and Katie Wilde for their input into the application for funding, and ongoing support.

PY - 2016

Y1 - 2016

N2 - ObjectivesMost UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UK Clinical Aptitude Test (UKCAT: http://www.ukcat.ac.uk), and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.MethodsThis was an observational study of 6294 medical students from 30 UK medical programmes who took the UKCAT from 2006 to 2008, for whom selection data from the UK Foundation Programme (UKFPO), the next stage of UK medical education training, were available in 2013. We included candidate demographics, UKCAT (cognitive domains; total scores), UKFPO Educational Performance Measure (EPM) and national exit situational judgement test (SJT). Multilevel modelling was used to assess relationships between variables, adjusting for confounders.ResultsThe UKCAT—as a total score and in terms of the subtest scores—has significant predictive validity for performance on the UKFPO EPM and SJT. UKFPO performance was also affected positively by female gender, maturity, white ethnicity and coming from a higher social class area at the time of application to medical school An inverse pattern was seen for a contextual measure of school, with those attending fee-paying schools performing significantly more weakly on the EPM decile, the EPM total and the total UKFPO score, but not the SJT, than those attending other types of school.ConclusionsThis large-scale study, the first to link 2 national databases—UKCAT and UKFPO, has shown that UKCAT is a predictor of medical school outcome. The data provide modest supportive evidence for the UKCAT's role in student selection. The conflicting relationships of socioeconomic contextual measures (area and school) with outcome adds to wider debates about the limitations of these measures, and indicates the need for further research.

AB - ObjectivesMost UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UK Clinical Aptitude Test (UKCAT: http://www.ukcat.ac.uk), and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.MethodsThis was an observational study of 6294 medical students from 30 UK medical programmes who took the UKCAT from 2006 to 2008, for whom selection data from the UK Foundation Programme (UKFPO), the next stage of UK medical education training, were available in 2013. We included candidate demographics, UKCAT (cognitive domains; total scores), UKFPO Educational Performance Measure (EPM) and national exit situational judgement test (SJT). Multilevel modelling was used to assess relationships between variables, adjusting for confounders.ResultsThe UKCAT—as a total score and in terms of the subtest scores—has significant predictive validity for performance on the UKFPO EPM and SJT. UKFPO performance was also affected positively by female gender, maturity, white ethnicity and coming from a higher social class area at the time of application to medical school An inverse pattern was seen for a contextual measure of school, with those attending fee-paying schools performing significantly more weakly on the EPM decile, the EPM total and the total UKFPO score, but not the SJT, than those attending other types of school.ConclusionsThis large-scale study, the first to link 2 national databases—UKCAT and UKFPO, has shown that UKCAT is a predictor of medical school outcome. The data provide modest supportive evidence for the UKCAT's role in student selection. The conflicting relationships of socioeconomic contextual measures (area and school) with outcome adds to wider debates about the limitations of these measures, and indicates the need for further research.

KW - Education & Training

KW - Medical Education & Training

U2 - 10.1136/bmjopen-2016-011313

DO - 10.1136/bmjopen-2016-011313

M3 - Article

VL - 6

JO - BMJ Open

JF - BMJ Open

SN - 2044-6055

IS - 10

M1 - e011313

ER -