Predictive validity of the UK Clinical Aptitude Test in the final years of medical school

a prospective cohort study

Adrian Husbands, Alistair Mathieson, Jonathan Dowell, Jennifer Cleland, Rhoda MacKenzie

Research output: Contribution to journalArticle

14 Citations (Scopus)
4 Downloads (Pure)

Abstract

BACKGROUND: The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.

METHODS: The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.

RESULTS: Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.

CONCLUSIONS: Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

Original languageEnglish
Article number88
Number of pages6
JournalBMC Medical Education
Volume14
DOIs
Publication statusPublished - 23 Apr 2014

Fingerprint

aptitude test
school
examination
student
age group
regression analysis
performance
working life
gender
interview

Keywords

  • UKCAT
  • predictive validity
  • psychometric
  • assessment
  • selection
  • admissions
  • aptitude

Cite this

Predictive validity of the UK Clinical Aptitude Test in the final years of medical school : a prospective cohort study. / Husbands, Adrian; Mathieson, Alistair; Dowell, Jonathan; Cleland, Jennifer; MacKenzie, Rhoda.

In: BMC Medical Education, Vol. 14, 88, 23.04.2014.

Research output: Contribution to journalArticle

@article{4a85a134835c490e8ffbd47860f8a5cc,
title = "Predictive validity of the UK Clinical Aptitude Test in the final years of medical school: a prospective cohort study",
abstract = "BACKGROUND: The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.METHODS: The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.RESULTS: Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.CONCLUSIONS: Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.",
keywords = "UKCAT, predictive validity, psychometric, assessment , selection, admissions, aptitude",
author = "Adrian Husbands and Alistair Mathieson and Jonathan Dowell and Jennifer Cleland and Rhoda MacKenzie",
year = "2014",
month = "4",
day = "23",
doi = "10.1186/1472-6920-14-88",
language = "English",
volume = "14",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Predictive validity of the UK Clinical Aptitude Test in the final years of medical school

T2 - a prospective cohort study

AU - Husbands, Adrian

AU - Mathieson, Alistair

AU - Dowell, Jonathan

AU - Cleland, Jennifer

AU - MacKenzie, Rhoda

PY - 2014/4/23

Y1 - 2014/4/23

N2 - BACKGROUND: The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.METHODS: The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.RESULTS: Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.CONCLUSIONS: Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

AB - BACKGROUND: The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.METHODS: The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson's correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.RESULTS: Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.CONCLUSIONS: Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT's role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

KW - UKCAT

KW - predictive validity

KW - psychometric

KW - assessment

KW - selection

KW - admissions

KW - aptitude

U2 - 10.1186/1472-6920-14-88

DO - 10.1186/1472-6920-14-88

M3 - Article

VL - 14

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

M1 - 88

ER -