Incorporating patient partner scores into high stakes assessment: an observational study into opinions and attitudes

Fiona C Thomson, Rhoda K MacKenzie, Marie Anderson, Alan R Denison, Graeme P Currie

Research output: Contribution to journalArticle

1 Citation (Scopus)
11 Downloads (Pure)

Abstract

BACKGROUND: Volunteer patients (also known as patient partners (PPs)) play a vital role in undergraduate healthcare curricula. They frequently take part in objective structured clinical examinations (OSCE) and rate aspects of students' performance. However, the inclusion and weighting of PP marks varies, while attitudes and opinions regarding how (and if) they should contribute towards the pass/fail outcome are uncertain.

METHODS: A prospective observational study was conducted to explore beliefs of PPs regarding inclusion of their scores in a high stakes undergraduate OSCE in a single UK medical school. All PPs delivering components of the local MBChB curriculum were asked to participate in the questionnaire study. Quantitative and qualitative data were analysed using descriptive statistics and framework analysis respectively.

RESULTS: Fifty out of 160 (31% response rate) PPs completed the questionnaire; 70% had participated in a final year OSCE. Thirty (60%) felt their marks should be incorporated into a student's overall score, while 28% were uncertain. The main reasons for inclusion were recognition of the patient perspective (31%) and their ability to assess attitudes and professionalism (27%), while reasons against inclusion included lack of PP qualification/training (18%) and concerns relating to consistency (14%). The majority of PPs were uncertain what proportion of the total mark they should contribute, although many felt that 5-10% of the total score was reasonable. Most respondents (70%) felt that globally low PP scores should not result in an automatic fail and many (62%) acknowledged that prior to mark inclusion, further training was required.

CONCLUSION: These data show that most respondents considered it reasonable to "formalise their expertise" by contributing marks in the overall assessment of students in a high stakes OSCE, although what proportion they believe this should represent was variable. Some expressed concerns that using marks towards progress decisions may alter PP response patterns. It would therefore seem reasonable to compare outcomes (i.e. pass/fail status) using historical data both incorporating and not incorporating PP marks to evaluate the effects of doing so. Further attention to existing PP training programmes is also required in order to provide clear instruction on how to globally rate students to ensure validity and consistency.

Original languageEnglish
Article number214
Pages (from-to)1-7
Number of pages7
JournalBMC Medical Education
Volume17
Issue number1
DOIs
Publication statusPublished - 15 Nov 2017

Fingerprint

inclusion
examination
student
curriculum
questionnaire
weighting
descriptive statistics
qualification
training program
expertise
instruction
lack
ability
school
performance

Keywords

  • Journal Article
  • “Patient partner”
  • “Volunteer or simulated patient”
  • “Assessment”

Cite this

Incorporating patient partner scores into high stakes assessment : an observational study into opinions and attitudes. / Thomson, Fiona C; MacKenzie, Rhoda K; Anderson, Marie; Denison, Alan R; Currie, Graeme P.

In: BMC Medical Education, Vol. 17, No. 1, 214, 15.11.2017, p. 1-7.

Research output: Contribution to journalArticle

Thomson, Fiona C ; MacKenzie, Rhoda K ; Anderson, Marie ; Denison, Alan R ; Currie, Graeme P. / Incorporating patient partner scores into high stakes assessment : an observational study into opinions and attitudes. In: BMC Medical Education. 2017 ; Vol. 17, No. 1. pp. 1-7.
@article{b29365f0de4f46fda55e89499c0fc983,
title = "Incorporating patient partner scores into high stakes assessment: an observational study into opinions and attitudes",
abstract = "BACKGROUND: Volunteer patients (also known as patient partners (PPs)) play a vital role in undergraduate healthcare curricula. They frequently take part in objective structured clinical examinations (OSCE) and rate aspects of students' performance. However, the inclusion and weighting of PP marks varies, while attitudes and opinions regarding how (and if) they should contribute towards the pass/fail outcome are uncertain.METHODS: A prospective observational study was conducted to explore beliefs of PPs regarding inclusion of their scores in a high stakes undergraduate OSCE in a single UK medical school. All PPs delivering components of the local MBChB curriculum were asked to participate in the questionnaire study. Quantitative and qualitative data were analysed using descriptive statistics and framework analysis respectively.RESULTS: Fifty out of 160 (31{\%} response rate) PPs completed the questionnaire; 70{\%} had participated in a final year OSCE. Thirty (60{\%}) felt their marks should be incorporated into a student's overall score, while 28{\%} were uncertain. The main reasons for inclusion were recognition of the patient perspective (31{\%}) and their ability to assess attitudes and professionalism (27{\%}), while reasons against inclusion included lack of PP qualification/training (18{\%}) and concerns relating to consistency (14{\%}). The majority of PPs were uncertain what proportion of the total mark they should contribute, although many felt that 5-10{\%} of the total score was reasonable. Most respondents (70{\%}) felt that globally low PP scores should not result in an automatic fail and many (62{\%}) acknowledged that prior to mark inclusion, further training was required.CONCLUSION: These data show that most respondents considered it reasonable to {"}formalise their expertise{"} by contributing marks in the overall assessment of students in a high stakes OSCE, although what proportion they believe this should represent was variable. Some expressed concerns that using marks towards progress decisions may alter PP response patterns. It would therefore seem reasonable to compare outcomes (i.e. pass/fail status) using historical data both incorporating and not incorporating PP marks to evaluate the effects of doing so. Further attention to existing PP training programmes is also required in order to provide clear instruction on how to globally rate students to ensure validity and consistency.",
keywords = "Journal Article, “Patient partner”, “Volunteer or simulated patient”, “Assessment”",
author = "Thomson, {Fiona C} and MacKenzie, {Rhoda K} and Marie Anderson and Denison, {Alan R} and Currie, {Graeme P}",
year = "2017",
month = "11",
day = "15",
doi = "10.1186/s12909-017-1063-4",
language = "English",
volume = "17",
pages = "1--7",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Incorporating patient partner scores into high stakes assessment

T2 - an observational study into opinions and attitudes

AU - Thomson, Fiona C

AU - MacKenzie, Rhoda K

AU - Anderson, Marie

AU - Denison, Alan R

AU - Currie, Graeme P

PY - 2017/11/15

Y1 - 2017/11/15

N2 - BACKGROUND: Volunteer patients (also known as patient partners (PPs)) play a vital role in undergraduate healthcare curricula. They frequently take part in objective structured clinical examinations (OSCE) and rate aspects of students' performance. However, the inclusion and weighting of PP marks varies, while attitudes and opinions regarding how (and if) they should contribute towards the pass/fail outcome are uncertain.METHODS: A prospective observational study was conducted to explore beliefs of PPs regarding inclusion of their scores in a high stakes undergraduate OSCE in a single UK medical school. All PPs delivering components of the local MBChB curriculum were asked to participate in the questionnaire study. Quantitative and qualitative data were analysed using descriptive statistics and framework analysis respectively.RESULTS: Fifty out of 160 (31% response rate) PPs completed the questionnaire; 70% had participated in a final year OSCE. Thirty (60%) felt their marks should be incorporated into a student's overall score, while 28% were uncertain. The main reasons for inclusion were recognition of the patient perspective (31%) and their ability to assess attitudes and professionalism (27%), while reasons against inclusion included lack of PP qualification/training (18%) and concerns relating to consistency (14%). The majority of PPs were uncertain what proportion of the total mark they should contribute, although many felt that 5-10% of the total score was reasonable. Most respondents (70%) felt that globally low PP scores should not result in an automatic fail and many (62%) acknowledged that prior to mark inclusion, further training was required.CONCLUSION: These data show that most respondents considered it reasonable to "formalise their expertise" by contributing marks in the overall assessment of students in a high stakes OSCE, although what proportion they believe this should represent was variable. Some expressed concerns that using marks towards progress decisions may alter PP response patterns. It would therefore seem reasonable to compare outcomes (i.e. pass/fail status) using historical data both incorporating and not incorporating PP marks to evaluate the effects of doing so. Further attention to existing PP training programmes is also required in order to provide clear instruction on how to globally rate students to ensure validity and consistency.

AB - BACKGROUND: Volunteer patients (also known as patient partners (PPs)) play a vital role in undergraduate healthcare curricula. They frequently take part in objective structured clinical examinations (OSCE) and rate aspects of students' performance. However, the inclusion and weighting of PP marks varies, while attitudes and opinions regarding how (and if) they should contribute towards the pass/fail outcome are uncertain.METHODS: A prospective observational study was conducted to explore beliefs of PPs regarding inclusion of their scores in a high stakes undergraduate OSCE in a single UK medical school. All PPs delivering components of the local MBChB curriculum were asked to participate in the questionnaire study. Quantitative and qualitative data were analysed using descriptive statistics and framework analysis respectively.RESULTS: Fifty out of 160 (31% response rate) PPs completed the questionnaire; 70% had participated in a final year OSCE. Thirty (60%) felt their marks should be incorporated into a student's overall score, while 28% were uncertain. The main reasons for inclusion were recognition of the patient perspective (31%) and their ability to assess attitudes and professionalism (27%), while reasons against inclusion included lack of PP qualification/training (18%) and concerns relating to consistency (14%). The majority of PPs were uncertain what proportion of the total mark they should contribute, although many felt that 5-10% of the total score was reasonable. Most respondents (70%) felt that globally low PP scores should not result in an automatic fail and many (62%) acknowledged that prior to mark inclusion, further training was required.CONCLUSION: These data show that most respondents considered it reasonable to "formalise their expertise" by contributing marks in the overall assessment of students in a high stakes OSCE, although what proportion they believe this should represent was variable. Some expressed concerns that using marks towards progress decisions may alter PP response patterns. It would therefore seem reasonable to compare outcomes (i.e. pass/fail status) using historical data both incorporating and not incorporating PP marks to evaluate the effects of doing so. Further attention to existing PP training programmes is also required in order to provide clear instruction on how to globally rate students to ensure validity and consistency.

KW - Journal Article

KW - “Patient partner”

KW - “Volunteer or simulated patient”

KW - “Assessment”

U2 - 10.1186/s12909-017-1063-4

DO - 10.1186/s12909-017-1063-4

M3 - Article

C2 - 29141624

VL - 17

SP - 1

EP - 7

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

IS - 1

M1 - 214

ER -