Getting it wrong most of the time?: Comparing trialists’ choice of primary outcome with what patients and health professionals want

Shaun Treweek; Viviane  Miyakoda; Dylan  Burke; Frances Shiely

doi:10.1186/s13063-022-06348-z

Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want

Shaun Treweek^* (Corresponding Author), Viviane Miyakoda, Dylan Burke, Frances Shiely

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

10 Citations (Scopus)

5 Downloads (Pure)

Abstract

Background
Randomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials.

Methods
The work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary.

Results
In our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome.

Conclusions
The primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.

Original language	English
Article number	537
Number of pages	28
Journal	Trials
Volume	23
Issue number	1
Early online date	27 Jun 2022
DOIs	https://doi.org/10.1186/s13063-022-06348-z
Publication status	Published - 27 Jun 2022

Bibliographical note

Acknowledgements
We would like to acknowledge Prof. Joe Eustace, consultant nephrologist, for facilitating data collection in the renal outpatient clinic and for assisting in re-wording the outcomes for the patient document. Thank you also to Ellen Murphy, who assisted in data collection at two clinics. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates.

Funding
There was no direct funding received for this research. The HRB CRF-C at UCC facilitated the placement of an undergraduate pharmacy student, DB, who participated in the nephrology study.

Data Availability Statement

The datasets used and/or analysed during the current study are available at https://osf.io/xkad6/.

The online version contains supplementary material available at https://doi.org/10.1186/s13063-022-06348-z.

Keywords

outcomes
Trial methodology
Clinical Trials

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1186/s13063-022-06348-zLicence: CC BY

Treweek_etal_T_Getting_It_Wrong_VoR
© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Final published version, 1.13 MBLicence: CC BY

Data from: Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want
Treweek, S. (Creator), Miyakoda, V. (Creator), Burke, D. (Creator) & Shiely, F. (Creator), Figshare, 2022
DOI: 10.6084/m9.figshare.c.6067962, https://springernature.figshare.com/collections/Getting_it_wrong_most_of_the_time_Comparing_trialists_choice_of_primary_outcome_with_what_patients_and_health_professionals_want/6067962
Dataset
Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want
Treweek, S. (Creator), Miyakoda, V. (Creator), Burke, D. (Creator) & Shiely, F. (Creator), Figshare, 2022
DOI: 10.6084/m9.figshare.c.6067962.v1, https://springernature.figshare.com/collections/Getting_it_wrong_most_of_the_time_Comparing_trialists_choice_of_primary_outcome_with_what_patients_and_health_professionals_want/6067962/1
Dataset

Cite this

@article{616c4094c4db4ee8bbb0001765baa883,

title = "Getting it wrong most of the time?: Comparing trialists{\textquoteright} choice of primary outcome with what patients and health professionals want",

abstract = "BackgroundRandomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials.MethodsThe work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary.ResultsIn our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent{\textquoteright}s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome.ConclusionsThe primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.",

keywords = "outcomes, Trial methodology, Clinical Trials",

author = "Shaun Treweek and Viviane Miyakoda and Dylan Burke and Frances Shiely",

note = "Acknowledgements We would like to acknowledge Prof. Joe Eustace, consultant nephrologist, for facilitating data collection in the renal outpatient clinic and for assisting in re-wording the outcomes for the patient document. Thank you also to Ellen Murphy, who assisted in data collection at two clinics. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates. Funding There was no direct funding received for this research. The HRB CRF-C at UCC facilitated the placement of an undergraduate pharmacy student, DB, who participated in the nephrology study.",

year = "2022",

month = jun,

day = "27",

doi = "10.1186/s13063-022-06348-z",

language = "English",

volume = "23",

journal = "Trials",

issn = "1745-6215",

publisher = "BioMed Central",

number = "1",

}

TY - JOUR

T1 - Getting it wrong most of the time?

T2 - Comparing trialists’ choice of primary outcome with what patients and health professionals want

AU - Treweek, Shaun

AU - Miyakoda, Viviane

AU - Burke, Dylan

AU - Shiely, Frances

N1 - Acknowledgements We would like to acknowledge Prof. Joe Eustace, consultant nephrologist, for facilitating data collection in the renal outpatient clinic and for assisting in re-wording the outcomes for the patient document. Thank you also to Ellen Murphy, who assisted in data collection at two clinics. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates. Funding There was no direct funding received for this research. The HRB CRF-C at UCC facilitated the placement of an undergraduate pharmacy student, DB, who participated in the nephrology study.

PY - 2022/6/27

Y1 - 2022/6/27

N2 - BackgroundRandomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials.MethodsThe work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary.ResultsIn our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome.ConclusionsThe primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.

AB - BackgroundRandomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials.MethodsThe work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary.ResultsIn our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome.ConclusionsThe primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.

KW - outcomes

KW - Trial methodology

KW - Clinical Trials

U2 - 10.1186/s13063-022-06348-z

DO - 10.1186/s13063-022-06348-z

M3 - Article

C2 - 35761293

SN - 1745-6215

VL - 23

JO - Trials

JF - Trials

IS - 1

M1 - 537

ER -

Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want

Abstract

Bibliographical note

Data Availability Statement

Keywords

UN SDGs

Access to Document

Fingerprint

Datasets

Data from: Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want

Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want

Cite this