Estimating Site Performance (ESP)

can trial managers predict recruitment success at trial sites? An exploratory study

Hanne Bruhn (Corresponding Author), Shaun Treweek, Anne Duncan, Kirsty Shearer, Sarah Cameron, Karen Campbell, Karen Innes, Dawn McRae, Seonaidh C. Cotton

Research output: Contribution to journalArticle

6 Downloads (Pure)

Abstract

Background
Multicentre randomised trials provide some of the key evidence underpinning healthcare practice around the world. They are also hard work and generally expensive. Some of this work and expense are devoted to sites that fail to recruit as many participants as expected. Methods to identify sites that will recruit to target would be helpful.

Methods
We asked trial managers at the Centre for Healthcare Randomised Trials (CHaRT), University of Aberdeen to predict whether a site would recruit to target. Predictions were made after a site initiation visit and were collected on a form comprising a simple ‘Yes/No’ prediction and a reason for the prediction. We did not provide guidance as to what trial managers might want to think about when making predictions.

After a minimum of eight months of recruitment at each site for which a prediction had been made, all trial mangers in CHaRT were invited to a group discussion where predictions were presented together with sites’ actual recruitment performance over that period. Individual trial managers reflected on their predictions and there was a general discussion about predicting site recruitment. The prediction reasons from the forms and the content of the group discussion were used to identify features linked to correct predictions of recruitment failure.

Results
Ten trial managers made predictions for 56 site visits recruiting to eight trials. Trial managers’ sensitivity was 82% and their specificity was 32%, correctly identifying 65% of sites that would hit their recruitment target and 54% of those that did not. Eight ‘red flags’ for recruitment failure were identified: previous poor site performance; slow approvals process; strong staff/patient preferences; the site recruitment target; the trial protocol and its implementation at the site; lack of staff engagement; lack of research experience among site staff; and busy site staff. We used these red flags to develop a guided prediction form.

Conclusions
Trial managers’ unguided recruitment predictions were not bad but were not good enough for decision-making. We have developed a modified prediction form that includes eight flags to consider before making a prediction. We encourage anyone interested in contributing to its evaluation to contact us.
Original languageEnglish
Article number192
JournalTrials
Volume20
DOIs
Publication statusPublished - 3 Apr 2019

Fingerprint

Delivery of Health Care
Patient Preference
Clinical Protocols
Decision Making
Research

Keywords

  • recruitment
  • clinical trials
  • trial managers
  • recruitment sites
  • recruitment performance
  • Recruitment sites
  • Recruitment performance
  • Trial managers
  • Clinical trials
  • Recruitment
  • PREVALENCE

ASJC Scopus subject areas

  • Pharmacology (medical)
  • Medicine (miscellaneous)

Cite this

Estimating Site Performance (ESP) : can trial managers predict recruitment success at trial sites? An exploratory study. / Bruhn, Hanne (Corresponding Author); Treweek, Shaun; Duncan, Anne; Shearer, Kirsty; Cameron, Sarah; Campbell, Karen; Innes, Karen; McRae, Dawn; Cotton, Seonaidh C.

In: Trials, Vol. 20, 192, 03.04.2019.

Research output: Contribution to journalArticle

@article{4c1dc215f8f4485a94679665c1b0cf6d,
title = "Estimating Site Performance (ESP): can trial managers predict recruitment success at trial sites? An exploratory study",
abstract = "BackgroundMulticentre randomised trials provide some of the key evidence underpinning healthcare practice around the world. They are also hard work and generally expensive. Some of this work and expense are devoted to sites that fail to recruit as many participants as expected. Methods to identify sites that will recruit to target would be helpful.MethodsWe asked trial managers at the Centre for Healthcare Randomised Trials (CHaRT), University of Aberdeen to predict whether a site would recruit to target. Predictions were made after a site initiation visit and were collected on a form comprising a simple ‘Yes/No’ prediction and a reason for the prediction. We did not provide guidance as to what trial managers might want to think about when making predictions.After a minimum of eight months of recruitment at each site for which a prediction had been made, all trial mangers in CHaRT were invited to a group discussion where predictions were presented together with sites’ actual recruitment performance over that period. Individual trial managers reflected on their predictions and there was a general discussion about predicting site recruitment. The prediction reasons from the forms and the content of the group discussion were used to identify features linked to correct predictions of recruitment failure.ResultsTen trial managers made predictions for 56 site visits recruiting to eight trials. Trial managers’ sensitivity was 82{\%} and their specificity was 32{\%}, correctly identifying 65{\%} of sites that would hit their recruitment target and 54{\%} of those that did not. Eight ‘red flags’ for recruitment failure were identified: previous poor site performance; slow approvals process; strong staff/patient preferences; the site recruitment target; the trial protocol and its implementation at the site; lack of staff engagement; lack of research experience among site staff; and busy site staff. We used these red flags to develop a guided prediction form.ConclusionsTrial managers’ unguided recruitment predictions were not bad but were not good enough for decision-making. We have developed a modified prediction form that includes eight flags to consider before making a prediction. We encourage anyone interested in contributing to its evaluation to contact us.",
keywords = "recruitment, clinical trials, trial managers, recruitment sites, recruitment performance, Recruitment sites, Recruitment performance, Trial managers, Clinical trials, Recruitment, PREVALENCE",
author = "Hanne Bruhn and Shaun Treweek and Anne Duncan and Kirsty Shearer and Sarah Cameron and Karen Campbell and Karen Innes and Dawn McRae and Cotton, {Seonaidh C.}",
note = "Availability of data and materials All quantitative data generated and analysed during this study are included in this published article [and its supplementary information files] Additional file 3. The dataset of predictions used and analysed during the current study are available from the corresponding author on reasonable request. The transcript of the group discussion generated and analysed during the current study is not publicly available due it containing information that could compromise research participant consent (it would be a relatively simple matter to identify trials and trial managers) but are available from the corresponding author on reasonable request.",
year = "2019",
month = "4",
day = "3",
doi = "10.1186/s13063-019-3287-6",
language = "English",
volume = "20",
journal = "Trials",
issn = "1745-6215",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Estimating Site Performance (ESP)

T2 - can trial managers predict recruitment success at trial sites? An exploratory study

AU - Bruhn, Hanne

AU - Treweek, Shaun

AU - Duncan, Anne

AU - Shearer, Kirsty

AU - Cameron, Sarah

AU - Campbell, Karen

AU - Innes, Karen

AU - McRae, Dawn

AU - Cotton, Seonaidh C.

N1 - Availability of data and materials All quantitative data generated and analysed during this study are included in this published article [and its supplementary information files] Additional file 3. The dataset of predictions used and analysed during the current study are available from the corresponding author on reasonable request. The transcript of the group discussion generated and analysed during the current study is not publicly available due it containing information that could compromise research participant consent (it would be a relatively simple matter to identify trials and trial managers) but are available from the corresponding author on reasonable request.

PY - 2019/4/3

Y1 - 2019/4/3

N2 - BackgroundMulticentre randomised trials provide some of the key evidence underpinning healthcare practice around the world. They are also hard work and generally expensive. Some of this work and expense are devoted to sites that fail to recruit as many participants as expected. Methods to identify sites that will recruit to target would be helpful.MethodsWe asked trial managers at the Centre for Healthcare Randomised Trials (CHaRT), University of Aberdeen to predict whether a site would recruit to target. Predictions were made after a site initiation visit and were collected on a form comprising a simple ‘Yes/No’ prediction and a reason for the prediction. We did not provide guidance as to what trial managers might want to think about when making predictions.After a minimum of eight months of recruitment at each site for which a prediction had been made, all trial mangers in CHaRT were invited to a group discussion where predictions were presented together with sites’ actual recruitment performance over that period. Individual trial managers reflected on their predictions and there was a general discussion about predicting site recruitment. The prediction reasons from the forms and the content of the group discussion were used to identify features linked to correct predictions of recruitment failure.ResultsTen trial managers made predictions for 56 site visits recruiting to eight trials. Trial managers’ sensitivity was 82% and their specificity was 32%, correctly identifying 65% of sites that would hit their recruitment target and 54% of those that did not. Eight ‘red flags’ for recruitment failure were identified: previous poor site performance; slow approvals process; strong staff/patient preferences; the site recruitment target; the trial protocol and its implementation at the site; lack of staff engagement; lack of research experience among site staff; and busy site staff. We used these red flags to develop a guided prediction form.ConclusionsTrial managers’ unguided recruitment predictions were not bad but were not good enough for decision-making. We have developed a modified prediction form that includes eight flags to consider before making a prediction. We encourage anyone interested in contributing to its evaluation to contact us.

AB - BackgroundMulticentre randomised trials provide some of the key evidence underpinning healthcare practice around the world. They are also hard work and generally expensive. Some of this work and expense are devoted to sites that fail to recruit as many participants as expected. Methods to identify sites that will recruit to target would be helpful.MethodsWe asked trial managers at the Centre for Healthcare Randomised Trials (CHaRT), University of Aberdeen to predict whether a site would recruit to target. Predictions were made after a site initiation visit and were collected on a form comprising a simple ‘Yes/No’ prediction and a reason for the prediction. We did not provide guidance as to what trial managers might want to think about when making predictions.After a minimum of eight months of recruitment at each site for which a prediction had been made, all trial mangers in CHaRT were invited to a group discussion where predictions were presented together with sites’ actual recruitment performance over that period. Individual trial managers reflected on their predictions and there was a general discussion about predicting site recruitment. The prediction reasons from the forms and the content of the group discussion were used to identify features linked to correct predictions of recruitment failure.ResultsTen trial managers made predictions for 56 site visits recruiting to eight trials. Trial managers’ sensitivity was 82% and their specificity was 32%, correctly identifying 65% of sites that would hit their recruitment target and 54% of those that did not. Eight ‘red flags’ for recruitment failure were identified: previous poor site performance; slow approvals process; strong staff/patient preferences; the site recruitment target; the trial protocol and its implementation at the site; lack of staff engagement; lack of research experience among site staff; and busy site staff. We used these red flags to develop a guided prediction form.ConclusionsTrial managers’ unguided recruitment predictions were not bad but were not good enough for decision-making. We have developed a modified prediction form that includes eight flags to consider before making a prediction. We encourage anyone interested in contributing to its evaluation to contact us.

KW - recruitment

KW - clinical trials

KW - trial managers

KW - recruitment sites

KW - recruitment performance

KW - Recruitment sites

KW - Recruitment performance

KW - Trial managers

KW - Clinical trials

KW - Recruitment

KW - PREVALENCE

UR - http://www.scopus.com/inward/record.url?scp=85063959138&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/estimating-site-performance-esp-trial-managers-predict-recruitment-success-trial-sites-exploratory-s

U2 - 10.1186/s13063-019-3287-6

DO - 10.1186/s13063-019-3287-6

M3 - Article

VL - 20

JO - Trials

JF - Trials

SN - 1745-6215

M1 - 192

ER -