Use of methods for specifying the target difference in randomised controlled trial sample size calculations

Two surveys of trialists' practice

Jonathan A Cook, Jennifer M Hislop, Doug G Altman, Andrew H Briggs, Peter M Fayers, John D Norrie, Craig R Ramsay, Ian M Harvey, Luke D Vale, for the DELTA group

Research output: Contribution to journalArticle

5 Citations (Scopus)
3 Downloads (Pure)

Abstract

BACKGROUND: Central to the design of a randomised controlled trial (RCT) is a calculation of the number of participants needed. This is typically achieved by specifying a target difference, which enables the trial to identify a difference of a particular magnitude should one exist. Seven methods have been proposed for formally determining what the target difference should be. However, in practice, it may be driven by convenience or some other informal basis. It is unclear how aware the trialist community is of these formal methods or whether they are used.

PURPOSE: To determine current practice regarding the specification of the target difference by surveying trialists.

METHODS: Two surveys were conducted: (1) Members of the Society for Clinical Trials (SCT): participants were invited to complete an online survey through the society's email distribution list. Respondents were asked about their awareness, use of, and willingness to recommend methods; (2) Leading UK- and Ireland-based trialists: the survey was sent to UK Clinical Research Collaboration registered Clinical Trials Units, Medical Research Council UK Hubs for Trial Methodology Research, and the Research Design Services of the National Institute for Health Research. This survey also included questions about the most recent trial developed by the respondent's group.

RESULTS: Survey 1: Of the 1182 members on the SCT membership email distribution list, 180 responses were received (15%). Awareness of methods ranged from 69 (38%) for health economic methods to 162 (90%) for pilot study. Willingness to recommend among those who had used a particular method ranged from 56% for the opinion-seeking method to 89% for the review of evidence-base method. Survey 2: Of the 61 surveys sent out, 34 (56%) responses were received. Awareness of methods ranged from 33 (97%) for the review of evidence-base and pilot methods to 14 (41%) for the distribution method. The highest level of willingness to recommend among users was for the anchor method (87%). Based upon the most recent trial, the target difference was usually one viewed as important by a stakeholder group, mostly also viewed as a realistic difference given the interventions under evaluation, and sometimes one that led to an achievable sample size.

LIMITATIONS: The response rates achieved were relatively low despite the surveys being short, well presented, and having utilised reminders.

CONCLUSION: Substantial variations in practice exist with awareness, use, and willingness to recommend methods varying substantially. The findings support the view that sample size calculation is a more complex process than would appear to be the case from trial reports and protocols. Guidance on approaches for sample size estimation may increase both awareness and use of appropriate formal methods.

Original languageEnglish
Pages (from-to)300-308
Number of pages9
JournalClinical Trials
Volume11
Issue number3
Early online date6 Mar 2014
DOIs
Publication statusPublished - Jun 2014

Fingerprint

Sample Size
Randomized Controlled Trials
Clinical Trials
Surveys and Questionnaires
Research
National Institutes of Health (U.S.)
Clinical Protocols
Ireland
Biomedical Research
Research Design
Economics

Cite this

Use of methods for specifying the target difference in randomised controlled trial sample size calculations : Two surveys of trialists' practice. / Cook, Jonathan A; Hislop, Jennifer M; Altman, Doug G; Briggs, Andrew H; Fayers, Peter M; Norrie, John D; Ramsay, Craig R; Harvey, Ian M; Vale, Luke D; for the DELTA group.

In: Clinical Trials, Vol. 11, No. 3, 06.2014, p. 300-308.

Research output: Contribution to journalArticle

Cook, Jonathan A ; Hislop, Jennifer M ; Altman, Doug G ; Briggs, Andrew H ; Fayers, Peter M ; Norrie, John D ; Ramsay, Craig R ; Harvey, Ian M ; Vale, Luke D ; for the DELTA group. / Use of methods for specifying the target difference in randomised controlled trial sample size calculations : Two surveys of trialists' practice. In: Clinical Trials. 2014 ; Vol. 11, No. 3. pp. 300-308.
@article{6ae0e954f2fc40dfb2ad59ba9d6c3277,
title = "Use of methods for specifying the target difference in randomised controlled trial sample size calculations: Two surveys of trialists' practice",
abstract = "BACKGROUND: Central to the design of a randomised controlled trial (RCT) is a calculation of the number of participants needed. This is typically achieved by specifying a target difference, which enables the trial to identify a difference of a particular magnitude should one exist. Seven methods have been proposed for formally determining what the target difference should be. However, in practice, it may be driven by convenience or some other informal basis. It is unclear how aware the trialist community is of these formal methods or whether they are used.PURPOSE: To determine current practice regarding the specification of the target difference by surveying trialists.METHODS: Two surveys were conducted: (1) Members of the Society for Clinical Trials (SCT): participants were invited to complete an online survey through the society's email distribution list. Respondents were asked about their awareness, use of, and willingness to recommend methods; (2) Leading UK- and Ireland-based trialists: the survey was sent to UK Clinical Research Collaboration registered Clinical Trials Units, Medical Research Council UK Hubs for Trial Methodology Research, and the Research Design Services of the National Institute for Health Research. This survey also included questions about the most recent trial developed by the respondent's group.RESULTS: Survey 1: Of the 1182 members on the SCT membership email distribution list, 180 responses were received (15{\%}). Awareness of methods ranged from 69 (38{\%}) for health economic methods to 162 (90{\%}) for pilot study. Willingness to recommend among those who had used a particular method ranged from 56{\%} for the opinion-seeking method to 89{\%} for the review of evidence-base method. Survey 2: Of the 61 surveys sent out, 34 (56{\%}) responses were received. Awareness of methods ranged from 33 (97{\%}) for the review of evidence-base and pilot methods to 14 (41{\%}) for the distribution method. The highest level of willingness to recommend among users was for the anchor method (87{\%}). Based upon the most recent trial, the target difference was usually one viewed as important by a stakeholder group, mostly also viewed as a realistic difference given the interventions under evaluation, and sometimes one that led to an achievable sample size.LIMITATIONS: The response rates achieved were relatively low despite the surveys being short, well presented, and having utilised reminders.CONCLUSION: Substantial variations in practice exist with awareness, use, and willingness to recommend methods varying substantially. The findings support the view that sample size calculation is a more complex process than would appear to be the case from trial reports and protocols. Guidance on approaches for sample size estimation may increase both awareness and use of appropriate formal methods.",
author = "Cook, {Jonathan A} and Hislop, {Jennifer M} and Altman, {Doug G} and Briggs, {Andrew H} and Fayers, {Peter M} and Norrie, {John D} and Ramsay, {Craig R} and Harvey, {Ian M} and Vale, {Luke D} and {for the DELTA group}",
note = "{\circledC} The Author(s), 2014.",
year = "2014",
month = "6",
doi = "10.1177/1740774514521907",
language = "English",
volume = "11",
pages = "300--308",
journal = "Clinical Trials",
issn = "1740-7745",
publisher = "Sage Publications",
number = "3",

}

TY - JOUR

T1 - Use of methods for specifying the target difference in randomised controlled trial sample size calculations

T2 - Two surveys of trialists' practice

AU - Cook, Jonathan A

AU - Hislop, Jennifer M

AU - Altman, Doug G

AU - Briggs, Andrew H

AU - Fayers, Peter M

AU - Norrie, John D

AU - Ramsay, Craig R

AU - Harvey, Ian M

AU - Vale, Luke D

AU - for the DELTA group

N1 - © The Author(s), 2014.

PY - 2014/6

Y1 - 2014/6

N2 - BACKGROUND: Central to the design of a randomised controlled trial (RCT) is a calculation of the number of participants needed. This is typically achieved by specifying a target difference, which enables the trial to identify a difference of a particular magnitude should one exist. Seven methods have been proposed for formally determining what the target difference should be. However, in practice, it may be driven by convenience or some other informal basis. It is unclear how aware the trialist community is of these formal methods or whether they are used.PURPOSE: To determine current practice regarding the specification of the target difference by surveying trialists.METHODS: Two surveys were conducted: (1) Members of the Society for Clinical Trials (SCT): participants were invited to complete an online survey through the society's email distribution list. Respondents were asked about their awareness, use of, and willingness to recommend methods; (2) Leading UK- and Ireland-based trialists: the survey was sent to UK Clinical Research Collaboration registered Clinical Trials Units, Medical Research Council UK Hubs for Trial Methodology Research, and the Research Design Services of the National Institute for Health Research. This survey also included questions about the most recent trial developed by the respondent's group.RESULTS: Survey 1: Of the 1182 members on the SCT membership email distribution list, 180 responses were received (15%). Awareness of methods ranged from 69 (38%) for health economic methods to 162 (90%) for pilot study. Willingness to recommend among those who had used a particular method ranged from 56% for the opinion-seeking method to 89% for the review of evidence-base method. Survey 2: Of the 61 surveys sent out, 34 (56%) responses were received. Awareness of methods ranged from 33 (97%) for the review of evidence-base and pilot methods to 14 (41%) for the distribution method. The highest level of willingness to recommend among users was for the anchor method (87%). Based upon the most recent trial, the target difference was usually one viewed as important by a stakeholder group, mostly also viewed as a realistic difference given the interventions under evaluation, and sometimes one that led to an achievable sample size.LIMITATIONS: The response rates achieved were relatively low despite the surveys being short, well presented, and having utilised reminders.CONCLUSION: Substantial variations in practice exist with awareness, use, and willingness to recommend methods varying substantially. The findings support the view that sample size calculation is a more complex process than would appear to be the case from trial reports and protocols. Guidance on approaches for sample size estimation may increase both awareness and use of appropriate formal methods.

AB - BACKGROUND: Central to the design of a randomised controlled trial (RCT) is a calculation of the number of participants needed. This is typically achieved by specifying a target difference, which enables the trial to identify a difference of a particular magnitude should one exist. Seven methods have been proposed for formally determining what the target difference should be. However, in practice, it may be driven by convenience or some other informal basis. It is unclear how aware the trialist community is of these formal methods or whether they are used.PURPOSE: To determine current practice regarding the specification of the target difference by surveying trialists.METHODS: Two surveys were conducted: (1) Members of the Society for Clinical Trials (SCT): participants were invited to complete an online survey through the society's email distribution list. Respondents were asked about their awareness, use of, and willingness to recommend methods; (2) Leading UK- and Ireland-based trialists: the survey was sent to UK Clinical Research Collaboration registered Clinical Trials Units, Medical Research Council UK Hubs for Trial Methodology Research, and the Research Design Services of the National Institute for Health Research. This survey also included questions about the most recent trial developed by the respondent's group.RESULTS: Survey 1: Of the 1182 members on the SCT membership email distribution list, 180 responses were received (15%). Awareness of methods ranged from 69 (38%) for health economic methods to 162 (90%) for pilot study. Willingness to recommend among those who had used a particular method ranged from 56% for the opinion-seeking method to 89% for the review of evidence-base method. Survey 2: Of the 61 surveys sent out, 34 (56%) responses were received. Awareness of methods ranged from 33 (97%) for the review of evidence-base and pilot methods to 14 (41%) for the distribution method. The highest level of willingness to recommend among users was for the anchor method (87%). Based upon the most recent trial, the target difference was usually one viewed as important by a stakeholder group, mostly also viewed as a realistic difference given the interventions under evaluation, and sometimes one that led to an achievable sample size.LIMITATIONS: The response rates achieved were relatively low despite the surveys being short, well presented, and having utilised reminders.CONCLUSION: Substantial variations in practice exist with awareness, use, and willingness to recommend methods varying substantially. The findings support the view that sample size calculation is a more complex process than would appear to be the case from trial reports and protocols. Guidance on approaches for sample size estimation may increase both awareness and use of appropriate formal methods.

U2 - 10.1177/1740774514521907

DO - 10.1177/1740774514521907

M3 - Article

VL - 11

SP - 300

EP - 308

JO - Clinical Trials

JF - Clinical Trials

SN - 1740-7745

IS - 3

ER -