Assessing methods to specify the target difference for a randomised controlled trial

DELTA (Difference ELicitation in TriAls) review

Jonathan Alistair Cook, Jennifer Margaret Hislop, Temitope E Adewuyi, Kirsten Ann Harrild, Douglas G Altman, Craig R Ramsay, Cynthia Fraser, Brian Buckley, Peter Fayers, Ian Harvey, Andrew H Briggs, John D Norrie, Dean Fergusson, Ian Ford, Luke David Vale

Research output: Contribution to journalArticle

27 Citations (Scopus)
3 Downloads (Pure)

Abstract

BACKGROUND: The randomised controlled trial (RCT) is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to the design and validity of a RCT is a calculation of the number of participants needed (the sample size). The value used to determine the sample size can be considered the 'target difference'. From both a scientific and an ethical standpoint, selecting an appropriate target difference is of crucial importance. Determination of the target difference, as opposed to statistical approaches to calculating the sample size, has been greatly neglected though a variety of approaches have been proposed the current state of the evidence is unclear.

OBJECTIVES: The aim was to provide an overview of the current evidence regarding specifying the target difference in a RCT sample size calculation. The specific objectives were to conduct a systematic review of methods for specifying a target difference; to evaluate current practice by surveying triallists; to develop guidance on specifying the target difference in a RCT; and to identify future research needs.

DESIGN: The biomedical and social science databases searched were MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, Education Resources Information Center (ERIC) and Scopus for in-press publications. All were searched from 1966 or the earliest date of the database coverage and searches were undertaken between November 2010 and January 2011. There were three interlinked components: (1) systematic review of methods for specifying a target difference for RCTs - a comprehensive search strategy involving an electronic literature search of biomedical and some non-biomedical databases and clinical trials textbooks was carried out; (2) identification of current trial practice using two surveys of triallists - members of the Society for Clinical Trials (SCT) were invited to complete an online survey and respondents were asked about their awareness and use of, and willingness to recommend, methods; one individual per triallist group [UK Clinical Research Collaboration (UKCRC)-registered Clinical Trials Units (CTUs), Medical Research Council (MRC) UK Hubs for Trials Methodology Research and National Institute for Health Research (NIHR) UK Research Design Services (RDS)] was invited to complete a survey; (3) production of a structured guidance document to aid the design of future trials - the draft guidance was developed utilising the results of the systematic review and surveys by the project steering and advisory groups.

SETTING: Methodological review incorporating electronic searches, review of books and guidelines, two surveys of experts (membership of an international society and UK- and Ireland-based triallists) and development of guidance.

PARTICIPANTS: The two surveys were sent out to membership of the SCT and UK- and Ireland-based triallists.

INTERVENTIONS: The review focused on methods for specifying the target difference in a RCT. It was not restricted to any type of intervention or condition.

MAIN OUTCOME MEASURES: Methods for specifying the target difference for a RCT were considered.

RESULTS: The search identified 11,485 potentially relevant studies. In total, 1434 were selected for full-text assessment and 777 were included in the review. Seven methods to specify the target difference for a RCT were identified - anchor, distribution, health economic, opinion-seeking, pilot study, review of evidence base (RoEB) and standardised effect size (SES) - each having important variations in implementation. A total of 216 of the included studies used more than one method. A total of 180 (15%) responses to the SCT survey were received, representing 13 countries. Awareness of methods ranged from 38% (n =69) for the health economic method to 90% (n =162) for the pilot study. Of the 61 surveys sent out to UK triallist groups, 34 (56%) responses were received. Awareness ranged from 97% (n =33) for the RoEB and pilot study methods to only 41% (n =14) for the distribution method. Based on the most recent trial, all bar three groups (91%, n =30) used a formal method. Guidance was developed on the use of each method and the reporting of the sample size calculation in a trial protocol and results paper.

CONCLUSIONS: There is a clear need for greater use of formal methods to determine the target difference and better reporting of its specification. Raising the standard of RCT sample size calculations and the corresponding reporting of them would aid health professionals, patients, researchers and funders in judging the strength of the evidence and ensuring better use of scarce resources.

FUNDING: The Medical Research Council UK and the National Institute for Health Research Joint Methodology Research programme.

Original languageEnglish
Pages (from-to)v-vi, 1-175
JournalHealth Technology Assessment
Volume18
Issue number28
DOIs
Publication statusPublished - May 2014

Fingerprint

Randomized Controlled Trials
Sample Size
Clinical Trials
Health
National Institutes of Health (U.S.)
Databases
Ireland
MEDLINE
Biomedical Research
Research Design
Economics
Research
Surveys and Questionnaires
Information Centers
Textbooks
Social Sciences
Clinical Protocols
Publications
Joints
Research Personnel

Cite this

Assessing methods to specify the target difference for a randomised controlled trial : DELTA (Difference ELicitation in TriAls) review. / Cook, Jonathan Alistair; Hislop, Jennifer Margaret; Adewuyi, Temitope E; Harrild, Kirsten Ann; Altman, Douglas G; Ramsay, Craig R; Fraser, Cynthia; Buckley, Brian; Fayers, Peter; Harvey, Ian; Briggs, Andrew H; Norrie, John D; Fergusson, Dean; Ford, Ian; Vale, Luke David.

In: Health Technology Assessment, Vol. 18, No. 28, 05.2014, p. v-vi, 1-175.

Research output: Contribution to journalArticle

Cook, JA, Hislop, JM, Adewuyi, TE, Harrild, KA, Altman, DG, Ramsay, CR, Fraser, C, Buckley, B, Fayers, P, Harvey, I, Briggs, AH, Norrie, JD, Fergusson, D, Ford, I & Vale, LD 2014, 'Assessing methods to specify the target difference for a randomised controlled trial: DELTA (Difference ELicitation in TriAls) review', Health Technology Assessment, vol. 18, no. 28, pp. v-vi, 1-175. https://doi.org/10.3310/hta18280
Cook, Jonathan Alistair ; Hislop, Jennifer Margaret ; Adewuyi, Temitope E ; Harrild, Kirsten Ann ; Altman, Douglas G ; Ramsay, Craig R ; Fraser, Cynthia ; Buckley, Brian ; Fayers, Peter ; Harvey, Ian ; Briggs, Andrew H ; Norrie, John D ; Fergusson, Dean ; Ford, Ian ; Vale, Luke David. / Assessing methods to specify the target difference for a randomised controlled trial : DELTA (Difference ELicitation in TriAls) review. In: Health Technology Assessment. 2014 ; Vol. 18, No. 28. pp. v-vi, 1-175.
@article{fa2c9a868e2b42118732af54513ea94c,
title = "Assessing methods to specify the target difference for a randomised controlled trial: DELTA (Difference ELicitation in TriAls) review",
abstract = "BACKGROUND: The randomised controlled trial (RCT) is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to the design and validity of a RCT is a calculation of the number of participants needed (the sample size). The value used to determine the sample size can be considered the 'target difference'. From both a scientific and an ethical standpoint, selecting an appropriate target difference is of crucial importance. Determination of the target difference, as opposed to statistical approaches to calculating the sample size, has been greatly neglected though a variety of approaches have been proposed the current state of the evidence is unclear.OBJECTIVES: The aim was to provide an overview of the current evidence regarding specifying the target difference in a RCT sample size calculation. The specific objectives were to conduct a systematic review of methods for specifying a target difference; to evaluate current practice by surveying triallists; to develop guidance on specifying the target difference in a RCT; and to identify future research needs.DESIGN: The biomedical and social science databases searched were MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, Education Resources Information Center (ERIC) and Scopus for in-press publications. All were searched from 1966 or the earliest date of the database coverage and searches were undertaken between November 2010 and January 2011. There were three interlinked components: (1) systematic review of methods for specifying a target difference for RCTs - a comprehensive search strategy involving an electronic literature search of biomedical and some non-biomedical databases and clinical trials textbooks was carried out; (2) identification of current trial practice using two surveys of triallists - members of the Society for Clinical Trials (SCT) were invited to complete an online survey and respondents were asked about their awareness and use of, and willingness to recommend, methods; one individual per triallist group [UK Clinical Research Collaboration (UKCRC)-registered Clinical Trials Units (CTUs), Medical Research Council (MRC) UK Hubs for Trials Methodology Research and National Institute for Health Research (NIHR) UK Research Design Services (RDS)] was invited to complete a survey; (3) production of a structured guidance document to aid the design of future trials - the draft guidance was developed utilising the results of the systematic review and surveys by the project steering and advisory groups.SETTING: Methodological review incorporating electronic searches, review of books and guidelines, two surveys of experts (membership of an international society and UK- and Ireland-based triallists) and development of guidance.PARTICIPANTS: The two surveys were sent out to membership of the SCT and UK- and Ireland-based triallists.INTERVENTIONS: The review focused on methods for specifying the target difference in a RCT. It was not restricted to any type of intervention or condition.MAIN OUTCOME MEASURES: Methods for specifying the target difference for a RCT were considered.RESULTS: The search identified 11,485 potentially relevant studies. In total, 1434 were selected for full-text assessment and 777 were included in the review. Seven methods to specify the target difference for a RCT were identified - anchor, distribution, health economic, opinion-seeking, pilot study, review of evidence base (RoEB) and standardised effect size (SES) - each having important variations in implementation. A total of 216 of the included studies used more than one method. A total of 180 (15{\%}) responses to the SCT survey were received, representing 13 countries. Awareness of methods ranged from 38{\%} (n =69) for the health economic method to 90{\%} (n =162) for the pilot study. Of the 61 surveys sent out to UK triallist groups, 34 (56{\%}) responses were received. Awareness ranged from 97{\%} (n =33) for the RoEB and pilot study methods to only 41{\%} (n =14) for the distribution method. Based on the most recent trial, all bar three groups (91{\%}, n =30) used a formal method. Guidance was developed on the use of each method and the reporting of the sample size calculation in a trial protocol and results paper.CONCLUSIONS: There is a clear need for greater use of formal methods to determine the target difference and better reporting of its specification. Raising the standard of RCT sample size calculations and the corresponding reporting of them would aid health professionals, patients, researchers and funders in judging the strength of the evidence and ensuring better use of scarce resources.FUNDING: The Medical Research Council UK and the National Institute for Health Research Joint Methodology Research programme.",
author = "Cook, {Jonathan Alistair} and Hislop, {Jennifer Margaret} and Adewuyi, {Temitope E} and Harrild, {Kirsten Ann} and Altman, {Douglas G} and Ramsay, {Craig R} and Cynthia Fraser and Brian Buckley and Peter Fayers and Ian Harvey and Briggs, {Andrew H} and Norrie, {John D} and Dean Fergusson and Ian Ford and Vale, {Luke David}",
year = "2014",
month = "5",
doi = "10.3310/hta18280",
language = "English",
volume = "18",
pages = "v--vi, 1--175",
journal = "Health Technology Assessment",
issn = "1366-5278",
publisher = "National Co-ordinating Centre for HTA",
number = "28",

}

TY - JOUR

T1 - Assessing methods to specify the target difference for a randomised controlled trial

T2 - DELTA (Difference ELicitation in TriAls) review

AU - Cook, Jonathan Alistair

AU - Hislop, Jennifer Margaret

AU - Adewuyi, Temitope E

AU - Harrild, Kirsten Ann

AU - Altman, Douglas G

AU - Ramsay, Craig R

AU - Fraser, Cynthia

AU - Buckley, Brian

AU - Fayers, Peter

AU - Harvey, Ian

AU - Briggs, Andrew H

AU - Norrie, John D

AU - Fergusson, Dean

AU - Ford, Ian

AU - Vale, Luke David

PY - 2014/5

Y1 - 2014/5

N2 - BACKGROUND: The randomised controlled trial (RCT) is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to the design and validity of a RCT is a calculation of the number of participants needed (the sample size). The value used to determine the sample size can be considered the 'target difference'. From both a scientific and an ethical standpoint, selecting an appropriate target difference is of crucial importance. Determination of the target difference, as opposed to statistical approaches to calculating the sample size, has been greatly neglected though a variety of approaches have been proposed the current state of the evidence is unclear.OBJECTIVES: The aim was to provide an overview of the current evidence regarding specifying the target difference in a RCT sample size calculation. The specific objectives were to conduct a systematic review of methods for specifying a target difference; to evaluate current practice by surveying triallists; to develop guidance on specifying the target difference in a RCT; and to identify future research needs.DESIGN: The biomedical and social science databases searched were MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, Education Resources Information Center (ERIC) and Scopus for in-press publications. All were searched from 1966 or the earliest date of the database coverage and searches were undertaken between November 2010 and January 2011. There were three interlinked components: (1) systematic review of methods for specifying a target difference for RCTs - a comprehensive search strategy involving an electronic literature search of biomedical and some non-biomedical databases and clinical trials textbooks was carried out; (2) identification of current trial practice using two surveys of triallists - members of the Society for Clinical Trials (SCT) were invited to complete an online survey and respondents were asked about their awareness and use of, and willingness to recommend, methods; one individual per triallist group [UK Clinical Research Collaboration (UKCRC)-registered Clinical Trials Units (CTUs), Medical Research Council (MRC) UK Hubs for Trials Methodology Research and National Institute for Health Research (NIHR) UK Research Design Services (RDS)] was invited to complete a survey; (3) production of a structured guidance document to aid the design of future trials - the draft guidance was developed utilising the results of the systematic review and surveys by the project steering and advisory groups.SETTING: Methodological review incorporating electronic searches, review of books and guidelines, two surveys of experts (membership of an international society and UK- and Ireland-based triallists) and development of guidance.PARTICIPANTS: The two surveys were sent out to membership of the SCT and UK- and Ireland-based triallists.INTERVENTIONS: The review focused on methods for specifying the target difference in a RCT. It was not restricted to any type of intervention or condition.MAIN OUTCOME MEASURES: Methods for specifying the target difference for a RCT were considered.RESULTS: The search identified 11,485 potentially relevant studies. In total, 1434 were selected for full-text assessment and 777 were included in the review. Seven methods to specify the target difference for a RCT were identified - anchor, distribution, health economic, opinion-seeking, pilot study, review of evidence base (RoEB) and standardised effect size (SES) - each having important variations in implementation. A total of 216 of the included studies used more than one method. A total of 180 (15%) responses to the SCT survey were received, representing 13 countries. Awareness of methods ranged from 38% (n =69) for the health economic method to 90% (n =162) for the pilot study. Of the 61 surveys sent out to UK triallist groups, 34 (56%) responses were received. Awareness ranged from 97% (n =33) for the RoEB and pilot study methods to only 41% (n =14) for the distribution method. Based on the most recent trial, all bar three groups (91%, n =30) used a formal method. Guidance was developed on the use of each method and the reporting of the sample size calculation in a trial protocol and results paper.CONCLUSIONS: There is a clear need for greater use of formal methods to determine the target difference and better reporting of its specification. Raising the standard of RCT sample size calculations and the corresponding reporting of them would aid health professionals, patients, researchers and funders in judging the strength of the evidence and ensuring better use of scarce resources.FUNDING: The Medical Research Council UK and the National Institute for Health Research Joint Methodology Research programme.

AB - BACKGROUND: The randomised controlled trial (RCT) is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to the design and validity of a RCT is a calculation of the number of participants needed (the sample size). The value used to determine the sample size can be considered the 'target difference'. From both a scientific and an ethical standpoint, selecting an appropriate target difference is of crucial importance. Determination of the target difference, as opposed to statistical approaches to calculating the sample size, has been greatly neglected though a variety of approaches have been proposed the current state of the evidence is unclear.OBJECTIVES: The aim was to provide an overview of the current evidence regarding specifying the target difference in a RCT sample size calculation. The specific objectives were to conduct a systematic review of methods for specifying a target difference; to evaluate current practice by surveying triallists; to develop guidance on specifying the target difference in a RCT; and to identify future research needs.DESIGN: The biomedical and social science databases searched were MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, Education Resources Information Center (ERIC) and Scopus for in-press publications. All were searched from 1966 or the earliest date of the database coverage and searches were undertaken between November 2010 and January 2011. There were three interlinked components: (1) systematic review of methods for specifying a target difference for RCTs - a comprehensive search strategy involving an electronic literature search of biomedical and some non-biomedical databases and clinical trials textbooks was carried out; (2) identification of current trial practice using two surveys of triallists - members of the Society for Clinical Trials (SCT) were invited to complete an online survey and respondents were asked about their awareness and use of, and willingness to recommend, methods; one individual per triallist group [UK Clinical Research Collaboration (UKCRC)-registered Clinical Trials Units (CTUs), Medical Research Council (MRC) UK Hubs for Trials Methodology Research and National Institute for Health Research (NIHR) UK Research Design Services (RDS)] was invited to complete a survey; (3) production of a structured guidance document to aid the design of future trials - the draft guidance was developed utilising the results of the systematic review and surveys by the project steering and advisory groups.SETTING: Methodological review incorporating electronic searches, review of books and guidelines, two surveys of experts (membership of an international society and UK- and Ireland-based triallists) and development of guidance.PARTICIPANTS: The two surveys were sent out to membership of the SCT and UK- and Ireland-based triallists.INTERVENTIONS: The review focused on methods for specifying the target difference in a RCT. It was not restricted to any type of intervention or condition.MAIN OUTCOME MEASURES: Methods for specifying the target difference for a RCT were considered.RESULTS: The search identified 11,485 potentially relevant studies. In total, 1434 were selected for full-text assessment and 777 were included in the review. Seven methods to specify the target difference for a RCT were identified - anchor, distribution, health economic, opinion-seeking, pilot study, review of evidence base (RoEB) and standardised effect size (SES) - each having important variations in implementation. A total of 216 of the included studies used more than one method. A total of 180 (15%) responses to the SCT survey were received, representing 13 countries. Awareness of methods ranged from 38% (n =69) for the health economic method to 90% (n =162) for the pilot study. Of the 61 surveys sent out to UK triallist groups, 34 (56%) responses were received. Awareness ranged from 97% (n =33) for the RoEB and pilot study methods to only 41% (n =14) for the distribution method. Based on the most recent trial, all bar three groups (91%, n =30) used a formal method. Guidance was developed on the use of each method and the reporting of the sample size calculation in a trial protocol and results paper.CONCLUSIONS: There is a clear need for greater use of formal methods to determine the target difference and better reporting of its specification. Raising the standard of RCT sample size calculations and the corresponding reporting of them would aid health professionals, patients, researchers and funders in judging the strength of the evidence and ensuring better use of scarce resources.FUNDING: The Medical Research Council UK and the National Institute for Health Research Joint Methodology Research programme.

U2 - 10.3310/hta18280

DO - 10.3310/hta18280

M3 - Article

VL - 18

SP - v-vi, 1-175

JO - Health Technology Assessment

JF - Health Technology Assessment

SN - 1366-5278

IS - 28

ER -