Practical help for specifying the target difference in sample size calculations for RCTs: The DELTA2 five-stage study, including a workshop

Jonathan A. Cook*, Steven A. Julious, William Sones, Lisa V. Hampson, Catherine Hewitt, Jesse A. Berlin, Deborah Ashby, Richard Emsley, Dean A. Fergusson, Stephen J. Walters, Edward C.F. Wilson, Graeme Maclennan, Nigel Stallard, Joanne C. Rothwell, Martin Bland, Louise Brown, Craig R. Ramsay, Andrew Cook, David Armstrong, Douglas AltmanLuke D. Vale

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Background: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

Original languageEnglish
Number of pages118
JournalHealth Technology Assessment
Volume23
Issue number60
DOIs
Publication statusPublished - 30 Oct 2019

Fingerprint

Sample Size
Education
Randomized Controlled Trials
Clinical Protocols
Research Personnel
Health
Population

ASJC Scopus subject areas

  • Health Policy

Cite this

Practical help for specifying the target difference in sample size calculations for RCTs : The DELTA2 five-stage study, including a workshop. / Cook, Jonathan A.; Julious, Steven A.; Sones, William; Hampson, Lisa V.; Hewitt, Catherine; Berlin, Jesse A.; Ashby, Deborah; Emsley, Richard; Fergusson, Dean A.; Walters, Stephen J.; Wilson, Edward C.F.; Maclennan, Graeme; Stallard, Nigel; Rothwell, Joanne C.; Bland, Martin; Brown, Louise; Ramsay, Craig R.; Cook, Andrew; Armstrong, David; Altman, Douglas; Vale, Luke D.

In: Health Technology Assessment, Vol. 23, No. 60, 30.10.2019.

Research output: Contribution to journalArticle

Cook, JA, Julious, SA, Sones, W, Hampson, LV, Hewitt, C, Berlin, JA, Ashby, D, Emsley, R, Fergusson, DA, Walters, SJ, Wilson, ECF, Maclennan, G, Stallard, N, Rothwell, JC, Bland, M, Brown, L, Ramsay, CR, Cook, A, Armstrong, D, Altman, D & Vale, LD 2019, 'Practical help for specifying the target difference in sample size calculations for RCTs: The DELTA2 five-stage study, including a workshop', Health Technology Assessment, vol. 23, no. 60. https://doi.org/10.3310/hta23600
Cook, Jonathan A. ; Julious, Steven A. ; Sones, William ; Hampson, Lisa V. ; Hewitt, Catherine ; Berlin, Jesse A. ; Ashby, Deborah ; Emsley, Richard ; Fergusson, Dean A. ; Walters, Stephen J. ; Wilson, Edward C.F. ; Maclennan, Graeme ; Stallard, Nigel ; Rothwell, Joanne C. ; Bland, Martin ; Brown, Louise ; Ramsay, Craig R. ; Cook, Andrew ; Armstrong, David ; Altman, Douglas ; Vale, Luke D. / Practical help for specifying the target difference in sample size calculations for RCTs : The DELTA2 five-stage study, including a workshop. In: Health Technology Assessment. 2019 ; Vol. 23, No. 60.
@article{6cdfe5a2b593414090aacd0d16702536,
title = "Practical help for specifying the target difference in sample size calculations for RCTs: The DELTA2 five-stage study, including a workshop",
abstract = "Background: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.",
author = "Cook, {Jonathan A.} and Julious, {Steven A.} and William Sones and Hampson, {Lisa V.} and Catherine Hewitt and Berlin, {Jesse A.} and Deborah Ashby and Richard Emsley and Fergusson, {Dean A.} and Walters, {Stephen J.} and Wilson, {Edward C.F.} and Graeme Maclennan and Nigel Stallard and Rothwell, {Joanne C.} and Martin Bland and Louise Brown and Ramsay, {Craig R.} and Andrew Cook and David Armstrong and Douglas Altman and Vale, {Luke D.}",
note = "Acknowledgements This project was funded by the MRC NIHR Methodology Research Programme in the UK in response to a commissioned call to lead a workshop on this topic in order to produce guidance. The members of the original DELTA2 group were Associate Professor Jonathan Cook, Professor Douglas Altman, Dr Jesse Berlin, Professor Martin Bland, Professor Richard Emsley, Dr Dean Fergusson, Dr Lisa Hampson, Professor Catherine Hewitt, Professor Craig Ramsay, Miss Joanne Rothwell, Dr Robert Smith, Dr William Sones, Professor Luke Vale, Professor Stephen Walters and Professor Steve Julious. As part of the process of developing this document, a 2-day workshop was held in Oxford in September 2016. The workshop participants were Professor Douglas Altman, Professor David Armstrong, Professor Deborah Ashby, Professor Martin Bland, Dr Andrew Cook, Professor Jonathan Cook, Dr David Crosby, Professor Richard Emsley, Dr Dean Fergusson, Professor Andrew Grieve, Dr Lisa Hampson, Professor Catherine Hewitt, Professor Steve Julious, Professor Graeme MacLennan, Professor Tim Maughan, Professor Jon Nicholl, Dr Jos{\'e} Pinheiro, Professor Craig Ramsay, Miss Joanne Rothwell, Dr William Sones, Professor Nigel Stallard, Professor Luke Vale, Professor Stephen Walters and Dr Ed Wilson. The authors would like to acknowledge and thank the participants in the Delphi exercise and the one-off engagement sessions with various groups, including the SCT, PSI and JSM conference session attendees, along with the other workshop participants who kindly provided helpful input and comments on the scope and content of this document. We would also like to thank, in particular, Dr Robert Smith in his role as a member of the public who provided a helpful public perspective during the workshop and in the development and revision of this document. Finally, the authors would like to thank Stefano Vezzoli for in-depth comments that helped to refine this document, helpful feedback from the MRC Methodological Research Programme Advisory Group, and representatives of the Medicines and Healthcare products Regulatory Agency and Health and Social Care, Northern Ireland. Dedication This work is dedicated to Douglas Altman, an inspirational researcher, friend and colleague. Contributions of authors Jonathan A Cook (Professor) drafted the initial version of the manuscript, and read and approved the final version. Steven A Julious (Professor) drafted the initial version of the manuscript and read and approved the final version. William Sones (Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Lisa V Hampson (Associate Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Catherine Hewitt (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Jesse A Berlin (Vice President and Global Head of Epidemiology) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Deborah Ashby (Co-Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Richard Emsley (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Dean A Fergusson (Senior Scientist and Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Stephen J Walters (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Edward CF Wilson (Senior Research Associate in Health Economics) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Graeme MacLennan (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Nigel Stallard (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Joanne C Rothwell (PhD student) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Martin Bland (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Louise Brown (Senior Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Craig R Ramsay (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Andrew Cook (Consultant in Public Health Medicine and Fellow in Health Technology Assessment) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. David Armstrong (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Douglas Altman (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Luke D Vale (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.",
year = "2019",
month = "10",
day = "30",
doi = "10.3310/hta23600",
language = "English",
volume = "23",
journal = "Health Technology Assessment",
issn = "1366-5278",
publisher = "National Co-ordinating Centre for HTA",
number = "60",

}

TY - JOUR

T1 - Practical help for specifying the target difference in sample size calculations for RCTs

T2 - The DELTA2 five-stage study, including a workshop

AU - Cook, Jonathan A.

AU - Julious, Steven A.

AU - Sones, William

AU - Hampson, Lisa V.

AU - Hewitt, Catherine

AU - Berlin, Jesse A.

AU - Ashby, Deborah

AU - Emsley, Richard

AU - Fergusson, Dean A.

AU - Walters, Stephen J.

AU - Wilson, Edward C.F.

AU - Maclennan, Graeme

AU - Stallard, Nigel

AU - Rothwell, Joanne C.

AU - Bland, Martin

AU - Brown, Louise

AU - Ramsay, Craig R.

AU - Cook, Andrew

AU - Armstrong, David

AU - Altman, Douglas

AU - Vale, Luke D.

N1 - Acknowledgements This project was funded by the MRC NIHR Methodology Research Programme in the UK in response to a commissioned call to lead a workshop on this topic in order to produce guidance. The members of the original DELTA2 group were Associate Professor Jonathan Cook, Professor Douglas Altman, Dr Jesse Berlin, Professor Martin Bland, Professor Richard Emsley, Dr Dean Fergusson, Dr Lisa Hampson, Professor Catherine Hewitt, Professor Craig Ramsay, Miss Joanne Rothwell, Dr Robert Smith, Dr William Sones, Professor Luke Vale, Professor Stephen Walters and Professor Steve Julious. As part of the process of developing this document, a 2-day workshop was held in Oxford in September 2016. The workshop participants were Professor Douglas Altman, Professor David Armstrong, Professor Deborah Ashby, Professor Martin Bland, Dr Andrew Cook, Professor Jonathan Cook, Dr David Crosby, Professor Richard Emsley, Dr Dean Fergusson, Professor Andrew Grieve, Dr Lisa Hampson, Professor Catherine Hewitt, Professor Steve Julious, Professor Graeme MacLennan, Professor Tim Maughan, Professor Jon Nicholl, Dr José Pinheiro, Professor Craig Ramsay, Miss Joanne Rothwell, Dr William Sones, Professor Nigel Stallard, Professor Luke Vale, Professor Stephen Walters and Dr Ed Wilson. The authors would like to acknowledge and thank the participants in the Delphi exercise and the one-off engagement sessions with various groups, including the SCT, PSI and JSM conference session attendees, along with the other workshop participants who kindly provided helpful input and comments on the scope and content of this document. We would also like to thank, in particular, Dr Robert Smith in his role as a member of the public who provided a helpful public perspective during the workshop and in the development and revision of this document. Finally, the authors would like to thank Stefano Vezzoli for in-depth comments that helped to refine this document, helpful feedback from the MRC Methodological Research Programme Advisory Group, and representatives of the Medicines and Healthcare products Regulatory Agency and Health and Social Care, Northern Ireland. Dedication This work is dedicated to Douglas Altman, an inspirational researcher, friend and colleague. Contributions of authors Jonathan A Cook (Professor) drafted the initial version of the manuscript, and read and approved the final version. Steven A Julious (Professor) drafted the initial version of the manuscript and read and approved the final version. William Sones (Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Lisa V Hampson (Associate Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Catherine Hewitt (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Jesse A Berlin (Vice President and Global Head of Epidemiology) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Deborah Ashby (Co-Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Richard Emsley (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Dean A Fergusson (Senior Scientist and Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Stephen J Walters (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Edward CF Wilson (Senior Research Associate in Health Economics) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Graeme MacLennan (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Nigel Stallard (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Joanne C Rothwell (PhD student) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Martin Bland (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Louise Brown (Senior Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Craig R Ramsay (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Andrew Cook (Consultant in Public Health Medicine and Fellow in Health Technology Assessment) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. David Armstrong (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Douglas Altman (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version. Luke D Vale (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

PY - 2019/10/30

Y1 - 2019/10/30

N2 - Background: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

AB - Background: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

UR - http://www.scopus.com/inward/record.url?scp=85074259328&partnerID=8YFLogxK

U2 - 10.3310/hta23600

DO - 10.3310/hta23600

M3 - Article

C2 - 31661431

AN - SCOPUS:85074259328

VL - 23

JO - Health Technology Assessment

JF - Health Technology Assessment

SN - 1366-5278

IS - 60

ER -