Partial factorial trials: comparing methods for statistical analysis and economic evaluation

Helen A Dakin, Alastair M Gray, Graeme S MacLennan, Richard W Morris, David W Murray

Research output: Contribution to journalArticle

5 Downloads (Pure)

Abstract

BACKGROUND: Partial factorial trials compare two or more pairs of treatments on overlapping patient groups, randomising some (but not all) patients to more than one comparison. The aims of this research were to compare different methods for conducting and analysing economic evaluations on partial factorial trials and assess the implications of considering factors simultaneously rather than drawing independent conclusions about each comparison.

METHODS: We estimated total costs and quality-adjusted life years (QALYs) within 10 years of surgery for 2252 patients in the Knee Arthroplasty Trial who were randomised to one or more comparisons of different surgical types. We compared three analytical methods: an "at-the-margins" analysis including all patients randomised to each comparison (assuming no interaction); an "inside-the-table" analysis that included interactions but focused on those patients randomised to two comparisons; and a Bayesian vetted bootstrap, which used results from patients randomised to one comparison as priors when estimating outcomes for patients randomised to two comparisons. Outcomes comprised incremental costs, QALYs and net benefits.

RESULTS: Qualitative interactions were observed for costs, QALYs and net benefits. Bayesian bootstrapping generally produced smaller standard errors than inside-the-table analysis and gave conclusions that were consistent with at-the-margins analysis, while allowing for these interactions. By contrast, inside-the-table gave different conclusions about which intervention had the highest net benefits compared with other analyses.

CONCLUSIONS: All analyses of partial factorial trials should explore interactions and assess whether results are sensitive to assumptions about interactions, either as a primary analysis or as a sensitivity analysis. For partial factorial trials closely mirroring routine clinical practice, at-the-margins analysis may provide a reasonable estimate of average costs and benefits for the whole trial population, even in the presence of interactions. However, such conclusions will be misleading if there are large interactions or if the proportion of patients allocated to different treatments differs markedly from what occurs in clinical practice. The Bayesian bootstrap provides an alternative to at-the-margins analysis for analysing clinical or economic endpoints from partial factorial trials, which allows for interactions while making use of the whole sample. The same techniques could be applied to analyses of clinical endpoints.

TRIAL REGISTRATION: ISRCTN, ISRCTN45837371 . Registered on 25 April 2003.

Original languageEnglish
Article number442
JournalTrials
Volume19
DOIs
Publication statusPublished - 16 Aug 2018

Fingerprint

Cost-Benefit Analysis
Quality-Adjusted Life Years
Costs and Cost Analysis
Knee Replacement Arthroplasties
Economics
Therapeutics
Research
Population

Keywords

  • Journal Article
  • randomised controlled trial
  • factorial design
  • cost-utility analysis
  • Bayesian bootstrap
  • partial factorial trial

Cite this

Partial factorial trials : comparing methods for statistical analysis and economic evaluation. / Dakin, Helen A; Gray, Alastair M; MacLennan, Graeme S; Morris, Richard W; Murray, David W.

In: Trials, Vol. 19, 442, 16.08.2018.

Research output: Contribution to journalArticle

Dakin, Helen A ; Gray, Alastair M ; MacLennan, Graeme S ; Morris, Richard W ; Murray, David W. / Partial factorial trials : comparing methods for statistical analysis and economic evaluation. In: Trials. 2018 ; Vol. 19.
@article{1c5c74d8cd824fefa26fd47c2f825971,
title = "Partial factorial trials: comparing methods for statistical analysis and economic evaluation",
abstract = "BACKGROUND: Partial factorial trials compare two or more pairs of treatments on overlapping patient groups, randomising some (but not all) patients to more than one comparison. The aims of this research were to compare different methods for conducting and analysing economic evaluations on partial factorial trials and assess the implications of considering factors simultaneously rather than drawing independent conclusions about each comparison.METHODS: We estimated total costs and quality-adjusted life years (QALYs) within 10 years of surgery for 2252 patients in the Knee Arthroplasty Trial who were randomised to one or more comparisons of different surgical types. We compared three analytical methods: an {"}at-the-margins{"} analysis including all patients randomised to each comparison (assuming no interaction); an {"}inside-the-table{"} analysis that included interactions but focused on those patients randomised to two comparisons; and a Bayesian vetted bootstrap, which used results from patients randomised to one comparison as priors when estimating outcomes for patients randomised to two comparisons. Outcomes comprised incremental costs, QALYs and net benefits.RESULTS: Qualitative interactions were observed for costs, QALYs and net benefits. Bayesian bootstrapping generally produced smaller standard errors than inside-the-table analysis and gave conclusions that were consistent with at-the-margins analysis, while allowing for these interactions. By contrast, inside-the-table gave different conclusions about which intervention had the highest net benefits compared with other analyses.CONCLUSIONS: All analyses of partial factorial trials should explore interactions and assess whether results are sensitive to assumptions about interactions, either as a primary analysis or as a sensitivity analysis. For partial factorial trials closely mirroring routine clinical practice, at-the-margins analysis may provide a reasonable estimate of average costs and benefits for the whole trial population, even in the presence of interactions. However, such conclusions will be misleading if there are large interactions or if the proportion of patients allocated to different treatments differs markedly from what occurs in clinical practice. The Bayesian bootstrap provides an alternative to at-the-margins analysis for analysing clinical or economic endpoints from partial factorial trials, which allows for interactions while making use of the whole sample. The same techniques could be applied to analyses of clinical endpoints.TRIAL REGISTRATION: ISRCTN, ISRCTN45837371 . Registered on 25 April 2003.",
keywords = "Journal Article, randomised controlled trial , factorial design, cost-utility analysis, Bayesian bootstrap, partial factorial trial",
author = "Dakin, {Helen A} and Gray, {Alastair M} and MacLennan, {Graeme S} and Morris, {Richard W} and Murray, {David W}",
note = "The KAT trial was funded by the National Institute Health for Research (NIHR) Health Technology Assessment Programme (project number 95/10/01) and has been published in full in Health Technology Assessment. The NIHR provided partial funding of the Health Economics Research Centre during the time this research was undertaken. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR, NHS or the Department of Health.",
year = "2018",
month = "8",
day = "16",
doi = "10.1186/s13063-018-2818-x",
language = "English",
volume = "19",
journal = "Trials",
issn = "1745-6215",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Partial factorial trials

T2 - comparing methods for statistical analysis and economic evaluation

AU - Dakin, Helen A

AU - Gray, Alastair M

AU - MacLennan, Graeme S

AU - Morris, Richard W

AU - Murray, David W

N1 - The KAT trial was funded by the National Institute Health for Research (NIHR) Health Technology Assessment Programme (project number 95/10/01) and has been published in full in Health Technology Assessment. The NIHR provided partial funding of the Health Economics Research Centre during the time this research was undertaken. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR, NHS or the Department of Health.

PY - 2018/8/16

Y1 - 2018/8/16

N2 - BACKGROUND: Partial factorial trials compare two or more pairs of treatments on overlapping patient groups, randomising some (but not all) patients to more than one comparison. The aims of this research were to compare different methods for conducting and analysing economic evaluations on partial factorial trials and assess the implications of considering factors simultaneously rather than drawing independent conclusions about each comparison.METHODS: We estimated total costs and quality-adjusted life years (QALYs) within 10 years of surgery for 2252 patients in the Knee Arthroplasty Trial who were randomised to one or more comparisons of different surgical types. We compared three analytical methods: an "at-the-margins" analysis including all patients randomised to each comparison (assuming no interaction); an "inside-the-table" analysis that included interactions but focused on those patients randomised to two comparisons; and a Bayesian vetted bootstrap, which used results from patients randomised to one comparison as priors when estimating outcomes for patients randomised to two comparisons. Outcomes comprised incremental costs, QALYs and net benefits.RESULTS: Qualitative interactions were observed for costs, QALYs and net benefits. Bayesian bootstrapping generally produced smaller standard errors than inside-the-table analysis and gave conclusions that were consistent with at-the-margins analysis, while allowing for these interactions. By contrast, inside-the-table gave different conclusions about which intervention had the highest net benefits compared with other analyses.CONCLUSIONS: All analyses of partial factorial trials should explore interactions and assess whether results are sensitive to assumptions about interactions, either as a primary analysis or as a sensitivity analysis. For partial factorial trials closely mirroring routine clinical practice, at-the-margins analysis may provide a reasonable estimate of average costs and benefits for the whole trial population, even in the presence of interactions. However, such conclusions will be misleading if there are large interactions or if the proportion of patients allocated to different treatments differs markedly from what occurs in clinical practice. The Bayesian bootstrap provides an alternative to at-the-margins analysis for analysing clinical or economic endpoints from partial factorial trials, which allows for interactions while making use of the whole sample. The same techniques could be applied to analyses of clinical endpoints.TRIAL REGISTRATION: ISRCTN, ISRCTN45837371 . Registered on 25 April 2003.

AB - BACKGROUND: Partial factorial trials compare two or more pairs of treatments on overlapping patient groups, randomising some (but not all) patients to more than one comparison. The aims of this research were to compare different methods for conducting and analysing economic evaluations on partial factorial trials and assess the implications of considering factors simultaneously rather than drawing independent conclusions about each comparison.METHODS: We estimated total costs and quality-adjusted life years (QALYs) within 10 years of surgery for 2252 patients in the Knee Arthroplasty Trial who were randomised to one or more comparisons of different surgical types. We compared three analytical methods: an "at-the-margins" analysis including all patients randomised to each comparison (assuming no interaction); an "inside-the-table" analysis that included interactions but focused on those patients randomised to two comparisons; and a Bayesian vetted bootstrap, which used results from patients randomised to one comparison as priors when estimating outcomes for patients randomised to two comparisons. Outcomes comprised incremental costs, QALYs and net benefits.RESULTS: Qualitative interactions were observed for costs, QALYs and net benefits. Bayesian bootstrapping generally produced smaller standard errors than inside-the-table analysis and gave conclusions that were consistent with at-the-margins analysis, while allowing for these interactions. By contrast, inside-the-table gave different conclusions about which intervention had the highest net benefits compared with other analyses.CONCLUSIONS: All analyses of partial factorial trials should explore interactions and assess whether results are sensitive to assumptions about interactions, either as a primary analysis or as a sensitivity analysis. For partial factorial trials closely mirroring routine clinical practice, at-the-margins analysis may provide a reasonable estimate of average costs and benefits for the whole trial population, even in the presence of interactions. However, such conclusions will be misleading if there are large interactions or if the proportion of patients allocated to different treatments differs markedly from what occurs in clinical practice. The Bayesian bootstrap provides an alternative to at-the-margins analysis for analysing clinical or economic endpoints from partial factorial trials, which allows for interactions while making use of the whole sample. The same techniques could be applied to analyses of clinical endpoints.TRIAL REGISTRATION: ISRCTN, ISRCTN45837371 . Registered on 25 April 2003.

KW - Journal Article

KW - randomised controlled trial

KW - factorial design

KW - cost-utility analysis

KW - Bayesian bootstrap

KW - partial factorial trial

U2 - 10.1186/s13063-018-2818-x

DO - 10.1186/s13063-018-2818-x

M3 - Article

C2 - 30115104

VL - 19

JO - Trials

JF - Trials

SN - 1745-6215

M1 - 442

ER -