Learning from Cochrane systematic reviews: what improvements do these suggest for the design of trials?

Stefania Pirosca; Mike Clarke; Shaun Treweek

doi:10.12688/f1000research.22635.2

Learning from Cochrane systematic reviews: what improvements do these suggest for the design of trials?

Stefania Pirosca, Mike Clarke, Shaun Treweek^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

3 Downloads (Pure)

Abstract

Background: Many randomised trials have serious methodological flaws that fatally undermine their results, which makes the research wasteful. This is of concern for many, including those doing systematic reviews that include trials. Cochrane systematic reviews all have a section called ' Implications for research', which allows authors of the review to present their conclusions on how future research might be improved. Looking at these conclusions might highlight priority areas for improvement. Methods: We focused on the Cochrane Schizophrenia Review Group and the Multiple sclerosis and rare diseases of the central nervous system Review Group (the MS Review Group). Reviews with citations between 2009 and 2019 were identified and the recommendations given by review authors in ' Implications for research' were put into categories. Results: Between 2009 and 2019 we identified 162 reviews for the Schizophrenia Review Group and 43 reviews for the MS Review Group. We created 22 categories in total, of which 12 were common to both groups. The five most used categories were the same for both: better choice of outcomes; better choice of intervention/comparator; longer follow-up; larger sample size; use of validated scales. Better choice of outcomes and/or intervention/comparator was recommended in over 50% of reviews. Longer follow-up and larger sample size were recommended in over a third, with use of validated scales being suggested in around a fifth of reviews. There was no obvious pattern of improvement over time for trials included in systematic reviews published by both groups. Conclusions: We suggest that trialists working in these areas (and others too) ask themselves why they have chosen their particular outcomes, intervention and comparator, whether follow-up is long enough, if the sample size is big enough and whether the scales they choose to measure their outcomes are appropriate.

Original language	English
Article number	178
Number of pages	14
Journal	F1000Research
Volume	9
Early online date	11 Mar 2020
DOIs	https://doi.org/10.12688/f1000research.22635.2
Publication status	Published - 25 Jun 2020

Bibliographical note

Grant information:

The study was funded by the Health Research Board (grant ‘Capacity Building for Evidence Synthesis – Evidence Synthesis Ireland 2018’) as part of Trial Forge (http://trialforge.org), a non-commercial research initiative led by ST.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability Statement

Underlying data

Open Science Framework: Data extracted from the Implications for Research sections of reviews from two Cochrane Review Groups. https://doi.org/10.17605/OSF.IO/XCJ7R

This project contains the following underlying data:
- Data from Implications for Research section of Cochrane MS Review Group 2009–2019.csv (Data extracted from Cochrane MS Review Group)
- Data from Implications for Research section of Cochrane Schizophrenia Review Group 2009–2019.csv (Data extracted from Cochrane Schizophrenia Review
Group)

Extended data

Open Science Framework: Data extraction form used in ‘What improvements do Cochrane systematic reviewers suggest for the design of trials?’. https://doi.org/10.17605/OSF.IO/FNJVZ

This project contains the following extended data:
- Data extraction form.csv (Study data extraction form) Open Science Framework: Full data extraction from the Implications for Research sections of reviews from two Cochrane Review Groups. https://doi.org/10.17605/OSF.IO/P9CW6

This project contains the following data:
1. Data extraction MS 12-6-2020.csv
2. Data extraction Schizophrenia 12-6-2020.csv

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Keywords

Multiple sclerosis
Randomised controlled trial
Schizophrenia
Systematic reviews
Trial methodology

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.12688/f1000research.22635.2Licence: CC BY

Pirosca_et_al_F1000Res_LearningFromCochrane_VOR
This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/)
Final published version, 1.15 MBLicence: CC BY

Cite this

@article{697b4e3ac5dc449aa5d0de3c5d7637bb,

title = "Learning from Cochrane systematic reviews: what improvements do these suggest for the design of trials?",

abstract = "Background: Many randomised trials have serious methodological flaws that fatally undermine their results, which makes the research wasteful. This is of concern for many, including those doing systematic reviews that include trials. Cochrane systematic reviews all have a section called ' Implications for research', which allows authors of the review to present their conclusions on how future research might be improved. Looking at these conclusions might highlight priority areas for improvement. Methods: We focused on the Cochrane Schizophrenia Review Group and the Multiple sclerosis and rare diseases of the central nervous system Review Group (the MS Review Group). Reviews with citations between 2009 and 2019 were identified and the recommendations given by review authors in ' Implications for research' were put into categories. Results: Between 2009 and 2019 we identified 162 reviews for the Schizophrenia Review Group and 43 reviews for the MS Review Group. We created 22 categories in total, of which 12 were common to both groups. The five most used categories were the same for both: better choice of outcomes; better choice of intervention/comparator; longer follow-up; larger sample size; use of validated scales. Better choice of outcomes and/or intervention/comparator was recommended in over 50% of reviews. Longer follow-up and larger sample size were recommended in over a third, with use of validated scales being suggested in around a fifth of reviews. There was no obvious pattern of improvement over time for trials included in systematic reviews published by both groups. Conclusions: We suggest that trialists working in these areas (and others too) ask themselves why they have chosen their particular outcomes, intervention and comparator, whether follow-up is long enough, if the sample size is big enough and whether the scales they choose to measure their outcomes are appropriate.",

keywords = "Multiple sclerosis, Randomised controlled trial, Schizophrenia, Systematic reviews, Trial methodology",

author = "Stefania Pirosca and Mike Clarke and Shaun Treweek",

note = "Grant information: The study was funded by the Health Research Board (grant {\textquoteleft}Capacity Building for Evidence Synthesis – Evidence Synthesis Ireland 2018{\textquoteright}) as part of Trial Forge (http://trialforge.org), a non-commercial research initiative led by ST. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. ",

year = "2020",

month = jun,

day = "25",

doi = "10.12688/f1000research.22635.2",

language = "English",

volume = "9",

journal = "F1000Research",