TY - JOUR
T1 - Rating of Included Trials on the Efficacy-Effectiveness Spectrum
T2 - development of a new tool for systematic reviews
AU - Wieland, L. Susan
AU - Berman, Brian M.
AU - Altman, Douglas G.
AU - Barth, Jürgen
AU - Bouter, Lex M.
AU - D’Adamo, Christopher R.
AU - Linde, Klaus
AU - Moher, David
AU - Mullins, Daniel
AU - Treweek, Shaun
AU - Tunis, Sean
AU - van der Windt, Danielle A.
AU - Zwarenstein, Merrick
AU - Witt, Claudia
N1 - Funding: This work was supported by the National Institutes of Health National Center for Complementary and Integrative Health (R24 AT001293)
PY - 2017/4
Y1 - 2017/4
N2 - Background: Randomized trials may be designed to provide evidence more related to intervention efficacy or effectiveness. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of information from the included trials.
Objective: To develop a tool to characterize randomized trials information in a systematic review on an efficacy-effectiveness continuum.
Methods: We extracted rating domains and descriptors from existing tools, and used a modified Delphi procedure to condense domains and develop a new tool. The feasibility and inter-rater reliability of the tool was tested on trials from 4 systematic reviews.
Results: The RITES (Rating of Included Trials on the Efficacy-effectiveness Spectrum) tool rates clinical trials on a 5-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, (4) clinical relevance of interventions. When RITES was piloted on trials from 3 reviews by unaffiliated raters, ratings were variable (Intraclass Correlation Coefficients (ICCs) 0.23-0.45 for the four domains; 0.25-0.66 after excluding outliers), but when used on one review by the review authors with expertise on the topic the ratings were consistent (ICCs >0.80).
Conclusion: RITES may help characterize the efficacy-effectiveness nature of trial information in systematic reviews.
AB - Background: Randomized trials may be designed to provide evidence more related to intervention efficacy or effectiveness. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of information from the included trials.
Objective: To develop a tool to characterize randomized trials information in a systematic review on an efficacy-effectiveness continuum.
Methods: We extracted rating domains and descriptors from existing tools, and used a modified Delphi procedure to condense domains and develop a new tool. The feasibility and inter-rater reliability of the tool was tested on trials from 4 systematic reviews.
Results: The RITES (Rating of Included Trials on the Efficacy-effectiveness Spectrum) tool rates clinical trials on a 5-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, (4) clinical relevance of interventions. When RITES was piloted on trials from 3 reviews by unaffiliated raters, ratings were variable (Intraclass Correlation Coefficients (ICCs) 0.23-0.45 for the four domains; 0.25-0.66 after excluding outliers), but when used on one review by the review authors with expertise on the topic the ratings were consistent (ICCs >0.80).
Conclusion: RITES may help characterize the efficacy-effectiveness nature of trial information in systematic reviews.
KW - Comparative Effectiveness Research
KW - Systematic reviews
KW - Randomized controlled trials
KW - Effectiveness
KW - Efficacy
U2 - 10.1016/j.jclinepi.2017.01.010
DO - 10.1016/j.jclinepi.2017.01.010
M3 - Article
VL - 84
SP - 95
EP - 104
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
SN - 0895-4356
ER -