Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree

Richard J. Sylvester; Steven E. Canfield; Thomas B. L. Lam; Lorenzo Marconi; Steven MacLennan; Yuhong  Yuan; Graeme MacLennan; John Norrie; Muhammad Imran Omar; Harman M. Bruins; Virginia Hernandez; Karin Plass; Hendrik  Van Poppel; James N'Dow

doi:10.1016/j.eururo.2016.11.023

Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree

Richard J. Sylvester, Steven E. Canfield, Thomas B. L. Lam, Lorenzo Marconi, Steven MacLennan, Yuhong Yuan, Graeme MacLennan, John Norrie, Muhammad Imran Omar, Harman M. Bruins, Virginia Hernandez, Karin Plass, Hendrik Van Poppel, James N'Dow

Research output: Contribution to journal › Article › peer-review

22 Citations (Scopus)

15 Downloads (Pure)

Abstract

Context
Clinicians and treatment guideline developers are faced with a dilemma when the results of a new, large, well-conducted randomized controlled trial (RCT) are in direct conflict with the results of a previous systematic review (SR) and meta-analysis (MA).

Objective
To explore and discuss possible reasons for disagreement in results from SRs/MAs and RCTs and to provide guidance to clinicians and guideline developers for making well-informed treatment decisions and recommendations in the face of conflicting data.

Evidence acquisition
The advantages and limitations of RCTs and SRs/MAs are reviewed. Two practical examples that have a direct bearing on European Association of Urology guidelines on treatment recommendations are discussed in detail to illustrate the points to be considered when conflicts exist between the results of large RCTs and SRs/MAs.

Evidence synthesis
RCTs are the gold standard for providing evidence of the effectiveness of interventions. However, concerns regarding the internal and external validity of an RCT may limit its applicability to clinical practice. SRs/MAs synthesize all evidence related to a given research question, but two urologic examples show that the validity of the results depends on the quality of the individual studies, the clinical and methodological heterogeneity of the studies, and publication bias.

Conclusions
Although SRs/MAs can provide a higher level of evidence than RCTs, the quality of the evidence from both RCTs and SRs/MAs should be investigated when their results conflict to determine which source provides the better evidence. Guideline developers should have a well-defined and robust process to assess the evidence from MAs and RCTs when such conflicts exist.

Patient summary
We discuss the advantages and limitations of using data from randomized controlled trials and systematic reviews/meta-analyses in informing clinical practice when there are conflicting results. We provide guidance on how such conflicts should be dealt with by guideline organizations.

Original language	English
Pages (from-to)	811-819
Number of pages	9
Journal	European Urology
Volume	71
Issue number	5
Early online date	30 Nov 2016
DOIs	https://doi.org/10.1016/j.eururo.2016.11.023
Publication status	Published - May 2017

Bibliographical note

Financial disclosures: Richard J. Sylvester certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.

Funding/Support and role of the sponsor: None.

Keywords

conflict of evidence
meta-analyses
randomized controlled trials
systematic reviews
treatment guidelines

Access to Document

10.1016/j.eururo.2016.11.023Licence: Unspecified

Accepted Manuscript
© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/
Accepted author manuscript, 1.75 MBLicence: CC BY-NC-ND

Steven MacLennan
Person: Academic Related - Research

Cite this

Sylvester, R. J., Canfield, S. E., Lam, T. B. L., Marconi, L., MacLennan, S., Yuan, Y., MacLennan, G., Norrie, J., Omar, M. I., Bruins, H. M., Hernandez, V., Plass, K., Van Poppel, H., & N'Dow, J. (2017). Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree. European Urology, 71(5), 811-819. https://doi.org/10.1016/j.eururo.2016.11.023

Sylvester, RJ, Canfield, SE, Lam, TBL, Marconi, L, MacLennan, S, Yuan, Y, MacLennan, G, Norrie, J, Omar, MI, Bruins, HM, Hernandez, V, Plass, K, Van Poppel, H & N'Dow, J 2017, 'Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree', European Urology, vol. 71, no. 5, pp. 811-819. https://doi.org/10.1016/j.eururo.2016.11.023

@article{3fa9359d0a874a8db0dee10b49efff75,

title = "Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree",

abstract = "ContextClinicians and treatment guideline developers are faced with a dilemma when the results of a new, large, well-conducted randomized controlled trial (RCT) are in direct conflict with the results of a previous systematic review (SR) and meta-analysis (MA).ObjectiveTo explore and discuss possible reasons for disagreement in results from SRs/MAs and RCTs and to provide guidance to clinicians and guideline developers for making well-informed treatment decisions and recommendations in the face of conflicting data.Evidence acquisitionThe advantages and limitations of RCTs and SRs/MAs are reviewed. Two practical examples that have a direct bearing on European Association of Urology guidelines on treatment recommendations are discussed in detail to illustrate the points to be considered when conflicts exist between the results of large RCTs and SRs/MAs.Evidence synthesisRCTs are the gold standard for providing evidence of the effectiveness of interventions. However, concerns regarding the internal and external validity of an RCT may limit its applicability to clinical practice. SRs/MAs synthesize all evidence related to a given research question, but two urologic examples show that the validity of the results depends on the quality of the individual studies, the clinical and methodological heterogeneity of the studies, and publication bias.ConclusionsAlthough SRs/MAs can provide a higher level of evidence than RCTs, the quality of the evidence from both RCTs and SRs/MAs should be investigated when their results conflict to determine which source provides the better evidence. Guideline developers should have a well-defined and robust process to assess the evidence from MAs and RCTs when such conflicts exist.Patient summaryWe discuss the advantages and limitations of using data from randomized controlled trials and systematic reviews/meta-analyses in informing clinical practice when there are conflicting results. We provide guidance on how such conflicts should be dealt with by guideline organizations.",

keywords = "conflict of evidence, meta-analyses, randomized controlled trials, systematic reviews, treatment guidelines",

author = "Sylvester, {Richard J.} and Canfield, {Steven E.} and Lam, {Thomas B. L.} and Lorenzo Marconi and Steven MacLennan and Yuhong Yuan and Graeme MacLennan and John Norrie and Omar, {Muhammad Imran} and Bruins, {Harman M.} and Virginia Hernandez and Karin Plass and {Van Poppel}, Hendrik and James N'Dow",

note = "Financial disclosures: Richard J. Sylvester certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None. Funding/Support and role of the sponsor: None.",

year = "2017",

month = may,

doi = "10.1016/j.eururo.2016.11.023",

language = "English",

volume = "71",

pages = "811--819",

journal = "European Urology",

issn = "0302-2838",

publisher = "Elsevier",

number = "5",

}

TY - JOUR

T1 - Conflict of Evidence

T2 - Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree

AU - Sylvester, Richard J.

AU - Canfield, Steven E.

AU - Lam, Thomas B. L.

AU - Marconi, Lorenzo

AU - MacLennan, Steven

AU - Yuan, Yuhong

AU - MacLennan, Graeme

AU - Norrie, John

AU - Omar, Muhammad Imran

AU - Bruins, Harman M.

AU - Hernandez, Virginia

AU - Plass, Karin

AU - Van Poppel, Hendrik

AU - N'Dow, James

N1 - Financial disclosures: Richard J. Sylvester certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None. Funding/Support and role of the sponsor: None.

PY - 2017/5

Y1 - 2017/5

N2 - ContextClinicians and treatment guideline developers are faced with a dilemma when the results of a new, large, well-conducted randomized controlled trial (RCT) are in direct conflict with the results of a previous systematic review (SR) and meta-analysis (MA).ObjectiveTo explore and discuss possible reasons for disagreement in results from SRs/MAs and RCTs and to provide guidance to clinicians and guideline developers for making well-informed treatment decisions and recommendations in the face of conflicting data.Evidence acquisitionThe advantages and limitations of RCTs and SRs/MAs are reviewed. Two practical examples that have a direct bearing on European Association of Urology guidelines on treatment recommendations are discussed in detail to illustrate the points to be considered when conflicts exist between the results of large RCTs and SRs/MAs.Evidence synthesisRCTs are the gold standard for providing evidence of the effectiveness of interventions. However, concerns regarding the internal and external validity of an RCT may limit its applicability to clinical practice. SRs/MAs synthesize all evidence related to a given research question, but two urologic examples show that the validity of the results depends on the quality of the individual studies, the clinical and methodological heterogeneity of the studies, and publication bias.ConclusionsAlthough SRs/MAs can provide a higher level of evidence than RCTs, the quality of the evidence from both RCTs and SRs/MAs should be investigated when their results conflict to determine which source provides the better evidence. Guideline developers should have a well-defined and robust process to assess the evidence from MAs and RCTs when such conflicts exist.Patient summaryWe discuss the advantages and limitations of using data from randomized controlled trials and systematic reviews/meta-analyses in informing clinical practice when there are conflicting results. We provide guidance on how such conflicts should be dealt with by guideline organizations.

AB - ContextClinicians and treatment guideline developers are faced with a dilemma when the results of a new, large, well-conducted randomized controlled trial (RCT) are in direct conflict with the results of a previous systematic review (SR) and meta-analysis (MA).ObjectiveTo explore and discuss possible reasons for disagreement in results from SRs/MAs and RCTs and to provide guidance to clinicians and guideline developers for making well-informed treatment decisions and recommendations in the face of conflicting data.Evidence acquisitionThe advantages and limitations of RCTs and SRs/MAs are reviewed. Two practical examples that have a direct bearing on European Association of Urology guidelines on treatment recommendations are discussed in detail to illustrate the points to be considered when conflicts exist between the results of large RCTs and SRs/MAs.Evidence synthesisRCTs are the gold standard for providing evidence of the effectiveness of interventions. However, concerns regarding the internal and external validity of an RCT may limit its applicability to clinical practice. SRs/MAs synthesize all evidence related to a given research question, but two urologic examples show that the validity of the results depends on the quality of the individual studies, the clinical and methodological heterogeneity of the studies, and publication bias.ConclusionsAlthough SRs/MAs can provide a higher level of evidence than RCTs, the quality of the evidence from both RCTs and SRs/MAs should be investigated when their results conflict to determine which source provides the better evidence. Guideline developers should have a well-defined and robust process to assess the evidence from MAs and RCTs when such conflicts exist.Patient summaryWe discuss the advantages and limitations of using data from randomized controlled trials and systematic reviews/meta-analyses in informing clinical practice when there are conflicting results. We provide guidance on how such conflicts should be dealt with by guideline organizations.

KW - conflict of evidence

KW - meta-analyses

KW - randomized controlled trials

KW - systematic reviews

KW - treatment guidelines

U2 - 10.1016/j.eururo.2016.11.023

DO - 10.1016/j.eururo.2016.11.023

M3 - Article

SN - 0302-2838

VL - 71

SP - 811

EP - 819

JO - European Urology

JF - European Urology

IS - 5

ER -

Conflict of Evidence: Resolving Discrepancies When Findings from Randomized Controlled Trials and Meta-analyses Disagree

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Profiles

Steven MacLennan

Cite this