The quality of research synthesis in surgery: the case of laparoscopic surgery for colorectal cancer

Guillaume Martel, Suleena Duhaime, Jeffrey S Barkun, Robin P Boushey, Craig R Ramsay, Dean A Fergusson

Research output: Contribution to journalArticle

16 Citations (Scopus)
3 Downloads (Pure)

Abstract

Background
Several systematic reviews and meta-analyses populate the literature on the effectiveness of laparoscopic surgery for colorectal cancer. The utility of this body of work is unclear. The objective of this study was to synthesize all such systematic reviews in terms of clinical effectiveness, to appraise their quality, and to determine whether areas of duplication exist across reviews.

Methods
Systematic reviews comparing laparoscopic and open surgery for colorectal cancer were identified using a comprehensive search protocol (1991 to 2008). The primary outcome was overall survival. The methodological quality of reviews was appraised using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Abstraction and quality appraisal was carried out by two independent reviewers. Reviews were synthesized, and outcomes were compared qualitatively. A citation analysis was carried out using simple matrices to assess the comprehensiveness of each review.

Results
In total, 27 reviews were included; 13 reviews included only randomized controlled trials. Rectal cancer was addressed exclusively by four reviews. There was significant overlap between review purposes, populations and, outcomes. The mean AMSTAR score (out of 11) was 5.8 (95% CI: 4.6 to 7.0). Overall survival was evaluated by ten reviews, none of which found a significant difference. Three reviews provided a selective meta-analysis of time-to-event data. Previously published systematic reviews were poorly and highly selectively referenced (mean citation ratio 0.16, 95% CI: 0.093 to 0.22). Previously published trials were not comprehensively identified and cited (mean citation ratio 0.56, 95% CI: 0.46 to 0.65).

Conclusions
Numerous overlapping systematic reviews of laparoscopic and open surgery for colorectal cancer exist in the literature. Despite variable methods and quality, survival outcomes are congruent across reviews. A duplication of research efforts appears to exist in the literature. Further systematic reviews or meta-analyses are unlikely to be justified without specifying a significantly different research objective. This works lends support to the registration and updating of systematic reviews.

Original languageEnglish
Pages (from-to)14
JournalSystematic reviews
Volume1
Issue number1
DOIs
Publication statusPublished - Feb 2012

Fingerprint

Laparoscopy
Meta-Analysis
Colorectal Neoplasms
Research
Rectal Neoplasms
Randomized Controlled Trials
Population

Cite this

The quality of research synthesis in surgery : the case of laparoscopic surgery for colorectal cancer. / Martel, Guillaume; Duhaime, Suleena; Barkun, Jeffrey S; Boushey, Robin P; Ramsay, Craig R; Fergusson, Dean A.

In: Systematic reviews , Vol. 1, No. 1, 02.2012, p. 14.

Research output: Contribution to journalArticle

Martel, Guillaume ; Duhaime, Suleena ; Barkun, Jeffrey S ; Boushey, Robin P ; Ramsay, Craig R ; Fergusson, Dean A. / The quality of research synthesis in surgery : the case of laparoscopic surgery for colorectal cancer. In: Systematic reviews . 2012 ; Vol. 1, No. 1. pp. 14.
@article{61b5fec6b57c4e5b83357e04f1dadde6,
title = "The quality of research synthesis in surgery: the case of laparoscopic surgery for colorectal cancer",
abstract = "Background Several systematic reviews and meta-analyses populate the literature on the effectiveness of laparoscopic surgery for colorectal cancer. The utility of this body of work is unclear. The objective of this study was to synthesize all such systematic reviews in terms of clinical effectiveness, to appraise their quality, and to determine whether areas of duplication exist across reviews. Methods Systematic reviews comparing laparoscopic and open surgery for colorectal cancer were identified using a comprehensive search protocol (1991 to 2008). The primary outcome was overall survival. The methodological quality of reviews was appraised using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Abstraction and quality appraisal was carried out by two independent reviewers. Reviews were synthesized, and outcomes were compared qualitatively. A citation analysis was carried out using simple matrices to assess the comprehensiveness of each review. Results In total, 27 reviews were included; 13 reviews included only randomized controlled trials. Rectal cancer was addressed exclusively by four reviews. There was significant overlap between review purposes, populations and, outcomes. The mean AMSTAR score (out of 11) was 5.8 (95{\%} CI: 4.6 to 7.0). Overall survival was evaluated by ten reviews, none of which found a significant difference. Three reviews provided a selective meta-analysis of time-to-event data. Previously published systematic reviews were poorly and highly selectively referenced (mean citation ratio 0.16, 95{\%} CI: 0.093 to 0.22). Previously published trials were not comprehensively identified and cited (mean citation ratio 0.56, 95{\%} CI: 0.46 to 0.65). Conclusions Numerous overlapping systematic reviews of laparoscopic and open surgery for colorectal cancer exist in the literature. Despite variable methods and quality, survival outcomes are congruent across reviews. A duplication of research efforts appears to exist in the literature. Further systematic reviews or meta-analyses are unlikely to be justified without specifying a significantly different research objective. This works lends support to the registration and updating of systematic reviews.",
author = "Guillaume Martel and Suleena Duhaime and Barkun, {Jeffrey S} and Boushey, {Robin P} and Ramsay, {Craig R} and Fergusson, {Dean A}",
year = "2012",
month = "2",
doi = "10.1186/2046-4053-1-14",
language = "English",
volume = "1",
pages = "14",
journal = "Systematic reviews",
issn = "2046-4053",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - The quality of research synthesis in surgery

T2 - the case of laparoscopic surgery for colorectal cancer

AU - Martel, Guillaume

AU - Duhaime, Suleena

AU - Barkun, Jeffrey S

AU - Boushey, Robin P

AU - Ramsay, Craig R

AU - Fergusson, Dean A

PY - 2012/2

Y1 - 2012/2

N2 - Background Several systematic reviews and meta-analyses populate the literature on the effectiveness of laparoscopic surgery for colorectal cancer. The utility of this body of work is unclear. The objective of this study was to synthesize all such systematic reviews in terms of clinical effectiveness, to appraise their quality, and to determine whether areas of duplication exist across reviews. Methods Systematic reviews comparing laparoscopic and open surgery for colorectal cancer were identified using a comprehensive search protocol (1991 to 2008). The primary outcome was overall survival. The methodological quality of reviews was appraised using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Abstraction and quality appraisal was carried out by two independent reviewers. Reviews were synthesized, and outcomes were compared qualitatively. A citation analysis was carried out using simple matrices to assess the comprehensiveness of each review. Results In total, 27 reviews were included; 13 reviews included only randomized controlled trials. Rectal cancer was addressed exclusively by four reviews. There was significant overlap between review purposes, populations and, outcomes. The mean AMSTAR score (out of 11) was 5.8 (95% CI: 4.6 to 7.0). Overall survival was evaluated by ten reviews, none of which found a significant difference. Three reviews provided a selective meta-analysis of time-to-event data. Previously published systematic reviews were poorly and highly selectively referenced (mean citation ratio 0.16, 95% CI: 0.093 to 0.22). Previously published trials were not comprehensively identified and cited (mean citation ratio 0.56, 95% CI: 0.46 to 0.65). Conclusions Numerous overlapping systematic reviews of laparoscopic and open surgery for colorectal cancer exist in the literature. Despite variable methods and quality, survival outcomes are congruent across reviews. A duplication of research efforts appears to exist in the literature. Further systematic reviews or meta-analyses are unlikely to be justified without specifying a significantly different research objective. This works lends support to the registration and updating of systematic reviews.

AB - Background Several systematic reviews and meta-analyses populate the literature on the effectiveness of laparoscopic surgery for colorectal cancer. The utility of this body of work is unclear. The objective of this study was to synthesize all such systematic reviews in terms of clinical effectiveness, to appraise their quality, and to determine whether areas of duplication exist across reviews. Methods Systematic reviews comparing laparoscopic and open surgery for colorectal cancer were identified using a comprehensive search protocol (1991 to 2008). The primary outcome was overall survival. The methodological quality of reviews was appraised using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Abstraction and quality appraisal was carried out by two independent reviewers. Reviews were synthesized, and outcomes were compared qualitatively. A citation analysis was carried out using simple matrices to assess the comprehensiveness of each review. Results In total, 27 reviews were included; 13 reviews included only randomized controlled trials. Rectal cancer was addressed exclusively by four reviews. There was significant overlap between review purposes, populations and, outcomes. The mean AMSTAR score (out of 11) was 5.8 (95% CI: 4.6 to 7.0). Overall survival was evaluated by ten reviews, none of which found a significant difference. Three reviews provided a selective meta-analysis of time-to-event data. Previously published systematic reviews were poorly and highly selectively referenced (mean citation ratio 0.16, 95% CI: 0.093 to 0.22). Previously published trials were not comprehensively identified and cited (mean citation ratio 0.56, 95% CI: 0.46 to 0.65). Conclusions Numerous overlapping systematic reviews of laparoscopic and open surgery for colorectal cancer exist in the literature. Despite variable methods and quality, survival outcomes are congruent across reviews. A duplication of research efforts appears to exist in the literature. Further systematic reviews or meta-analyses are unlikely to be justified without specifying a significantly different research objective. This works lends support to the registration and updating of systematic reviews.

U2 - 10.1186/2046-4053-1-14

DO - 10.1186/2046-4053-1-14

M3 - Article

C2 - 22588035

VL - 1

SP - 14

JO - Systematic reviews

JF - Systematic reviews

SN - 2046-4053

IS - 1

ER -