Meta-analyses triggered by previous (false-)significant findings: Problems and solutions

Ewoud Schuit*, Kit C.B. Roes, Ben W.J. Mol, Anneke Kwee, Karel G.M. Moons, Rolf H.H. Groenwold

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)

Abstract

Background: Meta-analyses are typically triggered by a (potentially false-significant) finding in one of the preceding primary studies. We studied consequences of meta-analysis investigating effects when primary studies that triggered such meta-analysis are also included. Methods: We analytically determined the bias of the treatment effect estimates obtained by meta-analysis, conditional on the number of included primary and false-significant studies. The type I error rate and power of the meta-analysis were assessed using simulations. We applied a method for bias-correction, by subtracting an analytically derived bias from the treatment effect estimated in meta-analysis. Results: Bias in meta-analytical effects and type I error rates increased when increasing numbers of primary studies with false-significant effects were included. When 20% of the primary studies showed false-significant effects, the bias was 0.33 (z-score) instead of 0, and the type I error rate was 23% instead of 5%. After applying a bias-correction, the type I error rate became indeed 5%. Conclusions: Inclusion of primary studies with false-significant effects leads to biased effect estimates and inflated type I error rates in the meta-analysis, depending on the number of false-significant studies. This bias can be adjusted for.

Original languageEnglish
Article number57
JournalSystematic reviews
Volume4
Issue number1
DOIs
Publication statusPublished - 25 Apr 2015

Keywords

  • Bias
  • Meta-analysis
  • Power
  • Type I error

Fingerprint

Dive into the research topics of 'Meta-analyses triggered by previous (false-)significant findings: Problems and solutions'. Together they form a unique fingerprint.

Cite this