The Perceived Feasibility of Methods to Reduce Publication Bias

Harriet A. Carroll (Corresponding Author), Zoi Toumpakari, Laura Johnson, James A. Betts

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Publication bias is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce publication bias, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce publication bias by gauging opinions of journal editors (n = 73) and other academics/researchers (n = 160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing publication bias for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6% of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.
Original languageEnglish
Article numbere0186472
Number of pages19
JournalPloS ONE
Volume12
Issue number10
DOIs
Publication statusPublished - 24 Oct 2017

Fingerprint

Publication Bias
Research Personnel
Publications
researchers
methodology
Gaging
Accreditation
peers
Research
Literature
Peer Review
questionnaires

Cite this

The Perceived Feasibility of Methods to Reduce Publication Bias. / Carroll, Harriet A. (Corresponding Author); Toumpakari, Zoi; Johnson, Laura; Betts, James A.

In: PloS ONE, Vol. 12, No. 10, e0186472, 24.10.2017.

Research output: Contribution to journalArticle

Carroll, Harriet A. ; Toumpakari, Zoi ; Johnson, Laura ; Betts, James A. / The Perceived Feasibility of Methods to Reduce Publication Bias. In: PloS ONE. 2017 ; Vol. 12, No. 10.
@article{40a93e6598c842d39a208c114244548c,
title = "The Perceived Feasibility of Methods to Reduce Publication Bias",
abstract = "Publication bias is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce publication bias, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce publication bias by gauging opinions of journal editors (n = 73) and other academics/researchers (n = 160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing publication bias for editors (25{\%}), and was the third most popular choice for academics/researchers (14{\%}). The most common selection for academics/researchers was two-stage review (26{\%}), but fewer editors prioritised this (11{\%}). Negative results journals/articles were the second and third most common choices for academics/researchers (21{\%}) and editors (16{\%}), respectively. Editors more commonly chose research registration as ‘most effective’ (21{\%}), which was favoured by only 6{\%} of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.",
author = "Carroll, {Harriet A.} and Zoi Toumpakari and Laura Johnson and Betts, {James A.}",
note = "Funding: The primary researcher (HAC) is supported by the Economic and Social Research Council (http://www.esrc.ac.uk/ grant number: ES/J50015X/1). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.",
year = "2017",
month = "10",
day = "24",
doi = "10.1371/journal.pone.0186472",
language = "English",
volume = "12",
journal = "PloS ONE",
issn = "1932-6203",
publisher = "PUBLIC LIBRARY SCIENCE",
number = "10",

}

TY - JOUR

T1 - The Perceived Feasibility of Methods to Reduce Publication Bias

AU - Carroll, Harriet A.

AU - Toumpakari, Zoi

AU - Johnson, Laura

AU - Betts, James A.

N1 - Funding: The primary researcher (HAC) is supported by the Economic and Social Research Council (http://www.esrc.ac.uk/ grant number: ES/J50015X/1). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

PY - 2017/10/24

Y1 - 2017/10/24

N2 - Publication bias is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce publication bias, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce publication bias by gauging opinions of journal editors (n = 73) and other academics/researchers (n = 160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing publication bias for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6% of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.

AB - Publication bias is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce publication bias, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce publication bias by gauging opinions of journal editors (n = 73) and other academics/researchers (n = 160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing publication bias for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6% of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.

U2 - 10.1371/journal.pone.0186472

DO - 10.1371/journal.pone.0186472

M3 - Article

VL - 12

JO - PloS ONE

JF - PloS ONE

SN - 1932-6203

IS - 10

M1 - e0186472

ER -