Reporting methodological search filter performance comparisons

a literature review

Jennifer Harbour, Cynthia Fraser, Carol Lefebvre, Julie Glanville, Sophie Beale, Charles Boachie, Steven Duffy, Rachael McCool, Lynne Smith

Research output: Contribution to journalArticle

12 Citations (Scopus)
4 Downloads (Pure)

Abstract

BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.

METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.

RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.

CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

Original languageEnglish
Pages (from-to)176-194
Number of pages19
JournalHealth Information & Libraries Journal
Volume31
Issue number3
Early online date1 Aug 2014
DOIs
Publication statusPublished - Sep 2014

Fingerprint

Data Display
performance comparison
Routine Diagnostic Tests
Cost-Benefit Analysis
Randomized Controlled Trials
Databases
Sensitivity and Specificity
Research
performance
gold standard
research method
website
diagnostic
inclusion
evaluation
economics
Forests
literature

Keywords

  • bibliographic databases
  • information storage and retrieval
  • methodological filters
  • precision
  • recall
  • review
  • literature

Cite this

Harbour, J., Fraser, C., Lefebvre, C., Glanville, J., Beale, S., Boachie, C., ... Smith, L. (2014). Reporting methodological search filter performance comparisons: a literature review. Health Information & Libraries Journal, 31(3), 176-194. https://doi.org/10.1111/hir.12070

Reporting methodological search filter performance comparisons : a literature review. / Harbour, Jennifer; Fraser, Cynthia; Lefebvre, Carol; Glanville, Julie; Beale, Sophie; Boachie, Charles; Duffy, Steven; McCool, Rachael; Smith, Lynne.

In: Health Information & Libraries Journal, Vol. 31, No. 3, 09.2014, p. 176-194.

Research output: Contribution to journalArticle

Harbour, J, Fraser, C, Lefebvre, C, Glanville, J, Beale, S, Boachie, C, Duffy, S, McCool, R & Smith, L 2014, 'Reporting methodological search filter performance comparisons: a literature review', Health Information & Libraries Journal, vol. 31, no. 3, pp. 176-194. https://doi.org/10.1111/hir.12070
Harbour, Jennifer ; Fraser, Cynthia ; Lefebvre, Carol ; Glanville, Julie ; Beale, Sophie ; Boachie, Charles ; Duffy, Steven ; McCool, Rachael ; Smith, Lynne. / Reporting methodological search filter performance comparisons : a literature review. In: Health Information & Libraries Journal. 2014 ; Vol. 31, No. 3. pp. 176-194.
@article{a7ddfdb2b5dd43e59261367590d3f812,
title = "Reporting methodological search filter performance comparisons: a literature review",
abstract = "BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.",
keywords = "bibliographic databases, information storage and retrieval, methodological filters , precision, recall, review, literature",
author = "Jennifer Harbour and Cynthia Fraser and Carol Lefebvre and Julie Glanville and Sophie Beale and Charles Boachie and Steven Duffy and Rachael McCool and Lynne Smith",
note = "{\circledC} 2014 The authors. Health Information and Libraries Journal {\circledC} 2014 Health Libraries Journal.",
year = "2014",
month = "9",
doi = "10.1111/hir.12070",
language = "English",
volume = "31",
pages = "176--194",
journal = "Health Information & Libraries Journal",
issn = "1471-1834",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - Reporting methodological search filter performance comparisons

T2 - a literature review

AU - Harbour, Jennifer

AU - Fraser, Cynthia

AU - Lefebvre, Carol

AU - Glanville, Julie

AU - Beale, Sophie

AU - Boachie, Charles

AU - Duffy, Steven

AU - McCool, Rachael

AU - Smith, Lynne

N1 - © 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Journal.

PY - 2014/9

Y1 - 2014/9

N2 - BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

AB - BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

KW - bibliographic databases

KW - information storage and retrieval

KW - methodological filters

KW - precision

KW - recall

KW - review

KW - literature

U2 - 10.1111/hir.12070

DO - 10.1111/hir.12070

M3 - Article

VL - 31

SP - 176

EP - 194

JO - Health Information & Libraries Journal

JF - Health Information & Libraries Journal

SN - 1471-1834

IS - 3

ER -