A test of performance of breast MRI interpretation in a multicentre screening study.

R. M. Warren; C. Hayes; L. J. Pointon; R. J. Hoff; Fiona Jane Gilbert; A. R. Padhani; C. Rubin; G. Kaplan; K. Raza; L. Wilkinson; M. Hall-Craggs; P. Kessar; S. Rankin; A. Dixon; J. Walsh; L. W. Turnbull; P. Britton; R. Sinnatamby; D. Easton; D. Thompson; S. R. Lakhani; M. O. Leach; Collaborators UK MRC study MRI br

doi:10.1016/j.mri.2006.03.004

A test of performance of breast MRI interpretation in a multicentre screening study.

R. M. Warren, C. Hayes, L. J. Pointon, R. J. Hoff, Fiona Jane Gilbert, A. R. Padhani, C. Rubin, G. Kaplan, K. Raza, L. Wilkinson, M. Hall-Craggs, P. Kessar, S. Rankin, A. Dixon, J. Walsh, L. W. Turnbull, P. Britton, R. Sinnatamby, D. Easton, D. ThompsonS. R. Lakhani, M. O. Leach, Collaborators UK MRC study MRI br

Applied Medicine

Research output: Contribution to journal › Article › peer-review

17 Citations (Scopus)

Abstract

Objectives: The aim of this study was to assess the consistency and performance of radiologists interpreting breast magnetic resonance imaging (MRI) examinations.

Materials and Methods: Two test sets of eight cases comprising cancers, benign disease, technical problems and parenchymal enhancement were prepared from two manufacturers' equipment (X and Y) and reported by 15 radiologists using the recording form and scoring system of the UK MRI breast screening study [(MAgnetic Resonance Imaging in Breast Screening (MARIBS)]. Variations in assessments of morphology, kinetic scores and diagnosis were measured by assessing intraobserver and interobserver variability and agreement. The sensitivity and specificity of reporting performances was determined using receiver operating characteristic (ROC) curve analysis.

Results: Intraobserver variation was seen in 13 (27.7%) of 47 of the radiologists' conclusions (four technical and seven pathological differences). Substantial interobserver variation was observed in the scores recorded for morphology, pattern of enhancement, quantification of enhancement and washout pattern. The overall sensitivity of breast MRI was high [88.6%, 95% confidence interval (CI) 77.4-94.7%], combined with a specificity of 69.2% (95% CI 60.5-76.7%). The sensitivities were similar for the two test sets (P=.3), but the specificity was significantly higher for the Manufacturer X dataset (P <.001). ROC curve analysis gave an area under the curve of 0.85 (95% CI 0.79-0.92)

Conclusions: Substantial variation in all elements of the scoring system and in the overall diagnostic conclusions was observed between radiologists participating in MARIBS. High overall sensitivity was achieved with moderate specificity. Manufacturer-related differences in specificities possibly occurred because the numerical thresholds set for the scoring system were not optimised for both equipment manufacturers. Scoring systems developed on one equipment software may not be transferable to other manufacturers. (c) 2006 Elsevier Inc. All rights reserved.

Original language	English
Pages (from-to)	917-929
Number of pages	12
Journal	Magnetic Resonance Imaging
Volume	24
DOIs	https://doi.org/10.1016/j.mri.2006.03.004
Publication status	Published - 2006

Keywords

breast MRI
quality control
reporting performance
OBSERVER VARIABILITY
DATA SYSTEM
MAMMOGRAPHY
AGREEMENT
PROTOCOL
CANCER
RISK

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.mri.2006.03.004

Cite this

Warren, R. M., Hayes, C., Pointon, L. J., Hoff, R. J., Gilbert, F. J., Padhani, A. R., Rubin, C., Kaplan, G., Raza, K., Wilkinson, L., Hall-Craggs, M., Kessar, P., Rankin, S., Dixon, A., Walsh, J., Turnbull, L. W., Britton, P., Sinnatamby, R., Easton, D., ... Collaborators UK MRC study MRI br (2006). A test of performance of breast MRI interpretation in a multicentre screening study. Magnetic Resonance Imaging, 24, 917-929. https://doi.org/10.1016/j.mri.2006.03.004

Warren, RM, Hayes, C, Pointon, LJ, Hoff, RJ, Gilbert, FJ, Padhani, AR, Rubin, C, Kaplan, G, Raza, K, Wilkinson, L, Hall-Craggs, M, Kessar, P, Rankin, S, Dixon, A, Walsh, J, Turnbull, LW, Britton, P, Sinnatamby, R, Easton, D, Thompson, D, Lakhani, SR, Leach, MO & Collaborators UK MRC study MRI br 2006, 'A test of performance of breast MRI interpretation in a multicentre screening study.', Magnetic Resonance Imaging, vol. 24, pp. 917-929. https://doi.org/10.1016/j.mri.2006.03.004

@article{f6734276c0de4657a4aacccec89e66eb,

title = "A test of performance of breast MRI interpretation in a multicentre screening study.",

abstract = "Objectives: The aim of this study was to assess the consistency and performance of radiologists interpreting breast magnetic resonance imaging (MRI) examinations.Materials and Methods: Two test sets of eight cases comprising cancers, benign disease, technical problems and parenchymal enhancement were prepared from two manufacturers' equipment (X and Y) and reported by 15 radiologists using the recording form and scoring system of the UK MRI breast screening study [(MAgnetic Resonance Imaging in Breast Screening (MARIBS)]. Variations in assessments of morphology, kinetic scores and diagnosis were measured by assessing intraobserver and interobserver variability and agreement. The sensitivity and specificity of reporting performances was determined using receiver operating characteristic (ROC) curve analysis.Results: Intraobserver variation was seen in 13 (27.7%) of 47 of the radiologists' conclusions (four technical and seven pathological differences). Substantial interobserver variation was observed in the scores recorded for morphology, pattern of enhancement, quantification of enhancement and washout pattern. The overall sensitivity of breast MRI was high [88.6%, 95% confidence interval (CI) 77.4-94.7%], combined with a specificity of 69.2% (95% CI 60.5-76.7%). The sensitivities were similar for the two test sets (P=.3), but the specificity was significantly higher for the Manufacturer X dataset (P <.001). ROC curve analysis gave an area under the curve of 0.85 (95% CI 0.79-0.92)Conclusions: Substantial variation in all elements of the scoring system and in the overall diagnostic conclusions was observed between radiologists participating in MARIBS. High overall sensitivity was achieved with moderate specificity. Manufacturer-related differences in specificities possibly occurred because the numerical thresholds set for the scoring system were not optimised for both equipment manufacturers. Scoring systems developed on one equipment software may not be transferable to other manufacturers. (c) 2006 Elsevier Inc. All rights reserved.",

keywords = "breast MRI, quality control, reporting performance, OBSERVER VARIABILITY, DATA SYSTEM, MAMMOGRAPHY, AGREEMENT, PROTOCOL, CANCER, RISK",

author = "Warren, {R. M.} and C. Hayes and Pointon, {L. J.} and Hoff, {R. J.} and Gilbert, {Fiona Jane} and Padhani, {A. R.} and C. Rubin and G. Kaplan and K. Raza and L. Wilkinson and M. Hall-Craggs and P. Kessar and S. Rankin and A. Dixon and J. Walsh and Turnbull, {L. W.} and P. Britton and R. Sinnatamby and D. Easton and D. Thompson and Lakhani, {S. R.} and Leach, {M. O.} and {Collaborators UK MRC study MRI br}",

year = "2006",

doi = "10.1016/j.mri.2006.03.004",

language = "English",

volume = "24",

pages = "917--929",

journal = "Magnetic Resonance Imaging",

issn = "0730-725X",

publisher = "Elsevier",

}

TY - JOUR

T1 - A test of performance of breast MRI interpretation in a multicentre screening study.

AU - Warren, R. M.

AU - Hayes, C.

AU - Pointon, L. J.

AU - Hoff, R. J.

AU - Gilbert, Fiona Jane

AU - Padhani, A. R.

AU - Rubin, C.

AU - Kaplan, G.

AU - Raza, K.

AU - Wilkinson, L.

AU - Hall-Craggs, M.

AU - Kessar, P.

AU - Rankin, S.

AU - Dixon, A.

AU - Walsh, J.

AU - Turnbull, L. W.

AU - Britton, P.

AU - Sinnatamby, R.

AU - Easton, D.

AU - Thompson, D.

AU - Lakhani, S. R.

AU - Leach, M. O.

AU - Collaborators UK MRC study MRI br

PY - 2006

Y1 - 2006

N2 - Objectives: The aim of this study was to assess the consistency and performance of radiologists interpreting breast magnetic resonance imaging (MRI) examinations.Materials and Methods: Two test sets of eight cases comprising cancers, benign disease, technical problems and parenchymal enhancement were prepared from two manufacturers' equipment (X and Y) and reported by 15 radiologists using the recording form and scoring system of the UK MRI breast screening study [(MAgnetic Resonance Imaging in Breast Screening (MARIBS)]. Variations in assessments of morphology, kinetic scores and diagnosis were measured by assessing intraobserver and interobserver variability and agreement. The sensitivity and specificity of reporting performances was determined using receiver operating characteristic (ROC) curve analysis.Results: Intraobserver variation was seen in 13 (27.7%) of 47 of the radiologists' conclusions (four technical and seven pathological differences). Substantial interobserver variation was observed in the scores recorded for morphology, pattern of enhancement, quantification of enhancement and washout pattern. The overall sensitivity of breast MRI was high [88.6%, 95% confidence interval (CI) 77.4-94.7%], combined with a specificity of 69.2% (95% CI 60.5-76.7%). The sensitivities were similar for the two test sets (P=.3), but the specificity was significantly higher for the Manufacturer X dataset (P <.001). ROC curve analysis gave an area under the curve of 0.85 (95% CI 0.79-0.92)Conclusions: Substantial variation in all elements of the scoring system and in the overall diagnostic conclusions was observed between radiologists participating in MARIBS. High overall sensitivity was achieved with moderate specificity. Manufacturer-related differences in specificities possibly occurred because the numerical thresholds set for the scoring system were not optimised for both equipment manufacturers. Scoring systems developed on one equipment software may not be transferable to other manufacturers. (c) 2006 Elsevier Inc. All rights reserved.

AB - Objectives: The aim of this study was to assess the consistency and performance of radiologists interpreting breast magnetic resonance imaging (MRI) examinations.Materials and Methods: Two test sets of eight cases comprising cancers, benign disease, technical problems and parenchymal enhancement were prepared from two manufacturers' equipment (X and Y) and reported by 15 radiologists using the recording form and scoring system of the UK MRI breast screening study [(MAgnetic Resonance Imaging in Breast Screening (MARIBS)]. Variations in assessments of morphology, kinetic scores and diagnosis were measured by assessing intraobserver and interobserver variability and agreement. The sensitivity and specificity of reporting performances was determined using receiver operating characteristic (ROC) curve analysis.Results: Intraobserver variation was seen in 13 (27.7%) of 47 of the radiologists' conclusions (four technical and seven pathological differences). Substantial interobserver variation was observed in the scores recorded for morphology, pattern of enhancement, quantification of enhancement and washout pattern. The overall sensitivity of breast MRI was high [88.6%, 95% confidence interval (CI) 77.4-94.7%], combined with a specificity of 69.2% (95% CI 60.5-76.7%). The sensitivities were similar for the two test sets (P=.3), but the specificity was significantly higher for the Manufacturer X dataset (P <.001). ROC curve analysis gave an area under the curve of 0.85 (95% CI 0.79-0.92)Conclusions: Substantial variation in all elements of the scoring system and in the overall diagnostic conclusions was observed between radiologists participating in MARIBS. High overall sensitivity was achieved with moderate specificity. Manufacturer-related differences in specificities possibly occurred because the numerical thresholds set for the scoring system were not optimised for both equipment manufacturers. Scoring systems developed on one equipment software may not be transferable to other manufacturers. (c) 2006 Elsevier Inc. All rights reserved.

KW - breast MRI

KW - quality control

KW - reporting performance

KW - OBSERVER VARIABILITY

KW - DATA SYSTEM

KW - MAMMOGRAPHY

KW - AGREEMENT

KW - PROTOCOL

KW - CANCER

KW - RISK

U2 - 10.1016/j.mri.2006.03.004

DO - 10.1016/j.mri.2006.03.004

M3 - Article

SN - 0730-725X

VL - 24

SP - 917

EP - 929

JO - Magnetic Resonance Imaging

JF - Magnetic Resonance Imaging

ER -

A test of performance of breast MRI interpretation in a multicentre screening study.

Abstract

Keywords

UN SDGs

Access to Document

Fingerprint

Cite this