External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme

K A Goatman, S Philip, A D Fleming, R D Harvey, K K Swa, C Styles, M Black, G Sell, N Lee, P F Sharp, J A Olson

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Aims: To develop and evaluate an image grading external quality assurance system for the Scottish Diabetic Retinopathy Screening Programme. Method: A web-based image grading system was developed which closely matches the current Scottish national screening software. Two rounds of external quality assurance were run in autumn 2008 and spring 2010, each time using the same 100 images. Graders were compared with a consensus standard derived from the top-level graders' results. After the first round, the centre lead clinicians and top-level graders reviewed the results and drew up guidance notes for the second round. Results: Grader sensitivities ranged from 60.0 to 100% (median 92.5%) in 2008, and from 62.5 to 100% (median 92.5%) in 2010. Specificities ranged from 34.0 to 98.0% (median 86%) in 2008, and 54.0 to 100% (median 88%) in 2010. There was no difference in sensitivity between grader levels, but first-level graders had a significantly lower specificity than level-two and level-three graders. In 2008, one centre had a lower sensitivity but higher specificity than the majority of centres. Following the feedback from the first round, overall agreement improved in 2010 and there were no longer any significant differences between centres. Conclusions: A useful educational tool has been developed for image grading external quality assurance.
Original languageEnglish
Pages (from-to)776-783
Number of pages8
JournalDiabetic Medicine
Volume29
Issue number6
Early online date16 May 2012
DOIs
Publication statusPublished - Jun 2012

Fingerprint

Diabetic Retinopathy
Software
Sensitivity and Specificity

Keywords

  • diabetic retinopathy
  • quality assurance
  • retinal screening
  • sensitivity
  • specificity

Cite this

External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme. / Goatman, K A; Philip, S; Fleming, A D; Harvey, R D; Swa, K K; Styles, C; Black, M; Sell, G; Lee, N; Sharp, P F; Olson, J A.

In: Diabetic Medicine, Vol. 29, No. 6, 06.2012, p. 776-783.

Research output: Contribution to journalArticle

Goatman, KA, Philip, S, Fleming, AD, Harvey, RD, Swa, KK, Styles, C, Black, M, Sell, G, Lee, N, Sharp, PF & Olson, JA 2012, 'External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme', Diabetic Medicine, vol. 29, no. 6, pp. 776-783. https://doi.org/10.1111/j.1464-5491.2011.03504.x
Goatman, K A ; Philip, S ; Fleming, A D ; Harvey, R D ; Swa, K K ; Styles, C ; Black, M ; Sell, G ; Lee, N ; Sharp, P F ; Olson, J A. / External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme. In: Diabetic Medicine. 2012 ; Vol. 29, No. 6. pp. 776-783.
@article{c0ed0a06431e4034baed4c8d1dc98e57,
title = "External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme",
abstract = "Aims: To develop and evaluate an image grading external quality assurance system for the Scottish Diabetic Retinopathy Screening Programme. Method: A web-based image grading system was developed which closely matches the current Scottish national screening software. Two rounds of external quality assurance were run in autumn 2008 and spring 2010, each time using the same 100 images. Graders were compared with a consensus standard derived from the top-level graders' results. After the first round, the centre lead clinicians and top-level graders reviewed the results and drew up guidance notes for the second round. Results: Grader sensitivities ranged from 60.0 to 100{\%} (median 92.5{\%}) in 2008, and from 62.5 to 100{\%} (median 92.5{\%}) in 2010. Specificities ranged from 34.0 to 98.0{\%} (median 86{\%}) in 2008, and 54.0 to 100{\%} (median 88{\%}) in 2010. There was no difference in sensitivity between grader levels, but first-level graders had a significantly lower specificity than level-two and level-three graders. In 2008, one centre had a lower sensitivity but higher specificity than the majority of centres. Following the feedback from the first round, overall agreement improved in 2010 and there were no longer any significant differences between centres. Conclusions: A useful educational tool has been developed for image grading external quality assurance.",
keywords = "diabetic retinopathy, quality assurance, retinal screening, sensitivity, specificity",
author = "Goatman, {K A} and S Philip and Fleming, {A D} and Harvey, {R D} and Swa, {K K} and C Styles and M Black and G Sell and N Lee and Sharp, {P F} and Olson, {J A}",
note = "Journal compilation {\circledC} 2011 Diabetes UK.",
year = "2012",
month = "6",
doi = "10.1111/j.1464-5491.2011.03504.x",
language = "English",
volume = "29",
pages = "776--783",
journal = "Diabetic Medicine",
issn = "0742-3071",
publisher = "Wiley",
number = "6",

}

TY - JOUR

T1 - External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme

AU - Goatman, K A

AU - Philip, S

AU - Fleming, A D

AU - Harvey, R D

AU - Swa, K K

AU - Styles, C

AU - Black, M

AU - Sell, G

AU - Lee, N

AU - Sharp, P F

AU - Olson, J A

N1 - Journal compilation © 2011 Diabetes UK.

PY - 2012/6

Y1 - 2012/6

N2 - Aims: To develop and evaluate an image grading external quality assurance system for the Scottish Diabetic Retinopathy Screening Programme. Method: A web-based image grading system was developed which closely matches the current Scottish national screening software. Two rounds of external quality assurance were run in autumn 2008 and spring 2010, each time using the same 100 images. Graders were compared with a consensus standard derived from the top-level graders' results. After the first round, the centre lead clinicians and top-level graders reviewed the results and drew up guidance notes for the second round. Results: Grader sensitivities ranged from 60.0 to 100% (median 92.5%) in 2008, and from 62.5 to 100% (median 92.5%) in 2010. Specificities ranged from 34.0 to 98.0% (median 86%) in 2008, and 54.0 to 100% (median 88%) in 2010. There was no difference in sensitivity between grader levels, but first-level graders had a significantly lower specificity than level-two and level-three graders. In 2008, one centre had a lower sensitivity but higher specificity than the majority of centres. Following the feedback from the first round, overall agreement improved in 2010 and there were no longer any significant differences between centres. Conclusions: A useful educational tool has been developed for image grading external quality assurance.

AB - Aims: To develop and evaluate an image grading external quality assurance system for the Scottish Diabetic Retinopathy Screening Programme. Method: A web-based image grading system was developed which closely matches the current Scottish national screening software. Two rounds of external quality assurance were run in autumn 2008 and spring 2010, each time using the same 100 images. Graders were compared with a consensus standard derived from the top-level graders' results. After the first round, the centre lead clinicians and top-level graders reviewed the results and drew up guidance notes for the second round. Results: Grader sensitivities ranged from 60.0 to 100% (median 92.5%) in 2008, and from 62.5 to 100% (median 92.5%) in 2010. Specificities ranged from 34.0 to 98.0% (median 86%) in 2008, and 54.0 to 100% (median 88%) in 2010. There was no difference in sensitivity between grader levels, but first-level graders had a significantly lower specificity than level-two and level-three graders. In 2008, one centre had a lower sensitivity but higher specificity than the majority of centres. Following the feedback from the first round, overall agreement improved in 2010 and there were no longer any significant differences between centres. Conclusions: A useful educational tool has been developed for image grading external quality assurance.

KW - diabetic retinopathy

KW - quality assurance

KW - retinal screening

KW - sensitivity

KW - specificity

U2 - 10.1111/j.1464-5491.2011.03504.x

DO - 10.1111/j.1464-5491.2011.03504.x

M3 - Article

C2 - 22023553

VL - 29

SP - 776

EP - 783

JO - Diabetic Medicine

JF - Diabetic Medicine

SN - 0742-3071

IS - 6

ER -