Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy

G. S. Scotland*, P. McNamee, A. D. Fleming, K. A. Goatman, S. Philip, G. J. Prescott, P. F. Sharp, G. J. Williams, W. Wykes, G. P. Leese, J. A. Olson, Scottish Diabetic Retinopathy Clin

*Corresponding author for this work

Research output: Contribution to journalArticle

26 Citations (Scopus)

Abstract

Aims To assess the cost-effectiveness of an improved automated grading algorithm for diabetic retinopathy against a previously described algorithm, and in comparison with manual grading.

Methods Efficacy of the alternative algorithms was assessed using a reference graded set of images from three screening centres in Scotland (1253 cases with observable/referable retinopathy and 6333 individuals with mild or no retinopathy). Screening outcomes and grading and diagnosis costs were modelled for a cohort of 180 000 people, with prevalence of referable retinopathy at 4%. Algorithm (b), which combines image quality assessment with detection algorithms for microaneurysms (MA), blot haemorrhages and exudates, was compared with a simpler algorithm (a) (using image quality assessment and MA/dot haemorrhage (DH) detection), and the current practice of manual grading.

Results Compared with algorithm (a), algorithm (b) would identify an additional 113 cases of referable retinopathy for an incremental cost of 68 pound per additional case. Compared with manual grading, automated grading would be expected to identify between 54 and 123 fewer referable cases, for a grading cost saving between 3834 pound and 1727 pound per case missed. Extrapolation modelling over a 20-year time horizon suggests manual grading would cost between 25 pound 676 and 267 pound 115 per additional quality adjusted life year gained.

Conclusions Algorithm (b) is more cost-effective than the algorithm based on quality assessment and MA/DH detection. With respect to the value of introducing automated detection systems into screening programmes, automated grading operates within the recommended national standards in Scotland and is likely to be considered a cost-effective alternative to manual disease/no disease grading.

Original languageEnglish
Pages (from-to)712-719
Number of pages8
JournalBritish Journal of Ophthalmology
Volume94
Issue number6
Early online date3 Dec 2009
DOIs
Publication statusPublished - Jun 2010

Fingerprint

Diabetic Retinopathy
Costs and Cost Analysis
Scotland
Hemorrhage
Quality-Adjusted Life Years
Exudates and Transudates
Cost-Benefit Analysis

Keywords

  • Algorithms
  • Cost-Benefit Analysis
  • Decision Trees
  • Diabetic Retinopathy
  • Diagnosis, Computer-Assisted
  • Diagnostic Techniques, Ophthalmological
  • Exudates and Transudates
  • Health Care Costs
  • Humans
  • Image Interpretation, Computer-Assisted
  • Mass Screening
  • Quality-Adjusted Life Years
  • Retinal Hemorrhage
  • Scotland
  • Severity of Illness Index
  • Screening Program
  • Disease

Cite this

Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy. / Scotland, G. S.; McNamee, P.; Fleming, A. D.; Goatman, K. A.; Philip, S.; Prescott, G. J.; Sharp, P. F.; Williams, G. J.; Wykes, W.; Leese, G. P.; Olson, J. A.; Scottish Diabetic Retinopathy Clin.

In: British Journal of Ophthalmology, Vol. 94, No. 6, 06.2010, p. 712-719.

Research output: Contribution to journalArticle

Scotland, GS, McNamee, P, Fleming, AD, Goatman, KA, Philip, S, Prescott, GJ, Sharp, PF, Williams, GJ, Wykes, W, Leese, GP, Olson, JA & Scottish Diabetic Retinopathy Clin 2010, 'Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy', British Journal of Ophthalmology, vol. 94, no. 6, pp. 712-719. https://doi.org/10.1136/bjo.2008.151126
Scotland, G. S. ; McNamee, P. ; Fleming, A. D. ; Goatman, K. A. ; Philip, S. ; Prescott, G. J. ; Sharp, P. F. ; Williams, G. J. ; Wykes, W. ; Leese, G. P. ; Olson, J. A. ; Scottish Diabetic Retinopathy Clin. / Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy. In: British Journal of Ophthalmology. 2010 ; Vol. 94, No. 6. pp. 712-719.
@article{c9cf3a01832d406780d58fb8cf2bf892,
title = "Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy",
abstract = "Aims To assess the cost-effectiveness of an improved automated grading algorithm for diabetic retinopathy against a previously described algorithm, and in comparison with manual grading.Methods Efficacy of the alternative algorithms was assessed using a reference graded set of images from three screening centres in Scotland (1253 cases with observable/referable retinopathy and 6333 individuals with mild or no retinopathy). Screening outcomes and grading and diagnosis costs were modelled for a cohort of 180 000 people, with prevalence of referable retinopathy at 4{\%}. Algorithm (b), which combines image quality assessment with detection algorithms for microaneurysms (MA), blot haemorrhages and exudates, was compared with a simpler algorithm (a) (using image quality assessment and MA/dot haemorrhage (DH) detection), and the current practice of manual grading.Results Compared with algorithm (a), algorithm (b) would identify an additional 113 cases of referable retinopathy for an incremental cost of 68 pound per additional case. Compared with manual grading, automated grading would be expected to identify between 54 and 123 fewer referable cases, for a grading cost saving between 3834 pound and 1727 pound per case missed. Extrapolation modelling over a 20-year time horizon suggests manual grading would cost between 25 pound 676 and 267 pound 115 per additional quality adjusted life year gained.Conclusions Algorithm (b) is more cost-effective than the algorithm based on quality assessment and MA/DH detection. With respect to the value of introducing automated detection systems into screening programmes, automated grading operates within the recommended national standards in Scotland and is likely to be considered a cost-effective alternative to manual disease/no disease grading.",
keywords = "Algorithms, Cost-Benefit Analysis, Decision Trees, Diabetic Retinopathy, Diagnosis, Computer-Assisted, Diagnostic Techniques, Ophthalmological, Exudates and Transudates, Health Care Costs, Humans, Image Interpretation, Computer-Assisted, Mass Screening, Quality-Adjusted Life Years, Retinal Hemorrhage, Scotland, Severity of Illness Index, Screening Program, Disease",
author = "Scotland, {G. S.} and P. McNamee and Fleming, {A. D.} and Goatman, {K. A.} and S. Philip and Prescott, {G. J.} and Sharp, {P. F.} and Williams, {G. J.} and W. Wykes and Leese, {G. P.} and Olson, {J. A.} and {Scottish Diabetic Retinopathy Clin}",
year = "2010",
month = "6",
doi = "10.1136/bjo.2008.151126",
language = "English",
volume = "94",
pages = "712--719",
journal = "British Journal of Ophthalmology",
issn = "0007-1161",
publisher = "BMJ Publishing Group",
number = "6",

}

TY - JOUR

T1 - Costs and consequences of automated algorithms versus manual grading for the detection of referable diabetic retinopathy

AU - Scotland, G. S.

AU - McNamee, P.

AU - Fleming, A. D.

AU - Goatman, K. A.

AU - Philip, S.

AU - Prescott, G. J.

AU - Sharp, P. F.

AU - Williams, G. J.

AU - Wykes, W.

AU - Leese, G. P.

AU - Olson, J. A.

AU - Scottish Diabetic Retinopathy Clin

PY - 2010/6

Y1 - 2010/6

N2 - Aims To assess the cost-effectiveness of an improved automated grading algorithm for diabetic retinopathy against a previously described algorithm, and in comparison with manual grading.Methods Efficacy of the alternative algorithms was assessed using a reference graded set of images from three screening centres in Scotland (1253 cases with observable/referable retinopathy and 6333 individuals with mild or no retinopathy). Screening outcomes and grading and diagnosis costs were modelled for a cohort of 180 000 people, with prevalence of referable retinopathy at 4%. Algorithm (b), which combines image quality assessment with detection algorithms for microaneurysms (MA), blot haemorrhages and exudates, was compared with a simpler algorithm (a) (using image quality assessment and MA/dot haemorrhage (DH) detection), and the current practice of manual grading.Results Compared with algorithm (a), algorithm (b) would identify an additional 113 cases of referable retinopathy for an incremental cost of 68 pound per additional case. Compared with manual grading, automated grading would be expected to identify between 54 and 123 fewer referable cases, for a grading cost saving between 3834 pound and 1727 pound per case missed. Extrapolation modelling over a 20-year time horizon suggests manual grading would cost between 25 pound 676 and 267 pound 115 per additional quality adjusted life year gained.Conclusions Algorithm (b) is more cost-effective than the algorithm based on quality assessment and MA/DH detection. With respect to the value of introducing automated detection systems into screening programmes, automated grading operates within the recommended national standards in Scotland and is likely to be considered a cost-effective alternative to manual disease/no disease grading.

AB - Aims To assess the cost-effectiveness of an improved automated grading algorithm for diabetic retinopathy against a previously described algorithm, and in comparison with manual grading.Methods Efficacy of the alternative algorithms was assessed using a reference graded set of images from three screening centres in Scotland (1253 cases with observable/referable retinopathy and 6333 individuals with mild or no retinopathy). Screening outcomes and grading and diagnosis costs were modelled for a cohort of 180 000 people, with prevalence of referable retinopathy at 4%. Algorithm (b), which combines image quality assessment with detection algorithms for microaneurysms (MA), blot haemorrhages and exudates, was compared with a simpler algorithm (a) (using image quality assessment and MA/dot haemorrhage (DH) detection), and the current practice of manual grading.Results Compared with algorithm (a), algorithm (b) would identify an additional 113 cases of referable retinopathy for an incremental cost of 68 pound per additional case. Compared with manual grading, automated grading would be expected to identify between 54 and 123 fewer referable cases, for a grading cost saving between 3834 pound and 1727 pound per case missed. Extrapolation modelling over a 20-year time horizon suggests manual grading would cost between 25 pound 676 and 267 pound 115 per additional quality adjusted life year gained.Conclusions Algorithm (b) is more cost-effective than the algorithm based on quality assessment and MA/DH detection. With respect to the value of introducing automated detection systems into screening programmes, automated grading operates within the recommended national standards in Scotland and is likely to be considered a cost-effective alternative to manual disease/no disease grading.

KW - Algorithms

KW - Cost-Benefit Analysis

KW - Decision Trees

KW - Diabetic Retinopathy

KW - Diagnosis, Computer-Assisted

KW - Diagnostic Techniques, Ophthalmological

KW - Exudates and Transudates

KW - Health Care Costs

KW - Humans

KW - Image Interpretation, Computer-Assisted

KW - Mass Screening

KW - Quality-Adjusted Life Years

KW - Retinal Hemorrhage

KW - Scotland

KW - Severity of Illness Index

KW - Screening Program

KW - Disease

U2 - 10.1136/bjo.2008.151126

DO - 10.1136/bjo.2008.151126

M3 - Article

VL - 94

SP - 712

EP - 719

JO - British Journal of Ophthalmology

JF - British Journal of Ophthalmology

SN - 0007-1161

IS - 6

ER -