External quality assurance for image grading in the Scottish Diabetic Retinopathy Screening Programme

Aims: To develop and evaluate an image grading external quality assurance system for the Scottish Diabetic Retinopathy Screening Programme. Method: A web-based image grading system was developed which closely matches the current Scottish national screening software. Two rounds of external quality assurance were run in autumn 2008 and spring 2010, each time using the same 100 images. Graders were compared with a consensus standard derived from the top-level graders' results. After the first round, the centre lead clinicians and top-level graders reviewed the results and drew up guidance notes for the second round. Results: Grader sensitivities ranged from 60.0 to 100% (median 92.5%) in 2008, and from 62.5 to 100% (median 92.5%) in 2010. Specificities ranged from 34.0 to 98.0% (median 86%) in 2008, and 54.0 to 100% (median 88%) in 2010. There was no difference in sensitivity between grader levels, but first-level graders had a significantly lower specificity than level-two and level-three graders. In 2008, one centre had a lower sensitivity but higher specificity than the majority of centres. Following the feedback from the first round, overall agreement improved in 2010 and there were no longer any significant differences between centres. Conclusions: A useful educational tool has been developed for image grading external quality assurance.