Pre-validation methods for developing a patient reported outcome instrument

Maria E Prior, Jemaima Che Hamzah, Jillian J Francis, Craig R Ramsay, Mayret M Castillo, Susan E Campbell, Augusto Azuara-Blanco, Jennifer M Burr

Research output: Contribution to journalArticle

8 Citations (Scopus)
3 Downloads (Pure)

Abstract

BACKGROUND: Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology. METHODS: The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains. RESULTS: The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items. CONCLUSIONS: This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.
Original languageEnglish
Pages (from-to)112
JournalBMC Medical Research Methodology
Volume11
Issue number1
DOIs
Publication statusPublished - 9 Aug 2011

Fingerprint

Glaucoma
Health Services Needs and Demand
Research
Randomized Controlled Trials
Outcome Assessment (Health Care)
Surveys and Questionnaires
Patient Reported Outcome Measures
Health

Cite this

Prior, M. E., Che Hamzah, J., Francis, J. J., Ramsay, C. R., Castillo, M. M., Campbell, S. E., ... Burr, J. M. (2011). Pre-validation methods for developing a patient reported outcome instrument. BMC Medical Research Methodology, 11(1), 112. https://doi.org/10.1186/1471-2288-11-112

Pre-validation methods for developing a patient reported outcome instrument. / Prior, Maria E; Che Hamzah, Jemaima; Francis, Jillian J; Ramsay, Craig R; Castillo, Mayret M; Campbell, Susan E; Azuara-Blanco, Augusto; Burr, Jennifer M.

In: BMC Medical Research Methodology, Vol. 11, No. 1, 09.08.2011, p. 112.

Research output: Contribution to journalArticle

Prior, ME, Che Hamzah, J, Francis, JJ, Ramsay, CR, Castillo, MM, Campbell, SE, Azuara-Blanco, A & Burr, JM 2011, 'Pre-validation methods for developing a patient reported outcome instrument', BMC Medical Research Methodology, vol. 11, no. 1, pp. 112. https://doi.org/10.1186/1471-2288-11-112
Prior, Maria E ; Che Hamzah, Jemaima ; Francis, Jillian J ; Ramsay, Craig R ; Castillo, Mayret M ; Campbell, Susan E ; Azuara-Blanco, Augusto ; Burr, Jennifer M. / Pre-validation methods for developing a patient reported outcome instrument. In: BMC Medical Research Methodology. 2011 ; Vol. 11, No. 1. pp. 112.
@article{a742b85cf77240c39b285dd82fc1cd50,
title = "Pre-validation methods for developing a patient reported outcome instrument",
abstract = "BACKGROUND: Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology. METHODS: The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains. RESULTS: The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items. CONCLUSIONS: This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.",
author = "Prior, {Maria E} and {Che Hamzah}, Jemaima and Francis, {Jillian J} and Ramsay, {Craig R} and Castillo, {Mayret M} and Campbell, {Susan E} and Augusto Azuara-Blanco and Burr, {Jennifer M}",
year = "2011",
month = "8",
day = "9",
doi = "10.1186/1471-2288-11-112",
language = "English",
volume = "11",
pages = "112",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Pre-validation methods for developing a patient reported outcome instrument

AU - Prior, Maria E

AU - Che Hamzah, Jemaima

AU - Francis, Jillian J

AU - Ramsay, Craig R

AU - Castillo, Mayret M

AU - Campbell, Susan E

AU - Azuara-Blanco, Augusto

AU - Burr, Jennifer M

PY - 2011/8/9

Y1 - 2011/8/9

N2 - BACKGROUND: Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology. METHODS: The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains. RESULTS: The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items. CONCLUSIONS: This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.

AB - BACKGROUND: Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology. METHODS: The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains. RESULTS: The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items. CONCLUSIONS: This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.

U2 - 10.1186/1471-2288-11-112

DO - 10.1186/1471-2288-11-112

M3 - Article

VL - 11

SP - 112

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

ER -