Discriminant content validity: A quantitative methodology for assessing content of theory-based measures, with illustrative applications

Marie Johnston, Diane Dixon, Jo Hart, Liz Glidewell, Carin Schroder, Beth Pollard

Research output: Contribution to journalArticle

28 Citations (Scopus)

Abstract

Objectives

In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies.

Methods

Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions.

Results

The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format.

Conclusions

The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in AppendixS1 can be used in item or measure selection.

Statement of contribution

What is already known on this subject?

There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner.

What does this study add?

The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct.

In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats.

AppendixS1 gives content validity indices for each item of each questionnaire investigated.

Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

Original languageEnglish
Pages (from-to)240-257
Number of pages18
JournalBritish Journal of Health Psychology
Volume19
Issue number2
Early online date15 Mar 2014
DOIs
Publication statusPublished - May 2014

Fingerprint

Cognition
Reproducibility of Results
Theoretical Models
TimeLine
Surveys and Questionnaires

Keywords

  • theory
  • psychometrics
  • measurement
  • content validity
  • questionnaires
  • validity
  • illness perception questionnaire
  • social cognitive theory
  • planned behavior
  • confounded measures
  • perceived control
  • past behavior
  • chronic pain
  • perspective
  • models
  • disability

Cite this

Discriminant content validity : A quantitative methodology for assessing content of theory-based measures, with illustrative applications. / Johnston, Marie; Dixon, Diane; Hart, Jo; Glidewell, Liz; Schroder, Carin; Pollard, Beth.

In: British Journal of Health Psychology, Vol. 19, No. 2, 05.2014, p. 240-257.

Research output: Contribution to journalArticle

@article{f46cdaed07a64f88b7ab5b76f0269713,
title = "Discriminant content validity: A quantitative methodology for assessing content of theory-based measures, with illustrative applications",
abstract = "ObjectivesIn studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies.MethodsDiscriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions.ResultsThe IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format.ConclusionsThe DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in AppendixS1 can be used in item or measure selection.Statement of contributionWhat is already known on this subject?There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner.What does this study add?The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct.In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats.AppendixS1 gives content validity indices for each item of each questionnaire investigated.Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.",
keywords = "theory, psychometrics, measurement, content validity, questionnaires, validity, illness perception questionnaire, social cognitive theory, planned behavior, confounded measures, perceived control, past behavior, chronic pain, perspective, models, disability",
author = "Marie Johnston and Diane Dixon and Jo Hart and Liz Glidewell and Carin Schroder and Beth Pollard",
note = "{\circledC} 2014 The British Psychological Society.",
year = "2014",
month = "5",
doi = "10.1111/bjhp.12095",
language = "English",
volume = "19",
pages = "240--257",
journal = "British Journal of Health Psychology",
issn = "1359-107X",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Discriminant content validity

T2 - A quantitative methodology for assessing content of theory-based measures, with illustrative applications

AU - Johnston, Marie

AU - Dixon, Diane

AU - Hart, Jo

AU - Glidewell, Liz

AU - Schroder, Carin

AU - Pollard, Beth

N1 - © 2014 The British Psychological Society.

PY - 2014/5

Y1 - 2014/5

N2 - ObjectivesIn studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies.MethodsDiscriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions.ResultsThe IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format.ConclusionsThe DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in AppendixS1 can be used in item or measure selection.Statement of contributionWhat is already known on this subject?There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner.What does this study add?The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct.In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats.AppendixS1 gives content validity indices for each item of each questionnaire investigated.Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

AB - ObjectivesIn studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies.MethodsDiscriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions.ResultsThe IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format.ConclusionsThe DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in AppendixS1 can be used in item or measure selection.Statement of contributionWhat is already known on this subject?There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner.What does this study add?The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct.In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats.AppendixS1 gives content validity indices for each item of each questionnaire investigated.Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

KW - theory

KW - psychometrics

KW - measurement

KW - content validity

KW - questionnaires

KW - validity

KW - illness perception questionnaire

KW - social cognitive theory

KW - planned behavior

KW - confounded measures

KW - perceived control

KW - past behavior

KW - chronic pain

KW - perspective

KW - models

KW - disability

U2 - 10.1111/bjhp.12095

DO - 10.1111/bjhp.12095

M3 - Article

VL - 19

SP - 240

EP - 257

JO - British Journal of Health Psychology

JF - British Journal of Health Psychology

SN - 1359-107X

IS - 2

ER -