Reliability of Identification of Behavior Change Techniques in Intervention Descriptions

Charles Abraham, Caroline E Wood, Marie Johnston, Jill Francis, Wendy Hardeman, Michelle Richardson, Susan Michie

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.

METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.

RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.

CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

Original languageEnglish
Pages (from-to)885-900
Number of pages16
JournalAnnals of Behavioral Medicine
Volume49
Issue number6
Early online date20 Aug 2015
DOIs
Publication statusPublished - Dec 2015

Fingerprint

Reproducibility of Results

Keywords

  • behavior change
  • intervention
  • behavior change technique
  • taxonomy
  • inter-coder reliability
  • test-retest reliability

Cite this

Reliability of Identification of Behavior Change Techniques in Intervention Descriptions. / Abraham, Charles; Wood, Caroline E; Johnston, Marie; Francis, Jill; Hardeman, Wendy; Richardson, Michelle; Michie, Susan.

In: Annals of Behavioral Medicine, Vol. 49, No. 6, 12.2015, p. 885-900.

Research output: Contribution to journalArticle

Abraham, C, Wood, CE, Johnston, M, Francis, J, Hardeman, W, Richardson, M & Michie, S 2015, 'Reliability of Identification of Behavior Change Techniques in Intervention Descriptions', Annals of Behavioral Medicine, vol. 49, no. 6, pp. 885-900. https://doi.org/10.1007/s12160-015-9727-y
Abraham, Charles ; Wood, Caroline E ; Johnston, Marie ; Francis, Jill ; Hardeman, Wendy ; Richardson, Michelle ; Michie, Susan. / Reliability of Identification of Behavior Change Techniques in Intervention Descriptions. In: Annals of Behavioral Medicine. 2015 ; Vol. 49, No. 6. pp. 885-900.
@article{9741bf817fdf4c829fe030760182964b,
title = "Reliability of Identification of Behavior Change Techniques in Intervention Descriptions",
abstract = "PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the {"}Behavior Change Technique Taxonomy version 1{"} (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 {\%}) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 {\%}) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 {\%}) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.",
keywords = "behavior change , intervention, behavior change technique, taxonomy, inter-coder reliability, test-retest reliability",
author = "Charles Abraham and Wood, {Caroline E} and Marie Johnston and Jill Francis and Wendy Hardeman and Michelle Richardson and Susan Michie",
note = "Acknowledgements The work was carried out as part of the BCT Taxonomy project funded by the Medical Research Council [G0901474/1]. We are grateful to Kate Sheals (KS) for help with coding and to Professor Tania Huedo-Medina of the Department of Statistics, University of Connecticut for sharing her AC1 calculator with us. We acknowledge funding from the UK National Institute for Health Research (NIHR) and the Peninsula Collaboration for Leadership in Applied Health Research and Care",
year = "2015",
month = "12",
doi = "10.1007/s12160-015-9727-y",
language = "English",
volume = "49",
pages = "885--900",
journal = "Annals of Behavioral Medicine",
issn = "0883-6612",
publisher = "Springer New York",
number = "6",

}

TY - JOUR

T1 - Reliability of Identification of Behavior Change Techniques in Intervention Descriptions

AU - Abraham, Charles

AU - Wood, Caroline E

AU - Johnston, Marie

AU - Francis, Jill

AU - Hardeman, Wendy

AU - Richardson, Michelle

AU - Michie, Susan

N1 - Acknowledgements The work was carried out as part of the BCT Taxonomy project funded by the Medical Research Council [G0901474/1]. We are grateful to Kate Sheals (KS) for help with coding and to Professor Tania Huedo-Medina of the Department of Statistics, University of Connecticut for sharing her AC1 calculator with us. We acknowledge funding from the UK National Institute for Health Research (NIHR) and the Peninsula Collaboration for Leadership in Applied Health Research and Care

PY - 2015/12

Y1 - 2015/12

N2 - PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

AB - PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability.CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

KW - behavior change

KW - intervention

KW - behavior change technique

KW - taxonomy

KW - inter-coder reliability

KW - test-retest reliability

U2 - 10.1007/s12160-015-9727-y

DO - 10.1007/s12160-015-9727-y

M3 - Article

C2 - 26290002

VL - 49

SP - 885

EP - 900

JO - Annals of Behavioral Medicine

JF - Annals of Behavioral Medicine

SN - 0883-6612

IS - 6

ER -