Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

Tim Rapley (Corresponding Author), Melissa Girling, Frances S Mair, Elizabeth Murray, Shaun Treweek, Elaine McColl, Ian Nicholas Steen, Carl R May, Tracy L. Finch

Research output: Contribution to journalArticle

5 Citations (Scopus)
4 Downloads (Pure)

Abstract

Background
Understanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.

Methods
An iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.

Results
We initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: ‘not relevant at this stage’, ‘not relevant to my role’ and ‘not relevant to this intervention’ and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items.

Conclusion
The process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants’ experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.
Original languageEnglish
Article number133
JournalBMC Medical Research Methodology
Volume18
DOIs
Publication statusPublished - 15 Nov 2018

Fingerprint

Education
Research Personnel
Interviews
Surveys and Questionnaires

Keywords

  • normalization process theory
  • NPT
  • implementation process
  • questionnaire
  • instrument development
  • complex Interventions
  • NoMAD

Cite this

Improving the normalization of complex interventions : part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). / Rapley, Tim (Corresponding Author); Girling, Melissa; Mair, Frances S; Murray, Elizabeth; Treweek, Shaun; McColl, Elaine; Steen, Ian Nicholas ; May, Carl R; Finch, Tracy L.

In: BMC Medical Research Methodology, Vol. 18, 133, 15.11.2018.

Research output: Contribution to journalArticle

Rapley, Tim ; Girling, Melissa ; Mair, Frances S ; Murray, Elizabeth ; Treweek, Shaun ; McColl, Elaine ; Steen, Ian Nicholas ; May, Carl R ; Finch, Tracy L. / Improving the normalization of complex interventions : part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). In: BMC Medical Research Methodology. 2018 ; Vol. 18.
@article{292be18637cd468f956f4af88eeb16e3,
title = "Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)",
abstract = "BackgroundUnderstanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.MethodsAn iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.ResultsWe initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: ‘not relevant at this stage’, ‘not relevant to my role’ and ‘not relevant to this intervention’ and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100{\%} completion rate of items.ConclusionThe process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants’ experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.",
keywords = "normalization process theory, NPT, implementation process, questionnaire, instrument development, complex Interventions, NoMAD",
author = "Tim Rapley and Melissa Girling and Mair, {Frances S} and Elizabeth Murray and Shaun Treweek and Elaine McColl and Steen, {Ian Nicholas} and May, {Carl R} and Finch, {Tracy L.}",
note = "Funding This study is funded by the Economic and Social Research Council Study [Grant Number RES-062-23-3274] which is gratefully acknowledged. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates. Elizabeth Murray receives funding from the NIHR School of Primary Care Research and the NIHR Collaboration for Leadership in Applied Health Research and Care North Thames. Availability of data and materials The datasets generated and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.",
year = "2018",
month = "11",
day = "15",
doi = "10.1186/s12874-018-0590-y",
language = "English",
volume = "18",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Improving the normalization of complex interventions

T2 - part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

AU - Rapley, Tim

AU - Girling, Melissa

AU - Mair, Frances S

AU - Murray, Elizabeth

AU - Treweek, Shaun

AU - McColl, Elaine

AU - Steen, Ian Nicholas

AU - May, Carl R

AU - Finch, Tracy L.

N1 - Funding This study is funded by the Economic and Social Research Council Study [Grant Number RES-062-23-3274] which is gratefully acknowledged. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates. Elizabeth Murray receives funding from the NIHR School of Primary Care Research and the NIHR Collaboration for Leadership in Applied Health Research and Care North Thames. Availability of data and materials The datasets generated and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.

PY - 2018/11/15

Y1 - 2018/11/15

N2 - BackgroundUnderstanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.MethodsAn iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.ResultsWe initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: ‘not relevant at this stage’, ‘not relevant to my role’ and ‘not relevant to this intervention’ and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items.ConclusionThe process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants’ experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.

AB - BackgroundUnderstanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.MethodsAn iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.ResultsWe initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: ‘not relevant at this stage’, ‘not relevant to my role’ and ‘not relevant to this intervention’ and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items.ConclusionThe process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants’ experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.

KW - normalization process theory

KW - NPT

KW - implementation process

KW - questionnaire

KW - instrument development

KW - complex Interventions

KW - NoMAD

U2 - 10.1186/s12874-018-0590-y

DO - 10.1186/s12874-018-0590-y

M3 - Article

VL - 18

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

M1 - 133

ER -