An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma

Anne Burden, Nicolas Roche, Cristiana Miglio, Elizabeth V Hillyer, Dirkje S Postma, Ron MC Herings, Jetty A Overbeek, Javaria Mona Khalid, Daniela van Eickels, David B Price

Research output: Contribution to journalArticle

13 Downloads (Pure)

Abstract

Background: Cohort matching and regression modeling are used in observational studies to control for confounding factors when estimating treatment effects. Our objective was to evaluate exact matching and propensity score methods by applying them in a 1-year pre–post historical database study to investigate asthma-related outcomes by treatment. Methods: We drew on longitudinal medical record data in the PHARMO database for asthma patients prescribed the treatments to be compared (ciclesonide and fine-particle inhaled corticosteroid [ICS]). Propensity score methods that we evaluated were propensity score matching (PSM) using two different algorithms, the inverse probability of treatment weighting (IPTW), covariate adjustment using the propensity score, and propensity score stratification. We defined balance, using standardized differences, as differences of <10% between cohorts. Results: Of 4064 eligible patients, 1382 (34%) were prescribed ciclesonide and 2682 (66%) fine-particle ICS. The IPTW and propensity score-based methods retained more patients (96%–100%) than exact matching (90%); exact matching selected less severe patients. Standardized differences were >10% for four variables in the exact-matched dataset and <10% for both PSM algorithms and the weighted pseudo-dataset used in the IPTW method. With all methods, ciclesonide was associated with better 1-year asthma-related outcomes, at one-third the prescribed dose, than fine-particle ICS; results varied slightly by method, but direction and statistical significance remained the same. Conclusion: We found that each method has its particular strengths, and we recommend at least two methods be applied for each matched cohort study to evaluate the robustness of the findings. Balance diagnostics should be applied with all methods to check the balance of confounders between treatment cohorts. If exact matching is used, the calculation of a propensity score could be useful to identify variables that require balancing, thereby informing the choice of matching criteria together with clinical considerations.
Original languageEnglish
JournalPragmatic and Observational Research
Volume8
DOIs
Publication statusPublished - 22 Mar 2017

Fingerprint

Propensity Score
Adrenal Cortex Hormones
Asthma
Therapeutics
Databases
Medical Records
Observational Studies
Cohort Studies

Keywords

  • asthma
  • exact matching
  • propensity score
  • observational

Cite this

An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma. / Burden, Anne; Roche, Nicolas; Miglio, Cristiana; Hillyer, Elizabeth V; Postma, Dirkje S; Herings, Ron MC ; Overbeek, Jetty A; Khalid, Javaria Mona ; van Eickels, Daniela ; Price, David B.

In: Pragmatic and Observational Research, Vol. 8, 22.03.2017.

Research output: Contribution to journalArticle

Burden, Anne ; Roche, Nicolas ; Miglio, Cristiana ; Hillyer, Elizabeth V ; Postma, Dirkje S ; Herings, Ron MC ; Overbeek, Jetty A ; Khalid, Javaria Mona ; van Eickels, Daniela ; Price, David B. / An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma. In: Pragmatic and Observational Research. 2017 ; Vol. 8.
@article{3c267b1e39fb4f9bb8a045501aef6699,
title = "An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma",
abstract = "Background: Cohort matching and regression modeling are used in observational studies to control for confounding factors when estimating treatment effects. Our objective was to evaluate exact matching and propensity score methods by applying them in a 1-year pre–post historical database study to investigate asthma-related outcomes by treatment. Methods: We drew on longitudinal medical record data in the PHARMO database for asthma patients prescribed the treatments to be compared (ciclesonide and fine-particle inhaled corticosteroid [ICS]). Propensity score methods that we evaluated were propensity score matching (PSM) using two different algorithms, the inverse probability of treatment weighting (IPTW), covariate adjustment using the propensity score, and propensity score stratification. We defined balance, using standardized differences, as differences of <10{\%} between cohorts. Results: Of 4064 eligible patients, 1382 (34{\%}) were prescribed ciclesonide and 2682 (66{\%}) fine-particle ICS. The IPTW and propensity score-based methods retained more patients (96{\%}–100{\%}) than exact matching (90{\%}); exact matching selected less severe patients. Standardized differences were >10{\%} for four variables in the exact-matched dataset and <10{\%} for both PSM algorithms and the weighted pseudo-dataset used in the IPTW method. With all methods, ciclesonide was associated with better 1-year asthma-related outcomes, at one-third the prescribed dose, than fine-particle ICS; results varied slightly by method, but direction and statistical significance remained the same. Conclusion: We found that each method has its particular strengths, and we recommend at least two methods be applied for each matched cohort study to evaluate the robustness of the findings. Balance diagnostics should be applied with all methods to check the balance of confounders between treatment cohorts. If exact matching is used, the calculation of a propensity score could be useful to identify variables that require balancing, thereby informing the choice of matching criteria together with clinical considerations.",
keywords = "asthma, exact matching, propensity score, observational",
author = "Anne Burden and Nicolas Roche and Cristiana Miglio and Hillyer, {Elizabeth V} and Postma, {Dirkje S} and Herings, {Ron MC} and Overbeek, {Jetty A} and Khalid, {Javaria Mona} and {van Eickels}, Daniela and Price, {David B}",
year = "2017",
month = "3",
day = "22",
doi = "10.2147/POR.S122563",
language = "English",
volume = "8",
journal = "Pragmatic and Observational Research",
issn = "1179-7266",
publisher = "Dove Medical Press Ltd.",

}

TY - JOUR

T1 - An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma

AU - Burden, Anne

AU - Roche, Nicolas

AU - Miglio, Cristiana

AU - Hillyer, Elizabeth V

AU - Postma, Dirkje S

AU - Herings, Ron MC

AU - Overbeek, Jetty A

AU - Khalid, Javaria Mona

AU - van Eickels, Daniela

AU - Price, David B

PY - 2017/3/22

Y1 - 2017/3/22

N2 - Background: Cohort matching and regression modeling are used in observational studies to control for confounding factors when estimating treatment effects. Our objective was to evaluate exact matching and propensity score methods by applying them in a 1-year pre–post historical database study to investigate asthma-related outcomes by treatment. Methods: We drew on longitudinal medical record data in the PHARMO database for asthma patients prescribed the treatments to be compared (ciclesonide and fine-particle inhaled corticosteroid [ICS]). Propensity score methods that we evaluated were propensity score matching (PSM) using two different algorithms, the inverse probability of treatment weighting (IPTW), covariate adjustment using the propensity score, and propensity score stratification. We defined balance, using standardized differences, as differences of <10% between cohorts. Results: Of 4064 eligible patients, 1382 (34%) were prescribed ciclesonide and 2682 (66%) fine-particle ICS. The IPTW and propensity score-based methods retained more patients (96%–100%) than exact matching (90%); exact matching selected less severe patients. Standardized differences were >10% for four variables in the exact-matched dataset and <10% for both PSM algorithms and the weighted pseudo-dataset used in the IPTW method. With all methods, ciclesonide was associated with better 1-year asthma-related outcomes, at one-third the prescribed dose, than fine-particle ICS; results varied slightly by method, but direction and statistical significance remained the same. Conclusion: We found that each method has its particular strengths, and we recommend at least two methods be applied for each matched cohort study to evaluate the robustness of the findings. Balance diagnostics should be applied with all methods to check the balance of confounders between treatment cohorts. If exact matching is used, the calculation of a propensity score could be useful to identify variables that require balancing, thereby informing the choice of matching criteria together with clinical considerations.

AB - Background: Cohort matching and regression modeling are used in observational studies to control for confounding factors when estimating treatment effects. Our objective was to evaluate exact matching and propensity score methods by applying them in a 1-year pre–post historical database study to investigate asthma-related outcomes by treatment. Methods: We drew on longitudinal medical record data in the PHARMO database for asthma patients prescribed the treatments to be compared (ciclesonide and fine-particle inhaled corticosteroid [ICS]). Propensity score methods that we evaluated were propensity score matching (PSM) using two different algorithms, the inverse probability of treatment weighting (IPTW), covariate adjustment using the propensity score, and propensity score stratification. We defined balance, using standardized differences, as differences of <10% between cohorts. Results: Of 4064 eligible patients, 1382 (34%) were prescribed ciclesonide and 2682 (66%) fine-particle ICS. The IPTW and propensity score-based methods retained more patients (96%–100%) than exact matching (90%); exact matching selected less severe patients. Standardized differences were >10% for four variables in the exact-matched dataset and <10% for both PSM algorithms and the weighted pseudo-dataset used in the IPTW method. With all methods, ciclesonide was associated with better 1-year asthma-related outcomes, at one-third the prescribed dose, than fine-particle ICS; results varied slightly by method, but direction and statistical significance remained the same. Conclusion: We found that each method has its particular strengths, and we recommend at least two methods be applied for each matched cohort study to evaluate the robustness of the findings. Balance diagnostics should be applied with all methods to check the balance of confounders between treatment cohorts. If exact matching is used, the calculation of a propensity score could be useful to identify variables that require balancing, thereby informing the choice of matching criteria together with clinical considerations.

KW - asthma

KW - exact matching

KW - propensity score

KW - observational

U2 - 10.2147/POR.S122563

DO - 10.2147/POR.S122563

M3 - Article

VL - 8

JO - Pragmatic and Observational Research

JF - Pragmatic and Observational Research

SN - 1179-7266

ER -