Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation: An Illustration and Tutorial

Martin Hecht*, Christian Gische, Daniel Vogel, Steffen Zitzmann

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multi-purpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.

Original languageEnglish
Number of pages11
JournalStructural Equation Modeling
Early online date24 Sep 2019
DOIs
Publication statusE-pub ahead of print - 24 Sep 2019

Fingerprint

Efficient Estimation
Nuisance Parameter
Bayesian Estimation
Structural Equation Modeling
Intercept
Bayesian Model
Scatter
Model
Covariance matrix
Null
Choose
Simulation Study
Model-based
Software
Nuisance parameter
Bayesian estimation
Tutorial
Sampling
instruction
simulation

Keywords

  • Bayesian analysis
  • multi-level modeling
  • nuisance parameters
  • run time optimization
  • sampler monitoring
  • structural equation modeling
  • TOO
  • MODEL

ASJC Scopus subject areas

  • Economics, Econometrics and Finance(all)
  • Decision Sciences(all)
  • Sociology and Political Science
  • Modelling and Simulation

Cite this

Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation : An Illustration and Tutorial. / Hecht, Martin; Gische, Christian; Vogel, Daniel; Zitzmann, Steffen.

In: Structural Equation Modeling, 24.09.2019.

Research output: Contribution to journalArticle

@article{004bf43e5fff4a0189322bd7b07e2161,
title = "Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation: An Illustration and Tutorial",
abstract = "Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multi-purpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.",
keywords = "Bayesian analysis, multi-level modeling, nuisance parameters, run time optimization, sampler monitoring, structural equation modeling, TOO, MODEL",
author = "Martin Hecht and Christian Gische and Daniel Vogel and Steffen Zitzmann",
note = "Funding We acknowledge support by the Open Access Publication Fund of Humboldt-Universit{\"a}t zu Berlin.",
year = "2019",
month = "9",
day = "24",
doi = "10.1080/10705511.2019.1647432",
language = "English",
journal = "Structural Equation Modeling",
issn = "1070-5511",

}

TY - JOUR

T1 - Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation

T2 - An Illustration and Tutorial

AU - Hecht, Martin

AU - Gische, Christian

AU - Vogel, Daniel

AU - Zitzmann, Steffen

N1 - Funding We acknowledge support by the Open Access Publication Fund of Humboldt-Universität zu Berlin.

PY - 2019/9/24

Y1 - 2019/9/24

N2 - Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multi-purpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.

AB - Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multi-purpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.

KW - Bayesian analysis

KW - multi-level modeling

KW - nuisance parameters

KW - run time optimization

KW - sampler monitoring

KW - structural equation modeling

KW - TOO

KW - MODEL

UR - http://www.scopus.com/inward/record.url?scp=85073991630&partnerID=8YFLogxK

U2 - 10.1080/10705511.2019.1647432

DO - 10.1080/10705511.2019.1647432

M3 - Article

AN - SCOPUS:85073991630

JO - Structural Equation Modeling

JF - Structural Equation Modeling

SN - 1070-5511

ER -