Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats

Julia Jerke, David Johann, Heiko Rauhut, Kathrin Thomas

Research output: Contribution to journalArticle

Abstract

Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.
Original languageEnglish
Pages (from-to)319-351
Number of pages33
JournalSurvey Research Methods
Volume13
Issue number3
DOIs
Publication statusPublished - 2019

Fingerprint

social scientist
privacy
comprehension
instruction
interview

Keywords

  • Item Count Technique
  • Crosswise Model
  • Triangular Model
  • Cognitive Interviews
  • Academic Misconduct

Cite this

Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats. / Jerke, Julia; Johann, David; Rauhut, Heiko; Thomas, Kathrin.

In: Survey Research Methods, Vol. 13, No. 3, 2019, p. 319-351.

Research output: Contribution to journalArticle

@article{636855ac711240869a41e30f48bee72d,
title = "Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats",
abstract = "Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.",
keywords = "Item Count Technique, Crosswise Model, Triangular Model, Cognitive Interviews, Academic Misconduct",
author = "Julia Jerke and David Johann and Heiko Rauhut and Kathrin Thomas",
year = "2019",
doi = "10.18148/srm/2019.v13i3.7453",
language = "English",
volume = "13",
pages = "319--351",
journal = "Survey Research Methods",
number = "3",

}

TY - JOUR

T1 - Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats

AU - Jerke, Julia

AU - Johann, David

AU - Rauhut, Heiko

AU - Thomas, Kathrin

PY - 2019

Y1 - 2019

N2 - Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.

AB - Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.

KW - Item Count Technique

KW - Crosswise Model

KW - Triangular Model

KW - Cognitive Interviews

KW - Academic Misconduct

U2 - 10.18148/srm/2019.v13i3.7453

DO - 10.18148/srm/2019.v13i3.7453

M3 - Article

VL - 13

SP - 319

EP - 351

JO - Survey Research Methods

JF - Survey Research Methods

IS - 3

ER -