TY - JOUR
T1 - Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats
AU - Jerke, Julia
AU - Johann, David
AU - Rauhut, Heiko
AU - Thomas, Kathrin
N1 - Acknowledgements
We thank the anonymous reviewers for their concise comments and constructive feedback on our work. We also thank Nicola Babamov, Christophe Heger, Beatrice Schulz, Madeleine Siegel, Nikita Sorgatz, and Anastasiia Tcypina
for their assistance with the transcription of the interviews as well as Brian Cooper for proofreading. David Johann would like to thank the German Center for higher Education Research and Science (DZHW) for their support in providing
fund for interviewing and transcription. Kathrin Thomas was Research Associate at City, University of London, when the initial phase of the project started and would like to thank City for its support.
Heiko Rauhut acknowledges funding from the SNSF by the Starting Grant BSS-GIO 155981 “Social norms, cooperation and conflict in scientific collaborations”.
Authors in alphabetical order. Idea: David Johann. Design: Julia Jerke, David Johann, Heiko Rauhut, Kathrin Thomas. Interviews: Julia Jerke, David Johann, Kathrin Thomas. Coding and Analysis: Julia Jerke, David Johann, Heiko Rauhut, Kathrin Thomas. Text: Julia Jerke, David Johann, Heiko Rauhut, Kathrin Thomas. Final approval of the paper: Julia Jerke, David Johann, Heiko Rauhut, Kathrin
Thomas
PY - 2019/6/20
Y1 - 2019/6/20
N2 - Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.
AB - Misreporting of sensitive characteristics in surveys is a major concern among survey methodologists and social scientists across disciplines. Indirect question formats, such as the Item Count Technique (ICT) and the Randomized Response Techniques (RRT), including the Crosswise Model (CM) and the Triangular Model (TM), have been developed to protect respondents’ privacy by design to elicit more truthful answers. These methods have also been praised to produce more valid estimates than direct questions. However, recent research has revealed a number of problems, such as the occurrence of false negatives, false positives, and dependencies on socioeconomic characteristics, indicating that at least some respondents may still cheat or lie when asked indirectly. This article systematically investigates (1) how well respondents comprehend and (2) to what extent they trust the ICT, CM and TM. We conducted cognitive interviews with academics across disciplines, investigating how respondents perceive, think about and answer questions on academic misconduct using these indirect methods. The results indicate that most respondents comprehend the basic instructions, but many fail to understand the logic and principles of these techniques. Furthermore, the findings suggest that comprehension and honest self-reports are unrelated, thus violating core assumptions about the effectiveness of these techniques.
KW - Item Count Technique
KW - Crosswise Model
KW - Triangular Model
KW - Cognitive Interviews
KW - Academic Misconduct
U2 - 10.18148/srm/2019.v13i3.7453
DO - 10.18148/srm/2019.v13i3.7453
M3 - Article
VL - 13
SP - 319
EP - 351
JO - Survey Research Methods
JF - Survey Research Methods
IS - 3
ER -