How noisy is lexical decision?

Kevin Diependaele, Marc Brysbaert, Peter Neri

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Lexical decision is one of the most frequently used tasks in word recognition research. Theoretical conclusions are typically derived from a linear model on the reaction times (RTs) of correct word trials only (e.g., linear regression and ANOVA). Although these models estimate random measurement error for RTs, considering only correct trials implicitly assumes that word/non-word categorizations are without noise: words receive a yes-response because they have been recognized, and they receive a no-response when they are not known. Hence, when participants are presented with the same stimuli on two separate occasions, they are expected to give the same response. We demonstrate that this not true and that responses in a lexical decision task suffer from inconsistency in participants' response choice, meaning that RTs of "correct" word responses include RTs of trials on which participants did not recognize the stimulus. We obtained estimates of this internal noise using established methods from sensory psychophysics (Burgess and Colborne, 1988). The results show similar noise values as in typical psychophysical signal detection experiments when sensitivity and response bias are taken into account (Neri, 2010). These estimates imply that, with an optimal choice model, only 83-91% of the response choices can be explained (i.e., can be used to derive theoretical conclusions). For word responses, word frequencies below 10 per million yield alarmingly low percentages of consistent responses (near 50%). The same analysis can be applied to RTs, yielding noise estimates about three times higher. Correspondingly, the estimated amount of consistent trial-level variance in RTs is only 8%. These figures are especially relevant given the recent popularity of trial-level lexical decision models using the linear mixed-effects approach (e.g., Baayen et al., 2008).

Original languageEnglish
Article number348
JournalFrontiers in Psychology
Volume3
DOIs
Publication statusPublished - 24 Sep 2012

Fingerprint

Reaction Time
Noise
Linear Models
Psychophysics
Analysis of Variance
Research

Keywords

  • internal noise
  • lexical decision
  • signal detection
  • megastudies
  • lexicon projects

Cite this

Diependaele, K., Brysbaert, M., & Neri, P. (2012). How noisy is lexical decision? Frontiers in Psychology, 3, [348]. https://doi.org/10.3389/fpsyg.2012.00348

How noisy is lexical decision? / Diependaele, Kevin; Brysbaert, Marc; Neri, Peter.

In: Frontiers in Psychology, Vol. 3, 348, 24.09.2012.

Research output: Contribution to journalArticle

Diependaele, K, Brysbaert, M & Neri, P 2012, 'How noisy is lexical decision?', Frontiers in Psychology, vol. 3, 348. https://doi.org/10.3389/fpsyg.2012.00348
Diependaele, Kevin ; Brysbaert, Marc ; Neri, Peter. / How noisy is lexical decision?. In: Frontiers in Psychology. 2012 ; Vol. 3.
@article{37b27885d82b4eeeaa250ec1ae5b5b08,
title = "How noisy is lexical decision?",
abstract = "Lexical decision is one of the most frequently used tasks in word recognition research. Theoretical conclusions are typically derived from a linear model on the reaction times (RTs) of correct word trials only (e.g., linear regression and ANOVA). Although these models estimate random measurement error for RTs, considering only correct trials implicitly assumes that word/non-word categorizations are without noise: words receive a yes-response because they have been recognized, and they receive a no-response when they are not known. Hence, when participants are presented with the same stimuli on two separate occasions, they are expected to give the same response. We demonstrate that this not true and that responses in a lexical decision task suffer from inconsistency in participants' response choice, meaning that RTs of {"}correct{"} word responses include RTs of trials on which participants did not recognize the stimulus. We obtained estimates of this internal noise using established methods from sensory psychophysics (Burgess and Colborne, 1988). The results show similar noise values as in typical psychophysical signal detection experiments when sensitivity and response bias are taken into account (Neri, 2010). These estimates imply that, with an optimal choice model, only 83-91{\%} of the response choices can be explained (i.e., can be used to derive theoretical conclusions). For word responses, word frequencies below 10 per million yield alarmingly low percentages of consistent responses (near 50{\%}). The same analysis can be applied to RTs, yielding noise estimates about three times higher. Correspondingly, the estimated amount of consistent trial-level variance in RTs is only 8{\%}. These figures are especially relevant given the recent popularity of trial-level lexical decision models using the linear mixed-effects approach (e.g., Baayen et al., 2008).",
keywords = "internal noise, lexical decision, signal detection, megastudies, lexicon projects",
author = "Kevin Diependaele and Marc Brysbaert and Peter Neri",
note = "PMID: 23015793 [PubMed] PMCID: PMC3449292 Free PMC Article",
year = "2012",
month = "9",
day = "24",
doi = "10.3389/fpsyg.2012.00348",
language = "English",
volume = "3",
journal = "Frontiers in Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - How noisy is lexical decision?

AU - Diependaele, Kevin

AU - Brysbaert, Marc

AU - Neri, Peter

N1 - PMID: 23015793 [PubMed] PMCID: PMC3449292 Free PMC Article

PY - 2012/9/24

Y1 - 2012/9/24

N2 - Lexical decision is one of the most frequently used tasks in word recognition research. Theoretical conclusions are typically derived from a linear model on the reaction times (RTs) of correct word trials only (e.g., linear regression and ANOVA). Although these models estimate random measurement error for RTs, considering only correct trials implicitly assumes that word/non-word categorizations are without noise: words receive a yes-response because they have been recognized, and they receive a no-response when they are not known. Hence, when participants are presented with the same stimuli on two separate occasions, they are expected to give the same response. We demonstrate that this not true and that responses in a lexical decision task suffer from inconsistency in participants' response choice, meaning that RTs of "correct" word responses include RTs of trials on which participants did not recognize the stimulus. We obtained estimates of this internal noise using established methods from sensory psychophysics (Burgess and Colborne, 1988). The results show similar noise values as in typical psychophysical signal detection experiments when sensitivity and response bias are taken into account (Neri, 2010). These estimates imply that, with an optimal choice model, only 83-91% of the response choices can be explained (i.e., can be used to derive theoretical conclusions). For word responses, word frequencies below 10 per million yield alarmingly low percentages of consistent responses (near 50%). The same analysis can be applied to RTs, yielding noise estimates about three times higher. Correspondingly, the estimated amount of consistent trial-level variance in RTs is only 8%. These figures are especially relevant given the recent popularity of trial-level lexical decision models using the linear mixed-effects approach (e.g., Baayen et al., 2008).

AB - Lexical decision is one of the most frequently used tasks in word recognition research. Theoretical conclusions are typically derived from a linear model on the reaction times (RTs) of correct word trials only (e.g., linear regression and ANOVA). Although these models estimate random measurement error for RTs, considering only correct trials implicitly assumes that word/non-word categorizations are without noise: words receive a yes-response because they have been recognized, and they receive a no-response when they are not known. Hence, when participants are presented with the same stimuli on two separate occasions, they are expected to give the same response. We demonstrate that this not true and that responses in a lexical decision task suffer from inconsistency in participants' response choice, meaning that RTs of "correct" word responses include RTs of trials on which participants did not recognize the stimulus. We obtained estimates of this internal noise using established methods from sensory psychophysics (Burgess and Colborne, 1988). The results show similar noise values as in typical psychophysical signal detection experiments when sensitivity and response bias are taken into account (Neri, 2010). These estimates imply that, with an optimal choice model, only 83-91% of the response choices can be explained (i.e., can be used to derive theoretical conclusions). For word responses, word frequencies below 10 per million yield alarmingly low percentages of consistent responses (near 50%). The same analysis can be applied to RTs, yielding noise estimates about three times higher. Correspondingly, the estimated amount of consistent trial-level variance in RTs is only 8%. These figures are especially relevant given the recent popularity of trial-level lexical decision models using the linear mixed-effects approach (e.g., Baayen et al., 2008).

KW - internal noise

KW - lexical decision

KW - signal detection

KW - megastudies

KW - lexicon projects

U2 - 10.3389/fpsyg.2012.00348

DO - 10.3389/fpsyg.2012.00348

M3 - Article

C2 - 23015793

VL - 3

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 348

ER -