Synchronization of Speech and Gesture: Evidence for Interaction in Action

Mingyuan Chu, Peter Hagoort

Research output: Contribution to journalArticle

11 Citations (Scopus)
10 Downloads (Pure)

Abstract


Language and action systems are highly interlinked. A critical piece of evidence is that speech and its accompanying gestures are tightly synchronized. Five experiments were conducted to test 2 hypotheses about the synchronization of speech and gesture. According to the interactive view, there is continuous information exchange between the gesture and speech systems, during both their planning and execution phases. According to the ballistic view, information exchange occurs only during the planning phases of gesture and speech, but the 2 systems become independent once their execution has been initiated. In all experiments, participants were required to point to and/or name a light that had just lit up. Virtual reality and motion tracking technologies were used to disrupt their gesture or speech execution. Participants delayed their speech onset when their gesture was disrupted. They did so even when their gesture was disrupted at its late phase and even when they received only the kinesthetic feedback of their gesture. Also, participants prolonged their gestures when their speech was disrupted. These findings support the interactive view and add new constraints on models of speech and gesture production. (PsycINFO Database Record (c) 2014 APA, all rights reserved)
Original languageEnglish
Pages (from-to)1726-1741
Number of pages16
JournalJournal of Experimental Psychology: General
Volume143
Issue number4
DOIs
Publication statusPublished - Aug 2014

    Fingerprint

Keywords

  • gesture-speech synchronization
  • interactive view
  • pointing gesture
  • virtual reality
  • forward model

Cite this