Evidence for Communicative Compensation in Debt Advice with Reduced Multimodality

Nicole Andelic, Aidan Feeney, Gary McKeown

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.
Original languageEnglish
Title of host publicationICMI '19 2019 International Conference on Multimodal Interaction
EditorsWen Gao, Helen Mei Ling Meng, Matthew Turk, Susan R Fussell, Bjorn Schuller, Yale Song, Kai Yu
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Pages210-219
Number of pages10
ISBN (Electronic)9781450368605
ISBN (Print)978-1-4503-6860-5
DOIs
Publication statusPublished - 14 Oct 2019
Event21st ACM International Conference on Multimodal Interaction
- Suzhou, Jiangsu, China
Duration: 14 Oct 201918 Oct 2019

Conference

Conference21st ACM International Conference on Multimodal Interaction
Abbreviated titleICMI
CountryChina
CitySuzhou, Jiangsu
Period14/10/1918/10/19

    Fingerprint

Keywords

  • Advice-giving
  • Affective computing
  • Arousal
  • Valence

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Computer Science Applications

Cite this

Andelic, N., Feeney, A., & McKeown, G. (2019). Evidence for Communicative Compensation in Debt Advice with Reduced Multimodality. In W. Gao, H. M. Ling Meng, M. Turk, S. R. Fussell, B. Schuller, Y. Song, & K. Yu (Eds.), ICMI '19 2019 International Conference on Multimodal Interaction (pp. 210-219). New York: Association for Computing Machinery (ACM). https://doi.org/10.1145/3340555.3353757