Personal Storytelling

Using Natural Language Generation for Children with Complex Communication Needs, in the Wild...

Nava Tintarev, Ehud Reiter, Rolf Black, Annalu Waller, Joe Reddington

Research output: Contribution to journalArticle

9 Citations (Scopus)
8 Downloads (Pure)

Abstract

This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibility
study of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for nlg in augmented and alternative communication are also discussed.
Original languageEnglish
Pages (from-to)1-16
Number of pages16
JournalInternational Journal of Human Computer Studies
Volume92-93
Early online date16 Apr 2016
DOIs
Publication statusPublished - 30 Sep 2016

Fingerprint

communication
Communication
language
school
recording
narrative
Sensors
parents
evaluation
experience
time

Keywords

  • assistive technology
  • augmented and alternative communication
  • natural language generation
  • communication aids
  • user-centered design

Cite this

Personal Storytelling : Using Natural Language Generation for Children with Complex Communication Needs, in the Wild... . / Tintarev, Nava; Reiter, Ehud; Black, Rolf; Waller, Annalu; Reddington, Joe .

In: International Journal of Human Computer Studies, Vol. 92-93, 30.09.2016, p. 1-16.

Research output: Contribution to journalArticle

@article{581631f1e88b4ab3aa5c9e390b39c6ec,
title = "Personal Storytelling: Using Natural Language Generation for Children with Complex Communication Needs, in the Wild...",
abstract = "This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibilitystudy of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for nlg in augmented and alternative communication are also discussed.",
keywords = "assistive technology, augmented and alternative communication, natural language generation, communication aids, user-centered design",
author = "Nava Tintarev and Ehud Reiter and Rolf Black and Annalu Waller and Joe Reddington",
note = "Acknowledgements The authors thank the children, their parents and school staff, who participated in this research, and who so willingly gave us their time, help and support. They also thank Steven Knox and Alan Clelland for their work on programming the mobile phone application. Additional thanks to DynaVox Inc. for supplying the Vmax communication devices to run our system on and Sensory Software Ltd for supplying us with their AAC software. This research was supported by the Research Council UKs Digittal Economy Programme and EPSRC (Grant numbers EP/F067151/1, EP/F066880/1, EP/E011764/1, EP/H022376/1, and EP/H022570 /1).",
year = "2016",
month = "9",
day = "30",
doi = "10.1016/j.ijhcs.2016.04.005",
language = "English",
volume = "92-93",
pages = "1--16",
journal = "International Journal of Human Computer Studies",
issn = "1071-5819",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Personal Storytelling

T2 - Using Natural Language Generation for Children with Complex Communication Needs, in the Wild...

AU - Tintarev, Nava

AU - Reiter, Ehud

AU - Black, Rolf

AU - Waller, Annalu

AU - Reddington, Joe

N1 - Acknowledgements The authors thank the children, their parents and school staff, who participated in this research, and who so willingly gave us their time, help and support. They also thank Steven Knox and Alan Clelland for their work on programming the mobile phone application. Additional thanks to DynaVox Inc. for supplying the Vmax communication devices to run our system on and Sensory Software Ltd for supplying us with their AAC software. This research was supported by the Research Council UKs Digittal Economy Programme and EPSRC (Grant numbers EP/F067151/1, EP/F066880/1, EP/E011764/1, EP/H022376/1, and EP/H022570 /1).

PY - 2016/9/30

Y1 - 2016/9/30

N2 - This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibilitystudy of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for nlg in augmented and alternative communication are also discussed.

AB - This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibilitystudy of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for nlg in augmented and alternative communication are also discussed.

KW - assistive technology

KW - augmented and alternative communication

KW - natural language generation

KW - communication aids

KW - user-centered design

U2 - 10.1016/j.ijhcs.2016.04.005

DO - 10.1016/j.ijhcs.2016.04.005

M3 - Article

VL - 92-93

SP - 1

EP - 16

JO - International Journal of Human Computer Studies

JF - International Journal of Human Computer Studies

SN - 1071-5819

ER -