Natural language generation for augmentative and assistive technologies

Many people are not able to communicate easily, because of diversity in their physical and/or intellectual abilities. For example, someone with cerebral palsy may not have enough control over their vocal tract and mouth to speak (or enough manual dexterity control to use sign language); and someone with autism may be physically capable, but unable to speak because of their cognitive profile.

Augmentative and alternative communication (AAC) is a sub-field of assistive technology focusing on tools to help people with major motor and/or cognitive impairments communicate better. Traditionally, AAC has used technology to provide speech output for non-speaking users, either by converting text to speech or speaking out pre-stored phrases. The initial focus of computer-based AAC systems was on giving users access to a broad set of symbols/pictures (by using some kind of menu or search system), and on using speech synthesis to automatically speak out phrases corresponding to the selected symbols. However, computers can in principle do much more than this.

In particular, since many AAC systems are intended to help users produce utterances, there seems to be a natural role for Natural Language Generation (NLG) to play, as it is after all a technology for producing language. However, the use of NLG in AAC is somewhat different from most uses of NLG. Since the goal of AAC is to help the user communicate, the NLG system must be used interactively, under the user’s control; we want to assist the user in communication, not replace the user with an automatic communicator. Also, since most human communication is social, NLG AAC systems often need to generate texts whose communicative goal is social interaction. Hence, NLG AAC systems are very different from systems that summarize information in a task-oriented context, which has been the focus of most NLG research for interactive systems.