Generation of referring expressions: assessing the incremental algorithm

Kees van Deemter, Albert Gatt, Ielka van der Sluis, Richard Power

Research output: Contribution to journalArticle

44 Citations (Scopus)

Abstract

A substantial amount of recent work in natural language generation has focused on the generation of ‘‘one-shot’’ referring expressions whose only aim is to identify a target referent. Dale and Reiter's Incremental Algorithm (IA) is often thought to be the best algorithm for maximizing the similarity to referring expressions produced by people. We test this hypothesis by eliciting referring expressions from human subjects and computing the similarity between the expressions elicited and the ones generated by algorithms. It turns out that the success of the IA depends substantially on the ‘‘preference order’’ (PO) employed by the IA, particularly in complex domains. While some POs cause the IA to produce referring expressions that are very similar to expressions produced by human subjects, others cause the IA to perform worse than its main competitors; moreover, it turns out to be difficult to predict the success of a PO on the basis of existing psycholinguistic findings or frequencies in corpora. We also examine the computational complexity of the algorithms in question and argue that there are no compelling reasons for preferring the IA over some of its main competitors on these grounds. We conclude that future research on the generation of referring expressions should explore alternatives to the IA, focusing on algorithms, inspired by the Greedy Algorithm, which do not work with a fixed PO.
Original languageEnglish
Pages (from-to)799-836
Number of pages37
JournalCognitive Science
Volume36
Issue number5
Early online date31 Oct 2011
DOIs
Publication statusPublished - Jul 2012

Fingerprint

Psycholinguistics
Computational complexity
Language

Keywords

  • TUNA project
  • generation/production of referring expression
  • evaluation metrics for generation algorithms
  • psycholinguistics
  • reference
  • incremental algorithm

Cite this

Generation of referring expressions : assessing the incremental algorithm. / van Deemter, Kees; Gatt, Albert; Sluis, Ielka van der; Power, Richard.

In: Cognitive Science, Vol. 36, No. 5, 07.2012, p. 799-836.

Research output: Contribution to journalArticle

van Deemter, Kees ; Gatt, Albert ; Sluis, Ielka van der ; Power, Richard. / Generation of referring expressions : assessing the incremental algorithm. In: Cognitive Science. 2012 ; Vol. 36, No. 5. pp. 799-836.
@article{02b156b7e8c6418e902c7e25fee6b501,
title = "Generation of referring expressions: assessing the incremental algorithm",
abstract = "A substantial amount of recent work in natural language generation has focused on the generation of ‘‘one-shot’’ referring expressions whose only aim is to identify a target referent. Dale and Reiter's Incremental Algorithm (IA) is often thought to be the best algorithm for maximizing the similarity to referring expressions produced by people. We test this hypothesis by eliciting referring expressions from human subjects and computing the similarity between the expressions elicited and the ones generated by algorithms. It turns out that the success of the IA depends substantially on the ‘‘preference order’’ (PO) employed by the IA, particularly in complex domains. While some POs cause the IA to produce referring expressions that are very similar to expressions produced by human subjects, others cause the IA to perform worse than its main competitors; moreover, it turns out to be difficult to predict the success of a PO on the basis of existing psycholinguistic findings or frequencies in corpora. We also examine the computational complexity of the algorithms in question and argue that there are no compelling reasons for preferring the IA over some of its main competitors on these grounds. We conclude that future research on the generation of referring expressions should explore alternatives to the IA, focusing on algorithms, inspired by the Greedy Algorithm, which do not work with a fixed PO.",
keywords = "TUNA project, generation/production of referring expression, evaluation metrics for generation algorithms, psycholinguistics, reference, incremental algorithm",
author = "{van Deemter}, Kees and Albert Gatt and Sluis, {Ielka van der} and Richard Power",
year = "2012",
month = "7",
doi = "10.1111/j.1551-6709.2011.01205.x",
language = "English",
volume = "36",
pages = "799--836",
journal = "Cognitive Science",
issn = "0364-0213",
publisher = "Wiley-Blackwell",
number = "5",

}

TY - JOUR

T1 - Generation of referring expressions

T2 - assessing the incremental algorithm

AU - van Deemter, Kees

AU - Gatt, Albert

AU - Sluis, Ielka van der

AU - Power, Richard

PY - 2012/7

Y1 - 2012/7

N2 - A substantial amount of recent work in natural language generation has focused on the generation of ‘‘one-shot’’ referring expressions whose only aim is to identify a target referent. Dale and Reiter's Incremental Algorithm (IA) is often thought to be the best algorithm for maximizing the similarity to referring expressions produced by people. We test this hypothesis by eliciting referring expressions from human subjects and computing the similarity between the expressions elicited and the ones generated by algorithms. It turns out that the success of the IA depends substantially on the ‘‘preference order’’ (PO) employed by the IA, particularly in complex domains. While some POs cause the IA to produce referring expressions that are very similar to expressions produced by human subjects, others cause the IA to perform worse than its main competitors; moreover, it turns out to be difficult to predict the success of a PO on the basis of existing psycholinguistic findings or frequencies in corpora. We also examine the computational complexity of the algorithms in question and argue that there are no compelling reasons for preferring the IA over some of its main competitors on these grounds. We conclude that future research on the generation of referring expressions should explore alternatives to the IA, focusing on algorithms, inspired by the Greedy Algorithm, which do not work with a fixed PO.

AB - A substantial amount of recent work in natural language generation has focused on the generation of ‘‘one-shot’’ referring expressions whose only aim is to identify a target referent. Dale and Reiter's Incremental Algorithm (IA) is often thought to be the best algorithm for maximizing the similarity to referring expressions produced by people. We test this hypothesis by eliciting referring expressions from human subjects and computing the similarity between the expressions elicited and the ones generated by algorithms. It turns out that the success of the IA depends substantially on the ‘‘preference order’’ (PO) employed by the IA, particularly in complex domains. While some POs cause the IA to produce referring expressions that are very similar to expressions produced by human subjects, others cause the IA to perform worse than its main competitors; moreover, it turns out to be difficult to predict the success of a PO on the basis of existing psycholinguistic findings or frequencies in corpora. We also examine the computational complexity of the algorithms in question and argue that there are no compelling reasons for preferring the IA over some of its main competitors on these grounds. We conclude that future research on the generation of referring expressions should explore alternatives to the IA, focusing on algorithms, inspired by the Greedy Algorithm, which do not work with a fixed PO.

KW - TUNA project

KW - generation/production of referring expression

KW - evaluation metrics for generation algorithms

KW - psycholinguistics

KW - reference

KW - incremental algorithm

U2 - 10.1111/j.1551-6709.2011.01205.x

DO - 10.1111/j.1551-6709.2011.01205.x

M3 - Article

VL - 36

SP - 799

EP - 836

JO - Cognitive Science

JF - Cognitive Science

SN - 0364-0213

IS - 5

ER -