Evaluating algorithms for the generation of referring expressions

going beyond toy domains

Ielka van der Sluis, Albert Gatt, Kees van Deemter

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We describe a corpus-based evaluation methodology, applied to a number of classic algorithms in the generation of referring expressions. Following up on earlier work involving very simple domains, this paper deals with the issues associated with domains that contain ‘real-life’ objects of some complexity. Results indicate that state of the art algorithms perform very differently when applied to a complex domain. Moreover, if a version of the Incremental Algorithm is used then it becomes of huge importance to select a good preference order. These results should contribute to a growing debate on the evaluation of nlg systems, arguing in favour of carefully constructed balanced and semantically transparent corpora.
Original languageEnglish
Title of host publicationProceedings of Recent Advances in Natural Language Processing
Place of PublicationBorovets, Bulgaria
Number of pages7
Publication statusPublished - 2007

Keywords

  • generation of referring expressions
  • corpus-based evaluation of algorithms
  • Natural Language Generation

Cite this

Sluis, I. V. D., Gatt, A., & van Deemter, K. (2007). Evaluating algorithms for the generation of referring expressions: going beyond toy domains. In Proceedings of Recent Advances in Natural Language Processing Borovets, Bulgaria.

Evaluating algorithms for the generation of referring expressions : going beyond toy domains. / Sluis, Ielka van der; Gatt, Albert; van Deemter, Kees.

Proceedings of Recent Advances in Natural Language Processing. Borovets, Bulgaria, 2007.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sluis, IVD, Gatt, A & van Deemter, K 2007, Evaluating algorithms for the generation of referring expressions: going beyond toy domains. in Proceedings of Recent Advances in Natural Language Processing. Borovets, Bulgaria.
Sluis IVD, Gatt A, van Deemter K. Evaluating algorithms for the generation of referring expressions: going beyond toy domains. In Proceedings of Recent Advances in Natural Language Processing. Borovets, Bulgaria. 2007
Sluis, Ielka van der ; Gatt, Albert ; van Deemter, Kees. / Evaluating algorithms for the generation of referring expressions : going beyond toy domains. Proceedings of Recent Advances in Natural Language Processing. Borovets, Bulgaria, 2007.
@inproceedings{67193f7a8f26461f830e85452638ccd0,
title = "Evaluating algorithms for the generation of referring expressions: going beyond toy domains",
abstract = "We describe a corpus-based evaluation methodology, applied to a number of classic algorithms in the generation of referring expressions. Following up on earlier work involving very simple domains, this paper deals with the issues associated with domains that contain ‘real-life’ objects of some complexity. Results indicate that state of the art algorithms perform very differently when applied to a complex domain. Moreover, if a version of the Incremental Algorithm is used then it becomes of huge importance to select a good preference order. These results should contribute to a growing debate on the evaluation of nlg systems, arguing in favour of carefully constructed balanced and semantically transparent corpora.",
keywords = "generation of referring expressions, corpus-based evaluation of algorithms, Natural Language Generation",
author = "Sluis, {Ielka van der} and Albert Gatt and {van Deemter}, Kees",
year = "2007",
language = "English",
booktitle = "Proceedings of Recent Advances in Natural Language Processing",

}

TY - GEN

T1 - Evaluating algorithms for the generation of referring expressions

T2 - going beyond toy domains

AU - Sluis, Ielka van der

AU - Gatt, Albert

AU - van Deemter, Kees

PY - 2007

Y1 - 2007

N2 - We describe a corpus-based evaluation methodology, applied to a number of classic algorithms in the generation of referring expressions. Following up on earlier work involving very simple domains, this paper deals with the issues associated with domains that contain ‘real-life’ objects of some complexity. Results indicate that state of the art algorithms perform very differently when applied to a complex domain. Moreover, if a version of the Incremental Algorithm is used then it becomes of huge importance to select a good preference order. These results should contribute to a growing debate on the evaluation of nlg systems, arguing in favour of carefully constructed balanced and semantically transparent corpora.

AB - We describe a corpus-based evaluation methodology, applied to a number of classic algorithms in the generation of referring expressions. Following up on earlier work involving very simple domains, this paper deals with the issues associated with domains that contain ‘real-life’ objects of some complexity. Results indicate that state of the art algorithms perform very differently when applied to a complex domain. Moreover, if a version of the Incremental Algorithm is used then it becomes of huge importance to select a good preference order. These results should contribute to a growing debate on the evaluation of nlg systems, arguing in favour of carefully constructed balanced and semantically transparent corpora.

KW - generation of referring expressions

KW - corpus-based evaluation of algorithms

KW - Natural Language Generation

M3 - Conference contribution

BT - Proceedings of Recent Advances in Natural Language Processing

CY - Borovets, Bulgaria

ER -