Generating Expressions that Refer to Visible Objects

Margaret Mitchell, Kees Van Deemter, Ehud Reiter

Research output: Chapter in Book/Report/Conference proceedingConference contribution

23 Citations (Scopus)
5 Downloads (Pure)

Abstract

We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

Original languageEnglish
Title of host publicationProceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics
Subtitle of host publicationHuman Language Technologies
EditorsLucy Vanderwende, Hal Daumé III, Katrin Kirchhoff
PublisherAssociation for Computational Linguistics (ACL)
Pages1174-1184
Number of pages11
ISBN (Print)9781937284473
Publication statusPublished - Jun 2013
Event2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013 - Atlanta, United States
Duration: 9 Jun 201314 Jun 2013

Conference

Conference2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013
CountryUnited States
CityAtlanta
Period9/06/1314/06/13

Fingerprint

Computer vision
Visible
Color
evaluation
Referring Expressions
Graph
Evaluation
Computer Vision

ASJC Scopus subject areas

  • Language and Linguistics
  • Computer Science Applications
  • Linguistics and Language

Cite this

Mitchell, M., Van Deemter, K., & Reiter, E. (2013). Generating Expressions that Refer to Visible Objects. In L. Vanderwende, H. Daumé III, & K. Kirchhoff (Eds.), Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1174-1184). [N13-1137] Association for Computational Linguistics (ACL).

Generating Expressions that Refer to Visible Objects. / Mitchell, Margaret; Van Deemter, Kees; Reiter, Ehud.

Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ed. / Lucy Vanderwende; Hal Daumé III; Katrin Kirchhoff. Association for Computational Linguistics (ACL), 2013. p. 1174-1184 N13-1137.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mitchell, M, Van Deemter, K & Reiter, E 2013, Generating Expressions that Refer to Visible Objects. in L Vanderwende, H Daumé III & K Kirchhoff (eds), Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies., N13-1137, Association for Computational Linguistics (ACL), pp. 1174-1184, 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013, Atlanta, United States, 9/06/13.
Mitchell M, Van Deemter K, Reiter E. Generating Expressions that Refer to Visible Objects. In Vanderwende L, Daumé III H, Kirchhoff K, editors, Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics (ACL). 2013. p. 1174-1184. N13-1137
Mitchell, Margaret ; Van Deemter, Kees ; Reiter, Ehud. / Generating Expressions that Refer to Visible Objects. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. editor / Lucy Vanderwende ; Hal Daumé III ; Katrin Kirchhoff. Association for Computational Linguistics (ACL), 2013. pp. 1174-1184
@inproceedings{d9e9212b42334aa1a0f7113de3af68f1,
title = "Generating Expressions that Refer to Visible Objects",
abstract = "We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.",
author = "Margaret Mitchell and {Van Deemter}, Kees and Ehud Reiter",
note = "Funding for this research has been provided by SICSA and ORSAS. We thank the anonymous reviewers for useful comments on this paper.",
year = "2013",
month = "6",
language = "English",
isbn = "9781937284473",
pages = "1174--1184",
editor = "Lucy Vanderwende and {Daum{\'e} III}, Hal and Katrin Kirchhoff",
booktitle = "Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics",
publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - Generating Expressions that Refer to Visible Objects

AU - Mitchell, Margaret

AU - Van Deemter, Kees

AU - Reiter, Ehud

N1 - Funding for this research has been provided by SICSA and ORSAS. We thank the anonymous reviewers for useful comments on this paper.

PY - 2013/6

Y1 - 2013/6

N2 - We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

AB - We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

UR - http://www.scopus.com/inward/record.url?scp=84908171705&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781937284473

SP - 1174

EP - 1184

BT - Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics

A2 - Vanderwende, Lucy

A2 - Daumé III, Hal

A2 - Kirchhoff, Katrin

PB - Association for Computational Linguistics (ACL)

ER -