Error Analysis of ToTTo Table-to-Text Neural NLG Models

Barkavi Sundararajan, Somayajulu Sripada, Ehud Reiter

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

3 Downloads (Pure)

Abstract

We report error analysis of outputs from seven Table-to-Text generation models fine-tuned on ToTTo, an open-domain English language dataset. A manual error annotation of a subset of outputs (a total of 5,278 sentences) belonging to the topic of Politics generated by these seven models has been carried out. Our error annotation focused on eight categories of errors. The error analysis shows that more than 45% of sentences from each of the seven models have been error-free. It uncovered some of the specific classes of errors such as WORD errors that are the dominant errors in all the seven models, NAME and NUMBER errors are more committed by two of the GeM benchmark models, whereas DATE-DIMENSION and OTHER category of errors are more common in our Table-to-Text models.
Original languageEnglish
Title of host publicationProceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)
Place of PublicationAbu Dhabi, United Arab Emirates (Hybrid)
PublisherAssociation for Computational Linguistics
Pages456-470
Number of pages15
Publication statusPublished - 1 Dec 2022

Fingerprint

Dive into the research topics of 'Error Analysis of ToTTo Table-to-Text Neural NLG Models'. Together they form a unique fingerprint.

Cite this