Abstract
Despite two high profile series of challenges devoted to question answering technologies there remains no formal study into the representativeness that question corpora bear to real end-user inputs. We examine the corpora used presently and historically in the TREC and QALD challenges in juxtaposition with two more from natural sources and identify a degree of disjointedness between the two. We analyse these differences in depth before discussing a candidate approach to question corpora generation and provide a juxtaposition on its own representativeness. We conclude that these artificial corpora have good overall coverage of grammatical structures but the distribution is skewed, meaning performance measures may be inaccurate.
Original language | English |
---|---|
Title of host publication | Information Access Evaluation. Multilinguality, Multimodality, and Interaction |
Subtitle of host publication | CLEF 2014. |
Editors | Evangelos Kanoulas, Mihai Lupu, Paul Clough, Mark Sanderson, Mark Hall, Allan Hanbury, Elaine Toms |
Publisher | Springer-Verlag |
Pages | 1-6 |
Number of pages | 6 |
ISBN (Electronic) | 9783319113821 |
ISBN (Print) | 9783319113814 |
DOIs | |
Publication status | Published - 2014 |
Event | 5th International Conference of the CLEF Initiative, CLEF 2014 - Sheffield, United Kingdom Duration: 15 Sep 2014 → 18 Sep 2014 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 8685 LNCS |
ISSN (Print) | 03029743 |
ISSN (Electronic) | 16113349 |
Conference
Conference | 5th International Conference of the CLEF Initiative, CLEF 2014 |
---|---|
Country | United Kingdom |
City | Sheffield |
Period | 15/09/14 → 18/09/14 |
Fingerprint Dive into the research topics of 'Making test corpora for question answering more representative'. Together they form a unique fingerprint.
Profiles
-
Andrew Starkey
- Engineering, Engineering - Senior Lecturer
- Engineering (Research Theme)
- Centre for Applied Dynamics Research (CADR)
Person: Academic