Abstract
Bar exams provide are a key watershed by which legal professionals demonstrate their knowledge of the law and its application. Passing the bar entitles one to practice the law in a given jurisdiction. The bar provides a excellent benchmark for the performance of legal information systems since passing the bar would arguably signal that the system has acquired key aspects of legal reason on a par with a human lawyer. The paper presents a first experiment with actual bar exam material, treating it as a form of textual entailment from the question to an answer. The Gold Standard is provided by the company which provides the bar exam. The paper reports a textual entailment student on a US bar exam, running ‘out of the box’ the Excitement Open Platform for textual entailment. The results and evaluation show that the tool can identify wrong answers (non-entailment) with a high F1 score, it performs poorly in identifying the correct answer (entailment). The results provide a baseline performance measure against which to evaluate future improvements. The reasons for the poor performance are examined, and proposals are made to augment the tool in the future.
Original language | English |
---|---|
Title of host publication | Legal Knowledge and Information Systems, JURIX 2015: 28th Annual Conference |
Publisher | IOS Press |
Pages | 179-180 |
Number of pages | 2 |
Volume | 279 |
ISBN (Electronic) | 9781614996088 |
DOIs | |
Publication status | Published - 2015 |
Event | 28th Annual International Conference on Legal Knowledge and Information Systems, JURIX 2015 - Braga, Portugal Duration: 10 Dec 2015 → 11 Dec 2015 |
Publication series
Name | Frontiers in Artificial Intelligence and Applications |
---|---|
Volume | 279 |
ISSN (Print) | 09226389 |
Conference
Conference | 28th Annual International Conference on Legal Knowledge and Information Systems, JURIX 2015 |
---|---|
Country/Territory | Portugal |
City | Braga |
Period | 10/12/15 → 11/12/15 |
Bibliographical note
The authors appreciate support from the NCBE and the Niger Delta University.Fingerprint
Dive into the research topics of 'Passing a USA national bar exam - A first experiment'. Together they form a unique fingerprint.Datasets
-
Passing a USA National Bar Exam: a First Corpus for Experimentation: Article Data
Wyner, A. Z. (Creator), Fawei, B. J. (Creator), Pan, J. Z. (Creator) & Edwards, P. (Data Manager), University of Aberdeen, 31 Mar 2016
Dataset