Automatically Predicting Quiz Difficulty Level Using Similarity Measures

Chenghua Lin; Dong  Liu; Wei Pang; Edward Apeh

doi:10.1145/2815833.2815842

Automatically Predicting Quiz Difficulty Level Using Similarity Measures

Chenghua Lin, Dong Liu, Wei Pang, Edward Apeh

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

5 Citations (Scopus)

1 Downloads (Pure)

Abstract

In this paper, we present a semi-automatic system (Sherlock) for quiz generation using Linked Data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its ability to control the difficulty level of the generated quizzes. We cast the problem of perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed similarity measure outperforms four strong baselines in both the pilot evaluation using a synthetic gold standard as well as with human evaluation, giving more than 47% gain in clustering accuracy over the baselines.

Original language	English
Title of host publication	Proceedings of The 8th International Conference on Knowledge Capture (K-Cap)
Publisher	ACM
Pages	1-8
Number of pages	8
ISBN (Print)	978-1-4503-3849-3
DOIs	https://doi.org/10.1145/2815833.2815842
Publication status	Published - 2015
Event	K-CAP 2015 - The 8th International Conference on Knowledge Capture - USA, New York, United States Duration: 7 Oct 2015 → 10 Oct 2015

Conference

Conference	K-CAP 2015 - The 8th International Conference on Knowledge Capture
Country/Territory	United States
City	New York
Period	7/10/15 → 10/10/15

Access to Document

10.1145/2815833.2815842Licence: Unspecified

Cite this

@inproceedings{0b88aa5f05ac4de1a3f14acb6d5d7c0f,

title = "Automatically Predicting Quiz Difficulty Level Using Similarity Measures",

abstract = "In this paper, we present a semi-automatic system (Sherlock) for quiz generation using Linked Data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its ability to control the difficulty level of the generated quizzes. We cast the problem of perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed similarity measure outperforms four strong baselines in both the pilot evaluation using a synthetic gold standard as well as with human evaluation, giving more than 47% gain in clustering accuracy over the baselines.",

author = "Chenghua Lin and Dong Liu and Wei Pang and Edward Apeh",

note = "ACKNOWLEDGMENTS This work is supported by the BBC Connected Studio programme the award made by the RCUK Digital Economy theme to the dot.rural; K-CAP 2015 - The 8th International Conference on Knowledge Capture ; Conference date: 07-10-2015 Through 10-10-2015",

year = "2015",

doi = "10.1145/2815833.2815842",

language = "English",

isbn = "978-1-4503-3849-3",

pages = "1--8",

booktitle = "Proceedings of The 8th International Conference on Knowledge Capture (K-Cap)",

publisher = "ACM",

}

TY - GEN

T1 - Automatically Predicting Quiz Difficulty Level Using Similarity Measures

AU - Lin, Chenghua

AU - Liu, Dong

AU - Pang, Wei

AU - Apeh, Edward

N1 - ACKNOWLEDGMENTS This work is supported by the BBC Connected Studio programme the award made by the RCUK Digital Economy theme to the dot.rural

PY - 2015

Y1 - 2015

N2 - In this paper, we present a semi-automatic system (Sherlock) for quiz generation using Linked Data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its ability to control the difficulty level of the generated quizzes. We cast the problem of perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed similarity measure outperforms four strong baselines in both the pilot evaluation using a synthetic gold standard as well as with human evaluation, giving more than 47% gain in clustering accuracy over the baselines.

AB - In this paper, we present a semi-automatic system (Sherlock) for quiz generation using Linked Data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its ability to control the difficulty level of the generated quizzes. We cast the problem of perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed similarity measure outperforms four strong baselines in both the pilot evaluation using a synthetic gold standard as well as with human evaluation, giving more than 47% gain in clustering accuracy over the baselines.

U2 - 10.1145/2815833.2815842

DO - 10.1145/2815833.2815842

M3 - Published conference contribution

SN - 978-1-4503-3849-3

SP - 1

EP - 8

BT - Proceedings of The 8th International Conference on Knowledge Capture (K-Cap)

PB - ACM

T2 - K-CAP 2015 - The 8th International Conference on Knowledge Capture

Y2 - 7 October 2015 through 10 October 2015

ER -

Automatically Predicting Quiz Difficulty Level Using Similarity Measures

Abstract

Conference

Access to Document

Fingerprint

Cite this