Sherlock: a Semi-Automatic Framework for Quiz Generation Using a Hybrid Semantic Similarity Measure

Chenghua Lin, Dong Liu, Wei Pang, Zhe Wang

Research output: Contribution to journalArticle

9 Citations (Scopus)
7 Downloads (Pure)


In this paper, we present a semi-automatic system (Sherlock) for quiz generation using linked data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its generic framework for domain-independent quiz generation as well as in the ability of controlling the difficulty level of the generated quizzes. Difficulty scaling is non-trivial, and it is fundamentally related to cognitive science. We approach the problem with a new angle by perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed semantic similarity measure outperforms four strong baselines with more than 47 % gain in clustering accuracy. In addition, we discovered in the human quiz test that the model accuracy indeed shows a strong correlation with the pairwise quiz similarity.
Original languageEnglish
Pages (from-to)667-679
Number of pages13
JournalCognitive Computation
Issue number6
Early online date4 Aug 2015
Publication statusPublished - Dec 2015



  • quiz generation
  • linked data
  • RDF
  • educational games
  • semantic similarity
  • text analytics

Cite this