Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We describe the methodological considerations that arose
over a series of experiments evaluating the eectiveness of explanations
for recommendations. In particular, we look at issues relating to: criteria,
metrics, product domain used, choice of materials, possible confound-
ing factors, and approximation of experience versus real experience. We
generalize the problems we found and the solutions that we applied to
adaptive systems. We illustrate the learned lessons with examples from
our previous work on adaptive systems (ranging from adaptive learning
to persuasive technologies).
Original languageEnglish
Title of host publicationProceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems
EditorsStephan Weibelzahl, Judith Masthoff, Alexandros Paramythis, Lex van Velsen
PublisherCEUR-WS
Pages54-63
Number of pages10
Publication statusPublished - Jun 2009
EventUCDEAS Workshop associated with UMAP - Trento, Italy
Duration: 22 Jun 200926 Jun 2009

Conference

ConferenceUCDEAS Workshop associated with UMAP
CountryItaly
CityTrento
Period22/06/0926/06/09

Fingerprint

Adaptive systems
Experiments

Cite this

Tintarev, N., & Masthoff, J. (2009). Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems. In S. Weibelzahl, J. Masthoff, A. Paramythis, & L. van Velsen (Eds.), Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems (pp. 54-63). CEUR-WS.

Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems. / Tintarev, Nava; Masthoff, Judith.

Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems. ed. / Stephan Weibelzahl; Judith Masthoff; Alexandros Paramythis; Lex van Velsen. CEUR-WS, 2009. p. 54-63.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tintarev, N & Masthoff, J 2009, Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems. in S Weibelzahl, J Masthoff, A Paramythis & L van Velsen (eds), Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems. CEUR-WS, pp. 54-63, UCDEAS Workshop associated with UMAP, Trento, Italy, 22/06/09.
Tintarev N, Masthoff J. Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems. In Weibelzahl S, Masthoff J, Paramythis A, van Velsen L, editors, Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems. CEUR-WS. 2009. p. 54-63
Tintarev, Nava ; Masthoff, Judith. / Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems. Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems. editor / Stephan Weibelzahl ; Judith Masthoff ; Alexandros Paramythis ; Lex van Velsen. CEUR-WS, 2009. pp. 54-63
@inproceedings{c6e163baeae8434ab56e16db003521f7,
title = "Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems",
abstract = "We describe the methodological considerations that arose over a series of experiments evaluating the eectiveness of explanations for recommendations. In particular, we look at issues relating to: criteria, metrics, product domain used, choice of materials, possible confound- ing factors, and approximation of experience versus real experience. We generalize the problems we found and the solutions that we applied to adaptive systems. We illustrate the learned lessons with examples from our previous work on adaptive systems (ranging from adaptive learning to persuasive technologies).",
author = "Nava Tintarev and Judith Masthoff",
year = "2009",
month = "6",
language = "English",
pages = "54--63",
editor = "Stephan Weibelzahl and Judith Masthoff and Alexandros Paramythis and {van Velsen}, Lex",
booktitle = "Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems",
publisher = "CEUR-WS",

}

TY - GEN

T1 - Evaluating recommender explanations: Problems experienced and lessons learned for the evaluation of adaptive systems

AU - Tintarev, Nava

AU - Masthoff, Judith

PY - 2009/6

Y1 - 2009/6

N2 - We describe the methodological considerations that arose over a series of experiments evaluating the eectiveness of explanations for recommendations. In particular, we look at issues relating to: criteria, metrics, product domain used, choice of materials, possible confound- ing factors, and approximation of experience versus real experience. We generalize the problems we found and the solutions that we applied to adaptive systems. We illustrate the learned lessons with examples from our previous work on adaptive systems (ranging from adaptive learning to persuasive technologies).

AB - We describe the methodological considerations that arose over a series of experiments evaluating the eectiveness of explanations for recommendations. In particular, we look at issues relating to: criteria, metrics, product domain used, choice of materials, possible confound- ing factors, and approximation of experience versus real experience. We generalize the problems we found and the solutions that we applied to adaptive systems. We illustrate the learned lessons with examples from our previous work on adaptive systems (ranging from adaptive learning to persuasive technologies).

M3 - Conference contribution

SP - 54

EP - 63

BT - Proceedings of the Sixth Workshop on User-Centred Design and Evaluation of Adaptive Systems

A2 - Weibelzahl, Stephan

A2 - Masthoff, Judith

A2 - Paramythis, Alexandros

A2 - van Velsen, Lex

PB - CEUR-WS

ER -