Layered evaluation of interactive adaptive systems

framework and formative methods

Alexandros Paramythis, Stephan Weibelzahl, Judith Masthoff

Research output: Contribution to journalArticle

93 Citations (Scopus)
4 Downloads (Pure)

Abstract

The evaluation of interactive adaptive systems has long been acknowledged
to be a complicated and demanding endeavour. Some promising approaches
in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.
Original languageEnglish
Pages (from-to)383-453
Number of pages71
JournalUser Modelling and User-Adapted Interaction
Volume20
Issue number5
Early online date28 Nov 2010
DOIs
Publication statusPublished - 1 Dec 2010

Fingerprint

Adaptive systems
evaluation
literature

Keywords

  • layered evaluation
  • evaluation framework
  • formative evaluation methods
  • design

Cite this

Layered evaluation of interactive adaptive systems : framework and formative methods . / Paramythis, Alexandros; Weibelzahl, Stephan; Masthoff, Judith.

In: User Modelling and User-Adapted Interaction, Vol. 20, No. 5, 01.12.2010, p. 383-453.

Research output: Contribution to journalArticle

@article{f39f4c66824b499baff049f04f5ee273,
title = "Layered evaluation of interactive adaptive systems: framework and formative methods",
abstract = "The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.",
keywords = "layered evaluation, evaluation framework, formative evaluation methods, design",
author = "Alexandros Paramythis and Stephan Weibelzahl and Judith Masthoff",
year = "2010",
month = "12",
day = "1",
doi = "10.1007/s11257-010-9082-4",
language = "English",
volume = "20",
pages = "383--453",
journal = "User Modelling and User-Adapted Interaction",
issn = "0924-1868",
publisher = "Springer Netherlands",
number = "5",

}

TY - JOUR

T1 - Layered evaluation of interactive adaptive systems

T2 - framework and formative methods

AU - Paramythis, Alexandros

AU - Weibelzahl, Stephan

AU - Masthoff, Judith

PY - 2010/12/1

Y1 - 2010/12/1

N2 - The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.

AB - The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.

KW - layered evaluation

KW - evaluation framework

KW - formative evaluation methods

KW - design

U2 - 10.1007/s11257-010-9082-4

DO - 10.1007/s11257-010-9082-4

M3 - Article

VL - 20

SP - 383

EP - 453

JO - User Modelling and User-Adapted Interaction

JF - User Modelling and User-Adapted Interaction

SN - 0924-1868

IS - 5

ER -