Best-worst scaling improves measurement of first impressions

Nichola Burton (Corresponding Author), Michael Burton, Dan Rigby, Clare A M Sutherland, Gillian Rhodes

Research output: Contribution to journalArticle

Abstract

A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants’ facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.

Original languageEnglish
Article number36
JournalCognitive Research: Principles and Implications
Volume4
Early online date23 Sep 2019
DOIs
Publication statusE-pub ahead of print - 23 Sep 2019

Fingerprint

Aptitude
Research
Reproducibility of Results
Individuality
Psychology
Power (Psychology)

Cite this

Best-worst scaling improves measurement of first impressions. / Burton, Nichola (Corresponding Author); Burton, Michael; Rigby, Dan; Sutherland, Clare A M; Rhodes, Gillian.

In: Cognitive Research: Principles and Implications, Vol. 4, 36, 12.2019.

Research output: Contribution to journalArticle

Burton, Nichola ; Burton, Michael ; Rigby, Dan ; Sutherland, Clare A M ; Rhodes, Gillian. / Best-worst scaling improves measurement of first impressions. In: Cognitive Research: Principles and Implications. 2019 ; Vol. 4.
@article{268e823838f9446294cf82e3377d09a2,
title = "Best-worst scaling improves measurement of first impressions",
abstract = "A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants’ facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.",
author = "Nichola Burton and Michael Burton and Dan Rigby and Sutherland, {Clare A M} and Gillian Rhodes",
note = "This research was supported by the Australian Research Council (ARC) Centre of Excellence for Cognition and its Disorders (CE110001021), an ARC Discovery Project grant to GR and CS (DP170104602) and an ARC Discovery Outstanding Researcher Award to GR (DP130102300). The datasets analysed during the current study are available from the corresponding author on reasonable request.",
year = "2019",
month = "9",
day = "23",
doi = "10.1186/s41235-019-0183-2",
language = "English",
volume = "4",
journal = "Cognitive Research: Principles and Implications",

}

TY - JOUR

T1 - Best-worst scaling improves measurement of first impressions

AU - Burton, Nichola

AU - Burton, Michael

AU - Rigby, Dan

AU - Sutherland, Clare A M

AU - Rhodes, Gillian

N1 - This research was supported by the Australian Research Council (ARC) Centre of Excellence for Cognition and its Disorders (CE110001021), an ARC Discovery Project grant to GR and CS (DP170104602) and an ARC Discovery Outstanding Researcher Award to GR (DP130102300). The datasets analysed during the current study are available from the corresponding author on reasonable request.

PY - 2019/9/23

Y1 - 2019/9/23

N2 - A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants’ facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.

AB - A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants’ facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.

U2 - 10.1186/s41235-019-0183-2

DO - 10.1186/s41235-019-0183-2

M3 - Article

VL - 4

JO - Cognitive Research: Principles and Implications

JF - Cognitive Research: Principles and Implications

M1 - 36

ER -