How feedback biases give ineffective medical treatments a good reputation

Micheal De Barra, Kimmo Eriksson, Pontus Strimling

Research output: Contribution to journalArticle

7 Citations (Scopus)
3 Downloads (Pure)

Abstract

Background: Medical treatments with no direct effect (like homeopathy) or that cause harm (like bloodletting) are common
across cultures and throughout history. How do such treatments spread and persist? Most medical treatments result in a range of
outcomes: some people improve while others deteriorate. If the people who improve are more inclined to tell others about their
experiences than the people who deteriorate, ineffective or even harmful treatments can maintain a good reputation.
Objective: The intent of this study was to test the hypothesis that positive outcomes are overrepresented in online medical
product reviews, to examine if this reputational distortion is large enough to bias people’s decisions, and to explore the implications
of this bias for the cultural evolution of medical treatments.
Methods: We compared outcomes of weight loss treatments and fertility treatments in clinical trials to outcomes reported in
1901 reviews on Amazon. Then, in a series of experiments, we evaluated people’s choice of weight loss diet after reading different
reviews. Finally, a mathematical model was used to examine if this bias could result in less effective treatments having a better
reputation than more effective treatments.
Results: Data are consistent with the hypothesis that people with better outcomes are more inclined to write reviews. After 6
months on the diet, 93% (64/69) of online reviewers reported a weight loss of 10 kg or more while just 27% (19/71) of clinical
trial participants experienced this level of weight change. A similar positive distortion was found in fertility treatment reviews.
In a series of experiments, we show that people are more inclined to begin a diet with many positive reviews, than a diet with
reviews that are representative of the diet’s true effect. A mathematical model of medical cultural evolution shows that the size
of the positive distortion critically depends on the shape of the outcome distribution.
Conclusions: Online reviews overestimate the benefits of medical treatments, probably because people with negative outcomes
are less inclined to tell others about their experiences. This bias can enable ineffective medical treatments to maintain a good
reputation
Original languageEnglish
Article numbere193
JournalJournal of Medical Internet Research
Volume16
Issue number8
DOIs
Publication statusPublished - Aug 2014

Fingerprint

Cultural Evolution
Therapeutics
Diet
Fertility
Weight Loss
Theoretical Models
Bloodletting
Reducing Diet
Homeopathy
Reading
History
Clinical Trials
Weights and Measures

Keywords

  • bias
  • social media
  • behavioural sciences
  • reputation systems
  • cultural evolution

Cite this

How feedback biases give ineffective medical treatments a good reputation. / De Barra, Micheal; Eriksson, Kimmo; Strimling, Pontus.

In: Journal of Medical Internet Research, Vol. 16, No. 8, e193, 08.2014.

Research output: Contribution to journalArticle

De Barra, Micheal ; Eriksson, Kimmo ; Strimling, Pontus. / How feedback biases give ineffective medical treatments a good reputation. In: Journal of Medical Internet Research. 2014 ; Vol. 16, No. 8.
@article{4edc4cb540094a20beb552442487cdca,
title = "How feedback biases give ineffective medical treatments a good reputation",
abstract = "Background: Medical treatments with no direct effect (like homeopathy) or that cause harm (like bloodletting) are commonacross cultures and throughout history. How do such treatments spread and persist? Most medical treatments result in a range ofoutcomes: some people improve while others deteriorate. If the people who improve are more inclined to tell others about theirexperiences than the people who deteriorate, ineffective or even harmful treatments can maintain a good reputation.Objective: The intent of this study was to test the hypothesis that positive outcomes are overrepresented in online medicalproduct reviews, to examine if this reputational distortion is large enough to bias people’s decisions, and to explore the implicationsof this bias for the cultural evolution of medical treatments.Methods: We compared outcomes of weight loss treatments and fertility treatments in clinical trials to outcomes reported in1901 reviews on Amazon. Then, in a series of experiments, we evaluated people’s choice of weight loss diet after reading differentreviews. Finally, a mathematical model was used to examine if this bias could result in less effective treatments having a betterreputation than more effective treatments.Results: Data are consistent with the hypothesis that people with better outcomes are more inclined to write reviews. After 6months on the diet, 93{\%} (64/69) of online reviewers reported a weight loss of 10 kg or more while just 27{\%} (19/71) of clinicaltrial participants experienced this level of weight change. A similar positive distortion was found in fertility treatment reviews.In a series of experiments, we show that people are more inclined to begin a diet with many positive reviews, than a diet withreviews that are representative of the diet’s true effect. A mathematical model of medical cultural evolution shows that the sizeof the positive distortion critically depends on the shape of the outcome distribution.Conclusions: Online reviews overestimate the benefits of medical treatments, probably because people with negative outcomesare less inclined to tell others about their experiences. This bias can enable ineffective medical treatments to maintain a goodreputation",
keywords = "bias, social media, behavioural sciences, reputation systems, cultural evolution",
author = "{De Barra}, Micheal and Kimmo Eriksson and Pontus Strimling",
year = "2014",
month = "8",
doi = "10.2196/jmir.3214",
language = "English",
volume = "16",
journal = "Journal of Medical Internet Research",
issn = "1439-4456",
publisher = "Journal of medical Internet Research",
number = "8",

}

TY - JOUR

T1 - How feedback biases give ineffective medical treatments a good reputation

AU - De Barra, Micheal

AU - Eriksson, Kimmo

AU - Strimling, Pontus

PY - 2014/8

Y1 - 2014/8

N2 - Background: Medical treatments with no direct effect (like homeopathy) or that cause harm (like bloodletting) are commonacross cultures and throughout history. How do such treatments spread and persist? Most medical treatments result in a range ofoutcomes: some people improve while others deteriorate. If the people who improve are more inclined to tell others about theirexperiences than the people who deteriorate, ineffective or even harmful treatments can maintain a good reputation.Objective: The intent of this study was to test the hypothesis that positive outcomes are overrepresented in online medicalproduct reviews, to examine if this reputational distortion is large enough to bias people’s decisions, and to explore the implicationsof this bias for the cultural evolution of medical treatments.Methods: We compared outcomes of weight loss treatments and fertility treatments in clinical trials to outcomes reported in1901 reviews on Amazon. Then, in a series of experiments, we evaluated people’s choice of weight loss diet after reading differentreviews. Finally, a mathematical model was used to examine if this bias could result in less effective treatments having a betterreputation than more effective treatments.Results: Data are consistent with the hypothesis that people with better outcomes are more inclined to write reviews. After 6months on the diet, 93% (64/69) of online reviewers reported a weight loss of 10 kg or more while just 27% (19/71) of clinicaltrial participants experienced this level of weight change. A similar positive distortion was found in fertility treatment reviews.In a series of experiments, we show that people are more inclined to begin a diet with many positive reviews, than a diet withreviews that are representative of the diet’s true effect. A mathematical model of medical cultural evolution shows that the sizeof the positive distortion critically depends on the shape of the outcome distribution.Conclusions: Online reviews overestimate the benefits of medical treatments, probably because people with negative outcomesare less inclined to tell others about their experiences. This bias can enable ineffective medical treatments to maintain a goodreputation

AB - Background: Medical treatments with no direct effect (like homeopathy) or that cause harm (like bloodletting) are commonacross cultures and throughout history. How do such treatments spread and persist? Most medical treatments result in a range ofoutcomes: some people improve while others deteriorate. If the people who improve are more inclined to tell others about theirexperiences than the people who deteriorate, ineffective or even harmful treatments can maintain a good reputation.Objective: The intent of this study was to test the hypothesis that positive outcomes are overrepresented in online medicalproduct reviews, to examine if this reputational distortion is large enough to bias people’s decisions, and to explore the implicationsof this bias for the cultural evolution of medical treatments.Methods: We compared outcomes of weight loss treatments and fertility treatments in clinical trials to outcomes reported in1901 reviews on Amazon. Then, in a series of experiments, we evaluated people’s choice of weight loss diet after reading differentreviews. Finally, a mathematical model was used to examine if this bias could result in less effective treatments having a betterreputation than more effective treatments.Results: Data are consistent with the hypothesis that people with better outcomes are more inclined to write reviews. After 6months on the diet, 93% (64/69) of online reviewers reported a weight loss of 10 kg or more while just 27% (19/71) of clinicaltrial participants experienced this level of weight change. A similar positive distortion was found in fertility treatment reviews.In a series of experiments, we show that people are more inclined to begin a diet with many positive reviews, than a diet withreviews that are representative of the diet’s true effect. A mathematical model of medical cultural evolution shows that the sizeof the positive distortion critically depends on the shape of the outcome distribution.Conclusions: Online reviews overestimate the benefits of medical treatments, probably because people with negative outcomesare less inclined to tell others about their experiences. This bias can enable ineffective medical treatments to maintain a goodreputation

KW - bias

KW - social media

KW - behavioural sciences

KW - reputation systems

KW - cultural evolution

U2 - 10.2196/jmir.3214

DO - 10.2196/jmir.3214

M3 - Article

VL - 16

JO - Journal of Medical Internet Research

JF - Journal of Medical Internet Research

SN - 1439-4456

IS - 8

M1 - e193

ER -