The role of automated feedback in training and retaining biological recorders for citizen science

René van der Wal*, Nirwan Sharma, Chris Mellish, Anne Robinson, Advaith Siddharthan

*Corresponding author for this work

Research output: Contribution to journalArticle

23 Citations (Scopus)
6 Downloads (Pure)

Abstract

The rapid rise of citizen science, with lay people forming often extensive biodiversity sensor networks, is seen as a solution to the mismatch between data demand and supply while simultaneously engaging citizens with environmental topics. However, citizen science recording schemes require careful consideration of how to motivate, train, and retain volunteers. We evaluated a novel computing science framework that allowed for the automated generation of feedback to citizen scientists using natural language generation (NLG) technology. We worked with a photo-based citizen science program in which users also volunteer species identification aided by an online key. Feedback is provided after photo (and identification) submission and is aimed to improve volunteer species identification skills and to enhance volunteer experience and retention. To assess the utility of NLG feedback, we conducted two experiments with novices to assess short-term (single session) and longer-term (5 sessions in 2 months) learning, respectively. Participants identified a specimen in a series of photos. One group received only the correct answer after each identification, and the other group received the correct answer and NLG feedback explaining reasons for misidentification and highlighting key features that facilitate correct identification. We then developed an identification training tool with NLG feedback as part of the citizen science program BeeWatch and analyzed learning by users. Finally, we implemented NLG feedback in the live program and evaluated this by randomly allocating all BeeWatch users to treatment groups that received different types of feedback upon identification submission. After 6 months separate surveys were sent out to assess whether views on the citizen science program and its feedback differed among the groups. Identification accuracy and retention of novices were higher for those who received automated feedback than for those who received only confirmation of the correct identification without explanation. The value of NLG feedback in the live program, captured through questionnaires and evaluation of the online photo-based training tool, likewise showed that the automated generation of informative feedback fostered learning and volunteer engagement and thus paves the way for productive and long-lived citizen science projects.

Original languageEnglish
Pages (from-to)550-561
Number of pages12
JournalConservation Biology
Volume30
Issue number3
Early online date25 Apr 2016
DOIs
Publication statusPublished - 1 Jun 2016

Fingerprint

volunteers
learning
supply balance
science
citizen
questionnaires
biodiversity
train
programme
sensor
species identification
experiment

Keywords

  • Biological recording
  • Bumblebee identification
  • Natural language generation
  • Training
  • Volunteer motivation and retention

ASJC Scopus subject areas

  • Ecology
  • Nature and Landscape Conservation
  • Ecology, Evolution, Behavior and Systematics

Cite this

The role of automated feedback in training and retaining biological recorders for citizen science. / van der Wal, René; Sharma, Nirwan; Mellish, Chris; Robinson, Anne; Siddharthan, Advaith.

In: Conservation Biology, Vol. 30, No. 3, 01.06.2016, p. 550-561.

Research output: Contribution to journalArticle

van der Wal, René ; Sharma, Nirwan ; Mellish, Chris ; Robinson, Anne ; Siddharthan, Advaith. / The role of automated feedback in training and retaining biological recorders for citizen science. In: Conservation Biology. 2016 ; Vol. 30, No. 3. pp. 550-561.
@article{ba50d9a23850439da567e0886e9f3fb8,
title = "The role of automated feedback in training and retaining biological recorders for citizen science",
abstract = "The rapid rise of citizen science, with lay people forming often extensive biodiversity sensor networks, is seen as a solution to the mismatch between data demand and supply while simultaneously engaging citizens with environmental topics. However, citizen science recording schemes require careful consideration of how to motivate, train, and retain volunteers. We evaluated a novel computing science framework that allowed for the automated generation of feedback to citizen scientists using natural language generation (NLG) technology. We worked with a photo-based citizen science program in which users also volunteer species identification aided by an online key. Feedback is provided after photo (and identification) submission and is aimed to improve volunteer species identification skills and to enhance volunteer experience and retention. To assess the utility of NLG feedback, we conducted two experiments with novices to assess short-term (single session) and longer-term (5 sessions in 2 months) learning, respectively. Participants identified a specimen in a series of photos. One group received only the correct answer after each identification, and the other group received the correct answer and NLG feedback explaining reasons for misidentification and highlighting key features that facilitate correct identification. We then developed an identification training tool with NLG feedback as part of the citizen science program BeeWatch and analyzed learning by users. Finally, we implemented NLG feedback in the live program and evaluated this by randomly allocating all BeeWatch users to treatment groups that received different types of feedback upon identification submission. After 6 months separate surveys were sent out to assess whether views on the citizen science program and its feedback differed among the groups. Identification accuracy and retention of novices were higher for those who received automated feedback than for those who received only confirmation of the correct identification without explanation. The value of NLG feedback in the live program, captured through questionnaires and evaluation of the online photo-based training tool, likewise showed that the automated generation of informative feedback fostered learning and volunteer engagement and thus paves the way for productive and long-lived citizen science projects.",
keywords = "Biological recording, Bumblebee identification, Natural language generation, Training, Volunteer motivation and retention",
author = "{van der Wal}, Ren{\'e} and Nirwan Sharma and Chris Mellish and Anne Robinson and Advaith Siddharthan",
note = "Acknowledgments The authors thank H. H. Nguyen for his early development work on the BeeWatch interface; E. O'Mahony, I. Pearce, and R. Comont for identifying numerous photographed bumblebees; B. Darvill, D. Ewing, and G. Perkins for enabling our partnership with the Bumblebee Conservation Trust; and S. Blake for his investments in developing the NLG feedback. The study was part of the Digital Conservation project of dot.rural, the University of Aberdeen's Digital Economy Research Hub, funded by RCUK (grant reference EP/G066051/1).",
year = "2016",
month = "6",
day = "1",
doi = "10.1111/cobi.12705",
language = "English",
volume = "30",
pages = "550--561",
journal = "Conservation Biology",
issn = "0888-8892",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - The role of automated feedback in training and retaining biological recorders for citizen science

AU - van der Wal, René

AU - Sharma, Nirwan

AU - Mellish, Chris

AU - Robinson, Anne

AU - Siddharthan, Advaith

N1 - Acknowledgments The authors thank H. H. Nguyen for his early development work on the BeeWatch interface; E. O'Mahony, I. Pearce, and R. Comont for identifying numerous photographed bumblebees; B. Darvill, D. Ewing, and G. Perkins for enabling our partnership with the Bumblebee Conservation Trust; and S. Blake for his investments in developing the NLG feedback. The study was part of the Digital Conservation project of dot.rural, the University of Aberdeen's Digital Economy Research Hub, funded by RCUK (grant reference EP/G066051/1).

PY - 2016/6/1

Y1 - 2016/6/1

N2 - The rapid rise of citizen science, with lay people forming often extensive biodiversity sensor networks, is seen as a solution to the mismatch between data demand and supply while simultaneously engaging citizens with environmental topics. However, citizen science recording schemes require careful consideration of how to motivate, train, and retain volunteers. We evaluated a novel computing science framework that allowed for the automated generation of feedback to citizen scientists using natural language generation (NLG) technology. We worked with a photo-based citizen science program in which users also volunteer species identification aided by an online key. Feedback is provided after photo (and identification) submission and is aimed to improve volunteer species identification skills and to enhance volunteer experience and retention. To assess the utility of NLG feedback, we conducted two experiments with novices to assess short-term (single session) and longer-term (5 sessions in 2 months) learning, respectively. Participants identified a specimen in a series of photos. One group received only the correct answer after each identification, and the other group received the correct answer and NLG feedback explaining reasons for misidentification and highlighting key features that facilitate correct identification. We then developed an identification training tool with NLG feedback as part of the citizen science program BeeWatch and analyzed learning by users. Finally, we implemented NLG feedback in the live program and evaluated this by randomly allocating all BeeWatch users to treatment groups that received different types of feedback upon identification submission. After 6 months separate surveys were sent out to assess whether views on the citizen science program and its feedback differed among the groups. Identification accuracy and retention of novices were higher for those who received automated feedback than for those who received only confirmation of the correct identification without explanation. The value of NLG feedback in the live program, captured through questionnaires and evaluation of the online photo-based training tool, likewise showed that the automated generation of informative feedback fostered learning and volunteer engagement and thus paves the way for productive and long-lived citizen science projects.

AB - The rapid rise of citizen science, with lay people forming often extensive biodiversity sensor networks, is seen as a solution to the mismatch between data demand and supply while simultaneously engaging citizens with environmental topics. However, citizen science recording schemes require careful consideration of how to motivate, train, and retain volunteers. We evaluated a novel computing science framework that allowed for the automated generation of feedback to citizen scientists using natural language generation (NLG) technology. We worked with a photo-based citizen science program in which users also volunteer species identification aided by an online key. Feedback is provided after photo (and identification) submission and is aimed to improve volunteer species identification skills and to enhance volunteer experience and retention. To assess the utility of NLG feedback, we conducted two experiments with novices to assess short-term (single session) and longer-term (5 sessions in 2 months) learning, respectively. Participants identified a specimen in a series of photos. One group received only the correct answer after each identification, and the other group received the correct answer and NLG feedback explaining reasons for misidentification and highlighting key features that facilitate correct identification. We then developed an identification training tool with NLG feedback as part of the citizen science program BeeWatch and analyzed learning by users. Finally, we implemented NLG feedback in the live program and evaluated this by randomly allocating all BeeWatch users to treatment groups that received different types of feedback upon identification submission. After 6 months separate surveys were sent out to assess whether views on the citizen science program and its feedback differed among the groups. Identification accuracy and retention of novices were higher for those who received automated feedback than for those who received only confirmation of the correct identification without explanation. The value of NLG feedback in the live program, captured through questionnaires and evaluation of the online photo-based training tool, likewise showed that the automated generation of informative feedback fostered learning and volunteer engagement and thus paves the way for productive and long-lived citizen science projects.

KW - Biological recording

KW - Bumblebee identification

KW - Natural language generation

KW - Training

KW - Volunteer motivation and retention

UR - http://www.scopus.com/inward/record.url?scp=84964721035&partnerID=8YFLogxK

U2 - 10.1111/cobi.12705

DO - 10.1111/cobi.12705

M3 - Article

VL - 30

SP - 550

EP - 561

JO - Conservation Biology

JF - Conservation Biology

SN - 0888-8892

IS - 3

ER -