Deep Bayesian Self-Training

Fabio De Sousa Ribeiro, Francesco Calivá, Mark Swainson, Kjartan Gudmundsson, Georgios Leontidis* (Corresponding Author), Stefanos Kollias

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Downloads (Pure)

Abstract

Supervised deep learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real-world problems, manual annotation is practically intractable due to time/labour constraints; thus, the development of automated and adaptive data annotation systems is highly sought after. In this paper, we propose both a (1) deep Bayesian self-training methodology for automatic data annotation, by leveraging predictive uncertainty estimates using variational inference and modern neural network (NN) architectures, as well as (2) a practical adaptation procedure for handling high label variability between different dataset distributions through clustering of NN latent variable representations. An experimental study on both public and private datasets is presented illustrating the superior performance of the proposed approach over standard self-training baselines, highlighting the importance of predictive uncertainty estimates in safety-critical domains.
Original languageEnglish
Pages (from-to)4275-4291
Number of pages17
JournalNeural Computing and Applications
Volume32
Early online date10 Jul 2019
DOIs
Publication statusPublished - 2020

Keywords

  • Machine Learning
  • Deep Learning
  • Deep learning
  • Representation learning
  • Bayesian CNN
  • Variational inference
  • Clustering
  • Self-training
  • Adaptation
  • Uncertainty weighting

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Deep Bayesian Self-Training'. Together they form a unique fingerprint.

  • Cite this

    Ribeiro, F. D. S., Calivá, F., Swainson, M., Gudmundsson, K., Leontidis, G., & Kollias, S. (2020). Deep Bayesian Self-Training. Neural Computing and Applications, 32, 4275-4291. https://doi.org/10.1007/s00521-019-04332-4