TY - JOUR
T1 - Deep Bayesian Self-Training
AU - Ribeiro, Fabio De Sousa
AU - Calivá, Francesco
AU - Swainson, Mark
AU - Gudmundsson, Kjartan
AU - Leontidis, Georgios
AU - Kollias, Stefanos
N1 - Acknowledgements
The authors would like to thank Mr. George Marandianos, Mrs. Mamatha Thota and Mr. Samuel Bond-Taylor for manually annotating datasets used in this study and of course the reviewers for their constructive feedback that helped to improve the manuscript. We would also like to thank Professor Luc Bidaut for enabling this collaboration.
Funding
The research presented in this paper was funded by Engineering and Physical Sciences Research Council (Reference Number EP/R005524/1) and Innovate UK (Reference Number 102908), in collaboration with the Olympus Automation Limited Company, for the project Automated Robotic Food Manufacturing System.
PY - 2020/5
Y1 - 2020/5
N2 - Supervised deep learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real-world problems, manual annotation is practically intractable due to time/labour constraints; thus, the development of automated and adaptive data annotation systems is highly sought after. In this paper, we propose both a (1) deep Bayesian self-training methodology for automatic data annotation, by leveraging predictive uncertainty estimates using variational inference and modern neural network (NN) architectures, as well as (2) a practical adaptation procedure for handling high label variability between different dataset distributions through clustering of NN latent variable representations. An experimental study on both public and private datasets is presented illustrating the superior performance of the proposed approach over standard self-training baselines, highlighting the importance of predictive uncertainty estimates in safety-critical domains.
AB - Supervised deep learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real-world problems, manual annotation is practically intractable due to time/labour constraints; thus, the development of automated and adaptive data annotation systems is highly sought after. In this paper, we propose both a (1) deep Bayesian self-training methodology for automatic data annotation, by leveraging predictive uncertainty estimates using variational inference and modern neural network (NN) architectures, as well as (2) a practical adaptation procedure for handling high label variability between different dataset distributions through clustering of NN latent variable representations. An experimental study on both public and private datasets is presented illustrating the superior performance of the proposed approach over standard self-training baselines, highlighting the importance of predictive uncertainty estimates in safety-critical domains.
KW - Machine Learning
KW - Deep Learning
KW - Deep learning
KW - Representation learning
KW - Bayesian CNN
KW - Variational inference
KW - Clustering
KW - Self-training
KW - Adaptation
KW - Uncertainty weighting
UR - http://www.scopus.com/inward/record.url?scp=85069661389&partnerID=8YFLogxK
U2 - 10.1007/s00521-019-04332-4
DO - 10.1007/s00521-019-04332-4
M3 - Article
VL - 32
SP - 4275
EP - 4291
JO - Neural Computing and Applications
JF - Neural Computing and Applications
SN - 0941-0643
ER -
