Capsule Routing via Variational Bayes

Fabio De Sousa Ribeiro, Georgios Leontidis, Stefanos Kollias

Research output: Contribution to conferencePaperpeer-review

21 Downloads (Pure)

Abstract

Capsule networks are a recently proposed type of neural network shown to outperform alternatives in challenging shape recognition tasks. In capsule networks, scalar neurons are replaced with capsule vectors or matrices, whose entries represent different properties of objects. The relationships between objects and their parts are learned via trainable viewpoint-invariant transformation matrices, and the presence of a given object is decided by the level of agreement among votes from its parts. This interaction occurs between capsule layers and is a process called routing-by-agreement. In this paper, we propose a new capsule routing algorithm derived from Variational Bayes for fitting a mixture of transforming gaussians, and show it is possible transform our capsule network into a Capsule-VAE. Our Bayesian approach addresses some of the inherent weaknesses of MLE based models such as the variance-collapse by modelling uncertainty over capsule pose parameters. We outperform the state-of-the-art on smallNORB using 50% fewer capsules than previously reported, achieve competitive performances on CIFAR-10, Fashion-MNIST, SVHN, and demonstrate significant improvement in MNIST to affNIST generalisation over previous works.
Original languageEnglish
Pages1-8
Number of pages8
DOIs
Publication statusPublished - 10 Nov 2019
EventThirty-Fourth AAAI Conference on Artificial Intelligence - New York, United States
Duration: 7 Feb 202012 Feb 2020
Conference number: 34
https://aaai.org/Conferences/AAAI-20/

Conference

ConferenceThirty-Fourth AAAI Conference on Artificial Intelligence
Abbreviated titleAAAI
CountryUnited States
CityNew York
Period7/02/2012/02/20
Internet address

Keywords

  • Capsule Networks
  • Deep Learning
  • Variational Bayes

Fingerprint Dive into the research topics of 'Capsule Routing via Variational Bayes'. Together they form a unique fingerprint.

Cite this