Abstract
Variational Autoencoder (VAE) is a powerful method for learning representations of high-dimensional data. However, VAEs can suffer from an issue known as latent variable collapse (or KL loss vanishing), where the posterior collapses to the prior and the model will ignore the latent codes in generative tasks. Such an issue is particularly prevalent when employing VAE-RNN architectures for text modelling (Bowman et al., 2016). In this paper, we present a simple architecture called holistic regularisation VAE (HR-VAE), which can effectively avoid latent variable collapse. Compared to existing VAE-RNN architectures, we show that our model can achieve much more stable training process and can generate text with significantly better quality.
Original language | English |
---|---|
Title of host publication | Proceedings of the 12th International Conference on Natural Language Generation |
Place of Publication | Tokyo, Japan |
Publisher | Association for Computational Linguistics (ACL) |
Pages | 594-599 |
Number of pages | 6 |
Edition | W19-8673 |
ISBN (Electronic) | 9781950737949 |
DOIs | |
Publication status | Published - 30 Nov 2019 |
Event | The 12th International Conference on Natural Language Generation (INLG 2019) - National Museum of Emerging Science and Innovation (Miraikan), Tokyo, Japan Duration: 29 Oct 2019 → 1 Nov 2019 |
Publication series
Name | INLG 2019 - 12th International Conference on Natural Language Generation, Proceedings of the Conference |
---|
Conference
Conference | The 12th International Conference on Natural Language Generation (INLG 2019) |
---|---|
Country/Territory | Japan |
City | Tokyo |
Period | 29/10/19 → 1/11/19 |
Bibliographical note
AcknowledgementThis work is supported by the award made by the UK Engineering and Physical Sciences Research Council (Grant number: EP/P011829/1).