arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Statistics · stat.ML

Semi-Amortized Variational Autoencoders

Yoon Kim, Sam Wiseman, Andrew C. Miller, David Sontag, Alexander M. Rush

See on the map

Abstract

Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameters. We propose a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them. Crucially, the local SVI procedure is itself differentiable, so the inference network and generative model can be trained end-to-end with gradient-based optimization. This semi-amortized approach enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation. Experiments show this approach outperforms strong autoregressive and variational baselines on standard text and image datasets.

References

The reference list for this paper has not been collected yet.

Cited by 5

5 papers in the index cite this (Semantic Scholar counts 271 in total).

The cited-by list is still filling in while reference lists are collected.

More like this