Overview
This posting series is a study note that records the process of learning the book "Understanding Deep Learning".
This time, I will cover Chapter 17, Variational autoencoders.
1. Probability Distribution
In Chapter 17, many kinds of probability distributions appeared while dealing with VAE.
Distribution functions such as and appeared, making it not easy to interpret them.
Although these distributions appeared occasionally in other chapters, it was more difficult to understand as terms like prior, posterior, and likelihood were mixed.
2. ELBO
In VAE, due to the nature of sampling from a Gaussian Distribution, a problem arises where the likelihood or loss cannot be calculated during training.
Therefore, in reality, learning was conducted by defining a new likelihood function called ELBO.
Here, it was amazing to see the process of defining the ELBO function as by applying Jensen’s inequality and making it tight.
Also, the part of variational approximation of in the process of defining ELBO was also amazing.
Reference
[1] Prince, S. J. D. (2023). Understanding Deep Learning. The MIT Press. Retrieved from http://udlbook.com