🎨 Autoencoders & GANs

240 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# 🎨 Autoencoders & GANs ## 1. 🎯 Learning Objectives - Explain autoencoder architecture: encoder → bottleneck → decoder - Distinguish VAE (probabilistic) from standard autoencoder - Describe GAN: generator vs discriminator, adversarial training ## 2.

🎨 Autoencoders & GANs

1. 🎯 Learning Objectives

Explain autoencoder architecture: encoder → bottleneck → decoder
Distinguish VAE (probabilistic) from standard autoencoder
Describe GAN: generator vs discriminator, adversarial training

2. 📖 Core Content

3.1 Autoencoder

Encoder:

h = f(W_e x + b_e)

— compresses input to latent code. Decoder:

\hat{x} = g(W_d h + b_d)

— reconstructs input from code. Loss:

L = ||x - \hat{x}||^2

— reconstruction error.

3.2 Variational Autoencoder (VAE)

VAE learns a probabilistic latent space: encoder outputs mean μ and variance σ². Latent variable z ~ N(μ, σ²). Loss:

L = \underbrace{||x - \hat{x}||^2}_{\text{reconstruction}} + \underbrace{KL(N(\mu, \sigma^2) || N(0,1))}_{\text{regularization}}

The KL term encourages latent space to be continuous and well-structured.

3.3 GAN (Generative Adversarial Network)

Generator G: Creates fake data from random noise z. Discriminator D: Distinguishes real from fake data. Minimax game:

\min_G \max_D V(D, G) = \mathbb{E}_{x \sim p_{data}}[\log D(x)] + \mathbb{E}_{z \sim p_z}[\log(1 - D(G(z)))]

Training: Alternate between:

Train D to distinguish real from fake
Train G to fool D

4. 📝 Practice Questions

Q1: What is the purpose of the KL divergence term in VAE loss?

Answer: It regularizes the latent space to follow a standard normal distribution, ensuring continuity (similar inputs map to nearby latent points) and completeness (every point in latent space produces valid outputs). Join Discord PreviousRNNs & LSTMsNextAutoencoders & GAN Details