GANs for Computer Vision: Generator, Discriminator, DCGAN, CGAN, WGAN, CycleGAN

🎯 Learning Objectives

• Understand the GAN framework for image generation
• Implement DCGAN with convolutional layers
• Generate conditional images with CGAN
• Stabilize training with Wasserstein GAN (WGAN)
• Apply CycleGAN for unpaired image-to-image translation

📋 Prerequisites

• CNN Fundamentals (Week 1): Convolution, transposed convolution
• Transfer Learning (Week 3): Feature extraction concepts
• Object Detection (Week 4): Feature map understanding

1. 📖 Core Content

1.1 GAN Framework for Images

• Generator: Creates fake images from random noise
• Discriminator: Distinguishes real from fake images (Diagram)

1.2 DCGAN Architecture

python
Run Inline
# runnable
import torch.nn as nn
class DCGANGenerator(nn.Module):
    def __init__(self, latent_dim=100, feature_dim=64, img_channels=3):
        super().__init__()
        self.main = nn.Sequential(
            # Input: latent_dim x 1 x 1
            nn.ConvTranspose2d(latent_dim, feature_dim*8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(feature_dim*8),
            nn.ReLU(True),
            # 4x4
            nn.ConvTranspose2d(feature_dim*8, feature_dim*4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim*4),
            nn.ReLU(True),
            # 8x8
            nn.ConvTranspose2d(feature_dim*4, feature_dim*2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim*2),
            nn.ReLU(True),
            # 16x16
            nn.ConvTranspose2d(feature_dim*2, feature_dim, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim),
            nn.ReLU(True),
            # 32x32
            nn.ConvTranspose2d(feature_dim, img_channels, 4, 2, 1, bias=False),
            nn.Tanh()
            # 64x64
        )
    def forward(self, z):
        return self.main(z.view(z.size(0), -1, 1, 1))

python
Run Inline
# runnable
class DCGANDiscriminator(nn.Module):
    def __init__(self, feature_dim=64, img_channels=3):
        super().__init__()
        self.main = nn.Sequential(
            nn.Conv2d(img_channels, feature_dim, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(feature_dim, feature_dim*2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim*2),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(feature_dim*2, feature_dim*4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim*4),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(feature_dim*4, feature_dim*8, 4, 2, 1, bias=False),
            nn.BatchNorm2d(feature_dim*8),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(feature_dim*8, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()
        )
    def forward(self, x):
        return self.main(x).view(-1)

1.3 Training Tips for GANs

1.4 WGAN-GP (Wasserstein GAN with Gradient Penalty)

python
Run Inline
# runnable
def gradient_penalty(critic, real, fake, device):
    """WGAN-GP gradient penalty."""
    batch_size = real.size(0)
    # Random interpolation
    epsilon = torch.rand(batch_size, 1, 1, 1, device=device)
    interpolated = epsilon * real + (1 - epsilon) * fake
    interpolated.requires_grad_(True)
    # Critic score for interpolated images
    d_interpolated = critic(interpolated)
    # Gradients
    gradients = torch.autograd.grad(
        outputs=d_interpolated,
        inputs=interpolated,
        grad_outputs=torch.ones_like(d_interpolated),
        create_graph=True,
        retain_graph=True
    )[0]
    # GP loss: (||grad|| - 1)^2
    gradients = gradients.view(batch_size, -1)
    grad_norm = gradients.norm(2, dim=1)
    gp = ((grad_norm - 1) ** 2).mean()
    return gp

1.5 Why This Matters

2. 📐 Key Formulas / Concepts

3. ⚠️ Common Pitfalls

Pitfall 1: Using Sigmoid at the End of the Generator

Pitfall 2: Balancing Generator and Discriminator

4. 📝 Practice Questions

Q1: Your GAN generates the same cherry-red sports car for every noise input. What's wrong and how do you fix it?

Problem: Mode collapse — the generator found one plausible image that fools the discriminator and sticks to it.

Fixes:

1. Minibatch discrimination: Add feature statistics across the batch so the discriminator can detect when the generator produces uniform outputs.
2. Label smoothing: Use soft labels (0.9/0.1 instead of 1/0) to prevent the discriminator from being overconfident.
3. WGAN loss: Wasserstein loss doesn't saturate like binary cross-entropy, making mode collapse less likely.
4. Increase latent dimension: More noise dimensions give the generator more room to diversify.
5. Add noise to discriminator inputs: Inject Gaussian noise to force the discriminator to tolerate variation.

5. 🔗 Cross-References

• Previous: Segmentation (Week 5)
• Next: Vision Transformers (Week 7)
• Related: GenAI GAN notes
• External: Radford et al., "Unsupervised Representation Learning with DCGAN" (2016) Join Discord PreviousSegmentationNextData Augmentation