🏗️ CNN Architectures & Transfer Learning

251 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# 🏗️ CNN Architectures & Transfer Learning ## 1. 🎯 Learning Objectives - Explain ResNet's skip connections and why they work - Describe Inception module design - Apply transfer learning: feature extraction vs fine-tuning ## 2.

🏗️ CNN Architectures & Transfer Learning

1. 🎯 Learning Objectives

Explain ResNet's skip connections and why they work
Describe Inception module design
Apply transfer learning: feature extraction vs fine-tuning

2. 📖 Core Content

3.1 ResNet: Skip Connections

Problem: Very deep networks (50+ layers) suffer from vanishing gradients. Training error INCREASES with more layers (degradation problem). Solution: Skip connections (residual blocks):

y = F(x) + x

The network learns the RESIDUAL F(x) = H(x) - x instead of the direct mapping H(x). If identity is optimal, the network can set F(x) = 0.

3.2 Inception (GoogLeNet)

Uses parallel convolutions of different sizes (1×1, 3×3, 5×5) plus max pooling, then concatenates outputs. 1×1 convolutions: Reduce channel dimensions before expensive 3×3 and 5×5 convs (bottleneck).

3.3 Transfer Learning

Feature extraction: Freeze pretrained model's weights, train only new classifier layers. Fine-tuning: Unfreeze some/all layers and train with small learning rate.


python
# Feature extraction
model = torchvision.models.resnet18(pretrained=True)
for param in model.parameters():
    param.requires_grad = False
model.fc = nn.Linear(512, num_classes)
# Fine-tuning
for param in model.parameters():
    param.requires_grad = True
# Use lower learning rate for pretrained layers

4. 📝 Practice Questions

Q1: Why do skip connections help train very deep networks?

Answer: Skip connections provide a direct gradient path from output to earlier layers, bypassing intermediate layers. This prevents vanishing gradients. The network can learn the identity function easily (by setting F(x)=0), ensuring deeper layers don't hurt performance. Join Discord PreviousCNNsNextResNet