🏗️ CNN Architectures & Transfer Learning
251 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# 🏗️ CNN Architectures & Transfer Learning ## 1. 🎯 Learning Objectives - Explain ResNet's skip connections and why they work - Describe Inception module design - Apply transfer learning: feature extraction vs fine-tuning ## 2.
🏗️ CNN Architectures & Transfer Learning
1. 🎯 Learning Objectives
- Explain ResNet's skip connections and why they work
- Describe Inception module design
- Apply transfer learning: feature extraction vs fine-tuning
2. 📖 Core Content
3.1 ResNet: Skip Connections
Problem: Very deep networks (50+ layers) suffer from vanishing gradients. Training error INCREASES with more layers (degradation problem).
Solution: Skip connections (residual blocks):
The network learns the RESIDUAL F(x) = H(x) - x instead of the direct mapping H(x). If identity is optimal, the network can set F(x) = 0.
3.2 Inception (GoogLeNet)
Uses parallel convolutions of different sizes (1×1, 3×3, 5×5) plus max pooling, then concatenates outputs.
1×1 convolutions: Reduce channel dimensions before expensive 3×3 and 5×5 convs (bottleneck).
3.3 Transfer Learning
Feature extraction: Freeze pretrained model's weights, train only new classifier layers. Fine-tuning: Unfreeze some/all layers and train with small learning rate.
python# Feature extraction model = torchvision.models.resnet18(pretrained=True) for param in model.parameters(): param.requires_grad = False model.fc = nn.Linear(512, num_classes) # Fine-tuning for param in model.parameters(): param.requires_grad = True # Use lower learning rate for pretrained layers
4. 📝 Practice Questions
Q1: Why do skip connections help train very deep networks?Answer: Skip connections provide a direct gradient path from output to earlier layers, bypassing intermediate layers. This prevents vanishing gradients. The network can learn the identity function easily (by setting F(x)=0), ensuring deeper layers don't hurt performance. Join Discord PreviousCNNsNextResNet