Data Augmentation: Transformations, MixUp, CutMix, RandAugment, AutoAugment

824 words

4 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Data Augmentation: Transformations, MixUp, CutMix, RandAugment, AutoAugment ## 🎯 Learning Objectives - Apply geometric and color transformations for data augmentation - Implement MixUp and CutMix for regularized training - Use RandAugment for automated augmentation selection - Understand AutoAugment's learned aug...

Data Augmentation: Transformations, MixUp, CutMix, RandAugment, AutoAugment

🎯 Learning Objectives

Apply geometric and color transformations for data augmentation
Implement MixUp and CutMix for regularized training
Use RandAugment for automated augmentation selection
Understand AutoAugment's learned augmentation policies
Build robust training pipelines with augmentation

📋 Prerequisites

Classification models — Understanding of overfitting and generalization
PyTorch transforms — Basic image transformations

1. 📖 Core Content

1.1 Intuition: Why Augment?

More data = better models. But collecting more data is expensive. Data augmentation creates new training examples by applying label-preserving transformations to existing data. A random crop of a cat is still a cat. A horizontally flipped cat is still a cat. This creates an infinite supply of training data from a finite dataset.

1.2 Basic Transforms


python
# runnable
from torchvision import transforms
# Training transforms (heavy augmentation)
train_transform = transforms.Compose([
    transforms.RandomResizedCrop(224, scale=(0.08, 1.0)),
    transforms.RandomHorizontalFlip(p=0.5),
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),
    transforms.RandomRotation(degrees=15),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406],
                         std=[0.229, 0.224, 0.225])
])
# Validation transforms (minimal, consistent)
val_transform = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406],
                         std=[0.229, 0.224, 0.225])
])

1.3 MixUp

MixUp creates virtual training examples by mixing two images and their labels:


python
# runnable
import torch
import numpy as np
def mixup_data(x, y, alpha=0.2):
    """Returns mixed inputs, pairs of targets, and lambda."""
    lam = np.random.beta(alpha, alpha)
    batch_size = x.size(0)
    index = torch.randperm(batch_size)
    mixed_x = lam * x + (1 - lam) * x[index]
    y_a, y_b = y, y[index]
    return mixed_x, y_a, y_b, lam
def mixup_criterion(criterion, pred, y_a, y_b, lam):
    return lam * criterion(pred, y_a) + (1 - lam) * criterion(pred, y_b)
# Usage in training:
# for (inputs, targets) in dataloader:
#     inputs, targets_a, targets_b, lam = mixup_data(inputs, targets)
#     outputs = model(inputs)
#     loss = mixup_criterion(criterion, outputs, targets_a, targets_b, lam)

1.4 CutMix

CutMix replaces a rectangular region in one image with a patch from another image:


python
# runnable
def cutmix_data(x, y, alpha=1.0):
    """CutMix: replace square region with patch from another image."""
    lam = np.random.beta(alpha, alpha)
    batch_size = x.size(0)
    index = torch.randperm(batch_size)
    # Random box coordinates
    H, W = x.size(2), x.size(3)
    cut_ratio = np.sqrt(1. - lam)
    cut_h = int(H * cut_ratio)
    cut_w = int(W * cut_ratio)
    cx = np.random.randint(W)
    cy = np.random.randint(H)
    bbx1 = np.clip(cx - cut_w // 2, 0, W)
    bby1 = np.clip(cy - cut_h // 2, 0, H)
    bbx2 = np.clip(cx + cut_w // 2, 0, W)
    bby2 = np.clip(cy + cut_h // 2, 0, H)
    # Mix images
    x_mixed = x.clone()
    x_mixed[:, :, bby1:bby2, bbx1:bbx2] = x[index, :, bby1:bby2, bbx1:bbx2]
    # Adjust lambda to actual area ratio
    lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (W * H))
    return x_mixed, y, y[index], lam

1.5 RandAugment

RandAugment randomly applies N augmentations with magnitude M:

Augmentation	Description
Identity	No change
AutoContrast	Maximize image contrast
Equalize	Histogram equalization
Rotate	Rotate ±M degrees
Solarize	Invert pixels above threshold
Color	Adjust saturation by ±M
Posterize	Reduce bit depth
Contrast	Adjust contrast by ±M
Brightness	Adjust brightness by ±M
Sharpness	Adjust sharpness by ±M
ShearX/Y	Shear by ±M degrees
TranslateX/Y	Translate by ±M pixels

1.6 Why This Matters

Data augmentation is free accuracy. Proper augmentation can improve model accuracy by 2-5% on ImageNet without any architectural changes. Modern Vision Transformer (ViT) training heavily depends on RandAugment and MixUp/CutMix for state-of-the-art performance.

2. 📐 Key Formulas / Concepts

Method	Description	Best For	Implementation
Geometric	Flip, rotate, crop, scale	Position invariance	`torchvision.transforms`
Color	Brightness, contrast, hue	Appearance robustness	`ColorJitter`
MixUp	Linear interpolation of images + labels	Regularization	Custom in training loop
CutMix	Patch replacement + area-weighted labels	Local feature robustness	Custom in training loop
RandAugment	Random N augs of magnitude M	General robustness	`torchvision.RandAugment`

3. ⚠️ Common Pitfalls

Pitfall 1: Augmenting Validation/Test Data

Mistake: Applying the same augmentations to validation data. Why: Validation should measure performance on the original data distribution. Augmented validation data doesn't reflect real-world performance. Fix: Apply only ToTensor() and Normalize to validation data.

Pitfall 2: Over-Augmenting

Mistake: Using too many aggressive augmentations simultaneously. Why: When augmentations are too strong, the augmented images no longer resemble the original class. A cat rotated 90° with extreme color jitter might not look like a cat. Fix: Validate augmentation strength visually. If you can't recognize the object, the augmentation is too strong.

4. 📝 Practice Questions

Q1: You add RandAugment to your training pipeline. Validation accuracy drops from 85% to 83% after 50 epochs. What's happening?

Likely causes:

Augmentation too strong: Reduce magnitude M from 15 to 9. Start conservatively.

Training epochs insufficient: Stronger augmentation makes each epoch harder. Train for more epochs (2×-3× longer).

Learning rate not adjusted: Strong augmentation changes the loss landscape. Try a lower LR with cosine schedule.

Inconsistent distribution: Augmented images might not match the class distribution (e.g., extreme crops removing the object). Reduce augmentation strength selectively.

Fix: Start with RandAugment(N=2, M=9), train for 3× longer, use cosine LR schedule.

5. 🔗 Cross-References

Previous: GANs (Week 6) — GAN-based augmentation
Next: Image Processing (Week 9)
Related: Vision Transformers (Week 7) Join Discord PreviousGANs for CVNextVision Transformers