Quiz 3

SGD Variants

41 words
1 min read
Python Week 1: the first filter for runtime behavior
Visual companion
Python
Type and operator map

Python Week 1: the first filter for runtime behavior

Open galleryRelated surface
View
Revision summary

What this note is really saying

Short form

# SGD Variants ## Momentum $v_{t+1} = \beta v_t + \nabla L(\theta_t)$ $\theta_{t+1} = \theta_t - \eta v_{t+1}$ ## Adam (Adaptive Moment Estimation) Combines momentum + adaptive learning rates. Default: $\beta_1 = 0.9, \beta_2 = 0.999$.

SGD Variants

Momentum

vt+1=βvt+∇L(θt)v_{t+1} = \beta v_t + \nabla L(\theta_t) θt+1=θt−ηvt+1\theta_{t+1} = \theta_t - \eta v_{t+1}

Adam (Adaptive Moment Estimation)

Combines momentum + adaptive learning rates. Default: β1=0.9,β2=0.999\beta_1 = 0.9, \beta_2 = 0.999. Join Discord PreviousGradient DescentNextEM Algorithm
Document outline

Keep your place and jump directly to a heading.

Table of contents

No note selected

Open a chapter to see its outline, backlinks, and sources here.

Open libraryCheckpoint desk