🔄 RNN Variants: GRU, Bidirectional, Deep RNNs
103 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# 🔄 RNN Variants: GRU, Bidirectional, Deep RNNs ## 1. 🎯 Learning Objectives - Explain GRU: reset gate, update gate - Describe bidirectional RNN for full context - Stack RNN layers for deeper representations ## 2.
🔄 RNN Variants: GRU, Bidirectional, Deep RNNs
1. 🎯 Learning Objectives
- Explain GRU: reset gate, update gate
- Describe bidirectional RNN for full context
- Stack RNN layers for deeper representations
2. 📖 Core Content
3.1 GRU
z_t = σ(W_z·[h_{t-1}, x_t]) (update gate) r_t = σ(W_r·[h_{t-1}, x_t]) (reset gate) h̃_t = tanh(W·[r_t⊙h_{t-1}, x_t]) h_t = (1-z_t)⊙h_{t-1} + z_t⊙h̃_t
3.2 Bidirectional RNN
Forward: h→_t, Backward: h←_t. Output: h_t = [h→_t; h←_t]
3.3 Deep (Stacked) RNN
Multiple RNN layers on top of each other. Each layer processes the previous layer's hidden states.
Join Discord
PreviousResNetNextLSTM & GRU Details