๐ RNN Variants: GRU, Bidirectional, Deep RNNs
103 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# ๐ RNN Variants: GRU, Bidirectional, Deep RNNs ## 1. ๐ฏ Learning Objectives - Explain GRU: reset gate, update gate - Describe bidirectional RNN for full context - Stack RNN layers for deeper representations ## 2.
๐ RNN Variants: GRU, Bidirectional, Deep RNNs
1. ๐ฏ Learning Objectives
- Explain GRU: reset gate, update gate
- Describe bidirectional RNN for full context
- Stack RNN layers for deeper representations
2. ๐ Core Content
3.1 GRU
z_t = ฯ(W_zยท[h_{t-1}, x_t]) (update gate) r_t = ฯ(W_rยท[h_{t-1}, x_t]) (reset gate) hฬ_t = tanh(Wยท[r_tโh_{t-1}, x_t]) h_t = (1-z_t)โh_{t-1} + z_tโhฬ_t
3.2 Bidirectional RNN
Forward: hโ_t, Backward: hโ_t. Output: h_t = [hโ_t; hโ_t]
3.3 Deep (Stacked) RNN
Multiple RNN layers on top of each other. Each layer processes the previous layer's hidden states.
Join Discord
PreviousResNetNextLSTM & GRU Details