Neural Sync Active
๐ RNN Variants: GRU, Bidirectional, Deep RNNs
Registry Synced
๐ RNN Variants: GRU, Bidirectional, Deep RNNs
103 words
1 min read
Reading compass
Now ยท 1. ๐ฏ Learning Objectives
๐ RNN Variants: GRU, Bidirectional, Deep RNNs
1. ๐ฏ Learning Objectives
- Explain GRU: reset gate, update gate
- Describe bidirectional RNN for full context
- Stack RNN layers for deeper representations
2. ๐ Core Content
3.1 GRU
z_t = ฯ(W_zยท[h_{t-1}, x_t]) (update gate) r_t = ฯ(W_rยท[h_{t-1}, x_t]) (reset gate) hฬ_t = tanh(Wยท[r_tโh_{t-1}, x_t]) h_t = (1-z_t)โh_{t-1} + z_tโhฬ_t
3.2 Bidirectional RNN
Forward: hโ_t, Backward: hโ_t. Output: h_t = [hโ_t; hโ_t]
3.3 Deep (Stacked) RNN
Multiple RNN layers on top of each other. Each layer processes the previous layer's hidden states.
Join Discord
PreviousResNetNextLSTM & GRU Details