🔄 Perceptron to Multi-Layer Perceptron
114 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# 🔄 Perceptron to Multi-Layer Perceptron ## 1. 🎯 Learning Objectives - Explain why single perceptron fails at XOR - Implement MLP with hidden layers - State universal approximation theorem ## 2.
🔄 Perceptron to Multi-Layer Perceptron
1. 🎯 Learning Objectives
- Explain why single perceptron fails at XOR
- Implement MLP with hidden layers
- State universal approximation theorem
2. 📖 Core Content
3.1 XOR Problem
Single perceptron draws linear boundary. XOR requires non-linear boundary. Solution: 2-layer MLP.
3.2 MLP Architecture
Input → Hidden Layer(s) → Output Each layer: z = Wx + b, a = f(z) where f is non-linear (ReLU, sigmoid, tanh).
3.3 Universal Approximation Theorem
A feedforward network with one hidden layer (sufficient width) can approximate any continuous function to arbitrary accuracy. Width may need to be exponentially large.
Join Discord
PreviousActivation FunctionsNextForward Prop & Loss