🔦 Beam Search & Variable Neighborhood Descent
436 words
2 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# 🔦 Beam Search & Variable Neighborhood Descent ## 1. 🎯 Learning Objectives - Explain Beam Search as limited-memory BFS with heuristic guidance - Trace Beam Search with different beam widths - Understand Variable Neighborhood Descent (VND) with multiple neighborhood structures - Compare Beam Search to Best First S...
🔦 Beam Search & Variable Neighborhood Descent
1. 🎯 Learning Objectives
- Explain Beam Search as limited-memory BFS with heuristic guidance
- Trace Beam Search with different beam widths
- Understand Variable Neighborhood Descent (VND) with multiple neighborhood structures
- Compare Beam Search to Best First Search and Hill Climbing
2. 📖 Core Content
3.1 Beam Search: Intuition
Beam Search keeps only the k most promising states at each level (beam width = k). It is like BFS but with limited memory: at each depth, only the best k nodes survive.
- k = 1: Equivalent to Hill Climbing (greedy, one path)
- k = ∞: Equivalent to BFS (exhaustive level-by-level)
- k = moderate: Beam Search (limited parallel exploration)
3.2 Beam Search Algorithm
textBeamSearch(initial, goal_test, move_gen, heuristic, beam_width): beam = [initial] while True: if beam is empty: return NO_SOLUTION // Generate all successors of all states in beam all_successors = [] for state in beam: if goal_test(state): return state all_successors.extend(move_gen(state)) // Keep only top-k by heuristic beam = sorted(all_successors, key=h)[:beam_width]
3.3 Beam Search Trace
Example: tree with b=2, beam_width=2. Goal at depth 2. States with h values: A(5), B(4), C(3), D(6), E(2), F(1), G(8), H(0=goal).
Level 0: Beam = [A(5)] Level 1: Successors of A: B(4), C(3). Beam = [C(3), B(4)] (top 2 by h) Level 2: Successors: from C → D(6), E(2); from B → F(1), G(8). All: D(6), E(2), F(1), G(8). Top 2: [F(1), E(2)] Level 3: From F → H(0=goal!). Found!
3.4 Variable Neighborhood Descent (VND)
VND systematically switches between different neighborhood structures when stuck in a local optimum:
textVND(initial_state, neighborhoods[N1, N2, ..., Nk]): current = initial_state i = 1 while i ≤ k: find best neighbor in neighborhood Ni(current) if neighbor is better than current: current = neighbor i = 1 // Reset to first neighborhood else: i++ // Try next neighborhood return current
3.5 Comparison
| Algorithm | Memory | Complete? | Neighborhoods |
|---|---|---|---|
| Hill Climbing | O(1) | No | Single |
| Beam Search | O(k·b) | No (with small k) | N/A (breadth) |
| VND | O(1) | No | Multiple, systematic |
| Tabu Search | O(tabu size) | No | Single + memory |
4. 📝 Practice Questions
Q1: With beam width k=3 and branching factor b=10, how many nodes are stored at each level?Answer: At most k·b = 30 nodes (3 nodes survive, each generates 10 successors). Without pruning: 10^d nodes. Beam Search with k=3 drastically reduces memory. Q2: How does VND differ from random restart?Answer: VND systematically changes neighborhood structures (e.g., from 2-opt to 3-opt exchanges in TSP) when stuck, rather than jumping to a random new start. It is more systematic but can still get stuck. Join Discord PreviousSimulated Annealing & TabuNextGenetic Algorithms