Exact Algorithms — Branch & Bound, DP for TSP, Local Search
538 words
3 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Exact Algorithms — Branch & Bound, DP for TSP, Local Search ## 🎯 Learning Objectives - Apply branch and bound to combinatorial optimization - Implement Held-Karp DP for TSP in O(n²2ⁿ) time - Design local search algorithms with neighborhood structures - Understand when exact algorithms are feasible * * * ## 1. Bra...
Exact Algorithms — Branch & Bound, DP for TSP, Local Search
🎯 Learning Objectives
- Apply branch and bound to combinatorial optimization
- Implement Held-Karp DP for TSP in O(n²2ⁿ) time
- Design local search algorithms with neighborhood structures
- Understand when exact algorithms are feasible
1. Branch and Bound
1.1 Framework
(Diagram)
1.2 TSP via Branch and Bound
- Branch: Choose an edge to include or exclude
- Bound: Compute MST of remaining graph (lower bound = MST cost + current path cost)
- Prune: If bound ≥ best tour found, discard
2. Held-Karp DP for TSP
2.1 Recurrence
Where is set of visited cities (including 1), is ending city.
Base:
Answer:
2.2 Tracing: 4 cities
| Distance matrix: | | | | | | | | |
| S | i | DP[S][i] | Previous |
|---|---|---|---|
| {1,2} | 2 | 10 | 1 |
| {1,3} | 3 | 15 | 1 |
| {1,4} | 4 | 20 | 1 |
| {1,2,3} | 2 | 15+35=50 | 3 |
| {1,2,3} | 3 | 10+35=45 | 2 |
| {1,2,4} | 2 | 20+25=45 | 4 |
| {1,2,4} | 4 | 10+25=35 | 2 |
| {1,3,4} | 3 | 20+30=50 | 4 |
| {1,3,4} | 4 | 15+30=45 | 3 |
| {1,2,3,4} | 2 | 45+25=70 | 4 |
| {1,2,3,4} | 3 | 35+30=65 | 4 |
| {1,2,3,4} | 4 | 45+25=70 | 2 |
Minimum = min(DP[{1,2,3,4},2]+d(2,1), DP[{1,2,3,4},3]+d(3,1), DP[{1,2,3,4},4]+d(4,1)) = min(70+10, 65+15, 70+20) = min(80, 80, 90) = 80
Optimal tour: 1→3→4→2→1 = 15+30+25+10 = 80
3. Local Search
3.1 Framework
- Start with feasible solution
- Find improving neighbor (defined by neighborhood structure)
- If found, move to neighbor; repeat
- If no improving neighbor → local optimum
3.2 TSP: 2-opt Neighborhood
Remove two edges, reconnect in other way.
Example: Tour 1→2→3→4→1. If edges (1,2) and (3,4) cross, swap to 1→3→2→4→1.
4. Common Pitfalls
Pitfall 1: Bound Too Weak
The mistake: Using a bound that doesn't prune enough.
Correct approach: MST bound for TSP is effective. LP relaxation is stronger but slower.
Pitfall 2: Local Search Getting Stuck
The mistake: Accepting local optimum as final answer.
Correct approach: Use simulated annealing (accept worsening moves with decreasing probability) or restart from different start points.
5. Key Concepts Reference
| Method | Time | Space | Guarantee |
|---|---|---|---|
| Branch & Bound | Exponential | Polynomial | Optimal |
| Held-Karp DP | O(n²2ⁿ) | O(n2ⁿ) | Optimal |
| 2-opt Local Search | O(n³) per iteration | O(n) | Local optimum |
| Simulated Annealing | Depends | O(n) | Probabilistic |
6. 📝 Practice Questions
Q1: Held-Karp DP for 6 cities needs how many subsets?Answer: Sum of C(5, k) for k=1..5 = 2^5 = 32 subsets (excluding empty set, always including city 1). For each subset, up to n-1 ending cities. Total states: O(n × 2^{n-1}) ≈ O(n2^n). Q2: Why does Held-Karp take O(n²2ⁿ) and not O(n2ⁿ)?Answer: For each DP[S][i] (n × 2^n states), we minimize over j ∈ S{i}. In the worst case, |S| ≈ n, giving O(n) per state. Total = O(n² × 2^n). Q3: When would you use local search over branch and bound?Answer: Local search: when approximate solution is acceptable, problem instance is large, and good solutions are quickly found. B&B: when optimality is required and instance is small enough (n ≤ 20 for TSP).
7. 🔗 Cross-References
- Week 7 - Approximation: When to approximate
- Week 10 - Parameterized: Beyond exact for hard problems
- BSCS4020 (DSA): Graph theory basics Join Discord PreviousAdvanced DSNextOnline & Parallel