🌡️ Simulated Annealing & Tabu Search

1. 🎯 Learning Objectives

• Explain the analogy between annealing in metallurgy and search
• Trace Simulated Annealing with a cooling schedule
• Implement Tabu Search with tabu list management
• Compare metaheuristics: when each is appropriate
• Explain how SA escapes local optima

2. 📖 Core Content

3.1 Simulated Annealing: Intuition

• Start with high temperature: accept bad moves with high probability (explore)
• Gradually cool down: accept fewer bad moves (exploit)
• At T=0: only accept improving moves (like Hill Climbing)

3.2 SA Algorithm

text
SimulatedAnnealing(initial_state, move_gen, cost, schedule):
    current = initial_state
    T = initial_temperature
    for iteration = 1 to max_iterations:
        if T == 0: return current
        neighbor = random_neighbor(current, move_gen)
        delta_E = cost(neighbor) - cost(current)
        if delta_E < 0:  // Better neighbor
            current = neighbor
        else:  // Worse neighbor — accept with probability
            if random(0,1) < exp(-delta_E / T):
                current = neighbor
        T = schedule(T, iteration)
    return current

3.3 Acceptance Probability

• $\Delta E > 0$: how much worse the move is
• $T$: current temperature Analysis:
• High T: $e^{-\Delta E / T} \approx 1$ — almost all moves accepted
• Low T: $e^{-\Delta E / T} \approx 0$ — only improving moves accepted
• Large $\Delta E$: lower acceptance probability (proportional to how bad)

3.4 Cooling Schedules

3.5 Tabu Search: Intuition

3.6 Tabu Search Algorithm

text
TabuSearch(initial_state, move_gen, cost, tabu_size):
    current = initial_state
    best = initial_state
    tabu_list = FIFO_queue(tabu_size)
    while not stopping_condition:
        // Generate neighbors not in tabu list
        candidates = [n for n in move_gen(current) if n not in tabu_list]
        if candidates is empty: break
        // Pick best neighbor (even if worse than current)
        current = argmin(cost(n) for n in candidates)
        // Update tabu list
        tabu_list.add(current)
        if len(tabu_list) > tabu_size:
            tabu_list.pop_oldest()
        // Update global best
        if cost(current) < cost(best):
            best = current
    return best

3.7 Key Concepts

3.8 Comparison of Metaheuristics

4. 📝 Practice Questions

Q1: At T=100, what is the probability of accepting a move with ΔE=10?

Answer: P = exp(-10/100) = exp(-0.1) ≈ 0.905. High probability because temperature is high. Q2: At T=1, what is the probability of accepting ΔE=5?

Answer: P = exp(-5/1) = exp(-5) ≈ 0.0067. Very low probability — at low temperature, SA mostly accepts only improving moves. Q3: What is the aspiration criterion in Tabu Search?

Answer: If a move is tabu but leads to a state better than the global best found so far, it is still allowed. This prevents the algorithm from permanently rejecting the best possible solution just because it was recently visited. Join Discord PreviousSAT / CNFNextBeam Search & VND