18. Heaps & Priority Queues

What problem does this solve? A priority queue needs to efficiently find and remove the minimum (or maximum) element. Unsorted list: insert O(1), find-min O(n). Sorted list: insert O(n), find-min O(1). A binary heap gives us O(log n) for both operations.

1. The Heap Property

• Min-heap: Parent ≤ children (root = minimum)
• Max-heap: Parent ≥ children (root = maximum) (Diagram)

Array Representation

pseudo
Index:    0   1   2   3   4   5   6
Heap:    [1,  3,  5,  7,  9,  8, 11]

• Parent of node i: (i - 1) // 2
• Left child of node i: 2 * i + 1
• Right child of node i: 2 * i + 2

python
Run Inline
# runnable
def parent(i):
    return (i - 1) // 2
def left_child(i):
    return 2 * i + 1
def right_child(i):
    return 2 * i + 2
# Verify with heap [1, 3, 5, 7, 9, 8, 11]
heap = [1, 3, 5, 7, 9, 8, 11]
print(f"Parent of 5 (idx 2): {heap[parent(2)]} = {heap[1]}")  # 3
print(f"Left child of 3 (idx 1): {heap[left_child(1)]} = {heap[3]}")  # 7

2. Min-Heap Implementation

python
Run Inline
# runnable
class MinHeap:
    """Binary min-heap implementation."""
    def __init__(self):
        self._data = []
    def __len__(self):
        return len(self._data)
    def __repr__(self):
        return str(self._data)
    def peek(self):
        """Return minimum element. O(1)."""
        if not self._data:
            raise IndexError("Peek from empty heap")
        return self._data[0]
    def insert(self, val):
        """Insert value into heap. O(log n).
        Strategy: Add to end, bubble up (sift up) while parent > value.
        """
        self._data.append(val)
        self._sift_up(len(self._data) - 1)
    def extract_min(self):
        """Remove and return minimum element. O(log n).
        Strategy: Swap root with last element, pop last, sift down.
        """
        if not self._data:
            raise IndexError("Extract from empty heap")
        min_val = self._data[0]
        last = self._data.pop()
        if self._data:
            self._data[0] = last
            self._sift_down(0)
        return min_val
    def _sift_up(self, i):
        """Bubble element at i upward to maintain heap property."""
        while i > 0:
            p = parent(i)
            if self._data[i] < self._data[p]:
                self._data[i], self._data[p] = self._data[p], self._data[i]
                i = p
            else:
                break
    def _sift_down(self, i):
        """Bubble element at i downward to maintain heap property."""
        n = len(self._data)
        while True:
            smallest = i
            l = left_child(i)
            r = right_child(i)
            if l < n and self._data[l] < self._data[smallest]:
                smallest = l
            if r < n and self._data[r] < self._data[smallest]:
                smallest = r
            if smallest != i:
                self._data[i], self._data[smallest] = self._data[smallest], self._data[i]
                i = smallest
            else:
                break
# Test
h = MinHeap()
for v in [5, 3, 8, 1, 9, 2]:
    h.insert(v)
    print(f"After insert {v}: {h}")
print(f"Min: {h.peek()}")  # 1
print(f"Extract min: {h.extract_min()}")  # 1
print(f"Heap now: {h}")    # [2, 3, 5, 8, 9]

Trace: Insert 3, 5, 1

pseudo
Insert 3: [3]
Insert 5: [3, 5]  (5 > 3, OK)
Insert 1: [3, 5, 1] → 1 < 3 → swap → [1, 5, 3]

Trace: Extract-Min from [1, 3, 5, 7, 9]

pseudo
Step 1: swap 1 with 9 → [9, 3, 5, 7, 1], pop → [9, 3, 5, 7]
Step 2: sift-down 9: 9 > 3 → swap → [3, 9, 5, 7]
Step 3: sift-down 9: 9 > 7 → swap → [3, 7, 5, 9]
Result: [3, 7, 5, 9]

3. Building a Heap from an Array — Heapify

Naive: Insert each element O(n log n)

python
Run Inline
# runnable
def build_heap_naive(arr):
    h = MinHeap()
    for v in arr:
        h.insert(v)
    return h

Smart: Floyd's Algorithm O(n)

python
Run Inline
# runnable
def build_heap(arr):
    """Turn array into min-heap in-place. O(n)."""
    n = len(arr)
    # Start from last non-leaf node and sift down
    for i in range(n // 2 - 1, -1, -1):
        _sift_down(arr, n, i)
    return arr
def _sift_down(arr, n, i):
    """Sift down element at index i in array of size n."""
    while True:
        smallest = i
        l = 2 * i + 1
        r = 2 * i + 2
        if l < n and arr[l] < arr[smallest]:
            smallest = l
        if r < n and arr[r] < arr[smallest]:
            smallest = r
        if smallest != i:
            arr[i], arr[smallest] = arr[smallest], arr[i]
            i = smallest
        else:
            break
# Test
arr = [10, 3, 5, 1, 8, 2, 7]
print(f"Original: {arr}")
build_heap(arr)
print(f"Heapified: {arr}")  # [1, 3, 2, 10, 8, 5, 7]

Why Floyd's Algorithm is O(n)

4. Max-Heap (Just Reverse Comparisons)

python
Run Inline
# runnable
class MaxHeap:
    def __init__(self):
        self._data = []
    def insert(self, val):
        self._data.append(val)
        self._sift_up(len(self._data) - 1)
    def extract_max(self):
        if not self._data:
            raise IndexError("Extract from empty heap")
        max_val = self._data[0]
        last = self._data.pop()
        if self._data:
            self._data[0] = last
            self._sift_down(0)
        return max_val
    def _sift_up(self, i):
        while i > 0:
            p = (i - 1) // 2
            if self._data[i] > self._data[p]:  # Reversed comparison
                self._data[i], self._data[p] = self._data[p], self._data[i]
                i = p
            else:
                break
    def _sift_down(self, i):
        n = len(self._data)
        while True:
            largest = i
            l = 2 * i + 1
            r = 2 * i + 2
            if l < n and self._data[l] > self._data[largest]:
                largest = l
            if r < n and self._data[r] > self._data[largest]:
                largest = r
            if largest != i:
                self._data[i], self._data[largest] = self._data[largest], self._data[i]
                i = largest
            else:
                break

5. Priority Queue Operations Comparison

Practice Questions

Answers

A1.

pseudo
Start: [12, 5, 8, 3, 10, 1, 7]
Sift-down at index 2 (value 8): [12, 5, 1, 3, 10, 8, 7]  (8 ↔ 1)
Sift-down at index 1 (value 5): [12, 3, 1, 5, 10, 8, 7]  (5 ↔ 3)
Sift-down at index 0 (value 12): [1, 3, 7, 5, 10, 8, 12] (12↔1, then 12↔7)
Final heap: [1, 3, 7, 5, 10, 8, 12]

A2.

pseudo
After 1st extract: [3, 5, 7, 12, 10, 8]
After 2nd extract: [5, 8, 7, 12, 10]

A3. Height = ⌊log₂(100)⌋ = 6 (since 2⁶ = 64 ≤ 100 < 128 = 2⁷).

A4. Use a max-heap (priority = value). Higher priority items have larger values.

A5. Most nodes are near the bottom with small heights. The total sum of (nodes × height) is O(n), not O(n log n).

A6. The minimum in a max-heap is always at a leaf. Check all leaves (indices n/2 to n-1) — O(n).

A7. See next section (heap sort). Build max-heap (O(n)), then repeatedly extract max and place at end (O(n log n)). Join Discord Previous17. AVL Trees — Self-Balancing BSTsNext19. Heap Sort