Neural Sync Active
HyperLogLog
Registry Synced
HyperLogLog
96 words
1 min read
Reading compass
Now · The Distinct Counting Problem
HyperLogLog
The Distinct Counting Problem
Given a stream of elements, count the number of distinct elements using minimal memory.
Trivial solution: Keep a hash set — O(n) memory.
HyperLogLog: O(loglogn) memory, 1−2% accuracy.
Algorithm
- Hash each element to a uniform binary string
- Find the position r of the leftmost 1-bit
- Track the maximum r across all elements: R
- Estimate: n≈αmm⋅2Rˉ Where m is the number of registers (typically 214=16384), αm is a bias correction constant. Join Discord PreviousStreaming AlgorithmsNextRandomized Optimization