Johnson-Lindenstrauss Lemma

138 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Johnson-Lindenstrauss Lemma ## 4.1 Statement For any $\epsilon \in (0, 1/2)$ and any set of $n$ points in $\mathbb{R}^d$, there exists a map $f: \mathbb{R}^d \to \mathbb{R}^m$ with $m = O(\log n / \epsilon^2)$ such that for all $u, v$: $$ (1-\epsilon)||u-v||^2 \leq ||f(u)-f(v)||^2 \leq (1+\epsilon)||u-v||^2 $$ ##...

Johnson-Lindenstrauss Lemma

4.1 Statement

For any

\epsilon \in (0, 1/2)

and any set of

n

points in

\mathbb{R}^d

, there exists a map

f: \mathbb{R}^d \to \mathbb{R}^m

with

m = O(\log n / \epsilon^2)

such that for all

u, v

(1-\epsilon)||u-v||^2 \leq ||f(u)-f(v)||^2 \leq (1+\epsilon)||u-v||^2

4.2 Random Projection


python
import numpy as np
def random_projection(X, epsilon=0.1):
    n, d = X.shape
    m = int(4 * np.log(n) / (epsilon**2 / 2 - epsilon**3 / 3))
    R = np.random.randn(d, m) / np.sqrt(m)
    return X @ R
# Example
X = np.random.randn(1000, 10000)
X_proj = random_projection(X, epsilon=0.1)
print(f"Reduced from {X.shape[1]} to {X_proj.shape[1]} dimensions")

4.3 Key Insight

The target dimension

m

depends only on

n

(number of points), not on

d

(original dimension). This makes JL extremely powerful for high-dimensional data. Join Discord PreviousSpectral Graph TheoryNextJL Proof