Randomized Regression
102 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Randomized Regression ## 7.1 Leverage Score Sampling **Leverage score:** $h_{ii} = \mathbf{x}_i^T(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{x}_i$ Higher leverage = more influential point. Sample rows proportional to leverage for a fast, accurate regression.
Randomized Regression
7.1 Leverage Score Sampling
Leverage score:
Higher leverage = more influential point. Sample rows proportional to leverage for a fast, accurate regression.
pythonimport numpy as np def leverage_score_sampling(X, y, k): n, p = X.shape U, _, _ = np.linalg.svd(X, full_matrices=False) leverage = np.sum(U**2, axis=1) prob = leverage / np.sum(leverage) idx = np.random.choice(n, size=k, replace=False, p=prob) return X[idx], y[idx]
7.2 Sketched Regression
Use random projection to reduce before regression:
Where is a random sketching matrix (count-sketch, Gaussian, or Hadamard).
Join Discord
PreviousMinHash, SimHash, BloomNextMatrix Sketching