Matrix Sketching

127 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Matrix Sketching ## 8.1 Frequent Directions Maintain a sketch of $A \in \mathbb{R}^{n \times d}$ that approximates its covariance structure. Processes rows one at a time, keeping only $2k$ rows.

Matrix Sketching

8.1 Frequent Directions

Maintain a sketch of

A \in \mathbb{R}^{n \times d}

that approximates its covariance structure. Processes rows one at a time, keeping only

2k

rows.


python
import numpy as np
def frequent_directions(A, k):
    n, d = A.shape
    sketch = np.zeros((2*k, d))
    for i in range(n):
        sketch[0] = A[i]
        U, S, Vt = np.linalg.svd(sketch, full_matrices=False)
        delta = S[k-1]**2
        S = np.sqrt(S**2 - delta)
        sketch = np.diag(S[:k]) @ Vt[:k]
    return sketch

8.2 Count-Min Sketch

Purpose: Approximate frequency counts in a stream. Structure:

d \times w

table of counters,

d

hash functions

h_j

mapping items to

[w]

. Update: For item

i

, increment

C[j][h_j(i)]

for

j = 1, \dots, d

. Query:

\hat{f}_i = \min_j C[j][h_j(i)]

(always an overestimate). Join Discord PreviousRandomized RegressionNextFrequency Estimation

What this note is really saying

Matrix Sketching

8.1 Frequent Directions

8.2 Count-Min Sketch

Intelligence Hub