Rank, Dimension & the Rank-Nullity Theorem

🎯 Learning Objectives

• Compute the rank of a matrix using Gaussian elimination
• Find the nullity of a matrix
• State and apply the rank-nullity theorem
• Relate rank to the dimension of the column space
• Use rank to determine properties of linear systems

📋 Prerequisites

• Gaussian Elimination (Week 2) — finding pivots and RREF
• Basis & Dimension (this week) — dimension of a vector space
• Subspaces (Week 3) — null space and column space

1. Intuition: Counting Independent Directions

• A line (rank 1)
• A plane (rank 2)
• All of $\mathbb{R}^3$ (rank 3) The nullity tells you the opposite: how many directions in the domain get crushed to zero. (Diagram) The rank-nullity theorem says: $\text{rank}(A) + \text{nullity}(A) = \text{number of columns}$. It's a conservation law: every column either contributes to the output (rank) or gets crushed (nullity).

2. Rank of a Matrix

2.1 Definition

Definition (Rank). The rank of a matrix $A$ is:

• The number of non-zero rows in its RREF
• Equivalently, the number of pivot columns
• Equivalently, the dimension of the column space $C(A)$
• Equivalently, the dimension of the row space

2.2 Computing Rank

Example 1: Computing rank

Row reduce:

Two non-zero rows: $\text{rank}(A) = 2$.

2.3 Properties of Rank

3. Nullity

3.1 Definition

Definition (Nullity). The nullity of a matrix $A$ is the dimension of the null space $N(A)$:

Example 2: Computing nullity

For

(RREF from Example 1):

Pivots in columns 1 and 3. Free variables: columns 2 and 4. $\text{nullity}(A) = 2$.

Check: $A$ has 4 columns. Rank = 2. $2 + 2 = 4$ ✓ (rank-nullity).

4. The Rank-Nullity Theorem

Theorem (Rank-Nullity). For an $m \times n$ matrix $A$:

where $n$ is the number of columns of $A$. Proof Sketch

Row reduce $A$ to RREF. Let $r = \text{rank}(A)$ = number of pivot columns. Then there are $n - r$ non-pivot columns, each corresponding to a free variable. The null space has dimension $n - r$ (each free variable contributes one basis vector). So $\text{nullity}(A) = n - r = n - \text{rank}(A)$. Rearranging gives $\text{rank}(A) + \text{nullity}(A) = n$. Example 3: Applying rank-nullity

$A$ is $5 \times 7$ with rank 3. Find nullity.

$\text{nullity}(A) = 7 - 3 = 4$.

The null space is a 4-dimensional subspace of $\mathbb{R}^7$.

4.1 Consequences

5. Finding Bases for Row Space, Column Space, Null Space

5.1 Row Space Basis

Example 4: Row space basis

Basis for row space: $\{(1,2,0), (0,0,1)\}$.

5.2 Column Space Basis

Example 5: Column space basis

Same $A$ as above. Pivot columns are 1 and 3. Original columns: $\{(1,2,1), (1,3,0)\}$ form a basis for $C(A)$.

5.3 Null Space Basis

Example 6: Null space basis

Actually let's use a better example.

(RREF).

Pivots in columns 1 and 3. Free variables: $x_2$, $x_4$.

$x_1 = -2x_2 + x_3 = -2x_2 - (-x_4) = -2x_2 + x_4$? Wait, let's solve properly:

From RREF: $x_1 + 2x_2 - x_3 = 0$, $x_3 + x_4 = 0$. From row 2: $x_3 = -x_4$. From row 1: $x_1 = -2x_2 + x_3 = -2x_2 - x_4$.

Free: $x_2, x_4$.

Basis for $N(A)$: $\{(-2,1,0,0), (-1,0,-1,1)\}$. Nullity = 2.

6. Full Column Rank vs. Full Row Rank

Example 7: Full column rank

is $3 \times 2$. Row reduce shows pivots in both columns. $\text{rank}(A) = 2 = n$. Nullity = $2 - 2 = 0$. $N(A) = \{\mathbf{0}\}$.

7. Edge Cases & Gotchas

8. Common Pitfalls

❌ Pitfall 1: Confusing rank with number of rows

❌ Pitfall 2: Forgetting rank-nullity applies to columns

❌ Pitfall 3: Taking RREF rows as column space basis

9. Formula Summary Table

10. 📝 Practice Questions

Q1: Find rank from RREF

This is already in REF with 3 pivots. $\text{rank}(A) = 3$. Q2: Rank and nullity

Solution: Rows 1 and 3 are identical; row 2 is $2 \times$ row 1. RREF:

. Rank = 1, Nullity = $4 - 1 = 3$. Q3: Rank-Nullity application

If $A$ is $3 \times 5$ with rank 2, what are the dimensions of $N(A)$ and $C(A)$?

Solution: $\dim(C(A)) = \text{rank}(A) = 2$. $\text{nullity}(A) = 5 - 2 = 3$. $N(A)$ is a 3-dim subspace of $\mathbb{R}^5$, $C(A)$ is a 2-dim subspace of $\mathbb{R}^3$. Q4: Finding null space basis

Find a basis for $N(A)$ where

.

Solution: Already in RREF. Pivots in columns 1, 3. Free: columns 2, 4.

Row 1: $x_1 + x_2 + x_4 = 0 \Rightarrow x_1 = -x_2 - x_4$ Row 2: $x_3 + 2x_4 = 0 \Rightarrow x_3 = -2x_4$

Basis: $\{(-1,1,0,0), (-1,0,-2,1)\}$. Q5: Full row rank

For what $k$ does

have full row rank?

Solution: Full row rank means $\text{rank}(A) = 3$ (number of rows). But $A$ is $3 \times 2$, so max rank is $\min(3,2) = 2$. It's impossible for a $3 \times 2$ matrix to have rank 3.

The question should be: when does it have full column rank?

Row reduce:

If $k \neq 6$, there are 2 pivots ⇒ rank 2 (full column rank). If $k = 6$, rank 1. Q6: Rank of a product

If $A$ is $3 \times 4$ with rank 2 and $B$ is $4 \times 2$ with rank 2, what can you say about $\text{rank}(AB)$?

Solution: $\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B)) = \min(2, 2) = 2$.

Also, $\text{rank}(AB) \leq \min(3, 2) = 2$ (dimensions of $AB$ are $3 \times 2$).

So $\text{rank}(AB) \leq 2$. It could be 0, 1, or 2. Q7: Rank from determinant

If $A$ is $4 \times 4$ with $\det(A) = 0$, what is the maximum possible rank?

Solution: $\det(A) = 0$ means $A$ is singular, so $\text{rank}(A) < 4$. The maximum possible rank is 3. Q8: Invertibility and rank

If $A$ is $5 \times 5$ with $\text{rank}(A) = 5$, what is $\text{nullity}(A)$? Is $A$ invertible?

Solution: $\text{nullity}(A) = 5 - 5 = 0$. Yes, $A$ is invertible (full rank square matrix). Q9: Row space dimension

If $A$ is $7 \times 4$ with rank 3, what is the dimension of the row space?

Solution: $\dim(\text{row space}) = \text{rank}(A) = 3$. Q10: Rank of transpose

If $A$ is $5 \times 3$ with rank 2, what is $\text{rank}(A^T)$?

Solution: $\text{rank}(A^T) = \text{rank}(A) = 2$.

🔗 Cross-References

• Next topic: Null Space & Column Space
• Week 5 (Linear Transformations): Rank-nullity generalizes to linear transformations
• Week 6 (Kernel & Image): Kernel = null space, image = column space
• Week 11 (Hessian): The Hessian matrix's rank determines if a critical point is degenerate
• BSCS2004 (ML Foundations): Low-rank approximations, PCA, dimensionality reduction Join Discord Previous4.1 Basis & DimensionNext5.1 Null Space & Column Space