Rank, Dimension & the Rank-Nullity Theorem

2283 words

11 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Rank, Dimension & the Rank-Nullity Theorem ## 🎯 Learning Objectives After this topic you will be able to: - Compute the rank of a matrix using Gaussian elimination - Find the nullity of a matrix - State and apply the rank-nullity theorem - Relate rank to the dimension of the column space - Use rank to determine p...

Rank, Dimension & the Rank-Nullity Theorem

🎯 Learning Objectives

After this topic you will be able to:

Compute the rank of a matrix using Gaussian elimination
Find the nullity of a matrix
State and apply the rank-nullity theorem
Relate rank to the dimension of the column space
Use rank to determine properties of linear systems

📋 Prerequisites

Gaussian Elimination (Week 2) — finding pivots and RREF
Basis & Dimension (this week) — dimension of a vector space
Subspaces (Week 3) — null space and column space

1. Intuition: Counting Independent Directions

The rank of a matrix answers the question: how many independent directions does this matrix capture? If you have a

3 \times 5

matrix, its columns live in

\mathbb{R}^3

. The rank tells you whether the columns span:

A line (rank 1)
A plane (rank 2)
All of $\mathbb{R}^3$ (rank 3) The nullity tells you the opposite: how many directions in the domain get crushed to zero. (Diagram) The rank-nullity theorem says: $\text{rank}(A) + \text{nullity}(A) = \text{number of columns}$ . It's a conservation law: every column either contributes to the output (rank) or gets crushed (nullity).

2. Rank of a Matrix

2.1 Definition

Definition (Rank). The rank of a matrix $A$ is:

The number of non-zero rows in its RREF

Equivalently, the number of pivot columns

Equivalently, the dimension of the column space $C(A)$

Equivalently, the dimension of the row space

\text{rank}(A) = \dim(C(A)) = \dim(R(A)) = \#\text{pivots}

2.2 Computing Rank

The simplest way: row reduce and count pivots.

Example 1: Computing rank

> A = \begin{bmatrix} 1 & 2 & 3 & 1 \\ 2 & 4 & 7 & 3 \\ 3 & 6 & 10 & 4 \end{bmatrix} >

Row reduce:

> \begin{bmatrix} 1 & 2 & 3 & 1 \\ 2 & 4 & 7 & 3 \\ 3 & 6 & 10 & 4 \end{bmatrix} \xrightarrow{R_2 - 2R_1, R_3 - 3R_1} \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 1 & 1 \end{bmatrix} \xrightarrow{R_3 - R_2} \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix} >

Two non-zero rows: $\text{rank}(A) = 2$ .

2.3 Properties of Rank

Property	Explanation
$0 \leq \text{rank}(A) \leq \min(m, n)$	Cannot exceed number of rows or columns
$\text{rank}(A^T) = \text{rank}(A)$	Row space = column space of transpose
$\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))$	Composition can't increase rank
$\text{rank}(A+B) \leq \text{rank}(A) + \text{rank}(B)$	Subadditivity
$\text{rank}(A) = n$ for $A_{n \times n}$ invertible	Full rank means invertible

3. Nullity

3.1 Definition

Definition (Nullity). The nullity of a matrix $A$ is the dimension of the null space $N(A)$ :

> \text{nullity}(A) = \dim(N(A)) >

Nullity = number of free variables in the solution of

A\mathbf{x} = \mathbf{0}

. Nullity = number of columns without a pivot.

Example 2: Computing nullity

For

> A = \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix} >

(RREF from Example 1):

Pivots in columns 1 and 3. Free variables: columns 2 and 4. $\text{nullity}(A) = 2$ .

Check: $A$ has 4 columns. Rank = 2. $2 + 2 = 4$ ✓ (rank-nullity).

4. The Rank-Nullity Theorem

Theorem (Rank-Nullity). For an $m \times n$ matrix $A$ :

> \text{rank}(A) + \text{nullity}(A) = n >

where $n$ is the number of columns of $A$ . Proof Sketch

Row reduce $A$ to RREF. Let $r = \text{rank}(A)$ = number of pivot columns. Then there are $n - r$ non-pivot columns, each corresponding to a free variable. The null space has dimension $n - r$ (each free variable contributes one basis vector). So $\text{nullity}(A) = n - r = n - \text{rank}(A)$ . Rearranging gives $\text{rank}(A) + \text{nullity}(A) = n$ . Example 3: Applying rank-nullity

$A$ is $5 \times 7$ with rank 3. Find nullity.

$\text{nullity}(A) = 7 - 3 = 4$ .

The null space is a 4-dimensional subspace of $\mathbb{R}^7$ .

4.1 Consequences

Situation	What rank-nullity tells us
Square $n \times n$ , invertible	$\text{rank} = n$ , $\text{nullity} = 0$ — unique solution to $A\mathbf{x}=\mathbf{b}$
Square $n \times n$ , singular	$\text{rank} 0$ — either 0 or ∞ solutions
More columns than rows ( $m < n$ )	$\text{nullity} \geq n - m > 0$ — always non-trivial null space
More rows than columns ( $m > n$ )	$\text{rank} \leq n$ , nullity could be 0

5. Finding Bases for Row Space, Column Space, Null Space

5.1 Row Space Basis

Row reduce to RREF. The non-zero rows of the RREF form a basis for the row space.

Example 4: Row space basis

> A = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 3 \\ 1 & 2 & 0 \end{bmatrix} \xrightarrow{\text{RREF}} \begin{bmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix} >

Basis for row space: $\{(1,2,0), (0,0,1)\}$ .

5.2 Column Space Basis

The original columns corresponding to pivot columns form a basis for the column space.

Example 5: Column space basis

Same $A$ as above. Pivot columns are 1 and 3. Original columns: $\{(1,2,1), (1,3,0)\}$ form a basis for $C(A)$ .

5.3 Null Space Basis

Solve

A\mathbf{x} = \mathbf{0}

, express in parametric form. The vectors multiplying the free parameters form a basis.

Example 6: Null space basis

> A = \begin{bmatrix} 1 & 2 & 0 \\ 0 & 0 & 0 \end{bmatrix} >

Actually let's use a better example.

> A = \begin{bmatrix} 1 & 2 & -1 & 0 \\ 0 & 0 & 1 & 1 \end{bmatrix} >

(RREF).

Pivots in columns 1 and 3. Free variables: $x_2$ , $x_4$ .

$x_1 = -2x_2 + x_3 = -2x_2 - (-x_4) = -2x_2 + x_4$ ? Wait, let's solve properly:

From RREF: $x_1 + 2x_2 - x_3 = 0$ , $x_3 + x_4 = 0$ . From row 2: $x_3 = -x_4$ . From row 1: $x_1 = -2x_2 + x_3 = -2x_2 - x_4$ .

Free: $x_2, x_4$ .

> \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix} = x_2\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix} + x_4\begin{bmatrix} -1 \\ 0 \\ -1 \\ 1 \end{bmatrix} >

Basis for $N(A)$ : $\{(-2,1,0,0), (-1,0,-1,1)\}$ . Nullity = 2.

6. Full Column Rank vs. Full Row Rank

Type	Condition	Properties
Full column rank	$\text{rank}(A) = n$ (number of columns)	Columns are independent; $N(A) = \{\mathbf{0}\}$
Full row rank	$\text{rank}(A) = m$ (number of rows)	Rows are independent; $C(A) = \mathbb{R}^m$
Full rank (square)	$\text{rank}(A) = m = n$	Invertible matrix

Example 7: Full column rank

> A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{bmatrix} >

is $3 \times 2$ . Row reduce shows pivots in both columns. $\text{rank}(A) = 2 = n$ . Nullity = $2 - 2 = 0$ . $N(A) = \{\mathbf{0}\}$ .

7. Edge Cases & Gotchas

Situation	What Happens
Zero matrix	$\text{rank} = 0$ , $\text{nullity} = n$
Identity matrix	$\text{rank} = n$ , $\text{nullity} = 0$
$A$ is $m \times n$ with $m < n$	$\text{rank} \leq m 0$ (always has non-trivial null space)
Rank of a product	$\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))$

8. Common Pitfalls

❌ Pitfall 1: Confusing rank with number of rows

Rank is the dimension of the column space, not the number of rows. A

5 \times 3

matrix can have rank at most 3, not 5.

❌ Pitfall 2: Forgetting rank-nullity applies to columns

Students sometimes use number of rows instead of columns.

\text{rank}(A) + \text{nullity}(A) = n

(columns), not

m

❌ Pitfall 3: Taking RREF rows as column space basis

The RREF rows are a basis for the row space, not the column space. For the column space, use original pivot columns.

9. Formula Summary Table

Concept	Formula	Notes
Rank	$\#\text{pivots} = \dim(C(A))$	$0 \leq \text{rank} \leq \min(m,n)$
Nullity	$\dim(N(A))$	Number of free variables
Rank-Nullity	$\text{rank}(A) + \text{nullity}(A) = n$	$n$ = number of columns
Full column rank	$\text{rank}(A) = n$	$N(A) = \{\mathbf{0}\}$
Full row rank	$\text{rank}(A) = m$	$C(A) = \mathbb{R}^m$
Invertible (square)	$\text{rank}(A) = n$	$A^{-1}$ exists

10. 📝 Practice Questions

Q1: Find rank from RREF

> A = \begin{bmatrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix} >

This is already in REF with 3 pivots. $\text{rank}(A) = 3$ . Q2: Rank and nullity

> A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 2 & 4 & 6 & 8 \\ 1 & 2 & 3 & 4 \end{bmatrix} >

Solution: Rows 1 and 3 are identical; row 2 is $2 \times$ row 1. RREF:

> \begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix} >

. Rank = 1, Nullity = $4 - 1 = 3$ . Q3: Rank-Nullity application

If $A$ is $3 \times 5$ with rank 2, what are the dimensions of $N(A)$ and $C(A)$ ?

Solution: $\dim(C(A)) = \text{rank}(A) = 2$ . $\text{nullity}(A) = 5 - 2 = 3$ . $N(A)$ is a 3-dim subspace of $\mathbb{R}^5$ , $C(A)$ is a 2-dim subspace of $\mathbb{R}^3$ . Q4: Finding null space basis

Find a basis for $N(A)$ where

> A = \begin{bmatrix} 1 & 1 & 0 & 1 \\ 0 & 0 & 1 & 2 \end{bmatrix} >

.

Solution: Already in RREF. Pivots in columns 1, 3. Free: columns 2, 4.

Row 1: $x_1 + x_2 + x_4 = 0 \Rightarrow x_1 = -x_2 - x_4$ Row 2: $x_3 + 2x_4 = 0 \Rightarrow x_3 = -2x_4$

> \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix} = x_2\begin{bmatrix} -1 \\ 1 \\ 0 \\ 0 \end{bmatrix} + x_4\begin{bmatrix} -1 \\ 0 \\ -2 \\ 1 \end{bmatrix} >

Basis: $\{(-1,1,0,0), (-1,0,-2,1)\}$ . Q5: Full row rank

For what $k$ does

> A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & k \end{bmatrix} >

have full row rank?

Solution: Full row rank means $\text{rank}(A) = 3$ (number of rows). But $A$ is $3 \times 2$ , so max rank is $\min(3,2) = 2$ . It's impossible for a $3 \times 2$ matrix to have rank 3.

The question should be: when does it have full column rank?

Row reduce:

> \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & k \end{bmatrix} \xrightarrow{R_2-2R_1, R_3-3R_1} \begin{bmatrix} 1 & 2 \\ 0 & 0 \\ 0 & k-6 \end{bmatrix} >

If $k \neq 6$ , there are 2 pivots ⇒ rank 2 (full column rank). If $k = 6$ , rank 1. Q6: Rank of a product

If $A$ is $3 \times 4$ with rank 2 and $B$ is $4 \times 2$ with rank 2, what can you say about $\text{rank}(AB)$ ?

Solution: $\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B)) = \min(2, 2) = 2$ .

Also, $\text{rank}(AB) \leq \min(3, 2) = 2$ (dimensions of $AB$ are $3 \times 2$ ).

So $\text{rank}(AB) \leq 2$ . It could be 0, 1, or 2. Q7: Rank from determinant

If $A$ is $4 \times 4$ with $\det(A) = 0$ , what is the maximum possible rank?

Solution: $\det(A) = 0$ means $A$ is singular, so $\text{rank}(A) < 4$ . The maximum possible rank is 3. Q8: Invertibility and rank

If $A$ is $5 \times 5$ with $\text{rank}(A) = 5$ , what is $\text{nullity}(A)$ ? Is $A$ invertible?

Solution: $\text{nullity}(A) = 5 - 5 = 0$ . Yes, $A$ is invertible (full rank square matrix). Q9: Row space dimension

If $A$ is $7 \times 4$ with rank 3, what is the dimension of the row space?

Solution: $\dim(\text{row space}) = \text{rank}(A) = 3$ . Q10: Rank of transpose

If $A$ is $5 \times 3$ with rank 2, what is $\text{rank}(A^T)$ ?

Solution: $\text{rank}(A^T) = \text{rank}(A) = 2$ .

🔗 Cross-References

Next topic: Null Space & Column Space
Week 5 (Linear Transformations): Rank-nullity generalizes to linear transformations
Week 6 (Kernel & Image): Kernel = null space, image = column space
Week 11 (Hessian): The Hessian matrix's rank determines if a critical point is degenerate
BSCS2004 (ML Foundations): Low-rank approximations, PCA, dimensionality reduction Join Discord Previous4.1 Basis & DimensionNext5.1 Null Space & Column Space