Matrices: Algebra, Types & Operations

3227 words

16 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Matrices: Algebra, Types & Operations ## 🎯 Learning Objectives After this topic you will be able to: - Identify a matrix by its dimensions $m \times n$ and refer to its entries $a_{ij}$ - Add and subtract matrices of the same size - Multiply a matrix by a scalar and multiply two matrices (when dimensions are comp...

Matrices: Algebra, Types & Operations

🎯 Learning Objectives

After this topic you will be able to:

Identify a matrix by its dimensions $m \times n$ and refer to its entries $a_{ij}$
Add and subtract matrices of the same size
Multiply a matrix by a scalar and multiply two matrices (when dimensions are compatible)
Compute the transpose of a matrix
Recognise diagonal, triangular, identity, symmetric, and scalar matrices
Represent a system of linear equations as a matrix equation $A\mathbf{x} = \mathbf{b}$
Apply the algebraic properties of matrix operations

📋 Prerequisites

Vectors Introduction (this week, previous topic) — vectors are the building blocks of matrices
Basic algebra — operations on real numbers
Matrices organise data in a rectangular grid; they are the natural language of linear algebra

1. What is a Matrix?

1.1 Intuition: A Spreadsheet of Numbers

Think of a matrix as a rectangular spreadsheet. If you run a small business selling three products across four cities, you might arrange your monthly revenue in a table:

	City A	City B	City C	City D
Product 1	1200	850	2100	950
Product 2	1800	920	1750	1100
Product 3	950	1050	1300	780

That table is a

3 \times 4

(3 rows, 4 columns) matrix. Each entry records one number — the revenue for a specific product in a specific city. Matrices are everywhere in data science: datasets (rows = observations, columns = features), images (pixels), neural network weights, and more.

1.2 Formal Definition

Definition (Matrix). An $m \times n$ matrix is a rectangular array of real numbers with $m$ rows and $n$ columns. We write

> A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} >

The entry in row $i$ , column $j$ is $a_{ij}$ or $A_{ij}$ . We say $A \in \mathbb{R}^{m \times n}$ . Key terminology:

Term	Meaning	Example
Dimension	$m \times n$ (rows $\times$ columns)	$3 \times 4$
Square matrix	$m = n$	$3 \times 3$
Row vector	$1 \times n$ matrix	$[1\; -2\; 5]$
Column vector	$m \times 1$ matrix	$\begin{bmatrix}1\\-2\\5\end{bmatrix}$
Zero matrix	All entries are $0$	$0_{m \times n}$
Main diagonal	Entries $a_{11}, a_{22}, \dots, a_{kk}$	Where row = column

1.3 Notation Shortcuts

We often write a matrix as:

$A = [a_{ij}]$ — shorthand for "the matrix with entry $a_{ij}$ at position $(i,j)$ "
$A_{m \times n}$ — emphasising the dimensions

2. Matrix Operations

2.1 Matrix Addition (and Subtraction)

Two matrices of the same dimensions can be added entry-wise:

(A + B)_{ij} = a_{ij} + b_{ij}

Example 1: Matrix addition

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} >

> A + B = \begin{bmatrix} 1+5 & 2+6 \\ 3+7 & 4+8 \end{bmatrix} = \begin{bmatrix} 6 & 8 \\ 10 & 12 \end{bmatrix} >

Properties:

Commutativity: $A + B = B + A$
Associativity: $(A + B) + C = A + (B + C)$
Identity: $A + 0 = A$ (zero matrix is the additive identity)
Inverse: $A + (-A) = 0$

2.2 Scalar Multiplication

Multiply every entry by the scalar

c

(cA)_{ij} = c \cdot a_{ij}

Example 2: Scalar multiplication

> A = \begin{bmatrix} 1 & -2 \\ 0 & 4 \end{bmatrix},\quad 3A = \begin{bmatrix} 3 & -6 \\ 0 & 12 \end{bmatrix} >

2.3 Matrix Multiplication

This is the most important (and most confusing) operation. Unlike addition, matrix multiplication is not entry-wise.

Definition (Matrix Multiplication). If $A$ is $m \times n$ and $B$ is $n \times p$ , then the product $C = AB$ is $m \times p$ with entries:

> c_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj} >

The $(i,j)$ -entry of $C$ is the dot product of row $i$ of $A$ with column $j$ of $B$ . Critical requirement: The number of columns of $A$ must equal the number of rows of $B$ . (Diagram) Example 3: Matrix multiplication (2×2)

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} >

AB &= \begin{bmatrix} 1(0) + 2(1) & 1(1) + 2(0) \\ 3(0) + 4(1) & 3(1) + 4(0) \end{bmatrix} \\ &= \begin{bmatrix} 0 + 2 & 1 + 0 \\ 0 + 4 & 3 + 0 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix} \end{aligned}

> **Example 4: Matrix multiplication (2×3 by 3×2)** > >

A = \begin{bmatrix} 1 & 0 & -1 \ 2 & 1 & 3 \end{bmatrix}{2 \times 3},\quad B = \begin{bmatrix} 1 & 2 \ 0 & 1 \ -2 & 0 \end{bmatrix}{3 \times 2}

> > c_{11} &= 1(1) + 0(0) + (-1)(-2) = 1 + 0 + 2 = 3 \\\\ c_{12} &= 1(2) + 0(1) + (-1)(0) = 2 + 0 + 0 = 2 \\\\ c_{21} &= 2(1) + 1(0) + 3(-2) = 2 + 0 - 6 = -4 \\\\ c_{22} &= 2(2) + 1(1) + 3(0) = 4 + 1 + 0 = 5 \\end{aligned}

AB = \begin{bmatrix} 3 & 2 \ -4 & 5 \end{bmatrix}_{2 \times 2} Example 5: Matrix × Column Vector

This is the most common case in linear algebra — a matrix multiplying a column vector produces another column vector (a linear combination of the columns).

> A = \begin{bmatrix} 1 & 0 & -1 \\ 2 & 1 & 3 \end{bmatrix},\quad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} >

> A\mathbf{x} = 2\begin{bmatrix}1\\2\end{bmatrix} + (-1)\begin{bmatrix}0\\1\end{bmatrix} + 0\begin{bmatrix}-1\\3\end{bmatrix} = \begin{bmatrix}2\\4\end{bmatrix} + \begin{bmatrix}0\\-1\end{bmatrix} + \begin{bmatrix}0\\0\end{bmatrix} = \begin{bmatrix}2\\3\end{bmatrix} >

Notice: $A\mathbf{x}$ is a linear combination of the columns of $A$ with coefficients from $\mathbf{x}$ .

2.4 Properties of Matrix Multiplication

Property	Formula	Notes
Associativity	$(AB)C = A(BC)$	Always true when defined
Distributivity	$A(B+C) = AB + AC$	Left and right distributivity
NOT commutative	$AB \neq BA$ in general	This is a critical difference from numbers!
Identity	$AI = IA = A$	$I$ = identity matrix
Zero	$A0 = 0$ , $0A = 0$	Zero matrix

Example 6: Non-commutativity

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} >

> AB = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix},\quad BA = \begin{bmatrix} 3 & 4 \\ 1 & 2 \end{bmatrix} >

$AB \neq BA$ in this case.

3. Special Matrix Types

3.1 Identity Matrix

The

n \times n

identity matrix

I_n

has 1's on the main diagonal and 0's elsewhere:

I_n = \begin{bmatrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & 1 \end{bmatrix}

I_n

acts like the number 1 in multiplication:

A I_n = A

and

I_m A = A

for

m \times n

matrix

A

3.2 Diagonal Matrix

A square matrix where all off-diagonal entries are zero:

D = \begin{bmatrix} d_1 & 0 & \dots & 0 \\ 0 & d_2 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & d_n \end{bmatrix} = \text{diag}(d_1, d_2, \dots, d_n)

3.3 Scalar Matrix

A diagonal matrix where all diagonal entries are equal:

D = c I_n

3.4 Triangular Matrices

Upper triangular: all entries below the main diagonal are zero. Lower triangular: all entries above the main diagonal are zero.

U = \begin{bmatrix} 1 & 2 & 3 \\ 0 & 4 & 5 \\ 0 & 0 & 6 \end{bmatrix} \text{(upper)},\qquad L = \begin{bmatrix} 1 & 0 & 0 \\ 2 & 3 & 0 \\ 4 & 5 & 6 \end{bmatrix} \text{(lower)}

3.5 Symmetric Matrix

A square matrix

A

is symmetric if

A = A^T

, i.e.

a_{ij} = a_{ji}

for all

i, j

A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 5 \\ 3 & 5 & 6 \end{bmatrix} \text{is symmetric}

Symmetric matrices appear everywhere: distance matrices, adjacency matrices, covariance matrices, Hessian matrices (Week 11).

3.6 Zero Matrix

All entries are 0. Acts as the additive identity:

A + 0 = A

4. Transpose of a Matrix

Definition (Transpose). The transpose of an $m \times n$ matrix $A$ is the $n \times m$ matrix $A^T$ where $(A^T)_{ij} = a_{ji}$ (rows become columns). Example 7: Transpose

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix}_{3 \times 2},\quad A^T = \begin{bmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{bmatrix}_{2 \times 3} >

Properties of transpose:

Property	Formula
Involution	$(A^T)^T = A$
Addition	$(A + B)^T = A^T + B^T$
Scalar	$(cA)^T = cA^T$
Product reversal	$(AB)^T = B^T A^T$

The product reversal rule is famously easy to remember: "put your socks on before your shoes" ⇒ the transpose reverses the order.

5. Representing Linear Systems as Matrix Equations

A system of

m

linear equations in

n

unknowns:

\begin{cases} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n = b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n = b_2 \\ \vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n = b_m \end{cases}

can be written compactly as:

A\mathbf{x} = \mathbf{b}

where

A

is the

m \times n

coefficient matrix,

\mathbf{x}

is the

n \times 1

unknown vector, and

\mathbf{b}

is the

m \times 1

right-hand side vector.

Example 8: System → Matrix form

> \begin{cases} x + 2y - z = 4 \\ 3x - y + 2z = -1 \\ -x + y + z = 2 \end{cases} \;\Longleftrightarrow\; \begin{bmatrix} 1 & 2 & -1 \\ 3 & -1 & 2 \\ -1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \\ 2 \end{bmatrix} >

6. Edge Cases & Gotchas

Situation	What Happens	Why It Matters
Dimensions mismatch for addition	Operation undefined	Both matrices must be $m \times n$
Inner dimensions don't match for multiplication	$AB$ undefined if $A$ is $m \times n$ and $B$ is $r \times p$ with $n \neq r$	Always check: columns of first = rows of second
Multiplying by zero matrix	Always gives zero matrix	$A \cdot 0_{n \times p} = 0_{m \times p}$
$AB = 0$ but neither $A$ nor $B$ is zero	Possible with matrices!	Unlike real numbers, matrix multiplication has zero divisors
$AB = AC$ does not imply $B = C$	Cancellation fails without invertibility	Matrices have no division in general

7. Common Pitfalls

❌ Pitfall 1: Assuming commutativity

The most common mistake.

AB

and

BA

are usually different, and one may be defined when the other isn't.

❌ Pitfall 2: Confusing multiplication dimensions

For

AB

, the inner dimensions must match:

(m \times \mathbf{n})(\mathbf{n} \times p) = m \times p

❌ Pitfall 3: Forgetting the transpose reversal rule

(AB)^T = B^T A^T

, not

A^T B^T

. Students often forget the reversal.

8. Formula Summary Table

Operation	Formula	Dimension
Addition	$(A + B)_{ij} = a_{ij} + b_{ij}$	Same $m \times n$
Scalar multiplication	$(cA)_{ij} = c \cdot a_{ij}$	Same $m \times n$
Matrix multiplication	$(AB)_{ij} = \sum_k a_{ik} b_{kj}$	$m \times p$ (if $A$ is $m \times n$ , $B$ is $n \times p$ )
Transpose	$(A^T)_{ij} = a_{ji}$	$n \times m$
Matrix × vector	$(A\mathbf{x})_i = \sum_j a_{ij} x_j$	$m \times 1$

9. 📝 Practice Questions

Q1: Matrix addition

> A = \begin{bmatrix} 2 & -1 \\ 0 & 3 \end{bmatrix} >

,

> B = \begin{bmatrix} -2 & 1 \\ 4 & -3 \end{bmatrix} >

. Compute $A + B$ and $A - B$ .

Strategy: Add/subtract corresponding entries.

Solution:

> A + B = \begin{bmatrix} 2+(-2) & -1+1 \\ 0+4 & 3+(-3) \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 4 & 0 \end{bmatrix} >

> A - B = \begin{bmatrix} 2-(-2) & -1-1 \\ 0-4 & 3-(-3) \end{bmatrix} = \begin{bmatrix} 4 & -2 \\ -4 & 6 \end{bmatrix} >

Q2: Matrix multiplication

> A = \begin{bmatrix} 1 & 0 \\ 2 & -1 \end{bmatrix} >

,

> B = \begin{bmatrix} 3 & 1 \\ 0 & -2 \end{bmatrix} >

. Compute $AB$ and $BA$ .

Strategy: Use dot product of rows with columns.

Solution:

> AB = \begin{bmatrix} 1(3)+0(0) & 1(1)+0(-2) \\ 2(3)+(-1)(0) & 2(1)+(-1)(-2) \end{bmatrix} = \begin{bmatrix} 3 & 1 \\ 6 & 4 \end{bmatrix} >

> BA = \begin{bmatrix} 3(1)+1(2) & 3(0)+1(-1) \\ 0(1)+(-2)(2) & 0(0)+(-2)(-1) \end{bmatrix} = \begin{bmatrix} 5 & -1 \\ -4 & 2 \end{bmatrix} >

Note: $AB \neq BA$ — matrices do not commute. Q3: Matrix-vector product

Compute $A\mathbf{x}$ where

> A = \begin{bmatrix} 1 & 2 & 0 \\ -1 & 0 & 3 \end{bmatrix} >

and

> \mathbf{x} = \begin{bmatrix} 2 \\ 1 \\ -1 \end{bmatrix} >

.

Strategy: Linear combination of columns of $A$ using entries of $\mathbf{x}$ .

Solution:

> A\mathbf{x} = 2\begin{bmatrix}1\\-1\end{bmatrix} + 1\begin{bmatrix}2\\0\end{bmatrix} + (-1)\begin{bmatrix}0\\3\end{bmatrix} = \begin{bmatrix}2\\-2\end{bmatrix} + \begin{bmatrix}2\\0\end{bmatrix} + \begin{bmatrix}0\\-3\end{bmatrix} = \begin{bmatrix}4\\-5\end{bmatrix} >

Q4: Transpose properties

Verify $(AB)^T = B^T A^T$ for

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} >

,

> B = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix} >

.

Strategy: Compute both sides independently.

Solution:

> AB = \begin{bmatrix} -2 & 1 \\ -4 & 3 \end{bmatrix},\quad (AB)^T = \begin{bmatrix} -2 & -4 \\ 1 & 3 \end{bmatrix} >

> A^T = \begin{bmatrix} 1 & 3 \\ 2 & 4 \end{bmatrix},\quad B^T = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix},\quad B^T A^T = \begin{bmatrix} -2 & -4 \\ 1 & 3 \end{bmatrix} >

They match! ✓ Q5: Symmetric matrix check

Is

> A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 5 \\ 3 & 5 & 6 \end{bmatrix} >

symmetric?

Strategy: Check if $A = A^T$ (i.e., $a_{ij} = a_{ji}$ ).

Solution: $a_{12} = 2 = a_{21}$ , $a_{13} = 3 = a_{31}$ , $a_{23} = 5 = a_{32}$ . Yes, it is symmetric. Q6: System to matrix form

Write the system as $A\mathbf{x} = \mathbf{b}$ :

> \begin{cases} 2x - 3y + z = 5 \\ x + y - 2z = 0 \\ 3x + 2y + 4z = -1 \end{cases} >

Solution:

> \begin{bmatrix} 2 & -3 & 1 \\ 1 & 1 & -2 \\ 3 & 2 & 4 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 5 \\ 0 \\ -1 \end{bmatrix} >

Q7: Diagonal matrix multiplication

If $D = \text{diag}(2, -1, 3)$ and

> A = \begin{bmatrix} 1 & 0 & 2 \\ 3 & 1 & -1 \\ 0 & 1 & 1 \end{bmatrix} >

, compute $DA$ and $AD$ .

Strategy: $D$ times $A$ scales rows of $A$ ; $A$ times $D$ scales columns of $A$ .

Solution:

> DA = \begin{bmatrix} 2(1) & 2(0) & 2(2) \\ -1(3) & -1(1) & -1(-1) \\ 3(0) & 3(1) & 3(1) \end{bmatrix} = \begin{bmatrix} 2 & 0 & 4 \\ -3 & -1 & 1 \\ 0 & 3 & 3 \end{bmatrix} >

> AD = \begin{bmatrix} 1(2) & 0(-1) & 2(3) \\ 3(2) & 1(-1) & -1(3) \\ 0(2) & 1(-1) & 1(3) \end{bmatrix} = \begin{bmatrix} 2 & 0 & 6 \\ 6 & -1 & -3 \\ 0 & -1 & 3 \end{bmatrix} >

Q8: Zero divisors

Find $A, B \neq 0$ such that $AB = 0$ .

Strategy: Try a matrix with a row of zeros, or use projection matrices.

Solution:

> A = \begin{bmatrix} 1 & 0 \\ 2 & 0 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 0 \\ 1 & 1 \end{bmatrix} >

> AB = \begin{bmatrix} 1(0)+0(1) & 1(0)+0(1) \\ 2(0)+0(1) & 2(0)+0(1) \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} >

Both $A$ and $B$ are non-zero, yet their product is zero! Q9: Idempotent matrix

A matrix $P$ is idempotent if $P^2 = P$ . Show that

> P = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} >

is idempotent.

Solution:

> P^2 = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} 1(1)+0(0) & 1(0)+0(0) \\ 0(1)+0(0) & 0(0)+0(0) \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = P >

Q10: Trace of a matrix

The trace of a square matrix $A$ is $\text{tr}(A) = \sum_i a_{ii}$ . Compute $\text{tr}(A)$ for

> A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} >

. Also show $\text{tr}(AB) = \text{tr}(BA)$ for $2 \times 2$ matrices.

Solution: $\text{tr}(A) = 1 + 5 + 9 = 15$ .

For $A, B$ $2 \times 2$ :

> \text{tr}(AB) = a_{11}b_{11} + a_{12}b_{21} + a_{21}b_{12} + a_{22}b_{22} >

> \text{tr}(BA) = b_{11}a_{11} + b_{12}a_{21} + b_{21}a_{12} + b_{22}a_{22} >

These are the same four terms in a different order, so $\text{tr}(AB) = \text{tr}(BA)$ . ✓

🔗 Cross-References

Next topic: Determinants — a single number that captures key properties of a square matrix
Week 2 (Gaussian Elimination): Row operations transform matrices into simpler forms
Week 7 (Similarity): Two matrices can represent the same linear transformation in different bases
BSMA1001 (Maths 1): Matrix methods for solving systems
BSCS2004 (ML Foundations): Design matrices, feature matrices, weight matrices
BSCS3004 (Deep Learning): Neural network weights are matrices; backpropagation uses matrix calculus Join Discord Previous1.1 Vectors IntroductionNext1.3 Determinants

What this note is really saying

Matrices: Algebra, Types & Operations

🎯 Learning Objectives

📋 Prerequisites

1. What is a Matrix?

1.1 Intuition: A Spreadsheet of Numbers

1.2 Formal Definition

1.3 Notation Shortcuts

2. Matrix Operations

2.1 Matrix Addition (and Subtraction)

2.2 Scalar Multiplication

2.3 Matrix Multiplication

2.4 Properties of Matrix Multiplication

3. Special Matrix Types

3.1 Identity Matrix

3.2 Diagonal Matrix

3.3 Scalar Matrix

3.4 Triangular Matrices

3.5 Symmetric Matrix

3.6 Zero Matrix

4. Transpose of a Matrix

5. Representing Linear Systems as Matrix Equations

6. Edge Cases & Gotchas

7. Common Pitfalls

❌ Pitfall 1: Assuming commutativity

❌ Pitfall 2: Confusing multiplication dimensions

❌ Pitfall 3: Forgetting the transpose reversal rule

8. Formula Summary Table

9. 📝 Practice Questions

🔗 Cross-References

Intelligence Hub