Matrices: Algebra, Types & Operations

3227 words

16 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Matrices: Algebra, Types & Operations ## 🎯 Learning Objectives After this topic you will be able to: - Identify a matrix by its dimensions $m \times n$ and refer to its entries $a_{ij}$ - Add and subtract matrices of the same size - Multiply a matrix by a scalar and multiply two matrices (when dimensions are comp...

Matrices: Algebra, Types & Operations

🎯 Learning Objectives

After this topic you will be able to:

Identify a matrix by its dimensions $m \times n$ and refer to its entries $a_{ij}$
Add and subtract matrices of the same size
Multiply a matrix by a scalar and multiply two matrices (when dimensions are compatible)
Compute the transpose of a matrix
Recognise diagonal, triangular, identity, symmetric, and scalar matrices
Represent a system of linear equations as a matrix equation $A\mathbf{x} = \mathbf{b}$
Apply the algebraic properties of matrix operations

📋 Prerequisites

Vectors Introduction (this week, previous topic) — vectors are the building blocks of matrices
Basic algebra — operations on real numbers
Matrices organise data in a rectangular grid; they are the natural language of linear algebra

1. What is a Matrix?

1.1 Intuition: A Spreadsheet of Numbers

Think of a matrix as a rectangular spreadsheet. If you run a small business selling three products across four cities, you might arrange your monthly revenue in a table:

	City A	City B	City C	City D
Product 1	1200	850	2100	950
Product 2	1800	920	1750	1100
Product 3	950	1050	1300	780

That table is a

3 \times 4

(3 rows, 4 columns) matrix. Each entry records one number — the revenue for a specific product in a specific city. Matrices are everywhere in data science: datasets (rows = observations, columns = features), images (pixels), neural network weights, and more.

1.2 Formal Definition

Definition (Matrix). An $m \times n$ matrix is a rectangular array of real numbers with $m$ rows and $n$ columns. We write

> A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix} >

The entry in row $i$ , column $j$ is $a_{ij}$ or $A_{ij}$ . We say $A \in \mathbb{R}^{m \times n}$ . Key terminology:

Term	Meaning	Example
Dimension	$m \times n$ (rows $\times$ columns)	$3 \times 4$
Square matrix	$m = n$	$3 \times 3$
Row vector	$1 \times n$ matrix	$[1\; -2\; 5]$
Column vector	$m \times 1$ matrix	$\begin{bmatrix}1\\-2\\5\end{bmatrix}$
Zero matrix	All entries are $0$	$0_{m \times n}$
Main diagonal	Entries $a_{11}, a_{22}, \dots, a_{kk}$	Where row = column

1.3 Notation Shortcuts

We often write a matrix as:

$A = [a_{ij}]$ — shorthand for "the matrix with entry $a_{ij}$ at position $(i,j)$ "
$A_{m \times n}$ — emphasising the dimensions

2. Matrix Operations

2.1 Matrix Addition (and Subtraction)

Two matrices of the same dimensions can be added entry-wise:

(A + B)_{ij} = a_{ij} + b_{ij}

Example 1: Matrix addition

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} >

> A + B = \begin{bmatrix} 1+5 & 2+6 \\ 3+7 & 4+8 \end{bmatrix} = \begin{bmatrix} 6 & 8 \\ 10 & 12 \end{bmatrix} >

Properties:

Commutativity: $A + B = B + A$
Associativity: $(A + B) + C = A + (B + C)$
Identity: $A + 0 = A$ (zero matrix is the additive identity)
Inverse: $A + (-A) = 0$

2.2 Scalar Multiplication

Multiply every entry by the scalar

c

(cA)_{ij} = c \cdot a_{ij}

Example 2: Scalar multiplication

> A = \begin{bmatrix} 1 & -2 \\ 0 & 4 \end{bmatrix},\quad 3A = \begin{bmatrix} 3 & -6 \\ 0 & 12 \end{bmatrix} >

2.3 Matrix Multiplication

This is the most important (and most confusing) operation. Unlike addition, matrix multiplication is not entry-wise.

Definition (Matrix Multiplication). If $A$ is $m \times n$ and $B$ is $n \times p$ , then the product $C = AB$ is $m \times p$ with entries:

> c_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj} >

The $(i,j)$ -entry of $C$ is the dot product of row $i$ of $A$ with column $j$ of $B$ . Critical requirement: The number of columns of $A$ must equal the number of rows of $B$ . (Diagram) Example 3: Matrix multiplication (2×2)

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} >

AB &= \begin{bmatrix} 1(0) + 2(1) & 1(1) + 2(0) \\ 3(0) + 4(1) & 3(1) + 4(0) \end{bmatrix} \\ &= \begin{bmatrix} 0 + 2 & 1 + 0 \\ 0 + 4 & 3 + 0 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix} \end{aligned}

> **Example 4: Matrix multiplication (2×3 by 3×2)** > >

A = \begin{bmatrix} 1 & 0 & -1 \ 2 & 1 & 3 \end{bmatrix}{2 \times 3},\quad B = \begin{bmatrix} 1 & 2 \ 0 & 1 \ -2 & 0 \end{bmatrix}{3 \times 2}

> > c_{11} &= 1(1) + 0(0) + (-1)(-2) = 1 + 0 + 2 = 3 \\\\ c_{12} &= 1(2) + 0(1) + (-1)(0) = 2 + 0 + 0 = 2 \\\\ c_{21} &= 2(1) + 1(0) + 3(-2) = 2 + 0 - 6 = -4 \\\\ c_{22} &= 2(2) + 1(1) + 3(0) = 4 + 1 + 0 = 5 \\end{aligned}

AB = \begin{bmatrix} 3 & 2 \ -4 & 5 \end{bmatrix}_{2 \times 2} Example 5: Matrix × Column Vector

This is the most common case in linear algebra — a matrix multiplying a column vector produces another column vector (a linear combination of the columns).

> A = \begin{bmatrix} 1 & 0 & -1 \\ 2 & 1 & 3 \end{bmatrix},\quad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} >

> A\mathbf{x} = 2\begin{bmatrix}1\\2\end{bmatrix} + (-1)\begin{bmatrix}0\\1\end{bmatrix} + 0\begin{bmatrix}-1\\3\end{bmatrix} = \begin{bmatrix}2\\4\end{bmatrix} + \begin{bmatrix}0\\-1\end{bmatrix} + \begin{bmatrix}0\\0\end{bmatrix} = \begin{bmatrix}2\\3\end{bmatrix} >

Notice: $A\mathbf{x}$ is a linear combination of the columns of $A$ with coefficients from $\mathbf{x}$ .

2.4 Properties of Matrix Multiplication

Property	Formula	Notes
Associativity	$(AB)C = A(BC)$	Always true when defined
Distributivity	$A(B+C) = AB + AC$	Left and right distributivity
NOT commutative	$AB \neq BA$ in general	This is a critical difference from numbers!
Identity	$AI = IA = A$	$I$ = identity matrix
Zero	$A0 = 0$ , $0A = 0$	Zero matrix

Example 6: Non-commutativity

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} >

> AB = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix},\quad BA = \begin{bmatrix} 3 & 4 \\ 1 & 2 \end{bmatrix} >

$AB \neq BA$ in this case.

3. Special Matrix Types

3.1 Identity Matrix

The

n \times n

identity matrix

I_n

has 1's on the main diagonal and 0's elsewhere:

I_n = \begin{bmatrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & 1 \end{bmatrix}

I_n

acts like the number 1 in multiplication:

A I_n = A

and

I_m A = A

for

m \times n

matrix

A

3.2 Diagonal Matrix

A square matrix where all off-diagonal entries are zero:

D = \begin{bmatrix} d_1 & 0 & \dots & 0 \\ 0 & d_2 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & d_n \end{bmatrix} = \text{diag}(d_1, d_2, \dots, d_n)

3.3 Scalar Matrix

A diagonal matrix where all diagonal entries are equal:

D = c I_n

3.4 Triangular Matrices

Upper triangular: all entries below the main diagonal are zero. Lower triangular: all entries above the main diagonal are zero.

U = \begin{bmatrix} 1 & 2 & 3 \\ 0 & 4 & 5 \\ 0 & 0 & 6 \end{bmatrix} \text{(upper)},\qquad L = \begin{bmatrix} 1 & 0 & 0 \\ 2 & 3 & 0 \\ 4 & 5 & 6 \end{bmatrix} \text{(lower)}

3.5 Symmetric Matrix

A square matrix

A

is symmetric if

A = A^T

, i.e.

a_{ij} = a_{ji}

for all

i, j

A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 5 \\ 3 & 5 & 6 \end{bmatrix} \text{is symmetric}

Symmetric matrices appear everywhere: distance matrices, adjacency matrices, covariance matrices, Hessian matrices (Week 11).

3.6 Zero Matrix

All entries are 0. Acts as the additive identity:

A + 0 = A

4. Transpose of a Matrix

Definition (Transpose). The transpose of an $m \times n$ matrix $A$ is the $n \times m$ matrix $A^T$ where $(A^T)_{ij} = a_{ji}$ (rows become columns). Example 7: Transpose

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix}_{3 \times 2},\quad A^T = \begin{bmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{bmatrix}_{2 \times 3} >

Properties of transpose:

Property	Formula
Involution	$(A^T)^T = A$
Addition	$(A + B)^T = A^T + B^T$
Scalar	$(cA)^T = cA^T$
Product reversal	$(AB)^T = B^T A^T$

The product reversal rule is famously easy to remember: "put your socks on before your shoes" ⇒ the transpose reverses the order.

5. Representing Linear Systems as Matrix Equations

A system of

m

linear equations in

n

unknowns:

\begin{cases} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n = b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n = b_2 \\ \vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n = b_m \end{cases}

can be written compactly as:

A\mathbf{x} = \mathbf{b}

where

A

is the

m \times n

coefficient matrix,

\mathbf{x}

is the

n \times 1

unknown vector, and

\mathbf{b}

is the

m \times 1

right-hand side vector.

Example 8: System → Matrix form

> \begin{cases} x + 2y - z = 4 \\ 3x - y + 2z = -1 \\ -x + y + z = 2 \end{cases} \;\Longleftrightarrow\; \begin{bmatrix} 1 & 2 & -1 \\ 3 & -1 & 2 \\ -1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \\ 2 \end{bmatrix} >

6. Edge Cases & Gotchas

Situation	What Happens	Why It Matters
Dimensions mismatch for addition	Operation undefined	Both matrices must be $m \times n$
Inner dimensions don't match for multiplication	$AB$ undefined if $A$ is $m \times n$ and $B$ is $r \times p$ with $n \neq r$	Always check: columns of first = rows of second
Multiplying by zero matrix	Always gives zero matrix	$A \cdot 0_{n \times p} = 0_{m \times p}$
$AB = 0$ but neither $A$ nor $B$ is zero	Possible with matrices!	Unlike real numbers, matrix multiplication has zero divisors
$AB = AC$ does not imply $B = C$	Cancellation fails without invertibility	Matrices have no division in general

7. Common Pitfalls

❌ Pitfall 1: Assuming commutativity

The most common mistake.

AB

and

BA

are usually different, and one may be defined when the other isn't.

❌ Pitfall 2: Confusing multiplication dimensions

For

AB

, the inner dimensions must match:

(m \times \mathbf{n})(\mathbf{n} \times p) = m \times p

❌ Pitfall 3: Forgetting the transpose reversal rule

(AB)^T = B^T A^T

, not

A^T B^T

. Students often forget the reversal.

8. Formula Summary Table

Operation	Formula	Dimension
Addition	$(A + B)_{ij} = a_{ij} + b_{ij}$	Same $m \times n$
Scalar multiplication	$(cA)_{ij} = c \cdot a_{ij}$	Same $m \times n$
Matrix multiplication	$(AB)_{ij} = \sum_k a_{ik} b_{kj}$	$m \times p$ (if $A$ is $m \times n$ , $B$ is $n \times p$ )
Transpose	$(A^T)_{ij} = a_{ji}$	$n \times m$
Matrix × vector	$(A\mathbf{x})_i = \sum_j a_{ij} x_j$	$m \times 1$

9. 📝 Practice Questions

Q1: Matrix addition

> A = \begin{bmatrix} 2 & -1 \\ 0 & 3 \end{bmatrix} >

,

> B = \begin{bmatrix} -2 & 1 \\ 4 & -3 \end{bmatrix} >

. Compute $A + B$ and $A - B$ .

Strategy: Add/subtract corresponding entries.

Solution:

> A + B = \begin{bmatrix} 2+(-2) & -1+1 \\ 0+4 & 3+(-3) \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 4 & 0 \end{bmatrix} >

> A - B = \begin{bmatrix} 2-(-2) & -1-1 \\ 0-4 & 3-(-3) \end{bmatrix} = \begin{bmatrix} 4 & -2 \\ -4 & 6 \end{bmatrix} >

Q2: Matrix multiplication

> A = \begin{bmatrix} 1 & 0 \\ 2 & -1 \end{bmatrix} >

,

> B = \begin{bmatrix} 3 & 1 \\ 0 & -2 \end{bmatrix} >

. Compute $AB$ and $BA$ .

Strategy: Use dot product of rows with columns.

Solution:

> AB = \begin{bmatrix} 1(3)+0(0) & 1(1)+0(-2) \\ 2(3)+(-1)(0) & 2(1)+(-1)(-2) \end{bmatrix} = \begin{bmatrix} 3 & 1 \\ 6 & 4 \end{bmatrix} >

> BA = \begin{bmatrix} 3(1)+1(2) & 3(0)+1(-1) \\ 0(1)+(-2)(2) & 0(0)+(-2)(-1) \end{bmatrix} = \begin{bmatrix} 5 & -1 \\ -4 & 2 \end{bmatrix} >

Note: $AB \neq BA$ — matrices do not commute. Q3: Matrix-vector product

Compute $A\mathbf{x}$ where

> A = \begin{bmatrix} 1 & 2 & 0 \\ -1 & 0 & 3 \end{bmatrix} >

and

> \mathbf{x} = \begin{bmatrix} 2 \\ 1 \\ -1 \end{bmatrix} >

.

Strategy: Linear combination of columns of $A$ using entries of $\mathbf{x}$ .

Solution:

> A\mathbf{x} = 2\begin{bmatrix}1\\-1\end{bmatrix} + 1\begin{bmatrix}2\\0\end{bmatrix} + (-1)\begin{bmatrix}0\\3\end{bmatrix} = \begin{bmatrix}2\\-2\end{bmatrix} + \begin{bmatrix}2\\0\end{bmatrix} + \begin{bmatrix}0\\-3\end{bmatrix} = \begin{bmatrix}4\\-5\end{bmatrix} >

Q4: Transpose properties

Verify $(AB)^T = B^T A^T$ for

> A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} >

,

> B = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix} >

.

Strategy: Compute both sides independently.

Solution:

> AB = \begin{bmatrix} -2 & 1 \\ -4 & 3 \end{bmatrix},\quad (AB)^T = \begin{bmatrix} -2 & -4 \\ 1 & 3 \end{bmatrix} >

> A^T = \begin{bmatrix} 1 & 3 \\ 2 & 4 \end{bmatrix},\quad B^T = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix},\quad B^T A^T = \begin{bmatrix} -2 & -4 \\ 1 & 3 \end{bmatrix} >

They match! ✓ Q5: Symmetric matrix check

Is

> A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 5 \\ 3 & 5 & 6 \end{bmatrix} >

symmetric?

Strategy: Check if $A = A^T$ (i.e., $a_{ij} = a_{ji}$ ).

Solution: $a_{12} = 2 = a_{21}$ , $a_{13} = 3 = a_{31}$ , $a_{23} = 5 = a_{32}$ . Yes, it is symmetric. Q6: System to matrix form

Write the system as $A\mathbf{x} = \mathbf{b}$ :

> \begin{cases} 2x - 3y + z = 5 \\ x + y - 2z = 0 \\ 3x + 2y + 4z = -1 \end{cases} >

Solution:

> \begin{bmatrix} 2 & -3 & 1 \\ 1 & 1 & -2 \\ 3 & 2 & 4 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 5 \\ 0 \\ -1 \end{bmatrix} >

Q7: Diagonal matrix multiplication

If $D = \text{diag}(2, -1, 3)$ and

> A = \begin{bmatrix} 1 & 0 & 2 \\ 3 & 1 & -1 \\ 0 & 1 & 1 \end{bmatrix} >

, compute $DA$ and $AD$ .

Strategy: $D$ times $A$ scales rows of $A$ ; $A$ times $D$ scales columns of $A$ .

Solution:

> DA = \begin{bmatrix} 2(1) & 2(0) & 2(2) \\ -1(3) & -1(1) & -1(-1) \\ 3(0) & 3(1) & 3(1) \end{bmatrix} = \begin{bmatrix} 2 & 0 & 4 \\ -3 & -1 & 1 \\ 0 & 3 & 3 \end{bmatrix} >

> AD = \begin{bmatrix} 1(2) & 0(-1) & 2(3) \\ 3(2) & 1(-1) & -1(3) \\ 0(2) & 1(-1) & 1(3) \end{bmatrix} = \begin{bmatrix} 2 & 0 & 6 \\ 6 & -1 & -3 \\ 0 & -1 & 3 \end{bmatrix} >

Q8: Zero divisors

Find $A, B \neq 0$ such that $AB = 0$ .

Strategy: Try a matrix with a row of zeros, or use projection matrices.

Solution:

> A = \begin{bmatrix} 1 & 0 \\ 2 & 0 \end{bmatrix},\quad B = \begin{bmatrix} 0 & 0 \\ 1 & 1 \end{bmatrix} >

> AB = \begin{bmatrix} 1(0)+0(1) & 1(0)+0(1) \\ 2(0)+0(1) & 2(0)+0(1) \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} >

Both $A$ and $B$ are non-zero, yet their product is zero! Q9: Idempotent matrix

A matrix $P$ is idempotent if $P^2 = P$ . Show that

> P = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} >

is idempotent.

Solution:

> P^2 = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} 1(1)+0(0) & 1(0)+0(0) \\ 0(1)+0(0) & 0(0)+0(0) \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = P >

Q10: Trace of a matrix

The trace of a square matrix $A$ is $\text{tr}(A) = \sum_i a_{ii}$ . Compute $\text{tr}(A)$ for

> A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} >

. Also show $\text{tr}(AB) = \text{tr}(BA)$ for $2 \times 2$ matrices.

Solution: $\text{tr}(A) = 1 + 5 + 9 = 15$ .

For $A, B$ $2 \times 2$ :

> \text{tr}(AB) = a_{11}b_{11} + a_{12}b_{21} + a_{21}b_{12} + a_{22}b_{22} >

> \text{tr}(BA) = b_{11}a_{11} + b_{12}a_{21} + b_{21}a_{12} + b_{22}a_{22} >

These are the same four terms in a different order, so $\text{tr}(AB) = \text{tr}(BA)$ . ✓

🔗 Cross-References

Next topic: Determinants — a single number that captures key properties of a square matrix
Week 2 (Gaussian Elimination): Row operations transform matrices into simpler forms
Week 7 (Similarity): Two matrices can represent the same linear transformation in different bases
BSMA1001 (Maths 1): Matrix methods for solving systems
BSCS2004 (ML Foundations): Design matrices, feature matrices, weight matrices
BSCS3004 (Deep Learning): Neural network weights are matrices; backpropagation uses matrix calculus Join Discord Previous1.1 Vectors IntroductionNext1.3 Determinants