Directional Derivatives & The Gradient

🎯 Learning Objectives

• Compute the gradient vector $\nabla f$ of a multivariable function
• Find the directional derivative in any direction
• Determine the direction of steepest ascent/descent
• Use the gradient to find tangent planes
• Understand the relationship $\nabla f$ points in the direction of greatest increase

📋 Prerequisites

• Partial Derivatives (this week) — components of the gradient
• Dot Product (Week 1) — directional derivative = $\nabla f \cdot \mathbf{u}$
• The gradient is the vector calculus generalisation of the derivative

1. Intuition: Which Way is Up?

• Which direction is steepest uphill (the gradient direction)
• How steep it is (the gradient magnitude) A directional derivative answers: if I walk in a specific direction, how fast does the elevation change? (Diagram)

2. The Gradient Vector

Definition (Gradient). For $f: \mathbb{R}^n \to \mathbb{R}$, the gradient is the vector of all first partial derivatives:

Example 1: Gradient of $f(x,y,z) = x^2y + e^z$

2.1 Properties of the Gradient

3. Directional Derivatives

3.1 Definition

Definition (Directional Derivative). The directional derivative of $f$ at $\mathbf{x}$ in the direction of unit vector $\mathbf{u}$ is:

Example 2: Directional derivative

$f(x,y) = x^2 + y^2$ at $(1,2)$ in direction $(3,4)$.

$\nabla f = (2x, 2y)$, so $\nabla f(1,2) = (2,4)$.

Direction $\mathbf{v} = (3,4)$, unit vector $\mathbf{u} = \frac{(3,4)}{5} = \left(\frac{3}{5}, \frac{4}{5}\right)$.

$D_{\mathbf{u}} f = (2,4) \cdot \left(\frac{3}{5}, \frac{4}{5}\right) = \frac{6}{5} + \frac{16}{5} = \frac{22}{5} = 4.4$.

3.2 Steepest Ascent and Descent

Example 3: Steepest ascent

$f(x,y) = x^2 + y^2$ at $(1,2)$: $\nabla f(1,2) = (2,4)$. Direction of steepest ascent: $\frac{(2,4)}{\sqrt{20}} = \left(\frac{1}{\sqrt{5}}, \frac{2}{\sqrt{5}}\right)$. Rate of steepest ascent: $\|\nabla f\| = \sqrt{4+16} = \sqrt{20} \approx 4.47$. Direction of steepest descent: $-\left(\frac{1}{\sqrt{5}}, \frac{2}{\sqrt{5}}\right)$.

4. Gradient and Level Sets

Theorem. The gradient $\nabla f$ at a point is perpendicular to the level set (contour) passing through that point. This is why $\nabla f$ gives the normal vector to the tangent plane of $f(x,y,z) = c$. Example 4: Gradient perpendicular to level curve

$f(x,y) = x^2 + y^2$ at $(1,1)$. Level curve through $(1,1)$: $x^2 + y^2 = 2$ (a circle). $\nabla f(1,1) = (2,2)$, which points outward from the circle, perpendicular to it ✓.

5. Tangent Planes via Gradient

6. Edge Cases & Gotchas

7. Common Pitfalls

❌ Pitfall 1: Forgetting to normalise the direction vector

❌ Pitfall 2: Thinking $\nabla f$ points uphill at the same rate everywhere

❌ Pitfall 3: Confusing gradient with directional derivative

8. Formula Summary Table

9. 📝 Practice Questions

Q1: Gradient computation

$f(x,y) = 3x^2y - y^3$. Find $\nabla f$.

Solution: $\nabla f = (6xy, 3x^2 - 3y^2)$. Q2: Directional derivative

$f(x,y) = xe^y$ at $(2,0)$ in direction toward $(5,4)$.

Solution: $\nabla f = (e^y, xe^y)$, $\nabla f(2,0) = (1, 2)$. Direction $\mathbf{v} = (5-2, 4-0) = (3,4)$, $\|\mathbf{v}\| = 5$, $\mathbf{u} = (3/5, 4/5)$. $D_{\mathbf{u}} f = (1,2) \cdot (3/5, 4/5) = 3/5 + 8/5 = 11/5 = 2.2$. Q3: Steepest ascent

$f(x,y) = \sin(xy)$ at $(\pi/2, 1)$. Find direction of steepest ascent.

Solution: $\nabla f = (y\cos(xy), x\cos(xy))$. At $(\pi/2, 1)$: $\nabla f = (1\cdot\cos(\pi/2), (\pi/2)\cos(\pi/2)) = (1\cdot 0, (\pi/2)\cdot 0) = (0,0)$. $\nabla f = \mathbf{0}$! This is a critical point. There's no direction of increase. Q4: Rate of change

In what direction does $f(x,y) = x^2 - y^2$ increase most rapidly at $(1,2)$? What's the rate?

Solution: $\nabla f = (2x, -2y)$, $\nabla f(1,2) = (2, -4)$. Direction: $\frac{(2,-4)}{\sqrt{20}} = \left(\frac{1}{\sqrt{5}}, -\frac{2}{\sqrt{5}}\right)$. Rate: $\|\nabla f\| = \sqrt{20} = 2\sqrt{5}$. Q5: Gradient perpendicular to level set

For $f(x,y) = x^2 + y^2$, show $\nabla f$ is perpendicular to the level curve at $(1,1)$.

Solution: $\nabla f(1,1) = (2,2)$. Level curve: $x^2 + y^2 = 2$. Tangent vector to level curve: differentiate implicitly: $2x + 2y y' = 0 \Rightarrow y' = -x/y = -1$ at $(1,1)$. Tangent direction: $(1,-1)$. Dot product: $(2,2)\cdot(1,-1) = 2-2 = 0$ ✓. Q6: Tangent plane

Find the tangent plane to $z = x^2 + 3xy + y^2$ at $(1,1,5)$.

Solution: $f_x = 2x + 3y$, $f_x(1,1) = 5$. $f_y = 3x + 2y$, $f_y(1,1) = 5$. $z - 5 = 5(x-1) + 5(y-1)$ ⇒ $z = 5x + 5y - 5$. Q7: Direction of no change

For $f(x,y) = x^2 + y^2$ at $(1,2)$, find a direction where the directional derivative is 0.

Solution: We need $\nabla f \cdot \mathbf{u} = 0$. $\nabla f(1,2) = (2,4)$. A perpendicular vector: $\mathbf{u} = \frac{(-4,2)}{\sqrt{20}} = \left(-\frac{2}{\sqrt{5}}, \frac{1}{\sqrt{5}}\right)$. Check: $(2,4)\cdot(-2/\sqrt{5}, 1/\sqrt{5}) = (-4+4)/\sqrt{5} = 0$ ✓. Q8: Gradient in 3D

$f(x,y,z) = xyz$. Find $\nabla f(1,2,3)$ and the directional derivative toward $(2,1,0)$.

Solution: $\nabla f = (yz, xz, xy)$, $\nabla f(1,2,3) = (6, 3, 2)$. Direction: $\mathbf{v} = (2-1, 1-2, 0-3) = (1,-1,-3)$, $\|\mathbf{v}\| = \sqrt{11}$. $\mathbf{u} = (1/\sqrt{11}, -1/\sqrt{11}, -3/\sqrt{11})$. $D_{\mathbf{u}} f = (6,3,2)\cdot(1/\sqrt{11}, -1/\sqrt{11}, -3/\sqrt{11}) = (6 - 3 - 6)/\sqrt{11} = -3/\sqrt{11}$. Q9: Maximum rate of change

Find the maximum rate of change of $f(x,y,z) = x\ln(yz)$ at $(1,1,1)$.

Solution: $\nabla f = (\ln(yz), x/z, x/y)$. At $(1,1,1)$: $\nabla f = (0, 1, 1)$. Maximum rate = $\|\nabla f\| = \sqrt{0+1+1} = \sqrt{2}$. Q10: Gradient descent direction

If you're at $(2,1)$ on $f(x,y) = x^2 + 2y^2$, which direction should you move to descend most steeply?

Solution: $\nabla f = (2x, 4y)$, $\nabla f(2,1) = (4,4)$. Direction of steepest descent = $-\nabla f = (-4,-4)$, or unit vector $\left(-\frac{1}{\sqrt{2}}, -\frac{1}{\sqrt{2}}\right)$.

🔗 Cross-References

• Next topic: Limits & Continuity
• Week 10 (Tangent Planes): Direct application of gradient
• Week 11 (Hessian): Gradient's derivative is the Hessian
• BSCS3004 (Deep Learning): Gradient descent is the core optimisation algorithm Join Discord Previous9.1 Multivariable FunctionsNext9.3 Limits & Continuity