Expectation and Variance — Properties and Applications

🎯 Learning Objectives

• Apply linearity of expectation $E(aX + bY) = aE(X) + bE(Y)$
• Compute $E(g(X))$ for various functions g
• Understand the variance formula and its properties
• Compute covariance and understand its relationship to variance
• Apply these concepts to portfolio theory and decision making

📋 Prerequisites

• Discrete Random Variables (15-discrete-random-variables) — basic expectation and variance
• Continuous Random Variables (16-continuous-random-variables) — expectation via integration
• Basic algebra

---

📖 Core Content

18.1 Intuition: Why Expectation Properties Matter

• Compute expected values of complex expressions without starting from scratch
• Understand how random variables combine (sums, products, etc.)
• Simplify calculations dramatically

Everyday analogy: If you know the expected return of stocks and bonds separately, linearity of expectation tells you the expected return of a portfolio that combines them. 🔑 Key Insight: Expectation is linear, but variance is NOT. This distinction is crucial.

18.2 Properties of Expected Value

18.2.1 Constants

$$E(c) = c$$

18.2.2 Linearity (The Most Important Property)

$$E(aX + bY) = aE(X) + bE(Y)$$

18.2.3 Expected Value of a Function

$$E(g(X)) = \begin{cases} \sum g(x) \cdot p(x) & \text{(discrete)} \\ \int g(x) \cdot f(x) \, dx & \text{(continuous)} \end{cases}$$

18.3 Properties of Variance

18.3.1 Definition

$$Var(X) = E[(X - \mu)^2] = E(X^2) - [E(X)]^2$$

18.3.2 Constant

$$Var(c) = 0$$

18.3.3 Linear Transformation

$$Var(aX + b) = a^2 Var(X)$$

18.3.4 Sum of Two Variables

$$Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)$$ $$Var(X + Y) = Var(X) + Var(Y)$$

18.3.5 Difference of Two Variables

$$Var(X - Y) = Var(X) + Var(Y) - 2Cov(X, Y)$$ $$Var(X - Y) = Var(X) + Var(Y)$$

18.4 Covariance

18.4.1 Definition

$$\text{Cov}(X, Y) = E[(X - \mu_X)(Y - \mu_Y)] = E(XY) - E(X)E(Y)$$

18.4.2 Interpretation

• Positive covariance: X and Y tend to move in the same direction
• Negative covariance: X and Y tend to move in opposite directions
• Zero covariance: No linear relationship (but could still have non-linear dependence!)

18.4.3 Properties

1. $Cov(X, Y) = Cov(Y, X)$ (symmetric)
2. $Cov(X, X) = Var(X)$
3. $Cov(aX, bY) = ab \cdot Cov(X, Y)$
4. $Cov(X + Y, Z) = Cov(X, Z) + Cov(Y, Z)$

18.5 Correlation Coefficient

$$\rho(X, Y) = \frac{Cov(X, Y)}{\sigma_X \sigma_Y}$$

• $-1 \leq \rho \leq 1$
• $\rho = \pm 1$ means perfect linear relationship
• $\rho = 0$ means no linear relationship
• Unitless (unlike covariance)

18.6 Worked Examples

Example 1: Linearity of Expectation (Easy)

$$E(2X + 3Y - 5) = 2E(X) + 3E(Y) - 5 = 2(10) + 3(20) - 5 = 20 + 60 - 5 = 75$$

Example 2: Variance of a Linear Combination (Medium)

Example 3: Portfolio Risk (Harder)

$$E(R_P) = 0.4(8\%) + 0.6(12\%) = 3.2\% + 7.2\% = 10.4\%$$ $$Var(R_P) = Var(0.4A + 0.6B)$$ $$= 0.4^2Var(A) + 0.6^2Var(B) + 2(0.4)(0.6)Cov(A,B)$$ $$Var(R_P) = 0.16(225) + 0.36(400) + 2(0.24)(90)$$ $$= 36 + 144 + 43.2 = 223.2$$ $$\sigma_P = \sqrt{223.2} \approx 14.94\%$$

18.7 Moment Generating Functions (Brief Intro)

• 1st moment = mean
• 2nd moment about mean = variance The moment generating function (MGF) $M_X(t) = E(e^{tX})$ generates all moments. We'll use this in BSMA1004.

18.8 Chebyshev's Inequality

$$P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}$$

18.9 Decision Theory Applications

---

📐 Key Formulas / Concepts

---

⚠️ Common Pitfalls

Pitfall 1: Assuming $E(g(X)) = g(E(X))$

Pitfall 2: Adding Variances Instead of Standard Deviations

Pitfall 3: Forgetting Covariance in Variance of Sums

---

📝 Practice Questions

Q1: Linearity

</strong>

$E(X) = 5$, $E(Y) = -2$. Find $E(3X - 2Y + 7)$.

<details> <strong>Solution</strong>

$E(3X - 2Y + 7) = 3E(X) - 2E(Y) + 7 = 3(5) - 2(-2) + 7 = 15 + 4 + 7 = 26$

$\boxed{26}$

</details> > **Q2: Variance of Sum** > > </strong> > > $Var(X) = 9$, $Var(Y) = 16$, $Cov(X,Y) = 4$. Find $Var(X+Y)$. > > <details> <strong>Solution</strong> > > $Var(X+Y) = 9 + 16 + 2(4) = 9 + 16 + 8 = 33$ > > $\boxed{33}$ </details> > **Q3: Covariance Calculation** > > </strong> > > Joint distribution of X and Y: > >

Find $Cov(X,Y)$.

<details> <strong>Solution</strong>

Marginal PMFs: $P(X=0) = 0.1+0.2+0 = 0.3$, $P(X=1) = 0.1+0.3+0.3 = 0.7$ $E(X) = 0(0.3) + 1(0.7) = 0.7$

$P(Y=0) = 0.1+0.1 = 0.2$, $P(Y=1) = 0.2+0.3 = 0.5$, $P(Y=2) = 0+0.3 = 0.3$ $E(Y) = 0(0.2) + 1(0.5) + 2(0.3) = 0 + 0.5 + 0.6 = 1.1$

$E(XY) = \sum\sum xy \cdot p(x,y)$ $= 0(0)(0.1) + 0(1)(0.2) + 0(2)(0) + 1(0)(0.1) + 1(1)(0.3) + 1(2)(0.3)$ $= 0 + 0 + 0 + 0 + 0.3 + 0.6 = 0.9$

$Cov(X,Y) = 0.9 - (0.7)(1.1) = 0.9 - 0.77 = 0.13$

$\boxed{Cov(X,Y) = 0.13}$

</details> > **Q4: Correlation** > > </strong> > > Using Q3, find $\rho(X,Y)$ if $Var(X) = 0.21$ and $Var(Y) = 0.49$. > > <details> <strong>Solution</strong> > > $\rho = \frac{0.13}{\sqrt{0.21} \times \sqrt{0.49}} = \frac{0.13}{0.458 \times 0.7} = \frac{0.13}{0.321} \approx 0.405$ > > There's a moderate positive correlation. > > $\boxed{\rho \approx 0.405}$ </details> > **Q5: Chebyshev's Inequality** > > A distribution has mean 100 and standard deviation 15. Use Chebyshev to find the minimum percentage of data within 70 to 130. > > <details> <strong>Solution</strong> > > 70 and 130 are 30 units from the mean. 30/15 = 2 standard deviations. > > $k = 2$, so at most $1/2^2 = 1/4 = 25\%$ of data is more than 2 s.d. away. > > Therefore, at least $1 - 1/4 = 75\%$ of data falls within 70 to 130. > > **Better than Chebyshev?** If the distribution is known to be approximately normal, the empirical rule gives 95%. But Chebyshev works for ANY distribution. > > $\boxed{\text{At least } 75\%}$ </details> > **Q6: Portfolio Diversification** > > </strong> > > Two assets: $Var(A) = 100$, $Var(B) = 400$, $\rho = -0.5$. Find the portfolio weight w that minimizes variance for a portfolio $P = wA + (1-w)B$. > > <details> <strong>Solution</strong> > > $Var(P) = w^2(100) + (1-w)^2(400) + 2w(1-w)(-0.5)(\sqrt{100}\sqrt{400})$ > > $= 100w^2 + 400(1 - 2w + w^2) + 2w(1-w)(-0.5)(200)$ > > $= 100w^2 + 400 - 800w + 400w^2 + 2w(1-w)(-100)$ > > $= 100w^2 + 400 - 800w + 400w^2 - 200w(1-w)$ > > $= 500w^2 - 800w + 400 - 200w + 200w^2$ > > $= 700w^2 - 1000w + 400$ > > **Minimize:** Differentiate and set to 0: $1400w - 1000 = 0$ $w = 1000/1400 = 5/7 \approx 0.714$ > > So invest about 71.4% in A and 28.6% in B for minimum variance. > > $\boxed{w = 5/7 \approx 0.714}$ </details> > **Q7: Expected Value of Dice Game** > > Roll a fair die. You win ₹10 if you get a 6, otherwise you roll again. If you roll again, you win ₹4 if you get 4-6, ₹1 if you get 1-3. Find the expected winnings. > > <details> <strong>Solution</strong> > > **Strategy:** First roll: > > - P(6) = 1/6, win ₹10. > - P(not 6) = 5/6, go to second roll. > > **Second roll (given we got here):** > > - P(4-6) = 3/6 = 1/2, win ₹4 > - P(1-3) = 3/6 = 1/2, win ₹1 > > $E(\text{second roll}) = 4(1/2) + 1(1/2) = 2.5$ > > **Overall:** $E(X) = 10(1/6) + 2.5(5/6) = 10/6 + 12.5/6 = 22.5/6 = 3.75$ > > Expected winnings are ₹3.75. > > $\boxed{₹3.75}$ </details> > **Q8: Decision Tree** > > A company can develop Product A (cost ₹2M, revenue depends on demand) or Product B (cost ₹1M). For Product A, high demand (P=0.7) gives ₹10M revenue, low demand gives ₹1M. For Product B, high demand (P=0.4) gives ₹5M revenue, low demand gives ₹0.5M. Which product maximizes expected profit? > > <details> <strong>Solution</strong> > > **Product A:** $E(\text{revenue}) = 10(0.7) + 1(0.3) = 7 + 0.3 = 7.3$M $E(\text{profit}) = 7.3 - 2 = 5.3$M > > **Product B:** $E(\text{revenue}) = 5(0.4) + 0.5(0.6) = 2 + 0.3 = 2.3$M $E(\text{profit}) = 2.3 - 1 = 1.3$M > > **Product A** has higher expected profit (5.3M vs 1.3M). > > But Product A is riskier — could lose 1M (if low demand, revenue 1M minus cost 2M). Risk-averse decision-makers might prefer Product B. > > $\boxed{\text{Product A: } ₹5.3\text{M expected profit}}$ </details> > **Q9: Variance of Sample Mean** > > $X_1, X_2, ..., X_n$ are independent with $E(X_i) = \mu$ and $Var(X_i) = \sigma^2$. > > Find $E(\bar{X})$ and $Var(\bar{X})$ where $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$. > > <details> <strong>Solution</strong> > > **Mean of sample mean:** $E(\bar{X}) = E\left(\frac{1}{n}\sum X_i\right) = \frac{1}{n}\sum E(X_i) = \frac{1}{n}(n\mu) = \mu$ > > The sample mean is an **unbiased estimator** of the population mean. > > **Variance of sample mean:** $Var(\bar{X}) = Var\left(\frac{1}{n}\sum X_i\right) = \frac{1}{n^2}\sum Var(X_i) = \frac{1}{n^2}(n\sigma^2) = \frac{\sigma^2}{n}$ > > **Key insight:** The variance of the sample mean decreases as sample size increases. This is why larger samples give more precise estimates. > > $\boxed{E(\bar{X}) = \mu,\ Var(\bar{X}) = \sigma^2/n}$ </details> > **Q10: Application — Insurance Risk** > > An insurance company sells 10,000 policies. Each policy has: > > - P(no claim) = 0.9, profit = ₹1000 > - P(small claim) = 0.08, profit = ₹-5000 > - P(large claim) = 0.02, profit = ₹-20,000 > > Policies are independent. Find the expected total profit and the standard deviation. > > <details> <strong>Solution</strong> > > Let $X_i$ = profit from policy i. > > $E(X_i) = 1000(0.9) + (-5000)(0.08) + (-20000)(0.02)$ $= 900 - 400 - 400 = 100$ > > $E(X_i^2) = 1000^2(0.9) + (-5000)^2(0.08) + (-20000)^2(0.02)$ $= 900,000 + 2,000,000 + 8,000,000 = 10,900,000$ > > $Var(X_i) = 10,900,000 - 100^2 = 10,900,000 - 10,000 = 10,890,000$ > > **Total profit:** $T = \sum_{i=1}^{10000} X_i$ > > $E(T) = 10,000 \times 100 = ₹1,000,000$ > > $Var(T) = 10,000 \times 10,890,000 = 108,900,000,000$ > > $\sigma_T = \sqrt{108,900,000,000} \approx ₹330,000$ > > **Interpretation:** The expected profit is ₹10 lakh, with a standard deviation of about ₹3.3 lakh. The insurance company is very likely to be profitable (profit is many standard deviations above zero). > > $\boxed{E(T) = ₹10,00,000,\ \sigma_T \approx ₹3,30,000}$ </details> * * * ## 🔗 Cross-References - **Next topic:** [Bernoulli & Binomial Distribution](/courses/bsma1002/notes/19-bernoulli-binomial) — the first named distribution - **Previous:** [Discrete & Continuous RVs](/notes/01-foundation-bsma1002-stats-1-week09-15-discrete-random-variables) — foundation for expectations - **Week 11 (Distributions):** All distributions have specific mean and variance formulas - **BSMA1004 (Stats 2):** Law of Large Numbers, Central Limit Theorem - **BSCS2004 (ML Foundations):** Cost functions as expectations, bias-variance tradeoff [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**Continuous Random Variables**](/notes/01-foundation-bsma1002-stats-1-week09-16-continuous-random-variables)[Next**Bernoulli & Binomial**](/notes/01-foundation-bsma1002-stats-1-week11-18-bernoulli-binomial)