Time Series Analysis & ARIMA

🎯 Learning Objectives

• Decompose a time series into trend, seasonality, and residuals
• Test for stationarity using the Dickey-Fuller test
• Build ARIMA models for forecasting
• Evaluate time series forecasts

📖 Core Content

5.1 Intuition: Predicting the Future from the Past

5.2 Time Series Components

• $T_t$: Trend — long-term direction
• $S_t$: Seasonality — repeating patterns (weekly, yearly)
• $R_t$: Residuals — random noise

5.3 Stationarity

• $H_0$: Series has a unit root (non-stationary)
• $H_1$: Series is stationary
• If p-value < 0.05: reject $H_0$, series is stationary Making a series stationary:
• Differencing: $Y_t' = Y_t - Y_{t-1}$
• Log transformation (stabilizes variance)
• Seasonal differencing

5.4 ARIMA Model

• p (AR order): Use p past values as predictors
• d (Integration): Number of differencing steps
• q (MA order): Use q past forecast errors AR(p): $Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \dots + \phi_p Y_{t-p} + \varepsilon_t$ MA(q): $Y_t = c + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \dots + \theta_q \varepsilon_{t-q}$ ARIMA(p,d,q): AR on differenced series + MA components. Selecting p, q: Use ACF and PACF plots:
• ACF cuts off after q → MA(q)
• PACF cuts off after p → AR(p)

5.5 Implementation

python
Run Inline
# runnable
import numpy as np
import matplotlib.pyplot as plt
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller
from sklearn.metrics import mean_squared_error
# Generate sample time series
np.random.seed(42)
n = 200
t = np.arange(n)
y = 0.005 * t + np.sin(2 * np.pi * t / 50) * 0.5 + np.random.randn(n) * 0.3
# Train/test split
train, test = y[:150], y[150:]
# Fit ARIMA
model = ARIMA(train, order=(2, 1, 2))
fitted = model.fit()
print(fitted.summary())
# Forecast
forecast = fitted.forecast(steps=len(test))
rmse = np.sqrt(mean_squared_error(test, forecast))
print(f"Test RMSE: {rmse:.3f}")
# Check stationarity
result = adfuller(y)
print(f"ADF statistic: {result[0]:.3f}, p-value: {result[1]:.3f}")

📝 Practice Questions

Q1: What's the difference between AR and MA components?

AR uses past observed values as predictors: $Y_t$ depends on $Y_{t-1}, Y_{t-2}, \dots$. MA uses past forecast errors: $Y_t$ depends on $\varepsilon_{t-1}, \varepsilon_{t-2}, \dots$. AR captures persistence of values; MA captures shock effects that last q periods. Q2: Why must a time series be stationary for ARIMA?

Non-stationary series have time-varying mean/variance, so AR/MA coefficients learned from past data don't apply to future data. Differencing removes trends to achieve stationarity. Without it, forecasts would be biased. Q3: What does d=1 mean in ARIMA(p,1,q)?

The series is differenced once before applying AR and MA components. If original series has a linear trend, one differencing removes it. d=2 removes quadratic trends (rarely needed). Join Discord PreviousKernel MethodsNextAnomaly Detection