Exploratory Data Analysis & Visualization

🎯 Learning Objectives

• Profile a dataset systematically (shape, types, missing, distributions)
• Create effective visualizations for each data type
• Detect patterns, outliers, and relationships through visual EDA
• Summarize findings for stakeholders

📖 Core Content

3.1 The EDA Workflow

3.2 Initial Data Profile

python
Run Inline
# runnable
import pandas as pd
import numpy as np
# Create sample dataset
df = pd.DataFrame({
    'age': np.random.randint(18, 80, 100),
    'income': np.random.lognormal(mean=10, sigma=0.5, size=100),
    'gender': np.random.choice(['M', 'F'], 100),
    'score': np.random.randn(100) * 10 + 50
})
df.loc[0:5, 'income'] = np.nan  # Add some missing
print(f"Shape: {df.shape}")
print(f"\nDtypes:\n{df.dtypes}")
print(f"\nMissing:\n{df.isnull().sum()}")
print(f"\nDescribe:\n{df.describe()}")
print(f"\nHead:\n{df.head()}")

3.3 Univariate Analysis

3.4 Bivariate & Multivariate Analysis

python
Run Inline
import matplotlib.pyplot as plt
import seaborn as sns
# Correlation matrix
plt.figure(figsize=(8, 6))
sns.heatmap(df.select_dtypes(include=[np.number]).corr(), annot=True, cmap='coolwarm')
plt.title('Correlation Matrix')
plt.show()
# Pairplot
sns.pairplot(df, diag_kind='kde')
plt.show()

📝 Practice Questions

Q1: What does skewness tell you about a numeric feature?

• Skew > 0: Right-skewed (long tail of large values). Common for income, prices.
• Skew < 0: Left-skewed (long tail of small values). Uncommon in practice.
• Skew ≈ 0: Roughly symmetric (normal-like).

Skew affects model choice — many models assume symmetric distributions. Log transform can fix right skew. Q2: When would you use a boxplot vs histogram vs KDE?

• Boxplot: Quick summary of median, quartiles, outliers. Good for comparing many categories.
• Histogram: Shows exact bin counts. Sensitive to bin size choice.
• KDE: Smooth density estimate. Good for seeing distribution shape. Can be misleading with small data.

Use boxplot for outlier detection, histogram for exact distribution, KDE for shape comparison. Join Discord PreviousData CleaningNextModel Deployment