Introduction to Machine Learning

🎯 Learning Objectives

• Define machine learning and distinguish it from traditional programming
• Identify the three main types of learning: supervised, unsupervised, and reinforcement learning
• Explain what features, labels, and datasets are with concrete examples
• Describe the complete ML workflow from problem definition to deployment
• Recognize when a problem is solvable with ML vs. when it isn't

📋 Prerequisites

• Basic Python (variables, functions, lists, loops) — we'll use Python throughout for implementation
• Basic probability & statistics (mean, median, standard deviation, probability) — foundational for understanding how models learn from data
• Linear algebra basics (vectors, matrices, dot products) — the mathematical language of ML

📖 Core Content

1.1 Intuition: What is Machine Learning?

pseudo
Rules + Data → Answers

pseudo
Data + Answers → Rules (the model)

💡 Why this matters: ML has transformed industries because it allows computers to handle tasks that are easy for humans but hard to describe step-by-step. Vision, language, recommendation systems — all would be nearly impossible with rules-based programming.

1.2 Formal Definition

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. For example, in a spam filter:

• Task T: Classify emails as spam or not spam
• Experience E: A collection of emails that users have marked as spam/not spam
• Performance P: The percentage of emails correctly classified

1.3 Types of Machine Learning

Supervised Learning

Unsupervised Learning

Reinforcement Learning

1.4 Data, Features, and Labels

• X (features matrix): All input columns (Sq. Ft., Bedrooms, Location Score)
• y (labels/targets): The column we want to predict (Price)
• Training set: Data used to teach the model
• Test set: Data used to evaluate the model (held back during training) Each row in X is an example or instance, often denoted as $x^{(i)}$ (the $i$-th training example). The $j$-th feature of the $i$-th example is $x_j^{(i)}$.

1.5 The ML Workflow

1. Problem Definition: What are we trying to predict? Is it classification or regression? What's the business goal?
2. Data Collection: Where does the data come from? Databases, APIs, web scraping, sensors?
3. Data Cleaning: Handle missing values, fix inconsistencies, remove duplicates.
4. EDA: Visualize distributions, find correlations, spot anomalies.
5. Feature Engineering: Create new features from existing data, transform variables.
6. Model Selection: Choose an algorithm suited to the problem and data size.
7. Model Training: The algorithm learns patterns from the training data.
8. Model Evaluation: Test on held-out data using appropriate metrics.
9. Deployment: Put the model into production.
10. Monitoring: Track performance over time; retrain as needed.

1.6 When to Use ML (and When NOT To)

• You have a pattern to recognize but can't define rules manually
• You have sufficient data (typically thousands of examples)
• The problem has clear input-output mapping possibilities
• A small margin of error is acceptable (ML isn't 100% accurate)
• You need adaptability (models can be retrained as new data arrives) Don't use ML when:
• You need perfect accuracy (safety-critical systems)
• You have very little data (custom rules or heuristics are better)
• The problem can be solved with simple arithmetic (e.g., "calculate average")
• Interpretability is legally required (e.g., credit denial reasons)
• The data doesn't exist or is too expensive to collect

📐 Key Formulas / Concepts

⚠️ Common Pitfalls

Pitfall 1: Confusing Correlation with Causation

Pitfall 2: Leaking Future Information

Pitfall 3: Training-Serving Skew

📝 Practice Questions

Q1: What type of ML problem is "predicting whether a customer will churn based on their usage patterns"?

Answer: Supervised Learning — specifically binary classification (churn vs. not churn). We have historical data where we know which customers churned (labels), and we want to predict this for new customers.

Reasoning: Since we have labeled examples (customer features + known outcomes), this is supervised. Since the output is a yes/no decision, it's classification. Q2: You have a dataset of customer purchases with no labels. You want to group customers with similar purchasing behavior. What type of ML is this?

Answer: Unsupervised Learning — specifically clustering. There are no predefined categories; we want the algorithm to discover natural groupings. Q3: Which of the following is NOT a suitable ML problem? a) Predicting tomorrow's stock price b) Calculating the average age of a population c) Detecting fraudulent transactions d) Recommending movies to users

Answer: b) Calculating the average age of a population — this is a simple calculation that doesn't require ML. You just sum ages and divide by count. No learning from examples needed. Q4: In the definition of ML, what does E, T, and P stand for?

Answer:

• E = Experience (the data the model is exposed to)
• T = Task (what we want the model to do)
• P = Performance (how we measure success)

For a spam filter: E = labeled emails, T = classifying spam vs. not spam, P = accuracy/precision. Q5: You have 10 data points and need perfect accuracy. Should you use ML?

Answer: No. With only 10 data points, ML cannot learn reliable patterns. You would be better off writing explicit rules or using a simple heuristic. ML requires sufficient data to generalize beyond the training examples. Q6: A model predicts house prices with 95% accuracy on training data but 70% on test data. What's happening?

Answer: This is overfitting — the model has memorized the training data (including noise) but failed to learn generalizable patterns. The solution involves regularization, more training data, or a simpler model. Q7: What's the key difference between supervised and unsupervised learning?

Answer: Supervised learning uses labeled data (input-output pairs) where the model learns to map inputs to known outputs. Unsupervised learning uses unlabeled data where the model must find structure without guidance. Think of supervised as learning with a teacher vs. unsupervised as exploring on your own. Q8: In a dataset of patients, what are the features and label for predicting diabetes risk?

Answer:

• Features: Age, BMI, blood pressure, family history, glucose level, physical activity level
• Label: "Has diabetes" (yes/no), or "Glucose level after 2 hours" (if continuous regression)

Features are the input variables we use to make the prediction; the label is what we want to predict. Q9: Which ML type would you use for a chess-playing AI?

Answer: Reinforcement Learning — the AI (agent) plays chess (environment), makes moves (actions), and either wins, loses, or draws (rewards). It learns by trial and error to maximize its win rate. The state is the current board position, and the policy determines which move to make. Q10: What is the difference between regression and classification?

Answer:

• Regression predicts a continuous numerical value (e.g., temperature 72.5°F, price $350,000)
• Classification predicts a discrete class label (e.g., "spam" or "not spam", "cat" or "dog")

If the output is a number where order and magnitude matter, it's regression. If the output is a category, it's classification. Q11: Why is strong performance on the training set not enough?

Answer: A model can memorize the training data perfectly (100% training accuracy) but fail on new, unseen data because it hasn't learned general patterns — it has memorized noise. This is overfitting. We always evaluate on a held-out test set that wasn't seen during training. Q12: Give an example where ML would NOT be appropriate.

Answer: Computing the exact gravitational force between two objects. Newton's law $F = G \frac{m_1 m_2}{r^2}$ gives an exact answer. There's no need to "learn" from data since the rules are known precisely. ML is for problems where rules are hard to specify.

🔗 Cross-References

• Next Topic: Linear Regression with One Variable — the simplest supervised learning algorithm
• Related: Types of Learning — deeper dive into supervised vs unsupervised vs reinforcement
• External: IITM BSCS2004 Week 1 Lecture, Hands-On Machine Learning with Scikit-Learn Ch. 1 Join Discord NextLinear Regression