Week 4: Association Between Variables

3344 words

17 min read

2026-09-12T00:00:00.000Z

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

Scatter plots, covariance, Pearson correlation coefficient, positive and negative association, and correlation vs causation. # Statistics for Data Science I — Week 4: Association Between Variables IITM’s Week 4 covers **association between variables**.

Statistics for Data Science I — Week 4: Association Between Variables

IITM’s Week 4 covers association between variables.

TL;DR

Week 4 asks:

Do two variables move together in some systematic way?

The highest-value ideas are:

association vs no association
positive vs negative association
categorical–categorical association
numerical–numerical association
scatter plots
correlation intuition
strength vs direction
nonlinear relationships
association does not imply causation
lurking/confounding variables

1. What is association?

Two variables are associated if knowing the value of one gives useful information about the other.

Examples:

height and weight
study time and marks
smoking status and disease status
income and expenditure

If the variables behave independently of each other, there is little or no association.

2. Association is about patterns

Suppose:

Study hours	Marks
1	42
2	50
3	58
4	66
5	73

As study hours increase, marks also tend to increase.

This suggests:

\boxed{\text{positive association}}

3. Positive association

Two variables have positive association when larger values of one tend to occur with larger values of the other.

Informally:

X\uparrow \Rightarrow Y\uparrow

Example:

height and weight
advertising and sales, in some settings
study time and score

4. Negative association

Negative association means larger values of one tend to occur with smaller values of the other.

Informally:

X\uparrow \Rightarrow Y\downarrow

Examples:

speed and travel time for a fixed distance
price and demand, in some contexts

5. No association

If changes in one variable do not correspond to a systematic change in the other, there may be little or no association.

Example:

shoe size and exam score

You may see a cloud of points with no clear directional pattern.

6. Numerical–numerical association

When both variables are numerical, a scatter plot is one of the main tools.

Each observation becomes one point:

(x_i,y_i).

Example:

$x$ = study hours
$y$ = marks

7. Scatter plot

Conceptually:


text
Marks
 ^
 |                 *
 |              *
 |           *
 |        *
 |     *
 +--------------------> Study hours

This shows a positive association.

8. Negative scatter pattern


text
Y
 ^
 | *
 |   *
 |      *
 |         *
 |            *
 +--------------------> X

X

rises,

Y

falls.

So:

\boxed{\text{negative association}}

9. No clear pattern


text
Y
 ^
 |    *      *
 | *      *
 |       *      *
 |   *       *
 |      *
 +--------------------> X

No clear upward/downward structure.

This suggests weak or no association.

10. Three things to inspect in a scatter plot

When looking at a scatter plot, think about:

Direction
Strength
Form

Direction:

positive
negative
none

Strength:

weak
moderate
strong

Form:

linear
nonlinear

11. Strong association

A strong association means points follow a clear pattern closely.

Example:


text
Y
 ^
 |            *
 |         *
 |      *
 |   *
 | *
 +----------------> X

The points are close to a line.

12. Weak association

A weak association may still have an overall direction, but the points are much more scattered.


text
Y
 ^
 |       *      *
 |  *        *
 |      *
 | *             *
 |         *
 +----------------> X

13. Linear association

If the pattern is approximately straight-line shaped, the relationship is linear.

A model might look like:

Y\approx a+bX.

This connects directly to straight lines from Maths 1.

14. Nonlinear association

Variables can be strongly associated without following a straight line.

Example:

Y=X^2.

A scatter plot may look U-shaped.

So:

strong association does not necessarily mean strong linear association.

This distinction becomes important when interpreting correlation.

15. Correlation intuition

Correlation is a numerical measure of the direction and strength of linear association between two numerical variables.

Common notation:

r

for sample correlation.

Its values lie between:

\boxed{-1\leq r\leq1}

16. Interpreting correlation sign

If:

r>0,

positive linear association.

If:

r<0,

negative linear association.

If:

r\approx0,

little or no linear association.

That last word matters.

17. Perfect positive correlation

If:

r=1,

all points lie exactly on a straight line with positive slope.

Example:

Y=2X+3.

18. Perfect negative correlation

If:

r=-1,

all points lie exactly on a straight line with negative slope.

Example:

Y=10-2X.

19. Zero correlation does not mean no relationship

This is a major trap.

Consider:

Y=X^2

with symmetric

X

values around zero.

There is a clear nonlinear relationship.

Yet linear correlation may be near:

0.

So:

\boxed{ r\approx0 \not\Rightarrow \text{no association} }

It means no strong linear association.

20. Correlation magnitude

Rough intuition:

|r|\approx1

strong linear relationship.

|r|\approx0

weak linear relationship.

But avoid treating arbitrary cutoffs as universal.

Context matters.

21. Correlation is unitless

Correlation does not have units.

If one variable is measured in:

cm
meters
inches

the correlation stays the same under positive linear rescaling.

This makes it useful for comparing strength across variables.

22. Correlation is symmetric

Correlation between

X

and

Y

equals correlation between

Y

and

X

So:

\boxed{ r_{XY}=r_{YX} }

There is no dependent/independent direction built into correlation itself.

23. Effect of changing units

Suppose height is converted from:

\text{cm}

to:

\text{m}.

That is a positive linear transformation.

The correlation with weight stays unchanged.

Similarly, adding a constant does not change correlation.

24. Negative scaling

If one variable is multiplied by a negative number, the sign of correlation reverses.

Example:

If:

Y=-X,

then:

r=-1.

25. Outliers and correlation

Correlation can be highly sensitive to outliers.

Suppose most points show little pattern, but one extreme point lies far away.

That one point can significantly change:

r.

So always inspect the scatter plot.

Do not trust correlation alone.

26. Association between categorical variables

For two categorical variables, we do not use ordinary scatter plots.

Instead we compare:

contingency tables
conditional percentages
bar charts

This connects directly to Week 2.

27. Example

Suppose we observe:

	Pass	Fail	Total
Group A	80	20	100
Group B	50	50	100

Pass rates:

Group A:

80\%

Group B:

50\%.

Because conditional distributions differ substantially, the variables appear associated.

28. What would no association look like?

Suppose:

	Pass	Fail	Total
Group A	80	20	100
Group B	160	40	200

Pass rate for A:

\frac{80}{100}=80\%.

Pass rate for B:

\frac{160}{200}=80\%.

The conditional distributions are identical.

That suggests no association between group and result.

29. Marginal vs conditional distribution

Marginal distribution:

overall distribution of one variable.

Conditional distribution:

distribution within one subgroup of another variable.

Association is usually detected by comparing conditional distributions.

30. Numerical–categorical association

Sometimes one variable is numerical and the other categorical.

Example:

salary = numerical
department = categorical

Then ask:

Does the distribution of salary differ by department?

Useful summaries:

mean by group
median by group
box plots by group

31. Example

Suppose median salary:

Department A: ₹50k
Department B: ₹80k
Department C: ₹52k

That suggests department and salary may be associated.

32. Side-by-side box plots

A common way to compare one numerical variable across categories is multiple box plots.

Conceptually:


text
A: |---[---|---]----|
B:     |----[----|----]------|
C: |---[----|---]---|

This lets you compare:

center
spread
outliers

across groups.

33. Association vs causation

One of the most important principles:

\boxed{ \text{Association does not imply causation} }

If two variables move together, that alone does not prove one causes the other.

34. Example

Suppose ice-cream sales and drowning incidents both rise in summer.

You might observe:

\text{ice cream sales}\uparrow

and:

\text{drownings}\uparrow.

It would be absurd to conclude:

ice cream causes drowning.

A third variable:

\text{hot weather}

affects both.

35. Lurking variable

A lurking variable is an unmeasured or unconsidered variable that influences the observed relationship.

In the previous example:

\text{temperature}

is lurking behind both outcomes.

36. Confounding

Confounding occurs when the effect of one variable becomes mixed with the effect of another.

Suppose people who exercise more also tend to eat healthier diets.

If health outcomes differ, it may be difficult to isolate:

exercise effect
diet effect

without better study design.

37. Why observational data is tricky

In observational data, researchers do not control who receives which exposure.

So associated groups may differ in many other ways.

This makes causal conclusions difficult.

38. Experimental data

In a controlled experiment, researchers actively assign treatments.

Random assignment can help balance other variables across treatment groups.

This strengthens causal interpretation.

You will encounter these ideas more formally later in statistics, but the distinction matters already.

39. Correlation vs causation example

Suppose:

r=0.9

between two variables.

This means:

strong positive linear association.

It does not mean:

a one-unit increase in $X$ causes $Y$ to increase.

Correlation is descriptive.

Causality requires more.

40. Direction of causation can be reversed

Suppose:

X

and:

Y

are associated.

Possibilities include:

$X$ causes $Y$
$Y$ causes $X$
third variable causes both
feedback between $X$ and $Y$
coincidence / sampling variation

Association alone cannot distinguish these.

41. Ecological association trap

An association observed for groups does not necessarily hold for individuals.

Example:

A country-level relationship between:

average income
average life expectancy

does not automatically describe the exact relationship between individual income and individual lifespan.

Always know the observational unit.

42. Scatter plot does not establish causality

Even a beautiful near-perfect line only establishes a pattern in the data.

It cannot by itself reveal causal mechanism.

Graph shape and causal structure are different questions.

43. Covariance intuition

A related quantity is covariance.

Very roughly, covariance asks:

Do $X$ and $Y$ tend to deviate from their means in the same direction?

A common population-style expression is:

\operatorname{Cov}(X,Y) = E[(X-E[X])(Y-E[Y])].

You do not need to over-focus on this formula yet if IITM has not formally introduced expectation.

44. Covariance sign

If:

\operatorname{Cov}(X,Y)>0,

they tend to move together.

If:

\operatorname{Cov}(X,Y)<0,

they tend to move oppositely.

If near zero:

little linear co-movement.

Correlation is essentially a standardized covariance.

45. Why standardize covariance?

Covariance depends on the units of

X

and

Y

Correlation rescales it by variability:

\boxed{ r = \frac{\text{covariance}} {\text{SD of }X\times\text{SD of }Y} }

This produces a unitless number between:

-1

and:

1.

Treat this as useful intuition if the exact formula has not yet been emphasized in your lectures.

46. Scatter-plot reading workflow

When you see a scatter plot:


text
1. Direction?
   ├── positive
   ├── negative
   └── none

2. Form?
   ├── linear
   └── nonlinear

3. Strength?
   ├── strong
   ├── moderate
   └── weak

4. Any outliers/clusters?

5. Does context justify causality?
   Usually: not from scatter alone.

47. Clusters

Sometimes a scatter plot contains separate groups.

Example:


text
Y
^
|  ****
|  ****
|
|             ****
|             ****
+------------------> X

There may be another categorical variable separating the observations.

Ignoring group structure can produce misleading conclusions.

48. Subgroup effects

Suppose overall data shows positive association.

But within each subgroup, association may be weak or even negative.

This is another route to Simpson-type reversals.

So:

always ask whether aggregation is hiding structure.

49. Range restriction

Correlation can change when only a narrow range of

X

values is observed.

Example:

Height and weight may correlate substantially across a broad population.

If you restrict to people whose heights are almost identical, observed correlation may shrink.

So correlation depends partly on the sampled range.

50. Correlation does not measure slope

Suppose:

Y=100X.

Perfect positive linear relationship:

r=1.

Suppose instead:

Y=0.001X.

Still:

r=1.

The slopes are wildly different.

Correlation measures linear tightness and direction, not slope magnitude.

51. Correlation does not measure prediction error directly

Two datasets may have similar correlations but different scales and prediction errors.

Correlation is one summary, not a full model evaluation.

This becomes more important in regression later.

Common mistake:

r=0

means independence

Not necessarily.

A nonlinear dependency can produce:

r=0.

So:

\boxed{ r=0 \not\Rightarrow \text{independence} }

In some special settings, stronger conclusions may hold, but not generally.

Common mistake: high correlation means causal effect

No.

High association may arise due to:

confounding
common cause
reverse causality
selection bias

Correlation alone cannot resolve them.

Common mistake: negative correlation means weak relationship

No.

Example:

r=-0.95

is a very strong negative linear association.

Strength depends on:

|r|.

Direction depends on sign.

Common mistake:

r=0.8

means 80% of points fit

No.

Correlation is not a percentage of data explained in that simple interpretation.

Do not read:

r=0.8

as "80% correlated."

Common mistake: correlation works for categorical variables

Ordinary Pearson-style correlation is designed for numerical variables.

For categorical variables, use:

contingency tables
proportions
other categorical association measures

Do not mechanically assign numbers to categories and correlate them.

Common mistake: association is symmetric but causation is too

Association:

X\leftrightarrow Y

is symmetric descriptively.

Causation:

X\to Y

is directional.

That distinction is fundamental.

58. Example: positive association

Data:

$X$	$Y$
1	2
2	4
3	6
4	8

Here:

Y=2X.

So:

\boxed{r=1}

perfect positive linear association.

59. Example: negative association

$X$	$Y$
1	8
2	6
3	4
4	2

Here:

Y=10-2X.

So:

\boxed{r=-1}

perfect negative linear association.

60. Example: nonlinear association

Suppose:

$X$	$Y$
-2	4
-1	1
0	0
1	1
2	4

Clearly:

Y=X^2.

Strong deterministic relationship.

But there is no overall linear upward/downward trend across the symmetric range.

So correlation can be:

0

despite perfect nonlinear dependence.

61. Comparing group means carefully

Suppose:

Group A mean:

70

Group B mean:

75.

That alone does not tell us how separated the groups are.

Their spreads may be:

Case 1

A tightly around

70

, B tightly around

75

Strong group distinction.

Case 2

Both spread from

0

150

Huge overlap.

So association involves more than just comparing means.

62. Association and variability

If groups have very different centers relative to within-group variability, the categorical and numerical variables may have stronger association.

If distributions overlap almost completely, association may be weaker.

This is why side-by-side box plots are useful.

63. Association summary by variable types

Variable 1	Variable 2	Typical tool
categorical	categorical	two-way table / conditional percentages
numerical	numerical	scatter plot / correlation
categorical	numerical	grouped summaries / side-by-side box plots

This table is extremely useful.

64. Full conceptual map


text
Association
│
├── Categorical + Categorical
│   ├── contingency table
│   └── conditional distributions
│
├── Numerical + Numerical
│   ├── scatter plot
│   ├── direction
│   ├── form
│   ├── strength
│   └── correlation
│
└── Categorical + Numerical
    ├── compare centers
    ├── compare spread
    └── side-by-side plots

65. Correlation fact sheet

\boxed{-1\leq r\leq1}

r>0 \Rightarrow \text{positive linear association}

r<0 \Rightarrow \text{negative linear association}

|r|\to1 \Rightarrow \text{stronger linear association}

r\to0 \Rightarrow \text{weaker linear association}

but:

\boxed{ r\approx0 \not\Rightarrow \text{no relationship} }

66. Causality checklist

Before saying:

$X$ causes $Y$

ask:

Was this observational or experimental?
Was treatment randomly assigned?
Could another variable affect both?
Could causation run backward?
Could selection bias explain the pattern?
Is the association reproducible?
Is there a plausible mechanism?

For basic Stats 1, the core rule remains:

\boxed{ \text{Association}\neq\text{causation} }

67. Weeks 1–4 progression


text
Week 1
What kind of data?
      ↓
Week 2
How is categorical data distributed?
      ↓
Week 3
How is numerical data centered/spread?
      ↓
Week 4
How do two variables relate?

That is the first major block of Statistics 1.

68. What you should actually master

By the end of Stats 1 Week 4, you should be able to:

Define association between variables.
Distinguish positive and negative association.
Recognize no clear association.
Read scatter plots.
Describe direction, form, and strength.
Distinguish linear from nonlinear association.
Interpret correlation qualitatively.
Know: $-1\leq r\leq1.$
Distinguish strength from sign.
Understand why $r=0$ does not imply no relationship.
Understand how outliers affect association summaries.
Compare two categorical variables using conditional distributions.
Compare a numerical variable across categories.
Distinguish association from causation.
Recognize lurking/confounding variables.

If those are comfortable, Stats 1 Week 4 is under control.

Quick Recall & Flashcards

Test your active recall before proceeding to practice problems. Format:

Question :: Answer.

Week 4 Diagnostic: What is the possible range of values for Pearson's correlation coefficient r? :: Answer: B — Pearson's r is always bounded between -1 and +1 inclusive, where -1 indicates perfect negative linear correlation and +1 indicates perfect positive linear correlation.
Week 4 Diagnostic: If variables X and Y have a correlation coefficient r = 0, does that guarantee they are completely independent? :: Answer: B — Pearson's r only measures linear relationships. Two variables can have a strong nonlinear relationship (e.g., Y = X^2 on [-1, 1]) and still yield r = 0.

Practice & Self-Check

Attempt these diagnostic questions to verify your operational understanding.

Question 1

What is the possible range of values for Pearson's correlation coefficient r?

MCQ

3 Unit Assessment

A. [0, 1]

B. [-1, 1]

C. (-inf, +inf)

D. [0, 100]

Question 2

If variables X and Y have a correlation coefficient r = 0, does that guarantee they are completely independent?

MCQ

3 Unit Assessment

A. Yes, r = 0 implies statistical independence

B. No, r = 0 only indicates no linear association (nonlinear dependence may still exist)

C. Only if both variables are categorical

D. Yes, if sample size > 30

What this note is really saying

Statistics for Data Science I — Week 4: Association Between Variables

TL;DR

1. What is association?

2. Association is about patterns

3. Positive association

4. Negative association

5. No association

6. Numerical–numerical association

7. Scatter plot

8. Negative scatter pattern

9. No clear pattern

10. Three things to inspect in a scatter plot

11. Strong association

12. Weak association

13. Linear association

14. Nonlinear association

15. Correlation intuition

16. Interpreting correlation sign

17. Perfect positive correlation

18. Perfect negative correlation

19. Zero correlation does not mean no relationship

20. Correlation magnitude

21. Correlation is unitless

22. Correlation is symmetric

23. Effect of changing units

24. Negative scaling

25. Outliers and correlation

26. Association between categorical variables

27. Example

28. What would no association look like?

29. Marginal vs conditional distribution

30. Numerical–categorical association

31. Example

32. Side-by-side box plots

33. Association vs causation

34. Example

35. Lurking variable

36. Confounding

37. Why observational data is tricky

38. Experimental data

39. Correlation vs causation example

40. Direction of causation can be reversed

41. Ecological association trap

42. Scatter plot does not establish causality

43. Covariance intuition

44. Covariance sign

45. Why standardize covariance?

46. Scatter-plot reading workflow

47. Clusters

48. Subgroup effects

49. Range restriction

50. Correlation does not measure slope

51. Correlation does not measure prediction error directly

58. Example: positive association

59. Example: negative association

60. Example: nonlinear association

61. Comparing group means carefully

Case 1

Case 2

62. Association and variability

63. Association summary by variable types

64. Full conceptual map

65. Correlation fact sheet

66. Causality checklist

67. Weeks 1–4 progression

68. What you should actually master

Quick Recall & Flashcards

Practice & Self-Check

Question 1

Question 2

Active Recall Session