Week 12: NumPy & Pandas Foundations

4526 words

23 min read

2026-09-12T00:00:00.000Z

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

NumPy arrays, vectorized math, Pandas Series & DataFrame basics, filtering records, and tabular aggregation. # Programming in Python — Week 12: NumPy & Pandas Foundations The official IITM syllabus concludes with **Week 12: Basic Pandas and NumPy Processing**.

Visual strip

Relevant recall pieces for this note

Open gallery

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Python

Predicate trap map

Python Week 5: string checks without false confidence

Python

Data processing pipeline

Python Week 7: from raw input to clean output

Programming in Python — Week 12: NumPy & Pandas Foundations

The official IITM syllabus concludes with Week 12: Basic Pandas and NumPy Processing. The syllabus file gives only this broad heading, so the detailed material below is a standard introductory expansion of those tools, not a claim about the exact lecture ordering.

TL;DR

Until now, you processed data mostly with:


python
for x in values:
    ...

NumPy and Pandas provide higher-level ways to work with many values at once.


text
Python lists
    ↓
NumPy arrays
    ↓
fast numerical operations

tabular records
    ↓
Pandas DataFrame
    ↓
filter / select / aggregate

Core distinction:

\boxed{ \text{NumPy} \approx \text{numerical arrays} }

\boxed{ \text{Pandas} \approx \text{labelled/tabular data} }

1. Why NumPy?

Suppose:


python
values = [1, 2, 3, 4]

To double every value manually:


python
result = []

for x in values:
    result.append(2 * x)

With NumPy:


python
import numpy as np

a = np.array([1, 2, 3, 4])

result = 2 * a

Conceptually:


text
[1,2,3,4]
   × 2
↓
[2,4,6,8]

The operation is expressed directly over the entire array.

2. Import convention

NumPy is conventionally imported as:


python
import numpy as np

Pandas is conventionally:


python
import pandas as pd

So later you commonly see:


python
np.array(...)
pd.DataFrame(...)

3. NumPy array

A NumPy array stores numerical data in an array structure.


python
a = np.array([10, 20, 30])

Conceptually:


text
index:  0   1   2
value: 10  20  30

Like a list, it supports indexing.

4. Array indexing


python
a = np.array([10, 20, 30])

print(a[0])
print(a[2])

Outputs:


text
10
30

Zero-based indexing remains.

5. Negative indexing


python
a[-1]

returns the last element.

So familiar Python sequence ideas continue to apply.

6. NumPy array vs Python list

A NumPy array is not simply another spelling for a list.

Broadly:

Feature	Python list	NumPy array
General-purpose values	Yes	Mainly homogeneous numerical data
Element-wise numeric operations	Usually manual	Natural/vectorized
Multidimensional numeric data	Nested lists	Native array structure
Numerical library operations	Limited	Extensive

7. Element-wise arithmetic


python
a = np.array([1, 2, 3])

print(a + 10)

Result:


text
[11 12 13]

Each element receives the operation.

Mathematically:

[1,2,3]+10 = [11,12,13].

8. Multiplication


python
a * 2

gives:


text
[2 4 6]

Compare Python list:


python
[1, 2, 3] * 2

which gives repetition:


python
[1, 2, 3, 1, 2, 3]

This difference is important.

9. Array-to-array arithmetic


python
a = np.array([1, 2, 3])
b = np.array([10, 20, 30])

print(a + b)

Result:


text
[11 22 33]

Element-wise:

[1+10,\ 2+20,\ 3+30].

10. Other element-wise operations


python
a - b
a * b
a / b
a ** 2

operate element-by-element when shapes are compatible.

11. Vectorization

Writing:


python
a * 2

instead of manually looping over every element is often called vectorized computation.

Conceptually:


text
manual:
for each x:
    x * 2

vectorized:
array * 2

The second expresses the computation at a higher level.

12. Creating arrays

From a list:


python
a = np.array([1, 2, 3])

Zeros:


python
np.zeros(5)

Conceptually:


text
[0. 0. 0. 0. 0.]

Ones:


python
np.ones(4)

13. `np.arange()`

NumPy provides:


python
np.arange(...)

which resembles Python's range() but returns an array.

Example:


python
np.arange(5)

conceptually gives:


text
[0 1 2 3 4]

14. Start, stop, step


python
np.arange(2, 10, 2)

gives:


text
[2 4 6 8]

Like range():

\boxed{ \text{stop is excluded} }

15. Array shape

NumPy arrays have a:


python
a.shape

property.

For:


python
a = np.array([10, 20, 30])

shape is conceptually:


text
(3,)

meaning one dimension with 3 elements.

16. Two-dimensional array


python
a = np.array([
    [1, 2, 3],
    [4, 5, 6]
])

This has:

2

rows and:

3

columns.

Shape:


text
(2, 3)

17. Matrix-style indexing

For:


python
a = np.array([
    [1, 2, 3],
    [4, 5, 6]
])

you can access:


python
a[0, 1]

which gives:


text
2

Row 0, column 1.

18. Contrast with nested list syntax

Nested Python list:


python
x[0][1]

NumPy commonly supports:


python
a[0, 1]

This directly expresses multidimensional indexing.

19. Slicing arrays


python
a = np.array([10, 20, 30, 40, 50])

a[1:4]

gives:


text
[20 30 40]

Same half-open slicing:

1\le i<4.

20. Slice a row

For matrix-like array:


python
a[0, :]

means:

row 0, all columns.

Conceptually:


text
[1 2 3]

21. Slice a column


python
a[:, 1]

means:

all rows, column 1.

For:


text
1 2 3
4 5 6

result:


text
[2 5]

22. Colon `:`

In array indexing:


python
:

roughly means:

take the full range along this dimension.

So:


python
a[:, 0]

means every row, first column.

23. Array dimensions

Useful properties include:


python
a.ndim
a.shape
a.size

Conceptually:

ndim → number of dimensions
shape → size along each dimension
size → total number of elements

24. Example

For:


python
a = np.array([
    [1, 2, 3],
    [4, 5, 6]
])

we have conceptually:


text
ndim  = 2
shape = (2,3)
size  = 6

25. Reshaping

An array with 6 elements:


python
a = np.array([1, 2, 3, 4, 5, 6])

can conceptually be reshaped:


python
a.reshape(2, 3)

into:


text
1 2 3
4 5 6

The total number of elements must remain consistent.

26. Shape compatibility

You cannot reshape 6 elements into:


text
4 × 2

because:

4\cdot2=8\neq6.

Always check:

\prod dimensions = \text{number of elements}.

27. NumPy aggregation

NumPy provides operations such as:


python
np.sum(a)
np.mean(a)
np.min(a)
np.max(a)

These summarize arrays.

28. Example


python
a = np.array([2, 4, 6, 8])

np.sum(a)

gives:

20.


python
np.mean(a)

gives:

5.

29. Axis intuition

For two-dimensional data:


text
1 2 3
4 5 6

you may aggregate across different dimensions.

Conceptually:

row-wise
column-wise
whole array

NumPy uses an axis argument for such operations.

30. Whole-array sum


python
a.sum()

for:


text
1 2 3
4 5 6

gives:

21.

31. Column-style aggregation

For a matrix, an axis operation can produce totals such as:

[1+4,\ 2+5,\ 3+6] = [5,7,9].

The key conceptual lesson is:

an axis tells NumPy which dimension to collapse during aggregation.

32. Boolean comparisons on arrays


python
a = np.array([10, 20, 30])

a > 15

gives a Boolean array:


text
[False True True]

Comparison happens element-wise.

33. Boolean masking

You can use that Boolean result to filter an array:


python
a[a > 15]

Result:


text
[20 30]

This is one of the most important NumPy patterns.

34. Mental expansion of mask


python
a[a > 15]

means conceptually:


python
result = []

for x in a:
    if x > 15:
        result.append(x)

NumPy expresses the whole operation compactly.

35. Multiple conditions

Conceptually, to select values satisfying both conditions:

x>10

and:

x<30.

NumPy uses element-wise Boolean operations.

A common form is:


python
a[(a > 10) & (a < 30)]

Parentheses matter.

36. Why not ordinary `and`?

For array-wide element-wise comparison, NumPy commonly uses:


python
&
|

with Boolean arrays rather than Python's scalar-style and / or.

This is an important distinction when moving from single values to vectorized arrays.

37. NumPy datatype

Arrays have a datatype:


python
a.dtype

A numerical array may contain:

integers
floating-point numbers
other fixed numerical types

NumPy usually works most naturally with homogeneous data.

38. Automatic conversion

If you create:


python
np.array([1, 2, 3.5])

NumPy may promote values into a common compatible type, such as floating point.

So conceptually the array may become:


text
[1.0, 2.0, 3.5]

39. Why homogeneous types help

When every element follows a common representation, NumPy can process large blocks of numerical data efficiently.

This differs from general Python lists, which may contain arbitrary mixtures of objects.

40. NumPy missing-value intuition

Numerical data sometimes contains missing values represented with concepts such as:


python
np.nan

NaN broadly means:

not a valid ordinary numerical value / missing numerical value.

Detailed missing-data treatment becomes especially relevant with Pandas.

41. Pandas

Pandas is a library for working with labelled and tabular data.

Its two famous structures are:


text
Series
DataFrame

For this course, the DataFrame is especially important.

42. DataFrame

A DataFrame can be thought of as a table:

name	mark	city
A	80	X
B	92	Y
C	70	X

Rows are records.

Columns are variables/fields.

43. Create DataFrame


python
import pandas as pd

df = pd.DataFrame({
    "name": ["A", "B", "C"],
    "mark": [80, 92, 70],
    "city": ["X", "Y", "X"]
})

Now df represents a table.

44. Dictionary-of-lists connection

Notice:


python
{
    "name": [...],
    "mark": [...],
    "city": [...]
}

uses ideas from Weeks 6–8.

Pandas builds a richer table abstraction on top of familiar collection concepts.

45. Inspect DataFrame

Simply:


python
print(df)

shows a tabular representation.

Pandas also provides tools for quickly inspecting portions of a dataset.

46. `.head()`


python
df.head()

shows the first few rows.

Useful for answering:

What does this dataset look like?

without printing everything.

47. `.tail()`


python
df.tail()

shows the final few rows.

48. DataFrame shape


python
df.shape

returns:


text
(number_of_rows, number_of_columns)

For our 3-row, 3-column table:


text
(3,3)

49. Column names


python
df.columns

provides the DataFrame's column labels.

Conceptually:


text
name
mark
city

50. Select a column


python
df["mark"]

selects the mark column.

Conceptually:


text
80
92
70

This usually produces a Pandas Series.

51. Series

A Series is roughly a one-dimensional labelled data structure.

Think:


text
index → value

For the mark column:


text
0 → 80
1 → 92
2 → 70

52. Select multiple columns


python
df["name", "mark"](/courses/python/notes/%22name%22%2C%20%22mark%22)

returns another DataFrame containing only those columns.

Note the double brackets:


python
["name", "mark"]

is itself a list of column names.

53. Filtering rows

Suppose:


python
df["mark"] >= 80

produces conceptually:


text
True
True
False

Then:


python
df[df["mark"] >= 80]

returns only matching rows.

54. Filtering mental model


python
df[df["mark"] >= 80]

corresponds conceptually to:


python
result = []

for row in rows:
    if row["mark"] >= 80:
        result.append(row)

Pandas provides a tabular, vectorized form.

55. Multiple conditions

Example:


python
df[(df["mark"] >= 80) & (df["city"] == "X")]

This means:

rows where mark is at least 80 and city is X.

Again, parentheses matter.

56. Select a filtered column

You can combine row filtering and column selection.

Conceptually:


text
filter rows
↓
select desired field

For example, get names of high-scoring students from the filtered table.

57. Aggregation in Pandas

For one numerical column:


python
df["mark"].mean()
df["mark"].sum()
df["mark"].min()
df["mark"].max()

These are table-level versions of the same aggregation patterns you've already learned.

58. Example mean

For:


text
80, 92, 70

mean:

\frac{80+92+70}{3} = \frac{242}{3} \approx80.67.

Pandas can compute this directly:


python
df["mark"].mean()

59. `value_counts()`

For categorical data:


python
df["city"].value_counts()

conceptually builds a frequency table.

For:


text
X
Y
X

result conceptually:


text
X → 2
Y → 1

This is the Pandas equivalent of a frequency dictionary pattern.

60. Connection to manual dictionary frequency

Earlier:


python
freq = {}

for x in values:
    freq[x] = freq.get(x, 0) + 1

Now:


python
series.value_counts()

encapsulates the same broad task.

61. Sorting DataFrame

Conceptually:


python
df.sort_values("mark")

sorts rows by the mark column.

Descending:


python
df.sort_values("mark", ascending=False)

62. Sorting keeps records together

This is important.

If one row is:


text
B,92,Y

sorting by mark moves the whole row.

It does not sort the marks independently from names/cities.

This preserves row relationships.

63. Add a new column


python
df["passed"] = df["mark"] >= 40

Now the table gains a Boolean column.

Conceptually:

name	mark	passed
A	80	True
B	92	True
C	70	True

64. Transform a column

Suppose:


python
df["bonus_mark"] = df["mark"] + 5

The operation applies to the entire column.

Again, this is vectorized processing.

65. Column arithmetic


python
df["percentage"] = (
    df["obtained"] / df["total"] * 100
)

Pandas applies arithmetic row-by-row according to column alignment.

This replaces many explicit loops.

66. Read CSV

A major Pandas feature:


python
df = pd.read_csv("data.csv")

This handles CSV parsing at a much higher level than manually:


python
for line in f:
    line.split(",")

67. Manual files vs Pandas

Weeks 9–10:


python
with open(...) as f:
    for line in f:
        ...

Week 12:


python
df = pd.read_csv(...)

The lower-level skills still matter because they explain what the library is abstracting away.

68. CSV becomes table

File:


text
name,mark
A,80
B,92
C,70

Then:


python
df = pd.read_csv("marks.csv")

conceptually produces:

name	mark
A	80
B	92
C	70

69. Write CSV

Pandas can also write tabular data:


python
df.to_csv("output.csv", index=False)

The index=False part commonly means:

do not add the DataFrame's index as an extra CSV column.

70. DataFrame index

Pandas rows normally have an index.

Conceptually:

index	name	mark
0	A	80
1	B	92
2	C	70

The index acts as row labels.

It is not necessarily the same thing as a data column.

71. Index vs ordinary column

Suppose student ID is real data:


text
student_id

That is not automatically the same thing as Pandas' internal row index.

Keep the conceptual distinction:


text
index = row label
column = dataset variable

72. `.loc`

Pandas provides label-oriented selection using:


python
df.loc[...]

Conceptually:

select rows/columns by labels/conditions.

Example style:


python
df.loc[df["mark"] >= 80, ["name", "mark"]]

73. `.iloc`

Pandas also provides position-oriented selection:


python
df.iloc[...]

Conceptually:

select by integer position.

This is closer to Python list/NumPy positional indexing.

74. `loc` vs `iloc`

Broad mental model:


text
loc  → labels
iloc → integer positions

This distinction becomes important once DataFrame indices are not simply 0,1,2,....

75. Example `iloc`


python
df.iloc[0]

means:

first row by position.


python
df.iloc[:, 1]

means:

all rows, second column by position.

76. Missing values

Real tabular data may contain missing values.

Pandas commonly represents missing numerical/tabular entries using values such as NaN.

Example conceptual table:

name	mark
A	80
B	NaN
C	70

77. Detect missing values

Pandas provides operations such as:


python
df.isna()

or column-level:


python
df["mark"].isna()

These create Boolean masks identifying missing entries.

78. Drop missing data

A common operation:


python
df.dropna()

returns data with missing rows removed according to the chosen rules.

Do not treat dropping as automatically correct—it depends on the problem.

79. Fill missing data

Another operation:


python
df.fillna(...)

replaces missing values with chosen replacements.

Again, choice of replacement depends on the data meaning.

80. Data cleaning

A basic Pandas workflow often involves:


text
load
↓
inspect
↓
clean
↓
filter
↓
transform
↓
aggregate
↓
save

This is the higher-level version of the manual file pipeline from Weeks 9–10.

81. Data types in Pandas

Columns have data types.

Examples conceptually:


text
mark  → numeric
name  → text
passed → Boolean

Wrong datatypes can prevent calculations.

For example, marks stored as strings cannot be meaningfully averaged without conversion.

82. Conversion intuition

Pandas provides mechanisms for converting columns into appropriate datatypes.

The important conceptual lesson:

\boxed{ \text{correct datatype} \rightarrow \text{correct available operations} }

This is the same idea you saw with:


python
int(input())

and:


python
float(...)

83. Grouping

Suppose:

city	mark
X	80
Y	92
X	70

A natural question:

What is average mark per city?

Manual approach:


python
sum_by_city = {}
count_by_city = {}

Pandas provides higher-level grouping operations.

84. `groupby()` intuition

Conceptually:


python
df.groupby("city")

means:

partition rows by city.

Then aggregation can be applied to each group.

Conceptually:


text
X → rows with X
Y → rows with Y

85. Grouped mean

Conceptual Pandas form:


python
df.groupby("city")["mark"].mean()

For data:


text
X → 80,70
Y → 92

gives:

X\rightarrow75

Y\rightarrow92.

86. Connection to Week 7 dictionary grouping

Earlier manual pattern:


python
total[group] = total.get(group, 0) + value
count[group] = count.get(group, 0) + 1

Pandas groupby() expresses the same higher-level idea.

87. NumPy ↔ Pandas connection

Pandas uses array-oriented ideas heavily.

A DataFrame column often behaves like an array-like labelled vector.

So:


text
NumPy
numerical arrays
      ↓
Pandas
labelled/tabular structures

They complement rather than replace each other.

88. NumPy is lower-level numerically

Use NumPy naturally for:

vectors
matrices
numerical transformations
array arithmetic
numerical algorithms

Use Pandas naturally for:

CSV/tabular records
named columns
row filtering
grouping
data cleaning

89. Example complete Pandas pipeline

File:


text
name,city,mark
A,X,80
B,Y,35
C,X,90
D,Y,75

Code conceptually:


python
import pandas as pd

df = pd.read_csv("marks.csv")

passed = df[df["mark"] >= 40]

average = passed["mark"].mean()

passed.to_csv("passed.csv", index=False)

This performs:


text
read
→ filter
→ aggregate
→ write

without manual file loops.

90. Equivalent manual pipeline

Weeks 9–10 version:


python
total = 0
count = 0

with open("marks.csv", "r") as src:
    with open("passed.csv", "w") as dst:
        header = src.readline()
        dst.write(header)

        for line in src:
            name, city, mark = line.strip().split(",")
            mark = int(mark)

            if mark >= 40:
                dst.write(f"{name},{city},{mark}\n")
                total += mark
                count += 1

Same conceptual algorithm; Pandas provides higher-level primitives.

91. Why learn manual processing first?

Without the earlier weeks:


python
df[df["mark"] >= 40]

may look like magic.

But you can interpret it as:


text
for each row:
    check mark >= 40
    retain matching row

The abstraction becomes understandable because you know the lower-level algorithm.

92. Common NumPy mistake: list semantics assumed


python
np.array([1,2,3]) * 2

does not duplicate the sequence.

It multiplies each numerical element.

Always track the datatype:


text
list
vs
NumPy array

93. Common NumPy mistake: incompatible shapes

Element-wise operations require compatible shapes.

For example, arrays containing unrelated dimensions may not combine the way you expect.

Shape is part of the data's structure.

Always inspect:


python
a.shape

when array operations behave unexpectedly.

94. Common NumPy mistake: `and` with arrays

Element-wise conditions typically use:


python
&
|

with parentheses:


python
(a > 0) & (a < 10)

rather than scalar:


python
and
or

95. Common NumPy mistake: missing parentheses

Wrong:


python
a > 0 & a < 10

Operator precedence can produce unintended interpretation.

Safer:


python
(a > 0) & (a < 10)

96. Common Pandas mistake: one vs many columns

One column:


python
df["mark"]

usually produces a Series.

Multiple:


python
df["name", "mark"](/courses/python/notes/%22name%22%2C%20%22mark%22)

produces a DataFrame.

Single vs double brackets matter.

97. Common Pandas mistake: filtering without parentheses

For multiple conditions, prefer:


python
df[(condition1) & (condition2)]

Parentheses around each comparison are important.

98. Common Pandas mistake: `and`/`or` in vector conditions

Like NumPy arrays, Pandas Series conditions commonly use:


python
&
|

for element-wise logical combination.

Not ordinary scalar and / or.

99. Common Pandas mistake: modifying unrelated copy/view assumptions

DataFrame selections can have subtleties around whether operations affect original data.

At introductory level, prefer explicit assignments and transformations rather than relying on implicit mutation behavior.

100. Common Pandas mistake: assuming missing means zero

A missing value:


text
NaN

is not automatically:

0.

Missing and zero are conceptually different.

101. Common Pandas mistake: ignoring datatype

A column containing:


text
"10"
"20"
"30"

may be textual.

It is not necessarily the same as numerical:


text
10
20
30

Inspect/convert datatypes when needed.

102. Common Pandas mistake: row index treated as data

The DataFrame index is metadata for row identification unless intentionally used as a data field.

Don't automatically interpret:


text
0,1,2,...

as a meaningful dataset variable.

103. Trace NumPy example


python
a = np.array([2, 5, 8, 1])

mask = a > 4
b = a[mask]
c = b * 2

Step 1:


text
a = [2,5,8,1]

Step 2:


text
mask = [False, True, True, False]

Step 3:


text
b = [5,8]

Step 4:


text
c = [10,16]

104. Trace Pandas example

Initial:

name	mark
A	40
B	90
C	75

Condition:


python
df["mark"] >= 75

produces conceptually:


text
False
True
True

Filtering gives:

name	mark
B	90
C	75

105. Transformation example


python
df["double"] = df["mark"] * 2

Result:

name	mark	double
A	40	80
B	90	180
C	75	150

No explicit row loop is required.

106. Aggregation example


python
df["mark"].max()

returns:

90.

If you need the whole row associated with the maximum, the problem is different: preserve the row relationship rather than only calculating the scalar maximum.

107. High-level pattern equivalence

Manual filtering


python
result = []

for x in values:
    if x > 10:
        result.append(x)

NumPy


python
a[a > 10]

Pandas


python
df[df["column"] > 10]

Same computational idea, different abstraction layers.

108. High-level aggregation equivalence

Python


python
sum(values) / len(values)

NumPy


python
np.mean(a)

Pandas


python
df["column"].mean()

The mathematical concept is unchanged.

109. Full course progression

The Python course now forms a fairly coherent ladder:


text
Week 1
Algorithms
   ↓
Weeks 2–3
Conditionals
   ↓
Weeks 4–5
Iteration
   ↓
Weeks 6–8
Collections
   ↓
Weeks 9–10
Files
   ↓
Week 11
Modules
   ↓
Week 12
NumPy + Pandas

Each layer builds on the previous one.

110. Computational hierarchy

You can now think about Python at several levels:


text
VALUE
5

↓

VARIABLE
x = 5

↓

CONTROL FLOW
if / for / while

↓

COLLECTION
list / dict / set

↓

PERSISTENCE
files

↓

CODE ORGANIZATION
modules

↓

DATA ABSTRACTION
NumPy / Pandas

111. Compact NumPy cheat sheet


python
import numpy as np

a = np.array([1, 2, 3])

a.shape
a.ndim
a.size

a[0]
a[1:3]

a + 10
a * 2
a ** 2

a > 1
a[a > 1]

np.sum(a)
np.mean(a)
np.min(a)
np.max(a)

np.arange(10)
np.zeros(5)
np.ones(5)

For 2D:


python
a[row, column]

a[:, 0]
a[0, :]

a.reshape(...)

112. Compact Pandas cheat sheet


python
import pandas as pd

df = pd.read_csv("data.csv")

df.head()
df.tail()

df.shape
df.columns

df["mark"]
df["name", "mark"](/courses/python/notes/%22name%22%2C%20%22mark%22)

df[df["mark"] >= 40]

df["mark"].mean()
df["mark"].sum()
df["mark"].min()
df["mark"].max()

df["city"].value_counts()

df.sort_values("mark")

df.groupby("city")["mark"].mean()

df.to_csv("output.csv", index=False)

113. Which abstraction should I use?

Simple handful of values


python
list

Key-value associations


python
dict

Unique membership


python
set

Numerical arrays/matrices


python
NumPy

Named columns / tables


python
Pandas

Persistent storage


text
files

The data structure should match the problem.

114. What you should actually master

By the end of Python Week 12, you should be comfortable with:

Explaining why NumPy is useful for numerical arrays.
Importing NumPy as np.
Creating NumPy arrays.
Indexing and slicing arrays.
Understanding array shape/dimensions.
Performing element-wise arithmetic.
Understanding vectorized computation.
Aggregating with sum/mean/min/max.
Performing Boolean comparisons on arrays.
Filtering arrays with Boolean masks.
Understanding basic multidimensional indexing.
Explaining what a Pandas DataFrame represents.
Importing Pandas as pd.
Creating or reading tabular data.
Selecting DataFrame columns.
Filtering rows using Boolean conditions.
Computing column aggregates.
Sorting tabular data.
Adding/transformation columns.
Understanding Series vs DataFrame at a basic level.
Understanding DataFrame indices.
Recognizing missing values.
Understanding the purpose of groupby().
Reading and writing CSV files using Pandas.
Connecting NumPy/Pandas operations to the manual algorithms from earlier weeks.

If these are comfortable, Programming in Python Week 12 is under control.

Quick Recall & Flashcards

Test your active recall before proceeding to practice problems. Format:

Question :: Answer.

Week 12 Diagnostic: If A = np.array([1, 2, 3]), what is the result of A * 2? :: Answer: B — NumPy arrays perform element-wise vectorized arithmetic, so A * 2 multiplies each element: array([2, 4, 6]).
Week 12 Diagnostic: In Pandas, which method returns the first 5 rows of a DataFrame df? :: Answer: B — df.head() displays the top 5 rows of a Pandas DataFrame by default.

Practice & Self-Check

Attempt these diagnostic questions to verify your operational understanding.

Question 1

If A = np.array([1, 2, 3]), what is the result of A * 2?

MCQ

3 Unit Assessment

A. [1, 2, 3, 1, 2, 3]

B. array([2, 4, 6])

C. TypeError

D. array([1, 4, 9])

Question 2

In Pandas, which method returns the first 5 rows of a DataFrame df?

MCQ

3 Unit Assessment

A. df.start(5)

B. df.head()

C. df.top(5)

D. df.first(5)

What this note is really saying

Relevant recall pieces for this note

Programming in Python — Week 12: NumPy & Pandas Foundations

TL;DR

1. Why NumPy?

2. Import convention

3. NumPy array

4. Array indexing

5. Negative indexing

6. NumPy array vs Python list

7. Element-wise arithmetic

8. Multiplication

9. Array-to-array arithmetic

10. Other element-wise operations

11. Vectorization

12. Creating arrays

13. np.arange()

14. Start, stop, step

15. Array shape

16. Two-dimensional array

17. Matrix-style indexing

18. Contrast with nested list syntax

19. Slicing arrays

20. Slice a row

21. Slice a column

22. Colon :

23. Array dimensions

24. Example

25. Reshaping

26. Shape compatibility

27. NumPy aggregation

28. Example

29. Axis intuition

30. Whole-array sum

31. Column-style aggregation

32. Boolean comparisons on arrays

33. Boolean masking

34. Mental expansion of mask

35. Multiple conditions

36. Why not ordinary and?

37. NumPy datatype

38. Automatic conversion

39. Why homogeneous types help

40. NumPy missing-value intuition

41. Pandas

42. DataFrame

43. Create DataFrame

44. Dictionary-of-lists connection

45. Inspect DataFrame

46. .head()

47. .tail()

48. DataFrame shape

49. Column names

50. Select a column

51. Series

52. Select multiple columns

53. Filtering rows

54. Filtering mental model

55. Multiple conditions

56. Select a filtered column

57. Aggregation in Pandas

58. Example mean

59. value_counts()

60. Connection to manual dictionary frequency

61. Sorting DataFrame

62. Sorting keeps records together

63. Add a new column

64. Transform a column

65. Column arithmetic

66. Read CSV

67. Manual files vs Pandas

68. CSV becomes table

69. Write CSV

70. DataFrame index

71. Index vs ordinary column

72. .loc

73. .iloc

74. loc vs iloc

75. Example iloc

76. Missing values

13. `np.arange()`

22. Colon `:`

36. Why not ordinary `and`?

46. `.head()`

47. `.tail()`

59. `value_counts()`

72. `.loc`

73. `.iloc`

74. `loc` vs `iloc`

75. Example `iloc`

84. `groupby()` intuition

94. Common NumPy mistake: `and` with arrays

98. Common Pandas mistake: `and`/`or` in vector conditions