Week 11: Concurrency, Caching & Architecture

4117 words

21 min read

2026-09-12T00:00:00.000Z

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

Message passing, RPC, cache memory hierarchy, concurrency vs parallelism, preemption, and atomicity. # Computational Thinking — Week 11: Concurrency, Caching & Architecture IITM’s CT Week 11 covers **Message Passing, Remote Procedure Call (RPC), Cache Memory, Parallelism, Concurrency, Polling, Preemption, Multithrea...

Computational Thinking — Week 11: Concurrency, Caching & Architecture

IITM’s CT Week 11 covers Message Passing, Remote Procedure Call (RPC), Cache Memory, Parallelism, Concurrency, Polling, Preemption, Multithreading, Producer–Consumer, and Atomicity.

TL;DR

Week 10 focused on one object and its internal organization.

Week 11 shifts to multiple computations interacting at the same time or across boundaries.

The core ideas are:

Concept	Main idea
Message passing	Components communicate by sending data/messages
RPC	Call a procedure on another machine/system as if it were local
Cache	Keep frequently used data closer/faster
Parallelism	Actually execute multiple tasks simultaneously
Concurrency	Manage multiple tasks whose execution overlaps
Polling	Repeatedly check whether something has happened
Preemption	System interrupts one task to let another run
Multithreading	Multiple execution threads within a process
Producer–Consumer	One side creates data, another consumes it
Atomicity	Operation appears indivisible

The unifying theme is:

\boxed{ \text{multiple actors} + \text{shared resources} + \text{coordination} }

1. Why Week 11 matters

So far, many CT algorithms looked conceptually like:


text
do A
then B
then C

But real systems often look more like:


text
Task A running
Task B waiting
Task C receives data
Task A resumes
Task D sends message

The difficulty is no longer only:

Is each individual step correct?

It becomes:

What happens when several activities interact?

2. Message passing

Message passing means components communicate by explicitly sending information to one another.

Conceptually:


text
Process A
   |
   |  message
   v
Process B

Instead of directly modifying B's internal state, A sends B a request or piece of data.

3. Example

Suppose:


text
Client

wants weather information from:


text
Server

Client sends:


text
"Give me today's weather"

Server replies:


text
"28°C"

The interaction is based on messages.

4. Message contents

A message might contain:

command
data
identifier
result
error
acknowledgement

Conceptually:


text
{
    operation: "withdraw",
    account: 123,
    amount: 500
}

5. Sender and receiver

Message passing has at least:

\boxed{ \text{sender} \rightarrow \text{message} \rightarrow \text{receiver} }

Important questions:

Who sends?
Who receives?
When?
Does sender wait?
What if receiver is unavailable?
What if messages arrive out of order?

Week 11 starts exposing these systems questions.

6. Direct shared state vs message passing

Two components can interact in different ways.

Shared state

Both access the same data:


text
shared_counter

Message passing

One component sends:


text
"increase counter"

to another.

These approaches lead to different coordination problems.

7. Message passing and OOP connection

Week 10 already gave an object-call intuition:


text
account.withdraw(500)

can be viewed conceptually as:

send the account object a request to withdraw 500.

Week 11 generalizes that idea to components that may be separate processes or machines.

8. Remote Procedure Call

RPC stands for:

\boxed{ \text{Remote Procedure Call} }

The idea:

Call a procedure that actually executes somewhere else.

Conceptually:


text
result = remoteMachine.compute(x)

looks similar to a local call, but computation happens remotely.

9. Local call vs remote call

Local


text
y = f(x)

Same process/machine.

Remote


text
y = remote_f(x)

Request must travel to another process/system, execute there, then return a response.

10. RPC conceptual flow


text
Caller
  |
  | request: f(10)
  v
Remote system
  |
  | execute f(10)
  |
  | result = 25
  v
Caller

So RPC uses message passing underneath.

11. Why RPC is convenient

Without RPC-like abstraction, caller may explicitly handle:


text
construct message
send message
wait
decode response
handle protocol

RPC tries to make remote interaction look more like:


text
result = function(argument)

That is an abstraction.

12. But remote is not truly local

This is important.

A remote call can fail in ways a normal local function call usually cannot:

network unavailable
server down
timeout
duplicated request
delayed response
partial failure

So:

\boxed{ \text{RPC resembles local calls syntactically, but not operationally} }

13. Cache memory

A cache stores copies of useful data in a location that is faster to access.

Basic hierarchy intuition:


text
fast + small + expensive
        ↑
      cache
        ↑
main memory / slower storage
        ↑
slow + large

The goal is to avoid repeatedly fetching the same information from a slower source.

14. Cache example

Suppose fetching data from server takes:

100\text{ ms}

but cache lookup takes:

1\text{ ms}.

If recently requested data is cached:


text
request
  ↓
cache hit
  ↓
return quickly

Otherwise:


text
request
  ↓
cache miss
  ↓
fetch from slower source
  ↓
store in cache
  ↓
return

15. Cache hit

A cache hit occurs when required data is already present in the cache.


text
requested item ∈ cache

So the fast copy can be used.

16. Cache miss

A cache miss occurs when the requested item is not present.

Then the system must fetch it from a slower level.

17. Why caching works

Caching depends on locality.

Two useful intuitions:

Temporal locality

If something was used recently, it may be used again soon.

Spatial locality

If one location/item is used, nearby ones may be used soon.

These are standard reasons caches are effective.

18. Cache tradeoff

Caches improve speed but introduce problems:

limited capacity
stale data
replacement decisions
synchronization with source

So caching is not simply "store everything."

19. Stale cache

Suppose source says:


text
balance = 1000

Cache stores:


text
balance = 1000

Source later changes to:


text
balance = 800

but cache still says:


text
1000

Now cache is stale.

This illustrates the consistency problem.

20. Parallelism

Parallelism means multiple computations genuinely execute at the same time.

Example with two processors:


text
Processor 1: Task A
Processor 2: Task B

Both execute simultaneously.

21. Concurrency

Concurrency means multiple tasks are in progress during overlapping periods.

They need not literally execute at the same instant.

Example with one CPU:


text
time →
A A A | B B | A A | C C | B

The CPU rapidly switches among tasks.

A, B and C are concurrent even if only one executes at each instant.

22. Concurrency vs parallelism

This distinction matters.

Concurrency

\boxed{ \text{multiple tasks make progress in overlapping time} }

Parallelism

\boxed{ \text{multiple tasks execute simultaneously} }

Parallel execution is concurrent, but concurrency does not require physical simultaneity.

23. Everyday analogy

Imagine one cook preparing several dishes.

The cook:

boils pasta
while waiting, chops vegetables
returns to pasta
checks oven

That is concurrency.

Two cooks working at the same time on two dishes is closer to parallelism.

24. Why concurrency exists

Tasks often spend time waiting for:

user input
disk
network
timer
another task

Instead of letting CPU sit idle, system can work on something else.

25. Polling

Polling means repeatedly checking whether an event or condition has occurred.

Example:


text
while message not available:
    check again

Conceptually:


text
Ready?
No.
Ready?
No.
Ready?
No.
Ready?
Yes.

26. Polling example

Suppose a program waits for a file to appear:


text
while file does not exist:
    check again

This is polling.

27. Polling drawback

Repeated checking can waste resources.

If the condition changes only once every minute but you check millions of times, much work is useless.

So polling frequency creates a tradeoff:

frequent polling → responsive but expensive
infrequent polling → cheaper but slower detection

28. Event-driven alternative intuition

Instead of asking:


text
Are you ready?
Are you ready?
Are you ready?

a system may conceptually say:

Notify me when you are ready.

That avoids repeated checks.

The syllabus names polling specifically, so the key is recognizing its repeated-check behavior.

29. Preemption

Preemption means the system can interrupt a currently running task and give execution time to another task.

Example:


text
Task A running
     ↓
OS interrupts A
     ↓
Task B runs
     ↓
later A resumes

A does not necessarily voluntarily give up control.

30. Why preemption matters

Suppose one task runs an infinite loop:


text
while True:
    compute

Without preemption, it might monopolize the processor.

A preemptive scheduler can interrupt it and let other tasks execute.

31. Time slicing intuition

A scheduler may give each task a small time slice:


text
A → B → C → A → B → ...

This creates the appearance that many tasks run together.

That is a common mechanism for concurrency.

32. Context switching

When CPU moves from one task/thread to another, it must preserve enough state to resume later.

Conceptually:


text
save A state
load B state
run B
save B state
load A state
resume A

This is a context switch.

33. Multithreading

A thread is an execution flow within a process.

A process may contain several threads:


text
Process
├── Thread 1
├── Thread 2
└── Thread 3

Threads may share much of the process's memory/resources.

34. Why threads are useful

One program may simultaneously handle:

user interface
network request
computation
background processing

Separate threads can represent these activities.

35. Shared memory among threads

Threads in the same process often share state.

Example:


text
shared counter = 0

Thread A and Thread B can both access it.

This is convenient but dangerous.

36. Race condition intuition

Suppose:


text
counter = 0

Two threads both execute:


text
counter = counter + 1

You might expect:

counter=2.

But that increment may internally involve:


text
read counter
add 1
write counter

Those steps can interleave badly.

37. Lost update example

Initial:

counter=0.

Thread A reads:

0.

Thread B reads:

0.

A computes:

1

and writes:

1.

B also computes from its old read:

1

and writes:

1.

Final:

\boxed{counter=1}

instead of:

2.

This is a classic race condition.

38. Race condition

A race condition occurs when program outcome depends on the timing/interleaving of concurrent operations.

Different schedules may produce different results.

That makes bugs difficult to reproduce.

39. Atomicity

An operation is atomic if it behaves as one indivisible action.

Conceptually:


text
START operation
[ nobody observes/interleaves partial state ]
END operation

Either it happens as a whole or not at all, from the relevant perspective.

40. Atomic increment

If:


text
counter = counter + 1

were atomic, no second thread could interfere halfway through its read-modify-write sequence.

Then two atomic increments from 0 would safely produce:

2.

41. Atomicity does not mean "fast"

Atomic means:

indivisible with respect to relevant concurrent interactions.

An atomic operation may still take time.

The concept is about visibility/interleaving, not speed.

42. Critical section intuition

A critical section is a piece of code accessing shared state that should not be executed concurrently by conflicting threads.

Example:


text
read balance
subtract amount
write balance

If two withdrawals interleave incorrectly, account state may become inconsistent.

43. Example bank race

Balance:

100.

Two threads each try to withdraw:

80.

Thread A reads:

100.

Thread B reads:

100.

Each believes sufficient funds exist.

Each writes:

20.

Final balance:

20

but total money withdrawn:

160.

The intended invariant was violated.

44. Need for synchronization

Concurrent shared-state systems often need mechanisms that coordinate access.

The Week 11 syllabus does not explicitly list mutexes/locks, so the essential point is simply:

\boxed{ \text{shared state + concurrency requires coordination} }

Atomicity is one such core concept.

45. Producer–Consumer problem

The Producer–Consumer pattern contains:

producer → creates items
consumer → uses/removes items
buffer → holds items between them

Conceptually:


text
Producer
   |
   v
[ Buffer ]
   |
   v
Consumer

46. Example

Producer:


text
generates video frames

Buffer:


text
queue of frames

Consumer:


text
displays frames

They may operate at different speeds.

47. Why a buffer helps

Without buffer, producer and consumer must synchronize perfectly.

With a buffer:


text
Producer faster temporarily
        ↓
items accumulate in buffer
        ↓
Consumer catches up

The buffer decouples their exact timing.

48. Bounded buffer

A buffer may have finite capacity.

Suppose capacity:

3.

State:


text
[item1, item2, item3]

is full.

Producer cannot safely add another item until space becomes available.

49. Empty buffer

If:


text
buffer = []

consumer has nothing to consume.

It must wait or take some defined alternative action.

50. Producer–Consumer coordination

Two conditions matter:

Producer

Needs:

\text{buffer not full}

Consumer

Needs:

\text{buffer not empty}

So concurrent coordination is necessary.

51. Queue connection

Producer–Consumer often uses a queue:

\boxed{\text{FIFO}}

Producer appends items.

Consumer removes oldest item.

This connects to the queue concept associated with graph traversal.

52. Polling in Producer–Consumer

A naive consumer might do:


text
while buffer is empty:
    check buffer again

That is polling/busy waiting.

A producer might similarly poll while buffer is full.

This shows how Week 11 concepts interact.

53. Busy waiting

When a process repeatedly checks a condition without doing useful work:


text
while not ready:
    pass

it is busy waiting.

Polling can lead to busy waiting.

54. Concurrency + Producer–Consumer

Imagine:


text
Producer thread
Consumer thread

Both operate concurrently.

Potential issues:

producer updates buffer while consumer reads it
consumer sees inconsistent state
two producers insert simultaneously
two consumers remove same item

Again atomicity becomes important.

55. Example non-atomic queue update

Suppose adding an item requires:

read current tail
write item
increment tail

If multiple producers perform these steps concurrently without coordination, they may interfere.

So a logically simple operation:


text
enqueue(x)

may need to be atomic from users' perspective.

56. Sequential execution

Sequential execution:


text
Task A fully completes
then
Task B fully completes

has one obvious order.

Concurrent execution may interleave.

For example:


text
A1
B1
A2
B2

or:


text
B1
A1
A2
B2

or many other schedules.

57. Interleaving

An interleaving is one possible ordering of steps from multiple concurrent tasks.

Suppose:

Thread A:


text
A1
A2

Thread B:


text
B1
B2

Valid order preserving each thread's local order could be:


text
A1 B1 B2 A2

Another:


text
B1 A1 A2 B2

Concurrency introduces many possible interleavings.

58. Why interleavings are hard

If each thread has several steps, number of possible schedules can become large.

A bug may occur only under one unusual ordering.

This is why concurrency bugs can appear "random."

59. Preemption creates unexpected interleavings

A thread may be interrupted between apparently simple operations.

Example:


text
read x
--- preempted here ---
modify x
write x

Another thread can run during that gap.

So you cannot assume uninterrupted execution unless atomicity/synchronization guarantees it.

60. Cache and concurrency interaction

Suppose two processors each have cached copies of shared data.

Processor 1 modifies its copy.

Processor 2 may still see an older copy unless the system maintains consistency/coherence.

The Week 11 syllabus does not explicitly list cache coherence, but this is a useful connection between caching and concurrency.

61. RPC + concurrency

A server may receive many remote calls:


text
Client A ─┐
Client B ─┼→ Server
Client C ─┘

The server may process requests:

sequentially
concurrently
in parallel

So RPC systems naturally lead into concurrency topics.

62. Message passing vs shared memory

Two common communication models:

Shared memory


text
Task A ↔ shared data ↔ Task B

Requires careful synchronization.

Message passing


text
Task A → message → Task B

Communication is explicit.

Neither eliminates all complexity, but the coordination style differs.

63. Example: shared counter

Shared-memory style:


text
counter = counter + 1

Multiple threads may race.

Message-passing style:


text
send IncrementCounter to CounterService

One service might serialize updates.

This can avoid some shared-memory races at the cost of messaging overhead/complexity.

64. Cache analogy

Think of cache as:

keep useful information nearby so you do not repeatedly travel to a slower source.

But when source changes, nearby copies can become inconsistent.

So caching creates a fundamental tension:

\boxed{ \text{speed} \leftrightarrow \text{freshness/consistency} }

65. Polling analogy

Polling is like repeatedly asking:

"Has the package arrived yet?"

every second.

It detects arrival eventually, but constant checking can be wasteful.

66. Preemption analogy

Preemption is like a teacher stopping one student mid-presentation:

"Your turn is paused; someone else goes now."

The first student later resumes from where they stopped.

67. Thread analogy

A process is like a workshop.

Threads are workers inside it.

They share workshop resources.

This sharing makes cooperation easy but creates conflicts if two workers modify the same thing simultaneously.

68. Atomicity analogy

Imagine transferring money:


text
subtract ₹100 from A
add ₹100 to B

If the overall operation should be atomic, outsiders should not observe a permanently half-completed transfer.

Conceptually:

\boxed{ \text{all-or-nothing logical operation} }

69. Atomicity at different levels

What counts as "one atomic operation" depends on the abstraction.

At machine level:


text
increment

may involve multiple lower-level actions.

At application level:


text
bank transfer

may involve many database operations but still need to behave atomically.

So atomicity is relative to the interface/observer.

Common mistake: concurrency = parallelism

No.

A single-core system can support concurrency by interleaving tasks.

Parallelism requires simultaneous execution capability.

Keep:

\boxed{ \text{concurrency} \neq \text{parallelism} }

Common mistake: threads are independent processes

Threads typically share more state/resources than separate processes.

That shared memory is why thread coordination matters.

Common mistake: preemption means task terminates

No.

Preemption usually means:

pause now, possibly resume later.

Termination means execution ends.

Different ideas.

Common mistake: polling means one check

Polling requires repeated checks.

One check:


text
is ready?

is merely a test.

Repeated periodic checking is polling.

Common mistake: cache is permanent storage

A cache is generally a performance-oriented copy or temporary faster storage layer.

The authoritative source may exist elsewhere.

Cached data may be evicted or become stale.

Common mistake: RPC has no communication cost

RPC may look like:


text
f(x)

but actually involves network/process communication.

It is typically much more expensive and failure-prone than an ordinary local function call.

Common mistake: atomic = one source-code line

A line such as:


text
x = x + 1

looks like one statement but may not be atomic.

It can decompose into several lower-level operations.

Never infer atomicity purely from source-code appearance.

Common mistake: race condition always causes wrong result

A race creates the possibility that outcome depends on scheduling.

Some executions may coincidentally produce the expected result.

That unpredictability is exactly what makes races dangerous.

Common mistake: buffer eliminates synchronization

A buffer helps decouple producer/consumer timing, but concurrent access to the buffer itself still requires correct coordination.

Common mistake: producer must always be faster

No.

Either side may be faster.

If producer is faster:


text
buffer fills

If consumer is faster:


text
buffer empties

The design must handle both.

80. Trace example — race condition

Initial:

X=10.

Thread A:


text
X = X + 5

Thread B:


text
X = X * 2

If operations execute atomically and sequentially:

A then B

10+5=15

then:

15\cdot2=30.

B then A

10\cdot2=20

then:

20+5=25.

Different orders already produce:

30

or:

25.

So order matters.

81. Non-atomic version is even worse

If each statement is split into reads/writes, additional outcomes may become possible because operations can overlap internally.

This is the central concurrency danger:

\boxed{ \text{interleaving can change results} }

82. Producer–Consumer trace

Buffer capacity:

2.

Initially:


text
[]

Producer creates A:


text
[A]

Producer creates B:


text
[A,B]

Buffer full.

Consumer removes A:


text
[B]

Producer can now insert C:


text
[B,C]

This is the basic bounded-buffer dynamic.

83. Cache trace

Suppose cache capacity is 2.

Requests:


text
A, B, A

Start:


text
[]

Request A:


text
miss
cache = [A]

Request B:


text
miss
cache = [A,B]

Request A:


text
hit

No slower fetch required for the third request.

84. System design mental map


text
MULTIPLE COMPUTATIONS
│
├── Communication
│   ├── Message Passing
│   └── RPC
│
├── Performance
│   ├── Cache
│   ├── Parallelism
│   └── Multithreading
│
├── Scheduling
│   ├── Concurrency
│   ├── Polling
│   └── Preemption
│
└── Coordination
    ├── Producer–Consumer
    ├── Shared state
    ├── Race conditions
    └── Atomicity

85. Week 10 → Week 11 connection

Week 10:


text
object
├── state
└── methods

Week 11:


text
many objects/processes/threads
       ↕
communication
       ↕
shared resources
       ↕
coordination

The focus expands from internal organization to interaction between independently progressing computations.

86. What you should actually master

By the end of CT Week 11, you should be able to:

Explain message passing.
Identify sender, receiver and message.
Explain RPC conceptually.
Understand why a remote call can fail differently from a local call.
Explain why caches improve performance.
Distinguish cache hit from cache miss.
Understand stale-cache intuition.
Distinguish concurrency from parallelism.
Explain polling.
Explain preemption.
Understand basic context switching.
Explain what a thread is.
Understand shared-state problems in multithreading.
Recognize a race condition.
Explain Producer–Consumer and the role of a buffer.
Understand full-buffer and empty-buffer cases.
Explain atomicity.
Understand why apparently simple operations may not be atomic.
Recognize that concurrent outcomes can depend on interleaving.
Connect communication, scheduling, caching and synchronization into one systems picture.

If these are comfortable, CT Week 11 is under control.

Quick Recall & Flashcards

Test your active recall before proceeding to practice problems. Format:

Question :: Answer.

Week 11 Diagnostic: What is the key difference between Concurrency and Parallelism? :: Answer: A — Concurrency structures tasks to make progress independently (interleaving or parallel); parallelism physically executes multiple tasks simultaneously across multiple cores.
Week 11 Diagnostic: An operation that either completes entirely or leaves state unchanged without intermediate observable states is called: :: Answer: B — Atomic operations are indivisible and guarantee safe state transitions in concurrent systems.

Practice & Self-Check

Attempt these diagnostic questions to verify your operational understanding.

Question 1

What is the key difference between Concurrency and Parallelism?

MCQ

3 Unit Assessment

A. Concurrency is about dealing with lots of things at once (structure); parallelism is about doing lots of things at once (execution)

B. Parallelism can only run on a single-core CPU

C. Concurrency requires multiple physical machines; parallelism does not

D. They are mathematically identical terms

Question 2

An operation that either completes entirely or leaves state unchanged without intermediate observable states is called:

MCQ

3 Unit Assessment

A. Asynchronous

B. Atomic

C. Preemptive

D. Polled