Memory Hierarchy and Cache
462 words
2 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Memory Hierarchy and Cache ## 🎯 Learning Objectives - Explain the memory hierarchy (L1, L2, L3 cache, RAM, disk) - Calculate cache hit/miss rates for different mappings - Differentiate direct-mapped, fully associative, and set-associative caches - Analyze cache performance with locality * * * ## 1. Memory Hierarc...
Memory Hierarchy and Cache
🎯 Learning Objectives
- Explain the memory hierarchy (L1, L2, L3 cache, RAM, disk)
- Calculate cache hit/miss rates for different mappings
- Differentiate direct-mapped, fully associative, and set-associative caches
- Analyze cache performance with locality
1. Memory Hierarchy
(Diagram)
1.2 Locality
| Type | Description | Example |
|---|---|---|
| Temporal locality | Recently accessed data will be accessed again soon | Loop variables, counters |
| Spatial locality | Data near recently accessed data will be accessed | Array traversal |
2. Cache Organization
2.1 Direct-Mapped Cache
Each memory block maps to exactly one cache line.
pseudoMemory address: [ Tag | Index | Offset ] Index = address % num_cache_lines Tag = address / num_cache_lines
Example: 16KB cache, 64-byte blocks, 32-bit address
- Block offset: log₂(64) = 6 bits
- Index: log₂(16384/64) = log₂(256) = 8 bits
- Tag: 32 - 6 - 8 = 18 bits
2.2 Set-Associative Cache
An n-way set-associative cache has n blocks per set.
| Associativity | Advantages | Disadvantages |
|---|---|---|
| Direct-mapped | Simple, fast | Conflict misses |
| 2-way | Fewer conflicts | More complex |
| 4-way | Good balance | Higher latency |
| Fully associative | Fewest conflicts | Expensive, slow |
3. 📝 Practice Questions
Q1: Calculate the cache size given: 32-bit address, 2-way set-associative, 64-byte blocks, 1024 sets.Answer: Number of blocks = sets × associativity = 1024 × 2 = 2048 blocks. Block size = 64 bytes. Total cache size = 2048 × 64 = 128 KB (excluding tag storage). Q2: What causes a conflict miss?Answer: Conflict misses occur when multiple memory blocks map to the same cache set and are accessed alternately. Increasing associativity (going from direct-mapped to 2-way) can reduce conflict misses. Q3: How does spatial locality affect cache performance?Answer: When one word is accessed, the entire block (e.g., 64 bytes) is loaded into cache. If subsequent accesses are to nearby addresses, they hit in the cache. This is why sequential array access is much faster than random access. Q4: What is the difference between a cache hit and a cache miss?Answer: A cache hit occurs when the requested data is found in cache (fast). A cache miss occurs when it's not found — the data must be fetched from the next level of memory (slow). Miss penalty can be 10-100× the hit latency. Q5: Calculate AMAT (Average Memory Access Time) for hit rate 95%, hit time 2ns, miss penalty 100ns.Answer: AMAT = hit_time + miss_rate × miss_penalty = 2 + 0.05 × 100 = 2 + 5 = 7 ns. Even 5% misses triple the average access time!
4. 🔗 Cross-References
- BSCS4022 (OS): Virtual memory, TLB, page tables
- Week 6 - Processor Design: Cache in CPU datapath
- BSCS4022 - Week 7: Memory management, page replacement Join Discord PreviousPipeliningNextI/O Systems