Advanced File Operations & CSV Processing

Why read this? Beyond basic read/write, you often need to navigate within a file (seek), parse structured formats (CSV), or process files too large to fit in memory. This topic covers those real-world scenarios.

🎯 Learning Objectives

1. Use seek() and tell() for random access within a file
2. Parse CSV files using csv module and manually
3. Process large files line by line (memory efficient)
4. Work with binary files

📋 Prerequisites

• File Operations

📖 Core Content

26.1 File Navigation: tell() and seek()

python
Run Inline
# runnable
with open("example.txt", "w") as f:
    f.write("Hello World\nSecond Line\n")
with open("example.txt", "r") as f:
    print(f"Position: {f.tell()}")  # 0 (start)
    print(f.read(5))                 # "Hello"
    print(f"Position: {f.tell()}")  # 5
    f.seek(0)                        # Go to start
    print(f.read())                  # "Hello World\nSecond Line\n"
    f.seek(6)                        # Skip "Hello "
    print(f.read())                  # "World\nSecond Line\n"

26.2 CSV Parsing with csv Module

python
Run Inline
# runnable
import csv
# Write CSV
with open("students.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["Name", "Age", "Grade"])
    writer.writerow(["Alice", 25, "A"])
    writer.writerow(["Bob", 22, "B"])
    writer.writerow(["Charlie", 24, "A"])
# Read CSV
with open("students.csv", "r") as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

26.3 Manual CSV Parsing

python
Run Inline
# runnable
csv_data = """Name,Age,City
Alice,25,NYC
Bob,30,LA
Charlie,22,Chicago"""
lines = csv_data.strip().split("\n")
header = lines[0].split(",")
print("Header:", header)
for line in lines[1:]:
    values = line.split(",")
    print(f"{values[0]} is {values[1]} years old, lives in {values[2]}")

26.4 Large File Processing

python
Run Inline
# runnable
# Process file line by line (memory efficient)
with open("large_file.txt", "r") as f:
    for line in f:
        # Process each line — only ONE line in memory at a time
        if "ERROR" in line:
            print(f"Found error: {line.strip()}")

26.5 Binary Files

python
Run Inline
# runnable
with open("binary.dat", "wb") as f:
    f.write(bytes([0, 1, 2, 3, 4, 255]))
with open("binary.dat", "rb") as f:
    data = f.read()
    print(list(data))  # [0, 1, 2, 3, 4, 255]

⚠️ Common Pitfalls

Pitfall 1: CSV with Commas in Data

Pitfall 2: Forgetting newline="" When Writing CSV

Pitfall 3: Loading Entire Large File into Memory

📝 Practice Questions

Q1: What does f.tell() return?

Answer: The current position (byte offset) in the file from the beginning. Q2: Write code to read the last 10 bytes of a file.

Answer:

python
Run Inline
with open("file.txt", "rb") as f:
    f.seek(-10, 2)  # 2 = from end
    print(f.read())

Q3-10: Additional advanced file questions.

(Following pattern.)

🔗 Cross-References

• Next Topic: Exception Handling
• Previous Topic: File Operations
• Reference: Python for Everybody, Chapter 7 (Sections 7.7-7.8)
• Video: L70: Very big files a tip, L74: File handling genetic sequences Join Discord Previous25. File OperationsNext27. Exception Handling