Time & Space Complexity

1503 words

8 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Time & Space Complexity ## 🎯 Learning Objectives - Define time and space complexity classes (P, NP, PSPACE, EXP) - Explain the time hierarchy theorem - Analyze relationships between complexity classes - Distinguish between deterministic and nondeterministic complexity - Understand the P vs. NP problem * * * ## 1.

Time & Space Complexity

🎯 Learning Objectives

Define time and space complexity classes (P, NP, PSPACE, EXP)
Explain the time hierarchy theorem
Analyze relationships between complexity classes
Distinguish between deterministic and nondeterministic complexity
Understand the P vs. NP problem

1. Introduction to Complexity

1.1 Intuition

Decidability asks: "Can a computer solve this problem?" Complexity asks: "How efficiently can it solve this problem?" Some problems are solvable but require so much time (millions of years) that they're effectively impossible. Complexity theory classifies problems by the resources they need.

1.2 Asymptotic Notation

Notation	Meaning	Example
$O(g(n))$	At most $c \cdot g(n)$ for large $n$	$3n^2 + n \in O(n^2)$
$\Omega(g(n))$	At least $c \cdot g(n)$ for large $n$	$n^3 \in \Omega(n^2)$
$\Theta(g(n))$	Exactly $c \cdot g(n)$ for large $n$	$n^2 + n \in \Theta(n^2)$
$o(g(n))$	Strictly less than $g(n)$	$n^2$ is $o(n^3)$ but NOT $o(n^2)$

2. Time Complexity Classes

2.1 Definition

TIME(t(n)) = \{L \mid L \text{ is decided by an } O(t(n)) \text{ time TM}\}

Class	Definition	Example Problems
P	$\bigcup_k TIME(n^k)$	Path in graph, sorting, CFG parsing
NP	$\bigcup_k NTIME(n^k)$	SAT, TSP, coloring, subset sum
EXP	$\bigcup_k TIME(2^{n^k})$	Generalized chess, Presburger arithmetic

2.2 The Time Hierarchy Theorem

Theorem: For time-constructible functions

f(n)

TIME(f(n)) \subsetneq TIME(f(n)^2)

. This means: MORE TIME allows solving STRICTLY MORE problems. There's always a problem that takes just a bit more time. Consequence:

P \subsetneq EXP

(strict containment — there are problems in EXP not in P).

2.3 P vs. NP

(Diagram) Key question: Does

P = NP

? If yes, all efficiently verifiable problems are efficiently solvable. If no, some problems (like SAT, TSP) have no efficient algorithm.

3. Space Complexity Classes

3.1 Definition

SPACE(s(n)) = \{L \mid L \text{ is decided by an } O(s(n)) \text{ space TM}\}

Class	Definition	Example
L	$SPACE(\log n)$	Determining if a graph is connected
NL	$NSPACE(\log n)$	Path finding in directed graph
PSPACE	$\bigcup_k SPACE(n^k)$	Quantified Boolean formulas (QBF)
EXPSPACE	$\bigcup_k SPACE(2^{n^k})$	Presburger arithmetic

3.2 Space vs. Time

(Diagram) Key relationships:

$P \subseteq NP \subseteq PSPACE = NPSPACE$ (Savitch's theorem)
$P \subsetneq EXP$ (time hierarchy theorem)
$PSPACE \subseteq EXP$ (space ≤ time)
$P \neq EXPSPACE$ (if $P = PSPACE$ , then $P = EXP$ too, which contradicts the hierarchy theorem)

3.3 Savitch's Theorem

Theorem:

NSPACE(f(n)) \subseteq SPACE(f(n)^2)

Intuition: Nondeterministic space can be simulated deterministically with only a quadratic space overhead. This is much better than time (where exponential slowdown occurs). So

PSPACE = NPSPACE

, but we don't know if

P = NP

4. The Landscape

Class	Characterization	Complete Problems
P	Efficiently solvable	Linear programming, matching
NP	Efficiently verifiable	SAT, 3SAT, clique, vertex cover
PSPACE	Polynomial space	QBF, geography
EXP	Exponential time	Generalized chess
L	Log space	Graph connectivity
NL	Nondeterministic log space	Directed graph path

5. Common Pitfalls

Pitfall 1: Confusing NP with "Not Polynomial"

The mistake: Thinking NP stands for "not polynomial." Why students make it: The name is misleading. How to catch it: NP = Nondeterministic Polynomial time. Problems in NP can be SOLVED by an NTM in polynomial time, or VERIFIED by a DTM in polynomial time. Correct approach: NP = polynomial-time verification, not "exponential."

Pitfall 2: Assuming P ≠ NP is the Default

The mistake: Treating P ≠ NP as proven fact. Why students make it: Everyone talks about "hard" NP problems as if they're impossible. How to catch it: P ≠ NP is a conjecture, not a theorem. It's overwhelmingly believed but unproven. Both possibilities (P = NP and P ≠ NP) are consistent with current knowledge. Correct approach: "If P ≠ NP" when discussing NP-completeness. Be careful what follows from the assumption vs. what's proven.

Pitfall 3: Equating "Hard" with "Intractable"

The mistake: Concluding that NP-complete problems are always intractable. Why students make it: NP-completeness proofs suggest worst-case hardness. How to catch it: NP-completeness is about WORST-CASE instances. Many NP-complete problems have practical algorithms that work well on real-world instances (SAT solvers, integer programming). Correct approach: NP-completeness says "hard in the worst case." Real instances may be easy. Use specialized algorithms, heuristics, and approximation.

6. Key Concepts Reference

Concept	Definition	Key Insight
P	$O(n^k)$ time on DTM	Efficient computation
NP	$O(n^k)$ time on NTM	Verifiable in poly time
PSPACE	$O(n^k)$ space	Includes NP, QBF-complete
EXP	$O(2^{n^k})$ time	Strictly larger than P
Time hierarchy	More time → more problems	$P \neq EXP$
Savitch's theorem	$NSPACE(f) \subseteq SPACE(f^2)$	$PSPACE = NPSPACE$
Polynomial reduction	$A \leq_p B$	Used for NP-completeness

7. 📝 Practice Questions

Q1: Why do we use polynomial time as the definition of "efficient" rather than linear or quadratic?

Answer: Polynomial time $(n^k)$ is robust: (1) It's closed under composition (polynomial of polynomial is polynomial). (2) It's independent of machine model (polynomial time on any reasonable model is polynomial on a TM). (3) Most practical algorithms run in polynomial time. (4) Problems with super-polynomial lower bounds are truly intractable. Linear time is too restrictive (many algorithms are O(n log n) or O(n²)), and exponential time includes truly infeasible problems. Q2: Show that any problem in NP can be solved in exponential time.

Answer: If $L \in NP$ , there's an NTM N that decides $L$ in $O(n^k)$ time. We can simulate N using a DTM via breadth-first search of the computation tree. The branching factor of N is at most $b$ (finite), and the tree depth is $O(n^k)$ . The total number of configurations is at most $b^{O(n^k)}$ which is $O(2^{n^k})$ for some $k$ . So $L$ can be decided in $O(2^{n^k})$ time → $NP \subseteq EXP$ . This is proven, not conjectured. Q3: State and explain Savitch's theorem.

Answer: Savitch's theorem: $NSPACE(f(n)) \subseteq SPACE(f(n)^2)$ for $f(n) \geq \log n$ . Intuition: Nondeterministic space can be simulated deterministically with only quadratic space overhead. The proof uses divide-and-conquer: to check if a configuration can reach another in $t$ steps, check if there's a midpoint that can be reached in $t/2$ steps from the start and can reach the end in $t/2$ steps. This recursive approach reuses space (depth $O(f(n))$ , each level $O(f(n))$ space → total $O(f(n)^2)$ ).

Key implication: $PSPACE = NPSPACE$ (polynomial space is unaffected by nondeterminism), unlike time where $P \neq NP$ is believed. Q4: What is the difference between a reduction $A \leq_p B$ and $A \leq_m B$ ?

Answer: $A \leq_m B$ (mapping reduction) requires only that the reduction function $f$ is computable (in finite time). $A \leq_p B$ (polynomial-time reduction) requires that $f$ is computable in polynomial time. $\leq_p$ is more restrictive — it preserves polynomial-time computability. $\leq_m$ is used in undecidability proofs; $\leq_p$ is used in NP-completeness proofs. All $\leq_p$ reductions are also $\leq_m$ reductions, but not vice versa. Q5: Why don't we know if P = PSPACE?

Answer: We know $P \subseteq NP \subseteq PSPACE$ , but we can't prove proper inclusion for either containment. Proving $P \subsetneq PSPACE$ would imply $P \neq NP$ (since $NP \subseteq PSPACE$ ). Proving $P = PSPACE$ would imply $NP = PSPACE$ (since $NP \subseteq PSPACE$ ). Either result would be revolutionary, and both seem far beyond current techniques. The time hierarchy theorem gives $P \neq EXP$ but nothing separates $P$ from $PSPACE$ or $NP$ from $PSPACE$ with current methods. Q6: Place these problems in the most likely complexity class: (a) finding a path in a graph, (b) 3SAT, (c) QBF, (d) generalized chess.

Answer: (a) Path in graph → P (BFS/DFS runs in O(n+m) time) (b) 3SAT → NP (verifiable in polynomial time, believed not in P) (c) QBF → PSPACE-complete (quantified boolean formulas are the canonical PSPACE-complete problem) (d) Generalized chess → EXP (on an n×n board, generalized chess is EXP-complete) Q7: Explain the Time Hierarchy Theorem's significance.

Answer: The Time Hierarchy Theorem proves that $TIME(f(n)) \subsetneq TIME(f(n)^2)$ for time-constructible $f$ . This means: (1) More time allows solving strictly more problems — there's always a decidable problem that needs more than $O(f(n))$ time. (2) $P \subsetneq EXP$ (since $P = \bigcup_k TIME(n^k)$ and $EXP = \bigcup_k TIME(2^{n^k})$ , and the hierarchy theorem shows $TIME(n^k) \subsetneq TIME(2^n)$ for any $k$ ). (3) The proof uses diagonalization — constructing a TM on the fly that disagrees with every $O(f(n))$ time TM. Q8: If someone proves P = PSPACE, what other problems would be efficiently solvable?

Answer: If P = PSPACE, then every problem solvable in polynomial space is also solvable in polynomial time. This would include: (1) All problems in NP (SAT, TSP, graph coloring) — P = NP follows from P = PSPACE since NP ⊆ PSPACE. (2) All PSPACE-complete problems (QBF, generalized geography, many games). (3) Problems in PH (polynomial hierarchy) — the entire hierarchy collapses to P. This would be revolutionary — the biggest discovery in computer science — but is widely believed to be false.

8. 🔗 Cross-References

Week 11 - NP-Hardness: Polynomial reductions
Week 12 - NP-Completeness: Complete problems
BSCS4021 (Advanced Algorithms): Algorithm analysis Join Discord PreviousReductionsNextNP-Hardness