Context-Free Grammars

1971 words

10 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Context-Free Grammars ## 🎯 Learning Objectives - Design context-free grammars for various languages - Generate parse trees and derivations (leftmost, rightmost) - Detect and eliminate ambiguity in grammars - Convert CFGs to Chomsky Normal Form - Apply closure properties of context-free languages * * * ## 1. Intro...

Context-Free Grammars

🎯 Learning Objectives

Design context-free grammars for various languages
Generate parse trees and derivations (leftmost, rightmost)
Detect and eliminate ambiguity in grammars
Convert CFGs to Chomsky Normal Form
Apply closure properties of context-free languages

1. Introduction to CFGs

1.1 Intuition

A context-free grammar is like a set of LEGO building instructions. You start with one piece (the start symbol) and repeatedly replace pieces with more detailed assemblies using rules (productions). The rules are "context-free" because you can apply them regardless of what's around the symbol — just like LEGO instructions don't depend on what other bricks are nearby. CFGs describe programming languages, natural language syntax, and protocols. They're more powerful than regular expressions/DFAs — they can handle nested structures like parentheses, HTML tags, and mathematical expressions.

1.2 Formal Definition

A CFG is a 4-tuple

G = (V, \Sigma, R, S)

where:

$V$ : Finite set of variables (nonterminals)
$\Sigma$ : Finite set of terminals (alphabet)
$R$ : Finite set of rules (productions) of form $A \to \alpha$ where $A \in V, \alpha \in (V \cup \Sigma)^*$
$S \in V$ : Start symbol

1.3 Derivation Example

Grammar for balanced parentheses:

S \to SS \mid (S) \mid \varepsilon

Derivation of "( ) ( )":

Step	Sentential Form	Rule Applied
1	S	Start
2	SS	S → SS
3	(S)S	S → (S)
4	()S	S → ε
5	()(S)	S → (S)
6	()()	S → ε

(Diagram)

2. Derivation Types

2.1 Leftmost vs. Rightmost Derivation

Type	Description	Example (for $S \to 0S1 \mid \varepsilon$ )
Leftmost	Replace leftmost variable first	$S \Rightarrow 0S1 \Rightarrow 00S11 \Rightarrow 0011$
Rightmost	Replace rightmost variable first	$S \Rightarrow 0S1 \Rightarrow 0S01 \Rightarrow 0011$

2.2 Worked Example

Grammar:

E \to E+E \mid E*E \mid (E) \mid a

Leftmost derivation of $a+a*a$ :

E \Rightarrow E+E \Rightarrow a+E \Rightarrow a+E*E \Rightarrow a+a*E \Rightarrow a+a*a

Rightmost derivation of $a+a*a$ :

E \Rightarrow E+E \Rightarrow E+E*E \Rightarrow E+E*a \Rightarrow E+a*a \Rightarrow a+a*a

Note: The parse tree for both derivations is the same: (Diagram)

3. Ambiguity

3.1 Intuition

A grammar is ambiguous if a string has more than one parse tree. This is bad for programming languages — if a compiler can parse 3 + 4 × 5 in two ways, it gives different results (either 35 or 23).

3.2 Ambiguous Grammar

The expression grammar

E \to E+E \mid E*E \mid (E) \mid a

is ambiguous. Two parse trees for $a+a*a$ : Tree 1 (addition first):

(a+a)*a

→ evaluates as

(a+a)*a

Tree 2 (multiplication first):

a+(a*a)

→ evaluates as

a+(a*a)

3.3 Eliminating Ambiguity

Unambiguous grammar (respects precedence):

$E \to E + T \mid T$
$T \to T * F \mid F$
$F \to (E) \mid a$ Now $a+a*a$ has only one parse tree (multiplication binds tighter).

Rule	Level	Associativity
$E \to E + T$	Lowest precedence	Left-associative
$T \to T * F$	Medium precedence	Left-associative
$F \to (E) \mid a$	Highest precedence	—

4. Chomsky Normal Form (CNF)

4.1 Intuition

CNF is a simplified form where every rule has exactly two nonterminals or one terminal. This restriction makes parsing algorithms (like CYK) simpler — they only need to consider combining two adjacent sub-results, like binary tree operations.

4.2 CNF Rules

Every production is of the form:

$A \to BC$ (two nonterminals)
$A \to a$ (one terminal)
$S \to \varepsilon$ (only if ε is in the language, and S never appears on RHS)

4.3 Conversion Algorithm

Step 1: Add new start variable

S_0

Step 2: Eliminate ε-rules (

A \to \varepsilon

) Step 3: Eliminate unit rules (

A \to B

) Step 4: Convert remaining rules to CNF

4.4 Worked Example

Original grammar:

S \to ASA \mid aB

A \to B \mid S

B \to b \mid \varepsilon

Step 1 — New start:

S_0 \to S

S \to ASA \mid aB

A \to B \mid S

B \to b \mid \varepsilon

Step 2 — Eliminate ε-rules ( $B \to \varepsilon$ ): Remove

B \to \varepsilon

. For each rule with B on RHS, add version without B:

S \to aB

becomes

S \to aB \mid a

A \to B

becomes

A \to B \mid \varepsilon

(but we also need to handle ε in A) Step 3 — Eliminate unit rules:

S_0 \to S

A \to B

A \to S

are unit rules. Replace:

A \to B

(and

B \to b

) with

A \to b

. Etc. Step 4 — Convert to CNF:

S_0 \to AS_1 \mid aB \mid a

(where

S_1 = SA

) ... Final CNF:

S_0 \to AS_1 \mid aB \mid a

S_1 \to SA

S \to AS_1 \mid aB \mid a

A \to b \mid AS_1 \mid aB \mid a

B \to b

5. Common Pitfalls

Pitfall 1: Left Recursion in Top-Down Parsing

The mistake: Using

A \to A\alpha

(left recursion) in a grammar for top-down parsing. Why students make it: Left recursion is natural for left-associative operations (

E \to E + T

). How to catch it: Top-down parsers (like LL parsers) infinite loop on left recursion. Correct approach: Use right recursion or EBNF for top-down grammars, or use table-driven parsers (LR) that handle left recursion.

Pitfall 2: Confusing Derivation with Parse Tree

The mistake: Thinking two different derivations mean the grammar is ambiguous. Why students make it: Different derivation orders (leftmost vs. rightmost) give different derivations. How to catch it: Ambiguity means different parse trees, not different derivation orders. Leftmost and rightmost derivations of the same string may differ but have the same parse tree. Correct approach: Two derivations with the same parse tree ≠ ambiguity. Different parse trees = ambiguity.

Pitfall 3: Incorrect ε-Rule Elimination

The mistake: Removing ε-rules without adding replacement rules for all affected productions. Why students make it: The algorithm seems simple — just remove ε and add new versions. How to catch it: Some derivations that were valid before elimination become invalid because required replacements were missed. Correct approach: Find all nullable variables systematically. For every rule

A \to X_1X_2...X_n

, generate all combinations of replacing nullable

X_i

with ε.

6. Key Concepts Reference

Concept	Definition	Application
CFG	$G = (V, \Sigma, R, S)$	Describes context-free languages
Derivation	Step-by-step replacement of variables	Generating strings
Parse tree	Tree representation of derivation	Syntax analysis
Ambiguity	Multiple parse trees for same string	Problem for compilers
CNF	$A \to BC$ or $A \to a$ only	CYK parsing algorithm
Leftmost/Rightmost	Order of variable replacement	Derivation strategies

7. 📝 Practice Questions

Q1: Design a CFG for L = {0^n1^n | n ≥ 0}.

Answer: S → 0S1 | ε

Derivation for 0011: S ⇒ 0S1 ⇒ 0(0S1)1 ⇒ 00ε11 = 0011 Q2: Is the grammar S → 0S1 | 0S | ε ambiguous?

Answer: Yes. The string "01" can be derived in two ways: (1) S ⇒ 0S1 ⇒ 0(ε)1 = 01 (2) S ⇒ 0S ⇒ 0(0S1) ⇒ 0(0ε1) = 001 — wait that's "001", not "01".

Actually let me re-check. For grammar S → 0S1 | 0S | ε: String "01": S ⇒ 0S1 ⇒ 0ε1 = 01 (only this way) String "0": S ⇒ 0S ⇒ 0ε = 0 (only this way) String "00": S ⇒ 0S ⇒ 0(0S) ⇒ 0(0ε) = 00 OR S ⇒ 0S ⇒ 0(0ε) = 00 (same) This grammar is actually unambiguous. Q3: Convert S → aSa | bSb | ε to CNF.

Answer:

Step 1: New start S₀ → S. Step 2: No ε-rules (except S → ε, which is already in CNF form). Step 3: No unit rules. Step 4: Convert: S → aSa becomes S → A₁A₂ where A₁ → a, A₂ → Sa, and Sa → S A₁ Similarly for bSb.

Final CNF: S₀ → A₁A₃ | A₄A₆ | ε S → A₁A₃ | A₄A₆ | ε A₁ → a A₃ → SA₁ A₄ → b A₆ → SA₄

(where A₃ = Sa, A₆ = Sb) Q4: Why are CFGs called "context-free"?

Answer: They are "context-free" because the replacement rule for a variable A → α can be applied regardless of the surrounding symbols (context). Contrast with context-sensitive grammars where the rule may depend on neighboring symbols (e.g., aA → bc means "replace A with c only if preceded by a"). Q5: Design a CFG for L = {a^i b^j c^k | i ≠ j}.

Answer: This is tricky because we can't directly compare. Approach: generate a^i b^j where i ≠ j, then add matching c's.

S → AC | BC A → aA | a (at least one a) C → cC | ε (any number of c's) B → bB | b (at least one b)

Wait, this doesn't capture i ≠ j properly. Better:

S → aSc | aXc | Y (where X generates more a's, Y generates more b's) X → aX | a Y → bYc | bY | b

Actually let me think again:

L = {a^i b^j c^k | i ≠ j}

S → aSc | T T generates {a^i b^j | i ≠ j} (with extra c's? No, c's handled by S)

Better approach: S → aSc | U U → aU | aUb | Ub | ab (generate strings where a count ≠ b count)

Hmm, this is complex. Let me simplify:

S → aSc | T T → aT | aTb | Tb | ab But T needs to ensure i ≠ j.

The standard approach: generate three regions separately: S → aSc | T (S matches c's with outer a's) T → aTb | A | B (T generates inner a's and b's) A → aA | a (more a's) B → bB | b (more b's)

This generates a^i b^j c^k where i = (k + #inner a's), j = #inner b's, and if inner a's ≠ inner b's then i ≠ j. Q6: What is the Pumping Lemma for context-free languages?

Answer: If L is a CFL, there exists p > 0 such that any string s ∈ L with |s| ≥ p can be written as s = uvxyz where:

|vy| ≥ 1 (at least one of v or y is non-empty)

|vxy| ≤ p (the pumped region is bounded)

uv^i xy^i z ∈ L for all i ≥ 0

This differs from the regular pumping lemma: we're pumping TWO parts (v and y) simultaneously, reflecting the binary branching structure of CFG parse trees. Q7: Design a CFG for the language of palindromes over {0,1}.

Answer: S → 0S0 | 1S1 | 0 | 1 | ε

This generates strings that read the same forwards and backwards. For "0110": S ⇒ 0S0 ⇒ 0(1S1)0 ⇒ 0(1ε1)0 = 0110 Q8: Why is the grammar E → E + T | T, T → T * F | F, F → (E) | a unambiguous for expressions?

Answer: It enforces precedence: + is at the top level (E), * is one level lower (T), and atoms/literals are innermost (F). Since each string forces a specific derivation order (you must expand E first to add, T to multiply, F for parentheses), there's exactly one parse tree. Left recursion in E → E + T forces left associativity — there's no alternative way to group the same string.

8. 🔗 Cross-References

Week 3 - DFA Minimization: Regular languages → CFGs
Week 5 - Pushdown Automata: PDA = CFG equivalence
Week 6 - Turing Machines: Beyond context-free
BSCS4032 (Compiler Design): Parse trees, top-down parsing Join Discord PreviousDFA Min & Pumping LemmaNextCFG ↔ PDA