Context-Free Grammars

1971 words

10 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Context-Free Grammars ## 🎯 Learning Objectives - Design context-free grammars for various languages - Generate parse trees and derivations (leftmost, rightmost) - Detect and eliminate ambiguity in grammars - Convert CFGs to Chomsky Normal Form - Apply closure properties of context-free languages * * * ## 1. Intro...

Context-Free Grammars

🎯 Learning Objectives

Design context-free grammars for various languages
Generate parse trees and derivations (leftmost, rightmost)
Detect and eliminate ambiguity in grammars
Convert CFGs to Chomsky Normal Form
Apply closure properties of context-free languages

1. Introduction to CFGs

1.1 Intuition

A context-free grammar is like a set of LEGO building instructions. You start with one piece (the start symbol) and repeatedly replace pieces with more detailed assemblies using rules (productions). The rules are "context-free" because you can apply them regardless of what's around the symbol — just like LEGO instructions don't depend on what other bricks are nearby. CFGs describe programming languages, natural language syntax, and protocols. They're more powerful than regular expressions/DFAs — they can handle nested structures like parentheses, HTML tags, and mathematical expressions.

1.2 Formal Definition

A CFG is a 4-tuple

G = (V, \Sigma, R, S)

where:

$V$ : Finite set of variables (nonterminals)
$\Sigma$ : Finite set of terminals (alphabet)
$R$ : Finite set of rules (productions) of form $A \to \alpha$ where $A \in V, \alpha \in (V \cup \Sigma)^*$
$S \in V$ : Start symbol

1.3 Derivation Example

Grammar for balanced parentheses:

S \to SS \mid (S) \mid \varepsilon

Derivation of "( ) ( )":

Step	Sentential Form	Rule Applied
1	S	Start
2	SS	S → SS
3	(S)S	S → (S)
4	()S	S → ε
5	()(S)	S → (S)
6	()()	S → ε

(Diagram)

2. Derivation Types

2.1 Leftmost vs. Rightmost Derivation

Type	Description	Example (for $S \to 0S1 \mid \varepsilon$ )
Leftmost	Replace leftmost variable first	$S \Rightarrow 0S1 \Rightarrow 00S11 \Rightarrow 0011$
Rightmost	Replace rightmost variable first	$S \Rightarrow 0S1 \Rightarrow 0S01 \Rightarrow 0011$

2.2 Worked Example

Grammar:

E \to E+E \mid E*E \mid (E) \mid a

Leftmost derivation of $a+a*a$ :

E \Rightarrow E+E \Rightarrow a+E \Rightarrow a+E*E \Rightarrow a+a*E \Rightarrow a+a*a

Rightmost derivation of $a+a*a$ :

E \Rightarrow E+E \Rightarrow E+E*E \Rightarrow E+E*a \Rightarrow E+a*a \Rightarrow a+a*a

Note: The parse tree for both derivations is the same: (Diagram)

3. Ambiguity

3.1 Intuition

A grammar is ambiguous if a string has more than one parse tree. This is bad for programming languages — if a compiler can parse 3 + 4 × 5 in two ways, it gives different results (either 35 or 23).

3.2 Ambiguous Grammar

The expression grammar

E \to E+E \mid E*E \mid (E) \mid a

is ambiguous. Two parse trees for $a+a*a$ : Tree 1 (addition first):

(a+a)*a

→ evaluates as

(a+a)*a

Tree 2 (multiplication first):

a+(a*a)

→ evaluates as

a+(a*a)

3.3 Eliminating Ambiguity

Unambiguous grammar (respects precedence):

$E \to E + T \mid T$
$T \to T * F \mid F$
$F \to (E) \mid a$ Now $a+a*a$ has only one parse tree (multiplication binds tighter).

Rule	Level	Associativity
$E \to E + T$	Lowest precedence	Left-associative
$T \to T * F$	Medium precedence	Left-associative
$F \to (E) \mid a$	Highest precedence	—

4. Chomsky Normal Form (CNF)

4.1 Intuition

CNF is a simplified form where every rule has exactly two nonterminals or one terminal. This restriction makes parsing algorithms (like CYK) simpler — they only need to consider combining two adjacent sub-results, like binary tree operations.

4.2 CNF Rules

Every production is of the form:

$A \to BC$ (two nonterminals)
$A \to a$ (one terminal)
$S \to \varepsilon$ (only if ε is in the language, and S never appears on RHS)

4.3 Conversion Algorithm

Step 1: Add new start variable

S_0

Step 2: Eliminate ε-rules (

A \to \varepsilon

) Step 3: Eliminate unit rules (

A \to B

) Step 4: Convert remaining rules to CNF

4.4 Worked Example

Original grammar:

S \to ASA \mid aB

A \to B \mid S

B \to b \mid \varepsilon

Step 1 — New start:

S_0 \to S

S \to ASA \mid aB

A \to B \mid S

B \to b \mid \varepsilon

Step 2 — Eliminate ε-rules ( $B \to \varepsilon$ ): Remove

B \to \varepsilon

. For each rule with B on RHS, add version without B:

S \to aB

becomes

S \to aB \mid a

A \to B

becomes

A \to B \mid \varepsilon

(but we also need to handle ε in A) Step 3 — Eliminate unit rules:

S_0 \to S

A \to B

A \to S

are unit rules. Replace:

A \to B

(and

B \to b

) with

A \to b

. Etc. Step 4 — Convert to CNF:

S_0 \to AS_1 \mid aB \mid a

(where

S_1 = SA

) ... Final CNF:

S_0 \to AS_1 \mid aB \mid a

S_1 \to SA

S \to AS_1 \mid aB \mid a

A \to b \mid AS_1 \mid aB \mid a

B \to b

5. Common Pitfalls

Pitfall 1: Left Recursion in Top-Down Parsing

The mistake: Using

A \to A\alpha

(left recursion) in a grammar for top-down parsing. Why students make it: Left recursion is natural for left-associative operations (

E \to E + T

). How to catch it: Top-down parsers (like LL parsers) infinite loop on left recursion. Correct approach: Use right recursion or EBNF for top-down grammars, or use table-driven parsers (LR) that handle left recursion.

Pitfall 2: Confusing Derivation with Parse Tree

The mistake: Thinking two different derivations mean the grammar is ambiguous. Why students make it: Different derivation orders (leftmost vs. rightmost) give different derivations. How to catch it: Ambiguity means different parse trees, not different derivation orders. Leftmost and rightmost derivations of the same string may differ but have the same parse tree. Correct approach: Two derivations with the same parse tree ≠ ambiguity. Different parse trees = ambiguity.

Pitfall 3: Incorrect ε-Rule Elimination

The mistake: Removing ε-rules without adding replacement rules for all affected productions. Why students make it: The algorithm seems simple — just remove ε and add new versions. How to catch it: Some derivations that were valid before elimination become invalid because required replacements were missed. Correct approach: Find all nullable variables systematically. For every rule

A \to X_1X_2...X_n

, generate all combinations of replacing nullable

X_i

with ε.

6. Key Concepts Reference

Concept	Definition	Application
CFG	$G = (V, \Sigma, R, S)$	Describes context-free languages
Derivation	Step-by-step replacement of variables	Generating strings
Parse tree	Tree representation of derivation	Syntax analysis
Ambiguity	Multiple parse trees for same string	Problem for compilers
CNF	$A \to BC$ or $A \to a$ only	CYK parsing algorithm
Leftmost/Rightmost	Order of variable replacement	Derivation strategies

7. 📝 Practice Questions

Q1: Design a CFG for L = {0^n1^n | n ≥ 0}.

Answer: S → 0S1 | ε

Derivation for 0011: S ⇒ 0S1 ⇒ 0(0S1)1 ⇒ 00ε11 = 0011 Q2: Is the grammar S → 0S1 | 0S | ε ambiguous?

Answer: Yes. The string "01" can be derived in two ways: (1) S ⇒ 0S1 ⇒ 0(ε)1 = 01 (2) S ⇒ 0S ⇒ 0(0S1) ⇒ 0(0ε1) = 001 — wait that's "001", not "01".

Actually let me re-check. For grammar S → 0S1 | 0S | ε: String "01": S ⇒ 0S1 ⇒ 0ε1 = 01 (only this way) String "0": S ⇒ 0S ⇒ 0ε = 0 (only this way) String "00": S ⇒ 0S ⇒ 0(0S) ⇒ 0(0ε) = 00 OR S ⇒ 0S ⇒ 0(0ε) = 00 (same) This grammar is actually unambiguous. Q3: Convert S → aSa | bSb | ε to CNF.

Answer:

Step 1: New start S₀ → S. Step 2: No ε-rules (except S → ε, which is already in CNF form). Step 3: No unit rules. Step 4: Convert: S → aSa becomes S → A₁A₂ where A₁ → a, A₂ → Sa, and Sa → S A₁ Similarly for bSb.

Final CNF: S₀ → A₁A₃ | A₄A₆ | ε S → A₁A₃ | A₄A₆ | ε A₁ → a A₃ → SA₁ A₄ → b A₆ → SA₄

(where A₃ = Sa, A₆ = Sb) Q4: Why are CFGs called "context-free"?

Answer: They are "context-free" because the replacement rule for a variable A → α can be applied regardless of the surrounding symbols (context). Contrast with context-sensitive grammars where the rule may depend on neighboring symbols (e.g., aA → bc means "replace A with c only if preceded by a"). Q5: Design a CFG for L = {a^i b^j c^k | i ≠ j}.

Answer: This is tricky because we can't directly compare. Approach: generate a^i b^j where i ≠ j, then add matching c's.

S → AC | BC A → aA | a (at least one a) C → cC | ε (any number of c's) B → bB | b (at least one b)

Wait, this doesn't capture i ≠ j properly. Better:

S → aSc | aXc | Y (where X generates more a's, Y generates more b's) X → aX | a Y → bYc | bY | b

Actually let me think again:

L = {a^i b^j c^k | i ≠ j}

S → aSc | T T generates {a^i b^j | i ≠ j} (with extra c's? No, c's handled by S)

Better approach: S → aSc | U U → aU | aUb | Ub | ab (generate strings where a count ≠ b count)

Hmm, this is complex. Let me simplify:

S → aSc | T T → aT | aTb | Tb | ab But T needs to ensure i ≠ j.

The standard approach: generate three regions separately: S → aSc | T (S matches c's with outer a's) T → aTb | A | B (T generates inner a's and b's) A → aA | a (more a's) B → bB | b (more b's)

This generates a^i b^j c^k where i = (k + #inner a's), j = #inner b's, and if inner a's ≠ inner b's then i ≠ j. Q6: What is the Pumping Lemma for context-free languages?

Answer: If L is a CFL, there exists p > 0 such that any string s ∈ L with |s| ≥ p can be written as s = uvxyz where:

|vy| ≥ 1 (at least one of v or y is non-empty)

|vxy| ≤ p (the pumped region is bounded)

uv^i xy^i z ∈ L for all i ≥ 0

This differs from the regular pumping lemma: we're pumping TWO parts (v and y) simultaneously, reflecting the binary branching structure of CFG parse trees. Q7: Design a CFG for the language of palindromes over {0,1}.

Answer: S → 0S0 | 1S1 | 0 | 1 | ε

This generates strings that read the same forwards and backwards. For "0110": S ⇒ 0S0 ⇒ 0(1S1)0 ⇒ 0(1ε1)0 = 0110 Q8: Why is the grammar E → E + T | T, T → T * F | F, F → (E) | a unambiguous for expressions?

Answer: It enforces precedence: + is at the top level (E), * is one level lower (T), and atoms/literals are innermost (F). Since each string forces a specific derivation order (you must expand E first to add, T to multiply, F for parentheses), there's exactly one parse tree. Left recursion in E → E + T forces left associativity — there's no alternative way to group the same string.

8. 🔗 Cross-References

Week 3 - DFA Minimization: Regular languages → CFGs
Week 5 - Pushdown Automata: PDA = CFG equivalence
Week 6 - Turing Machines: Beyond context-free
BSCS4032 (Compiler Design): Parse trees, top-down parsing Join Discord PreviousDFA Min & Pumping LemmaNextCFG ↔ PDA

What this note is really saying

Context-Free Grammars

🎯 Learning Objectives

1. Introduction to CFGs

1.1 Intuition

1.2 Formal Definition

1.3 Derivation Example

2. Derivation Types

2.1 Leftmost vs. Rightmost Derivation

2.2 Worked Example

3. Ambiguity

3.1 Intuition

3.2 Ambiguous Grammar

3.3 Eliminating Ambiguity

4. Chomsky Normal Form (CNF)

4.1 Intuition

4.2 CNF Rules

4.3 Conversion Algorithm

4.4 Worked Example

5. Common Pitfalls

Pitfall 1: Left Recursion in Top-Down Parsing

Pitfall 2: Confusing Derivation with Parse Tree

Pitfall 3: Incorrect ε-Rule Elimination

6. Key Concepts Reference

7. 📝 Practice Questions

8. 🔗 Cross-References

Intelligence Hub