CFGs, PDAs, and the Pumping Lemma

Outline Review Chomsky Normal Form Equivalence between PDAs and CFG CFGs⇒ PDAs Non-context-free languages

CFGs, PDAs, and the Pumping Lemma204213 Theory of Computation

Jittat Fakcharoenphol

Kasetsart University

December 6, 2008


Outline

1 Review

2 Chomsky Normal Form

3 Equivalence between PDAs and CFG

4 CFGs ⇒ PDAs

5 Non-context-free languages


Quiz

Let A = {a(n2)|n ≥ 0}.

E.g., a, aaaa, aaaaaaaaa∈ A.

Prove that A is not regular.

Hint: using the pumping lemma (together with somecalculations)


Quiz

Let A = {a(n2)|n ≥ 0}.E.g., a, aaaa, aaaaaaaaa∈ A.




Quiz

Let A = {a(n2)|n ≥ 0}.E.g., a, aaaa, aaaaaaaaa∈ A.




More Example

Let B = {0i1j |i > j ≥ 0}Prove that B is not regular.

Hint: try to pump down.

More hint: Let p be the pumping length. Try 0p+11p.


More Example

Let B = {0i1j |i > j ≥ 0}Prove that B is not regular.

Hint: try to pump down.

More hint: Let p be the pumping length. Try 0p+11p.


Review: Chomsky normal form

CNF

A context-free grammar is in Chomsky normal form is every ruleis of the form

A→ BC

A→ a

where a is any terminal and A, B, and C are any variables,

except that B and C cannot be the start variable.We also permit the rule S → ε, where S is the start variable.



CNF


A→ BC

A→ a

where a is any terminal and A, B, and C are any variables,except that B and C cannot be the start variable.

We also permit the rule S → ε, where S is the start variable.



CNF


A→ BC

A→ a

where a is any terminal and A, B, and C are any variables,except that B and C cannot be the start variable.We also permit the rule S → ε, where S is the start variable.


Any CFGs can be converted into CNF

Theorem 1

Any context-free grammar is generated by a context-free grammarin Chomsky normal form.

We shall not do the full proof, but will show how to do so byexample. (See also Example 2.10 on the book.)


Any CFGs can be converted into CNF

Theorem 1

Any context-free grammar is generated by a context-free grammarin Chomsky normal form.

We shall not do the full proof, but will show how to do so byexample. (See also Example 2.10 on the book.)


Step 1: The start variable cannot be on the right-hand side

Suppose that S is the start variable.

An example of violated rules: S → aS , or A→ BS .

We introduce a new start variable S0 and add rule

S0 → S


Step 1: The start variable cannot be on the right-hand side

Suppose that S is the start variable.

An example of violated rules: S → aS , or A→ BS .

We introduce a new start variable S0 and add rule

S0 → S


Step 2: ε rules

Sample rules:B → aAb|bAcA

A→ c |aA|ε

Remove A→ ε and on any occurrence of A add new ruleswhere A replaced by ε.

Resulting rules:

B → aAb ⇒ B → aAb|ab

B → bAcA⇒ B → bAcA|bcA|bAc|bc

A→ aA⇒ A→ aA|a


Step 2: ε rules


A→ c |aA|ε


Resulting rules:

B → aAb ⇒

B → aAb|ab




Step 2: ε rules


A→ c |aA|ε


Resulting rules:





Step 2: ε rules


A→ c |aA|ε


Resulting rules:


B → bAcA⇒

B → bAcA|bcA|bAc|bc



Step 2: ε rules


A→ c |aA|ε


Resulting rules:


B → bAcA⇒ B → bAcA|bcA|bAc |bc



Step 2: ε rules


A→ c |aA|ε


Resulting rules:



A→ aA⇒

A→ aA|a


Step 2: ε rules


A→ c |aA|ε


Resulting rules:





Step 3: unit rules

Sample rules:C → Ba|Ac

B → aAb|bAcA

A→ B|c

Remove A→ B and on any occurrence of A add new ruleswhere A replaced by B.

Resulting rules:

C → Ba|Ac ⇒ C → Ba|Ac|Bc

B → aAb|bAcA⇒ B → aAb|aBb|bAcA|bBcA|bAcB|bBcB

A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:

C → Ba|Ac ⇒

C → Ba|Ac|Bc


A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:



A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:


B → aAb|bAcA⇒

B → aAb|aBb|bAcA|bBcA|bAcB|bBcB

A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:


B → aAb|bAcA⇒ B → aAb|aBb|

bAcA|bBcA|bAcB|bBcB

A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:



A→ c


Step 3: unit rules


B → aAb|bAcA

A→ B|c


Resulting rules:



A→ c


Step 4: long rules

Sample rules:C → abC |asbdB

Split rules into short rules and add more variables to connectthem.

Resulting rules:

C → abC ⇒ C → aC1, C1 → bC

C → asbdB ⇒


Step 4: long rules



Resulting rules:

C → abC ⇒ C → aC1, C1 → bC

C → asbdB ⇒


Step 4: long rules



Resulting rules:

C → abC ⇒

C → aC1, C1 → bC

C → asbdB ⇒


Step 4: long rules



Resulting rules:

C → abC ⇒ C → aC1, C1 → bC

C → asbdB ⇒


Step 4: long rules



Resulting rules:

C → abC ⇒ C → aC1, C1 → bC

C → asbdB ⇒


Step 5: remove rules with terminal

Sample rules:C → aC

D → ab|a

Replace terminals with new variables and add rules that thenew variables derive to that terminals.

Resulting rules:C → AC

A→ a

D → AB|a

B → b




D → ab|a



A→ a

D → AB|a

B → b




D → ab|a



A→ a

D → AB|a

B → b


Context-free languages

CFL

A language described by some context-free grammar is called acontext-free language.


Equivalence

Theorem 2

A language is context-free if and only if some pushdownautomaton recognizes it.


Again, two directions to prove the equivalence

Only-if: If a language is context-free, it is recognized by somepushdown automaton.

Given a CFG G , construct a PDA P that recognizes thelanguage generated by G .

If: A language is context-free if it is recognized by somepushdown automaton.

Given a PDA P, construct a CFG G that generates a languagerecognized by P.




















Plan for today

Today we’ll cover only the only-if part, i.e., given a CFL describedby CFG G , we’ll construct a PDA P that recognizes G .


Any CFLs can be recognized by PDAs

Take an example CFG G :

S → AB

A→ aAb|ε

B → cB|c

How can we recognize string generated by G?

Consider aabbccc.


Any CFLs can be recognized by PDAs

Take an example CFG G :

S → AB

A→ aAb|ε

B → cB|c

How can we recognize string generated by G?

Consider aabbccc.


Generating: aabbccc

CFG G

S → AB

A→ aAb|ε

B → cB|c

and aabbccc.

Maybe we can try to generate itusing a PDA:

S ⇒ AB

⇒ aAbB

⇒ aaAbbB

⇒ aaεbbB

⇒ aabbcB

⇒ aabbccB

⇒ aabbccc


Generating: aabbccc

CFG G

S → AB

A→ aAb|ε

B → cB|c

and aabbccc.

Maybe we can try to generate itusing a PDA:

S ⇒ AB

⇒ aAbB

⇒ aaAbbB

⇒ aaεbbB

⇒ aabbcB

⇒ aabbccB

⇒ aabbccc


Generating a string

So, we want to generate a string using a PDA.

How can we generate the correct derivation?

We guess the rule. We always make a correct guess, becausePDAs are nondeterministic machines.

Where should we put the string (and its intermediatederivations)? How can we remember it?

A memory. Yes, we have a memory: a stack

But a stack has a very limited access rule. How can I do thederivation from aAbB ⇒ aaAbbB.

What do you want to do? aAbB ⇒ aAbB ⇒ aaAbbBOkay, why are you stuck at a?Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string



We guess the rule.

We always make a correct guess, becausePDAs are nondeterministic machines.






Generating a string









Generating a string









Generating a string





A memory.

Yes, we have a memory: a stack




Generating a string









Generating a string









Generating a string







What do you want to do?

aAbB ⇒ aAbB ⇒ aaAbbBOkay, why are you stuck at a?Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string







What do you want to do? aAbB ⇒

aAbB ⇒ aaAbbBOkay, why are you stuck at a?Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string







What do you want to do? aAbB ⇒ aAbB ⇒

aaAbbBOkay, why are you stuck at a?Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string







What do you want to do? aAbB ⇒ aAbB ⇒ aaAbbB

Okay, why are you stuck at a?Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string







What do you want to do? aAbB ⇒ aAbB ⇒ aaAbbBOkay, why are you stuck at a?

Because it’s not a variable.So, anything we can do to get rid of it?


Generating a string







What do you want to do? aAbB ⇒ aAbB ⇒ aaAbbBOkay, why are you stuck at a?Because it’s not a variable.

So, anything we can do to get rid of it?


Generating a string









Generate and match

aabbccc S

aabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc AB

aabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbB

aabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbB

aabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbB

aabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbB

aabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbB

aabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbB

aabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbB

aabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcB

aabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcB

aabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccB

aabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccB

aabbccc aabbcccaabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbccc

aabbccc aabbccc


Generate and match

aabbccc Saabbccc ABaabbccc aAbBaabbccc aAbBaabbccc aaAbbBaabbccc aaAbbBaabbccc aabbBaabbccc aabbBaabbccc aabbBaabbccc aabbcBaabbccc aabbcBaabbccc aabbccBaabbccc aabbccBaabbccc aabbcccaabbccc aabbccc


The algorithm for PDA

1 Push empty stack symbol $ on the stack

2 Push start variable on the stack

3 Repeat

4 — Depending on the top of stack:

5 — — If it’s a terminal,— — — match with the same terminal on the input

6 — — If it’s a variable,— — — pick some substitution rule and put that on the stack

7 Until nothing’s left on the stack (you’ll see $).

8 Accept if $ is on top of the stack.





3 Repeat










3 Repeat










3 Repeat










3 Repeat










3 Repeat







Overall structure

qstart

qloop

qaccept

e, A --> we,e --> S

e,$ --> e

e,e --> $

a, a --> e

for rule A --> w

for each terminal a


Practice:

CFG G1

S → AB

A→ aAb|ε

B → cB|c


Practice:

CFG G2

S → aTb|b

T → Ta|ε


Formal proof


Non-context-free language

Can you find a CFG describing the language {anbncn|n ≥ 0}?

I bet you can’t.


Non-context-free language

Can you find a CFG describing the language {anbncn|n ≥ 0}?I bet you can’t.


Pumping lemma for CFL

Theorem 3 (pumping lemma for CFL)

If A is a context-free language, then there is a pumping length psuch that for any string s ∈ A of length at least p, s can be dividedinto 5 pieces s = uvxyz satisfying the following conditions

1 for each i ≥ 0, uv ixy iz ∈ A,

2 |vy | > 0, and

3 |vxy | ≤ p.


Parse tree for s

S

R

R

S

R

R

S

R

R

xu v y z

xv y

z

x

u v y z u


C = {anbncn|n ≥ 0} is not context-free (1)

We’ll prove by contradiction. Assume that C is context-free.

Thus, there exists a pumping length p.

Consider s = apbpcp ∈ C . Note that |s| ≥ p.

The pumping lemma states that we can divide s = uvxyz ,such that uv ixy iz ∈ C for any i ≥ 0.

We’ll show that this leads to a contradiction.






























First proof

There are two cases.

Case 1, if each of v and y contains only one kind ofalphabets.

Then consider s ′ = uv2xy2z . Note that at leastone alphabet appears in s ′ in fewer times than the others;thus, s ′ 6∈ C .

Case 2, if v or y contains two kinds of alphabets. Note thats ′ = uv2xy2z contains alphabets in the wrong order. Again,s ′ 6∈ C .

Note that in either case, s cannot be pumped, and this contradictsthe assumption that C is context-free.


First proof


Case 1, if each of v and y contains only one kind ofalphabets. Then consider s ′ = uv2xy2z . Note that at leastone alphabet appears in s ′ in fewer times than the others;thus, s ′ 6∈ C .




First proof



Case 2, if v or y contains two kinds of alphabets.

Note thats ′ = uv2xy2z contains alphabets in the wrong order. Again,s ′ 6∈ C .



First proof






First proof






Second proof

Note that the first proof doesn’t use the 3rd property, stating that|vxy | ≤ p.

Given that fact, we know that v and y cannot containall three types of alphabets. Therefore, s ′ = uv2xy2z containsdifferent numbers of a’s, b’s, or c’s, and s ′ 6∈ C . This, again, leadsto a contradiction.


Second proof

Note that the first proof doesn’t use the 3rd property, stating that|vxy | ≤ p. Given that fact, we know that v and y cannot containall three types of alphabets.

Therefore, s ′ = uv2xy2z containsdifferent numbers of a’s, b’s, or c’s, and s ′ 6∈ C . This, again, leadsto a contradiction.


Second proof

Note that the first proof doesn’t use the 3rd property, stating that|vxy | ≤ p. Given that fact, we know that v and y cannot containall three types of alphabets. Therefore, s ′ = uv2xy2z containsdifferent numbers of a’s, b’s, or c’s, and s ′ 6∈ C .

This, again, leadsto a contradiction.


Second proof

Note that the first proof doesn’t use the 3rd property, stating that|vxy | ≤ p. Given that fact, we know that v and y cannot containall three types of alphabets. Therefore, s ′ = uv2xy2z containsdifferent numbers of a’s, b’s, or c’s, and s ′ 6∈ C . This, again, leadsto a contradiction.

Documents

CFGs, PDAs, and the Pumping Lemma