Computational Models - Lecture 5tau-cm2019.wdfiles.com/local--files/course-schedule/CFL2.pdf · Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 8 / 47. Proof ofThm

Computational Models - Lecture 5

Handout Mode

Roded Sharan.

Tel Aviv University.

April, 2019

Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 1 / 47

Talk Outline

I Pumping Lemma for context free languages

I Push Down Automata (PDA)

I Closure properties of CFLs

I Sipser’s book, 2.1, 2.2 & 2.3


CFGs and CFLs, reminder

A context-free grammar is a 4-tuple (V ,Σ,R,S), where

I V is a finite set of variables

I Σ is a finite set of terminals (V ∩ Σ = ∅)I R is a finite set of rules of the form A→ x , where A ∈ V and

x ∈ (V ∪ Σ)∗.

I S ∈ V is the start symbol.

I u ∗→ v if u = v , or u → u1 → . . .→ uk → v for some sequenceu1,u2, . . . ,uk

Definition 1

The language of the grammar G, denoted L(G), is {w ∈ Σ∗ : S ∗→ w}


Part I

Non-Context-Free Languages


Proving a Language is not a CFL

I The pumping lemma for finite automata and Myhill-Nerode theorem areour tools for showing that languages are not regular.

I We will now show a similar pumping lemma for context-free languages.

I It is slightly more complicated . . .


Pumping Lemma for CFL (also known as, the uvxyz Theorem)

Theorem 2

For any CFL L there exists ` ∈ N (“critical length"), such that for any w ∈ Lwith |w | ≥ `, there exit u, v , x , y , z ∈ Σ∗ such that w = uvxyz and

I For every i ≥ 0: uv ixy iz ∈ LI |vy | > 0, (“non-triviality")

I |vxy | ≤ `

Basic Intuition:Let L be a CFL and a let w be a “very long” string in L. Then w must have a“tall” parse tree

Hence, some root-to-leaf path must repeat a symbol.


Basic intuition cont.

T

R

u v x y z

R

We have: T ∗→ uRz, R ∗→ vRy , and R ∗→ x .


Proof of Thm 2

Let G be a CFG and let L = L(G).

I Let b be the max number of symbols in right-hand-side of any rule (whatis b for a CNF grammar?).

I We assume wlg. that b > 1 (?)

I Since no node in a parse tree of G has more than b children, at depth dsuch tree has at most bd leaves.

I Let |V | be the number of variables in G, and set ` = b|V |+1. . .

Let w be a string with |w | ≥ `, and let T be parse tree for w (with respect toG) with fewest nodes

I T has height ≥ |V |+ 1

I Some path in T has length ≥ |V |+ 1

I This path has ≥ |V |+ 1 variables

I This path repeats a variable R


Proof of Thm 2 cont.

Set w = uvxyz

T

R

u v x y z

R

I Each occurrence of R produces a string

I Upper produces string vxy

I Lower produces string x


Proving uv ixy iz ∈ L for all i > 1

Replacing smaller by larger yields uv ixy iz, for i > 0.

T

R

u v x y z

R

v yx

R


Proving uv ixy iz ∈ L for i = 0

Replacing larger by smaller yields uxz.

T

R

u

x

z

Together, they establish:

I uv ixy iz ∈ L for all i ≥ 0


Proving |vy | > 0

If v and y are both ε, then

T

R

u

x

z

is a parse tree for w with fewer nodes than T , a contradiction.


Proving |vxy | ≤ `

T

R

u v x y z

R |V|+1

b |V|+1

I Without loss of generality both occurrences of R lie in bottom |V |+ 1variables on the path.

I The upper occurrence of R (from now on R1) generates vxy .I Subtree rooted at R1 is of height at most |V |+ 1.

Hence, |vxy | ≤ b|V |+1 = `.Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 13 / 47

Non CFL example #1

Claim 3L1 = {anbncn : n ∈ N} is not a CFL.

Proof: By contradiction. Assume L1 is a CFL with grammar G, let ` be thecritical length of G and consider w = a`b`c`. Let u, v , x , y , z be the stringswith w = uvxyz guaranteed by Thm 2 for w .

I Both v and y are monochromatic — repetitions of a single letter.

Proof: Otherwise, uv2xy2z would have out-of-order symbols.

I Hence, uv2xy2z is imbalanced

♣


Non CFL example #2

Claim 4

L2 = {aibjck : 0 ≤ i ≤ j ≤ k} is not context free.

Proof: By contradiction. Assume L2 is a CFL with grammar G, let ` be thecritical length of G and consider w = a`b`c`. Let u, v , x , y , z be the stringswith w = uvxyz guaranteed by Thm 2 for w .

I Both v and y are monochromatic.

I If neither v nor y contains a, then uv0xy0z has too many a’s.

I If neither v nor y contains c, then uv2xy2z has too few c’s.

=⇒ non empty v contains only a’s, and non empty y contains only c’s

=⇒ uv2xy2z has more a’s than b’s.

♣


Non CFL example #3

Claim 5L3 = {ww : w ∈ {0,1}∗} is not context-free.

Proof: By contradiction. Assume L3 is a CFL with grammar G, let ` be thecritical length of G and consider w = 0`1`0`1`. Let u, v , x , y , z be the stringswith w = uvxyz guaranteed by Thm 2 for w .

I Assuming vxy is in the first half of w .=⇒ w ′ = uv2xy2z is of the form 0w ′′1`0`1` with ` < |0w ′′| ≤ 2`=⇒ second half of w ′ starts with a 1 . . .

I Similarly, assuming vxy is in the second half of w . Then the last positionof the first half of w ′ = uv2xy2z is a 0. . .

I Assuming vxy straddles the midpoint of w . Then pumping down to uxzyields w ′ = 0`1i0j1` where i and j cannot both be `.

I Clearly, w ′ /∈ L3 (?)

♣Note that {wwR : w ∈ {0,1}∗} is a CFL.


Part II

Push-Down Automata


String generators and string acceptors

I Regular expressions are string generators – they tell us how to generateall strings in a language L

I Finite automata (DFA, NFA) are string acceptors – they tell us if aspecific string w is in L

I CFGs are string generators

I Are there string acceptors for CFLs?

I YES! Push-down automata


Finite automaton


PushDown automaton

(ignore the ’$’ sign)Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 20 / 47

Example #1 — PDA for L1 = {0n1n : n ≥ 0}Informally:

1. Read input symbols

1.1 Push each read 0 on the stack1.2 Pop a 0 for each read 1

2. Accept if stack is empty after last symbol read, and no 0 appears after 1

Recall that L1 is not regular


Example #2 — PDA for L2 = {aibjck : i = j ∨ i = k}Informally:

Read and push a’sEither pop and match with b’s or pop and match with c’s

A non-deterministic choice


PDA – Formal definition

A PDA is a 6-tuple (Q,Σ, Γ, δ,q0,F ), where

I Q is a finite set called the states,

I Σ is a finite set called the input alphabet,

I Γ is a finite set called the stack alphabet,

I δ : Q × Σε × Γε 7→ P(Q × Γε) is the transition function,1

I q0 ∈ Q is the starting state, and

I F ⊆ Q is the set of accepting states.

1Xε := X ∪ {ε}.Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 23 / 47

Formal model of computation

Recall that for a ∈ (Xε)∗, we let d(a) ∈ X ∗ be a without the ε symbols.

Definition 6 (δ̃)

For PDA P = (Q,Σ, Γ, δ,q0,F ), define δ̃P : Q × Σε × Γ∗ 7→ P(Q × Γ∗) by:(q′, s′) ∈ δ̃P(q, σ, s), if ∃a,b ∈ Γε and t ∈ Γ∗ s.t.:

I (q′,b) ∈ δ(q, σ,a)

I s = d(at) and s′ = d(bt)

Namely, reading the symbol σ from the input, can change configuration (stateand stack value) (q, s) into configuration (q′, s′).

It might be that

I |s′| = |s|: a,b 6= ε or a = b = ε

I |s′| > |s|: a = ε and b 6= ε

I |s′| < |s|: a 6= ε and b = ε

As usual, we omit the subscript P when clear from the context.


Formal model of computation, cont.

Definition 7A PDA P = (Q,Σ, Γ, δ,q0,F ) accepts w ∈ Σ∗, if exist a1, . . . ,ak ∈ Σε,r0, . . . , rk ∈ Q and s0, . . . , sk ∈ Γ∗, s.t.

I w = d(a1, . . . ,ak ).

I r0 = q0 and rk ∈ F .

I s0 = ε.

I (ri+1, si+1) ∈ δ̃P(ri ,ai+1, si ), for all 0 ≤ i < k .

Definition 8The language accepted by PDA P = (Q,Σ, Γ, δ,q0,F ), denoted L(P), is theset of all strings w ∈ Σ∗ accepted by P.


An equivalent definition

Definition 9

PDA P = (Q,Σ, Γ, δ,q0,F ) accepts w ∈ Σ∗, if (δ̂P(q0,w , ε))1 ∩ F 6= ∅

Definition 10 (ε-environment)

For PDA P = (Q,Σ, Γ, δ,q0,F ) and (q, s) ∈ Q × Γ∗, letE(q, s) = {(q′, s′) ∈ Q × Γ∗ : ∃(q1, s1), . . . , (qk , sk ) s.t.(q1, s1) = (q, s) ∧ (qk , sk ) = (q′, s′) ∧ ∀i ∈ [k − 1] : (qi+1, si+1) ∈ δ̃(qi , ε, si )}.E(A ⊆ Q × Γ∗) =

⋃(q,s)∈A E(q, s)

Definition 11 (δ̂)

For PDA P = (Q,Σ, Γ, δ,q0,F ) , define δ̂P : Q × Σ∗ × Γ∗ 7→ P(Q × Γ∗) by:

δ̂P(q,w , s) =

{E(q, s), w = ε,

E(⋃

(q′,s′)∈δ̂P(q,w1,...,n−1,s)δ̃P(q′,wn, s′)

), n = |w | ≥ 1. .


PDA languages

The Push-Down Automata Languages, LPDA, is the set of all languages thatcan be described by some PDA:

I LPDA = {L(M) : M is a PDA}

It is immediate that LPDA ) LDFA: every DFA is just a PDA that ignores thestack.

I LPDA = LCFG !!!

Proof sketch next class.


Diagram notation

When drawing the automaton diagram, we use the following notation

I Transition a,b → c from state q to q′ means (q′, c) ∈ δ(q,a,b),and informally means the automaton

I read a from inputI pop b from stackI push c onto stack

I Meaning of ε transitions (informally):

I a = ε : don’t read inputI b = ε: don’t pop any symbolI c = ε: don’t push any symbol


Knowing when stack is empty

It is convenient to be able to know when the stack is empty, but there is nobuilt-in mechanism to do that.

Solution

1. Start by pushing $ onto stack.

2. When you see it again, stack is empty.


A PDA for L1 = {0n1n : n ≥ 0}

Claim 120011 ∈ L(P).

Proof: takew ′1 = ε w ′2 = 0 w ′3 = 0 w ′4 = 1 w ′5 = 1 w ′6 = ε

s0 = ε s1 = $ s2 = 0$ s3 = 00$ s4 = 0$ s5 = $ s6 = εr0 = q1 r1 = q2 r2 = q2 r3 = q2 r4 = q3 r5 = q3 r6 = q4


A PDA for L2 = {aibjck : i = j ∨ i = k}

I Non-determinism is essential here!


Part III

Non-determinism adds power


Non-Determinism Adds Power

A deterministic PDA has at most one transition option per state, stack valueand input (accounting also for ε-transitions).

Theorem: There are context free languages that are accepted only bynon-deterministic push down automata.

Comment: Recall that all regular languages are accepted by deterministicfinite automata. For finite automata, non-determinism does not add power,even though it may make life more convenient.


Non-Determinism Adds Power: Proof

Theorem: Let M be a PDA that accepts

L = {xnyn|n ≥ 0} ∪ {xny2n | n ≥ 0} .

Then M is non-deterministic.Proof: Suppose, by way of contradiction, that M is deterministic.

I Create two copies of this PDA, denoted by M1 and M2, respectively.

I Two states in M1 and M2 are called “cousins” if they are copies of thesame state in the original PDA.


Non-Determinism Adds Power (cont.)

I We now modify these PDA copies to make them into one PDA, M0, overthe alphabet {x , y , z}.

I The stack alphabet is not changed.

I States of the new M0 are those of M1 union M2.

I Start state of the new M0 is the start state of M1.

I The accepting states of the new M0 are the accepting states of M2.



Modifications:

I Erase all x transitions of M2 (formally, on input letter x and any stackoption, M2 moves to a new “dead end” state).

I Replace every existing y transition of M2 by a new z transition (how isthis implemented formally?).

I At this point M2 got only z transition (so while in M2, the letters x and ylead immediately to rejection).

I Erase all x transitions out of accepting states of M1.

I Replace every existing y transition leading out of an accepting state ofM1 to some state q, by a new z transition, and redirect it to the “cousin”of q in M2.



I What language does M0 recognizes?

I If M0 accepts a string, it must be of the form (x ∪ y)∗z∗.

I Surely not all strings of that form are accepted by M0.

I For example, the prefix in (x ∪ y)∗ must be accepted by the original M.

I Otherwise there will be no switch to M2, and no acceptance by M0.

I So the prefix of an accepted string is either of the form xnyn or xny2n.

I And the whole string is of the form xnynz i or xny2nz j .



I What can we say about the z i part? First, i must be greater than 0 for atransition to take place.

I By construction, M2 on z i imitates the actions of M on y i from the samestarting point.

I This means that if M0 accepts xnynz i then M accepts xnyn+i .

I Which is only possible if i = n, so M0 accepts xnynzn, n > 0.

I Notice that M0 cannot accepts xny2nz j , because M does not accepts anystring of the form xny2n+j , j > 0.


Conclusion of Proof

I We just showed that the PDA M0 accepts the language {xnynzn|n ≥ 1}.I By the equivalence theorem, the language that a PDA accepts is a

context free language. By the so called uvxyz pumpinglemma, {xnynzn|n ≥ 1} is not a CFL. Contradiction. ♣

I This means that our initial supposition that the language{xnyn} ∪ {xny2n} is accepted by a deterministic PDA, does not hold.

While thinking about the proof, where would it fail if the original M werenon-deterministic? Specifically, why is it true that for deterministic M,L(M0) 6= ∅, but for non deterministic M, L(M0) = ∅ is possible?


Part IV

Closure Properties ofContext-Free Languages


Simple closure properties of CFL’s

I CFL’s are closed under

I Union: S → S1 | S2I Concatenation: S → S1S2I Star: Snew → SoldSnew | ε

I What about complement and intersection?


Intersection

S1 → A1B1 S2 → A2B2A1 → 0A11|ε A2 → 0A2|εB1 → 2B1|ε B2 → 1B22|ε

L1 = 0n1n2∗ L2 = 0∗1n2n

I L1 ∩ L2 = 0n1n2n

I L1 and L2 are CFLs (why?),

I But L1 ∩ L2 is not a CFL.

I But can’t we run two PDA’s in parallel, and accept iff both accept??

I What about intersection of a CFL with a regular language?


When CFL intersects a regular language

I Are the context free languages closed under intersection with a regularlanguage?

I That is, if L1 is context free languages, and L2 is regular, must L1 ∩ L2be context free languages?

I YES!

I Run PDA L1 and DFA L2 “in parallel” (just like the intersection oftwo regular languages).

I Formal details omitted (but you should be able to figure them out).


Applications

I Is L = {w ∈ {0,1,2}∗ : #0(w) = #1(w) = #2(w)} context free?

I L ∩ 0∗1∗2∗ = {0n1n2n : n ≥ 0} is not context free.I 0∗1∗2∗ is regular.I Context free languages intersected with a regular languages are

context free.I So L is not a context free language

I This could also be established using pumping lemma, but proof above ismore elegant.


Complementation

The fact that CFLs are not closed under intersection, but are closed underunion, implies they are not closed under complementation, asL1 ∩ L2 = L1 ∪ L2.

Here is a direct proof for the above, presenting a non CFL L such that L is aCFL.

I Take L = {ww : w ∈ {0,1}∗}.I L is not a CFL.

I We prove that L is a CFL


Complementation cont.

I For any y ∈ L, either

I y ’s length is odd, orI y ’s length is even, 2`, and ∃i ≥ 1 such that yi 6= y`+i .

I It suffices to construct a CFG for Leven – the even length members of L(why?)

I Let Lσeven = {{0,1}kσ{0,1}j{0,1}kσ{0,1}j : k , j ≥ 0}

I Note that Leven = L0even ∪ L

1even

I and that Lσeven = {{0,1}kσ{0,1}k{0,1}jσ{0,1}j : k , j ≥ 0}

I CFG for L0even left as an ex.


A PDA for L0even

Idea: Guess k , j ≥ 0, and accept w if it is of the form:{0,1}k 0{0,1}k{0,1}j1{0,1}j

A simpler PDA:

I Loops from q1 and q3 changed to 0/1, ε→ 2

I Loops from q2 and q4 changed to 0/1,2→ ε


Documents

Computational Models - Lecture 5tau-cm2019.wdfiles.com/local--files/course-schedule/CFL2.pdf · Roded Sharan (TAU) Computational Models, Lecture 5 April, 2019 8 / 47. Proof ofThm