Handout Mode Iftach Haitner and Yishay Mansour.tau-cm2014.wdfiles.com/local--files/course-schedule/BLecture5_h.pdf · Computational Models - Lecture 51 Handout Mode Iftach Haitner

Computational Models - Lecture 5 1

Handout Mode

Iftach Haitner and Yishay Mansour.

Tel Aviv University.

March 17/19, 2014

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.

Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 1 / 55

Outline

Push Down Automata (PDA)

Closure properties for CFL and testing properties.

Equivalence of CFGs and PDAs

Sipser’s book, 2.1, 2.2 & 2.3

Announcement: Two new recitations, Thursday 12-13 and 14-15.


Part I

Push-Down Automata


String Generators and String Acceptors

Regular expressions are string generators – they tell us how togenerate all strings in a language L

Finite Automata (DFA, NFA) are string acceptors – they tell us if aspecific string w is in L

CFGs are string generators

Are there string acceptors for CFLs?

YES! Push-down automata


A Finite Automaton


A PushDown Automaton

(ignore the ’$’ sign)Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 6 / 55

Example 1 — PDA for L1 = {0n1n : n ≥ 0}

Informally:

1 Read input symbols1 Push each read 0 on the stack2 Pop a 0 for each read 1

2 Accept if stack is empty after last symbol read, and no 0 appearsafter 1

Recall that L1 is not regular


Example 2 — PDA for L2 = {aibjck : i = j ∨ i = k}

Informally:

Read and push a’sEither pop and match with b’s or pop and match with c’s

A non-deterministic choice


Pushdown Automaton (PDA) — Formal Definition

A PDA is a 6-tuple (Q,Σ, Γ, δ,q0,F ), where

Q is a finite set called the states,

Σ is a finite set called the input alphabet,

Γ is a finite set called the stack alphabet,

δ : Q × Σε × Γε → P(Q × Γε) is the transition function,2

q0 ∈ Q is the starting state, and

F ⊆ Q is the set of accepting states.

2Xε := X ∪ {ε}.Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 9 / 55

The language accepted by a PDA

A pushdown automaton (PDA) M accepts a string w , if there is a“computation" of M on w (see next slide) that leads to anaccepting state.

The language accepted by M, denoted L(M), is the set of allstrings w ∈ Σ∗ accepted by M.

A (non-deterministic) PDA may have many computations on asingle string


Model of ComputationThe following is with respect to M = (Q,Σ, Γ, δ,q0,F ).

Definition 1 ( δ∗)

For w ∈ Σ∗ let δ̂(q,w , s) be all pairs (q′, s′) ∈ Q × Γ∗ for which existw ′

1, . . . ,w′m ∈ Σε, states r1, . . . , rm ∈ Q and strings s0, s1, . . . sm ∈ Γ∗

s.t.:1 w = w ′

1, . . . ,w′m, r0 = q, rm = q′, s0 = s and sm = s′

2 For every i ∈ {0, . . . ,m − 1} exist a,b ∈ Γε and t ∈ Γ∗ s.t.:1 (ri+1, b) ∈ δ(ri ,w ′

i+1, a)2 si = at and si+1 = bt

Namely, (q′, s′) ∈ δ̂(q0,w , ε) if after reading w (possibly within-between ε moves), M can find itself in state q′ and stack values′.

M accepts w ∈ Σ∗ if ∃q′ ∈ F such that (q′, t) ∈ δ̂(q0,w , ε) forsome t .


Diagram Notation

When drawing the automata diagram, we use the following notation

Transition a,b → c from state q to q′ means (q′, c) ∈ δ(q,a,b),and informally means the automata

◮ read a from input◮ pop b from stack◮ push c onto stack

Meaning of ε transitions ((informally):◮ a = ε : don’t read input◮ b = ε: don’t pop any symbol◮ c = ε: don’t push any symbol


A PDA for L1 = {0n1n : n ≥ 0}

Claim 20011 ∈ L(P).

Proof: takew ′

1 = ε w ′2 = 0 w ′

3 = 0 w ′4 = 1 w ′

5 = 1 w ′6 = ε

s0 = ε s1 = $ s2 = 0$ s3 = 00$ s4 = 0$ s5 = $ s6 = ε

r0 = q1 r1 = q2 r2 = q2 r3 = q2 r4 = q3 r5 = q3 r6 = q4


A PDA for L1 = {0n1n : n ≥ 0}

We want to show that L(P) = L1 = {0n1n : n ≥ 0}What do we need to prove?

Claim 3

δ̂(q1, ε, ε) = {(q1, ε), (q2, $)}.

δ̂(q1,0k , ε) = {(q2,0k$)}, for k ≥ 1.

δ̂(q1,0k1i , ε) = {(q3,0k−i$)}, for k > i ≥ 1.

δ̂(q1,0k1k , ε) = {(q3, $), (q4, ε)}, for k ≥ 1.

δ̂(q1,w , ε) = ∅, for w 6∈ {0k1i |k ≥ i ≥ 0}.


Knowing when stack is empty

It is convenient to be able to know when the stack is empty, but there isno built-in mechanism to do that.

Solution1 Start by pushing $ onto stack.2 When you see it again, stack is empty.


A PDA for L2 = {aibjck : i = j ∨ i = k}


A PDA for L2 = {aibjck : i = j ∨ i = k}, cont.

Non-determinism is essential here!

Unlike finite automata, non-determinism does add power.

But we saw deterministic algorithm to deicide any CFL (and as wesee later, CFLs are exactly the languages decided by PDAs)!

How to prove that non-determinism adds power?...Does not seem trivial or immediate.

Another example: L = {xnyn : n ≥ 0} ∪ {xny2n : n ≥ 0} isaccepted by a non-deterministic PDA, but not by a deterministicone. (Proof? Book!)


PDA Languages

The Push-Down Automata Languages, LPDA, is the set of alllanguages that can be described by some PDA:

LPDA = {L(M) : M is a PDA}

It is immediate that LPDA ) LDFA: every DFA is just a PDA that ignoresthe stack.

LCFG ⊆ LPDA ?

LPDA ⊆ LCFG ?

LPDA = LCFG !!!

Proof in last hour of class.


Part II

Closure Properties


Simple Closure Properties of Context-Free Languages

CFL’s are closed under◮ Union:S → S1 | S2◮ Concatenation:S → S1S2◮ Star:Snew → ε | Sold | Sold Snew

What about complement and intersection?


Intersection

S1 → A1B1 S2 → A2B2

A1 → 0A11|ε A2 → 0A2|εB1 → 2B1|ε B2 → 1B22|ε

L1 = 0n1n2∗ L2 = 0∗1n2n

L1 ∩ L2 = 0n1n2n

L1 and L2 are CFLs (why?),

But L1 ∩ L2 is not a CFL.

But can’t we run two PDA’s in parallel, and accept iff bothaccept??

What about intersection of a CFL with a regular language?


When CFL Intersects Regular Language

Are the context free languages closed under intersection with aregular language?

That is, if L1 is context free languages, and L2 is regular, mustL1 ∩ L2 be context free languages?

Run PDA L1 and DFA L2 “in parallel” (just like the intersection oftwo regular languages).

Formal details omitted (but you should be able to figure them out).


Applications

Is L = {w ∈ {0,1,2}∗ : #0(w) = #1(w) = #2(w)} context free?◮ L ∩ 0∗1∗2∗ = {0n1n2n : n ≥ 0} is not context free.◮ 0∗1∗2∗ is regular.◮ Context free languages intersected with a regular languages are

context free.◮ So L is not a context free language

This could also be established using pumping lemma, but proofabove is more elegant.


Complementation

The fact that CFLs are not closed under intersection, but are closedunder union, implies they are not closed under complementation, asL1 ∩ L2 = L1 ∪ L2.

We give a simple example where L is not CFL but L is.

Take L = {ww : w ∈ {0,1}∗}.

L is not a CFL (why?)

We prove that L is a CFL


Complementation cont.

For any y ∈ L, either◮ y ’s length is odd, or◮ y ’s length is even, 2ℓ, and ∃i ≥ 1 such that yi 6= yℓ+i .

It suffice to construct a PDA/CFG for Leven – the even lengthmembers of L (why?)

Let Lσ

even = {{0,1}kσ{0,1}j{0,1}kσ{0,1}j : k , j ≥ 0}

Note that Leven = L0even ∪ L

1even

and that Lσ

even = {{0,1}kσ{0,1}k{0,1}jσ{0,1}j : k , j ≥ 0}

CFG for L0even

S → ABA → CAC | 0B → CBC | 1C → 0 | 1


A PDA for L0even

Idea: Guess k , j ≥ 0, and accept w if it is of the form:{0,1}k 0{0,1}k{0,1}j1{0,1}j


Homomorphism and Inverse Homomorphism

Homomorphism: replaces each letter with a word

Example: h(1) = aba, h(0) = aah(010) = aa aba aaL1 = {0n1n|n ≥ 0}, h(L1) = {a2n(aba)n|n ≥ 0}.

Claim: Assuming that L is a CFL, then so is h(L)

Proof? use the grammar

Inverse homomorphism: h−1(w) = {x : h(x) = w},h−1(L) = {x : h(x) ∈ L}

Example: L2 = {anbnai |n, i ≥ 0}, h−1(L2) = {10i ,0i |i ≥ 0}.

Claim: Assuming that L is a CFL, then so is h−1(L)

Proof? use the automata (see next)


CFL’s are Closed Under Inverse Homomorphism

Idea: let P be a PDA for L. On input word w , emulate P(h(w))

But we cannot afford to store h(w)!

Solution, compute h(w) “on demand":

Algorithm 4

Input w :

1 Initialized a “buffer" Buff to h(a), where a is the first letter of w

2 Emulate a running of P with Buff as its input string.

Each time Buff is fully read by P, set Buff = h(a), where a is the nextletter in w (if exists)

3 Accept iff P does

How do we implement Buff?


CFL’s are Closed Under Inverse Homomorphism

Given PDA (Q,Σ, Γ, δ,q0,F ) and homomorphism h : Σ1 → Σ∗, wedefine a new PDA (Q′,Σ1, Γ, δ

′,q′0,F

′).

Let k = maxσ1∈Σ1 |h(σ1)| and Σ̄ = ∪0≤i≤kΣi .

Q′ = Q × Σ̄.

q′0 = [q0, ε].

F ′ = F × {ε}.δ′ is define as follows:

◮ if (p, γ) ∈ δ(q, ε,Y ) then ([p, x ], γ) ∈ δ′([q, x ], ε,Y ), for any x ∈ Σ̄.◮ if (p, γ) ∈ δ(q, a,Y ) then ([p, x ], γ) ∈ δ′([q, ax ], ε,Y ), for any x ∈ Σ̄.◮ ([q, h(a)],Y ) ∈ δ′([q, ε], a,Y ), for a ∈ Σ1 and Y ∈ Γ.


Part III

Algorithmic Questions


Emptiness of CFGs

Question 5Given a CFG, G, is L(G) = ∅?

In other words, is there a string generated by G?

Theorem 6There is an algorithm that solves this problem (and always halts).

Possible approaches for a proof:

Bad Idea: We know how to test whether w ∈ L(G) for any stringw , so just try it for each w ...

Better Idea: Can the start variable generate a string of terminals?

A more holistic approach: Can a particular variable generate astring of terminals?


Removing redundant variables and terminals of a CFG

1 Mark all terminal symbols in G.2 Repeat until no new variable become marked:

Mark any A where A → U1U2 . . .Uk and all Ui have already beenmarked.

3 Remove all unmarked variables, and any rule they appear in.4 If S is removed, then L(G) = ∅.5 Remove any variable A not reachable from S.6 Remove any terminal which does not appear in some rule.


Checking Emptiness

Algorithm 7

Input a CFG G1 Remove redundant variables and terminals to get G′.2 Return TRUE (i.e., L(G′) = ∅) iff S is removed

Correctness?


CFGs Fullness

Question 8Given a CFG G, is L(G) = Σ∗?

We just saw an algorithm to determine, given a CFG G, whetherL(G) = ∅

L(G) = Σ∗ iff L(G) = ∅. Why not modify the algorithm so itdetermines emptiness of the complement?

Unfortunately, CFGs are not closed under complement.

Fact 9There is no algorithm to solve CFG fullness.

We are not prepared to prove this remarkable fact (yet).


Finiteness of CFGs

Question 10Given a CFG G, is |L(G)| finite?

1 Remove redundant variables and terminals.2 Turn into a CNF form3 Create a graph C whose nodes are variables and its directed

edges are derivations.4 Return TRUE iff C has a no cycles.

Correctness?


CFGs Inherent Ambiguity

Question 11Given a CFG G, is L(G) inherently ambiguous?

This means that for any CFG that generates L(G), there is a word inthe language with two different parse trees.

Fact 12There is no algorithm to solve CFG inherent ambiguity.

We will not prove this fact, yet you want to know it to put things incontext.


When Are Two CFGs equivalent?

Question 13Given two CFG G1 and G2, test if L(G1) = L(G2).Is there an algorithm to solve this problem?


Part IV

Equivalence Theorem


The CFG–PDA Equivalence Theorem

Theorem 14LPDA = LCFG : A language is context free if and only if somepushdown automata accepts it.

This time (unlike the regular expression vs. regular languagestheorem), both the proof “if” part and of the “only if” part are non trivial.

Proof sketch follows.


CFL =⇒ PDA

Lemma 15

LCFG ⊆ LPDA : If a language is context free, then some PDA accepts it.

Let L be a context-free language, and let G = (V ,Σ,R,S) becontext-free grammar for L

We build a PDA P = (Q,Σ, Γ, δ,q0,F ), such that on input w it“figures out" if there is a derivation of w using G.

Question 16How does P figure out which substitution to make?

Answer: It guesses.


Simplifying Assumptions

1 In a single move, a PDA can push a whole word (from some fixedset) into the stack (first letter at the top)

Can we justify it?2 When deriving a word form a CFL, we always substitute the left

most variable

Does it change the derived language?


Informal Description of P

Algorithm 17 ( P)

1 Push S$ on stack2 While top of the stack t is not $:3 If t is variable A,

(non-deterministically) select rule A → α and substitute.4 If t is a terminal a,

read next input and compare; Reject if different.5 Accept if end of input and stack is empty


State Diagram for P


Example

consider the CFG:S → 0S1|ε.

The related PDA:


Claim: L(P) = L(G)

Claim 18

S ∗→ α iff α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, ε).

Does the above yields that L(P) = L(G)?


S ∗→ α =⇒ α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $)

Proof by induction on the number of derivations steps used to yield α

from S.

0 derivation steps: hence α = S. Since (qloop,S$) ∈ δ̂(qloop, ε, $),the proof follows for α1 = ε and α2 = S.

Assume S ∗→ α in k > 0 derivation steps, and let α′ be the string

derived by the first (k − 1) steps.

By i.h α′ = α′1α

′2 such that (qloop, α

′2$) ∈ δ̂(qloop, α

′1, $)

Write α′2 = w1Aw2 where A is the left most variable in α′

2.

The k ’th derivation step replaces this occurrence of A with a stringt (why?)

It is easy to see that (qloop, tw2$) ∈ δ̂(qloop, α′1w1, $).

To complete the proof take α1 = α′1w1 and α2 = tw2.


α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $) =⇒ S ∗→ α

Proof by induction on the number of steps used by P to process α1.

A single step: α1 = ε and α2 = S$, and the proof follows sinceS ∗→ S.

Assume α1 was processed in k > 1 steps, and let α′1 and α′

2 bethe input string read and the stack value before the last step

Note that (qloop, α′2$) ∈ δ̂(qloop, α

′1, $).

By i.h S ∗→ α′ = α′

1α′2.

In case the k ’th move of P is reading an input character, thenα1α2 = α′

1α′2, and therefore S ∗

→ α1α2

Otherwise, α′1 = α1, α′

2 = Aw and α2 = tw for some ruleA → t ∈ R

Hence S ∗→ α1α2


PDA =⇒ CFL

Lemma 19LPDA ⊆ LCFG: If a PDA accepts a language then it is context free.

We prove the lemma by constructing a CFG G for a language Laccepted by a PDA PLet P = (Q,Σ, Γ, δ,q0,F ). We assume wlg. that:

◮ A single accepting state qa ∈ F .◮ P empties the stack before accepting◮ Each transition either pops or pushes

Can we justify the above?


Defining G = (V ,Σ,R,S)

V = {Apq : p,q ∈ Q}

Idea: Apq will generate all strings that take P from p with an emptystack, to q with an empty stack

S = Aq0,qa

Initially R = ∅ and1 Add {Apq → Ap,r Ar ,q : p, q, r ∈ Q} to R2 Add {Aqq → ε : q ∈ Q} to R3 For all p, r , s, q ∈ Q, a, b ∈ Σε and t ∈ Γ such that

1 (r , t) ∈ δ(p, a, ε) and2 (q, ε) ∈ δ(s, b, t)

add Apq → aAr ,sb to R


Example PDA to CFG

Aq1,q4 → Aq2,q3

Aq2,q3 → 0Aq2,q31Aq2,q3 → 0Aq2,q21.Aq2,q2 → ε.


PDA Computation corresponding to Apq → Ap,rAr ,q


PDA Computation corresponding to Apq → aAr ,sb


Claim: L(G) = L(P)

Claim 20

Apq∗→ w ∈ Σ∗ iff (q, ε) ∈ δ̂(p,w , ε)

Proof by induction on the number of derivation rules/ transitions


A Short Summary

Regular Languages ≡ Finite Automata.

Context Free Languages ≡ Push Down Automata.

Closure properties of regular languages and of CFLs.

Most algorithmic problems for finite automata are solvable.

Some algorithmic problems for finite automata are not solvable.

Pumping lemmata for both classes of languages.

There are additional languages out there.


The View Over The Horizon

context free

regular

decidable

enumerable


Documents

Handout Mode Iftach Haitner and Yishay Mansour.tau-cm2014.wdfiles.com/local--files/course-schedule/BLecture5_h.pdf · Computational Models - Lecture 51 Handout Mode Iftach Haitner