Computational Models - Lecture 21tau-cm2014.wdfiles.com/local--files/course-schedule/BLecture2_h.pdf · Computational Models - Lecture 2 Non-Deterministic Finite Automata (NFA) Closure

Computational Models - Lecture 2 1

Handout Mode

Iftach Haitner and Yishay Mansour.

Tel Aviv University.

February 24/26, 2014

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by MauriceHerlihy, Brown University.

Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 2 February 24/26, 2014 1 / 74

Some administrative Notes

MidTerm Exam (final date): Friday, March 28.

Homework assignment 1 was published (or to be published soon).

It is unlikely you’ll be able to solve it on your own, if the first timeyou think about is the night before the deadline.

While we can hardly detect dependencies in preparation of thehomework (still, this sometimes happens), we actively enforcemutual independence in the midterm and final exams.


Computational Models - Lecture 2

Non-Deterministic Finite Automata (NFA)

Closure of Regular Languages Under⋃, ‖, ∗

Regular expressions

Equivalence with finite automata

Sipser’s book, 1.1 − 1.3


Part I

Non-Deterministic Finite Automata


DFA – formal definition, (reminder)

Definition 1 (DFA)

A deterministic finite automaton (DFA) is a 5-tuple (Q,Σ, δ,q0,F ),where

Q is a finite set called the states,

Σ is a finite set called the alphabet,

δ : Q × Σ 7→ Q is the transition function,

q0 ∈ Q is the start state, and

F ⊆ Q is the set of accept states.


Formal model of computation, (reminder)

Definition 2

M = (Q,Σ, δ,q0,F ) accepts w ∈ Σ∗ if δ̂(q0,w) ∈ F .

Definition 3 ( δ̂)

For DFA M = (Q,Σ, δ,q0,F ), define δ̂ : Q × Σ∗ 7→ Q by

δ̂(q,w) =

δ(q,w), |w | = 1δ(δ̂(q,w1,...,n−1),wn), n = |w | > 1q, w = ε.

.


The language of a DFA, (reminder)

Definition 4The language of a DFA M, denoted L(M), is the set of strings that Maccepts.

Definition 5A language is called regular, if some deterministic finite automatonaccepts it.


The Regular operations (reminder)

Let A and B be languages.

The union operation:

A∪B = {x : x ∈ A or x ∈ B}

The concatenation operation:

A‖B = {xy : x ∈ A and y ∈ B}

The star operation:

A∗ = {x1x2 . . . xk : k ≥ 0 and each xi ∈ A}


Closure under union (reminder)

Theorem 6If A1 and A2 are regular languages, then so is A1 ∪A2.

Approach to Proof:

Some DFA M1 accepts A1

Some DFA M2 accepts A2

Construct DFA M that accepts A1 ∪ A2.

In our construction:

states of M were Cartesian product of M1 and M2 states.

M “Emulates" simultaneously both M1 and M2

The accepting states of M include an accepting state of either M1

or M2.


Closure under concatenation

Theorem 7If L1, L2 are regular languages, then so is L1‖L2.

This is much harder to prove.

Idea: Emulate M1 for a while, then switch to M2.

Problem : But when do you switch?

Seems hard to do with DFAs.

This leads us into non-determinism.


NFA — Non-deterministic Finite Automata

q4q1 q

2 q30

0,1 0,1

10

An NFA may have more than one transition labeled with the samesymbol,

An NFA may have no transitions labeled with a certain symbol,

An NFA may have transitions labeled with ε, the symbol of theempty string. Will deal with this latter

Every DFA is also an NFA. (?)


Non-deterministic computation

q4q1 q

2 q30

0,1 0,1

10

What happens when more than one transition is possible?

The machine “splits” into multiple copies

Each branch follows one possibility

Together, branches follow all possibilities.

If the input doesn’t appear, that branch “dies”.

Automaton accepts if some branch accepts.


Non-Deterministic Computation

q4q1 q

2 q30

0,1 0,1

10

What happens on string 1001?


The String 1001

q4q1 q

2 q30

0,1 0,1

10

q1

q1

q2q1

q2q1 q3

q1 q4

1

0

0

1

symbol


Why non-determinism?

Theorem 8 (Informal, to be proved soon)

Deterministic and non-deterministic finite automata, accept exactly thesame set of languages.

Q.: So why do we need NFA’s?

Design a finite automaton for the language L — all binary strings witha 1 in their third-to-the-last position?


NFA for L

q4q1 q

2 q31

0,1

0,10,1

“Guesses” which symbol is third from the last, and

checks that indeed it is a 1.

If guess is premature, that branch “dies”, and no harm occurs.


DFA for L

Have 8 states, encoding the last three observed letters.

A state for each string in {0, 1}3.

Add transitions on modifying the suffix, give the new letter.

Mark as accepting, the strings 1 ∗ ∗

010


NFA – Formal Definition

Transition function δ is going to be different.

Let P(Q) denote the powerset of Q.

Definition 9 (NFA)

A non-deterministic finite automaton is a 5-tuple (Q,Σ, δ,q0,F ), where



δ : Q × Σ 7→ P(Q) is the transition function,



We will sometimes consider multiple starting states.


Example

q4q1 q

2 q30

0,1 0,1

10

N1 = (Q,Σ, δ,q1,F ) where

Q = {q1,q2,q3,q4}, Σ = {0,1},

δ is

0 1q1 {q1,q2} {q1}q2 {q3} ∅q3 ∅ {q4}q4 {q4} {q4}

q1 is the start state, and F = {q4}.


Formal model of computation

Definition 10

N = (Q,Σ, δ,q0,F ) accepts w ∈ Σ∗, if δ̂(q0,w) ∩ F 6= ∅.

Definition 11 ( δ̂)

For NFA N = (Q,Σ, δ,q0,F ), define δ̂ : Q × Σ∗ 7→ P(Q), byq′ ∈ δ̂(q,w = w1, . . . ,wn) if exists r0, . . . , rn ∈ Q, such that

r0 = q.

ri+1 ∈ δ(ri ,wi+1), for all 0 ≤ i < n.

rn = q′.

Naturally extends to the case that q0 is a subset of Q.

What is δ̂(q, ε)? {q}.


Alternative definition for δ̂

Definition 12 ( δ̂, alternative definition)

For NFA N = (Q,Σ, δ,q0,F ), define δ̂ : Q × Σ∗ 7→ P(Q), by

δ̂(q,w) =

{{q}, w = ε,⋃

r∈δ̂(q,w1,...,n−1)δ(r ,wn), n = |w | ≥ 1. .

Not hard to prove that the two definitions are equal.


Equivalence of NFA’s and DFA’s

Theorem 13For any NFA N there exists a DFA M such that L(N) = L(M).

Given an NFA N, we construct a DFA M, that accepts the samelanguage.

Make DFA emulates all possible NFA states.

As consequence of the construction, if the NFA has k states, theDFA has 2k states (an exponential blow up).


Equivalence of NFA’s and DFA’s, the DFA

Let N = (Q,Σ, δ,q0,F ).

Construction 14 ( M = (Q′,Σ, δ′,q′

0,F′))

Q′ = {[R] : R ⊆ Q}.

For [R] ∈ Q′ and a ∈ Σ, let δ′([R],a) =[⋃

r∈R δ(r ,a)].

q′

0 = [{q0}]

F ′ = {[R] ∈ Q′ : R ∩ F 6= ∅}

To prove equivalence, we need to prove that δ̂(q0,w) ∈ F iffδ̂′(q′

0,w) ∈ F ′.

Claim 15

δ̂′(q′

0,w) = [δ̂(q0,w)] for any w ∈ Σ∗.

The above claim yields the equivalence.


Proving Claim 15

The proof is by induction on the length of w .

|w | = 0, by definition.

Assume for words of length (m − 1), and let x = yσ, where y is aword of length (m − 1) and σ ∈ Σ.

Let Ry = δ̂(q0, y). By i.h., δ̂′(q′

0, y) = [Ry ]

Compute

δ̂′(q′

0, x) = δ′([Ry ], σ)

=

⋃

r∈Ry

δ(r , σ)

=

⋃

r∈δ̂(q0,y)

δ(r , σ)

= [δ̂(q0, x)]


Example: NFA ⇒ DFANon-Deterministic Automata:


Example: NFA ⇒ DFADeterministic automata - set of states:


Example: NFA ⇒ DFADeterministic automata - transitions from {q0}:


Example: NFA ⇒ DFADeterministic automata:


NFA with ε-moves

q4q1 q

2 q30

0,1 0,1

10,ε

What is the interpretation of ε transitions ?

What will happen with 101 ?


Example: NFA with ε-moves

L = {aibjck |i , j , k ≥ 0}


NFA – Formal definition with ε-moves

Transition function δ is going to be different.

Let P(Q) denote the powerset of Q.

Let Σε denote the set Σ ∪ {ε}.

Definition 16 (NFA, with ε-moves)

A non-deterministic finite automaton is a 5-tuple (Q,Σ, δ,q0,F ), where



δ : Q × Σε 7→ P(Q) is the transition function,




Example

q4q1 q

2 q30

0,1 0,1

10,ε

N1 = (Q,Σ, δ,q1,F ) where

Q = {q1,q2,q3,q4}, Σ = {0,1}

q1 is the start state, and F = {q4}.

δ is

0 1 ε

q1 {q1,q2} {q1} ∅q2 {q3} ∅ {q3}q3 ∅ {q4} ∅q4 {q4} {q4} ∅


Formal model of computation, with ε-moves

Definition 17

N = (Q,Σ, δ,q0,F ) accepts w ∈ Σ∗, if δ̂(q0,w) ∩ F 6= ∅.

Definition 18 ( δ̂, with ε-moves)

For NFA M = (Q,Σ, δ,q0,F ), define δ̂ : Q × Σ∗ 7→ P(Q) by q′ ∈ δ̂(q,w)if exist

parsing w = w1w2 · · ·wk , where each wi ∈ Σε, and

r0, . . . , rk ∈ Q,

such that

r0 = q.

ri+1 ∈ δ(ri ,wi+1), for all 0 ≤ i < n.

rn = q′.

When does N accept the empty string?


Alternative definition for δ̂

Definition 19For NFA N = (Q,Σ, δ,q0,F ), letE(q) = {q′ ∈ Q : q′ can be reached from q by 0 or more ε transitions}

E(Q′) =⋃

q∈Q′ E(q).

Q: is it always the case that q ∈ E(q)? Yes

Definition 20 ( δ̂, alternative definition)

For NFA N = (Q,Σ, δ,q0,F ), define δ̂ : Q × Σ∗ 7→ P(Q), by

δ̂(q,w) =

{E(q), w = ε,⋃

r∈δ̂(q,w1,...,n−1)E(δ(r ,wn)), n = |w | ≥ 1. .

Not hard to prove that the two definitions are equal.


Removing ε-transitions

Given NFA N = (Q,Σ, δ,q0,F ) with ε-transitions, we create anequivalent NFA N ′ = Q,Σ, δ′,q′

0,F ) with no ε-transitions.2

q′

0 = E(q0) (?)

δ′(q,a) = E(δ(q,a))

It is not hard to prove that δ̂(q0,w) =⋃

r∈q′

0δ̂′(r ,w) for any w ∈ Σ∗.

Thus, L(N) = L(N ′).

2The following is a variant of what presented for the first group (both variants arefine).


Example: Removing ε-transitions

Non-Deterministic Automata with ε-transitions

The non-Deterministic automata without ε-transitions

q′

0 = {q0, q1, q2, q3}

δ′ is

a b cq0 ∅ ∅ ∅q1 {q1, q2, q3} ∅ ∅q2 ∅ {q2, q3} ∅q3 ∅ ∅ {q3}


Part II

Closure of Regular Languages, Revisited


Regular languages, revisited

By definition, a language is regular if it is accepted by some DFA.

Corollary 21

A language is regular if and only if it is accepted by some NFA.

This is an alternative way of characterizing regular languages.

We will now use the equivalence to show that regular languages areclosed under the regular operations (union, concatenation, star).


Closure under union (alternative proof)

N1

N2


Closure under union , cont.

N1

N2

ε

ε


Closure under union cont..

Suppose

N1 = (Q1,Σ, δ1,q1,F1) accept L1, and

N2 = (Q2,Σ, δ2,q2,F2) accept L2.

Define N = (Q,Σ, δ,q0,F ):

Q = {q0} ∪ Q1 ∪ Q2 (we assume Q1 ∩ Q2 = ∅)

Σ is the same, q0 is the start state

F = F1 ∪ F2

δ′(q,a) =

δ1(q, a) q ∈ Q1

δ2(q, a) q ∈ Q2

{q1, q2} q = q0 and a = ε∅ q = q0 and a 6= ε


Closure under concatenation

N2

N1


Closure under concatenation , cont.

N2N1ε

ε

Remark: Final states are exactly those of N2.


Closure under concatenation , cont..

Suppose

N1 = (Q1,Σ, δ1,q1,F1) accept L1, and

N2 = (Q2,Σ, δ2,q2,F2) accept L2.

Define N = (Q,Σ, δ,q1,F2):

Q = Q1 ∪ Q2

q1 is the start state of N

F2 is the set of accept states of N

δ′(q,a) =

δ1(q, a) q ∈ Q1 and a 6= εδ1(q, a) q /∈ F1 and a = εδ1(q, a) ∪ {q2} q ∈ F1 and a = εδ2(q, a) q ∈ Q2


Closure under Star


Closure under star , cont.

Suppose N1 = (Q1,Σ, δ1,q1,F1) accepts L1.Define N = (Q,Σ, δ,q0,F ):

Q = {q0} ∪ Q1

q0 is the new start state.

F = {q0} ∪ F1

δ′(q,a) =

δ1(q,a) q ∈ Q1 and a 6= εδ1(q, ε) q /∈ F1 and a = εδ1(q, ε) ∪ {q0} q ∈ F1 and a = ε{q1} q = q0 and a = ε∅ q = q0 and a 6= ε


Summary

Regular languages are closed under◮ union◮ concatenation◮ star

Non-deterministic finite automata◮ are equivalent to deterministic finite automata◮ but much easier to use in some proofs and constructions.


Part III

Regular Expressions


Regular expressionsA notation for building up languages by describing them asexpressions, e.g. (0 ∪ 1)0∗.

0 and 1 are shorthand for the set (languages) {0} and {1}

so 0 ∪ 1 = {0,1}.

0∗ is shorthand for {0}∗.

Concatenation, is implicit. So 0∗10∗ stands for{w ∈ {0,1} : w havs exactly a single 1}.Just like in arithmetic, operations have precedence:

◮ star first◮ concatenation next◮ union last◮ parentheses used to change default order i.e., ab∗ 6= (ab)∗

Q.: What does (0 ∪ 1)0∗ stand for?

Remark : Regular expressions are often used in text editors or shellscripts.


Regular expressions – formal definition

Definition 22A string R is a regular expression over Σ, if R is of form

a for some a ∈ Σ

ε

∅

(R1 ∪ R2) for regular expressions R1 and R2

(R1‖R2) for regular expressions R1 and R2

(R∗

1) for regular expression R1

Parenthesis and ‖ are omitted when their role is clear from the context.

We denote the empty regular expression with ∅


Formal Definition, cont.

Definition 23The language L(R) of regular expression R, is defined byR L(R)

a {a}ε {ε}∅ ∅(R1 ∪ R2) L(R1) ∪ L(R2)(R1R2) L(R1)‖L(R2)(R1)

∗ L(R1)∗

Isn’t this definition circular?


Examples of regular expressions

For Σ = {0,1}, write regular expression for the following languages:

The third letter from the end is 1

(0 ∪ 1)∗1(0 ∪ 1)2

The number of 1’s is even

(0∗ ∪ 10∗1)∗

The number of 1’s is odd

(0∗ ∪ 10∗1)∗10∗

No three consecutive 1’s and ends in 0, or the empty word

(0∗0 ∩ 0∗10∗0 ∩ 0∗110∗0)∗


Part IV

Regular Expressions and Regular Languages


Remarkable Fact

Theorem 24A language is described by a regular expression, iff it is regular.

⇐=: Given a regular language, construct a regular expressiondescribing it.

=⇒: Given a regular expression, construct an NFA accepting itslanguage.


Given RE R, build NFA Accepting it (=⇒)

1. R = a , for some a ∈ Σ

a

2. R = ε

3. R = ∅


Given R, Build NFA Accepting It (=⇒), cont.

N1

N2

ε

εN2N1

ε

ε

R = (R1 ∪ R2) R = (R1‖R2)


Example

a

a

b

b

ab εba

ab ε

a

ε

εba U a

Formal proof by induction on the length of the regular expresssion


Regular Expression from an NFA (⇐=)

We now define generalized non-deterministic finite automata (GNFA).An NFA:

Each transition labeled with a symbol or ε,

reads zero or one symbols,

takes matching transition, if any.

A GNFA:

Each transition labeled with a regular expression,

reads zero or more symbols,

takes transition whose regular expression matches string, if any.

GNFAs are natural generalization of NFAs.


A Special Form of GNFA

We will focus on a special type of GNFA

Start state has outgoing arrows to every other state, but noincoming arrows.

Unique accept state has incoming arrows from every other state,but no outgoing arrows.

Except for start and accept states, an arrow goes from every stateto every other state, including itself.

Easy to transform any GNFA into special form. (?)


GNFA – Formal Definition

Definition 25A generalized deterministic finite automaton (GNFA) is (Q,Σ, δ,qs ,qa),where

Q is a finite set of states,

Σ is the alphabet,

δ : (Q − {qa})× (Q − {qs}) 7→ R is the transition function.

qs ∈ Q is the start state, and

qa ∈ Q is the unique accept state.


GNFA – Model of Computation

Definition 26A GNFA G = (Q,Σ, δ,qs ,qa) accepts a string w ∈ Σ∗, if there exists

parsing w = w1w2 · · ·wk , where each wi ∈ Σ∗, and

r0, . . . , rk ∈ Q,

such that

r0 = qs,

rk = qa, and

wi ∈ L(δ(ri−1, ri )), for every 0 < i ≤ k .


The Transformation: DFA → Regular Expression

Strategy – sequence of equivalent transformations

Given a k-state DFA

Transform into (k + 2)-state GNFA (how?)

While GNFA has more than 2 states, transform it into equivalentGNFA with one fewer state

eventually reach 2-state GNFA (states are just start and accept).

Label on single transition is the desired regular expression. ♠


Converting strategy (⇐=)

3-stateDFA

5-stateGNFA

4-stateGNFA

3-stateGNFA

2-stateGNFA

regularexpression


Removing One State

We remove one state qr , and then repair the machine by alteringregular expression of other transitions.

qi jq

qr

R1 3R

4R

qi jq

2R

2R*R1 3R U 4R


The StateReduce and Convert algorithms

Algorithm 27 ( StateReduce)

Input: a (k > 2)-state GNFA G = (Q,Σ, δ,qs ,qa).

Select any state qr ∈ Q \ {qs,qa}.

Let Q′ = Q \ {qr}.

For any qi ∈ Q′ \ {qa} and qj ∈ Q′ \ {qs}, let◮ R1 = δ(qi , qr ), R2 = δ(qr , qr ),◮ R3 = δ(qr , qj), and R4 = δ(qi , qj).

Define δ′(qi ,qj) = (R1)(R2)∗(R3) ∪ (R4).

Return the resulting (k − 1)-state GNFA G′ = (Q′,Σ, δ′,qs,qa).

Algorithm 28 ( Convert)

Input: a (k ≥ 2)-state GNFA G.

If k = 2, return the regular expression labeling the only arrow of G.

Otherwise, return Convert(StateReduce(G)).


Conversion - Example








Correctness proof

Claim 29G and Convert(G) accept the same language.

Proof: By induction on k – the number of states of G.

Basis. k = 2: Immediate by the definition of GFNA.

Induction step: Assume claim for (k − 1)-state GNFA, where k > 2,prove for k-state GNFA.

Let G′ = StateReduce(G) (note that G′ has k − 1 states), and let qr bethe removed state.We prove (in a very high level) that L(G) = L(G′) (i.e., G and G′

accept the same language). We show

w ∈ L(G) =⇒ w ∈ L(G′)

w ∈ L(G′) =⇒ w ∈ L(G)


w ∈ L(G) =⇒ w ∈ L(G′)

Let w ∈ L(G) and let p = qs,q1, . . . ,qℓ,qa be (a possible) “path ofstates” traversed by G on w .

If qr /∈ p, then G′ accepts w (the new regular expression on eachedge of G′ contains the old regular expression in the “union part”.)

If p = qS, . . . ,qi ,qr ,qj , . . . ,qa, the regular expression(Ri ,r )(Rr ,r )

∗(Rr ,j) linking qi and qj in G′, causes G′ to accept w .

Hence, w ∈ L(G′).


w ∈ L(G′) =⇒ w ∈ L(G)

Let w ∈ Σ∗ and let p = q0, . . . ,qn be be (a possible) “path of states”traversed by G′ on w .

There exists parsing w = w1, . . . ,wn such that wi ∈ L(R′

i ) forR′

i = δG′(qi−1,qi).

For qi ,qj ∈ Q, let Ri ,j = δG(qi ,qj).

Hence, δG′(qi−1,qi) =(Ri−1,rR∗

r ,rRr ,i)∪ Ri−1,i .

If wi ∈ L(Ri−1,i), then wi ∈ δG(qi−1,qi).If wi ∈ L(Ri−1,rR∗

r ,rRr ,i), then wi = u1, . . . uℓ such that◮ u1 ∈ L(Ri−1,r )◮ uℓ ∈ L(Rr ,i)◮ uj ∈ L(Rr ,r ) for 2 ≤ j ≤ ℓ− 1.

In both cases, wi corresponds to possible traverse from qi−1 to qi in G.

Hence, w ∈ L(G′) =⇒ w ∈ L(G).


Summing it up

We proved L(G) = L(G′).

Hence, G and (the regular expression) Convert(G) accept thesame language.

Thus, we proved: A language is described by a regular expressioniff it is regular.


Summary

Non-Deterministic Automata (with ε-moves)◮ Equivalence to DFA

Closure properties◮ union◮ concatenation◮ star

Regular expressions.◮ Equivalence to DFA


Documents

Computational Models - Lecture 21tau-cm2014.wdfiles.com/local--files/course-schedule/BLecture2_h.pdf · Computational Models - Lecture 2 Non-Deterministic Finite Automata (NFA) Closure