COMP3630/COMP6363

Theory of Computation

Prerequisites: COMP1140 and COMP 1600

Course assumes one knows:

(Foundations of Computing)

- Sets, functions, relations

- Mathematical induction

Textbook: Introduction to Automata Theory, Languages and ComputationJohn E. Hopcroft, Rajeev Motwani, and Jeffrey D. Ullman [HMU].

- Any other background material related to Chapter 1 of HMU.

1

The First Half of the Course...

Covered by Badri Vellambibadri.vellambi@anu.edu.au

- Automata and Regular Languages [1.5 weeks]

- Pushdown Automata and Context-free Languages [1.5 weeks]

- Decidability, Undecidability, and Intractable Problems [1.5 weeks]

- Turing Machines and Recursively Enumerable Languages [1.5 weeks]

Models of Computation and Languages:

Computational Problems:

2

• Nondeterministic Finite Automaton

• NFA with ›-transitions

Reading (from HMU): All of Chapter 2.

• Deterministic Finite Automaton

• An Equivalence among the above three.

This Lecture Covers Finite Automata

(Chapter 2 of HMU)

3

Preliminary Concepts

E.g., ⌃ = {0; 1} (binary alphabet)⌃ = {a; b; : : : ; z} (Roman alphabet)

• String (or word) is a finite sequence of symbols

• Alphabet ⌃: A finite set of symbols

-Usually represented without commas, e.g., 0011 instead of (0; 0; 1; 1)

• Concatenation of strings

› is the identity element for concatenation, i.e., ›x = x› = x .

• A (formal) language is a subset of ⌃⇤.

• Kleene ⇤ or closure operator: ⌃⇤ = {›} [ ⌃ [ ⌃2 [ ⌃3 · · · denotes the set of all strings.

Concatenation of sets of strings: AB = {ab : a 2 A; b 2 B}

x and y is the string xy = x followed by y

Concatenation of the same set: A2 = AA; A3 = (AA)A, etc

4

Deterministic Finite AutomatonInformally:

s1 s2 s3 s‘: : : [Input tape]

Finite Control q0q1

q3

q4

q5

q2

[Movable Reading Head]

• The input is read from left to right

• Each read operation changes the internal state of the FSM

• Input is accepted/rejected based on the final state after reading all symbols

• The device consisting of: (a) input tape; (b) reading head; and (c) finite control(Finite-state machine)

5

Deterministic Finite Automaton (DFA)

• A DFA A = (Q;⌃; ‹; q0; F )

Q: A finite set (of internal states)

⌃: The alphabet corresponding to the input

‹ : Q⇥ ⌃ ! Q

q0: The (unique) starting state of the DFA (prior to any reading). (q0 2 Q)

F ⇢ Q is the set of final (or accepting) states

Transition Table: Transition Diagram:

q0

q1

q2

0 1

q2

q2

q0

q1 q1

q1

⇤

F = {q1}‹(q0; 0) = q2

‹(q0; 1) = q0

q0 q1q2

1

10

0; 10

[If present state is q 2 Q, and a 2 ⌃ is read, the DFA moves to ‹(q; a).](Transition Function)

Remark: Each state has exactly one outgoing edge labelled by a symbol

6

Language Accepted by a DFA

• The language L(A) accepted by a DFA A = (Q;⌃; ‹; q0; F ) is:

The set of all input strings that move the state of the DFA from q0 to a state in F

• This is formalized via the extended transition function ‹̂ : Q⇥ ⌃⇤ ! Q:

‹̂(q; ›) = q [No state change]

‹̂(q; s) = ‹(q; s) s 2 ⌃

s1 2 ⌃; w 2 ⌃⇤if ‹̂(q; w) = p, then ‹̂(q; ws) = ‹(p; s).

Basis: 1)

2)

Induction: 3)

• L(A) := all strings that take q0 to some final state

= {w 2 ⌃⇤ : ‹̂(q0; w) 2 F}:

w = s1s2 · · · sk 2 L(A) , q0 �! P1 �! P2 �! · · · �! Pk 2 Fs1 s2 sk

› 2 L(A) , q0 2 F

For k > 0,

In other words,

(a)

(b)

7

An Example

q0 q1q2

1

10

0; 10

• The only way one can reach q1 from q0 is if the string contains 01.

• L(A) is the set of strings containing 01.

A:

Is 00 accepted by A?

- Need to determine ‹̂(q0; 00)

q0 �! q2 �! q2 =2 F0 0

Is 001 accepted by A?

Thus, 00 is not accepted by A

q0 �! q2 �! q2 �! q1 2 F0 0 1

Thus, 001 is accepted by A.

Remark 1: In general, each string corresponds to a unique path of states.

The converse isn’t true. For example, 0010 and 0011 have the same sequence ofstates.

Remark 2:

8

Limitations of DFAs

• DFAs have a finite number of states (and hence finite memory).

• Can generalize DFAs in one of many ways:

- Allow transitions to multiple states at each symbol reading.

- Allow transitions without reading any symbol

- Allow the device to have an additional tape to store symbols

- Allow the device to edit the input tape

- Allow bidirectional head movement

• Can all languages be accepted by DFAs?

e.g., L = {0n1n : n 2 N} cannot be accepted by any DFA.

• Given a DFA, there is always a long pattern it cannot ’remember’ or ’track’

9

- Allow transitions to multiple states at each symbol reading.

Non-deterministic Finite Automaton (NFA)

- Multiple transitions allows the device to:

(a) clone itself, traverse through and consider all possible parallel outcomes.

(b) hypothesize/guess multiple eventualities concerning its input.

- Seems bizarre, but aids the implication of describing the automaton.

• Formal Definition: A = (Q;⌃; ‹; q0; F )

‹ : Q⇥ ⌃ ! 2Q

If ‹(·; ·) is a singleton for all argument pairs, then NFA is a DFA.

‹(q; s) can be a set with two or more states, or even be empty!

[Transition Function]

[So every DFA is an NFA, by definition!]

Remark 3:

Remark 4:

10

Language Accepted by an NFA• This is formalized via the extended transition function ‹̂ : Q⇥ ⌃⇤ ! 2Q:

‹̂(q; ›) = {q} [No state change]

‹̂(q; s) = ‹(q; s) s 2 ⌃

s1 2 ⌃; w 2 ⌃⇤

• L(A) := {w 2 ⌃⇤ : ‹̂(q0; w) \ F 6= ;}.

‹̂(q; ws) =

⇢[ki=1‹(pi ; s); ‹̂(q; w) = {p1; : : : ; pk}

; ‹̂(q; w) = ;

Basis: 1)

2)

Induction: 3)

q

‹̂(q; w)p1

p2...

pk

...

...

...

‹(p1; s)

‹(pk ; s)

w

s

s

‹̂(q; ws)

w = s1s2 · · · sk 2 L(A) , q0 �! P1 �! P2 �! · · · �! Pk 2 Fs1 s2 sk

› 2 L(A) , q0 2 F

For k > 0,

In other words,

(a)(b)

11

An Example

q0

q1

q2

0

q2

⇤q0 q1 q2

1

0; 1

0; 1

1

q0 {q0; q1}q2

; ;

‹̂(q0; 01) = {q0; q1}

q00�! q0

0�! q0

q00�! q0

1�! q1 q00�! q0

1�! q0

‹̂(q0; 10) = {q0; q2} q01�! q0

0�! q0 q01�! q1

0�! q2

‹̂(q0; 100) = {q0} q01�! q1

0�! q00�! q0

• An input can move the state from q0 to q2 only if it ends in 10 or 11.

• L(A) = {w : penultimate symbol in w is a 1}.

(a) the 1 is the penultimate symbol.

(b) the 1 is not the penultimate symbol.

‹̂(q0; 00) = {q0}

[These paths die if the 1 is not actually the penultimate symbol]

• Each time the NFA reads a 1 (in state q0) it considers two parallel possibilities:

12

Is Non-determinism Better?Non-determinism was introduced to increase the computational power.

So is there a language L that is accepted by an NDA, but not by any DFA?

Every Language L that is accepted by an NFA is also accepted by some DFA.Theorem 1:

13

Proof of Theorem 11) Let N = (QN ;⌃; ‹N ; q0; FN) generate the given language L

3) Hence,

QD = 2QN qD;0 = {q0} FD = {S ✓ QN : S \ FN 6= ;}

q0 q1 q21

0; 1

0; 1 ;

{q0}

{q1}

{q2}

{q1; q2}

{q0; q1}

{q0; q2}

{q0; q1; q2}

D :

N :

Example:

2) Define DFA D = (QD;⌃; ‹D; qD;0; FD) from N using the following subset construction:

› 2 L(N) , q0 2 F

, {q0} 2 FD , › 2 L(D)

Idea: Devise a DFA D such that at any time instant the state of the DFA is the setof all states that NFA N can be in.

14

Proof of Theorem 14) To define ‹D(P; s) for each P ✓ Q and s 2 ⌃:

-Assume NFA N is simultaneously in all states of P

-Set ‹D(P; s) := P 0 =S

p2P ‹N(p; s).

P P 0

s

N: D :

P P 0�!s,

-Let P 0 be the states to which N can transition from states in P upon reading s

5) Induction step:

Basis: Let s 2 ⌃

‹̂N(q0; s)def= ‹N(q0; s) =

[

p2{q0}

‹N(p; s)def= ‹D({q0}; s)

def= ‹̂D({q0}; s)

Induction: Let ‹̂N(q0; w) = ‹̂D({q0}; w) for w 2 ⌃⇤ \ {›}

‹̂N(q0; ws)def=

[

p2‹̂N(q0;w)

‹N(q0; s)ind=

[

p2‹̂D({q0};w)

‹N(q0; s)def= ‹D(‹̂D({q0}; w); s)

def= ‹̂D({q0}; ws)

6) Thus, ‹̂N(q0; ·) = ‹̂D({q0}; ·), and hence the languages have to be identical. [QED]15

q0 q1 q21

0; 1

0; 1

;

{q0}

{q1}

{q2}

{q1; q2}

{q0; q1}

{q0; q2}

{q0; q1; q2}

0; 1

0

1

0; 1

0; 1

0; 1

10

1

0 0

1

D :

Comments about Subset construction

• Generally, the DFA constructed using subset construction has 2n states

(n = number of states in the NFA)

• Not all states are reachable! (see example below)

• The state corresponding to the empty set is never a final state.

16

›-Transitions

• State transitions occur without reading any symbols.

• An ›-Nondeterministic Finite Automaton is a 5-tuple (Q;⌃; ‹; q0; F ), where:

Q;⌃; q0; and F are as in an NFA

‹ : Q⇥ (⌃ [ {›}) ! 2Q

q0

q1 q2 q3

q4 q5 q6

›

› ›

›

›a

b

Example:

Without reading any input symbols, the state of the ›-NFA can transition

- From q0 to q1, q4, q2, or q3. - From q1 to q2, or q3.

- From q5 to q6.- From q2 to q3.

⇤

q0

q1

q2

› a b

q3

q4

q5

q6

{q2}

{q3}

{q5}

{q6}

;

{q1; q4}

;;

;

;;

{q3}

;; ;; ;;

; ;;

17

Language accepted by an ›-NFA• ›-closure of a state

ECLOSE(q) = all states that are reachable from q by ›-transitions alone.

q0

q1 q2 q3

q4 q5 q6

›

›

›a

b

ECLOSE(q0) = {q0; q1; q4; q2; q3}ECLOSE(q1) = {q1; q2; q3}ECLOSE(q2) = {q2; q3}ECLOSE(q3) = {q3}ECLOSE(q4) = {q4}ECLOSE(q5) = {q5; q6}ECLOSE(q6) = {q6}

› ›

18

Language accepted by an ›-NFA

• extended transition function ‹̂ : Q⇥ ⌃⇤ ! 2Q by

‹̂(q; ›) = ECLOSE(q)

› ›q

›: : :

› ›q

›: : :

s › ›: : :

• w 2 L(N) if and only if ‹̂(q0;w) \ F 6= ;

Given ›-NFA N = (Q;⌃; ‹; q0; F )

›

‹̂(q; s) =[

p2ECLOSE(q)

0

@[

p02‹(p;s)

ECLOSE(p0)

1

A

‹̂(q; ws) =[

p2‹̂(q;w)

0

@[

p02‹(p;s)

ECLOSE(p0)

1

A

Basis: 1)

2)

Induction: 3)

[s = › · · · ›| {z }finitely many

s › · · · ›| {z }finitely many

]

› = ›2 = ›3 = · · ·q1 q0

q1 q0 p0 p1 p

q

w

‹̂(q; w)

s›

‹̂(q; ws)

Language accepted by an ›-NFA• w 2 L(N) if and only if ‹̂(q0;w) \ F 6= ;

w = s1s2 · · · sk 2 L(A) ,

For k > 0,

In other words,

(a)

(b)

› 2 L(N) , ECLOSE(q0) \ F 6= ;› › ›

: : :

› ›q0

›: : :

›

p1

p2

s1

s2

sk

p1

...

: : :

: : :

› › ›

› › ›

: : :›

›

› ›

pkpk�1

pk

q0 p1 pr 2 F

qF 2 F

20

Every Language L that is accepted by an ›-NFA is also accepted by some DFA.Theorem 2:

Are ›-NFAs Better?

Proof: Given L that is accepted by some ›-NFA, we must find an NFA that accepts L.

[NFA to DFA conversion can be done as in Theorem 1].

Let ›-NFA N = (QN ;⌃; ‹N ; q0; FN) accept L.

Let us devise NFA N 0 = (QN0 ;⌃; ‹N0 ; q00; FN0) as follows:

‹N0 : QN0 ⇥ ⌃ ! 2QN0 defined by:

› ›q

›: : :

spN : p0

QN0 = QN : q00 = q0 F 0N = {q 2 QN : ECLOSE(q) \ FN 6= ;}

‹N0(q; s) =[

p2ECLOSE(q)

‹(p; s)

N 0 : p0qs

m

N :

N 0: q can transition to p0 after reading s.

q can transition to p0 after a few ›-transitions, and a single read of s 2 ⌃.

21

Are ›-NFAs Better?

(Proof continued)

N :

m

p1

p2

s2

sk

pk

,

qs1

...

ECLOSE(pk) \ FN 6= ;

N 0 :

ms1 : : : sk is accepted by ›-NFA N s1 : : : sk is accepted by NFA N 0

› ›q0

›: : :

›

p1

p2

s1

s2

sk

p1

...

: : :

: : :

› › ›

› › ›

: : :›

›

› ›

pkpk�1

pk qF 2 F

[QED]

[Handwavy, but can be formalized!]

22

23

Languages acceptedby DFAs

Languages acceptedby NFAs

Languages acceptedby ›-NFAs= =

• Nondeterminism and ›-transitions do not offer computational benefits

To Summarize...