View
10
Download
0
Category
Preview:
Citation preview
Automata & Complexity
Jörg Endrullis
Vrije Universiteit Amsterdam
Information
Lecturer:Jörg Endrullis
room: NU-12A-65
email: j.endrullis@vu.nl
Teaching assistant:Geoffrey Frankhuizen
Johannes van der Meer
Alexander van Oostveen
Course information:all relevant course information is on Canvas
link to the slides on Canvas
Course Structure
2 lectures per week
2 exercise classes per week
Homework:80% of the homework in an online system
20% of the homework written assignments(one written assignment at the end of the course)
70% of homework points to qualify for the exam
90% of homework points for 0.5 points bonus
Written exam
Material
Peter LinzAn Introduction to Formal Languages and Automata (5th edition)
Jones and Bartlett
Course Motivation
Computer is based on a universal computation mechanism.
Different applications =⇒ different formalisms:
Pattern recognition: regular languagesregular grammars, regular expressions, finite automata
Parsing: context-free languagescontext-free grammars, pushdown automata
Computation: recursively enumerable languagesTuring machines
Course Motivation: Automata are Ubiquitous
q0 q1
0
1
0
1
Theory of computation
Pattern matching
Parsing
Hardware design
Software modelling
Compiler construction
Formal verification
Number theory
Course Motivation: What can a Computer do?
What can a computer do?
Some (at first glance simple) problems are undecidable.
For example:program termination,Post correspondence problem,validity in predicate logic.
Some problems (NP-complete problems)are (probably) not efficiently solvable bya computer.
For example:travelling salesman problem,satisfiability in propositional logic.
Course Motivation
Typical questions that we will answer
What is a (programming) language?
How can languages be recognised by computers?(automata, regular expressions, grammars)
What problems can be solved by what types of automata?
How much time/memory is needed for solving a problem?
Aspects of languages:syntax: the form of the words in the language
semantics: the meaning of the words in the language
We will focus on the syntax.
Words
WordsWord = finite sequence of symbols from a finite alphabet Σ.
notation for symbols: a,b, c, . . .
symbol from the alphabet: a ∈ Σ
notation for words: u, v ,w , x , y , z
empty word: λ
Note that, in programming, words are often called strings.
A computer program:is itself a word (a finite sequence of bits/bytes)
takes input word
produces output word
Operations on Words (1)
ConcatenationIf v = a1 · · · an and w = b1 · · · bm, then
vw = a1 · · · anb1 · · · bm
LengthIf v = a1 · · · an, then |v | = n.
The length can be defined inductively:
|λ| = 0|va| = |v |+ 1
Operations on Words (2)
PowerThe power vk consists of k concatenations of v ’s:
v0 = λ
vk+1 = vkv
ReverseThe reverse of a1 · · · an is
(a1 · · · an)R = an · · · a1
The reverse can be inductively defined
λR = λ
(va)R = a(vR)
Formal Languages
Formal language = set of words
ExampleAll parseable C programs form a language.
More precisely:Σ∗ is the set of all words over Σ.
Formal languageA (formal) language L is a subset of Σ∗, that is, L ⊆ Σ∗.
Examples of Formal Languages
Let Σ = {a,b}.
{ab, aab, bbaaabb } is a finite language
{abna | n ≥ 1 } is an infinite language:
{aba, abba, abbba, abbbba, . . . }
{anbn | n ≥ 0 } is an infinite language:
{ λ, ab, aabb, aaabbb, aaaabbbb, . . . }
Operations on Languages (1)
A language is a set of words.So the usual set operations have meaning for languages:
∈, ⊆, ∩, ∪, \, . . .
Complement
The complement L = all words that are not in the language L:
L = Σ∗ \ L
ConcatenationThe concatenation of languages L1 and L2 is defined as
L1L2 = { xy | x ∈ L1 ∧ y ∈ L2 }
ReverseThe reverse of a language L is
LR = { xR | x ∈ L }
Operations on Languages (2)
PowerThe n-th power of a language L is defined by induction on n:
L0 = { λ }
Ln+1 = LnL (n ≥ 0)
Attention: L2 = {uv | u, v ∈ L} 6= {uu | u ∈ L}
Kleene star
L∗ =
∞⋃i=0
Li = L0 ∪ L1 ∪ L2 ∪ L3 ∪ · · ·
L+ =
∞⋃i=1
Li = L1 ∪ L2 ∪ L3 ∪ · · ·
Thus L∗ = L+ ∪ {λ}.
Exercise
LetΣ = {a,b }
L1 = {a, bb }
L2 = {ab, ba }
(In groups of two, 2 minutes)
Describe the following languages as sets:L1L2 = {aab, aba, bbab, bbba }
L21 = {aa, abb, bba, bbbb }
L31 = answered in the lecture
Exercise
LetΣ = {a,b }
L = {abn | n ≥ 0 }
(In groups of two, 3 minutes)
Describe the following languages as sets:LR = {bna | n ≥ 0 }
L = { λ } ∪ {bw | w ∈ Σ∗ } ∪ {awau | w ,u ∈ Σ∗ }
LR= answered in the lecture
LR = answered in the lecture
ConclusionSets are not ideal to describe languages.
Deterministic Finite Automata
A deterministic finite automaton, short DFA, consists of:a finite set Q of statesa finite input alphabet Σa transition function δ : Q × Σ→ Qa starting state q0 ∈ Qa set F ⊆ Q of final states
Understanding the transition function δ : Q × Σ→ QIf the automaton in state q reads the symbol a, then theresulting state is δ(q,a).
Example DFALet M = (Q, Σ, δ, q0,F ) with Q = {q0,q1}, Σ = {a,b}, F = {q0},
δ(q0,a) = q0 δ(q1,a) = q1
δ(q0,b) = q1 δ(q1,b) = q0
DFA’s Reading Words
Let M = (Q, Σ, δ, q0,F ) be a DFA. We write
(q,aw) ` (q ′,w) if δ(q,a) = q ′
Here, (q,w) indicates that M is in state q and reads word w .
Let M = (Q, Σ, δ, q0,F ) with Q = {q0,q1}, Σ = {a,b}, F = {q0},
δ(q0,a) = q0 δ(q1,a) = q1
δ(q0,b) = q1 δ(q1,b) = q0
Then we have
(q0,abba) ` (q0,bba) ` (q1,ba) ` (q0,a) ` (q0, λ)
We define `∗ as the reflexive transitive closure of `.
Continuing the above example, we have
(q0,abba) `∗ (q0, λ)
Transition Function in Table Notation
Let M = (Q, Σ, δ, q0,F ) with Q = {q0,q1}, Σ = {a,b}, F = {q0},
δ(q0,a) = q0 δ(q1,a) = q1
δ(q0,b) = q1 δ(q1,b) = q0
Hint: transition function δ can be written in the form of a table:
δ q0 q1a q0 q1b q1 q0
DFA’s as Transition Graphs
A DFA can be visualised as a transition graph, consisting of:states are the nodes of the graph
arrows with labels from Σ
starting state indicated by an extra incoming arrow
final states indicated by double circleWe have an arrow from q to q ′ with label a if δ(q,a) = q ′.
Σ = {a,b }
q0 q2 q4
q1
q3
a
b
a bab
a
b
a
b
Exercise (1)
We define a DFA M = (Q, Σ, δ, q0,F ) with
Q = {q0,q1} Σ = {a,b} F = {q0}
and the transition function given by
δ q0 q1a q0 q1b q1 q0
(In groups of two, 1 minute)
Visualise M as a transition graph.
q0 q1b
b
a a
Exercise (2)
z0 z2
z1
a
ba,ba
b
Note that the transition with label a,bis shorthand for two transitions: onewith label a and one with label b.
(In groups of two, 1 minute)
Translate the graph to a deterministic finite automaton:states Q = ?alphabet Σ = ?transition function δ : Q × Σ→ Q is given by
δ z0 z1 z2ab
starting state ?final states F = ?
DFA’s Reading Words continued...
If (q,w) `∗ (q ′, λ), we also write
q w−→ q ′
In other words,
q w−→ q ′
means that there is a path from q to q ′ in the automaton withtransitions labelled a1, . . . ,an such that w = a1 · · · an.
Regular Languages
A DFA defines (accepts) a language!
The language accepted by DFA M = (Q, Σ, δ, q0,F ) is
L(M) = {w ∈ Σ∗ | (q0,w) `∗ (q, λ) with q ∈ F }
= {w ∈ Σ∗ | q0w−→ q with q ∈ F }
Let M = (Q, Σ, δ, q0,F ) with Q = {q0,q1}, Σ = {a,b}, F = {q0},
δ(q0,a) = q0 δ(q1,a) = q1
δ(q0,b) = q1 δ(q1,b) = q0
Then we have
(q0,abba) ` (q0,bba) ` (q1,ba) ` (q0,a) ` (q0, λ)
The word abba is accepted by M, that is, abba ∈ L(M).
A language L is regular if there exists a DFA M with L(M) = L.
Exercise
Let M be the following DFA:
q0 q1b
b
a a
(In groups of two, 2 minutes)
What is the language accepted by M? (describe in words)
L(M) consists of all words that contain an even number of b’s.
Determinism
DFAs are deterministicFor every state q ∈ Q and every symbol a ∈ Σ, the state q hasprecisely one outgoing arrow with label a.
Recall that δ is a function from Q × Σ to Q.
Hence, for every input word, there is precisely one path fromthe starting state through the transition graph.
The following picture shows the path for aaba:
q0 q2 q4
q1
q3
a
b
a bab
a
b
a
b
Exercise (1)
(Individually, 10 seconds)
Show that the following language is regular.
Construct a deterministic finite automaton for the language:
{ λ }
Exercise (2)
(Individually, 1 minutes)
Show that the following language is regular.
Construct a deterministic finite automaton for the language:
{anb | n ≥ 0 }
Exercise (3)
(Individually, 2 minutes)
Show that the following language is regular.
Construct a deterministic finite automaton for the language:
{a2n+1 | n ≥ 0 } ∪ {b2n | n ≥ 0 }
Exercise (4)
(Individually, 4 minutes)
Show that the following languages are regular.
Construct deterministic finite automata for the languages:
{w ∈ {a,b}∗ | w contains the subword bab }
and
{w ∈ {a,b}∗ | w does not contain the subword bab }
Theorems about Regular Languages (1)
TheoremIf L is a regular language, then L is also regular.
Proof.Let L be regular.
Then there exists a DFA M = (Q, Σ, δ, q0,F ) with L(M) = L.
Then N = (Q, Σ, δ, q0,Q \ F ) is a DFA with L(N) = L.
Here it is important that for every input word w :There is precisely one path starting at q0 labelled with w .
There is precisely one state q with q0w−→ q, and
w ∈ L ⇐⇒ w ∈ L(M) ⇐⇒ q ∈ F
w ∈ L ⇐⇒ w ∈ L(M) ⇐⇒ q ∈ (Q \ F ) ⇐⇒ w ∈ L(N)
Theorems about Regular Languages (2)
TheoremIf L1 and L2 are regular, then L1 ∪ L2 is regular.
Construction (Product)There exists a DFAs
M1 = (Q1, Σ, δ1,q0,1,F1) M2 = (Q2, Σ, δ2,q0,2,F2)
such that L(M1) = L1 and L(M2) = L2.
Idea: We run M1 and M2 in parallel.
We define a DFA N = (Q, Σ, δ, q0,F ) whereQ = Q1 ×Q2 = { (q1,q2) | q1 ∈ Q1, q2 ∈ Q2 }
δ((q1,q2),a) = (δ1(q1,a), δ2(q2,a))q0 = (q0,1, q0,2)
F = {(q1,q2) ∈ Q | q1 ∈ F1 or q2 ∈ F2}
Then it follows that L(N) = L(M1) ∪ L(M2) = L1 ∪ L2.
Theorems about Regular Languages (3)
QuestionLet L1 and L2 be regular languages.
How to change the product construction to show thatL1 ∩ L2 is regular, and
L1 \ L2 is regular ?
TheoremIf L is regular, then LR is also regular.
This proof will be an exercise.
Exercise
Is the following language regular?
{anbn | n ≥ 0 }
This language is not regular!
Intuition: a DFA has only a finite memory (the states).
(Groups of two, 2 minutes)
Give a deterministic automaton with infinitely many states for
{anbn | n ≥ 0 }
Answer in the lecture.
QuestionIs every finite language regular ?
Nondeterministic Finite Automata
Nondeterministic finite automaton (NFA)
A state can have zero or more outgoing arrows with thesame label.
Allows for empty steps: arrows with label λ that do not‘eat’ a symbol from the input word.
DFA’s and NFA’s are used everywhere:in software engineering,
for modelling of hardware circuits,
in compilers, and
in network protocols.
Nondeterministic Finite Automata
NFA’s are defined like DFA’s, except for the transition function!
A nondeterministic finite automaton, short NFA, consists of:a finite set Q of states
a finite input alphabet Σ
a transition function δ : Q × (Σ ∪ {λ}) → 2Q
a starting state q0 ∈ Q
a set F ⊆ Q of final states
Here 2Q is the set of all subsets of Q:
2Q = {S | S ⊆ Q }
NFA’s Reading Words
Let M = (Q, Σ, δ, q0,F ) be a NFA. We write
(q, αw) ` (q ′,w) if q ′ ∈ δ(q, α) with α ∈ Σ ∪ {λ}
Note that if α = λ, thenthe state changes (q to q ′), but
the input word stays the same (λw = w).
Again we write
q w−→ q ′
whenever (q,w) `∗ (q ′, λ).
It means that there is a path from q to q ′ with transitionslabelled α1, . . . , αn ∈ (Σ ∪ {λ}) such that w = α1 · · ·αn.
NFA’s Reading Words
The language accepted by NFA M = (Q, Σ, δ, q0,F ) is
L(M) = {w ∈ Σ∗ | (q0,w) `∗ (q, λ) with q ∈ F }
= {w ∈ Σ∗ | q0w−→ q with q ∈ F }
But now paths are not unique!
q0 q1a
λ
b
b
This automaton can accept the word ab in 2 different ways!
(q0,ab) ` (q1,b) ` (q1, λ)
(q0,ab) ` (q1,b) ` (q0, λ) ` (q1, λ)
DFA’s and NFA’s are Equally Expressive
TheoremA language L is accepted by a NFA ⇐⇒ L is regular.
Construction (Powerset)Let M = (Q, Σ, δ, q0,F ) be a NFA.Idea: state of DFA = set of all states the NFA can be inWe construct a DFA ND = (QD, Σ, δD,q0D,FD) where
QD = 2Q
δD(X ,a) = {q ′ ∈ Q | (q,a) `∗ (q ′, λ) for some q ∈ X }
q0D = {q ′ ∈ Q | (q0, λ) `∗ (q ′, λ) }FD = {X ⊆ Q | X ∩ F 6= ∅ }
For every w ∈ Σ∗ and q ∈ Q it holds that
(q0,w) `∗ (q, λ) ⇐⇒ (q0D,w) `∗ (X , λ) with q ∈ X .
From this property it follows that L(ND) = L(M).
Exercise
(Groups of two, 2 minutes)
Given is the following NFA:
q0 q1a
λ
b
b
Construct a DFA that accepts the same language.
Looking Forward
Read:Linz 1.2–1.2, 2.1–2.3
Do the following exercises:Linz 1.2: 2, 4, 8, 10
Linz 2.1: 1, 2d, 3, 7b, 9b,f, 11
Linz 2.2: 12
Linz 2.3: 2, 3, 6, 12
Following lecture:Alternative descriptions of regular languagse:
regular expressions
grammars
Recommended