Theory of Languages and - Sharifce.sharif.edu/courses/93-94/2/ce415-1/resources/root...Theory of Languages and Automata Prof. Movaghar 2 Grading Policy O Assignments 30% O First Quiz

Theory of Languages

and Automata

Chapter 0 - Introduction

Sharif University of Technology

References O Main Reference

M. Sipser, ”Introduction to the Theory of Computation,” 3nd Ed.,

Cengage Learning , 2013.

O Additional References

P. Linz, “An Introduction to Formal Languages and Automata,” 3rd

Ed., Jones and Barlett Publishers, Inc., 2001.

J.E. Hopcroft, R. Motwani and J.D. Ullman, “Introduction to

Automata Theory, Languages, and Computation,” 2nd Ed.,

Addison-Wesley, 2001.

P.J. Denning, J.B. Dennnis, and J.E. Qualitz, “Machines,

Languages, and Computation,” Prentice-Hall, Inc., 1978.

P.J. Cameron, “Sets, Logic and Categories,” Springer-Verlag,

London limited, 1998.

Prof. Movaghar Theory of Languages and Automata 2

Grading Policy

O Assignments 30%

O First Quiz 15%

O Second Quiz 15%

O Final Exam 40%


Main Topics

O Complexity Theory

O Computability Theory

O Automata Theory

O Mathematical Notions

O Alphabet

O Strings

O Languages


Complexity Theory

O The scheme for classifying problems according to their computational difficulty

O Options:

Alter the problem to a more easily solvable one by understanding the root of difficulty

Approximate the perfect solution

Use a procedure that occasionally is slow but usually runs quickly

Consider alternatives, such as randomized computation

O Applied area:

Cryptography


Computability Theory

O Determining whether a problem is solvable by

Computers

O Classification of problems as solvable ones and

unsolvable ones


Automata Theory

O Definitions and properties of mathematical models

of computation

Finite automaton

Usage: Text Processing, Compilers, Hardware Design

Context-free grammar

Usage: Programming Languages, Artificial Intelligence


Logic

O The science of the formal principles of

reasoning


History of Logic

O Logic was known as 'dialectic' or 'analytic' in Ancient Greece. The word 'logic' (from the Greek logos, meaning discourse or sentence) does not appear in the modern sense until the commentaries of Alexander of Aphrodisias, writing in the third century A.D.

O While many cultures have employed intricate systems of reasoning, and logical methods are evident in all human thought, an explicit analysis of the principles of reasoning was developed only in three traditions: those of China, India, and Greece.


History of Logic (cont.)

O Although exact dates are uncertain, particularly in the case of

India, it is possible that logic emerged in all three societies by

the 4th century BC.

O The formally sophisticated treatment of modern logic

descends from the Greek tradition, particularly Aristotelian

logic, which was further developed by Islamic Logicians and

then medieval European logicians.

O The work of Frege in the 19th century marked a radical

departure from the Aristotlian leading to the rapid

development of symbolic logic, later called mathematical

logic.


Modern Logic

O Descartes proposed using algebra, especially techniques for solving for unknown quantities in equations, as a vehicle for scientific exploration.

O The idea of a calculus of reasoning was also developed by Leibniz. He was the first to formulate the notion of a broadly applicable system of mathematical logic.

O Frege in his 1879 work extended formal logic beyond propositional logic to include quantification to represent the "all", "some" propositions of Aristotelian logic.


Modern Logic (cont.)

O A logic is a language of formulas.

O A formula is a finite sequence of symbols with a

syntax and semantics.

O A logic can have a formal system.

O A formal system consists of a set of axioms and

rules of inference.


Logic Types of Interest

O Propositional Logic

O Predicate Logic


Syntax of Propositional Logic

O Let {p0, p1, ..} be a countable set of

propositional variables,

O {, , ¬, →, ↔} be a finite set of

connectives,

O Also, there are left and right brackets,

O A propositional variable is a formula,

O If φ and ψ are formulas, then so are (¬φ), (φ ψ),

(φ ψ), (φ → ψ), and (φ ↔ ψ).


Semantics of Propositional Logic

O Any formula, which involves the propositional

variables p0,...,pn , can be used to define a function

of n variables, that is, a function from the set {T, F}n

to {T, F}. This function is often represented as the

truth table of the formula and is defined to be the

semantics of that formula.


Examples

ψ φ (¬φ) (φ ψ) (φ ψ) (φ → ψ) (φ ↔ ψ)

T T F T T T T

F T F F T F F

T F T F T T F

F F T F F T T


Other Definitions

O A formulae is a tautology if it is always true.

(P (¬P)) is a tautology.

O A formulae is a contradiction if it is never true.

(P (¬P)) is a contradiction.

O A formulae is a contingency if it is sometimes true.

(P → (¬P)) is a contingency.


Formal System

A formal system includes the following:

O An alphabet A, a set of symbols.

O A set of formulae, each of which is a string of

symbols from A.

O A set of axioms, each axiom being a formula.

O A set of rules of inference, each of which takes as

„input‟ a finite sequence of formulae and produces as

output a formula.


Proof

O A proof in a formal system is just a finite

sequence of formulae such that each formula

in the sequence either is an axiom or is

obtained from earlier formulae by applying a

rule of inference.


Theorem

O A theorem of the formal system is just the

last formula in a proof.

O Example:

For any formula φ, the formula (φ→φ) is a

theorem of the propositional logic.


A Formal System for Propositional

Logic

O There are three „schemes‟ of axioms, namely:

(A1) (φ→(ψ → φ))

(A2) (φ→(ψ → θ)) → ((φ→ ψ) → (φ → θ))

(A3) (((¬φ) →(¬ψ)) → (ψ → φ))

O Each of these formulas is an axiom, for all choices

of formulae φ, ψ, θ.

O There is only one rule of inference, namely Modus

Ponens: From φ and (φ→ ψ), infer ψ.


Example of a Proof

O Using (A2), taking φ, ψ, θ to be φ, (φ→φ) and φ respectively

((φ→((φ→φ)→φ))→((φ→(φ→φ))→(φ →φ)))

O Using (A1), taking φ, ψ to be φ and (φ→φ) respectively

(φ→((φ→φ)→φ))

O Using Modus Ponens

((φ→(φ→φ))→(φ →φ))

O Using (A1), taking φ, ψ to be φ and φ respectively

(φ→(φ→φ))

O Using Modus Ponens

(φ→φ)


Soundness & Completeness

O A formal system is said to be sound if all

theorems in that system are tautology.

O A formal system is said to be complete if all

tautologies in that system are theorems.


Predicate

O A proposition involving some variables,

functions and relations

O Example

P(x) = “x > 3”

Q(x,y,z) = “x2 + y2 = z2”


Quantifier

O Universal: “for all” ∀

∀x P(x) ⇔ P(x1) ∧ P(x2) ∧ P(x3) ∧ …

O Existential: “there exists” ∃

∃x P(x) ⇔ P(x1) ∨ P(x2) ∨ P(x3) ∨ …

O Combinations:

∀x ∃y y > x


Quantifiers: Negation

O ¬ (∀x P(x)) ⇔ ∃x ¬ P(x)

O ¬ (∃x P(x)) ⇔ ∀x ¬ P(x)

O ¬ ∃x ∀y P(x,y) ⇔ ∀x ∃y ¬ P(x,y)

O ¬ ∀x ∃y P(x,y) ⇔ ∃x ∀y ¬ P(x,y)


Rules of Inference

1. Direct Proof

2. Indirect Proof

3. Proof by Contradiction

4. Proof by Cases

5. Induction


Direct Proof

O If the two propositions (premises) p and p → q are

theorems, we may deduce that the proposition q is

also a theorem.

O This fundamental rule of inference is called modus

ponens by logicians.

p → q

p

∴ q


Indirect Proof

O Proves p → q by instead proving the contra-

positive, ~q → ~p

p → q

~q

∴ ~p


Proof by Contradiction

O The rule of inference used is that from

theorems p and ¬q → ¬p, we may deduce

theorem q.

p

~q → ~p

∴ q


Proof by Cases

O You want to prove p → q

O P can be decomposed to some cases:

p ↔ p1 ν p2 ν …ν pn

O Independently prove the n implications given

by

pi → q for 1 ≤ i ≤ n.


Proof by Induction

O To prove

1. Proof P(0),

2. Proof

( )xP x

[ ( ) ( 1)]x P x P x


Constructive Existence Proof

O Want to prove that

O Find an a and then prove that P(a) is true

)(xPx


Non-Constructive Existence Proof

O Want to prove that

O You cannot find an a that P(a) is true

O Then, you can use proof by contradiction:

)(xPx

FxPxxPx )(~)(~


Intuitive (Naïve) Set Theory

There are three basic concepts in set theory:

O Membership

O Extension

O Abstraction


Membership

O Membership is a relation that holds between a set

and an object

O x∈A to mean “the object x is a member of the set A”, or “x belongs to A”. The negation of this assertion

is written x∉A as an abbreviation for the proposition

¬(x∈A)

O One way to specify a set is to list its elements. For

example, the set A = {a, b, c} consists of three

elements. For this set A, it is true that a∈A but d∉A


Extension

O The concept of extension is that two sets are

identical if and if only if they contain the

same elements. Thus we write A=B to mean

∀x [x∈A ⟺ x∈B]


Abstraction

Each property defines a set, and each set defines a

property

O If p(x) is a property then we can define a set A

A = {x ∣ p(x)}

O If A is a set then we can define a predicate p(x)

p(x) = x∈A


Intuitive versus axiomatic set theory

O The theory of set built on the intuitive concept of

membership, extension, and abstraction is known as

intuitive (naïve) set theory.

O As an axiomatic theory of sets, it is not entirely

satisfactory, because the principle of abstraction

leads to contradictions when applied to certain

simple predicates.


Russell‟s Paradox

Let p(X) be a predicate defined as

P(X) = (XX)

Define set R as

R={X|P(X)}

O Is P(R) true?

O Is P(R) false?


Gottlob Frege‟s Comments

O The logician Gottlob Frege was the first to develop

mathematics on the foundation of set theory. He

learned of Russell Paradox while his work was in

press, and wrote, “A scientist can hardly meet with

anything more undesirable than to have the

foundation give way just as the work is finished. In

this position I was put by a letter from Mr. Bertrand

Russell as the work was nearly through the press.”


Power Set

O The set of all subsets of a given set A is

known as the power set of A, and is denoted

by P(A):

P(A) = {B | B A}


Set Operation-Union

O The union of two sets A and B is

A B = {x | (x A) (x B)}

and consists of those elements in at least one of A

and B.

O If A1, …, An constitute a family of sets, their union is

= (A1 … An)

= {x| x Ai for some i, 1≤ i ≤ n}


Set Operation-Intersection

O The intersection of two sets A and B is

A B = {x | (x A) ∧ (x B)}

and consists of those elements in at least one of A and B.

O If A1, …, An constitute a family of sets, their intersection is

= (A1 … An)

= {x| x Ai for all i, 1≤ i ≤ n}


Set Operation-Complement

O The complement of a set A is a set Ac defined as:

Ac = {x | x ∉ A}

O The complement of a set B with respect to A, also

denoted as A-B, is defined as:

A-B = {x ∈ A | x ∉ B}


Ordered Pairs and n-tuples

O An ordered pair of elements is written

(x,y)

where x is known as the first element, and y is known

as the second element.

O An n-tuple is an ordered sequence of elements

(x1, x2, …, xn)

And is a generalization of an ordered pair.


Ordered Sets and Set Products

O By Cartesian product of two sets A and B, we mean

the set

A×B = {(x,y) | xA , yB}

O Similarly,

A1×A2×…An = {x1A1, x2A2, …, xnAn}


Relations

A relation ρ between sets A and B is a subset of A×B:

ρ A×B

O The domain of ρ is defined as

Dρ = {x A | for some y B, (x,y) ρ}

O The range of ρ is defined as

Rρ = {y B | for some x A, (x,y) ρ}

O If ρ A×A, then ρ is called a ”relation on A”.


Types of Relations on Sets

O A relation ρ is reflexive if

(x,x) ρ, for each x A

O A relation ρ is symmetric if, for all x,y A

(x,y) ρ implies (y,x) ρ

O A relation ρ is antisymmetric if, for all x,yA

(x,y) ρ and (y,x) ρ implies x=y

O A relation ρ is transitive if, for all x,y,z A

(x,y) ρ and (y,z) ρ implies (x,z) ρ


Partial Order and Equivalence Relations

O A relation ρ on a set A is called a partial

ordering of A if ρ is reflexive, anti-symmetric,

and transitive.

O A relation ρ on a set A is called an equivalence

relation if ρ is reflexive, symmetric, and

transitive.


Total Ordering

O A relation ρ on a set A is a total ordering if ρ

is a partial ordering and, for each pair of

elements (x,y) in A×A at least one of (x,y)ρ

or (y,x)ρ is true.


Inverse Relation

O For any relation ρ A×B, the inverse of ρ is

defined by

ρ-1 = {(y,x) | (x,y) ρ}

O If Dρ and Rρ are the domain and range of ρ, then

Dρ-1 = Rρ and Rρ-1 = Dρ


Equivalence

O Let ρ A×A be an equivalence relation on A. The

equivalence class of an element x is defined as

[x] = {y A | (x,y) ρ}

O An equivalence relation on a set partitions the set.


Functions

A relation f A× B is a function if it has the property:

for all x,y,z, (x,y) f and (x,z) f implies y = z

O If f A× B is a function, we write

f: A → B

and say that f maps A into B. We use the common

notation

Y = f(x)

to mean (x,y) f.


Functions (cont.)

O As before, the domain of f is the set

Df = {x A | for some y B, (x,y) f}

and the range of f is the set

Rf = {y B | for some x A, (x,y) f}

O If Df A, we say the function is a partial function; if Df = A, we say that f is a total function.


Functions (cont.)

O If x Df, we say that f is defined at x; otherwise f is undefined at x.

O If Rf = B, we say that f maps Df onto B.

O If a function f has the property

for all x,y,z, f(x) = z and f(y) = z implies x = y

then f is a one-to-one function.

If f: A → B is a one-to-one function, f gives a one-to

one correspondence between elements of its domain

and range.


Functions (cont.)

O Let X be a set and suppose A X. Define function

CA: X → {0,1}

such that CA(x) = 1, if x A; CA(x) = 0, otherwise. CA(x) is called the characteristic function of set A

with respect to set X.

O If f A× B is a function, then the inverse of f is the

set

f--1 = {(y,x) | (x,y) f}

f--1 is a function if and only if f is one-to-one.


Functions (cont.)

O Let f: A → B be a function, and suppose that X A. Then the set

Y = f(X) = {y B | y = f(x) for some x X}

is known as the image of X under f.

O Similarly, the inverse image of a set Y included in

the range of f is

f -1(Y) = {x A | y = f(x) for some y Y}


Cardinality

O Two sets A and B are of equal cardinality, written as

|A| = |B|

if and only if there is a one-to-one function f: A → B

that maps A onto B.

O We write

|A| ≤ |B|

if B includes a subset C such that |A| = |C|.

O If |A| ≤ |B| and |A| ≠ |B|, then A has cardinality less than that of B, and we write

|A| < |B|


Cardinality (cont.)

O Let J = {1, 2, …} and Jn = {1, 2, …, n}.

O A sequence on a set X is a function f: J → X. A sequence

may be written as

f(1), f(2), f(3), …

However, we often use the simpler notation

x1, x2, x3, …., xi X

O A finite sequence of length n on X is a function

f: Jn → X, usually written as

x1, x2, x3, …., xn, xi X

O The sequence of length zero is the function f: → X.


Finite and Infinite Sets

O A set A is finite if |A| = |Jn| for some integer n ≥ 0, in which case we say that A has cardinality n.

O A set is infinite if it is not finite.

O A set X is denumerable if |X| = |J|.

O A set is countable if it is either finite or denumerable.

O A set is uncountable if it is not countable.


Some Properties

O Proposition: Every subset of J is countable.

Consequently, each subset of any denumerable set is

countable.

O Proposition: A function f: J → Y has a countable

range. Hence any function on a countable domain

has a countable range.

O Proposition: The set J × J is denumerable.

Therefore, A × B is countable for arbitrary

countable sets A and B.


Some Properties (cont.)

O Proposition: The set A B is countable whenever A

and B are countable sets.

O Proposition: Every infinite set X is at least

denumerable ; that is |X| ≥ |J|.

O Proposition: The set of all infinite sequence on {0, 1}

is uncountable.


Some Properties (cont.)

O Proposition: (Schröder-Bernestein Theorem)

For any set A and B, if |A| ≥ |B| and |B| ≥ |A|, then

|A| = |B|.

O Proposition: (Cantor’s Theorem)

For any set X,

|X| < |P(X)|.


1-1 correspondence Q↔N

O Proof (dove-tailing):


Countable Sets

O Any subset of a countable set

O The set of integers, algebraic/rational numbers

O The union of two/finite number of countable sets

O Cartesian product of a finite number of countable sets

O The set of all finite subsets of N

O Set of binary strings


Diagonal Argument


Uncountable Sets

O R, R2, P(N)

O The intervals [0,1), [0, 1], (0, 1)

O The set of all real numbers

O The set of all functions from N to {0, 1}

O The set of functions N → N

O Any set having an uncountable subset


Transfinite Cardinal Numbers

O Cardinality of a finite set is simply the number of

elements in the set.

O Cardinalities of infinite sets are not natural numbers,

but are special objects called transfinite cardinal

numbers.

O 0:|N|, is the first transfinite cardinal number.

O continuum hypothesis claims that |R|=1, the second transfinite cardinal.


Alphabet

O A finite and nonempty set of symbols

(usually shown by ∑ or Г).

O Example:

= {𝑎, 𝑏, 𝑐, … , 𝑧}


Strings

O A finite list of symbols from an alphabet

O Example: house

O If ω is a string over ∑, the length of ω, written |ω|, is

the number of symbols that it contains.

O The empty string (ε or λ) is the string of length zero.

εω = ωε = ω

O String z is a substring of ω if z appears consecutively

within ω.


Operations on Strings O Reverse of a string:

w = a1a2…an ababaaabbb

wR = an…a2a1 bbbaaababa

O Concatenation:

w = a1a2…an abba

v = b1b2…bm bbbaaa

wv = a1a2…an b1b2…bm abbabbbaaa

|wv| = |w| + |v|


Operations on Alphabet

O * :

The set of all strings that can be produced from ∑.

O + :

The set of all strings, excluding λ, that can be produced from ∑.

Suppose that λ={λ}. Then:


Languages

O A set of strings

O Any language on the alphabet ∑ is a subset of ∑*.

O Examples:


Operations on Languages

Languages are a special kind of sets and operations on sets can be defined on them as well.

O Union

O Intersection

O Relative Complement

O Complement

λ, b, aa, ab, bb, aaa, …}


Operations on Languages (cont.)

O Reverse of a language

O Example:



O Concatenation

O Example:



O *: Kleene * The set of all strings that can be produced by concatenation

of strings of a language.

Example:

O +:

The set of all strings, excluding λ, that can be produced by concatenation of strings of a language.


Gödel Numbering

O Let ∑ be an alphabet containing n objects. Let h: ∑ → Jn

be an arbitrary one-to-one correspondence. Define

function f as:

f: ∑* → N

such that f(ε) =0; f(w.v) = n f(w) + h(v), for w ∑* and v ∑.

f is called a Gödel Numbering of ∑*.

O Proposition: ∑* is denumerable.

O Proposition: Any language on an alphabet is countable.


Documents

Theory of Languages and - Sharifce.sharif.edu/courses/93-94/2/ce415-1/resources/root...Theory of Languages and Automata Prof. Movaghar 2 Grading Policy O Assignments 30% O First Quiz