Computational Models - Introduction1

Handout Mode

Instructors:Prof. Iftach Haitner iftach.haitner at cs.tau.ac.il

Teaching Assistants:Noam Mazor noammaz@gmail.com

Mark Roznov markroza at post.tau.ac.il

Tel Aviv University. Fall Semester, 2017-2018. Mondays, 13–16

October 23, 2017

1Based on slides by Benny Chor, Tel Aviv University, modifying slides by Maurice Herlihy, Brown University.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 1 / 28

Part I

Administrativia

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 2 / 28

Administrativia

Course website:http://moodle.tau.ac.il/enrol/index.php?id=368220001

Site (containing a forum) is our sole mean of disseminating information.

Course Requirements:

I 6 problem sets (10% of grade).Submission via Moodle (see the course website)

I Readable, concise, correct answers expected.I Late submission will not be accepted. (You have between one and

two weeks, start working when you get them. Any excuse has tocover all the period.)

I See more instructions on the course website.

I Solving problems independently is highly recommended.I (variants of) questions from HW in exams!

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 3 / 28

AdministraTrivia II

I Midterm, covering first 6 lectures (up to computability). We will discuss itwhen we get closer.

I Midterm (magen) is scheduled to Friday, December 1, 2017.

I Final exam, covering all course material.

I You must pass the final exam to get a passing course grade.

I Final Grade: 0.75 · Exam + 0.15 ·max{Midterm,Exam}+ 0.10 · HW .

I Prerequisites (formally): Extended introduction to computer scienceI But most importantly is “mathematical maturity”.I Students from other disciplines with mathematical background

encouraged to contact the instructor.

I Textbook: Sipser — Introduction to the theory of computation, first orsecond editions.

I Other (excelent) book: Hopcroft, Motwani, and Ullman —Introduction to Automata Theory, Languages, and Computation.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 4 / 28

Part II

Course overview

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 5 / 28

Why study theory?

I Basic computer science issues

I What is a computation?I Are computers omnipotent?I What are the fundamental capabilities and limitations of computers?

I Pragmatic reasons

I Avoid intractable or impossible problems.I Apply efficient algorithms when possible.I Learn to tell the difference.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 6 / 28

Course Topics

I Automata Theory: Basic model of computation.

I Re-invented in many other disciplines.

I Computability Theory: What can computers do?

I True impossibility results.

I Complexity Theory: What makes some problems computationally hardand others easy?

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 7 / 28

Automata theory – Simple models

I Finite automata.

I Related to controllers and hardware design.I Useful in text processing and finding patterns in strings.I Probabilistic (Markov) versions useful in modeling various natural

phenomena (e.g. speech recognition).

I Push down automata.

I Tightly related to a family of languages known as context freelanguages.

I Play important role in compilers, design of programming languages,and studies of natural languages.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 8 / 28

Computability Theory

In the first half of the 20th century, mathematicians such as Kurt Göedel, AlanTuring, and Alonzo Church discovered that some fundamental problemscannot be solved by computers.

I Proof verification of statements can be automated.

I It is natural to expect that determining validity can also be done by acomputer.

I Theorem: A computer cannot determine if mathematical statement trueor false.

I Results needed theoretical models for computers.

I These theoretical models helped lead to the construction of realcomputers.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 9 / 28

Complexity Theory

Key notion: tractable vs. intractable problems.

I A problem is a general computational question:

I description of parametersI description of solution

I An algorithm is a step-by-step procedure

I a recipeI a computer programI a mathematical object

I We want the most efficient algorithms

I fastest (usually)I most economical with memory (sometimes)I expressed as a function of problem size

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 10 / 28

Example: Traveling Salesman Problem

10 9

36

9

a

b

c

d

5

Roger WilliamsZoo

Brown UniversityAl FornoRestaurant

StateCapitol

(not drawn to scale)

Input:I set of citiesI set of inter-city distances

Goal: want the shortest tour through the citiesIftach Haitner (TAU) Computational Models - Introduction October 23, 2017 11 / 28

Example: Traveling Salesman Problem

Example: a,b,d , c,a has length 27

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 12 / 28

Part IV

AppendixNot Taught in Class

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 13 / 28

Section 1

A (Very) Short Math Review

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 14 / 28

A Very Short Math Review

I Graphs

I Strings and languages

I Mathematical proofs

I Mathematical notations (sets, sequences, . . . )√

I Functions and predicates√

√= will be done in recitation.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 15 / 28

Graphs

1

2

3

4

5

1

234

I G = (V ,E), where

I V is set of nodes or vertices, andI E is set of edgesI degree of a vertex is number of edges

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 16 / 28

Graphs

2

3

5

1

4 subgraph

graph tree

root

2

3

5

1

4

cycle14

23

path

leaves

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 17 / 28

Directed Graphs

2

3

5

1

4

6

scissors

rockpaper

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 18 / 28

Directed Graph and its Adjacency Matrix

Which directed graph is represented by the following 6-by-6 matrix?

A =

0 1 0 0 0 01 0 1 0 0 00 0 0 1 0 00 0 0 0 1 01 0 0 0 0 10 0 0 0 0 0

A(i , j) = 1←→ (i , j) ∈ E

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 19 / 28

Strings and Languages

I an alphabet is a finite set of symbols

I a string over an alphabet is a finite sequence of symbols from thatalphabet.

I the length of a string is the number of symbols

I the empty string ε

I reverse: abcd reversed is dcba.

I substring: xyz in xyzzy .

I concatenation of xyz and zy is xyzzy .

I xk is x · · · x , k times.

I a language L is a (possibly infinite) set of strings.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 20 / 28

Proofs

We will use the following basic kinds of proofs.

I by construction

I by contradiction

I by induction

I by reduction

I we will often mix them.

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 21 / 28

Proof by Construction

A graph is k -regular if every node has degree k .

Theorem 1For every even n > 2, there exists a 3-regular graph with n nodes.

1

2

3

4

5

0

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 22 / 28

Proof by Construction

1

2

3

4

5

0

Proof: Construct G = (V ,E), where V = {0,1, . . . ,n − 1} and

E = {{i , i + 1} : for 0 ≤ i ≤ n − 2} ∪ {n − 1,0}∪{{i , i + n/2} : for 0 ≤ i ≤ n/2− 1}.

degree(i) = |{(i , i + 1), (i , i − 1), (i , i + n/2)}| = 3

Missing details?

Note: A picture is helpful, but it is not a proof!Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 23 / 28

Proof by Contradiction

Theorem 2√

2 is irrational.

Proof: Suppose not, then√

2 = mn , where m and n are relatively prime.

n√

2 = m =⇒ 2n2 = m2

So m2 is even =⇒ m is even (?). Let m = 2k .2n2 = (2k)2 = 4k2

=⇒ n2 = 2k2

Thus n2 is even, and so is n. Therefore both m and n are even, and notrelatively prime!

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 24 / 28

Proof by Induction

Prove properties of elements of an infinite set.

N = {1,2,3, . . .}

To prove that ℘ holds for each element, show:

I base step: show that ℘(1) is true.

I induction step: show that if ℘(i) is true (the induction hypothesis), thenso is ℘(i + 1).

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 25 / 28

Induction Example

Theorem 3All cows are the same color.

Base step: A single-cow set is definitely the same color.Induction Step: Assume all sets of i cows are the same color.Divide the set {1, . . . , i + 1} into U = {1, . . . , i}, and V = {2, . . . , i + 1}.

All cows in U are the same color by the induction hypothesis.All cows in V are the same color by the induction hypothesis.All cows in U ∩ V are the same color by the induction hypothesis.

Hence, all cows are the same color.Quod Erat Demonstrandum (QED).

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 26 / 28

Induction Example (cont.)

(cows’ images courtesy of www.crawforddirect.com/ cows.htm)

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 27 / 28

Proof by Reduction

We can sometime solve problem A by reducing it to problem B, whosesolution we already know.

Example: k th element using MAX element

Input: S = {7,12,3,15,9,5,19}, k = 2

Reduction to MAX

I For i = 1, . . . , k − 1 Do:

I M = MAX (S)

I S = S \ {M}I Next i

I Output MAX (S)

Iftach Haitner (TAU) Computational Models - Introduction October 23, 2017 28 / 28