MELJUN CORTES AUTOMATA THEORY chapter21

8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter21

1/36

CSC 3130: Automata theory and formal languages

Polynomial time

Fall 2008

MELJUN P. CORTES MBA MPA BSCS ACS

MELJUN CORTES


2/36

Efficient algorithms

The running time of an

algorithm depends on

the input

For longer inputs, we

allow more time

Efficiency is measured

as a function of input

size

ATM

PCP


3/36

Examples of running time

parsingproblem

running time

0n1n

algorithm LR(1)

O(n) O(n) O(n log n)

short paths

Dijkstra

matching

Edmonds

O(n3)

CYK

O(n2)

n= input size

running time

problem routing

2O(n)

scheduling

2O(n logn) 2O(n)

theorem proving


4/36

Input representation

Since we measure efficiency in terms of inputsize, how the input is represented will make a

difference

For us, any reasonable representation will be

okayThe number 17 17

10001 (17in base two)

11111111111111111

OK

OK

NO

This graph

0000,0010,0001,0010

1 2

3 4OK

(2,3),(3,4) OK


5/36

Measuring running time

What does it mean when we say:

One step in

all mean different things!

This algorithm runs in 1000 steps

java RAM machine Turing Machine

if (x > 0)

y = 5*y + x;

write r3; d(q3, a) = (q7, b, R)


6/36

Example

L= {0n1n: n> 0}

in java:

M(string x) {n = x.len;

if n % 2 == 0 reject;

else

for (i = 0; i


7/36

Efficiency and the Church-Turing thesis

The Church-Turing thesis says all these modelsare equivalent in power

but notin running time!

java

RAM machine

Turing Machine

multitape TM

UNIVAC


8/36

The Cobham-Edmonds thesis

However, there is an extension to the Church-Turing thesis that says

For any realisticmodels of computationM1and

M2:

So any task that takes time TonM1can be done

in time (say) T2or T3onM2

M1can be simulated onM2with at most

polynomial slowdown


9/36

Efficient simulation

The running time of a program dependson themodel of computation

but in the grand scheme, this is irrelevant

javaRAM machinemultitape TMordinary TM

fastslow

Every reasonable model of computation can be

simulated efficientlyon every other


10/36

Example of efficient simulation

Recall simulating multiple tapeson a single tape

M

0 1 0

0 1

1 0 0

G= {0, 1, }

S 0 1 0 10 # # 0 #1 0

G= {0, 1, , 0, 1, , #}

#


11/36


12/36

Simulation slowdown

Cobham-Edmonds Thesis:

multi-tape

TMjava

single tape TMRAM machine

O(t) O(t)

O(t2)

O(t2)

O(t)O(t)

M1can be simulated onM2with at most

polynomial slowdown


13/36

Running time of nondeterministic TM

What about nondeterministicTMs?

For ordinaryTMs, the running time ofMon input

xis the number of transitionsMmakes before it

halts

But a nondeterministic TM can run for a different

time on different computation paths


14/36

Example

Definition of running time for nondeterministic TM

qacc

q0 1/1R

0/0Rq1

10001

what is the running time?

qrej

running time =

computation path: any possible sequence of transitions

max length of any computation path

5


15/36

Simulation of nondeterministic TM

nondet TM

multi-tape

TM

1 00

1 00

2 21

input tape x

1

simulation tape z

address tape aFor all k> 0

For all possible strings aof length k

Copy xto z.

SimulateNon input zusing a as choices

If aspecifies an invalid choice or

simulation loops/rejects, abandon simulation.

IfNenters its accept state, accept and halt.

IfNrejected on all as of length k, reject and halt.

represents

possible choices

at each step

each a

describes

a possible

computation

path

N M


16/36

Simulation slowdown for

nondeterminismFor all k> 0

For all possible strings aof length k

Copy xto z.

SimulateNon input zusing a as choices

If aspecifies an invalid choice or

simulation loops/rejects, abandonsimulation.

IfNenters its accept state, accept and

halt.

IfNrejected on all as of length k, reject and

halt.simulation will halt when k= trunning time ofNis t

running time of simulation= (running time for specific a)

(number of as of length t)

= O(t)2O(t) = 2O(t)


17/36

Simulation slowdown

multi-tape

TMjava

single tape TMRAM machine

O(t) O(t)O(t2)

O(t2)O(t)

O(t)

nondeterministic TM

2O(t)

Do nondeterministic TM violate the Cobham-Edmonds thesis


18/36

Nondeterminism and the CE thesis

Cobham-Edmonds Thesis says:

But is nondetermistic computation realistic?

Any two realisticmodels of computation

can be simulated with polynomial slowdown


19/36

Example

Recall the scheduling problem

Scheduling with nondeterminism:

CSC 3230 CSC 2110

CSC 3160CSC 3130

Can you schedule final exams

so that there are no conflicts?

Exams vertices

Slots colors

Conflicts edges

Y R B

schedule(int n, Edges edges) {

for i := 1 to n:

choose { c[i] := Y; }

or { c[i] := R; }

or { c[i] := B; }for all e in edges:

if c[e.left] == c[e.right]

reject;

accept;

}


20/36

Example

... but if we had it, we could schedule in linear

time!

schedule(int n, Edges edges) {

for i := 1 to n:choose{ c[i] := Y; }

or{ c[i] := R; }

or{ c[i] := B; }

for all e in edges:

if c[e.left] == c[e.right]

reject;

accept;

}

In reality, programming

languages dont allow usto choose

We have to tellthe

computer how to make

these choices

Nondeterminism does not seem like a realistic

feature of a programming language or computer


21/36

Nondeterministic simulation

If we can do better, this would improve all known

combinatorial optimization algorithms!

nondeterministic TMmulti-tape

TM

2O(t) slowdown

Is this the best we can do?


22/36

Millenium prize problems

Recall how in 1900, Hilbert gave 23 problemsthat guided mathematics in the 20thcentury

In 2000, the Clay Mathematical Institute gave 7

problems for the 21stcentury

1 P versus NP

2 The Hodge conjecture

3 The Poincar conjecture

4 The Riemann hypothesis5 YangMills existence and mass gap

6 NavierStokes existence and

smoothness

7 The Birch and Swinnerton-Dyer

conjecture

$1,000,000

Hilberts 8thproblem

Perelman 2006 (refused money)

computer science


23/36

The P versus NP question

Among other things, this asks:

Is nondeterminism a realistic feature of computation?

Can the chooseconstruct be efficiently implemented?

Can we efficiently optimize any well-posed problem?

nondeterministic TM ordinary TM

Can nondeterministic TM be simulated onordinary TM with polynomial slowdown?

poly(t)

Most people think not,

but nobody knows for sure!


24/36

The class P

Pis the class of all languagesthat can be decided on an

ordinary TM whose running

time is some polynomialin the length of the input

By the CE thesis, we can replace

ordinary TM by any realistic

model of computation

multi-tape TMjava RAM


25/36

Examples of languages in P

parsingproblem

running time

0n1n

algorithm LR(1)

O(n) O(n) O(n log n)

short paths

Dijkstra

matching

Edmonds

O(n3)

CYK

O(n2)

n= input size

L01= {0n1n: n> 0}

LG= {x: x is generated by G}

PATH= {(G, a, b,L): G is a graph with apath of lengthL from ato b}

G is some CFG

MATCH= {G, a, b,L: G is a graph with aperfect matching} context-free

P(efficient)

decidable

L01

LGPATH

MATCH


26/36

Languages believed to be outside P

running timeof best-known algorithm

problem routing

2O(n)scheduling

2O(n) 2O(n)thm-proving

We do not know if these

problems have faster algorithms,

but we suspect not

P(efficient)

decidable

LGPATH

MATCH

ROUTE

SCHED

PROVE

?

To explain why, first we need

to understand what these

problems have in common


27/36


28/36

Boolean formula satisfiability

A boolean formulais an expression made up ofvariables, ands, ors, and negations, like

The formula is satisfiable if one can assign

values to the variables so the expression

evaluates to true

(x1x2) (x2x3x4) (x1)

x1 = Fx2 = Fx3 = T x4 = TAbove formula is satisfiable because this assignmentmakes it true:


29/36

Status of these problems

CLIQUE= {(G, k): G is a graph with a clique ofk vertices}

IS= {(G, k): G is a graph with an independent set ofk vertices}

VC= {(G, k): G is a graph with a vertex cover ofk vertices}

SAT= {f:f is a satisfiable Boolean formula}

running time

of best-known algorithm

problem CLIQUE

2O(n)

IS

2O(n)

SAT

2O(n)

VC

2O(n)

What do these problems have in common?


30/36

Checking solutions efficiently

We dont know how to solve them efficiently

But if someone told us the solution, we would be

able to check it very quickly

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Is (G, 5) in CLIQUE?

1,5,9,12,14

Example:


31/36

Cliques via nondeterminism

Checking solutions efficiently is equivalent todesigning efficient nondeterministic algorithms

Is (G, k) in CLIQUE?Example:

clique(Graph G, int k) {C = {}; % potential clique

for i := 1 to G.n: % choose clique

choose{ C := union(C, {i}); }

or{}

if size(C) != k reject; % check size is k

for i := 1 to G.n: % check all edges

for j := 1 to G.n: % are inif i in C and j in C

if G.isedge(i,j) == false

reject;

accept;

}


32/36

Example: Formula satisfiability

(x1

x2)

(x2

x3

x4)

(x1)f=

Checking solution: Nondeterministic algorithm:

FFTTsubstitutex1 = F x2 = Fx3 = T x4 = T

evaluate formula

(F T) (FTF) (T)f=

can be done in linear time

sat(Formula f) {

x = new bool[f.n];for i := 1 to n:

choose{ x[i] := true; }

or { x[i] := false; }

if f.eval(x) == true accept;

else reject;

}


33/36

The class NP

The class NP:

Lcan be solved on a nondeterministic TM inpolynomial time iff its solutions can be

checked in

time polynomial in the input length

NPis the class of all languages that can be

decided on a nondeterministic TM whose

running time is some polynomial in the length

of the input


34/36

P versus NP

because an ordinary

TM is only weaker

than anondeterministic one

Conceptually, findingsolutions can only be

harder than checking

them

P(efficient)

decidable

LGPATH

MATCH

CLIQUE

SAT IS

NP(efficiently checkable)

VC

Pis contained inNP


35/36

P versus NP

The answer to the question

is not known. But one reason it is believed to benegative is because, intuitively, searchingis

harder than verifying

For example, solvinghomework problems

(searching for solutions) is harder than grading

(verifying the solution is correct)

Is Pequal toNP?

$1,000,000


36/36

Searching versus verifying

Mathematician:

Given a mathematical claim, come up with a proof for it.

Scientist:

Given a collection of data on some phenomena, find a

theory explaining it.

Engineer:

Given a set of constraints (on cost, physical laws, etc.)

come up with a design (of an engine, bridge, etc) whichmeets them.

Detective:

Given the crime scene, find whos done it.

Documents

MELJUN CORTES AUTOMATA THEORY chapter21