Theory of Computer Algorithms (4005-800-01): Introduction

Theory of Computer Algorithms (4005-800-01):

IntroductionProf. Richard Zanibbi

Course Administrative Information

Course Web Page

http://www.cs.rit.edu/~rlaz/algorithms20082

Course Syllabus and Tentative Schedule

Available from course web pages; handout

Resources

Additional textbooks/references/URLs listed on course web pages; some books to be placed on 2hr reserve in Watson library 2

Why and How do We Study Algorithms?

How we study algorithms: Space and Time

The Time-Space Tradeoff

Generally speaking, the more space used to store information, the less time needed to compute desired information, and vice versa.

• Simple example: table lookup for exponents, vs. function computing exponents iteratively

• Depending on the problem, at times we also want to consider other resources (e.g. power on mobile devices)

4

Emphasis for Algorithm Analysis:Worst-Case Time RequirementsWhy emphasize time?

In general, unless the space requirements are truly excessive, we want the fastest algorithm

• Analyses related to time often generalize quite directly to space and other resource types

• Note: while fast algorithms are useful, speed is not the only criteria for selecting an algorithm

Worst-case? Why so pessimistic?

Worst-case analyses are useful in practice and provide guarantees 5

A Benchmark:Brute-Force Search

Operation

Brute-force search enumerates the set of possible solutions, and checks each one

• e.g. to sort a list of numbers, generate all permutations until a sorted list is found

• Note: while inelegant and inefficient, brute-force search is correct, making it a useful for understanding and comparison

Our Goal

Preserve correctness, while finding faster solutions; apply knowledge to produce more informed algorithms 6

!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./011%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"

!"#$%&'(#$)*+,-.&"/%$)0($12-3,-/)0425

2 3456'7$8%9(8"4#(:9($6(:;<"'9$=;<(!"#$#%&$&'(0

2 >"'?6'%=;@"(6?$";(<'=A9($8"(47;"(&"$A"";(A8=$(79(?"=97&4"(=;<(A8=$(79(7%#6997&4"0

2 3456'7$8%7@(%=$8"%=$7@9(#'6B7<"9(=($#)*+#*,?6'($=4C7;5(=&6:$(#'65'=%(&"8=B76'0

2 >"'?6'%=;@"(79($8"("+--,)"( 6?(@6%#:$7;50

2 D8"(4"996;9(6?(#'65'=%(#"'?6'%=;@"(5";"'=47E"($6(6$8"'(@6%#:$7;5('"96:'@"90(

2 !#""<(79(?:;F

Measuring Algorithm Performance

T(n)

The time, in “primitive” computations needed by an algorithm for input size n (normally worst-case analysis)

• Primitive computations might be: assembly instructions, a line of C/Java, a basic operation in pseudo code (e.g. assignment, addition)

• Goal: machine-independent performance measure

8

!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0+,1%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"

!"#$%&'(&)#)*+%,%

-'.%/01)%,2&123245567

8 A1"7(9 %4:;%2%($;%"(<=(45><';$?%(<@(4@6(;@#2$(<=(3;A"("0

34,.)5,01)%,2&13<%"$;%"37

8 A1"7(9 ":#"B$"C($;%"(<=(45><';$?%(<D"'(455(;@#2$3(<=(3;A"("0

8 E""C(4332%#$;<@(<=(3$4$;3$;B45(C;3$';&2$;<@(<=(;@#2$30

6,%/01)%,2&1&<>237

8 F?"4$(G;$?(4(35<G(45><';$?%($?4$(G<'H3(=43$(<@(0%/> ;@#2$0

Efficient AlgorithmsPolynomial Time Algorithm

If there exist constants c and d (c, d > 0) such that running time is bounded by cnd for all input sizes (n)

Efficient Algorithm (Page 33 K&T)

An algorithm is efficient if it has a polynomial running time

• This measure (normally) reflects efficiency of algorithms and tractability of problems in practice

• Problems with polynomial solutions usually require low order polynomials (e.g. n, n2, n3) 10

Copyright © 2005 Pearson Addison-Wesley. All rights reserved.

Example: The Problem of Stable Matching

(Chapter 1.1, K&T)

Problem Definition

Task

Design a procedure to match individuals in two sets that is self-enforcing (i.e. stable)

Example

Matching job applicants to employers (e.g. for co-op positions)

13

Problem FormulationIssues in the General Case

Asymmetric matching: companies need multiple employees, applicants need one job

Sizes: we may have a different sized sets

Simplification

Sets to be matched are of the same size, and each individual is in exactly one match (pair)

All individuals have a preference list ranking matches with individuals of the other set

14

Matches

Perfect Match

All individuals in both sets (e.g. men and women) are paired with a unique partner

Stable Match (Our Goal)

A matching that is perfect and stable, where no two individuals mutually prefer to leave their matching in order to join together

15

Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-2

A Solution:Gale-Shapley Algorithm

17Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-4

Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-3

Properties of theG-S Algorithm

Upper Bound on Worst-Case Run-time (Performance)

(1.3) G-S Terminates after at most n2 iterations of the while loop

Match Properties (Correctness)

(1.1) Each woman remains engaged from their first proposal, and the sequence of partners to which she becomes engaged gets better and better (according to her preference list)

(1.4) If m is free at some point, then there is a woman to whom he has not proposed

(1.5) The set S returned at termination is a perfect matching

(1.6) The set S returned at termination is a stable matching19

Additional Properties

(1.7) Every execution of the G-S algorithm yields the same matching S*, where S* = {(m, best(m)) : m ε M)}.

(1.8) In stable matching S*, each woman is paired with her worst valid partner

valid partner: partner in a stable matching

best valid partner: highest ranked partner in a stable matching

20

Overview: Five Representative Problems

(Chapter 1.2 K&T)

I. Interval Scheduling Problem

Task

Assign a resource (e.g. lecture hall) to requests with known time intervals, maximizing the number of satisfied requests that do not conflict

Solved By

Greedy algorithm: sort requests, then single pass produces solution

22

Copyright © 2005 Pearson Addison-Wesley. All rights reserved.

2. Weighted Interval Scheduling Problem

Modification

Requests have an associated weight vi > 0. New goal is to maximize weight of satisfied requests that do not conflict

Special Case

If for all i vi = 1, instance of regular Interval Scheduling problem

Solved By (not greedy alg!)

Dynamic Programming: build optimal value over all possible solutions using an efficient table-based strategy 23

3. Bipartite Matching Problem

Task

Find a the largest set of edges producing disjoint pairs of nodes in a bipartite graph

• e.g. each x paired with a unique y

Solution (not greedy or dynamic!)

Augmentation: Inductively construct larger and larger matchings with backtracking (used in network flow problems)

24

Copyright © 2005 Pearson Addison-Wesley. All rights reserved.

4. Independent Set Problem

Task

Identify max no. of nodes not joined by an edge in a graph

• Edges represent ‘conflicts’

• Interval scheduling, bipartite matching instances of I.Set problem

Solution (general case)

No efficient algorithm believed to exist (can use brute force). Problem is NP-complete 25

Copyright © 2005 Pearson Addison-Wesley. All rights reserved.

5. Competitive Facility Location

Task

Two companies take turns selecting locations (nodes) forming an independent set, trying to maximize the value of selected nodes. Is there a strategy for player 2 guaranteeing a node set with at least value B?

26

Copyright © 2005 Pearson Addison-Wesley. All rights reserved.

(Comp. Facil. Location, Cont’d)

Solution

No short ‘proof’ for a solution; requires detailed case analysis (i.e. game traces); problem is PSPACE-complete (believed harder than NP-complete problems)

Many game playing and planning problems belong to PSPACE

27

Asymptotic Order of Growth(Section 2.2, K&T)

Asymptotic ComplexityDefinition

Characterizes worst-case running time of an algorithm in the limit, i.e. as input size goes to infinity

• Rate of growth defined as proportional to some function f(n) (i.e. within a constant multiple of f(n))

• f(n) normally simple (e.g. n2), not a detailed characterization such as: 1.62n2 + 3.5n + 8

• Consider upper (big ‘O’), lower (Ω), and tight (Θ) bounds

(One) Purpose

Identify sets of algorithms with similar behavior 29

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/*

!"#$%&'&()*+'&,&('+

=012$3$421(56##"'(&261789:

;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/

;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/0

!"#$%&'&()*+'&,&('+

=123$4$532(67##"'(&37289:;

<"(='5$"(>68:(>(=6'68:: 5?($@"'"("A59$(B329$42$9(? C(,+(8, C(, 97B@($@4$(,(! >68:(! ?'68: ?3'(4DD(8 " 8,/

<"(='5$"(>68:(>(=6'68:: 5?($@"'"("A59$(B329$42$9(? C(,+(8, C(, 97B@($@4$(,(! >68:(! ?'68: ?3'(4DD(8 " 8,/

-.!/01-2 *8* >(=680: 6? >()+(8, >(*:

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/-

!"#$%&'&()*+'&,&('+

=012$3$421(56##"'(&261789:

;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/

;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/

-.!/01-2 *8* =(=58D9 5? =()+(8, =(*9

>@8?)&"8;A*8")*B7:@5;

>@88$A*C"85/D7$E5F@7:&)$

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/0

!"#$%"&'('#')($)&$*+()#,#')(

=1'1822 3(4(>182(5 $6"'"("789$(:;<9$=<$9(? >(,+(8, >(, 9?:6($6=$(,(! >182(! ?'182@;'(=AA(8 " 8, B

=1'1822 3(4(>182(5 $6"'"("789$(:;<9$=<$9(? >(,+(8, >(, 9?:6($6=$(,(! >182(! ?'182@;'(=AA(8 " 8, B

-./012-3 *8* # =18C2

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/),

!"#$%&'()'*+*(*+%,

!"#$%#&'"#( 0(1"$(23(4(56'%784('"#'"1"3$1(43(43639%671(573:$263(23($;"(1"$/

=<8=(>(8? @(><8*=(

%"431(

=<8=(>(8? @((<8=

56'(16%"((<8=(! ><8*= /

-./!01-2

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/))

!"#$%&'()'*+*(*+%,

!"#$%#&'"#( 0(1"$(23(4(56'%784('"#'"1"3$1(43(43639%671(573:$263(23($;"(1"$/

8* <(==8>(?(==8*>

%"431

56'(439(>=8>(! ==8>@

8* <(>=8>(?((=8>

56'(16%"((=8>(! ==8*> /

-./!01-2

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0

!!"#$%$&#"'()#*+,'-#."/01

=123$4$532(56(42(>##5%/?">89 23$4$532/((7$(%48"6(23(6"26"($3(649(@:8; 56(4$(<"46$(=:8*;/

!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F

!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0

!!"#$%$&#"'()#*+,'-#."/01

=123$4$532(56(42(>##5%/?">89 23$4$532/((7$(%48"6(23(6"26"($3(649(@:8; 56(4$(<"46$(=:8*;/

!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F

!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F

23456728 :A =()+(8, =()G;;:<H88 !%

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)-

!!"#$%$&#"'($&)*$'+#,"-./

!"'0811(2(30'0811((# $0'0811!"'0811(2(30'0811((# $0'0811

f(n) = Θ(g(n)) means that both f(n) = O(g(n)) and f(n) = Ω(g(n))

!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0

!!"#$%$&#"'($&)*$'+#,"-./

!"'1822(3(41'1822((# $1'1822!"'1822(3(41'1822((# $1'1822

21* **

*

)888 !%&01234506

f(n) = Θ(g(n)) means that both f(n) = O(g(n)) and f(n) = Ω(g(n))

!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0++1%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"

Θ!"#$%$&#"

1 2'3#(43563'7"'($"'%89(:;<3'"(4"=7:<;(>3<8$=<$80

1 ?@=%#4"A(B"B(C(D,"5 E -" C(F,GF(H(ΘI"BJ

!"#$%ΘI-I"JJ(H(K(A*I"J A $L"'"("@:8$(#38:$:M"(>3<8$=<$8((/*((+*(=<7(

", 8N>L($L=$(,(≤ (/(-I"J(≤ A*I"J(≤ (+(-I"JO3'(=44(" ≥ ", P

&'()'**+)'(%

!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0+11%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"

!"#$%&'&()*%+,-',$./)+

"

A2"3

",

4 5"(6789:;<=$(>?<8'"(@6A%#$8$>B@::A(6:8C"'(@:?8'>$7%6*(78C"D"'0

4 E"@:FC8':;(;"6>?<(6>$9@$>8<6(8G$"<(B@::(G8'(@(B@'"G9:(&@:@<B><?(8G("<?><""'><?(8&H"B$>D"60

4 I6A%#$8$>B(@<@:A6>6(>6(@(96"G9:($88:($8(7":#($8(6$'9B$9'"(89'($7><J><?0

57"<(" ?"$6(:@'?"("<89?7*(@(Θ2"+3 @:?8'>$7%(?,B?30 &"@$6(@(Θ2"13 @:?8'>$7%0