-
8/9/2019 Analysis of Discrete Algorithms
1/23
A S u rvey o f An a l ys is T ech n i q u es fo r D iscre te A l
g o r ith ms *B R U C E WEIDEDepartment of Computer Science,
Carneg~e-MeUon UnwersLty, PLttsburgh, Pennsylvanm 15213
T h i s s u r v e y i n c l u d e s a n i n t r o d u c t i o n
t o t h e c o n c e p t s o f p r o b l e m c o m p l e x i t y , a
n a l y s i s o fa l g o r i t h m s t o f in d b o u n d s o n c o
m p l e x i t y , a v e r a g e - c a s e b e h a v i o r , a n d a
p p r o x i m a t i o na l g o m t h m s T h e m a j o r t e c h m
q u e s u s e d m a n a l y s i s o f a l g o r i t h m s a r e r e
v i e w e d a n de x a m p l e s o f t h e u s e o f t h e s e m e
t h o d s a r e p r e s e n t e d . A b r i e f e x p l a n a t i o
n o f t h e p r o b l e mc l a s s e s P a n d N P , a s w e l l a
s t h e c l a s s o f N P - c o m p l e t e p r o b l e m s , i s a
l s o p r e s e n t e d .Keywords and Phrases A n a l y s i s o f a
l g o r i t h m s , c o m p u t a t i o n a l c o m p l e x i ty ,c
o m b m a t o r lc s , N P - c o m p l e t e p r o b le m s , a p p
r o x i m a t i o n a l g o r i th m s .CR Categories 5.25, 5 30,
5.39
INTRODUCTIONWe shall express our darker purpose
- W i l l i a m S h a k e s p e a r eI t h a s l o n g b e e n r
e c o g n iz e d t h a t t h e s t u d yo f t h e b e h a v i o r o
f a l g o r i t h m s p l a y s a c r u -c i a l r o l e i n i n t
e l l i g e n t a l g o r i t h m d e s i g n .A h o , H o p c r o
f t , a n d U l l m a n [ 1 ] b e g i n t h ep r e f a c e o f t h
e i r r e c e n t b o o k o n a l g o r i t h m i cd e s i g n a n
d a n a l y s i s b y p o i n t i n g o u t t b a t" t h e s t u d
y o f a l g o r i t h m s i s a t t h e v e r yh e a r t o f c o m
p u t e r s c ie n c e. " A t t h e f o u n d a -t io n o f e v e r
y c o m p u t a t i o n a l d i s c ip l in e i s ac o ll ec ti o n
o f a l g o r i t h m s . A f t e r a p r o b l e m -f or e x a m p
l e , u n d e r s t a n d i n g s p e ec h , a n a -l y zi n g d a
t a , o r c o m p il in g a p r o g r a m - i sa n a l y z e d a t
a h i g h l e v e l a n d d e s i g n d e c i -s io n s a r e f in
a l iz e d , t h e a l g o r i t h m m u s t b ei m p l e m e n t e
d o n a r e a l m a c h i n e . O n e j obo f t h e c o m p u t e r
s c i e n t i s t is t o i s o l a te a n ds t u d y t h e s e a l
g o r i t h m s , w h i c h a b o u n d i ng r a p h t h e o r y ,
s t a t i s t i c s , o p e r a t i o n s r e -s e a r ch , a n d m
a n y o t h e r a re a s . H e n c e th ep e r v a s iv e n a t u r
e o f a n a l y s i s o f a lg o r i th m s .D e s p i te ( o r p e
r h a p s b e c a u s e o f) m a n y* T h in w o r k w a s s u p p
o r t e d i n p a r t b y a N a t i o n a lS c m n c e F o u n d a
t i o n G r a d u a t e F e l l o w s h i p .
s i g n i f ic a n t n e w r e s u l t s i n a n a l y s i s o f
a lg o -r i t h m s i n t h e p a s t f e w y e a r s , t h e r e
is n oc u r r e n t s u r v e y o f t h e m a t h e m a t i c a l t
e ch -n i q u e s u s e d i n a l g o r i t h m i c a n a l y s i s
. F o rt h e r e a d e r w h o w a n t s t o s e e t h e g o r yd e
t a il s o f t h e a n a l y s i s o f m a n y a l g o r it h m s
,a n d is w i l l i n g to a n d c a p a b l e o f s u p p l y i n
gm a n y m o r e d e t a il s h im s e l f , t h e t h r e e v o
l-u m e s o f The Art of Computer Program-m~ng b y K n u t h [ 4 3
, 4 4 , 4 6 ] a r e u n s u r -p a s s e d f o r c o m p l e t e n
e s s . A h o , H o p c r o f t ,a n d U l l m a n [1] i s a n e x
c e l l e n t t e x t w i t hm a n y e x a m p l e s o f b o t h d
e s ig n a n d a n a l y -s is b u t , l i k e K n u t h , d o e s
n o t p r o v i d e a no v e r v i e w o f t h e a r e a .F o r s o
m e o n e w h o w a n t s a n o v e r v i ew o ft e c h n i q u e s
a n d a r e v i e w o f s o m e i m p o r t a n tr e s u l t s , t
h e l i t e r a t u r e i s s p a r s e . T h e a r t i -c l e s b
y K n u t h [ 4 5 ] , R e i n g o l d [ 56 ], a n dF r a z e r [ 2
2 ] p r o v i d e s o m e r e l i e f, b u t c o n-c e n t r a t e
h e a v i l y o n p r e s e n t i n g t h e d e t a il so f o n e o
r tw o e x a m p l e a l g o r i t h m s o r t e c h -n i q u e s .
A f i n e p a p e r b y B o r o d i n [ 6] t r e a t sp r i m a r i
l y t h e t h e o r e t i c a l a s p e c t s o f c o m -p u t a t
i o n a l c o m p l e x i t y , r e v i e w i n g t h e d e fi -n i
t io n s a n d p r o p e r t i e s o f c o m p l e x i t yc l a s s
e s f o r v a r i o u s a u t o m a t a . H o p c r o f t ' ss u r
v e y [ 2 9] o f p r i n c ip l e s o f a l g o r i t h m d e -s i
g n c o m e s c l o s e s t t o f i t t i n g t h e b i l l , b u
t
C o p y r i g h t 1 97 7, A s s o c i a t io n f or C o m p u t
i n g M a c h i n e r y , I n c G e n e r a l p e r m i s s i o n t
o r e p u b l i s h , b u t n o t fo rp r o fi t , a l l o r p a r
t o f t h i s m a t e r i a l i s g r a n t e d p r o v i d e d t h
a t A C M ' s c o p y r i g h t n o t i c e is g i v e n a n d t h
a tr e f e r e n c e is m a d e t o t h e p u b h c a t l o n , t o
it s d a t e o f i s su e , a n d t o t h e f a c t t h a t r e p r
i n t i n g p r i v i l e g e s w e r eg r a n t e d b y p e r m i
s m o n o f t h e A s s o c i a t io n f o r C o m p u t i n g M a
c h i n e r y
Computing Surveys, Vol. 9, No. 4, December 1977
-
8/9/2019 Analysis of Discrete Algorithms
2/23
292 B . W e i d eC O N T E N T SI N T R O D U C T I O N1 T H E N
O T I O N O F C O M P L E X I T Y
B o u n d s o n C o m p l ex l t yWorst-case ersus Average-Case
Anal ymsMode ls of omputatlonM e a s u r i n g P r o b l e m S i z
e2 L O W E R B O U N D ST r] vl al o w e r B o u n d
sInforrnatlon-Theorehc Bo un ds an d Declslon TreesOraclesProblem
RedactmnMiscel laneou s Tr~cks3 U P P E R B O U N D SI d en t i fy
in g a W o rs t C aseC o u n t i n g O p e r a t m n sRecurrencesT
h e Bru te F o rce M eth o d4 T H E A V E R A G E C A S EPros and
Cons of Average-Case AnalysisS o m e E x am p les o f A v erag e -C
ase A n a ly s i s5 APPROXIMATION ALGORITHMSP ro b lem C las ses an
d Red u c lb lh ty" G u a r a n t e e d " A p p r o x l m a t m n A
l g o r it h m sP ro b ab f l l s t lc Beh aw o r o f A p p ro x lm
a tm n A lg o r i t h m sC O N C L U S I O N SA P P E N DI X G lo
ssa ry o f P ro b lem sA C K N O W L E D G M E N T SR E F E R E N C
E S
n e w r e s u l t s s i n c e i t a p p e a r e d a r e a b u n
-d a n t , e s p e c i a ll y in t h e a r e a o f a p p r o x i m
a -t i o n a l g o r i t h m s . I n o r d e r t o a v o i d d u p
l i c a -t i o n , t h e m o r e d e t a i l e d e x a m p l e s g
i v e nh e r e d is c u s s a l g o r it h m s o t h e r t h a n t
h eo n e s t r e a t e d i n t h o s e p a p e r s . T h i s s u r
v e yi s a n a t t e m p t t o co l le c t s o m e o f t h e i m p
o r -t a n t t e c h n i q u e s u s e d i n a l g o r i t h m i c
a n a l -y s i s a n d t o l is t s o m e o f t h e r e s u l t s p
r o -d u c e d , a n d t h e r e b y ( a l b e i t t e m p o r a r
i l y )h e l p f i ll t h e g a p i n t h i s a r e a . I t is d e
s i g n e dt o b e p r i m a r i l y a s u r v e y , w i t h t u t
o r i a lc o m m e n t s w h e r e a p p r o p r i a te .T h e r e
a d e r i s a s s u m e d t o b e fa m i l i a rw i t h t h e n o t
io n s o f a l g o r it h m s a n d d a t as t r u c t u r e s , t
o h a v e b e e n i n t r o d u c e d t oc o m p u t a t io n a l m
o d e l s su c h a s T u r i n g m a -c h in e s , a n d t o k n o
w s o m e t h i n g a b o u ta s y m p t o t i c b o u n d s o n f
u n c t i o n s a n d r e cu r -r e n c e r e l a ti o n s . H e o
r s h e s h o u l d a l s or e co g n i z e s o m e o f t h e i m p
o r t a n t d i s c r et e( c o m b i n a t o r i a l ) p r o b l e
m s . T h e s e i n c l u d es o r t i n g , s e a r c h i n g , g
r a p h p r o b l e m s , d i s -c r e t e o p t i m i z a t io n ,
s e t m a n i p u l a t i o n , a n dp a t t e r n m a t c h i n g
. T h e A p p e n d i x i s a g lo s-s a r y o f p r o b l e m s m
e n t i o n e d i n t h e t e x t ,
w h i c h s h o u l d h e lp i n t h i s r e g a r d . S o m ef
a m i l i a r i t y w i t h t h e n o t a t io n u s e d b y a l-g
o r i t h m i c a n a l y s t s w o u l d a l s o b e h e l p f u l
,b u t i s n o t e s s e n t i a l. T h e p a p e r i s n o ti n t
e n d e d a s a r e v i e w o f a l g o r i th m o r d a t as t r u
c t u r e d e s i g n , b u t a s a s u r v e y o f a n a l y -s i
s t e c h n i q u e s . T h e r e a d e r i s u r g e d t oc o n s
u l t t h e o r i g i n a l s o u r c e s f o r d e t a i l e da l
g o r i t h m s a n d a n a l y s e s w h i c h c a n o n l yb e b
r ie f l y m e n t i o n e d h e r e .1 . T H E N O T I O N O F C O
M P L E X I T Y
I f y o u w ~ s h to c o n v e r se w ~ t h m e , d e f i n ey o
u r t e r m s . - V o l t a i r e
I h a t e d e f i n ~ t w n s .- B e n ja min Di s r a e l iT h
e p r i m a r y d e f in i t io n o f c o m p l e x i t y isr u n n
i n g t im e , F o r a g i v e n p r o b l em , " t im ec o m p l e
x i t y " i s a f u n c t i o n t h a t m a p s p r o b -l e m s i
z e i n t o t h e t i m e r e q u i r e d t o s o l v et h e p r o
b l e m . W e w i l l n o t c o n s i d e r o t h e rm e a s u r e
s o f p r o b l e m d i f fi c u lt y , s u c h a st h e a m o u n
t o f s p a c e r e q u i r e d , n o r w i l l w ea t t e m p t t
o m e a s u r e c o m p r e h e n s i b i l i t y o fa l g o r i t
h m s o r t h e s c o re s o f o t h e r f a c to r sw h i c h m u
s t b e co n s id e re d w h e n d e s i g n in ga n a l g o r i t
h m . T o m a k e e v e n th i s i n tu i -t i v e l y s i m p l e
q u a n t i t y , t i m e , a u s e f u l m e a -s u r e , w e n e
e d t o c o n s i d e r s u c h q u e s t i o n sa s u p p e r a n
d l o w e r b o u n d s o n c o m p l e x it y ,t h e m o d e l o f
c o m p u t a t i o n , a n d t h e m a n n e ro f p r o b l e m r
e p r e s e n t a t io n . T h e m o d e l o fc o m p u t a t i o n
i s i m p o r t a n t b e c a u s e w ec o u n t i ts o p e r a ti
o n s w h e n m e a s u r i n g t im e .T h e p r o b l e m r e p r
e s e n t a t io n i s i m p o r t a n tb e c a u s e i t a f f e c
ts t h e p r o b l e m s iz e . T h e s eq u e s t i o n s a r e c
o n s i d e r e d i n t h e f o l l o w i n gs u b s e c t i o n s
.B o u n d s o n C o m p l e x i t yT y p i c a l l y , w e a r e i
n t e r e s t e d i n t h e ( in h e r -e n t ) c o m p l e x i t y
o f c o m p u t i n g t h e s o l u t i o nt o p r o b l e m s i n
a p a r t i c u l a r c l as s . F o r e x -a m p l e , w e m i g h
t w a n t t o k n o w h o w f a s tw e c a n h o p e t o s o r t a
l i s t o f n i t e m s ,i n i t i a l l y i n a n a r b i t r a r
y o r d e r , r e g a r d l e s so f t h e a l g o r i t h m w e u
s e . I n t h i s c a s e , w es e e k a " l o w e r b o u n d " L
( n ) o n s o r t i n g ,
C o m p u t i n g S u r v e y s , V o l . 9 , N o . 4 , D e c e
m b e r 1 9 7 7
-
8/9/2019 Analysis of Discrete Algorithms
3/23
A n a l y s~s T e c h n i qu e s f o r D ~sc re te A l g o r i t
h m s 29 3w h i c h i s a p r o p e r t y o f t h e s o r t i n g p
r o b l e ma n d n o t o f a n y p a r t i c u l a r a l g o r i th
m . T h i sl o w e r b o u n d s a y s t h a t n o a l g o r i th m
c a nd o t h e j o b i n f e w e r t h a n L ( n ) t i m e u n i t
sf o r a r b i t r a r y i n p u t s , i .e . , t h a t e v e r y s
o r t-i n g a l g o r i t h m m u s t t a k e a t l e a s t L ( n
)t i m e i n t h e w o r s t c a s e . S o m e a p p r o a c h e st
o r e f in i n g l o w e r b o u n d s a r e s u r v e y e d i nS e
c t i o n 2 .O n t h e o t h e r h a n d , w e m i g h t a l s o l
ik et o k n o w h o w l o n g i t w o u l d t a k e t o s o r ts u
c h a l is t u s i n g a k n o w n s o r t i n g a lg o -r i t h m
w i t h a w o r s t -c a s e i n p u t . H e r e , w ea r e a f t e
r a n " u p p e r b o u n d " U ( n ) , w h i c hs a y s t h a t f
o r a r b i t r a r y i n p u t s w e c an a l-w a y s s o r t i n
t i m e a t m o s t U(n) . T h a t i s ,i n o u r c u r r e n t s t
a t e o f k n o w l e d g e , w e n e e dn o t s e t t l e f o r a
n a l g o r i t h m w h i c h t a k e sm o r e t h a n U(n) t i m e
, b e c a u s e a n a l g o -r i t h m w h i c h o p e r a te s in
t h a t m a n y s te p si s k n o w n . F o r t h i s re a s o n ,
a l g o r i t h m s a r en o r m a l l y a n a l y ze d t o d e t e
r m i n e t h e i rw o r s t - c a s e b e h a v i o r , i n t h e
h o p e o f r e d u c -in g U(n) e v e n f u r t h e r b y d e m o
n s t r a t i n gt h a t s o m e n e w a l g o r i t h m h a s w o
r s t - c a s ep e r f o rm a n c e w h i c h i s b e t t e r t h a
n a n y p r e -v i o u s a lg o r i t h m . T e c h n i q u e s f o
r d o i n g th i sa r e r e v i e w e d i n S e c t i o n 3.O n e w
a y o f s e e i n g t h e d i s t i n c t io n b e -t w e e n l o w
e r a n d u p p e r b o u n d s i s t o n o t et h a t b o t h b o
u n d s a r e m i n i m a o v e r t h em a x i m u m c o m p l e x
i ty o f i n p u t s o f s iz e n .H o w e v e r , L ( n ) i s t h
e m i n i m u m , o v e r a llposs ib le a l g o r i th m s , o f t
h e m a x i m u m c om -p l e x i t y , w h i l e U(n) i s t h e m
i n i m u m , o v e ra l l k n o w n a l g o ri th m s , o f t h e
m a x i m u mc o m p l e x i t y . I n t r y i n g t o p r o v e b
e t t e r l o w e rb o u n d s , w e c o n c e n t r a t e o n t e
c h n i q u e sw h i c h w i l l a l l o w u s t o i n c r e a s e
t h e p r e c i -s io n w i t h w h i c h th e m i n i m u m , o v
e r a llposs ib le a l g o r i t h m s , c a n b e b o u n d e d .
I m -p r o v in g a n u p p e r b o u n d m e a n s f in d i n g a
na l g o r i t h m w i t h b e t t e r w o r s t - c a s e p e r fo
r m -a n c e . T h i s d i f f e r e n c e l e a d s t o t h e d i
f f e r -e n c e s in t e c h n i q u e s d e v e l o p e d in c o
m p l e x -i t y a n a l y s i s .W h i le t h e r e a r e a p p a
r e n t l y t w o co m -p l e x i t y f u n c t i o n s f o r p r o
b l e m s , l o w e r a n du p p e r b o u n d s , t h e u l t i m
a t e g o a l is t o m a k et h e s e t w o f u n c t io n s c o i
n ci de . W h e n t h i s i sd o n e , th e " o p t i m a l " a l g
o r i t h m w i ll h a v eL(n) = U(n) . F o r m o s t o f t h e p r
o b l e m s w ew i l l m e n t i o n , t h i s g o a l i s n o t y
e t r e a l iz e d .
W o r s t -C a s e v e r s u s A v e r a g e - C a s e A n a l y
s isT r a d i t i o n a l ly , t h e w o r s t - c a s e c o m p l
e x i t yh a s b e e n o f m a j o r t h e o r e t ic a l i n t e r
e s t f o rt h e r e a s o n s j u s t c i t e d . O c c a s i o n
a l l y i t i s o fp r a c t ic a l i m p o r t a n c e t o k n o w
t h e w o r s t -c a s e b e h a v i o r o f a n a l g o r i th m .
F o r e x a m -p l e , a n a i r - t ra f f i c c o n t r o l s y s
t e m w i t h g o o de x p e c t e d p e r f o r m a n c e m i g h
t n o t b e co n si d -e r e d u s e f u l i f o c c u r r e n c e
o f i t s w o r s t c a s e ,h o w e v e r u n l i k e l y , c o u
l d c a u s e a n a c c id e n t .R e c e n t ly , h o w e v e r ,
t h e r e h a s b e e n g r e a t e re f fo r t i n t h e a n a l y
s i s o f t h e b e h a v i o r o fa l g o r i t h m s " o n t h e
a v e r a g e , " s i n ce t h ea v e r a g e i s o f t e n m o r e
u s e f u l i n p r a c t i c e .F o r e x a m p l e , t h e s i m
p l e x a l g o r i t h m f o rl in e a r p r o g r a m m i n g i s
k n o w n t o r e q u i r ea n a m o u n t o f t i m e w h i c h is
a n e x p o n e n t i a lf u n c t io n o f t h e p r o b l e m s
iz e i n t h e w o r s tc a s e , b u t f o r p r o b l e m s e n c
o u n t e r e d i np r a c t i c e i t a l m o s t a l w a y s d o
e s e x t r e m e l yw e l l .T h i s a p p r o a c h h a s d i ff
ic u l ti e s, h o w e v e r .F o r o n e t h in g , a v e r a g i
n g o v e r m a n y c a s e sc o m p l i c a t e s t h e a n a l y
s i s c o n s i d e r a b l y .F u r t h e r , w h i l e t h e a v
e r a g e a lo n e m i g h tb e o f s o m e v a l u e , f i n d in
g t h e d i s t r i b u t i o no f s o l u t io n t i m e s o r e v
e n t h e v a r i a n c e i sa n a d d e d b u r d e n , a n d t h
e r e f o r e o f t e n n e -g l e c t e d i n p r a c t i c e . P
e r h a p s t h e b i g g e s to b j e c ti o n o f a l l is t h a
t t h e t y p i c a l a s s u m p -t i o n s r e g a r d i n g t h
e p r o b a b i l i t y d i s t r i b u -t io n o v e r a l l p o s
s i b le i n p u t s ( u s u a l l y s i m -p l e o n e s t o m a k
e t h e a n a l y s i s t r a c t a b l e )a r e o f t e n u n r e
a l i s t i c . T h e o b j e c t io n s w i l lb e c o v e r e d i
n m o r e d e t a i l i n S e c t i o n 4 .D e s p i t e t h e o b
j e c t i o n s , a v e r a g e - c a s e a n a l -y s i s i s i m
p o r t a n t a n d c o n t i n u e s t o g r o w .M o de l s o f C
o mpu ta t i o nD o e s c o m p l ex i ty m e a s u r e t h e n u m
b e r o fs te p s on a T u r i n g m a c h i n e , o r t h e n u m
b e ro f s e c o n d s o n a n I B M 3 7 0 /1 9 5 ? W e a r e u s u
-a l l y n o t i n t e r e s t e d i n e i t h e r o f t h e s e f
ig -u r e s e x a c t l y , a l t h o u g h e a c h i s a l e g i t
i m a t em e a s u r e o f c o m p l e x i t y i n c e r t a i n c
a se s .T h e i s s u e a t h a n d i s t h e " m o d e l o f c o m
p u -t a ti o n . " T u r i n g m a c h i n e c o m p l e x i t y i
ss o m e t i m e s i m p o r t a n t , a s w e w i l l s e e i nS e
c ti o n 5 w h e n e x a m i n i n g t h e p r o b l e mc l a s s e
s P a n d N P . A h i g h l y s o p h i s t i c a t e dm a c h i n
e l i k e t h e I B M 3 7 0/1 9 5 i s p r o b a b l ya b a d c h o
ic e b e c a u s e it m a k e s t h e a n a l y s i s
Computing Surveys,Vol. 9, No 4, December1977
-
8/9/2019 Analysis of Discrete Algorithms
4/23
294 B . Weideeven more complicated than it needs to befor most
purposes. Knuth [43, 44, 46]chooses an intermediate ground for
hisMIX machine, and derives runnin g timesfor particular
implementations of algo-rithms. Between the two extremes,
resultscan differ by more than simply a constantscale factor. In
choosing a model of com-putation, we try to balance realismagainst
mathematical tractability. How-ever, our experience is that results
can beuseful despite what appear to be overlysimplified
computational models.Rather than finding exact runningtimes on
particular machines, most anal-yses count only certain ~elementary"
op-erations. The complexity measure reportsthe asymptotic growth of
this operationcount. Mathematicians have used variousnotations for
such "order" results, as re-ported by Knu th [47], who suggests
thefollowing generally accepted version. Toexpress the fact tha t a
function g ( n ) growsasymptotically no faster than anotherfunction
/~n), we write g ( n ) = O(~n)),which reads, "of order at most
f(n)". It ishelpful to regard O(f (n ) ) as the set offunctions
with the property that for anyfunction g ( n ) in O(f (n ) ) there
is a constantc such that g(n ) ~ c f (n ) . Similarly, ~(f(n))Cof
order at least f(n)") is the set of func-tions with the property
that for any func-tion g ( n ) in ~(f(n)) there is a constant c
>0 such that g ( n ) > - c f ( n ) . Finally, 0(f(n))Cof
order exact ly f(n)") is the set of func-tions with the property
that for any func-tion g ( n ) in #(/~n)) there are constants
c1>0 and c2 such t ha t cgC(n)
-
8/9/2019 Analysis of Discrete Algorithms
5/23
A n a l y s i s T e c h n i q u e s f o r D i s c r e t e A l g
o r i t h m s 295importance in graph algorithms as well.Determining
if a graph with n nodes isplanar can be done in #(n) time if
thegraph is represented by adjacency lists,but requires ~(n 2) time
i f the representa-tion is an adjacency matrix [72]. In con-trast
with the unary/binary choice for therepresentation of integers,
both of thesegraph representations seem like reasona-ble choices,
and indeed each is appropriatefor various problems.These points
illustra te tha t the problemsize must be explicitly defined in
eachcase in order for results to have any mean-ing. For example, in
sorting problems, nis the number of items to be sorted; forgraph
problems, it may be the number ofvertices. Describing the problem
size mayeven be more convenient if two or moreparameters are used,
for example thenumber of edges and the number of ver-tices in a
graph. If a graph has V verticesand E edges, then it is clear what
is meantby "an algorithm which requires O ( V +E) steps."For the
results to be of practical interest,definitions of both the measure
of problemsize and the measure of computing timeshould be closely
related to the well-de-fined meanings which these terms havefor
actual machines. Thus, the random-access machine model is more
commonlyused than the Turing machine model forcalculations on most
computers. On theother hand, the Turing machine is usefulin
theoretical studies and in modelingcomputations which are
tape-bound. Aho,Hopcroft, and Ullman [1] present a goodaccount of
the similarities and differencesbetween these two models of
computation.2 . L O W E R B O U N D S
. . abou ndin g i n t n tu i tmns w~thou tmet hod . . . -George
Santayana
The more difficult of the bounds on prob-lem complexity is the
lower bound. Thereis no algorithm to analyze, few generalprinciples
to apply; proofs of results inthis area often require outright
cleverness.The results must apply to any algorithm,
including undiscovered ones. Still a fewtechniques have been
found useful, andothers are promising.T r i v i a l L o w e r B o u
n d sThe most obvious, and also the weakest,method produces what
are appropriatelycalled trivial lower bounds. The methodconsists of
simply counting the number ofinputs that must be examined and
thenumber of outputs tha t must be produced,and noting that any
algorithm must, atleast, read its inputs and write its
outputs.There are many examples of the use ofsuch a technique. One
interesting graphproblem is the single-source shortest-pathproblem:
given a directed graph G withnonnegative edge weights, and a
distin-guished vertex v, find the minimum-weight path from v to
each other vertex ofG. Dijkstra [14] gives an algorithm forthis
version of the problem, where all edgeweights are known to be
nonnegative. Amore interesting variation allows nega-tive-weight
edges but no negative-weightcycles; by definition of the problem,
acorrect algorithm must be able to detectnegative-weight cycles.
Let n be the num-ber of vertices of G. Then there may be asmany as
n ( n - 1) edges inG, and anyalgorithm for solving the modified
problemmust inspect each of them. If some edgewere ignored by any
algorithm, we couldchange its weight so that a shortest pathor a
negative-weight cycle would bemissed and force the algorithm to
give awrong answer Hence there are inputswhich require ft(n 2) time
for any algo-rithm to solve the modified single-sourceshortest-path
problem.Similarly, multiplication of a pair of n n matrices
requires tha t n 2 outputs beproduced, and is therefore 12(n2).
This saysnothing about the number of multiplica-tions required to
solve the problem, butonly tha t s o m e operation (namely
output)must be performed f~(n 2) times; therefore,the dominant
operation must be performedat least that many times.Trivial lower
bounds are genera lly easyto come by and, therefore, are of
lessinterest than sharper bounds which can
C o m p u t i n g S u r v e y s , V o l . 9 , N o . 4 , D e c e
m b e r 1 9 7 7
-
8/9/2019 Analysis of Discrete Algorithms
6/23
2 9 6 B. Weides o m e t i m e s b e p r o v e d b y m o r e s o
p h is ti -c a t e d m e t h o d s . I n m a n y c a s e s , h o w
e v e r ,t r i v i a l b o u n d s a r e t h e o n l y l o w e r b
o u n d sa v a il a b le . B e c a u s e t h e y a r e u s u a l l
y e a s yt o p r o v e , t h e y s h o u l d b e t r i e d b e f o
r e m o r ed i f f ic u l t t e c h n i q u e s a r e a p p l i e d
.I n fo r m a t i o n - T h e o r e t ic B o u n d s a n d D e c i
s io nT r e e sS e v e r a l a u t h o r s h a v e u s e d a r g u
m e n t sf ro m i n f o r m a t i o n t h e o r y t o sh o w t h a
t a n ya l g o r i t h m f or s o lv i n g so m e p r o b l e m m u
s td o so m e m i n i m a l a m o u n t o f w o r k . T h em o s t
u s e f u l p r i n c ip l e o f t h i s k i n d i s t h a tt h e o
u t c o m e o f a c o m p a r i s o n b e t w e e n t w oi t e m s
c o n t a i n s a t m o s t o n e b i t o f i n f o rm a -t i o n (
w h e r e ~ b i t " d e n o t e s t h e v a l u e s 0 o r1 ). H e n
c e , i f t h e r e a r e m p o s s i b l e i n p u ts t r in g s ,
a n d a n a l g o r i t h m p u r p o r t s t o i d e n -t i f y w
h i c h o n e i t w a s g i v e n s o l e l y o n t h eb a s i s o
f c o m p a r i s o n s b e t w e e n i n p u t s y m -b o ls , t h
e n [ lo g m ] c o m p a r i s o n s a r e n e e d e d .T h i s i s
b e c a u s e [ lo g rn ] b i t s a r e n e c e s s a r yt o s p e
c i f y o n e o f t h e m p o s s i b i l i t ie s ( ins t a n d a
r d b i n a r y n o t a t i o n , f o r e x a m p l e ; f o rt h i
s r e a s o n , a l l l o g a r i t h m s i n t h i s p a p e ra r
e t o t h e b a s e 2 ).T h e b e s t - k n o w n e x a m p l e o f
a l o w e rb o u n d o n c o m p u t a t i o n a l c o m p l e x i
t y f r o mi n f o r m a t i o n t h e o r y i s f or t h e p r o b
l e m o fs e a r c h i n g a n o r d e r e d t a b l e w i t h n e
l e -m e n t s f o r t h e p o s i t io n o f a p a r t i c u l a r
i t em .T h e r e a r e n p o s s i b l e o u t c o m e s , s o u n
i q u ei d e n t i f i c a t io n o f a n i n d e x i n t h e t a b
l e r e -q u i r e s [ lo g n ] b i ts . H e n c e , a t l e a s t
[ lo g n ]c o m p a r i s o n s a r e r e q u i r e d a c c o r d i
n g t o t h ep r i n c i p l e m e n t i o n e d a b o v e . A s i
m i l a r a r -g u m e n t a p p l ie d t o t h e p r o b l e m o f
s o r t i n ga l i n e a r l y o r d e r e d s e t g i v e s a l o
w e r b o u n do f l l ( n l o g n ) f o r t h a t p r o b l e m .T
h e s a m e b a s i c p r i n c ip l e o f t e n a p p e a r si n a
d i f f e r e n t g u i s e i n l o w e r b o u n d p r o o f sf o
r c o m p a r i s o n - b a s e d p r o b l e m s . F o r e x -a m
p l e , K r m t h [4 6] u s e s a ~ d e ci s io n tr e e "m o d e l
f o r t h e s o r t i n g p r o b l e m , i n w h i c ha n y s o r
ti n g a l g o r i t h m c a n b e v i e w e d a s ab i n a r y t r
e e . E a c h i n t e r n a l n o d e o f t h e t r e er e p r e s
e n t s a c o m p a r i so n b e t w e e n t w o in -p u t e l e m
e n t s , a n d i t s t w o s o n s c o r r e s p o n dt o t h e t
w o p o s s i b l e c o m p a r i s o n s w h i c h a r em a d e n
e x t , d e p e n d i n g o n t h e o u t c o m e o ft h e p r e v
i o u s c o m p a r i s o n . E a c h l e a f n o d e
s p ec if ie s a n i n p u t p e r m u t a t i o n t h a t i su
n i q u e l y i d e n ti f i e d b y t h e o u t c o m e s o f t h
ec o m p a r i s o n s a l o n g t h e p a t h f r o m t h e r o o
to f t h e t r e e . S i n c e t h e r e m u s t b e n! l e a fn o
d es , s o m e p a t h o f t h e t r e e m u s t b e o fl e n g t h
a t l e a s t [ lo g n !] T h i s f o l l o w s b e c a u s et h e
n u m b e r o f n o d e s a t a n y l ev e l o f ab i n a r y t r e
e c an b e a t m o s t t w i c e t h e n u m -b e r o n t h e p r e
v i o u s l e v e l . T h e w o r s t c a s eo f a n y s o r t in g
a l g o r i t h m m u s t t h e r e fo r eb e f ~ ( lo g n !) = f ~
( n l o g n ) .R e i n g o l d [5 7] h a s e x t e n d e d t h i s
a p -p r o a c h t o a l l o w f o r c o m p a r i s o n s b e t w
e e nf u n c ti o n s o f t h e i n p u t s , r a t h e r t h a n s
i m p l yb e t w e e n i n p u t v a l u e s t h e m s e lv e s . H
es h o w s , fo r e x a m p l e , t h a t d e c id i n g w h e t h
e rt w o s e t s o f n r e a l n u m b e r s a r e i d e n t i c a
lr e q u i r e s l ~ (n l o g n ) c o m p a r i s o n s , e v e n i
fc o m p a r i s o n s b e t w e e n l i n e a r f u n c t io n s o
ft h e i n p u t s a r e a l l o w e d . S i m i l a r l y , D o b
k i na n d L i p t o n [1 5] s h o w t h a t t h e ~ e l e m e n tu
n i q u e n e s s " p r o b l e m ( d e t e r m i n i n g a m o n
gn re a l n u m b e r s w h e t h e r a n y t w o n u m b e r sa r
e e q u a l ) r e q u i r e s l ) ( n l o g n ) s t e p s , e v e
ni f c o m p a r i s o n s b e t w e e n l i n e a r f u n c t i o
n s o ft h e i n p u t s a r e a l l o w e d .O r a c l e sK n u t
h [46 ] p o i n t s o u t t h a t a b o u n d c a n b eo b t a i n
e d f o r t h e p r o b l e m o f m e r g i n g t w oo r d e re d l
is t s b y a n o t h e r t e c h n i q u e w h i c hh e c a l ls t
h e c o n s t r u c t i o n o f a n ~ 'o ra cl e" , o r~ a d v e r
s a ry " . A n o r a c l e is a f i e n d i s h e n e m yo f a n a
l g o r i th m w h i c h a t e v e r y o p p o r tu -n i t y t r i
e s to m a k e t h e a l g o r i t h m d o a sm u c h w o r k a s p
o s s i b l e . C o n s i d e r c o m p a r i -s o n - b as e d a l
g o r it h m s f or m e r g i n g t h e t w ol i s t s A 1 < A 2
< < A , a n d B 1 < B 2
-
8/9/2019 Analysis of Discrete Algorithms
7/23
Analysis Techniques for Discrete Algorithms 297m u s t p r o d u
c e th i s o u t p u t i f i t w o r k sp r o p e r l y .S u p p o
s e t h a t o n e o f t h e c o m p a r i so n sb e t w e e n a d j
a c e n t e l e m e n t s f r o m t h i s f i n a ll i s t h a d n
o t b e e n m a d e d u r i n g t h e c o u r s eo f e x e c u t i
o n o f t h e a l g o r it h m ; s a y , A,:B2.T h e n t h e c o n
f i g u r at i o n :
BI
-
8/9/2019 Analysis of Discrete Algorithms
8/23
298 B. We~deu p o f k . I n f a c t , f o r c e r t a i n c o m
p u t a t i o n st h e s p e e d u p i s e v e n l e s s ; f o r a
d d i n g u p al i st o f k n u m b e r s , i t is o n l y k / lo g
k . T h el a c k o f a g o o d m o d e l f or p a r a l l e l c o m
p u t a -t i o n h a s h i n d e r e d f u r t h e r d e v e l o p
m e n t o fw a y s o f d e c o m p o s i n g p r o b l e m s f o r
p a r a l l e ls o l u t i o n s a n d p r o v i n g b o u n d s s
u c h a st h e s e , e v e n t h o u g h t h e p r o s p e c t o f
i n ex -p e n s i v e p ar a l l e l h a r d w a r e c o m p e l s
u s t os t u d y s u c h a l g o r i th m s . L a w l e r [51 ] e x
-p r e s s e s c o n f i d e n c e t h a t s u c h g r a p h - t h
e o -r e t ic a r g u m e n t s w i l l c o n t i n u e t o p r o v
eu s e f u l i n d e m o n s t r a t i n g l o w e r b o u n d s ,
a n dr e c e n t r e s u l t s ( s ee , f o r e x a m p l e , [7
5]) s h o wt h i s o p t i m i s m t o b e w e l l -f o u n d e d
.A n i n t e r e s t in g l o w e r b o u n d w h i c h re -l a t e
s t o p r o b l e m r e p r e s e n t a t i o n s i s t h e s o -c
a l l e d A a n d e r a a - R o s e n b e r g c o n j e c t u r e[
5 9 ] . S i m p l y s t a t e d , i t a s s e r t s t h a t d e t e
c t -i n g a n y n o n t r i v i a l m o n o t o n e g r a p h p r
o p -e r t y ] r e q u i r e s ~ ( n 2) s t e p s i f t h e g r a p
h i sr e p r e s e n t e d b y a n n n a d j a c e n c y m a t r i
x .A p r o o f o f t h i s c o n j e c t u re b y R i v e s t a n
dV u i l l e m i n [ 5 8 ] i s b a s e d o n a d e c i s i o n t r
e em o d e l fo r t h e e v a l u a t i o n o f B o o l e a n fu n
c -t io n s a n d o n p r o p e r ti e s o f p e r m u t a t i o ng
r o u p s .
A n o t h e r n e w a p p r o a c h u s e s t h e o r e m sf ro
m c o m p l e x a n a ly s i s . S h a m o s a n d Y u v a l[67 ] s
h o w t h a t f i n d i n g t h e a v e r a g e o f a llt h e i n t
e r p o i n t d i s t a n c e s f o r n p o i n t s i n t h ep l a
n e r e q u i r e s ~ ( n 2) s q u a r e - r o o t o p e r a-t io n
s . T h e i r p r o o f i s b a s e d o n t h e a m b i g u -i t y
o f t h e s q u a r e - r o o t f u n c t i o n . T h e p r i-m a r
y s i g n i fi c a n c e o f t h i s r e s u l t i s t h a tw h i l
e i t h a d p r e v i o u s l y b e e n a l m o s t i m p o s-s i b
le to o b t a i n l o w e r b o u n d s e x c e p t f o r t h ef o
u r c o m m o n a r i t h m e t i c o p e r a t i o n s a n df o r
c o m p a r i s o n s , t h e n e w a p p r o a c h a p p l i e st
o a n y m u l t i p l e - v a l u e d f u n c t i o n s u c h a ss
q u a r e r o o t , i n v e r s e t r i g o n o m e t r i c s , a n
dl o g a r i t h m s .I t r e m a i n s t o b e s e e n w h e t h e
r t h e s e a n do t h e r t r i c k s w i l l b e a p p l i c a b
l e t o e n o u g hp r o b l e m s t o b e c a l l e d " m e t h o
d s " f o r p r o v -i n g lo w e r b o u n d s . A t p r e s e n t
, n o n t r i v i a lr e s u l t s a n d g e n e r a l t e c h n i
q u e s a r e q u i t es p a r s e .' A g raph property is nontm
vlal ]f at least on e, butn o t all, graphs have it. It ]s m onoton
e ]f addingnew edges to the grap h does not change the prop-erty. T
hus, for exam ple, nonp lanam ty is a n o n t r i v i a lm o n o t
o n e property.
3 . U P P E R B O U N D SMethod is good ~n all things.Order
governs the world.- J o n a t h a n S w i f t
T h e r e a r e t w o p o w e r f u l m e t h o d s f o r p r o
v -i n g u p p e r b o u n d s b y a n a ly z i n g t h e w o r st
-c a s e b e h a v i o r o f a n a l g o r it h m : c o u n t i n
gi n s t r u c t io n s , o r s o lv i n g r e c u r r e n c e r e
la -t io n s . E i t h e r a p p r o a c h m a y r e q u i r e i d
e n -t i f ic a t i o n o f a w o r s t - c a s e i n p u t . A t h
i r da l t e r n a t i v e , u s i n g b r u t e fo r c e c o m p u
t i n gp o w e r t o f i n d b o t h a w o r s t - c a s e a n d a
na s s o c i a t e d o p t i m a l a l g o r i t h m , s o m e t i
m e sw o r k s .I d e n t i f y i n g a W o r s t C a s eI n s e e
k i n g a n u p p e r b o u n d o n p r o b l e mc o m p l e x i t
y , t h e f i r s t t a s k i s t o i d e n t i f y a" w o r s t ca
s e " , i .e . , a n i n p u t o f s i z e n w h i c hm a x i m i z
e s t h e a m o u n t o f w o r k t h e a lg o -r i t h m m u s t d
o f or t h a t v a l u e o f n . T h ea l g o r i t h m w i t h t h
i s i n p u t t h e n d e f i n e sU(n), p r o v i d e d t h a t n
o o t h e r a l g o r i t h m ' sw o r s t - c a s e b e h a v i o
r i s b e t t e r .W h e n t h e a l g o r i t h m ' s w o r k i s
t h e s a m ef o r a l l i n p u t s o f s iz e n , f i n d i n g t
h e w o r s tc a s e i s e a s y . T h i s p h e n o m e n o n i s
e a s i l yr e c o g n i ze d w h e n t h e f lo w o f c o n t r o
l d o e sn o t d e p e n d o n th e d a t a . T h e n e v e r y c a
sei s a " w o r s t " o n e , b e c a u s e a l l c a s e s a r e t
h es a m e f r o m t h e s t a n d p o i n t o f t h e a n a l y si
s .F o r e x a m p l e , a s t r a ig h t f o r w a r d a l g o r i
th mf o r f i n d i n g t h e l a r g e t e l e m e n t i n a s e t
S i s :
p r o c e d u r e largest (S);b e g i nb~g := first elemen t in
S,for each remaining element x o f S do b~g :=max(b,g,
x);return(b,g)end;C l e a r l y , f o r e v e r y s e t S w i t h n
e l e m e n t s ,t h e a l g o r i th m m a k e s n - 1 c o m p a r
i s o n s( a n d t h i s i s o p t i m a l s i n c e t h e l o w e
r b o u n d
i s a l s o n - 1 ). S i m i l a r l y , m u l t i p l i c a t i
o n o ft w o n n m a t r i c e s i n th e c l a s s ic a l w a yt a
k e s 0 ( n 3) s t e p s r e g a r d l e s s o f t h e d a t a .T h
e r e a r e m a n y m o r e e x a m p l e s o f s u c ha l g o r i
t h m s f o r w h i c h i d e n t i f y i n g th e w o r s
tComputing Surveys,Vol. 9, No. 4, December1977
-
8/9/2019 Analysis of Discrete Algorithms
9/23
A n a l y s i s T e c h n i q u e s f o r D i sc r et e A l g o
r i t h m s 299c a s e m e a s y b e c a u s e e v e r y c a s e i
s t h es a m e .S o m e t i m e s d a t a - d e p e n d e n t d e c
is io n s ob -s c u r e t h e f a c t t h a t e v e r y c a s e is
t h e s a m e .M a n y s e a rc h a l g o r i th m s h a v e t h is
f e a tu r e ;f o r e x a m p l e , t h e b i n a r y s e a r c h a
l g o r i t h ms e a r c h e s t h r o u g h a s o r te d a r r a y
A c o n si st -i n g o f n e l e m e n t s w i t h i n d ic e s ~ t
h r o u g h j( w h e r e n - j - ~ + 1 ), f o r t h e p o s i t i o
n o f ap a r t i c u l a r " k e y " i te m . F o r s im p l i c i
ty , a s -s u m e t h a t t h e k e y it e m i s i n t h e t a b le
.T h e p r o c e d u r e b e l o w r e t u r n s t h e i n d e x i
nA o f t h e k e y i t e m :
p r o c e d u r e bmsearch (A,L3,key);i f L = jt h e n r e t u r
n (Delse
b e g i ni n t e g e r mLddle = L(L+j)/2J ;i f key
-
8/9/2019 Analysis of Discrete Algorithms
10/23
30 0 B. WeideCounting OperationsE a c h o f t h e a b o v e e x
a m p l e s l e a d s d i r e c t l yt o a w a y o f c o u n t in g
t h e n u m b e r o f s t ep sr e q u i r e d i n t h e w o r s t c
a se . O f t e n , h o w -e v e r , t h i s c o u n t i n g s t a g
e i s a t l e a s t a sd i f f ic u l t a s f i n d i n g a w o r s
t - c a s e i n p u t . I nf a c t , i t s e e m s v e r y s i m i
l a r t o t h e p r o b l e mo f f i n d i n g i n v a r i a n t r
e l a ti o n s f o r p r o o fs o fp r o g r a m c o r r e c t n e
s s .C o n s i d e r a n a l g o r i t h m f o r f i n d i n g t h
ec o n v e x h u l l o f n p o i n t s i n t h e p l a n e . Ac o r
r e c t s o l u t i o n ( b y d e f i n i t i o n o f t h e c o n
-v e x h u ll ) r e q u i r e s t h a t t h e v e r t i c e s o f t
h eh u l l b e o u t p u t i n t h e o r d e r th e y w o u l d b
ee n c o u n t e r e d b y t r a c in g t h e c o n v e x h u l li
n , s a y , a c l o c k w i s e d i r e c t io n . S h a m o s [6
5]s h o w s t h a t a l o w e r b o u n d i s f ~( n lo g n ) ,a n
d G r a h a m [26 ] h a s g i v e n a n a l g o r i t h mw h i c h
a c h ie v e s th i s. G r a h a m ' s m e t h o d o p-e r a t e s
i n t h r e e s t a g e s . F i r s t , a l l n p o i n t sa r e c
o n v e r t e d t o p o l a r c o o r d i n a t e s a b o u ts o m
e p o i n t w h i c h i s i n t e r i o r t o t h e h u l l .S i n
c e a n y p o i n t w h i c h l i e s w i t h i n t h et r i a n g
l e f o r m e d b y t h r e e n o n c o l l i n e a rp o i n t s o
f t h e s e t s e r v e s a s s u c h a n o r ig i n ,t h e c o n v
e r s i o n s t e p c a n b e d o n e i n O(n)t i m e . N e x t , t
h e p o i n t s a r e s o r t e d b y p o l a ra n g l e i n O(n l
o g n ) t i m e .F o r t h e f i n a l p h a s e , e a c h p o i n
t i s o ne i th e r o n e o r tw o l i s t s : t h e UN CON SI DER
EDl i s t a n d t h e ( c i r c u l a r ) T EN TA T IV E l i s t .
T h eUN CON SIDER ED l i s t i s t h e o u tp u t o f t h es o r t
i n g s t a g e , w h i l e t h e T EN TA TIV E l i s t i si n i t
i a l l y e m p t y . A s c a n i s t h e n m a d et h r o u g h t
h e U N CO N SID ER ED p o i n t s . F o re a c h n e w p o i n t ,
t h e a l g o r i t h m d e t e r m i n e sw h e t h e r t h e p a
t h t h r o u g h t h e l a s t t w op o i n t s o n t h e T EN TA
TIV E l i s t a n d t h e nt h r o u g h t h e n e w p o i n t t u
r n s t o t h e r i g h to r t h e l e ft . I f i t t u r n s t o t
h e r i g h t , t h ea l g o r i t h m a d d s t h e n e w p o i n
t to th e T E N-T AT IV E l i s t a n d c o n t i n u e s s c a n n
i n g t h eUNCONSIDER ED l i s t . I f i t t u r n s t o t h e l e
f t ,t h e n t h e m i d d l e p o i n t o f t h e t h r e e c a n
n o tb e o n t h e h u l l , s o t h e a l g o r i t h m d e l e t
e st h e l a s t p o i n t o f t h e T EN TA TIV E l i s t f r o mt
h a t l i s t . A g a i n , a p a t h t h r o u g h t h e l a s tt
w o T EN TA TIV E p o i n t s a n d t h e n e w p o i n ti s c h e
c k e d , a n d t h i s c o n t i n u e s u n t i l a r ig h tt u r
n i s d i s c o v e r e d o r u n t i l t h e r e a r e f e w e rt
h a n t w o p o i n t s o n t h e T EN TA TIV E l is t .W h e n a l
l p o i n t s o n t h e U N C O N S I D E R E D i s th a v e b e e
n s c a n n e d , t h e T EN TA T IV E l i s t
c o n t a i n s t h e p o i n t s o n t h e c o n v e x h u l l
i nt h e p r o p e r o r d e r.I t i s n o t i m m e d i a t e l y
c l e a r h o w l o n gt h i s s t e p m a y t a k e , s i n c e f
o r e a c h n e wp o i n t t h e s c a n c a n a p p a r e n t l y
b a c k u p t ot h e b e g i n n i n g o f t h e T EN T AT IV E l i
s t. Ar e a s o n a b l e ( b u t p e s si m i s ti c ) g u e s s f
or t h ec o m p l e x i t y o f t h e f i n a l p h a s e w o u l d
t h e nb e O ( n2 ), w h i c h i s n o t s u f f i c i e n t fo r o
u rp u r p o s e s . I t i s n o t p a r t i c u l a r l y e a s y
t oi d e n t i fy a w o r s t - c a se i n p u t b e c a u s e o f
t h et r a d e o f f s i n v o l v e d .G r a h a m n o t i c e d t
h a t a l l t h e w o r k i sd o n e i n c r e a t i n g a n d t e
s t i n g p a t h s b e -t w e e n t h r e e c o n s e c u t i v e
p o i n t s , a n d t h a tt h e t o t a l t i m e i s p r o p o r
t io n a l t o t h e n u m -b e r o f t i m e s t h i s m u s t b e
d o ne . E v e r yt i m e s u c h a p a t h i s c r e a t e d a n d
t e s t e d ,e i t h e r a n e w p o i n t i s a d d e d t o t h e
T E N T A -T IV E l i s t , o r a p o i n t i s d e l e t e d f r o
m i t . T h en u m b e r o f p o i n t s a d d e d c a n n o t e x
c ee d n,a n d t h e n u m b e r d e l e te d a ls o c a n n o t e
x c e e dn b e c a u s e o n c e d e le t e d , a p o i n t i s n e
v e rc o n s i d e re d a g a i n . H e n c e , a n e w t h r e e
-p o i n t p a t h i s c r e a t e d a t m o s t 2 n ti m e s .T h
e a r g u m e n t i s t a n t a m o u n t t o f in d i n ga n i n v
a r i a n t r e l a t i o n a m o n g t h e r e l e v a n tq u a n
t i t i e s: P , t h e n u m b e r o f p a t h s c o n s id -e r e
d s o f a r; A , t h e n u m b e r o f p o i n t s w h i c hh a v e
b e e n a d d e d t o t h e T EN TA TIV E l i s t S Of a r ; a n d
D, t h e n u m b e r w h i c h h a v e b e e nd e l e t e d f r o m
t h e T EN T AT IV E l i s t SO f a r .T h e q u a n t i t y P - A
- D , c a l c u l a t e d a f t e ra n y p a t h i s c o n s i d e
re d , i s c o n s t a n t . H e n c e ,P =O(A +D) , ands i nceA
+D
-
8/9/2019 Analysis of Discrete Algorithms
11/23
A n a l y s i s T e c h n i q u e s f o r D ~ sc re te A l g o r
i t h m s 301Another version of the arg ume nt is usedby Hopcroft
and Ull man [32] to analyzethe behavior of a data structure and
asso-ciated algorithms for set manipulation. Asequence of n UNION
and FIND ins truct ionsis input; a UNION inst ruc tion requests th
attwo disjoint sets be merged, and a FIND
asks for the name of the set containing aparticular element.
Such a sequence canbe processed in O ( n G ( n ) ) time, where G (
n ) .is the least k such that the k-th iteratedlogarithm of n does
not exceed one. Theproof of this bound relies on an ~account-ing
trick" in ana lyz ing the FIND algorithm.The tota l cost of all
FIND inst ruc tions isdivided between the instructions them-selves
and the elements inspected by theinstructions. Each account is then
ana-lyzed separa tely and shown to be chargedO ( n G ( n ) ) units;
thus the total is O ( n G ( n ) )for n FIND instructions, which
dominatesthe time for the UNmN instructions. Thereader is urged to
consult Hopcroft andUl lman [32] or Aho, Hopcroft, and Ul lman[1]
for more detail s.R e c u r r e n c e s
Although analysis of worst-case behav-ior by directly counting
the number ofsteps is greatly simplified by a concretedescription
of the algorithm, it is not al-ways necessary to be so explicit. In
deriv-ing recurrence relations for solution times,it is sometimes
more convenient to thinkin abstract terms about what the algo-rithm
does. This is especially true whenthe algorithm itself is not
writte n recur-sively.In accordance with established usage,let the
running time be denoted by T ( n ) .It is sometimes possible to
find a recur-rence rel ation (difference equation) for T ( n )and
to solve it exactly-or even just ap-proximately, concluding that T
( n ) isO ( f ( n ) ) , for example-to discover theworst-case
behavior of an algorithm.Recall the first example of the
previoussubsection, where the problem was to findthe l argest
element in a set S of n ele-ments. Although the a lgorithm is not
writ-ten as a recursive procedure, it can never-theless be viewed
as finding the largesteleme nt of a set S' consisting of the first
n
- 1 elements, then comparing the resultto the nt h element of S.
The recurrenceobtained isT ( n ) = T ( n - 1) + 1 for n > 1,T(1)
= 0
where the initial condition is zero becauseno comparisons are
needed to find themax imu m element of a singleton set. Thesolution
to this recurrence is clearly T ( n )= n - 1, the same resu lt as
before.Next consider the binary search algo-rithm, where it is true
thatT ( n ) 1,T(1) = 0.
Again, the initial condition is zero becausethe process is
finished without any morecomparisons when only one element re-mains
in the array. The recurrence isdiscovered by recognizing th at in
the worstcase, one comparison is used to dete rminewhich remaining
part to search recur-sively, so th at the t otal nu mbe r of
compar-isons T ( n ) is the s um of this comparisonplus the number
T ( [ n / 2 ] ) required to findthe key in an array essentially
half aslarge. Since the r emainin g part can neverbe larger than
[n/2] whether n is odd oreven, we use '~' rather than ~='.
Thesolution T ( n ) < - [log n] can be discoveredby computing
the value of T ( n ) for thefirst several values of n, and can th
en beproved by induction. As menti oned before,the binary search
algorithm is optimal,since this solution coincides wit h the
lowerbound.
A more complicated recurrence resultsfrom analyzing the worst
case of Quick-sort. Here, the equati on isT ( n ) = T ( n - 1) + P
( n ) + C ( n )forn > 1,T(1) = T(0) = 0,
where C ( n ) is the number of comparisonsrequired to choose an
element x from S,and P(n) is the nu mber needed to partitionS on
the chosen elemen t x. Since we arecounting only comparisons, C ( n
) = 0 andP ( n ) = n - 1; thus,
T ( n ) = T ( n - 1)+n - 1 fo rn> l ,T(1) = T(0) =
0,Computing S u r v e y s , V Q I. 9 , N o . 4 , D e c e m b e r 1
9 7 7
-
8/9/2019 Analysis of Discrete Algorithms
12/23
302 B. Weidefor which the exact solution is T(n) = n(n- 1)/2. Of
course, i t is clearly not sufficientto choose x arbitrarily if a
worst case of0(n 2) must be avoided. Rather, x shouldpartition S
into approximately equalparts. Aho, Hopcroft, and Ullman [1]
callthis the ~principle of balancing. " It can beaccomplished by
choosing x as the med ianelement of S, whereupon the
recurrencebecomes
T(n) 1,T(1) = T(0) = 0.As in the case of binary search , ~---'
ra therthan '= ' is used because n may be odd andbecause the
partitioning element is al-ready in place and need not be
consideredin the recursive step. T(n) still providesan upper bound
since neither remaininglist can contain more than half the
ele-ments. It is convenient to thi nk of T asbeing defined not only
for integer argu-ments, but on the enti re real line. Since Tis a
nondecreasing function, T([xJ)- T(x),so the inequality remains
valid if In/2] isreplaced by n/2. This step greatly simpli-fies the
task of solving the recurrence.Now, P(n) = n - 1 as before; however
,C(n) is no longer zero but the number ofcomparisons necessary to
find the medi anof n elements. Blum et al., [5] presen t
analgorithm which finds the median in atmost 5.43n comparisons, and
Schonhage,Paterson, and Pippenger [63] have an al-gorithm which
uses at most 3n compari-sons. Taking C(n) 1,T(1) = T(0) = 0,for
which the solution is T(n) 1,T(1) = 1.
Here, the recursive step requires sevenapplications of the
algori thm to (n/2) x(n/2) matrices, plus 15 additions of
suchmatrices. Each matrix addition takes (n/2) ~ scalar operations.
The init ial conditionis obvious, since multip lication of two 1 x1
matrices consists of a single scalar mul-tiplication. The solution
is T(n) = 6n g7 -5 n 2 ; t h u s T ( n ) i s O (n gT) = O(n 2"Sl) ,
com-pared to O(n 3) for the classical method.Because of the factor
of 6, though, theclassical a lgori thm (which takes 2n 3 - n
2operations) is still faster for n less thanabout 300. By using a
hybrid schemewhich uses Strassen's algorithm for largematrices and
the classical algorithm forsmaller ones, this crossover point can
be
C om put ing S u r ve ys , Vo l 9 , No 4 , D e c e m b e r 1
977
-
8/9/2019 Analysis of Discrete Algorithms
13/23
An a ly s i s Te c h n iq u e s f o r D i sc r et e A lg o r i t
h m s 303reduced (for a real implementation) toabout n = 40
[11].Simply finding a recurrence is only partof the problem; the
other half , of course, issolving it. It is relatively easy to find
anupper bound on the solution by simplyguessing a solution and then
trying it.For example, given the recurrence
T(n) = 2T(n/2) + n log nwith some initial condition T(1), we mig
htguess that T( n ) should be no larger thanO(n2). If we assume
that T(n ) = cn 2 andcan show tha t the right- hand side of
therecurrence is at most cn 2 + lower-orderterms , then O(n 2) is
an upper bound onT(n ) . That this is true for the presentexample
is easily verified.A better guess in this case is that T( n )is O(n
log2n); this means t ha t our guess ofthe value of T( n ) is cn
log2n, resulting in:
T(n ) = 2T(n /2 ) + n log n= cn log~(n/2) + n log n= en logan +
(1 - 2c) n log n + cn
so that O( n logZn) is an upper bound forT(n) . In fact, since
the coefficient of nlog2n is the same on both sides of theequation,
T(n) = O(n log2n).This computation can be extended tofind the exact
solution. Suppose that T( n )is a linear combination of linearly
inde-pendent functions, the dominant one ofwhich is n log2n.
Substituting cn log2ninto the recurrence gives rise to terms inn
log n and in n, which appear on theright -hand side but not on the
left. Conse-quently, T( n ) must also have ter ms a n logn + bn,
which, when expanded nn theright, produce no new lower-order
terms.Now it is a simple ma tt er to equate coeffi-cients of like
functions to get the solution:T(n) = (n log2n + n log n)/2 +
T(1)n.More powerful techniques must some-times be applied. Gene rat
ing functions (Z-transforms), which are also of value insolving
problems associated with average-case analysis, are among the most
usefulof these tools. K nu th [43], Liu [52], andKleinrock [42]
give excellent accounts ofhow to use this method. Some
relativelyeasy recurrences can also be solved byreferring to
standard formulas (see, for
example, [13] on difference equations),while at least references
to others can befound by iterating the recurrence to findthe first
few terms and then looking upthe sequence in Sloane [68].T h e B r
u t e F o r c e M e t h o dEven though the method to be
describedhere is not often practical, it is interestingbecause it
is possible at all only with theaid of high-speed computers and,
there-fore, could only recently have been at-tempted. The question
of how to sort usinga mi ni mum number of comparisons is
con-sidered in detai l by Knut h [46], who pointsout tha t the
merge-insertion algorithm ofFord and Johnson [21] is optimal for
n
-
8/9/2019 Analysis of Discrete Algorithms
14/23
304 B. We~det e c h n i q u e s r e p o r t e d i n t h e p r e
v i o u s s e c t i o na r e s ti ll a p p l i c a b le , a l t h o
u g h s o m e o f t h er e c u r r e n c e s a r e t o u g h e r t
o h a n d l e a n dt h e r e f o r e s t r o n g e r s o l u t i o
n m e t h o d s m a yn e e d t o b e u s e d .
P r o s an d C o n s o f A v e r a g e - C a s e A n a l y s i
sT h e p r i m a r y r e a s o n f o r a n a l y z in g t h e b e
-h a v i o r o f a l g o r i t h m s " o n t h e a v e r a g e " is
,o f c o u r s e , t h a t a w o r s t c a s e m a y a r i s e s or
a r e l y ( p e r h a p s n e v e r ) i n p r a c t i c e t h a ts
o m e o t h e r c o m p l ex i ty m e a s u r e w o u ld b em o r e
u s e f u l . A n a l t e r n a t i v e t o w o r s t - c a s ea n
a l y s i s t h a t i m m e d i a t e l y co m e s to m i n di s s
o m e s o r t o f a v e r a g e - c a s e a n a l y s i s .R a t h
e r t h a n t r y t o d e f i n e a n d a n a l y z e ap a r t i c
u l a r c a s e w h i c h i s s o m e h o w ~ a v e r-a g e , " t h
e a p p r o a c h i s t o s i m u l t a n e o u s l ya n a l y z e
a l l c a s e s a n d t o w e i g h t t h e i n d i -v i d u a l c
a s e c o m p l e x i t i e s w i t h t h e a p p r o -p r i a t e
p r o b a b i l it i e s o f e a c h c a s e o c c u r r i n g .O b
v i o u s l y , t h i s c o m p l i c a t e s t h e m a t h e -m a
t i c s c o n s i d e r a b ly . I f t h i s w e r e t h e o n l yo
b j e c t i o n t o d o i n g a v e r a g e - c a s e a n a l y s i
s ,a l l t h a t w o u l d b e r e q u i r e d w o u l d b e m o r
es o p h i s t i c a t e d t o o l s , w h i c h c o u l d b e d i
s -c u s s e d in d e t a i l h e r e . H o w e v e r , m o r e s e
ri -o u s q u e s ti o n s h a v e b e e n r a i s e d w h i c h t
e n dt o c a s t c o n s i d e r a b l e d o u b t o n t h e e n t
i r ev e n t u r e ; t h i s i s t h e m a i n r e a s o n f o r n
o tg o i n g i n t o m o r e d e t a i le d d e s c r i p ti o n o
f t h em e t h o d s u s e d i n a v e r a g e - c a s e a n a l y
s i s .R a t h e r , t h e r e a d e r is u r g e d t o c o n s u l
t t h eo r i g i n a l s o u r c e s .T h e m o s t i m p o r t a n
t o b j e c t i o n i s t h a t ,t y p i c a l ly , t h e r e is n
o w a y t o i d e n t i f y t h ep r o b a b i l i t y d i s t r i
b u t i o n o v e r a l l p r o b l e mo c c u r r e n c e s . W h
i l e i t m a y b e r e a s o n a b l ei n s o m e s i t u a t i o
n s t o a s s u m e t h a t e v e r yp o s s i b l e a b s t r a c
t p r o b l e m i n s t a n c e i se q u a l l y li k e l y ( fo r
e x a m p l e , t h a t e v e r yi t e m i s e q u a l l y l i k e
l y t o b e t h e k e y i n ab i n a r y s e a r c h , o r th a t e
v e r y p e r m u t a t i o ni s e q u a l l y l i k e l y to b e t
h e i n p u t t o Q u i ck -s o rt , o r t h a t e v e r y n - v e
r t e x g r a p h ise q u a l l y l i k e l y ) , t h i s a s s u m
p t i o n r e a l l yo n l y m a k e s s e n s e i f t h e p r o b
l e m s p a ce i sf in i te . I t c l e a r l y m a k e s n o s e n
s e , f o r e x -a m p l e , t o s a y t h a t e v e r y i n t e g
e r p r o g r a mi s e q u a l l y li k e ly . F u r t h e r m o r
e , e v e n w h e ni t d o es h a v e m e a n i n g , t h e a s s u
m p t i o n o f au n i f o r m d i s t r ib u t i o n o v e r a l l
p o ss ib l e i n -p u t s m a y n o t b e a t a l l r e a l i s t
i c i f w e h a v e
p r i o r k n o w l e d g e a b o u t t h e l i k e l i h o o d
o fv a r i o u s i n p u t s . W h i l e a u n i f o r m d i st r
ib u -t i o n i s n o t t h e o n l y p o s si b le a s s u m p t i
o n , i ti s t h e o n e m o s t o f t e n e n c o u n t e r e d .
E x c e p -t i o n s a r e S p i r a [7 0] o n t h e e x p e c t e
d t i m ef o r f i n d i n g a l l s h o r t e s t p a t h s i n a
g r a p h ,a n d B e n t l e y a n d S h a m o s [4] o n t h e e x
-p e c t e d b e h a v i o r o f s o m e g e o m e t r i c al go -r
i t h m s . I n b o t h o f t h e s e c a se s , t h e o n l yi m p
o r t a n t a s s u m p t i o n s d e a l w i t h i n d e -p e n d
e n c e o f t h e r a n d o m v a r i a b le s i n v o lv e da n d
n o t w i t h t h e i r d i s tr i b u t io n . T h e a s -s u m p
t i o n s m a d e i n a n a l y z i n g s c h e d u l i n ga l g o
r i th m s o r p a r a l l e l c o m p u t a t i o n s o f te ni n
c l u d e e x p o n e n t i a l l y d i s t r i b u t e d o r E r
-l a n g i a n d i s t r i b u t e d s e r v i c e s t i m e s f o
rt a s ks [8 , 2 ] .I n o n e a t t e m p t t o a n s w e r t h i s
o b je c ti on ,Y u v a l [ 79 ] h a s s u g g e s t e d t h a t a
l g o r i t h m sm i g h t ~ r a n d o m i z e " t h e i r i n p u
t s i n o r d e rt o m a k e t h e a s s u m p t i o n a p p e a r
v a l id . H eh a s p o i n t e d o u t t h a t i f s u i ta b l e
r a n d o ms te p s w e r e b e i n g t a k e n a t c e r t a i n s
t a g e s,a n a l g o r i t h m c o u l d h a v e g o o d e x p e c
t e db e h a v i o r f or e v e r y in p u t , a n d t h e r e b y
a s-s u r e g o o d e x p e c t e d - c a s e s o l u t io n t i m
e s r e -g a r d l e s s o f th e p r o b a b i l i ty d i s t r i
b u t i o n
b e i n g a s s u m e d ( s e e a l s o [ 5 5 ] ) . F o r e x a
m -p l e , Q u i c k s o r t c o u ld c h o o se t h e p a r t i t
io n -i n g e l e m e n t r a n d o m l y ( t h i s i d e a w a s o
f -f e r e d b y H o a r e [2 8] i n h i s o r i g i n a l p a p e
ro n Q u i c k s o r t ) . E v e n t h o u g h t h i s m i g h ts e
e m a c a s e o f t h e t a il w a g g i n g t h e d o g ,t h e r e
is s o m e j u s t i f ic a t i o n f o r s u c h a na p p r o a c
h i n t h i s i n s t a n c e . I n o r d e r tom a k e t h e a n a
l y s i s o f t h e a l g o r i t h m t r a c t -a b l e, S e d g e
w i c k [6 4] a s s u m e s t h a t t h e f i le s~o b e s o r t e
d a r e r a n d o m , a n d p r e s e n t se v i d e n c e t h a t
t h e a l g o r i th m w o r k s b e t t e ri f t h e y a r e ! C l
e a r l y , n o t a l l a l g o r i t h m s c a nb e m o d i f ie d
i n t h is m a n n e r . T h e k e y e le -m e n t i n b i n a r y
s e a r c h c a n n o t b e ~ 'r an do m -i z e d " ; i t i s g i v
e n a s t h e s o l e i n p u t . S i m i -l a r l y , t h e a r r
a y i n w h i c h t h e s e a r c h i s t ob e m a d e is c e r t a
i n l y f ix e d d u r i n g t h es e a r c h , s o t h e r e i s n
o r o o m to m a n i p u l a t et h e a l g o r i t h m i n t h i s
w a y .D e s p i t e t h e s h a k y b a s i s f o r a s s u m i n
gr a n d o m i n p u ts , t h e r e s u l t s o f a n a l y s i s u
s-i n g t h i s a s s u m p t i o n m a y b e r e a s o n a b l ea
p p r o x i m a t i o n s f o r o t h e r d i s t r i b u t i o n s
.M o r e o v e r , n o n u n i f o r m d i s t r ib u t i o n s c o
m -p l i c a t e t h e r e c u r r e n c e s .K n o w i n g t h e a
v e r a g e b e h a v i o r o f a n
C o m p u t i n g S u r v e y s , V o l . 9 , N o 4 , D e c e m
b e r 19 7 7
-
8/9/2019 Analysis of Discrete Algorithms
15/23
A n a l y s i s T e c h n i q u e s f o r D i sc re te A l g o r
i t h m s 3 0 5a l g o r i t h m i s h e l p fu l , b u t k n o w i
n g t h e v a r -i a n c e a s w e l l w o u l d b e e v e n m o r
e s o . F e wa v e r a g e - c a s e a n a l y s e s h a v e c o n
s i d e r e dh i g h e r m o m e n t s o f t h e s o l u t io n t i
m e . O n en o t a b l e e x c e p t i o n i s S e d g e w i c k '
s a n a l y s i so f Q u i c k s o r t [6 4] ( s e e a l s o K n u
t h [4 6]), i nw h i c h h e s h o w s t h a t t h e a v e r a g e
n u m b e ro f c o m p a r i s o n s i s a b o u t p = 1 .3 9 n lo
g n ,a n d t h e s t a n d a r d d e v i a t i o n i s a p p r o x
i -m a t e l y ~ = 0 . 6 5n . S i n c e ~ / /z d i m i n i s h e sw
i t h i n c r e a s i n g n , o u r c o n f i d e n c e t h a t t h
ea l g o r i t h m w i l l b e e f f i c i e n t g r o w s w i t h
n .T h i s e x p l a i n s i n p a r t w h y Q u i c k s o r t w o
r k ss o w e l l i n p r a c t i c e . S p i r a [7 0] a l s o f in
d st h e v a r i a n c e o f t h e t i m e r e q u i r e d b y h i
sa l l - s h o r t e s t - p a t h s a l g o r i t h m .S o m e E x
a m p l e s o f A v e r a g e - C a s e A n a l y s isT h e r e i s
a l a r g e a n d g r o w i n g n u m b e r o fa l g o r it h m s t
h a t h a v e b e e n s u b j e c t e d t oa n a l y s i s o f a v
e r a g e - c a s e c o m p l e x it y .W h i l e o n l y a f e w w
i l l b e d i s c u s s e d h e r e ,m a n y o t h e r s c a n b e
f o u n d i n t h e l i t e r a -t u r e .S u r e l y t h e s i n g
l e m o s t c o m p r e h e n s i v ea n a l y s i s o f a n y a l
g o r i t h m i s p r e s e n t e d f o rQ u i c k s o r t b y S e
d g e w i c k in h i s P h D t h e s i s[ 6 4 ] . H e a n a l y z e
s t h e a v e r a g e n u m b e r o fc o m p a r i s o n s , e x c
h a n g e s , a n d p a r t i t i o n i n gs t a g e s t h r o u g
h t h e u s e o f r e c u r r e n c e s , j u s ta s fo r t h e s i
m p l e w o r s t - c a s e a n a l y s i s o ft h e n u m b e r o
f c o m p a r is o n s d e s c r i b e d i nS e c t io n 3 . A l o
n g w i t h S e d g e w i c k 's A p p e n -d i x B , K n u t h [4
3, 4 6] i s a g o o d r e f e r e n c ef o r t h e s e t e c h n i
q u e s .O b v i o u s l y , n o t a l l r e c u r r e n c e s a r
e e a s yt o s o l v e . I n a n a l y z i n g r a d i x e x c h a
n g e s o r t -i n g, K n u t h [4 6] u s e s p r o p e r t i e s o
f t h eg a m m a f u n c t i o n a n d c o m p l e x - v a r i a b
l et h e o r y t o d e r i v e a s y m p t o t i c r e s u l t s f
o r ar e c u r r e n c e t h a t l o o k s f a i r ly s i m p l e ,
b u tc a n n o t b e s o lv e d w i t h t r a d i t io n a l t e ch
-n i q u e s .O n e a l g o r i th m t h a t h a s b e e n a n a ly
z e di n a t l e a s t t h r e e d i f f e r e n t w a y s i s t h
ea l p h a - b e t a s e a r c h a l g o r i t h m f o r g a m et r
e e s . T h e p r o b l e m w h i c h t h e a l g o r i t h ms o l
v e s i s a s e a r c h t h r o u g h a g a m e t r e e .T h e r e
a r e t w o p l a y e r s w h o a l t e r n a t e l ym a k e m o v
e s , t h e f i r s t t r y i n g t o m a x i m i z es o m e fu n c
t i o n o f t h e p o s it io n , t h e o t h e rt r y i n g t o m
i n i m i z e i t . T h e c l a s s i c a l e x a m -p l e i s t h
e g a m e o f c h e s s; a ll s o p h i s t i c a t e d
c h e s s - p l a y i n g p r o g r a m s u s e a l p h a - b e
t as e a rc h . G o o d d e s c r ip t i o n s o f t h e a l g o r
i t h mc a n b e f o u n d i n F u l l e r , G a s c h n i g , a n
dG i l l o g l y [ 2 3 ] a n d i n K n u t h a n d M o o r e[4 8].
F u l l e r , e t al . , a s s u m e t h a t t h e g a m et r e e i
s a c o m p l e t e t r e e w i t h b r a n c h i n gf a c to r N a
n d d e p t h D , a n d t h a t e a c h p er -m u t a t i o n o f t
h e r a n k s o f t h e v a l u e s o f t h el e a f n o d e s i s
e q u a l l y l i k e ly . T h e y d e r i v ee x p r e s s i o n s
f o r t h e p r o b a b i l i t y o f e x p a n d -i n g i n d i v
i d u a l n o d e s , a n d t h e e x p e c t e dn u m b e r o f e
v a l u a t e d b o t t o m p o s it io n s .W h i l e t h e a n s
w e r s i n t h i s c a s e l o o k s i m p l ee n o u g h b e c a
u s e o f c o n c is e n o t a t i o n , t h ea u t h o r s p o i n
t o u t t h e c o m p u t a t i o n a l i n -f e a s i b il i ty o
f c a l c u l a t i n g t h e s e q u a n t i t i e sf or a n y b u
t v e r y s m a l l v a l u e s o f N a n d D .H o w e v e r , t h
e y s u r m i s e f r o m s i m u l a t i o nr e s u l t s t h a t
t h e a v e r a g e n u m b e r o f n o d e se x a m i n e d i s a
b o u t O (N '7 2D ).K n u t h a n d M o o r e [48] m a k e t h e s
a m ea s s u m p t i o n s a b o u t t h e t r e e a n d t h e r an
-d o m o r d e r i n g o f l e a f- n o d e v a l u e s , a n ds h
o w t h a t t h e a v e r a g e b e h a v i o r o f t h ea l g o r
i t h m i s O ( ( N / l o g N ) ' ) . T h e y s u g g e s tt h a t
t h e s i m u l a t i o n r e s u l t s b y F u l l e r , e ta l .
, r e su l t i n a f i t t o N '~2~ b e c a u s e N i s sos m a l l
.N e w b o r n [5 3] u s e s a m o d e l i n w h i c h t h eb r a n
c h ( r a t h e r t h a n t h e n od e ) v a l u e s a r er a n d o
m l y r a n k e d ; h e o b t a i n s r e s u l t s f o rt h e c a
s e s D = 2 , 3 , 4 , a n d i n e a c h c a s et h e y d i f fe r f
ro m t h o s e o f K n u t h a n d M o o r e.T h i s i s y e t a n
o t h e r e x a m p l e o f t h e d e p en d -e n ce o f r e s u l
t s o n t h e a s s u m p t i o n s o f t h em o d e l b e i n g u
s e d i n t h e a n a l y s i s : t h e t h r e ed i f f e r e n t
a n a l y s e s fo r D = 2 g i v e c o m p l e x -i t i e s o f O (
N 144), O ( ( N / l o g N ) 2 ), a n d O ( Nl o g N ) , r e s p e c
t i v e l y .O t h e r e x a m p l e s o f a v e r a g e -c a s e a
n a l y s i si n c l u d e G u i b a s a n d S z e m e r e d i [2
7] o n d o u -b l e h a s h i n g , O ' N e i l a n d O ' N e i l [
54 ] o nB o o l e an m a t r i x m u l t i p li c a t io n , a n d
K n u t h[4 5] a n d F l o y d a n d R i v e s t [ 20 ] o n s e l e
c t i o n .5 . A P P R O X IM A T E A L G O R IT H M S
Tro uble creates a capacity to ha nd le st.- Ol iv e r W e n d e
l l Ho lme s , J r .
U n t i l v e r y r e c e n t l y , t h e f o c u s o f a t t e
n t i o ni n a l g o r it h m i c a n a l y s i s h a s b e e n o n
" t ra c t -a b l e " c o m b i n a t o r i a l p r o b l e m s s u
c h a ss e a r c h in g , s o r ti n g , a n d m a t r i x m u l t
i p li c a -ComputingS u r v e y s , V o L 9 , N o . 4 , D e c e m
b e r 1 97 7
-
8/9/2019 Analysis of Discrete Algorithms
16/23
306 B. Weidetion, which have been mentioned above.These are
among the "easy" problems(which in current terminology means tha
ttheir complexity is bounded by a polyno-mial in n); on the other
hand, many opti-mization and graph problems are "hard"(i.e., thei r
complexity is apparent ly notbounded by any such polynomial).
Sinceso many important problems are, unfortu-nately, in the latter
category, an entirenew group of algorithms that find approx-imate
solutions to hard problems has beendeveloped. They are known as
"approxi-mation" or "heuristic" algorithms. Withthese algorithms
have come new measuresof "goodness" as well as techniques fortheir
design and analysis. Two classes ofapproximation, one guaranteeing
a near-optimal solution always, and the otherproducing an optimal
or near-optimal so-lution "almost everywhere," are discussed.Pro b
l e m C l a s s e s a n d R e du c i b i l i t yFor the discussion
that follows we requiresome concepts regarding Turing machines(TM)
and formal languages. It is some-what artificial, but convenient,
to pose aproblem in terms of a language-recogni-tion task. This is
done by formulating itto require a yes-no solution; for
example,"Does this traveling salesman problemhave a solution with
cost less than k?" ATM can be asked to accept the input prob-lem
description if the answer is "yes." Tosolve the problem, such a TM
then acceptsonly input strings from some language Lwhich comprises
precisely those probleminstances with "yes" answers. It is
helpfulto refer to the original problem and thelanguage L
interchangeably in the contextof the classes P and NP (see [1] for
moredetails).Formally, the class P (for "Polynomial")is the set of
languages which are recog-nized by some deterministic TM that
al-ways halts in a number of steps which isbounded by a fixed
polynomial in thelength of the input (i.e., in "polynomialtime").
Similarly, the class NP (for "Non-deterministic Polynomial") is the
set oflanguages which are recognized by somenondeterministic TM in
polynomial time.A nondeterministic TM operates in poly-
nomial time if all sequences of choices ofmoves are of
polynomial bounded length;a string is accepted by such a machine
ifthere exists any such sequence of stepswhich leads to an
accepting state (seeHopcroft and Ullman [31]).Tha t P C NP is clear
from the defini-tions. Undoubtedly the most intriguingopen question
in the complexity area iswhether P = NP, or whether there
areproblems in NP which cannot be solved inpolynomial time by a
deterministic TM.Problems known to be in P include the"easy"
problems previously discussed.Other problems in NP which might
alsobe in P are most of the so-called optimiza-tion and graph
problems, such as 0/1 inte-ger programming, certain
schedulingproblems, finding Hamiltonian circuits,graph coloring,
and many others (see Karp[39, 40]). So far as is known, there is
nodeterministic polynomial-time algorithmfor solving any of these
"hard" problems;all known algorithms have a worst-casecomplexity
that is not bounded by a poly-nomial function of the input size.One
method of proving the equivalenceP = NP might be to demonstrate a
deter-ministic polynomial-time simulation of anondeterministic TM.
Since this wouldamount to a systematic search through atree of move
sequences of polynomialdepth, the total number of nodes (andhence
the time for the simulation) couldbe exponential. Hence, it appears
that itis not possible to simulate an arbitrarynondeterministic TM
operating in polyno-mial time by a determinis tic TM operatingin
polynomial time. Of course this doesnot by any means show that P ~
NP;there might well be other approaches, notresembling a
deterministic simulation,which could solve one of the hard
problemsin NP in polynomial time.Fortunately, it is sufficient to
considersolving these problems in polynomial timeon a normal
random-access computerrather than on a TM; the problems in Premain
the same [1], because each ma-chine can simula te the o ther in
polynomialtime. Roughly speaking, problems thatseem to require some
sort of backtrack-searching through a tree of polynomiallybounded
depth are the difficult problems
C o m p u t i n g S u r ve y s , V o l. 9 , N o . 4 , D e c e m
b e r 1 9 7 7I
-
8/9/2019 Analysis of Discrete Algorithms
17/23
An alysis Techniques for Discrete Algorith m s 307o f N P . W h
i le t h e r e is o v e r w h e l m i n g c ir -c u m s t a n t i a
l e v i d e n c e t h a t s u c h p r o b le m sa r e n o t a l s o
i n P , n o p r o o f o f t h i s c o n j e c t u r eh a s y e t b
e e n p r o d u c e d.A l a n g u a g e L 1 i s " p o l y n o m i a
l l y r e d u c i -b l e " to L 2 i f t h e r e i s a d e t e r m i
n i s t i c p o ly -n o m i a l - t i m e a l g o r i t h m w h i c
h t r a n s f o r m sa s t r i n g x i n t o a s t r i n g f(x) s u
c h t h a t x i si n L I i f f f(x) i s i n L 2 . A m o n g t h e c
o n s e -q u e n c e s o f r e d u c i b i l it y i s t h e f a c t
t h a t i ft h e r e i s a p o l y n o m i a l -t i m e d e t e r m
i n i s t i ca l g o r i t h m t o r e c o g n i z e L 2 a n d L 1
is r e d u c i -b l e to L 2, t h e n a p o l y n o m i a l - t i m
e d e t e r -m i n i s t i c a l g o r i t h m t o r e c o g n iz e
L ~ c a n b ec o n s t r u c t e d . I t c o n s i s ts o f a p p l
y i n g t h et r a n s f o r m a t i o n f to t h e i n p u t x a n
d t h e nc h e c ki n g w h e t h e r f(x) i s i n L ~, a l l o f w
h i c hc a n b e d o n e d e t e r m i n i s t i c a l l y i n p o
l y n o -m i a l t im e .T h e k e y t o t h e a r g u m e n t t h
a t P ~ N Pis a r e m a r k a b l e t h e o r e m b y C o o k [ 1
2]s t a t in g t h a t e v e r y p ro b l e m i n N P c a n b ep o
l y n o m i a l ly r e d u c e d t o B o o l e a n s a ti s fi a -b
i li ty . T h i s p r o b l e m i s v e r y s i m p l y s t a t e d
:I s t h e r e a n a s s i g n m e n t o f t r u t h v a l u e s t
ot h e l i te r a l s o f a B o o l e a n e x p r e s s io n w h i
c hm a k e s t h e e x p r e s s i o n t r u e ? T h i s m e a n st
h a t e v e r y p r o b l e m w h i c h c a n b e s o lv e d i np o
l y n o m i a l t i m e o n a n o n d e t e r m i n i s t i c T Mc
a n a ls o b e s o l v ed b y s u b j e c t in g t h e i n p u ts t
r in g t o a t r a n s f o r m a t i o n ( d o ne d e t e r m i n
-i s ti c a ll y i n p o l y n o m i a l ti m e ) t h a t c o n v e
r t si t to a n i n s t a n c e o f s a t is f i a b il i t y , a n
d t h e ns o l v i n g t h e r e s u l t i n g s a t i s f i a b i
l i t y p r o b -l e m . M a n y o t h e r p r o b l e m s h a v e
s u b s e -q u e n t l y b e e n s h o w n t o h a v e t h e s a m
ep r o p e r t y . A p r o b l e m s u c h a s s a t i s f i a b i
l i t yi s c al le d " N P - c o m p l e t e ." O n e w a y t o p r
o v et h a t a p r o b l e m i s N P - c o m p l e t e i s t o d e
m -o n s t r a t e t h a t t h e p r o b l e m i s i n N P , a n dt
h a t a n o t h e r N P - c o m p l e te p r o b l e m i s r e -d u
c i b l e t o i t . A l l N P - c o m p l e t e p r o b l e m sar e
a l s o ~ 'P -com pl e t e" ( s ee Sa h n i [60 ]) , i . e . ,i n t
h e c l a ss o f p r o b l e m s s o l v a b l e i n d e t e r -m i
n i s t i c p o l y n o m i a l t i m e i f f P = N P . T h ed i s
t i n c t i o n b e t w e e n t h e s e d e f i n i t i o n s h a
sa l l b u t d i s a p p e a r e d i n t h e l i t e r a t u r e ,
a n dt h e t e r m " N P - c o m p l e te " h a s c o m e to b eu s
e d f o r e i t h e r o n e . U n d e r e i t h e r d e f i n i -t
io n , i f t h e r e i s a p o l y n o m i a l - t i m e d e t e r
-m i n i s t i c a l g o r i t h m f o r a n y N P - c o m p l e t
ep r o b l e m , t h e n P = N P .K a r p [ 3 9 ] s h o w s r e d u
c i b i li t i e s a m o n gp r o b le m s w h i c h d e m o n s t
r a t e t h a t t h e c la ss
o f N P - c o m p l e t e p r o b l e m s i s q u i t e l a r g
e ,a n d i n c l u d e s a l l t h e o p t i m i z a t i o n a n dg
r a p h p r o b l e m s m e n t i o n e d a bo v e. T h i sf a c t
fo r m s t h e b a s i s f o r b e l i e v i n g ( e v e n i fo n e
c a n n o t p r o ve ) t h a t P ~ N P , s i n c en o n e o f t h e
h u n d r e d s o f a l g o r i t h m s f o r th es co r es o f p r
o b le m s t h a t a r e N P - c o m p l e ter u n s i n p o l y n
o m i a l t i m e . I f a n y o f t h e s ep r o b l e m s c o u l
d b e s o l v e d q u i c k l y , a l l o ft h e m c o u ld b e; t
h e f a c t t h a t s o f a r n o n e o ft h e m c a n b e is a c o
n v in c i n g a r g u m e n t( a l t h o u g h n o t a p ro o f) t
h a t t h e y n e v e r w i l lb e .W h e t h e r P = N P i s n o t
t h e o n l y o p e nq u e s t i o n i n t h i s a r e a . S o m e
p r o b l e m s r e -m a i n u n c l a s s i f i e d . F o r e x a
m p l e , d e c i d i n gw h e t h e r t w o r e g u l a r e x p r
e s s i o n s a r ee q u i v a l e n t , o r w h e t h e r a s t r
i n g i s i n ag i v e n c o n t e x t - s e n s it i v e l a n g u
a g e , a r ep r o b le m s a t l e a s t a s h a r d a s a n y p r
o b l e m si n N P , b u t a r e n o t k n o w n t o b e i n N Pt h
e m s e l v e s ( s ee A h o , H o p c r o f t , a n d U l l -m a n
[ 1 ] ) . S u c h p r o b l e m s a r e c a l l e d " N P -h a r d
" b e c a u s e t h e i r i n c l u s i o n i n P w o u l di m p l
y t h a t P = N P . O n t h e o t h e r h a n d ,d e c i d i n g w
h e t h e r t w o g i v e n g r a p h s a r ei s o m o r p h i c ,
w h e t h e r a g i v e n i n t e g e r i s ap r im e , o r w h e t
h e r a g i v e n in t e g e r i s n o t ap r i m e , h a v e n o t
b e e n s h o w n t o b e N P -c o m p le te , e v e n th o u g h t
h e y a r e i n N P a n dc a n n o t c u r r e n t l y b e s o lv e
d i n p o l y n o m i a lt i m e . H e n c e , t h e s e m a y b e
p r o b l e m s i nN P b u t n o t i n P , w h o s e so l u t io n
s w i l l n o th e l p s o lv e t h e o t h e r p r o b l e m s i n
N P . L a d -h e r [ 50 ] h a s s h o w n t h a t P @ N P i m p l
ie st h e e x i s t e n c e o f s u c h ' ~ i n te r m e d i a t e
" p r o b -l e m s , b u t i t i s n o t k n o w n w h e t h e r t
h ea b o v e p r o b l e m s a r e i n t h i s c l as s ."Gua
ranteed" Approximat ion Algor ithmsA l l k n o w n a l g o r i t h
m s f o r s o l v in g N P - c o m -p l e te p r o b l e m s r u n
i n n o n p o l y n o m i a l d e -t e r m i n i s t i c t i m e .
I t i s t h e r e f o r e i m p r a c t i -c a l to s o l v e v e r
y l a r g e i n s t a n c e s o f t h e s ep r ob le m s , e v e n
t h o u g h s o m e o f t h e m a r ea m o n g t h e m o s t i m p
o r t a n t p r o b l e m s o fg r a p h t h e o r y a n d o p e r
a t i o n s r e s e a r c h .F o r t u n a t e l y m a n y a p p l
i c a t i o n s r e q u i r i n gs o l u ti o n o f s u c h p r o b
l e m s d o n o t r e q u i r ee x a c t s o lu t io n s . T o t a
k e a d v a n t a g e o f t h i sf a c t, a l g o r i t h m d e s i
g n e r s h a v e d e v e l o p e dm a n y n e w a p p r ox i m a t
i o n m e t h o d s. M o s to f t h e s e a l g o r i t h m s s e e
k n e a r - o p t i m a l s o-l u t i o n s t o a ll i n s t a n c
e s o f a p r o b l e m .
C om put ing S u r ve ys , V o l . 9 , No . 4 , D e c e m be r 1
977
-
8/9/2019 Analysis of Discrete Algorithms
18/23
308 B . W e i d eA g e n e r a l l y a c c e p t e d e r r o r m
e a s u r e f o rt h e s e a l g o r i th m s i s t h e m a x i m u
m r e l a ti v ee r r o r o v e r a l l p r o b l e m i n s t a n c
e s [3 7]. N o r -m a l l y , t h e q u a n t i t i e s f r o m w h
i c h t h e r e l a -t i v e e r r o r i s c o m p u t e d a r e o
b v i o u s : a ni n t e g e r p r o g r a m ' s o b j e c t iv e f
u n c t i o n v a l u e ,a g r a p h 's c h r o m a t i c n u m b e
r f o r t h e g r a p h
c o l o r i n g p r o b l e m , o r a s c h e d u l e ' s l e n
g t h .S o m e t i m e s , a s f o r t h e s a t i s f ia b i l i t
y p r ob -l e m , t h e o r i g i n a l p r o b l e m m u s t b e r
e-p h r a s e d a s a n o p t i m i z a t i o n p r o b l e m i no
r d e r to s t u d y a p p r o x i m a t e s o l u ti o n s . T h
er a t i o o f t h e a b s o l u t e e r r o r t o t h e e x a c ts
o l u t i o n v a l u e i s g i v e n t h e s y m b o l ~ .I n d e
s ig n i n g a p p r o x i m a t io n a l g o r i th m so f t h i s
t y p e , o n e i s c o n c e r n e d w i t h g u a r -a n t e e i
n g t h a t t h e r e l a t i v e e r r o r ~ i s n e v e rl a rg e
r t h a n s o m e p r e s cr i b ed m a x i m u m ,w h i c h e s s
e n t i a l ly m e a n s t h a t e is t a k e n a sg i v e n . T h
e g o a l i s to d e v e lo p a n a l g o r i t h mt h a t a l w a
y s s o l ve s t h e p r o b l e m t o w i t h i na f a c t o r o f
1 -+ ~ . F o r t h i s r e a s o n , m a n y o ft h e s e a l g o r
i th m s t e n d t o r e s e m b l e t h e p a t-t e r n s c o m m
o n l y e n c o u n t e re d i n c a lc u l u sp r oo f s, w i t h
s t r a n g e f u n c t i o n s o f ~ a p p e a r -i n g, a s i f b
y m a g i c , i n t h e e a r l y s t a g e s o ft h e a l g o r i
th m i n o r d er to a s s u r e t h a t t h ef i n a l s o l u t i
o n w i l l b e w i t h i n a f a c t o r o f 1- ~ o f o p t i m a
l . R e c o g n i z i n g t h i s t a k e sm u c h o f t h e m y s
t e r y o u t o f w h a t c a na p p e a r t o b e v e r y c o m p
l i c a te d a l g o r i th m s .B e c a u s e e r ro r a n a l y s
i s i s p a r t o f t h ed e s ig n o f s u c h a l g o r i t h m s
, u n d e r s t a n d i n gt h e a n a l y s i s is t a n t a m o u
n t t o u n d e r s t a n d -i n g t h e o p e r a t i o n o f t h
e a l g o r i t h m . S a h n i[6 2] n i c e l y s u m m a r i z e
s t h e t e c h n i q u e s i nt h i s c l as s , d iv i d i n g t
h e m e t h o d s i n t o t h r e ec a t eg o r i es . T h e s e a
r e e s p e c i a l ly u s e f u l f o rt h e k n a p s a c k p r o
b l e m , p a c k i n g p r o b l e m s ,a n d c e r t a i n s c h
e d u l i n g p r o b l e m s . T h ew o r s t - c a s e c o m p l
e x i t i e s t h e y p r o d u c e a r eu s u a l l y l ik e O ( n
a / ~ ) f o r c o n s t a n t s a a n d bw h i c h d e p e n d o n
t h e p r o b le m .A n e x a m p l e o f o n e o f S a h n i ' s t
ec h -n i q u e s , c a l l e d i n t e r v a l p a r t i t i o n i
n g , i s as i m p l e a l g o r i t h m f o r s o l v i n g t h e
0 /1 k n a p -s a c k p r o b l e m :
m a x i m i z e : c " xs u b j e c t t o : a . x
-
8/9/2019 Analysis of Discrete Algorithms
19/23
A n a l y s i s T e c h n i q u e s f o r D t s c r e t e A l g
o r i t h m s 3 0 9s o n [ 37 ] p o i n t s o u t t h a t f o r s o
m e p r o b l e m s ,i n c l u d i n g t h o s e t o w h i c h S a
h n i ' s t e c h -n i q u e s c a n b e a p p l ie d , t h e r e l
a t i v e e r r o r ec a n b e b o u n d e d b y a c o n s ta n t i
n d e p e n d e n to f p r o b l e m s iz e. H o w e v e r , f o r
o t h e r s ( s u c ha s t h e m a x i m u m c l i q u e p r o b l
e m ) n o a l g o -r i t h m h a s y e t b e e n f o u n d fo r w h
i c h d o e sn o t g r o w a t l e a s t a s f a s t a s n c f o r
s o m ec > 0 . G a r e y a n d J o h n s o n [2 4] s h o w t h a
ta p p r o x i m a t i n g t h e c h r o m a t ic n u m b e r o f
ag r a p h t o w i t h i n a f a c to r o f t w o i s N P -c o m p
l e t e ; i n f a c t , n o k n o w n p o l y n o m i a l -t i m e
a l g o r i t h m s o l v e s t h e p r o b l e m t ow i t h i n a
n y b o u n d e d r a t i o . S i m i l a r l y d i s -c o u r a g
i n g i s a n o t h e r r e s u l t r e p o r t e d i nt h e s a m
e p a p e r : i f t h e r e i s a p o l y n o m i a l -t i m e a l
g o r i t h m f o r a p p r o x i m a t i n g t h e si zeo f t h e
m a x i m u m c l i q u e o f a g r a p h t ow i t h i n s o m e b
o u n d e d r at i o , t h e n t h e r e i sa p o l y n o m i a l -
ti m e a l g o r i t h m f o r a p p ro x i -m a t i n g i t t o w
i t h i n a n y b o u n d e d r a t io .E x a m p l e s o f a p p r
o x i m a t i o n a l g o r it h m sa n d a n a l y s i s o f e r r
o r s a n d c o m p l e x i t m sh a v e a l so b e e n g i v e n b
y J o h n s o n [38 ],S a h n i [6 1], a n d C o f f m a n a n d S
e t h i [ 10 ],a m o n g m a n y o th e rs . G a r e y a n d J o h
n s o n[2 5] h a v e c o m p i l e d a f in e a n n o t a t e d b i
b l i-o g r a p h y f o r t h i s a r e a .Probabi l i s t i c
Behav ior o f Approx imat ionA l g o r i th msI t i s n o t e a s y
t o g u a r a n t e e g o o d a p p r o x i -m a t e s o l u t i o
n s t o c e r t a in N P - c o m p l e t ep r o b l em s . T h i s
f a c t h a s m o t i v a t e d a s e a r c hf o r a l t e r n a t
i v e s t o t h e g u a r a n t e e d a p p ro x -i m a t i o n a p
p r o a c h . O n e a l t e r n a t i v e i s t om a k e s u c h a
s o p h i s ti c a t e d g u e s s a b o u t t h es o l u t i o n t
h a t t h e l i k e l i h o o d o f e r r o r i sn e g l i g i b l
y s m a l l . I f w e d e f i n e P n t o b e t h ep r o b a b i l
i t y t h a t s u c h a n a l g o r i t h m g i v e sa n u n a c c
e p t a b l y b a d a n s w e r t o a ra n d o m l yc h o se n p r
o b le m o f s iz e n - s o m e d i s t r i b u -t io n o f p r o b
l e m i n s t a n c es i s a s s u m e d -t h e n t h e a l g o r i
t h m i s s a i d t o w o r k c o r -r e c t l y " a l m o s t e v
e r y w h e r e " i f ~ p n i s f i -n i t e , w h e r e t h e s u
m i s o v e r a l l p r o b l e msiz es n = 1, 2, - . ..O p t i m a
l i t y o r n e a r - o p t im a l i t y " a l m o s te v e r y w h
e r e " i s th e o r e t i c a l l y a s t r o n g c on -d i ti o n
. W e r e w e t o r a n d o m l y c h o o s e o n ep r o b l e m i
n s t a n c e o f e a c h s i ze n , f o r n = 1 ,2 , . . . , a n d
r u n t h e a l g o r i t h m o n e a c h o ft h e m , t h e n n o
t o n l y w o u l d t h e a l g o r i t h m
g i v e g o o d a n s w e r s i n f i n i t e l y o f t e n , b
u t( w i t h p r o b a b i l i t y o n e ) i t w o u l d f a i l t
o g i v eg o od a n s w e r s o n l y f i n i t e ly o f t e n .K a
r p [4 1] i l l u s t r a t e s s u c h a l g o r i t h m s .H e d
e m o n s t r a t e s a n O ( n l o g n ) a l g o r i t h mf o r s
o l v in g r a n d o m E u c l i d e a n t r a v e l i n gs a l e s
m a n p r o b l e m s w h i c h , f or e v e r y e > 0 ,f i n d
s a s o l u t i o n w i t h i n a f a c t o r o f 1 + o fb e i n g
o p t i m a l a l m o s t e v e r y w h e r e . T h ea n a l y s is
d e p e n d s h e a v i l y o n a n i n t e re s t in gt h e o r e
m b y B e ar d w o o d , H a l t e n , a n d H a m -m e r s l e y
[3 ], s t a t i n g t h a t f o r n p o i n t s c h o -s e n a t r
a n d o m i n a p l a n e f i g u r e o f a r e a A ,t h e r e i s
a c o n s t a n t c s u c h t h a t t h e l e n g t ho f t h e s
h
![Algorithms Lecture 1: Discrete Probability [Sp’20] · 2020-04-07 · Algorithms Lecture 1: Discrete Probability [Sp’20] Moregenerally,acountablesetofeventsfAi ji 2Igisfullyormutuallyindependentifand](https://img.pdfslide.net/doc/110x75/5f8ded7df702d4365b6a0d5e/algorithms-lecture-1-discrete-probability-spa20-2020-04-07-algorithms-lecture.jpg)
