Amortized 4up

8/9/2019 Amortized 4up

1/12

Princeton University COS 423 Theory of Algorithms Spring 2001 Kevin Wayne

Amortized Analysis

2

Beyond Worst Case Analysis

Worst-case analysis.

s Analyze running time as function of worst input of a given size.

Average case analysis.

s Analyze average running time over some distribution of inputs.

s Ex: quicksort.

Amortized analysis.

s Worst-case bound on sequence of operations.

s Ex: splay trees, union-find.

Competitive analysis.s Make quantitative statements about online algorithms.

s Ex: paging, load balancing.

3

Amortized Analysis

Amortized analysis.

s Worst-case bound on sequence of operations.

no probability involved

s Ex: union-find.

sequence of m union and find operations starting with n

singleton sets takes O((m+n) (n)) time.single union or find operation might be expensive, but only (n)

on average

4

Dynamic Table

Dynamic tables.

s Store items in a table (e.g., for open-address hash table, heap).

s Items are inserted and deleted.

too many items inserted copy all items to larger tabletoo many items deleted copy all items to smaller table

Amortized analysis.

s Any sequence of n insert / delete operations take O(n) time.

s Space used is proportional to space required.

s Note: actual cost of a single insert / delete can be proportional to n

if it triggers a table expansion or contraction.

Bottleneck operation.

s We count insertions (or re-insertions) and deletions.

s Overhead of memory management is dominated by (or

proportional to) cost of transferring items.


2/12

5

Aggregate method.

s Sequence of n insert ops takes O(n) time.

s Let ci = cost of ith insert.

Initialize table size m = 1.

INSERT(x)

IF (number of elements in table = m)Generate new table of size 2m.

Re-insert m old elements into new table.

m 2m

Insert x into table.

Dynamic Table Insert

Dynamic Table: Insert

=otherwise1

2ofpowerexactanis1if -iici n

nn

ncn

j

jn

ii

3

)12(

22

log

01

x} .

Implementing Find(x, S).s Call Splay(x, S).

s If x is root, then return x; otherwise return NO.

18

Splay

Implementing Join(S, S).

s Call Splay(+, S) so that largest element of S is at root and allother elements are in left subtree.

s Make S the right subtree of the root of S.

Implementing Delete(x, S).

s Call Splay(x, S) to bring x to the root if it is there.

s Remove x: let S and S be the resulting subtrees.

s Call Join(S, S).

Implementing Insert(x, S).

s

Call Splay(x, S) and break tree at root to form S and S.s Call Join(Join(S, {x}), S).

19

Implementing Splay(x, S)

Splay(x, S): do following operations until x is root.

s ZIG: If x has a parent but no grandparent, then rotate(x).

s ZIG-ZIG: If x has a parent y and a grandparent, and if both x and y

are either both left children or both right children.

s ZIG-ZAG: If x has a parent y and a grandparent, and if one of x, y

is a left child and the other is a right child.

A B

xC

y

CB

y

A

x

ZIG(x)

ZAG(y)

root

20



s ZIG: If x has a parent but no grandparent.





ZIG-ZIG

A B

x

C

y

D

z

DC

z

B

y

A

x


6/12

21



s ZIG: If x has a parent but no grandparent.





ZIG-ZAG

B C

x

D

y

z

DC

y

x

A

BA

z

22

Splay Example

Apply Splay(1, S) to tree S:10

9

8

7

6

5

4

3

2

1

ZIG-ZIG

23

Splay Example

Apply Splay(1, S) to tree S:

ZIG-ZIG

10

9

8

7

6

5

4

1

2

3

24

Splay Example

Apply Splay(1, S) to tree S:

ZIG-ZIG

10

9

8

7

6

1

2

3

4

5


7/12


8/12

29

Splay Tree Analysis

Definitions.

s Let S(x) denote subtree of S rooted at x.

s |S| = number of nodes in tree S.

s (S) = rank = log |S| .

s (x) = (S(x)). 2

8

4

63

10

1

9

5 7

|S| = 10(2) = 3(8) = 3(4) = 2

(6) = 1(5) = 0

S(8)

30

Splay Tree Analysis

Splay invariant: node x always has at least (x) credits on deposit.

Splay lemma: each splay(x, S) operation requires 3((S) - (x)) + 1credits to perform the splay operation and maintain the invariant.

Theorem: A sequence of m operations involving n inserts takesO(m log n) time.

Proof:

s (x) log n at most 3 log n + 1 credits are needed foreach splay operation.

s Find, insert, delete, join all take constant number of splays plus

low-level operations (pointer manipulations, comparisons).

s

Inserting x requires log n credits to be deposited to maintaininvariant for new node x.

s Joining two trees requires log n credits to be deposited tomaintain invariant for new root.

31

Splay Tree Analysis

Splay invariant: node x always has at least (x) credits on deposit.

Splay lemma: each splay(x, S) operation requires 3((S) - (x)) + 1credits to perform the splay operation and maintain the invariant.

Proof of splay lemma: Let (x) and (x) be rank before and singleZIG, ZIG-ZIG, or ZIG-ZAG operation on tree S.

s We show invariant is maintained (after paying for low-level

operations) using at most:

3((S) - (x)) + 1 credits for each ZIG operation. 3((x) - (x)) credits for each ZIG-ZIG operation. 3((x) - (x)) credits for each ZIG-ZAG operation.

s Thus, if a sequence of of these are done to move x up the tree, we

get a telescoping sum total credits 3((S) - (x)) + 1.

32

Splay Tree Analysis

Proof of splay lemma (ZIG): It takes 3((S) - (x)) + 1 credits toperform a ZIG operation and maintain the splay invariant.

s In order to maintain invariant, we must pay:

s Use extra credit to pay for

low-level operations.))()((3

))()((3

)()(

)()()()()()(

xS

xx

xx

xyyxyx

=

=+

A B

x

C

y

CB

y

A

xS S

(y) = (x)

ZIG

root

(x) = (S)


9/12

33

Splay Tree Analysis

Proof of splay lemma (ZIG-ZIG): It takes 3((x) - (x)) credits toperform a ZIG-ZIG operation and maintain the splay invariant.

s If (x) > (x), then can afford topay for constant number of low-level

operations and maintain invariant using 3((x) - (x)) credits.

))()((2

))()(())()((

))()(())()((

)()()()()()()()()()(

xx

xxxx

yzxy

yxzyzyxzyx

=

+

+=

+=++

S

A B

xC

yD

z

DC

zB

yA

x S

ZIG-ZIG

34

Splay Tree Analysis

Proof of splay lemma (ZIG-ZIG): It takes 3((x) - (x)) credits toperform a ZIG-ZIG operation and maintain the splay invariant.

s Nasty case: (x) = (x).

s We show in this case (x) + (y) + (z) < (x) + (y) + (z).dont need any credit to pay for invariant

1 credit left to pay for low-level operations

so, for contradiction, suppose (x) + (y) + (z) (x) + (y) + (z).

s Since (x) = (x) = (z), by monotonicity (x) = (y) = (z).

s After some algebra, it follows that (x) = (z) = (z).

s Let a = 1 + |A| + |B|, b = 1 + |C| + |D|, then

log a = log b = log (a+b+1)

s WLOG assume b a.S

A B

xC

yD

z

DC

zB

yA

xS

a

a

aba

log

log1

)2log()1log(

>

+=

++

ZIG-ZIG

35

Splay Tree Analysis

Proof of splay lemma (ZIG-ZAG): It takes 3((x) - (x)) credits toperform a ZIG-ZAG operation and maintain the splay invariant.

s Argument similar to ZIG-ZIG.

ZIG-ZAG

B C

xD

y

z

DC

y

x

A

BA

z

Princeton University COS 423 Theory of Algorithms Spring 2001 Kevin Wayne

Augmented Search Trees


10/12

37

Support following operations.

Interval-Insert(i, S): Insert interval i = (li, ri ) into tree S.

Interval-Delete(i, S): Delete interval i = (li, ri ) from tree S.

Interval-Find(i, S): Return an interval x that overlaps i, or

report that no such interval exists.

Interval Trees

(7, 10)

(5, 11)

(4, 8) (21, 23)

(15, 18)

(17, 19)

38

(4, 8)

Key ideas:

s Tree nodes contain interval.

s BST keyed on left endpoint.

Interval Trees

(7, 10)

(5, 11)

(4, 8) (21, 23)

(15, 18)

(17, 19)

(17, 19)

Key Interval

(5, 11) (21, 23)

(15, 18)

(7, 10)

39

(4, 8)

Key ideas:

s Tree nodes contain interval.

s BST keyed on left endpoint.

s Additional info: store max

endpoint in subtree rooted

at node.

Interval Trees

(7, 10)

(5, 11)

(4, 8) (21, 23)

(15, 18)

(17, 19)

(17, 19)

max insubtree

(5, 11) 18 (21, 23) 23

8 (15, 18) 18

(7, 10) 10

23

40

x root(S)

WHILE (x != NULL)

IF (x overlaps i)

RETURN t

IF (left[x] = NULL OR

max[left[x]] < li)

x right[x]ELSE

x left[x]RETURNNO

Splay last node on path

traversed.

Interval-Find (i, S)

Finding an Overlapping Interval

Interval-Find(i, S): return an interval x that overlaps i = (li, ri ), orreport that no such interval exists.

(4, 8)

(17, 19)

(5, 11) 18 (21, 23) 23

8 (15, 18) 18

(7, 10) 10

23


11/12

41



Case 1 (right). If search goes right,

then there exists an overlap in rightsubtree or no overlap in either.

Proof. Suppose no overlap in right.

s left[x] = NULL no overlap in left.

smax[left[x]] < li no overlap in left.

x root(S)

WHILE (x != NULL)

IF (x overlaps i)

RETURN t


max[left[x]] < li)

x right[x]ELSE

x left[x]RETURNNO


traversed.


i = (li, ri )left[x]

max

42


Case 2 (left). If search goes left,

then there exists an overlap in leftsubtree or no overlap in either.

Proof. Suppose no overlap in left.

s li max[left[x]] = rj forsome interval j in left subtree.

s Since i and j dont overlap, we have

li ri lj rj.

s

Tree sorted byl

for any intervalk in right subtree: ri lj lk no overlap in right subtree.


x root(S)

WHILE (x != NULL)

IF (x overlaps i)

RETURN x


max[left[x]] < li)

x right[x]ELSE

x left[x]RETURNNO


traversed.


i = (li, ri ) j = (lj, rj)

k = (lk, rk)

43

Interval Trees: Running Time

Need to maintain augmented data structure during tree-modifying ops.

s Rotate: can fix sizes in O(1) time by looking at children:

A14

B19

C

30 ZIG

ZAG

(11, 35) 35

(6, 20) 20

C30

B19

A

14

(6, 20) ?

(11, 35) ?

35

35

=

xr

xright

xleft

x ]][max[

]][max[

max]max[

44

VLSI Database Problem

VLSI database problem.

s Input: integrated circuit represented as a list of rectangles.

s Goal: decide whether any two rectangles overlap.

Algorithm idea.

s Move a vertical "sweep line" from left to right.

s Store set of rectangles that intersect the sweep line in an interval

search tree (using y interval of rectangle).


12/12

45

Sort rectangle by x coordinate (keep two copies of

rectangle, one for left endpoint and one for right).

FORi = 1 to 2N

IF (ri is "left" copy of rectangle)

IF (Interval-Find(ri, S))

RETURN YES

ELSE

Interval-Insert(ri, S)

ELSE (ri is "right" copy of rectangle)

Interval-Delete(ri

, S)

VLSI (r1, r2 ,..., rN)

VLSI Database Problem

46

Order Statistic Trees

Add following two operations to BST.

Select(i, S): Return ith smallest key in tree S.

Rank(i, S): Return rank of x in linear order of tree S.

Key idea: store size of subtrees in nodes.

m 8

c 5 P 2

b 1 f 2

d 1 h 1

q 1

Key Subtree size

47

Order Statistic Trees

Need to ensure augmented data structure can be maintained duringtree-modifying ops.

s Rotate: can fix sizes in O(1) time by looking at children.

V6

X4

Z17

ZIG

ZAG

y 29

w 11

Z17

X4

V6

w29

y22

Documents

Amortized 4up