Amortized analysis

Amortized analysis

Amortized analysis

In amortized analysis the goal is to bound the worst casetime of a sequence of operations on a data-structure.

If n operations take T (n) time (worst case), theamortized cost of an operation is T (n)/n.

In the aggregate analysis method, we directly boundT (n).

Amortized analysis

Stack with Multipop operation

A stack with Multipop operation supports the standard stackoperations (Push, Pop, IsEmpty) and the a Multipopoperation:

Multipop(S , k)

(1) while not IsEmpty(S) and k > 0(2) Pop(S)(3) k ← k − 1

Given n operations (starting with an empty stack):

There are at most n push operations.

Each element is popped from the stack at most once foreach time we pushed it into the stack

Therefore, there are at most n pop operations, includingthose performed during Multipop.

Total cost is Θ(n). Amortized cost is Θ(n)/n = Θ(1).

Amortized analysis

Binary counter

Suppose we store a counter in an array A[0..k − 1] ofbits, where A[0] is the least significant bit.We implement operation Increment(A) which increasesthe value of the counter by 1, as follows.

Increment(A)

(1) i ← 0(2) while i < A.length and A[i ] = 1(3) A[i ]← 0(4) i ← i + 1(5) if i < A.length(6) A[i ]← 1

To bound the time complexity of n increment operations(starting with A containing only zeros), it suffices tobound the number of bit flips in these operations.

Amortized analysis

Binary counter

0 0 0 0 0 00 0 0 0 0 10 0 0 0 1 00 0 0 0 1 10 0 0 1 0 00 0 0 1 0 1

01234567

0 0 0 1 1 00 0 0 1 1 10 0 1 0 0 08

A[0]A[1]A[2]A[3]A[4]A[5]

bit flip freq. number of flips0 1 n1 1/2 bn/2c2 1/4 bn/4c3 1/8 bn/8c

...i 1/2i bn/2ic

...

The total number of flips isk−1∑i=0

bn/2ic <∞∑i=0

n/2i = n∞∑i=0

1/2i = 2n.

Total cost of n increments is Θ(n). Amortized cost Θ(1).

Amortized analysis

Accounting method

Each operation has a cost which is its actual timecomplexity.

When an operation is performed, we charge a certainamount, which may be different than the cost.

Part of a charge can be stored in objects of thedata-structures.

To pay the cost of an operation, we can use both thecharge or stored credit.

The total cost of the operation is at most the sum of thecharges.

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

push(S,4)$2

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,4)

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,5)

5 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,2)

5 $12 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

multipop(S,2)

5 $12 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows:

operation cost chargePush 1 2Pop 1 0Multipop s 0

s is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

multipop(S,2)

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 0 0A[0]A[1]A[2]A[3]A[4]A[5]

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 0 1A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 1 0A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 1 1A[0]A[1]A[2]A[3]A[4]A[5]

$1$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 1 0 0A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Hash table

Suppose that we use a hash table with chaining.

Let α = n/m, where n is the number of elements in thetable and m is the size of the table.

We want to maintain the invariant 1/4 ≤ α ≤ 1:

α ≤ 1 gives Θ(1) expected search time.α ≥ 1/4 gives Θ(n) space.

Assume we only have insertions (no deletions).

We can maintain the invariant by performing rehashingwhen the m + 1-th element is inserted to the table: Wecreate a new table of size 2m and move all elements tothe new table.

Amortized analysis

Hash table

The cost of Insert is 1 for inserting the element to thetable, and n for performing rehashing.

The charge of Insert is $3.

When we insert an element to the table, we use $1 fromthe charge to pay for the cost, and we store the remaining$2 in the table.

When rehashing is performed, the cost is payed usingdollars stored in the table.

Immediately after rehashing, no dollars are stored in thetable.

The next rehashing occurs after m/2 insertions, so thetable stores m dollars at the time of the next rehashing.

Amortized analysis

Example

T

6

$0

Amortized analysis

Example

3

T

6

insert(T,3)$2

Amortized analysis

Example

T insert(T,4)6

3

$0

Amortized analysis

Example

T insert(T,4)6

4 3

$2

Amortized analysis

Example

T insert(T,2)6

4 32

$4

Amortized analysis

Example

T insert(T,7)

6

3

4

2

$0

Amortized analysis

Example

T insert(T,7)

6

3

4

27

$2

Amortized analysis

BB[α] tree — definition

Let size(v) be the number of vertices in the subtree of v .A vertex v is of bounded balance α if

size(v .left) ≥ bα·size(v)c and size(v .right) ≥ bα·size(v)cA BB[α] tree (α < 0.5) is a binary search tree such thatevery vertex v is of bounded balance α.The height of a BB[α] tree with n vertices is at mostlog1/(1−α) n.

α = 1/3

9

1 1

35

3 1

114

2 6

10

168

14

12

20

Amortized analysis

Insertion

To insert an element to a BB[α] tree, insert it as a newleaf. Let w be the highest vertex that is not of boundedbalance α. If w exists, replace the subtree of w by abalanced tree containing the same elements.The time complexity is O(depth(w) + size(w)), which isΘ(n) in the worst case.

α = 1/3

9

1 1

35

3 1

114

2 6

10

168

14

12

20

Amortized analysis

Insertion

To insert an element to a BB[α] tree, insert it as a newleaf. Let w be the highest vertex that is not of boundedbalance α. If w exists, replace the subtree of w by abalanced tree containing the same elements.The time complexity is O(depth(w) + size(w)), which isΘ(n) in the worst case.

α = 1/3

10

1 1

36

4 1

12

w

3

4

2 6

10

168

14

12

20

Amortized analysis

Insertion

To insert an element to a BB[α] tree, insert it as a newleaf. Let w be the highest vertex that is not of boundedbalance α. If w exists, replace the subtree of w by abalanced tree containing the same elements.The time complexity is O(depth(w) + size(w)), which isΘ(n) in the worst case.

α = 1/3

10

1 1

36

3 2

11 12 10

16

8 14

12

203

4

6

Amortized analysis

Amortized complexity

The cost of insert is depth(w) + size(w), where w is thevertex whose subtree was replaced (or the new leaf).

The charge of insert is(1 +

1

1− 2α

)log1/(1−α) n +

2

1− 2α.

We use ≤ log1/(1−α) n dollars from the charge to pay forthe depth(w) cost.

In each node on the path from the root to the new leaf,we put 1/(1− 2α) dollars from the charge. This requires≤ 1

1−2α log1/(1−α) n dollars.

We use the dollars stored in w and 2/(1− 2α) dollarsfrom the charge to pay for the size(w) cost.

We need to show that w contains at least size(w)− 21−2α

dollars.Amortized analysis

Example

After the insertion, we add $3 to the nodes on the path fromthe root to the new leaf.

α = 1/3

$6

$6w

4

2 6

10

168

14

12

20

Amortized analysis

Example

After the insertion, we add $3 to the nodes on the path fromthe root to the new leaf.

α = 1/3

$9

$9

$3

$3

$3

w

3

4

2 6

10

168

14

12

20

Amortized analysis

Example

To pay for the balancing of w , we use the dollars stored in w .

α = 1/3

$9

w

2 10

16

8 14

12

203

4

6

Amortized analysis

Analysis

Consider the tree before the balancing of w .Let M = size(w).Without loss of generality, size(w .left) ≥ size(w .right).Since w is not of bounded balance αsize(w .right) = bαMc − 1

size(w .left) = M − size(w .right)− 1 = M − bαMcConsider the insert operations of elements to the subtreeof w since the last balancing operation involving w .The least number of such insertions is when all elementswere inserted to w .left.In that case, after last balancing operation involving w ,

size(w .right) = bαMc − 1 and size(w .left) ≤ bαMc.Therefore, the number of insert operations is at least

(M − bαMc)− bαMc ≥ M(1− 2α)− 2.Amortized analysis

Analysis

Each insert operation of an element to the subtree of wadds 1/(1− 2α) dollars to w .

Therefore, w contains at least

1

1− 2α(M(1− 2α)− 2) = M − 2

1− 2α

dollars.

Amortized analysis