Evaluation of Caching Strategies Based on Access Statistics on Past Requests

Least Recently Used (LRU): Simple Standard Method - Analysis, Simulation: Deficits of LRU Cache Hit Rate

Statistics-based Caching Strategies - Window: over the last K Requests - Geometrical Aging: Geom. Decreasing Weight per Request - Criteria: Hit Rate and Effort for Alternative Strategies

Summary on hit rates and effort of web caching strategies

Evaluation of Caching Strategies based on Access Statistics on Past Requests

Gerhard Haßlinger, Konstantinos Ntougias

[email protected]; [email protected]

Com

merc

ial in

Con

fiden

ce

© 2013 The SmartenIT Consortium

Evaluation of 3.7 billion accesses on 1.65 million YouTube files

Cache Efficiency for YouTube Video Traces

Sources: M. Cha et al., I tube, you tube, everybody tubes: Analyzing the world’s largest usergenerated content video system, Internet measurement conference IMC, San Diego, USA (2007)

Efficiency of caching for IP-based Content Delivery (G. Haßlinger, O. Hohlfeld, ITC 2010)

Results confirmed by N. Megiddo and S. Modha, Outperforming LRU with an adaptivereplacement cache algorithm, IEEE Computer, (Apr. 2004) 4-11

Com

merc

ial in

Con

fiden

ce


0%

10%

20%

30%

40%

50%

60%

0.0078% 0.031% 0.124% 0.5% 2%

Cache Size: Fraction of videos in the cache

Cac

he

Hit

Rat

e

Optimal Cache Strategy: Most Popular Data in CacheZipf Law Approximation: 0.004*R**(-5/8)LRU Cache Strategie: Least Recently Used

Cache Strategies incl. Statistics on Past Requests

Sliding Window: Cache holds objects with highest request frequency over a sliding window of the last K requests

Geometric Fading: Cache holds objects that have the highest sum of weights for past requests, where the kth request in the past has a geometrically decreasing weight r

k (0 < r < 1).

Com

merc

ial in

Con

fiden

ce


Statistics over window of the last K requests

Converges to caching of the most popular objects for large K Reacts to dynamic change in population, after delay until requests to new item are relevant in the statistics Implementation: The request sequence in the window has to be stored; for a new request one request is falling out of the window and has to be removed from statistics 2 objects change their statistics score per new request: Updates in cache still have constant effort per request, although more than for LRU

Com

merc

ial in

Con

fiden

ce


Statistics with geometrical aging

The k-th request in the past is weighted by k (<1) The weight of an object is the sum of the weights of request Objects are ordered according to their weights Implementation: In principle, all weights should be multiplied by for each request; instead, the new weight can be multiplied by

1/ i.e. weights are (1/k for the k-th request One object changes rank per request;

Effort for update rank in sorted list: O(ln(M))

Faster approx.: Requested object to step up noly one rank;

or rank updates only e.g. per hour or per day

Com

merc

ial in

Con

fiden

ce


Basic Assumptions on Cache Modeling & EvaluationWe assume a set of N objects and a cache for M (< N) objects of fixed size (objects of different size are handled as k unit size chunks; bin-packing problems are almost irrelevant in large caches) Random independent requests with static popularity pk: Request Probability to object k in the order of popularity

Optimum strategy holds the most popular objects in cache

Static popularity is favourable for the cache hit rate, since unforeseen changes in popularity detract from cache efficiency Measurement traces of request to Youtube show only slowly varying popularity, a few percent of new top 100 items appear per day/week

Com

merc

ial in

Con

fiden

ce


Results on LRU Caching Strategy

An LRU cache is implemented as a stack of dept M; A new request puts the object on top LRU is simple and frequently used (Squid, DropBox etc.)

Analysis of the hit rate for static distribution is possible:

but has complex evaluation feasible only for small size M < 15 Approximations by Towsley et al. (1999), Ha. & Ho. (2010), Fricker, Robert, Roberts (2011) seem to be good for arbitrary static request distribution but verified only by simulation

.1

...11

)(1

,...,1

1

1,111

LRU

11213

3 21

3

12

2 1

2

1

1

M

jk

N

kkkk

M

j k

kN

kkkk kk

kN

kkk k

kN

kk j

nM

Mj

M pp

p

pp

p

p

ppMh

Com

merc

ial in

Con

fiden

ce


Worst Case Analysis of LRU Caching Strategy

Cache size M =1 with only one popular popularity p1 >> > p2 , … When most popular item is always in cache optimum hit rate: p1; LRU hit rate is smaller: p1

2.

Arbitrary cache size M with a set IPop of M popular objects

p1 = p2 = … =pM = p/M >> > pM+1, pM+2, …

pLRU(j, k): probability of j popular items from the set IPop are found in an LRU cache of size k. We can analyse pLRU(j, k) iteratively:

.)()1(1

)1()1,1(

)1(1)1(1

)1,(),(LRU

jkpjpjM

kjpjkjpjkMp

kjpkjp

LRU hit rate hLRU = j pLRU(j, M)[ j p + (M – j) ].

pLRU(j, k-1) Last request to an object X not in the cache of size k-1

pLRU(j+1, k) XTop

+Cache of size k-1

XTop IPop

pLRU(j, k)XTop IPop

LRUCache

of size k

=

Com

merc

ial in

Con

fid

ence

Com

merc

ial in

Con

fiden

ce


Exact analysis of LRU worst case hit rate is feasible

Worst Case Analysis of LRU Caching

0%

20%

40%

60%

80%

100%

0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1

Worst Case LRU Scenario: Probability of a request to the set of popular objects

Cac

he H

it R

ate

Most popular items in cacheLRU Worst Case for Cache of Size 1LRU Worst Case for Cache of Size 2LRU Worst Case for Cache of Size 10LRU Worst Case for Cache of Size 50

28.9% max. absolute deficit severe relative deficits for small cache hit rate

Com

merc

ial in

Con

fiden

ce


Com

merc

ial in

Con

fiden

ce


Simulation Results for Caching Strategies

Hit rate of the caching strategies (N = 1000 objects; K = 1000) for Zipf distributed requests A(R) = R– (= 0.6; = 2.7%)

Com

merc

ial in

Con

fiden

ce


0%

10%

20%

30%

40%

5 10 20 50 100

Ca

che

Hit

Ra

te

M =

Most popular objects in the cacheGeometrical fadingSliding windowLRU ApproximationLRU Simulation

Com

merc

ial in

Con

fiden

ce



Hit rate of the caching strategies (N = 1000; K = 1000) for Zipf distributed requests A(R) = R– (= 0.99; = 13.9%)

Com

merc

ial in

Con

fiden

ce


0%

10%

20%

30%

40%

50%

60%

70%

5 10 20 50 100

Ca

che

Hit

Ra

te

M =

OptimumGeometrical fadingSliding windowLRU ApproximationLRU Simulation

Com

merc

ial in

Con

fiden

ce


Sliding Window and Geometrical Fading:Hit rate depending on the window size K, ( = K/(K + 1))

Com

merc

ial in

Con

fiden

ce


35%

40%

45%

50%

55%

60%

1 4 16 64 128 256 512 1024 2048

Cac

he

Hit

Rat

e

K =

Optimum for i.i.d. requests

Geometrical fading

Sliding window

LRU


Hit rate of the caching strategies (N = 1000) for Zipf distributed requests A(R) = R– (= 0.99; = 6.5%)

Conclusions on Cache Replacement Strategies

LRU seems most often used in web caches (Squid, DropBox) For static popularity, LRU is below the maximal hit rate by - 28.9% in the worst case - 10-20% for large content sites (YouTube; Zipf-like requests) LRU performance is poor especially for small caches Statistics over a fixed size window and geometric aging can converge to optimum hit rate of the static popularity case

Implementation: - Statistics over window needs some storage, has constant update effort per request but more than LRU - Geometric aging has effort O(ln(M)) Zipf law popularity makes (small) caches efficient

Com

merc

ial in

Con

fiden

ce


Technology

Evaluation of Caching Strategies Based on Access Statistics on Past Requests