NUS.SOC.CS5248 Ooi Wei Tsang 1 Proxy Caching for Streaming Media

1

NUS.SOC.CS5248Ooi Wei Tsang

Proxy Caching for Streaming Media

NUS.SOC.CS5248Ooi Wei Tsang

2

You Are Here

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248Ooi Wei Tsang

3

Cache Proxies for Web

A

NUS.SOC.CS5248Ooi Wei Tsang

4

Hierarchical Caching

A

B

NUS.SOC.CS5248Ooi Wei Tsang

5

Cooperative Caching

A B

NUS.SOC.CS5248Ooi Wei Tsang

6

Distributed Caching

A B

NUS.SOC.CS5248Ooi Wei Tsang

7

Streaming Media vs. Webpage

NUS.SOC.CS5248Ooi Wei Tsang

8

Video Access Pattern

by S. Acharya and B. Smith in 1999

Study at Lulea University, Sweden 55% complete, 45% stop very early High temporal locality

NUS.SOC.CS5248Ooi Wei Tsang

9

Prefix Access Distribution

10 30 50 70 90

% Played

NUS.SOC.CS5248Ooi Wei Tsang

10

Video Popularity

Assume Zipf “Law”Probability of access to i-th most popular video is

11

NUS.SOC.CS5248Ooi Wei Tsang

Benefits of Caching

NUS.SOC.CS5248Ooi Wei Tsang

12

Reduce Access Latency

:) :(

NUS.SOC.CS5248Ooi Wei Tsang

13

Reduce Server Load

NUS.SOC.CS5248Ooi Wei Tsang

14

Reduce Start-up Latency

NUS.SOC.CS5248Ooi Wei Tsang

15

Hide Network Congestion

NUS.SOC.CS5248Ooi Wei Tsang

16

Other Issues

What to cache?Who to fetch from?When cache is full, who to kick out?

How to measure popularity?Can cache adapt to popularity?

17

NUS.SOC.CS5248Ooi Wei Tsang

What to Cache?

NUS.SOC.CS5248Ooi Wei Tsang

18

Segmentation

Cache “all or none” is badDivide media file into segments and consider each segment individually

NUS.SOC.CS5248Ooi Wei Tsang

19

Effects of Size S

Large S : Low utilizationSmall S : Lots of gaps

NUS.SOC.CS5248Ooi Wei Tsang

20

Prefix Caching Policy

1 Chunk = k segments

NUS.SOC.CS5248Ooi Wei Tsang

21

Caching Policy

Basic unit of caching: segmentCache prefix in chunkReplace suffix in chunkNever replace segments in curr chunk

22

NUS.SOC.CS5248Ooi Wei Tsang

Where To Fetch From?

NUS.SOC.CS5248Ooi Wei Tsang

23

Cooperative Caching

A B

NUS.SOC.CS5248Ooi Wei Tsang

24

Fetch from server

B

A

Server

Client 2Client 1

NUS.SOC.CS5248Ooi Wei Tsang

25

Fetch from fellow proxy

B

A

Server

Client 2Client 1

NUS.SOC.CS5248Ooi Wei Tsang

26

Issues

How to advertise?How to choose “helper”?

NUS.SOC.CS5248Ooi Wei Tsang

27

How to Advertise?

Balance betweennetwork loadfreshness of information

NUS.SOC.CS5248Ooi Wei Tsang

28

Scalable Advertisement

Expanding Ring Advertisement

16 132 264 4128 8

TTL PERIOD

NUS.SOC.CS5248Ooi Wei Tsang

29

How to Choose Helper?

Consideration for Static Cache network distance (1,2,3,4) number of streams being served avoid frequent switches

Build a cost function, integrating the metrics

NUS.SOC.CS5248Ooi Wei Tsang

30

Cost Function

Cost for retrieving a segment from node X to node Y=

NUS.SOC.CS5248Ooi Wei Tsang

31

Algorithm

Consider the next gap in local caches

Find the next helper with minimum cost, and can fill in at least k segments in gaps

32

NUS.SOC.CS5248Ooi Wei Tsang

Distributed Caching

Y. Chae et. al.JSAC 2000

NUS.SOC.CS5248Ooi Wei Tsang

33

Cooperative vs DistributedCooperative caching caches independently

Distributed caching caches as a team

NUS.SOC.CS5248Ooi Wei Tsang

34

Cold Start

B

A

Server

new clip!

NUS.SOC.CS5248Ooi Wei Tsang

35

Segment Map

Local segment mapWhich segment I should cache?

Global segment mapWho is suppose to cache what?

NUS.SOC.CS5248Ooi Wei Tsang

36

Cache Hit

B

A

Server

NUS.SOC.CS5248Ooi Wei Tsang

37

Cache Miss

B

A

Server

NUS.SOC.CS5248Ooi Wei Tsang

38

Distributed Caching

B

A

Server

NUS.SOC.CS5248Ooi Wei Tsang

39

Problems

Who should cache what?

Which segment to kick out?

How to adapt segment distribution?

NUS.SOC.CS5248Ooi Wei Tsang

40

Who should cache what?

RCache schemeSegment video into equal size segments

A proxy will cache each segment with some probability

NUS.SOC.CS5248Ooi Wei Tsang

41

RCache

Np proxies

video of length Lv

divide into Ns equal segments

Each proxy caches each segment with a/Np probability

NUS.SOC.CS5248Ooi Wei Tsang

42

Analysis

Probability that whole video is cached is

Ns:: num of segmentsNp: num of proxiesa/Np: prob of caching 1 segment

NUS.SOC.CS5248Ooi Wei Tsang

43

RCache’s Segmentation

Video is divided into segments of equal length

Can we do better?

NUS.SOC.CS5248Ooi Wei Tsang

44

Bimodal Distribution

10 30 50 70 90

% Played

NUS.SOC.CS5248Ooi Wei Tsang

45

Silo

segment size probabilityof storage

NUS.SOC.CS5248Ooi Wei Tsang

46

Further Improvement

segment size probabilityof storage

NUS.SOC.CS5248Ooi Wei Tsang

47

Problems

Who should cache what?

Which segment to kick out?

How to redistribute data?

NUS.SOC.CS5248Ooi Wei Tsang

48

Segment “Popularity”

For each video iFor each segment j

F(i,j) = Prob(i is accessed)*Prob(j is accessed)

NUS.SOC.CS5248Ooi Wei Tsang

49

Rainbow Algorithm

less popular

NUS.SOC.CS5248Ooi Wei Tsang

50

Rainbow Algorithm

Problem: Many large video – too many segments

computationally expensive to sort

Solution: Just approximate by quantizing popularity

NUS.SOC.CS5248Ooi Wei Tsang

51

Rainbow Algorithm

less popular

NUS.SOC.CS5248Ooi Wei Tsang

52

Problems

Who should cache what?

Which segment to kick out?

How to redistribute data?

NUS.SOC.CS5248Ooi Wei Tsang

53

Data Redistribution

When popularity changes, need to redistribute.

Redistribute “on-demand” (lazy)

NUS.SOC.CS5248Ooi Wei Tsang

54

Cache Token

Each segment have two bits(T,C)

T: I am suppose to have the segment

C: I have the segment

NUS.SOC.CS5248Ooi Wei Tsang

55

Data Redistribution

(T=1,C=1)(T=0,C=0)(T=1,C=0) (T=0,C=1)

NUS.SOC.CS5248Ooi Wei Tsang

56

Example

I J

A CB D

old

new

T=0,C=1 T=0,C=1

T=1,C=0 T=1,C=0 T=1,C=0 T=1,C=0

NUS.SOC.CS5248Ooi Wei Tsang

57

Example

I J

A CB D

old

new

T=1,C=1 T=0,C=1

T=0,C=0 T=1,C=0 T=1,C=0 T=1,C=0

NUS.SOC.CS5248Ooi Wei Tsang

58

Example

I J

A CB D

old

new

T=1,C=1 T=1,C=1

T=0,C=0 T=1,C=0 T=1,C=0 T=0,C=0

NUS.SOC.CS5248Ooi Wei Tsang

59

Example

I J

A CB D

old

new

T=1,C=1 T=1,C=1

T=0,C=0 T=1,C=1 T=1,C=0 T=0,C=0

60

NUS.SOC.CS5248Ooi Wei Tsang

Quality Adaptive Caching

Reza Rajaie et. al.INFOCOM 2000

NUS.SOC.CS5248Ooi Wei Tsang

61

Scenario (10am)

A

Server

Client 2

NUS.SOC.CS5248Ooi Wei Tsang

62

Scenario (1am)

A

Server

Client 2

NUS.SOC.CS5248Ooi Wei Tsang

63

Problem:Cache interfere with congestion control algorithm

Solution:Make cache aware of quality adaptation

NUS.SOC.CS5248Ooi Wei Tsang

64

Solutions

Making cache “quality-aware”PrefetchReplacement Algorithm

NUS.SOC.CS5248Ooi Wei Tsang

65

Cache Miss

A

Server

Client 2

NUS.SOC.CS5248Ooi Wei Tsang

66

Cache Hit

A

Server

Client 2

repair +prefetch

NUS.SOC.CS5248Ooi Wei Tsang

67

Peeking Inside the Cache

NUS.SOC.CS5248Ooi Wei Tsang

68

Cache Hit: Repair

NUS.SOC.CS5248Ooi Wei Tsang

69

Cache Hit: Prefetch

NUS.SOC.CS5248Ooi Wei Tsang

70

Prefetch Algorithm

prefetchwindow

playbackpoint

NUS.SOC.CS5248Ooi Wei Tsang

71

Proxy Request to Server

Multiple requests (for different clients) are batched.

NUS.SOC.CS5248Ooi Wei Tsang

72

Server Response

Missing segments are sent in decreasing priority

1

2

3 4

NUS.SOC.CS5248Ooi Wei Tsang

73

Server Response

Send as many segments as possible until next prefetch request

NUS.SOC.CS5248Ooi Wei Tsang

74

Trade-offs

How far in the future should we prefetch?

NUS.SOC.CS5248Ooi Wei Tsang

75

Solutions

Making cache “quality-aware”PrefetchReplacement Algorithm

NUS.SOC.CS5248Ooi Wei Tsang

76

Goal: converge to efficient state

if a stream is popularaverage quality is highvariation in quality is low

Goal of Replacement

NUS.SOC.CS5248Ooi Wei Tsang

77

The Algorithm

NUS.SOC.CS5248Ooi Wei Tsang

78

Trashing and Locking

NUS.SOC.CS5248Ooi Wei Tsang

79

Choosing Victim

whit (weighted hit) =Tplay/Ttotal

Calculate whit for each layer in a stream over a popularity window

NUS.SOC.CS5248Ooi Wei Tsang

80

Example

WHIT Lock? Name Layer

5.9 1 Nemo 0

4.3 1 Nemo 1

4.0 0 Matrix 2 0

3.9 0 Matrix 2 1

1.1 0 Gigli 0

NUS.SOC.CS5248Ooi Wei Tsang

81

Static Cache

Cache segments in proxy do not changed over time

Can we exploit further properties of streaming media?

82

NUS.SOC.CS5248Ooi Wei Tsang

Dynamic Caching

NUS.SOC.CS5248Ooi Wei Tsang

83

Motivating Scenario

t = 0, R1 requests for stream M

t = , R2 requests for stream M

Ideally, R1 and R2 should share a multicast of M

NUS.SOC.CS5248Ooi Wei Tsang

84

needs to be“patched”

share withR1 from cache

NUS.SOC.CS5248Ooi Wei Tsang

85

Using Dynamic Cache

R2 request stream MProxy allocate a ring bufferCache the most recent -seconds of M sent to R1

R2 get prefix of M from other places, and rest from proxy

NUS.SOC.CS5248Ooi Wei Tsang

86

Extending to N Receivers