Upload
charlene-long
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
1
Quality of Service Guarantees for Multimedia Digital Libraries
and Beyond
Gerhard Weikum
http://www-dbs.cs.uni-sb.de
2
Vannevar Bush’s Memex (1945)Collect all human knowledge into computer storage
Size of today‘s and tomorrow‘s applications:
Everything you see or hear: 1 MB/s * 50 years 2 PB
Library of Congress: 20 TB books + 200 TB maps + 500 TB video + 2 PB audio
Challenges: size of data• performance & QoS• intelligent search
3
Multimedia Data Management
Discrete Data Index Data Continuous Data
ParallelDiskSystem
ServerMemoryBuffer
Clients
High-speed Networkwith QoS Guarantees
. . .
QoSGuarantees
byData Server
4
Internal Server Error.Our system administrator has been notified. Please try later again.
Check Availability(Look-Up Will Take 8-25 Seconds)
The Need for Performance and QoS Guarantees
• Service performance is best-effort only• Response time is unacceptable during peak load because of queueing delays• Performance is mostly unpredictable !
Observations:
5
From Best Effort To Performance & QoSGuarantees
”Our ability to analyze and predict the performance of the enormously complex software systems ...are painfully inadequate”
(Report of the US President’s Technology Advisory Committee)
• Very slow servers are like unavailable servers• Tuning for peak load requires predictability of workload config performance function• Self-tuning requires mathematical models• Stochastic guarantees for huge #clients
6
Outline
The Need for Performance Guarantees
Towards a Science of QoS Guarantees
QoS for Continuous-Data Streams
Caching and Prefetching for Discrete Data
Self-tuning Servers using Stochastic Predictions
7
Performance and Service Qualityof Continuous-Data Streams
Quality of service (QoS): (almost) no "glitches"
High throughput (= concurrently active streams)
admission control
8
Data Placement and SchedulingPartitioning of C-data Objects with VBR (Variable Bit Rate)into CTL Fragments (of Constant Time Length)Coarse-grained Striping with Round-robin AllocationPeriodic, Variable-order SchedulingOrganized in Rounds of Duration T (= Fragment Time Length)
0 T 3T2T
...
0 T 3T2T
0 T 3T2T
2 2 2
1
2
3
1
2
1
2
3
1
2
1
2
3
1 1 1
3 3 3
1
2
3
1
1
2
Admission control:
No way!Now go ahead!
1
Yes, go ahead!
1 1
9
Admission Control
Stochastic QoS: Admit at most N streams such thatP [ total service time > T ]
- tolerable by most multimedia applications- appropriate with many workload and system parameters being random variables- allows much better resource utilization compared to worst-case modeling
threshold
Worst-case QoS: Admit at most N streams such thatN * Tmax T
with Stochastic QoS Guarantees
10
Mathematical Tools
X, Y, ...: continuous random variables with non-negative, real values
(cumulative) distribution function of X :][)( xXPxFX
probability density function of X :)(')( xFxf XX
:][)()(*0 sX
Xsx
X eEdxxfesf Laplace-Stieltjes transform(LST) of X
z
YXYX dxxzFxfzF0
)()()(Convolution
)(*)(*)(* sfsfsf YXYX
0|)(*inf][ X
t fetXP Chernoff bound
11
Total Service Time Per Round(With N Streams Per Disk)
T T T Tserv seek rot i trans ii
N
i
N
, ,
11f f f fserv seek rot
Ntrans
N* * * *
T N tseekZ
NSEEKseek
( ) :1
1 f s eseek
s SEEK* ( )
f se
s ROTrot
s ROT
* ( ) 1
f t
ROTrot ( ) 1
f sC ROT
C ROT strans* ( )/
/
F t F
tROT
Ctrans size( )
with
f x x esizex( ) ( ) / ( ) 1
P T t e fservt
serv[ ] inf * ( )
0
12
Total Service Time Per Round(With N Streams)
T T Tserv seek rot ii
N
i
N
,
11
T N tseekZ
NSEEKseek
( ) :1
1
f tROTrot ( )
1
F t Ft
ROTCtrans size( )
with
f x x esizex( ) ( ) / ( ) 1
P T t e fservt
serv[ ] inf * ( )
0
f f f fserv seek rotN
transN* * * *
f s eseeks SEEK* ( )
f se
s ROTrot
s ROT
* ( ) 1
f sC ROT
C ROT strans* ( )/
/
Ttrans,i
13
Stochastic versus Worst-Case QoS Guarantees
0
0,2
0,4
0,6
0,8
1
1,2
12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
analyticreal
N
p late
14
Stochastic versus Worst-Case QoS Guarantees
0
0,05
0,1
0,15
0,2
0,25
0,3
12 14 16 18 20 22 24 26 28 30
analyticreal
N
p late
analytic realworst-case
15
Generalization to Mixed-Workload Servers
0 T 3T2T
arrivals ofdiscrete-datarequests
departures of completed discrete-datarequests
4T
response time
response time
response time
Additional performance guarantee for discrete-data requests: ][ ttimeresponseP (e.g., with t = 2s and = 0.95)
Needs clever scheduling and sophisticated stochastic model,to provide both continuous-data and discrete-data guarantees
16
QoS & Performance Guarantees for Mixed Workload Servers
P [ glitch frequency of a stream > tolerance ] threshold
P [ admission/startup delay of a stream > tolerance ] threshold
forContinuousData
forDiscreteData
P [ response time > tolerance t ] threshold (e.g., t = 2 seconds, = 5 percent)
Detailed analytic model can derive minimum-cost server configurationfor specified QoS & performance requirementsincl. differentiated QoS for multiple user/request classes
Auto-Configuration of Data Server:
17
Outline
The Need for Performance Guarantees
Towards a Science of QoS Guarantees
QoS for Continuous-Data Streams
Caching and Prefetching for Discrete Data
Self-tuning Servers using Stochastic Predictions
18
The Need for Caching in Storage Hierarchies
Searchengine
Internet
Proxy
Clients
...
DL server
Ontologies,XML etc.
Very high access latency ! CachingPrefetching
100 TB
5 TB
50 GB
19
Basic Caching PoliciesLRU: Drop page that has been least recently used
Example:
time
AB
C DX Y X Y
AB
C DX Y X Y
AB
C DX Y X Y
1 2 3 4 5 10 15 20 24 now
LRU-k: Drop page with the oldest k-th last reference
estimates heat (p) =
optimal for IRM)( ptnow
k
k
20
LRU-k OptimalityIRM: pages 1 ... n with ref. probabilities 1 ... n (i i+1)
and backward distances b1 ... bn
timenow
3 2 13221323
b2
]|..[ dbprobrefhasxP xi
n
hhhx
iix
hasxPhasxdbP
hasxPhasxdbP
1][]|[
][]|[
]|..[ dbxofprobrefE x
n
hxhh dbhasxP
1]|[
Theorem: ...][......][... yExEbb yx
n
h
kdh
khk
d
kdi
kik
d
n
n
111
11
/1)1(
/1)1(
21
LRU-k as Maximum Likelihood Estimator
n
iibPL
1]|[
1 ii
for observation b1, ..., bn with bi < b i+1
maximize
n
i
kbi
kik
ib i
111 )1(
0)ln(
i
L 1/
/1
kb
bk
i
ii
ii b
k for k << bi
IRM: pages 1 ... n with ref. probabilities 1 ... n (i i+1)and backward distances b1 ... bn
timenow
3 2 13221323
b2
22
Cache Size Configuration
170
$50064
$5032
sMBKB 1min21.0
Keep page in cache if diskcache CC
Cost / throughput consideration:
yh
sMB
KB 1$
10164
$5032 19 y
Keep page in cache if waitcache CC
Cost / response-time consideration:
Minimum cache size M such that
goalpercentile RTMgfratiohitfRT ...)),((...),(
Response-time guarantee:
23
LRU-k Cache Hit Rate (for Cache Size M)][:)( WwindowintimeskleastatreferencedpagesdistinctEWP
jWi
ji
n
i
W
kjj
W
)1(1
)(:~ 1 MPW
][: cacheinresidesipagePpi jWi
ji
W
kj jW
~~ ~)1(
][:)( hitcacheisreferencePMH
n
iii p
1
20
40
60
80 LRU
LRU-2
LRU-3
A0
cache size M
hit rate H(M) [%]
24
Stochastic Response Time Guaranteewith cache size M, block size S, and multi-zone disk with known seek-time function, Z tracks of capacity Cmin Ci Cmax, rotation time T
n
iRdiskiiRcacheiiR tfptfptf
1)()1()()(
n
iRdiskiiR sfpsf
1
** )()1()(
0
* )()(t
Xst
X dttfesfwith LST
)(
)1()(
***
sfs
ssff
servdiskdiskservRdisk
n
iiidisk p
1)1(
][ servdisk tE
with
)()()()( **** sfsfsfsf transrotseekserv
M/G/1 queue:
)( inf ][
0
*RfetRP tChernoff bound:
25
Extended LRU-k-based Policies
Generalization to variable-size documents:
Generalization to non-uniform / hierarchical storage:
temperature (d) =
benefit (d) =
drop documents with lowest
drop documents with lowest
)()(
psizepheat
)(cos)( dtdetemperatur fetch
Generalization to cooperative cachingin computer cluster
singletisdifdtreplicaisdifdt
fetch diskcacheremotedt )(cos
)(cos)(cos
Speculative PrefetchingArchive
Cache
Mask high access latency
Speculative prefetching
Keep long-term beneficial data in cache
Throttling of prefetching
Prefetch x iff benefit(x,T) > {benefit(y,T) | y victims}
with benefit (x,T) =
))()(()(
][#xRTxRT
xsizeTtimeinxtoaccessesE
cachearchive
with time horizon T = „max“ (RTarchive)
Context-aware Prefetching and Caching
Session 1 Session 2 ...
accessdoc. i
accessdoc. k
doc. f doc. g doc. h
... ...
Pif=0.80.1
0.9
0.30.1 ...
P[time in i t]Hi=E[...]=10s
Hk=10s
Hf=30s
Sessionarrival rate
newsessions
...
...
HN+1=c/
...
0.1
with continuous state-residence timesModel session behavior as Markov chain
Superimpose CTMCs of all active sessions
Incorporate arrivals of new sessions
CTMC-based Access PredictionGiven: states di (i=1, ..., N+c) with transition probabilites pij and mean residence times Hi (departure rates i=1/Hi)
Uniformization:
ijfor
ijforpij
i
iijp
/1with = max{i}
N(x,T) = E[#accesses to x in T] = s j
jxjsstate p
TE
/1
)(),(
where
1
1
0
)(
!)(1
)(n
n
m
mij
nt
ij pnt
etE
cN
kkj
mik
mij ppp
1
)1()( jiifjiif
ijp 0
1)0(
Transient analysis for time horizon T:
MCMin Prefetching and Caching Algorithm
access tracking and online bookkeeping for statistics
periodic evaluation of N(state(s),T) for active sessionsbased on approximative CTMC transient analysis
prefetching candidates
Prefetch x iff benefit(x,T) > {benefit(y,T) | y victims}
with benefit (x,T) =
))()()(()(),(
xxRTxRTxsizeTxN
penaltycachearchive
)()()( xtxtx servservpenalty with
+ appropriate device scheduling at server
Overhead: • size of bookkeeping data < 0.02%• compute time per access 1 ms• both dynamically adjustable
Performance ExperimentsSimulations based on WWW-server access patterns
05
101520253035404550
0,20% 1% 2%
LazyTempMCMin
Mean response time [s]
Cache size / archive size
Applicability of LRU-k and MCMin Familyfor Internet and intranet proxies and clients
for data hoarding in mobile clients
for (stochastically) guaranteed response time
for caching of (partial) search results
w.r.t. heterogeneous data servers,as opposed to best-effort caching
when client goes on low (or zero) connectivity,prefetch near-future relevant data and programs
with careful management of access statistics
in data warehouses, digital libraries, etc.
for adaptive broadcast of data feedsin networks with asymmetric bandwidth
Interesting Research Problems
Response-time guarantee for MCMin
Optimal (online) decisions about amoung of bookkeeping
?
?
Caching and prefetching fordifferentiated QoS (multiple user/request classes)
?
Caching of (partial) search results andprefetching for (speculative) query evaluationin ranked (XML) retrieval
?
????
33
Outline
The Need for Performance Guarantees
Towards a Science of QoS Guarantees
QoS for Continuous-Data Streams
Caching and Prefetching for Discrete Data
Self-tuning Servers using Stochastic Predictions
34
Advancing the State of the Art on QoS
+ Substantially Better Cost/Performance
+ Major Building Blocks for Configuration Tool for Specified QoS Guarantees and Self-tuning, Zero-admin Operation
Benefit of stochastic models and derived algorithms/systemsover commercial state-of-the-art systems(e.g., Oracle Media Server, MS NetShow Theater Server, etc.):
+ Predictable Performance
35
QoS in (Web) Query Processing
Credibility
Timeliness
Responsiveness & Cost-effectivity (Performance)
AccuracyExample: Select ... ).( AmountOSum
ComprehensivenessExample: association rules of the kind
Software Engineering & Y2K Astrology
Combined with IR & MultimediaExamples: Where ... P About {„Mining“, „19th Century“} ...
Where ... P.Category =„CDs“ And P Sounds Like
36
The End
„low-hanging fruit“ engineering: 90% solution with 10% intellectual effort
self-tuningservers withguaranteedperformance
„Web engineering“ for end-to-end QoSwill rediscover stochastic modeling or will fail
need libraries of composable building blocks withpredictable behavior and (customizable) QoS guarantees
Conceivable killer argument:Infinite RAM & network bandwidth and zero latency (for free)
But:• An engineer is someone who can do for a dime what any fool can do for a dollar.• Predictions are very difficult, especially about the future.