A QoS approach for Multimedia Consumer Terminals - A case for Conditionally Guaranteed Budgets -

Reinder J. Bril, [email protected]/e Informatica, System Architecture and Networking

1

Reinder J. Bril

A QoS approach forMultimedia Consumer Terminals

- A case for Conditionally Guaranteed Budgets -

23-11-2004


2

Multimedia Consumer Terminals and QoS

• Multimedia Consumer Terminals– audio/video: perception is key– high volume electronics: cost-constrained– requires average-case resource allocation

• High quality audio and video:– have real-time requirements

• Quality of Service (QoS)– “collective effort of service performances that

determine the degree of satisfaction of the user of that service”(International Telecommunications Union)


3

Quality of Service Resource Management

(QoS-RM)

Scalable Video Algorithms

(SVA)

V-QoS

University of Madrid (dit/UPM)

University of Illinois at Urbana- Champaign (UIUC)

University of Mannheim

University of St. Petersburg

ITEA/Europa, ITEA/Robocop, OZONE, …

Multi-disciplinary QoS approach


4

Overview

• Multimedia Consumer Terminals• A QoS approach• Conditionally Guaranteed Budgets• Conclusion


5

Overview

• Multimedia Consumer Terminals– Media processing from dedicated HW to SW– Platforms are resource constraint– High quality video has real time requirements

• A QoS approach• Conditionally Guaranteed Budgets• Conclusion


6

Overview

• Multimedia Consumer Terminals• A QoS approach

– Adaptive applications– Budget-based resource manager– Control hierarchy

• Conditionally Guaranteed Budgets• Conclusion


7

Overview

• Multimedia Consumer Terminals• A QoS approach• Conditionally Guaranteed Budgets

– Resource allocation conflict– Extension of QoS approach– Analysis

• Conclusion


8

Multimedia Consumer Terminals

DVD CDxfront end

YC interface

IEEE 1394interface

DVB Tuner

Cable modem

CVBSinterface

VGA

RF Tuner

Focus:

Receivers in broad-cast environments

High-quality video applications


9

Traditional High-End TV Architecture

TXT CPUacq control

progROM

RAM

PICNICenhancement

PALplusIC16:9 helper

MEM MEM

HFelectr.

standarddecoder

PICNICenhancement

picturecontrol

audiodemod.

Audioproces.

cableantenna

PAL/NTSC

display

YUV1fh

YUV2fh

RGB2fh

picturecontrol

standarddecoder

PiP+

Mem RGB2fh

CVBS

MEM

100HzFALCON

IC

MEM

MEM

NICAMdecod.

demod

MPEGaudio

MPEGvideo

MEM

chan.decod.

transp.demux

channelbits

Traditional TV sets and

Set-Top Boxes:

• Fixed algorithms for fixed HW architectures

• Upgrade for new services and applications is problematic

• Systems are not flexible


10

Digital video platform

Bus

VLIWCPU

MIPSCPU

Memory

Coprocessors

Expectations:

• Upgradeable for new servicesand applications

• Fast time-to-market for new features

• Enabling approach for product families


11

HW Architectures vs. SW Applications

Low-

end

Mid-

range

High-

end

Resources

ProductFamilies

SW-Modules

Algorithm1

Algorithm2

Algorithm4

Algorithm3

Algorithm1 m

in max

Algorithm3

Algorithm2

Algorithm4


12

Low-

endAlgorithm

1

Algorithm2

Algorithm4

Algorithm3

Algorithm1

Mode 1 Mode 2 Mode 3

Flexibility


13

Low-

end

Mode 1 Mode 2

Algorithm3

Alg. 4

Algorithm1 Alg. 1

Algorithm3

Scalability


14

Low-

end

Initial Transitory Final

Algorithm3

Algorithm1 Alg. 1

Algorithm3

Alg. 4

Alg. 1

Algorithm3

Smooth mode transition


15

Platform constraints

Cost

Functionalitytarget limit

traditionalsystems

scalableapproach

Functionality

Quality

target limit

traditionalsystems

scalableapproach


16


• Cost-effectiveness requirement– High volume: low bill of material– Low power– Software solutions are relatively expensive

(mm2 silicon / power)– Average case and worst-case are far apart


17

Fluctuating resource requirements

time

load

structural load

running average

temporal load

worst-case


18

Fluctuating resource requirements

• Temporal load changes (very frequent)– from I frame to B frame– more or less motion– transient high peaks

• Structural load changes (less frequent)– scene change– from movie to camera– from news to commercial


19


• Cost-effectiveness requirement– High volume: low bill of material– Low power– Software solutions are relatively expensive

(mm2 silicon / power)– Average case and worst-case are far apart

• Conclusion:Aim for average-case resource allocation


20

High quality video

original

up-scaled

Rendered stream: 60 Hz (TV screen)

Input stream: 24 Hz (movie)

TV companies invest heavily in video enhancement,e.g. temporal up-scaling


21

High quality video

original

up-scaled



displayed

• Deadline miss leads to “wrong” picture.

• Deadline misses tend to come in bursts (heavy load).

• Valuable work may be lost.


22

High quality video

original

up-scaled


QoS trade-off (at run-time):

Lesser picture quality often better than temporal incorrectness.


displayed


23

High quality video

• QoS = degree of user satisfaction

• User satisfaction has to do with perception:– Lesser picture quality often better than temporal

incorrectness.– Quality fluctuations are perceived as non-quality.– With a scene change, the brain needs some time to re-

focus.– Most people focus on one thing at a time (user focus)– User focus normally is at

• the center of a window,

• the window that received the latest (remote) command.

• Only video specialists can make the necessary trade-offs.


24

Multimedia Consumer Terminals - Summary

• Media processing from dedicated HW to SW– Flexibility & scalability– Fast time-to-market– Product families

• Platform constraints– Aim for average-case resource allocation

• High quality video– Has real-time requirements;– is about perception -> QoS;– real-time is a QoS parameter;– QoS is primarily an application-domain issue.


25

Overview

• Multimedia Consumer Terminals• A QoS approach

– Adaptive applications– Budget-based resource manager– Control hierarchy

• Conditionally Guaranteed Budgets• Conclusion


26

User applications

Media processing

pip

mode 2: main + pip

main

mode 1: main

disk

mode 3: main + pip + disk

Modes

Input Output

DVD

In mode 3, main + pip + disk can not run at highest quality.


27

disk : non-scalable

main: scalable

mixer : non-scalabledigitizer: non-scalable pip: scalable

hierarchical task

digit

Application execution model (prototype)

DVD dmux

audiodec.

dec.sharpenh.

mixer

audiorend

read

scaler

enc.hwscaler

enc. writer

dec

enc.

scalable task

task

connection to HW IO

buffer

data transferBoth main and pip are scalable.


28

Scalable Algorithm

FUNCTION 1 FUNCTION 2

FUNCTION 4

FUNCTION 3

QUALITY CONTROL

control signal for

quality level

signal in

signal out

ALGORITHM FOR MEDIA PROCESSING

Quality

Resource needs


29

s1 s2

DetailFilter

NoiseMeasurement

AmplitudeControl

Filter AC NM

No No No

1D No No

2D No No

2D Yes Yes

QL0

QL1

QL2

QL3

Example: Sharpness Enhancement


30

Sharpness Enhance.

0 10 20 30Resource needs [MIPS]

Qua

l. es

timat

e

QL0 QL1 QL4

Down-Scaler

0 5 10 15Resource needs [MIPS]

Qua

l. es

timat

eQL0 QL2 QL4

Resource range for SVA examples


31

Budget-based resource manager

CPU timeapp. 1

time

budget 1

CPU timeapp. 2 budget 2

• Temporal isolation:– Reserved, guaranteed, and enforced budgets

• e.g. 20% of the CPU time every 20 ms;

– Applications run on a “virtual platform”;


32

Dynamic load (reminder)

• Temporal load changes (very frequent)– from I frame to B frame– more or less motion– transient high peaks

• Structural load changes (less frequent)– scene change– from movie to camera– from news to commercial

• Mode change (infrequent)– functional change– external trigger


33

Control hierarchy

• Dynamic load at different time-scales:– Temporal load changes;– Structural load changes;– Mode change.

• Layers of control, e.g.– local quality control of applications;– global system utility control.


34

Adaptive applications

Provide quality levels + estimated resource req.

Co-operative QoS approach

Resource managerProvides guaranteed resource budgets

Local quality control

SVAs…

Global system utility controlOptimizes system utility, sets quality levels + allocates resources


35

Overview

• Multimedia Consumer Terminals• A QoS approach• Conditionally Guaranteed Budgets

– Why: Resource allocation conflict– How: Extension of QoS approach– Analysis

• Conclusion


36

Resource requirements (reminder)

time

load

structural load

running average

temporal load


37

time

load

Resource allocation: worst-case

Not cost-effective

structural load

running average

temporal load


38

time

load

Resource allocation: close to average

Instantaneous budget increase

Not yet feasible

structural load

running average

temporal load


39

time

load


Instable output quality

Not acceptable forimportant applications

structural load

running average

temporal load


40

time

load


“wasted”

Not cost-effective

structural load

running average

temporal load


41

Resource allocation conflict

Structural load increase

+close-to-average resource allocation

yields– either instable output quality

not acceptable for important applications– or “wasted” resources

not cost-effective


42

time

load

Conditionally guaranteed budgets: Why?

Instantaneous budget increase

B

structural load

running average

temporal load

B

Anticipated increase


43

Conditionally guaranteed budgets: How?

Basic approach (MIA and LIA):– Two modes of quality settings + allocation:

QMIA, BMIA+BMIAQMIA, BMIA

QLIA,N, BLIA +BLIA QLIA,A, BLIA

Anticipated mode(high load for MIA)

Normal mode(low load for MIA)

MIA

LIA

CGB


44



Resource manager (RM)

Global system utility control

MIA LIA

Inform MIA, LIA, and RM about both modes

Normal mode

Anticipated mode

Normal mode

Anticipated mode

Normal mode

Anticipated mode

modes


45





MIANormal mode

Anticipated mode

Normal mode

Anticipated mode

LIANormal mode

Anticipated mode

Running in normal mode


46


MIA detects load increase,

claims BMIA, and switches

mode




MIANormal mode

Anticipated mode

Normal mode

Anticipated mode

LIANormal mode

Anticipated mode

Claim BMIA

Mode transition


47





MIANormal mode

Anticipated mode

Normal mode

Anticipated mode

LIANormal mode

Anticipated mode

RM switches mode instantaneously, andinforms LIA

Inform LIA

Mode transition


48





MIANormal mode

Anticipated mode

Normal mode

Anticipated mode

LIANormal mode

Anticipated mode

LIA switches mode

Running in anticipated mode


49


• Summary basic approach:– Assumption:

• Anticipation of resource needs and modes;

– Allocation phase:• Informing MIA, LIA, and RM about modes;• Delegation of mode changes to MIA;

– Execution phase:• Release and claim of resources by MIA;• Instantaneous mode change by RM.


50


• How to change budgets instantaneously ?• In-the-place-of resource consumption

– LIA consumes BLIA exactly whenMIA would have consumed BMIA


51

In-the-place-of budget consumption

BMIA BMIA

MIA

LIA

Anticipated mode


52


BMIA BMIA

MIA

LIA

Normal mode


53


BMIA BMIA

MIA

LIA

Normal mode


54


BMIA BMIA

MIA

LIA

Claim BMIA

Mode switch


55

Conditionally guaranteed budgets

• Analysis

• How to determine BLIA?– Worst-case (i.e. minimal) amount that can be

guaranteed on a periodic basis.

• Cognac-glass algorithm


56

Cognac-glass algorithm

• How to determine the worst-case BLIA?

MIA

LIA

BMIA BMIA

R

TMIA

TLIA


57

• How to determine the worst-case BLIA?

• Worst-case for R when BMIA is available:– as early as possible for first overlapping interval;

• “best-case” analysis

– as late as possible for last overlapping interval.• “worst-case” analysis

– based on notion of advancement

• Minimize for all values of R.



58


• Example– fixed-priority preemptive scheduling– set of four applications

?

MIA

LIA

A2

A1

29

31

14

6

9

-

2

1

7

period budget

Bi BiTi


59

Advancement

5 10 15 20 25 30

5

10

15

time t

BMIA

BMIA + BMIA

MIA

AMIA(t)


60

Advancement

5 10 15 20 25 30

5

10

15

time t

BMIA

BMIA + BMIA

WAMIA(t)

Worst-case advancement


61

Advancement

5 10 15 20 25 30

5

10

15

time t

BMIA

BMIA + BMIA

Worst-case and best-case advancement

WAMIA(t)BAMIA(t)


62

Situation of worst-case availability

5 15 20 25 30

5

10

15

time t

35 40 45 50 55 60 10

TMIA TMIA

BMIA

BMIA + BMIA

BAMIA(t) WAMIA(t TMIA)


63


5 15 20 25 30

5

10

15

time t

35 40 45 50 55 60 10

TLIA

BMIA

BMIA + BMIA

R

BBMIA(R)

BWMIA(R+TLIATMIA)


64


5

5 15 20 25 30 time t

35 40 45 50 55 60 10

TLIA

BMIA

R

BBMIA(R)

BWMIA(R+TLIATMIA)

BLIA = R

min(BBMIA(R) + BW

MIA(R+ TLIA – TMIA))


65


5

5 15 20 25 30 time t

35 40 45 50 55 60 10

TLIA

BMIA

R

BBMIA(R) BW

MIA(R+TLIA-TMIA)

BLIA = R

min(BBMIA(R) + BW

MIA(R+ TLIA – TMIA))


66


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R

The curve looks like the shape of a glass.

Changing the relative phasing R is like tilting the glass…

hence, cognac-glass algorithm.


67


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R,lwb

Range for R


68


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R,upb

Range for R : [R,lwb, R,upb]


69


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R

Range for R

Domination values of R

: [R,lwb, R,upb]


70


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R

Range for R


: [R,lwb, R,upb]


71


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R

Range for R


: [R,lwb, R,upb]


72


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R

Range for R


: [R,lwb, R,upb]


73


5 15 20 25 30 time t

35 40 45 50 55 60 10

5

TLIA

BMIA

R,min = 17

Range for R


: [R,lwb, R,upb]

BLIA

BLIA = 5


74

• Summary of the analysis:– Based on notion of advancement;– requires best-case next to worst-case analysis;

– restricted to subset of values for R.

• See [Bril 04] for:– a generalization to arbitrary periods TMIA and TLIA;

– formalization;– efficient calculation.



75

Overview

• Multimedia Consumer Terminals• A QoS approach• Conditionally Guaranteed Budgets• Conclusion


76

Conclusion

[Bril 04] R.J. Bril, Real-time scheduling for media processing using conditionally guaranteed budgets, PhD thesis TU/e, IPA Dissertation Series 2004 – 13, Sept. 2004, http://alexandria.tue.nl.extra2/200412419.pdf.


77

Acknowledgement

• Research (PhD):– Emile H.L. Aarts (TU/e);

– Gerhard Fohler (Mälardalen University, Sweden);

– Wim F.J. Verhaegh (Philips Research);

– Christian Hentschel (Brandenburger University, Germany);

– Johan J. Lukkien (TU/e);

– Peter D.V. v.d. Stok (TU/e).

• V-QoS program:– All program members and partners;

– Clara M. Otero Pérez;

– Clemens C. Wűst.

Documents

A QoS approach for Multimedia Consumer Terminals - A case for Conditionally Guaranteed Budgets -