Winter 2008 Internet QoS1 Internet Quality of Service Principles for QoS Traffic Policing and Scheduling Policies –Lucky Buckets, Weighted Fair Queueing,

winter 2008 Internet QoS 1

Internet Quality of Service• Principles for QoS• Traffic Policing and Scheduling Policies

– Lucky Buckets, Weighted Fair Queueing,

• Basic QoS Theory– fluid model, arrival and service curves– bandwidth and delay guarantees

• Internet QoS Architectures and beyond– InterServ Architecture– DiffServ Architecture– Core Stateless (see optional readings [SZ99][ZDH00])


Improving QOS in IP NetworksThus far: “making the best of best effort”Future?: next generation Internet with QoS guarantees

– RSVP: signaling for resource reservations– Differentiated Services: differential guarantees– Integrated Services: firm guarantees

• simple model for sharing and congestion studies:


Principles for QoS Guarantees

• Consider a phone application at 1Mbps and an FTP application sharing a 1.5 Mbps link. – bursts of FTP can congest the router and cause audio packets to be dropped. – want to give priority to audio over FTP

• PRINCIPLE 1: Marking of packets is needed for router to distinguish between different classes; and new router policy to treat packets accordingly

e.g. MPLS, Diffserv, RSVP


Principles for QoS Guarantees (cont’d)• Applications misbehave (audio sends packets at a rate higher than 1Mbps

assumed above); • PRINCIPLE 2: provide protection (isolation) for one class from other classes • Require Policing Mechanisms to ensure sources adhere to bandwidth requirements; Marking

and Policing need to be done at the edges:e.g. leaky bucket,WFQ


Principles for QoS Guarantees (more)

• Alternative to Marking and Policing: allocate a set portion of bandwidth to each application flow; can lead to inefficient use of bandwidth if one of the flows does not use its allocation

• PRINCIPLE 3: While providing isolation, it is desirable to use resources as efficiently as possible


Principles for QoS Guarantees (more)

• Cannot support traffic beyond link capacity• PRINCIPLE 4: Need a “Signaling” or “Call

Admission” Process; application flow declares its needs, network may block call if it cannot satisfy the needs


Four Pillars of QoS


Traffic Policing Mechanisms

• Three criteria: – (Long term) Average Rate (100 packets

per sec or 6000 packets per min??), crucial aspect is the interval length

– Peak Rate: e.g., 6000 pkts per minute Avg and 1500 pkts per sec Peak

– (Max.) Burst Size: Max. number of pkts sent consecutively, ie over a short period of time


Leaky/Token Bucket Mechanism

• Leaky/Token Bucket mechanism, provides a means for limiting input to specified Burst Size and Average Rate– Leaky: when token bucket full, tokens lost


Dual Leaky/Token Bucket• Limiting input to specified Burst Size Average

Rate and Peak Rate R– one with buffer: token rate r and buffer size b– another with “no” buffer: token rate p in practice: needs to be “packetized” - buffer of max.

packet size M

b

M

minpackets

p tokens/sec

rtokens/sec


Packet Scheduling

• Scheduling: choosing the next packet for transmission on a link can be done following a number of policies

• pre-emptive vs. non-preemptive:– non-preemptive: packet currently being transmitted will not

be terminated --- assumed in most scheduling algorithms

• work-conserving vs. non-work-conserving– work-conserving: whenever there are packets queued,

scheduler will schedule a packet for transmission– non-work-conserving: transmission may be idle even there

may be packets queued


Scheduling Policy: FIFO• FIFO: in order of arrival to the queue; packets

that arrive to a full buffer are either discarded, or a discard policy is used to determine which packet to discard among the arrival and those already queued

• Simplest scheduling policy, default policy


Scheduling Policy: Priority Queueing• Priority Queuing: classes have different priorities; class

may depend on explicit marking or other header info, eg IP source or destination, TCP Port numbers, etc.

• Transmit a packet from the highest priority class with a non-empty queue

• Preemptive and non-preemptive versions


Scheduling Policy: Round Robin

• Round Robin: scan class queues serving one from each class that has a non-empty queue• Extension: “weighted” round robin


Scheduling Policy: WFQ• Weighted Fair Queuing (WFQ): is a generalized Round

Robin– no concept of “round”, no fixed order of serving queues

– based on “fluid model”: if queue assigned a weight wi, and is served with (a minimum) rate wiC when backlogged (non-empty)

• C link capacity, wi =1

– if all queues backlogged, each queue served at rate of wiC;

– otherwise, “spare capacity” proportionally allocated to each backlogged queue (thus the name “fair queueing”!)


WFQ Implementation• Concept of Virtual Time:

– Packet arrival times of flow i packets: aij; size Lij

– What would be the finish times of these packets if flow i were served by a dedicated link of C_i = w iC?

Let s_{ij} be the time packet j of flow i being servicedand f_{ij} be the time it finishes transmission

for 1st packet: si1 = ai1; f i1 = si1 +Li1/Ci

for jth packet: sij = max {aij, fi,j-1 }; fij = sij + Lij/Ci

Note that jth packet queued if aij < fi,j-1

aij sij fij

Ci

“Ideal” dedicatedlink for flow i


Virtual Clock Scheduling Algorithm

• Virtual Clock: a “first” approximation of WFQ– Given N flows, each assigned with weight w i, wi =1

– Calculate the virtual finish time fij of each packet of each flow as if it were served at a rate of C i =wiC

– Schedule packets based on fij, i.e., packet with smallest fij among all queued packets is selected for transmission

• why not scheduled based on sij ?

• What is the key difference between Virtual Clock and Weighted Fair Queueing – WFQ: also know as Generalized Processor Sharing (GPS)


WFQ Implementation (cont’d)• “Ideal dedicated link” model does not take into account the

distribution of “spare capacity” when some queues are empty– Need to keep track which queues are backlogged and which are

empty

Let B(t) ½ {1,…,N} denote the set of queues backlogged at time tThen for flow/queue i 2 B(t), its actual service rate is

which is time-independent

Modification of the “ideal dedicated link model”: sij = max {aij, fi,j-1 }; fij = sij + Lij/Ci(t)

• Schedule packets based on the (modified) virtual finish time fij

• What states do we need to maintain to compute fij’s ?

wiPk wk

wiPk

wk

wiPk

wk

Ci(t) := wiPk2 B (t) wk

C


Basic QoS Theory• Concept of Arrival Curve (or “arrival

envelope”)– Let x(t) be cumulative amount of traffic sent by a

flow by time t (>=0); t=0 is the starting time.– We say (t) is an arrival curver (“envelope”) of x(t) if and

only if for all s · t, we have x(t) –x(s) · (t-s)

• Examples of arrival curves:– a constant bit rate flow: (t) = Rt– a <r,b>-leaky-bucket-policed flow: (t) = rt +b – a dual-leaky-bucket-policed flow: (t) = min{M+pt,rt+b} where p is the peak rate, M is the max. pkt size

STEXPH2:12:1

Equivalently, x(t) · inf0 · s · t{(t-s)+x(s)}: = ( x)(t) What does this equation intuitively mean?


Illustration of Arrival Curve• Arrival curve – maximum amount of bits

transmitted during an interval of time Δt

Δt

bitsArrival curve

time

bps

bits Arrival curve

time

bps

0 1 2 3 4 5

1

2

1 2 3 4 5

1

2

3

4

(R=2,b=1,r=1)

Δt


Service Curve • Concept of Service Curve

– Let a link (or rather a scheduler) as a “server” S– y(t): cumulative amount of traffic (in bits) of a flow leaving scheduler S by time t (¸ 0)

– We say scheduler S offers the flow a service curve (t) if and only if for all t ¸ 0, there exists t0 ¸ 0, with t0 · t, such that

y(t) –x(t0) ¸ (t-t0)

STEXPH2:12:1

Sflow x(t) y(t)

Equivalently, y(t) ¸ inf0· s · t {(t-s)+x(s)} =( x)(t) What does this equation intuitively mean?


Service Curve: Example • A fixed delay server (defined in IETF IntServ)

– For a flow with a bandwidth reservation R at a router,IETF IntServ assumes a router S will guarantee a

service curve of the following form:

where T represents the packetization and other processing delay incurred by router S in processing packets of the flow

STEXPH2:12:1

¯R ;T (t) := R[t ¡ T]+ =

½R(t ¡ T) if t > T0 otherwise

Δt

bitsArrival curve

time

bps

TR


Backlog and Delay Bounds

• Given a flow has an arrival curve , and is offered a service curve then – backlog at time t is bounded by – (virtual) delay at time, d(t): = inf{¸ 0: x(t) · y(t+)} is bounded by h(,) where

w(t) := x(t) ¡ y(t) · w(t) := sups¸ 0f®(t) ¡ ¯(s)g

STEXPH2:12:1

t

bitsarrival curve

service curve

w(t)d(t)

h(®;¯) := sups¸ 0d(s)andd(s) := inff ¿ ¸ 0 : ®(s) · ¯(s + ¿)g:


Backlog & Delay Bounds: Example

• given arrival curve and service curve Then

®(t) = minf rt + b;pt + M g,¯ (t) = R[t ¡ T ]+

bitsslope r arrival curve

slope pM

b

T

slope R

wmax

dma

x

service curve

t

and

wmax := b+ rmaxµ

b¡ Mp¡ r

;T¶

dmax :=M + b¡ M

p¡ r(p¡ R)+

R+ T


End-to-End QoS Guarantees• Each flow i: traffic characterized/policed by dual leaky-bucket traffic policer: i(t) = min{ rit+bi, pit+Mi}• Each hop h, router (with capacity Ch)

– uses WFQ + dual-leaky bucket traffic shaper for each flow– allocate bandwidth Ri > ri (thus wi: = Ri/ Ch) and buffer space Bi for each flow i

• Use the previous formula, we can guarantee flow i a maximum delay bound of Dmax: = h=1Hi Dh

where Hi is the no. of hops in the path of flow i and Dh is the delay bound for flow i at the hth router• and no loss if Bi ¸ wmax for flow i


IETF Integrated Services

• Architecture for providing QOS guarantees in IP networks for individual application sessions

• Resource reservation: routers maintain state info (a la VC) of allocated resources, QoS requests

• admit/deny new call setup requests:

Question: can newly arriving flow be admitted with performance guarantees while not violated QoS guarantees made to already admitted flows?


Intserv: QoS Guarantee Scenario• Resource reservation

– call setup, signaling (RSVP)– traffic, QoS declaration– per-element admission control

– QoS-sensitive scheduling (e.g.,

WFQ)

request/reply


Call AdmissionArriving session must :• declare its QOS requirement

– R-spec: defines the QOS being requested

• characterize traffic it will send into network – T-spec: defines traffic characteristics

• signaling protocol: needed to carry R-spec and T-spec to routers (where reservation is required)– RSVP


Intserv QoS: Service models [rfc2211, rfc 2212]

Guaranteed service:• worst case traffic arrival:

leaky-bucket-policed source • simple (mathematically

provable) bound on delay

Controlled load service:• "a quality of service

closely approximating the QoS that same flow would receive from an unloaded network element."

WFQ

token rate, r

bucket size, b

per-flowrate, R

D = b/Rmax

arrivingtraffic


IETF Differentiated ServicesConcerns with Intserv:• Scalability: signaling, maintaining per-flow router state difficult with large number of

flows • Flexible Service Models: Intserv has only two classes. Also want “qualitative” service

classes– “behaves like a wire”– relative service distinction: Platinum, Gold, Silver

Diffserv approach: • simple functions in network core, relatively complex functions at edge routers (or

hosts)• Don’t define define service classes, provide functional components to build service

classes


Edge router: per-flow traffic

management

marks packets as in-profile and out-profile

Diffserv Architecture

Core router: per class traffic

management buffering and scheduling

based on marking at edge preference given to in-

profile packets Assured Forwarding

scheduling

...

r

b

marking


Edge-Router Packet Marking

• class-based marking: packets of different classes marked differently

• intra-class marking: conforming portion of flow marked differently than non-conforming one

• profile: pre-negotiated rate A, bucket size B• packet marking at edge based on per-flow profile

Possible usage of marking:

User packets

Rate A

B


Classification and Conditioning• Packet is marked in the Type of Service (TOS) in

IPv4, and Traffic Class in IPv6• 6 bits used for Differentiated Service Code Point

(DSCP) and determine PHB that the packet will receive

• 2 bits are currently unused


Classification and Conditioning (cont’d)

may be desirable to limit traffic injection rate of some class:

• user declares traffic profile (e.g., rate, burst size)

• traffic metered, shaped if non-conforming


Forwarding (PHB)

• PHB result in a different observable (measurable) forwarding performance behavior

• PHB does not specify what mechanisms to use to ensure required PHB performance behavior

• Examples: – Class A gets x% of outgoing link bandwidth over time

intervals of a specified length– Class A packets leave first before packets from class B


Forwarding (PHB) in DiffServ

PHBs being developed:• Expedited Forwarding: pkt departure

rate of a class equals or exceeds specified rate – logical link with a minimum guaranteed rate

• Assured Forwarding: 4 classes of traffic– each guaranteed minimum amount of bandwidth– each with three drop preference partitions


Signaling in the Internet

• New requirement: reserve resources along end-to-end path (end system, routers) for QoS for multimedia applications

• RSVP: Resource Reservation Protocol [RFC 2205]– “ … allow users to communicate requirements to

network in robust and efficient way.” i.e., signaling !

• earlier Internet Signaling protocol: ST-II [RFC 1819]

connectionless(stateless)forwarding by IP routers

best effortservice

no network signaling protocolsin initial IP design

+ =


RSVP Reservation Protocol• Performs signaling to set up reservation

state for a session

• A session is a simplex data flow sent to a unicast or a multicast address, characterized by– <IP dest, protocol number, port number>

• Multiple senders and receivers can be in same session


Design Goal1. Accommodate heterogeneous receivers (different

bandwidth along paths)2. Accommodate different applications with different

resource requirements3. Make multicast a first class service, with

adaptation to multicast group membership4. Leverage existing multicast/unicast routing, with

adaptation to changes in underlying unicast, multicast routes

5. Control protocol overhead to grow (at worst) linear in no. of receivers

6. Modular design for heterogeneous underlying technologies


RSVP does not …

• specify how resources are to be reserved– rather: a mechanism for communicating needs

• determine routes packets will take– that’s the job of routing protocols– signaling decoupled from routing

• interact with forwarding of packets– separation of control (signaling) and data

(forwarding) planes


RSVP Basic Operations• Sender: sends PATH message via the data delivery

path– Set up the path state each router including the address of

previous hop

• Receiver sends RESV message on the reverse path– Specifies the reservation style, QoS desired (RSpec)– Set up the reservation state at each router

• Things to notice– Receiver initiated reservation– Decouple routing from reservation


PATH and RESV messages

• PATH also specifies – Source traffic characteristics

• Use token bucket

• RESV specifies – Service requirements – Source traffic characteristics (from PATH)– Filter specification, i.e., what senders can use

reservation– Based on these routers perform reservation

winter 2008 43

The Big Picture

NetworkSender

Receiver

PATH Msg

winter 2008 44

The Big Picture (2)

NetworkSender

Receiver

PATH Msg

RESV Msg


Route Pinning• Problem: asymmetric routes

– You may reserve resources on RS3S5S4S1S, but data travels on SS1S2S3R !

• Solution: use PATH to remember direct path from S to R, i.e., perform route pinning

S1S1

S2S2

S3S3

SSRR

S5S5S4S4PATH

RESV

IP routing


RSVP Path Messages: sender-to-network signaling

• path message contents:– address: unicast destination, or multicast group– flowspec: bandwidth requirements spec.– filter flag: if yes, record identities of upstream

senders (to allow packets filtering by source)– previous hop: upstream router/host ID– refresh time: time until this info times out

• path message: communicates sender info, and reverse-path-to-sender routing info– later upstream forwarding of receiver

reservations


Reservation Style

• Motivation: achieve more efficient resource • Observation: in a video conferencing when there

are M senders, only a few are active simultaneously– Multiple senders can share the same

reservation• Various reservation styles specify different rules

for sharing among senders• Key distinction:

– Reserved resources (bandwidth)– Which packets use those resources


Reservation Styles: Filters• Wildcard filter: all session packets share resources

– Good for small number of simultaneously active senders

• Fixed filter: no sharing among senders, sender explicitly identified in reservation– Sources cannot be modified over time– Allows reserved resources to be targeted to particular

paths

• Dynamic filter: resource shared by senders that are (explicitly) specified– Sources can be modified over time– Switching between speakers at a conference


RSVP: simple audio conference• H1, H2, H3, H4, H5 both senders and receivers• multicast group m1• no filtering: packets from any sender forwarded• audio rate: b• only one multicast routing tree possible

H2

H5

H3

H4H1

R1 R2 R3


inout

inout

inout

RSVP: building up path state• H1, …, H5 all send path messages on

m1: (address=m1, Tspec=b, filter-spec=no-

filter,refresh=100)

• Suppose H1 sends first path message

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7

L5 L7L6

L1L2 L6 L3

L7L4m1:

m1:

m1:


RSVP: building up path state• next, H5 sends path message, creating

more state in routers

inout

inout

inout

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7

L5 L7L6

L1L2 L6 L3

L7L4

L5

L6L1

L6

m1:

m1:

m1:


RSVP: building up path state• H2, H3, H5 send path msgs, completing

path state tables

inout

inout

inout

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7

L5 L7L6

L1L2 L6 L3

L7L4

L5

L6L1

L6L7

L4L3L7

L2m1:

m1:

m1:


Reservation Messages: receiver-to-network signaling

• reservation message contents:– desired bandwidth: – filter type:

• no (wildcard) filter: any packets address to multicast group can use reservation

• fixed filter: only packets from specific set of senders can use reservation

• dynamic filter: senders who’s packets can be forwarded across link will change (by receiver choice) over time.

– filter spec

• reservations flow upstream from receiver-to-senders, reserving resources, creating additional, receiver-related state at routers


RSVP: receiver reservation example

H1 wants to receive audio from all other senders• H1 reservation msg flows uptree to sources• H1 only reserves enough bandwidth for 1 audio

stream• reservation is of type “no filter” – any sender can

use reserved bandwidth

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7


inout

RSVP: receiver reservation example …

• H1 reservation msgs flows uptree to sources • routers, hosts reserve bandwidth b needed on

downstream links towards H1

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7

L1L2 L6

L6L1(b)

inout

L5L6 L7

L7L5 (b)

L6

inout

L3L4 L7

L7L3 (b)

L4L2

b

bb

b

bb

b

m1:

m1:

m1:


inout

RSVP: receiver reservation example …• next, H2 makes no-filter reservation for

bandwidth b• H2 forwards to R1, R1 forwards to H1 and R2 (?)• R2 takes no action, since b already reserved on

L6

H2

H5

H3

H4H1

R1 R2 R3L1

L2 L3

L4L5

L6 L7

L1L2 L6

L6L1(b)

inout

L5L6 L7

L7L5 (b)

L6

inout

L3L4 L7

L7L3 (b)

L4L2

b

bb

b

bb

b

b

b

(b)m1:

m1:

m1:


RSVP: Operation Summary• senders, receiver join a multicast group

– done outside of RSVP– senders need not join group

• sender-to-network signaling– Path message: make sender presence known to routers– path teardown: delete sender’s path state from routers

• receiver-to-network signaling– Resv message: reserve resources from sender(s) to

receiver– reservation teardown: remove receiver reservations

• network-to-end-system signaling– path error– reservation error


RSVP Reflections• multicast as a “first class” service• receiver-oriented reservations• use of soft-state

We did we miss?• Make aggregation central to design

– In core, don’t want to keep track of each flow– Don’t want to process each RESV message

• Economics: user/provider and provider/provider– We talked about it (at great length) but didn’t realize how

inflexible the providers would be

• Too complicated: filter styles a waste of time• Multicast focus?• Over-provisioning vs. network-layer QoS mechanisms

Documents

Winter 2008 Internet QoS1 Internet Quality of Service Principles for QoS Traffic Policing and Scheduling Policies –Lucky Buckets, Weighted Fair Queueing,