Copyright © 2004 Juniper Networks, Inc. www.juniper.net 1
SEEREN2 Summer SchoolHeraklion, Sept 25th
Routing Issues: QoS/CoS
Jean-Marc UzéLiaison Research & Education, EMEA
2Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Agenda: QoS/CoS Workshop
Module 1: Overview of QoS/CoS Module 2: JUNOS CoS implementation (J/M/T-
Series) Module 3: Introduction to JUNOS CLI Module 4: GEANT2 QoS services
ImplementationThe content of this module is
courtesy of Dante (http://www.dante.net)
http://www.dante.net/nep/geantqos/http://www.dante.net/tf-ngn/activities.html
3Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Module 4:GEANT2 QoS services implementation
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
4Copyright © 2004 Juniper Networks, Inc. www.juniper.net
10 Gb/s IP/MPLS backbone with Juniper T640s, M160s, M40s
4 x 10 Gb/s to North America
Dark fiber and WDM optical technology
Connecting 34 European Countries and 30 National R&E Networks
European connectivity to over 3000 R&E institutions
Advanced Services: IPv6 Premium IP Multicast v4 + v6 Best Effort Less Than Best Effort Layer 2 VPN
GEANT2 / Dante
6Copyright © 2004 Juniper Networks, Inc. www.juniper.net
IP QoS Services on GÉANT
Premium IP upper-bounded one-way delay upper-bounded IPDV negligible packet loss guaranteed capacity
Less than Best Effort class of traffic using the un-utilised Best Effort
and higher classes of service bandwidth
http://www.geant.net/server/show/nav.00700a009
7Copyright © 2004 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
8Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Premium IP Model
End-to-end service across multiple management domains using diffserv, ATM CBR or over-provisioning(!) packet tagged DSCP 46 (EF - 101110) destination aware service packet with other DSCP are left untouched (packets
from other service) Premium IP bandwidth limited to 10% of the link
capacity can cope with 20% in case of circuit failure
9Copyright © 2004 Juniper Networks, Inc. www.juniper.net
GEANT and IP Premium Service
Source: http://www.dante.net/sequin
10Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Premium IP on GEANT
Protection of authorised Premium IP traffic under normal circumstances, the Premium IP traffic
of a circuit is limited to 10% of the circuit capacity 20% in case of another circuit failure
bullet-proof all the GÉANT accesses against unauthorised Premium IP traffic (tagged DSCP 46) on all the ingress interfaces
if DSCP 46 packet arrives on GÉANT and part of an unauthorised flow: classify the packet into the Best Effort queue and remark it as Best Effort (DSCP 0)
if DSCP 46 packet arrives on GÉANT and is part of an authorised flow: check against policer according capacity requested in the SLA (in-profile accepted, out-of-profile dropped)
11Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Premium IP on GEANT Protection of authorised Premium IP traffic [cont]
per next AS rate-limitation (implemented by Juniper for GÉANT) can also do source-destination IP addresses when NREN don’t
do it. (NREN = National Research & Education Network, a Dante customer network directly connected to GEANT)
Trust the Premium IP traffic received from a GÉANT backbone interface. Traffic checked at the GÉANT ingresses.
Configure queuing mechanism on the backbone and access interfaces. strict-high priority is configured to the Premium IP queue.
Don’t forget that the amount of Premium traffic expected in the Premium IP queue is 10% of the link capacity (service over-provisioned by a factor 9); this is assured by ingress policing.
90% for the BE and 5% for the network control (and 5% for LBE)
12Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Jitter distribution in VBR traffic - BE & Premium IP
0%
10%
20%
30%
40%
50%
60%
70%
0.0
0
0.1
9
0.3
8
0.5
6
0.7
5
0.9
4
1.1
3
1.3
1
1.5
0
1.6
9
1.8
8
2.0
7
2.2
5
2.4
4
2.6
3
2.8
2
3.0
1
3.1
9
3.3
8
3.5
7
3.7
6
Premium IP BE jitter[ms]
pe
rce
nta
ge
of
pa
ck
ets
Avg. jitter vs. packet size - BE & Premium IP
0
2
4
6
8
10
12
14
Premium IP BE
avg
. jit
ter
[ms]
packet size [bytes]
Test result end-to-end IP Premium
13Copyright © 2004 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
14Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Less than Best Effort
Class of traffic using the un-utilised Best effort and higher classes of service bandwidth in case of competition for resources, the LBE traffic will de
discarded before any Best-Effort or higher classes of traffic.
use the DSCP 8 (001000) - same as Internet2 scavenger service.
Congestion on an interface due to LBE should be transparent to the BE or higher classes of
services no BE or higher classes of services packet loss
15Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Less than Best Effort
No end-to-end guarantees no metric needed to quantitatively describe the service
Can be supported on one interface anywhere else, the LBE tagging should be passed
transparently.
Application scenarios mirroring, test traffic, some GRID data transfers, network
backups, protection of research traffic from student dormitory one.
16Copyright © 2004 Juniper Networks, Inc. www.juniper.net
LBE Queuing Technique
For algorithm with bandwidth shared assignment, as Weighted Wound Robin and Weighted Fair Queuing, a very small bandwidth share is allocated to the LBE queue. Typically between 0% and 5%
17Copyright © 2004 Juniper Networks, Inc. www.juniper.net
LBE: Measurement with congestion
One-way delay Increase of LBE maximum one-way delay of 1.5ms Increase of BE maximum one-way delay of 400µs
18Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Normal Traffic
Normal Traffic +Less Than Best Effort2.0 Gbit/s
Normal Traffic +
Radio Astronomy Data500 Mbit/s
Normal Traffic +
Radio Astronomy Data +Less Than Best Effort2.0 Gbit/s
LBE live test:ER2002 Demo - VLBI - dataGRID
19Copyright © 2004 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
20Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Queuing Technique
WRR - Juniper M-series
Weight Assure the queue to be given a minimum amount of
bandwidth proportional to the weight. Priority
queue with high priority are served before the low priority
allow the BE (and other high priority queues) to be served first until empty before serving the LBE one.
WRED is used to limit the queuing delay in case of congestion use to protect one class of traffic over the other within
a queue.
21Copyright © 2004 Juniper Networks, Inc. www.juniper.net
DSCP/ToS Values used by GEANT
Service DSCP value ToS value Juniper alias ToS (hex) DSCP-ToS binary
Premium IP 46 184 ef B8 101110 - 101110xx
LBE 8 32 cs1 20 001000 - 001000xx
DWS 32 128 cs4 80 100000 - 100000xx
Network control 1
48 192 cs6 C0 110000 - 110000xx
Network control 2
56 224 cs7 E0 111000 - 111000xx
The DSCP/ToS values used in GÉANT to classify the traffic of the different QoS classes are shown in the table below. In addition to the three service classes offered to transiting traffic there is a DWS (IP commodity service) and a Network Control class, which are traffic classes used internally to the GÉANT network.
22Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Juniper Networks and CoS ServicesGEANT with IP Premium + LBE Service
Junos CoS features include policing, (strict) priority queuing, weighted round robin (WRR), precedence/DSCP field rewrite, and random early drop RED.
On a Juniper M-series Router each port has 4 Queues Weighted Round Robin Percentages can be set for each Queue
New generation Q-PICs offers multiples queues per logical interfaces (Ethernet VLAN, ATM PVC, etc.)
WRWRRR
Source: http://www.dante.net/nep/geantqos/ and http://www.dante.net/tf-ngn/activities.html
Queue FC LP Service DSCP Weight Priority Buffer low Best Effort / 0 BE high DWS 32
90% Low 50%
low Premium IP 46 1 EF high / /
N/A Strict-high
15%
low Less than BE 8 2 LBE high Retag to BE 0
5% low 30%
low Network control 48 3 NC high Network control 56
5% high 5%
23Copyright © 2004 Juniper Networks, Inc. www.juniper.net
QoS Configuration on GEANT The configuration has completed on most of
the GEANT routers allowing Premium IP, BE and LBE to co-exist.
The routers where the three services have been enable are represented as green on the following map.
The routers coloured yellow are Juniper routers where “old” FPCs have been re-used from TEN-155 (1999) These old FPC’s that do not allow for the full functionality of QoS.
As such BE is not ideally protected by LBE and the bandwidth is effectively shared. Premium IP only is supported.
25Copyright © 2004 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
26Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Router Configuration
Each router in the GÉANT network contains certain QoS building blocks in order to configure Per Hop Behaviors (PHB). The configuration shown here is taken from a Juniper M160 router with JUNOS 5.7 and with E-FPC (enhanced FPCs) and SDH interfaces.
Classifiers, schedulers and rewrite rules can be associated to each interface. In GÉANT two types of interface configurations are used for QoS a backbone interface an access interface (i.e. the interface where the traffic
from an NREN is entering GÉANT)
27Copyright © 2004 Juniper Networks, Inc. www.juniper.net
DSCP and ToS Values
Type of Service field illustration
Illustration of DSCP Type of Service field
configuration
Service DSCP value ToS value Juniper alias ToS (hex) DSCP-ToS binary
Premium IP 46 184 ef B8 101110 - 101110xx
LBE 8 32 cs1 20 001000 - 001000xx
DWS 32 128 cs4 80 100000 - 100000xx
Network control 1 48 192 cs6 C0 110000 - 110000xx
Network control 2 56 224 cs7 E0 111000 - 111000xx
28Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Router Interfaces
Backbone interface
so-7/0/0 { scheduler-map MAP-BASIC; unit 0 { classifiers { dscp backbone-classifier; } rewrite-rules { dscp basic-rewrite-rules; } }}
Access interface
so-0/2/3 { scheduler-map MAP-BASIC; unit 0 { classifiers { dscp access-classifier; } rewrite-rules { dscp basic-rewrite-rules; } }}
In addition, the access interface may contain filters in order to classify and police Premium IP traffic.The following configurations apply to all (access and backbone) interfaces.
29Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Drop Profiles Drop profile define the parameters used by
the Random Early Detection (RED) mechanism that MAY be used in a queue
dws-drop-profile { fill-level 35 drop-probability 10; fill-level 40 drop-probability 100;}be-drop-profile { fill-level 15 drop-probability 30; fill-level 19 drop-probability 50; fill-level 24 drop-probability 70; fill-level 30 drop-probability 100;}less-than-be-drop-profile { fill-level 25 drop-probability 30; fill-level 30 drop-probability 50; fill-level 40 drop-probability 70; fill-level 50 drop-probability 100;}
30Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers1. Associate a name with each queue
Note: a queue is sometimes also called a forwarding class
forwarding-classes { queue 0 best-effort; queue 1 expedited-forwarding; queue 2 less-than-best-effort; queue 3 network-control;}
The Premium IP traffic is classified into the expedited-forwarding queue. The naming of the queues is performed once and applies to all interfaces of the router.
31Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers1. Define scheduler configurations
scheduler weight, queue size and priority as set at the GÉANT router for each queue:sch-best-effort { transmit-rate percent 90; buffer-size percent 50; priority low;}sch-expedited-forwarding { buffer-size percent 15; priority strict-high;}sch-less-than-best-effort { transmit-rate percent 5; buffer-size percent 30; priority low; drop-profile-map loss-priority low protocol any drop-profile less-than-be-drop-profile; drop-profile-map loss-priority high protocol any drop-profile be-drop-profile;}sch-network-ctrl { transmit-rate percent 5; buffer-size percent 5; priority high;}
32Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Queues and Schedulers3. Associate a scheduler with a queue (FC)
The main advantage of the scheduler-map is that it can be applied to more than one interface.
MAP-BASIC { forwarding-class best-effort scheduler sch-best-effort; forwarding-class expedited-forwarding scheduler sch-expedited-forwarding; forwarding-class less-than-best-effort scheduler sch-less-than-best-effort; forwarding-class network-control scheduler sch-network-ctrl;}
33Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Classification The classifier is a functional block located at the input interface that sets
three internal bits for each IP packet Two bits that select the output-queue. There are four output queues, also
called forwarding-classes, per port. One bit to indicate the loss-priority, the packets classified in an output-queue
can have two different values of loss-priority (low or high). Note that it is also possible to classify packets by means of an input
filter.
Backbone Classifierdscp backbone-classifier { import default; forwarding-class best-effort { loss-priority low code-points [ af11 af12 af13 ]; loss-priority high code-points cs4; } forwarding-class less-than-best-effort { loss-priority low code-points cs1; }}
Access Classifierdscp access-classifier { import default; forwarding-class best-effort { loss-priority low code-points [ af11 af12 af13 ]; } forwarding-class less-than-best-effort { loss-priority low code-points cs1; loss-priority high code-points [ ef cs4 ]; }
34Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Marking The marking of packets with a DSCP value is the last
QoS action performed before the transmission of the packet (hence after firewall filter evaluation)
dscp basic-rewrite-rules { forwarding-class best-effort { loss-priority high code-point cs4; } forwarding-class expedited-forwarding { loss-priority low code-point ef; } forwarding-class network-control { loss-priority low code-point nc1; loss-priority high code-point nc2; } forwarding-class less-than-best-effort { loss-priority high code-point be; loss-priority low code-point cs1; }}
ServiceIncoming DSCP
valueNew DSCP
value
Authorised Premium IP 46 46/drop
Un-authorised Premium IP
46 0/5
DWS 32 0
LBE 8 8
Network Control 48/56 48
Best Effort other values Unchanged
35Copyright © 2004 Juniper Networks, Inc. www.juniper.net
AGENDA
GÉANT Network and Services Premium IP Less than Best Effort Queuing on GÉANT and status Router Configuration Premium IP Management
36Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Useful Tool Feature of the NANOG traceroute to discover
the DSCP changes along the path (Simon Leinen from Switch:-)
[root]# ./traceroute -t 184 193.171.2.1 traceroute to 193.171.2.1 (193.171.2.1), 30 hops max, 40 byte packets 1 css7-ATM4-0-0-101-dmsk.man.poznan.pl (150.254.160.62) 1 ms 1 ms 1 ms 2 150.254.163.118 (150.254.163.118) 2 ms 2 ms 2 ms 3 z-pozmanu-oc3.poznan-gw.pol34.pl (212.191.127.49) 2 ms 2 ms 2 ms 4 pol-34.pl1.pl.geant.net (62.40.103.109) 2 ms 2 ms 2 ms 5 pl.cz1.cz.geant.net (62.40.96.45) 22 ms (TOS=0!) 22 ms 22 ms 6 cz.de1.de.geant.net (62.40.96.38) 30 ms 30 ms 30 ms 7 de1-1.de2.de.geant.net (62.40.96.130) 30 ms 30 ms 31 ms 8 de.at1.at.geant.net (62.40.96.5) 43 ms 43 ms 43 ms 9 aconet-gw.at1.at.geant.net (62.40.103.2) 43 ms 43 ms 43 ms 10 193.171.2.1 (193.171.2.1) 45 ms * 45 ms
37Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Need for Automation
Service management was done manually Service set up, maintenance and
termination was done by phone calls and emails
Considerable manual effort required Complexity in keeping track of:
Path information Current and future reservations Premium IP utilisation levels Changes in network topology
Multi-party communication
38Copyright © 2004 Juniper Networks, Inc. www.juniper.net
System Architecture
Java web-based architecture Using Apache 1.3 web server, Tomcat
servlet container & MySQL Database
39Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Premium IP Reservation Tool Features (1)
Authentication & Authorisation Path Finder
Find shortest path between two end pointsDynamic based upon configured IS-IS cost
Utilisation Monitoring Check Premium IP reservation levels on
each intermediate link along the path Take into account all active reservation
during the given time period Display the available Premium IP capacity
40Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Premium IP Reservation Tool Features (2)
Reservation Management View, Request, Modify, Cancel Based upon available Premium IP capacity
Contact managementIP Address management
Automated email notification to User, DANTE - Premium IP team, NOC upon Reservation Request, Modification,
Cancellation Router Configuration update (script) Others: IS-IS cost, Reports, Archival,
System Administration...
42Copyright © 2004 Juniper Networks, Inc. www.juniper.net
Thank you
Jean-Marc UzéLiaison Research & Education, EMEA
Mobile: +3361543251231 Place Ronde, 92986 Paris-La-Defense, France