International Telecommunication Union Workshop on End-to-End Quality of Service.What is it? How do we get it? Geneva, 1-3 October 2003 IP Performance Metrics:

International Telecommunication Union

Workshop on End-to-End Quality of Service.What is it? How do we get it?Geneva, 1-3 October 2003

IP Performance IP Performance Metrics:Metrics:

Definitions and Implementation ExamplesDefinitions and Implementation Examples Al MortonAT&T Labs

21-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?

ITU-T

Outline

o Performance Management Framework• Relationship to the E2E QoS goal

o IP Parameters/Metrics Summaryo In-progress Metric Developmento Implementations

1. Service Providers2. Customers3. 3rd Parties

1. Performance for MPLS-enabled IP Nets


ITU-T

Network Performance Management Framework

o Fault Monitoring -- failure detectiono Passive Info Collection (single point)

• Read MIB counters or control data• Sample Traffic

o Active Measurements• Synthetic Traffic Dedicated to meas.

o Customer Measurements• Live or Synthetic traffic


ITU-T

Relationship to E2E QoS: Provide answers to ...

o Network Provider• Is the design meeting requirements for

various traffic classes or applications?• How can I demonstrate the superior

performance of my service offering?o Customer

• Is Network Performance Agreement?o 3rd Parties

• What does “the net” look like? Hot spots?

• What Network Provider is “best”?


ITU-T

Packet

Perf

. P

ara

mete

rsIPRE1

t Tmax

t Tmax

t > Tmax

ingress MP egress MP

SuccessfulIP packet outcome

ErroredIP packet outcome

SpuriousIP packet outcome

LostIP packet outcome

LostIP packet outcome

IPRE2Corrupted header or

errored payload

IPRE2(Note)

IPRE1

NOTE – Outcome occurs independent of IP packet contents

Never delivered ordelivered to an unpermitted

egress MP

IPRE1

IPRE1

IPRE2Valid header anderror-free payload

IPRE2(Note)


ITU-T

Metric/Parameter Definition Summary

IETF IPPM RFCs ITU-T Recs.Framework 2330 Y.1540 cl 1 thru 5Sampling 2330 Poisson

3432 Periodic(future work inSG4 ?)

Loss 2680 Y.1540 cl 5.5.6Delay 2679 (1-way)

2681 (Round Trip)Y.1540 cl 6.2

Delay Variation 3393 Y.1540 cl 6.2.2

Availability 2678 Y.1540 cl 7Bulk Transfer Cap 3148Loss Patterns 3357 Possibly in G.IPP


ITU-T

Comparison of IETF and ITU-T Delay Variation Metrics

Tx Rcv Playout

Time spent in: Transit Rcv Buffer

1

2

3

4

t

1

2

3

4

Inter packetarrival time,longer thansend interval

IETF IPDV is a measure of transfer delay variation w.r.t. previous packet.For Packet n,IPDV(n) = Delay(n) - Delay(n-1) or = R(n) - R(n-1) - T(n) - T(n-1)If the nominal transfer time is =10msec, and packet 2 is delayed in transit for an additional 5 msec, then two IPDV values will be affected.IPDV(2) = 15 - 10 = 5 msecIPDV(3) = 10 - 15 = -5 msecIPDV(4) = 10 - 10 = 0 msec

ITU-T SG 13 PDV is delay w.r.t. a reference, usually minimum delay.PDV(n) = Delay(n) - Min[Delay(*)]PDV(1,3,4)=0 PDV(2)=5


ITU-T

Transient Delay Variation caused by burst traffic

-20

-10

0

10

20

30

40

50

60

70

80

90

100

120 220 320 420 520 620 720 820 920 1020 1120

Time in ms

Del

ay o

r Ji

tter

(m

s)

delta ms IPDV ms RT-delay msITU PDV IETF


ITU-T

Packet Metrics for VoIP and other voiceband applications

o *new* metrics in G.IPP• Consecutive Packet Loss• Degraded Seconds• Short-term Delay Variation• Overall VoIP Parameters

o Alan Clark’s Presentation


ITU-T

What is Packet Reordering?

Packets arrive at Dst, but not in send order.

1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering

1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering

In the “world of order” all these packets are of interest.

1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,... | Early | Late |

No reordering until Late Packets Arrive# of Early Packets => Reordering Extent


ITU-T

Affect of Reordered Packets on most applications

o Receivers must perform work to restore order

1, 2, 3, 7, 8, 9,10, 4, 5, 6, 11, 12,...

| Buffered ||Reordered|

1 2 3 7 8 9 10 4 5 6 11

4 5 6, 11 (& 7 to 10)

1 2 3

Dst Time axis

Higherlayers


ITU-T

Definition of Reordered Packeto Packet n is designated reordered

when its sequence number is less than the Next Expected threshold (set by the arrival of a previous packet).

1

2

3

7

8

9

10

4

5

6

11

0

2

4

6

8

10

12

1 2 3 4 5 6 7 8 9 10 11

Arrival Order (i)

Se

nd

Ord

er

(s)

Next Expected


ITU-T

Failure Recovery Time

o When recovery was a simple outage, characterization was simple, too.

o IETF Benchmarking Methodology WG has identified 5 possible recovery scenarios:Lost packets Induced delay

Out-of-order packetsDuplicate packets123 45 67123456 4

1267 1234

Errored packets

1267


ITU-T

Implementations: Customers, Service Providers & 3rd Parties

o Select Ping Target - make Round-trip connectivity and RTT measurement

o Accuracy Issues include path through router, path through net (asymmetries), response time at target, sampling rates

o Compare to current perf. to “normal”

$ ping R2 (or R3)

R1 R2

R3

R4or

or


ITU-T

Beyond ping: ICMP Timestamp or Timestamp Reply Message

o Time spent processing packet at target can be removed, for more accurate RTT.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Code | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identifier | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Originate Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Receive Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transmit Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

R1 R2

R3

R4

OriginateCode=13

Receive TransmitCode=14


ITU-T

Implementations:AT&T Global IP Measurements

BR(City 1)

MEASUREMENTCOLLECTION

SERVER (MCS)

BR(City 2)

BR(City n)

Measurementprobe

AT&T GLOBAL IP BACKBONE INFRASTRUCTURE

WEBCLIENTS(for reportviewing)

MEASUREMENTAGGREGATION& REPORTINGSERVER (MRS)

AGGREGATEDMEASUREMENT

DATA

Measurement Probes

http://www.att.com/ipnetwork


ITU-T

AT&T’s IP Measurement Design

o Poisson Sequence (RFC2330)• 15 minute duration = 0.3 pkts/sec• Type UDP, IPv4• 278 bytes total• ~300 packets sent• unbiased sample

o Periodic Sequence (RFC3432)• 1 minute duration • Random Start Time• 20 ms packet spacing• Type UDP, IPv4 • 60 bytes total• ~3000 packets sent

24 hours. . .

15 minutes


ITU-T

Technical Collaborators at AT&T

o Len Ciavattoneo George Holubeco Madhukar Kshirsagaro Ron Kulpero Arvind Ramarajano Gomathi Ramachandran


ITU-T

New Measurement Challenges for MPLS-enabled IP Networks

o Most (all?) IP/Packet Network challenges

o Two main categories of MPLS Domains:• LDP-based, connection-less• Traffic Engineering, connection oriented

o Label Switched Paths are Unidirectionalo point to point and multi-point to pointo Many options for Failure Recoveryo LSP identity optionally removed (PHP)o Work in progress in SG 13 =

Y.MPLSperf


ITU-T


Networksection

Networksection

Networksection

MPLS Edge Node, or MPLS Ingress Node, or

LSR if both IP and MPLS are enabled

MPLS Node

Exchangelink

Exchangelink

Exchangelink

Exchangelink

Exchangelink

Network Section Ensemble (NSE)

Label Switched Paths

Scope of OA&M Measurements: single Network Section or MPLS Domain

MPLS Domain MPLS Domain

MPLS Network


ITU-T


o New Protocols = New Opportunities to Blackhole Traffic

o Detect this new class of failures with• Y.1711 MPLS OA&M Connectivity Verific.

• First version approved, adding fast failure detection

• LSP-Ping, Like ICMP Echo Request, plus• One-way Delay measurement possible• LSP Traceroute possible


ITU-T


o New Availability Definition? Crossroad:• Connection-Oriented Transport has

used a 10 second sliding window• Connection-Less Packet Transport has

used a 5 minute fixed windowo MPLS Networks => both transport

types• When Connection-oriented Services use

a Connection-less transport, which precedent should the Availability Definition follow?


ITU-T

Summary

o Performance Management Framework• Measurement Systems are a key step

toward the goal of E2E QoS

o Summary of existing Parameters/Metrics

o In-progress Metric Developmento Active Measurement Implementations

• Ping for connectivity and ...• Dedicated Measurement Systems

o Parameter Framework for MPLS has new challenges


ITU-T

Resources and References

o L. Ciavattone, A. Morton and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone," IEEE Communications Magazine, June 2003.

o Geoff Huston, “Measuring IP Network Performance,” The Internet Protocol Journal, vol 6, no.1, March 2003 http://www.cisco.com/ipj

o X.Xiao, et al., “A Practical Approach for Providing QoS in the Internet Backbone,” IEEE Communications Magazine, December 2002.

o D. Meyer, et al., “Trends in Measurement and Monitoring of Internet Backbones,” Panel at NANOG 26, slides etc. at http://www.nanog.org/mtg-0210/measurement.html

o ITU-T Rec. Y.1540, “Internet Protocol Data Communication Service – IP Packet Transfer and Availability Performance Parameters,” 2003.

o IETF IP Performance Metrics Working Group, links to RFC 2330, other IPPM RFCs and Internet Draft on Reordering: http://www.ietf.cnri.reston.va.us/html.charters/ippm-charter.html

o Draft New Recommendation Y.MPLSperf, “Performance and Availability Parameters for MPLS Networks”

o Draft New Recommendation G.IPP, “Performance Parameter Definitions for Quality of Speech and other Voiceband Applications Utilising IP Networks”

o RFC 792, “Internet Control Message Protocol,” J. Postel, September 1981.

Documents

International Telecommunication Union Workshop on End-to-End Quality of Service.What is it? How do we get it? Geneva, 1-3 October 2003 IP Performance Metrics: