1 VoiceCon Fall 2005
VoIP Performance Management
Alan ClarkCEO, Telchemy
VoiceCon Fall 2005
2 VoiceCon Fall 2005
Agenda
• Voice over IP Performance Problems– Voice quality– IP problems– Non-IP problems
• VoIP Performance Management Framework• RTCP XR• Defining Requirements• Summary
3 VoiceCon Fall 2005
Expectations of service quality
• Listening quality– Clarity, no distracting noise/ pops/ distortion
• Conversational quality– No noticeable delay or echo
• Availability– Always available, does not drop calls
• Signaling quality– Low call setup delay, features work
4 VoiceCon Fall 2005
Voice Quality - performance measures?
• The “mantra”– Packet Loss– Jitter– Delay
• The reality…..
5 VoiceCon Fall 2005
Jitter
0
50
100
150
200
0 0.5 1 1.5 2
Time (Seconds)
De
lay (
mS
)
Average jitter level (PPDV) = 4.5mS
6 VoiceCon Fall 2005
Jitter
• Jitter is strongly time varying
• Often measured using simple “Packet to Packet Delay Variation (PPDV)” metric– Only valid if jitter has a constant level– Meaningless for time varying jitter (normally the
case)
• Other metrics exist - MAPDV, Y.1541….
• Ensure that metrics used will capture information on highly variable jitter levels
7 VoiceCon Fall 2005
Packet Loss - also time varying
0
10
20
30
40
50
30 35 40 45 50 55 60 65 70
Time (seconds)
50
0m
S A
vg
e P
ack
et
Loss
Rate
Average packet loss rate = 2.1%
8 VoiceCon Fall 2005
Leads To Time Varying Call Quality
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Time
MO
S
0
100
200
300
400
500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Ban
dw
idth
(kb
it/
s)
Voice
DataHigh jitter/ loss/ discard
Codecdependant
9 VoiceCon Fall 2005
Packet Loss & Jitter
• Packet loss is strongly time varying
• Packet loss typically occurs at the same time as jitter– Both due to network congestion
• Need to measure the length and severity of bursts of lost/ discarded packets– Burst = period of high loss/discard rate
• Discard Rate is a good measure of the impact of jitter
• Jitter is a good predictor of congestion problems
10 VoiceCon Fall 2005
Delay
2
3
4
5
0 100 200 300 400 500 600
Round trip delay (milliseconds)
MO
S S
core
55dB Echo Return Loss
35dB Echo Return Loss
Conversational problemsEcho problems
11 VoiceCon Fall 2005
Where does echo come from?
IP
EchoCanceller
Gateway
Echo
Round trip delay - typically 50mS+
Additional delay introduced by VoIP makes existing echo problems more obvious
12 VoiceCon Fall 2005
Delay
• Impact of delay strongly depends on how much echo is present on the call– No echo - user can tolerate 200+mS of delay– Noticeable echo - delays over 50mS are problematic
• User perceived delay includes delays in end-equipment, which can be 50mS or more
• Therefore specifying a simple delay metric does not ensure good quality
13 VoiceCon Fall 2005
Signal Level Problems
Temporal Clipping occurs with VAD or Echo Suppressors -- gaps in speech, start/end of words missing
Amplitude Clipping occurs -- speech sounds loud and “buzzy”
-16 dBm0
-36 dBm0
14 VoiceCon Fall 2005
Typical VoIP SLA
Jitter < 20mS
Loss < 0.1%
Latency < 100mS
Availability 99.9%
15 VoiceCon Fall 2005
A Better VoIP SLA
99.9% of calls/intervals haveMOS-LQ > 3.9MOS-CQ > 3.8
Degraded Service QualityEvents < 0.1/ hour[DSQ = ….]
Latency < 100mS
Availability 99.9%
Based on either referenceor actual endpoint
Also reflected in MOS-CQ
Availability of media ANDSignaling path
Transient quality problems
Based on all available data
16 VoiceCon Fall 2005
Enterprise Scenario
IP Phones
IP Phones
IP Phones
IP VPN
Gateway
PSTN
17 VoiceCon Fall 2005
Measuring at ISP Demarcation - active test
Active Test Functions
Test call
18 VoiceCon Fall 2005
Active test for IP Service SLA
• Uses VoIP calls, to ensure packets are treated identically to “real” VoIP calls
• Use a Reference endpoint - I.e. a fixed configuration, known, virtual IP endpoint
• Test:– Peak times - to understand quality under load
conditions– Off-peak times - to detect problems before they
impact users
19 VoiceCon Fall 2005
Measuring at user desktop - passive test
RTCP XR
SIP QoSReport
EmbeddedMonitoringFunction
20 VoiceCon Fall 2005
Non-intrusive VoIP Performance Monitoring
• Most effective for end-to-end measurement
• Embedded quality monitoring function in IP endpoint
• Can measure service quality, signaling reliability……
• Collect data via RTCP XR or SIP QoS reports
21 VoiceCon Fall 2005
VoIP Performance Management Framework
Media Path Reporting(RTCP XR)
Call Server andCDR database
VoIPEndpoint
VoIPGateway
SNMP
NetworkManagementSystem
Signaling Based QoS Reporting
Embedded Monitoring
Network Probeor Analyzer
VQ
VQ
Embedded Monitoring
VQ
22 VoiceCon Fall 2005
RTCP XR
Loss Rate Discard Rate Burst Density Gap Density
Burst Duration (mS) Gap Duration (mS)
Round Trip Delay (mS) End System Delay (mS)
Signal level RERL Noise Level Gmin
R Factor Ext R MOS-LQ MOS-CQ
Rx Config - Jitter Buffer Nominal
Jitter Buffer Max Jitter Buffer Abs Max
23 VoiceCon Fall 2005
RTCP XR-Based Protocols
• Media Path– RFC3611 RTCP XR -- published Nov 2003
• Signaling– SIP QoS Reporting -- last call, final by late 2005– H.323 reporting (H.460.9) -- published Apr 04– Megaco reporting (H.248.30) -- published Apr 04– MGCP -- draft, final by end 2005?
• SNMP– RTCP XR MIB -- draft, final by end 2005
24 VoiceCon Fall 2005
Enterprise Application Using New Framework
Branch Office
IP Phone
IP VPN
IP Phone
Teleworker
IP
IP Phones
VQ
VQ VQ
VQ
VQ
VQ
VQ
VQ
Gateway
Probe
SLA MetricsNMS
25 VoiceCon Fall 2005
Defining Requirements
• Do predeployment testing!!• Request support for RFC3611 VoIP Metrics in
IP Phones and Gateways• Use same monitoring technology in IP
endpoints, probes and analyzers• Use VoIP Performance Management
Framework• Request SLA metrics from service providers
that are meaningful for VoIP• Remember that IP problems are transient!!
26 VoiceCon Fall 2005
Agenda
• Voice over IP Performance Problems– Voice quality– IP problems– Non-IP problems
• VoIP Performance Management Framework• RTCP XR• Defining Requirements• Summary