Upload
yossi-cohen
View
7.252
Download
5
Tags:
Embed Size (px)
DESCRIPTION
IPTV QoE and QoS Monitoring, the standards, Monitoring goals, hierarchies and domains.
Citation preview
www.dsp-ip.comFast Forward Your Development
IPTV QoS Monitoring
www.dsp-ip.comFast Forward Your Development
Overview
•QoS and QoE•Measurement and Monitoring Goals
▫Lab, pre-deployment, Monitoring, failure analysis
•Measurement Methods▫Objective, Subjective, payload/codec based
•Monitoring Levels▫Transport, Transaction, content
•Monitoring Domains ▫Head-end, network, Home
www.dsp-ip.comFast Forward Your Development
Challenges in providing QoE
Name Challenge Answer
network Overcome Packet loss, jitter, network congestion
Improved protocols
QoS mapping schemes
Bandwidth No bandwidth in the network and at the home
increased capacity inside the home and the network level
Monitoring Measure hundreds of video streams in Gb Eth & IP environment
Video over IP monitoring, new protocols & devices
CRM costs Lower MTBF, call center calls and “truck rolls”
Automatic maintenance, Automatic firmware update
Home network monitoring
www.dsp-ip.comFast Forward Your Development
User quality measurements•What does subscribers want?
▫Content Content quality Content pricing Content availability
▫Viewing experience Video Quality Channel Zapping delay
▫Integration of services Usability One bill New services
▫Customer services
Technical Factors
www.dsp-ip.comFast Forward Your Development
Voice/Video Quality?
We can look at quality in different ways:
1. Perceptual subjective Quality (MOS)2. Numerical Quality parameters (PSNR)3. How to connect (relation matrix
between 1 and 2)
www.dsp-ip.comFast Forward Your Development
MEASUREMENT & MONITORINGIn the Lab & In The FieldsFor Pre-Deployment/monitoring/Failure Analysis
www.dsp-ip.comFast Forward Your Development
Measurement & Monitoring phases
AnalysisProblem Solving
Tuning
PreDeployment
Testing
LabTesting
Design
24/7Monitoring
DeploymentDeploymentPhasePhase
Pre-Pre-DeploymentDeploymentPhasePhase
www.dsp-ip.comFast Forward Your Development
Measurement & Monitoring phases
•Design & Lab testing▫Simulation and Emulation of the network▫Lab and testing tools
•Pre Deployment Stage▫Work on actual network▫Load testing▫Lab, testing, diagnosing and monitoring tools
•Deployment (production) Phase▫Mostly monitoring (probes) equipment,
management systems, data filtering and diagnostics equipment
www.dsp-ip.comFast Forward Your Development
MEASUREMENT TAXONOMYSubjectiveObjectivePayload based, codec aware, codec anaware
www.dsp-ip.comFast Forward Your Development
Measurement methods review
•Subjective▫Accurate▫Expensive, not for monitoring
•Objective▫Repeatable▫For both testing and monitoring
www.dsp-ip.comFast Forward Your Development
Multimedia monitoring methods
ObjectiveSubjective
MOS(Voice)
BT500(Video)
Codec independentPacket based
MDI
MonitoringTesting
BroadcastWorld
HSI and Data World
NetworkMonitoring
Delay, JitterPacket loss
Codec awarePacket based
VQI
V-Factor
VQSTelchemy
www.dsp-ip.comFast Forward Your Development
Objective methods
www.dsp-ip.comFast Forward Your Development
Payload Based Methods
www.dsp-ip.comFast Forward Your Development
Full Reference: Video Quality AssessmentITU-T J.144 and ITU-R BT.1683 •Full-reference perceptual models•Digital TV•Rec. 601 image resolution (PAL/NTSC)•Bit rates: 768 kbps ~ 5 Mbps•Compression errors
www.dsp-ip.comFast Forward Your Development
Voice Quality Assessment – with/out reference
•ITU-T P.862 (Feb 2001) - Full Reference▫Full-reference perceptual model (PESQ)▫Signal-based measurement▫Narrow-band telephony and speech codecs▫P.862.1 provides output mapping for
prediction on MOS scale•ITU-T P.563 (May 2004)
▫No-reference perceptual model▫Signal-based measurement ▫Narrow-band telephony applications
www.dsp-ip.comFast Forward Your Development
Voice Quality Assessment
• ITU-T P.862.2 (Nov 2005):▫Extension of ITU-T P.862 ▫Wide-band telephony and speech codecs (5
~7Khz)• ITU-T P.VTQ (on-going):
▫Targeted at VoIP applications▫Minimum performance framework for no-
reference packet-based measurement▫Models analyze packet statistics; speech
payload is assumed▫Uses P.862 as a measurement reference
www.dsp-ip.comFast Forward Your Development
Codec Aware Methods
Codec awarePacket based
VQI
V-Factor
VQSTelchemy
www.dsp-ip.comFast Forward Your Development
Packet – Codec Aware•Monitoring technique•Codec dependent•Incorporates network parameters data
with codec behavior data•Scales- could monitor thousands of
channels•Examples:
▫ VQS (Telchemy)▫ VQI(Brix)▫ V-Factor (QoSMetrics)
0
5
10
15
20
25
30
35
0 5 10 15 20
Packet Loss (%)
PS
NR
(d
B)
Problem area
Robust codec
“Raw”codec
The need a codec aware metrics
www.dsp-ip.comFast Forward Your Development
Packet – Codec aware
0
20
40
60
80
100
0 10 20 30 40 50Time
Pack
et
Loss
/D
isca
rd R
ate
Packet loss/discard typicallyoccurs in high density periods
Average can bemisleading
1
2
3
4
5
0 5 10 15 20
Time
Mea
n O
pin
ion
Sco
re
Impact of Burst ofPacket Loss
Base quality level depends on frame rate, codec type, bit rate
Poor quality during burst of loss/discards
5-8 seconds 15-30
seconds
Subjective compensation for variance between human and testing equipment view of loss
www.dsp-ip.comFast Forward Your Development
Example V-Factor
•Based on MPQM (Moving Picture Quality Metrics) – high quality video measurement standard
•V = f(QER, PLR, R)▫QER – relative video codec quality▫PLR – Packet loss ratio (based on actual
packet loss, jitter data and jitter buffer model)
▫R – Image complexity factor (2-3)•Adopted by Spirnet
www.dsp-ip.comFast Forward Your Development
Packet – Codec Independent
•Monitoring only•Codec independent•Based on network parameters data only•Scales - could monitor thousands of
channels•Examples:
▫MDI IneoQuest standardized by IETF
www.dsp-ip.comFast Forward Your Development
MONITORING LEVELS
www.dsp-ip.comFast Forward Your Development
Measurement Levels
•Transport Level•Service (transaction) Level•Media Quality Level
Add Your Text
Add Your Text
Add Your Text
Add Your Text
Content Quality
Video Stream Quality
Transport Quality
Transaction Quality
www.dsp-ip.comFast Forward Your Development
Transport Level
•Example: ▫Packet Loss▫Loss Patterns▫Jitter▫Delay
•Well understood•Defined by ITU and IETF
www.dsp-ip.comFast Forward Your Development
Transaction Level•Examples
▫Post-dial delay in PSTN/mobile networks▫Video start time
•Requires understanding in both network monitoring and signaling (IGMP, SIP) and in media coding (analysis of the media to discover dial tone or I frame)
www.dsp-ip.comFast Forward Your Development
Channel zapping delay
•Multicast saves bandwidth but creates signaling delays:▫Multicast Leave +▫Multicast Join + ▫First I Frame +▫Up to 2 seconds buffering time
Leave latency
Join latency
Signaling Latency
First I Frame
Media Latency
Total Channel zapping Latency
Buffering latency
First frame viewed
www.dsp-ip.comFast Forward Your Development
Microsoft improved channel zapping
•Unicast server is used to serve the stream till the second multicast stream has an “I Frame”▫Immediate I Frame▫Shorter buffering time
Leave latency
Join latency
Multicast stream
Total Channel zapping Latency
I Frame unicast
unicast stream
I Frame Multicast
Shorter buffering time
First frame viewed
www.dsp-ip.comFast Forward Your Development
VoD Signaling
Play to server
Total VoD Signaling Latency
Buffering latency
First frame viewed
•Unicast server is used to serve the stream till the second multicast stream has an “I Frame”▫Immediate I Frame▫Shorter buffering time
www.dsp-ip.comFast Forward Your Development
Content Level• Content quality is a payload based measurement.
▫ Requires decoding of the video stream▫ Understanding of the buffering and error concealment
algorithms of the decoder • CPU intensive – Does not scale• Accurate• Used mostly is Lab equipment and diagnostic
equipment• Examples:
▫ PSNR ▫ ITU-T J.144
• Usually requires the reference (original) stream• Tests:
▫ Source artifacts▫ Source quality
www.dsp-ip.comFast Forward Your Development
Standardization landscape
DSL
DSL Forum
TR-64, TR-69LAN and WAN monitoring standards
ITU
Study Group 12
Algorithms for end-to-end transmission performance
ITU
VQEG – Video Quality Expert Group
Video performance measurement based on Subjective tests Database
Used in Diagnostics / Lab
ATIS
IIF – IPTV Interoperability Forum
QoS Metrics Standardization
Used for Monitoring
www.dsp-ip.comFast Forward Your Development
Example: ATIS IIF Quality Metrics
VSTQ - Video Service Transmission Quality ▫ Transmission Quality - codec/ content independent▫ Based on the rate and distribution of effective packet loss and
discardVSPQ - Video Service Picture Quality
▫ Estimated viewing quality▫ Considers the impact of VSTQ, video codec type and rate,
resolutionVSAQ - Video Service Audio Quality
▫ audio listening quality▫ Considers the impact of VSTQ, audio codec type, sample rate, …..
VSMQ - Video Service Multimedia Quality ▫ overall user experience▫ Combined effect of VSPQ, VSAQ, audio-video synchronization..
VSCQ - Video Service Control (Plane) Quality ▫ Considers responsiveness and reliability of control plane (trick
play)
www.dsp-ip.comFast Forward Your Development
Monitoring levels• J.144 and PSNR examines the video content only (payload
measurements)• TR101290 examines only transport stream data and
coherence without examining the video content• V -Factor and VQS looks at packet loss, jitter and loss patterns
data and incorporate it with codec information and video header information
• MDI – Examines only packet loss and packet loss patterns without considering the codec or video information
TR101290MPEG2TS Headers
V-Factor, VQSMDI
J.144, PSNRVideo payload
www.dsp-ip.comFast Forward Your Development
MEASUREMENT DOMAINSHeadendNetworkVoDHome
www.dsp-ip.comFast Forward Your Development
DVB TR 101290
•DVB Transport stream monitoring standard•Alerts and KPI:
▫TS sync loss▫Sync byte error▫PAT error▫Continuity counter errors▫PMT error▫PID error▫Transport error
▫CRC error▫PCR error▫PCR
discontinuity▫PCR
accuracy error
▫PTS error
www.dsp-ip.comFast Forward Your Development
Network measurement points
• Head-end▫ VoD Servers▫ Encoders▫ TS muxers
• Network core▫ Routers
• Network edge▫ ATM switches▫ GBE Switches
• Access▫ DSLAM / BRAS
• CPE▫ STBs▫ Home network
Source: Brix-Video Quality Measurement Algorithms WP
• Network Probe interface GBE, OC3/12, STM1/4
• Raw Video input MPEG2 TS - DVB ASI,
DVBoIP Digital and analog video
www.dsp-ip.comFast Forward Your Development
Headend
•Video source quality•Video encoding quality•MPEG2 Transport stream parameters
▫TR100290 1st-3rd level priority alerts
•Inputs▫Raw analog video in a variety of forms
(SD/HD)▫DVB-ASI▫DVBoIP▫DVBoATM (AAL1, AAL5)
www.dsp-ip.comFast Forward Your Development
VoD
•Pre-deployment▫RTSP signaling latency (delay)▫Performance under stress (user load)▫Trick Mode (FF, REW)
Signaling delay Video quality Buffer state (overflow and underflow)
• Monitoring▫Loss of data (under stress)
•Interfaces:▫GBE▫10GBE
www.dsp-ip.comFast Forward Your Development
Home network monitoring challenges•Scalability – Millions of STBs
▫Cost – Monitoring hardware and/or software costs must not add to STB costs (multiply by number of STBs)
▫Data – Large amounts of data which needs RT processing, filtering, logging and analysis
•Unclear network structure•Non “standard” equipment•Installation, support and debugging
problems
www.dsp-ip.comFast Forward Your Development
Home network monitoring roots
•Home network monitoring is currently a minor feature in IPTV “Support Automation”
•Support Automation handles:▫Installation▫Preventive maintenance▫STB and CPE remote management (TR69 and
more)▫Service verification for field technician▫CRM support▫Interaction logging▫Home Network monitoring and repair
www.dsp-ip.comFast Forward Your Development
CPE network management•2Wire solution -Three data views
OSS/BSS Customer care Data Analysis
•Other companies in this field▫SupportSoft▫Enure ▫Motive
www.dsp-ip.comFast Forward Your Development
Low level network monitoring
www.dsp-ip.comFast Forward Your Development
Vendors specialization
Lab Testing Monitoring
Incumbent Telecom
Agilent
Ixia
Spirnet
Cable
Rhode & Schwartz
Telecom
Agilent
JDSU/ Acterna
Spirnet
Cable
Thomson
Snell&Wilcox
New entrants
Telecom Shenick
Telecom Telchemy
Bridge Technologies
iBrixQoSMetrix
www.dsp-ip.comFast Forward Your Development
DSP-IP Contact informationFor Course materials & lecture request contact: For Course materials & lecture request contact:
For projects development services contact: For projects development services contact:
Adi YakovAdi YakovTraining and Courses ManagerTraining and Courses [email protected]+972-9-8651933+972-9-8651933
www.dsp-ip.com Mail : Mail : [email protected] Phone: Phone: +972-9-8850956, +972-9-8850956, Fax : +972-50- 8962910Fax : +972-50- 8962910
Alona AshkenaziAlona AshkenaziTechnology Management Technology Management Services Services : : [email protected]+972-9-8850956+972-9-8850956