Qosmet 20160219

VTT TECHNICAL RESEARCH CENTRE OF FINLANDLTD

Qosmet – A solution formeasuring Quality of Service

Contact:Dr. Jarmo Prokkola

[email protected]: +358 20 722 2346VTT (Technical Research Centre of Finland)P.O. Box 1100 (Street: Kaitoväylä 1, Oulu)FI-90571 Oulu, Finland

19.02.2016

19/02/2016 2

Qosmet Overview

Qosmet is a solution for passive measuring/monitoring of QoS (Quality of Service)performance of a network path.

Allows also QoE (Quality of Experience) estimation for selected applications(e.g., VoIP and video)

The measurement results can be evaluated in real-time or stored for later analysisThe architecture of Qosmet allows to trigger events in (e.g. alarms or trafficrerouting) real-timeQosmet is at it’s best when measuring the QoS of real-time applications (e.g., VoIP,video conferencing, IPTV), but practically any IP based application can bemeasuredThe solution is based on light-weight measurement agents (software components),enabling the measurements to be remotely invoked and controlled when needed.Possible users include network testing experts, operators, service providers, servicedevelopers, administrators, network and measurement equipment manufacturers,researchers, and end-users

The solution is developed by a neutral research organization, VTT

19/02/2016 3

Benefits

Discover how the applications/services really perform over the networkThe measurements are passive, i.e., overhead is minimizedEnables quality based alerts & actionsQosmet can be integrated as an information source to a larger networkmanagement/monitoring solution

A 3rd party solution can fully control the measurementsEnables also bottleneck tracingSupports long-term performance monitoring, e.g.,

Are there certain periods of time, when the service quality is not tolerable?The solution uses light-weight SW agents, which could stand by in all thepotentially interesting network nodesMeasurements can be carried out also over NATs (Network AddressTranslation)Can be used to provide information on CEM (Customer ExperienceManagement)

19/02/2016 4

Qosmet in a Measurement point

Measurement point = a network node,which has the QosmetService installedQosmetService is the main componentof Qosmet, executing the measurements

A light-weight piece of software that canbe installed to any network device, whoseQoS is of interestA single QosmetService supports manyindependent measurements

QMCP (QoS Measurement ControlProtocol) is the controlling interface ofQosmet

QosmetServices can be controlled by anyQMCP capable SW (API provided)

Qosmet’s capturing process makesQosmet independent of the measuredapplications

No privacy issues: The content of theapplication data packets are not stored

Network device (Terminal, Server, Router, etc.)

Networkinterface

Network stack of the operating system

Application 1(e.g., VoIP)

QMCPinterface

QosmetServicesoftware

Application 2(e.g., Video)

...

Control stream

Qosmet’s capturing process

19/02/2016 5

QosmetServices can be located whereversoftware installations are possible, e.g., in

User terminalsServersRoutersExternal measurement computers connectedwith network taps or mirror ports of switches

QosmetServices should be located at pointswhere the application traffic streams ofinterest are flowingThe more measurement points there arewithin a network path, the higher accuracy ofthe QoS measurement can be reachedOne can select what to measure, e.g.,

A single application flowAggregation of selected flowsAll the traffic between the measurement points

Connecting Qosmet

QosmetService

QosmetService

QosmetService

QosmetService

QosmetService

User terminal

Measurement computerwith network taps

Measurement computerconnected to monitoring

port of a switch

Router

Application servers

...

19/02/2016 6

The use of Qosmet: Basics

A typical Qosmet measurement is a two-point measurement betweenQosmetServices

Gives QoS of the measured applicationstream between the measurement pointsOne can measure also in a single point,but then the statistic set is limited

The measurements are set up by Thecontrolling entity on-demand

An example of the controlling entity isQosmet’s Graphical User Interface (GUI)It can be located in same device withQosmetService or in another device

When QosmetService is idle (i.e., nomeasurement activated), no resourcesare consumed

One can install QosmetServices to all theinteresting locations to be ready to beactivated when needed.

IP Network

QosmetService(activated)

Qosmet GUI

Application stream

QMCP

connectionduring

ongoingm

easurement

QoSResults


QosmetService(idle)

QosmetService(idle)

19/02/2016 7

The use of Qosmet: Topologies

Only a single QosmetService is requiredper measurement point

Each measurement is independent anda single QosmetService can serve manyof them simultaneouslyOne can e.g., create a starmeasurement topology, where a singleQosmetService runs severalmeasurements towards different clientdevices

Qosmet supports many different kinds ofdevices of Windows or Linux basedplatforms (including Android)If accurate delay statistics are desired,clock synchronization between themeasurement points needs to behandled

Qosmet GUI

QoSResults

ofmeas. 3





Application servers

Application client Application client

Application client

Measurement controller

Qosmet GUI

QoSResults

ofmeas. 1

&meas. 2

Measurem

ent 2

Measurement 3

19/02/2016 8

The use of Qosmet : The controllingentity and databases

The controlling entity can be anyQMCP-capable software, e.g.,

Qosmet’s GUIThird party implemented networkmonitoring solution

QMCP supports also Listening modeA light-weight implementation of QMCPresults reception onlyCannot control Qosmet

Qosmet DB is a dedicated database forQosmet’s measurement results

Operates QMCP in Listening modeThis is an optional component: Notrequired to run Qosmet

VTT provides the SW API library for theQMCPSecurity issues: one can limit, whichControlling entities are allowed toconnect QosmetServices.

QMCPconnections

during ongoing

measurement

19/02/2016 9

The use of Qosmet: Service registration

A QosmetService can be nominated asThe central service

The same software as anyQosmetServiceThis is an optional feature, whosepurpose is solely to ease themeasurement setup

The central service keeps upinformation of the availablemeasurement points in the networkRegistered QosmetServices can beeasily waked up for measurementsTracking of QosmetServices gives alsoinformation of whichconnections/servers are up and running

Coarse network status information

19/02/2016 10

Measurement output (1/2)

The mostly used statistics include:

QoS statistics:DelayJitter (average and absolute)Packet lossConnection break statistics

Traffic statistics:LoadVolume of dataPacket sizes

Flow statistics:Flows seen in the measurement nodePer flow: load, duration, transmitted data, etc.

QoE: (Pseudo-subjective analysis)Accurate VoIP and video quality with PSQA in MOS scaleGeneral application QoE with GQoSM

19/02/2016 11

Measurement output (2/2)

All the collected statistics are one-way statisticsSend and received directions are evaluated separately

Qosmet produces average as well as per packet accurate resultsThe results are calculated in real-timeThe measurement results can be also output to a dedicated data baseor simply to a text file

Rough post analysis can be made with Qosmet GUIAccurate post analysis with common statistical tools likeMATLAB, R, or MS Excel

Delay, jitter, and packet loss calculations follow the principlespresented in IETF RFC’sWith a GPS device, the results can be tied to location and alsovisualized on a map with the GUI

19/02/2016 12

Features of the Qosmet GUI

Real-time and offline analysisAny of the statistics can be visualized as

A heat mapMetersGraphs

QoE estimation bar visualizationControl of the parametersShows availableQosmetServicesNumerical resultsFlow analysis

19/02/2016 13

Qosmet Measurement Examples:WLAN performance during a drive test

Setup:The QoS of a VoIPconnection ismeasured from amoving vehicle to acontrol center behinda reliable fixednetwork connectionThe WLAN consistsof many accesspoints

Findings:WLAN handovers(with this particularnetwork) causeserious QoSdisturbance and longconnection breaks

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.1

1

10

100

1000

10000

440 480 520 560 600

Pack

etlo

ss

Dela

y[m

s]

Measurment duration [s]

UL delayDL delayUL Packet lossDL Packet loss

Further actions are needed considering the WLANsetup in order to solve the found performanceproblem regarding the handovers

19/02/2016 14

Qosmet Measurement Examples:Basic QoS performance of a cellular network

Setup:A field test with a commercialWCDMA/HSPA (3G) networkGeneral QoS is visualized on amapSignal strength was measured tobe strong throughout the area ofinterest

Findings:Basic QoS test revealed seriousproblems in Uplink (UL) direction.Downlink (DL) performance issomewhat tolerable even thoughthere are also some poorperformance spots found.

The network problems can bedirectional, i.e., studying roundtrip performance is not enough.Figure: 3G DL delay performance of one operator

Figure: 3G UL delay performance of one operator

20 ms 60 ms 100 ms

Legend:

19/02/2016 15

Qosmet Measurement Examples: Theapplication’s perspective on QoS

Setup:A field test with a commercial LTE networkGeneral QoS is visualized on a mapSignal strength was measured to be strongthroughout the area of interest

Findings:With low rate traffic, the QoS is mostly good inthe area measuredWhen data rates is higher, already rate above1 Mb/s significantly decreases the QoS: TheQoS coverage of the same network looks nowcompletely different!

Network performance is important to bevalidated with real target applications

Fig. QoS with low rate traffic (~100 kb/s)

Fig. QoS with higher rate traffic (~1-2 Mb/s)

19/02/2016 16

References: Research Work

Research project cooperationTekes 5GTN, 2015 – 2016

Qosmet is used for testing and verifying the 5G test network function and qualitycoverage area

Celtic NOTTS, 2013-2016Qosmet is used,e.g. for developing QoE models for HTTP-based video streaming andtesting live commercial OTT services.

Tekes CORE project series, 2011 – 2017Qosmet is used for performance measurementsQosmet is a part of cognitive network control system.

EU UniverSelf, 2010 – 2013Qosmet is used to verify the performance of the developed load balancing solution

Celtic IPNQSIS, 2011 – 2013Qosmet is used for performance measurements

EU TALOS, 2008 – 2012Qosmet is used to evaluate communications quality in real-time in unmanned groundvehicle (UGV) system.

Six other research projects between 2004 – 2014Publications

Qosmet has been an essential part of research work reported in over 30 publications

19/02/2016 17

References: Commercial Work

Projects (measurement campaigns, consultation, development)A project for SME company in Finland, 2015

Real-time VoIP QoS evaluation with QosmetA project for PSA International, Singapore, 2013 – 2014

Qosmet used for performance measurements and as a part of anintelligent mobility solution

A project for ESA (European Space Agency), 2013 – 2014Qosmet used for video quality measurement / estimation

A project for Finnish Defense Forces, 2013Qosmet used for performance measurements

EU/SME DOCKINGASSIST, 2012 – 2013Qosmet used for WiMAX maritime performance evaluations

Projects for NSN, 2011 – 2012Qosmet used for performance measurements

Three other projects between 2008 – 2011Licensing

Qosmet is in use of several organizations

19/02/2016 18

Offerings

Qosmet solutionLicensing the solution as suchIntegrating the solution to a 3rd party tool

Light integration: QMCP-capability is implemented to athird party tool, bringing the capability to controlmeasurements.Full integration: Qosmet core functionalities are directlyintegrated to a third party tool.

Implementing Qosmet-like features to a third party toolCustomization is possible

Measurement servicesTechnology validation & performance tests of new applicationsover different networks running different devices, etc.

19/02/2016 19

More information

Up-to-date information about Qosmet is provided inhttp://www.cnl.fi/qosmet.html

Documents

Qosmet 20160219