VoIP on WIFI - TU Berlin · Berlin URL: VoIP on WIFI Christian Hoene September 13th, 2004 UNITN – DIT, Trento. 2 Content ... Verified software in various research projects, e.g

1

TechnischeUniversität Berlin URL: www.tkn.tu-berlin.de

VoIP on WIFI

Christian HoeneSeptember 13th, 2004UNITN – DIT, Trento

2

Content

Motivation: Semantic data-linkDetermining VoIP Quality

Perceptual quality of VoIP packet tracesListening-only tests

On the Importance of a VoIP PacketAudio DemosPredicting the Importance

Voice over WLANExperiments: Semantic data-linkSimulations: Voice over EDCA

Conclusions

3

Phone Calls on WLAN?

State of the Art:: Two technologies for voice-(DECT) and data services (IEEE 802.11)

Increased total cost of ownership (TCO) for 2x purchase, 2x operation, 2x administrationDo customers prefer combined products?

Data over DECT?Voice over WLAN?A new wireless technologyfor voice and data?

Digital packet telephone services (with VoIP) on wireless local networks using IEEE 802.11+

4

Research Challenges

WLAN phones are often worse than DECT phones:Higher energy consumptionCoverage is smallerTel-systems are more complex and expensive.Optimization of speech transmission over WLANin order to achieve:

1. Better speech quality for a given capacity2. Higher capacity at the same level of speech quality.3. Reduction of energy consumption

5

How to Improve Speech Communication?

New speech coding algorithms (“codecs”)Classic approaches (e.g. in UMTS)

combined source channel codingunequal bit error protectionsoft-bit decoding

Algorithms in packet networksSeparating different data flows (IntServ, DiffServ)Overprovision (increase bandwidth)Medium access protocols (PCF, 802.11e)High priority for delay-sensitive flows

6

Enhancing Voice over WLAN

Where to enhance?Backbones are fast. They are not a problem.WLAN access is the bottleneck!

What to change?Codecs are given: ITU standardsTransport/Network: IETF RFC standardsData-link and physical layer: IEEE 802.11... standards.

But: Change parameters sets on the fly!

7

Semantic Transmission

Utilize the temporal variance of speech:Of course: transmitted during voice activityand wait during silenceBut: even during voice activity speech segments differ greatly.

Transmit every speech frameTransmit every speech frameaccording to its importance. according to its importance. Drop the unimportant frames sooner than more important ones

(e.g. voiced sounds are more important than unvoiced sounds ).

8

Classic Protocol Stack

RTP+FECUDP/IP

Encoding

Packet Importance

Back-bone

Access Point IEEE 802.11...

Decoding + concealment

Playout Scheduler

Wire-less

Data Link IEEE 802.11...

RTP+FEC UDP/IP

VoIP - Quality Assessment

9

Semantic Data-Link

RTP+FECUDP/IP

Encoding

Packet Importances

Back-bone

Access Point IEEE 802.11...

Decoding + concealment

Playout Scheduler

Wire-less

Data Link IEEE 802.11...

RTP UDP,IP

VoIP - Quality Assessment

10

Content





Conclusions

11

How to Judge the Quality of a Call?

Measure the quality of a Wifi-Voice system:Gather packet loss rates, mean delay, etc.

Easy but inaccurate

In the end the service quality is importantHuman based listening tests are extensive

ITU P.862 (PESQ algorithm) measures speech qualityCompares original sample with the transmitted versioncalculates Mean Option Score (MOS) (1=bad, 5=excellent)

ITU G.107 (E-Model) predicts quality of tel.-systemConsiders echo, loudness, coding, packet loss rate, delay, …Result: R Factor (0=bad, 70=toll quality, 100=excellent)

12

Assessing a Trace of VoIP Packets

PESQ and E-Model cannot judge the impact of:Variable packet delay adaptive playout timePacket losses are considered to be random but depend on the content and context of the speech frameNot applicable to assess VoIP packets traces resulting from simulations or experiments

Our solution:Using different playout schedulers and encoding schemescombine von E-Model und PESQ (approach approved by the ITU)PESQ calculate speech qualityE-Model to combine MOS rating and transmission delays

13

Speech Quality and Transmission Delay

Speech Recordings

SpeechCoding

(coding distortion)

Experimentor Simulation

(loss or delay)

Decoder PLC

PESQReferenceDegraded

Audio

AudioMOS

Playout Buffer

(loss or scheduling)

E-ModelIe Result:Id R Factor

Degraded Audio

Mean Delay

14

Equations – referring to the SPECTS’04 paper…

15

PESQ not verfied conduct listening tests

Can PESQ measure playout rescheduling?Or non-random packet losses?

We conducted formal listening-option testsResult: Correlation between PESQ and humans is for

non random packet losses: R=0.94playout re-scheduling: R=0.87(R=0 no correlation; R=1 perfect correlation)

16

Assessing VoIP: Summary

Implemented all common playout schedulersVerified software in various research projects, e.g. voice over WLAN, impact of handover, ad-hoc.Open-source softwarehttp://www.tkn.tu-berlin.de/reseach/qofis

C. Hoene, S. Wiethölter, and A. Wolisz, "Predicting the Perceptual Service Quality Using a Trace of VoIP Packets", In Proceedings of QofIS’04, Barcelona, Spain, September 2004.C. Hoene, H. Karl, and A. Wolisz, "A Perceptual Quality Model for Adaptive VoIP Applications", In Proceedings of International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS'04), San Jose, California, USA, July 2004, Paper won the Best Paper Award of the conference. C. Hoene and E. Dulamsuren-Lalla, "Predicting Performance of PESQ in Case of Single Frame Losses", In Proc. MESAQIN 2004, Prague, CZ, June 2004.S. Möller and C. Hoene, "Information About a New Method For Deriving the Transmissio Rating Factor R From MOS in Closed Form", ITU, May 2002, Temporary Document for the study group 12.

17

Content





Conclusions

18

Introduction

Losing one Voice-Over-IP packet impairs the perceptual quality in a wide range, depending on

the frame speech propertiesthe encoder/decoder/concealment algorithmsdecoders resynchronization time after loss (especially low-rate decoders might maintain a wrong state after loss for the following frames.)the surrounding speech.

Example: Discontinuous Transmission (DTX)Speech frames during silence are less importantLower frame rate during silence

19

Measurement Setup (1)

Large sample database (ITU P suppl. 23)4 Languages x 4 speakers x 52 samples = 8328s each, two sentences

Codec's: ITU G.711 + Appendix II (64 kbit/s)ITU G.729 (8 kbit/s)3GPP Adaptive Multi-Rate (4.75...12.2 kbit/s)

20

Measurement Setup (2)

Loss Generatordifferent positions (800) and loss lengths (1,2,3,4)

Totally: some millions different testsWill take several years if humans conduct these speech quality tests.Thus, use ITU P.832 PESQ.PESQ calculates a Mean Opinion Source

21

3.4 3.6 3.8 4.0 4.2 4.4

0.0

0.5

1.0

1.5

2.0

2.5

3.0

(a) PESQ LQO−MOS, G.711, 64kbps

PD

F: k

erne

l den

sity

est

imat

es

3.4 3.6 3.8 4.0 4.2 4.4

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.4 3.6 3.8 4.0 4.2 4.4

0.0

0.5

1.0

1.5

2.0

2.5

3.0

no packet lossone loss of 10msone loss of 20ms

3.4 3.6 3.8 4.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

(b) PESQ LQO−MOS, G.729, 8kbps

3.4 3.6 3.8 4.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.4 3.6 3.8 4.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.4 3.6 3.8 4.0 4.2

0.0

0.5

1.0

1.5

2.0

2.5

3.0

(c) PESQ LQO−MOS, AMR, 12.2 kbps

PD

F: k

erne

l den

sity

est

imat

es

3.4 3.6 3.8 4.0 4.2

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.4 3.6 3.8 4.0 4.2

0.0

0.5

1.0

1.5

2.0

2.5

3.0

no packet lossone loss of 20msone loss of 40ms

2.8 3.0 3.2 3.4 3.6

0.0

0.5

1.0

1.5

2.0

(c) PESQ LQO−MOS, AMR, 4.75 kbps

2.8 3.0 3.2 3.4 3.6

0.0

0.5

1.0

1.5

2.0

2.8 3.0 3.2 3.4 3.6

0.0

0.5

1.0

1.5

2.0

22

Definition: Metric for ImportanceThe packet’s importance is the quality degradation that its loss would cause.

Definition:The importance of frame losses is the difference between the speech quality due to coding loss and the quality due to coding loss and frame losses, times the length of the analyzed sample:

s: samplet(s): samples length (s)c: codec implementationlx: loss event, one or multiple correlated frame losses

23

Measuring the Importance of a VoIP packetUsing PESQ to measure the loss of one speech frameSpeech frames differ largely

AMR 4.75 G.729 AMR 12.2 G.711 active 0.389 0.655 0.923 1.338 silence 0.003 0.004 0.008 0.016

Mean Importance [MOS*s]

24

PESQ not verified conduct listening tests

Can PESQ measure packet importances? Nobody knows…Thus: Conduct formal listening-only tests!

Problem: Humans can not hear single packet losses!

Human just can hear multiple packet losses.Thus, drop multiple similar frames…PESQ can identify similar frames.

25

Just try it: www.tkn.tu-berlin.de/research/mongoliaSelect predefined parameter sets

chose sample

Compression?

Listen to it!

amount of loss

Choose an another loss pattern

Drop only frames with an importance between

min. and max.

Talking or silence? Voiced or unvoiced?

Judge the speech quality by your self!

Sample statistics

26

27

28

29

30

Human LQS-MOS vs. PESQ LQO-MOS

trend line: R = 0.94

2,4

2,6

2,8

3,0

3,2

3,4

3,6

3,8

4,0

2,4 2,6 2,8 3,0 3,2 3,4 3,6 3,8 4,0MOS (PESQ)

MO

S (h

uman

s)

31

MOS Variance vs. Prediction Performance

trend line: R = 0,876

0,00

0,05

0,10

0,15

0,7 0,8 0,9 1,0Corellation between PESQ and humans

PESQ

MO

S va

rianc

e

G.729G.729

G.711G.711

AMR 4.75AMR 4.75

AMR 12.2AMR 12.2

32

Content





Conclusions

33

Understanding the Importance

Frame loss distortion is due to two effectsImperfect frame loss concealment (2+ ≠ 3)Error Propagation (4…7 ≠ ~4…~7)

tX+1

1 32 4 65 7 98

2+2 98

Time axistX tX+t

Original(Sender)

Degraded(Receiver)

Frame loss Error propagation

1 ~4 ~5 ~6 ~7

Legend:1,2,… 9 Frame sequence number2+ Concealed frame

(extrapolating “2")~3 … ~7 frames falsified by

error propagation Transmission successfulFrame loss

34

Length of Error Propagation After a Frame Loss

35

Concealment (left) vs. Error-Prop. (right)

Importance of a Speech Frame depends onImperfect concealment (9/10)Error propagation (1/10)

Good news, because Imperfect concealment can be measuredError propgation cannot be measured

36

Predicting the Importance at Real-Time

Algorithms by Sanneck, De Martin, HoeneAssessment, analysis, more details to come(Not yet published)

37

ContentMotivation: Semantic data-linkDetermining VoIP Quality



Voice over WLANIntermezzoExperiments: Semantic data-linkSimulations: Voice over EDCA

Conclusions

38

Impact of Movement on Link Quality?

Measured loss rate and TX delay A bidirectional phone call, consisting solely of RTP packetsCommunication between an access point and mobile nodeExperimental enforced controlled motion (Node Mover)

simulated call

stationary base station

wireless link

Node Mover

WLAN phone

39

Measurement Environments (Office and Gym)

border position mean position

near position

20m

10m

wall

laboratory

hall

20m 10m

2m

Stationary PC

5m

Measuring positions: 5, 10, 15, 20, 25, 30, 35, 40m

Stationary PC

Mobile PC

Office

Gym:

40

Node Mover (1)

How to enforce controlled motion?reproducable, over long time spans?

Human based experiments not feasible…

Solution: A prepared toy train

How to supply notebook withpower?

1. Separate Cable2. Supplementary,

power carrying rails

41

Node Mover (2)

42

43

1: Office Environment using vanilla RM conf.

1m 5m 1-5m 8m 12m 8-12m 14m 18m 14-18m-1

0

1

2

3

4

5

6

7

8

9

10lo

ss ra

te [%

]

moving

44

3: Overall loss rate (mean from all experiments)

0

2

4

6

8

10

StationaryMoving

1 MBit/s 11 MBit/s 1 MBit/s 11 MBit/s 1 MBit/s 11 MBit/s 1 MBit/s 11 MBit/s

pack

et lo

ss [%

]

45

Summary

We presented experimental set-up and methodology measuring IEEE 802.11b wireless linkDevelop software Snuffle, which traces protocol messages as well as states on multiple layers.

Measured packet loss and transmission delay, influenced by modulation type, ARQ, environment, slow motion, and others factors.

Slow motion usually decreases link qualityIn a Rayleight fading environment using automatic rate selection loss rate were lower! Mere chance?

46

Introduction: QoS in WLAN

How to provide time-bounded services like VoIP / audio / video streaming over WLANs?

hard requirements: low delay, low jitter and constant throughput

Medium access in legacy 802.11: • point coordination function (PCF)• distributed coordination function (DCF)

Medium access in 802.11e:• hybrid coordination function (HCA)• enhanced DCF (EDCA)• packet frame grouping (CFB, not include in std. anymore)

47

Content





Conclusions

48

Adaptive Data Link: ImplementationIInntteerrnneett PPrroottooccooll SSttaacckk

DDeevviiccee DDrriivveerr

Packet Classifier

RRaaddiioo MMooddeemm

tx. packets rcv. packets

tx. Mode

Feasibility study and prototypeVoice sounds are protected by a higher number of retransmissionsChanged device driver of 802.11b WLAN cardURL http://www.tkn.tu-berlin.de/research/easysnuffle/

49

Semantic Data-Link Protocol

Often improvement of speech qualityOften improvement of speech quality,even when using a simple algorithmHow large is the gain that can be reached with a sophisticated algorithm?

50

55

60

65

70

75

POS5 POS3 POS4 POS1 POS2 Measurement Position

R F

acto

r(0

bad,

100

exc

elle

nt)

default MAC

VoIP ARQ

11MBit/s+no ARQ

11MBit/s+VoIPARQ

50

Content





Conclusions

51

Simulating IEEE 802.11e

Implementation of a simulation model supporting EDCF and CFB for the ns-2.26 network simulatorSubset: Correctness is testedMangold simulator is not free available.Repeated the research of Mangold et al. (RWTH-Aachen) no major differenceOpen-source software: 500 downloads since December 2003http://www.tkn.tu-berlin.de/research/802.11e_ns2/

52

Voice over IEEE 802.11e EDCA

Open source, modular and verified IEEE 802.11e simulation model for ns-2.26VoIP Results- Strong improvements with EDCA, EDCA+CFB- Low costs for EDCA compared to DCF- Max. number of VoIP calls with

DCF: 10; EDCA: 10, EDCA + CFB: 11

Simple EDCA is adequate for voice calls andHCA only for a high number of calls and for the downlink.

53

Speech Frames – Dropping Strategy

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

AMR 12.2 : Frame loss rate [%]

PE

SQ

MO

S

bestworstrandomDTX

Drop packets in cases ofCongestionWireless fadingSaving Energy

Best: dropping the unimportant frames firstWorst: dropping the important frames firstRandom frames lossesDTX: drop first silences frames, then active frames (randomly)

54

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

G.711 : Loss rate [%] (only ON)

PE

SQ

MO

S

bestrandomDTX

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

G.729 : Loss rate [%] (only ON)

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

AMR 12.2 : Loss rate [%] (only ON)

PE

SQ

MO

S

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

AMR 4.75 : Loss rate [%] (only ON)

PE

SQ

MO

S

bestrandomDTX

BestDTX

55

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

MR122 : Frame loss rate [%]

LQO

−M

OS

R= 0R= 0.2R= 0.4R= 0.6R= 0.8R= 1

Impact of Prediction Error (Importance is Falsified by Random Noise)

56

Conclusion

Enhancing speech quality on VoIP over WLANwith semantic data-link protocolsupporting packet importances

Packet importance for GSM an UMTS to reduce power consumption?

Thanks to our hard-working students

Questions?

Documents

VoIP on WIFI - TU Berlin · Berlin URL: VoIP on WIFI Christian Hoene September 13th, 2004 UNITN – DIT, Trento. 2 Content ... Verified software in various research projects, e.g