Effect of FEC mechanisms in the Performance of Low Bit Rate Codecs in Lossy Mobile Environments Rolando Herrero, PhD Northeastern University, Boston, MA,

Effect of FEC mechanisms in the Performance of Low Bit Rate Codecs in Lossy Mobile Environments

Rolando Herrero, PhDNortheastern University, Boston, MA, U.S.A.

Martin Cadirola, Director/VP Business DevelopmentEcotronics Ventures LLC, Clarksburg, MD, U.S.A.

Background

Developers of Kapanga (Voice/Video/Fax SIP client)

Field work with OEM, QA and military environments

Collaboration with academia

© 2014 - IPT Communications Conference - September 29, 2014

Motivation


Motivation


Voyager 1-Launched in 1977-130 AU from the Sun-14 hours to Tx/Rx-38,000 mph = 61,000 kph

Motivation


HOW? FEC!

Mobile Networks


Source: http://www.ice.rwth-aachen.de/

- Multipath loss- Slow/Fast fading channels- Applicable in RT over 3G/4G/LTE

Use of Forward Error Correction (FEC) techniques in media transmission are critical here too!

Motivation

Mobile networks

Interest in creating real-time QA tools

Room for innovation in improving performance


Motivation

Work based on Barton, Lemberg, Sarraf, Hamilton, “Performance analysis of packet loss concealment in mobile environments”, in Communications Quality and Reliability (CQR), 2010 IEEE International Workshop Technical Committee.

Network impairments in mobile networks are mostly due to fading, especially in high multipath environments

The characteristics of fading channels are a well known phenomenon and their statistics have been modeled mathematically by a two-state Markov process


How do LBR codecs perform in such a mobile network? How can we assess quality of

speech?

Theoretical Model: Markov Process

2-state Markov Process models a fading channel mobile network environment


• Model has 2 states:• High Packet Loss State (H)• Low Packet Loss State (L)

• Model has 5 parameters:• Transition probability from low to high loss

states (p)• Probability that the channel stays in a high

loss state (α)• Distance between an RTP packet and the

transmission of the corresponding FEC RTP packet in packet times (D)

• Packet loss probability when the channel is in the low loss state (β)

• Packet loss probability when the channel is in the high loss state (γ)

Experimental Framework

We implemented 2 kinds of FEC techniques in the UA

XOR-based

Optimal for lower bandwidths

Repetition-based

Best quality for higher bandwidths

Assumptions wrt Markov process: β = 0 and γ = 1

All RTP packets are dropped in the high loss state

No RTP packets are lost in the low loss state


A little FEC background


Pn

160 bytes

Speech Packet

Repetition-based FEC …

160 bytes 320 bytes

P2 P2 P3

320 bytes

P1 P1

XOR-based FEC

160 bytes

P1 P2 …

RTP PayloadRTP

Header

RTP Packet

P2-P1

160 bytes

For each packet (on each case)

P3-P2P3

160 bytes 160 bytes 160 bytes


Modify a SIP UA to implement FEC mechanisms under study

Setup UA with Linux box running Netem

Use ITU-based speech waveforms *no tones nor signals*

Use of G.723.1, G.729A, AMR-NB, AMR-WB, EVRC, SILK, Opus



Netem takes the transition probabilities of the two-state Markov model, namely α and p, as input parameters

In order to validate different bursty scenarios the following combination of parameters is considered; α = 0.01, 0.05, 0.1 and 0.15 and p = 0.25, 0.5 and 0.75

In order to minimize latency, in part due to the restrictions imposed by playout buffers in regards to tolerable speech quality, the packet distance between the transmitted RTP and the FEC RTP is D = 2


Comparative Analysis


Theoretical Experimental G.723.1 G.729A AMR-NB EVRC AMR-WB SILK Opus0.01 0.25 95.01 94.79 3.91 3.84 3.89 3.82 3.75 3.66 3.650.01 0.50 83.39 81.36 3.46 3.10 3.17 3.23 3.29 3.12 3.270.01 0.75 67.99 72.62 2.87 2.61 2.43 2.68 2.70 2.64 2.710.05 0.25 95.00 93.21 3.83 3.78 3.81 3.74 3.68 3.55 3.590.05 0.50 83.53 80.78 3.38 3.01 3.09 3.12 3.20 3.07 3.180.05 0.75 68.46 71.86 2.81 2.53 2.35 2.59 2.61 2.56 2.640.10 0.25 94.89 91.05 3.75 3.72 3.74 3.67 3.61 3.47 3.510.10 0.50 83.57 79.29 3.32 2.91 3.01 3.04 3.11 2.93 3.090.10 0.75 68.86 70.42 2.73 2.45 2.29 2.52 2.54 2.48 2.530.15 0.25 94.66 90.72 3.68 3.66 3.67 3.61 3.52 3.41 3.440.15 0.50 83.43 78.59 3.24 2.83 2.94 2.95 3.04 2.83 3.010.15 0.75 69.06 69.64 2.62 2.39 2.23 2.46 2.47 2.41 2.45

Success % PESQ PESQ-WBα p

Bursty Packet Loss (Repetition-based FEC)



Theoretical Experimental G.723.1 G.729A AMR-NB EVRC AMR-WB SILK Opus0.01 0.25 95.01 94.79 3.77 3.69 3.73 3.66 3.59 3.41 3.420.01 0.50 83.39 81.36 3.32 2.97 3.04 3.07 3.14 2.97 3.100.01 0.75 67.99 72.62 2.73 2.44 2.28 2.52 2.54 2.48 2.570.05 0.25 95.00 93.21 3.71 3.63 3.66 3.60 3.52 3.35 3.370.05 0.50 83.53 80.78 3.27 2.90 2.95 3.02 3.08 2.92 3.010.05 0.75 68.46 71.86 2.63 2.84 2.89 2.93 3.01 2.84 2.930.10 0.25 94.89 91.05 3.64 3.54 3.58 3.55 3.46 3.29 3.310.10 0.50 83.57 79.29 3.21 2.82 2.88 2.96 3.01 2.87 2.930.10 0.75 68.86 70.42 2.54 2.77 2.81 2.86 2.96 2.76 2.880.15 0.25 94.66 90.72 3.57 3.48 3.52 3.49 3.40 3.22 3.240.15 0.50 83.43 78.59 3.15 2.75 2.80 2.91 2.97 2.82 2.890.15 0.75 69.06 69.64 2.46 2.71 2.74 2.81 2.90 2.69 2.81

Success % PESQ PESQ-WBα p

Bursty Packet Loss (XOR-based FEC)



Rate-Distortion (for a set of α and p)

Conclusions

We presented a framework to evaluate and compare the experimental and theoretical performance of RTP FEC applied to a set of narrowband and wideband codecs in a lossy mobile environment

It can be verified that the experimental probability of success of an endpoint receiving an RTP packet is within 15% of the analytical value (error) obtained by modeling the system with a two-state Markov process

Speech quality scores, based on PESQ and PESQ-WB metrics, follow the probability of decoding success for both when repetition FEC and XOR FEC schemes are used


Conclusions

When put in the context of rate-distortion plots narrowband codec G.723.1 and wideband codec Opus provide better performance by decoding same quality speech at lower transmission rates

Although the probability of success (from the theoretical model) does not take into account the codec type nor the characteristics of the playout mechanism it provides a good approximation to estimate the overall quality of the communication

When comparing XOR-based vs Repetition-based FEC it is clear that the latter gives better media quality at the cost of significantly increasing the transmission rate

FEC and other error correcting techniques offer improvements to the quality of *any* real-time media application: RTC, any rtp-based communications


Next Steps

Challenges (as any new technology) -> standarization, unless everybody uses one kind of UA

Continue research on new FEC mechanisms (TurboCodes, Reed Salomon)

Utilization of these techniques in:

UAV communications for civilian and military fields

LTE deployments overseas

Space applications


Earth Science

Drones/UAV/UAS

Thank you! Q & A!


Documents

Effect of FEC mechanisms in the Performance of Low Bit Rate Codecs in Lossy Mobile Environments Rolando Herrero, PhD Northeastern University, Boston, MA,