Colombia, 23-24 September 2013 QoS assessment for mobile speech services including user interfaces in vehicles and hands-free communication Dr.-Ing. H

Colombia, 23-24 September 2013

QoS assessment for mobile speech services including user interfaces in vehicles and

hands-free communication

Dr.-Ing. H. W. GierlichHead of Telecom Division

CITEL (PCC.I)/ ITU Forum on Information and

Communication Technology Service:

Quality, Control and Surveillance

(Cartagena de Indias, Colombia, 23-24 September 2013)


2

Outline

IntroductionThe impact of terminal design on speech qualityAssessment methods and standards for mobile terminalsDriver distraction and speech services in carsSummary

Users Expectation (QoE)

Colombia, 23-24 September 2013 3

SpeechQuality

... ...

Sound Quality &Naturalness

Intelligibility

TalkingEffort

ConversationalEffort

Double TalkPerformance

Expectation

Transmission ofBackground Noise

SpeechCharacteristics

Individual Listening

NetworkConditions

EnvironmentalConditions

ListeningEffort

Signal Processing in Modern Terminals


Modulation/

Demodulation

H(f )

-NLP

CNEC

Decoder

Coder

LoudspeakerLoudspeaker

DecoderDecoderComfort Comfort NoiseNoise

Mikrofon –Mikrofon –(Array)(Array)

ECEC

AGC/ AGC/ companding companding

…… CoderCoder

Noise Noise cancellatiocancellatio

nn

+

NR

z.B. GSM/UMTS

SE

Speech- Speech- “enhancement”“enhancement”speech recognition/

control

EQEQ

Important: All signal processing is non-linear, time variant and

targeted to speech enhamcement

Speech Quality


TESTINGTESTING…in general should be based on users perception…may be based on simple signals and procedures modelling aspects of users perception (e.g. loudness)…may be based on complex signals and advanced procedures modelling aspects of users perception (e. g. switching and double talk tests)…may be based on perceptual models using psychoacoustically motivated procedures (models of the human hearing)

USERS PERCEPTIONUSERS PERCEPTION…is subjective and generally different between subjects…is based on experience…is cultural dependant


6

Outline


Speech Quality Evaluation


Speech-Quality

EchoEchoPerformancePerformance

ReceivingReceiving

Double TalkDouble Talk

Quality of Quality of Background Background

Noise Transm.Noise Transm.

SendingSending

Terminal Testing

Bring the world to the lab:Simulate userSimulate user behaviourSimulate user environmentsSimulate network and access pointsSimulate network conditionsValdidate correct NTP access


The Typical Lab Setup


Sub-woofer

Testsystem

Network Simulation

Background

Noise

Simulation

Relevant Standards

GPP TS 51.010 (GSM 11.10)3GPP TS 26.131 & TS 26.1323GPP2 C.S0056TIA 928 (CDMA/TDMA)CTIAGSMA HDvoice standardFor more advanced testing:

ETSI TS 103 737 – TS 103740 standard series


Moste Important Test Parameters Enabling a Good QoS Performance

Basic acoustic tests e.g. Loudness Ratings (ITU-T P.79), Frequency response tests…Delay and echo testsSpeech quality tests – with and without background noiseConversational test including double talk, switching…Important: Test signals are mostly speech based – ITU-T P.501


The Impcat of Delay on Conversational Dynamics


Best case between fixed & mobile network (no satelite):~150 ms Worst case across networks:>> 500 ms

For superior conversational quality: delay < For superior conversational quality: delay < ~150 ms~150 ms(from the users perspective)

Network

Terminal A Terminal B0 ms 0 ms (.wav)

150 ms 150 ms (.wav)500 ms 500 ms

(.wav)

Conversational/Double Talk Test Signal


Word fully masked, one speaker continuosly talking

Word in partially masked

Word conicident with the other channel speaker, Simulating delay and unwanted double talk

Double talk sentences

Receive

Send

Male Female Male Female

Short isolated words

Word in a speech pause

Summary of Quality Parameters Using „Quality Pies“ according to ITU-T P.505


TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1

Subj. Exp. Assessmentpoorfairgood

TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

2

3,45

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 4

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1


TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

2

3,45

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 3

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1


TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

2

3,45

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 2

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1


TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

2

3,45

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 1

Conv. Quality

Background Noise PerformanceBackground Noise Performance


15

Outline


Driver Distraction


Physical/manual distractionPhysical/manual distractionManual operation of devicesManual operation of devices

Visual distraction Visual distraction Watching information other than roadWatching information other than road

Auditory distractionAuditory distractionFocusing on auditory events not related to Focusing on auditory events not related to the driving taskthe driving task

Cognitive distractionCognitive distractionOccupied by non driving related tasksOccupied by non driving related tasks

Technology Based Distraction in Cars


Car entertainment systemsCar entertainment systems

Navigation systemsNavigation systems

Phones for speech communicationPhones for speech communication

Car information systemsCar information systems

Text messanging systemsText messanging systems

„„Texting while driving“Texting while driving“

::

To be Avoided


In general all activities distracting In general all activities distracting from the driving task, esp.:from the driving task, esp.:

Visual cues which require different focus Visual cues which require different focus than roadthan road

Non intuitive manual operation of the car Non intuitive manual operation of the car

Loading the human auditory system by Loading the human auditory system by distracting cuesdistracting cues

Unnatural and low quality dialogs and Unnatural and low quality dialogs and communications communications

Driving Task


The driving taskThe driving task

=> Auditory Channel and Speech => Auditory Channel and Speech “available”“available”

mostly occupied: mostly occupied: visual systemvisual system

mainly involved: mainly involved: hands and legshands and legs

involved: involved: auditory systemauditory system

not involved: not involved: talkingtalking

Speech Based Services in Cars


o The main speech applications:The main speech applications:

• Speech recognition systemsSpeech recognition systems• Speech dialog systemsSpeech dialog systems• Text to speech systemsText to speech systems• Speech enhancement for Speech enhancement for

communication systemscommunication systems• Hands-free communicationHands-free communication• Enhanced in-car communication Enhanced in-car communication

systems between passengerssystems between passengers

Communicational Quality – Drivers Distraction


Superior Superior Communication Communication

Quality -> Reduced Quality -> Reduced Drivers DistractionDrivers Distraction

Seamless Interaction Seamless Interaction -> Low Delay, Perfekt -> Low Delay, Perfekt

Double Talk Double Talk PerformancePerformance

Perfekt Intelligibility, Perfekt Intelligibility, High Speech High Speech

Naturalness at the Naturalness at the Far EndFar End

Perfekt Intelligibility, Perfekt Intelligibility, High Speech High Speech

Naturalness in the Naturalness in the CarCar

Hands-Free in Cars


Why hands-free in Why hands-free in cars?cars?

Reduce phsyical Reduce phsyical distraction distraction (dialing, holding (dialing, holding the phone…)the phone…)

Reduce visual Reduce visual distraction distraction (watching display, (watching display, keyboard …)keyboard …)

Superior Sound Quality & Intelligibility in the Car


o Wideband services in mobile networks available soon->->

o Enabling wideband telephony (100 Hz- 8 kHz) in cars• Fullband• Narrow band (car)• Wideband (car)

o Efficient use of the high quality audio systems in Efficient use of the high quality audio systems in cars:cars:• Getting superior sound quality• Increasing speech intelligibility• Increasing naturalness of a conversation• Reduce drivers distraction due to poor speech sound quality


Bluetooth Link

Phoneadapter &antennaamplifier

Radio Link

Hands-free signal

processing

Caraudio

system

Car Hands-Free Testing


analysisanalysis

Mobiles Sound Playback System (can be installed in any

vehicle)

Subwoofer

HFTACQUA

BackgroundNoise

Generatorand Analyzer

MFE VIUSBFrontend

PEQ IVProgrammableEqualizer



Radio Tester

(Analog)

(Digital)AES / EBU

(Digital)AES / EBU

(Digital)USB

(Digital)AES / EBU

Subwoofer

HFT

Subwoofer

HFTACQUA

BackgroundNoise

Generatorand Analyzer

MFE VIUSBFrontend




Radio Tester

(Analog)

(Digital)AES / EBU

(Digital)AES / EBU

(Digital)USB

(Digital)AES / EBU

Test System

ITU-T Standards Contributing to Reduced Drivers Distraction


ITU-T standards for Hands-Free Communication:ITU-T standards for Hands-Free Communication:

• ITU-T P.1100 for narrowband hands-freeITU-T P.1100 for narrowband hands-free• ITU-T P.1110 for wideband hands-freeITU-T P.1110 for wideband hands-free

ITU-T standards work ITU-T standards work in FG CarCOMin FG CarCOM::• New work on subsystem requirements for Hands-Free New work on subsystem requirements for Hands-Free

Systems in CarsSystems in Cars

ITU-T standards for Dialog Systems:ITU-T standards for Dialog Systems:

• P.851: Subj. evaluation of dialog systemsP.851: Subj. evaluation of dialog systems• Suppl. 24 to P. Rec.: Parameters describing Suppl. 24 to P. Rec.: Parameters describing

the interaction with spoken dialog systemsthe interaction with spoken dialog systems

The Main Section in ITU-T P.1100 & P.1110


Definition of the test arrangement in a carDescription of a digital interface concept for development and debuggingMicrophone test specification for separate microphones in a carMeasurement parameters and requirements for hands-free terminalsBluetooth test interface for validation of telephone performanceSubjective test strategy for car to car communication

Conclusions and Recommendations

Mobile terminal testing is essential to provide sufficient or even superior QoE to the user – many operator have specific terminal standards for their suppliers A variety of parameters contribute to the overall (conversational) speech qualityNetwork conditions have to be considered in the testingLow driver distraction is essential to offer mobile services Speech based operation is the key to the success of mobile services in carsBoth, terminals and networks have to be thoroughly engineered and adapted to each other


28

Tandeming of Signal Processing in Terminals & Networks


BaseStation

EC

ES

NR

Mobile Phone

EC

ES

NR

gain gain

Mobile Network

Netw. 1

Netw. 2

Documents

Colombia, 23-24 September 2013 QoS assessment for mobile speech services including user interfaces in vehicles and hands-free communication Dr.-Ing. H