Motorola QVoice R3 1 en Speech

ascom

80000840-QS31 5.4.2 en / © 1 released Ascom (Switzerland) Ltd., 20.5.09

QVoice™ Symphony QVoice™ Companion Symphony Multi

'Speech' Tests

User Manual For Release 5.4.2 and higher

«Speech» Tests: General 3 Chapter 1

Carrying out «Speech» Tests 5 Chapter 2

Measured Value Displays 19 Chapter 3

«Speech» Measurement Programs 23 Chapter 4

Call Handling 31 Chapter 5

Speech Quality Evaluation 43 Chapter 6

'SPEECH' TESTS

Version Overview Vers. Date Author Modification

5.0 26. 9. 06 Peter Beiner First edition for Release 5.0

5.1 18. 1. 07 Peter Beiner Adaptations for Release 5.1

5.2 31. 8. 07 Peter Beiner Adaptations for Release 5.2

5.3 4. 4. 08 Peter Beiner Viewers replaced, Call Handling

5.4 22. 10. 08 Peter Beiner Channel Capabilities complemented

5.4.2 20. 5. 09 Peter Beiner Symphony Multi added, adaptations

Document Management Document: 80000840-QS31 5.4.2 en

Created by: Peter Beiner

Word text file: QV_R3-1_en_Speech_542.doc

Number of pages: 52

Release: This document was released following formal review on: May 20, 2009 by: Johan De Bruyne

© Ascom (Switzerland) Ltd. Solothurn, Switzerland and Lake Forest, California All rights reserved

2 © 80000840-QS31 5.4.2 en /Ascom (Switzerland) Ltd., 20.5.09 released

'SPEECH' TESTS

Chapter 1 «Speech» Tests: General

This Chapter provides an overview of all the possible speech connections and tests.

Measurement Connections 4


'SPEECH' TESTS «SPEECH» TESTS: GENERAL

Measurement Connections

ISDN / PSTNCellularNetwork

MM

MS

M

SS

S

QVS–Speech /LU–Speech

QVoice Symphony /QVoice Companion /Symphony Multi

QVoice Symphony /QVoice Companion /Symphony Multi

SpeechAutomatic

Answer Station

M = Master S =Slave

Connection to an Answer Station

MOC only, no speech quality evaluation.

Connection to QVS-Speech / LU-Speech

• MOC / MTC1)

• MOS with Uplink / Downlink / Half-duplex

Mobile–to-Mobile Connection, same MU

• MOC / MTC1)


• 2 air interfaces per link!

Mobile–to-Mobile Connection, second MU / QVS (LU)

• MOC / MTC1)


• 2 air interfaces per link!

1) Configuration of the Slave via inband signalling, with «Direct inband» can also be used.


'SPEECH' TESTS

Chapter 2 Carrying out «Speech» Tests

This Chapter describes the settings required for speech measure-ments.

Configuring the Measurement 6

Destinations 7

Creating a Measurement Configuration 8

Speech Delay 9

Speech Sample Recording 9

Call Audio Recording 11

Trace AMR 11

Connect Recognition Mode 11

Call Progress 12

Use Direct Inband 12

Force Config Call on GSM 13

Voice Codec Forcing 13

C/I Rate 13

Synchronising Measurement Channels 14


'SPEECH' TESTS CARRYING OUT «SPEECH» TESTS


Configuring the Measurement All measurements are defined using the «Measurement Config Editor».

The procedure and the modules required are described under «Operation – Configuring Measurements».

The following modules contain speech-specific settings:

Destination

Destination determines the remote station.

Destinations are to be defined in accordance with your environ-ment. See next page.

Measurement program

The measurement program determines the measurement sequence.

A number of standard measurement programs are supplied with the system. You can adapt them to your requirements or create your own. See «Speech» Measurement Programs on page 23

Measurement Configuration

The Measurement Configuration is used to determine which measurement program is used to measure on what channel and for what Destination.

You can also select the «Channel Capabilities» here (Trace, Forcing, etc.). See Creating a Measurement Configuration on page 8


Destinations Open the Destination Editor.

CS Destinations (circuit-switched) are required for speech measurements. They can consist of:

• QVS / Land Unit / another mobile phone as active Slave (remote station)

• Answer station (e.g. time announcement): Call statistics only, no speech quality evaluation

Destination Editor: Speech Destination

Speech Number Phone number of the Destination.

Data Number For Speech: No input required.



Creating a Measurement Configuration Open the Measurement Config Editor:

Measurement Config Editor: Measurement Program and Destination

Select the channel (Master)

If the channel is not assigned a default Subscription under «HW Configuration – Channels», you need to select the appropriate Subscription here.

Select a Speech measurement program

Decision The selection determines…

• which service the channel is to measure

• whether the channel is Master or Slave

Select the Destination

Select a CS Destination, e.g.:

• a QVS/LU channel configured accordingly

• a channel configured accordingly, with mobile phone (Slave)

• an answer station



Select the Channel Capabilities

Select the Channel Capabilities to suit your requirements:

Measurement Config Editor: Channel Capabilities

General Channel Capabilities see «Operation – Configuring Measurements – The Measurement Config Editor».

Speech Delay 0 / 5…10 [s] This allows you to set a delay between «Connect» and the start of the speech sample transmission.

Speech Sample Recording Used to record speech samples received. The speech samples can be played back during the evaluation with QVP or exported as a wav-file and examined with special analysis tools. You can choice whether to record all speech samples or only those with specific characteristics.



The measurement files are considerably larger if Speech Sample Recording is activated.

Data volume

A 1 hour measurement can generate up to 20 MB of data per channel.

If you intend to transfer the data to QVP using «Wireless File Transfer», we recommend that you switch off Speech Sample Recording.

Switching off Speech Sample Recording Lower MOS Threshold 5 Record on Error False

Select criteria

«Speech Sample Recording» Criteria

Lower MOS threshold Upper MOS threshold Speech samples whose speech quality is within the defined range are recorded.

Max Count

0 All speech samples will be recorded.

>0 This number of speech samples will be recorded.

Record on Error True Speech samples will be recorded if certain errors occur. See description in the editor.



Call Audio Recording Used to record audio signals from connection setup to connection clear-down. The audio signals can be played back in QVP for analysis.

Direction Uplink Downlink Both Uplink + Downlink None No recording

Audio Format PCM, 16 bit linear, 8 kSamples/s MP3 PCM, 8 bit a-Law, 8 kSamples/s PCM, 8 bit µ-Law, 8 kSamples/s

Condition All Calls Bad Calls Failed Calls, Dropped Calls, Failed Con-

fig Call, MOS <2, Echo, Gap, Silence

Data volume The measurement files are considerably larger if Call Audio Recording is activated.

Trace AMR With certain mobile phones only.

True AMR values are written into the measurement file and can be displayed.

False AMR values are not measured.

Connect Recognition Mode Detects the audio connect by transmitting tones (needed for example with CDMA as no «Connect» appears).


Master + Slave must have the same setting! Master + Slave

Automatic Tones are generated and detected during measurement calls, depending on the Master Device and Slave Device.

Always On Tones are generated and detected during configu-ration and measurement calls.

Always Off Deactivated

Recommen-dation

Switch on the Connect Recognition Mode with all Speech tests, if you compare a CDMA network with other networks.


Call Progress Detects signalling tones and sequences and writes them into the measurement file as QVoiceInfo.

Busy On Time Busy Off Time Busy Tone Freq

Fast Busy On Time Fast Busy Off Time Fast Busy Tone Freq

Reorder On Time Reorder Off Time Reorder Tone Freq

Ring On Time Ring Off Time Ring Tone Freq

Error On Time Error Off Time Error Tone Freq

Dial Tone Freq

Use Direct Inband False Slave controlled by configuration calls.

True Slave controlled directly from MEC to MEC (SBC to SBC).


If Direct Inband is to be used… Direct inband

• Master and Slave must be connected to the same MU

• «Use Direct Inband» must be set to True on Master AND Slave

• the destination on the Master must be identical to the Slave subscription (incl. prefix!)



Force Config Call on GSM Allows configuration calls to be made via GSM if the inband signalling communications via a WCDMA network are proving faulty.

False Automatic selection of the mobile network for configu-ration calls.

True Configuration calls are made via GSM network.

Voice Codec Forcing With certain mobile phones only.

No Forcing Automatic selection of the codec mode. Force FR Force FR EFR Force FR AMR

C/I Rate With certain mobile phones only.

0 No C/I values.

2…255 Number of burst measurements to create the mean.

The number allows you to adapt the C/I measurement to the situation. You indirectly select the time for creating the mean value:

C/I Rate For Measuring… Bursts per Result

2 Spiky Interference1)

255 Continuous Interference2)

1) «Spiky Interference» caused by bursts, e.g. by TCH with «Frequency Hopping».

2) «Continuous Interference» caused by continuous sources, e.g. by each BCCH.


Synchronising Measurement Channels You can configure measurement channels with Speech or Video Telephony in such a way that the connection setup is always synchronised.

Same locations The synchronised channels make the calls at the same time.

With measurement trips by car: The calls are made at the same locations.

Schematic

� � � � � � � � � � � �

� � � � � � � � � � � �

� � � � � � � �

� � � � � � � �

� � � � � � � � � � � � � � �

� � � � � � � � � � � � � � �

� � � � � � � � ��

� � � � � � � � ��

��

��

��

� � � � � � � � �

Call Synchronisation: The Call Windows are synchronised

The Call Windows of the measuring channels are synchronised and thus the dial commands as well. The simultaneity of the calls depends on the time needed to establish a connection (setup time).



Requirements

Measurement program / Test Operation The following settings must be in place:

Test Operation = Speech / VT Delay between Series = Call Window Pause Duration = Call Window

Measurement Configurations of the channels that are to be synchronised:

They must all contain the same measurement program.

Procedure

Measurement program

Measurement Program for Channel Synchronisation

80000840-QS31 5.4.2 en / released Ascom (Switzerland) Ltd., 20.5.09 © 15


Measurement Configurations with this measurement pro-gram

Measurement Configurations for Channel Synchronisation

16 © Ascom (Switzerland) Ltd., 20.5.09 released 80000840-QS31 5.4.2 en /


Create a «Channel Sync Configuration»

The command opens the «Channel Sync Config Editor»:

«Channel Sync Configuration» with 2 of 3 possible Measurement Configurations

1. Open a new «Channel Sync Configuration».

The Measurement Configurations that satisfy the requirements are displayed.

2. Select the Measurement Configurations you want.

Init Duration Max. waiting time for the measurement channels to be ready to start.

3. Save the inputs.



Start the «Channel Sync Configuration»

On the main toolbar, click the «Start Channels [Synchronized]» button:


'SPEECH' TESTS

Chapter 3 Measured Value Displays

This Chapter shows typical screenshots of measurements.

«Viewers» 20

Map Display 22


'SPEECH' TESTS MEASURED VALUE DISPLAYS

«Viewers» Creating Viewers: See «Operation – Displaying Measurement Data».

The Default Workspace contains predefined Viewers. The figure shows the «Speech / VT» Viewer:

«Speech / VT» Viewer: WCDMA Speech Test with AMR



«Speech / VT» Viewer: Speech Test with «Street Noise»



Map Display

«Map» Viewer: • Map • Route travelled, with Speech Quality (Colour-coded) • Call Markers


'SPEECH' TESTS

Chapter 4 «Speech» Measurement Programs

This Chapter describes the measurement program parameters and explains how they correlate.

Measurement Program Parameters 24

One call per Call Window 29

Number of speech samples per call 30


'SPEECH' TESTS «SPEECH» MEASUREMENT PROGRAMS

Measurement Program Parameters Open the Measurement Program Editor.

Measurement Program Editor: Speech Parameters

1st call setup delay 1…65 535 s Delay prior to the first call with every Call Window.

Call duration 10…65 535 s Normal call duration. Can be shortened in the measurement procedure if the connection was only established after x attempts and the «Guard Time» forced a premature clear-down.

With Half-duplex: min. 20 s. Speech quality is not evaluated with smaller values (see also Estimation of call duration on page 30).

Half-duplex + evaluation



Call window 60…65 535 s Time window that provides a constant rhythm (compulsory for MIX mode). The Call Window applies to the Master AND the Slave. Call Windows follow one another without gaps. Within a Call Window all MOC or all MTC are set up.

� ��

� ��

� ��

� ��

� ��

� ��

� � � � � � � � ! � � � � � � " � � � � � � � � # �

� � � � � � � � � � �

� � � � � � � � � � � $

� � ��

� � ��

� � ��

� � ��

� � ��

� � ��

� � � � � � � � � ! � � � � # � � " � � � � � � � � �

� � � � � � � � � � $

� � � � � � � � � � � �

� ��

� � ��

� ��

� � ��

� ��

� � ��

� % & � � � � � � � ' � � ! � � � ( � � � � � � � � � � � )

� � � � � � � � � � �

� � � � � � � � � � � �

� ��

� ��

� ��

� � ��

� � ��

� ��

� % & � � � � � � � ' �

� � � � � � � � � � �

� � � � � � � � � � � �

MOC, MTC and MIX Mode

The total number of Call Windows is determined by the duration of the measurement.

MIX mode: The numbers determine the number of successive MOC or MTC Windows. The cycle begins always with MOC.

The Call Window must be defined at least so long as: «1st call setup delay» + «Call duration» + «Min time left» + «Guard time»

Call Window duration

See figure on next page.

Connect fail timer 10…255 s The timer starts with the dialling. If there is no «Connect» from the network within the defined time, the call is classified as a «Failed Call». The duration of the call set-up depends on the net-work’s characteristics and its level of capacity utilisation. Typical values are around 8 s.

4.2 en / © 25 released Ascom (Switzerland) Ltd., 20.5.09 80000840-QS31 5.


Delay between call attempts 1…65 535 s

Delay between series 5…65 535 s Delay after one series of failed call attempts, which is then followed by a new series. Subscriber behaviour can be well simulated in this way. The process is repeated until a connection is established or the «Minimum time left» is reached. Only two series of call attempts are shown on the chart so that clear legibility is maintained.

Guard time 10…65 535 s Guard time for the connection clear-down at the end of the Call Window. If a call runs into this time, it will be terminated.

� � � ��

� * � � � � � � � � + � � � � � � � � � � � � � � " " � � � + � �

� � �� +

, � � ��

� � � � � - � � � � �

� � � � � � � � � � � �� .

� � � ��

� � � � �� ) � � � � � �

The Third Call will be Abridged

Language of speech sample Language of the speech samples that are transmitted for measuring the quality of speech e.g. English.




Make calls in no-service area

True Call attempts are also made in areas without coverage, following strictly the measurement program. This setting is useful for obtaining comparative measurements of cellular networks: In borderline areas for example it may be possible to use one particular network – albeit with poor quality – and not at all using another. Since the measure-ment program goes ahead for both networks, the com-parative values are not corrupted.

False The call only starts if the mobile phone is logged on.

Max nbr of call attempts in 1 series 1…255 Max. number of failed call attempts in one series.

Min time left 30…65 535 s Minimum time remaining. A call cannot be set up within this time range. This allows you to specify the shortest call duration. If under «Minimum time left» you select the same value as for «Call duration», only calls with a normal duration will occur.

However it is not possible to expect a specific number of calls to be established during a Call Window. The reason is that the duration of the call set-up varies.

MOS algorithm Algorithm for speech quality evaluation.

MOS direction

Downlink The Slave channel sends speech samples / the Master evaluates.

Uplink The Master channel sends speech samples / the Slave evaluates.

Half-duplex Both transmission directions are alternately evaluated. Each call begins with Downlink (Slave channel sends).


Nbr of MOC windows 0…255 Number of successive Call Windows for the Master channel.

Nbr of MTC windows 0…255 Number of successive Call Windows for the Slave channel (the Slave calls the Master).

No connect shutdown time 60…65 535 s The measurement is terminated for this channel if this time is exceeded without any successful calls.

Make sure the «No Connect Shutdown Time» you select is significantly greater than the «Pause duration»; otherwise the measurement will be terminated after one call.

Noise DL Noise UL dB value = Signal/Noise Noise can be transmitted on channels with mobile phones (optional); with mobile-to-mobile connections, in the Uplink and Downlink.

There is a selection of noise types and levels. The noise is added to the speech sequences and also sent during pauses.

Pause duration 5…65 535 s Pause duration if several connections are set up and cleared down within a Call Window.

Use DSP

True For calls with speech quality evaluation

False For calls to a service number




One call per Call Window Select the settings as follows if you only want one call per Call Window:

• «Delay between series» same as for «Call window». This creates only one series of call attempts.

• «Pause duration» same as for «Call window». This creates only one call.

Call rhythm min. 60 s results from the duration of the Call Window.

Interval between two connections min. 12 s


Number of speech samples per call Normal speech samples last 5 s. The number of speech samples and the call duration cannot be matched to one another as the duration of the various signalling that occurs at the beginning of the call can vary.

QVoice handles started speech samples as follows:

During transmission The call duration is observed; the speech sample is terminated prematurely.

During analysis with QVP The abridged speech sample is ignored.

� � � � � � � � � � �

/ � � � " 0 � � � � � " � � � � � � � � �% � " � � � � � �

� � � � " � � � � � � � �

1 # � � � � � � �

� # � � � � � �

� � � � � � � � �

� � � � � � " � � � � � )

Abridged Speech Sample at the End of a Call

Estimation of call duration To achieve a specific number of speech samples per call, you can estimate the necessary call duration as follows:

Call duration Transmission Direction Estimated Call Duration

Uplink / Downlink (n × 5 s) + 4 s

Half-duplex (n × 5,25 s) + 9 s

n = Number of speech samples


'SPEECH' TESTS

Chapter 5 Call Handling

This Chapter describes how QVoice interprets and records events during connection set-up and clear-down.

Call Duration 32

Normal Connection Procedure 33

Faulty Connection Procedure 34

Call Error Marker 34

Connect Fail Marker 35

Break Marker 36

Release Fail Marker 37

Call Markers and their Causes 38

Call Markers with the mobile network 38

Call Markers with ISDN 41


'SPEECH' TESTS CALL HANDLING

Call Duration Call duration = effective dura-tion of the call

The call duration starts with «Connect» and ends with «Dis-connect».

Input range 10…65 535 s. 65 535 s = approx. 18 hours.

The input range is used for both realistic connections lasting 2…3 minutes and long-term connections lasting several hours.

User Break Any call that is disconnected because the user wants to stop the measuring process is given the User Break call status. Connections with User Break do not appear in the statistics during evaluation with QVP.



Normal Connection Procedure QVoice interprets messages and events of the connection pro-cedure and stores the corresponding information in the measuring file as «Call Markers». These Call Markers are used to reconstruct the connection procedure during the subsequent evaluation with QVP. For example Call Markers provide the basis for compiling call statistics with «QVP Report».

Call Markers are recorded wherever a connection is set up or disconnected: in MOC mode the Call Markers are recorded by the Master; in MTC mode, by the Slave. In the fixed network the time interval between Call Markers differs in part from the time interval on the mobile network. Certain marker types can only occur in the mobile network.

The figure below shows the Call Markers of a normal connection that are set automatically:

GSM / WCDMA / CDMA

Dia

l

Set

up

Call

Call Setup

Con

nect

Rel

ease

Dis

conn

ect

Call Duration Release

ISDN / PSTN

Dia

l+

Setu

p

CallCall

Setup

Con

nect

+ R

elea

seD

isco

nnec

t

Call Duration

Call Markers for a Normal Connection

QVoice hands over the dialling to the terminal and the connection set-up is successful. Once the call duration is reached, QVoice gives the clear command to the terminal and the connection clear-down is successful.

Call status «Completed Call»

33 80000840-QS31 5.4.2 en / released Ascom (Switzerland) Ltd., 20.5.09 ©


Faulty Connection Procedure QVoice monitors the connection set-up, the observance of the call duration and the connection clear-down. If QVoice detects any errors in the procedure, it sets additional Call Markers.

Shown below are a few typical instances of faulty connection procedures, with the corresponding marker combinations.

Call Error Marker

GSM / WCDMA / CDMA

Dia

l

«Connect Fail Timer»

Cal

l Erro

r

GSM: No Layer 3 «CC:Setup» within a defined time OR a Layer 3 «CC:Channel Release»

CDMA: No Layer 3 «Service Connect» within a defined time

OR

Status «No Network»

Call Error Marker

QVoice hands over the dialling to the mobile phone, but receives no connection set-up message within 10…255 s (time selectable using the «Connect Fail Timer»).

Call status «Failed Call»



Connect Fail Marker

GSM / WCDMA / CDMA

Dia

l

«Connect Fail Timer» (Restarted)

Con

nect

Fai

l

Set

up

GSM: No Layer 3 «CC:Connect Acknow-ledge» within a defined time after sending a Layer 3 «CC:Setup» OR a Layer 3 «CC:Release» OR a Layer 3 «CC:Channel Release»

CDMA: No audio detected within a defined time after dialling

OR


ISDN / PSTN

«Connect Fail Timer»

No ISDN Layer 3 «Connect Acknowledge» within a defined time OR

an ISDN Layer 3 «Disconnect»

Con

nect

Fai

l

Dia

l+

Set

up

Connect Fail Marker

QVoice hands over the dialling to the terminal, but receives no Connect message within 10…255 s (time selectable using the «Connect Fail Timer»).

Call status «Failed Call»



Break Marker

GSM / WCDMA / CDMA

< Call Duration

Call

Dia

l

Set

up

Con

nect

Bre

ak

GSM: Phone goes to «Idle Mode» (Layer 3 «RR:System Information 3») OR Layer 3 «Idle Report» OR Layer 3 «CC:DL Disconnect»

CDMA: Phone state «Idle» OR Layer 3 «DL (Fwd) Release Order» OR Phone state «Disconnected»

OR


ISDN / PSTN

ISDN Layer 3 «UL Disconnect» before call duration is achieved

< Call Duration

Call

Con

nect

Bre

ak

Dia

l+

Set

up

Break Marker

The connection set-up is successful. However, the connection is disconnected before the call duration is reached.

Call status «Dropped Call»



Release Fail Marker

GSM / WCDMA / CDMA

Dia

l

Set

up

Call

Con

nect

Dis

conn

ect

Max. 5 s

Rel

ease

Fai

l

Release Fail Marker

GSM No Layer 3 «Release» within 5 s after sending a Layer 3 «Disconnect» OR a Layer 3 «CC:Channel Release» OR a Layer 3 «Idle Report» OR a Layer 3 «RR:System Information 3»

CDMA: No Layer 3 «DL (Fwd) Release Order» within 5 s after sending a Layer 3 «UL (Rev) Release Order» OR Phone state «Idle» OR Phone state «Disconnected»

OR


The connection set-up is successful. Once the call duration is reached, QVoice gives the Clear command to the mobile phone. The mobile phone sends a Layer 3 Disconnect command, but does not receive a Layer 3 Release message from the network within 5 s.

Call status «Release Failed Call»




Call Markers and their Causes

Call Markers with the mobile network

«Dial» will be written…

if QVoice sends a dial command to the mobile phone.

«Setup» will be written…

GSM: if QVoice receives a Layer 3 «CC:Setup» message from the mobile phone

CDMA: if QVoice receives a Layer 3 «Service Connect» message from the mobile phone

«Call Error» will be written…

GSM: if QVoice receives no Layer 3 «CC:Setup» message within 10…255 s after the Dial Marker (time selectable using the «Connect Fail Timer») OR if QVoice receives a Layer 3 «CC:Channel Release»

CDMA: if QVoice receives no Layer 3 «Service Connect» message within 10…255 s after the Dial Marker (time selectable using the «Connect Fail Timer»)

OR if QVoice is in the «No Network» status.

«Connect» will be written…

GSM: if QVoice receives a Layer 3 «CC:Connect Acknowledge» message

CDMA: if QVoice receives a detection of audio



«Connect Fail» will be written…

GSM: if QVoice receives no Layer 3 «CC:Connect Acknow-ledge» message within 10…255 s after the Setup Marker (time selectable using the «Connect Fail Timer») OR if QVoice receives a Layer 3 «CC:Release» message OR if QVoice receives a Layer 3 «CC:Channel Release» message

CDMA: if QVoice does not detect audio within 10…255 s after the Dial Marker (time selectable using the «Connect Fail Timer»)


«Break» will be written…

GSM: if the mobile phone goes to the Idle Mode (Layer 3 «RR:System Information 3») OR if QVoice receives Layer 3 «Idle Reports» OR if QVoice receives a Layer 3 «CC:DL Disconnect» message

CDMA: if the mobile phone goes to Idle Mode (Phone state «Idle») OR if QVoice receives a Layer 3 «DL (Fwd) Release Order» OR if QVoice receives a phone state «Disconnected»


«Disconnect» will be written…

GSM: if QVoice receives a Layer 3 «CC:UL Disconnect» message

CDMA: if QVoice receives a phone state «UL Disconnect»



«Release» will be written…

GSM: if QVoice receives a Layer 3 «CC:Release Complete» message

CDMA: if QVoice receives a Layer 3 «UL (Rev) Release Order»

«Release Fail» will be written…

GSM: if QVoice receives one of the following messages before a Layer 3 «CC:Release Complete» message: • Layer 3 «CC:Channel Release» message • Layer 3 «Idle Report» • Layer 3 «RR:System Information 3»

CDMA: if QVoice receives one of the following messages before a Layer 3 «DL (Fwd) Release Order»: • Phone state «Idle» • Phone state «Disconnected»

OR if QVoice is in the «No Network» status

OR if 5 s timeout after Disconnect Marker has elapsed.

«User Break» will be written, if the user stops the measure-ment…

during the Dedicated Mode

OR during connection set-up

OR during connection clear-down.

«System Release» will be written…

if the call was terminated by QVoice, for instance because the battery is discharged or the operating temperature has been exceeded.



Call Markers with ISDN

«Dial» will be written…

if the QVS / LU sends a dial command to the ISDN board.

«Setup» will be written…

together with the Dial Marker (no meaning).

«Connect» will be written…

if the QVS / LU receives an ISDN Layer 3 «Connect Acknowledge» message.

«Connect Fail» will be written…

if the QVS / LU receives no ISDN Layer 3 «Connect Acknowledge» message within 10…255 s after the Dial Marker (time selectable using the «Connect Fail Timer»)

OR if the QVS / LU receives an ISDN Layer 3 «Disconnect» message.

«Break» will be written…

if the QVS / LU receives an ISDN Layer 3 «UL Disconnect» message before the call duration is reached.

Checks during import in QVP: During data import the status on the mobile network side is also included. If the mobile phone was already in idle mode, the «Break» is put forward chronologically. This can turn an MTC Success into an MTC Break.

«Disconnect» will be written…

if the QVS / LU receives an ISDN Layer 3 «DL Disconnect» message (= Local Hang-up)

OR if the QVS / LU receives an ISDN Layer 3 «UL Disconnect» message (= Remote Hang-up) and the call duration is reached.



«Release» will be written…

together with the Disconnect Marker (no meaning).

«User Break» will be written, if the user stops the measure-ment…

during call establishing

OR during the conversation phase.

«System Release» will be written…

if the call was terminated by QVS / LU.

'SPEECH' TESTS

Chapter 6 Speech Quality Evaluation

This Chapter provides an overview of the speech quality evalu-ation with QVoice.

Overview of Evaluation Methods 44

MOS scale 44

Mode of Operation with QVoice 46

Speech samples 46

PESQ 47

P.862.1 (Mapped PESQ) 48

Ascom Class 49

Special Audible Effects, Noise 50

Echo 50

Gain 51

Silence 51

Gap 51

SNRI 52

TNLR 52


'SPEECH' TESTS SPEECH QUALITY EVALUATION

Overview of Evaluation Methods

� 1 � 2 � � � ) � � � �3 � � 4 � " � # � 5

� �

/

�

� � � � � � )

� � � � � � � ) � � � � 3 � � � 4 � " � # � 5

, � � � � � � � � � � � �

� � � � � � � � � � 6 � � � � �

7 0 /

! � $ 0 /

� � � � � � )

7 0 /

�

� 1 � 2 � � 0 8 9 � � 1 � 2 � � 0 8 9 � 0 �

� � " � � � � � � � �

�

/

Evaluation of Speech Quality: Methods and Rating Scales

MOS Mean Opinion Score = average evaluation of speech quality.

MOS values are obtained when a sizeable group of people are asked to evaluate speech quality in a listening test. MOS is therefore a subjective evaluation method. The method and the rating scale are specified in the following documents:

ITU-T Rec. P.830 ITU-T Rec. P.800

MOS scale 1 bad 2 poor 3 fair 4 good 51) excellent

1) A 5.0 rating cannot be achieved in telecommunications as coding already results in a certain reduction in quality. The maximum rating achievable with PESQ P.862.1 is 4.5



Objective methods such as PESQ use the appropriate algorithms to determine speech quality.

P.862 Also referred to as «Raw PESQ».

The PESQ algorithm determines speech quality on a separate scale (see ITU-T Rec. P.862). The results correlate very well with MOS.

Due to differing scale ranges numerical values are not directly comparable with the MOS scale.

Not comparable

P.862.1 Also referred to as «Mapped PESQ».

Numerical values are comparable with MOS. For P.862.1 the P.862 values are mapped on the MOS rating scale. The allocation is unambiguous and reversible (see also the function on page 48).

Ascom Class The «Ascom Class» of speech quality uses its own rating scale. The Ascom Class is calculated from the recorded PESQ values during evaluation with QVP (see figure on page 49).



Mode of Operation with QVoice

DSP required For a measurement of speech quality, both the Master and the Slave channels have to be equipped with DSPs.

On the send side a Digital Signal Processor transmits voice se-quences of a specific duration (e.g. 5 s). Longer calls are simulated by repeating the sequence.

On the receive side a Digital Signal Processor evaluates the inco-ming voice sequences in accordance with the PESQ algorithm.

QVoice measures the P.862 values as floating point values. The floating point values can be evaluated with QVP. The measure-ment range of PESQ also provides a good evaluation of EFR Codecs.

The speech quality of a connection is evaluated by QVoice in the same way as it is experienced by the subscriber. The algorithms used guarantee an absolutely realistic speech quality evaluation due to the psychoacoustic modelling. As a result QVoice is ca-pable of evaluating the speech quality of a connection objectively and with reproducible results.

Real evaluation of speech quality

Speech samples The speech samples are supplied as standard in:

• Cantonese

• Deutsch (GermanNew)

• English (EnglishNew)

• French

• Mandarin

The speech samples are composed of both male and female spoken segments. The spoken text has been very carefully chosen for each language so that it is phonetically balanced, i.e. it con-tains the mostly used phonemes and diphones of the language.



PESQ

References ITU-T Rec. P.862 «Perceptual Evaluation of Speech Quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs»

Principle Speech output and speech quality evaluation are carried out in real time on the DSP board. In a first stage the P.862 value is determined. Illustrated below are the main functional blocks:

% � � � : � � �

� � # � �� ) �

� � � � � .� � � � + � �

% � � � : � � �

� � # � �� ) �

� � � � � � � " �� " � � � � � )

� � � � � .� � � � + � �

� � � �� ) �

; � � � " �� ) �

� � �% � � # � � �

� � � � " �2 � � � � .

< � + � � � " �� ) � � �

� � ) � � � �� ) � � �

< � = � � � ) � � � � � � % � � # � � �� 1 � 2

Principle of the PESQ Algorithm

80000840-QS31 5.4.2 en / released Ascom (Switzerland) Ltd., 20.5.09 © 47


P.862.1 (Mapped PESQ)

References

ITU-T Rec. P.862.1 «Mapping function for transforming P.862 raw result scores to MOS-LQO» LQO = Listening Quality, Objective

Principle

In the second stage QVoice uses the «P.862» result to calculate the «P.862.1» value so that it too can be displayed. Conversion function:

PESQ P.862.1 = 0.999 +4.999 – 0.999

1 + e–1.4945*(PESQ P.862)+4.6607

2

2.5

3

3.5

4

4.5

PESQ

P.8

62.1

1

1.5

-0.5 0 0.5 1 1.5 2 2.5PESQ P.862

3 3.5 4 4.5

Records P.862 In measurement files P.862 values only are recorded. P.862.1 can be calculated from these values at any time.

With QVP too, P.862.1 is calculated from the recorded P.862 values.

48 80000840-QS31 5.4.2 en / © Ascom (Switzerland) Ltd., 20.5.09 released


Ascom Class

Classification Ascom Class 1 excellent

2 good 3 fair 4 poor 5 bad

The Ascom Class speech quality is calculated from the recorded PESQ values during evaluation with QVP. A classification is carried out according to this chart:

P.862.1 to Ascom Class

��)�2

��.

�1�2��089�0�

��"��

� * " � � � � �

� � �

/

7

�

�

�

� 0 /, � � � : � � � � < � �

, � � � > � � + � < � �

7 0 /

�

7

�

�

7 0 �

� 0 8

� 0 /

� 0 �

, � � � 1 : <

% � �

< � + � � � " � � � � ) � � �

� 0 ?

� 0 �

� 0 /

�

�

�

�

Measurement Ranges of P.862.1 and Ascom Class

Values depend on the network

The typical values (GSM EFR: ~3.8; FR: ~3.5; HR: ~3.2) depend on the supplementary components in the network (Audio Enhancer) such as Noise Suppression, Echo Cancellation, AGC.

80000840-QS31 5. © 49 4.2 en / released Ascom (Switzerland) Ltd., 20.5.09



Special Audible Effects, Noise The powerful Audio options can be used to detect the causes of poor speech quality.

QVoice can measure the following effects:

• Echo

• Gain Also referred to as Signal Level or Loudness

• Silence

• Gap

• SNRI Signal to Noise Ratio Improvements

• TNLR Total Noise Level Reduction

The effects are measured by the DSP. The results are handed over from the DSP to the main computer where they are stored in the measurement files.

The individual effects are briefly described below.

Echo The talker hears his own voice as an echo. This is perceived as a disruption.

Echo can occur: – during 4-wire/2-wire conversion (e.g. in the PSTN) – if the echo cancellation in the ISDN is defective

QVoice measures the talker echo only on the mobile phone side… with Uplink on the Master with Downlink on the Slave

Measurement ranges:

• Level of the echo signal: ≥ – 45 dB in relation to the transmitted signal.

• Delay with regard to the transmitted signal: 100…750 ms

The echo delay is over 180 ms since the minimum transmission time for a GSM link is 90 ms. Values below 180 ms indicate a local echo in the terminal.



Gain Insufficient loudness impairs intelligibility and requires greater attention on the part of the listener; excessive loudness is per-ceived as a disruption. The optimum loudness level is somewhere in between. The loudness at the receiver’s end is greatly in-fluenced by the change in volume on the transmission path.

Measurement range: –28…+28 dB in relation to the transmitted signal.

Silence Despite good radio conditions, connections can occur in which the voice channel is interrupted (e.g. by transcoder errors).

The interruptions are detected by monitoring the voice level.

Level during a speech sample < –28 dB

Gap In certain cases (Handover, high FER) gaps can occur in the voice signal.

These gaps are determined by monitoring the voice level and the duration of the interruption.

Level ≤ – 30 dB in relation to the transmitted signal & duration ≥ 8 ms, resolution: multiple of 8 ms



SNRI bad good Signal to Noise Ratio Improvements –6…+12 dB in relation to the transmitted signal.

The measurement complies with Recommendation ITU G.160 (Testing of Voice Enhancement Devices).

Requirement «Noisy Samples» licence enabled.

To measure SNRI and TNLR, speech sequences with noise must be selected on the send side: In the measurement program under «Noise …» select the type (Street Noise / Vehicle Noise / White Noise) and the required level (10 dB / 15 dB).

The receive side evaluates the incoming signal. The MOS value is usually somewhat poorer than without noise (which is more in keeping with reality).

TNLR bad good Total Noise Level Reduction +10…–20 dB in relation to the transmitted signal.

Same requirement and measurement conditions as for SNRI (above).

Documents

Motorola QVoice R3 1 en Speech