3G UMTS NSN Test Cases

1 © Nokia Siemens Networks Presentation / Author / DateFor internal use

Nelmon and Traffica for MGW use casesHow to troubleshoot and solve end customer voice quality problems

This material can be found from:

https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/358877511

https://share.inside.nokiasiemensnetworks.com/groups/traffica-for-mgw-nelmon?view=overview


Use case examples for Nelmon/Traffica for MGW• Nelmon/Traffica for MGW is new NSN capability for real-time monitoring of user

plane connections. • Traffic News tool provides functionality for correlating the control plane and user

plane information that enables operator the see subscriber and call level quality information. This slideset explains how the metrics available in Traffic News can be interpreted when solving customer problems

• This slideset shows 5 different call cases from real network where end customer has called operator customer care to complain problems in call quality.

• At the end of the material there are recommended actions for operator when troubleshooting user plane problems

• In these cases data collected by Nelmon provides invaluable information needed to either solver problem immediately or direct troubleshooting to right direction or element.

• Measurement data shown in the slides are as reported by Nelmon to Traffica, i.e those are real measured values

• Following cases are studied in following slides. – Case 1 Call was just muted during the call– Case 2 (Undefined) Noise on the line– Case 3 Unidirectional call, cannot hear at all the other party– Case 4 Unidirectional call, bad voice quality in one direction – Case 5 Dropped call


A-TDM

Double Talk 9Active Speech Level -18Speech Activity Factor Ing 32Speech Activity Factor Egr 25Saturation Clipping 0.00Noise Level -78Flat Noise Level -66

Nb-SIP-I

Frequency Offset 284Time Offset 10Total FER 64.66Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 1636Peak To Peak Jitter 0Inter Arrival Jitter 0Quantile Jitter 0Frequency Offset Reliable FalsePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable FalseUsed codec G.711 a-law TruePacket Loss Ratio 0.00

Case1: Call was just muted during call, B-party cannot hear A-party

1

2

B partyMGW1 MGW2

BSC3G

PSTNA party

MSS

Frequency Offset 284Time Offset 10Total FER 64.66Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 1636Peak To Peak Jitter 0Inter Arrival Jitter 0Quantile Jitter 0Frequency Offset Reliable FalsePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable FalseUsed codec G.711 a-law TruePacket Loss Ratio 0.00

Serv. A Call Hold FalseServ. A Retrieve False

Serv. B Call Hold TrueServ. B Retrieve True

Call length 33sec


Case1: Analysis- In this case, measurement data is unfortunately available only from MGW1 for further

analysis. However even if measurements from MGW 2 are not available, we are able to provide key indications for troubleshooting.

- Based on MGW measurements- From measurement report 1 we can see that Speech Activity Factor Ing = 32. This

means from A interface there is speech in uplink direction- From measurement report 2 we can see high Total FER (~65%) towards A-party, which

leads to assume A-party cannot hear B-party note difference to what end customer has reported

- Parameters Packet Loss Ratio and Link FER are 0 which indicate that Nb SIP towards A party and link from MGW2 to MGW1 are working ok. MGW1 has received all RTP packets which MGW 2 has sent..

- In PSTN NB-SIP direction there cannot be speech missing because PSTN is continuoisly sending packets.

- High Total FER could be explained by either problem in MGW2 (for which there is no report available) This could be caused by either MSS or MGW problem. However when we look at MSS measurement….

- Based on MSS measurements - We can see that call was ended normally but B-party has put call on hold during call

(Serv B Call Hold = True) and also released hold (Serv B Retrieve = True) before ending call. This may also explain measurement 2 high Total FER values (this value would be zero if call is ended during hold)

Conclusion: - End customer has accidentally pressed call to hold from terminal and thought that

there is something wrong with the speech path.


RNC BSC

Iu-ATM

Frequency Offset -23Time Offset -2Double Talk 17Active Speech Level egr -15Speech Activity Factor Ing 50Speech Activity Factor Egr 32Saturation Clipping 0.00Noise Level -47Flat Noise Level -49Total FER 0.28Link FER 0.04MOS NB 3.82Lost Frame 1Bad Frame 6Total Frame Count 2435Peak To Peak Jitter 6Inter Arrival Jitter 12Quantile Jitter 6Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueFlat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable TrueUsed codec UMTS AMR 2

A-TDM

Double Talk 16Active Speech Level egr -15Speech Activity Factor Ing 32Speech Activity Factor Egr 50Saturation Clipping 0.00Noise Level -53Flat Noise Level -52Flat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable True

Case 2: There is noise on the line

A-party B party

1

2

MGW1

Call length

1min 11sec


- Based on end customer problem description, it is not clear which party (or both) is hearing noise/cracling on the line

- For fault finding it is needed to have detailed fault description, including nature of impairment- Call length in this case is relatively long so call call quality has not been so bad that speech would

have been totally understandableBased on MGW measurements- Speech Activity Factor values indicate that speech is going through in both direction- Active Speech Level and Double Talk values are normal- Saturation Clipping is also zero (this parameter would indicate noise on the line)- Measuremet 1 FER values are also normal. High FER could be also interpreted by somebody as

noise/cracling sound. - From all measurements you can see that both access interface parameters (in uplink direction)

are within normal range, so you can conclude that access interfaces in MGW’s are ok.Based on MSS measurements- Call is ended normally and there is no indication of any problems in control plane

Conclusion- Based on MGW measurements there is nothing indicating major user plane problem in the call- Based on parameter values, core network is performing ok- Reason for degraded voice quality is most propably coming from either from Iu downlink or

terminal. - Further troubleshooting should be focused on Iu downlink direction and terminal related problems

(faulty microphone or other).

Case 2: Analysis


PBX

MGW2MGW3 RNC

PSTN

PSTN

Double Talk 12Active Speech Level -18Speech Activity Factor Ing 56Speech Activity Factor Egr 30Saturation Clipping 0.00Noise Level -60Flat Noise Level -58Flat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable False

PSTN

Double Talk 20Active Speech Level -14Speech Activity Factor Ing 43Speech Activity Factor Egr 67Saturation Clipping 0.00Noise Level -71Flat Noise Level -65Flat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable True

Nb-SIP-I

Frequency Offset -15Time Offset -1Total FER 0.08Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 2308Peak To Peak Jitter 18Inter Arrival Jitter 1Quantile Jitter 8Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable TrueLast Used Codec UMTS AMR 2

Frequency Offset 169Time Offset 10Total FER 0.00Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 1427Out Of Order Packets 0Invalid Packets 0Peak To Peak Jitter 20Inter Arrival Jitter 9Quantile Jitter 10Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable True

Iu-ATM

Frequency Offset -33Time Offset -2Double Talk 15Active Speech Level -18Speech Activity Factor Ing 70Speech Activity Factor Egr 32Saturation Clipping 0.00Noise Level -59Flat Noise Level -57Total FER 0.81Link FER 0.09Lost Frame 2Bad Frame 15Total Frame Count 2081Peak To Peak Jitter 7Inter Arrival Jitter 4Quantile Jitter 5Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueFlat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable FalseInter Arrival Jitter Reliable TrueQuantile Jitter Reliable True


MGW1

MSSMSS

Case 3: Unidirectional call

3G

PSTN

12

3

5

4

6

PSTN no speech

PSTN ok

Call length 52 sec


Case 3. Analysis

Based on end customer complaint there is no speech in direction from PSTN to 3G. However no complaint has no information if call was mute in that direction from beginning or was muted during callBased on MGW measurements- Speech is passing through core in both directions. We can compare the ingress side Speech

activity factor and egress speech activity factor in both directions, In this case values are almost same so we can say that core network does not have problem in this case.

- From measurement 5 we can see that Nb-SIP-I termination is ok , since Total FER=0 (no RTP packets missing between PSTN and MGW2) core network seems to work ok.

- Also PSTN side Ingress and egress values (measurement 1) match each other. Same amount of speech is sent as is receivved. PSTN side seems to be ok.

- As core network measurements do not indicate any problems, failure is most propably after MGW 6, either in RAN or in Ue (meaning Iu-DL/Iub/-DL/Uu-DL-transmission, RNC, BTS or in Ue). However from measurement 6 (MGW3) we can see that Iu-ATM uplink FER is small, therefore most propably also downlink radio interface quality is good (these correlate typically). Radio network quality seems to be ok.

Based on MSS measurements- Call is ended normally and there is no indication of any problems in control planeConclusion:- Based on MGW reports, speech is passing through core network- Frther troubleshooting should focus first on end user terminal (broken microphone or other

problem), possibly also radio network problems.


PSTN


Nb-SIP-I

Frequency Offset 129Time Offset 18Total FER 0.04Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 4861Peak To Peak Jitter 32Inter Arrival Jitter 16Quantile Jitter 17Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable TrueUsed codec UMTS AMR 2 True

Frequency Offset 65Time Offset 9Total FER 0.00Link FER 0.00Lost Frame 0Bad Frame 0Total Frame Count 5648Peak To Peak Jitter 24Inter Arrival Jitter 6Quantile Jitter 13Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable TrueUsed codec UMTS AMR 2 True

Iu-ATM

Frequency Offset -14Time Offset -2Double Talk 42Active Speech Level -12Speech Activity Factor Ing 60Speech Activity Factor Egr 77Saturation Clipping 0.00Noise Level -59Flat Noise Level -55Total FER 0.57Link FER 0.02Bad Frame 23Total Frame Count 4148Peak To Peak Jitter 7Inter Arrival Jitter 6Quantile Jitter 5Frequency Offset Reliable TruePeak To Peak Jitter Reliable TrueFlat Noise Level Reliable FalseNoise Level Reliable FalseActive Speech Level Reliable TrueInter Arrival Jitter Reliable TrueQuantile Jitter Reliable True

Case 4: Unidirectional call, bad voice quality in one direction

1

2

3

4

Call length 2 minutes

MGW1 MGW2

RNC

3G


Case 4: Analysis

Based on customer complaint there is bad voice quality from PSTN direction towards 3G terminal. It is not clear if bad quality was on all of the time or only part of the time.(Relatively long call indicates that quality was reasonable during the call and possibly only became poor at very end)Based on MGW measurements:- Relatively high Speech Activity Factor values throughout the path of the call (from PSTN to 3G)

indicate that noise is coming from PSTN and transported through core. - Also Double Talk value is increased indicating problems- Measurements 2 and 3 do not indicate FER in NB-SIP-I interface therefore we can conclude

that backbone is performing ok.- Measurement 4 indicate Total FER value is only small in Iu-ATM uplink, which very likely

means also that Iu-ATM downlink quality is ok.Based on MSS measurements- Call is ended normally and there is no indication of any problems in control plane. - From MSS report we can see that call is coming from PSTN direction but actually end

subscriber is other operator mobile subscriberConclusion:- Based on MGW parameters we can see that core network is not cause for degraded voice

quality, and noise is coming from PSTN direction- In this case call is coming from PSTN direction to MGW but is actually end subscriber is behind

other operators mobile network (we can see this from MSS report Aid value)- Further troubleshooting actions should be focused on PSTN direction, possibly leading to that

failure is coming from other operator network


RNC – BSC HO

PSTNIu-ATM

Report Time 17.08.2010 09:42:01.000Start Time 17.08.2010 09:41:20.970Frequency Offset -51Time Offset -2Double Talk 8Active Speech Level -16Speech Activity Factor Ing 19Speech Activity Factor Egr 60Saturation Clipping 0.00Noise Level -64Flat Noise Level -62Total FER 0.14Link FER 0.14Lost Frame 1Bad Frame 0Total Frame Count 702Peak To Peak Jitter 5Inter Arrival Jitter 5Quantile Jitter 4 PSTN

Case 5: Dropped call

MGW1

PSTNRNC

BSC

MSS


Case 5: Analysis

Based on customer complaint call was simply dropped after becoing completely silent.Initial assumption would be that there has been handover which has failed for some reason.

Based on MGW measurements- For this type of handover related problems failure can be typically seen from control plane

reports. User plane report is typically not giving much additional information for this type of problems as speech quality parameters are within normal range (like also in this case).

Based on MSS measurements- We can see that handover has failed (we can see error code for failed handover)

Conclusion- Handover from 3G to 2G has failed most propably due to problem in RNC. No failure is seen

in MGW side. - Typically troubleshooting failed handover problems would need to be done from control plane

using e.g. clear code: GSM Radio Link failure


Problem solving process

How to interpret measurements• Find out call route (based on CP and UP measurements)

• Normally first task is to look ALC measurements (Active speech level and Speech activity factor) from ingress interface and egress interface of the call and cross check values. If values match roughly, there is speech path from core perspective is working.

• For mute calls look for:– FER measurements on different interacaces (Link FER/Total FER)– ALC measurements on different interfaces (Active Speech Level/Speech Activity

Factor)▪ Invcestigate one direction at a time a b and b a, both speech activity and level ▪ If FER is high look for jitter measurement values (is packet loss caused by high jitter or are

packets just lost on the way)– Has call been put to hold?

▪ See MSS report for supplementary service report related to call

• For bad voice quality calls– ALC measurements on different interfaces (level, activity, saturation clipping values)

• Breaking calls/ weak signal – FER measurements on different interfaces (area NSN recommended values

exceeded) • Dropped calls

– Look reason for call drop from MSS report


Call center issuesWhat information need to be recorded from end customer complaint

It is essential that when recording initial end customer complaint, an accurate and detail fault description is made. This makes troubleshooting and fault fixing process within operator much faster.

Examples of questions customer care may need to ask from end customer:

• How often end customer receives calls with bad quality

• Is problem always same or similar/repeated

• Does end customer always get problem when calling to particular person

• Which party is experiencing the disturbance/problem

• How does disturbance sound: crackling sound, robotic sound, breaking line, “bubbling sound”, echo (all these may lead to different conclusion)

• Was disturbance present from beginning or introduced during call

• Was disturbance present only on top of call, during silence or whole time

• Is there certain location where disturbance is introduced?


Other notes

• Echo measurements from PSTN require relatively long time before they find correct value. If Echo delay measurement flag has value false, measurement is not to be used (measurement shows random value). Typically only if call is terminated to PSTN switch, measurement is reliable.

• Speech activity factor measurements are done only for the access interfaces and not for Nb interface, because measurement on Nb interface would have significant impact on MGW DSP capacity.

• Call ringing tone is included in speech activity calculation, which for short calls may represent quite large portion and therefore lead to wrong conclusion.

Documents

3G UMTS NSN Test Cases