7. GB_TS011_E01_1 ZXG10 iBSC Troubleshooting 60.ppt

ZXG10 iBSC Troubleshooting

ZTE University

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Collect fault information

Fault analysis

Fault locating

Troubleshooting

1 2 3 4

Troubleshooting procedure

Common methods to locate faults

Views of alarm, notification and operation log Analysis of status indicator Performance analysis Analysis of signaling tracing Analysis with instruments and meters Plugging/unplugging and pressing Isolation method Self-test method

Content


Platform faults

Database faults

Configuration and application faults

Alarm application faults

Performance application faults

1

2

3

4

5

NMS fault category

Troubleshooting guide of NMS faults

Client can not connect server NetNumen M31 server startup failure Database startup faults Failure of observing status attribute in dynamic

data management Failure of alarm box displaying audio/visual alarms Abnormal performance data report

Network between server and client is disconnectted

The client version differs from the server version

Language setting in deploy-gsmomcr01.properties and deploy-010muep.properties differs from that on the server

Disk is full

…

Use ping command to check whether the connection is OK

Use ipconfig and netstat commands to check whether server IP address configuration is proper

Check whether client version is consistent with server version

Check whether language setting is consistent with that in server

Use df –kh to check disk space

…Fault cause Troubleshooting

Client can not connect server

Server ini file doesn‘t have execution permission

Java virtual machine is not started successfully

Server profile is improperOracle database can not

start properly

…

Use ls –l to check whether the ini file has executable permission

Check whether compressed file under idk-solaris folder have been decompressed

Check whether the server configuration file is correct

Check whether it is successfully connected with Oracle database

Check whether database server name, IP address and hostname configured in tnsnames.ora are consistent with actual value

Use pkill -9 java to kill java process and restart the server


NetNumen M31 server startup failure

Database is already runningDtabase parameters are not

set correctly, which cause insufficient resources needed for startup

The system data file needed for startup is corrupt

Use data file is corrupt when the database is opened

…

Execute sqlplus sys/oracle@gomcr as sysdba to check whether Oracle can be connected

If the database does not properly start, check the returned result

Check whether the disk is full by using command df –k

If the database starts properly but cannot be connected, check whether the database monitor has started


Database startup faults

Communication between foreground and background is improper

Foreground and background data inconsistent

…

Check whether the communication link has been properly established between foreground and background

Do synchronization for whole table to keep consistence for foreground and background data


Failure of observing status attribute in dynamic data management

Probable reason is improper connection state in alarm box

…

Check the alarm box setting in NetNumen M31 alarm management, to check whether IP address/TCP port number of alarm box is consistent with actual value

Chek whether ping can get response from the network about alarm box, preventing from network cable or IP address conflict

Check whether the port number of alarm box has conflict with the port occupied by other processes

Check whether there are the setting for this alarm box on NMS server and communication state is OK


Failure of alarm box displaying audio/visual alarms

Link is disconnected between foreground and background

Performance file reported from foreground is improper or performace data has not stored yet

Nsf detect process is failed to connect foreground

…

Check physical link to ensure the foreground-background link is normal

After logging in SBCX board, switch to corresponding measurement types directory to view whether original performance data zip file for granularity is missing

check whether the data file for performance measurement is missing

Check nsf service state after logining OMP

Execute ps –ef | grep nsfdetect check whether nsfdetect process exists

View and print nsfdetect log


Abnormal performance data report

Alarm cases on NMS faults

After restarting iBSC OMP, background can not establish link with OMP

Software version cannot be synchronized to NE and system prompts FTP download failed

"License file invalid or missing" appears while configuring transceiver

Wrong time zone configuration causes performance report issue

NetNumen M31 upgrade fails

Fault analysis

The reason is IP address of O&M board is not consistent with that of OMP at background while configuring data.

Change OMP IP address at background. Restart OMP after synchronizing data, and the fault is solved.

Troubleshooting

After restarting iBSC OMP,background can not establish link with OMP

Fault analysis

The reason is ftp server IP in deploy-gsmomcr01.properties is inconsistent with that of IMP communication.

Modify ftpserver IP in deploy-gsmomcr01.properties and enable it consistent with IP address for IMP communication.

Troubleshooting

Software version cannot be synchronized to NE and system prompts FTP download failed

Fault analysis License file is invalid or missing.

Open Configuration Management view, License management appears in menu bar.

Click View License in License Management, and go to Update License interface.

Select the path to save license, find license.dat. license.dat file is located under ums-svr\deploy\runtime\,and click the Update button.

Troubleshooting

"License file invalid or missing" appears while configuring transceiver

Fault analysis

Time zone on client is not consistent with that on NetNumen(TM) M31 server. The system modifies the reported data according to time zone setting, so the time in performance report is not consistent with current time and performance data is displayed in two columns.

Querying whether performance data is proper after modifying time zone.

Troubleshooting

Wrong time zone configuration causes performance report issue

Fault analysis

After upgrade fails, check the server and input echo $OMCHOME command, null string is returned, not the value of $OMCHOME. From this, it is determined that the installation fault is caused by wrong $OMCHOME setting.

solaris has strict requirements on environment variable. The user shall not only modify /etc/profile file, also shall logout after performing /etc/profile command, and then use root user to login again.

After adding $OMCHOME value into /etc/profile file and making it effect, perform graphical installation process.

Troubleshooting

NetNumen M31 upgrade fails

Content

Overview NMS fault troubleshooting Data and software loading fault

troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Fault from data configuration

Fault caused by settings of software version

Category of data and software loading faults

Troubleshooting guide about data and software loading faults Mismatched board type and version configuration Version file does not exist Version file check error IPA resource unavailable NetNumen M31 data synchronization faults

Mismatch error between board version information configuration and board type causes impossible to download version

Board type configuration error

Access type error supported by the board

…

Check the version configuration and physical configuration of corresponding board, and correct corresponding errors


Mismatched board type and version configuration

Such a fault is usually caused by MP changeover or MP replacement

…

Check the data configuration to confirm whether the board configuration is correct

Wait for a period of time (more than 15 minutes) and then check whether the board is running properly

In software management on NMS, perform NE synchronization action for software, and upload the version file to OMP


Version file does not exist

The check error on the reception end often results from the version file error occurred during the transmission

…

Download the version again from the server

Contact the local ZTE office for the version file


Version file check error

Data configuration of user plane IP is different from that for MSC and BSC

AIPB board goes faultyDSP port configuration is

incorrect on AIPBIPA resource is blocked

manually

…

Check whether data configuration is consistent for MSC and iBSC and the address on user plane shall be connected

View IPA resource state; view the state of AIPB board and DSP

Check whether the ports correspond to DSP are correct

Check whether IPA resource is blocked manually


IPA resource unavailable

The link between NetNumen M31 and iBSC is disconnected

iBSC is performing writing action for previous data synchronization

…

Check whether the link between NetNumen M31 and iBSC is normal by Ping packets

Retry after waiting for a certain duration, and wait for the completion of writing action on previous synchronization


NetNumen M31 data synchronization faults

Fault analysis

Disconnected communication between server and OMP

OMP doesn't add FTP user; while trying to login OMP through FTP, it is found that zte/zte can't login

Normal communication between server and OMP

After logging into foreground OMP through zte/zte and adding users by addFtpUserInfo "zte","zte",3,3,0, it can synchronize with foreground

Troubleshooting

-------iBSC data failed to synchronize and download to NE

Alarm case on data and software loading faults

Content


Faulty physical transmission

Signaling faults

Trunkline fault at interface A

Fault caused by wrong data configuration

1

2

3

4

Category of connection faults at interface A

Troubleshooting Guide about Connection Faults at Interface A

Unstable No. 7 signaling BSC can not PING virtual address at interface A

successfully Synchronization fault at interface address Office is available and BSC message can't reach

MSC server

Transmission problems, including high bit error ratio of transmission equipments and bad contact of connectors

The fault is possibly due to message overflow resulting from high signaling load

…

Check whether connections are reliable. Test the bit error ratio of transmission

Check the signaling link load through performance measurement. Expand the link if the load is too high


Unstable No. 7 signaling

This is caused by inconsistent ARP entries between MAC address of BSC interface board and virtual IP address at interface A in MSC Server NE

…

Manually clear wrong MAC or reset BSC IPI interface board


BSC can not PING virtual address at interface A successfully

Print all interface information on RPU, it is found that IPI interface is UP and another interface is configured as unknown unit interface address for IPI, its state is DOWN. Through analysis, it is acknowledged that old data is not deleted completely, leading to interface address incorrectly synchronized to interface board

…

Delete ZDB file on RPU and restart it


Synchronization fault at interface address

SI0 location is inaccurate, the message transferred from exchange domain by BSC is implemented by keywords record in index SI0 location. If SI0 location is inaccurate, BSC may discard relevant messages

Network appearance configuration of BSC is inconsistent with that of MSC Server

…

Use NetNumen M31 client to view whether "Original point code" that configures "SIO location" at interface A is correct

Use NetNumen M31 client to view whether "network appearance" that configures "local office point" is consistent with network appearance setting on MSC Server. If inconsistent, you shall modify it to be consistent


Office is available and BSC message can't reach MSC server

Fault analysis

Check process parameters on application server for local office point in background configuration management, "Application property" is configured as "server"; verify it with the engineer at CN side, this property for MSC Server is also configured as “server", if the settings for both parties are same, coupling is impossible to connect

After changing "application property" on application server at BSC side to "Client", the problem is solved

Troubleshooting

Coupling disconnection due to data configurationAlarm case on connection faults at interface A

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb

interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Faults caused by data configuration: such as wrong data configuration at Gb port

Fault caused by hardware problem: such as GIPB board damaged

Category of Connection Faults at Interface Gb

Troubleshooting guide about connection Faults at interface Gb

Cannot ping iBSC successfully Cannot ping SGSN successfully

Subnet mask is not configured as 255.255.255.255 while configuring iBSC virtual address

Cross-network virtual address with iBSC, no static route or wrong static route configuration

IPGB board uses wrong network ports

…

Configure RPU subnet mask for iBSC virtual address as 255.255.255.255 for interface configuration in IP related configuration

Correctly configure static routeMake network ports used by

backplane of IPGB board matched with the ones which setting in NetNumen M31


Cannot ping iBSC successfully

Configuration incompleteSubnet mask is not

configured as 255.255.255.255 while configuring SGSN virtual address

Cross-network virtual address with SGSN, no static route or wrong static route configuration

…

Port, loopback address, NSEI, local and remote IP port, and NSVC shall be configured in order

Configure subnet mask for loopback address as 255.255.255.255 for interface configuration in protocol stack configuration

Correctly configure static route in NetNumen M31


Cannot ping SGSN successfully

Content


System faultsBoards faults

Category of hardware faults

System fault

Interconnected board link faults

Startup faults

External interface link faults

Environmental faults

Clock faults

System faults

Board

CMPGUIM

CHUB

SBCX

GUP2

GLI

PSN

ICM

UIMC

OMP

GIPI

Boards faults

Fault analysis

It is found that OMP module type is configured as OMP and RPU. MP type is selected as OMP_SMP_CMP while creating OMP version of ini file

Re-create OMP version of ini file and select its MP type as OMP for later solution

Troubleshooting

-------Instance of OMP restarting repeatedlyAlarm Case on board faults

Content


Category of CS service faults

Number Category of CS service faults

1 Access faults Fault caused by frequent location update

Faults of service failure

QoS fault

2 Handover faults

3 Congestion faults Ground resource congestion

Wireless resource congestion

4 Call drop faults Call drop due to hardware problem

Call drop due to data configuration and optimization

5 Voice faults Faults from wireless environment

Faults from data configuration

Faults from hardware issues

When MS is powered on, it finds the network and performs a location update

MS performs location update periodically

MS is moved to a new location area

…

It needs to determine whether location areas are too small, and plan the location areas again if necessary

It needs to adjust the radio coverage range, and if possible, increase the minimum access level of the cell

Cooperate with MSC to modify the value of timer T3212


-------Frequent Location Update

Troubleshooting guide about access faults(1)

BSC provides many ways for the high priority subscriber to occupy resource first, which includes reserving a few TCH/F for high priority subscribers, and making the low priority subscriber handover to adjacent cell. The fault that call fails even for a high priority subscriber indicates the above measures are not taken effectively

…

In Other property of Cell params, open Use cell EMLPP threshold to configure some reserved channel for high-priority user, and set Reserved rate of eMLPP to configure user priority threshold of reserved channel

Enable the cell parameters of allowing forced handover and allowing forced disconnection in Service process additional params


-------Problem about High Priority Subscriber AccessTroubleshooting guide about access faults(2)

Fault analysis

By signaling trace SCCP protocol, it is found that unmatched encryption algorithm is prompted in connection error

We verify and confirm with core network side that A5/1 voice encryption algorithm is opened and default iBSC parameters are closed

After modifying and using A5/1 encryption in Other property, the call is restored

Troubleshooting

-------Call disconnection due to different voice encryption modes at core network and radio ride

Alarm Case on access faults

The handover failure of radio interface is related to the radio quality of the handover-in cell, for example, the handover-in cell has severe interference, hardware fault, or failure of obtaining the timing advance

…

Find out towards which cell the handover failure ratio is high

Check whether the handover-in cell with handover failure has hardware faults in some carriers, improper power, or high assignment failure ratio

Check whether the handover-in cell with handover failure has interference

Check whether MARGIN settings for handover threshold window, slide window, and adjacent window are reasonable

Check whether the adjacent relation is correct


-------High Ratio of Handover FaultsTroubleshooting guide about handover faults

The location areas are too small

Burst traffic (short messages for example) causes SDCCH congestion

The traffic is too heavy and too few SDCCHs are configured

Hardware fault of the carrier causes SDCCH congestion

Interference on the radio interface causes congestion

Periodical location updating timer T3212 timeout is too short

…

Adjust the location area sizeEnable the SDCCH dynamic

conversion functionIncrease the number of SDCCHs in

configurationSet MAX number of re-transmitting

events times to reduce the number of random access retransmitted by MS

Troubleshoot the hardware fault or replace the carrier to restore the number of available SDCCHs

Check whether there is any TCH carrier with the same frequency and same color code as those of BCCH nearby


-------SDCCH congestionTroubleshooting guide about congestion faults(1)

By creating Performance Statistics at MAC Port and Performance Statistics at IP Port in background performance management, view transmitting & receiving bytes, its average rate, transmitted & received packet number, and discarded packet number

View resource occupancy condition in IPA Management dynamically managed from background BSC data

…

Add IPI boardAdd AIPB board


-------IPA resource congestionTroubleshooting guide about congestion faults(2)

Interference includes co-channel interference and adjacent-channel interference. When MS receives strong co-channel or adjacent-channel interference signals in the service cell, the signal quality degrades, which makes MS unable to accurately demodulate the BSIC of the adjacent cells or makes BTS unable to correctly receive the MS measurement report, thus the call quality becomes worse and call drop occurs

…

Interference can be avoided by modifying the BTS transmission power, downtilt of the antenna, or the frequency point plan of the cell concerned

Use DTX, frequency-hopping technology, power control, and diversity

Adjust the intra-cell handover parameters relevant with interference

Adjust the handover threshold and cell reselection parameters to lower the cell traffic


-------Call drop due to interfaceTroubleshooting guide about call drop faults

Echo suppressor fault or a few trunks not connecting echo suppressors may all cause the echo problem

The fault may be due to MS problems

The fault may be due to BTS hardware problems

The self loop of a few A-interface circuits or inter-office trunks will cause that only one’s own echo can be heard and the opposite party’s voice can not be heard

…

Ensure whether the fault exists only in a few MSs

Check whether the trunk self loop exists

Check whether the echo suppressor of MSC works properly

Decide which hardware of BTS is related to the fault through dialing test and signaling tracing, and replace relevant faulty hardware


-------Echo problemTroubleshooting guide about voice faults

Content


Insufficient radio channelInsufficient Abis transmission

resourcePoor UPPB DSP resource

…

Adjust wireless channel configuration and add PDCH channels

Add transmission at AbisAdjust UPPB DSP or add UPPB

boar


-------PDCH allocation faultsTroubleshooting guide about PS service faults(1)

Network of the computer connected to MS isn’t installed correctly

The dial-up network setting is incorrect

GPRS parameter setting of the MS is incorrect

…

Ensure the dial-up network is correctly installed in the network properties

Check the dial-up network setting to ensure the data configurations (such as port number, rate setting etc) are correct

Ensure the data configurations (such as APN, gateway setting) of MS are correct


-------MS PDP activation faultsTroubleshooting guide about PS service faults(2)

Documents

7. GB_TS011_E01_1 ZXG10 iBSC Troubleshooting 60.ppt