18
CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883

CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

  • Upload
    others

  • View
    35

  • Download
    0

Embed Size (px)

Citation preview

Page 1: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

CPS vDRA Troubleshooting Guide, Release 13.1.0First Published: 2017-08-18

Last Modified: 2017-08-18

Americas HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000 800 553-NETS (6387)Fax: 408 527-0883

Page 2: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITEDWARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITHTHE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain versionof the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDINGANYOTHERWARRANTYHEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS"WITH ALL FAULTS.CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OFMERCHANTABILITY, FITNESS FORA PARTICULAR PURPOSEANDNONINFRINGEMENTORARISING FROMACOURSEOFDEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUTLIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERSHAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, networktopology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentionaland coincidental.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnershiprelationship between Cisco and any other company. (1721R)

© 2017 Cisco Systems, Inc. All rights reserved.

Page 3: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

C O N T E N T S

P r e f a c e Preface v

About this Guide v

Audience v

Additional Support v

Conventions (all documentation) vi

Obtaining Documentation and Submitting a Service Request vii

C H A P T E R 1 Troubleshooting CPS vDRA 1

Overview 1

General Troubleshooting 1

Diameter Troubleshooting and Connections 1

DRA Plug-in Configuration in DRA Policy Builder (PB) 2

Troubleshooting Basics 3

Diameter Error Codes and Scenarios 3

Common Troubleshooting Steps 6

CPS vDRA Logs 6

Counters and Statistics 6

Frequently Encountered Troubles in CPS vDRA 7

Redis Not Working 7

Gx Bindings not happening on Mongo 8

Rx Call Failing at CPS vDRA 9

CPS vDRA Forwarding Message to Wrong Peer 9

PCRF Generated Messages not Reaching CPS vDRA 10

Issues in Reaching Ports and Setup IPs 10

CPS vDRA Troubleshooting Guide, Release 13.1.0 iii

Page 4: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

CPS vDRA Troubleshooting Guide, Release 13.1.0iv

Contents

Page 5: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

Preface

• About this Guide, page v

• Audience, page v

• Additional Support, page v

• Conventions (all documentation), page vi

• Obtaining Documentation and Submitting a Service Request, page vii

About this GuideThis document describes common methods and scenarios of correcting processing and production functionsfor CPS vDRA and its various components.

AudienceThis guide is best used by these readers:

• Network administrators

• Network engineers

• Network operators

• System administrators

This document assumes a general understanding of network architecture, configuration, and operations.

Additional SupportFor further documentation and support:

• Contact your Cisco Systems, Inc. technical representative.

• Call the Cisco Systems, Inc. technical support number.

CPS vDRA Troubleshooting Guide, Release 13.1.0 v

Page 6: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

•Write to Cisco Systems, Inc. at [email protected].

• Refer to support matrix at https://www.cisco.com/c/en/us/support/index.html and to other documentsrelated to Cisco Policy Suite.

Conventions (all documentation)This document uses the following conventions.

IndicationConventions

Commands and keywords and user-entered textappear in bold font.

bold font

Document titles, new or emphasized terms, andarguments for which you supply values are in italicfont.

italic font

Elements in square brackets are optional.[ ]

Required alternative keywords are grouped in bracesand separated by vertical bars.

{x | y | z }

Optional alternative keywords are grouped in bracketsand separated by vertical bars.

[ x | y | z ]

A nonquoted set of characters. Do not use quotationmarks around the string or the string will include thequotation marks.

string

Terminal sessions and information the system displaysappear in courier font.

courier font

Nonprinting characters such as passwords are in anglebrackets.

< >

Default responses to system prompts are in squarebrackets.

[ ]

An exclamation point (!) or a pound sign (#) at thebeginning of a line of code indicates a comment line.

!, #

Means reader take note. Notes contain helpful suggestions or references to material not covered in themanual.

Note

CPS vDRA Troubleshooting Guide, Release 13.1.0vi

PrefaceConventions (all documentation)

Page 7: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

Means reader be careful. In this situation, you might perform an action that could result in equipmentdamage or loss of data.

Caution

IMPORTANT SAFETY INSTRUCTIONS.

Means danger. You are in a situation that could cause bodily injury. Before you work on any equipment,be aware of the hazards involved with electrical circuitry and be familiar with standard practices forpreventing accidents. Use the statement number provided at the end of each warning to locate its translationin the translated safety warnings that accompanied this device.

SAVE THESE INSTRUCTIONS

Warning

Provided for additional information and to comply with regulatory and customer requirements.Warning

Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a servicerequest, and gathering additional information, see What's New in Cisco Product Documentation.

To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What'sNew in Cisco Product Documentation RSS feed. RSS feeds are a free service.

CPS vDRA Troubleshooting Guide, Release 13.1.0 vii

PrefaceObtaining Documentation and Submitting a Service Request

Page 8: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

CPS vDRA Troubleshooting Guide, Release 13.1.0viii

PrefaceObtaining Documentation and Submitting a Service Request

Page 9: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

C H A P T E R 1Troubleshooting CPS vDRA

• Overview, page 1

• General Troubleshooting, page 1

• Diameter Troubleshooting and Connections, page 1

• Troubleshooting Basics, page 3

• Common Troubleshooting Steps, page 6

• Frequently Encountered Troubles in CPS vDRA, page 7

OverviewCPS vDRA is a functional element that ensures that all Diameter sessions established over Gx, Rx interfacesand for unsolicited application reporting, the Sd interface for a certain IP-CAN session reach the same PCRFor destined PCRF when multiple and separately addressable PCRFs have been deployed in a Diameter realm.

General TroubleshootingRun the following command in CLI to view the diagnostics status. Verify that the status of all the nodes is inpassing state.admin@orchestrator[master-0]# show system diagnostics statusRun the following command in CLI to view the docker engines status. Verify that all docker engines are inCONNECTED state.admin@orchestrator[master-0]# show docker engine

Diameter Troubleshooting and ConnectionsFor messages belonging to particular interface, CPS vDRA should be ready to make diameter connection onthe configured application port. As CPS vDRA acts as a server, it should be listening on ports for differentapplications to accept any incoming diameter requests for the application.

If you are facing problems making diameter connections, check for the following configuration:

CPS vDRA Troubleshooting Guide, Release 13.1.0 1

Page 10: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

DRA Plug-in Configuration in DRA Policy Builder (PB)

Figure 1: DRA Endpoints

Step 1 Check status of application base port on active policy director (lb). It should be listening to diameter connections externallyon VIP and internally to Policy Servers (QNS).[root@lb01 ~]# netstat -na | grep 3868tcp 0 0 10.77.207.100:3868 0.0.0.0:* LISTENtcp 0 0 ::ffff:80.80.80.10:3868 :::* LISTEN

Step 2 Check haproxy-diameter.cfg file for proper entries:For Step 1, on page 2 and Step 2, on page 2 configuration, the entries should be as follows:

[root@lb01 ~]# cat /etc/haproxy/haproxy-diameter.cfgglobaldaemonnbproc 1 # number of processing coresstats socket /tmp/haproxy-diameter

defaultstimeout client 60000ms # maximum inactivity time on the client sidetimeout server 180000ms # maximum inactivity time on the server sidetimeout connect 5000ms # maximum time to wait for a connection attempt to a server to

succeedlog 127.0.0.1 local1 errlisten diameter-int1

bind 10.77.207.100:3868mode tcpoption tcpkabalance leastconnserver lb01-A lb01:3868 checkserver lb01-B lb01:3869 checkserver lb01-C lb01:3870 check

listen diameter-int2bind 10.77.207.100:4868mode tcpoption tcpkabalance leastconnserver lb01-A lb01:4868 checkserver lb01-B lb01:4869 checkserver lb01-C lb01:4870 check

listen stats_proxy_diameter lbvip01:5540mode httpoption httpclose

CPS vDRA Troubleshooting Guide, Release 13.1.02

Troubleshooting CPS vDRADRA Plug-in Configuration in DRA Policy Builder (PB)

Page 11: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

option abortonclose# enable web-statsstats enablestats uri /haproxy-diam?stats#stats auth haproxy:cisco123stats refresh 60sstats hide-version

Step 3 Listen for diameter traffic by logging into lb01 and lb02 and execute the following command:tcpdump -i any port 3868 -s 0 -vv

Troubleshooting BasicsTroubleshooting CPS vDRA consists of these types of basic tasks:

• Gathering Information

• Collecting Logs

• Running Traces

Diameter Error Codes and Scenarios

Table 1: Diameter Error Codes and Scenarios

DescriptionResult-Code ValueResult-Code

Informational

Subsequentmessages triggered by client shallalso used in Authentication and to get accessof required resources. Generally used inDiameter NAS.

1001DIAMETER_MULTI_ROUND_AUTH

Success

Request processed Successfully.2001DIAMETER_SUCCESS

Request is processed but some moreprocessing is required by Server to provideaccess to user.

2002DIAMETER_LIMITED_SUCCESS

Protocol Errors [E-bit set]

Server returns it if Diameter Command-Codeis un-recognized by server.

3001DIAMETER_COMMAND

_UNSUPPORTED

CPS vDRA Troubleshooting Guide, Release 13.1.0 3

Troubleshooting CPS vDRATroubleshooting Basics

Page 12: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

DescriptionResult-Code ValueResult-Code

Message cannot be delivered because thereis no Host with Diameter URI present inDestination-Host AVP in associated Realm.

3002DIAMETER_UNABLE

_TO_DELIVER

Intended Realm is not recognized.3003DIAMETER_REALM_NOT

_SERVED

Shall return by server only when serverunable to provide requested service, whereall the pre-requisites are also met. Clientshould also send the request to alternate peer.

3004DIAMETER_TOO_BUSY

-3005DIAMETER_LOOP_DETECTED

In Response from Redirect Agent.3006DIAMETER_REDIRECT

_INDICATION

-3007DIAMETER_APPLICATION

_UNSUPPORTED

It is sent when a request is received withinvalid bits combination for consideredcommand-code in DIAMETER Headerstructure. For example, Marking Proxy-Bitin CER message.

3008DIAMETER_INVALID_HDR_BITS

It is sent when a request is received withinvalid flag bits in an AVP.

3009DIAMETER_INVALID_AVP_BITS

A DIAMETER server can be configuredwhether it shall accept DIAMETERconnection from all nodes or only fromspecific nodes. If it is configured to acceptconnection from specific nodes and receivesCER frommessage from any node other thanspecified.

3010DIAMETER_UNKNOWN_PEER

Transient Failures [Could not satisfy request at this moment]

Returned by Server, most likely because ofinvalid password.

4001DIAMETER_AUTHENTICATION

_REJECTED

Returned by node, when it receivesaccounting information but unable to storeit because of lack of memory.

4002DIAMETER_OUT_OF_SPACE

CPS vDRA Troubleshooting Guide, Release 13.1.04

Troubleshooting CPS vDRADiameter Error Codes and Scenarios

Page 13: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

DescriptionResult-Code ValueResult-Code

Peer determines that it has lost election bycomparing Origin-Host value received inCER with its own DIAMETER IDENTITYand found that received DIAMETERIDENTITY is higher.

4003ELECTION_LOST

Permanent Failures [To inform peer, request is failed, should not be attempted again]

AVP marked with Mandatory Bit, but peerdoes not support it.

5001DIAMETER_AVP

_UNSUPPORTED

-5002DIAMETER_UNKNOWN

_SESSION_ID

User can not be authorized. For example,Comes in AIA on s6a interface.

5003DIAMETER_AUTHORIZATION

_REJECTED

-5004DIAMETER_INVALID_AVP_VALUE

Mandatory AVP in request message ismissing.

5005DIAMETER_MISSING_AVP

A request was received that cannot beauthorized because the user has alreadyexpended allowed resources. An example ofthis error condition is a user that is restrictedto one dial-up PPP port, attempts to establisha second PPP connection.

5006DIAMETER_RESOURCES

_EXCEEDED

Server has identified that AVPs are presentthat are contradictory to each other.

5007DIAMETER_CONTRADICTING

_AVPS

Message is received by node (Server) thatcontain AVP must not be present.

5008DIAMETER_AVP_NOT_ALLOWED

If message contains the a AVP number oftimes that exceeds permitted occurrence ofAVP in message definition.

5009DIAMETER_AVP_OCCURS

_TOO_MANY_TIMES

In response of CER if no commonapplication supported between the peers.

5010DIAMETER_NO_COMMON

_APPLICATION

Self explanatory.5011DIAMETER_UNSUPPORTED

_VERSION

CPS vDRA Troubleshooting Guide, Release 13.1.0 5

Troubleshooting CPS vDRADiameter Error Codes and Scenarios

Page 14: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

DescriptionResult-Code ValueResult-Code

Message rejected because of unspecifiedreasons.

5012DIAMETER_UNABLE

_TO_COMPLY

When an unrecognized bit in the Diameterheader is set to one.

5013DIAMETER_INVALID_BIT

_IN_HEADER

Self explanatory.5014DIAMETER_INVALID

_AVP_LENGTH

Self explanatory.5015DIAMETER_INVALID

_MESSAGE_LENGTH

For example, marking AVP to Mandatorywhile message definition doesn't say so.

5016DIAMETER_INVALID_AVP

_BIT_COMBO

In response of CER if no common securitymechanism supported between the peers.

5017DIAMETER_NO_COMMON

_SECURITY

Common Troubleshooting Steps

CPS vDRA Logs

Step 1 Use the following command in CLI to view the consolidated application logs.admin@orchestrator[master-0]# show log application

Step 2 Use the following command in CLI to view the consolidated engine logs.admin@orchestrator[master-0]# show log engine

Counters and StatisticsCheck for statistics generated at pcrfclient01/02 in /var/broadhop/stats and counters in beans at jmxterminal.

CPS vDRA Troubleshooting Guide, Release 13.1.06

Troubleshooting CPS vDRACommon Troubleshooting Steps

Page 15: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

Frequently Encountered Troubles in CPS vDRA

Redis Not Working

Step 1 Check redis status by executing the following command:[root@lb01 ~]# service redis statusredis-server (pid 22511) is running...

Step 2 Try starting redis process by executing the following command:[root@lb01 ~]# service redis start

Step 3 Check the following entries in /etc/broadhop/draTopology.ini file at policy directors (lb) and Policy Servers (QNS) forredis connecting on ports 6379, 6380, 6381, 6382:[root@lb02 ~]# cat /etc/broadhop/draTopology.inidra.redis.qserver.1=lb02:6379dra.redis.qserver.2=lb02:6380dra.redis.qserver.3=lb02:6381dra.redis.qserver.4=lb02:6382dra.redis.qserver.4=lb02:6383dra.local-control-plane.redis.1=lb02:6379dra.mongodb.binding.db.ipv6.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.ipv4.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.imsiapn.uri=mongodb://sessionmgr01:27718dra.mongodb.pcap.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.session.uri=mongodb://sessionmgr01:27718[root@lb02 ~]# cat /etc/broadhop/redisTopology.inidra.redis.qserver.1=lb02:6379dra.redis.qserver.2=lb02:6380dra.redis.qserver.3=lb02:6381dra.redis.qserver.4=lb02:6382dra.local-control-plane.redis.1=lb02:6379

Step 4 Redis process on active policy director (lb) should be established with all Policy Servers (QNS) as shown below:[root@lb01 ~]# netstat -na | grep 6379tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTENtcp 0 0 80.80.80.10:6379 80.80.80.10:37400 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38020 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38034 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37390 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38207 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.16:50597 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.14:35703 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.14:35711 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38188 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37375 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38174 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38229 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38211 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.14:35709 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.16:50590 ESTABLISHED

CPS vDRA Troubleshooting Guide, Release 13.1.0 7

Troubleshooting CPS vDRAFrequently Encountered Troubles in CPS vDRA

Page 16: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

tcp 0 0 80.80.80.10:6379 80.80.80.10:38032 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38172 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.16:50605 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38204 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38213 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38223 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38044 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38187 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38205 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38211 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37672 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.14:35710 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.17:59833 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37388 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37389 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37662 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.17:59824 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.16:50596 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38210 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.15:49162 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38231 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38230 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.15:49159 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38152 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37659 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38208 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.17:59832 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.15:49161 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38206 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38212 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:38033 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37650 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.15:49160 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38155 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.10:37660 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.17:59831 ESTABLISHEDtcp 0 0 80.80.80.10:6379 80.80.80.11:38186 ESTABLISHEDtcp 0 0 :::6379 :::* LISTENtcp 0 0 ::ffff:80.80.80.10:37660 ::ffff:80.80.80.10:6379 ESTABLISHED

Gx Bindings not happening on Mongo

Step 1 Check if the binding's exceptions are coming in consolidated-qns.log file.Step 2 Check for the entry -DdraBindingTier=true in qns.conf file on all Policy Servers (QNS).Step 3 Check for the entries in /etc/broadhop/draTopology.ini file.

dra.redis.qserver.1=lb02:6379dra.redis.qserver.2=lb02:6380

CPS vDRA Troubleshooting Guide, Release 13.1.08

Troubleshooting CPS vDRAGx Bindings not happening on Mongo

Page 17: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

dra.redis.qserver.3=lb02:6381dra.redis.qserver.4=lb02:6382dra.redis.qserver.4=lb02:6383dra.local-control-plane.redis.1=lb02:6379dra.mongodb.binding.db.ipv6.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.ipv4.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.imsiapn.uri=mongodb://sessionmgr01:27718dra.mongodb.pcap.uri=mongodb://sessionmgr01:27718dra.mongodb.binding.db.session.uri=mongodb://sessionmgr01:27718

For example, make sure if the primary binding server is 27718 only as per above example.

Step 4 Check for the Binding Keys entries in binding key type profile and the application attached to the profile.

Rx Call Failing at CPS vDRA

Step 1 Check for the Binding key Retriever for Rx Profile.Step 2 Check if the Gx Binding is available for that Binding key.Step 3 Check the consolidated-qns.log file if CPS vDRA is able to retrieve SRK from the bindings.Step 4 Check for any exception in consolidated-qns.log file during binding retrieval.Step 5 If Rx peer is available for the same SRK at CPS vDRA, CPS vDRA should forward the Rx message to that peer.Step 6 Check the connection for that peer and proper entries in Peer Group, Peer Routing, Peer Group Peer and Rx_Routing

for Rx New session rules.

CPS vDRA Forwarding Message to Wrong Peer

Step 1 Check the Control Center configuration in Gx_Routing for new session rules. Gx routing should have the AVP definedon the basis of which, one wants to route the traffic.

Step 2 Check whether the Control Center configuration for the Peer is bonded to correct Peer Group.Step 3 Check whether the Peer Group is assigned to correct Peer Route and Dynamic AVPs are properly aligned with Peer

Route in Gx New Session Rules.Step 4 Diameter Connection with the desired Destination Peer should be established with CPS vDRA.

CPS vDRA Troubleshooting Guide, Release 13.1.0 9

Troubleshooting CPS vDRARx Call Failing at CPS vDRA

Page 18: CPS vDRA Troubleshooting Guide, Release 13.1 · CPS vDRA Troubleshooting Guide, Release 13.1.0 First Published: 2017-08-18 Last Modified: 2017-08-18 Americas Headquarters Cisco Systems,

PCRF Generated Messages not Reaching CPS vDRA

Step 1 Make sure PCRF has the correct entry of CPS vDRA as next hop.

Figure 2: Next Hop Routes

Next Hop definition is mandatory in PCRF to forward the messages to CPS vDRA generated by PCRF itself.

For example, Gx-RAR, Sd-TSR

Step 2 Wild Card Entry not supported in Next Hop Routing configuration.

Issues in Reaching Ports and Setup IPs

Step 1 Check firewall is running or not.Step 2 Make sure the firewall configuration is OK.

a) To check if this is the problem, then stop the firewall./etc/init.d/iptables stop

CPS vDRA Troubleshooting Guide, Release 13.1.010

Troubleshooting CPS vDRAPCRF Generated Messages not Reaching CPS vDRA