Cassandra Adoption on Cisco UCS & Open stack

Full bleed image placeholder

Sukanta Nanda Sr. Database Admin. Cisco IT

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

About Cisco

2

300 locations in165+ countries

460 offices

70,000+ employees

2000+ Applications

1500+ Databases (Prod & Non-Prod)

HANA, Legacy EDW, Hadoop

Supporting Mission Critical Environments

32 data centers and server rooms

334,000 sq. ft. of data center space

35.6 MW of UPS power to raised floors

88% servers virtualized in new DCs, 82% overall

Virtualization goal = 95+%

30000 Virtual Machines


CSCC Application Architecture - Today

CCIX

Quote

CCW - 96TB,

12 node RAC,

ERP + Custom

OCM

X-track

SOM

Oracle CM- Offline Jobs

POM

Upload, Save,Order, SNIF,Convert contract, email

NGVS – VMs(2012+)

Validation JobsQAS

Advanced Search(Lucene VM)

SVE Adminsprice

AQS AQS Opportunity - 2014+ (Lucene VM)

Web requests

Advanced SearchRequests

$U Jobs

CAAS

ASFAIL

BID/Customer contract data

Opportunity data


CSCC – QTR by the numbers …

69Kunique users accessed CSCC/SMS332K external users

628K# of estimates/quotes

$2.7Bbooked (99.7% portal, <1% B2B)

18%order touch rate

10.6 hrsavg. order cycle time

1.3 millionhits per day

213Kservice orders

92%of services booked thru CSCC/SMS3

99.78%availability


CSCC Today - FeedbackConsistent Feedback on the need for Change

“I can't even see/find my contracts. I used to expect so much from Cisco and would have recommended your products to anyone, but not anymore. Why does everything, including managing contracts or getting support, have to be so complex. For being a company that I thought of as innovative and a big reason for the success of the internet, Cisco has fallen a long way. It's like Cisco is stuck in 1999.”

Source: CSCC Feedback Form

“I am new to services from the product team and have quite a few views on our tools”

Source: Services Sentiment Survey

“Simplify. I've spoken to many partners, and resellers and managing your portal is a full time job. Cisco needs less engineers developing your website and a few administrators with some common sense. Some of support folks I've spoken to have trouble with the website...”


This tool is extremely complex and slow. It takes hours and hours to do simple low value quotes, coming up with error after error, and regularly requiring manual intervention from Cisco to get it just to do something that should be simple. I really hope you can come up with something better and quickly.”


“Our Customers are telling us that they feel like they did back in the CSCC CAP days. (Policy, process & tool?)”

Source: Juli Clark, Cisco Director Management Operations – CPE

“We have agents that cry using CSCC (Sales turnaround expectations)”

Source: Katie, Convergys


Platform Challenges

Performance

• Performance : Online - multi-seconds to few minutes, Offline Processing - multi-minutes up to few hours

• Scalability : Non-linear degradation with large data

• Quarterly release : 16+ hours downtime• Weekend downtimes – backups, purge table jobs,

EBF - 2+ hours downtime• Unplanned downtime – (minimal)

• Quarterly releases mechanism • Large number of people effort• Stretched in doing 4 releases/year• Need downtime + DBAs + SCM + Manual

preparation for deploys

• Complex, non-intuitive • Difficult to change workflows

ScalabilityLatency Uptime

Agility Resiliency User Interface

• Data Reliability• Resilient across data center


1st Client Cisco Commerce Renewals Cloud Clients

IaaS - OpenStack StorageCompute Networking

Platform

Cassandra (Database)

ApplicationsNotifications Pricing Search Quoting

Data Loader Validation (drools)

Cisco UCE Browser App AndroidPartner app IOS

Upload

Conversion

Nginx(Web

Server)

Plat

form

, bui

ld, T

est A

utom

atio

n (P

uppe

t, Na

gios

, Jen

kins

Ordering Web

Tomcat(Java

appServer)

Elastic Search(Search Engine, Log

Mining)Rabbit MQ

(Messaging)

HAProxy(Load

Balancer)

Memcached(In Memory

Cache)

Logstash(Log

Forwarder)

Kibana(Log

Visualizer)Quartz

(Scheduler)


Platform Improvements

Performance

• Performance : Online - Can query multiple Service Orders using Elastic Search & Cassandra in few Seconds

• Scalability : Linear scalable by Addition of multiple nodes• Elastic Search Improved by 300%

• Quarterly release : Can follow ITDT model• Weekend downtimes – NONE• EBF - NONE• Unplanned downtime – NONE

• Quarterly releases mechanism • Large number of people effort• Stretched in doing 4 releases/year• Need downtime + DBAs + SCM + Manual

preparation for deploys

• Complex, non-intuitive • Difficult to change workflows

ScalabilityLatency Uptime

Agility Resiliency User Interface

• Complete Resiliant • Resilient across data center


Cassandra & Spark on Physical Server with SSD

4

1

6

5

2

3

(Transactional Physical/SSD) 3

1

2(ETL /Spark

Physical/SSD)

• C220 M4 Servers• 256 GB Memory each• 7 SSD Drives 960GB each• RHEL 6.5 OS 64bit• JBOD Configuration • Datastax 4.8.6



CISCO

DC2 [US]

HAProxy-1(Process :

8000)

HAProxy-1(Process : 80)


HAProxy-2(Process: 8000)

https://ccrc.cisco.com/

https://ccrc-internal-2.cisco.com/

Public VIP

Internal VIP

Module1

VM

Module2

VM

Module3

VMModule4

VM

Module5

VM

Module5VM

Elastic Search Cluster

CISCO

HAProxy-1(Process :

8000)



HAProxy-2(Process:

8000)

Public VIP

Internal VIP

Module1

VM

Module2

VM

Module3

VMModule4

VM

Module5

VM

Module5

VM

Elastic Search Cluster

DC1 [US]

https://ccrc-internal -1.cisco.com/

GSShttps://ccrc-external-1.cisco.com/ https://ccrc-external-2.cisco.com/

DMZ HAProxy-1 DMZ HAProxy-2

Internal VIP

DMZ HAProxy-1 DMZ HAProxy-2

Internal VIP

Cassandra Cluster

Cassandra DC1

Cassandra DC2

https://ccrc.cisco.com/

http://ccrc-int.cisco.com/ServiceContract/qotservice

http://ccrc-int.cisco.com/ServiceContract/qotservice

http://ccrc-external-1.cisco.com/

http://ccrc-external-1.cisco.com/



Thank you.

Technology

Cassandra Adoption on Cisco UCS & Open stack