a 300+ OpenStack Node Installing, Configuring, and Managing · Step 1: Design the Network (Cumulus) 1) Layer 3 networking throughout with Cumulus Linux 2) Scalability through Ansible

cumulusnetworks.com | redhat.com | dell.com

Nolan Leake — Co-Founder, Cumulus NetworksBrad Watkins — NFV Partner Engineer, Red Hat

Jaiwant Virk — Senior Product Manager, Dell

Installing, Configuring, and Managinga 300+ OpenStack Node...

April 28, 2016 @ OpenStack Summit Austin 2016

… network in 15 minutes… overcloud in 6 hours


Agenda

Why did we do this?

Project Planning

Virtual Prototyping

Virtual to Physical Deployment

Results/Lessons Learned

Links/Resources


Questions we wanted answered

■ Can we quickly deploy the compute nodes?■ Can the entire pod be prototyped virtually first?■ Can the entire pod be entirely Layer 3 with VXLAN?■ Can Ansible and Git be the common “language”

between Network Engineers and System Admins?■ Can RHOSP Director handle the bulk deployment?■ Can the entire project be done remotely with ease?

Spoiler Alert: Answer is YES to all of the above!


Step 0: Find a lab, thank you to Dell!


Project Inventory

Overcloud: 303 Dell R220 1U servers (compute, controllers)One Dell R630 server (undercloud)9 physical racks in Dell Santa Clara lab

Network: 6 Dell S6000 switches (spine)18 Dell S4048 switches (leaf)Cumulus Linux 2.5.6Cumulus Quagga Linux package

OpenStack Distro: Red Hat OpenStack Platform 7


Step 1: Design the Network (Cumulus)

1) Layer 3 networking throughout with Cumulus Linux2) Scalability through Ansible and Git — Linux “All the Way Down”3) Compute deployment simplicity with Cumulus Quagga Linux package4) Dell open networking switches with ONIE bootloader

...

...

...


Step 2: Build the Virtual Prototype (Red Hat)

■ Build and provision spine/leafs using Cumulus VX and Ansible

■ Deploy five compute nodes using Red Hat OSP-d■ Build Ansible inventory file from nova list bash script■ Bootstrap compute nodes by building management network■ Run Ansible site playbook to install Cumulus Quagga and

configure nodes to join L3 fabric


Step 3: Deploy to Physical (Cumulus/Red Hat)

■ Reuse all Ansible scripts from virtual and apply to physical■ Deploy overcloud on 60 Dell R220 node “batches” due to

undercloud and controller hardware configuration■ Stop at 300 compute nodes, and 3 controllers■ Create 1000 tenant networks across compute nodes


Step 4: Analyze Results

■ 100% Linux in the entire rack■ Built and provisioned the network in 15 minutes without

proprietary APIs or controllers■ All Layer 3 network using BGP Unnumbered, no VLANs

(VXLAN), and reduced IP address bookkeeping■ Built overcloud in less than 6 hours with hardware provided■ Network and overcloud deployed only with Ansible, Git, ZTP

and Red Hat OpenStack Project Director■ Stress test with Rally and analyze with Browbeat


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Screenshot!


Step 5: You can do it too!

● VirtualBox “Rack on a Laptop Part II” with RDO

https://support.cumulusnetworks.com/hc/en-us/articles/216634868-Demo-OpenStack-Cumulus-VX-Rack-on-a-Laptop-Part-II-L3-to-the-Host-

https://support.cumulusnetworks.com/hc/en-us/articles/216634868-Demo-OpenStack-Cumulus-VX-Rack-on-a-Laptop-Part-II-L3-to-the-Host-


Links and Resources

■ Project official joint press release ■ MovingPackets.net review of project■ Cumulus VX - it’s free!■ Git: Ansible playbooks, quagga package builds and deployment:

■ https://github.com/leifmadsen/quagga-config-deploy■ https://github.com/leifmadsen/quagga-rpm■ https://github.com/CumulusNetworks/dellrh_openstack_ansible

https://cumulusnetworks.com/press_releases/detail/20160414-cumulus-networks-collaborates-with-dell-and-red-hat-to-simplify-300-node-openstack-pod/

http://movingpackets.net/2016/04/14/cumulus-dell-red-hat-demo-linux-full-stack-sddc/

http://movingpackets.net/2016/04/14/cumulus-dell-red-hat-demo-linux-full-stack-sddc/

https://cumulusnetworks.com/cumulus-vx/


https://github.com/leifmadsen/quagga-config-deploy

https://github.com/leifmadsen/quagga-config-deploy

https://github.com/leifmadsen/quagga-rpm

https://github.com/leifmadsen/quagga-rpm

https://github.com/CumulusNetworks/dellrh_openstack_ansible

https://github.com/CumulusNetworks/dellrh_openstack_ansible


▪So, what’s next?▪ Keep on learning▪ Join the community http://community.cumulusnetworks.com/▪ Participate in discussions

▪Download Cumulus VX to work on proofs of concept, automation building, and continued practice: https://cumulusnetworks.com/cumulus-vx/

▪Use lab guides and demo modules to learn at your own pace:https://support.cumulusnetworks.com/hc/en-us/articles/201787686 and https://support.cumulusnetworks.com/hc/en-us/sections/200398866

Join the Cumulus Networks Community!

http://community.cumulusnetworks.com/


https://support.cumulusnetworks.com/hc/en-us/articles/201787686

https://support.cumulusnetworks.com/hc/en-us/sections/200398866

https://support.cumulusnetworks.com/hc/en-us/sections/200398866


Thank You!

© 2015 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.

Documents

a 300+ OpenStack Node Installing, Configuring, and Managing · Step 1: Design the Network (Cumulus) 1) Layer 3 networking throughout with Cumulus Linux 2) Scalability through Ansible