DELL EMC VXRACK SYSTEM SDDC TECHNOLOGY OVERVIEW€¦ · Dell EMC and VMware share this vision and together are developing new technologies to make this vision a reality for customers

WHITE PAPER 1

DELL EMC VXRACK SYSTEM SDDCTM TECHNOLOGY OVERVIEW

ABSTRACT

VxRack System SDDC powered by VMware Cloud Foundation is a turnkey, rack-scale,

hyper-converged engineered system. Fully integrated hardware and software provides

an agile and flexible infrastructure foundation that IT organizations can leverage as part

of their IT transformation into an easy to manage cloud operating model. This is

accomplished through system features such as workload domains and automated

infrastructure deployments that are comparable to public cloud offerings, providing

elasticity and scalability while reducing costs.

This paper provides an overview of VxRack SDDC, describes its major components,

and describes its value as a turnkey Software-Defined Data Center system.

October 2017

Document H15859-1

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copy, and distribution of any software described in this publication requires an applicable software license. Copyright © 2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA 2/17 VxRack SDDC Whitepaper.

Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Dell EMC VxRack System SDDC

2

TABLE OF CONTENTS

EXECUTIVE SUMMARY ...........................................................................................................3

INTENDED AUDIENCE .............................................................................................................3

INTRODUCTION ........................................................................................................................3

ENGINEERED SYSTEM OVERVIEW .......................................................................................5

CLOUD FOUNDATION SOFTWARE STACK ..........................................................................6

vSphere ........................................................................................................................................... 77

vSPHERE VIRTUAL NETWORKING ............................................................................................................................. 8

vCENTER SERVER FEATURES FOR AVAILABILITY AND PERFORMANCE ............................................................ 8

NSX ................................................................................................................................................... 9

NSX COMPONENTS ................................................................................................................................................... 10

vSAN ............................................................................................................................................... 10

vSAN Datastore ............................................................................................................................................................ 11

vSAN Storage Policy-Based Management ................................................................................................................... 11

vSAN Features ............................................................................................................................................................. 11

SDDC Manager ............................................................................................................................... 12

Simplified Resource Management with Workload Domains ........................................................................................ 12

Lifecycle Management .................................................................................................................................................. 14

vRealize Suite—VMware’s Cloud Management Platform (CMP) .................................................... 14

Horizon Suite—VMware’s VDI and App Virtualization Platform ...................................................... 15

HARDWARE CONFIGURATION ............................................................................................ 16

Hardware Components ................................................................................................................... 17

Cabinets ........................................................................................................................................................................ 17

Server Nodes ................................................................................................................................................................ 18

Server Node Storage Options ...................................................................................................................................... 18

Network Switches ......................................................................................................................................................... 19

NETWORK TOPOLOGY ........................................................................................................ 21

Network Topology for Single-Rack Configuration ............................................................................ 21

Network Topology for Multi-Rack Configuration .............................................................................. 22

Traditional and Rack Interconnect Network Architecture ................................................................. 22

CONCLUSION ........................................................................................................................ 25


3

EXECUTIVE SUMMARY

The Dell EMC VxRack™ SDDC consists of hyper-converged, rack-scale engineered systems with integrated networking to achieve the

scalability and management requirements of traditional and cloud-native workloads. Dell EMC and VMware have expanded the

industry’s broadest hyper-converged infrastructure portfolio to provide a turnkey rack-scale, hyper-converged system with VxRack

SDDC. The VxRack SDDC is a fully integrated hardware and software solution with VMware Cloud Foundation. A complete turnkey

solution at rack scale, the VxRack SDDC provides industry-best lifecycle management and assurance for VMware SDDC solutions

adding rack-scale capabilities to complement the Dell EMC Vblock® Systems and VxRail™ Appliances.

The hyper-converged VxRack SDDC is designed for enterprise-scale deployments of virtual infrastructure, IaaS, and virtual desktops

infrastructure (VDI) in a VMware environment. Each VxRack SDDC system is based on a standardized architecture that combines Dell

EMC and Cisco hardware with pre-loaded, pre-integrated software components in a complete and validated system. The VxRack

SDDC is a turnkey VMware SDDC system.

Together with VMware, only Dell EMC can provide a tightly engineered hyper-converged solution with storage, networking, computing,

and software. VxRack SDDC brings standardization, modular scale, lifecycle management, and industry-best support for on-demand IT

services that further accelerate business outcomes and the transformation to cloud computing.

INTENDED AUDIENCE

This document is intended for Dell EMC sales and field personnel, partners, and customers involved in designing, acquiring, and

managing a VxRack SDDC solution. It may also be a useful resource for System Administrators and Dell EMC Solutions Architects.

INTRODUCTION

The IT infrastructure market is undergoing unprecedented transformation. The most significant transformation is reflected by two major

trends: a deployment trend toward converged and hyper-converged infrastructure (HCI) and a design trend toward a software-defined

data center architecture. Both are responses to the IT realities of infrastructure clutter, complexity, and high cost; they represent

attempts to simplify IT and reduce the overall cost of infrastructure ownership.

Today’s legacy infrastructure environments are typically comprised of multiple hardware and software products from multiple vendors,

with each product offering a different management interface and requiring different training. Each product in this type of legacy stack is

likely to be grossly overprovisioned, using its own resources (CPU, memory, and storage) to address the intermittent peak workloads of

resident applications. The value of a single shared resource pool, offered by server virtualization, is still generally limited to the server

layer. All other products are islands of overprovisioned resources that are not shared. Therefore, low utilization of the overall stack

results in the ripple effects of high acquisition, space, and power costs. Too many resources can be wasted in traditional legacy

environments.

In the modern data center, software defined, converged, and hyper-converged infrastructure (HCI) are cornerstones of virtually every

customer’s IT strategy. Converged and hyper-converged infrastructures mean that multiple pre-engineered and pre-integrated

components operate under a single controlled architecture with a single point of management and a single source for end-to-end

support. HCI provides a localized single resource pool that enables a higher overall resource utilization than with a legacy island-based

infrastructure. There is no separate Storage Area Network (SAN) infrastructure layer to manage. Overall acquisition cost is lower and

management is simplified. In the data center, HCI typically has a smaller footprint with less cabling and can be deployed much faster

and at lower total cost than traditional infrastructure.

Industry infrastructure deployment is transforming as customers begin to shift from a “build” to a “buy” approach. This deployment shift

is being driven by the need for IT to focus limited economic and human capital resources on driving business innovation, resulting in

less resources available to focus on infrastructure. While a “build-your-own” deployment strategy can achieve a productive IT

infrastructure, this strategy can be difficult to maintain and vulnerable to higher operating costs. It is susceptible to greater risk related to

component integration, configuration, qualification, compliance, and management. A “buy” deployment strategy for HCI provides the

benefits of previously integrated, configured, qualified, and compliant components. Buying an HCI system provides a single optimized

IT infrastructure base which is quick and easy to deploy. A “buy” deployment strategy for HCI provides a simple and effective alternative

to “build-your-own”, and it is widely adopted.


4

The software-defined data center (SDDC) is a significant design trend driving an unprecedented transformation of the IT infrastructure

market. SDDC is VMware’s architectural vision for the modern, emerging software-centric data center. Dell EMC and VMware share

this vision and together are developing new technologies to make this vision a reality for customers. While the concept is still evolving,

the SDDC is a software-centric architectural design for the data center based on virtualization and automation. It logically defines all

data center infrastructure services by applying the widely successful principles of server virtualization—abstraction, isolation, and

pooling—to the remaining network and storage infrastructure services. SDDC management is automated through policy-based software

that controls resources both on premises and off premises. With SDDC, traditional enterprise applications can be supported in a more

flexible and cost-effective manner. SDDC represents the epitome of the agile digital business model, where pooled resources adapt

and respond to shifting application requirements. The SDDC architectural vision is being driven by VMware, a thought leader in the

industry. See Figure 1 below for a typical SDDC Architecture.

Figure 1: Software-Designed Data Center (SDDC) Architecture

In the SDDC, virtualized servers represent a familiar software-defined IT model, in which hypervisors running on a cluster of hosts

allocate hardware resources to virtual machines (VMs). In turn, VMs can function with a degree of autonomy from the underlying

physical hardware. Software-defined storage (SDS) and software-defined networking (SDN) are based on a similar premise: physical

resources are aggregated and dynamically allocated based on predefined policies with software abstracting control from the underlying

hardware. The result is the logical pooling of compute, storage, and networking resources. Physical servers function as a pool of CPU

resources hosting VMs, while network bandwidth is aggregated into logical resources, and pooled storage capacity is allocated by

specified service levels for performance and durability.

Once the SDDC has abstracted resources, SDDC services make the data center remarkably adaptable and responsive to business

demands. In addition to virtualized infrastructure, the SDDC includes automation, policy-based management, and hybrid cloud services.

The policy-based model eases compliance capabilities and insulates users from the underlying standardized technology, and policies

balance and coordinate resource delivery. Resources are allocated where needed, absorbing utilization spikes while maintaining

consistent and predictable performance. Conceptually, SDDC encompasses more than the IT infrastructure itself; it also represents an

essential departure from traditional methods of delivering and consuming IT resources. The SDDC vision requires changes in the IT

organization technology, people, and process which then enables a business’s transformation to a cloud model and toward a digital

transformation as a whole. In the SDDC, infrastructure, platforms, and software become cloud computing services and are called

Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), Software-as-a-Service (SaaS) and Container-as-a-Service (CaaS).

SDDC is the fundamental architecture that underpins the most sophisticated cloud computing environments.


5

Cloud services (IaaS, PaaS, SaaS, and CaaS) are accessed via a cloud management platform (CMP) that provides end users with a

self-service catalog of IT services with financial transparency. The VMware SDDC architectural vision includes CMP layers running on

top of SDDC infrastructure layers to create a private or hybrid cloud. This requires a SDDC infrastructure with an extremely high level of

efficiency and serviceability, such as the VxRack SDDC system. VxRack SDDC is the simplest and most complete underlying

infrastructure system needed when building a SDDC with private or hybrid cloud computing services. VMware’s cloud management

platform, vRealize Suite, when combined with VxRack SDDC delivers the most complete solution for a private or hybrid cloud

computing environment.

VxRack SDDC, a hyper-converged engineered system designed as one by Dell EMC and powered by VMware Cloud Foundation, is

comprised of innovative technology from three leading industry providers—Dell EMC, Cisco, and VMware. VxRack SDDC provides a

tightly integrated, turnkey hardware and software system with virtualized computing, networking and storage. VxRack SDDC provides

standardization, modular scale, lifecycle management, and industry-best support for on-demand IT services that further accelerate a

business’ transformation to cloud computing. VxRack SDDC is evolutionary in its design and engineering.

ENGINEERED SYSTEM OVERVIEW

The VxRack SDDC hyper-converged infrastructure system, shown in Figure 2, is powered by VMware’s Cloud Foundation, an

integrated virtualization software platform that includes vSphere, NSX, vSAN, and the SDDC Manager. The hyper-converged VxRack

SDDC is a system which is sized and optimized for customer workloads, and manufactured to customer specifications. The system

arrives on site ready to be configured and integrated into the data center network, and to provision virtual servers within hours.

This new hyper-converged system is designed for companies that want a complete cloud platform infrastructure with improved

economics and security. VxRack SDDC has been developed for companies that have moved to standardize on VMware virtualization

and cloud technologies and want a premier foundation for automated data center operations. VxRack SDDC is an agile system that

enables customers to build an IaaS platform based on a complete SDDC architecture. Its modular design allows datacenter scalability

and flexibility as budgets for next-generation cloud workloads and enterprise applications demand. It’s designed for deployments

involving large numbers of VMs and provides the following features:

Automated bring-up delivers best-in-class day 0 experience – from power-on to system ready in a few hours

Low provisioning complexity with simple-to-use functionality, optimized to consolidate thousands of applications with proven

VMware technologies

Flexible and variable configuration options to support a variety of application workloads

Horizontal scaling by adding, re-assigning, and extending nodes on the fly to extend compute, storage, and networking resources

exponentially

Easy day-to-day management and one-click software upgrades

Pre-configured, pre-loaded, pre-tested, and fully optimized hardware and software stack, delivered as a fully assembled and single

vendor supported system

Enhanced networking and security capabilities built-in and automatically configured

Future proof with full release certification of software and hardware upgrades to insure reliability

A complete, turnkey system, the Dell EMC hyper-converged VxRack SDDC provides the easiest and fastest way to stand up a VMware

based private cloud while at the same time introducing the performance, flexibility, and automation required to make IT departments

more agile. VxRack SDDC powered by VMware Cloud Foundation delivers a rack scale hyper-converged system that simplifies the

transformation to a cloud computing model.


6

Figure 2: VxRack SDDC

VMWARE CLOUD FOUNDATION SOFTWARE STACK

The VxRack SDDC architecture defines data center resources in terms of software. It minimizes compute, network, and storage

hardware constraints and increases infrastructure service agility. This is the evolution from simple server virtualization to complete

virtualization and automation of a modern data center. VMware SDDC software provides virtualization and automation across fully

integrated components.

VMware Cloud Foundation provides the core software platform for this virtualization and automation in the VxRack SDDC. Cloud

Foundation has been designed to extend and optimize the familiar VMware vSphere experience with robust software-defined solutions

for storage and networking as well as simple management. It provides the opportunity to leverage a single standardized SDDC platform

for both on-premises private cloud and off-premises public clouds which ensures compatibility for a true hybrid cloud architecture. It’s a

complete SDDC solution that provides a highly available, resilient, on-demand infrastructure.

Figure 3: VMware Cloud Foundation


7

Cloud Foundation is a unified SDDC software platform that combines leading virtual-computing technology into a natively and fully

integrated, enterprise-ready infrastructure for private and public clouds. It’s made up of vSphere, vSAN, and NSX, and is managed by

the SDDC Manager as shown in Figure 3 above. The following sections examine the Cloud Foundation software components as

implemented in VxRack SDDC.

vSPHERE

VMware vSphere, the compute virtualization component of VxRack SDDC, delivers server virtualization within a highly available,

resilient, and efficient on-demand infrastructure. vSphere is an established, industry-leading software platform with the following

advantages:

Continuous availability and fault tolerance with data protection and replication

Simplified customer experience for automation and management at scale

Optimal hypervisor supported by a broad ecosystem and scalable management

Comprehensive built-in security for protecting data, infrastructure, and access

Load-balanced workloads and prioritized access to resource for top performance and compliance

Rapid provisioning and deployment of workloads and desktops

VxRack SDDC leverages two key components of vSphere: ESXi and vCenter. ESXi is the hypervisor software which is installed directly

onto physical VxRack SDDC nodes, the nodes are powered by the Dell EMC PowerEdge server platform. Each physical server

functions as a virtual ESXi server (ESXi host) made up of partitioned logical servers referred to as virtual machines (VMs). VMs are

configured on top of the ESXi server. VMware vCenter is available under a separate license and is a centralized management

application that is used to manage the vSphere ESXi servers (hosts) and VMs. vCenter communicates with each ESXi host using an

agent that relays tasks to perform management operations directly on the hosts.

VMWARE vCENTER SERVER

The vCenter Server software provides the primary point of management for server virtualization and storage using vSAN, and is the

enabling technology for advanced capabilities such as vMotion, DRS and HA. vCenter scales to enterprise levels where a single

vCenter can support up to 1000 ESXi servers and 10,000 VMs. vCenter supports a logical hierarchy of data centers, clusters, and

hosts, which allows resources to be segregated by use cases or line of business and can also scale dynamically as needed.

vCenter is a single interface that provides a number of services, including task scheduling, statistics logging, alarms, event

management, and VM provisioning and configuration. vCenter also provides distributed services such as vSphere vMotion, vSphere

DRS, and vSphere HA. VxRack SDDC uses vCenter for its own system management cluster as well as for user provisioned workload

cluster management.

VMWARE vSPHERE ESXI

VMware vSphere ESXi is an enterprise-class hypervisor that deploys and services VMs. Figure 4 illustrates vSphere ESXi basic

architecture. ESXi partitions a physical server into multiple secure and portable VMs that can run side by side on the same physical

server. Each VM represents a complete system—with processors, memory, networking, storage, and BIOS—so any operating system

(guest OS) and applications can be installed and run on the virtual machine without any modification. The hypervisor provides physical

hardware resources, dynamically to support the operation of VMs. The hypervisor enables the VMs to operate independently from the

underlying physical hardware. For example, a virtual machine can be moved from one physical host to another. Also, the VM’s virtual

disks (VMDKs) can be moved from one type of physical storage to another without affecting the functioning of the virtual machine. ESXi

also isolates VMs from one another, so when a guest operating system running in one VM fails, other VMs on the same physical host

are unaffected and continue to run. VMs share access to CPUs, and the hypervisor is responsible for CPU scheduling. In addition, ESXi

assigns each VM a region of usable memory and provides shared access to the physical network cards and disk controllers associated

with the physical host.


8

Figure 4: vSphere ESXi Architecture

vSPHERE VIRTUAL NETWORKING

vSphere also provides a rich set of virtual networking capabilities, which are managed through vCenter. Virtual-switch technology

allows communication among ESXi servers and among VMs in a cluster, using the same protocols that would be used over physical

switches. The virtual switch supports VLANs and forwards frames at the data-link layer and virtual Ethernet adapters each have their

own IP and MAC address. As a result, VMs have the same properties as physical machines from a networking perspective. The virtual

adapters connect both VMs and the ESXi server console to external networks.

VxRack SDDC clusters use the VMware Virtual Distributed Switch (VDS), a single switch that spans across multiple ESXi hosts in the

same cluster. This switch enables VMs to maintain consistent network configurations as they migrate across multiple hosts. VDS is

configured in vCenter Server at the data-center level and makes the configuration consistent across all ESXi hosts. vCenter Server

stores the state of distributed ports in the vCenter Server database. Networking statistics and policies migrate with VMs when the VMs

are moved from host to host. As discussed in upcoming sections, vSAN relies on VDS for its storage virtualization functionality, and

VDS is also the underlying technology which NSX leverages for its network virtualization functionality.

vCENTER SERVER FEATURES FOR AVAILABILITY AND PERFORMANCE

VxRack SDDC leverages the following vSphere software suite technologies to ensure a high level of availability and load balanced

performance during planned and unplanned system outages.

vMotion for VM migration

Distributed Resource Scheduler (DRS) for VM load balancing

vSphere HA for node failover protection

vMotion. VMware vMotion™ enables live migration of running VMs from one physical server to another with no downtime, continuous

service availability, and complete transaction integrity. vMotion is a key enabling technology for creating a dynamic, automated, and

self-optimizing data center. vMotion continuously and automatically allocates VMs within resource pools. It also improves availability by

conducting maintenance without disrupting business operations. The advanced capability for migrating workloads without disruption is

one of the features that distinguish the VxRack SDDC system from other hyper-converged systems.


9

Distributed Resource Scheduler. The Distributed Resource Scheduler (DRS) balances computing capacity across a collection of

VxRack SDDS resources that have been aggregated into logical pools. It continuously balances and optimizes compute resource

allocation among the VMs. When a VM experiences an increased workload, DRS evaluates the VM priority against user-defined

resource-allocation rules and policies. If justified, DRS allocates additional resources. It can also be configured to dedicate consistent

resources to the VMs of particular business-unit applications to meet SLAs and business requirements. DRS allocates resources to the

VM either by migrating the VM to another server with more available resources or by making more “resources” for the VM on the same

server by migrating other VMs off that server. In VxRack SDDC, all ESXi hosts are part of a vMotion network. The live migration of VMs

to different node servers is completely transparent to end users through VMotion. DRS adds tremendous value to the VxRack SDDC

clusters by automating VM placement, ensuring consistent and predictable application-workload performance.

vSphere HA. vSphere HA failover technology protects VMs with rapid recovery from outages, providing cost-effective high availability

for VM applications. vSphere HA uses a Fault Domain Manager (FDM) agent to proactively monitor host availability and power state.

When a host fails, vSphere HA restarts the affected VMs on another host. Once vSphere HA is configured, all workloads are protected.

No actions are required to protect new VMs, and applications and VMs need no special software. vSphere HA provides several points

of protection for applications including:

Circumventing any server failure by restarting the VMs on other hosts within the cluster.

Continuous monitoring of VMs and resetting of any detected VM failures.

Protecting against datastore accessibility failures and providing automated recovery for affected VMs.

Protecting VMs against network isolation by restarting them if their host becomes isolated on the management or VMware vSAN

network. This protection is provided even if the network has become partitioned.

NSX

Physical and virtual network design and planning for growth when deploying a cloud model infrastructure is critical to maintaining

operational performance as the environment scales. The Dell EMC VxRack SDDC integrates the latest in networking best practices at

rack scale—both physical and software defined. Physical networking consists of a Cisco topology with top-of-rack (ToR) and rack

interconnect switches. Software-defined networking (SDN) consists of VMware NSX and VMware SDDC Manager for management

which provides network configuration, control, and management of the physical and virtual network elements.

NSX network virtualization delivers the operational model of a VM to VxRack SDDC’s network infrastructure. As an integral component

in the VxRack SDDC architecture, NSX SDN injects improved security into the entire data center infrastructure. With NSX, network

functions including switching, routing, and firewalling are embedded in the hypervisor and distributed across the environment. This

effectively creates a “network hypervisor” that acts as a platform for virtual networks and services as shown in Figure 5.

Figure 5: NSX Software-Defined Networking


10

NSX virtual networks leverage automated, policy-based provisioning and multi-tenant isolation to simplify network management, even

for complex multi-tier network topologies and multiple workload domains. NSX reproduces the entire network model in software,

enabling any network topology to be created and provisioned in seconds. Users can create multiple virtual networks with diverse

requirements, leveraging a combination of the services offered via NSX, to build inherently more agile and secure environments.

NSX COMPONENTS

The NSX virtualized SDN is built on several key components described below.

NSX Manager. NSX Manager provides a central point to deploy and configure the virtualized network components, including

controller cluster systems and VMware Installation Bundles (VIB) for ESXi, VXLAN, logical switching, logical firewall, and logical

routing. NSX Manager can also deploy and configure Edge gateway systems and its services.

NSX Manager is deployed as a single VM in the management domain and controller clusters are deployed on the workload domain

cluster for which the NSX Manager is responsible for managing. Each workload domain has an NSX Manager VM per vCenter

Server VM with NSX virtual switches installed in the hypervisors. SDDC Manager uses NSX APIs to configure a VXLAN with the IP

parameters specified for the workload domain, and to configure the ToR ports associated with the servers.

NSX Distributed Logical Router. The Distributed Logical Router DLR) is responsible for forwarding and routing all packets

through the virtualized SDN networks. The NSX LDR can provide virtual network segmentation. Creating multiple VMware NSX

LDRs enables multi-tenancy or separate security zones. Each LDR can create virtual switches that function in the same context as

a physical VLAN. The LDR is accessed independently on both the Control Plane and Data Plane. Each VMware host has a copy of

the NSX LDR running in the hypervisor. All the gateway interfaces and IP addresses are distributed throughout the VMware

cluster. This allows VMs to directly access their default gateway at the local hypervisor.

NSX Controller. NSX Controller (NSX-C) provides the Control Plane functionality to distribute logical routing, so VXLAN network

information can reach the underlying hypervisor. Controllers are deployed as virtual appliances and should reside in the same

vCenter that connects to NSX Manager. NSX-C nodes are deployed in sets of three within a cluster and they divide the workload

equally in an active-active cluster scenario. Additionally, NSX-C removes the dependency on multicast routing in the physical

network and suppresses broadcast traffic in VXLAN networks.

NSX Edge Services Gateway. The NSX Edge Services Gateway (ESG) offers a rich set of services that include network address

translation, routing, load balancing, firewall, L2/L3VPN and DHCP/DNS relay. Individual services can be deployed, configured, and

consumed on demand. The ESG is a virtual machine, deployed via NSX Manager and accessed using the vSphere web client.

Distributed Firewall. NSX provides a complete L2–L4 stateful distributed firewall that runs in the ESXi hypervisor kernel. Because

the firewall is a function of the ESXi kernel, it provides massive throughput and performs at near line-rate speeds. The distributed

firewall service (DFW) installs in the kernel by deploying the kernel VIB in conjunction with the VMware Inter-networking Service

Insertion Platform (VSIP). VISP is responsible for monitoring and enforcing security policies on all traffic flowing through the Data

Plane and provides anti-spoofing functionality and traffic redirection for third-party appliance extensions and services. DFW

protects virtual-to-virtual or virtual-to-physical traffic. DFW policies can also restrict traffic between VMs and external networks.

Individual VMs without a firewall-protection requirement can be added to the DFW exclusion list.

NSX delivers significant advantages to the VxRack SDDC system including, improved security, on-demand service delivery, faster and

more agile deployment, and operational efficiency. NSX allows IT organizations to break free of the constraints and limitations of

hardware-based data center networking infrastructure. Customers can choose to leverage these features with the VxRack SDDC at the

onset of deployment or can gradually introduce NSX capabilities as use case requires.

vSAN

vSAN is a Software Defined Storage (SDS) technology that includes deep integration with vSphere and the VMware ecosystem to

make it a compelling, effective storage solution, well suited for VxRack SDDC. vSAN decouples software from the underlying hardware

and it implements a notably efficient architecture, built directly into ESXi hypervisor. This distinguishes vSAN from solutions that

typically install a virtual storage appliance (VSA) that runs as a guest VM on each host. Embedding vSAN into the ESXi-kernel layer


11

has clear advantages in performance and memory requirements. It has very little impact on CPU utilization (less than 10 percent) and

self-balances based on workload and resource availability.

vSAN aggregates locally attached disks of VxRack SDDC nodes to create a pool of distributed shared storage. This enables on-

demand provisioning and consumption according to policy. VxRack SDDC supports multiple vSphere clusters with up to 64 nodes per

cluster and up to a total of 192 nodes in a single VxRack SDDC system. Storage characteristics are configured using Storage Policy

Based Management (SPBM), which allows VM object-level policies to be set and modified on the fly to control storage provisioning and

day-to-day management of storage service-level agreements (SLAs).

vSAN DATASTORE

vSAN technology creates a virtual SAN from the local datastore on clustered ESXi instances. The solution provides a single vSAN

datastore spanning all the hosts within each vSphere cluster. In the VxRack SDDC, it leverages the local disk storage on the Dell

PowerEdge servers, which function as ESXi cluster nodes. The resulting virtual SAN becomes shared storage for the ESXi hosts

contributing storage.

vSAN is preconfigured for the management domain when VxRack SDDC is initialized and managed through vCenter. After the

initialization of the management domain, the additional vSAN datastore for user workloads are dynamically provisioned by SDDC

manager from the selected nodes to create a vSphere cluster for that workload domain at the time of workload domain creation. The

VxRack SDDC system total storage capacity depends on the storage disk configuration of the VxRack SDDC server nodes. Each node

contains ten storage devices in two separate disk groups configured for either hybrid storage, with flash SSD drives for the cache tier

and HDD drives for the capacity tier, or for all-flash storage, with flash SSDs for both the cache and capacity tiers.

vSAN STORAGE POLICY-BASED MANAGEMENT

vSAN is entirely policy driven and designed to simplify storage provisioning and management. It automatically and dynamically matches

requirements with underlying storage resources based on VM-level storage policies. vSAN policies define VM storage requirements for

performance and availability. They determine how storage objects are provisioned and allocated within the datastore to guarantee the

required level of service. vSAN policies also define storage-specific attributes for VM objects, including availability and replication

settings.

vSAN FEATURES

Today’s data center management teams have high expectations of their storage platforms in terms of scalability, availability,

performance, and efficiency. vSAN delivers the same advanced technology found in the most robust storage systems.

Scalability. The vSAN distributed architecture suits the modular architecture of VxRack SDDC. Customers can non-disruptively

scale out by adding nodes for capacity and performance.

Availability. As a core VxRack technology, vSAN is tightly integrated with vSphere’s availability feature set, including vSphere HA,

DRS, vMotion, Fault Tolerance, and snapshot technology. vSAN enforces availability specifically at the storage level with efficient

data-replication functionality and Fault Domain technology, which protects the environment from rack-level failures.

Performance. vSAN’s tight kernel-level integration with vSphere gives vSAN a notable advantage in performance as it reduces

CPU and memory overhead. Furthermore, as part of the VMware SDDC software stack, vSAN dynamically self-tunes, adjusting to

ongoing changes in workload conditions to load balance storage resources, ensuring each VM adheres to its defined storage

policies. In addition, vSAN’s flash-optimized design minimizes storage latency by accelerating read and write IO with built-in

caching. The caching tier for each disk group functions as a read/write buffer for hybrid configurations and a write buffer for all-flash

configurations.

Efficiency. Storage-capacity requirements continue to grow exponentially and vSAN uses data deduplication and compression to

increase capacity utilization at a lower cost for all-flash configurations. Meanwhile, vSAN deduplication and compression have only

a minimal impact on CPU overhead and memory. Typical virtual-server workloads with highly redundant data such as full-clone

virtual desktops or homogenous-server operating systems benefit most from data deduplication and compression. vSAN’s kernel-

level integration with vSphere not only enhances performance, but it places the storage close to the application and removes the

complexity of storage management and provisioning. vSAN sits directly in the IO data path, delivering the highest levels of

performance without taxing CPU or memory resources. It’s the optimal SDS backbone for the VxRack SDDC.


12

SDDC MANAGER

SDDC Manager is a new, innovative system management solution for VxRack SDDC designed to deliver a radically simplified user

experience. SDDC Manager automates critical operations across physical and virtual infrastructure such as system initial build-up,

configuration of servers and switches, auto-discovery of new physical capacity, resource provisioning, and lifecycle management of

hardware and software components. SDDC Manager serves as the primary interface for an operator’s day-to-day tasks and provides

an integrated view of both the physical and virtual infrastructure. It complements well-known VMware management tools such as

vCenter Server and vRealize Operations that continue to be available for advanced administration tasks and integration with third-party

software tools.

SIMPLIFIED RESOURCE MANAGEMENT WITH WORKLOAD DOMAINS

The value of an SDDC platform is simple resource management. In VxRack SDDC, physical compute, storage and network

infrastructure becomes part of a single shared pool of virtual resources that is managed as one system using the SDDC Manager. From

this shared pool, customers can carve out separate pools of capacity called workload domains, each with its own set of specified CPU,

memory, network, and storage requirements to support various workloads types. Two types of customer workload domains are

supported currently, Virtual Desktop Infrastructure (VDI) and Virtual Infrastructure (VI). As new physical capacity is added to the

VxRack SDDC, added resources are automatically recognized by SDDC Manager and made available for consumption. The entire

system is managed as one thereby removing any physical constraints of a single physical server or rack.

SDDC Manager abstracts and aggregates physical resources into these customer defined logical workload domains which are actually

physical vSphere clusters provisioned from available nodes in the VxRack SDDC system. SDDC Manager then automates the

configuration of each physical vSphere cluster, creating a dedicated vCenter server, the vSAN datastore, and virtual network based on

the underlying configuration parameters specified by the administrator. Workload domains are a policy-driven approach for defining

performance, availability, and security parameters. SDDC Manager automatically implements a deployment workflow to translate the

workload domain specifications into the underlying pool of resources. Through the automation of tasks and workflows, that SDDC

Manager simplifies the provisioning, monitoring, and ongoing management of both the logical and physical resources of the VxRack

SDDC.

In the VxRack SDDC, system management utilities are designed to function via a management domain. The management domain is a

special purpose workload domain which runs Cloud Foundation infrastructure components including vCenter Servers, PSCs, NSX

Managers, SDDC Manager components, LCM, vR Ops, vR LI. The management domain normally consumes the first four nodes in rack

one. See Figure 6.

Figure 6: SDDC Manager abstracts and aggregates physical resources into workload domains.


13

The management cluster runs two virtual machine appliances that make up the SDDC Manager management services as shown in

Figure 7: The SDDC Manager Controller VM and the SDDC Manager Utility VM. These VM’s run the “system” level management

services. The SDDC Manager Controller VM runs the SDDC Manager user interface (UI), Hardware Management, Physical and Virtual

Resource Manager services, datastore and coordination services, and lifecycle management services.

The Hardware Management Service (HMS) runs as a service inside the SDDC Manager Controller VM instead of on the physical

management switch within the system. This service is responsible for being the translation layer to the physical hardware infrastructure

enabling SDDC Manager to perform operations such as configuring the VLANS on physical switch ports for example.

The datastore and coordination services track system inventory, coordinate activities, and provide a persistent datatore for the system

metadata and configuration information.

The SDDC Manager Utility VM is used for lifecycle management operations and is responsible for storing patches and upgrade bundles

as well as applying these updates to the workload domains. As new updates become available admins are notified to schedule a time

to download the updates and apply them. With Cloud Foundation, individual workload domains can be updated independently. The

entire patch/upgrade process for Cloud Foundation components is fully automated using SDDC Manager.

Other management components that run in the management domain include a dedicated vCenter server and a pair of Platform

Services Controllers (PSCs). Storage is provided by vSAN and NSX Manager and NSX Controllers are also deployed. A clustered

instance of vRealize Log Insight will also run on the management domain as well as other optional software such as vRealize

Operations.

In VxRack SDDC, a Cloud Foundation Standard Architecture is used. This means that the management domain cluster is physically

separated as its own vSphere cluster to run only system management workloads. By default, the first 4 nodes in Rack 1 of the system

will be reserved for the management domain cluster. User workloads would be then be deployed using physically separate VI/VDI

workload domain clusters.

While the deployment and configuration of the management cluster is fully automated, once running, management operations for the

workload domain clusters are just like any vSphere cluster using the vSphere Web Client.

Figure 7: Management Domain


14

All workload domains (VI/VDI and management) also include a dedicated NSX Manager and cluster of 3 NSX Controller VM’s. While

the NSX Managers for all workload domains get deployed and run on the management domain, the NSX Controllers for a give workload

domain will run on the workload domain cluster itself.

Specifically, for VDI workload domains, the desktop management software such as Horizon and AppVolumes will also be deployed on

the management domain cluster leaving the workload domain cluster to run the NSX controllers, other NSX components and desktop

VM’s themselves. Figure 7 illustrates the components that run on the management domain cluster.

LIFECYCLE MANAGEMENT

SDDC Manager also streamlines and automates lifecycle management for VxRack SDDC components as shown below in Figure 8.

Given the complexities of validating new firmware and patches in an interconnected, hyper-converged infrastructure, conventional

methods of performing upgrades and patches are unfortunately prone to configuration and implementation errors. SDDC Manager

automates upgrades and patch management for the logical infrastructure. Lifecycle management can be applied to the entire

infrastructure or to individual workload domains. vMotion and DRS allow administrators to perform upgrades while the system and the

VMs remain up and running. VxRack SDDC comes with 24/7 world-class support and includes physical and logical lifecycle

management for the entire system, both the hardware and software, through a single Dell EMC phone number.

Figure 8: SDDC Manager Lifecycle Management for VxRack SDDC

vREALIZE SUITE—VMWARE’S CLOUD MANAGEMENT PLATFORM (CMP)

Most software-defined data centers will consist of a workloads mix of traditional and modern application architectures. They will be

provisioned in an increasingly virtualized mix of physical and virtual environments managed both on-premises in private clouds and in

off-premises in public clouds. The concept of a cloud management platform has evolved as a response to this complex set of

management requirements. Available as a separate purchase, but fully supported to work with a VxRack SDDC environment VMware’s

vRealize cloud management platform delivers the management capabilities to effectively manage the complete lifecycle of services

delivered in a private cloud environment.

VMware’s vRealize cloud management platform includes:

vRealize Operations which provides intelligent health, performance, capacity, and configuration management. vRealize Operations

offers performance and health monitoring and capacity planning for the VxRack SDDC as well as custom dashboards, capacity

modeling, and customized alerting. These insights help administrators maintain compliance and efficiently detect and resolve any

operational issues that may arise.


15

vRealize Log Insight which provides real time log management and log analysis. vRealize Log Insight lets administrators monitor

physical and virtual infrastructure to avoid failures and performance issues. vRealize Log Insight provides centralized log aggregation

and analysis with search and filter capabilities for the entire VxRack SDDC system. This provides the ability to monitor all workload

domains from a single place. Each VxRack SDDC system contains a management domain cluster with an instance of the vRealize Log

Insight virtual appliance for management domain components. This cluster covers logging for the management domain only. When

deploying VxRack SDDC, vRealize Log Insight is deployed by default and is included as part of Cloud Foundation for use with

management domain only. Optional licensing can be purchased enabling Log Insight to be configured to receive and process log

events for all workload domains in the system

vRealize Automation which automates delivery of personalized infrastructure, applications and custom IT services.

vRealize Business for Cloud which automates costing, usage metering, and service pricing of virtualized infrastructure.

The vRealize Suite can be purchased with VxRack SDDC. As previously mentioned, Log Insight is deployed by default at time of

system installation. The rest of the vRealize Suite components are optional add ons. These are not deployed by SDDC Manager today,

but are planned for a future release of Cloud Foundation.

HORIZON SUITE—VMWARE’S VDI AND APP VIRTUALIZATION PLATFORM

Most IT environments offer end-user computing as a service. VMware Horizon Suite, the platform for workforce mobility, connects end

users to their data and applications on any device without sacrificing IT security and control. IT can transform technology silos of

desktops, data and applications into centralized IT services—and improve operational efficiency, security and agility through policy-

based management of those services. Horizon Suite is the market-leading desktop-virtualization, end user computing solution and

includes Horizon View and App Volumes. Horizon can be easily integrated and supported on the VxRack SDDC.

VMware Horizon View (Horizon) is VMware’s VDI and desktop-management environment. Horizon provisions user desktops using a

flexible and secure delivery model. The desktop environments are accessed by the user from almost any device, including mobile

devices, with the security and resiliency of the datacenter. Because the application software and data components reside in the

datacenter, traditional security, backup, and disaster recovery approaches may be applied. If a user's device is lost or the hardware

fails, the recovery is straight forward. The user simply restores the environment by logging in using another device. With no data saved

on the user's device, if the device is lost or stolen, there is much less chance that critical data could be retrieved and compromised.

Figure 9 below shows how Horizon View encapsulates the OS, applications, profiles, and user data into isolated layers and dynamically

assembles desktops on demand to provide users with a personalized view of their individual environments.

Figure 9: Highly Available and Secure Desktops


16

Availability and security, along with ease of management and support, are compelling reasons for moving from traditional physical

desktops and laptops to VDI.

VMware App Volumes provides application and user management and monitoring with enterprise-scale capabilities across virtual

desktop and published/remote application environments powered by VMware Horizon. VMware App Volumes supports real-time

application delivery to virtualized desktop environments. With Horizon 7 and App Volumes, IT can build a real-time application delivery

system that ensures all applications are centrally managed. Applications are delivered to virtual desktops through VMDK virtual disks

without modifying the VM or applications themselves and can be scaled out to virtual desktops with superior performance, at lower

costs and without compromising end-user experience. These are deployed as part of a VDI workload domain on the VxRack SDDC.

HARDWARE CONFIGURATION

VxRack SDDC implements an integrated, scalable hardware architecture (Figure 10), ready to deploy as a complete SDDC

infrastructure. Minimum configurations begin with eight nodes and scale to 192 nodes in eight racks.

Figure 10: VxRack SDDC Integrated and Scalable Architecture

The VxRack SDDC system includes industry-leading Cisco Nexus IP 9000 series network switches for ToR and rack interconnect

switches and Dell PowerEdge servers that run the VMware SDDC stack: ESXi hypervisor, NSX software-defined networking (SDN),

and vSAN software-defined storage (SDS).

A VxRack SDDC configuration contains a set of compute and storage components as well as fixed network resources. The minimum

single-rack configuration has eight server nodes, four of which are reserved for the management domain cluster, one Dell Networking

S3048-ON switch for management and a pair of Cisco Nexus 93180YC-EX switches. Storage is part of the hyper-converged

infrastructure and based solely on the associated nodes. Storage for the default four node management cluster is provided by vSAN

using a highly available configuration. The remaining four nodes that make up the eight node minimum in a VxRack SDDC system can

be leveraged to expand the management domain cluster, to more than the four node default or can be used as a separate workload

domain cluster entirely.

Multiple-rack configurations add a pair of Nexus 9332PQ switches in the second rack as a rack interconnect layer backbone. Two

switches maximum are needed for each system and are located in the second rack. This provides inter-rack connectivity at multiple

40GB links. The server or node connectivity to the ToR switching is 10GB via the Nexus 93180YC-EX, and these switches


17

accommodate customer uplinks (10Gbps or 40Gbps). Customers can dynamically scale VxRack SDDC as demands increase. To

simplify future expansion, best practice is to size the initial configuration with the maximum number of ports and include cross-rack

networking.

HARDWARE CONFIGURATION

The VxRack SDDC includes the following hardware components:

Panduit cabinet (or rack) with an IPI appliance and two PDUs

Support for a maximum configuration of 192 Dell PowerEdge R630 servers (or 8 fully populated racks)

o Minimum of eight server nodes per rack

o Maximum of 24 server nodes per rack

Two Cisco 93180YC-EX Top of the Rack (ToR) switches per rack

Two Cisco 9332PQ rack interconnect switches included in multi-rack configurations only

One Dell Networking S3048-ON switch per rack for management

HARDWARE COMPONENTS

VxRack SDDC takes advantage of the most reliable enterprise-quality server hardware from Dell and the industry’s top network

switches from Cisco. VxRack SDDC uses Panduit intelligent cabinets for rack hardware, and system components are pre-installed and

configured within the rack.

CABINETS

In each VxRack SDDC, the compute and network-layer components are pre-installed and distributed within Panduit cabinets as shown

in Figure 11. Distributing the components balances out the power draw and reduces the size of the required PDUs. This distributed

design improves flexibility during upgrades or expansion. In addition, each Panduit cabinet has an Intelligent Physical Infrastructure

(IPI) appliance which allows users to collect and monitor environmental data, power, and security.

Figure 11: VxRack SDDC uses Panduit cabinets equipped with an IPI appliance.


18

SERVER NODES

VxRack SDDC nodes are high-density, two-socket Dell PowerEdge R630 servers (picture in Figure 13) which utilize the latest Intel

based CPUs. These single unit (1RU), rack-mount servers come in a choice of configurations with either hybrid or all-flash storage.

Currently, hybrid and all-flash storage cannot be mixed in the same VxRack SDDC.

Front View

Rear View Figure 12: Dell PowerEdge 630 VxRack SDDC Node

SERVER NODE STORAGE OPTIONS

Local disks within the Dell PowerEdge R630 Server provide storage in the VxRack SDDC. Each server has direct attached storage

(DAS) that becomes aggregated by VMware vSAN to create a single datastore shared among all the VMs running on the VxRack

SDDC Workload Domain clusters. Each node contains ten disks in two separate vSAN disk groups configured either as a hybrid

storage with two Solid State Drives (SSDs) for the cache tier and eight Hard Disk Drives (HDD) for capacity tier or as all-flash storage

with two SSDs for the cache tier and eight SSDs for the capacity tier.

VXRACK SDDC ENCLOSURES SPECIFICS (ALL FLASH)

BASE PLATFORM PowerEdge R630

FORM FACTOR 1U1N

CPU 5th generation Intel™ Xeon™ E5-2600 Family

CORES PER CPU 6 - 22

MEMORY GB 768 GB – 1536 GB

TOTAL RAW CAPACITY (TB) 15 TB – 30 TB

CAPACITY STORAGE DEVICE 1.92 TB SSD 1WPD 2.5" or 3.84 TB SSD 1WPD 2.5”

NETWORK Dual 10 GbE SFP+

NIC Intel Ethernet X520 DP 10Gb DA/SFP/+ I350 DP

STORAGE CONTROLLER PERC H730 mini Integrated RAID Controller

CACHE SSD Dual 800 GB 10WPD 2.5"

BOOT DEVICE 64 GB SLC SATADOM

TPM Optional: TPM 1.2 pre-installed or no TPM

POWER SUPPLY Dual 750W PSU 100-240VAC


19

Figure 14 lists current VxRack SDDC node configurations with Dell PowerEdge R630 as of the writing of this paper. As Dell EMC

continues to qualify and build new Dell EMC PowerEdge systems these choices will grow. Check with your Dell EMC account team for

any new VxRack SDDC node configurations.

VXRACK SDDC ENCLOSURES SPECIFICS (HYBRID)

Base Platform PowerEdge R630

Form Factor 1U1N

CPU Dual E5-2680v4 2.4 Ghz

Cores per CPU 14

Memory GB 384 - 768

Total Raw Capacity (TB) 10 TB

Capacity Storage device 1.2 TB HDD 10K 2.5"

Network Dual 10 GbE SFP+

NIC Intel Ethernet X520 DP 10Gb DA/SFP/+ I350 DP

Storage Controller PERC H730 mini Integrated RAID Controller

Cache SSD Dual 800 GB 10WPD 2.5"

Boot Device 64 GB SLC SATADOM

TPM TPM 1.2 pre-installed or no TPM

Power Supply Dual 750W PSU 100-240VAC

Figure 13 VxRack SDDC Node Configuration Table

NETWORK SWITCHES

Physical networking for a VxRack SDDC consists of a rack interconnect topology with top-of-rack (ToR) switches and a single

management switch. Each physical rack contains two physical network switches that control network traffic, provide redundancy, and

manage out-of-band connectivity. With scale-out across multiple racks, east-west traffic is fully self-contained. Two interconnect rack

switches provide connectivity between racks. VxRack SDDC comes with Cisco Nexus 93180YC-EX switches installed in pairs at the

top of the rack.


20

Expansion racks require a pair of switches in the second rack (and only the second rack) to provide connectivity for inter-rack east-west

traffic. Both the Nexus 93180YC-EX top-of-rack switches and the Nexus 9332PQ rack interconnect switches have two links to the Dell

S3048-ON management switch to communicate with out-of-band components. Both the Nexus 9372PX-E switches and the Nexus

9332PQ switches have two links to the Dell S3048-ON management switch to communicate with out-of-band components. The table in

Figure 14 lists each VxRack SDDC network switch and their functions.

Switch Switch Type Function Ports

Cisco Nexus 93180YC-EX Top of Rack / ToR Switch Deployed in pairs in each rack

Northbound traffic to customer network is through these ToR switches in the 1st rack only

Customer data uplink

48x 1/10/25GbE + 6x

40Gb/100GbE

One 40Gbps QSFP per

93180YC-EX switch to

connect to each of the two

rack interconnect switches

in multi-rack configuration

Four 10GbE links between

each pair of 93180YC-EX

2x 10GbE links between

server and each 93180YC-

EX

For 93180YC-EX in the 1st

rack only - up to four

10GbE or 40GbE links to

customer data center

switches

Cisco Nexus 9332PQ Rack Interconnect Switch

(multi-rack configurations)

Inter-rack communications

Deployed in a pair in 2nd rack only

Only 2 needed per VxRack SDDC system

32x 40GbE

1x 40Gbps QSFP to

connect to each of the

9372PX-E ToR switches

2x 40Gbps QSFP to

connect to the other

9332PQ

Dell Switch S3048-ON Management Switch Single switch deployed on each rack

48x 1GbE + 4x 10GbE

1x 1GbE link to each

server nodes, 9372PX-E

and 9332PQ (if exists) in

the same rack

1x 10GbE link to each

9372PX-E in the same

rack

Figure 14: VxRack SDDC Switches

The ToR switches in the first rack connect to the data center aggregation switch using up to 4x10 or 40Gbps links per 93180YC-EX.


21

NETWORK TOPOLOGY

Each rack configuration uses a pair of Cisco Nexus 93180YC-EX ToR switches and a single Dell Switch 3048-ON management switch.

VxRack SDDC nodes connect to the ToR switches through 10G SFP+ (one per switch) to communicate with the in-band components of

the infrastructure. The 93180YC-EX switches have two links to the management switch to communicate with the out-of-band

components. Expansion rack(s) requires a single pair of Cisco Nexus 9332PQ switches for the rack interconnect network. These reside

only in rack #2 and allow connectivity between all racks. Each expansion rack also has two Cisco Nexus 93180YC-EX ToR switches

that connect to the Nexus 9332PQ rack interconnect switches in rack #2.

The server nodes connect to the Ethernet component of the network layer. The southbound interconnects link to the Cisco Nexus

switches in the Ethernet network through 10GbE port channels. All racks are part of the same VxRack SDDC management domain, up

to the supported maximum scale-out configuration of 192 nodes, or eight racks.

The uplinks used to connect to a customer’s datacenter network in rack 1 can be configured using L2 connectivity. This is the preferred

and default method used for VxRack SDDC systems. However, if required, connecting the uplinks using L3 is also available.

NETWORK TOPOLOGY FOR SINGLE-RACK CONFIGURATION

The diagram in Figure 15 illustrates the single-rack configuration network design.

Figure 15: Single Rack Network Design

ToR Pairs Cisco ToR Switch

93180YC-EX

Cisco ToR Switch

93180YC-EX


22

NETWORK TOPOLOGY FOR MULTI-RACK CONFIGURATION

The diagram in Figure 17 illustrates the multi-rack configuration network design.

Figure 16: Multi-Rack Network Design

TRADITIONAL AND RACK INTERCONNECT NETWORK ARCHITECTURE

In a conventional data center network hierarchy, hosts at the access layer connect to network switches which, in turn, rely on routing

services (data center services as well services outside the data center, including the Internet) from a third tier. This traditional core-

aggregate-access (three-tier) network model is efficient for traffic that travels “north-south,” which is traffic that travels in and out of the

data center. North-south traffic typically has a lot of remote client/server communication. This traditional network architecture is usually

built for redundancy and resiliency and is still very widely used for service-oriented types of traffic that travel north-south. However, the

trends in traffic patterns are changing as the workloads in today’s data centers are changing. Traffic in the new SDDC data center

typically is east-west traffic, or server-to-server traffic. This traditional network model becomes prone to bottlenecks for highly scalable

SDDC infrastructures.

Figure 17 shows the traditional network data center topology with a three-layer architecture: the access layer, where users connect to

the network; the aggregation layer, where access switches intersect; and the core, where aggregation switches interconnect to each

other and to networks outside of the data center. In this traditional network architecture, if a server connected to the left-most access

switch needs to communicates with a server connected to the right-most access switch, this “east-west” communication must travel all

the way to the core switch and back down again. Clearly this is not the most efficient path and will cause more latency while consuming

more bandwidth.

Rack #2

9332PQ Rack #2

9332PQ

Rack Interconnect Switches in Rack 2

Cisco ToR Switch

93180YC-EX

Cisco ToR Switch

93180YC-EX

Cisco ToR Switch

93180YC-EX

Cisco ToR Switch

93180YC-EX


23

Figure 17: Traditional Network Architecture

In modern SDDC data centers, compute and storage infrastructure alterations change the predominant network traffic patterns from

“north-south” to “east-west”. To address network bottleneck because of the shift from north-south to east-west traffic, one solution is to

create a rack interconnect architecture. This architecture has two main components: rack interconnect switches and top of rack (ToR)

switches as shown in Figure 18.

Figure 18: Top of Rack and Rack Interconnect Network Architecture

Rack interconnect switches are composed of high-throughput Layer 3 switches with high port density. ToR switches are like the access

layer; they provide network connection points for servers, as well as uplink to the rack interconnect switches. The most important part of

Rack Interconnect Switch Pair

ToR Switches


24

this architecture is that every ToR switch connects to both rack interconnect switches in rack two. This point is important because no

matter which ToR switch a server is connected to, it always has to cross the same amount of devices to get to another server (unless

the other server is located on the same rack). This keeps the latency down to a predictable level and provides much greater overall

bandwidth.

VxRack SDDC uses a unique network design. In Cloud Foundation all the physical racks are on the same L2 network. The ToR

switches are dual connected to the two rack interconnect switches through redundant 40Gbps links configured in a Multi-Chassis Link

Aggregation Group (MC-LAG). Spanning Tree Protocol is not used because looping is avoided using MC-LAG.

As an option, Layer 3 (northbound traffic) is only supported with the Cisco Nexus 93180YC-EX switches in the first rack. SDDC

Manager configures a Switched VLAN Interface (SVI) for each requested VLAN and configures a static route between the two

93180YC-EX ToR switches in the first rack and the upstream router. Full redundancy is accomplished by setting up iBGP between ToR

switches and an eBGP between each of the two 93180YC-EX ToR switches in the first rack and the upstream router.

VxRack SDDC with VMware Cloud Foundation integrates with an existing data center network infrastructure and provides full uplink

compatibility to existing switches such as Cisco, Juniper, and Brocade. L3 or L2 uplink connectivity is supported, however, L2 is

preferred. The VMware SDDC Manager provides support for configuring, controlling, and managing the physical network elements, and

software defined networking is delivered through VMware NSX.

In Figure 19 VxRack SDDC with VMware Cloud Foundation physical switches features are shown. Each rack contains two ToR

switches and all hosts (Server 1…24) in the physical rack are dual connected to these two ToR switches with 10Gb links. Each NIC on

all hosts is connected to one of the two ToR switches in a Multi-Link Aggregation (MLAG) bond configuration. MLAG is a type of link

aggregation group (LAG) with constituent ports that terminate on separate chassis, primarily for the purpose of providing redundancy in

the event one of the chassis fails. They operate as if they are connected to a single, logical switch. The BMC port on each host is

connected to the management switch over a 1G connection used for out-of-band (OOB) management. The ToR switches are

connected to each other over 2 links for control traffic and redundancy. The ToR switches are further connected to the rack interconnect

switches in a dual-MLAG (Multi-Link Aggregation) configuration. The data path between the hosts across multiple racks can tolerate a

failure of one link or one ToR switch or one rack interconnect switch using link aggregation between hosts and ToR switches and also

between ToR switches and rack interconnect switches. For uplink connectivity, ToR switches are connected to the existing LAN

environment using 40 or 10 GbE with MLAG/Channel over 160 GbE.

Figure 20: VxRack SDDC powered by Cloud Foundation Physical Networking

Rack Interconnect Switches

ToR ports 49, 50 for Interconnect switches


25

CONCLUSION

The modern CIO knows there are conflicting priorities for the IT organization: Quickly adopt new technology, but minimize technical

debt; be strategic and focus on business objectives, but maintain operational excellence; encourage innovation, but stay secure and

maintain compliance. The overarching challenge in the cloud era: Support business need for speed and agility, while maintaining

control and reducing IT costs.

The IT market is changing faster than ever, driven by continuous acceleration of business and modern data center initiatives. Business-

as-usual no longer works, and hyper-converged infrastructure (HCI) and cloud computing are now cornerstones of every business’

digital and IT transformation strategy. IT organizations undertaking modernization and transformation efforts are looking to use

technology to reduce costs as well as gain competitive advantage. The Dell EMC VxRack SDDC is a turnkey HCI engineered

infrastructure system with single vendor support and lifecycle management. It is the fastest and simplest way to deploy a VMware

Cloud Foundation based SDDC infrastructure. VxRack SDDC is unique in the industry and is offered exclusively by Dell EMC as a co-

engineered system with VMware. The VxRack SDDC powered by VMware Cloud Foundation delivers a rack scale hyper-converged

system that simplifies the journey to the private cloud.

Documents

DELL EMC VXRACK SYSTEM SDDC TECHNOLOGY OVERVIEW€¦ · Dell EMC and VMware share this vision and together are developing new technologies to make this vision a reality for customers