10,000-User Reference Implementation with Atlantis ILIO ... · and Atlantis ILIO Diskless VDI™ in a large-scale deployment, including the specific implementation decisions made

10,000-User Reference Implementation with Atlantis ILIO™ and VMware® Horizon View™ at a Global Service Provider

10,000-User Reference Implementation with Atlantis ILIO and VMware Horizon View at a Global Service Provider

2ATLANTIS ILIO / VMWARE HORIZON VIEW

Table of Contents

About Reference Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Deployment Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 VMware vSphere Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Compute Platform Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Server Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Desktop Workload Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 VDI Management Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Atlantis ILIO Diskless VDI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Stateless Desktops with Atlantis ILIO Diskless VDI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Server Sizing – User Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11ESXi Host Configuration and Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Hyper-Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 DNS and Network Time Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Host Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Network Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Logical Network Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Shared Storage Architecture and High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Image and Template Datastores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 High Availability (HA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17



About Reference ImplementationsReference implementations describe how particular customers have architected their virtual desktop environment and deployed VMware® Horizon View™ . As compared to reference architectures, reference implementations focus on actual customer environments and include deployment information regarding each layer, from client access devices, access infrastructure, virtual infrastructure, VMware Horizon View desktops, and session management .

Executive SummaryThis reference implementation illustrates the architecture a global service provider used with Horizon View and Atlantis ILIO Diskless VDI™ in a large-scale deployment, including the specific implementation decisions made to meet customer requirements .

The reference implementation:

• Delivers a fully stateless virtual desktop platform using Horizon View and Atlantis ILIO Diskless VDI

• Has a modular design that can scale to 10,000 users and beyond

• Gives the end user the best possible experience

• Delivers the most cost-effective desktop from both a CapEx and OpEx standpoint

• Provides a suitable level of redundancy to cope with disaster recovery

Deployment ArchitectureThe architecture is described by a logical design layout that is independent of hardware-specific details, as shown in Figure 1. Specifications of physical design components chosen for the logical design are also provided .

Architecture OverviewThe logical architecture follows the principles of the Horizon View pod and block approach—modular, scalable and repeatable blocks of management and desktop infrastructure .

This architecture includes:

• Two datacenters (Datacenter1 and Datacenter2) in an active-active configuration. In a Disaster Recovery (DR) scenario, 55 percent of users would continue to be serviced in the event of datacenter loss .

• Clusters of VMware vSphere® hosts supporting desktops for load balancing through the View Connection Servers .

• Considerations for storage and network requirements, including redundancy .



Datacenter 2Datacenter 1

Remote Access Layer

Management Layer

Desktop Compute Layer

Networking Layer

Storage Layer

Load BalancerSecurity Server Security ServerLoad Balancer

vCenterServer(s)

Print Database

Stretched Management vSphere Cluster

DHCP

ILIO Centers

Pro�le Management

Connection Servers

File Servers Pro�le Store Pro�le Store File Servers

DHCP Database Print vCenterServer(s)

Connection Servers

Pro�le Management

ILIO Centers

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

12 ESXHosts

StatelessDesktop

Block

VMware ESX

Figure 1: Architecture Overview



VMware vSphere Design The virtual infrastructure chosen in this architecture is VMware ESXi™ 5 .0 Update 1 . The ESXi layer is based on an approach that utilizes a single ESXi host design for both the desktop cluster and the management cluster . Figure 2 shows how the hosts would be used in this implementation, and the cluster management layout . Virtual desktops are hosted on desktop clusters running only virtual desktops and the Atlantis ILIO Diskless VDI virtual machine .

The server infrastructure required for management and the View Connection Server implementation are hosted on a separate cluster, known as the VDI management block . Exceptions to placement within the VDI management block are the load balancers and the remote access and Security Server, which are hosted in a vSphere cluster located in the DMZ . The VDI management block itself should be managed by existing vSphere infrastructure in the enterprise, thus avoiding managing vSphere from within itself .

Datacenter 1

Datacenter 2Existing ESX

Infrastructure

Manage

Manage

Manage

Connection Server

vCenterServer(s)

Connection Server

vCenterServer(s)

VDI Management

Block

vCenterServer(s)

VMware ESX

VMware ESX

StatelessDesktopClusters

StatelessDesktopClusters

Figure 2: vSphere Cluster Design



Compute Platform DesignThis reference architecture are designed using the Cisco UCS B230-M2 half-width blade .

The assembly, build, and configuration of each system have been standardized, with all components installed identically for all ESXi hosts. It is critical to standardize not only the model but also the physical configuration of the ESXi hosts . Standardization provides a manageable and supportable infrastructure by eliminating variability in such a large-scale deployment . A consistent hardware platform across the infrastructure also allows for a host to take the role of any other host without the need to consider its hardware specification, thereby offering a greater level of flexibility. The configuration for the hardware platform is detailed in Table 1.

ATTRIBUTE VDI STATELESS DESKTOP HOSTS SPECIFICATION

Vendor Model

Cisco UCS B230-M2

Number of CPU sockets Number of CPU cores Total number of cores Processor speed

2 8 16 2.70GHz

Memory 384GB

Network ports (outbound) Network speed

2 10 GbE (per port)

Number of local drives Total useable capacity

0 0

Datastores 10GB Boot from SAN LUN 10GB Atlantis ILIO OS LUN

Hypervisor version ESXi 5 U1, Build: 623860

Table 1: ESXi Server Specification

Server SizingTo determine the required number of ESXi hosts needed to accommodate the 10,000 stateless desktops (5,000 in each of the two datacenters), the service provider would collect performance and utilization data of existing users. If it is not possible to collect data from users (say, a greenfield deployment), then a production pilot of at least 30 days active use by users would provide the metrics for average and peak CPU, memory, network, storage, and IOPS utilization .

Taking into account the historical data collected by monitoring utilization and workloads through pilot or pre-production rollouts, the number of virtual machines per blade can be increased to meet demand In this deployment, 20 percent headroom has been left free on each host to have the capacity necessary should a host fail in a cluster or a larger DR event occurs .



Desktop Workload Assessment A typical user in this architecture runs a core set of office applications and requires 1.5GB RAM and 255MHz CPU to run their desktop at an acceptable performance level . Historically, workers have had a 7 .5K SATA drive in a physical machine, so drive performance would peak at 80 IOPS with an average consumption of 30 IOPS for an 80 percent write and 20 percent read split during steady state .

Table 2 summarizes key virtual desktop machine requirements of a typical user in this environment .

ATTRIBUTE VALUE

RAM 1.5GB

CPU 255MHz average

IOPS 30

Read/write ratio ~20 percent read, 80 percent write

Average throughput per virtual machine 487Kbps – 1,660Kbps

Total Throughput per LUN (50vms) 24,350Kbps – 83,000Kbps (23.8Mbps – 81Mbps)

Table 2: Summary of Key Virtual Machine Resource Requirements

Table 3 summarizes the CPU requirements and details the CPU consumption for each of the ESXi hosts to support the workload of 10,000 stateless virtual desktop machines .

DESKTOP PERFORMANCE METRIC RECORDED VALUE

Number of CPUs (sockets) per host 2

Number of cores per CPU 8

MHz per CPU core 2,700MHz

Total MHz per host 43,200MHz

Proposed maximum host CPU utilization 80 percent

Usable CPU MHz per host 34,560MHz

Atlantis ILIO CPU cores ‘overhead’ 2 (1 Reserved – 2700 MHz)

Average number of vCPUs per physical desktop system

1

Average CPU utilization per physical desktop system 255MHz

Number of virtual machines per host 100 (110 during host failure)

Total average CPU utilization per host 26,775MHz(29,325MHz during 1 host failure)

Total CPU requirement for 1 host, including additional 15 percent for virtualization considerations (display protocol, VMware Tools™, Atlantis ILIO, etc.)

33,491MHz per host(36,424MHz during 1 host failure)

Table 3: Hypervisor CPU Requirements



Table 4 summarizes the RAM availability and details the overall RAM requirements for the ESXi hosts to support the workloads of the Windows 7 virtual machines with Atlantis ILIO Diskless VDI . The key sizing component for an Atlantis ILIO Diskless server is RAM . Table 4 outlines how the blade was sized for this architecture to support 110 virtual machines (worst case for DR) .

DESKTOP PERFORMANCE METRIC RECORDED VALUE

ESXi host memory 2GB

Atlantis ILIO OS 6GB

Observed average memory utilization per desktop

1.5GB (1.05GB after transparent page sharing)

Hypervisor overhead per virtual machine 0.03GB (30.72MB)

Atlantis ILIO RAM per virtual machine 1.1GB

Number of virtual machines per host 100 (110 during 1 host failure)

Maximum number of replicas per datastore 3

Total replica size (3) 60GB

Total virtual machine runtime memory 171GB

Total Atlantis ILIO memory 191GB (allowance for 110 virtual machines)

Total host memory consumption 335GB (362GB during a host failure) Table 4: Hypervisor RAM Requirements

VDI Management BlockIn addition to the ESXi hosts required to support desktop virtual machines, a number of hosts are required for the VDI management block . They provide the Connection Servers and other management infrastructure required to deliver the VDI platform .

Eight Cisco UCS B230-M2 ESXi hosts are used in the management block . The block is a stretched vSphere cluster over the two datacenters with four hosts in each. This allows virtual machines to benefit from HA, DRS, and VMware vSphere vMotion® providing a highly available management platform .

The VMware vCenter™ server managing the VDI management block should be hosted external to the environment on existing ESXi infrastructure . This prevents the VDI management block’s vCenter server from being affected by host failure in the environment .

Atlantis ILIO Center is required to manage and update the individual Atlantis ILIO appliances . Atlantis ILIO Management Center should be able to communicate with the vCenter server managing the Atlantis ILIO appliances . A single ILIO Center is required per vCenter instance .



Atlantis ILIO Diskless VDIAtlantis ILIO is a virtual appliance that accelerates storage IO, provides compression and performs inline data de-duplication . In this design, each host contains an Atlantis ILIO virtual appliance, which presents an NFS datastore for the vSphere host .

Stateless Desktops with Atlantis ILIO Diskless VDIAtlantis ILIO Diskless VDI stores stateless virtual desktops on the datastore, which uses local server RAM as primary storage . The Atlantis ILIO virtual appliance presents a standard NFS datastore through the vSphere hypervisor. Datastores for the desktops reside entirely in physical memory which provides a significant performance gain over shared storage even when compared to a local solid-state disk .

Atlantis ILIO is deployed onto each blade allowing for a modular, scalable solution . It is provided in an OVF format, imported into vCenter and placed onto the appropriate blades using Atlantis ILIO Deployment Services to automate the deployment, installation, sizing and configuration of the ILIO virtual machines. The user data contained in the OS drive of the virtual desktops is placed onto a datastore on the shared storage platform .

VMware

ILIO Datastore

Master

Virtual Machine Clones

NFS

Shared Storage

ILIO OS

VMFS

OS

APP

VMwareESXi

Boot

Figure 3: Atlantis ILIO Diskless VDI Deployment



This deployment and configuration can be fully automated for Atlantis ILIO Diskless VDI deployment through PowerShell API or the use of Atlantis ILIO Deployment Services . This automation allows deployment and update of Atlantis ILIO components as required from Atlantis ILIO Center or a single script and configuration file.

The initial setup and configuration is documented in Table 5. If a manual build of ILIO were to be required, the settings in Table 5 would be set on the ILIO appliance before powering on the appliance .

ATTRIBUTE DETAIL SPECIFICATION

CPU vCPUs allocated 2

vCPUs reserved 1 – 2700MHz

Advanced CPU HT sharing – none

Memory 193GB

Network vNICs 2

Adapter type E1000

Networks NFS network VLAN Management VLAN

Disk VMFS datastore (shared storage) 10GB (Atlantis ILIO OS disk)

Table 5: Atlantis ILIO Appliance Specifications

After the Atlantis ILIO appliance is powered on, use the configuration settings in Table 6 during setup.

ATTRIBUTE SPECIFICATION

IP addressing Eth0: NFSNet Eth1: MgmtNet

Hostname ILIO-xxx-xxx

Geographic area United States

Timezone PDT

Keymap PC/QWERTY/US

Setup Standard

Compression Disabled

Export type NFS Table 6: Atlantis ILIO Appliance Configuration

After the Atlantis ILIO appliance has been configured, the storage is presented back to the ESXi host through a new NFS datastore . A CNAME is used to mount the datastore to enable changing IP addresses on Atlantis ILIO, if required, without having to un-mount and remount the datastore. Table 7 lists the datastore specifications (on a per ESXi host basis) .



ATTRIBUTE SPECIFICATION

Datastore type NFS

Server ILIO-xxx-xxx

Mount point /exports/ILIO_VirtualDesktops/

Datastore name ILIONFS-xxx

Table 7: Atlantis ILIO NFS Datastore Specification

Server Sizing – User DensityA synthetic workload generator was used to validate the reference implementation design and establish the maximum number of users that a single server could support for a given CPU and memory capacity . This test simulates a workload running a set of applications, including office productivity applications, Internet browsers, PDF readers, and multimedia tests. Testing was run on a blade with the specifications detailed in Table 1, but with 512GB of memory . This test concluded that 171 users could be supported on the server while maintaining an acceptable end-user experience . The number of users that could be supported with different memory amounts was also tested . With 384GB of memory, 115 users were possible . Taking into account other high-availability requirements within the environment, steady-state blade density was set at 100 users .

The charts in Figure 4 and Figure 5 show memory and CPU usage during the tests .

Figure 4: Server Memory Consumption During Test



Figure 5: Server CPU Consumption During Test



ESXi Host Configuration and OptimizationESXi was installed on SAN disks to boot from SAN for the server and take advantage of Cisco UCS Server Profiles. Server Profiles ease the management of hosts and enable host identity to be moved when required. As Boot from SAN is the preferred solution, no manually configured local storage partitions are required or created. Boot LUNs should only be masked to the individual host as opposed to the cluster configuration of the datastore LUNs. A separate VLAN to isolate boot traffic from virtual machine traffic is also recommended.

Hyper-ThreadingESXi supports Intel Hyper-Threading (HT) Technology on servers with Intel processors . HT improves processor performance by taking advantage of additional CPU interrupt controllers and registers, thereby enabling slightly higher utilization levels across the virtual infrastructure . Generally, a small improvement in performance can be gained by using HT . This performance gain is achieved by doubling the number of logical processors in an ESXi host . HT should be enabled as a default to offer a performance increase .

Note: HT improves performance by supporting concurrent thread processing on the same physical CPU to take advantage of idle thread cycles . It is important to recognize that any performance increase will not be equivalent to adding more physical processors . In very rare cases, enabling HT can actually hinder performance . To protect against this, performance should be monitored after changing this setting .

By default, some servers throttle the CPU at the BIOS level and hamper virtualization as a server will not be able to use 100 percent of the CPU . This has a direct impact on not only the performance of all desktops but also of Atlantis ILIO . Ensure that Power Management and CPU Power and Performance are both set to Maximum Performance in the BIOS .

DNS and Network Time ProtocolDNS and NTP should be configured appropriately on each ILIO and ILIO Center appliance. DNS configuration allows the Atlantis ILIO appliance to automatically register with ILIO Center. Configuration of a consistent set of NTP servers across the environment allows for consistent reporting and debugging . Performance graphs and statistics on ILIO and ESX can be matched, for example .

SERVER NAME IP ADDRESS

PDTDNS01 10.x.x.x

PDTDNS02 10.x.x.x

PDTNTP01 10.x.x.x

PDTNTP02 10.x.x.x

Table 8: NTP and DNS Servers

Host ProfilesHost profiles provide the ability to capture a gold image ESXi host configuration, and ensure compliance with this profile. This feature also provides the ability to remediate hosts that are not compliant, ensuring a consistent host configuration within the vSphere environment.

Host profiles should be applied at the cluster level; though when using a building block approach, different vSphere clusters (management, desktop) may contain different settings . Scaling the environment in the future might lead to introduction of new hardware at a later stage, and these clusters might require different host profiles.



Network ArchitectureThis section provides details on the networking configuration used for this architecture. Typically, the ESXi implementation uses three different types of network connection: Virtual Machine, Management Network and VMkernel . These networks connect to a virtual switch that has one or more physical adapters (although two is regarded as the minimum required for resilience) providing connectivity to existing physical networks . A fourth network is used to separate the NFS traffic that travels between ILIO and the ESXi hosts from other traffic.

Logical Network DesignThe following best practices were followed for this network design:

• Separate networks for vSphere management, vSphere vMotion, and virtual machine connectivity

• Use of multi-NIC vSphere networking

• Redundancy at the physical network switch level

• Redundant vSwitches with at least two active physical adapter ports

Management VLAN

VMK0

vmnic0vmnic0

vmnic1vmnic1

vmnic2

vmnic3

vSwitch0

vMotionVLAN

VMK1

Management Virtual Machines

VLAN

Distributed vSwitch

FabricInterconnect A

FabricInterconnect B

Figure 6: Logical Host Network Design (Management Cluster)

The VDI management block has a single standard vSwitch for vSphere vMotion traffic and the vSphere ESXi management network . It also has a vNetwork Distributed Switch (dvSwitch) that will be managed through vCenter .



Management VLAN xxx

VMK0

vmnic0

vmnic1

vmnic2

vmnic3

vSwitch0

vmnic4

vmnic5

vMotionVLAN xxx

VMK1

NFS (ILIO)VLAN xxx

vSwitch1

Desktop Virtual Machines

VLAN xxx


VLAN xxx


VLAN xxx


VLAN xxx

Distributed vSwitch

FabricInterconnect A

FabricInterconnect B

Figure 7: Logical Host Network Design (VDI Desktop Block)

Note: NFS NICs are presented to each host but NFS traffic will be intra-host only.

The dvSwitch spans many ESXi hosts and aggregates networking to a centralized cluster . This architecture incorporates a hybrid virtual networking solution that comprises both vSwitches and dvSwitches .

The VDI desktop blocks have two standard vSwitches: one for vSphere vMotion traffic and the vSphere ESXi management network, and the other for NFS storage presented by Atlantis ILIO . It also has a vNetwork Distributed Switch managed through vCenter .



Shared Storage Architecture and High AvailabilityThis deployment has a requirement for shared storage and its performance cannot be ignored, especially when considering folder redirection and the user profile. The core desktop performs very well, but if My Documents or the user profile is placed on poorly performing storage, the quality of the user experience will be degraded . Shared storage is required for the following data:

• Golden images for the desktop pools

• ESXi boot from SAN

• Atlantis ILIO Diskless VDI Virtual Machine boot drive and SnapClones

• Server infrastructure storage

• Template storage for desktops and servers

Image and Template DatastoresFor each datacenter there is a 1 .12TB datastore for templates and golden images . It is recommended that these datastores be configured in RAID 5, or utilize enterprise flash drives (EFDs).

High Availability (HA)vSphere HA was configured on the management block and disabled on the stateless desktops block due to the architecture deployed and the use of Atlantis ILIO Diskless VDI . In the event of a host failure, users simply log in and are assigned a to a new virtual desktop and a different physical host . When the host is operational, the Atlantis ILIO virtual machine will recreate its datastore automatically from SnapClone storage on shared storage .

For the management servers, HA will provide recovery of the virtual machines in the event of an ESXi host or blade failure . If an ESXi host fails for any reason, the virtual machines running on that server will go down, but will be restarted on another host within a few minutes . While there would be a service interruption perceivable to users, the impact is minimized by the automatic restarting of these virtual machines on other ESXi hosts . For the server virtual machines in the management block vSphere cluster, it is important that the management server datastores are presented to all hosts in the block and are in a single storage subsystem .

Atlantis Computing, Atlantis ILIO and Atlantis ILIO Diskless VDI, In-Memory VDI, Atlantis ILIO FlexCloud and Atlantis ILIO Persistent VDI are trademarks of Atlantis Computing, Inc.

VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.comCopyright © 2013 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.


ConclusionThis design accommodates large-scale deployments that expand to accommodate implementation requirements in a horizontal scale-out fashion . Using Atlantis ILIO Diskless VDI to create a datastore from local server memory provides an implementation that delivers better than PC performance and dramatically reduces storage OpEx by reducing costs for rack space, cooling, and ongoing maintenance . In addition, with this approach, you no longer need storage for virtual desktop images . This provides end users with an excellent desktop experience at low CapEx costs .

http://www.vmware.com/go/patents

http://www.vmware.com/go/patents