Presentation architecting a cloud infrastructure

Architecting a Cloud Infrastructure

Moderator: Chris Colotti, VMware, Inc.

Aidan Dalgleish, VMware, Inc.

Duncan Epping, VMware, Inc.

David Hill, VMware, Inc.

Rawlinson Rivera, VMware, Inc.

INF-VSP1168

#vmworldinf

2

Disclaimer

This session may contain product features that are currently under development.

This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product.

Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

Technical feasibility and market demand will affect final delivery.

Pricing and packaging for any new technologies or features discussed or presented have not been determined.

3

Agenda

Intros Gathering requirements Sizing and Scaling Host Design vCenter Design Cluster Design Networking and Security Storage Concluding

4

Introduction

Chris Colotti • Consulting Architect, VMware Global Cloud CoE

• VCDX 37 and owner of chriscolotti.us

• Twitter: @CColotti

David Hill – (Not Available for VMworld US) • Senior Solutions Architect, GTS

• Owner of Virtual-Blog.com

• Twitter @DaveHill99

Aidan Dalgleish • Consulting Architect, VMware Global Cloud CoE

• VCDX 10

• Twitter @AidersD

5

Introduction

Duncan Epping • Principal Architect, VMware Technical Marketing

• VCDX 7 and owner of Yellow-Bricks.com

• Twitter @DuncanYB

Rawlinson Rivera • Senior Consultant, VMware Professional Services

• VCDX 86 and owner of punchingclouds.com

• Twitter: @PunchingClouds

6

What is this session about?

Architecting a Cloud Infrastructure • See the CIM Case Study whitepaper. This is not a blueprint, it is an example!

Design Decisions Real world examples Understanding the potential pitfalls

Tweet / blog about this session, please use the #vmworldinf and

#VSP1168 hashtag Take pictures and tweet them!

• Best picture will get a free copy of the “vSphere 5.1 Clustering Deepdive”

7

Gathering Requirements

8

Talk to your customer

Most important part of any engagement Gather information and document! Categorize

• Requirements

• Nice to have

• Constraints

• Assumptions

Conceptualize Sound like VCDX to Anyone?

9

Example Requirements

Increasing agility / flexibility while decreasing cost of doing business Availability of services defined as 99.9% during core business

hours Security compliancy requires network isolation for specific

workloads from other services Minimal workload deployment time Should be able to guarantee resources to groups of workloads as

part of internal SLAs Recovery time objective in the case of a datastore failure should be

less than 8 hours Servers hosted in the DMZ should be protected within the virtual

environment

10

Example Constraints

Dell and AMD have been preselected as the platform of choice Eight 1GbE ports will be used per server NetApp’s NAS offering has been preselected as the solution of

choice All Tier 2 NAS volumes are de-duplicated Physical switches will not be configured for QoS Existing Cisco top of rack environment should be used for the

virtual infrastructure

11

Example Assumptions

Sufficient Switch Ports are available Current storage infrastructure can handle expected workload Staff properly trained on vSphere …

12

Historical Best Practices

It is best practice to have a 500GB LUN with a maximum of 15 VMs It is best practice to have an 8 host cluster

• vSphere

• vCloud Director (Fast Provisioning limitations)

It is best practice to have a maximum of 4 VMs per core It is best practice to have a dedicated 1GbE link for vMotion It is best practice to have “high / medium / low” resource pools It is best practice to have an isolated management It is best practice to configure vMotion and Management on a VSS

and the other traffic on a VDS It is best practice to …

13

Some Use cases

Server consolidation (power and cooling savings, green computing, lowering TCO) OPEX Savings on redundant tasks

• Self Service Provisioning

Server infrastructure resource optimization (load balancing, high availability) Standardization Business Agility (Rapid provisioning) Infrastructure as a Service (IaaS)

14

Conceptualize Your Design

Building blocks • Operations

• Time to market

• Compliance

15

Sizing / Scaling Exercise

16

Basic details

What does the environment look like today? • How many sites?

• How many potential virtualization candidates?

• Multiple waves?

How will this impact your Design / Project? • Different Cluster / Datacenter structure

• Within the limits?

• Sizing based on X waves / years?

What is the use case? • Server consolidation?

• IaaS?

• Service Level Agreements (SLA)?

17

Tooling Options

Use tools like • VMware Capacity Planner

• PlateSpin Recon

• Lanamark

Do we really need it? • Don’t all results just look the same?

• What is important?

What am I designing for? • Average vs Peak

• Consolidation vs Performance

18

Compute Considerations

How many eggs in on basket? • Two sockets vs four sockets

• Optimal Memory configurations • 8GB DIMMs are cheaper than 2 x 4GB • Triple channel configurations • Number of DIMM slots might be different per vendor / model

AMD vs Intel • AMD supports more cores, while Intel generally is faster

• VMmark can be used to make perf comparisons!

TPS vs no TPS • Using 64-bit Guest OS’es?

• Performance gain

Sweetspot? • Still seems to be dual socket – 96GB of memory

19

Network Sizing

Is this ever really a bottleneck? In most of the Capacity Planner reports we’ve seen…

• Expected average network bandwidth requirement ~ 4Mbps based on an average of 20 VMs per ESXi host.

10GbE will lift all (or most) constraints for a very long time! Use the report to identify anomalies!

20

Storage Sizing

Not only Size but Performance matters!

(TOTAL IOps × % READ)+ ((TOTAL IOps × % WRITE) × RAID Penalty) (42 x 62%) + (( 42 x 38%) x 2) (26.04) + ((15.96) x 2) 26.04 + 31.92 = 57.96 But what about size?

How does this drive your Storage Considerations?

21

Storage Considerations

RAID level used impacts IOps • IOps penalty can be severe

• Spindle count

RTO impacts amount of VMs per Datastore • Backup environment need to be capable of restoring within RTO window

• (RTO * restore speed) / avg VM Size

• Don’t confuse Mb/S with MB/S!

Adding it up… • 270 VMs from backup perspective vs 50 VMs from IOps

• What does your customer feel comfortable with?

Going SSD / Hybrid solutions? • Potential undesired results – vCloud Director Catalogs

22

Hosts

23

Design Considerations

Vendor AMD vs Intel Blade vs Rack

• Density increases

• Hot spots

• Costs

• Management

Additional considerations • Is embedded ESXi available?

• How much local SSD (capacity and IOPS) can it handle?

• Does it have built-in 2x 10 GE ports?

• Does the built-in NIC card have hardware iSCSI capability?

• Management integration

24

ESXi boot considerations

4 methods of booting ESXi • Local Disk

• Local SD / USB

• SAN Boot

• PXE Boot with Auto-Deploy • GUI Fling by Max Daneri!

Considerations • USB is cheap

• Local disk usually higher availability than USB

• SAN Boot easy to move identity, but what about costs

• Best of all worlds: PXE Boot! • Brand new and dependencies…

25

Platform Composition: ESXi base, drivers, CIM providers, …

Configuration: networking, storage, date/time, firewall, admin password, …

What Is Auto Deploy

No Boot Disk? Where does it go?

Boot Disk

All information on the state of the host is stored off the

host in vCenter

Running State: VM Inventory, HA state, License, DPM configuration

Event Recording: log files, core dump





26



What Is Auto Deploy

No Boot Disk? Where does it go?

Boot Disk

All information on the state of the host is stored off the

host in vCenter



vCenter Server Running State: VM Inventory, HA state, License, DPM configuration

Add-on Components Event Recording: log files, core dump

Host Profile Configuration: networking, storage, date/time, firewall, admin password, …

Image Profile Platform Composition: ESXi base, drivers, CIM providers, …

27

vCenter

28


How many VMs? • Do I need a dedicated vCenter Server?

• Can I still use the vCenter Appliance?

Is there a need for the Web Client? • Can I use the vCenter Appliance for that?

• Use it!

Will there be other products used like SRM / View / vCloud Director? • vCenter Heartbeat required?

Statistic Levels will impact performance / scaling

29

Sizing vCenter and Update Manager

Read the documentation! • 50 hosts / 500 VMs 2vCPUs – 4GB

• 300 hosts / 3000 VMs 4vCPUs – 8GB

• 1000 hosts / 10000 VMs 8 vCPUs – 16GB

Do we want to scale up or scale out? vSphere Update Manager on the same Server? How many users will be using vCenter? Use the Sizing Calculators for the database

• Consider Reservations

30

Clustering

31

Where do we start?

How many physical Datacenters will there be? Will each physical DC need a vCenter Server? For each vCenter, do we need multiple virtual Datacenters? For each DC, do we need multiple Clusters? For each Cluster, how many hosts?

Physical DC

vCenter

Datacenter Datacenter

Cluster Cluster

ESXi ESXi ESXi

vCenter

Datacenter

Physical DC vCenter Datacenter Cluster ESXi

32


Separate clusters for DMZ? • Why not use vShield App or vShield Edge?

Separate clusters for test? • Might also allow you to test vSphere patches!

vCenter Datacenter object is a vMotion boundary not the Cluster! vCenter Datacenter object is a VDS boundary not the Cluster! Will you be using HA / FT / DRS / DPM?

• Did you know that each DRS cluster has its own thread on the vCenter server?

• Did you know that with vSphere 5 there’s a thing called Datastore Heartbeating?

• There is no “primary / secondary” concept as of vSphere 5.0

• Admission Control is important!

33


Is 8 the perfect Cluster size? • Primary / secondary nodes (4.1 and prior) vs Master / Slave (5.x)

• Blade environment implication on design?

• LUN count vs Path count

• Linked Clones being used?

• DRS and DPM love big clusters

• HA benefits from big clusters

What about EVC? Should I turn it on by default… • EVC can only be enabled when all VMs are powered off

• Did you know that DRS requires EVC to be enabled to balance and place FT virtual machines in a cluster?

Is there a need for Resource Pools • How will you handle shares?

• Reservations / limits?

34

Networking

35


What type of vSwitch will be used? • VSS vs VDS vs Cisco Nexus 1000v

• If vCloud Director what Network Pools are required?

What are the pSwitch capabilities? Will VLANs be used? Will PVLANs be required?

• Consider vShield App?

Requirements for Jumbo Frames? • Helps support larger vCDNI packets and/or IP Storage

What type of load balancing will be used? • What type of traffic

• Load Based Teaming vs Virtual Port ID vs IP Hash

Additional Security requirements?

37


Network I/O Control • Even in 1GbE environment NIOC is useful

• Especially when connecting outbound you want to

• Did you know that Limits apply to a NIC pair level?

• Did you know that shares apply on a NIC Port level?

Additional Security Requirements? • vShield App

• vShield App with Data Security

• vShield Edge

3rd party security products? • Hy-trust

• 2-factor authentication • Audit trails • Rigid, Hierarchical Access Controls

38

vShield App Considerations

VNIC level firewall DVFilter used for in- and out-bound traffic vShield App Firewall per host! Set rules on vCenter Objects like Resource Pools and Portgroups

• Deploying a VM with the right rules is easy!

Did you know that the vShield Manager cannot be locked out? • The DVFilter is not applied to

the vShield Manager

You can exclude VMs from protection!

39

vShield Edge considerations

Multiple edge security services in a single appliance • Firewall (5 tuple)

• NAT

• DHCP

• VPN

• Load Balancing

Network Isolation Data Security options

• Useful for compliance

Think about “resiliency”! • vSE HA from within vCD in 5.1

vCloud Director leverages vShield Edge heavily!

Tenant A Tenant C Tenant X

VMware vShield Edge

VMware vShield Edge

VMware vShield Edge

VPN Load balancer Firewall

Secure Virtual

Appliance

Secure Virtual

Appliance

Secure Virtual

Appliance

40

vShield Manager considerations

Local Database Backup • vSM uses a MySQL local database

• If database is lost all configurations on vShield Edge are lost

Availability considerations • FT is supported (not from 5.1), and of course HA and VM Monitoring

• New 5.1 Appliance ships with 2 vCPUs

• vShield Manager Failure • All existing, published, rules continue to be enforced • All flow logging continues to be sent to syslog server • No changes to rules or settings can be made • Regular vShield Manager backups can be used to rebuild vShield Manager if needed

Security considerations • Default Passwords

41

Storage

42


Protocol Wars! Multiple Tiers?

• Or even Auto-Tiering, what is the impact?

vSphere Storage APIs – Array Integration (VAAI) • Does it impact sizing?

vSphere Storage APIs – Storage Awareness (VASA) • Will it impact operations?

Thin provisioning? • Thin, Thick and Eager Zeroed Thick

• vSphere vs Storage Array!

43


Can we use Storage DRS? • Impact on storage array features?

• Impact on sizing?

• Impact on other VMware products like vCloud Director?

Profile-Driven Storage? • How does it utilize VASA?

DR Requirements? • Or possibly in the future?

No more worrying about block sizes with VMFS-5 • When upgrading VMFS-3 to VMFS-5 block size does not change!

Did you know VAAI is T-10 compliant? • Makes leveraging it easier for lower-end devices

44

Impact of Features

Storage DRS has constraints… SRM does not support Storage vMotion / Storage DRS vCloud Director does support Storage DRS in 5.1!

Feature or Product Initial Placement Migration Recommendations

Array-based replication (SRDF, MirrorView, SnapMirror, etc ) Supported Manual: I/O and Space

Array-based snapshots Supported Manual: I/O and Space

Array-based Dedupe Supported Manual: I/O and Space

Array based thin provisioning Supported Fully Automated: I/O and

Space

Array-based auto-tiering (EMC FAST, Compellent Data Progression, etc) Supported Manual: Space

Array-based I/O balancing (Dell Equallogic) Supported Manual: Space

45

Questions

FILL OUT A SURVEY

EVERY COMPLETE SURVEY IS ENTERED INTO DRAWING FOR A

$25 VMWARE COMPANY STORE GIFT CERTIFICATE

Architecting a Cloud Infrastructure

Moderator: Chris Colotti, VMware, Inc.

Aidan Dalgleish, VMware, Inc.

Duncan Epping, VMware, Inc.

David Hill, VMware, Inc.

Rawlinson Rivera, VMware, Inc.

INF-VSP1168

#vmworldinf and #VSP1168

Documents

Presentation architecting a cloud infrastructure