EMC Cloud-enabled Infrastructure for SAP · VMware vCenter Chargeback Manager ... VMware Site Recovery Manager EMC RecoverPoint™ SAP automation any server and operations SAP system

White Paper

EMC Solutions Group

Abstract

This white paper focuses on the foundation layer of the on-premises Cloud-enabled Infrastructure for SAP, which can be enhanced with optional add-on bundles to address enterprise business challenges. This Cloud-enabled Infrastructure is driven by VMware vCloud Suite. End-to-end visibility, management and analysis are provided by the EMC Storage Resource Management Suite. The solution is powered by the EMC® Symmetrix VMAX® storage platform and technologies.

September 2013

EMC CLOUD-ENABLED INFRASTRUCTURE FOR SAP Foundation Bundle using EMC Symmetrix VMAX, EMC Storage Resource Management Suite, VMware vCloud Suite, and VMware vCenter Chargeback Manager

Build a virtual data center for SAP providing service catalogs, productized SLAs and multitenant resource pools

Implement chargeback, based on fully automated cost measurement, cost analysis, and cost reporting

Efficiently manage availability, capacity, performance, and health in the SAP landscape

EMC Cloud-enabled Infrastructure for SAP Foundation bundle using EMC Symmetrix VMAX, EMC Storage Resource Management Suite,

VMware vCloud Suite, and VMware vCenter Chargeback Manager

2

Copyright © 2013 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All trademarks used herein are the property of their respective owners.

Part Number H11635.2

3 EMC Cloud-enabled Infrastructure for SAP Foundation bundle using EMC Symmetrix VMAX, EMC Storage Resource Management Suite,


Table of contents

Executive summary ............................................................................................................................... 6

Business case .................................................................................................................................. 6

Solution overview ............................................................................................................................ 6

Key benefits ..................................................................................................................................... 7

EMC service offering—Cloud Readiness Accelerator ......................................................................... 7

Introduction.......................................................................................................................................... 9

Purpose ........................................................................................................................................... 9

Scope .............................................................................................................................................. 9

Audience ......................................................................................................................................... 9

Overview .......................................................................................................................................... 9

Prerequisites and supporting documentation................................................................................. 12

Terminology ................................................................................................................................... 12

Key components ................................................................................................................................. 14

Overview ........................................................................................................................................ 14

EMC Symmetrix VMAX .................................................................................................................... 14

EMC PowerPath/VE ........................................................................................................................ 14

EMC VSI ......................................................................................................................................... 14

EMC Storage Resource Management Suite ..................................................................................... 15

VMware vCloud Director ................................................................................................................. 15

VMware vCloud Networking and Security ....................................................................................... 15

VMware vCenter Chargeback Manager ........................................................................................... 16

VMware vCenter Operations Manager ............................................................................................ 16

EMC storage platform key values........................................................................................................ 17

Symmetrix VMAX ............................................................................................................................ 17

EMC FAST VP .................................................................................................................................. 17

Twenty-two percent CAPEX reductions ....................................................................................... 18

Thirty-seven percent OPEX reductions ....................................................................................... 18

Automated and optimized storage management with sustained performance ........................... 18

Federated Tiered Storage ............................................................................................................... 18

Host I/O Limits ............................................................................................................................... 19

Management .................................................................................................................................. 19

VMware integration ........................................................................................................................ 19

Virtual provisioning ........................................................................................................................ 20

Validated landscape ........................................................................................................................... 21

Overview ........................................................................................................................................ 21

Environment architecture ............................................................................................................... 21



4

Physical architecture ................................................................................................................. 21

Hardware resources................................................................................................................... 22

Logical architecture ................................................................................................................... 22

Software resources .................................................................................................................... 23

Key functional design and configuration ............................................................................................ 25

Overview ........................................................................................................................................ 25

Virtual data center for SAP ............................................................................................................. 25

Service levels ............................................................................................................................ 25

Designing vCloud Director ......................................................................................................... 26

Designing the vSphere .............................................................................................................. 29

Infrastructure chargeback .............................................................................................................. 33

Overview ................................................................................................................................... 33

vCenter ChargeBack Manager design ........................................................................................ 33

Integrated cloud management performance analysis ..................................................................... 36

Overview ................................................................................................................................... 36

Full visibility of the virtual data center ....................................................................................... 40

End-to-end performance analysis .............................................................................................. 43

Configuration validation and compliance .................................................................................. 45

Storage capacity analysis .......................................................................................................... 47

Predefined and customized intelligent reports .......................................................................... 48

Fully Automated Storage Tiering ..................................................................................................... 51

Cloud networking and security for SAP ........................................................................................... 54

Overview ................................................................................................................................... 54

VMware vCloud networking and security design ........................................................................ 54

SAP system design ............................................................................................................................. 59

Overview ........................................................................................................................................ 59

Compute ........................................................................................................................................ 59

Storage .......................................................................................................................................... 60

Network ......................................................................................................................................... 61

Installation type ............................................................................................................................. 61

Other ............................................................................................................................................. 61

Validation of functions ....................................................................................................................... 62

Overview ........................................................................................................................................ 62

Use case 1: Virtual data center for SAP ........................................................................................... 62

Test scenario ............................................................................................................................. 62

Success criteria ......................................................................................................................... 62

Validation procedure ................................................................................................................. 62

Summary ................................................................................................................................... 69

Use case 2: Automated provisioning, service catalogs, self-services, and chargeback ................... 69



Test scenario ............................................................................................................................. 69

Validation procedure ................................................................................................................. 69

Summary ................................................................................................................................... 75

Use case 3: Integrated cloud management and performance analysis ............................................ 75

Success criteria ......................................................................................................................... 75

Scenario 1: On-demand resource pooling .................................................................................. 75

Scenario 2: Root cause analysis and problem resolution ........................................................... 78

Summary ................................................................................................................................... 83

Conclusion ......................................................................................................................................... 84

Summary ....................................................................................................................................... 84

Findings ......................................................................................................................................... 84

Moving forward .............................................................................................................................. 84

References.......................................................................................................................................... 85

White papers ................................................................................................................................. 85

Product documentation .................................................................................................................. 85

Other documentation ..................................................................................................................... 85



6

Executive summary

Automation and optimization of IT processes running on a virtualized infrastructure are key enablers to fulfill the business needs of IT managers who want to provide information technology as a service.

Many case studies exemplify striking results after transforming to the EMC Cloud-enabled Infrastructure for SAP. One of the best examples of deploying a private cloud is EMC’s own SAP landscape, which became fully operational in 2012.

The EMC IT approach to the EMC Cloud-enabled Infrastructure for SAP has enabled a global SAP instance supporting over 300 global sites. It has resulted in a 50 percent performance increase, five times the scalability, and twenty times the integration throughput. In the process, EMC IT was able to retire 65 legacy applications and reduce the count of Application Integration Component (AIC) interfaces from 450 to 125. New virtual environment provisioning times have gone from days to hours, and the percentage of time available for primary value chain activities has doubled.

To support running your SAP operations as a business, the following basic functions would be required for the on-premises cloud:

A virtual datacenter for SAP providing service catalogs, productized service level agreements (SLAs), and multitenant resource pools

Chargeback services, based on fully automated cost measurement, cost analysis, and cost reporting

Integrated cloud management and performance analysis tools, enabling you to efficiently manage availability, capacity, performance, and health in the SAP landscape

Automatic resource management for server, network, and storage

A high level of security with simplified compliance

This white paper introduces the EMC solution that serves all of these functions.

VMware vCloud Suite, EMC Storage Resource Management Suite, and EMC®

Symmetrix VMAX® are the primary enabling technologies for this solution:

VMware vCloud Director applies the principles of pooling, abstraction, and automation to all data center services such as compute, storage, networking, and security. It enables the multi-tenancy business model for IT operation and provides the portal for self-service provisioning. VMware vCenter Chargeback Manager is used for metering the resource consumption of different organizations. VMware vCenter Operation Manager provides powerful functions to efficiently manage availability, capacity, and performance of the virtual datacenter.

Business case

Solution overview



The EMC Storage Resource Management Suite provides visualizations and reporting for understanding and managing the impact storage has in the on-premise cloud. It helps administrators to manage availability, capacity, and performance at the storage layer and provides powerful views of the relationships among host, virtual machine, network, and storage.

The Symmetrix VMAX platform is a powerful, trusted, and smart storage platform for your multiple mission-critical applications and provides automation and efficiency in tiered storage environments with Fully Automated Storage Tiering for Virtual Pools (FAST VP).

The solution builds the foundation layer of the EMC Cloud-enabled Infrastructure for SAP by providing the following functions:

Virtual datacenter for SAP

Infrastructure chargeback

Integrated cloud management and performance analysis

Fully Automated Storage Tiering

Cloud networking and security for SAP

The Cloud Readiness Accelerator (CRA) is a fast-track engagement with immediate value and actionable deliverables that allow IT organizations to accelerate their transformation strategies towards cloud computing.

CRA uses a proven methodology to examine an IT organization’s maturity by analyzing nine fundamental characteristics of the cloud data center. An EMC cloud architect with extensive experience and certifications in virtualization and cloud computing technologies guides the participants through a set of guided workshops.

Key benefits

EMC service offering—Cloud Readiness Accelerator



8

The workshops are designed to build a personalized cloud maturity model that describes the existing and desired readiness state of the IT environments on a scale from basic to federated, as shown in Figure 1. Participants define their own roadmaps for building their own Cloud-enabled Infrastructures for SAP.

Cloud maturity model Figure 1.



Introduction This white paper describes the foundation layer of the EMC Cloud-enabled Infrastructure for SAP solution.

This white paper:

Introduces the key enabling technologies for this solution

Describes the best practice and design consideration for this solution

Outlines how the key components are configured

Presents the test results performed to validate the solution

Identifies the key business benefits of the solution

Demonstrates the functionality within predefined use cases

This white paper is intended for SAP operations managers, administrators (SAP/server/network/storage), IT architects, and technical managers responsible for designing, creating, or managing mission-critical SAP landscapes.

The EMC Cloud-enabled Infrastructure for SAP solution is designed so that customers can flexibly choose the cloud functionalities (or “bundles”) they want to enable without losing sight of their end goal—an efficient and reliable private cloud.

The foundation bundle is a mandatory component upon which all the other cloud bundles are based. After establishing the foundation bundle, customers can plan their own roadmaps to determine their next steps. Table 1 outlines the foundation bundle and its functions.

Purpose

Scope

Audience

Overview



10

Table 1. EMC Cloud-enabled Infrastructure for SAP solution: foundation bundle

Bundle Function Benefits Technology

On-premises cloud

Virtual datacenter for SAP

Autonomy of business units and application operations

Service catalogs

SLAs

Management of vCloud tenants

Resource pooling

VMware vCloud Suite Enterprise


Cost measurement, analysis, and reporting of the use of compute, network, storage, and backup resources (in combination with data protection add-on bundle)

VMware vCenter Chargeback

Integrated cloud management and performance analysis

Manage availability, capacity, performance, and health in the SAP landscape

VMware vCenter Operations Manager, EMC SRM, EMC VSI

Storage tiering Automatically get the right data to the right place at the right time

EMC VMAX, FAST Suite


Cloud-enabled infrastructure security framework

Authorization concepts

Compliance and non-compliance tracking

VMware vCloud Networking and Security



Table 2 describes the add-on bundles that are described in separate solution white papers.

Table 2. EMC Cloud-enabled Infrastructure for SAP solution: add-on bundles

Bundle Function Benefit Technology

High availability and application mobility

High availability and application mobility for SAP

High availability within one datacenter and across two sites

Non-disruptive movement of applications from one datacenter to another

EMC VPLEX,

VMware HA

Disaster recovery

Disaster recovery for SAP

Disaster recovery protection for cloud management cluster and SAP systems

VMware Site Recovery Manager

EMC RecoverPoint™

SAP automation and operations

SAP system virtualization

Any service any time on any server

SAP NetWeaver Landscape Virtualization Management

Automated SAP system provisioning

Provision SAP systems on demand with automated end-to-end process

Mass operations Reduce downtime window during maintenance

Data protection

Backup and recovery for SAP

Provide data protection to cover the cloud management stack and SAP systems

Backup/recovery at local and far apart (hundreds of kilometers) sites

EMC Avamar® ,

EMC Data Domain® ,

EMC Data Protection Advisor (DPA)

Enhanced security

Enhanced security and compliance for SAP

Efficient, collaborative enterprise governance, risk and compliance (eGRC) program across IT, finance, operations and legal domains

Data loss prevention

Secure user to network authentication

RSA Archer eGRC,

RSA Data Loss Prevention (DLP),

RSA SecurID



12

Read the white paper EMC Transforms IT On-Premise Private Cloud Infrastructure for SAP–Reference Architecture, available on EMC.com, for a detailed overview of the Cloud-enabled Infrastructure for SAP, and to find out how enterprises can benefit, from both the technical and business perspectives.

Table 3 defines the terms used in this document.

Table 3. Terminology

Term Definition

CMCNE Connectrix Manager Converged Network Edition

Datastore A storage location for virtual machine files

DRS Distributed Resource Scheduler

FAST VP Fully Automated Storage Tiering for Virtual Pools. A feature of EMC Symmetrix VMAX that automatically optimizes performance in a tiered storage environment. FAST™ VP reduces costs, footprint, and management efforts and puts the right data in the right place at the right time. FAST VP maximizes the utilization of Flash drive capacity for high IOPS workloads and SATA drives for capacity intensive applications.

LDAP Lightweight Directory Access Protocol

Resource pool A logical abstraction for the flexible management of CPU and memory resources

Storage Resource Management Suite

The Storage Resource Management Suite provides visualizations and reporting solution for optimizing storage resources, managing service levels and improving productivity.

vApp Virtual application. A virtual system that contains one or more individual virtual machines and system parameters that define operational details

vCD vCloud Director. Software that enables customers to build a secure, multi-tenant cloud by pooling infrastructure resources into virtual data centers and enabling those resources to be consumed by users on demand

OvDC Organization Virtual Data Center. A vDC abstraction, which is partitioned from a PvDC that maps resources to an organization

PvDC Provider Virtual Data Center. A vDC abstraction that combines the computer and memory resources of a single vCenter™ Server resource pool with the storage resources of one or more datastores connected to that resource pool

VMware vCenter Chargeback Manager

VMware vCenter Chargeback Manager is a software component of vCloud Suite that allows you to customize rate cards and prices according to the processes and policies of different organizations.

VMware vCops The VMware vCenter Operations Manager™ software manages the operation of vSphere and physical and cloud infrastructures

Prerequisites and supporting documentation

Terminology



Term Definition

VMware vSphere DRS

The VMware vSphere Distributed Resource Scheduler™ continuously monitors the utilization across vSphere servers and intelligently allocates available resources among virtual machines according to business needs

VSI Virtual Storage Integrator, a VMware vCenter server plug-in



14

Key components

This paper introduces the following technology components:

EMC Symmetrix VMAX

EMC PowerPath® /VE

EMC VSI

EMC Storage Resource Management Suite

VMware vCloud Director



VMware vCenter Operations Manager

The EMC Symmetrix VMAX with the Enginuity operational environment is built to support enterprise customers’ top IT priorities, including the transformation to the private cloud and “big data” processing. The EMC Symmetrix VMAX system builds on the revolutionary scale-out Virtual Matrix Architecture that features unprecedented performance, availability, functionality, and cost efficiency.

EMC PowerPath® /VE for VMWARE vSphere delivers multipathing features that optimize VMware vSphere virtual environments. PowerPath/VE installs as a kernel module on the VMware ESXi host and works as a multipathing plug-in (MPP) that provides enhanced path management and load-balancing capabilities for ESXi hosts.

EMC VSI provides multiple feature sets including Storage Viewer (SV), Path Management, and Unified Storage Management. It facilitates the discovery and identification of all EMC storage devices allocated to VMware ESXi hosts and virtual machines. Unified Storage Management simplifies the provisioning of Symmetrix VMAX virtual pooled storage for data centers, ESXi server, and resource pools. Path Management allows you to control how users access datastores.

Overview

EMC Symmetrix VMAX

EMC PowerPath/VE

EMC VSI



The Storage Resource Management Suite provides visualizations and reporting for understanding and managing the impact storage your application services. Storage Resource Management Suite enables you to better:

View the performance and health status of your storage environment in real time

Monitor storage service levels by customizing service-level agreement (SLA) dashboards and setting alerts when service levels are about to be breached

Optimize capacity to align with performance and business requirements

Reduce application downtime by being aware of the relationships and dependencies between applications and storage

Align cost of service delivery with business requirements using charge back reports

Continuously validate compliance of the storage environment with design best practices and the EMC Support Matrix (ESM)

Analyze performance from the application/host perspective and correlate it with historical storage metrics to identify workload patterns and their impact

The Storage Resource Management Suite, delivers new insights into the storage environment to better understand the impact storage has on application services. It enables storage administrators to visualize application to storage dependencies, analyze configurations and capacity growth, and optimize their environment to improve return on investment. By using the suite, storage administrators can assure storage services by providing end-to-end performance analysis, monitoring, alerting, and configuration analysis. With this increased visibility, storage teams are in a better position to optimize storage utilization to avoid unnecessary purchases and justify new ones while meeting SLAs. The suite provides this visibility across datacenters for heterogeneous storage (EMC and third party) environments.

VMware vCloud Director orchestrates the provisioning of software-defined data center services as complete virtual data centers that are ready for consumption in a matter of minutes. Virtual data centers provide virtualized computing, networking, storage, and security. Administrators can provision the complete set of services necessary to make workloads operational in minutes. Software-defined data center services and the virtual data centers fundamentally simplify infrastructure provisioning, and enable IT to move at the speed of business.

By dramatically simplifying the provisioning of these services, VMware enables IT to provision a complete and operationally ready infrastructure without worrying about the physical configuration of its hardware.

VMware vCloud Networking and Security is the leading software-defined networking and security solution that enhances operational efficiency, unlocks agility, and enables extensibility to rapidly respond to business needs. It provides a broad range of services in a single solution, including virtual firewall, VPN, load balancing, and VXLAN extended networks.

EMC Storage Resource Management Suite

VMware vCloud Director




16

VMware vCenter Chargeback Manager improves utilization of your virtual infrastructure with accurate visibility into the true costs of virtualized workloads. This software enables line-of-business owners to have full cost transparency and accountability for self-service resource requests.

VMware vCenter Chargeback Manager allows you to customize rate cards and prices to the process and policies of different organizations. Virtual machine resource consumption data is collected from VMware vCenter Server, ensuring the most complete and accurate tabulation of resource costs.

VMware vCenter Operations Management Suite provides automated operations management using patented analytics and an integrated approach to performance, capacity and configuration management. vCenter Operations Management Suite enables IT organizations to get better visibility and actionable intelligence to proactively ensure service levels, optimum resource usage, and configuration compliance in dynamic virtual and cloud environments.





EMC storage platform key values

The EMC Symmetrix VMAX with Enginuity operating environment builds on EMC’s industry-leading Symmetrix storage array platform for powerful, trusted, smart storage that provides a higher level of performance, availability, and intelligence in the virtual data center, as described in Table 4. Symmetrix VMAX system offers a broad array of functionality and tools that simplify storage management and reduce costs to help accelerate the transformation to the private cloud.

Table 4. EMC Symmetrix VMAX with Enginuity

Storage feature Description

Powerful Symmetrix VMAX is a distributed multi-engine storage system that can non-disruptively scale from one to as many as eight VMAX Engines. It supports millions of virtual machines, high-demand online transaction processing (OLTP) and online analytical processing (OLAP) for enterprise data warehousing (EDW), and business intelligence (BI) applications.

VMAX 10K supports up to 4 engines, 512 gigabyte (GB) memory, and 1.5 PB capacity

VMAX 20K supports up to 8 engines, 1 TB memory, and 2 PB capacity

VMAX 40K supports up to 8 engines, 2 TB memory, and 4 PB capacity

Trusted Six-nines application availability 24x7xForever operations

Robust security with secure audit, secure access, and data encryption for cloud environments

Smart Automation and efficiency in tiered storage environments with FAST VP

Trusted Symmetrix services to consolidate and simplify operations with Federated Tiered Storage

Ease-of-management and integration into virtual environments with EMC Unisphere® for VMAX

EMC FAST VP optimizes array performance across all drive types (Flash, SAS, and NL SAS disk) in the array to improve system performance while reducing cost. FAST dynamically allocates workloads based on the configured service level, and non-disruptively moves workloads across storage types, optimizing overall system performance. FAST VP moves the most active parts of your workload to high-performance flash disks and the least frequently accessed data to lower cost drives, leveraging the best performance and cost characteristics of the different drive types. Compared to a traditional single-tier Fibre Channel solution, FAST VP provides higher performance but requires fewer drives, reducing energy and space requirements, helping to accelerate the transformation to a cloud environment.

FAST VP real-time performance monitoring and intelligent algorithms enable the VMAX storage arrays to relocate chunks of data as small as 7.6 MB. This very granular analysis and data movement enables cost-efficient and high-performing

Symmetrix VMAX

EMC FAST VP



18

configurations, continually tuned to the performance needs of the applications running on the storage array.

From previous proven SAP solution, you may see:

Twenty-two percent CAPEX reductions

A multi-tier Flash/SAS/NL SAS configuration with FAST VP results in a 22 percent cost reduction per gigabyte in storage capital expenditure (CAPEX) without compromising system performance or throughput. This figure was derived by evaluating and comparing the costs of the storage tier configurations with and without the use of FAST VP.

Thirty-seven percent OPEX reductions

A multi-tier Flash/SAS/NL SAS configuration with FAST VP results in 37 percent power consumption cost reduction per gigabyte in operational expenditure (OPEX). This figure was derived by using the EMC Power Calculator web-based tool to evaluate and compare the costs of the storage tier configurations with and without the use of FAST VP.

Automated and optimized storage management with sustained performance

FAST VP provides an automated solution to simplify optimization of storage resources while meeting the performance requirements of a realistic SAP workload. Storage is managed as pools instead of LUNs tracked with spreadsheets, and storage can be provisioned quickly with no downtime for data growth.

To learn more about the validation, refer to the Increase Efficiency for SAP Landscape with Symmetrix FAST VP White Paper.

Federated Tiered Storage (FTS) enables the on-premises private cloud by creating cooperating pools of resources and by enabling dynamic application and data movement. FTS technology allows you to leverage the benefits of EMC’s trusted innovations across the entire private cloud by consolidating, optimizing, and sharing both EMC and non-EMC storage resources. Using FTS, you can integrate legacy arrays to maximize efficiency, protect your storage investment, and simplify the management of your storage landscape.

FTS provides the capability to put heterogeneous arrays behind VMAX to consolidate and simplify operation in the private cloud. With FTS, customers can use a trusted Symmetrix feature such as FAST VP, Symmetrix Remote Data facility (SRDF), and VLUN to extend the life and value of existing storage arrays in the data center. In a FAST VP configuration, the external array can be used as a lower tier. FAST VP will automatically move data between VMAX and external storage as additional performance or reduced cost is required from the application.

Federated Tiered Storage



VMAX Host I/O Limits enable users to set maximum performance quotas for IOPS and throughput on a per-storage group basis. This ensures that each application does not affect the performance of other applications. Host I/O Limits enable:

Users to provide greater levels of control on performance allocation in multi-tenant environments

Service providers to articulate service-level characteristics, manage expectations, and sell new services

The VMAX monitors and manages I/O on each director in the masking view to ensure that the performance limit is not exceeded. When the limits are reached, VMAX slows I/O to discourage additional transactions from that storage group.

EMC Unisphere for VMAX is an intuitive management interface that allows IT managers to dramatically reduce the time required to provision, manage, and monitor Symmetrix storage assets. Unisphere delivers the simplification, flexibility, and automation that are key requirements to accelerate the transformation to the private cloud.

With the Performance monitoring options, Unisphere for VMAX provides tools for performing analysis and monitoring historical trending of Symmetrix system performance data.

VMAX supports VMware vStorage APIs for Array Integration (VAAI), which offloads virtual machine operation to the array to optimize server performance. In the VMware environments, VMAX 40K supports the following VAAI components:

Full copy: Offloads replication to VMAX to enable up to 10 times faster virtual machine deployments, clones, snapshots, and VMware Storage vMotion.

UNMAP: Enables more efficient space usage for virtual machines by reclaiming space on datastores that is unused and returns it to the thin provision pool from which it was originally drawn. UNMAP is only supported with vSphere 5.01 and higher.

VMware vSphere Storage APIs for Storage Awareness (VASA): Allows VMware administrators to view VMAX driver types (Flash, SAS, or NL SAS) as needed with VASA support. VMware administrators have a single management view into both the virtualization and storage infrastructures.

Block zero: Initializes file system block and virtual disk space more rapidly, with as much as 10 times less I/O for virtual machine file system (VMFS) formatting and relocation.

Hardware-assisted locking: Enables more efficient Meta data updates and assists virtual desktop deployments, allowing up to 10 times more virtual machines per data store.

Host I/O Limits

Management

VMware integration



20

Symmetrix virtual provisioning can reduce cost, improve capacity utilization, and simplify storage management. Users can present a large amount of capacity to a host and then consume space only as needed from a shared pool. This improves TCE for reducing initial over-allocation of storage capacity. Symmetrix virtual provisioning also offers automated pool rebalancing, helping to expand thin pools in small increments while protecting performance, as well as non-disruptive shrinking of thin pools to reuse space to improve capacity utilization. Virtual provisioning can reduce labor costs by simplifying data layout and reducing the steps required to accommodate capacity growth.

Virtual provisioning



Validated landscape

We validated the foundation bundle’s functions in the infrastructure and software environment shown in Figure 2.

Physical architecture

The foundation bundle is designed to run in a single site with basic cloud-ready functions that enable the virtual data center. You can seamlessly integrate the add-on bundles to the foundation bundle at a later time to configure a full cloud-enabled infrastructure.

Figure 2 illustrates the foundation bundle’s physical architecture.

Physical architecture of the foundation bundle Figure 2.

Overview

Environment architecture



22

Hardware resources

Table 5 lists the hardware resources used in this solution.

Table 5. Hardware resources

Hardware Quantity Product configuration

Storage 1 EMC Symmetrix VMAX

Compute 6 ESXi servers: 2 for Management Cluster, 4 for Resource Cluster

2 x four-core CPUs, 128 GB RAM

Network switching and routing platform

2 10 GbE (gigabit Ethernet) fabric switch

2 SAN fabric switch

Logical architecture

Figure 3 depicts the logical architecture diagram.

Logical architecture of the foundation bundle Figure 3.



We suggest separating the virtual infrastructure into two cluster groups:

The management group hosts all virtual machines used for cloud-enabled infrastructure management. This enables functions such as multi-tenancy (VMware vCloud Director), monitoring (EMC Storage Resource Management Suite, vCops, and SAP Solution Manager), networking, security (vCloud networking and security), and chargeback (vCenter Chargeback Manager).

The resource group hosts all SAP systems consumed by different organizations to support business needs.

Software resources

Table 6 lists the hardware components used in this solution.

Table 6. Software components

Software Version Description

Active Directory Windows 2008 Central hostname and user management

EMC PowerPath/VE 5.8 Multipathing software

EMC ProSphere 1.6 Storage management software

EMC Enginuity 5876 Operating environment for Symmetrix VMAX

EMC Solution Enabler 7.5 Symmetrix command line interface

EMC Storage Configuration Advisor

3.0.0.1.126 Configuration software

EMC Watch4Net 6.1u3 Enterprise reporting and analysis tool

EMC Unisphere 1.1.0.5 VMAX management software

SAP ERP 6.0 EHP 4/NW 7.01 SP 10

IDES/Unicode

SAP Solution Manager 7.1/NW 7.3 SAP Management tool

Oracle Database 11.2.0.3 Used on all SAP systems

VMware vSphere 5.1 Hypervisor hosting all virtual machines


2.5.0 Chargeback software

Microsoft SQL Server 2008 R2 Used by VMware vCenter Chargeback Manager and vCloud Director


5.6 User interface and analytics virtual machine

VMware vCenter Server 5.1 Managing resource and management clusters

VMware vCloud Director 5.1.0 Multi-tenancy software component of the VMware vCloud suite



24

Software Version Description


5.1.0 Security software

SUSE Linux Enterprise 11 SP2 Operating system for SAP



Key functional design and configuration The foundation bundle of a cloud-enabled infrastructure is a collaboration of functions that delivers essential characteristics of a private cloud including:

Virtual data center for SAP


Integrated performance analysis



The virtual data center aggregates compute, memory, storage, and network resources into resource pools for SAP systems. In this section, we describe the service level and corresponding virtual data center design.

Service levels

When enterprises transform their traditional data centers into a cloud-enabled data center, the first step is to define service levels that suit various SAP requirements for each organization.

The service levels presented in this section serve as a reference for building a differentiated service model. Service levels and their inherent virtual data center constructs provide an effective means of creating service differentiation within a broader vCloud service landscape. They deliver consistent service levels that invariably align with the unique business requirements of individual organizations, which are represented in our solution.

According to VMware vCloud Architecture Toolkit (vCAT) 3.1, the following is a summary of these service levels:

Basic—Based on the Pay As You Go allocation model, this service level lends itself to quick-start pilot projects or test and development application workloads that typically do not require persistent resource commitments or upfront resource reservations.

Committed—Based on the Allocation Pool allocation model, this service level provides consumers with a minimum initial resource commitment and the ability to exceed that minimum, with additional capacity available as needed. This minimum commitment level, expressed as a percentage of overall capacity, accommodates the customer’s varying application workloads when needed.

Dedicated—Based on the Reservation Pool allocation model, this service level provides a reserved resource capacity up front, fully dedicated to an individual organization. This resource level guarantees a high degree of service that assures the customer that the capacity required by their workload is available.

Overview

Virtual data center for SAP



26

To help you decide which service level makes the most sense for your (or your customer’s) business needs, refer to Table 7.

Table 7. Key service level attributes

Service attributes Service level

Basic Committed Dedicated

Target workloads Test and development Tier 2 and tier 3 production

Tier 1 production

Workload examples QAS and DEV systems

Training systems

Sandbox systems

Lightly used SAP systems

Moderate IOPS systems

High IOPS systems

CPU-intensive systems

Standalone database

Compute resource Allocation model

Pay As You Go Allocation pool Reservation pool

Compute resource Limit

CPU is not guaranteed, memory is guaranteed at high percentage

CPU and memory is guaranteed at high percentage

CPU and memory is fully guaranteed as allocated

Charge mode Usage Allocation

Physical server resource

Low-end physical server models

Latest high-end physical server models

Storage resource Medium performance High performance

We designed the following components to implement the service levels:

vCloud Director: PvDC, OvDC

vSphere: DRS, storage profile, storage DRS, networking

Designing vCloud Director

The virtual data center is a new construct that represents the standard container for a pool of compute and storage resources. There are two types of virtual data centers: Provider (PvDC) and Organization (OvDC). This section focuses on PvDC and OvDC design.

PvDC A PvDC is composed of resource pools and datastores from one or several vCenter Servers. A PvDC is the source for Organization vDCs.

This solution consists of two PvDCs:

Standard—The standard PvDC is built on distributed resource scheduler (DRS) clusters with low-end ESXi servers, which may come from existing physical servers or as replacements for high-end physical servers. Storage in the standard PvDC is categorized as Bronze tier, with relatively low performance compared to advanced PvDC. QAS/DEV SAP systems are hosted on this standard PvDC.



Advanced—The advanced PvDC is built on DRS clusters with high-end ESXi servers. Storage in advanced PvDC is categorized as Silver tier and Gold tier. Both Silver and Gold tier storage offer high performance that is suitable for production SAP systems. Gold tier storage uses more enterprise Flash drive capacity than Silver tier storage, making it best for high-IOPS SAP systems.

The advantages of separating PvDC into Standard and Advanced are:

Separation of non-productive SAP systems from productive SAP systems on different DRS clusters ensures enough resources for the PRD SAP systems when needed.

You can reuse existing x86 servers to construct a standard PvDC to protect your investment.

You can replace older servers in an advanced PvDC with new models. You can then transition the old models to a standard PvDC to support QAS/DEV SAP systems.

OvDC An Organization vDC allocates resources from a PvDC and makes it available for a given organization. Multiple Organization vDCs can share the resources of the same PvDC. The Allocation model is a core concept in vCloud Director 5.1 that determines how and when the vCloud compute and memory resources that you allocate are committed to an organization. There are three allocation models:

Reservation Pool Model—Reservation Pool resources allocated to the organization vDC is completely dedicated. All guarantees are set to 100 percent. Reservation Pool vDCs map to resource pools with the reservations set equivalent to the limits.

Allocation Pool Model—An Allocation pool is a pool of allocated resources with a certain percentage of resources guaranteed. When an organization vDC is created using the Allocation Pool Model, a dynamic resource pool is instantiated. This resource pool automatically adjusts available resources as new workloads are powered on, based on the values specified within the OvDC. Each value has a direct impact on how the related resource pool dynamically changes as new virtual machines are deployed.

Pay As You Go Model—The Pay As You Go model provides the illusion of an unlimited resource pool. This model maps to a sub-resource pool with no configured reservations or limits. Resources are committed only when vApps are deployed in the organization vDC.

For more information about allocation models, refer to the vCloud Director Administrator's Guide and vCloud Architecture Toolkit 3.1.

To demonstrate the vCloud Director design, we introduced the following business scenarios as an example:

ABC Corporation adopts the EMC Cloud-enabled Infrastructure for SAP solution to transform their traditional IT infrastructure running SAP to a cloud infrastructure.

There is a centralized IT in ABC Corporation that manages the cloud infrastructure and provides IT related services for its organizations.



28

Cloud-enabled infrastructure resources are shared between organizations of ABC Corporation.

Org A requires Committed and Basic service levels for their PRD/QAS/DEV SAP systems.

Org B requires Dedicated and Basic service levels for their PRD/QAS/DEV SAP systems.

Figure 4 shows the high-level architecture of the vCloud Director design.

vCloud Director design Figure 4.



OvDCs are mapped to service levels according to organization requirements as shown in Table 8.

Table 8. OvDC organizational mapping

Organization vDC Service level Compute resource allocation model SAP system

OrgA_Basic Basic Pay As You Go QAS/DEV

OrgA_Committed Committed Allocation Pool Medium workload

OrgB_Basic Basic Pay As You Go QAS/DEV

OrgB_Dedicated Dedicated Reservation Pool Dedicated service

Designing the vSphere

This solution includes the following vSphere design components:

DRS cluster

Storage profile

Storage DRS cluster

Networking

DRS cluster In this solution, we used two separate DRS clusters to support two PvDCs: one standard cluster and one advance cluster.

VMware recommends that for all DRS clusters, you:

Enable the High Availability (HA) feature to protect all virtual machines against ESXi host failures.

Configure the Automation Level as fully automated so that vSphere can balance the workload automatically.

When building DRS clusters for the cloud-enabled infrastructure, you should:

Use the same ESXi server model in a cluster with the same type and amount of CPU, memory, NICs, and HBAs to simplify deployment and management.

Configure multiple smaller clusters rather than a single cluster to achieve better performance.

Avoid setting DRS affinity rules with Virtual Machines to Hosts, as this may cause HA failure.



30

Storage profile vCloud Director leverages storage profiles defined in vSphere as storage resources. By using profile-driven storage, you can define various storage characteristics as tiers and apply them to individual virtual machines.

FAST VP policies on VMAX provide three storage tiers to meet different performance requirements:

Bronze—High Capacity datastores leveraging mix of SAS and NL SAS storage

Silver—High performance datastores with a mix of Flash, SAS, and NL SAS storage

Gold—Highest performance datastores leveraging Flash and SAS storage

The detailed storage and FAST VP design on VMAX is covered in the Fully Automated Storage Tiering section.

We defined three storage profiles to correspond with the defined FAST VP policies, Bronze, Silver, and Gold. Each storage profile maps to a user-defined storage capability. You can define the storage profile of a datastore automatically through VMware vStorage APIs for Storage Awareness (VASA) or you can manually define the storage profile for each datastore. In this solution, we manually defined the storage profile of each datastore. Refer to vSphere Storage Guide 5.1 for detailed procedures for configuring storage profiles.

We defined the storage profile of the SAP DEV/QAS applications as Bronze because these applications do not typically demand high performance. We used Gold and Silver storage profiles for SAP PRD applications that typically demand higher performance levels.

Datastore cluster Datastore clusters form the basis of storage DRS. As with a cluster of hosts, a datastore cluster is used to aggregate storage resources, enabling smart and rapid placement of new virtual machines and virtual disk drives and balancing existing workload.

We created three datastore clusters (gold, silver and bronze) in accordance with the storage profiles.

We set the DRS storage automation level to NO Automation to avoid automatic storage migration activities that will affect performance. Storage DRS then determines the initial placement of the virtual machine storage, leaving the performance optimization to EMC FAST VP.

We disabled the Storage DRS I/O metrics as shown in Figure 5. This allows SDRS to automatically perform capacity balancing and FAST VP to automatically relocate data on the EMC VMAX array. This simplifies the storage view from the vCloud Suite higher up the stack, and enables an efficient provisioning model that is easily consumed by the vCloud Suite.



Disabled I/O metric in Storage DRS settings Figure 5.

Networking vSphere networking is the foundation of vCloud Director. In This solution, each ESXi host has four network adapters:

Two 1 Gb Ethernet adapters are teamed together and are configured with a port group for Internet access using a standard switch.

Two 10 Gb Ethernet adaptors are teamed together and are configured with multiple port groups for virtual machines, vMotion, management, an external network for client access, and an administration network using a distributed switch.

Figure 6 depicts the network design.

Network design in vSphere Figure 6.



32

Table 9 describes each port group defined in Figure 6.

Table 9. Network details

Network name Port group name VLAN Traffic type Usage

Public network Public Network 10 Virtual machine Internet access

vMotion network dvPG_vMotion 11 vMotion vMotion traffic

Host network dvPG_Host 12 Host Management For communications among ESXi hosts, and between ESXi hosts and the vCenter server

Management network

dvPG_Management 100 Virtual machine Management virtual machine

Corp network dvPG_Corp 101 Virtual machine Corporate internal network for communication between SAP systems and client hosts

Admin Network dvPG_Corp_Admin 102 Virtual machine Corporate internal network for privileged communications between SAP systems and SAP administrators

For better network utilization and balance of network adapter bandwidth, we recommend that you enable Network I/O Control (NIOC) on the distributed switches to guarantee sufficient bandwidth for SAP systems.

Specify the following share values for the corresponding traffic types:

Fault Tolerance/vSphere Replication/NFS/iSCSI = 1

Management Traffic = 5

vMotion Traffic = 20

Virtual Machine Traffic = 10

We set the FT/VR/NFS/iSCSI share values to the minimum value because this solution does not use them. When a vApp is deployed, its virtual machine network traffic share value is set according to the Virtual Machine Traffic settings.

To minimize the latency in the process of DRS cluster balancing, when configuring the Network I/O control settings, be sure to set a higher share value for vMotion traffic and iSCSI/NFS traffic than other traffic values.

The above configuration is for illustration purposes only, and should not be considered as the default standard configuration used in all environments. To determine the share values, plan the configuration carefully.

For more details, refer to vSphere Networking Guide vSphere 5.1.



Overview

In this solution, vCenter Chargeback Manager provides the virtualized infrastructure metering functionality. It integrates with VMware vCloud Director, providing the service provider with the ability to track and charge for the resources used by customers, and provides the serviced consumer with the ability to track resource usage and generate a cost reports when needed.

vCenter ChargeBack Manager design

In vCenter Chargeback Manager, we defined the cost elements required to support the service level listed in Table 7, including:

Billing policy—Defines how to measure and charge for each compute resource

Pricing model—Defines the base computing resource rate and binds that rate with the relevant billing policy

Infrastructure specific cost—Defines the rate factor on specific infrastructure components, such as different storage tiers

Cost template—Defines the rate factor on specific compute resources for the purpose of discounts, promotion for an individual vApp, or organizations defined in the cloud hierarchy.

Entity cost—Binds the price model to a specific entity

Fixed cost—Defines the costs that Chargeback Manager cannot meter as resource usage but must be factored into the price, such as software licenses and maintenance fees.

Defining a billing policy We used vCenter Chargeback Manager to measure usage of the computing resources listed in Table 10.

Table 10. Computing resource and measurement

Resource Measurement

CPU The CPU usage is measured in GHz

Memory The memory usage is measured in GB

Storage The storage usage is measured in GB

vCPU The number of virtual CPUs in the virtual machine

External Network Transmit The external network usage for uploads measured in MB

External Network Receive The external network usage for downloads measured in MB




34

In Chargeback Manager, each computing resource can be charged using different attributes as shown in Table 11.

Table 11. Resource attributes

Attributes Description

Usage The virtual machine’s resource utilization. This attribute can be used only with CPU, memory, storage, disk I/O, network I/O, vCPU, and external network transmit and receive computing resources. For storage, the usage attribute indicates the total provisioned storage space for the virtual machine. For vCPU, the usage attribute indicates the vCPU count.

Reservation The amount of resources reserved for the virtual machine.

Allocation The amount of resources allocated for the virtual machine.

This attribute is useful when you want to enforce a minimum amount of resource to be charged, irrespective of the actual resource utilization or the resource reservation.

Size This attribute can be used only for memory.

You should name your billing policies according to service level, for example:

Billing policy—Basic: Set CPU=Usage, Memory=Usage

Billing policy—Committed: Set CPU= Allocation, Memory= Allocation

Billing policy—Dedicated: Set CPU= Allocation, Memory= Allocation1

Pricing model We suggest that you associate different pricing models with billing policies and set base rates for CPU/memory/network consumption. The base rate can differ depending on the cost variation of its DRS cluster. Table 12 provides an example pricing model.

Table 12. Pricing model settings

Pricing model Billing policy Base rate

PM_Foundation—Basic Billing Policy—Basic CPU: $3/GHz Memory: $0.5/GB

PM_Foundation—Committed Billing Policy—Committed CPU: $6/GHz Memory: $1/GB

PM_Foundation—Dedicated Billing Policy—Dedicated CPU: $8/GHz Memory: $1.5/GB

Note: The rates are samples and subject to change in each customer environment.

1 The parameter setting is based on VMware recommendation for vCloud director.



Define infrastructure specific cost We created different storage profiles by associating with different FAST VP policy in the datastore level. You can define different rate factors for each storage profile in addition to the allocation model as shown in Table 13.

Table 13. Infrastructure cost setting for datastores

Infrastructure cost—datastores Rate factor

Foundation_Bronze 1

Foundation_Silver 1.5

Foundation_Gold 2

Defining cost template You can define different cost templates with different rate factors for all kinds of resources. Apply the cost template to the entity for a specific period. You can customize it for a promotion or special discount for individual organizations or virtual machines, as shown in Figure 7.

Define cost templates Figure 7.

Defining entity cost After defining the cost elements, we bound them to a vCloud Director-synchronized hierarchy, as shown in Figure 8. When you bind the pricing model to the OvDC level, it automatically binds to all the associated vApps. The entity cost takes the highest precedence over any rate factors that are defined elsewhere.

Defining the entity cost Figure 8.



36

Defining fixed cost You can use fixed cost to meter the cost that cannot be measured directly, such as application license fees and hardware maintenance fees. You can define it as a one-time charge or a recurring charge.

After defining the elements, you can generate a cost report. A cost report provides the cost and utilization information for each computing resource in the organizational hierarchy.

You can also automatically generate on-demand cost reports and distribute them to users through the SMTP interface.

Overview

VMware vCenter Operations Manager together with the EMC Storage Resource Management Suite provide role-based dashboards and lead to an optimal team collaboration, as shown in Table 14. The combined software solutions provide efficient management of availability, performance, capacity, and health of the EMC cloud-enabled Infrastructure for SAP in part or as a whole.

This leads to fewer incidents, less downtime, fewer penalties and avoids multiple handoffs. In addition, you can identify unused applications and underutilized hardware.

Figure 9 presents an overview of the monitoring strategy in the foundation bundle. The SAP administrator monitors the application performance and availability through SAP Solution Manager. The administrator can log in to vCenter Operations Manager and EMC SRM with read-only authorization to speed up the root cause analysis process. The cloud administrator is responsible for monitoring and managing the underlying infrastructure and takes care of the vCloud director configuration. SAP administrators can serve individual or several organizations, as needed.

Integrated cloud management performance analysis



Monitoring strategy for the foundation bundle Figure 9.



38

Table 14 shows an example of role-based user profiles in the Cloud-enabled Infrastructure.

Table 14. Cloud-enabled Infrastructure user profile

Component Component authorization

Organization—end user

Enterprise—SAP administrator

Enterprise—Cloud administrator

SAP system X X

SAP Solution Manager

X

EMC Storage Resource Management

Read/report X X

Configure X

Alert X X

vSphere VM-OS X X

Server X

Network X

Storage X

vCloud Director Read X X

Ordering resources

X X

Configure X

vCenter Operations Manager

Read/report X X

Configure X



Table 15 lists the monitoring functions used in this solution.

Table 15. EMC SRM and VMware vCops monitoring functions

Function VMware vCops EMC Storage Resource Management Suite

Performance Performance data at the different levels include:

Data center

Cluster

ESXi server

Virtual machine level

Datastore

Metrics:

CPU usage

Disk I/O (virtual disk)

Memory usage

Network workload

EMC storage performance:

LUN throughput, IOPS, response time

FE Director, FE IOPS (VMAX)

SAN switch performance:

Port throughput/port link

Port utilization

Host performance:

Host IOPS, response time, throughput, queue length

Port link utilization

Host CPU utilization

Virtual machine performance:

Host devices—IOPS, throughput

Port link utilization

Host CPU utilization

Storage capacity

Cluster or host capacity

Virtual machine capacity

Datastore capacity

Metrics:

CPU

Memory

Disk I/O

Disk space

Network I/O

Raw capacity usage

Pool capacity

LUN capacity

Service level:

By array type

By FAST VP

By disk characteristics or RAID type

Reporting Host utilization report

Cluster or host capacity inventory report

Virtual machine capacity overview report:

Virtual machine list report

Waste, stress, and capacity report

Oversized virtual machines report

Storage real time and historical report

Storage capacity and performance report

Trending report

Enterprise-wide SLA report

Change tracking and reporting

Alert Badge score alerts

Fault alerts

Administrative alerts

Email or SNMP notifications

VMAX health

Switch health

EMC storage performance thresholds

Email or SNMP notifications



40

This chapter discusses the integrated management, which allows:

Full visibility of the virtual data center

End-to-end performance analysis

Configuration validation and compliance

Storage capacity analysis

Predefined and customized intelligence reports

Full visibility of the virtual data center

VMware vCenter Operations Enterprise is designed to simplify and automate operations management. It provides the automated performance analytics for enterprise IT systems and business processes. It also integrates patented analytics to provide the intelligence and visibility needed to ensure service levels are met in dynamic virtual and cloud environments.

VMware vCenter Operations offers the following values:

Real-time performance monitoring and analysis—Displays role-based performance dashboards

Smart alerts—Informs you of impending problems so you can avoid major performance or capacity issues

On-demand analytics and reporting—Provides capacity planning and optimization for long-term enterprise systems

Real-time performance monitoring and analysis The VMware vCenter Operations Enterprise role-based performance dashboards leverage the analysis of individual metric behaviors to determine the health of an enterprise in part or as a whole.

The flexibility of the vCenter Operations Enterprise architecture allows you to automatically calculate the health of any desired “container” of resources (for example, the individual servers and devices or the combined health of a collection of resources, including technology silos such as an “SAP tier” that contains database, central instance, dialog instance, and so on).

Figure 11 shows an example of a customized dashboard for the data center administrator:

In the vCenter relationship widget, the environment has two data centers with six ESXi servers hosts, 68 virtual machines, and 30 datastores. The dashboard shows that:

One virtual machine is very busy (red)

Some virtual machines are shut down (gray)

Some datastores are undergoing a heavy workload (yellow)

In the ESXi server health scoreboard, the status of ESXi servers is displayed as:

Green means a healthy status

Orange means there are serious problems



The virtual machine/datastore workload and response time heat map clearly shows how the workload is allocated across the data center. In this example, size represents workload and color represents the response time.

In the vCloud Director heat map, size represents each organization’s CPU usage, while color represents the CPU contention. The data center administrator can customize the resource scoreboard and the sapvsedb scoreboard.

vCenter Operations Manager performance dashboard Figure 11.

Smart alert When vCenter Operations Enterprise detects significant abnormal behavior exceeding its expected “noise” level, a smart alert pops up to prevent impact to end users. You can monitor Smart Alerts in a variety of ways, including viewing the vCenter Operations Enterprise alert summary dashboards or using email notifications.



42

Smart Alerts example Figure 12.

On-demand analytics and reporting The VMware vCenter Operations’ on-demand analytics and reporting offers the ability to gain a deep understanding of the known behavior patterns of your applications and supporting IT systems. These features enable the cloud administrator to optimize the operation infrastructure that supports mission-critical applications.

Figure 13 shows an example of how the most problematic or most volatile resources manifest in vCenter Operations Enterprise. In this example, the vCenter Operations Manager can determine that the virtual machine SAPVM3SQLCEN is the root cause of the problem. You can use this tool to determine not only the problematic resources, but also the technology tiers and applications that are causing problems.

On-demand analytics showing the least healthy or most volatile resources Figure 13.

Cloud capacity usage is critical for both real time monitoring and historical reporting that determines future hardware expenditures.

Capacity usage reporting can help cloud users determine the status of their resource pools, including:

Undersized or oversized virtual machines

Virtual machine resources that are nearing depletion



Figure 14 illustrates the report showing oversized and undersized virtual machines with recommended vCPU and memory.

Oversized and undersized virtual machines report Figure 14.

End-to-end performance analysis

If the underlying infrastructure fails to meet the workload requirements, the performance of the SAP system will be affected. EMC Storage Resource Management Suite enables you to analyze host, application, and storage performance to identify workloads and assess impacts on the application.

SAN performance To address the cloud administrator’s need for performance management, EMC SRM provides an end-to-end view, allowing insight into the complex relationships in large virtualized environments. EMC SRM automatically collects performance data for the entire path, including virtual machines, hosts, switches, and storage. It also integrates with EMC element managers, including Unisphere for VMAX and Connectrix Manager Converged Network Edition, to improve productivity.



44

Figure 15 shows the end-to-end performance view.

EMC SRM end-to-end performance view Figure 15.

As shown in Figure 15, customers can easily filter a specific SAP host and view its end-to-end topology. Figure 15 presents a compact, high-level topology map showing the SAN elements connected to the host, while filtering out all non-pertinent data. If a customer wants to see detailed performance or connectivity path information, they can select the object of interest such as the ESXi host, switches, or storage.

Consolidated Alert Console EMC Storage Resource Management presents a single view of performance and healthy alerts from EMC Unisphere and CMCNE. Alerts that appear or are updated in EMC SRM can be sent to a problem tracking application through SNMP traps for dispatch to the appropriate organization. As shown in Figure 16, the use space of the virtual pool “SAS15K_RAID1” in VMAX has crossed a predefined threshold, and the level is above 80 percent.



EMC SRM alert view Figure 16.

EMC SRM performance thresholds enable customers to set the threshold metrics of select Configuration Items (CI) in the storage environment and to keep track of the resource utilization limit and performance. If thresholds are breached, the software generates warnings or critical alerts and emails them to the storage administrator, so problems can be proactively resolved before issues affect users. Figure 17 displays the performance thresholds set for this solution.

Setting performance thresholds Figure 17.

Configuration validation and compliance

EMC SRM helps SAP and cloud administrators manage compliance with internal configurations policies, vendor design and interoperability guidelines, and industry best practices across virtual infrastructures. In this solution, EMC SRM helps to improve service levels and operational efficiency by:

Providing best practices policy management

Tracking changes

Flagging configuration breaches for further investigation



46

Best practices policy management EMC SRM validates storage configuration for the cloud-enabled infrastructure best practices. As shown in Figure 18, we created a policy to enforce array configuration best practices, validate Symmetrix port flag settings for VMware, and set notifications when customers breach the policy. This enables virtual data center administrators to provide consistent service to the business for the cloud-enabled infrastructure.

Creating a policy to validate Symmetrix port flag settings for VMware Figure 18.

Change tracking Figure 19 displays how EMC SRM detects the movement of virtual machines across physical servers and virtual disk configuration changes in a virtualized environment. Administrators can keep an audit trail of any change. This historical record enables administrators to analyze incident causes or trends and improve management processes and planning.



Virtual machine activity detection Figure 19.

Investigating configuration breaches EMC SRM continuously monitors the cloud-enabled infrastructure. If a change differs from the desired state for a rule, EMC SRM triggers a configuration breach. Figure 20 shows how EMC SRM identifies areas where the path of one virtual machine is missing, determines which objects caused the breach, and provides the steps needed to resolve the problem. Administrators can drill down into a breach to investigate its cause, view recommended remedial actions, and add or view notes and tags related to the breach.

EMC SRM Investigate Breach Figure 20.

Storage capacity analysis

EMC SRM allows you to report on several capacity metrics based on specific use cases that reflect the everyday needs of storage administrators. It provides a dashboard view that offers insight into where and how capacity is being consumed and when more will be required. You can monitor storage usage information including:

Used versus free storage

Total raw storage utilization

Array utilization by purpose

Array utilization by FAST policy



48

Array utilization by tier

Array-based storage chargeback

Total federated storage capacity

Figure 21 shows EMC SRM capacity monitoring.

Monitoring storage capacity with EMC SRM Figure 21.

Predefined and customized intelligent reports

EMC SRM provides an enterprise reporting and analysis tool that provides real-time, historical, and projected visibility into the network, data-center, and cloud infrastructure performance. Administrators can have a complete view of the reports from a single interface.

EMC SRM helps virtual data center administrators improve performance and capacity management processes, delivering high quality IT services at more-efficient costs. It is packaged with an extensive set of predefined Report Packs. In this case, EMC SRM provides the following reports:

Real-time and historical report

Investigate trending report

SLAs reports



Real-time and historical report EMC SRM monitors the performance, utilization, and capacity of virtual data center components in real time and provides a historical report as shown in Figure 22. In this case, we can make informed virtualization decisions by gathering configuration and performance data on virtualized environments.

Real time and historical reporting Figure 22.

We can also schedule reports to send at fixed times to a list of recipients. Administrators can choose a format for the report as shown in Figure 22.

Trending report Trending reports based on key metrics can help virtual data center administrators avoid SLA violations. As shown in Figure 23, we can identify potential issues before they occur and plan growth with accurate data using the trending reports.



50

Trending and Forecasting report and email configuration Figure 23.

SLA reports Virtual data center administrators can quickly measure operational results, check compliance with contracted service levels, and document SLA reports through EMC SRM. Figure 24 shows the actual and forecasted SLA value of the devices in the virtual data center. EMC SRM identifies SLA issues with check marks and color changes, and reduces the time spent monitoring service levels.

Monthly SLA overview Figure 24.



FAST VP allows the cloud administrator to leverage the storage tiering abilities of the storage array. It enables automatic storage tiering below both the virtual machine and VMDK level. It is the storage array rather than vCloud Director that provides the tiering. The storage array is responsible for managing the data placement, moving the data to the appropriate location where the vApps benefit from the performance required. FAST VP maximizes utilization of Flash drive capacity for high IOPS workloads and maximizes utilization of NL SAS drives for capacity-intensive applications.

FAST VP provides the following benefits for Cloud-enabled Infrastructures:

Service providers can offer a greater range of service levels

Service providers can optimize and protect their vCloud Director environments on Flash, SAS, and NL SAS drives

Multiple tiers can be aligned and integrated with storage profile

VMware vCenter Chargeback Manager can report on and bill for FAST VP enabled storage

Service providers can meet a greater number of SLAs by taking advantage of tight integration between EMC and VMware products

This section discusses the implementation of FAST VP for the cloud-enabled infrastructure. Table 16 details the tier design.

Table 16. Virtual pool configuration and FAST VP tiers

Storage pool Disk size and technology

RAID protection

Tier name

Flash drive 200 GB Flash Raid5 (3+1) Flash

SAS15K_RAID1 450 GB SAS RAID1 SAS

NLSAS_R6 2 TB NL SAS RAID6 (6+2) NL SAS

The EMC Solutions Enabler Symmetrix Array Controls CLI v7.5 Product Guide provides detailed information about VMAX virtual provisioning.

FAST VP for EMC Symmetrix VMAX Theory and Best Practices for Planning and Performance provides detailed information on best practices and design considerations for FAST VP.

We designed FAST VP policies to deliver service levels according to the customer’s performance needs. Table 17 shows the FAST VP policies used in this implementation. The FAST VP policy assigned and the storage consumption of a virtual machine serve as the base for the chargeback mechanism described in the Infrastructure chargeback section.




52

Table 17. FAST VP policies

Policy name Flash tier SAS tier NL SAS tier

Gold 15% 85% 0%

Silver 4% 28% 68%

Bronze 0% 20% 80%

Figure 25 shows the storage service-level relationship throughout the cloud-enabled infrastructure. At the storage layer, FAST VP polices define a level of service for sets of LUNs presented to the ESXi clusters. Within VMware vCenter, VMFS datastores are created on the EMC VMAX LUNs and are grouped into storage DRS clusters according to their service-level.

Service-level relationship throughout the cloud-enabled infrastructure Figure 25.

In this solution, we used a standard datastore size of 2-TB thin LUNs as the base configuration for the VMAX storage LUNs presented to ESXi hosts. This provides for up to 1 PB of shared storage per cluster (256 shared datastores). We configured the VMAX volumes as striped Meta devices (128 GB Meta member).

We used Unisphere Storage Templates to create the VMAX LUNs. This enables storage administrators to quickly and easily create standard volumes. Figure 26 shows the Unisphere storage template for creating a 2-TB Meta volume. We configured the details of the Meta volume under the Advanced Options, which is a one-time effort that can be reused multiple times.

Vmware SDRS

Datastore

Clusters

VMware ESXi HA/DRS Cluster

EMC VMAX with FAST VP

FAST Policy

High Performance

Medium Performance

Low Performance

Redundant

FC SAN

Gold

Bronze

Silver



Defining a Storage Template in Unisphere for VMAX Figure 26.



54

Overview

It is a critical requirement for every data center to have a robust networking and security solution. The cloud-enabled infrastructure uses VMware vCloud networking and security, which virtualizes the network and security layer of the IT infrastructure. vCloud networking and security integrates with vCenter and vCloud Director to centralize management and deploy logical constructs for a multi-tenant model, including firewalls and network configurations.

In this section, we demonstrate the following design:

How SAP system networks are isolated between organizations

How client hosts access their own SAP systems while access to other organization’s SAP systems is prohibited

VMware vCloud networking and security design

There are three types of networks in vCloud Director:

External network—External networks are used to provide a connection outside the vCloud environment. In this solution, external networks are networks that are connected to the corporate network. Through the external network, client hosts, admin hosts, and management servers can connect to SAP systems based on access rules.

Organization network—We customized the Organization networks according to organization requirements. This configuration ties vApps together to provide interconnection between vApps, outside connections between vApps, and connection to external networks.

vApp network—We used vApp networks to provide a connection between virtual machines that are within one vApp.

Predefined port groups in vSphere back the external networks. Port groups that are created dynamically by vCloud Director back organization networks and vApp networks. You can configure these networks to directly connect to a higher network level, to be routed to a higher network level, or to connect internally only. Figure 27 depicts a reference network design.




Networking and security design Figure 27.

Different installation options are supported in this solution. We used the centralized installation option when deploying SAP systems. Each SAP system is encapsulated into one vApp. Within the vApp, we deployed one virtual machine, with two vNICs: one for client access (CL) and one for administration purposes (Admin).

The above network configuration is one example of many options and not to be considered the default configuration for all environments. Networking and security should be carefully planned based on the requirements of each organization.

The following sections explain how we designed and connected these networks.

vApp network Figure 27 shows the vApps directly connected to OrgNet without a vApp network. This is based on the assumption that OrgNet is a trusted network inside the organization and that there is no need to isolate the SAP systems within the organization. VMware recommends that you keep the network configuration simple because the more networks you have, the more complex network management becomes.



56

Organization network Table 18 shows the organization networks in this solution along with their network names, IP pools, and gateways.

Table 18. Organization networks

Organization Network name IP pool Gateway

Org A OrgNet 192.168.30.10 - 250 192.168.30.1

Org A OrgNet_Admin 192.168.40.10 - 250 192.168.40.1

Org B OrgNet 192.168.10.10 - 250 192.168.10.1

Org B OrgNet_Admin 192.168.20.10 - 250 192.168.20.1

Two types of organization networks are defined in each organization:

OrgNet —A network that provides a link between the SAP systems inside the cloud and client hosts outside the cloud. OrgNet also provides interconnections between SAP systems, for example, between DEV and PRD inside the organization.

OrgNet is a routed network, which connects to the external network, CorpNet, through the edge gateway. Figure 27 shows Edge 02 and Edge 04 as the firewall and NAT device in each organization respectively. The main functions of these edge gateways are to:

Allow only the IP ranges of client hosts from their own organizations to access SAP systems.

Block all ports that are not necessary for client hosts to access SAP systems.

Open essential ports like SAP message service (port number: 36xx) and SAP dispatch service for client access (port number: 32xx). This protects SAP systems from internal attacks and threats.

Translate IP addresses from organization networks to external networks. For example, CL interface of SAP ERP DEV in Org A is configured with IP address 192.168.10.10 on OrgNet. It will be translated to 192.168.101.30, to which client hosts can connect.

OrgNet_Admin—A network that provides a link between the SAP systems inside the cloud and admin hosts outside the cloud. The admin host has more privileges than the client host. One of the privileges is direct access to OS through SSH.

OrgNet_Admin is a routed network, which connects to the external network CorpNet_Admin through the edge gateway. Figure 27 shows Edge 01 and Edge 03 as the firewall and NAT device in each organization respectively. The main functions of these edge gateways are to:

Allow only administrator hosts to access the SAP systems and OS.

Block all ports that are not necessary for administrator hosts to access SAP systems.

Open essential ports such as SAP message service, SAP dispatch service, and SSH service according to business requirements.



Translate IP addresses from organization networks to external networks.

External network We defined three external networks as shown in Table 19.

Table 19. External networks

Network name Backed by port group

VLAN IP pool Gateway

CorpNet dvPG_Corp 101 192.168.101.10 - 250 192.168.101.1

CorpNet_Admin dvPG_Corp_Admin

102 192.168.102.10 - 250 192.168.102.1

CorpNet_Public Public Network 10 10.110.86.110 - 250 10.110.86.1

The external networks in this solution include:

CorpNet—A network that allows client hosts to connect to SAP systems. Access rules are defined in Edge 02 and Edge 04, which block interconnections between SAP systems across organizations and allow access from the organization’s own client hosts and other management servers like SAP Solution Manager.

CorpNet_Admin—A network that allows administrator hosts from the SAP basis support team to access restricted resources such as OS and DB. Like CorpNet, access rules are defined in Edge 01 and Edge 03, which block the interconnection between SAP systems across organizations and allow access only from specific admin hosts in centralized IT.

CorpNet_Admin is backed by port group dvPG_Corp_Admin and is a trusted network between the SAP system and administrator hosts. Security devices should be used on this network to protect and isolate them from other external networks on the physical network layer and they should be properly tracked and audited.

CorpNet_Public—A network that provides Internet access.



58

Table 20 describes the IP address range of external networks for each organization.

Table 20. External network assignments for organizations

Organization Network Name IP Range

Org A CorpNet 192.168.101.80 - 129

Org A CorpNet_Admin 192.168.102.80 - 129

Org B CorpNet 192.168.101.30 - 79

Org B CorpNet_Admin 192.168.102.30 - 79

Network pool Network pools are used as a template to create networks at the organization and vApp levels. Two types of organization networks require network pools:

Routed organization networks that connect to an external network

Internal organization networks

All vApp networks are built on network pools. See the vCloud Director Administrator's Guide for detailed information about network pools.

In this solution, vCloud Director dynamically creates vAppNet, OrgNet, and OrgNet_Admin based on the network pool. We used a vCloud Director Network Isolated (VCD-NI) backed network pool to simplify the network design. Due to MAC-in-MAC encapsulation by VCD-NI, be sure to set MTU to 1524 in the following places to avoid package fragmentation and performance downgrade:

Distributed switches in vSphere that vCloud networks are using

Physical switches that connect ESXi hosts

Network pool settings in vCloud Director



SAP system design

This chapter describes the design considerations used to deploy SAP systems in the cloud-enabled infrastructure, including:

Compute

Storage

Network

Installation type

Allowing CPU and memory over-commitment would boost compute resource utilization but would also result in performance degradation for SAP systems. In general, SAP recommends not overcommitting CPU or memory for the following reasons:

For memory resources, the SAP kernel allocates memory permanently and does not release it again. If a memory outage on an ESXi server occurs, vSphere uses the swap file on the datastore as memory, which causes performance degradation.

For CPU resources, SAP successfully runs performance tests in virtual machines (utilizing all available virtual CPUs to 100 percent), which overcommits the host system up to 150 to 200 percent. The performance degradation of the virtual machine is linearly reciprocal to the over commitment.

In a cloud environment, the different service levels have their own design considerations:

For Dedicated service levels, compute resources are fully guaranteed. If the allocation of CPU and memory is 50 GHz and 100 GB, the guaranteed CPU and memory values are 50 GHz and 100 GB. Cloud administrators can set reservations and limitations on each single virtual machine. We recommend you set the "Memory Reservation" to the amount of memory configured for most critical SAP systems running in Dedicated OvDC.

For Committed service levels, compute resources are guaranteed at 75 percent of total allocation. This allows 25 percent of over-commitment on CPU and memory. Cloud administrators cannot set reservations and limitations on virtual machines. Since Committed service levels target less critical or low workload production SAP systems, it is reasonable to allow a certain degree of over-commitment, which increases resource utilization and lowers cost. The cloud infrastructure administrator should regularly monitor total resource usage on ESXi servers. If severe resource contention lasts, the cloud infrastructure administrator can increase total allocable resources by assigning additional compute and memory resources.

For Basic service levels, memory resource is guaranteed at 75 percent and there is no guarantee on CPU resource availability. This allows maximized compute resource utilization. We recommend you review the-size report in vCops regularly and adjust memory and CPU on each virtual machine to the appropriate value.

Overview

Compute



60

Here are some general guidelines for allocating compute resources:

Determine the proper sizing of vCPU before going live. More vCPUs in virtual machines can increase scheduling overhead.

If the exact workload is unknown, size the virtual machine with a smaller number of vCPUs initially and increase the number later if necessary.

Even though the current version of vCloud Director provides all storage of an SAP system out of the same storage profile, you still can follow SAP’s best practice of file systems set up (especially for critical SAP production systems). The file systems can lie on different VMDK files on single or multiple datastores of the same performance level as shown in Table 21. The additional advantage is: in case of possible changes of vCloud Director in future to also support file systems of an SAP system to be delivered from different performance levels that are already prepared.

Table 21. Sample file system configuration for database instance

Logical volume Volume group File system

fslvorastaging orastagingvg /oracle/stage

fslvoraclient oraclientvg /oracle/client

fslvorahome orahomevg /oracle/VSE

fslv11203 11203vg /oracle/VSE/11203

Fslvsaparch saparchvg /oracle/VSE/oraarch

fslvsaptrace saptracevg /oracle/VSE/saptrace

fslvoriglogA saporalogAvg /oracle/VSE/origlogA

fslvoriglogB saporalogBvg /oracle/VSE/origlogB

fslvmirrlogA sapmirrlogAvg /oracle/VSE/mirrlogA

fslvmirrlogB sapmirrlogBvg /oracle/VSE/mirrlogB

fslvsapdata1 sapdata1vg /oracle/VSE/sapdata1




Fslvsapmnt sapmntvg /sapmnt/VSE

Fslvusrsap usrsapvg /usr/sap/VSE

Fslvtrans transvg /usr/sap/trans

fslvusrhome usrhomevg /home/vseadm

Storage



Use multiple virtual SCSI Controllers for the database instance. The use of multiple virtual SCSI Controllers allows several parallel I/O operations inside the guest OS. We also highly recommend you separate the log I/O traffic from the datafile I/O traffic through separate virtual SCSI controllers. As a best practice, you can use one controller for OS and Swap, one controller for database log, and one or more controllers for database data files (depending on the amount and size of database data files).

For optimal throughput, use the paravirtual SCSI Controller (PVSCSI). Be sure that you install the VMware Tools first. See VMware KB 1010398 for details: Configuring disks to use VMware Paravirtual SCSI (PVSCSI) adapters.

For best network performance, use a vmxnet3 network card. See VMware KB1001805 for details: Choosing a network adapter for your virtual machine.

Central installation and distributed installation are supported. A High Availability configuration is explained in the HA/Mobility bundle.

Install the latest version of VMware Tools in the guest operating system for optimal performance. See SAP Note 1122388 - Linux: VMware vSphere configuration guidelines for detailed guidelines from SAP.

Network

Installation type

Other

http://kb.vmware.com/kb/1010398



https://service.sap.com/sap/support/notes/1122388

https://service.sap.com/sap/support/notes/1122388



62

Validation of functions

We tested the following use cases to validate the solution:

Use case 1: Virtual data center for SAP

Use case 2: Automated provisioning, service catalogs, self-services, and chargeback

Use case 3: Integrated cloud management and performance analysis

Test scenario

By implementing the foundation bundle, customers can pool different physical and virtual resources into a single abstraction layer that allows resources to be dynamically and elastically assigned and reassigned based on the multi-tenancy model.

Success criteria

The virtual datacenter for SAP should demonstrate the following benefits:

Multi-tenancy—Virtual machines from different organizations can occupy the same resource pool. You can regulate resource allocation using a set of policies.

Security—Using the design in the VMware vCloud networking and security design section, the foundation bundle must be able to:

Isolate SAP, DB, and OS data access within an organization

Restrict end user access from other organizations

Enable Admin access across organizations

Validation procedure

The validation procedure includes Multi-tenancy and Security testing.

Multi-tenancy We validated the following scenarios in the multi-tenancy function:

PvDC is configured and ready for consumption

Organization vDC is configured with different service levels

Overview

Use case 1: Virtual data center for SAP



As shown in Table 22, we built Standard and Advanced PvDCs to provide resources for constructing Organization vDCs.

Table 22. PvDC resources

Standard PvDC Advanced PvDC

DRS cluster Standard cluster-1 Advanced cluster-1

Storage profile Bronze Silver

Gold

Datastore cluster

Bronze Tier Silver Tier

Gold Tier

We built Organization vDCs according to the OvDC design mentioned in the Virtual data center for SAP section.

Table 23 depicts the detailed parameters of each OvDC.

Table 23. OvDC parameters

Org A

Basic

Org A

Committed

Org B

Basic

Org B

Dedicated

Allocation Model Pay as you go Allocation pool Pay as you go Reservation pool

CPU allocation 25 GHz 50 GHz 40 GHz 50 GHz

CPU resource guaranteed

0% 75% 0% 100%

vCPU speed 1 GHz 2 GHz 1 GHz Any

Memory allocation 100 GB 150 GB 100 GB 200 GB

Memory resources guaranteed

75% 75% 75% 100%

Maximum number of virtual machines

Unlimited Unlimited Unlimited Unlimited

Storage profile Bronze Silver

Gold

Bronze Silver

Gold

Thin provisioning Enabled Enabled Enabled Disabled

When thin provisioning is enabled in the OvDC settings, vDisks of virtual machines are created in ‘thin’ format. Otherwise, vDisks of virtual machines are created in “zeroedthick.”

We disabled thin provisioning in Dedicated OvDC to ensure storage performance. For Basic and Committed OvDCs, we enabled thin provisioning to maximize storage utilization.



64

Security We validated the following security scenarios:

Client hosts can access their own SAP systems in their organizations through SAPGUI

Client hosts cannot access other organizations’ SAP systems

Administrator hosts can access the operating systems of SAP systems through SSH across organizations

We built the organizations and external networks according to the network design in the Cloud networking and security for SAP section. Client hosts from different organizations and administrator hosts from centralized IT are external to the cloud environment. They access SAP systems through the external networks CorpNet and CorpNet_Admin. The IP addresses of all SAP systems within the cloud were translated by the vShield edge gateway, which serves as a firewall that controls access rules. Figure 28 shows an overview of the two organizations’ IP addresses.

IP addresses Figure 28.

Each SAP system has two IP addresses connected to the organization network; one for client access, and the other for administrator access. IP addresses are translated when the packages leave the cloud environment. Table 24 and Table 25 show the detailed information on NAT.



Table 24. NAT for Org A

SAP system Interface Original IP Translated IP Edge gateway

ERP QAS Client 192.168.30.10 192.168.101.80 Edge 01

ERP QAS Admin 192.168.40.10 192.168.102.80 Edge 02

ERP PRO Client 192.168.30.13 192.168.101.81 Edge 01

ERP PRO Admin 192.168.40.11 192.168.102.81 Edge 02

Table 25. NAT for Org B

SAP system Interface Original IP Translated IP Edge gateway

ERP DEV CL 192.168.10.10 192.168.101.30 Edge 03

ERP DEV Admin 192.168.20.10 192.168.102.30 Edge 04

ERP PRO CL 192.168.10.13 192.168.101.31 Edge 03

ERP PRO Admin 192.168.20.11 192.168.102.31 Edge 04

Client hosts and administrator hosts only know about external network IP addresses. Organization networks are transparent to the client hosts.

We created firewall rules on each edge gateway to control access. Table 26 to Table 28 show detailed information about our firewall rules.

Table 26. Firewall rules for client access on Edge 01

ID Name Source Destination Protocol Action

1 Dispatcher 10.2.0.0/16:Any 192.168.101.0/24:3200 TCP Allow

2 Message Server

10.2.0.0/16:Any 192.168.101.0/24:3600 TCP Allow

3 Outbound internal:Any Any:Any ANY Allow

Table 27. Firewall rules for client access on Edge 03



2 Message Server

10.1.0.0/16:Any 192.168.101.0/24:3600 TCP Allow




66

Table 28. Firewall rules for admin access on Edge02 and Edge 04


1 SSH 10.200.1.0/16:Any 192.168.102.0/24:22 TCP Allow


3 Message Server

10.200.1.0/16:Any 192.168.101.0/24:3600 TCP Allow


Verifying network access within an organization

Table 29 shows the result of client host in Org A accessing an SAP system in Org A.

Table 29. Client host in Org A accesses an SAP system in Org A

Client host IP 10.2.10.100

SAP system / IP ERP PRD / 192.168.101.81

Port 3200/TCP (SAP dispatcher)

Result Allowed

Figure 29 depicts the firewall logs:

Firewall logs for client host in Org A accessing SAP system in Org A Figure 29.

Access from a client host (10.2.10.100) in Org A to SAP ERP PRD (192.168.101.81) in Org A was allowed. IP address of SAP ERP PRD was translated to 192.168.30.13.



Verifying secure network in multi-tenancy

Table 30 shows the result of client host in Org A accessing an SAP system in Org B.

Table 30. Client host in Org A accesses SAP system in Org B


SAP system/IP ERP PRD/192.168.101.31


Result Denied

Figure 30 depicts the firewall logs:

Firewall logs for client host in Org A accessing SAP system in Org B Figure 30.

Access from a client host (10.2.10.100) in Org A to SAP ERP PRD (192.168.101.31) in Org B is denied. The SAP ERP PRD IP address is translated to 192.168.10.13.

We performed the same test for the client host in Org B accessing SAP systems in Org B and Org A. Table 31 and Table 32 show the test results.

Table 31. Client host in Org B accesses SAP system in Org B


SAP system/IP ERP PRD/192.168.101.31


Result Allowed



68

Table 32. Client host in Org B accesses SAP system in Org A


SAP system / IP ERP PRD / 192.168.101.81


Result Denied

Administrator host in centralized IT accesses OS in Org A and Org B

Table 33 shows the result of the administrator host in centralized IT accessing the OS in Org A and Org B.

Table 33. Administrator host in centralized IT accesses the OS in Org A and Org B


SAP system / IP Org A ERP PRD / 192.168.102.81

Org A ERP QAS / 192.168.102.80

Org B ERP DEV / 192.168.102.30

Org B ERP PRD / 192.168.102.31

Port 22/TCP (SSH)

Result Allowed

Figure 31 depicts the following firewall logs:

The first entry shows that access from the administrator host (10.200.1.100) to SAP ERP PRD (192.168.102.81) in Org A is allowed and that the SAP ERP PRD IP address is translated to 192.168.40.11.

The second entry shows that access from the administrator host (10.200.1.100) to SAP ERP QAS (192.168.102.80) in Org A is allowed and that the SAP ERP QAS IP address is translated to 192.168.40.10.

The third entry shows that access from the administrator host (10.200.1.100) to SAP ERP DEV (192.168.102.30) in Org B is allowed and that the SAP ERP DEV IP address is translated to 192.168.20.10.

The last entry shows that access from the administrator host (10.200.1.100) to SAP ERP PRD (192.168.102.31) in Org B is allowed and that the SAP ERP PRD IP address is translated to 192.168.20.11.



Firewall logs for the administrator host accessing SAP system in Org A and Figure 31.Org B

Summary

With vCloud Director, corporate IT can provide a multi-tenancy infrastructure for different organizations running SAP systems. Cloud administrators can setup vDCs with different service levels to meet the organizations’ business requirements. vCloud networking and security enables fast network provisioning within the cloud environment and secure access for clients outside of the cloud environment. NAT and firewall on the edge gateway provides the flexibility to implement access control.

Test scenario

This use case includes the following scenarios:

Simplified provisioning—The foundation bundle allows easier service allocation and provisioning. Demonstrate how the services are consumed by:

The SAP administrator

An SAP developer in Org B

Chargeback—The foundation bundle can:

Accurately measure resources and services consumed by organizations

Automatically reflect changing vApp resources in vCenter Chargeback and adjust the cost accordingly

Validation procedure

Simplified provisioning The SAP administrator for Org B can log on to the organization management link to manage, monitor and control vApps and resources that belong to the organization, as shown in Figure 32.

Use case 2: Automated provisioning, service catalogs, self-services, and chargeback



70

Organization management page in vCD Figure 32.

The SAP administrator can also deploy new applications with the vApp template with just a few clicks of a button. In this solution, we prepared a foundation operating system environment preconfigured to meet all the configuration and software requirements needed by SAP. All the agents, such as VMware Tools and backup agents were preinstalled in this environment, so this template was ready to deploy in the new SAP system without any additional configuration. Templates can be imported into vCloud Director either by importing a virtual machine from vCenter, or by importing an OVF file directly. We predefined templates, one for the SAP ABAP stack system and one for the SAP JAVA stack system, as shown in Figure 33.

Displaying the templates available in the vCD catalog Figure 33.



Figure 34 shows the steps used to deploy a new vApp from the catalog.

Adding a vApp from the catalog Figure 34.

Figure 35 shows the main steps needed to complete the deployment of a vAPP configuration.

Configure steps to add a vApp from the catalog Figure 35.

1. Select template

2. Naming vAPP

3. Naming VM

4. Configure Network



72

After the deployment was completed, we powered on the virtual machine to reset the specific static IP to the virtual machines related to the vApp as shown in Figure 36.

Managing the new created virtual machine in vCD Figure 36.

After deploying the new application, the cloud administrator can find the new entry under the organization of Org B in vCenter Chargeback Manager as shown in Figure 37.

Displaying the hierarchy in vCenter Chargeback Manager Figure 37.

The newly deployed vApp SAPDM1 was already reflected in the Chargeback hierarchy of Org B.

Accurately measure resources and services consumed by organizations We generated a weekly report for OrgA_Basic OvDC. SAP virtual machines under this OvDC were in the service category, Basic. As depicted in Figure 38, OrgA_Basic OvDC contains one vApp named SAP ERP 6.0 EHP QAS. CPU, memory, and storage resources are charged separately.



Cost report for OrgA_Basic Figure 38.

Changing the vApp resource automatically reflects in Chargeback Manager In the virtual machine event, we can find the virtual machine sapvsedb was off at 00:03:23 and was back on at 07:57:12 on April 9th. Figure 39 shows the event log in vCenter.

Event log in vCenter Figure 39.



74

The change in the virtual machine’s state appears in the cost report as shown in Figure 40.

Virtual machine state change appears in the cost report Figure 40.

We found two entries for CPU and memory: one entry was made before the power off, and the other entry was made after the power on. The time exactly matched what we observed in the tasks and events change infrastructure.

We also observed that from April 27th to April 28th, the datastore was changed three times. Figure 41 shows the datastore change reflected in the cost report.

Virtual machine datastore change reflected in the Chargeback report Figure 41.



The first change was to add a 200 GB disk from the silver datastore to extend PSAPTEMP table space to rebuild the index for a large table.

The storage profile of this SAP virtual machine was upgraded from silver level to gold level, which is reflected by the rate factor increasing from 1.5 to 2.

Everything was done non-disruptively.

Summary

The SAP administrator or SAP developer can easily provision virtual machines from a catalog in its own organization through vCloud Director. After being authorized by the cloud infrastructure administrator, the SAP developer can log into the vCloud Director portal to provision an SAP standard application on demand through the vCD catalog. The addition or removal of virtual machines is automatically reflected in the Chargeback Manager reports. Billing from the Chargeback Manager is charged according to the resources consumed.

An SAP system experiences an abnormally high workload. In the first step, we validated VMware's fully automated Distributed Resource Scheduling functionality. In the second scenario, we demonstrated how the SAP administrator performed a high speed root cause analysis, using vCops, EMC Storage Resource Management Suite, and Unisphere to analyze the performance and optimize the storage resources.

Success criteria

The evaluation is based on the following criteria:

Fully automated resource pooling—the CPU and storage react automatically by regulating the resources as needed, while allowing enough room for administrators to control the process.

Root cause analysis and problem resolution—validate the effectiveness of the monitoring tools in determining the root cause and make recommendations based on the data.

Scenario 1: On-demand resource pooling

For demonstration purposes, we set the VMware DRS Automation level to Manual before simulating a heavy SAP workload.

The SAP administrator received an incident report from EMC SRM indicating that the CPU utilization of the virtual machine SAPVSEDB had reached 80 percent, as shown in Figure 42.

Use case 3: Integrated cloud management and performance analysis



76

Virtual machine performance tab Figure 42.

By using vCops to investigate further, the SAP administrator observed high CPU usage on the ESXi host where the virtual machine resided, as shown in Figure 43.

ESXi CPU usage in vCops Figure 43.

We resolved this problem by moving the virtual machine to a host with a lower CPU utilization. We set the VMware DRS to automatically migrate the virtual machine to a different host to balance the workload and automatically avoid contention issues.



The Cloud administrator changed the Automation level setting from Manual to Fully automated, as shown in Figure 44.

DRS setting in vCenter Figure 44.

Using EMC SRM, the SAP administrator detected the virtual machine vMotion, as shown in Figure 45.

EMC SRM detects virtual machine migration Figure 45.

Analysis After the automatic migration, the host CPU usage dropped by 45 percent as shown in Figure 46.

CPU usage after automatic migration Figure 46.



78

Scenario 2: Root cause analysis and problem resolution

In this scenario, we demonstrate how the SAP administrator performed a high-speed root cause analysis, using vCops, EMC Storage Resource Management Suite, and Unisphere to analyze the performance and optimize the storage resources.

Validation procedure We used several SAP systems and set up a test environment as follows:

1. We measured the performance of an SAP ECC system under a Silver storage profile by generating an I/O-intensive workload on an SAP system. We used 1,000 dialog SD (Sales and Distribution module) users and 20 resource-intensive background jobs to simulate a real environment.

2. We benchmarked the ST03N response time to fall within 800ms to 1,000ms before the test, such that approximately 40 percent came from the database time. This was to ensure that the database time drove the overall system performance.

3. We doubled the number of users from 1,000 to 2,000 and used exactly the same workload before and after the performance issue was resolved.

4. The SAP administrator used the monitoring tools to analyze the performance and observed how the data center responded to this increase in workload. The administrator then identified the bottleneck and recommended a solution.

5. Following the recommendation, we observed the difference in performance subjected to the same 2,000-user workload, using the same tools.

Figure 47 shows the system performance after the host migration. Note that the response time was still high, even if the processing and CPU times were already normal. The database time contributed more than 46 percent to this value. Furthermore, the roll wait time was high.

Dialog response time Figure 47.

1. Identify affected area by VMware vCops The SAP administrator logs into vCops to identify the affected area in the cloud environment. As shown in Figure 48, in the Org_B view, the SAPVSEDB virtual machine presents a low healthy index score (35) and the datastore (G86_RG_silver_DS_823) used by this virtual machine had a heavy workload of up to



100%. We performed deeper analysis on the above affected virtual machines in EMC SRM.

Org_B’s virtual machine health scoreboard and datastore workload Figure 48.

2. Identify the topology path from virtual machine to storage In the cloud environment, virtual machines float in the ESXi Server to balance the workload. To efficiently locate the ESXi host and the storage of the affected virtual machine, the SAP administrator quickly retrieves an end-to-end view of the relationship between the virtual machine (SAPVSEDB) and storage in EMC SRM, as shown in Figure 49.

End-to-end view from the virtual machine to storage Figure 49.

3. End-to-end detail performance analysis on affected virtual machine Next, the administrator drilled down to see more details on the virtual machine, ESxi host, fabric switches, and arrays. By simply clicking on each of these objects and selecting the Performance tab, several KPI charts displayed the trends that enabled the administrator to detect irregularities in performance.



80

Table 34. Virtual machine to array performance analysis in EMC Storage Resource Management Suite

Measure Value

Virtual machine(sapvsedb)

CPU utilization 40%

IOPS abruptly increased from 2,000 to 4,000

ESXi server CPU utilization 20%

SAN switch CPU utilization 12%

port utilization 8%

Array LUN IOPS Abruptly increased from 2,000 to 4,000

LUN response time Increased from 2ms to 12ms

As shown in Table 34, the SAP administrator analyzed virtual-machine-to-array end-to-end performance using EMC Storage Resource Management Suite and detected the following performance activities:

The virtual machine’s I/O rate abruptly increased from 2,000 to 4,000 IOPS. However, the virtual machine’s CPU utilization was low (only 40 percent). Therefore, the virtual machine’s CPU did not cause the bottleneck and the administrator moved forward to the host level.

The ESXi host and SAN switch showed similar behavior as that of the virtual machine chart. The CPU and switch port utilization values were both very low. Neither the ESXi host nor the switches caused the bottleneck and the administrator moved forward to the array level.

This virtual machine used LUN 0823. As shown in Figure 40, the LUN’s IOPS increased from 2,000 to 4,000, and the response time also increased from less than 2ms to 12ms. At this point, the SAP administrator isolated the problem to a storage issue. The SAP administrator escalated the issue to the cloud administrator for further investigation.

EMC storage array Performance tab Figure 50.



The cloud administrator launched EMC Unisphere from Prosphere. In Prosphere, the response time of the Basic_Silver_Pool where LUN0823 resided increased with the IOPS. This indicates that the pool was unable to provide enough IOPS capacity to the virtual machine, which caused degraded performance across the hosts and up through the SAP system.

Storage group response time Figure 51.

4. Recommendation to decrease IO contention The cloud administrator recommended that the organization upgrade its storage profile from Silver to Gold. In this case, the organization accepted the recommendation to decrease I/O contention. As shown in Figure 52, the cloud administrator upgraded the storage profile in vCloud Director Console.

Change storage profile in vCloud Director Figure 52.

After the upgrade, the datastore migrated from VMAX storage pool named Basic_Silver_Pool to Basic_Gold_Pool non-disruptively. As shown in Figure 53 and Figure 54, the storage response time decreased to 8.2ms, with an IOPS increase from 4,000 to 6,182. The database time improved by 28 percent, and roll wait time was reduced by 70 percent—a 45 percent improvement overall, as shown in Figure 55 and Figure 56.

Virtual machine’s IOPS increased after the upgrade Figure 53.



82

Virtual machine’s response time decreased after the upgrade Figure 54.

Storage response time after the upgrade Figure 55.

Improved dialog response time Figure 56.



Summary

The DRS cluster regulates CPU and memory automatically and without any downtime, reducing maintenance time, effort, and loss in productivity.

vCloud Director works with Fast VP to dynamically allocate storage resources based on performance requirements. This allocation mechanism is regulated using policies, and does not need to be constrained to one storage profile. This allows automation and flexibility to better adapt to the demands of any application.

VMware vCops identifies the affected area in the cloud environment. The SRM dashboards enable the administrator to isolate where the issue lies, and directly recommends the right tool for analyzing issues in more detail, reducing the time and effort needed to resolve incidents.

It is obvious that increasing the IOPS capacity was the solution to the use case presented. The value, however, is that the cloud-enabled infrastructure allows you not only to increase the IOPS capacity, but also to decrease it when the workload is generally not as high during the rest of the month or year.

The cloud-enabled infrastructure improves capital and operational efficiency of both the IT department and its tenants. If the need to increase the performance is temporary, the increased charge to the organization for the resources will also be temporary. The tenant receives the performance benefit without overinvesting, while the IT department profits and conserves resources.



84

Conclusion This solution demonstrates how EMC recommends that customers transform their IT operations to an on-premises private cloud service level.

This solution illustrates best practices in:

Building a multi-tenancy-oriented virtual data center for SAP

Setting up infrastructure chargeback

Working with self-service virtual machine templates to expedite provisioning

Enabling unified/integrated performance analyses of a virtual data center’s components

Elastically provisioning resources using FAST VP and DRS technologies

Protecting the SAP cloud infrastructure by cloud networking and security

This solution demonstrates the following findings and benefits:

EMC Storage Resource Management Suite provides visibility into the physical and virtual relationships to ensure consistent storage service level when optimizing IT resources, both key attributes of successful cloud deployment.

VMware vCloud Suite applies the principles of pooling, abstraction, and automation to all data center services like storage, networking, and security. By dramatically simplifying the provisioning of these services, VMware enables IT to completely provision an operationally ready infrastructure without worrying about the physical configuration of hardware.

After the foundation bundle is in place, read the available additional technical white papers that discuss and validate the following add-on bundles:

High availability and application mobility bundle using EMC VPLEX to enable application mobility in a multi data center environment and provide high availability across data centers with the stretched cluster configuration.

Data protection bundle using EMC Data Domain and Avamar to protect your SAP databases, applications, virtual machines, and vCloud environments.

SAP automation and operation bundle using SAP Landscape Virtualization Management to enhance the SAP operation efficiency.

Disaster recovery bundle using EMC SRM and VMware SRM to enable the failover of the private cloud to remote data center.

Enhanced security bundle using RSA products to achieve enhanced security and compliance of the cloud-enabled infrastructure.

Summary

Findings

Moving forward



References

For additional information, see the white papers listed below.

EMC Transforms IT On-Premise Private Cloud Infrastructure for SAP–Reference Architecture

EMC Storage Resource Management Suite—How Storage Resource Management Suite Meets Today’s Storage Management Challenge

For additional information, see the product documents listed.

Watch4net Web Portal Guide

EMC Watch4net ReportPack for ProSphere-SCA installation Guide

EMC Ionix Storage Configuration Advisor Administration Guide

EMC ProSphere 1.7 Administrator-Guide

VMware vCloud Director Adapter Installation and Configuration Guide - vCenter Operations Manager

Increase Efficiency for SAP Landscapes with EMC Symmetrix FAST VP

For additional information, see the documents listed below.

Best Practices and Troubleshooting Guide vCenter Chargeback Manager 2.5

vCenter Chargeback Manager Installation and Upgrade Guide

vCenter Chargeback Manager User’s Guide

Technical Note: Using VMware vCenter Chargeback Manager with VMware vCloud Director

VMware vCloud Architecture Toolkit 3.1

vCloud Director Administrator's Guide vCloud Director 5.1

VMware vCloud Design Best Practices vCloud Director 1.5

vSphere Networking Guide vSphere 5.1

vSphere Storage Guide vSphere 5.1

VMware vCenter Operations Manager Administration Guide - Custom User Interface vCenter Operations Manager 5.6

VMware vCenter Operations Manager Getting Started Guide

Installation and Configuration Guide for Windows and Linux -vCenter Operations Manager 5.6

VMware vCenter Operations Enterprise Automated Operations Management for Your Enterprise—VMware Technical Paper

SAP Note 1122388 - Linux: VMware vSphere configuration guidelines

White papers

Product documentation

Other documentation

Documents

EMC Cloud-enabled Infrastructure for SAP · VMware vCenter Chargeback Manager ... VMware Site Recovery Manager EMC RecoverPoint™ SAP automation any server and operations SAP system