VMware Virtualized SAP HANA with Dell EMC Storage · 2018-12-04 · Executive summary 4 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up

VMware Virtualized SAP HANA with Dell EMC Storage

Storage configuration and connectivity

Scale-up and scale-out deployments November 2018

H14721.4

Deployment Guide

Abstract

This deployment guide describes best practices for deploying an SAP HANA system on Dell EMC storage arrays in VMware vSphere virtualized environments.

Dell EMC Solutions

Copyright

2 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA 11/18 Deployment Guide H14721.4.

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Contents

3 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments

Deployment Guide

Contents

Executive summary ....................................................................................................................... 4

Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage ................................ 6

SAP HANA storage devices in virtualized environments ......................................................... 11

Connecting an ESXi host to Dell EMC storage arrays .............................................................. 17

Creating data and log devices on vSphere datastores ............................................................. 19

Setting up an SAP HANA virtual machine in vSphere .............................................................. 21

References ................................................................................................................................... 24

Executive summary


Executive summary

Since the initial release of the SAP HANA in-memory, real-time platform as a physical

appliance, updates to the platform have provided more flexible deployment and scalability

for customers while reducing the total cost of ownership (TCO). The enhancements

include:

SAP HANA scale-outBreaks the RAM barrier of a single server by functioning as

a multinode cluster.

SAP HANA tailored datacenter integration (TDI)―Broadens the appliance model

toward shared enterprise storage arrays.

Virtualized SAP HANAEnables the installation of SAP HANA single-node (scale-

up) and multinode (scale-out) clusters in VMware virtualized environments. For

more information, see SAP Note 1788665: SAP HANA Support for

virtualized/partitioned (multitenant) environments (access requires an SAP

registration ID).

While these updates make the SAP HANA deployment easier and help customers with

the integration into their existing data center, organizations must still meet strict SAP

HANA requirements relating to their compute, network, and storage environments.

This VMware Virtualized SAP HANA with Dell EMC Storage solution offers SAP

customers a data center platform that provides agility, high availability (HA), cost savings,

and easy provisioning. The solution enables SAP customers to provision SAP HANA

instances more quickly in virtual machines.

Using the SAP HANA platform with the VMware vSphere virtualization infrastructure

provides the best environment for achieving a unique, cost-effective solution. This solution

is enabled by VMware vSphere vMotion, VMware vSphere Distributed Resource

Scheduler (DRS), and VMware vSphere High Availability, ensuring the best operational

performance and availability.

VMware provides best practices for deploying SAP HANA scale-up and scale-out systems

under vSphere. This deployment guide can be used in vSphere virtualized environments

as a supplement to the VMware documents and other Dell EMC documents that describe

storage configuration best practices for SAP HANA in physical TDI environments. See

References on page 24 for the titles of these documents.

VMware virtualized SAP HANA scale-up and scale-out systems can be deployed in TDI

environments on certified Dell EMC storage arrays. When running an SAP HANA

production system, the environment must meet SAP hardware and performance

requirements. You can use the SAP HANA hardware configuration check tool (hwcct) to

validate that the environment is correctly configured and that the storage performance

meets the SAP key performance indicators (KPIs). For more information, see SAP Note

1943937: Hardware Configuration Check Tool - Central Note (access requires an SAP

registration ID).

Note: The SAP performance requirements apply to production systems only.

Business

challenge

Solution

overview

http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enEN/enterprise-storage.html


https://launchpad.support.sap.com/#/notes/1943937

Executive summary


Deployment Guide

This guide describes best practices for implementing SAP HANA in vSphere virtualized

environments when using the following Dell EMC storage systems in TDI deployments:

VNX series unified and block storage arrays

VMAX, VMAX3, and VMAX All Flash

PowerMax

XtremIO

VPLEX

SC Series

Unity

This guide describes vSphere-specific storage connectivity and configuration procedures

to follow when connecting vSphere virtualized SAP HANA systems on ESXi hosts with

certified Dell EMC enterprise storage arrays.

Use this guide with the following VMware guide for virtualized SAP HANA scale-up and

scale-out deployments: Architecture Guidelines and Best Practices for Deployments of

SAP HANA on VMware vSphere.

This deployment guide is intended for administrators who are responsible for

implementing, maintaining, and protecting robust and virtualized SAP HANA in-memory

databases and storage systems on vSphere. It assumes that readers have some

familiarity with SAP HANA in-memory databases, Dell EMC storage hardware and

software, and vSphere virtual infrastructures.

Table 1 provides explanations for some abbreviations used in this deployment guide.

Table 1. Terminology

Term Explanation

RDM Raw Device Mapping

RHEL Red Hat Enterprise Linux

SLES SUSE Linux Enterprise Service

VMDK Virtual Machine Disk

VMFS Virtual Machine File System

Dell EMC and the authors of this document welcome your feedback on the solution and the

solution documentation. Contact the Dell EMC Solutions Team with your comments.

Authors: Werner Katzenberger, Aighne Kearney

Document

purpose

Audience

Terminology

We value your

feedback

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/sap_hana_on_vmware_vsphere_best_practices_guide-white-paper.pdf


mailto:[email protected]?subject=Feedback:%20VMware%20Virtualized%20SAP%20HANA%20with%20Dell%20EMC%20Storage%20Solution%20Guide%20(H14721.4)

Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage



Following the enhancement of SAP HANA deployment options with the TDI model, Dell

EMC certified the VMAX, VMAX3, VMAX All Flash, PowerMax, VNX, XtremIO, Unity,

VPLEX, and SC Series block storage platforms for production use in TDI deployments.

Tests with SAP HANA-certified Dell EMC block storage systems confirmed that these

arrays can be used in VMware virtualized environments with only a minor impact on

performance when you follow the guidelines provided n this deployment guide.

As in physical environments, you can install an SAP HANA system in virtualized

environments as a single-node (scale-up) system or as a multinode (scale-out) system.

The number of SAP HANA virtual machines that you can install on a single physical

server depends on the number of CPU sockets, the server RAM, the SAP HANA

installation use case (production or non-production), and the maximum scalability

supported by SAP and VMware, among other things.

Consult the relevant VMware best practices and recommendations when you create an

SAP HANA virtual machine and need to size and configure memory, CPU,

hyperthreading, and non-uniform memory access (NUMA) nodes for the machine.

Scale-up systems

Scale-up systems are primarily used for transactional SAP applications such as the SAP

Business Suite. The SAP HANA database is stored in the RAM of a single virtual

machine, while the SAP HANA persistence is stored on a data and log file system

belonging to this virtual machine.

Figure 1 shows the virtual disk devices used in SAP HANA virtual machines. The virtual

disk that holds the operating system is created during the initial machine configuration and

can reside on any available vSphere data store. Installing the SAP HANA software

requires a /hana/shared mount point. In scale-up installations, this mount point is

private to the virtual machine and can be a directory under the operating system root

device (if the size meets the requirements), or a separate virtual disk mounted under

/hana/shared. This virtual disk does not have specific performance requirements and

can reside on any multipurpose data store.

Overview

SAP HANA

deployment

options



Deployment Guide

Figure 1. Virtual disks in an SAP HANA scale-up scenario

Scale-out systems

Scale-out systems are used primarily for analytic applications such as SAP Business

Warehouse. The SAP HANA database is distributed across multiple virtual machines,

either on a single physical server or across a vSphere cluster consisting of multiple

physical servers. The system uses a “shared-nothing cluster architecture,” in which every

node in a scale-out deployment stores its database persistence on its own data and log

file systems. In a scale-out deployment, the /hana/shared file system must be shared

across all SAP HANA nodes that are part of the scale-out cluster. For configuration

guidelines, see SAP HANA shared file system on page 11.

High availability in a vSphere cluster

In a vSphere cluster, you must enable the vSphere High Availability (HA) feature. HA handles the restart of virtual machines on another host that has enough free resources if one of the hosts in the cluster fails. To ensure that SAP HANA starts automatically with the virtual machine, enable the SAP HANA service autostart feature, either at the time of

the SAP HANA installation or by setting the autostart option to 1 in the following file:

/hana/shared/<SID>/profile/<SID>_HDB<InstNo>_<hostname>

Figure 2 shows the minimum required vSphere HA settings.

High availability

considerations



Figure 2. vSphere HA Settings

Using vSphere host groups, VM groups, and affinity rules, the administrator can control which hosts in the vSphere cluster are allowed to run SAP HANA virtual machines.

HA for SAP HANA virtual machines

The SAP HANA service auto-restart watchdog function monitors the SAP HANA

application and the associated services within a virtual machine. The watchdog function

automatically detects a failure and restarts the corresponding SAP HANA process

(nameserver, indexserver, and so on).

The VMware HA monitoring feature (Guest not heartbeating) handles operating system

(OS) failures. The monitoring feature restarts the guest OS of the virtual machine as well

as SAP HANA on the same host, provided the SAP HANA autostart option is enabled.

Enable the monitoring feature when you enable vSphere HA, as shown in Figure 2.

Dell EMC recommends setting Heartbeat monitoring sensitivity to High, as shown in

Figure 3.



Deployment Guide

Figure 3. Enabling VMware HA VM Monitoring

To enable heartbeat monitoring, install and run VMware Tools in the virtual machine.

VMware Tools are installed either as part of the OS (open-vm-tools) or by using the

vSphere Web client and the context menu. Select Guest OS > Install VMware Tools.

In addition to heartbeat polling, VM Monitoring monitors the virtual machine’s I/O activity.

When heartbeats are not received and no disk or network activity has occurred over the

last 120 seconds, the virtual machine is reset by default. The administrator can change

the advanced setting das.iostatsInterval to modify this 120-second interval. Dell

EMC recommends aligning das.iostatsInterval with the failure interval you

selected in the vSphere Web client’s vSphere HA VM Monitoring section.

SAP HANA high availability in scale-out installations

In an SAP HANA scale-out installation on physical servers, you can deploy a standby

server to enable the SAP HANA host auto-failover functionality. If one of the active SAP

HANA servers (workers) fails, the SAP HANA nameserver triggers a failover to the

standby host.

In VMware virtualized environments, an SAP HANA standby host is not required and does

not work with the default SAP HANA storage connector. Instead, the VMware HA feature

restarts the SAP HANA virtual machines on another host in the cluster when a physical

host fails or on the same host if just the virtual machine operating system fails.

SAP HANA split-brain condition and host isolation

In physical SAP HANA scale-out installations with shared storage, the first installed node

becomes the master nameserver, Master 1, and the second and the third installed nodes

become master candidates Master 2 and Master 3. SAP HANA experiences a split-brain

situation if multiple hosts try to become the master nameserver or indexserver and to

access the same data (persistence) from disk. Such a situation might occur if the host of

the master nameserver becomes isolated from the network but can still access the shared

disk storage and a master candidate tries to take over the master role. Physical SAP

HANA installations use techniques such as SCSI-3 persistent reservation to prevent this

situation.



The standby master nameserver candidates are not required in VMware virtualized

environments. VMware HA restarts the existing master nameserver quickly enough that

promoting a new node to the master role is not necessary. To prevent SAP HANA from

using Master 2 and Master 3 candidates in scale-out installations, set the following entries

in the global.ini file:

[communication]

listeninterface = .global

[persistence]

basepath_shared = no

When installing SAP HANA using hdbinst, add these parameters to the

/usr/sap/<SID>/SYS/global/hdb/custom/config/global.ini file after the first

node is installed.

When using hdblcm for a SAP HANA scale-out installation, create a default

global.ini file with the above entries in a temporary directory. The --custom_cfg

installation parameter must point to this temporary directory.

In existing SAP HANA installations that already have multiple master candidates (Master

2 and Master 3), disable the failover mechanism by removing the nameserver roles

Master 2 and Master 3. Follow these steps:

1. Select SAP HANA Studio > Administration-Perspective > Landscape >

Hosts.

2. Select Configure Hosts for Failover Situation and change the master role to

Slave.

If a network failure occurs whereby a physical ESXi host becomes disconnected from the

network but all virtual machines on the host are still running, the VMware HA Host

Isolation rule and the associated response are applied. The default response leaves all

virtual machines powered on, as shown in Figure 2. Because a virtualized SAP HANA

scale-out cluster does not have a standby node to take over the function (master

nameserver) of the unresponsive node on the isolated host, a split-brain situation cannot

occur. The default response allows the administrator to investigate and fix the network

issue, or to power off the isolated host, which then causes VMware HA to restart the

virtual machines on another host.

You can also set the host isolation response to Power off and restart VMs. The host is

powered off and the virtual machines are restarted automatically on another host that still

has network connectivity.

With only one SAP HANA master nameserver candidate and the VMware HA Host

Isolation rules, a master nameserver can exist only once and therefore a split-brain

condition cannot occur.

SAP HANA storage devices in virtualized environments


Deployment Guide


SAP HANA installations on VMware vSphere require the following storage devices:

OS disk (root device) ‒ approximately 10 GB

The disk is created when the virtual machine is deployed. The virtual disk file

(VMDK) can reside on any data store that meets the installation requirements, that

is, has vMotion support.

SAP home directory: /usr/sap ‒ approximately 50 GB

The /usr/sap directory can be a mount point under the OS root directory. Adjust

the size of the root device correspondingly. Alternatively, you can use and mount a

separate virtual disk under /usr/sap/<SID>. Dell EMC recommends this method

for storage-based cloning and when using SAP instance cloning tools such as SAP

Landscape Management (LaMa).

SAP HANA shared file system: /hana/shared

The /hana/shared file system stores the SAP HANA installation binaries,

configuration files, and trace files. The size of the file system depends on the size of

the SAP HANA installation―that is, database size and number of nodes. For

guidelines about capacity sizing, see SAP HANA Storage Requirements. The SAP

HANA shared file system section on page 11 of this guide provides configuration

details.

SAP HANA database persistence (data and log)

Each active SAP HANA node, be it a scale-up node or worker node in scale-out

deployments, requires two devices to persist the in-memory database to disk and to

write the transaction log. Because the SAP performance requirements for the

storage devices apply only to these two file systems, it is important to consider

storage configuration best practices when you set up an SAP HANA production

environment. The size of the data and log devices depends on the actual database

size. For more information, see SAP HANA Storage Requirements.

SAP HANA scale-out scenarios require a shared file system that is mounted on every

SAP HANA node within the cluster. The file system stores the installation binaries,

configuration files, and trace files. In physical environments, this file system is generally

provided as a network file system (NFS) share by storage systems with network-attached

storage (NAS) capabilities, such as VNX series unified, Unity, SC Series storage, and

VMAX or PowerMax with eNAS.

If a NAS array is not available, vSphere with native Linux functionality offers an

alternative. A Linux virtual machine (non-SAP HANA node) running an NFS server

process provides the NFS share. This process exports a file system that is mounted on all

the SAP HANA cluster nodes. Reliability is achieved by using vSphere Fault Tolerance.

Overview

SAP HANA

shared file

system

http://go.sap.com/documents/2015/03/74cdb554-5a7c-0010-82c7-eda71af511fa.html




Figure 4. A /hana/shared file system using Linux NFS and vSphere fault tolerance

Setting up the NFS server

Follow these steps:

1. Create a Linux virtual machine (SLES or RHEL) and install the NFS server

package. See the operating system documentation for instructions. The Linux

virtual machine is not required to meet the high CPU and memory requirements of

an SAP HANA node.

2. Add a new VMDK to the virtual machine. The VMDK can reside on any vSphere

datastore. The size of the disk must be the required size of the shared file system

for the SAP HANA cluster.

3. Format the disk using the XFS file system.

4. Create a mount point in the format /hana_<sid>, replacing <sid> with the

system ID of the SAP HANA cluster that is using the export as the shared file

system. Then mount the disk under the new mount point.

5. Open the /etc/exports file. For each SAP HANA node (NFS client) in the

cluster, add the following line:

/hana_<sid> <fully qualified file server client host

name>(rw,sync,no_root_squash,no_subtree_check)

6. Restart the NFS server process to activate the changes

Note: The virtual machine running the NFS server can easily provide multiple SAP HANA

share exports for different SAP HANA installations.

Setting up the NFS client on the SAP HANA nodes

To set up the NFS client:

1. Ensure that the Linux NFS client is installed and running on each SAP HANA

node.

2. Open the /etc/fstab file in each SAP HANA node and add the following line:

<fully qualified file server host name>:/hana_<sid>

/hana/shared nfs hard,intr,nolock,proto=tcp 0 0



Deployment Guide

3. Create the local mount point on each SAP HANA node by using the following

command:

mkdir -p /hana/shared

4. Mount the shared file system on each node using the following command:

mount –a

Using fault tolerance for the NFS server virtual machine in vSphere

Because the SAP HANA nodes depend on the shared file system, Dell EMC recommends

that you turn on the fault tolerance function for the NFS server VM. To do this, go to the

vSphere Web client, highlight the NFS Server virtual machine, and select and enable

Fault Tolerance in the context menu.

Note: The fault-tolerance feature may require additional licenses in your vSphere environment.

Also, it presents some limitations on the number of virtual CPUs per protected virtual machine.

vSphere fault tolerance provides continuous availability for the SAP HANA share during

an NFS server failure. vSphere creates a live-shadow virtual machine instance that is

always up to date with the primary virtual machine. If a hardware outage occurs, vSphere

automatically triggers failover, ensuring zero downtime for the SAP HANA share and

preventing data loss. After failover, vSphere creates a secondary virtual machine to

deliver continuous protection for the SAP HANA share.

SAP HANA persists the in-memory database to the /hana/data file system at regular

intervals and writes the transaction log to the /hana/log file system. These two file

systems are the SAP HANA database persistence. The storage performance

requirements for SAP HANA TDI certification apply to the SAP HANA persistence for

production systems only. Dedicate the underlying storage device (RDM or a VMware

VMFS data store) to the persistence and then configure it using the best practices for SAP

HANA on that storage array. For more information, see the relevant document in

References on page 24 of this deployment guide.

SAP HANA production on vSphere

In production environments, a virtual SAP HANA node requires the same storage

performance and bandwidth as a physical node. The configuration of the LUNs and the

underlying pools or RAID groups must follow the same rules. Because the additional

virtualization layer affects the bandwidth and latency, the scalability of a particular Dell

EMC array for virtualized SAP HANA nodes might be lower than the number of SAP

HANA nodes supported in a physical environment.

For non-production virtualized SAP HANA environments, the storage performance does

not have to meet the SAP KPIs. Therefore, you have more flexibility when you configure

the virtual environment: RAID protection, disk types, pools, RAID groups, and so on. Dell

EMC recommends monitoring the storage performance and making adjustments if the

performance does not meet your requirements.

SAP HANA

database

persistence



Data stores and raw device mappings

When creating LUNs for virtualized SAP HANA deployments in production environments,

follow the storage configuration best practices for physical SAP HANA environments for

the corresponding SAP TDI-certified Dell EMC storage array. See Dell EMC

documentation on page 24 for links to these documents.

When Dell EMC block storage devices (LUNs) are connected to vSphere ESXi hosts, the

SAP HANA virtual machines can use them for the database persistence, either as data

stores formatted using a VMFS cluster or as RDMs. VMFS data stores are preferred

because they are easier to manage and provide more data growth flexibility.

This guide does not address the use of VMware virtual volumes (vvols).

Using VMFS datastores

Creating a VMFS data store on a block storage LUN is the preferred configuration option

for SAP HANA virtual machines. Creating dedicated and isolated datastores for SAP

HANA data and log file systems is a VMware best practice, which means the datastores

must be created on different LUNs in the storage array. Create all data devices (VMDKs)

of an SAP HANA system on one datastore, and all log devices on a second datastore.

When running multiple SAP HANA databases in a vSphere cluster, you can achieve

further workload isolation with dedicated data and log datastores for each SAP HANA

database system ID SID.

Each SAP HANA worker virtual machine has a data and a log file system residing on

VMDK files created on the corresponding VMFS datastore.

Figure 5 shows an example of an SAP HANA three-node scale-out deployment on a

vSphere cluster using VMFS datastores.



Deployment Guide

Figure 5. Deploying SAP HANA persistence on VMFS datastores

Using raw device mappings

RDMs are an alternative option for the SAP HANA database persistence. With RDMs,

every data and log file system resides on a dedicated LUN in the storage array.

RDMs enable you to configure the LUNs for virtualized SAP HANA nodes the same way

as for physical environments. RDMs also enable you to use the LUNs for physical SAP

HANA installations when the nodes are migrated to and from a virtualized environment.

Because the high number of LUNs might become difficult to manage for larger SAP HANA

deployments, consider VMFS datastores instead.

Figure 6 shows an example of an SAP HANA 3+1 scale-out deployment on RDMs.



Figure 6. SAP HANA persistence deployed on RDMs

Sizing storage LUNs for SAP HANA persistence

This section provides guidelines for correctly sizing the LUNs used as RDMs or the

storage LUNs used for the vSphere VMFS datastores. See SAP HANA Storage

Requirements for the capacity sizing for SAP HANA devices. Because the requirements

have been updated several times, use the most recent version of the guide when sizing a

new SAP HANA installation.

The examples in this guide use the following capacity sizing:

SAP HANA data file system: 1.2x net data of the SAP HANA node

SAP HANA log file system: 0.5x RAM of the SAP HANA node (maximum 512 GB)

Sizing storage LUNs for VMFS datastores

The LUNs for the VMFS datastores contain the SAP HANA persistence. When sizing the

LUNS, provide the following information:

Number of virtualized SAP HANA nodes, both scale-up and scale-out

RAM size of the SAP HANA nodes

Figure 5 shows a three-node SAP HANA scale-out cluster. Assuming each node has 128

GB RAM and holds approximately 50 GB of data, the capacity calculation is:

Data: 50 GB x 1.2 x 3 nodes = 180 GB

Log: 128 GB x 0.5 x 3 nodes = 194 GB



Connecting an ESXi host to Dell EMC storage arrays


Deployment Guide

All VMFS datastores must retain a level of free space to ensure that virtual machines

function correctly. Dell EMC recommends adding 25 percent free space to the previous

calculation. The size of the LUNs should be:

Data: 225 GB LUN size

Log: 242 GB LUN size

These are the minimum capacity requirements for the SAP HANA persistence. They do

not take account of future growth. With virtualized SAP HANA, you can increase the RAM

capacity of the SAP HANA nodes, requiring larger file systems for the database

persistence and therefore more capacity in the VMFS datastores. It is more efficient to

create larger LUNs at the beginning to prepare for future growth. It is also better to avoid

adding another LUN to a VMFS datastore because performance does not scale when

LUNs are concatenated.

Unlike physical environments, the underlying disk configuration of virtual environments,

that is, RAID groups, thin pools, and so on, does not change. You must still adjust the size

and configuration of the LUNs to meet the capacity requirements for datastores.

Sizing storage LUNs for RDMs

When using RDMs for the SAP HANA persistence, you can size the storage LUNs in

accordance with the SAP minimum requirements described in SAP HANA Storage

Requirements.

For the example shown in Figure 6, we created the following LUNs:

Three data LUNs: 50 GB x 1.2 = 60 GB for each LUN

Three log LUNs: 128 GB x 0.5 = 64 GB for each LUN


To connect Dell EMC storage to VMware ESXi servers, follow the best practices

described in the EMC Host Connectivity Guide for VMware ESX Server. When running

virtualized SAP HANA systems on vSphere, consider the additional connectivity options

that this section of the deployment guide describes.

Host bus adapters

For redundancy reasons, use at least two Fibre Channel (FC) host bus adapters (HBAs)

connected to the storage array over two switches. If you deploy multiple virtualized SAP

HANA production nodes on a single ESXi host, the storage bandwidth and latency must

fulfill the SAP TDI KPIs for each SAP HANA system1. In TDI deployments, a single SAP

HANA production node requires 400 MB/s maximum bandwidth. With four SAP HANA

production nodes on a single physical ESXi host, the combined bandwidth of all HBAs on

the host must be at least 4 x 400 MB/s (1600 MB/s)—for example, a single 8 Gb/s FC

HBA provides approximately 750 MB/s bandwidth.

1 See the relevant SAP and VMWare documentation for information about how many SAP HANA

production systems can be deployed per physical host.

Overview

ESXi

considerations



http://www.emc.com/collateral/TechnicalDocument/docu5265.pdf



ESXi HBA queue depth

Dell EMC recommends increasing the HBA queue depth on the ESXi host to 128:

1. Run the following command on the ESXi host:

esxcli system module parameters set -p

lpfc_lun_queue_depth=128 -m lpfc

2. To display the HBA queue depth setting, run the following command:

esxcli system module parameters list -m lpfc|grep

lpfc_lun_queue_depth

3. Restart the ESXi host to activate the changes.

vSphere Native Multipathing

If the ESXi host is using vSphere Native Multipathing (NMP), Dell EMC recommends the

round-robin policy for the multipath setting for VMAX, VMAX3, VMAX All Flash,

PowerMax, VNX, Unity, SC Series, and XtremIO storage arrays. For best performance,

set the multipath round-robin policy for the I/O operation limit to 1. For the SC Series, the

I/O operation limit must be set to 3. The current VMware default value for this setting is

1000.

To change the default policy setting from fixed to round-robin and the host I/O operation

limit to 1, follow these steps:

1. Open the vSphere CLI (recommended) or the service console.

2. Run the following command:

For VMAX, VMAX3, VMAX All Flash and PowerMax: esxcli storage nmp satp set --default-psp=VMW_PSP_RR --

satp=VMW_SATP_SYMM

For VNX2 and Unity using the ALUA protocol: esxcli storage nmp satp set --default-psp=VMW_PSP_RR --

satp=VMW_SATP_ALUA_CX

For XtremIO: esxcli storage nmp satp rule add -c tpgs_off -e "XtremIO

Active/Active" –M XtremApp -P VMW_PSP_RR -O iops=1 -s

VMW_SATP_DEFAULT_AA –t vendor -V XtremIO

3. To set the host I/O operation limit to 1, run the following commands:

List all devices connected to the ESXi host:

# esxcli storage nmp device list

Set the I/O operation limit to 1 for all Dell EMC storage devices used for the

SAP HANA persistence (either as datastores or as RDMs):

esxcli storage nmp psp roundrobin deviceconfig set -I 1 -

t iops -d <naa.deviceid>

Verify the device settings:

Creating data and log devices on vSphere datastores


Deployment Guide

esxcli storage nmp psp roundrobin deviceconfig get -d

<naa.deviceid>

Dell EMC PowerPath/VE

Dell EMC PowerPath/VE for VMware vSphere can be used instead of or with vSphere

NMP. PowerPath/VE is a multipathing extensions module for vSphere, providing software

that works with SAN storage to intelligently manage FC, iSCSI, and FC over Ethernet

(FCoE) I/O paths. PowerPath/VE is installed on the vSphere host and scales to the

maximum number of virtual machines on the host, thus improving I/O performance. The

virtual machines do not have PowerPath/VE installed and are not aware that

PowerPath/VE is managing I/O to storage. PowerPath/VE dynamically load-balances I/O

requests and automatically detects and recovers from path failures.

PowerPath/VE automatically assigns an optimized load-balancing policy for all devices

that it manages. Dell EMC recommends that you use the default optimized load-balancing

and failover policies that PowerPath/VE assigns to the devices based on the underlying

storage array type.

The number of front-end ports that are used for the ESXi hosts cluster depends on the

number of virtualized SAP HANA production nodes. Treat a virtualized SAP HANA node

like a physical SAP HANA node. For more information, see the storage configuration best

practices document for the corresponding Dell EMC storage array.

With ESXi hosts, use single-initiator zoning or single-initiator-single-target zoning. Single-

initiator-single-target zoning is preferred.

Note: Dell EMC does not support multi-initiator zones in a VMware ESX Server fabric

environment.

To perform zoning on the fabric, create zone sets that contain the initiator and the targets.


VMDKs on VMFS datastores are the preferred storage for the SAP HANA persistence on

SAP HANA virtual machines. After you have correctly sized and created the storage

LUNs, as described in Sizing storage LUNs for SAP HANA persistence on page 16, and

formatted VMFS datastores on these LUNs, you can create data and log devices for the

SAP HANA virtual machines on the datastores.

Each SAP HANA node uses its own data and log device and has access only to its pair of

devices, as Figure 1 shows. For performance reasons, VMware recommends provisioning

the SAP HANA data and log devices as “thick provision eager zeroed.” VMware also

recommends assigning dedicated SCSI controllers for each data and log device. See

vSphere SCSI controller on page 20 for more information.

Front-end ports

for storage

arrays

Fabric zoning

Introduction

Scale-up and

scale-out



For information about creating data and log VMDKs, see Setting up an SAP HANA virtual

machine in vSphere on page 21.

VMware recommends using multiple vSphere SCSI controllers for each SAP HANA virtual

machine. Assign SCSI controller 0 to the operating system disk, SCSI controller 1 to the

SAP HANA data devices, and SCSI controller 2 to the SAP HANA log devices.

vSphere offers several different types of SCSI controller to connect disks to a virtual

machine:

LSI Logic Parallel

LSI Logic SAS

VMware Paravirtual

BusLogic Parallel (this does not work with Dell EMC storage)

For best performance, VMware recommends using one VMware Paravirtual SCSI

controller for all data devices and one for all log devices.

With the VMWare Paravirtual SCSI controller used for the SAP HANA persistence, set the

following kernel parameters for the virtual machine running SAP HANA:

elevator=noop

vmw_pvscsi.cmd_per_lun=254

vmw_pvscsi.ring_pages=32

For information about how to set these kernel parameters, see:

Red Hat knowledge base article 5427: Suggested I/O scheduler to improve disk

performance when using Red Hat Enterprise Linux with virtualization (applies also

to SUSE).

Novell Hat knowledge base article 7009616: Virtual SLES11SP1 shows bad IO

performance when accessing raw disks (applies also to SLES versions other than

those mentioned).

vSphere SCSI

controller

Prepare an SAP

HANA virtual

machine in Linux

https://access.redhat.com/solutions/5427

https://access.redhat.com/solutions/5427

http://www.novell.com/support/kb/doc.php?id=7009616

http://www.novell.com/support/kb/doc.php?id=7009616

Setting up an SAP HANA virtual machine in vSphere


Deployment Guide


This section describes the configuration steps that you must complete when creating an

SAP HANA virtual machine with data and log persistence on Dell EMC storage as one of

the following:

Single-node scale-up system

Multinode scale-out system without a standby host

When deploying a new SAP HANA virtual machine, ensure that the operating system

version is supported by SAP and that you follow SAP and VMware best practices when

configuring the memory and CPU of the virtual machine.

Operating system virtual disk

The virtual disk on which the operating system is installed can reside on any datastore

that meets the functional requirements, namely, support for vSphere vMotion. This device

does not have to meet the SAP HANA performance requirements.

vSphere compatibility mode

It is important to use the correct vSphere compatibility mode (virtual-machine version). We

tested and validated compatibility mode ESXi 5.5 and later versions, virtual-machine

version 10.

To configure the disk:

1. In the vSphere Web client, edit the settings in the SAP HANA virtual machine.

2. Add two additional VMware Paravirtual SCSI controllers to the virtual machine.

Set SCSI Bus Sharing to None.

The virtual machine should now have three SCSI controllers assigned:

SCSI controller 0Used for the system device

SCSI controller 1To be used for the SAP HANA data devices

SCSI controller 2To be used for the SAP HANA log devices

3. Add a new SAP HANA data or log disk.

The SAP HANA persistence can be a vSphere VMDK residing on a VMFS

datastore, or an RDM pointing directly to the underlying storage LUN.

VMDK—Add a new hard disk to the virtual machine on the desired datastore

using the thick provision eager zeroed option. Assign an SAP HANA data

device to SCSI controller 1, as shown in Figure 7, and an SAP HANA log

device to SCSI controller 2, as shown in Figure 8.

Overview

Deploying an SAP

HANA virtual

machine

Assigning the

persistence to

the SAP HANA

virtual machine



Figure 7. Creating a new data hard disk

The virtual-machine data device is assigned to SCSI device 1:0

and the log device is assigned to SCSI device 2:0.

Figure 8. Creating a new hard disk log



Deployment Guide

RDM—Add the data and log RDM disks and assign them to SCSI controller 1

(data) and SCSI controller 2 (log), as shown in Figure 9.

Figure 9. Assigning an RDM to a virtual machine

Conclusion

Virtualizing SAP HANA scale-up and scale-out systems enables an organization to benefit

from all supported VMware virtualization solutions and options, such as HA and live

migration using vSphere vMotion, to increase service level agreements and lower the

TCO. We verified that virtualized SAP HANA installations on Dell EMC storage arrays

meet the SAP performance KPIs for SAP HANA production deployments when we

followed the storage configuration best practices and supplemental configuration

guidelines described in this document.

Summary

References


References

The following documentation on EMC.com or EMC Online Support provides additional

relevant information. Access to these documents depends on your login credentials. If you

do not have access to a document, contact your Dell EMC representative.

Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC VMAX and VMAX3 Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on Dell EMC PowerMax Arrays

Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC VNX Series Unified Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored Data CenterIntegration on EMC Unity Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC XtremIO Storage

EMC Host Connectivity Guide for VMware ESX Server

Dell EMC SC Series Best Practices with VMware vSphere 5.x-6.x

Storage Configuration Best Practices for SAP HANA TDI on Dell EMC VPLEXSystems

Dell EMC SC Series Storage Configuration Best Practices for SAP HANA TDIValidation Guide

For SAP Ready Solution documentation, see Solutions for SAP Server Info Hub:

Technical Documentation in the Everything SAP at Dell EMC Community forum.

The following documentation on the VMware website provides additional relevant

information:

Architecture Guidelines and Best Practices for Deployments of SAP HANA onVMware vSphere

The following documentation on the SAP website provides additional relevant information:

SAP HANA Installation Guide

SAP HANA Server Installation and Update Guide

SAP HANA Administration Guide

SAP HANA Storage Requirements

SAP HANA Certified Hardware Directory – Enterprise Storage

SAP Note 1788665: SAP HANA Support for virtualized/partitioned (multitenant)environments

SAP Note 800326: Certified EMC solutions for Linux environments

SAP Note 1943937: Hardware Configuration Check Tool - Central Note

SAP Community WIKI: SAP HANA on VMware vSphere

Dell EMC

documentation

VMware

documentation

SAP

documentation

http://www.emc.com/

https://support.emc.com/

http://www.emc.com/collateral/white-paper/h12342-storage-config-best-practices-sap-hana-tdi-vmax-vmax3-wp.pdf

http://www.emc.com/collateral/white-paper/h12342-storage-config-best-practices-sap-hana-tdi-vmax-vmax3-wp.pdf

http://www.emc.com/collateral/technical-documentation/h17290-sap-hana-tdi-on-powermax-vg.pdf

http://www.emc.com/collateral/technical-documentation/h17290-sap-hana-tdi-on-powermax-vg.pdf

http://www.emc.com/collateral/white-paper/h13047-storage-config-best-practices-sap-hana-tdi-vnx-wp.pdf

http://www.emc.com/collateral/white-paper/h13047-storage-config-best-practices-sap-hana-tdi-vnx-wp.pdf

http://www.emc.com/collateral/technical-documentation/h13984-storage-config-best-practices-sap-hana-tdi-xtremio-cg.pdf

http://www.emc.com/collateral/technical-documentation/h13984-storage-config-best-practices-sap-hana-tdi-xtremio-cg.pdf

http://www.emc.com/collateral/TechnicalDocument/docu5265.pdf

http://en.community.dell.com/techcenter/extras/m/white_papers/20441056/download

http://www.emc.com/collateral/technical-documentation/h17323-sap-hana-tdi-on-vplex-vg.pdf

http://www.emc.com/collateral/technical-documentation/h17323-sap-hana-tdi-on-vplex-vg.pdf

http://www.emc.com/collateral/technical-documentation/h17356-storage-config-sap-hana-tdi-sc-vg.pdf

http://www.emc.com/collateral/technical-documentation/h17356-storage-config-sap-hana-tdi-sc-vg.pdf

https://community.emc.com/docs/DOC-66864

https://community.emc.com/docs/DOC-66864

http://www.vmware.com/



http://help.sap.com/hana/

http://help.sap.com/hana/sap_hana_server_installation_guide_en.pdf

http://help.sap.com/saphelp_hanaplatform/helpdata/en/7e/b0167eb35e4e2885415205b8383584/frameset.htm

http://help.sap.com/saphelp_hanaplatform/helpdata/en/33/0e5550b09d4f0f8b6cceb14a64cd22/frameset.htm

http://scn.sap.com/docs/DOC-62595




https://launchpad.support.sap.com/#/notes/800326/E


http://scn.sap.com/docs/DOC-60470

Documents

VMware Virtualized SAP HANA with Dell EMC Storage · 2018-12-04 · Executive summary 4 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up