1

Installation runbook for Huawei Technologies Co., Ltd +

HuaweiDswareDriver

Partner Name: Huawei Technologies Co., Ltd

Product Name: FusionStorage

Product Version: V100R003C30

MOS Version: 6.1

OpenStack version: Juno on CentOS 6.5

Product Type: Storage

2

Contents Document History ................................................................................................................................................. 3 1. Introduction ................................................................................................................................................... 4

1.1 Objective ........................................................................................................................................... 4 1.2 Target Audience ................................................................................................................................ 4

2. Product Overview ......................................................................................................................................... 4 3. Joint reference architecture ........................................................................................................................... 4 4. Networking ................................................................................................................................................... 5

4.1 Network Topology .................................................................................................................................. 5 5. Installation and Configuration ...................................................................................................................... 6

5.1 Overview of MOS installation steps ....................................................................................................... 6 5.2 MOS Installation in details ..................................................................................................................... 6 5.3 Creation of OpenStack environment ....................................................................................................... 7 5.4 MOS Deployment ................................................................................................................................... 7

5.4.1 Nodes ........................................................................................................................................... 7 5.4.2 Network Setting ......................................................................................................................... 11 5.4.3 OpenStack Settings .................................................................................................................... 13 5.4.4 The followings are bridge names and their IP addresses after the deployment for the reference: ............................................................................................................................................................ 17

5.5 HuaweiDswareDriver Installation steps ................................................................................................ 17 5.5.1 Install FusionStorage ................................................................................................................. 17 5.5.2 Install Dsware Agent on all MOS nodes .................................................................................... 19 5.5.3 Get HuaweiDswareDriver .......................................................................................................... 22 5.5.4 Cinder installation and configuration on all cinder nodes .......................................................... 22 5.5.5 Nova installation and configuration on all MOS Compute nodes.............................................. 24

6. Testing ......................................................................................................................................................... 24 6.1 Test tools ............................................................................................................................................... 24 6.2 Test cases .............................................................................................................................................. 24

6.2.1 Target Use case(s) ...................................................................................................................... 25 6.2.2 Deployment modes and configuration options .......................................................................... 25

6.3 Test results (if FUEL HealthCheck is used) .......................................................................................... 26 6.3.1 Fuel HealthCheck Result ........................................................................................................... 26 6.3.2 Other testing result screenshots. ................................................................................................. 29

7. Appendix. .................................................................................................................................................... 32 7.1 How to download Fusion Storage software. ......................................................................................... 32 7.2 Configure FusionStorage HA mode and single plane network example............................................... 32 7.3 class LibvirtDswareVolumeDriver (please pay attention to python code format of the screenshots). .. 34

3

Document History

Version Revision Date Description

0.1 16-11-2015 Initial Version

1.0 20-11-2015 First Version

1.1 26-11-2015 Updated per Andrey Danin’s review

1.2 2-12-2015 Change FusionStorage Manager Portal network to Management plane network per MOS’s suggestion.

1.3 3-12-2015 Add missed integration steps.

1.4 11-12-2015 Update per demo review comments

1.5 18-12-2015 Add MOS nodes hardware information.

4

1. Introduction This document is to serve as a detailed Deployment Guide for HuaweiDswareDriver. Huawei Technologies Co., Ltd offers HuaweiDswareDriver (Storage solution). This document describes the reference architecture, installation steps for certified MOS+HuaweiDswareDriver, limitations and testing procedures.

1.1 Objective The objective of Mirantis OpenStack certification is to provide Mirantis program partners with an consistent and unified approach for acceptance of their solution into the Mirantis Technology Partner Program. Certification is designed within the context of Mirantis OpenStack infrastructure, including Mirantis Fuel deployment tool and supported cloud reference architectures.

1.2 Target Audience

This documentation is intended for OpenStack administrators and developers; it assumes that you have experience with network and cloud concepts.

2. Product Overview HuaweiDswareDriver include two part of FusionStorage driver and Nova-volume-driver. FusionStorage driver implemented OpenStack Cinder access FusionStorage storage service APIs, such as create volume, snapshot,etc. Nova-volume-driver implemented OpenStack Nova Compute access FusionStorage I/O APIs, such as conncet/disconnect volume, attach/detach volume,etc.

3. Joint reference architecture OpenStack Cinder & FusionStorage architecture. Print in yellow modules are implemented by Fusion Storage and HuaweiDswareDriver.

5

4. Networking 4.1 Network Topology

Each Server at least 2 NICs are required Eth0 - untagged port for PXE network (Fuel network) Eth1- port for networks: Public/Floating (untagged), Management (VLAN 101), Storage (VLAN 102), Private (VLAN 103)

6

5. Installation and Configuration 5.1 Overview of MOS installation steps ◇ Setup OpenStack Public network via a vSphere Virtual Machine as Gateway (172.16.0.2/24). ◇ Install Fuel Master Node on Physical Server. ◇ Deploy MOS Slave nodes on Fuel UI ◇ Run Fuel Health Check, make sure all cases pass. ◇ Manually integrated Fusion Storage cinder driver with MOS. ◇ Run Fuel Heath Check test on Fuel UI ◇ Verify HuaweiDswareDriver basic functions on MOS. Submit all cases test result.

5.2 MOS Installation in details ◇ Mount MirantisOpenStack-6.1.iso through IPMI on one Huawei E9000 blade server and power the

machine on. Set the boot order for the system with the Huawei DVD media. Using the Fuel User guide (https://docs.mirantis.com/openstack/fuel/fuel-6.1/user-guide.html#download-install-ug) go through the installation process until Fuel Setup will appear. Enter into it.

◇ Navigate to Network Setup menu, adjust configuration to one provided in the table below and then click ‘Check’ button to ensure the configuration is valid.

Interface IP address Netmask Default Gateway

eth0(PXE) 192.168.40.122 255.255.255.0 x

eth1 172.16.0.3 255.255.255.0 172.16.0.2

◇ Navigate to PXE Setup menu, adjust configuration to one provided in the table below and then click ‘Check’ button to ensure the configuration is valid.

7

Interface DHCP Pool Start DHCP Pool End DHCP Gateway

eth0(PXE) 192.168.40.2 192.168.40.254 192.168.40.122

◇ Navigate to DNS & Hostname menu, adjust configuration to one provided in the table below and then click ‘Check’ button to ensure the configuration is valid. It will also check that the Internet is reachable by the Fuel node because it will be required for the MOS installation later.

Hostname Domain Search Domain External DNS Hostname to test DNS

fuel Domain.tld Domain.tld 8.8.4.4, 223.5.5,5, 223.6.6.6

www.baidu.com

◇ Keep Time Sync menu untouched with default settings entered. Click to ‘Check’ button to ensure NTP

servers are reachable. ◇ Change Root password to make sure Fuel Health Check cases pass. ◇ Save and Quit Setup and boot the Fuel Master Node. ◇ When the installation is done Fuel UI become available at http://172.16.0.3:8000 as it’s shown at

screenshot 5.4.1 below.

5.3 Creation of OpenStack environment Name: mos_cert OS release: Juno on CentOS 6.5 Deployment mode: HA Cinder backend: Cinder LVM over iSCSI for volumes (Use default providers) Glance backend: Swift (Use default providers) Network setup: Neutron with GRE segmentation Murano and Sahara: Not Install, please install them if need.

5.4 MOS Deployment

5.4.1 Nodes All nodes: 3 Controller nodes combined with Storage-Cinder, Telemetry-MongoDB, and 1 Compute node. For this demo, hardware model information as follow: Controller Node 1 Controller Node 2 Controller Node 3 Compute Node E9000-CH121 E9000-CH121 E9000-CH121 RH2288 V2

8

Controller node (3a:ea) disks configuration:

9

Controller node (3a:ea) network Interfaces configuration:

10

Compute Node (10:ba) disks configuration:

Compute Node (10:ba) network interfaces configuration:

11

5.4.2 Network Setting

12

13

5.4.3 OpenStack Settings

14

15

16

17

5.4.4 The followings are bridge names and their IP addresses after the deployment for the reference:

Network Interface

Fuel Master Node

Controller (combine Cinder, Mongo)Node 1

Controller (combine Cinder, Mongo)Node 2

Controller (combine Cinder, Mongo)Node 3

Compute Node

eth0 (br-fw-admin)

192.168.40.122 192.168.40.5 192.168.40.6 192.168.40.4 192.168.40.2

eth1(br-ex/public)

172.16.0.3 IP:172.16.0.7 VIP:172.16.0.4

172.16.0.8 172.16.0.9 172.16.0.6

eth1 (br-mgnt)

x 192.168.0.5 192.168.0.6 192.168.0.7 192.168.0.4

eth1 (br-storage)

x 192.168.50.3 192.168.50.4 192.168.50.5 192.168.50.2

eth1 (br-mesh)

x 192.168.2.3 192.168.2.4 192.168.2.5 192.168.2.2

5.5 HuaweiDswareDriver Installation steps

5.5.1 Install FusionStorage 1) Hardware Preparation: At least 4 Physical Servers required. One Server for Fusion Storage Management Node (For this demo, we use vSphere VM to install Fusion Storage Management Node), the other three Physical Servers for Fusion Storage Node to make cluster. 2) Software Preparation: FusionStorage_V100R003C30SPC101_Manager.iso FusionStorage V100R003C30SPC100_Agent.zip dsware-kernel-centos-V100R003C30SPC200-1.x86_64.rpm xbb_net_config.sh FusionStorage_V100R003C30SPC100_Fsc_Cli_For_5.1.tar, For this demo purposes we use above packages and script to install and configure FusionStorage. Please, visit https://support.huawei.com to get the latest version of FusionStorage (the Huawei support account needed).How to download the software and product document, see detail in Appendix 7.1. 3) How to Install FusionStorage Manager VM on vSphere, see FusionStorage product document. 4) Once FusionStorage Manger VM install is done, ssh to FusionStorage with dsware user, su - to root user, configure FusionStorage Manager HA mode (For this demo, we use Fusion Storage single mode) and single plane network, see detail example in Appendix 7.2, and the followings are Fusion Storage Network setting for the references:

18

Network Interface

FusionStorage Manager VM

FusionStorage Node1

FusionStorage Node2

FusionStorage Node3

eth0 192.168.0.100 (Management VLAN 101)

192.168.0.101 (Management VLAN 101)

192.168.0.102 (Management VLAN 101)

192.168.0.103 (Management VLAN 101)

eth1 x 192.168.50.101 (Storage VLAN 102)

192.168.50.102 (Storage VLAN 102)

192.168.50.103 (Storage VLAN 102)

5) Copy xbb_net_config.sh script to each FusionStorage Node and run the command to setup Storage VLAN 102.

Ex: # sh xbb_net_config.sh add_vlan eth1 102 192.168.50.101 255.255.255.0 6) Configure Fusion Storage MDC Cluster and Storage Resource Pool in FusionStorage Portal: named mos_cluster.

19

5.5.2 Install Dsware Agent on all MOS nodes 1) Install Cent OS dependence package. # yum install sysstat Since no expect package in mos and mos-updates yum repository, create a local yum repository via mount CentOS 6.5 iso, install dependance package expect on all MOS nodes. (Please backup original yum repo files and rollback after expect installed) Check the package versions with ‘rpm -qa’ after have them installed. # rpm –qa |grep expect # rpm –qa |grep sysstat

2) Upload FusionStorage V100R003C30SPC100_Agent.zip and dsware-kernel-centos-V100R003C30SPC200-1.x86_64.rpm on Fusion Storage Portal (https://192.168.0.100:28443/fsportal/ ).

20

4) Add all MOS nodes via br-mgnt IP to FusionStorage on FusionStorage Portal

5) Install the following 3 packages on all MOS nodes from Fusion Storage Portal.

21

dsware-agent-V100R003C30SPC100-1.x86_64.rpm OMA-V100R001C00-1.x86_64.rpm dsware-kernel-centos-V100R003C30SPC200-1.x86_64.rpm

6) Add and save iptables rule on each MOS nodes:

#iptables -I INPUT 15 -p tcp -m multiport --ports 10000:30000 -j ACCEPT #/sbin/service iptables save

7) After the packages are installed check that the process dsware_agent is running on every OpenStack node. # ps –ef |grep dsware_agent

8) Select MOS Cinder and Compute nodes, create Block Storage Clients on Fusion Storage Portal. Once it’s done the nodes should look like as follow:

22

9) Ensure that the process dsware_vbs is running on every OpenStack node. # ps –ef |grep dsware_vbs

5.5.3 Get HuaweiDswareDriver Unzip FusionStorage_V100R003C30SPC100_Fsc_Cli_For_5.1.tar, it has the following 4 folders

5.5.4 Cinder installation and configuration on all cinder nodes 1) Copy from above folder.../driver/dsware.py, fspythonapi.py to /usr/lib/python2.6/site-packages/cinder/volume/drivers/. 2) Backup /usr/lib/python2.6/site-packages/cinder/volume/manage.py, add HuaweiDswareDriver mapping relationship as follow: 'cinder.volume.drivers.dsware.HuaweiDswareDriver': 'cinder.volume.drivers.dsware.DSWARELocalDriver',

23

3) Backup /etc/cinder/cinder.conf, modify cinder.conf, and add the following configuration: [Default] dsware_manager=192.168.0.100 fusionstorageagent=192.168.0.4 enabled_backends=dsware [dsware] dsware_manager=192.168.0.100 fusionstorageagent=192.168.0.4 volume_backend_name=dsware volume_driver=cinder.volume.drivers.dsware.HuaweiDswareDriver 4) Copy .../bin/fsc_cli to /usr/bin/, chmod 755 fsc_cli. 5) Copy .../ssl folder and all files to /usr/lib/python2.6/site-packages/cinder/volume/drivers/, chmod 755 *.* to all ssl folder files 6)Copy .../handle/fsc_handle.sh to /usr/lib/python2.6/site-packages/cinder/volume/drivers/, chmod 755 fsc_handle.sh 7) mkdir /etc/cinder/rootwrap.d, create file volume.filters, add configuration as follow: [Filters] fsc_cli:CommandFilter,/usr/bin/fsc_cli,root vbs_cli:CommandFilter,/usr/bin/vbs_cli,root 8) Restart cinder volume service #service openstack-cinder-volume restart

9) Check cinder dsware type on OpenStack Dashboard:

FusionStorage Manger IP

MOS Compute Node, also Fusion Storage Compute node. If has multiple fusionstorage agent, setting with IP1, IP2,…

24

5.5.5 Nova installation and configuration on all MOS Compute nodes 1) Backup /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py, volume.py. 2) Modify driver.py,add FusionStorage driver implemented Class Name as follow: 'dsware=' 'nova.virt.libvirt.volume.LibvirtDswareVolumeDriver',

3) Modify volume.py, add class LibvirtDswareVolumeDriver(LibvirtBaseVolumeDriver) and all functions of the class. Find the code in Appendix 7.3 4)mkdir /etc/nova/rootwrap.d, create file volume.filters as follow: [Filters] fsc_cli:CommandFilter,/usr/bin/fsc_cli,root vbs_cli:CommandFilter,/usr/bin/vbs_cli,root 5) Restart openstack-nova-compute service #service openstack-nova-compute restart

6. Testing 6.1 Test tools

Equipment Purpose

Fuel health check scripts Functional/acceptance testing

other Test HuaweiDswareDriver Function

6.2 Test cases Fuel Health Check test cases, see Fuel Health Check test result 6.3.1 Other test cases:

25

Function Description Remark

Create Volume Create dsware type empty volume on OpenStack Dashboard or CLI

Show Volume Query dsware type volume on OpenStack Dashboard or CLI

Delete Volume Delete dsware type volume on OpenStack Dashboard or CLI

Create Snapshot Create dsware type volume snapshot on OpenStack Dashboard or CLI

Show Snapshot Query dsware type volume snapshot on OpenStack Dashboard or CLI

Delete Snapshot Delete dsware type volume snapshot on OpenStack Dashboard or CLI

Create Volume From Image Create dsware type volume from image on OpenStack Dashboard or CLI

Create Volume From Volume Create dsware type volume from volume on OpenStack Dashboard or CLI

Create Volume From Snapshot Create dsware type volume from snapshot on OpenStack Dashboard or CLI

Launch Instance From Volume Launch Instance From Volume on OpenStack Dashboard or CLI

Attach Volume to Instance Attach Volume to Instance on OpenStack Dashboard or CLI

Detach Volume to Instance Detach Volume to Instance on OpenStack Dashboard or CLI

6.2.1 Target Use case(s) 1) Large-scale cloud computing data centers FusionStorage organizes disks of x86 servers into large-scale storage resource pools, provides standard block storage data access interfaces SCSI, integrate with MOS, enabling on-demand resource allocation. 2) Critical enterprise IT infrastructure MOS integrate with Fusion Storage HuaweiDswareDriver, providing self-service infrastructure to enterprise.

6.2.2 Deployment modes and configuration options

OS Mode HV Network Storage

Neutron with GRE FusionStorage

CentOS HA KVM SUSE 11.3

26

6.3 Test results (if FUEL HealthCheck is used)

6.3.1 Fuel HealthCheck Result

27

28

29

6.3.2 Other testing result screenshots. Created cirros image list on Openstack Dashboard.

Created dsware type volume list on Openstack Dashboard.(Include of create empty volume, create volume from image, create volume from snapshot, create volume from volume, etc, also checked delete, modify, etc)

30

Created Volume Snapshots list on Openstack Dashboard.

Created Instances with FusionStorage Volumes on Openstack dashboard.(Also checked delete, modify, etc)

31

Created Instance console screenshot on Openstack dashboard.

Attach/Detach FusionStorage volume to instance on Openstack Dashboard

32

7. Appendix. 7.1 How to download Fusion Storage software.

1) Visit http://support.huawei.com/carrier/ , create an account and upgrade your account for authority to download the software if your company has bought the relevant products. Ex: Fusion Storage DSware. 2) Type dsware in the Product Support field, find the FusionStorage DSware software, and select the latest software and products documents to download.

7.2 Configure FusionStorage HA mode and single plane network example.

33

FSM:~ # cd /opt/dsware/manager/setup/forCommonServer/ FSM:/opt/dsware/manager/setup/forCommonServer # sh ConfigHAForCommonServer.sh Start to stop ha stop HA successfully. Please enter manager HA mode (double/single): [double](?)s Please enter network plane mode (double/single): [double](?)s Please enter hostname: [FSM](?) Please enter manager gateway ip : [192.168.40.1](?)192.168.0.1 Please enter local manager ip : [192.168.40.15](?)192.168.0.100 Please enter local manager port : [eth0](?) Please enter manager mask : [255.255.254.0](?)255.255.255.0 ####################summary################### ### ha_mode | single ### net_mode | single ### local_host_name | FSM ### local_manager_ip | 192.168.0.100 ### manager_gateway | 192.168.0.1 ### manager_mask | 255.255.255.0 ### local_manager_port | eth0 ############################################## ############start check configure############# Check success! ############################################## Please commit all options ,press [y/n] : y kernel.hostname = FSM InputInfo: senario : ha_mode : single local_name : DSM01 local_manager_ip : 192.168.0.100 manager_float_ip : 192.168.0.100 manager_gateway : 192.168.0.1 manager_mask : 255.255.255.0 local_manager_port : eth0 init_role : primary Start to modify manager-ha.properties End to modify manager-ha.properties Start to modify tomcat_rm_conf.xml Start to modify gsdb_rm_conf.xml Start to modify readconfig.xml End to modify readconfig.xml Start to modify internal manager float ip ha config End to modify internal manager float ip ha config Start to modify local internal manager ip ha config End to modify local internal manager ip ha config Start to modify external manager float ip ha config End to modify external manager float ip ha config Start to modify local external manager ip ha config End to modify local external manager ip ha config Start to clean old network configure info End to clean old network configure info

34

Start to set files properties End to set files properties Start to config omm ************************************************************************ * * * HUAWEI OMM Platform * * * * The OMM server package config program. * * * * Copyright (C) 2012 Huawei Technologies Co., Ltd * ************************************************************************ checking user identity... [done] The current user:[root]. setting log file... [done] The log file:[/var/log/omm/oms/readconfig.log] start to check configs... The input IP paramters are below: Storage Pool ID : 0 GM IP : OM Float IP : 192.168.0.100 OM Float extern IP : 192.168.0.100 OM Active IP : 192.168.0.100 OM Active extern IP : 192.168.0.100 OM Standby IP : OM Standby extern IP: HA network out time : 270 [done] start to config Gauss... [done] start to config HA... [done] start to update IP paramters into database... [done] Congratulations, config the OMS successfully. start HA successfully. Warning: HA monitor has been running already. Start to set monitor task. inittab add id:7 Finish setting monitor task. End to config omm Config FSM successfully

7.3 class LibvirtDswareVolumeDriver (please pay attention to python code format of the screenshots). class LibvirtDswareVolumeDriver(LibvirtBaseVolumeDriver): """Libvirt Dsware Volume Driver, access huawei Dsware""" def __init__(self, connection): """Create back-end to dsware.""" super(LibvirtDswareVolumeDriver, self).__init__(connection, is_block_dev=True) def _attach_volume(self, volume_name, dsw_manager_ip): cmd = ['vbs_cli', '-c', 'attachwithip', '-v', volume_name, '-i', dsw_manager_ip.replace('\n','') , '-p', 0] out, err = utils.execute(*cmd, run_as_root=True) analyse_result = self._analyse_output(out)

35

LOG.debug(_("_attach_volume out is %s") % analyse_result) return analyse_result def _detach_volume(self, volume_name, dsw_manager_ip): cmd = ['vbs_cli', '-c', 'detachwithip', '-v', volume_name, '-i', dsw_manager_ip.replace('\n','') , '-p', 0] out, err = utils.execute(*cmd, run_as_root=True) analyse_result = self._analyse_output(out) if (analyse_result is not None and int(analyse_result['ret_code']) not in (0, 50151601)) or analyse_result is None: msg = "detach volume failed." raise exception.NovaException(msg=msg) LOG.debug(_("_detach_volume out is %s") % analyse_result) return analyse_result def connect_volume(self, connection_info, disk_info): """Connect the volume. Returns xml for libvirt.""" LOG.info('connect_volume %s' % connection_info) conf = super(LibvirtDswareVolumeDriver, self).connect_volume(connection_info, disk_info) # get volume name, 50151401 volume or snapshot has been attached out = self._attach_volume(connection_info['data']['volume_name'],connection_info['data']['dsw_manager_ip']) if (out is not None and int(out['ret_code']) not in (0, 50151401)) or out is None: msg = "initialize_connection failed." raise exception.NovaException(msg=msg) conf.source_type = 'block' conf.source_path = out['dev_addr'] return conf def _analyse_output(self, out): if out is not None: analyse_result = {} out_temp = out.split('\n') for line in out_temp: if re.search('^ret_code=', line): analyse_result['ret_code'] = line[9:] elif re.search('^ret_desc=', line): analyse_result['ret_desc'] = line[9:] elif re.search('^dev_addr=', line): analyse_result['dev_addr'] = line[9:] return analyse_result else: return None def disconnect_volume(self, connection_info, disk_dev): """Disconnect the volume.""" LOG.info('disconnect_volume %s' % connection_info) super(LibvirtDswareVolumeDriver, self).disconnect_volume(connection_info, disk_dev) self._detach_volume(connection_info['data']['volume_name'],connection_info['data']['dsw_manager_ip'])

36