VIOS in action with IBM i

© 2012 IBM Corporation

VIOS in Action with IBM i

Janus HertzSenior IT SpecialistIBM EMEA ETS Center of Competency for Power Systems

Common Europe Conference – Vienna | June 2012

Rev. 10-06-2012



2

What is the VIOS?

� A special purpose appliance partition

– Provide I/O virtualization

– Advanced Partition Virtualization enabler

� First GAed 2004

� Built on top of AIX, but not an AIX partition

� IBM i first attached to VIOS in 2008 with the IBM i 6.1

� VIOS is licensed with PowerVM



3

Where Do I Start with Installing VIOS and IBM i on Power system?

• Latest version at:

http://www.ibm.com/systems/resources/systems_i_Virtualization_Open_Storage.pdf



4

I/O Virtualization on POWER

IO Bus Virtualization

with Dedicated Adapters

Hypervisor

Fabric

Func

PCI adapter

Port

Func

Port

LPAR A LPAR B

PhysicalAdapterDevDrv

PhysicalAdapterDevDrv

PCI adapter

IO Adapter Virtualization

with VIO Server

Hypervisor

VIOS LPAR

LPAR A

Physical

Adapter

DevDrv

Virtual Fabric

Virtual AdapterServer

Virtual AdapterDevDrv

Virtual AdapterServer

LPAR B

Virtual AdapterDevDrv

Fabric

Func

Port

PCI adapter

Increasing

Adapter BW

& LPAR

Density

per Slot



5

IBM i + VSCSI (Classic)

Source

VIOS IBM i Client

(System 1)

POWER6 with IBM i 6.1.1

System 1

System 2

System 3

FC HBA

IBM i Client

(System 2)

IBM i Client

(System 3)

Hypervisor

•Assign storage to the physical HBA

in the VIOS

•Hostconnect is created as an open

storage or AIX hosttype,

•Requires 512 byte per sector LUNs

to be assigned to the hostconnect

•Cannot Migrate existing direct

connect LUNs

•Many Storage options supported

6B22Device Type

6B22Device Type

6B22Device Type



6

IBM i + VSCSI (Classic) Storage Virtualizer

VIOS


FC HBA

IBM i Client

Hypervisor

6B22Device Type

VSCSI SERVER

VSCSI Client

vhostXXX hdisk1

hdisk2

•Storage is assigned to the VIOS partition

•Within the VIOS you map the hdisk (lun) to the vhost corresponding to the client partition

•Storage management allocation is done from both the external storage box and the VIOS

•Flexible disk sizes up to 2Tb -512

•16 disks per vscsi adapter



7

IBM i + VSCSI Tape and optical

VIOS


IBM i Client

Hypervisor

VSCSI SERVER

VSCSI Client

vhostXXX

•Storage is assigned to the VIOS partition

•Within the VIOS you map physical tape or optical or file backed virtual optical to the vhost corresponding to the client partition

•No tape library support with vscsi adpaters. Must use VFC adapers.

cd1

rmt1

CD1

RMT1TAP01

OPT01



8

IBM i + NPIV ( Virtual Fiber Chanel )

Source

VIOS IBM i Client

(System 1)


System 1

System 2

System 3

8Gbs HBA

IBM i Client

(System 1)

IBM i Client

(System 1)

Hypervisor

•Hypervisor assigns 2 unique

WWPNs to each Virtual fiber

•Hostconnect is created as an

iSeries hosttype,

•Requires 520 byte per sector LUNs

to be assigned to the iSeries

hostconnect on DS8K

•Can Migrate existing direct connect

LUNS

•DS8100, DS8300, DS8700,

DS8800, DS5100 and DS5300

supported

Virtual address example C001234567890001

Note: an NPIV ( N_port ) capable switch is required to connect the

VIOS to the DS8000 to use virtual fiber.



9

NPIV Configuration - Limitations

� Single client adapter per physical port per partition

– Intended to avoid single point of failure

– Documentation only – not enforced

� Maximum of 64 active client connections per physical port

– It is possible to map more than 64 clients to a single adapter port

– May be less due to other VIOS resource constraints

� 32K unique WWPN pairs per system platform

– Removing adapter does not reclaim WWPNs

Can be manually reclaimed through CLI (mksyscfg, chhwresJ)

“virtual_fc_adapters” attribute

– If exhausted, need to purchase activation code for more

� Device Limitations

– Maximum of 128 visible target ports

Not all visible target ports will necessarily be active

Redundant paths to a single DS8000 node

Device level port configuration

Inactive target ports still require client adapter resources

– Maximum of 64 target devices

Any combination of disk and tape

Tape libraries and tape drives are counted separately



10

Redundant VIOS with NPIV

VIOSVIOS

POWER6

IBM i

Physical FC

connections

SYSBASIASP

Server

VFC

adapters

Client

VFC

adapters

1

� Step 1: configure virtual and physical FC adapters

– Best Practice to make VIOS redundant or

separate individual VIOS partitions where a

single hardware failure would not take down

both VIOS partitions.

� Step 2: configure SAN fabric and storage

– Zone Luns to the virtual WWPNs.

– Each DASD sees a path through 2 VIOS

partitions

2

•Notes: Support up to 8 paths per LUN

•Not all paths have to go through separate VIOS partitions.

•New multi-path algorithm in 7.1 TR2



11

6B25 Adapter Look & Feel

� Similar in look & feel to

other IOPless storage

adapters

� Attached device resources

have real hardware CCINs



12

Support for IBM Storage Systems with IBM i

Notes

- This table does not list more detailed considerations, for example required levels of firmware or PTFs required or configuration performance considerations

- POWER7 servers require IBM i 6.1 or later

- This table can change over time as addition hardware/software capabilities/options are added

# DS3200 only supports SAS connection, not supported on Rack/Tower servers which use only Fibre Channel connections, supported on Blades with SAS

## DS3500 has either SAS or Fibre Channel connection. Ractk/Tower only uses Fibre Channel. Blades support either SAS or Fibre Channel (either BCS or BCH)

### Not supported on IBM i 7.1. But see SCORE System RPQ 846-15284 for exception support

* Supported with Smart Fibre Channel adapters – NOT supported with IOP-based Fibre Channel adapters

** NPIV requires Machine Code Level of 6.1.1 or later and requires NPIV capable HBAs (FC adapters) and switches

@ BCH supports DS3400, DS3500, DS3950 & BCS supports DS3200, DS3500

@@ N Series can only be used as file server. No load source/boot support. Support only through IFS. No IBM i data base support

% NPIV support for DS5100/DS5300 requires IBM i 7.1 TR2 and must have POWER7 firmware: Ax730_xxx or the POWER6 firmware Service Pack that will be

released 2Q2011

Table as of

April 5, 2011

N Series@@

DS3200

DS3400

DS3500

DS3950

DS4700

DS4800

DS5020

Storwize

V7000

DS5100

DS5300DS6800 SVC XIV

DS8100

DS8300

DS8700

DS8800

Rack /

Tower

Systems

IBM i

Version

Hardware

5.4 / 6.1 / 7.1

POWER5/6/7

6.1 / 7.1

POWER6/7

Not DS3200#,

Yes DS3500##

6.1 / 7.1

POWER6/7

6.1 / 7.1

POWER6/7

6.1 / 7.1

POWER6/7

5.4 / 6.1

POWER5/6/7

Not 7.1 ###

POWER5/6/7

6.1 / 7.1

POWER6/7

6.1 / 7.1

POWER6/7

5.4 / 6.1 / 7.1

POWER5/6/7

5.4 / 6.1 / 7.1

POWER5/6/7

IBM i

Attach

IFS / NFS

(NAS)VIOS VIOS VIOS

Direct* or VIOS

–VSCSI and

NPIV%

Direct VIOS VIOS

Direct or VIOS –

VSCSI and

NPIV**

Direct or VIOS –

VSCSI and

NPIV**

Power

Blades

IBM i

Version

Hardware

6.1 / 7.1

POWER6/7

IFS / NFS

(NAS)

6.1 / 7.1

POWER6/7 @,

#, ##

6.1 / 7.1

POWER6/7

(BCH)

6.1 / 7.1

POWER6/7

(BCH)

6.1 / 7.1

POWER6/7

(BCH)

Not

supported

6.1 / 7.1

POWER6/7

(BCH)

6.1 / 7.1

POWER6/7

(BCH)

6.1 / 7.1

POWER6/7

(BCH)

6.1 / 7.1

POWER6/7

(BCH)

IBM i

Attach

IFS

(NAS)VIOS VIOS VIOS VIOS n/a VIOS VIOS

VIOS

NPIV**

VIOS

NPIV**

For more details, use the System Storage Interoperability Center: www.ibm.com/systems/support/storage/config/ssic/Note there are currently some differences between the above table and the SSIC. The SSIC should be updated to reflect the above information



13

IBM PowerVM Virtual Ethernet

� PowerVM Ethernet switch

– Part of PowerVM Hypervisor

– Moves data between LPARs

� Shared Ethernet Adapter

– Part of the VIO server

– Logical device

– Bridges traffic to and from

external networks

� Additional capabilities

– VLAN aware

– Link aggregation for external networks

– SEA Failover for redundancy

PowerVM Hypervisor

Virtual I/O Server

ent(Phy)

SharedEthernetAdapter

ent(Vir)

VLAN-Aware Ethernet Switch

Client 2

CMN(Vir)

Client 1

CMN(Vir)

EthernetSwitch



14

SEA Failover Configuration

VIOS

VE

TH

Partition

VE

TH

SE

A

ET

H

Hypervisor

VE

TH

VIOS

SE

A

ET

H

VE

TH

VE

TH

ClientnetworkNetwork

Partition

VE

TH

VE

TH



15

Virtual Ethernet Limits

Description Limit

Maximum virtual Ethernet adapters per LPAR 256

Maximum number of VLANs per virtual adapter 21 VLAN (20 VID, 1 PVID)

Number of virtual adapter per single SEA sharing a single physical network adapter 16

Maximum number of VLAN IDs 4094

Maximum number of physical adapters in a link aggregation 8 primary, 1 backup



16

PowerVM Active Memory Sharing

� Supports over-commitment of logical memory with overflow going to a paging device

� Intelligently flow memory from one partition to another for increased utilization and flexibility

� Memory from a shared physical memory pool is dynamically allocated among logical partitions as needed to optimize overall memory usage

� Designed for partitions with variable memory requirements

� PowerVM Enterprise Edition on POWER6 and Power7 processor-based systems

– Partitions must use VIOS for I/O virtualization

� Make sure it’s a good fit for you!

* All statements regarding IBM's future direction and intent are subject to change or

withdrawal without notice, and represent goals and objectives only.

POWER Server

Virtual

I/O

Server

Paging

PowerVM Hypervisor AMS

Dedicated

Memory

CPU

Shared Memory

Shared CPU

�Reduce memory costs by improving memory utilization on Power Servers



17

LPAR Suspend/Resume – Customer Value

� Resource balancing for long-running batch jobs

– e.g. suspend lower priority and/or long running workloads to free resources.

� Planned CEC outages for maintenance/upgrades

– Suspend/resume may be used in place of or in conjunction with partition mobility.

– Suspend/resume may require less time and effort than manual database shutdown and restart,

for example.

Requirements:• All I/O is virtualized

• HMC version 7 releases 7.3

• FW: Ax730_xxx

• IBM i 7.1 TR2

• VIOS 2.2.1.0 FP24 SP2



18

StorageSubsystem

ReservedStorage Pool

LUN

Validate environment for appropriate resources

Power7 System #1

A

Hypervisor

VIOS

A

vscsi0

vtscsi0

vhost0

fcs0

en2(if)

VLAN

ent2SEA

ent0

ent1

en0(if)

ent1

MoverService

VASI

Suspended PartitionIBM i Client 1

Partition Suspended

Suspend Partition CPU and I/O

Ask partiton if it’s ready for suspend

M M M M M M

Partition Suspend/Resume supported on POWER7

IBM i 7.1 TR2

C

Move Memory and CPU to Storage Pool

C C

Partition Suspend/Resume



19

PowerVM Live Partition Mobility

•Move running partition from one system to another with almost no impact to end users

•Requires POWER7 systems or later, PowerVMEnterprise, and all I/O must be through the Virtual I/O Server

•Requires IBM i 7.1 with TR4

Potential Benefits• Eliminate planned outages • Balance workloads across systems• Energy Savings

Movement of the OS and

applications to a different server with no loss of service

Virtualized storage and Network Infrastructure

Live Partition Mobility



20

Requirements & Planning

� Source and destination must be mobility capable and compatible.

– Enhanced hardware virtualization capabilities.

– Identical or compatible processors.

– Compatible firmware levels.

� Source and destination must be LAN connected – same subnet.

� All resources (CPU, Memory, IO adapters) must be virtualized prior to migration.

– Hypervisor will handle CPU and Memory automatically, as required. Virtual IO adapters are pre-configured, and SAN-attached disks accessed through Virtual IO Server (VIOS)

� Source and destination VIOS must have symmetrical access to the partition’s disks.

– e.g. no internal or VIOS LVM-based disks.

� OS is migration enabled/aware.

– Certain tools/middleware can benefit from being migration aware also.

SAN

LAN

Boot

Paging

Application Data

LPAR

HMC



21

MOnce enough memory pages have been moved, suspend the source system

Create shell partition on target system

Validate environment for appropriate resources

Live Partition Mobility

Power7 System #2Power7 System #1

StorageSubsystem

A

HMC

Hypervisor

VIOS

A

vscsi0

vtscsi0

vhost0

fcs0

en2(if)

VLAN

ent2SEA

ent0

ent1

en0(if)

ent1

Hypervisor

VIOSfcs0

en2(if)

VLAN

ent2SEA

ent0

ent1

A

vscsi0

en0(if)

ent1

vtscsi0

vhost0

MoverService

VASI

MoverService

VASI

Shell PartitionSuspended Partition IBM i Client 1IBM i Client 1Finish the migration and remove the original LPAR definitions

Start migrating memory pages

Create virtual SCSI devices

M M M M M M M M M M M M MMM

Partition Mobility supported on POWER7

IBM i 7.1 TR



22

Installing VIOS

� PowerVM Standard or Enterprise Edition must be ordered

– VIOS install media

migration utility

install image

– activation code

� Partition Creation

– physical adapters

– virtual adapters

� Partition Activation to SMS

� DVD Install

– Can also install over network – NIM Server

� Post-Install Tasks



23

Check VIOS enablement

� In HMC, display managed system’s properties



24

VIOS enablement

� Select managed system �� Capacity on demand �� Advanced POWER

virtualization (now PowerVM)

� Enter 32-digit enablement code



25

LPAR configuration – create a partition



26

VIOS partition configuration – physical IO



27

VIOS install: SMS setup

� When the partition is powered on, the PFW screen will appear

� Hit 1 to enter the SMS menu and set up the install source for

VIOS



28

VIOS install: Select Install/Boot Device

� Select option 5, then option 1



29

VIOS install: Select DVD as boot device



30

� Choose option 2 to verify VIOS is going to be installed on the correct disk unit

VIOS install: verify disk unit



31

� hdisk0 is the first physical disk or LUN detected

� VIOS can be installed on mirrored integrated disk, or installed on the SAN

VIOS install: verify disk unit



32

System & Metode, Denmark

www.system-method.com

• IBM Business Partner

• Software Solutions & Hosting company

� Focuses on very small / old existing installations

• 1 BladeCenter S chassis

• 1 JS12 POWER6 blade

• 2 HS21 x86 blades

• Provides hosting services to several clients/companies

• 1 IBM Virtual IO Server 2.2 (VIOS) host LPAR

• 9 IBM i 6.1 client LPARs – for different customers

Pros:

• Cheap hardware compared to traditional Power servers

• Possible to get customers that would potentially have switched to the “dark sideJ”

• Flexible

Cons:

• More complex, requires three different skills sets (Blade, VIOS, IBM i)

IBM i and BladeCenter SPower Virtualization



33

References:• PowerVM Quicksheet - http://tablespace.net/quicksheet/powervm-quicksheet.pdf

• PowerVM Quickstart - http://tablespace.net/quicksheet/powervm-quickstart.html

• VIOS Cheat Sheet - http://www.ibm.com/developerworks/aix/library/au-

viocheatsheet/index.html?ca=drs-

• Get to know the VIOS CLI - http://www.ibm.com/developerworks/aix/library/au-

vioscli/index.html?cmp=dw&cpb=dwaix&ct=dwnew&cr=dwnen&ccy=zz&csr=111011#VIO%20is%2

0critical

• IBM i on a POWER blade read-me-first:

http://www.ibm.com/systems/resources/systems_power_hardware_blades_i_on_blade_readme.p

df



34

TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes:

Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.

All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel

Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.

IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not

actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.

Technology

VIOS in action with IBM i