LPAR Advanced Topics - IBM WWW Page · z990 with 11 CPs, 1 ICF, 2 IFLs, and 3 zAAPs – CP pool contains 11 CP engines – Specialty pool contains 6 engines – ICFs, IFLs, zAAPs

© 2009 IBM Corporation

SHARE 113 in Denver IBM Systems

Permission is granted to SHARE to publish this presentation in the SHARE Proceedings. IBM retains its right to distribute copies of this presentation to whomever it chooses.

IBM System z10 EC: LPAR Advanced Topics

Session 2247, August 24, 2009Harv Emery, IBM, Washington Systems Center Team Leader, Americas System z ATS

© 2009 IBM CorporationS2247HE 2

IBM System z

SHARE 113 | System z10 EC LPAR Advanced Topics

TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.


IBM System z


AgendaIBM System z10™ Update– System z10 EC New Function

• Announce review• Statements of Direction• z/VM-mode partition• CF-mode partition changes• LPAR IDs and “Partition Numbers”

Activation Profile Changes

Changing Running Partitions

Memory and Addressability– Fixed HSA


IBM System z


IBM System z10™ Update


IBM System z


Plan Ahead memory

Capacity Provisioning Enhancements – Capacity Tokens

Run production workload with CBU test. T & Cs apply

LPAR Dynamic PU reassignment

CPU Measurement Facility

Scheduled LPAR weight change

HMC Console Messenger

HMC and z/VM System Management

NTP Server on HMC

CFCC Level 16

STP Enhancements

InfiniBand 1x Coupling Links

High Performance FICON for System z

OSA-Express3 GbE and 1000BASE-T

Crypto TKE 5.3 Enhancements

z/VM LPAR Mode

QDIO Connection Isolation

System z10 Functions and Features (October 21, 2008)

System z10 Functions and Features (May and July, 2009)CBU Enhancements

STP Enhancements

World Wide port name (WWPN) Tool

Any z10 to any z9 – 12x InfiniBand SDR coupling

FICON Express8

Statements of Direction!


IBM System z


Statements of Direction* as of August, 2009 The System z10 will be the last server to support connections to the Sysplex Timer (9037). Servers that require time synchronization, such as to support a base or Parallel Sysplex, will require Server Time Protocol (STP). STP has been available since January 2007 and is offered on the System z10, System z9, and zSeries 990 and 890 servers The System z10 will be the last server to support Dynamic ICF expansion. This is consistent with the Statement of Direction in Hardware Announcement 107-190, (RFA44930) dated April 18, 2007: "IBM intends to remove the Dynamic ICF expansion function from future System z servers." ICB-4 links to be phased out. IBM intends to not offer Integrated Cluster Bus-4 (ICB-4) links on future servers. IBM intends for System z10 to be the last server to support ICB-4 links as originally stated in Hardware Announcement 108-154, dated February 26, 2008 ESCON channels to be phased out. It is IBM's intent for ESCON channels to be phased out. System z10 EC and System z10 BC will be the last server to support greater than 240 ESCON channels IBM intends to support optional water cooling on future high-end System z servers. This cooling technology will tap into building chilled water that typically exists within the datacenter for computer room air conditioning systems. External chillers or special water conditioning will typically not be required. Water cooling technology for high-end System z servers will be designed to deliver improved energy efficiencies IBM intends to support the ability to operate from High Voltage DC power on future System z servers. This will be in addition to the wide range of AC power already supported. A direct HV DC datacenter power design can improve data center energy efficiency by removing the need for an additional DC to AC inversion step

*All statements regarding IBM's plans, directions, and intent are subject to change or withdrawal without notice. Any reliance on these Statements of General Direction is at the relying party's sole risk and will not create liability or obligation for IBM.


IBM System z


Internal Coupling Facility (ICF)

1997

Integrated Facility for Linux® (IFL)

2000

System z Application Assist Processor (zAAP)

2004

Mainframe Innovation: Specialty Engines

Eligible for zIIP:DB2 remote access DB2 for BI/DWISVsIPSec encryptionz/OS XML System Servicesz/OS Global Mirror (XRC)HiperSockets for large messagesIBM GBS Scalable Architecture for Financial Reporting z/OS CIM ServerDB2 sort utilityzAAP on zIIP

Eligible for zAAP:Java execution environmentz/OS XML System Services

IBM System z Integrated Information Processor (zIIP)

2006

New! z/OS 1.11

http://www-03.ibm.com/systems/z/os/linux/ve/


IBM System z


PR/SM™ Hypervisor™ PU Dispatching “Pools”

PU Pool – Physical PUs to dispatch to online logical PUs z10 or z9 EC with 10 CPs, 1 ICF, 2 IFLs, 1 zIIP and 3 zAAPs– CP pool contains 10 CP engines – ICF pool contains 1 ICF– IFL pool contains 2 IFLs– zAAP pool contains 3 zAAPs– zIIP pool contains 1 zIIP – z/OS LPAR can have different CP, zAAP and zIIP weights– z/VM-mode LPAR (z10 only) can have different CP, zAAP, zIIP,

IFL and ICF weights z990 with 11 CPs, 1 ICF, 2 IFLs, and 3 zAAPs– CP pool contains 11 CP engines – Specialty pool contains 6 engines – ICFs, IFLs, zAAPs– z/OS LPAR zAAP weight is set equal to the initial CP weight


IBM System z


PR/SM Hipervisor PU Pool Rules

Logical PUs dispatched from supporting pool only– Logical CPs from CP pool only, for examplePool “width”– Width equals the number of physical PUs in the pool– Limits an LPAR’s maximum number of shared

logical PUs brought onlinePUs placed in pools by– Activate (POR)– Concurrent Upgrade – On/Off CoD, CBU, CIU, CUoD MES– Dedicated LPAR deactivation– Dedicated LPAR configure logical PU OFFPUs removed from pools by– Concurrent Downgrade - On/Off CoD, CBU, PU Conversion MES– Dedicated LPAR activation (“width” permitting)– Dedicated LPAR configure logical PU ON (“width” permitting)


IBM System z


New LPAR type for IBM System z10: z/VM-mode– Allows z/VM V5.4 users to configure all CPU types in a System z10 LPAR

Offers added flexibility for hosting mainframe workloads– Add IFLs to an existing standard-engine z/VM LPAR to host Linux workloads– Add CPs to an existing IFL z/VM LPAR to host z/OS, z/VSE, or traditional CMS

workloads– Add zAAPs and zIIPs to host eligible z/OS specialty-engine workloads– Test integrated Linux and z/OS solutions in the same LPAR

No change to software licensing– Software continues to be licensed according to CPU type

LPAR

z/VM

LPAR

z/OS

LPAR

z/VM

LPAR

z/OS

z/OS LinuxLinux

zAAP zAAP zAAP zIIP zIIP ICF ICFCP CP CP CP CP IFL IFL IFL IFL

LinuxLinuxLinuxLinux ProductionDev/Test and Optional Failover

z/OS Production

LPAR

z/OS

LPAR

CFCC

IFL

IBM System z10

z/VM-mode LPARCFCC CMSz/OS

System z10 z/VM-mode partitions – z/VM 5.4


IBM System z


System z10 Coupling Facility Processors

SoD: To be removed on System z “future”– Dynamic ICF Expansion

• Dedicated ICF and Shared CP in the same CF partition– Dynamic ICF Expansion Across ICFs

• Dedicated ICF and Shared ICF in the same CF partitionCF partition processor options– Recommended for production

• Dedicated ICFs (or dedicated CPs – Expensive!)– Not recommended (Exception: Backup or function test CF only)

• Shared ICFs or shared CPs• Dynamic ICF Expansion• Read and follow “PR/SM Planning”, SB10-7153, recommendations on

weights, Dynamic Dispatch, and capping VERY carefully to avoid performance problems, wasted resource, link checkstops, etc. “Big” ICFs on System z10 do NOT address these issues.


IBM System z


System z10 Activation Profile Changes


IBM System z


z990 HMC Reset Profile – General Page (OS2)

Logical Partition 'Suffix' Naming Convention

LPnameXX

where LPname is the first 6 characters of the customer required name

where xx = LPname suffix 1st character = LCSSid (0 = LCSS.0, 1 = LCSS.1)2nd character = same as MIFid of 1 to F

Logical partition is the only mode supported, basic mode is not available (HCD also provides only the LPAR mode option)


IBM System z


System z10 - Reset Profile - General (CEC TSYS z10 E64)

Logical Partition 'Suffix' Naming Convention

LPnameXX

where LPname is the first 6 characters of the customer required name

where xx = LPname suffix 1st character = CSSid (0 = CSS 0 to 3 = CSS 3)2nd character = same as MIFid of 1 to F

Logical partition is the only mode supported, basic mode is not available (HCD also provides only the LPAR mode option)


IBM System z


Shows customer available storage only on System z10. Fixed HSA is separate on System z10. HSA is included on System z9 and earlier.

System z10 - Reset Profile - Storage


IBM System z


System z10 Reset Profile - Dynamic

Global enable/disable of Dynamic I/O REMOVED on System z10. - Dynamic I/O always enabled at POR on System z10. - Dynamic I/O can be disabled/enabled later on System z10 globally or by partition on the HMC. S/390, z900 and z800 dynamic I/O expansion setting is removed. For z990 to z9, dynamic I/O expansion requirement is supported within HCD (IODF) by the MAXDEV option when defining a subchannel set in an LCSS. This impacts HSA size.On System z10, HSA is fixed with every supported LCSS defined with 15 partitions, both subchannel sets, and maximum devices in both subchannel sets. Dynamic LPAR add/delete is supported by renaming from “*” to add, to “*” to delete.Subchannel Set 0 – Up to 65,280 subchannels (63.75k)Subchannel Set 1 – Up to 65,535 subchannels (64k - 1)HSA Subchannels = MAXDEV times number of LPARs in LCSS


IBM System z


System z10 Reset Profile - Options

No functional change


IBM System z


System z10 Reset Profile - CP/SAP

New on System z10 EC: POR option to convert some purchased CPs to SAPs for the duration of the POR removed. Number of CPs, SAPs, zAAPs and zIIPs is now a comment on System z10.New on z9 EC: Option to view “Fenced Book” page for Enhanced Book Availability”


IBM System z


System z10 Reset Profile – Fenced Book Page

Shows purchased processors and available PUs if a 17 PU book is removed. Allows selection of processors to be available after POR with a fenced 17 PU book if all purchased processors do not fit.“LIC Processors” 75 at top: 64 + 11 SAPs + 2 spares = 77 total on E64 With 17 fenced, 59 useable plus 1 spare = 60 remaining on E64

This case: All processors remain available with a 17 PU book fenced.


IBM System z


System z10 Reset Profile – Fenced Book Page

Shows purchased processors and available PUs if a 20 PU book is removed. Allows selection of processors to be available after POR with a fenced 20 PU book if all purchased processors do not fit.“LIC Processors” 75 at top: 64 + 11 SAPs + 2 spares = 77 total on E64 With 20 fenced, 57 available plus 0 spares = 57 remaining on E64 Note default removal of zIIPs, which can be changed.


IBM System z


System z10 Reset Profile - Partitions

Remove partitions not to be activated automatically at POR. Change Order of activation as desired.Remember to activate Internal CFs before supported z/OS partitions.


IBM System z


New on System z10: z/VM-mode partition for z/VM 5.4 supports 5 different processor types. The Logical partition 'Partition Identifier' is a 1 or 2 digit unique hexadecimal value from 0 to 3F.Recommended convention: Assign first digit to partition’s LCSS ID – 0 to 3Assign second digit to “Partition Number” – 1 to F

System z10 Image Profile - General Page (Partition TOSP1)


IBM System z


System z10 Image Profile - General Page

Partition Time Offset from STP or ETR time

Time Offset Page


IBM System z


System z10 Image Profile – Time Offset

Typically used with STP or the ETR set to local time when a sysplex that is required to use LOCAL=GMT needs to be set to a different time zone than the CEC. Different sysplexes can also use different offsets to operate on local time in multiple different time zones. Note: This is somewhat unusual. Setting STP or the ETR to GMT, not local time, is recommended.


IBM System z


Select Capacity Group Profile Name

Example: System z10 EC ESA/390 mode partition: CPs, zAAPs, zIIPs Note: z10 EC allows Initial and Reserved to add up to 64 PUs. The operating system “sees” all those PUs (31 in this case). Don’t specify more engines than the operating system supports or engine types it doesn’t support.

System z10 Image Profile - Processor Page


IBM System z


System z10 Customize Group Capacity Profile

Use “Customize/Delete Activation Profiles” task to copy the Default Group Profile, customize it and save it with the new Group name. This creates a new Capacity Group.

Operating System Support – z/OS 1.8 and above

Group Capacity in MSUs


IBM System z


System z10 Image Profile - Processor Page

Example: System z10 EC z/VM-mode partition (z/VM 5.4 and later) which supports all PU types.

New October, 2008: z10 allows creation and save of an LPAR Image profile with an Initial processor definition that can’t be activated. Why? Allows creation of profiles to be used when CBU is active.


IBM System z


System z10 Image Profile - Security

Global performance data = “Partition Data Report” information on other partitions I/O configuration control = Write IOCDS and HCD dynamic hardware change. Counter Facility Security Options – NEW System z10 October, 2008 Sampling Facility Security Options – NEW System z10 October, 2008

z10 CBU Measurement Facility, Session 2136, John Berg, Thursday 3:00 PM


IBM System z


System z10 Image Profile - Storage

Central storage : z10 EC supports up to 1 TB Central Storage ( Initial + Reserved) maximum in an LPARCheck OS level for supported amounts.

Storage origin (Central and Expanded storage)It is recommended that you use the 'Determined by the system' option

Expanded Storage: Some OSs do not support. One example is z/OS (64-bit) running on a System z10.

Initial Storage: Brought ON at LPAR activation. Reserved Storage: Specify like reserved processors to add storage to a running partition. z/OS supports configuring storage ON and, if RSU specified, OFF. z/VM 5.4 supports configuring reserved central ON.

Recommended!

Recommended!


IBM System z


System z10 Image Profile - Options

Note: Sysplex cluster name is not required for a z/OS member of a parallel sysplex it is required for a non-member partition to be managed as a workload.

Cluster N

ame

I/O Priority Range

MSUs - WLC


IBM System z


System z10 Image Profile – Load “Classic”

“Load during activation” = IPL when activated Check box and provide IPL parameters (Recommended) Allow dynamic change to IPL address/parameter if wanted. Supports classic IPL or SCSI IPL (e.g. for Linux)


IBM System z


System z10 Image Profile - Crypto for Crypto Express2

Note candidate list for concurrent addition of Crypto Coprocessors.


IBM System z


Changing Running Partitions


IBM System z


System z10 EC Change LPAR Controls – CP Tab

Enter changes, “Change Running System” or “Save to Profiles” or do both.


IBM System z


System z10 Change Logical Partition I/O Priority Queuing


IBM System z


System z10 Logical Processor Add

New on System z10! Change is concurrent to the running partition.Operating System Support: Concurrent change to the running OS z/OS 1.10z/VM 5.3


IBM System z


System z10 Change LPAR Cryptographic Controls

New on System z10! Change is concurrent to the running partition.Operating System Supprt:z/OS ICSF: Concurrent change to the running z/OS image.


IBM System z


System z10 Change I/O Priority Queuing


IBM System z


System z10 Change Logical Partition Security

Counter Facility Security Options – NEW System z10 October, 2008 Sampling Facility Security Options – NEW System z10 October, 2008


IBM System z


System z10 Change LPAR Group Controls

Select “Edit” to reassign LPAR Group Membership


IBM System z


System z10 Change LPAR Group Controls Edit LPAR Group Membership


IBM System z


System z10 Memory and Addressability


IBM System z


Large Element Size Granularity

64 MB to 32 GB 64 MB>32 GB to 64 GB 128 MB>64 GB to 128 GB 256 MB>128 GB to 256 GB 512 MB

Rare to exceed today

Memory Granularity = Increment Size– Storage assignments/reconfiguration and HSA must be an even multiple– Physical increment size fixed at 64 MB– Expanded memory granularity always 64 MB– Central memory granularity is virtualized for each LP

• LP central memory increment is determined according to the size of the larger of the two central memory elements defined in the activation profile: Initial central memory or Reserved central memory

Single Storage Pool - All central storage– ES configured as needed from CS - No POR needed

Review MVS™ RSU parameter. Large z990 increment size may result in too much memory being reserved for reconfiguration after migration unless the new RSU options introduced in OS/390 2.10 are used.

System z9, z990, and z890 Memory Granularity


IBM System z


Large Element Size Granularity

>256 GB to 512 GB (EC only) 1 GB>512 GB to 1 TB (EC only) 2 GB

256 MB EC or > 64 GB BC up to 128 GB 256 MB>128 GB up to 256 GB (240 GB BC) 512 MB

Memory Granularity = Increment Size– Storage assignments/reconfiguration must be an even multiple– System z10 physical increment size is fixed at

• 256 MB on z10 EC and 128 MB on z10 BC (Was 64 MB on all z9, z990, z890)

– Expanded memory granularity always 256 MB (64 MB on z9, z990, z890)– Central memory granularity is virtualized for each LP

• LP central memory increment is determined according to the size of the larger of the two central memory elements defined in the activation profile: Initial central memory or Reserved central memory

• Virtualization determined by the limit of 512 increments per CS elementSingle Storage Pool - All central storage

– ES configured as needed from CS - No POR neededReview MVS™ RSU parameter. Large System z10 increment size may result in too much memory being reserved for reconfiguration after migration unless the new RSU options introduced in OS/390 2.10 are used.

System z10 Memory Granularity

128 MB up to 64 GB (BC only) 128 MB


IBM System z


storage increment sizeRSU = ---------------------------------------

CS amount to be reconfigured

In IEASYSxx. Specifies the number of central storage increments to be made available for central storage reconfiguration

– MVS attempts to keep this area free of long term fixed pages – calculated as follows for RSU = number

Or: Storage to be kept free = RSU * increment– If element size is upgraded, check the RSU parameter!

z/OS - Recommended RSU coding instead of RSU = numberto allow z/OS to calculate the number of increments

– RSU = % of storage, MBs or GBs– RSU = OFFLINE – Amount is Reserved Storage amount.

No storage is kept free of long term fixed pages except Reserved Storage that has been configured ON.

MVS RSU Parameter for System z


IBM System z


System z10 SE – Base System Storage Allocation

Note: Memory only adds up to 98% because this machine is in TEST mode running an IBM internal tool that steals 32 GB of memory.


IBM System z


8,192 GB

0 GB

LPAR/HSAAddressability

PartitionStorage

Addressability

CS/ESStorage

Pool

UnusedAddressability

Note: Gaps may exist due to storage reconfiguration or LPAR deactivation

Addressability for HSA is allocated top down from the highest supported address

System z10 – Highest address = 8 TB System z9/990/890 – Highest address = 1 TB

Partition Addressability is assigned below the fixed 64 GB HSA addressability when the LPAR is activated

Initial and Reserved CS are assigned contiguous CSaddressability at activation.Initial and Reserved ES (e.g. for z/VM) are assigned contiguous CS addressability at activation. This does NOT have to be contiguous with the partition’s CS addressability for CS. In addition they have contiguous ES addressability. Origin addresses are assigned top/down by default but a specific origin can be requested. Allowing default allocation is recommended in almost all cases.An addressability gap created, for example, by LPAR deactivation, can be “filled in” by a subsequent activation

Physical MemoryHSA starts at book/memory physical address 0 (in book 0)Physical memory is assigned for Initial ES and CS at LPAR activation. It is assigned to Reserved ES and CS if and when that memory is Configured ON. PR/SM is allowed to assign an LPAR any physical memory anywhere in the machine. Limit is “purchased” memory size. There is no requirement for LPAR physical memory to be contiguousEnhanced Book Availability is designed to be able to move partition memory and HSA to different book to enable removing a book from a running machine without disruption.

System z10 Central Storage Addressability

8,128 GB

64 GB (z10 EC)


IBM System z


System z10 SE - Logical Partition Storage AllocationCS Addressability Math in MB:

8,388,608 Top End = 8TB- 65,536 HSA = 64 GB- 32,768 Gap = 32 GB- 16,384 TOSPF CS = 16 GB

= 8,273,920 TOSPF Origin- 4,096 TOSP1 CS = 4 GB

= 8,269,824 TOSP1 Origin

Origin = Start of LPAR’s addressability Initial = Initial CS Maximum – Initial = Reserved CS Current = Physical Memory Assigned Gap = Unused addressability above

Expanded Storage Elements (eg z/VM) Take BOTH CS Addressability below the LPAR’s CS origin address (not shown) AND separate ES Addressability (shown).

HSA is allocated 64 GB of addressability but only 16 GB (8 on z10 BC) of storage.


IBM System z


zEnd

Documents

LPAR Advanced Topics - IBM WWW Page · z990 with 11 CPs, 1 ICF, 2 IFLs, and 3 zAAPs – CP pool contains 11 CP engines – Specialty pool contains 6 engines – ICFs, IFLs, zAAPs