Symantec StorageFoundation and HighAvailability Solutions 6.2Virtualization Guide - SolarisNovember 2014SymantecStorage Foundation and High AvailabilitySolutions Virtualization GuideThe software described in this book is furnished under a license agreement and may be usedonly in accordance with the terms of the agreement.Product version: 6.2Document version: 6.2 Rev 1Legal NoticeCopyright 2014 Symantec Corporation. All rights reserved.Symantec, the Symantec Logo, the Checkmark Logo, Veritas, Veritas Storage Foundation,CommandCentral, NetBackup, Enterprise Vault, and LiveUpdate are trademarks or registeredtrademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Othernames may be trademarks of their respective owners.The product described in this document is distributed under licenses restricting its use, copying,distribution, and decompilation/reverse engineering. No part of this document may bereproduced in any form by any means without prior written authorization of SymantecCorporation and its licensors, if any.THE DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIEDCONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIEDWARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE ORNON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCHDISCLAIMERSAREHELDTOBELEGALLYINVALID. SYMANTECCORPORATIONSHALLNOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTIONWITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THEINFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGEWITHOUT NOTICE.The Licensed Software and Documentation are deemed to be commercial computer softwareas defined in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227-19"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights inCommercial Computer Software or Commercial Computer Software Documentation", asapplicable, and any successor regulations, whether delivered by Symantec as on premisesor hosted services. Any use, modification, reproduction release, performance, display ordisclosure of the Licensed Software and Documentation by the U.S. Government shall besolely in accordance with the terms of this Agreement.Symantec Corporation350 Ellis StreetMountain View, CA 94043http://www.symantec.comTechnical SupportSymantec Technical Support maintains support centers globally. Technical Supportsprimary role is to respond to specific queries about product features and functionality.The Technical Support group also creates content for our online Knowledge Base.The Technical Support group works collaboratively with the other functional areaswithin Symantec to answer your questions in a timely fashion. For example, theTechnical Support group works with Product Engineering and Symantec SecurityResponse to provide alerting services and virus definition updates.Symantecs support offerings include the following: A range of support options that give you the flexibility to select the right amountof service for any size organization Telephone and/or Web-based support that provides rapid response andup-to-the-minute information Upgrade assurance that delivers software upgrades Global support purchased on a regional business hours or 24 hours a day, 7days a week basis Premium service offerings that include Account Management ServicesFor information about Symantecs support offerings, you can visit our website atthe following URL:www.symantec.com/business/support/index.jspAll support services will be delivered in accordance with your support agreementand the then-current enterprise technical support policy.Contacting Technical SupportCustomers with a current support agreement may access Technical Supportinformation at the following URL:www.symantec.com/business/support/contact_techsupp_static.jspBefore contacting Technical Support, make sure you have satisfied the systemrequirements that are listed in your product documentation. Also, you should be atthe computer on which the problem occurred, in case it is necessary to replicatethe problem.When you contact Technical Support, please have the following informationavailable: Product release level Hardware information Available memory, disk space, and NIC information Operating system Version and patch level Network topology Router, gateway, and IP address information Problem description: Error messages and log files Troubleshooting that was performed before contacting Symantec Recent software configuration changes and network changesLicensing and registrationIf your Symantec product requires registration or a license key, access our technicalsupport Web page at the following URL:www.symantec.com/business/support/Customer serviceCustomer service information is available at the following URL:www.symantec.com/business/support/Customer Service is available to assist with non-technical questions, such as thefollowing types of issues: Questions regarding product licensing or serialization Product registration updates, such as address or name changes General product information (features, language availability, local dealers) Latest information about product updates and upgrades Information about upgrade assurance and support contracts Information about the Symantec Buying Programs Advice about Symantec's technical support options Nontechnical presales questions Issues that are related to CD-ROMs or manualsSupport agreement resourcesIf you want to contact Symantec regarding an existing support agreement, pleasecontact the support agreement administration team for your region as follows:customercare_apj@symantec.com Asia-Pacific and Japansemea@symantec.com Europe, Middle-East, and Africasupportsolutions@symantec.com North America and Latin AmericaDocumentationYour feedback on product documentation is important to us. Send suggestions forimprovements and reports on errors or omissions. Include the title and documentversion (located on the second page), and chapter and section titles of the text onwhich you are reporting. Send feedback to:doc_feedback@symantec.comFor information regarding the latest HOWTO articles, documentation updates, orto ask a question regarding product documentation, visit the Storage and ClusteringDocumentation forum on Symantec Connect.https://www-secure.symantec.com/connect/storage-management/forums/storage-and-clustering-documentationAbout Symantec ConnectSymantec Connect is the peer-to-peer technical community site for Symantecsenterprise customers. Participants can connect and share information with otherproduct users, including creating forum posts, articles, videos, downloads, blogsand suggesting ideas, as well as interact with Symantec product teams andTechnical Support. Content is rated by the community, and members receive rewardpoints for their contributions.http://www.symantec.com/connect/storage-managementTechnical Support ............................................................................................... 4Section 1 Overview of SFHA Solutions used inSolaris virtualization ........................................... 15Chapter 1 Overview of Symantec Storage Foundation andHigh Availability Virtualization Solutions ................ 16Overview of the Symantec Storage Foundation and High AvailabilitySolutions Virtualization Guide ................................................... 16About Symantec Storage Foundation and High Availability support forSolaris virtualization technology ................................................ 17About SmartIO in the Solaris virtualized environments ................... 18Virtualization use cases addressed by Storage Foundation and HighAvailability Solutions ............................................................... 21Section 2 Zones and Projects ..................................................... 24Chapter 2 Storage Foundation and High Availability Solutionssupport for Solaris Zones ............................................ 25About Solaris Zones ...................................................................... 26About VCS support for zones .......................................................... 26Overview of how VCS works with zones ...................................... 26About the ContainerInfo service group attribute ............................ 27About the ContainerOpts resource type attribute ........................... 28About the ResContainerInfo resource type attribute ....................... 29Zone-aware resources ............................................................ 29About the Mount agent ............................................................ 30About networking agents .......................................................... 40About the Zone agent .............................................................. 41About configuring failovers among physical and virtualservers ........................................................................... 41Configuring VCS in zones .............................................................. 41Prerequisites for configuring VCS in zones .................................. 42ContentsDeciding on the zone root location ............................................. 43Performing the initial internal zone configuration ........................... 47About installing applications in a zone ......................................... 47Configuring the service group for the application ........................... 48Configuring a zone resource in a failover service group with thehazonesetup utility ............................................................ 51Configuring zone resource in a parallel service group with thehazonesetup utility ........................................................... 55Configuring multiple zone resources using same VCS user forpassword less communication ............................................. 59Modifying the service group configuration .................................... 60Verifying the zone configuration ................................................. 61Performing maintenance tasks .................................................. 61Troubleshooting zones ............................................................ 62Configuring for physical to virtual and virtual to physicalfailoversa typical setup ................................................... 62Adding VxFS file systems to a non-global zone .................................. 63Mounting VxFS as lofs into a non-global zone .................................... 64Mounting VxFS directly into a non-global zone from global zone ............ 64Mounting VxFS as VxFS inside a non-global zone ............................... 65Adding a direct mount to a zone's configuration .................................. 67Benefits of a VxFS mount in a non-global zone over VxFS mount fromglobal zone into the non-global zone .......................................... 69SFCFSHA mounts ........................................................................ 69Concurrent I/O access in non-global zones ........................................ 70Veritas Extension for Oracle Disk Manager ........................................ 71Exporting VxVM volumes to a non-global zone ................................... 78VxVM devices in Oracle Solaris global zones ............................... 79Removing a VxVM volume from a non-global zone ........................ 79About SF Oracle RAC support for Oracle RAC in a zoneenvironment .......................................................................... 80Supported configuration ........................................................... 81Known issues with supporting SF Oracle RAC in a zoneenvironment .................................................................... 82Setting up an SF Oracle RAC cluster with Oracle RAC on non-globalzones .................................................................................. 85Preparing to install non-global zones .......................................... 86Installing non-global zones ....................................................... 91Creating SF Oracle RAC configuration files inside non-globalzones ............................................................................ 92Enabling Oracle Disk Manager file access from non-global zoneswith Veritas File System ..................................................... 92Configuring high availability for non-global zones .......................... 938 ContentsConfiguring the cluster name for clustering non-global zones ........... 94Installing Oracle RAC inside the non-global zones ......................... 94Linking the ODM library ........................................................... 94Creating the Oracle database ................................................... 95Configuring non-global zones under VCS .................................... 95Sample VCS configuration with non-global zones .......................... 96Configuring Solaris non-global zones for disaster recovery .................. 116Software limitations of Storage Foundation support of non-globalzones ................................................................................. 118Administration commands are not supported in non-globalzone ............................................................................ 118VxFS file system is not supported as the root of a non-globalzone ............................................................................ 118QIO and CQIO are not supported ............................................. 118Package installation in non-global zones ................................... 119Package removal with non-global zone configurations ................. 119Root volume cannot be added to non-global zones ...................... 120Some Veritas Volume Manager operations can cause volumedevice names to go out of sync .......................................... 120Chapter 3 Storage Foundation and High Availability Solutionssupport for Solaris Projects ...................................... 122About Solaris Projects ................................................................. 122About VCS support for Solaris projects ............................................ 123Overview of how VCS works with Solaris projects ........................ 123About the ContainerInfo service group attribute ........................... 123About the ContainerOpts resource type attribute ......................... 124Project-aware resources ........................................................ 125About the Project agent .......................................................... 125Configuring VCS in Solaris projects ................................................ 125Prerequisites for configuring VCS in projects .............................. 125Chapter 4 Storage Foundation and High Availability Solutionssupport for Branded Zones ....................................... 127About branded zones .................................................................. 127System requirements ................................................................... 128Symantec Storage Foundation support for branded zones ................... 128About migrating VCS clusters on Solaris 10 systems .......................... 128Preparing to migrate a VCS cluster ................................................. 129Configuring VCS/SF in a branded zone environment .......................... 1309 ContentsSection 3 Oracle VM Server for SPARC ............................... 135Chapter 5 Storage Foundation and High Availability Solutionssupport for Oracle VM Server for SPARC ................ 136About Oracle VM Server for SPARC ............................................... 137Terminology for Oracle VM Server for SPARC .................................. 137Oracle VM Server for SPARC deployment models ............................. 139Split Storage Foundation stack ................................................ 139Guest-based Storage Foundation stack ..................................... 139Layered Storage Foundation stack ........................................... 140Benefits of deploying Storage Foundation High Availability solutionsin Oracle VM server for SPARC ............................................... 140Standardization of tools .......................................................... 140Array migration ..................................................................... 140Moving storage between physical and virtual environments ........... 140Boot Image Management ....................................................... 141Features ................................................................................... 141Storage Foundation features ................................................... 141Oracle VM Server for SPARC features ...................................... 144Split Storage Foundation stack model ............................................. 145How Storage Foundation and High Availability Solutions works inthe Oracle VM Server for SPARC ....................................... 145Storage Foundation features restrictions .................................... 146Guest-based Storage Foundation stack model .................................. 148HowSymantec Storage Foundation and High Availability Solutionsworks in the guest domains ............................................... 148About SFCFSHA in an Oracle VM Server for SPARCenvironment .................................................................. 150Symantec Storage Foundation features restrictions ..................... 154Layered Storage Foundation stack model ........................................ 155DMP as a multi-pathing solution in an Oracle VM Server forSPARC environment ....................................................... 156How Storage Foundation and High Availability Solutions works inthe Oracle VM Server for SPARC ....................................... 161Storage Foundation features restrictions .................................... 161System requirements ................................................................... 161Hardware requirements ......................................................... 161Symantec product release notes .................................................... 161Symantec Storage Foundation and High Availability ..................... 162Symantec Storage Foundation Cluster File System HighAvailability ..................................................................... 162Product licensing ........................................................................ 16210 ContentsInstalling Storage Foundation in a Oracle VM Server for SPARCenvironment ........................................................................ 162Installing and configuring Oracle VM Server for SPARC anddomains ....................................................................... 163Installing Storage Foundation in the control domain or guest .......... 163Installing Veritas File System in the guest domain ........................ 164Verifying the configuration ...................................................... 164Exporting a Veritas volume to a guest domain from the controldomain ............................................................................... 165Provisioning storage for a guest domain .......................................... 167Provisioning Veritas Volume Manager volumes as data disks forguest domains ............................................................... 167Provisioning Veritas Volume Manager volumes as boot disks forguest domains ............................................................... 169Using Veritas Volume Manager snapshots for cloning logical domainboot disks ........................................................................... 176Support of live migration for Solaris LDOMs with fencing configured inDMP mode .......................................................................... 181Configuring Oracle VM Server for SPARC guest domains for disasterrecovery ............................................................................. 183Software limitations ..................................................................... 186When an I/O domain fails, the vxdisk scandisks or vxdctl enablecommand take a long time to complete (2791127) ................. 186Resizing a Veritas Volume Manager volume (exported as a sliceor full disk) does not dynamically reflect the new size of thevolume in the guest ......................................................... 186Known issues ............................................................................ 187Guest-based known issues ..................................................... 187Split Storage Foundation stack known issues ............................. 188Chapter 6 Symantec Cluster Server support for using CVMwith multiple nodes in a Oracle VM Server forSPARC environment .................................................... 189Clustering using Cluster Volume Manager ........................................ 189Installing Storage Foundation on multiple nodes in a LogicalDomain .............................................................................. 190Reconfiguring the clustering agents for Cluster VolumeManager ....................................................................... 190Cluster Volume Manager in the control domain for providing highavailability ........................................................................... 192Provisioning storage to guests with Flexible Storage Sharingvolumes of control domain ................................................ 19411 ContentsChapter 7 VCS: Configuring Oracle VM Server for SPARC forhigh availability ........................................................... 196About VCS in a Oracle VM Server for SPARC environment ................. 196Benefits of using VCS in Oracle VM server for SPARCenvironments to manage logical domains ............................ 197Dynamic reconfiguration of memory and CPU of a guestdomain ......................................................................... 198AdaptiveHA for logical domains ............................................... 198Symantec Cluster Server requirements ..................................... 198Symantec Cluster Server limitations ......................................... 199Symantec Cluster Server known issues ..................................... 199About Symantec Cluster Server configuration models in an Oracle VMServer for SPARC environment ............................................... 200Symantec Cluster Server setup to fail over a logical domain on afailure of logical domain ................................................... 201Symantec Cluster Server setup to fail over an Application runninginside logical domain on a failure of Application ..................... 206Oracle VM Server for SPARC guest domain migration in VCSenvironment ........................................................................ 208Overview of a warm migration ................................................. 209Overview of a live migration .................................................... 209Prerequisites before you perform domain migration ..................... 213Supported deployment models for Oracle VM Server for SPARCdomain migration with VCS ............................................... 214Migrating Oracle VM guest when VCS is installed in the controldomain that manages the guest domain .............................. 214Migrating Oracle VM guest when VCS is installed in the controldomain and single-node VCS is installed inside the guestdomain to monitor applications inside the guest domain .......... 215Migrating Oracle VM guest when VCS cluster is installed in theguest domains to manage applications for Oracle VM Serverfor SPARC version 2.1 and above ...................................... 217Migrating Oracle VM guest when VCS cluster is installed in theguest domains to manage applications for Oracle VM Serverfor SPARC version 2.0 ..................................................... 218About configuring VCS for Oracle VM Server for SPARC with multipleI/O domains ......................................................................... 220About Alternate I/O domain ..................................................... 221Setting up the Alternate I/O domain .......................................... 222Configuring VCS to manage a Logical Domain with multiple I/Odomains ....................................................................... 22212 ContentsConfiguring VCS to manage a Logical Domain using services frommultiple I/O domains .............................................................. 222A typical setup for a Logical Domain with multiple I/Oservices ........................................................................ 223Identify supported storage and network services ......................... 224Determine the number of nodes to form VCS cluster .................... 225Install and configure VCS inside the control domain and alternateI/O domain .................................................................... 225Configuring storage services ................................................... 225Configure storage service groups ............................................. 230Configure network service groups ............................................ 233Configure a service group to monitor services from multiple I/Odomains ....................................................................... 236Configure the AlternateIO resource .......................................... 237Configure the service group for a Logical Domain ........................ 239Failover scenarios ................................................................. 241Recommendations while configuring VCS and Oracle VM Serverfor SPARC with multiple I/O domains .................................. 242Sample VCS configuration for AlternateIO resource configuredas a fail over type ........................................................... 244Configuring VCS on logical domains to manage applications usingservices from multiple I/O domains ........................................... 248Chapter 8 SF Oracle RAC support for Oracle VM Server forSPARC environments .................................................. 251About deploying SF Oracle RAC in Oracle VM Server for SPARCenvironments ....................................................................... 251Sample configuration scenarios ..................................................... 252Preparing to deploy SF Oracle RAC in logical domainenvironments ....................................................................... 253SF Oracle RAC with Oracle RAC database on I/O domains of twohosts .................................................................................. 254SF Oracle RAC with Oracle RAC database on guest domains of twohosts .................................................................................. 256SF Oracle RACwith Oracle RACdatabase on guest domains of singlehost ................................................................................... 258SF Oracle RAC with Oracle RAC database on I/O domain and guestdomain of single host ............................................................. 26113 ContentsChapter 9 Support for live migration in FSS environments ........ 265About live migration in Flexible Storage Sharing (FSS)environments ....................................................................... 265Performing live migration of Oracle VM Server for SPARC systems inFlexible Storage Sharing (FSS) environments ............................. 266Chapter 10 Symantec ApplicationHA: Configuring Oracle VMServer for SPARC for high availability ..................... 268About Symantec ApplicationHA ..................................................... 268Why Symantec ApplicationHA ...................................................... 269LDom configurations with Symantec ApplicationHA ........................... 270Symantec ApplicationHA in the guest domains (LDoms) ..................... 270VCS in the control domain and Symantec ApplicationHA in the guestdomain (LDOM) ................................................................... 271Installing and configuring ApplicationHA for applicationavailability ........................................................................... 273Additional documentation ............................................................. 274Oracle Solaris documentation .................................................. 274Symantec documentation ....................................................... 274Section 4 Reference ........................................................................ 275Appendix A Where to find more information .................................... 276Symantec Storage Foundation and High Availability Solutions productdocumentation ..................................................................... 276Solaris virtualization documentation ................................................ 277Service and support .................................................................... 277About Symantec Operations Readiness Tools ................................... 27714 ContentsOverview of SFHA Solutionsused in Solaris virtualization Chapter 1. Overview of Symantec Storage Foundation and High AvailabilityVirtualization Solutions1SectionOverview of SymantecStorage Foundation andHigh AvailabilityVirtualization SolutionsThis chapter includes the following topics: Overview of the Symantec Storage Foundation and High Availability SolutionsVirtualization Guide About Symantec Storage Foundation and High Availability support for Solarisvirtualization technology Virtualization use cases addressed by Storage Foundation and High AvailabilitySolutionsOverview of the Symantec Storage Foundation andHigh Availability Solutions Virtualization GuideVirtualization technologies use software partitioning to provide a means of virtualizingoperating system services. Partitioning enables the creation of isolated virtualmachine environments for running applications. This isolation prevents processesrunning in one virtual machine from affecting processes running in other virtualmachines. The virtualized computing environment is abstracted from all physicaldevices, enabling you to consolidate and centrally manage your workloads on asystem.1ChapterThis document provides information about Symantec Storage Foundation and HighAvailability (SFHA) Solutions support for Solaris virtualization technologies. Itcontains: High-level conceptual information for SFHA Solutions and how they function inSolaris virtual environments. High level implementation information for setting up SFHA products in Solarisvirtual environments. Use case chapters with examples of how SFHA Solutions can improveperformance outcomes for common Solaris virtualization use cases.The information in this guide supplements rather than replaces SFHA Solutionsproduct guides. It assumes you are a skilled user of Symantec products andknowledgeable concerning virtualization technologies.See Symantec Storage Foundation and High Availability Solutions productdocumentation on page 276.See Solaris virtualization documentation on page 277.About Symantec Storage Foundation and HighAvailability support for Solaris virtualizationtechnologyThis section describes the Storage Foundation and High Availability support forSolaris virtualization technology:Solaris Zones, also known as non-global zones is an operatingsystem-level virtualization technology, which lets you virtualizeoperating system services to create an isolated environment forrunning applications. Non-global zones function as completelyisolated virtual servers with a single operating system instance.The non-global zones also called as native zones on Solaris 10and "solaris" brand zones on Oracle Solaris 11.See About Solaris Zones on page 26.Solaris Zones17 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsAbout Symantec Storage Foundation and High Availability support for Solaris virtualization technologyThe Solaris operating system provides a facility called projectsto identify workloads. The project serves as an administrativetag, which you can use to group useful and related work. Forexample, you can create one project for a sales application andanother project for a marketing application. By placing allprocesses related to the sales application in the sales projectand the processes for the marketing application in the marketingproject, you can separate and control the workloads in a waythat makes sense to the business.See About Solaris Projects on page 122.Solaris ProjectsBranded zones are an extension of the Solaris Zoneinfrastructure. A branded zone is a non-native zone that letsindividual zones emulate an operating systemenvironment otherthan the native environment of the global operating system. Forexample, you can run a Solaris 10 operating environment on anOracle Solaris 11 operating system as a "solaris10" brandedzone.Note: Symantec Storage Foundation for Oracle RAC andSymantec Storage Foundation Cluster File System HighAvailability do not support Branded Zones.See About branded zones on page 127.Branded ZonesOracle VMServer for SPARCis technology that lets you allocatea system's various resources. The Oracle VMServer for SPARCwas formerly known as Solaris Logical Domains (LDOMs).See About Oracle VM Server for SPARC on page 137.See Clustering using Cluster Volume Manager on page 189.See About VCS in a Oracle VMServer for SPARCenvironmenton page 196.See About Symantec ApplicationHA on page 268.Oracle VM Server forSPARCAbout SmartIO in the Solaris virtualized environmentsStorage Foundation and High Availability Solutions are supported in the Solarisvirtualized environments. This section describes how you can use SmartIO in theSolaris virtualized environments.Table 1-1 shows how SmartIO can be used in the Oracle VM Server for SPARCenvironment.When you install Storage Foundation and High Availability Solutions in the guest,you can use SmartIOto cache data onto an SSDor any other supported fast device.18 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsAbout Symantec Storage Foundation and High Availability support for Solaris virtualization technologyThe SSDused for the cache can be either a PCIe or SAS device, or an array-basedSSD.When you install Storage Foundation and High Availability Solutions in the controldomain, you can use VxVM read caching at the control domain level.Storage Foundation for Oracle RAC is supported only on guest domains.If an array-based SSD is used, live migration is supported with SmartIO caching.With direct attached devices (PCIe), live migration is not supported if SmartIOcaching is enabled. If you need to performlive migration, you can use manual steps.See Performing live migration between LDOMs in the SmartIO environmenton page 20.Table 1-1 Solaris: Oracle VM Server for SPARCVxFS writebackcachingVxFSreadcaching VxVM readcachingCaching takesplace:Configurationin controldomainConfigurationin guestNo No Yes in the controldomainSF(VxVM/CVM/DMP)"Split stack"VxFSYes Yes Yes in the guest Any "Guest-basedstack"SFYes Yes Yes in the guest Any "Guest-basedstack"SFCFSNo Yes Yes in the guest No SF stack "Guest-basedstack"SFRACYes Yes Yes in the guest DMP "Layered stack"SFN/A N/A Yes in the controldomainSF, CVM,VxFS,or CFSZFSTable 1-2 shows how SmartIO can be used in the Solaris zones environment.19 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsAbout Symantec Storage Foundation and High Availability support for Solaris virtualization technologyTable 1-2 Solaris: zonesVxFS writebackcachingVxFSreadcaching VxVM readcachingCaching takesplace:Configurationin global zoneConfigurationin non-globalzonesYes Yes Yes in the globalzoneSF SFN/A N/A N/A N/A SF SFCFSPerforming live migration between LDOMs in the SmartIOenvironmentIf an array-based SSD is used, live migration is supported with SmartIO caching.With direct attached devices (PCIe), live migration is not supported if SmartIOcaching is enabled. If you need to performlive migration, you can use manual steps.To perform live migration in the SmartIO environment1 To prepare the LDOM for the live migration, perform the following steps: Offline the cache area that is created inside the LDOM.Ldom1:/root# sfcache offline cachearea_name Delete the cache area.Ldom1:/root# sfcache delete cachearea_name2 Remove the SSD device from the VxVM configuration so that the device canbe unexported from the LDOM.Ldom1:/root# vxdisk rm ssd_device_name3 Verify that the SSD device is removed from VxVM. The SSD device is notvisible in the output of the following command:Ldom1:/root# vxdisk list4 Unexport the device from the LDOM.Cdom1:/root> ldm remove-vdisk vdisk_name ldom15 After unexporting the local SSDdevice, performthe live migration of the LDOM.During the live migration, make sure that the application and mountpoints thatuse the SFHA objects are intact and running properly.20 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsAbout Symantec Storage Foundation and High Availability support for Solaris virtualization technology6 After the live migration completes, export the PCIe SSD devices that areavailable on the other control domain.Cdom1:/root> ldm add-vdsdev vxvm_device_path vds_device_name>@vdsCdom1:/root> ldm add-vdisk vdisk_name vds_device_name@vds ldom17 After exporting the local PCIe SSD devices, include the devices in the VxVMconfiguration that is inside the LDOM.Ldom1:/root> vxdisk scandisks8 Verify that the SSD device is visible in the output of the following command:Ldom1:/root# vxdisk list9 After the local PCIe device is available to the VxVM configuration, you cancreate the required SmartIO cache area.10To live migrate back the LDOM from target control domain to source controldomain, follow step 1 to step 9.Virtualization use cases addressed by StorageFoundation and High Availability SolutionsVirtualization use cases addressed by Storage Foundation and High AvailabilitySolutions:Table 1-3 Virtualization use casesDetails Use caseFor LDOMs:See Symantec Cluster Server setup to fail over an Applicationrunning inside logical domain on a failure of Applicationon page 206.For Zones:See Configuring VCS in zones on page 41.Application failover21 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsVirtualization use cases addressed by Storage Foundation and High Availability SolutionsTable 1-3 Virtualization use cases(continued)Details Use caseFor LDoms:See Symantec ApplicationHA in the guest domains (LDoms)on page 270.For Zones:See Configuring VCS in zones on page 41.Application monitoringand managementFor LDOMs:See Clustering using Cluster Volume Manager on page 189.For Zones:See SFCFSHA mounts on page 69.Fast failoverSee Oracle VM Server for SPARC guest domain migration inVCS environment on page 208.Live migrationPhysical to virtual migration in zone and LDomenvironments arebasically provided from the operating system.Physical to virtualmigrationFor LDOMs:See Configuring Oracle VM Server for SPARC guest domainsfor disaster recovery on page 183.For Zones:See Configuring Solaris non-global zones for disaster recoveryon page 116.Physical to virtualmigration for disasterrecoverySee Provisioning storage for a guest domain on page 167. Simplified management Consistent devicenaming Storage managementand provisioning Storage managementand boot diskprovisioningSee DMP as a multi-pathing solution in an Oracle VM Server forSPARC environment on page 156.See Oracle VM Server for SPARC deployment modelson page 139.Storage availability22 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsVirtualization use cases addressed by Storage Foundation and High Availability SolutionsTable 1-3 Virtualization use cases(continued)Details Use caseFor LDOMs:See About Oracle VM Server for SPARC on page 137.For Branded Zones:See About branded zones on page 127.For Zones:See About Solaris Zones on page 26.Server consolidationSee Array migration on page 140. Storage migration - arraymigrationFor LDoms:See About VCS in a Oracle VMServer for SPARCenvironmenton page 196.For Zones:See About VCS support for zones on page 26.Virtual machineavailability23 Overview of Symantec Storage Foundation and High Availability Virtualization SolutionsVirtualization use cases addressed by Storage Foundation and High Availability SolutionsZones and Projects Chapter 2. Storage Foundation and High Availability Solutions support for SolarisZones Chapter 3. Storage Foundation and High Availability Solutions support for SolarisProjects Chapter 4. Storage Foundation and High Availability Solutions support forBranded Zones2SectionStorage Foundation andHigh Availability Solutionssupport for Solaris ZonesThis chapter includes the following topics: About Solaris Zones About VCS support for zones Configuring VCS in zones Adding VxFS file systems to a non-global zone Mounting VxFS as lofs into a non-global zone Mounting VxFS directly into a non-global zone from global zone Mounting VxFS as VxFS inside a non-global zone Adding a direct mount to a zone's configuration Benefits of a VxFS mount in a non-global zone over VxFS mount from globalzone into the non-global zone SFCFSHA mounts Concurrent I/O access in non-global zones Veritas Extension for Oracle Disk Manager Exporting VxVM volumes to a non-global zone About SF Oracle RAC support for Oracle RAC in a zone environment2Chapter Setting up an SF Oracle RAC cluster with Oracle RAC on non-global zones Configuring Solaris non-global zones for disaster recovery Software limitations of Storage Foundation support of non-global zonesAbout Solaris ZonesSolaris Zones is a software partitioning technology, which provides a means ofvirtualizing operating systemservices to create an isolated environment for runningapplications. This isolation prevents processes that are running in one zone frommonitoring or affecting processes running in other zones.You can configure non-global zones with a shared-IP address or an exclusive-IPaddress. The shared-IP zone shares a network interface with global-zone and theexclusive-IP zone does not share network interface with global-zone.See the Oracle Solaris Administration: Oracle Solaris Zones, Oracle Solaris 10Zones, and Resource Management Solaris operating environment document.Oracle provides regular updates and patches for the Oracle Solaris Zones feature.Contact Oracle for more information.About VCS support for zonesSymantec Cluster Server (VCS) provides application management and highavailability to applications running in zones.With the Intelligent Monitoring Framework (IMF), VCS supports intelligent resourcemonitoring. The Zone agent is IMF-aware and uses the asynchronous monitoringframework (AMF) kernel driver for IMF notification.For more information about the Intelligent Monitoring Framework (IMF) and intelligentresource monitoring, refer to the Symantec Cluster Server Administrator's Guide.For more information about how to perform intelligent resource monitoring for theZone agent, see the Symantec Cluster Server Bundled Agents Reference Guide.Overview of how VCS works with zonesYou can use VCS to perform the following: Start, stop, monitor, and fail over a non-global zone. Start, stop, monitor, and fail over an application that runs in a zone.26 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout Solaris ZonesDescription TopicVCS provides zone aware resource types to monitor applicationsinside the zone. For the zone aware resources, C based entrypoints run fromglobal zone and fetch information fromnon-globalzone when ever required. The script based entry points run within non-global zone. If any resource faults, based on theconfiguration, VCS fails over either the service group with thezone or the service group with the application running in the zoneto another node.You can configure VCS to use Symantec Product AuthenticationService to run in a secure environment. Communication fromnon-global zones to global zones is secure in this environment.How VCS modelscontainersInstall and configure the zone. Create the service group with thestandard application resource types (application, storage,networking) and the Zone resource. VCS manages the zone asa resource. You then configure the service groups ContainerInfoattribute.You can use the hazonesetup and hazoneverify utilities tocreate and verify the configuration.Installing and configuringzones in VCSenvironmentsThe service group attribute ContainerInfo specifies informationabout the zone. When you have configured and enabled theContainerInfo attribute, you have enabled the zone-awareresources in that service group to work in the zone environment.VCS defines the zone information at the level of the service groupso that you do not have to define it for each resource. You mayspecify a per-system value for the ContainerInfo attribute.VCS has zone aware resource types which can be configured tomonitor resources inside the non-global zone or on the globalzone. For example, if a network interface is shared with the zone,the associated NIC resource should run in the global zone. If aresource is to be monitored inside non-global zone, you can defineservice group attribute ContainerInfo.Configuring theContainerInfo attributeIf you want to configure more than one zone resource in a servicegroup, you can set ResContainerInfo attribute at resource levelfor all the zone aware agents. In this case you must not setContainerInfo attribute at service group level.ResContainerInfoAbout the ContainerInfo service group attributeThe ContainerInfo attribute has the Name key, Type key, and Enabled key. TheName key defines the name of the container. The Type key lets you select the type27 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesof container that you plan to use. The Enabled key enables the Zone-awareresources within the service group. The ContainerInfo attribute specifies that theresources that belong to that Service Group will run inside the local zone, if theyare Zone-aware.Note: Zone-aware resources are resources capable of running inside a local zone.Assign the following values to the ContainerInfo attribute: NameThe name of the container. TypeThe type of container. You can set this to Zone. EnabledSpecify the value as 0, if you want to disable the container. Specify the valueas 1, if you want to enable the container. Specify the value as 2, to enablephysical to virtual and virtual to physical failovers. When the value is 2, the Zoneresource mimics a non-existent entity.You can set a per-system value for this attribute.About the ContainerOpts resource type attributeThe ContainerOpts resource attribute is pre-set for Zone-aware resource types. Itdetermines the following: Whether the zone-aware resource can run in the zone. Whether the container information that is defined in the service groupsContainerInfo attribute is passed to the resource.These values are only effective when you configure the ContainerInfo service groupattribute.Attribute's keys follow:The ContainerOpts resource type attribute's definitions for Zone-aware types containthe following values: RunInContainer (RIC)When the value of the RunInContainer key is 1, the agent function (entry point)for that resource runs inside of the local container.When the value of the RunInContainer key is 0, the agent function (entry point)for that resource runs outside the local container (in the global environment).A limitation for the RunInContainer value is that only script agent functions (entrypoints) can run inside a container.28 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zones PassCInfo (PCI)When the value of the PassCInfo key is 1, the agent function (entry point)receives the container information that is defined in the service groupsContainerInfo attribute. An example use of this value is to pass the name of thecontainer to the agent.About the ResContainerInfo resource type attributeSet ResContainerInfo attribute for each of the zone aware resources in a servicegroup. The ResContainerInfo attribute has the Name key, Type key, and Enabledkey. The Name key defines the name of the container.Assign the following values to this attribute: Name: The name of the container. Type: The type of container. You can set this to Zone. Enabled: Specify the value as 1 to enable the container. Specify the value as 0to disable the container. Specify the value as 2 to enable failovers from physicalcomputers to virtual machines and fromvirtual machines to physical computers.When ResContainerInfo attribute is set at any resource level, you need not setContainerInfo attribute at service group level. If you want to configure multiple zoneresources in a service group, you can use ResContainerInfo attribute.Zone-aware resourcesTable 2-1 1ists the ContainerOpts attributes default values for resource types.Zone-aware resources have predefined values for the ContainerOpts attribute.Note: Symantec recommends that you do not modify the value of the ContainerOptsattribute, with the exception of the Mount agent.See About the Mount agent on page 30.See About networking agents on page 40.Table 2-1 ContainerOpts attribute default values for applications andresourcetypesPassCInfo RunInContainer Resource type0 1 Apache0 1 Application29 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesTable 2-1 ContainerOpts attribute default values for applications andresourcetypes (continued)PassCInfo RunInContainer Resource type0 1 ASMInst0 1 ASMDG0 1 Db2udb1 0 NIC1 0 IP1 0 IPMultiNIC1 0 IPMultiNICB0 1 Process1 0 Zone0 1 Oracle0 1 Netlsnr0 1 Sybase0 1 SybaseBk0 1 ProcessOnOnly1 0 ProjectAbout the Mount agentYou may need to modify the ContainerOpts values for the Mount resource in certainsituations.In certain situations where the block device is not exported to zone, you can makethe file system available inside local zone. Mount the block device on the directorythat has a path that includes the zone root from global zone, for example:BlockDevice = /dev/vx/dsk/dg/vol1MountPoint = /zones/zone-test/root/mntptWhere /zones/zone-test is the zone root of the local zone.30 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesMount agent supports the following configuration for mount points1 Direct mount of file systemwith mount point as full path seen fromglobal zone.Typical mount resource configuration for this type of mount is shown below:group mntgrp (SystemList = { Sys1 = 0, Sys1 = 1 })Mount mnt-direct (MountPoint = "/zones/zone-test/root/mnt"BlockDevice = "/dev/vx/dsk/dg/vol"FSType = vxfsFsckOpt = "-y")2 Loop-back file system mount inside non-global zone for file system mountedin global zone. Typical mount resource configuration for this type of mount isshown below:group loopbacksg (SystemList = { sysA = 0, sysB = 1 }ContainerInfo@sysA = { Name = zone1, Type = Zone, Enabled = 1 }ContainerInfo@sysB = { Name = zone1, Type = Zone, Enabled = 1 })Mount zone_mnt (MountPoint = "/export/home/zone1/root/lofs_mnt"BlockDevice = "/mnt1/m1"FSType = lofs)Zone z1 ()Mount global_mnt (MountPoint = "/mnt1"BlockDevice = "/dev/vx/dsk/tdg/tvol1"FSType = vxfsFsckOpt = "-y")zone_mnt requires z1zone_mnt requires global_mnt31 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zones3 Direct mount of NFS based file system inside non-global zone. Typical mountresource configuration for this type of mount is shown below:group mntgrp (SystemList = { Sys1 = 0, Sys1 = 1 }ContainerInfo = { Name = zone-test, Type = Zone, Enabled = 1 })Mount mntnfs (MountPoint = "/mnt"BlockDevice = "system:/shared-dir"FSType = nfsFsckOpt = "-n"ContainerOpts = { RunInContainer = 1, PassCInfo = 0 })4 Support for direct mount of VxFS file system inside non-global zone. VCSMount agent supports the direct mount of VxFS file system inside non-globalzone. Typical mount resource configuration for this type of mount is shownbelow:group mntgrp (SystemList = { sys1 = 0, sys2 = 1 }ContainerInfo = { Name = zone-test, Type = Zone, Enabled = 1 }Administrators = { z_zoneres_sys1, z_zoneres_sys2 })Mount mnt-zone-direct (BlockDevice = "/dev/vx/dsk/data_dg/data_vol"MountPoint = "/mnt1"FSType = vxfsFsckOpt = "-y"ContainerOpts = { RunInContainer = 1, PassCInfo = 0 })Sample configurations for Physical-to-Virtual (P2V) scenariosIn the following sample configuration, the local zone (zone1) runs only on system1 (sys1) and it does not exist in system 2 (sys2). The mount resource comes onlineinside the zone. On system 1, the /export/home/zone1/root/mnt mount pointexists inside the zone. On system 1, the /export/home mount point is mounted on/export/home/zone1/root/mnt inside the zone. On system 2, the /export/homemount point is mounted on /mnt in the global zone.32 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesgroup mountgrp (SystemList = { sys1 = 0, sys2 = 1 }ContainerInfo @sys1 = { Name = zone1, Type = Zone, Enabled = 1 }ContainerInfo @sys2 = { Name = zone1, Type = Zone, Enabled = 2 }Administrators = { z_mountres_sys1, z_zoneres_sys1 })Mount loopback (MountPoint @sys1 = "/export/home/zone1/root/mnt"MountPoint @sys2 = "/mnt"BlockDevice = "/export/home"FSType = lofsFsckOpt = "-n")Zone zoneres ()loopback requires zoneresIn this sample configuration, there are two mount resources. The first mount resource(mountres) mounts the actual shared volume /dev/vx/dsk/vdg/nfs_vol on/lockinfo. Next, /lockinfo is loop-backed on /export/home/zone1/root/mntinside the zone on system 1. On system 2, the /dev/vx/dsk/vdg/nfs_vol volumeis mounted on /lockinfo and then /lockinfo is loop-backed and mounted on /mntin the global zone. In the following sample, localization has been done for the mountresource:group mountgrp (SystemList = { sys1 = 0, sys2 = 1 }ContainerInfo @sys1 = { Name = zone1, Type = Zone, Enabled = 1 }ContainerInfo @sys2 = { Name = zone1, Type = Zone, Enabled = 2 }Administrators = { z_mountres_sys1, z_zoneres_sys1 })DiskGroup dgres (DiskGroup = vdg)Mount loopback (MountPoint @sys1 = "/export/home/zone1/root/mnt"MountPoint @sys2 = "/mnt"33 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesBlockDevice = "/lockinfo"FSType = lofsFsckOpt = "-n")Mount mountres (MountPoint = "/lockinfo"BlockDevice = "/dev/vx/dsk/vdg/nfs_vol"FSType = vxfsFsckOpt = "-y")Zone zoneres ()loopback requires mountresloopback requires zoneresmountres requires dgresBringing a Mount resource online in the zoneThe Mount resource is brought online in the global zone by default (RunInContainer= 0). If you want to bring a mount resource online inside the non-global zone, performthe following: Export the block device to the zone through zone configuration. Ensure that theraw volume is used appropriately to prevent the possibility of data corruption. Modify the ContainerInfo attribute for the service group and set values for theName, Type, and Enabled keys.# hagrp -modify service_group ContainerInfo Name zone_name \Type Zone Enabled 1 Override the ContainerOpts attribute at the resource level. Set the value of the RunInContainer key to 1, for example:# hares -override Mountres ContainerOpts# hares -modify Mountres ContainerOpts \RunInContainer 1 PassCInfo 0For information on overriding resource type static attributes, see the SymantecCluster Server Administrator's Guide.34 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesSetting the attribute values for a Mount resource for NFSmountsFor NFS mounts, you must mount in the non-global zone. Modify the ContainerInfo attribute for the service group and set values for Name,Type and Enabled keys. Override the ContainerOpts attribute at the resource level. Set the value of the RunInContainer key to 1.Set the RICvalue to 1. When you set RIC=1, specify the value of the MountPointattribute relative to the zone root, for example:BlockDevice = abc:/fs1MountPoint = /mnt1The file system is mounted on /zone_root/mnt1.Configuring a direct mount of VxFS file systemin a non-globalzone with VCSTypical steps to configure a direct mount inside a non-global zone.To configure a direct mount inside a non-global zone1 Create a VxVM disk group and volume: Create a VxVM disk group from a device:global# vxdg init data_dg c0t0d1 Create a volume from a disk group:global# vxassist -g data_dg make data_vol 5GFor more information, see the Symantec Storage Foundation Administrator'sGuide.2 Create a zone: Create a root directory for a zone local-zone and change its permission to700:global# mkdir -p /zones/local-zoneglobal# chmod 700 /zones/local-zone On Solaris 10, configure a zone local-zone:35 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesglobal# zonecfg -z local-zonelocal-zone: No such zone configuredUse `create' to begin configuring a new zone.zonecfg:local-zone> createzonecfg:local-zone> set zonepath=/zones/local-zonezonecfg:local-zone> add netzonecfg:local-zone:net> set physical=eri0zonecfg:local-zone:net> set address=192.168.5.59zonecfg:local-zone:net> endzonecfg:local-zone > verifyzonecfg:local-zone > commitzonecfg:local-zone > exitThe zone is in configured state.On Solaris 11, configure a zone local-zone:global# zonecfg -z local-zonelocal-zone: No such zone configuredUse `create' to begin configuring a new zone.zonecfg:local-zone> createzonecfg:local-zone> set zonepath=/zones/local-zonezonecfg:local-zone> set ip-type=sharedzonecfg:local-zone> add netzonecfg:local-zone:net> set physical=eri0zonecfg:local-zone:net> set address=192.168.5.59zonecfg:local-zone:net> endzonecfg:local-zone > verifyzonecfg:local-zone > commitzonecfg:local-zone > exitThe zone is in configured state. Install the zone:global# zoneadm -z local-zone install Login to the zone console to setup the zone from terminal 1:global# zlogin -C local-zone36 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zones Boot the zone from another terminal:global# zoneadm -z local-zone boot Follow the steps on terminal 1 on the zone console to setup the zone.See the Oracle documentation for more information about creating a zone.3 Add VxVM volumes to the zone configuration: Check the zone status and halt the zone, if it is running:global# zoneadm list -cvID NAME STATUS PATH BRAND IP0 global running / native shared2 local-zone running /zones/myzone native sharedglobal# zoneadm -z myzone halt Add the VxVM devices to the zone's configuration:global# zonecfg -z local-zonezonecfg:local-zone:fs> add devicezonecfg:local-zone:fs> set match=/dev/vxportalzonecfg:local-zone:fs> endzonecfg:local-zone:fs> add devicezonecfg:local-zone:fs> set match=/dev/fddzonecfg:local-zone:fs> endzonecfg:local-zone:fs> add devicezonecfg:local-zone:fs> set match=/dev/vx/rdsk/data_dg/data_volzonecfg:local-zone:fs> endzonecfg:local-zone:fs> add devicezonecfg:local-zone:fs> set match=/dev/vx/dsk/data_dg/data_volzonecfg:local-zone:fs> endzonecfg:local-zone:fs> add fszonecfg:local-zone:fs> set dir=/etc/vx/licenses/liczonecfg:local-zone:fs> set special=/etc/vx/licenses/liczonecfg:local-zone:fs> set type=lofszonecfg:local-zone:fs> endzonecfg:local-zone> verifyzonecfg:local-zone> commitzonecfg:local-zone> exit On Solaris 11, you must set fs-allowed to vxfs and odm in the zone'sconfiguration:37 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesglobal# zonecfg -z myzonezonecfg:myzone> set fs-allowed=vxfs,odmzonecfg:myzone> commitzonecfg:myzone> exitSee Adding a direct mount to a zone's configuration on page 67.See Mounting VxFS as VxFS inside a non-global zone on page 65. Boot the zone:global# zoneadm -z myzone boot4 Create a VxFS file system on the volume inside a non-global zone: Login to the local-zone:global# zlogin myzone Create a VxFS file system on the block device:bash-3.00# mkfs -F vxfs /dev/vx/dsk/data_dg/data_vol5 Create a mount point inside the zone: Login to the local-zone:global# zlogin myzone Create a mount point inside the non-global zone:bash-3.00# mkdir -p /mydata Mount the VxFS file system on the mount point:bash-3.00# mount -F vxfs /dev/vx/dsk/data_dg/data_vol /mydata6 Configure the zone service group: On the first node, create the service group with password-lesscommunication with global zone:global# hazonesetup -g zone_grp -r zone_res -z myzone \-p password -s sysA,sysB Switch the service group from the first node to the second node and runthe hazonesetup command to setup password-less communication fromthe next node.38 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zones Repeat step 6 for all the nodes in the cluster where the zone can go online.See Configuring a zone resource in a failover service group with thehazonesetup utility on page 51.7 Create a mount, disk group, and volume resources into the service group: Add a disk group resource to the service group:global# hares -add dg_res DiskGroup zone_grpglobal# hares -modify dg_res DiskGroup data_dgglobal# hares -modify dg_res Enabled 1 Add a volume resource to the service group:global# hares -add vol_res Volume zone_grpglobal# hares -modify vol_res Volume data_volglobal# hares -modify vol_res DiskGroup data_dgglobal# hares -modify vol_res Enabled 1 Add a Mount resource to the service group:global# hares -add mnt_res Mount zone_grpglobal# hares -modify mnt_res BlockDevice \/dev/vx/dsk/data_dg/data_volglobal# hares -modify mnt_res MountPoint /mydataglobal# hares -modify mnt_res FSType vxfsglobal# hares -modify mnt_res FsckOpt %-yglobal# hares -modify mnt_res Enabled 1 Create a resource dependency between the resources in the service group:global# hares -link zone_res vol_resglobal# hares -link vol_res dg_resglobal# hares -link mnt_res zone_res8 Set the ContainerOpts attribute for the Mount resource for VxFS direct mount: Override the ContainerOpts attribute at the resource level for mnt_res:global# hares -override mnt_res ContainerOpts Set the value of the RunInContainer key to 1:global# hares -modify mnt_res ContainerOpts RunInContainer \1 PassCInfo 039 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesFor information on overriding resource type static attributes: see the SymantecCluster Server Administrator's Guide.9 Here is a sample configuration for the VxFS direct mount service groups in themain.cf file:group zone_grp (SystemList = {sysA = 0, sysB = 1 }ContainerInfo = { Name = local-zone, Type = Zone, Enabled = 1 }Administrators = { z_zoneres_sysA, z_zoneres_sysB })Mount mnt_res (BlockDevice = "/dev/vx/dsk/data_dg/data_vol"MountPoint = "/mydata"FSType = vxfsFsckOpt = "-y"ContainerOpts = { RunInContainer = 1, PassCInfo = 0 })DiskGroup dg_res (DiskGroup = data_dg)Volume vol_res (Volume = data_volDiskGroup = data_dg)Zone zone_res ()zone_res requires vol_resvol_res requires dg_resmnt_res requires zone_resAbout networking agentsEnable the attribute ExclusiveIPZone for resources of type IP and NIC when theseresources are configured to manage the IP and the NICinside an exclusive-IP zone.This attribute is disabled by default. The IP agent and the NIC agent assumes thenative zone (shared-IP) by default.40 Storage Foundation and High Availability Solutions support for Solaris ZonesAbout VCS support for zonesSymantec Cluster Server (VCS) brings resources online in the global zone bydefault.If you want to bring these resources online inside the exclusive-IP zone, performthe following tasks: Make sure that the resource is in a service group that has valid ContainerInfoattribute value configured. Set the value of the ExclusiveIPZone attribute to 1.Note: The exclusive-IP zone supports the IP and NIC networking agents. For moreinformation about these agents, see the Symantec Cluster Server Bundled AgentsReference Guide.About the Zone agentThe Zone agent monitors zones, brings them online, and takes them offline. Formore information about the agent, see the Symantec Cluster Server Bundled AgentsReference Guide.Use the hazonesetup utility to create user account with group administrativeprivileges. The DeleteVCSZoneUser attribute of zone resource controls removingthe user account when the zone resource is taken offline. For more information,see the Symantec Cluster Server Bundled Agents Reference Guide.About configuring failovers among physical and virtual serversYou can configure VCS to fail over from a physical system to a virtual system andvice versa. A physical to virtual failover gives an N + N architecture in an N + 1environment. For example, several physical servers with applications can fail overto containers on another physical server.See Configuring for physical to virtual and virtual to physical failoversa typicalsetup on page 62.Configuring VCS in zonesConfiguring VCS in zones involves the following tasks:Review the prerequisites.See Prerequisites for configuring VCS in zones on page 42.First41 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesDecide on the location of the zone root, which is either on local storageor shared storage.See Deciding on the zone root location on page 43.SecondInstall the application in the zone.See About installing applications in a zone on page 47.ThirdCreate the application service group and configure its resources.See Configuring the service group for the application on page 48.FourthPrerequisites for configuring VCS in zonesReview the following prerequisites for configuring VCS in zones: For Oracle Solaris 10, VCS supports UFS, ZFS, CFS, and VxFS, CFS mountsfor the zone root. For Oracle Solaris 11, VCS supports only ZFS for zone root.Method for file system access inside non-global zoneFile system mounts must meet one of the following two conditions: Use a loopback file system with zone configuration. All mounts that theapplication uses must be part of the zone configuration and must be configuredin the service group. For example, you can create a zone, z-ora, and define thefile systemcontaining the applications data to have the mount point as /oradata.When you create the zone, you can define a path in the global zone. An exampleis /export/home/oradata, which the mount directory in the non-global zonemaps to. The MountPoint attribute of the Mount resource for the application isset to /export/home/oradata. Confirm that /export/home/oradata maps to/oradata with the zonecfg -z zone_name info command. You can also lookinto the zone configuration /etc/zones/zone_name.xml file. The Zone resourcedepends on the Mount resource. Mount the file system inside zones root-path. All file system mount pointsthat the application uses that run in a zone must be set relative to the zonesroot. For example, if the Oracle application uses /oradata, and you create thezone with the zonepath as /z_ora, then the mount must be/z_ora/root/oradata. The MountPoint attribute of the Mount resource mustbe set to this path. The Mount resource depends on the Zone resource.Using custom agents in zonesIf you use custom agents, review the following information for their use in zones:42 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones If you use custom agents to monitor the applications that run in zones, you canuse C++ or script-based entry points. If you want the customagent to monitor an application in the zone, for the customagent type, set the following values for the ContainerOpts attribute:RunInContainer = 1 and the PassCInfo = 0. If you don't want the custom agent to monitor an application in the zone, for thecustom agent type, set the following values for the ContainerOpts attribute:RunInContainer = 0 and the PassCInfo = 0. Two main use cases exist where you might want to use a RunInContainer = 0and PassCInfo = 1, descriptions of these follow. The first is the Zone agent's use of these values. The Zone agent's entrypoints cannot run inside of the non-global zone but the agent itself managesthe zone. RunInContainer requires a value of 0 because the agent must runin the global zone. PassCInfo has a value of 1 because the Zone agentrequires the name of the container from the ContainerInfo service groupattribute. The second case is how the IP agent uses RunInContainer and PassCInfo.The IP agent's entry points must run outside of the non-global zone becausea shared-IP zone may cause the networking stack to not completely run inthe non-global zone. You cannot perform an ifconfig command and thenplumb the IP from inside of a non-global zone. When you run the ifconfigcommand in the global zone with the zone option - it plumbs the IP andmakes it available to the zone that you specify. The need for the container'sname comes from the use of this command, even though it cannot run inthe container. This is applicable to all networking agents.If the non-global zone is of type exclusive IP zone, the agent runs the scriptbased entrypoints inside the non-global zone if the zone is in running state.The agents runs the script based entrypoints in global zone if the zone isnot in running state.For NICassigned to exclusive IP Zone and IP to be plumbed inside exclusiveIP zone you can set ExclusiveIPZone attribute available to NIC and IPresources.Deciding on the zone root locationEach zone has its own section of the file systemhierarchy in the zone root directory.Processes that run in the zone can access files only within the zone root.You can set the zone root in the following two ways: Zone root on local storage43 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesIn this configuration, you must configure and install a zone on each node in thecluster.To configure a zone under VCS control using the hazonesetup utility when thezone root is on local storage Zone root on shared storageIn this configuration, configure and install a zone in shared storage from onesystem and duplicate the configuration on each node in the cluster.Setting the zone root on shared storage means you need to install the non-globalzone on shared storage from one system only. The zone root can fail over tothe other systems. To do this, the system software, including the patches, mustbe identical on each system during the existence of the zone.To configure a zone under VCS control using the hazonesetup utility when thezone root is on shared storageCreating a zone with root on local diskCreate a zone root on the local disk on each node in the cluster. The file systemfor application data is on a shared device and is either the loopback type or thedirect mount type. For a direct mount file system, run the mount command from theglobal zone with the mount point specified as the complete path that starts with thezone root. For a loopback file system, add it into the zone's configuration beforeyou boot the zone.To create a zone root on local disks on each node in the cluster1 Configure the zone with the zonecfg command.zonecfg -z newzonezonecfg:newzone> create2 Set the zonepath parameter to specify a location for the zone root.zonecfg:newzone> set zonepath=/export/home/newzone3 For exclusive IP zone, set the ip-tye to exclusive:zonecfg:newzone> set ip-type=exclusive44 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones4 Add network interface to zone configuration. This is required for non-globalzone to communicate with had running in global zone.zonecfg:newzone> add netzonecfg:newzone:net> set physical=bge1zonecfg:newzone:net> set address=192.168.1.10zonecfg:newzone:net> end5 Make sure global zone can be pinged from non-global zone with global zonehostname. You may need to add global zone hostname entry to /etc/hostsfile inside non-global zone or enable DNS access from inside the non-globalzone.6 If your application data resides on a loopback mount file system, create theloopback file system in the zone.7 Exit the zonecfg configuration.zonecfg> exit8 Create the zone root directory.mkdir zonepath9 Set permissions for the zone root directory.chmod 700 zonepath10Install the non-global zone.zoneadm -z newzone install11Repeat step 1 to step 10 on each system in the service groups SystemList.12If the application data is on a loopback file system, mount the file systemcontaining the applications data on shared storage.13Boot the zone.zoneadm -z newzone boot14If the application data is on a direct mount file system, mount the file systemfrom the global zone with the complete path that starts with the zone root.45 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesCreating a zone with root on shared storageCreate a zone with root which points to the shared disk's location on each node inthe cluster. The file system for application data is on a shared device and is eitherthe loopback type or the direct mount type. For a direct mount file system, run themount command fromthe global zone with the mount point specified as the completepath that starts with the zone root. For a loopback file system, add it into the zone'sconfiguration before you boot the zone.To create a zone root on shared disks on each node in the cluster1 Create a file system on shared storage for the zone root. The file system thatis to contain the zone root may be in the same disk group as the file systemthat contains the application data.2 Configure the zone with the zonecfg command.zonecfg -z newzonezonecfg:newzone> create3 Set the zonepath parameter to specify a location for the zone root.zonecfg:newzone> set zonepath=/export/home/newzone4 Add network interface to zone configuration. This is required for non-globalzone to communicate with had running in global zone.zonecfg:newzone> add netzonecfg:newzone:net> set physical=bge1zonecfg:newzone:net> set address=192.168.1.10zonecfg:newzone:net> end5 Make sure global zone can be pinged from non-global zone with global zonehostname. You may need to add global zone hostname entry to /etc/hostsfile inside non-global zone or enable DNS access from inside the non-globalzone.6 If your application data resides on a loopback mount file system, create theloopback file system in the zone.7 Exit the zonecfg configuration.zonecfg> exit8 Create the zone root directory.mkdir zonepath46 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones9 Set permissions for the zone root directory.chmod 700 zonepath10Repeat step 2 to step 9 on each system in the service groups SystemList.11Mount the file system that contains the shared storage on one of the systemsthat share the storage to the directory specified in zonepath.12Run the following command to install the zone on the system where the zonepath is mounted.zoneadm -z newzone install13If the application data is on a loopback file system, mount the file systemcontaining the applications data on shared storage.14Boot the zone.zoneadm -z newzone boot15If the application data is on a direct mount file system, mount the file systemfrom the global zone with the complete path that starts with the zone root.Performing the initial internal zone configurationWhen a zone is booted for the first time after installation, the zone is in anunconfigured state. The zone does not have an internal configuration for namingservices. Its locale and time zone have not been set, and various other configurationtasks have not been performed. You need to perform the initial internal zoneconfiguration after zone installation.You can perform the internal zone configuration in the following ways: sysidcfg tool Zone console loginFor more details refer to Oracle documentation about "Performing the Initial InternalZone Configuration" section in the Oracle Solaris Administration: Oracle SolarisZones, Oracle Solaris 10 Zones, and Resource Management guide.About installing applications in a zonePerform the following tasks to install the application in a zone: If you have created zones locally on each node in the cluster, install theapplication identically in all zones on all nodes. If you are installing an application47 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesthat is supported by a Veritas High Availability agent, see the installation andconfiguration guide for the agent. Install the agent packages on the global zone and the currently existing zones.Installs the agents in future zones when they are installed. On Solaris 10, if the non-global zone is in running state, installing the packageon the global zone updates the non-global zone also. If the non-global zoneis in configured state, the zone needs to be attached with the -U option. On Solaris 11, the package needs to be installed separately inside eachnon-global zone. You must define all the mount points that the application uses that are configuredin the zone in the service group's configuration.Configuring the service group for the applicationYou need to configure the application service group and the required resourcedependencies. The following diagrams illustrate different examples of resourcedependencies. In one case the zone root is set up on local storage. In the other,zone root is set up on shared storage.Resource dependency diagrams: zone root on local disksThe following resource dependency diagrams show zone configurations on localdisks configured for loopback and direct mounted file systems.If the zone root is on a cluster file system, ensure that service group containingzone resource is a fail-over type service group.Figure 2-1 depicts the dependency diagram when the zone root is set up on localstorage with the loopback file systemfor the application. You can replace the Mountresource with the CFSMount resource and the DiskGroup resource with theCVMVolDg resource in the following diagram. In this configuration, decide if youwant the service group to be a parallel service group. If so, you may need to localizecertain attributes for resources in the service group. For example, you have tochange the IP resource's Address attribute for each node.48 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesFigure 2-1 Zone root on local disks with loopback file systemDiskGroupMountZoneNICIPApplicationApplicationFigure 2-2 depicts the dependency diagram when the zone root is set up on localstorage with a direct mount file system for the application. You can replace theMount resource with the CFSMount resource and the DiskGroup resource with theCVMVolDg resource in the following diagram. In this configuration, decide if youwant the service group to be a parallel service group. If so, you may need to localizecertain attributes for resources in the service group. For example, you have tochange the IP resource's Address attribute for each node.Figure 2-2 Zone root on local disks with direct mount file systemMountZoneNICIPApplicationDiskGroupManagesmounting andumounting theApplication filesystemIn a typical use case where application runs in parallel inside zones, you can havefollowing configuration on Solaris. You can configure parallel service groupcontaining zone resource whose zone roots are on a VxVM volume or Zpool andnot shared between the nodes. The storage for the application is set up on sharedstorage between the nodes and managed using CFSMount and CVMVolDGresources. You may need to localize certain attributes for resources in the servicegroup. For example you may have to change the IP resources Address attribute,the DiskGroup resources DiskGroup attribute or the Zpool resources PoolNameattribute.49 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesResource dependency diagrams: zone root on shared disksThe following resource dependency diagrams show zone configurations on shareddisks configured for loopback and direct mounted file systems.Figure 2-3 depicts the dependency diagram when a zone root is set up on sharedstorage with the loopback file system. You can replace the Mount resource withthe CFSMount resource and the DiskGroup resource with the CVMVolDg resourcein the following diagram for application. In this configuration, decide if you want theservice group to be a parallel service group. If so, you may need to localize certainattributes for resources in the service group. For example, you have to change theIP resource's Address attribute for each node.Figure 2-3 Zone root on shared storage with loopback file systemDiskGroupMountZoneNICIPApplicationDiskGroup/ZpoolMountZonerootApplicationfile systemFigure 2-4 depicts the dependency diagram when a zone root is set up on sharedstorage with the direct mount file system for the application. You can replace theMount resource with the CFSMount resource and the DiskGroup resource with theCVMVolDg resource in the following diagram for application. In this configuration,decide if you want the service group to be a parallel service group. If so, you mayneed to localize certain attributes for resources in the service group. For example,you have to change the IP resource's Address attribute for each node.50 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesFigure 2-4 Zone root on shared storage a direct mounted file systemMountZoneNICIPApplicationDiskGroup MountDiskGroup/ZpoolApplicationfile systemApplicationdisk groupZone rootfile systemUse the following principles when you create the service group: Set the MountPoint attribute of the Mount resource to the mount path. If the application requires an IP address, configure the IP resource in the servicegroup.Configuring a zone resource in a failover service group with thehazonesetup utilityThe hazonesetup utility helps you configure a zone under VCS. This section coverstypical scenarios based on where the zone root is located.Two typical setups for zone configuration in a failover scenario follow: Zone root on local storageTo configure a zone under VCS control using the hazonesetup utility when thezone root is on local storage Zone root on shared storageTo configure a zone under VCS control using the hazonesetup utility when thezone root is on shared storageConsider an example in a two-node cluster (sysA and sysB). Zone local-zone isconfigured on both the nodes.51 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesTo configure a zone under VCS control using the hazonesetup utility when the zoneroot is on local storage1 Boot the non-global zone on first node outside VCS.sysA# zoneadm -z local-zone boot2 To use the hazonesetup utility, ensure you have a IP configured for thenon-global zone and hostname of the global zone is resolvable fromnon-globalzone.# zlogin local-zone# ping sysA3 Run the hazonesetup utility with correct arguments on the first node. This addsfailover zone service group and zone resource in VCS configuration.sysA# hazonesetup -g zone_grp -r zone_res -z local-zone\-p password -a -s sysA,sysBNote: If you want to use a particular user for password-less communicationuse -u option of the hazonesetup utility. If -u option is not specified a defaultuser is used for password-less communication.4 Switch the zone service group to next node in the cluster.sysA# hagrp -switch zone_grp -to sysB5 Run the hazonesetup utility with correct arguments on the node. Thehazonesetup utlity detects that the zone service group and zone resource arealready present in VCS configuration and update the configuration accordinglyfor password-less communication.sysB# hazonesetup -g zone_grp -r zone_res -z local-zone\-p password -a -s sysA,sysB6 Repeat step 4 and step 5 for all the remaining nodes in the cluster.52 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zonesTo configure a zone under VCS control using the hazonesetup utility when the zoneroot is on shared storage1 Configure a failover service group with required storage resources (DiskGroup,Volume, Mount, etc.) to mount the zone root on the node. Set the requireddependency between storage resources (DiskGroup->Volume->Mount). Makesure you configure all the required attributes of all the storage resources inorder to bring them online on cluster node.sysA# hagrp -add zone_grpsysA# hagrp -modify zone_grp SystemList sysA 0 sysB 1sysA# hares -add zone_dg DiskGroup zone_grpsysA# hares -add zone_vol Volume zone_grpsysA# hares -add zone_mnt Mount zone_grpsysA# hares -link zone_mnt zone_volsysA# hares -link zone_vol zone_dgsysA# hares -modify zone_dg DiskGroup zone_dgsysA# hares -modify zone_dg Enabled 1sysA# hares -modify zone_vol Volume volume_namesysA# hares -modify zone_vol DiskGroup zone_dgsysA# hares -modify zone_vol Enabled 1sysA# hares -modify zone_mnt MountPoint /zone_mntsysA# hares -modify zone_mnt BlockDevice /dev/vx/dsk/zone_dg/volume_namesysA# hares -modify zone_mnt FSType vxfssysA# hares -modify zone_mnt MountOpt rwsysA# hares -modify zone_mnt FsckOpt %-ysysA# hares -modify zone_mnt Enabled 1When the zone root is on a ZFS file system, use the following commands:sysA# hagrp -add zone_grpsysA# hagrp -modify zone_grp SystemList sysA 0 sysB 1sysA# hares -add zone_zpool Zpool zone_grpsysA# hares -modify zone_zpool AltRootPath /zone_root_mntsysA# hares -modify zone_zpool PoolName zone1_poolsysA# hares -modify zone_zpool Enabled 12 Bring the service group online on first node. This mounts the zone root on firstnode.sysA# hagrp -online zone_grp -sys sysA53 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones3 Boot the local zone on first node outside VCS.sysA# zoneadm -z local-zone boot4 To use the hazonesetup utility, ensure you have a IP configured for thenon-global zone and hostname of the global zone is resolvable fromnon-globalzone.# zlogin local-zone# ping sysA5 Run the hazonesetup utility with correct arguments on the first node. Use theservice group configured in step 1. This adds the zone resource to VCSconfiguration.sysB# hazonesetup -g zone_grp -r zone_res -z local-zone \-p password -a -s sysA,sysBNote: If you want to use a particular user for password-less communicationuse -u option of the hazonesetup utility. If -u option is not specified a defaultuser is used for password-less communication.6 Set the proper dependency between the Zone resource and other storageresources. The Zone resource should depend on storage resource (Mount orZpool ->Zone).sysA# hares -link zone_res zone_mntWhen the zone root is on a ZFS file system, use following command:sysA# hares -link zone_res zone_zpool7 Switch the service group to next node in the cluster.sysA# hagrp -switch zone_grp -to sysB54 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones8 Run the hazonesetup utility with correct arguments on the node. Thehazonesetup utility detects that the service group and the zone resource arealready present in VCS configuration and update the configuration accordinglyfor password-less communication.sysB# hazonesetup -g zone_grp -r zone_res -z local-zone\-p password -a -s sysA,sysB9 Repeat step 7 and step 8 for all the remaining nodes in the clusterConfiguring zone resource in a parallel service group with thehazonesetup utilityThe hazonesetup utility helps you configure a zone under VCS. This section coverstypical scenarios based on the location of the zone root.In the case of a zone resource in parallel service group, the zone root can be onlocal or shared storage that the node owns.Consider an example in a two-node cluster (sysA and sysB). Zone local-zone1 isconfigured on sysA and local-zone2 is configured on sysB.To configure a zone under VCS control using the hazonesetup utility when the zoneroot is on local storage1 Boot the local zone on all the nodes outside VCS.sysA# zoneadm -z local-zone1 bootsysB# zoneadm -z local-zone2 boot2 To use the hazonesetup utility, ensure you have a IP configured for thenon-global zone and hostname of the global zone is resolvable fromnon-globalzone.55 Storage Foundation and High Availability Solutions support for Solaris ZonesConfiguring VCS in zones3 Run the hazonesetup utility with correct arguments on all the nodessuccessively.sysA# hazonesetup -g zone_grp -r zone_res -z local-zone1\-p password -a -l -s sysA,sysBsysB# hazonesetup -g zone_grp -r zone_res -z local-zone2\-p password -a -l -s sysA,sysBNote: If you want to use a particular user for password-less communicationuse -u option of the hazonesetup command. If -u option is not specified adefault user is used for password-less communication.4 Running the hazonesetup utility on first node adds parallel zone service groupand zone resource in VCS configuration. Running the hazonesetup utility onother nodes detect that the zone service group and zone resource are alreadypresent in VCS configuration and update the configuration accordingly forpassword-less communication.Note: Run the hazonesetup utility on all the nodes in the cluster that have a zonerunning on that node. This is required as hazonesetup runs the halogin commandinside the local zone that enables password-less communication between localzone and global zone.You can