media54650_AppSync-2.0-Technical-Presentation.pptx

Embed Size (px)

Citation preview

TITLE 44 POINT META NORMAL LF ALL CAPS

EMC AppSync 2.0Technical PresentationP/N 302-001-13501June 27, 2014 #EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync 2.0Technical PresentationF1950: Oracle Database for AIX & LinuxAditya Sinha / Tony NastaDate #EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaBrief introduction to Oracle in AppSyncSupported platforms & versionsSupported Oracle SoftwareSupported Host Operating SystemsSupported Host SoftwareSupported Storage PlatformsConfiguration informationSupported filesystem-based configurationsSupported ASM-based configurationsSupported RAC configurationsSupported VCS/HACMP configurations#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaUNIX Agent/Host-Plugin Deployment OverviewUNIX Agent/Host-Plugin SettingsCopy managementProtection of Oracle databasesProtecting HA/RAC & failover Oracle databasesAppSync behavior during protectionVNX Consistency group limitationMount & recovery of Oracle copiesMount & recovery of Oracle copiesScript-assisted manual recovery of Oracle copiesMount & catalog of Oracle copies with RMANMount & recovery of Oracle copies as RAC databases

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaRestoring Oracle copiesRestore of Oracle copies in standalone environmentsRestore of Oracle copies in RAC/cluster environmentsRestore considerations & warningsTroubleshootingQuestions & Answers#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearIntroductionThis release of AppSync is the first to contain support for protection & copy management of Oracle databases. As part of this effort, broad support for the Linux and AIX host platforms were addedThe focus of this release was on delivering platform & feature parity with Replication Manager while enhancing feature support wherever possible.Large focus on scalability and reducing overhead for discovery and file mapping operations from RM.In addition, we have strived to include numerous enhancements from RM and reduce customer pain points on certain issues#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync vs. RM at-a-glanceAppSyncRMHost resource consumptionLightweight worker agent only runs to do workDaemon, runs all the time. Susceptible to memory leak/growth.CommunicationEncrypted & secure using SSH (Standardized administration protocol for UNIX OSes)Encrypted & proprietary protocolApplication SecurityApp/Host credentials are not stored nor transmitted across network connectionsOracle application credentials are stored and transmitted across network connectionsApplication host workloadApplication & host operations onlyApplication, host & storage operationsThird party software dependencyNoneSolutions Enabler requiredNon-root capableYesNoPush install capabilityAll (Releases, Patches & Hotfixes)NoFile transferMinimal, large files not transferredAll auxiliary files are transferred over the networkOracle configuration relianceNoneOCI, TNS#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearSupported Oracle VersionsOracle ProductVersions SupportedOracle Database11gR2 (11.2.0.X)12cR1 (12.1.0.1)Oracle Grid Infrastructure11gR2 (11.2.0.3 required for ASM disk group rename)12cR1 (12.1.0.1)#EMC CONFIDENTIALINTERNAL USE ONLYSupported Host Operating SystemsOperating SystemVersions SupportedRed Hat Enterprise Linux5.5-5.10, 6.0-6.5Oracle Linux5.5-5.10, 6.0-6.5SUSE Linux Enterprise Server11 (SP1-SP3)CentOS5.5-5.10, 6.0-6.5IBM AIX6.1 TL2-TL9, 7.1 TL0-TL3** RPQ Only#EMC CONFIDENTIALINTERNAL USE ONLYKingfisher: Need proper AIX TL coverage on this slide#TitleMonth YearSupported Host SoftwareMPIO PackageOperating SystemsVersionsEMC PowerPathLinux, AIX5.7Veritas DMPLinux6.0, 6.1Native MPIOLinux, AIXLVM PackageOperating SystemsVersionsVeritas VxVMLinux6.0, 6.1Native LVMLinux, AIXCluster SuiteOperating SystemsVersionsIBM PowerHAAIX6.1, 7.1Veritas VCSLinux5.5-6.1#EMC CONFIDENTIALINTERNAL USE ONLYKingfisher: Need proper HACMP version coverage

#TitleMonth YearSupported Storage PlatformsArray Type & VersionFeaturesVMAX V2 (5876 Microcode required)SMI-S server running 4.6.2.10 minimum also requiredVPSnap, TFClone, SRDF, RecoverPoint BookmarksVNX (Flare 33 or higher required)Advanced Snapshots, RecoverPoint BookmarksRecoverPoint 3.5 SP2, 4.0CDP/CRR Bookmarks#EMC CONFIDENTIALINTERNAL USE ONLYAnyone: Need proper array microcode/flare versioning, etc Any missing fields??

#TitleMonth Yeararch_vgdata_vgSupported Configurations (filesystems)OracleDatabaseControl filesRedo logsData filesArchive logs/datafs/redofs/archfsSupported filesystems:AIX: jfs, jfs2 - NFS is not supported in this releaseLinux: ext2, ext3, ext4, vxfs, XFS - NFS is not supported in this release

Key requirements of Oracle on FilesystemsArchive logs must reside on different storage devices from the rest of the databaseArchive logs should be separate filesystem from other database filesArchive log filesystems should be on separate volume groups from other database filesFast Recovery Area, if present (and marked for protection) must follow the same rules as the archive log destinations. It does not need to share the same filesystem or underlying storage as archive logs, but it must not use the same storage as the control files, redo logs and data filesTemp tablespaces are not protected and can be placed anywherelun1lun2lun3lun4lun5#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year+ARCH+DATASupported Configurations (ASM)OracleDatabaseControl filesRedo logsData filesArchive logsSupported device types:LINUX: Block devices via O_DIRECT, ASMLib disks, UDEV bindings, MPIO aliases (RHEL6 & SLES11 only)AIX: raw devices, mknod devices

Similar to filesystems, ASM has much of the same requirementsArchive logs should be on a separate diskgroup from other database filesA Fast Recovery Area, if present (and marked for protection) must follow the same rules as the archive log destinations. It does not need to share the same diskgroup or underlying storage as archive logs, but it must not use the same storage as the control files, redo logs and data filesTemp files are not protected and can be placed anywherelun1lun2lun3lun4#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearSupported Configurations (RAC)AppSync currently supports RAC databases on ASM only at this timeIt is required that the Grid infrastructure files, such as the Oracle Cluster Registry (OCR) and voting files be kept on a diskgroup separate from the database if being stored on ASM. AppSync does not protect or modify this diskgroup.It is strongly recommended that for multiple RAC databases on the same Grid cluster that different diskgroups be used for each database for restore granularity and storage efficiency+ARCH2RACDatabase 1lun7lun8+DATA2lun5lun6Host AHost BRACDatabase 2+ARCH1lun3lun4+DATA1lun1lun2#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year+ASM2Archive logsSupported Configurations (RAC)RAC databases follow the same layout restrictions as standalone ASM databasesJust like with standalone databases, for each database it is required that archive logs, and the Fast Recovery Area be located on separate ASM diskgroups (FRA only required if being protected)In the image below, we illustrate that the archive log diskgroup must reside on different underlying storage than the data files, control files & redo logs.+ARCHRACDatabaseData, control & redo logslun3lun4+DATAlun1lun2+ASM1DB instance 1DB instance 2Host AHost B#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearSupported Configurations (HA/Failover)AppSync has generic support for Oracle HA/failover configurationsIt will leverage all possible nodes from which it is discovered as a potential source for protection with a bias towards the last node to successfully discover/protect the database. Not truly cluster integrated, but cluster-intelligent. RAC databases on clustered ASM diskgroupsFailover cluster on HACMP clustered logical volumes (Active/passive only)Failover cluster on VCS clustered logical volumes (Active/passive only, no SFRAC)Some important notes to keep in mind with respect to failover clustering software:AppSync will not instrument clustered resource groups or in any way administer the cluster. The extent of its behavior is to use cluster commands to obtain the cluster name & membership information.With the exception of RAC database copies, clustered copies can only be mounted as standalone. Mount to another cluster is perfectly legal, but AppSync will not do clustered import of the volume groupBe careful for restore considerations talked about later

#EMC CONFIDENTIALINTERNAL USE ONLYAnyone: Need proper array microcode/flare versioning, etc Any missing fields??

#TitleMonth Year

AppSync DeploymentNew in 2.0 the add host button in the servers panel is now a drop-downTo add/deploy UNIX servers, select Unix Servers from this drop-downProvide single-use credentials for the UNIX server. The initial connection will set up key-based authentication for future connectionsWhen you are finished adding hosts, hit start to begin deploymentThe installation path can be selected here. appsync will always be appended to an installation path if it is not specifiedProgress will be displayed here#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year

AppSync DeploymentLike Windows, you may deploy several UNIX hosts simultaneously

However, while you can deploy Windows and UNIX hosts simultaneously, you may not deploy UNIX and Windows hosts together using the same panelThe Add Unix Servers panel is for UNIX hosts ONLY!#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year

AppSync DeploymentA warning icon & message will be displayed over any host which had one or more databases which could not be discoveredFollowing deployment, AppSync will display all deployed host in the Servers paneThe agent plug-in version as well as any platform/cluster information about the host is displayed in the columns to the right of the host#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync Agent/Host-plugin DeploymentThe UNIX agent or Host-plugin, unlike its Windows cousin (and unlike RMs IRCCD/RM Client), is not service or daemon. As such, it does not need to be running all the time. It launches when needed to perform work and then shuts down.AppSync uses SSH for deployment/agent communication for UNIX platforms. As such, there are some specific requirements for using AppSync in a UNIX environment that must be adhered to.The AIX or Linux host being deployed must have a SSHd daemon and any firewall rule must permit a connection from AppSync server host to the AppSync agent host on port 22.In addition to SSH, the server also utilizes SFTP connections over SSH authenticated sessions, so the SFTP server must also be configured and runningThe SSHd must support both password-based and key-based authentication mechanisms./etc/ssh/sshd_config settings:PasswordAuthentication yesPubkeyAuthentication yesNote: AppSync only uses key-based authentication after the initial deployment, so if necessary for security, password-based authentication can be disabled following deployment, however, it may need to be re-enabled for hotfix installation & version upgrades.If using root for the AppSync user, the SSHd must permit remote root login./etc/ssh/sshd_config setting:PermitRootLogin yes

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync Deployment as non-root UserAppSync supports deployment/installation of a remote UNIX with a non-root account using what is called a sudo-based configurationLeverages the popular sudo tool to grant privileges to a non-privileged userRequires pre-setup on the machine from the root accountThere are some gotchas/limitations on where AppSync can be installedA non-root account must be created, it must support password-based and key-based authentication over SSH.sudo must be installed/configured, the /etc/sudoers file must be edited (using visudo) to include the following lines:Defaults: !requirettyDefaults: !env_reset ALL = (root) NOPASSWD: /acpThe installation path selected during deployment must match the settings in the sudoers file, otherwise deployment will fail:Note: the agent deployment will append appsync to the install-path if it is omitted, so if you select /opt/emc/appsync, your sudoers file should contain /opt/emc/appsync/acp, while if you select /opt/emc, your install path would be changed to /opt/emc/appsync and your sudoers file should still contain /opt/emc/appsync/acp At this time, using a sudo-based configuration will still result in the agent logs being owned by the root user

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearUnder the Servers panel in the settings menu each host will display a configuration panel including settings for logging. These settings are manageable remotely using this panel

The following settings can be configured:Log directory: Points to the location containing AppSync host plugin log filesMaximum directory size (in Megabytes): The size the log directory may grow to before old log files are expiredNumber of days to retain log files: The number of days which log files will be retained before they are expiredNote: The two settings above are not mutually exclusive, whichever limit is encountered first will cause logs to be rotated. If an age-based log rotation policy is desired, it is recommended to set the value of the directory size very high, while if a disk-quota based log rotation policy is desired, it is recommended to set the number of days to retain the logs very high. If both are desired, then they can both be used.Logging level: Normal / Debug (Normal is typically suitable, even for most diagnostic sessions

Changing AppSync Host-plugin Settings

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year./acp c Configures the values for these settings, you may specify multiple settings on the command line if necessary. Here we are changing the log retention period to 60 days and the size to 200MBIf necessary, it is possible to set the host-plugin settings manually via the command line (similar to RM). However, this would only really be necessary in practice if there was an issue with the product preventing these settings from being set from inside the GUI.

[root@myhost appsync]# ./acp -vEMC AppSync ACP v2.0.0.0------------------------------------------------------------settings file: /home/emcaps/.appsync/settings.xml------------------------------------------------------------logDirectory=/appsync/logslogLevel=3logRetentionPeriod=30logRetentionSize=100

[root@myhost appsync]# ./acp -c logRetentionPeriod=60,logRetentionSize=200EMC AppSync ACP v2.0.0.0------------------------------------------------------------settings file: /home/emcaps/.appsync/settings.xml------------------------------------------------------------logDirectory=/appsync/logslogLevel=3logRetentionPeriod=60logRetentionSize=200

Changing AppSync Host-plugin SettingsFrom the agent install directory:./acp v Displays version info and settingsBe careful if you change stuff here. You can break the agent if you send it bogus settings.#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearThe first step in managing Oracle copies is creating themCreation of Oracle database copies is achieved through the use of Service Plans. Service plans tiers are tied to the level of protection provided by the copy. Bronze service plans will perform a local copy to the same storage systemSilver will perform a remote copy to a different storage systemGold plans will perform both local and remote copies, offering maximum protection should either array suffer loss of serviceThe next slide will cover the process of subscribing an Oracle database to a pre-existing service plan and using it to protect the Oracle databaseOnce created, copies made through service plans can be expired, mounted & unmounted, recovered as part of the mount or restored back to production.This provides a good foundation for any number of business operations that might require the use of these copies

Oracle database copy management#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAs part of deployment & discovery, any databases available to AppSync for protection will be displayed in the Copy Management->Oracle paneFrom this pane, any of the discovered databases can be subscribed to service plans for protection by AppSync. Once subscribed, the service plan can be run on-demand, or at the scheduled time.The successful run of the service plan will produce copies of the subscribed databases which will then be available for use in other operations

Subscribing Oracle databases

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearThe create copy options on the service plan settings provides various controls which will influence how the Oracle copy is created. The settings (displayed to the right) are as follows:Place the database in hot backup mode: When enabled, the protection will put the database in hot backup and create copies of the archive logs immediately after. Disabling this means the database will not be put in Hot backup mode and the copy will be created from the live unquiesced data without any instrumentation of the database.Copy the Fast Recovery Area: When enabled, this field will instruct AppSync to create a copy of the underlying storage used by the FRA when protecting the databases archive log files.

Index and copy the BCT file: If enabled, AppSync will create an entry in the Oracle block change tracking file and copy the file over as part of the protection. This file can then be leveraged as part of a mount & backup use-case to provide accelerated incremental backup. This option requires hot backup mode.Create the backup control file for RMAN cataloging: Enabling this field will instruct AppSync to issue the creation of a binary backup control file for use with a request to catalog the database contents in a remote RMAN catalog. This option requires hot backup mode.

Protecting Oracle databases

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAs mentioned before, AppSync has generic cluster intelligence, meaning that it will not instrument clustered resources as part creating copies from clustered hosts. It will however, tolerate and handle the relocation of a database to another host (As is common with failover databases).Right now, for RAC databases the procedure is simple. When operating on RAC databases, any node in question can go down and AppSync will perform the next protection off a different node which is determined to be up and running. Due to this, it is highly recommended that multiple nodes be registered to AppSync

Protecting Oracle RAC & failover databasesRACDatabaseHost AHost B+ARCHlun3lun4+DATAlun1lun2If host B is up, it will be used, if host B goes down, AppSync will look for another node to useAppSyncProtection transferred to host A since host B was down during discovery#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearHost BFailover works similarly, with the required manual step that if the database has never been discovered on the former passive host, a manual discovery on the failover host must be initiated prior to running the service plan. Once it has been discovered, seamless protection from either active node will be possible.Protecting Oracle RAC & failover databasesFailoverDatabaseHost A+ARCHlun3lun4+DATAlun1lun2AppSyncSince this is active/passive appsync has no knowledge of host B since the DB was down at the time the host is discoveredHost BFailoverDatabaseHost A+ARCHlun3lun4+DATAlun1lun2AppSyncIf host A goes down, AppSync will have no node to protect from. Discovering host B lets AppSync detect the DB running on itHost B#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync Behavior (Hot-backup)OracleDatabaseControl filesRedo logsData filesArchive logsFast Recovery AreaAppSync splits up Oracle copies into two distinct point-in-time copiesThe first point-in-time copy is created for all the database files (control files, redo logs & datafiles). This copy is made while the database is in hot-backup mode.The second point-in-time copy is created for the archive log volumes and (optionally) for the Fast Recovery Area. This copy is made after Oracle has been taken out of hot-backup mode.The fact there are two copies here is managed internally, users only have to manage a single copy from within the product.Due to this behavior, it is required that at minimum, these files reside on different storage devicesCopy 1Copy 2Optional#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync Behavior (No hot-backup)OracleDatabaseControl filesRedo logsData filesArchive logsFast Recovery AreaFor crash-consistent copies without hot backup, AppSync omits the archive logs and only creates a single copy unless the optionally included FRA is also marked for protectionIn a crash-consistent copy, there is no requirement that the archive logs reside on separate storage, since they are not protected, but the FRA still must adhere to this requirement if it is marked for protection.

Copy 1Not protectedOptional#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVNX ConsiderationsOracleDatabaseControl filesRedo logsData filesArchive logsFast Recovery AreaFor crash-consistent copies without hot backup, AppSync omits the archive logs and only creates a single copy unless the optionally included FRA is also marked for protectionIn a crash-consistent copy, there is no requirement that the archive logs reside on separate storage, since they are not protected, but the FRA still must adhere to this requirement if it is marked for protection.

Copy 1Not protectedOptional#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync Mount & RecoveryMount & recovery of Oracle databases can be done in two ways from AppSyncScheduled within a service plan runOn-demand

Several Mount & recovery options exist:Mount on standalone server (RM-equivalent: No recover)Mount on standalone server and create RMAN catalog entry (RM-equivalent: Catalog with RMAN)Mount on standalone server & recover database (RM-equivalent: Recover)Mount on standalone server & prepare scripts for manual database recovery (RM-equivalent: Prepare-only/Generate scripts for manual recovery)Mount on Grid cluster & recover as RAC database (RM-equivalent: Mount as RAC database)

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStandalone mount (no-recover)This option performs a filesystem-only mountOptions on this panel are common to all other recover types (with the exception of RAC mount)No database recovery is performed & no database files are copied to the systemOracle need not be installed on mount hostFor ASM diskgroups, a valid Grid installation is required for ASM disk provisioning, but Oracle database is notFor ASM diskgroups, ASM disks are only provisioned by this operation and no ASM diskgroups are mounted or renamedMount on Server: This option allows you to specify which host to mount the copy on. Original host will mount to the production host

Mount to path: This option allows you to specify an alternate mount point for the filesystems. The underlying directory structure will not be altered and only the mount point will changeImage access mode: Allows you to specify the RecoverPoint bookmark access mode#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStandalone mount & create RMAN catalog entryThis option performs a RMAN catalog operation of Oracle database filesRMAN username: RMAN catalog userRMAN password: RMAN catalog user passwordRMAN connect string: RMAN catalog TNS aliasTNS_ADMIN: Path pointing to directory containing valid tnsnames.ora / sqlnet.ora configuration files, by default: $ORACLE_HOME/network/adminORACLE_HOME: Path pointing to the Oracle database home directoryASM diskgroup name: Used to specify that the diskgroup will be renamed. The variable %DG% represents the original diskgroup name and can be prefixed/postfixed by other charactersSkip datafiles: Datafile copies will not be cataloged with RMAN

#EMC CONFIDENTIALINTERNAL USE ONLYKingfisher: Please review for accuracy/content#TitleMonth YearStandalone mount & recover databaseThis option performs a mount & recovery of an Oracle database copy.Open-mode: This setting controls whether the recovered database is opened read-only or read/write.ORACLE_HOME: Path pointing to Oracle database home directoryDatabase name: This setting specifies the database name. If %DB% is used, the databases original name will be used. If T%DB% is used, the database name will be prefixed by T.Example: If the original SID name is TEST and SID name is set to T%SID%, the new SID name will be TTESTSID name: This setting specifies the SID name. If %SID% is used, the databases original SID will be used. If T%SID% is used, the SID will be prefixed by the letter T.Example: If the original SID name is TEST and SID name is set to T%SID%, the new SID name will be TTEST

ASM diskgroup name: Used to specify that the diskgroup will be renamed. The variable %DG% represents the original diskgroup name and can be prefixed/postfixed by other charactersCustom Initialization Parameters: Any init file settings the user may wish to override on the mounted database

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStandalone mount & prepare scripts for manual recoveryThis option performs a mount & creates scripts in /tmp/ that can be used to recover the database manuallyOpen-mode: This setting controls whether the recovered database is opened read-only or read/write.ORACLE_HOME: Path pointing to Oracle database home directoryDatabase name: This setting specifies the database name. If %DB% is used, the databases original name will be used. If T%DB% is used, the database name will be prefixed by T.Example: If the original SID name is TEST and SID name is set to T%SID%, the new SID name will be TTESTSID name: This setting specifies the SID name. If %SID% is used, the databases original SID will be used. If T%SID% is used, the SID will be prefixed by the letter T.Example: If the original SID name is TEST and SID name is set to T%SID%, the new SID name will be TTEST

ASM diskgroup name: Used to specify that the diskgroup will be renamed. The variable %DG% represents the original diskgroup name and can be prefixed/postfixed by other charactersCustom Initialization Parameters: Any init file settings the user may wish to override on the mounted database

#EMC CONFIDENTIALINTERNAL USE ONLYAnkur: Please review for accuracy/content#TitleMonth YearManual recovery with scripts exampleUsing the same settings as on the previous slide, here is an example of a manual script based recovery.After AppSync is done with the mount and recover of the copy the recovery scripts can be found on the mount host under the /tmp//RecoveryScripts directory. There will be a total of four script files found there, each with a step number and operation in the filename.

The user has to now run the script files in order of the step numbers to recover the database.There are two different types of scripts, an SQL script (*.sql) and an RMAN script (*.rman). Before running any of the scripts the mount host has to be configured with the correct ORACLE_HOME and ORACLE_SID settings. (NOTE: The ORACLE_SID in this case is the new SID after the rename operation T%DB%)To run a SQL script the user has to login into sqlplus on the command line, this can be done as follows from inside the ORACLE_HOME/bin directory with the following command, ./sqlplus / as sysdbaThe command to run an sql script is, @ /tmp//RecoveryScripts/Step-_Operation.sql Similarly to login into rman the user would enter rman target=/ and enter the following to run the script.@ /tmp//RecoveryScripts/Step-_Operation.rman

#EMC CONFIDENTIALINTERNAL USE ONLYMay need help with scripted recovery example from Ankur#TitleMonth YearManual recovery with scripts example 2Expected output after running the scripts in sequence,Step1:

Step2:

#EMC CONFIDENTIALINTERNAL USE ONLYMay need help with scripted recovery example from Ankur#TitleMonth YearStep3:

Step4:

#EMC CONFIDENTIALINTERNAL USE ONLYMount on Grid cluster & recover as RAC database

You can mount to an entire cluster, or just a subset of the clusterThe rest of these settings are the same as with standalone recovery options with one notable exception, SID name will contain an incrementing digit for each node in the cluster#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMount on Grid cluster & recover as RAC database (continued)Important points about Oracle RAC mount in AppSyncAppSync will only mount to cluster nodes which are registered to it (Unlike RM which only required the registration of 1 cluster node)You can only perform RAC mounts if the original database is a RAC databaseAppSync can perform RAC mount from N->N, N->N+ or N->N- cluster configurations. For example:2-node to 2-node (N->N)2-node to 4-node (N->N+)4-node to 2-node (N->N-)The selected SID name serves as a base name for the SID and does not include the instance number, additionally, the %SID% variable in the mount dialog represents the base SID of the production RAC database.Example: User protects database racdb which has instances RACDB1 & RACDB2, the base SID here is RACDB.If a SID name setting of %SID%X is used and they are mounting to a 3-node Grid cluster, the resulting instances will be:RACDBX1RACDBX2RACDBX3

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestoring Oracle copiesAppSync will perform an check for databases affected by the copy restore operationA database is considered affected if it shares storage on one or more of the following management layersDevices in the same array consistency group (RP & VNX)Virtual disks in the same datastore/VMFSDevices in the same volume group (for Linux/AIX native LVM and Veritas diskgroups)Disks in the same ASM diskgroupFiles in the same filesystemFiles in the same ASM diskgroupRestoring a copy which has affected databases is considered dangerous, be aware.If affected databases will also be restored to the point in time the copy of the original was at.Any application consistency that was applied to the original copy will not be available for the impacted databases (they are crash consistent after restore).AppSync will not shut them down or dismount any filesystems unique to these databases, so this must be done manually prior to restore#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestoring Oracle database copies in a RAC/clustered environmentFor failover clusters, resource groups should be manually frozen and any agents monitoring the database for shutdown/failure should be temporarily halted when performing a restore as this operation will shut down the database and dismount filesystems.For RAC clusters, databases on all nodes but the restore node should be shut down and the affected ASM diskgroups should be dismounted on each node.Restore users should understand the implications of performing restore in environments where multiple databases are impacted. AppSync will only issue warnings about affected databases that it can see on its current host and any databases unique to non-restore hosts will not result in a warning being issued. Please be aware that this does not mean they will not be impacted by restore if they share the same underlying storage as the restored database.For this reason it is highly recommended that databases be laid out on different ASM diskgroups so that when a restore is performed it can be done so without impacting other databases on the cluster.#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestore considerations & warningsRestore of a copy which was previously mounted with or without recovery may result in a database which is not recoverable or permanently altered from its original source.Understand that recovery, especially if database rename is used is a destructive operation which results in many database attributes being changed.In some cases, even just mount operations without recovery can irreversibly change some disk-level signatures that may cause impact during restore. Notable examples are Veritas and AIX/Linux native volume group UUIDs.A gold copy (if available) should always be the first choice for a restore operation as it will have been untouched by mount & recoveryThe product does offer a warning during restore that the copy was mounted though it will not prevent the restore from going through.#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTroubleshooting TipsSymptom: Host was successfully added to AppSync, but no Oracle databases are available for copy management. Probable cause: Databases or instances were down, or entries not present in /etc/oratab fileSolution: Restart the databases or add the relevant entries to the /etc/oratab file, then perform a host discovery through the product.Symptom: Protection fails because the Oracle database could not be quiescedCause: Database is not in archivelog modeSolution: Enable archiving on the database itself or disable Place database in hot backup mode option in service plan copy phase settingsSymptom: Protection fails because the Oracle database could not be unquiescedProbable cause: Archive log destination is full, invalid or has some kind of permissions issueSolution: Free up space in the archive log directory and retry the protection

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTroubleshooting TipsSymptom: Mount & recovery fails with error Could not detect valid Grid homeProbable cause: Oracle Grid infrastructure is not installedSolution: Install Oracle grid infrastructure and make sure that a default ASM instance is runningSymptom: Mount & recovery fails with error Failed to mount ASM diskgroupProbable cause: Oracle SP fileSolution: Install Oracle grid infrastructure and make sure that a default ASM instance is runningSymptom: Mount & recovery fails with error Database needs more recovery to be consistentProbable cause: Using VNX advanced snaps without consistency groupsSolution: At minimum, the Oracle archive log disks must be contained within a VNX consistency group

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearQuestions & Answers

#EMC CONFIDENTIALINTERNAL USE ONLYTHANK YOU#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync 2.0Technical PresentationVMAX Support with AppSync2.0#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaVMAX Array discovery and configurationStorage mapping and protectionMount of VMAX copiesRestoreLun restoreVM restore

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

AppSync 2.0 uses EMC SMI-S Provider version 4.2.6.x toQuery all the required information( mapping, device information, SG, Pools etc.)perform the active management tasks such as snapshot and clone session creation, termination, restoration, LUN masking etc.SMI-S Provider should beinstalled on a separate host(not same as AppSync server)SMI-S provider can run on windows as well as Linux Hosts and AppSync supports both

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Array discoveryVMAX array is discovered using any of the SMI-S Provider which manages that VMAX arraySMI-S Provider host should have access to a minm. Of 6 GK devices from arrayUser can perform VMAX discovery by navigating to the setting page in AppSync UI as shown in next pic.As part of array discovery, AppSync establish connection with the SMI-S provider and gets a list of all the arrays discovered by SMI-S Provider along with detailed information about the arrays e.g. microcode information, VMAX Model number. It also saves the time when this discovery was performed.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Array discoverySMI-S provider credentials can be reset by AppSync settings page as described in pic. next

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Array discoverySince VMAX arrays can be managed by multiple SMI-S Providers, for every VMAX array discovered by AppSync using any SMI-S Provider, it maintains an association between SMI-S Providers and VMAX System as shown in the picture next.For VMAX arrays discovered by multiple providers, a preferred providers needs to be set which AppSync uses to manage that array.By default AppSync picks the first SMI-S provider which discovers a VMAX arrays(which has locally attached devices from VMAX) as the preferred provider. This can be changed by AppSync UI(Settings page) in case of multiple providers.For arrays who doesnt have a preferred SMI-S Provider selected, AppSync uses any of the provider to perform queries and active management tasks on VMAX. Sometime, this may not be the best provider for the Array and can result in failures, so it is highly recommended that for Arrays who preferred provider is not by default set by AppSync (normally for all REMOTE arrays), end user should select one.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Array discoveryPreventing the Automatic Discovery of EMC ArraysEMC Arrays (VNX,VMAX etc. ) are automatically discovered when they are connected through HBA to the servers where EMC SMI-S provider is installed. Performance of SMI-S provider is very critical to AppSync performance. Sometimes, we may need to disable discovery of other Arrays on SMI-S provider host which is not used by AppSync as it impacts SMI-S provider performance

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Array discoveryProcedure to prevent Automatic Discovery of EMC ArraysPlease dont perform this procedure if SMI-S provider setup is shared by other EMC applications e.g. ECC,SRM etc as they may need all the array information.Either stop the ECOM service on windows host or execute the following command to stop the EMC SMI-S provider service:\ECIM\ECOM\bin\sm_service stop ecom.exeExecute the command SYMCLI\binsymcfg list to list the Array IDs which we want to disableIn a file, list the IDs of the arrays that should not be auto-discoveredSave this file as symavoid in the SYMAPI/config directoryDelete the hosts SYMAPI database (the symapi_db_bin file) for the changes to take effect. The database will be recreated the next time the SMI-S provider tries to discover arrays. Start ECOM service or rxecute the following command to start the EMC SMI-S provider service:\ECIM\ECOM\bin\sm_service start ecom.exePerform a discovery for the changes to take effect.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationOnce an Array is discovered using SMI-S Provider, to create Successful application copies using VP SNAP or TF clone copy technology, AppSync requires copy devices. AppSync customer has two choices which he can use to configure copy device selection for AppSync. Customers can use either of them or can configure both.

Using dedicated StorageGroup (one or more) for copy devices onlyEnabling a virtual provisioning pool for AppSync use

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationUsing dedicated Storage Group (one or more) for copy devices only

Create one or more Storage Group with no host connectivity(not attached to any LUN masking view)Provision new target devices for application copies matching source device size and type and add this to Storage Group createdFor VP-SNAP, make sure target device is also thin provisioned. If source already has VP-SNAP from a thin pool, make sure target copy device is also provisioned in same poolFor META devices make sure target is also meta matching META type e.g. striped or concatenatedIn case application already has existing clone/VP-SNAP session created, put the target devices of those session in the one of the storage group created for AppSync purpose. AppSync can read existing session information and reuses them.All devices in the configured storage group(s) are used as copy devices, so never add any production/Source device into these Storage Groups as it will result in data loss erasing all the production data.Dont share same storageGroup(s) across multiple AppSync server

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationUsing dedicated Storage Group (one or more) for copy devices only

As shown in picture next, AppSync discovers all the storage group on selected VMAX system which doesnt have any host connectivity i.e. not attached to any LUN masking view and list all of them in the groups and Pools section of Add VMAX Storage wizard.

To configure a Storage Group for AppSync target device selection, user need to enable it by clicking on the check box next to the storage group. AppSync allow selecting multiple storage groups.

Once end user selects Storage Group(s)whose devices he intends to use as replication/copy storage devices, upon pressing finish he gets to see the warning screen below which warn end user to review his selection once more time as all the devices in the selected storage groups will be used as vp-snap or TF clone target copy devices and original data on those devices gets lost

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationUsing dedicated Storage Group (one or more) for copy devices only

At any point of time if end user needs to know what storage groups were selected for replication purpose and how many devices overall were enabled for replication(vp-snap and clone) target, he can select his vmax system in the settings tab and the pane below displays all those details(as shown in picture below

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationUsing dedicated Storage Group (one or more) for copy devices onlyUser can select also the Manage copy storage button to enable/disable storage groups for any discovered vmax array(as shown in picture next)At any time user wants to disable any enabled storage group so that AppSync no longer uses devices from those storage groups for replication purpose, he can use the same wizard and just deselect the check box. If any device is used as a valid appsync copies, AppSync performs the necessary checks and throw relevant error message and doesnt let user disable that storage group(s)AppSync doesnt terminate differential clone session when storge group(s) are disabled.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationEnabling a virtual provisioning poolCreate or use one of the existing VMAX virtual provisioning pool for AppSync use, where AppSync can provision target devices which it needs.If application already has pre-existing VP SNAP session in any of the existing VMAX pool and AppSync needs to create new VP SNAP session, Please make sure that existing VMAX pool (where VP SNAP exists) is configured for AppSync use to provision new target devicesIf Application resides on a VMAX META volume, target META volume provisioning requires a POOL matching the RAID type of source META volume. AppSync requires such pool to be configured

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationEnabling a virtual provisioning poolAs shown in picture next, AppSync discovers all the THIN Pools configured on the VMAX array and display it in the wizard. User can configure one or more pools for AppSync use.User is also allowed to select a threshold percentage value for each of the selected pool. This is used by AppSync while selecting a storagepool for target provisioning. If any pool used percentage is above this threshold limit, it is not used by AppSync for any target provisioningUser can enable/disable a pool any time using the above wizard. All the target devices provisioned by AppSync arenamed AppSync- String appended. This can be used to identify that a device is provisioned by AppSync (as shown in picture next from univmax).AppSync doesnt delete devices when pool is disabled.When a pool is disabled, user can select to choose to terminate clone session created by AppSync for the newly provisioned device.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationRemoving a VMAX array from AppSyncAs shown in picture next, AppSync allows removing a discovered VMAX arrays.If a VMAX array is selected for removal but it has valid copies associated with it or has storage group and pool configured, AppSync detects it and throws an appropriate error and fails removal of vmax arrays.If a VMAX array is removed from SMI-S provider and AppSync admin performs a storage discovery again using that SMI-S provider, AppSync removed the association between vmax array and SMI-S provider and if VMAX arrays is eligible for removal(refer above point), it removes VMAX array from its persistence.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

Storage ConfigurationRemoving an SMI-S ProviderAs shown in picture next, AppSync allows removing a SMI-S providerIf any VMAX array uses the SMI-S provider as preferred array or any Array is only managed by this SMI-S provider and has valid copies or storage configuration, AppSync detects it and fails the removal of SMI-S provider with an appropriate error message.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearVMAX Array discovery and configuration

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionStorage Mapping in AppSyncDuring storage mapping phase of a serviceplan execution cycle, AppSync needs to maps the source LUNs on which application resides to the underlying storage system devicesAppSync uses source LUN wwn to do a quick lookup in each of the configured SMI-S provider to map the wwn to corresponding VMAX array and get all the LUN detials e.g. thin/thick,RAID type etc.If any configured SMI-S provider is down, it may introduce a delay in mapping phase as it takes some time for AppSync to detect if an SMI-S provider is down and it shouldnt be considered for mapping phase.For Silver service plan, we create a LUN for the R2 device after getting the R1 and R2 details from the VMAX array.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection Affinitization of applicationsBefore creating VMAX copies appsync affintizes applications based on certain rules which enables it to group application together.Vmax affintization has rules as listed belowAffinitize by vmax : Like VNX storage system, VMAX affinitization rules also break applications on basis of the VMAX id(serialNumber). If two datastore/databases( on same host) are part of two diff VMAXes, they will split in two diff phase pitsAffintize by device type: e.g. thin(pool device) vs thick(Normal RAID devices): As we know we cant do vp-snap for non-pool devices. In case we have multiple databases on same symm spread across thin and thick luns and user pref. is snapshot first, We need to split application based on the type, so that for thick devices we can continue with clone option. Affinitize by RA Group (only for silver plans): All the devices belonging to the same RA group is protected together. Affinitizer does the split based on RA group when a silver service plan is run.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtectionAppSync 2.0 supports VP SNAP and TF Clone as the supported copy technologies for VMAX arraysVP SNAPS are supported for VMAX 10k,20k and 40K arrays only running microcode 5876 and above.VDEV snaps are not supportedRemote copies i.e. R2 VP SNAP and TF Clones are supported for SRDF/S session. ApPSync doesnt create/manage SRDF/S sessionCrash consistent Remote copies i.e. R2 VP SNAP and TF Clones are supported for SRDF/A session for certain application. Please check AppSync documentaion for details.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionAny SRDF mode other than synchronous or asynchronous is not supported.For SRDF/A the link state should be in Consistent or Synchronized state for replications to work. For SRDF/S, link state should be Synchronized.For creating copy at the R2 site, Silver service plan should be used. AppSync 2.0 does not support Gold service plans for SRDF.Application consistency is not yet available for SRDF/A copies. We will not be providing support for Microsoft Applications (SQL, Exchange).If you need to create copies of R2 devices if the link asynchronous, the following should be enabled.Device level write pacing should be enabled and supported on the SRDF/AAll the devices which constitute a single application should belong to the same RA Group(RDF group)Note: Please refer to the SRDF/TimeFinder product guide for detailed steps on configuring device pacing.

#EMC CONFIDENTIALINTERNAL USE ONLYStorage mapping and protectionProtection Target device selectionBefore creating VMAX copies appsync needs to pair each of the source devices on which application data resides to a corresponding target device. Appsync follows a set of rules for pairing source and target devices for VP SNAP and TF clone copies.Next slides have few of the rules captured while pairing source and target devices. The rules for VP Snap and TF clones are slightly different.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection Target device selectionIf the source device has an pre-existing session with any of target deviceconfigured for appsync use and it not already used for any other copy, it gets the highest priorityIn case rule 1 doesnt yield in a target device. AppSync queries SMI-S provider to find information about the pool in use for vp-snap copies.It makes use of this pool information to get a suitable ThinPool out of the list of all the enabled Thin pool for AppSync use. AppSync store this suitablePool information in memory so that in case it doesnt find any suitable target device from appsync persistence, Appsync can provision target for the sourceLun in this poolAppSync queries its database and find out all thin devices matching size and not in any vp snap of clone session. These devcies gets the next highest priority.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection Target device selectionIf AppSync doesnt find any target till 3 above and no valid pool is configured, it considers any target device which had a pre-existing session but is not being used for any copy and it terminates its earlier sesison and reuses

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection VP SNAPBy default bronze servicePlan has snapshot as the most preferred storage order(refer picture next)If application data resides on a VMAX array and the source storage devices are THIN luns, AppSync bronze plan will create VP-Snap copies as part of the serviceplan runIf application data resides on a mix of Thin and Thick(RAID) Luns, AppSync creates clone copy of the application as it cant create VP SNAP for Thick devices.AppSync allows to specify copy rotation value in serviceplan settings. End user needs ot make sure to configure appsync with atleast n+1 target devices for each source where n is the rotation value. This is required as AppSync expires a copy only after making sure that next copy is created first.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearProtection

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection VP SNAP workflowWhile creating VP SNAP copies typical workflow involves :1.mapping the source storage devices to VMAX devices and identifying the VMAX system,2.pairing the source devices with suitable target devices i.e. SPA. If required AppSync provisions the target LUN in the StoragePool configured for AppSync use.3.creating an unactivated snapshot between the source and target4.freezing the application5.activating the snapshot6.thawing the application7.cataloging the snapshot information#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year

Protection#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection TF CloneBy default bronze service plan has snapshot as the most preferred storage order, If end user wants to create TF Clone for all of there applications residing on VMAX array, they need to change the storage order pref. and push clone to the top of storage pref. Order.If application data resides on a VMAX array and the source storage devices are THICK luns, AppSync bronze plan will always create TF Clone copy even thou snapshot is higher in storage order pref.AppSync allows to specify copy rotation value in service plan settings. End user needs ot make sure to configure AppSync with at least n+1 target devices for each source where n is the rotation value. This is required as AppSync expires a copy only after making sure that next copy is created first.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year

Protection#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearStorage mapping and protectionProtection Clone workflowWhile creating TF Clone copies typical workflow involves :1.mapping the source storage devices to VMAX devices and identifying the VMAX system,2.pairing the source devices with suitable target devices i.e. SPA. If required AppSync provisions the target LUN in the StoragePool configured for AppSync use.3.creating an unactivated and differential TF clone session between the source and target4.freezing the application5.activating the snapshot6.thawing the application7.Waiting for the clone session to complete i.e. 100% synchronization8.cataloging the snapshot information

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year

Protection#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMount of VMAX copiesAppSync 2.0 supports dynamic mounting of VMAX copies (both vp-snap and TF clone) for all the application copies on all AppSync supported platform e.g. vSphere, Linux, AIX and Windows. AppSync doesn't support static mounting of VMAX copies except (RP protection for VMAX devices).AppSync relies on the VMAX Auto-Provisioning capability. AppSync requires the mount host to be zoned to the VMAX array. You should create a masking view with the appropriate initiator group, port group and storage group.To perform dynamic mount of VMAX copies, AppSync first find the mount host FC/iSCSI adapter information and then it interacts with VMAX via SMI-S provider to find an appropriate storage group for the mount host.Once AppSync find the appropriate storage group it adds the target copy devices to that storage group as shown in the attached picture. #EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMount of VMAX Copies

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMount of VMAX copiesRules used by AppSync to determine mount host Storage GroupFor virtual machines, AppSync performs mount by masking the copy devices to the ESX host and then hot-Adding the devices to the virtual machine as RDM/virtual disk (depending on application copy)In case host is connected to VMAX via multiple masking view and host is not an ESX host in cluster(for Data store or RDM mounts), AppSync first gives preference to a masking view which is dedicated for that host i.e. initiator group connected to the masking view has only initiator(s) for that host. In this scenario AppSync uses the storage group connected to this dedicated masking view to add all the copy devices during mount.If AppSync doesn't find a dedicated storage group, it creates a list of storage group with all storage group attached to each of the masking devices which is connected to the host and picks up the first storage group. If the selected storage group is dedicated for GK devices(i.e. has only GK devcies0, AppSync picks up the next storage group in sorted list. In case no other storage group exist except the one with GK devices only, AppSync fails back and select only this storage group.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMount of VMAX copiesRules used by AppSync to determine mount host Storage GroupIn case host is an ESX host in cluster and RDM/ Datastore mount is performed, AppSync tries to find a masking view which has connectivity to the maximum number of nodes of the ESX cluster . If it finds such a common masking view, it uses storage group connected to this masking view to add all the copy devices during mount. In case any ESX nodes in cluster is not connected to this common masking view, it tries to find masking view for that node as outlined in step 2 and 3 above. In case it doesn't find any storage group for any node of the cluster, it fails the mount operation with appropriate exception. In case the selected masking view is connected to a cascaded storage group i.e. storage group containing a list of other storage group, AppSync sorts the storage group by its name and picks up the first storage group with a check that storage group is not dedicated for GK devices only. If the selected storage group is dedicated for GK devices(i.e. ha sonly GK devices, AppSync picks up the next storage group in sorted list.In case of SQL cluster mount, if the AppSync administrator has selected for use dedicatedStorageGroup and if AppSync is unable to find any dedicated storage group for the mount host, it throws exception and fails the mount.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearunMount of VMAX copiesDuring unmount, AppSync again find the storage group using the same rules as described above and removes the devices from the storage group as shown in picture below.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearunMount of VMAX Copies

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestore of VMAX copiesLUN Level restore of VP-Snap and TF Clone copies AppSync support restore of application copies created using bronze plan(i.e. local copies only). LUN restore from remote copies i.e. silver plan is not supported in AppSync 2.0. Some of the important point to note while AppSync performs LUN level restore:Since AppSync supports only Snap-VP(and not VDEV snap) snapshots and while restoring from a snap-vp snapshot, VMAX consumes one TF clone session to perform restore, it is very important to make sure that if AppSync is configured to take TF clone copies of the same application, it shouldn't be configured to take more than 6 copies.In case user has configured servicePlans to take both snap-vp and tf clone copies of the same application, AppSync supports restoring from both the type of copies.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestore of VMAX copiesLUN Level restore of VP-Snap and TF Clone copies At the end of restore from clone copies, AppSync put the TF clone session in split state so that if user wants to perform restore from same copy again, it should be allowed as well as AppSync can perform recreate on the same copy(in case it is expired from AppSync as part of rotation) and can do differential copy during next servciePlan run.At the end of restore from snap-vp copies, VMAX create one additional session in "RESTORED" state for the same source-target pair. AppSync terminates this RESTORED session at the end of restore, so that the same copy can be sued again for restore as well as restore from other copy is allowed. if session is left in RESTORED state, it wont allow restore from other copies.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestore of VMAX copiesVM Level restore from VP-Snap and TF Clone copies VM restore from VP SNAP CopiesInstant restore option is not available for restore from VP Snap copies as we can't sue storage migration technique which we use for VNX and RP copies here as VMAX doesn't allow snap of vp snap copy. Due to this restriction, appsync mount vp snap datastores copy and register vm from the copy and then clone the vm from snapshot datastore to the restore location.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRestore of VMAX copiesVM Level restore from VP-Snap and TF Clone copies VM restore from TF Clone CopiesDepending on the configuration of source and clone devices involved in the TF Clone copy operation. AppSync can perform instant restore option in certain cases. If all the source devices and clone devices of the datastores are THIN, we can perform vp-snap of clone devices and use the vp snap of clone to mount datastore on the fly and perform storage vmotion and thsi enabled instant restore of vm.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTHANK YOU#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync 2.0Technical PresentationF:1923 RecoverPoint on VMAX#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaProtectionMountRestoreLicense

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearProtectionAll the use cases supported on RP-VNX is supported for RP-VMAX from AppSync 2.0 onwards.#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearMountOnly Logged access is supported due to restriction from symmetrix splitter#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearRP Dynamic MountDynamic mount of RP Targets is supported if the underlying storage is VMAX.AppSync uses VMAX auto provisioning capabilities to add LUNs dynamically to a mount host.Register the SMI-S Provider managing the VMAX array.Please refer to the VMAX Dynamic Mount instructions for details on the setup.#EMC CONFIDENTIALINTERNAL USE ONLYRestoreAll use cases similar to VNX is supported#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearLicenseNo License needed.#EMC CONFIDENTIALINTERNAL USE ONLYTroubleshooting TipsIf mount takes more than 30 minutes, then please increase the value of server setting variable max.timeout.enable.image. The value is accepted in seconds.

#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTHANK YOU#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync 2.0Technical PresentationF1940 AppSync and RM or NMM co-existence

Dee HollyJune 6, 2014#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaIntroductionAppSync VSS Hardware ProviderAppSync Exchange Interface serviceUpgradeCoexistenceTroubleshooting#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearIntroductionBefore AppSync 2.0, AppSync, RM, and NMM could not be installed on the same serverStarting with 2.0, AppSync can be installed on a server with RM or NMM or NetWorker ClientRM and NMM still cannot coexist.AppSync and NMM shared RMs code baseThe COM+ interfaces were the same for the hardware provider and Exchange interface serviceThese interfaces were changed in 2.0AppSync Exchange Interface service has to be reconfigured during upgrade.#EMC CONFIDENTIALINTERNAL USE ONLYHardware ProvidersAppSync 1.5 and 1.6Provider name: 'AW VSS Provider' Provider type: Hardware Provider Id: {e929a027-cf8c-47bf-90a3-cd4241c7cace} Version: 1.0Replication ManagerProvider name: 'ERM VSS Provider' Provider type: Hardware Provider Id: {e929a027-cf8c-47bf-90a3-cd4241c7cace} Version: 1.0AppSync 2.0Provider name: 'AppSync VSS Provider' Provider type: Hardware Provider Id: {f2a06f22-aa08-481c-a9ca-8c01761d4fa0} Version: 1.0#EMC CONFIDENTIALINTERNAL USE ONLYThe Provider name has changed from AW VSS Provider to AppSync VSS ProviderThe Provider Id has changed from e929a027-cf8c-47bf-90a3-cd4241c7cace to f2a06f22-aa08-481c-a9ca-8c01761d4fa0#TitleMonth YearRM and AppSync 1.x Exchange Interface service

#EMC CONFIDENTIALINTERNAL USE ONLYThe Application ID for the EMC AppSync Exchange Interface component has changed from 2CD8E1F9-FA09-4102-B36F-140E770894C5 #TitleMonth YearAppSync 2.0Exchange Interface service

#EMC CONFIDENTIALINTERNAL USE ONLYto BF521C03-1C2D-4C2F-9CBC-DAFBF47BE095#TitleMonth YearUpgradeAppSync Exchange Interface service has to be reconfigured during upgradeManual upgrade will ask for user credentials to reconfigure the AppSync Exchange Interface servicePush install upgrade will also ask for credentials.One set of credentials for all of the servers currently be upgraded#EMC CONFIDENTIALINTERNAL USE ONLYPush Install Upgrade ProcessSelect server to upgradeProvide local admin credentials for installProved Exchange credentials for reconfiguring the Exchange Interface serviceAppSync install:Unregisters the Exchange Interface serviceInstalls the 2.0 plug-in softwareRestarts the AppSync Host Plug-in to make sure the hardware provider registers properlyAppSync reconfigures the Exchange Interface service and reports success or failure

#EMC CONFIDENTIALINTERNAL USE ONLYUpgrade Example

#EMC CONFIDENTIALINTERNAL USE ONLYFirst AppSync asks for the credentials needed to run install

#TitleMonth YearUpgrade Example cont.

#EMC CONFIDENTIALINTERNAL USE ONLYThen, if an Exchange Mailbox server or Exchange utility server is in the list, AppSync asks for the credentials needed to reconfigure and run the EMC AppSync Exchange Interface service.#TitleMonth YearUpgrade Example cont.

#EMC CONFIDENTIALINTERNAL USE ONLYThere is a progress message when the service is being reconfiguredIf the reconfiguration is successful, AppSync displays a status that shows success for update and reconfigureIf the reconfiguration is not successful, AppSync displays a status that shows failed reconfigure and successful update#TitleMonth YearCoexistence in AppSync 2.0NetWorker Client 8.0 or 8.1 and NMM 2.4 or 3.0AppSync creates VSS hardware shadow copies and NetWorker creates VSS software shadow copiesSupported on same Windows server:AppSync Server and RM Server AppSync Server and NetWorker ServerAppSync Host Plug-in and RM ClientAppSync Host Plug-in and NMM or NetWorker FS Client#EMC CONFIDENTIALINTERNAL USE ONLYCoexistence in AppSync 2.0 cont.Recommendations:Use NetWorker to create Full and Incremental Exchange backups and AppSync to create CopyUse NetWorker to create Full, Differential, and Log SQL Server backups and AppSync to create CopyUse RM or AppSync to do log truncation, not bothSchedule backups so they do not run at same timeDo not run an AppSync mount at the same time as a NetWorker or RM mount

#EMC CONFIDENTIALINTERNAL USE ONLYCoexistence in AppSync 2.0 cont.EMC APPSYNC and NetWorker Coexistence whitepaper will be published#EMC CONFIDENTIALINTERNAL USE ONLYTroubleshooting TipsIf, after upgrade, customers are getting strange errors about the provider, use vssadmin list providers to verify that the provider is registered with the new name and IdIf customers are not able to discover databases, verify that the EMC AppSync Exchange Interface DCOM component is registered with the new AppId and the credentials are correct.Also make sure that the EMC AppSync Exchange Interface service is running#EMC CONFIDENTIALINTERNAL USE ONLYTroubleshooting Tips cont.Sometimes the service gets in a bad state and you have to do a manual cleanupOpen a cmd prompt and change to the AppSync host plug-in install directoryRun: awExchangeInterface.exe /unregisterCheck services.msc for EMC AppSync Exchange InterfaceIf the service is still there, run: sc delete appsyncexchangeinterfaceCheck dcomcnfg: Component Services->Computers->MyComputer->DCOM Config for EMC AppSync Exchange InterfaceIf it is there, select it in the middle box and press the Delete key#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTroubleshooting Tips cont.When upgrading it is best to cleanup the registry to get rid of the old interface. Put this is a .reg file or delete manually:

Windows Registry Editor Version 5.00

[-HKEY_CLASSES_ROOT\AppID\{2CD8E1F9-FA09-4102-B36F-140E770894C5}][-HKEY_CLASSES_ROOT\AppID\awExchangeInterface.EXE][-HKEY_CLASSES_ROOT\CLSID\{0E7FA443-52E1-4888-8597-67D5E99802B0}][-HKEY_CLASSES_ROOT\CLSID\{368AC9A1-47FD-4EA6-B798-D6569408E210}][-HKEY_CLASSES_ROOT\CLSID\{53594447-15F1-4EAF-A75D-8CB65E769709}][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_2010_Component][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_2010_Component.1][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Common][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Common.1][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Component][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Component.1][-HKEY_CLASSES_ROOT\TypeLib\{F0695E19-89CB-45CB-B85A-480E65936926}][-HKEY_CLASSES_ROOT\Wow6432Node\AppID\{2CD8E1F9-FA09-4102-B36F-140E770894C5}][-HKEY_CLASSES_ROOT\Wow6432Node\AppID\awExchangeInterface.EXE][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{0E7FA443-52E1-4888-8597-67D5E99802B0}][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{368AC9A1-47FD-4EA6-B798-D6569408E210}][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{53594447-15F1-4EAF-A75D-8CB65E769709}][-HKEY_CLASSES_ROOT\Wow6432Node\TypeLib\{F0695E19-89CB-45CB-B85A-480E65936926}]#EMC CONFIDENTIALINTERNAL USE ONLYTHANK YOU#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAppSync 2.0Technical PresentationF1940 AppSync and RM or NMM co-existence

Dee HollyJune 6, 2014#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearAgendaIntroductionAppSync VSS Hardware ProviderAppSync Exchange Interface serviceUpgradeCoexistenceTroubleshooting#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearIntroductionBefore AppSync 2.0, AppSync, RM, and NMM could not be installed on the same serverStarting with 2.0, AppSync can be installed on a server with RM or NMM or NetWorker ClientRM and NMM still cannot coexist.AppSync and NMM shared RMs code baseThe COM+ interfaces were the same for the hardware provider and Exchange interface serviceThese interfaces were changed in 2.0AppSync Exchange Interface service has to be reconfigured during upgrade.#EMC CONFIDENTIALINTERNAL USE ONLYHardware ProvidersAppSync 1.5 and 1.6Provider name: 'AW VSS Provider' Provider type: Hardware Provider Id: {e929a027-cf8c-47bf-90a3-cd4241c7cace} Version: 1.0Replication ManagerProvider name: 'ERM VSS Provider' Provider type: Hardware Provider Id: {e929a027-cf8c-47bf-90a3-cd4241c7cace} Version: 1.0AppSync 2.0Provider name: 'AppSync VSS Provider' Provider type: Hardware Provider Id: {f2a06f22-aa08-481c-a9ca-8c01761d4fa0} Version: 1.0#EMC CONFIDENTIALINTERNAL USE ONLYThe Provider name has changed from AW VSS Provider to AppSync VSS ProviderThe Provider Id has changed from e929a027-cf8c-47bf-90a3-cd4241c7cace to f2a06f22-aa08-481c-a9ca-8c01761d4fa0#TitleMonth YearRM and AppSync 1.x Exchange Interface service

#EMC CONFIDENTIALINTERNAL USE ONLYThe Application ID for the EMC AppSync Exchange Interface component has changed from 2CD8E1F9-FA09-4102-B36F-140E770894C5 #TitleMonth YearAppSync 2.0Exchange Interface service

#EMC CONFIDENTIALINTERNAL USE ONLYto BF521C03-1C2D-4C2F-9CBC-DAFBF47BE095#TitleMonth YearUpgradeAppSync Exchange Interface service has to be reconfigured during upgradeManual upgrade will ask for user credentials to reconfigure the AppSync Exchange Interface servicePush install upgrade will also ask for credentials.One set of credentials for all of the servers currently be upgraded#EMC CONFIDENTIALINTERNAL USE ONLYPush Install Upgrade ProcessSelect server to upgradeProvide local admin credentials for installProved Exchange credentials for reconfiguring the Exchange Interface serviceAppSync install:Unregisters the Exchange Interface serviceInstalls the 2.0 plug-in softwareRestarts the AppSync Host Plug-in to make sure the hardware provider registers properlyAppSync reconfigures the Exchange Interface service and reports success or failure

#EMC CONFIDENTIALINTERNAL USE ONLYUpgrade Example

#EMC CONFIDENTIALINTERNAL USE ONLYFirst AppSync asks for the credentials needed to run install

#TitleMonth YearUpgrade Example cont.

#EMC CONFIDENTIALINTERNAL USE ONLYThen, if an Exchange Mailbox server or Exchange utility server is in the list, AppSync asks for the credentials needed to reconfigure and run the EMC AppSync Exchange Interface service.#TitleMonth YearUpgrade Example cont.

#EMC CONFIDENTIALINTERNAL USE ONLYThere is a progress message when the service is being reconfiguredIf the reconfiguration is successful, AppSync displays a status that shows success for update and reconfigureIf the reconfiguration is not successful, AppSync displays a status that shows failed reconfigure and successful update#TitleMonth YearCoexistence in AppSync 2.0NetWorker Client 8.0 or 8.1 and NMM 2.4 or 3.0AppSync creates VSS hardware shadow copies and NetWorker creates VSS software shadow copiesSupported on same Windows server:AppSync Server and RM Server AppSync Server and NetWorker ServerAppSync Host Plug-in and RM ClientAppSync Host Plug-in and NMM or NetWorker FS Client#EMC CONFIDENTIALINTERNAL USE ONLYCoexistence in AppSync 2.0 cont.Recommendations:Use NetWorker to create Full and Incremental Exchange backups and AppSync to create CopyUse NetWorker to create Full, Differential, and Log SQL Server backups and AppSync to create CopyUse RM or AppSync to do log truncation, not bothSchedule backups so they do not run at same timeDo not run an AppSync mount at the same time as a NetWorker or RM mount

#EMC CONFIDENTIALINTERNAL USE ONLYCoexistence in AppSync 2.0 cont.EMC APPSYNC and NetWorker Coexistence whitepaper will be published#EMC CONFIDENTIALINTERNAL USE ONLYTroubleshooting TipsIf, after upgrade, customers are getting strange errors about the provider, use vssadmin list providers to verify that the provider is registered with the new name and IdIf customers are not able to discover databases, verify that the EMC AppSync Exchange Interface DCOM component is registered with the new AppId and the credentials are correct.Also make sure that the EMC AppSync Exchange Interface service is running#EMC CONFIDENTIALINTERNAL USE ONLYTroubleshooting Tips cont.Sometimes the service gets in a bad state and you have to do a manual cleanupOpen a cmd prompt and change to the AppSync host plug-in install directoryRun: awExchangeInterface.exe /unregisterCheck services.msc for EMC AppSync Exchange InterfaceIf the service is still there, run: sc delete appsyncexchangeinterfaceCheck dcomcnfg: Component Services->Computers->MyComputer->DCOM Config for EMC AppSync Exchange InterfaceIf it is there, select it in the middle box and press the Delete key#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth YearTroubleshooting Tips cont.When upgrading it is best to cleanup the registry to get rid of the old interface. Put this is a .reg file or delete manually:

Windows Registry Editor Version 5.00

[-HKEY_CLASSES_ROOT\AppID\{2CD8E1F9-FA09-4102-B36F-140E770894C5}][-HKEY_CLASSES_ROOT\AppID\awExchangeInterface.EXE][-HKEY_CLASSES_ROOT\CLSID\{0E7FA443-52E1-4888-8597-67D5E99802B0}][-HKEY_CLASSES_ROOT\CLSID\{368AC9A1-47FD-4EA6-B798-D6569408E210}][-HKEY_CLASSES_ROOT\CLSID\{53594447-15F1-4EAF-A75D-8CB65E769709}][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_2010_Component][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_2010_Component.1][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Common][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Common.1][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Component][-HKEY_CLASSES_ROOT\RM_ExchangeInterface.RM_ESM_Component.1][-HKEY_CLASSES_ROOT\TypeLib\{F0695E19-89CB-45CB-B85A-480E65936926}][-HKEY_CLASSES_ROOT\Wow6432Node\AppID\{2CD8E1F9-FA09-4102-B36F-140E770894C5}][-HKEY_CLASSES_ROOT\Wow6432Node\AppID\awExchangeInterface.EXE][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{0E7FA443-52E1-4888-8597-67D5E99802B0}][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{368AC9A1-47FD-4EA6-B798-D6569408E210}][-HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{53594447-15F1-4EAF-A75D-8CB65E769709}][-HKEY_CLASSES_ROOT\Wow6432Node\TypeLib\{F0695E19-89CB-45CB-B85A-480E65936926}]#EMC CONFIDENTIALINTERNAL USE ONLYTHANK YOU#EMC CONFIDENTIALINTERNAL USE ONLY#TitleMonth Year