IMPLEMENTING LOCAL REPLICATION USING EMC ... LOCAL REPLICATION USING EMC TIMEFINDER AND RECOVERPOINT 3 AND COEXISTENCE OF RP/CDP WITH SRDF ON SYMMETRIX VMAX …

White Paper

Abstract

This applied technology paper describes the use of EMC TimeFinder™, RecoverPoint™ and SRDF™ running on EMC Symmetrix VMAX™ including VMAX 10K, VMAX 20K and VMAX 40K. Highlighted are the most common use cases and how these products and associated features are deployed in the real world for business continuance. July 2013

IMPLEMENTING LOCAL REPLICATION USING EMC TIMEFINDER AND RECOVERPOINT AND COEXISTENCE OF RP/CDP WITH SRDF ON SYMMETRIX VMAX

2 IMPLEMENTING LOCAL REPLICATION USING EMC TIMEFINDER AND RECOVERPOINT AND COEXISTENCE OF RP/CDP WITH SRDF ON SYMMETRIX VMAX

Copyright © 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware is a registered trademark of VMware, Inc. All other trademarks used herein are the property of their respective owners. Part Number h8981.2


Table of Contents

Executive summary.................................................................................................. 4 Technology Overview .......................................................................................................... 4 EMC Symmetrix VMAX ........................................................................................................ 4 EMC RecoverPoint .............................................................................................................. 6

RecoverPoint components .............................................................................................. 7 RecoverPoint Consistency Groups .................................................................................. 8

EMC TimeFinder/Clone ....................................................................................................... 9 TimeFinder/Consistency Groups ................................................................................... 10 TimeFinder VP Snap...................................................................................................... 11

Symmetrix VMAX SRDF Overview ...................................................................................... 13 SRDF modes of operation ............................................................................................. 13

Co-existence of TimeFinder with RecoverPoint for VMAX ................................................... 17 Co-existence of SRDF and RP CDP .................................................................................... 17 Operations in TimeFinder, RecoverPoint and SRDF environments...................................... 18

Use Case 1 – Using TimeFinder for multiple local copies .............................................. 19 Use Case 2 – Rolling point-in-time copies ..................................................................... 21 Use Case 3 – Restoring the RP source volumes from TimeFinder copies ........................ 23 Use Case 4 – Symmetrix SRDF and RecoverPoint CDP co-existence on the same set of devices. ....................................................................................................................... 26

Conclusion ............................................................................................................ 36

Appendix: Solutions Enabler Command Line Reference .......................................... 37

References ............................................................................................................ 38


Executive summary Symmetrix VMAX offers in-frame local replication using array based TimeFinder technology. Local replication is also supported using RecoverPoint Continuous Data Protection (CDP) by integrating the RecoverPoint splitter in conjunction with the RecoverPoint Appliances (RPA). VMAX offers remote replication using Symmetrix Remote Data Facility (SRDF) and/or RecoverPoint Continuous Remote Replication (CRR). RecoverPoint also offers Concurrent Local and Remote (CLR) data protection.

This applied technology paper describes the use of RecoverPoint and TimeFinder running on EMC Symmetrix™ VMAX. Highlighted are the most common use cases and how these products and associated features are deployed in the real world for business continuance. The paper also discusses the co-existence of TimeFinder and RecoverPoint as well as SRDF and RecoverPoint using single set of source devices to offer different choices to the customer for local replication in conjunction with remote replication.

RecoverPoint allows point-in-time bookmarks of production data with a chosen policy and schedule or they can be defined by the user on an ad-hoc basis. Such bookmarks can be enabled for physical image access on target host(s) for faster error detection and recovery. These bookmarks can also be used to restore the production data to any point in time similar to DVR like rollback, improving RPO and RTO. TimeFinder operations can be performed on RecoverPoint bookmarks allowing users to access the clone rather than using RecoverPoint image access for use cases like backup offload and repurposing production environments for test and/or development.

Using TimeFinder to clone a RecoverPoint replica is recommended in certain situations, and with several advantages such as creating the gold copies from bookmarks for backup or re-purposing. SRDF and RecoverPoint CDP can also co-exist on a single source volume to provide local continuous data protection using RecoverPoint CDP while maintaining a valid D/R copy using SRDF. These business use cases will be described in detail later.in the paper.

Technology Overview

The following section describes the hardware and software technologies that are discussed in this paper.

EMC Symmetrix VMAX

The EMC® Symmetrix VMAX™ with Enginuity™ delivers a multi-controller, scale-out architecture for enterprise reliability, availability, and serviceability at an affordable price. Built on the strategy of simple, intelligent, modular storage, it incorporates a scalable Virtual Matrix™ interconnect that connects all shared resources across all VMAX Engines, allowing the storage array to grow seamlessly from an entry-level configuration with one engine up to eight engines. Each VMAX engine contains two


directors and a redundant interface to the Virtual Matrix interconnects for increased performance and availability.

EMC Symmetrix VMAX delivers enhanced capability and flexibility for deploying various database management systems throughout the entire range of business applications, from mission-critical applications to test and development. In order to support this wide range of performance and reliability at minimum cost, Symmetrix VMAX family offers full line of high end storage platforms with the VMAX 10K, 20K and 40K models to meet the availability, performance and capacity demands of any sized businesses. Symmetrix VMAX can start with as little as 24 drives with a VMAX 10K single engine (a single system bay) and scale up to eight engines supporting up to 3200 drives with VMAX 40K. Symmetrix VMAX arrays support multiple drive technologies that include Enterprise Flash Drives (EFDs), Fibre Channel (FC) drives, and SATA drives. Symmetrix VMAX with FAST for Virtual Pools (FAST VP) technology provides automatic policy-driven storage tiering allocation, based on the actual application workload.

For ease of deployment and improved performance, Symmetrix VMAX offers Virtual Provisioning™ technology to provide ease and speed of storage management, and native wide striping storage layout for higher performance. When oversubscription is used, it can highly improve storage capacity utilization with a seamless “grow as you go” thin provisioning model.

For business continuity and disaster recovery Symmetrix VMAX offers TimeFinder®/Clone for creating local copies of the data for recoverability and restartability. Symmetrix VMAX offers SRDF for creating remote restartable copies for Enterprise consistency and restart. Symmetrix VMAX also offers a native RecoverPoint splitter (named RecoverPoint™).

RecoverPoint provides local and remote replications with any-point-in-time recovery using CDP, CRR, or CLR RecoverPoint technology.


Figure 1 - VMAX maximum specifications

The Symmetrix VMAX provides storage management and provisioning capabilities, as well as replication, migration, and volume configuration options using tools including Virtual LUN VP mobility, and Fully Automated Storage Tiering for Virtual Pools (FAST VP).

To find more information on the products referenced in this document, refer to the References section.

EMC RecoverPoint

EMC RecoverPoint provides DVR-like point in-time recovery with three topologies: local continuous data protection (CDP), synchronous or asynchronous continuous remote replication (CRR), or a combination of both (CLR). RecoverPoint is the offering that simplifies continuous data protection and replication by using EMC Symmetrix VMAX with an Enginuity-based write splitter. RecoverPoint is an appliance-based, out-of-band data protection solution designed to ensure the integrity of production data at local and/or remote sites. It enables customers to centralize and simplify their data protection management and allows for the recovery of data to nearly any point in time.

Performance and Scalability for any size business

– VMAX 10K – VMAX 20K – VMAX 40K

Scalable performance and Tier 1 RAS

– Virtual Matrix Architecture

– Up to eight engines (16 directors/controllers)

– Scale from 24 to 3200 drives

Easy to order, bundled software

– Available virtually provisioned

– FAST VP

Leading replication

– Integrated RecoverPoint splitter

– EMC Symmetrix Remote Data Facility (SRDF)


RecoverPoint provides continuous replication of every write between a pair of local volumes residing on one or more arrays. RecoverPoint also provides remote replication between pairs of volumes residing in two different sites. For local replication and remote synchronous replication, every write is collected and written to a local and remote journal and then distributed to target volumes. For remote asynchronous replications multiple writes are collected at a local site, deduplicated, compressed, and sent across periodically to the remote site where they are uncompressed and written to the journals and then distributed to target volumes.

Figure 2 depicts the RecoverPoint configuration for local and remote replication.

Figure 2 - RecoverPoint configuration

RecoverPoint components

RecoverPoint appliance (RPA)

RPA is a server that runs RecoverPoint software and includes four 4 Gb FC connections and two 1 Gigabit Ethernet connections. For fault tolerance a minimum of two RPAs are needed per site that can be extended up to eight RPAs. RPAs are connected to the SAN and for updating the journal volumes. RPA ports are zoned to the same Symmetrix VMAX front end-adapters (FAs) that are zoned to a production host which has access to all writes originating from the production host.

Symmetrix VMAX write splitter for RecoverPoint

Symmetrix VMAX write splitter for RecoverPoint is an enhanced implementation of Open Replicator that sends all incoming host writes from the VMAX array to a local


RPA cluster for use in CDP local replication, CRR-based remote replication, or CLR, which is a combination of CDP and CRR.

RecoverPoint source volumes

RecoverPoint source volumes are the production volumes that are protected using RecoverPoint.

RecoverPoint replica volumes

RecoverPoint replica volumes are the target RecoverPoint volumes on any heterogeneous storage array containing a full copy of the production volumes. The replica volumes are normally write-disabled volumes but by providing image access functionality RecoverPoint enables direct read/write access on the replica volume to a secondary or standby host by allowing easy access to data at any point in time, in conjunction with the available journal. This any-point-in-time image of the production data can be used for test/development systems, reporting, backup, or many other use cases. Optional features include the ability to swap the roles of secondary (or standby) and primary host, and the direction of replication can be reversed.

RecoverPoint journal volumes

RecoverPoint journals store block-level changes to the source volumes and they are used in conjunction with the replica volumes to enable any-point-in-time recovery. RecoverPoint journal volumes are the Symmetrix devices visible only to the RPA cluster. Because all writes are journaled, the size of the journal depends on the desired period of protection and change rate at the production site.

RecoverPoint repository volumes Repository volumes are very small devices visible to the RPA cluster. They store management information required for RecoverPoint replication operations.

RecoverPoint Consistency Groups

Similar to TimeFinder consistency groups, RecoverPoint consistency groups also allow the creation of a write-order consistent copy of the set of production volumes. The consistency group(s) can be disabled at any time for maintenance operations on production volumes, and RecoverPoint will resynchronize the replica volumes once the consistency group(s) is re-enabled. The best practices for using consistency groups for RecoverPoint are similar to those described for TimeFinder consistency groups.

A consistency group consists of one or more replication sets. Each replication set consists of a production volume and the replica volumes to which it is replicating. The consistency group ensures that updates to the replicas are always consistent and in correct write order; that is, the replicas can always be used as a working set of data, or to restore the production source, in case it is damaged or destroyed.


The consistency group monitors all the volumes added to it to ensure consistency and write-order fidelity. If two data sets are dependent on one another (for example, a database and a database log) they must be included in the same consistency group.

A RecoverPoint consistency group consists of settings and policies, a replication set and journals that receive changes to data.

For the purposes of this white paper project, a RecoverPoint consistency group was created that contained 16 replication sets. The source devices in the replication sets were configured as part of a separate TimeFinder consistency group used to ensure write order consistency during the TimeFinder operations.

Figure 3 - RecoverPoint replication sets used in this test case

EMC TimeFinder/Clone

TimeFinder/Clone provides point-in-time copies of critical data which can be used for backups, decision support, data warehouse refreshes, or any other process that requires multi-application access to data. Each clone keeps its own copy of a source volume’s information at the point in time that it was activated.

TimeFinder/Clone allows users to make copies of data simultaneously on multiple target devices from a single source device. The data is copied from the source device, creating a physical backup copy called a clone. The data can be available to a target device’s host immediately upon activation of the TimeFinder copy session even if the copy process has not completed. The background copy process and host I/O, on the protected tracks on source devices, drive the copy process.


TimeFinder/VP Snap allows users to create space efficient clone when using virtual provisioning. TimeFinder/VP Snap achieves this efficiency by having the shared thin pool extents across multiple VP Snap targets. This technology allows independent access of source and target devices with efficient storage utilization (refer to TF/VP Snap for more information).

TimeFinder/Consistency Groups

TimeFinder/Consistency Groups (TF/CG) ensure dependent-write consistency of application data when creating point-in-time images across multiple devices associated with an application within a single VMAX array or applications that also span multiple VMAX storage arrays.

Note: The source and target device pairs used for both TimeFinder and RecoverPoint are different due to the nature of the replication technologies used. Although the same set of source devices can be part of both (RecoverPoint and TimeFinder) consistency groups, as was the case with this project. TimeFinder and SRDF on the other hand can share the same consistency group for local and remote data protection operations.

For the purposes of this white paper project, a TF/CG containing 16 source and target device pairs were created, activated, tested for backup processing offload, and then restore operations were invoked for various tests.

Steps for creating a TF/Consistency Group Using SYMCLI

symcg create cloneCG

symcg -sid 313 -cg cloneCG addall -devs 7C9:7CC

symcg -sid 313 -cg cloneCG -tgt addall -devs 849:84C

The following graphic represents the query output of a TimeFinder consistency group configured and used during this project.


Figure 4 - symclone query of TF/CG named cloneCG

TimeFinder VP Snap

TF/VP Snap extends the storage efficiency of virtual provisioning to the copies of the data. As described earlier thin devices are cache or pointer based devices that use the storage from the thin pool they are bound to. TF/VP Snap targets can optionally use the source thin pool. The TF/VP Snap is depicted in the Figure 5below.


Figure 5 – TF/VP Snap

As shown in the above Figure 5 when a TF/VP Snap session for 100 GB source device is created no allocation on the target pool takes place. When a VP Snap session is activated the target pool represents data copied from the source devices and subsequent host writes on source device will result in additional copy of data to the target pool. The additional VP Snap target will share the extents in target pool instead of resulting in additional copies.

When a protected track on the source device is modified the original thin pool extent will be copied to the target thin pool, and any active target snap sessions to which this data is newer will share the thin pool extent.

In the event of write to a target device extent that is shared only that extent will cease to remain in shared group and additional allocation in target thin pool will take place for that extent. Only one shared copy exists for multiple target snaps.

As a result of this for the multiple copies of the source data the amount of additional space allocated in the thin pool does not grow with respect to the size of the source or number of snaps created but only based on the writes to protected thin pool extents on source and/target devices. TF/VP Snap allows all the functions traditionally offered by full copy TF/Clone just with excellent storage efficiency. TF/VP Snap also greatly reduces the cache impact during traditional TF/Clone operations allowing many copies of the source devices for a variety of use cases like backup, repurposing or reporting use cases.

Refer to the ‘Use Case 3 – Restoring TF clones to RP source volumes’ section for technical details.

Multiple Clone VP snaps share thin pool allocations to maximize space savings.

... VP SNAP 1

100 TB

SNAP 1

SNAP 2

SNAP x

VP SNAP 2

Target Virtual Pool

Independent 768KB Extent

Shared 768KB

Shared Extent for new snap sessions


Symmetrix VMAX SRDF Overview

Symmetrix Remote Data Facility (SRDF) is a Symmetrix-based business continuance and disaster restart solution. In simplest terms, the purpose of SRDF is to maintain real-time copies of host devices in more than one physical Symmetrix. The Symmetrix units can be in the same room, in different buildings within the same campus, or hundreds of miles apart. SRDF provides data mobility and disaster restart spanning multiple host platforms, operating systems, and applications. It can scale to thousands of devices, can replicate while maintaining write-order consistency from multiple source arrays to multiple target arrays, and can support a variety of topologies and configurations, including support for FAST VP and SRDF Coordination.

The local SRDF device, known as the source (R1) device, is configured in a pairing relationship with a remote target (R2) device, forming an SRDF pair. When the R2 devices are mirrored with R1 devices, the R2 devices are write-disabled to the remote host. After the R2 devices are synchronized with its R1 devices, they can be split at any time, making the R2 devices fully accessible to their hosts. The R2 device can be either used directly by hosts, once they are split, or can be restored incrementally to the R1 devices. TimeFinder replicas can be taken from the R2 devices even while SRDF is replicating, without disturbing the R1 to R2 relationship.

Many other new performance and scalability features were added to SRDF with Enginuity release 5876, including a new protection mode called SRDF/Extended Distance Protection (SRDF/EDP). Please refer to the SRDF product guide for a full description.

SRDF modes of operation

SRDF/Synchronous (SRDF/S), SRDF/Asynchronous (SRDF/A), and SRDF Adaptive Copy are the basic operation modes of SRDF. The first two are valid for Oracle database protection and maintain dependent write-order consistency. The third is useful for bulk data transfers or in combination with more complex SRDF solutions such as SRDF/Automated Replication (SRDF/AR)

SRDF/Synchronous mode

SRDF/S is used to create a no data loss solution of committed transactions. It provides the ability to replicate multiple databases and applications data remotely while guaranteeing the data on both the source and target devices is exactly the same. SRDF/S can protect single or multiple source Symmetrix storage arrays with synchronous replication.

With SRDF/S Synchronous replication, shown in Figure 6, each I/O from the local host to the source R1 devices is first written to the local Symmetrix cache (1) and then it is sent over the SRDF links to the remote Symmetrix unit (2). Once the remote Symmetrix unit acknowledged it received the I/O in its cache successfully (3), the I/O is acknowledged to the local host (4). Synchronous mode guarantees that the remote image is an exact duplication of the source R1 device’s data.


Figure 6, SRDF/Synchronous replication

SRDF/Asynchronous replication mode

SRDF/Asynchronous (SRDF/A) provides a consistent point-in-time image on the target (R2) devices that is only slightly behind the source (R1) devices. SRDF/A allows replication over unlimited distance, with minimum to no effect on the performance of the local production database(s). SRDF/A can “ride” through workload peaks by utilizing the local Symmetrix cache and optionally spilling data to a disk pool (also called delta set extension, or DSE) and reducing the link bandwidth requirements.

SRDF/A session data is transferred to the remote Symmetrix array in timed cycles, also called delta sets, as illustrated in Figure 7. There are three cycles that work in unison – the capture cycle receives all new I/O from the hosts, the transmit/receive cycles on the R1 and R2, respectively, send and receive the previous captured cycle until it is fully received, and the apply cycle applies a previously fully received cycle to the R2 devices.

The SRDF/A cycle switching process is very efficient and scalable. Within a capture cycle if a piece of data is updated multiple times only the most recent update to the data is transmitted once. This process is called write folding. Also, there is no need to maintain write consistency of each I/O. Instead, consistency is maintained between cycles. If replication stops for any reason SRDF will make sure to either apply a fully received cycle to the target R2 devices, or discard the last incomplete cycle. This leaves the remote R2 devices always only one or two cycles behind the R1 devices. While the default minimum cycle switching time is 30 seconds, it can grow during peak workload, and shrink back to default afterward.

RDF links

Source Target

2

4

1 3

Production Database


Figure 7, SRDF/Asynchronous replication

SRDF/A Consistency Exempt

Enginuity has the ability to add or remove devices from an SRDF/A session without breaking the session consistency to perform that operation. When dynamic SRDF devices are added the consistency exempt flag is set, allowing them to synchronize without interrupting the consistency attributes of the other devices in the SRDF/A session. After they are in sync for two cycles the flag will be automatically removed, allowing them to join the session consistency attributes. When devices are suspended the consistency exempt flag will be automatically set, thus allowing them to be removed without interrupting the SRDF session consistency. These new and flexible abilities enhance database protection and availability.

SRDF/A Multi-Session Consistency

Like SRDF/S, SRDF/A can replicate from multiple source arrays to multiple target arrays while maintaining write-order consistency between cycles. When dependent write consistently across multiple Symmetrix arrays is required, the SRDF/A Multi-Session Consistency (MSC) option is used and the coordination of cycle switching across the arrays is performed with the assistance of SRDF redundant host daemons. The daemons merely wait for ready conditions on all the arrays and then send the switch cycle command, keeping communication light and efficient. Similar to TimeFinder consistent split, also when SRDF/A MSC is used there is a brief hold of write I/O on all the arrays simultaneously during cycle switch to preserve write-order consistency.

SRDF Adaptive Copy replication mode

SRDF Adaptive Copy replication facilitates long-distance data sharing and migration (see Figure 8). SRDF Adaptive Copy replication allows the primary and secondary volumes to be more than one I/O out of synchronization. The maximum number of I/Os that can be out of synchronization is known as the maximum skew value, and can be set using SRDF monitoring and control software. There is no attempt to preserve the ordering of write I/Os when using SRDF Adaptive Copy replication.

Source Target

1

Production Database

Transmit Receive

Capture SRDF links

Apply

R2 R1 2 3

4


Figure 8, SRDF Adaptive Copy mode

SRDF Adaptive Copy replication is useful as an interim step before changing to an Oracle-supported SRDF/S or SRDF/A replication. It is also used for point-in-time long-distance bulk transfer of data. For example, if the connection between the two sides is lost for a long period of time allowing the buildup of a large number of changes to accumulate, resumption of the links can cause a heavy surge in link traffic (created by the backlog of changes added to those generated by normal production traffic). By using SRDF Adaptive Copy replication, the backlog of invalid tracks is synchronized using the SRDF low priority queue, while new writes are buffered in cache and sent across using the high priority SRDF queue without impacting the host application. Once the backlog of changes has been transferred, or the total amount of changed tracks has reached a specified number, the mode can be changed to SRDF/S or SRDF/A replication to achieve database protection.

SRDF Adaptive Copy replication is not supported for database restart or database recovery solutions with Oracle databases. Using SRDF Adaptive Copy replication by itself for disaster protection of Oracle databases will lead to a corrupt and unusable remote database.

RDF links

Source Target

3

2

1 4

Production Database


Co-existence of TimeFinder with RecoverPoint for VMAX

The EMC TimeFinder software provides a local copy of VMAX data, independent of the host and operating system, application, and database. Production volumes of data can be the source for TimeFinder or RecoverPoint operations. Typical applications for using TimeFinder with RecoverPoint would be data warehousing or other applications that require full volume, independently host addressable, local point-in-time copies of VMAX production devices. The writes on source volumes are journaled by RecoverPoint and the journal space has to be recycled for continued data protection as they fill up. Using TimeFinder in conjunction with RecoverPoint will extend the period of data protection and enable point in time recovery of production data even beyond RecoverPoint based protection. It should be noted that when using TimeFinder based copies for restoring production data, the RecoverPoint consistency group should be disabled during the restore operation. RecoverPoint will automatically perform a full sweep once the consistency group is re-enabled to resume data protection.

Using TF to clone a RP replica is recommended in the following cases:

• When image access must be enabled for an extended period of time

• When RP must track a large number of changes to the replica

• When the image being accessed must support a heavy write workload

The advantages of using a TF/Clone in such cases are:

• RP distribution can continue as soon as the clone is activated

• The clone is not limited by available journal space

• Accessing a clone has no impact on RP or splitters

• Improved host performance (after the clone copying is complete)

Co-existence of SRDF and RP CDP The EMC RecoverPoint CDP provides local journaled copy of the VMAX data allowing DVR like any point in time recovery or specific point in time recovery when integrated with application consistency. The EMC SRDF provides Synchronous and Asynchronous replications at distance for remote disaster recovery and application restart with enterprise consistency. By using RP CDP for local protection logical errors that are detected can be easily corrected. Co-existence of RP CDP and SRDF for the same set of application data would offer local and remote data protection with great improvement in RPO and RTO. The typical scenario would be to define SRDF and RP consistency groups for the same set of application volumes and configure data protection using RP CDP and SRDF. The considerations for the co-existence of SRDF and RP CDP are:

• Both SRDF and RP pairs have to be synchronized completely to ensure that both data protection windows for SRDF and RP are active


• When the production volumes are restored using RP CDP bookmarks, SRDF will continue to transfer the invalid tracks from source to target and higher than normal activity may appear on SRDF link during such operation but this will not compromise the D/R capability of the solution.

• RP CDP consistency group must be disabled to continue with SRDF failover. The consistency group will be re-established after the failover finishes, to resume the RP CDP operation. This means that all the prior journals are invalidated and hence a new CDP protection window would start.

Operations in TimeFinder, RecoverPoint and SRDF environments

The following use cases discuss common tasks and procedures performed within organizations deploying this combination of product sets. Best practices and order of operations are described as well.

For this project, a standard RecoverPoint CDP environment was configured using 16 replication sets and a single clustered RPA as shown below.

Figure 9 – RecoverPoint configuration used in this test case

Note: This document does not discuss the configuration or implementation of the aforementioned products for VMAX storage arrays. Refer to the References section to find more information on installation and configuration guidelines.


Use Case 1 – Using TimeFinder for multiple local copies

Multiple copies of the production data can be created to be used for backup offload to another server, test and development, reporting and monitoring. Some of these copies will be routinely refreshed to get the most up-to-date production data. Even when starting from a single point in time, various replicas of data may represent different forms of production data. This use case illustrates the creation and activation of multiple VP Snap from a single set of source devices. The Solutions Enabler1 SYMCLI (Symmetrix Command Line Interface) command ‘symclone’ is used to create the clone using the source and target pairings defined via text file.

The Unisphere GUI can also be used to define the source and target relationships and perform all TimeFinder operations. The procedures executed throughout this document use the SYMCLI.

These are the steps required for creating multiple TimeFinder/Clone replicas;

A. Create a text file containing source and target device pairings B. Create the TimeFinder/Clone session for the device group C. Verify the clone session creation and mode of the copy operation D. Activate the clone to create point-in-time copy E. Repeat the process for all other copies

The steps for creating multiple TF/Clone are as follows:

1. Define a list of source and target device pairs to be used for the TimeFinder operations using an ordinary text file(s). A device can appear no more than once within each list, but it can be the source in different lists. The graphic below displays a sample file.

Note: For each additional VP Snap, a separate file needs to be created.

1 EMC Solutions Enabler software, when licensed for use with TimeFinder, enables the Symmetrix Command Line Interface (SYMCLI) utility to perform TimeFinder operations.


2. When creating multiple source/target clone pairs, separate files must be created where each line contains a source and target device id in the pairing list. For example; in order to clone source device 7c9 to three target devices 849, 84a and 84b, this operation will require three files as shown below.

File A.txt: 7c9 849

File B.txt: 7c9 84a

File C.txt: 7c9 84b

…

3. Create the TimeFinder/Clone session for the device group

symclone create –sid 313 –file <filename> [For TF/VP Snap use the switch: <-vse>]

4. Verify the clone session creation and mode of the copy operation by using the symclone list –v command as shown below.

5. Activate the clone

Activate the TimeFinder/Clone session to create a point-in-time replica from the source devices. VP Snap

symclone activate –sid 313 –file <filename> -tgt –consistent

Note: “consistent split” is chosen below to ensure a write order consistent copy from the production data.


Use Case 2 – Rolling point-in-time copies

This use case illustrates the creation and activation of TimeFinder clones/VP snaps at periodic time intervals. A source device will be cloned to multiple targets, but the clones are added one by one – at various time intervals. Typically the clones are created/recreated hourly or daily.

The procedures for creating rolling point-in-time clones are as follows:

1. Define the length of time and cycle times for each set of source/target pairings. A device can appear no more than once within each file, but it can be the source in different files. For example, a weekly cycle of daily clones will require 7 files. The number of files cannot exceed a TF/Clone limit of 16 or TF/VP Snap limit of 32 [Note: Please refer to support matrix for additional restrictions].

2. If the cycle is longer than number of allowable number of copies per source, the recommendation would to recycle the files by terminating the oldest clones. A complementary strategy with a larger granularity clone (e.g. quarterly cycle of weekly clones) may also be required to extend the time horizon.

3. Create a file for each set of clones you wish to create (as described above) where each line contains a source-target device pairing in the list. The graphic below shows an example of two files used to create the rolling point-in-time clone scenario.

4. Create the TimeFinder/Clone symclone create –sid 313 –file day1.txt [-vse]

symclone activate –sid 313 –file day1.txt -tgt –consistent


A. RecoverPoint bookmarks can also be used to create point in time copies. TimeFinder operations can be performed on RecoverPoint bookmarks allowing users to access the clone rather than using RecoverPoint image access. There are various ways to create a bookmark. One way, is to simply select the consistency group name (TF_CDP in this case), right-click and select ‘create bookmark’.

B. The new RP snapshot image is displayed under ‘Bookmark Details’.

5. Repeat the above steps in cyclic order for each set of clones. symclone create –sid 313 –file day2.txt [-vse]

symclone activate –sid 313 –file day2.txt -tgt –consistent

6. If the clone session limit (16) has been reached, the oldest set of clones can be terminated and the target devices can be recycled. Terminate a clone session as follows.

symclone terminate –sid 313 –file <filename> -tgt

7. After activating multiple clone sessions over time, each write to a source device will still cause N additional writes to the target devices, where N is the number of clone targets. The same considerations exist as with Use Case 1. However, the system can absorb the gradual phase-in of each additional clone much better. In particular, the creation of certain write pending tracks is less bursty with rolling


clones. As a result, the possibility of reaching the system write pending limit is greatly reduced.

Use Case 3 – Restoring the RP source volumes from TimeFinder copies

During this procedure, the TimeFinder copy targets are used to restore the RecoverPoint source volumes. The clone target devices must be in a copied state (meaning there are no outstanding tracks to replicate) before any restore can take place.

These are the steps required for using TimeFinder/Clone or VP Snap for the restore. They are described in more details below.

A. Create a TimeFinder/Clone B. Activate it to any point in time to be used for restore operation later C. Verify that the full copy operation has completed successfully D. Issue restore command

1. Create a TimeFinder clone using the procedure described in Use Case 1 2. Activate the TimeFinder Clone to make sure it is a point in time (PiT) copy to be preserved

for future use. The activation step at any time will mark the tracks to that point in time copy of the source data.

symclone –sid 313 -file <Device file> activate

3. Verify that the tracks are copied prior to continuing with the restore operation. In the example below, the verification interval is set to 15 seconds. The command will return the control when all protected tracks at the time of activation are copied from source to target.

symclone –sid 313 –file <filename> verify –copied –i 15

4. Once all tracks are copied, the consistency group can be restored, however the RecoverPoint consistency group must be disabled prior to a TimeFinder restore operation. To disable, simply right-click on the name of consistency group and select ‘disable image access’.


5. When you elect to disable image access, RecoverPoint journaling for the consistency group is terminated, and a full sweep of the CDP will be initiated after the consistency group is re-enabled.

6. During this operation, the RecoverPoint CDP is paused as shown below.

7. Now the environment is ready and the TimeFinder consistency group can be restored to its original source using the following command.

symclone restore –sid 313 –file <filename> -full

8. Applications can be started as soon as the restore operation is initiated (waiting for the restore to complete is not necessary). It is advisable to wait for restore completion prior to re-enabling the RecoverPoint consistency group (in the next step).

9. After the restore, the RecoverPoint consistency group can be enabled as shown below. When image access is enabled, meaning that the replicas are enabled, the device state is RW (read-write).


Note: Enabling the consistency group will invoke a full sweep of the CDP and can take a while depending on the number and size of the configured devices.

10. In the final step, the TimeFinder consistency group can be terminated.

symclone –sid 313 terminate –f <filename>


Use Case 4 – Symmetrix SRDF and RecoverPoint CDP co-existence on the same set of devices.

This use case illustrates the co-existence of Symmetrix SRDF and RecoverPoint CDP on the same set of devices to provide local point in time copies using CDP and have a valid D/R copy using SRDF to facilitate the recovery of a production database /application, in the event of a logical error or a disaster event occurring on the local side. The configuration of the system will use Symmetrix SRDF R1 devices tagged for use with RecoverPoint as the Source Copy devices for the RecoverPoint CDP Consistency Group. An Oracle database is used for the test environment.

Note: SRDF and CDP can also co-exist on remote (R2) Symm in which case CDP consistency group will be created on R2 devices.

High-level steps

1. Configure the RecoverPoint CDP consistency group using Unisphere for RecoverPoint on local Symmetrix.

2. Configure the SRDF consistency group to create the D/R copy of the application on the remote Symmetrix.

3. Ensure that both consistency groups are fully synchronized with relevant target volumes. 4. Perform production database restore and recovery using RecoverPoint CDP bookmark at the

desired point in time for logical error detection and correction. 5. Disable the RecoverPoint CDP consistency group to allow an SRDF restore of the production

database from the remote SRDF R2 target and restart the production application. 6. Re-enable the RecoverPoint CDP consistency group to re-establish RecoverPoint based local

protection for the production database.


Detailed Description A. Setup and Configuration 1. Configure the RecoverPoint CDP consistency group using the SRDF R1 devices as the Source

Copy devices. The R1 devices will need to be tagged for RecoverPoint use either from the SYMCLI or Unisphere for VMAX. The Figure 10 shows the R1 devices being used as the Source Copy for the RecoverPoint CDP replication sets.

Figure 10 SRDF and RP/CDP co-existence for an application

2. Configure the SRDF consistency group using the SYMCLI and an SRDF pairs group file indicating the R1 to R2 pairings.

Figure 11 SRDF pairings between R1 and R2 devices

Host Symcli

showing SRDF CG

RecoverPoint GUI Showing RP

CG


3. Ensure that the consistency groups for both the RecoverPoint CDP and SRDF are synchronized and active. Figure 12 shows the Active RecoverPoint CDP Consistency Group in the RecoverPoint GUI and the Synchronized SRDF devices from the SYMCLI.

Figure 12 Fully Synchronized SRDF and RP/CDP CG

B. Recovering from logical errors on production database using RecoverPoint/CDP copy

1. When a logical error is detected you can restore production from a point in time based on RecoverPoint’s continuous bookmarking process or from a user defined bookmark. Steps to create a user defined bookmark. From the RecoverPoint GUI select the Protecton tab. Then select the CG to work with from the Consistency Groups window. Then click the “Apply Bookmark” button at the bottom of the Status window. Provide a name for your bookmark and click OK.

SRDF CG

RecoverPoint CG


Figure 13 Create User Defined Bookmark

2. Start running an Online Transaction Processing workload to effect changes not reflected in the bookmark.

3. Check the initial state of the database using SQL queries.

4. When the Online Transaction Processing workload completes, stop the database.

5. Restore the Production from the user defined bookmark created earlier.

a. From the Protection Tab, select the CG.

b. From the status window select “Recover Production”

Figure 14 Start Recover Production from RP CDP bookmark


c. Select the Replica copy then click the “Next Select an Image” button.

Figure 15 Select Replica copy

d. Enter the search info to find the image to be used then click the “Next Start Recovery” button.

Figure 16 Search for User Defined Bookmark


e. The recovery process needs to prepare the image that will be used to recover production. Once the image is prepared hit the “Finish” button.

Figure 17 Prepare Image for recovery

You will get the following messages:

Select Yes.

Select Close.


f. This screen shows the production recovery process in action.

Activity – Recovering Production

Figure 18 Recover Production in process

6. After Production is recovered restart the database to ensure that it is in a restartable state.

7. Validate the recovered Production database by comparing the current state of the database with the initial state to ensure that restore completed successfully.

C. Recovering the production database in the event of a disaster using SRDF R2 copy

1. Stop the database.

2. From the RecoverPoint GUI disable the CG.

a. Confirm action to disable CG.


Figure 19 Disable CG – Confirm Action

b. Acknowledge the warning that the journal will be lost and a full sweep will be needed.

Figure 20 Acknowledge Warning – Full Sweep required


c. Figure 21shows that the CG is disabled

Figure 21 CG in Disabled state

3. Use the Unisphere for VMAX or SYMCLI to restore the Production from the remote R2 devices.

4. After the restore is complete, from the RecoverPoint GUI re-enable the CG to allow a full sweep sync of the CG.

a. Confirm action to enable CG

Figure 22 Enable CG – Confirm Action


b. Full sweep in progress after the CG was enabled.

Figure 23 CG Full Sweep in progress

5. When the full sweep sync is complete, restart the database. This will ensure that the restored copy is good.

6. Start running an Online Transaction Processing workload to ensure the database is usable.


Conclusion

EMC TimeFinder software provides a local copy of VMAX data, independent of the host and operating system, application, and database. RecoverPoint CDP provides a local copy of VMAX data with a rollback feature to any point in time. SRDF provides a remote copy of VMAX data for D/R purposes. Production volumes of data can be the source for TimeFinder, RecoverPoint or SRDF operations.

Deploying RecoverPoint to replicate an application remotely and then using TimeFinder at the remote site to create fully writeable clones for use with another application and/or database are the best suited operations for these product sets. RecoverPoint’s ability to provide continuous local and remote data protection with DVR-like rollback capabilities are very complimentary to TimeFinder/Clone features and functionality.

Also deploying RecoverPoint CDP using SRDF R1 devices as the source copy for the consistency group will provide the ability to rollback production applications to any point in time in the event of a logical or operational error occurring, while continuing to maintain a remote copy of your data in the event of a disaster at the local site.

When using thin provisioning, TimeFinder/VP Snap extends the space efficiency of virtually provisioned VMAX storage array to copies of the source data. Application workloads will benefit from multiple space efficient copies available using TimeFinder/VP Snap by having many copies of the target devices without significant increase in allocated thin pool space. These copies can then be used for offloading backups, repurposing source data for TEST/DEV or reporting or incremental restore allowing very quick RTO.


Appendix: Solutions Enabler Command Line Reference A. Checking the RDF link state prior to running restore

Operation: Prior to running the restore we need to verify that the SRDF link state is Split.

Command: symrdf –sid xxx list

Output:

B. Execute the SRDF restore on the R1 / RecoverPoint CDP source devices.

Operation: Execute the symcli command to do a full restore with the –rp flag due to the R1 devices being the source devices in the consistency group. RP/CDP CG must be disabled using Unisphere for RecoverPoint prior to SRDF restore. When the CG is re-enabled full sweep will be performed.

Command: symrdf -sid xxx restore -full -rp -f device_filename -rdfg x

Output:


C. Monitor SRDF restore process for completion

Operation: While the SRDF full restore is in process, continue to monitor with the list command for completion.

Command: symrdf –sid xxx list –i 15

Output:

References Reference information and product documentation can be found at www.powerlink.emc.com including;

• General

o EMC Symmetrix VMAX Series – Product Guide

o EMC Symmetrix TimeFinder – Product Guide

o EMC Symmetrix SRDF – Product Guide

o EMC RecoverPoint – Administration Guide

o EMC RecoverPoint for the VMAX Series – Applied Technology white paper

• Installation and Configuration

o EMC RecoverPoint Deploying with Symmetrix Arrays and Splitters – Technical Notes

o EMC RecoverPoint - Installation Guide

o EMC Solutions Enabler - Installation Guide

o EMC RecoverPoint & the Symmetrix array-based Splitter

http://www.powerlink.emc.com/

Documents

IMPLEMENTING LOCAL REPLICATION USING EMC ... LOCAL REPLICATION USING EMC TIMEFINDER AND RECOVERPOINT 3 AND COEXISTENCE OF RP/CDP WITH SRDF ON SYMMETRIX VMAX …