ETERNUS DX Advanced Copy Functions - Fujitsu · PDF fileETERNUS DX Advanced Copy Functions Author: Fujitsu ISS/ FTA Subject: REC $Remote Equivalent Copy$ Created Date: 3/12/2012

$Page 1: ETERNUS DX Advanced Copy Functions - Fujitsu · PDF fileETERNUS DX Advanced Copy Functions Author: Fujitsu ISS/ FTA Subject: REC $Remote Equivalent Copy$ Created Date: 3/12/2012$
0 Copyright Fujitsu, Release February 2012

ETERNUS DX Advanced Copy Functions

REC (Remote Equivalent Copy)

REC ETERNUS DX Advanced Copy Functions

Content

REC Overview REC Modes Synchronous Split and Recovery Sub-modes Asynchronous Transmission Sub-modes REC in Detail Differences Between REC Modes Skip Initial Copy Combinations of Advanced Copy

REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 1

REC Overview

REC Usage Business Continuity: Disaster Recovery REC Concept Diagram REC Suspend and Resume REC Process REC Concurrent Suspend


Copyright Fujitsu, Release February 2012

REC Usage (1) Remote copy between two ETERNUS systems High Availability Short distances

• Data transfer can be achieved using FC or iSCSI Separate, independent interconnection lines between the ETERNUS

systems are necessary

REC ETERNUS DX Advanced Copy Functions

SAN

Operation Volume

Brocade Fiber Channel Switch

Site B

ETERNUS DX

Management Server

SAN

Operation Volume


Site A

ETERNUS DX

Management Server

3

REC Usage (2) Remote copy with ETERNUS systems Disaster Recovery Very long distances

• 10+ km data transfer can be achieved by FC-WAN converter or iSCSI Separate, independent interconnection lines between the ETERNUS

systems are necessary

Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions

Management Server

SAN

Advanced Copy Function REC

Operation Volume

Main site

ETERNUS DX

WAN/LAN SAN

Backup Volume

Management Server

ETERNUS DX

Backup site


Management Server


4

Business Continuity: Disaster Recovery Recovery Point Objective, RPO How much data has been lost?

Recovery Time Objective, RTO How soon after the event is the application available again?


RTO RPO

Time

Last known good image

Application is available again

Application Downtime

Recover

Yesterday 24 hrs Now

All modifications that are lost

Analyze

Restore

5

REC Concept Diagram Background copy transfers sequentially all data of the Source

Volume to the target site While also updating both the Source and Target with any new write data

REC must be started and reach the equivalent state before the Destination Volume can be detached (suspend or cancel) The Destination Volume is again writeable after suspend or cancel


Source at

site A

Target at

site B

Start REC Write

Background Copy Paired

Initial Copy complete

Mirroring

Write Stop REC (Split）

I/O Req.

After Split, both Volumes can be

accessed independently

I/O Req.

6

Cancelling REC terminates the session and a complete re-mirroring will be required when a new REC session is started All blocks of the Source need to be copied After resuming from Suspend state only the changes made

during the suspended state are copied Changes to both the Source and Destination Volumes are recorded in a

bitmap during the suspended state

REC Suspend and Resume


Suspend REC

Re-mirroring

Equivalent state

S D

S D

Cancel REC Write to source

Copy All data

Using the bitmap information,

copy changed data only!

Start REC

Resume S D

Equivalent state

Changes to Source are recorded when suspended

or destination volume

and Destination volumes or overwrite data only!

7

REC Process REC is started Replication proceeds in the background

• Block level copy from the local Source to the remote Destination Copy rate depends on ETERNUS priority setting (high, medium or low)

• Parameter to be assigned via the ETERNUS GUI or CLI The Source Volume remains available The Destination Volume is inaccessible Throughput is degraded during initial copy

Replication is complete The mirrors are kept in Equivalent state

Detach request (Cancel or Suspend) Must be coordinated with the application to obtain a logically "consistent"

copy • For example, flush system buffer, shutdown application, etc.

Both mirror Volumes are available for write access while suspended or after cancelling

Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 8

REC Concurrent Suspend Concurrent Suspend function This allows consistent copies of multi-volume data objects, such as

databases All REC replication pairs that are involved have to be grouped Possible after REC session has reached Equivalent state Multiple REC sessions can be suspended at the same point in time


REC Modes

REC Synchronous Modes REC Asynchronous Modes


REC Synchronous Modes Write confirmation is responded to the host after data transfer

to the target ETERNUS DX is completed This mode guarantees that the copy process is fully completed

on both sides of the REC before an acknowledgement of a data write request is returned to the server


Source Volume Site A

Target Volume Site B

Write Acknowledged

Application

Copy

OK

11

REC Asynchronous Modes Write confirmation is responded to the host after data write to

local cache of the ETERNUS DX system This mode will not guarantee that the copy process is fully

completed on both sides of the REC before any update or write process response is returned to the server


Queue

Write Acknowledged Acknowledged Write



Application Application

12

Synchronous Split and Recovery Sub-modes

Transfer Mode "Automatic Split" Transfer Mode "Manual Split" Line error options/recovery with "Automatic Recovery" or

"Manual Recovery"


Transfer Mode "Automatic Split" During path failure Automatic disconnection of the REC session might occur

• For example if the transmission fails during the REC process Write access to Source Volume can continue even when the REC

session is in Hardware Suspend • Changes are tracked by the bitmap mechanism

Server continues to write on the copy Source only The source and target data are no longer in Equivalent state

• Source and Destination LUNs need to be synchronized after the connection problem has been solved


Source

Target

Write Write Write Write

Copy Copy Copy

Time

Write

Automatic Split mode

Not Copied Failure

14

Transfer Mode "Manual Split During path failure The REC session stops completely and all writes are suspended Source and Target Volumes remain in Equivalent state

• If the path failure is a result of a site disaster this prevents data losses but on the other hand means unscheduled downtime for the applications

To regain access to the Source LUN of the ETERNUS, "Suspend" needs to be executed


Source

Target

Write Write Write Write

Copy Copy Copy

Time

Write

Not Copied Failure

Manual Split mode

15

Line Error Options/Recovery Automatic Recovery REC session is automatically resumed after the data path functionality

has been recovered

Manual Recovery REC session must be manually "Resumed" after the data path is restored

• Before issuing "Resume", this option allows clone creation (OPC, EC) of the mirror Volumes • Ensures that the last known good data images are preserved before writing on the

Volumes is re-enabled

As both mode options, Manual Split and Manual Recovery, require manual or scripted intervention, they are recommended for experienced users only


Asynchronous Transmission Sub-modes

Stack Mode Example of Using Stack Mode Asynchronous Mode - Consistency REC Buffer Transfer Parameters Consistency Mode Buffer Configuration Consistency Mode Usage


Asynchronous - Stack Mode (1) This mode uses a bitmap Only updated block locations are recorded in the bitmap before sending

acknowledgement to the server • Data itself is not cached

Transfer of the changed blocks is performed by an independent transfer engine that scans the bitmap Write order is NOT maintained in the

destination Useful when limited bandwidth

is available but a large amount of data needs to be transferred


1. Write data to disk 2. Record updated block 3. Write acknowledgement

Write

...

Write Ack.

Write I/O

Bit map

1 1

18

Asynchronous - Stack Mode (2) When multiple writes have occurred in the same block

address, only the latest data at the time of the next patrol check is sent to the copy destination, helping to minimize the amount of transferred data This mode can lead to a loss of

data if a malfunction occurs in the source system This mode is mainly used for

backing up data to a remote system


...

Transfer Engine Process

Transfer Engine

4. Transfer engine search and find an updated block

5. Transfer updated block

1

1

1. Write 2. Write

19

Example of Using Stack Mode (1) Example: REC Multi Copy Two REC (Stack) sessions are set up

and alternately operate as Suspend / Active (2) Example: REC Cascade Copy Two Local Copy sessions are set up

and operate at the REC (Stack) copy destination

Creating a Snapshot to a Remote site (remote backup)

When a data failure occurs at source site • Restore is performed using the session volume

that was in the suspend state at that time Suspending a session not possible

in Stack mode, only in Async. mode Stack mode -> Force suspend Switch to through mode Resume -> Suspend


1

2

20

Asynchronous Mode - Consistency (1) This mode works always with REC Buffer Cache Receive buffer and send buffer as a pair The write order across multiple REC sessions can be

guaranteed at the copy destination i.e. write consistency over multiple LUNs or regions Updates to multiple LUNs on the ETERNUS are accumulated

in the REC buffer and then grouped in buffer units for transfer This reduces the communication sequences with the remote site and

enables more efficient use of line capabilities (increased performance)


Asynchronous Mode - Consistency (2) This mode uses ETERNUS REC buffers to store actual data Not only bitmap pointers

REC buffers are configured as either Receive or Transmit At check point, all access to

Source Volumes are stopped Less than 100 micro seconds Overhead is less than 0.1%

Check point interval can be set between 1 and 120 seconds This has an affect to the RPO

Buffer data is applied to the remote site only after the entire buffer is received


Write data copies to buffer

Check Point

Source

Destination

Buffer switch, then transfer

Transfer complete, then copy to dest.

Send Buffer

Receive Buffer

22

Asynchronous Mode - Consistency (3) In this mode a large amount of data accumulates in the REC

buffer All the accumulated data is transferred to the destination as a

block at user definable intervals There are three REC buffer parameters for the asynchronous

consistency mode that are set with the ETERNUS Web GUI


Item Description Forwarding Interval (sec) Specify the data forwarding interval

Watch (Monitor) Time (minutes)

Specify the monitoring time before transition changes to Halt status for a REC session when a REC buffer shortage occurs. If the REC buffer remains in high-load state (overload) for the specified time, the status of REC session will be automatically changed to Halt. When the Watch Time is set to 0, the REC buffer is not monitored.

Halt Wait Timer (seconds) Specify the maximal non-response time. The ETERNUS monitors the host I/O waiting time. During this time the host I/O is stopped from ETERNUS point of view. This gives priority to perform REC buffer transfers. If the waiting time exceeds the specified time, REC session status automatically changes to Halt status.

23

REC Buffer Transfer Parameters Buffers can be set to either Send or Receive with following

setup Send Buffer at REC source Receive Buffer at REC destination Up to eight1) buffers (in total) can be set per system In the REC asynchronous Consistency mode, there may be Eight connected ETERNUS DX pairs Each pair has one connection channel with a send and receive buffer This only allows copy in one direction

• But the direction can be reversed from the command line by changing the REC buffer role (swsrpbuffset)

For model specific configuration details please refer to the Remote Advanced Copy Function Guide Many ETERNUS DX models provide also REC Disk Buffer

functionality


1) Four with DX90 and DX90 S2, please refer to product specification for up-to-date information.

Consistency Mode Buffer Configuration Multiple pairs of REC buffers have been configured to enable

copying between multiple systems (1) Enables copying between multiple locations (1) Use single buffer pair for unidirectional data transfer only (2) Enables copying in both directions between systems (2) Use two buffer pairs for bidirectional data transfer

• For forward and reverse data transfer NOTE: It is not possible to configure 2 x send buffers addressing the

same destination ETERNUS system


Receive REC buffer

Site A

SEND REC buffer

SEND REC buffer

Receive REC buffer

Site B

Pair 1

Pair 2 SEND REC buffer

Site A

SEND REC buffer

Receive REC buffer

Site B

Receive REC buffer

Site C

Pair 2

1 2 Pair 1

25

Consistency Mode Usage (1) Consistency pairs can be

established between two systems and for each transfer direction Multiple applications can

share one consistency pair Each consistency pairs can

be individually controlled Backups can be made

separately for each system Restore can be used for recovery, independently of the consistency pair


Site A Site B

App-Sys A

App-Sys B

App-Sys A

App-Sys B

REC buffer

send

REC buffer

receive

REC buffer

receive

REC buffer

send

App2-Sys C App1-Sys C

App2-Sys C App1-Sys C

Consistency Pair 1

Consistency Pair 2

26

Consistency Mode Usage (2) REC Consistency mode preserves write order During the copy process the updated blocks are stored in different sets of

REC buffers located in the CM cache All REC buffer sets are transmitted in one transaction

• Either all buffer sets are copied or nothing is copied

The receiving ETERNUS system copies the data to the disk only after having received the complete buffer contents

The transmitting ETERNUS system will not discard buffer contents until the receiving ETERNUS system acknowledges the reception


REC in Detail

Asynchronous Mode Suspend and Resume Asynchronous Mode Line Failure Asynchronous Through Mode


Asynchronous Mode - Suspend/Resume (1) During Suspended state, modified blocks are marked at each

side in bitmap tables Marked blocks at target side are overwritten (nullified) after

resume




A*

Write

Bitmap Table

REC Re-Sync Copy

Keep track of modifications

REC Maintain Copy

A

A

REC Maintain Copy

A**

A**

A*

Suspend

C

Bitmap Table

Resume

Merge Bitmap Table

A*

1

1

Write

Write Write

Overwrite

29

Asynchronous Mode - Suspend/Resume (2) This picture illustrates the bitmap merge after Resume is

invoked


Suspend Source Destination

Cache

Bitmap

Cache

Bitmap

Source physical

disk

Destination physical

disk

Resume Source Destination

Cache

Bitmap

Cache

Bitmap

Source physical

disk

Destination physical

disk

Bitmap merge

30

Asynchronous Mode - Line Failure During path failure, the modified blocks are marked in

bitmap table After path becomes available again, the marked blocks are

transferred without preserving the write order


A*

Write

REC Re-Sync

Copy

REC Maintain

Copy

A

A

Write

REC Maintain

Copy

A**

A**

A*

Halt State

First Write after path failure returns sense information

Re-synchronization

A*

Path Failure Path Available

Keep track of modifications



Write

Bitmap Table

31

REC Asynchronous Through Mode This mode is a special asynchronous mode It is used to flush as-yet-untransferred data when a Stack or Consistency

mode session is in • Stop state (due to Cancel command) • Suspend state

This mode should be used when an interconnect path failure occurs after a Cancel or Suspend is initiated After the path is restored the buffered data must be flushed using the

REC Asynchronous Through Mode


Through Mode for Sequential Writing This diagram shows the impact to the server I/O when running

"Asynchronous Through Mode" The performance of this mode equals that of the REC

Synchronous Mode For these reasons, you should not use the REC Asynchronous Through

Mode under normal circumstances


Server ETERNUS DX Copy Source

ETERNUS DX Copy Destination

1. Update data

1. Acknowledge 2. Update data

2. Acknowledge

ETERNUS

1. Acknowledge

Source ETERNUS DX has to wait for the acknowledgement of the previous data transfer.

33

Differences Between REC Modes

Network Bandwidth Differences Between Modes REC Modes of Operation - Summary


Network Bandwidth Synchronous and Consistency modes: Match to peak times

Stack mode: Match to average load


Update quantity

Required bandwidth

Update quantity

Required bandwidth

35

Differences Between Modes Transfer mode comparison


REC Modes of Operation - Summary Synchronous REC Application waits until update reflected on local and remote volume Order guaranteed No data loss High impact on performance Automatic or manual split options Asynchronous REC (Stack mode) Application waits until update reflected locally and placed in move queue Independent process sends updates to destination No update order guarantee Data loss possible Low impact on performance level Asynchronous REC (Consistency mode) Application waits until update placed into local group send queue After timer interval (1, 2 or 4 seconds) transmitted to remote receive

queue Update order guaranteed Data loss possible Low impact on performance level


Skip Initial Copy

Start+Suspend / Resume+Remain Initial Copy Skip Mode


There is a special ACM command option available to support this feature for REC only Start+Suspend Resume+Remain

Start+Suspend / Resume+Remain


Suspend

Write

EC Maintain

Copy

Write



Start +

Suspend

Tape Backup

Tape Restore

Operation can be continued

Suspend

Resume +

Remain

Time line

39

When the Start+Suspend is issued the Copy Source data has to be consistent from the application point of view The Copy Destination data has to be 100% identical with the

Source data when the Resume+Remain is issued Note, ETERNUS does not verify the data consistency between the two

Volumes After the synchronization of the Destination is completed, ETERNUS

shows always equivalent state for the Volumes • Regardless if the data actually is consistent

Initial Copy Skip Mode


Combinations of Advanced Copy

Cascade Copy Multi Copy


Concatenates several copy sessions An existing copy Destination Volume becomes the copy

Source for a new copy session

Cascade Copy


OPC, QuickOPC EC

REC - Stack

REC - Synchronous - Stack - Consistency

OPC, QuickOPC SnapOPC EC

REC - Stack - Consistency

REC - Stack

REC - Consistency

REC - Synchronous

Source-1 Source-2

Destin-2 Destin-1

Source-2 Destin-2 Destin-1

Source-1 Source-2

Destin-2 Destin-1

Source-1

ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3

ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3

42

Copies the same copy source area to multiple copy destinations The destination can not be the source for another multi copy

session

Multi Copy


REC

ETERNUS DX-1

Destin-2

Destin-1

Source-1

ETERNUS DX-2

Destin-1

43


Documents

ETERNUS DX Advanced Copy Functions - Fujitsu · PDF fileETERNUS DX Advanced Copy Functions Author: Fujitsu ISS/ FTA Subject: REC \(Remote Equivalent Copy\) Created Date: 3/12/2012