VNX SRDF_A

EMCé VNX Series

Release 7.1

Using SRDFé/A with VNX

P/N 300-013-438 Rev 01

EMC CorporationCorporate Headquarters:

Hopkinton, MA 01748-9103

1-508-435-1000

www.EMC.com

Copyright â 1998 - 2012 EMC Corporation. All rights reserved.

Published July 2012

EMC believes the information in this publication is accurate as of its publication date. Theinformation is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATIONMAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TOTHE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIEDWARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires anapplicable software license.

For the most up-to-date regulatory document for your product line, go to the TechnicalDocumentation and Advisories section on EMC Powerlink.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks onEMC.com.

All other trademarks used herein are the property of their respective owners.

Corporate Headquarters: Hopkinton, MA 01748-9103

2 Using SRDF/A with VNX 7.1

Contents

Preface.....................................................................................................7

Chapter 1: Introduction.........................................................................11

System requirements..................................................................................12

Restrictions..................................................................................................12

User interface choices.................................................................................14

Related information.....................................................................................15

Chapter 2: Concepts.............................................................................17

SRDF...........................................................................................................19

Communication between Control Stations.........................................19

SRDF and logical volumes.................................................................20

SRDF/A.......................................................................................................22

SRDF/A delta sets.......................................................................................23

Important resource considerations.....................................................24

Differences between SRDF/A and SRDF/S................................................25

Comparison of VNX for file high-availability and replication products.........26

Planning considerations..............................................................................28

Symmetrix system configuration........................................................29

Adding Symmetrix storage devices to the existing system

configuration..................................................................................29

VNX volume and Data Mover decisions (flexibility)............................30

VNX Data Mover configuration checklist............................................31

Consideration when using applications that require transactional

consistency...................................................................................32

Consideration when using applications that can switch to the

NFS copy from the R2 without a restart........................................33

Using SRDF/A with VNX 7.1 3

Upgrading the VNX SRDF environment.............................................34

SRDF/A task overview................................................................................34

Overview of the sample configuration.........................................................39

Chapter 3: Configuring.........................................................................41

Preinitialize the configuration......................................................................42

Preinitialize from the source (first) VNX..............................................43

Preinitialize from the destination (second) VNX.................................43

Verify the preinitialization....................................................................44

Initialize the configuration (active/passive)..................................................45

Initialize the source VNX....................................................................45

Initialize the destination VNX..............................................................49

Verify SRDF/A on the source VNX (active/passive)...........................53

Activate a failover (active/passive)...............................................................56

Prepare for a graceful failover............................................................57

Activate a failover from the destination VNX.......................................59

Verify SRDF/A after failover activation................................................63

Ensure access after failover...............................................................69

Restore the source VNX..............................................................................69

Prepare for the restore.......................................................................70

Restore from the destination..............................................................70

Chapter 4: Troubleshooting..................................................................77

EMC E-Lab Interoperability Navigator.........................................................78

Known problems and limitations..................................................................78

Retrieve information from log files......................................................78

Resolve initialization failures..............................................................79

Resolve activation failures..................................................................84

Resolve restore failures......................................................................87

Resolve Data Mover failure after failover activation............................95

Handle additional error situations.......................................................98

Error messages...........................................................................................98

EMC Training and Professional Services....................................................99

Appendix A: Portfolio of High-Availability Options...........................101

Glossary................................................................................................107


Contents

Index.....................................................................................................111


Contents


Contents

Preface

As part of an effort to improve and enhance the performance and capabilities of its productlines, EMC periodically releases revisions of its hardware and software. Therefore, somefunctions described in this document may not be supported by all versions of the softwareor hardware currently in use. For the most up-to-date information on product features, referto your product release notes.

If a product does not function properly or does not function as described in this document,please contact your EMC representative.


Special notice conventions

EMC uses the following conventions for special notices:

Note: Emphasizes content that is of exceptional importance or interest but does not relate topersonal injury or business/data loss.

Identifies content that warns of potential business or data loss.

CAUTION Indicates a hazardous situation which, if not avoided, could result in minor ormoderate injury.

Indicates a hazardous situation which, if not avoided, could result in death or serious injury.

DANGER Indicates a hazardous situation which, if not avoided, will result in death or seriousinjury.

Where to get help

EMC support, product, and licensing information can be obtained as follows:

Product information — For documentation, release notes, software updates, or forinformation about EMC products, licensing, and service, go to the EMC Online Supportwebsite (registration required) at http://Support.EMC.com.

Troubleshooting — Go to the EMC Online Support website. After logging in, locatethe applicable Support by Product page.

Technical support — For technical support and service requests, go to EMC CustomerService on the EMC Online Support website. After logging in, locate the applicableSupport by Product page, and choose either Live Chat or Create a service request. Toopen a service request through EMC Online Support, you must have a valid supportagreement. Contact your EMC sales representative for details about obtaining a validsupport agreement or with questions about your account.

Note: Do not request a specific support representative unless one has already been assigned toyour particular system problem.


Preface

http://Support.EMC.com



Your comments

Your suggestions will help us continue to improve the accuracy, organization, and overallquality of the user publications.

Please send your opinion of this document to:

[email protected]


Preface


Preface

1

Introduction

EMC Symmetrix Remote Data Facility/Asynchronous (SRDF/A) is anextended-distance asynchronous replication facility. SRDF/A providesdependent write consistency for host writes from a source VNX/SymmetrixDMX system pair to a destination VNX/Symmetrix DMX system pair throughpredetermined time cycles (delta sets) for maintaining a restartable,point-in-time remote copy of data. Chapter 2 provides more details.

This document is part of the VNX documentation set and is intended forsystem administrators responsible for installing and managinghigh-availability storage configurations.Your local EMC Customer SupportRepresentative performs the Symmetrix storage system administrativetasks, such as installation and configuration of the Symmetrix DMX andSRDF volumes required to support SRDF/A, and is available to help ensureproper SRDF/A functionality as needed.

Topics include:

System requirements on page 12 Restrictions on page 12 User interface choices on page 14 Related information on page 15


System requirements

Table 1 on page 12 describes the EMCé VNX series software, hardware, network, andstorage configurations.

Table 1. System requirements

EMC Symmetrixé

Enginuity

version 5670 or later microcode on the source and destination Symmetrix

DMX

systems with SRDF/A.

VNX version 7.1 on the source and destination servers.

Software

Remote adapter (RA) for interconnecting the Symmetrix DMX systems. The connection can be over

wide area channels using Fibre Channel (FC), Dense Wavelength Division Multiplexing (DWDM), or

IP.

Production and standby Data Movers on the VNX systems to support the EMC SRDFé

/A active/passive

configuration.

Similar source and destination VNX models.

Hardware

IP data network for communication between Control Stations of source and destination VNX for File sys-

tems.

Network

Two attached VNX/Symmetrix DMX system pairs.Storage

Restrictions

SRDF/A requires a Symmetrix SRDF/A license.

SRDF/A does not work with Symmetrix 3xxx, 5xxx, or 8xxx versions. Only SymmetrixDMX systems (5670 or later microcode) are supported.

Note: The Solutions Enabler Symmetrix SRDF family documentation, which includes the EMCSymmetrix Remote Data Facility (SRDF) Product Guide, provides additional restrictions that applyto the SRDF/A configuration with Symmetrix DMX. This documentation is available athttp://Support.EMC.com, the EMC Online Support website.

The Symmetrix Enginuity version 5670 microcode supports only one SRDF/A group,which can be dedicated to either an open host or a VNX, but not a mix of both.The 5671release eases this restriction.

SRDF/A source and destination sites support only one VNX/Symmetrix system pair.

SRDF/A does not support partial failovers. When a failover occurs, all file systemsassociated with SRDF-protected Data Movers fail over. To avoid failover issues, it iscritical that SRDF-protected Data Movers mount only the file systems consisting of SRDF


Introduction


volumes mirrored at the destination. Local standard (STD) and business continuance(BCV) volumes should have dedicated, locally protected Data Movers and SRDF volumesshould have dedicated, SRDF-protected Data Movers. Chapter 4 provides informationon potential failover and restore issues.

IP Alias cannot be set up or used on the Control Stations when SRDF is configured fordisaster recovery purposes.

Management restrictions

SRDF/A cannot be managed with the EMC Unisphere software. SRDF/A can bemanaged only with the CLI by using the /nas/sbin/nas_rdf/nas_cel commands.

For sites with redundant Control Stations, all SRDF management commands, including/nas/sbin/nas_rdf/nas_cel commands that perform initialization, activation, and restoreoperations, must be run from CS0. Control Station 1 (CS1) must be powered off atboth sites before any activation or restore commands are run.

Solutions Enabler Symmetrix SYMCLI action commands are not invoked on a VNXdevice group by using the Control Station host component, laptop service processor,or EMC Ionix ControlCenteré. However, you can run informational SYMCLIcommands for the SRDF/A device group. The Solutions Enabler Symmetrix SRDFfamily documentation on the EMC Online Support website provides more information.

When you run the nas_rdf restore command, ensure that no command is accessingthe /nas/rdf/500 directory. NAS commands in rdfadmin environment may access the/nas/rdf/500 directory. If a command accesses /nas/rdf/500 during restore, the restorecommand will fail.

To use fs_timefinder in the activated state by using the rdfadmin account, straightRemote Standby Data Mover configuration such as slot 2 to slot 2, slot 3 to slot 3 isrequired.

When you create file systems or extend them, ensure that all remote replicas are ofthe same type, either FC disks or ATA disks.

SnapSure checkpoints restrictions

EMC SnapSure SavVol cannot be created on local storage if the production filesystem (PFS) is mounted on a Data Mover configured with a remote standby. If youplan to create checkpoints of a PFS that resides on an SRDF LUN, ensure that theentire SnapSure SavVol with the checkpoints resides in the same pool of SRDF LUNsused to create the PFS. If any part of the SavVol is stored on a local volume ratherthan completely on the pool of SRDF LUNs, the checkpoints are not failed over, andare recoverable in the event of a failover. Evaluate the use of checkpoints carefully.

After an SRDF/A failover is activated, checkpoint scheduling is not supported until arestore is performed.

Restrictions 13

Introduction

If SnapSure checkpoints are used in the rdfadmin environment, the SavVol volumecan only be extended manually using the symm_std_rdf_tgt storage pool. If SavVolfills to capacity, writes to the PFS continue, while the oldest checkpoint is deactivated.

The NDMP automatic checkpoint create and delete feature will not work in the activatedstate (rdfadmin account environment). Manual checkpoint create and delete will work.

VNX feature-specific restrictions

Automatic File System Extension will not work in the activated state (rdfadmin accountenvironment). Manual extension will work.

SRDF/A with Timefinder/FS NearCopy or FarCopy is currently available with therequest for price quotation (RPQ) process. Contact the local EMC Representation orthe local EMC Service Provider for more information.

EMC VNX Replicator works with disaster recovery replication products such asSRDF/Synchronous (SRDF/S) and SRDF/Asynchronous (SRDF/A) or EMCMirrorView/Synchronous (MirrorView/S).You can run SRDF or MirrorView/S productsand VNX Replicator on the same data. However, if there is an SRDF or MirrorView/Ssite failover, you cannot manage Replicator sessions on the SRDF or MirrorView/Sfailover site. Existing Replicator sessions will continue to run on the failed over DataMover and data will still be replicated. On the primary site, you can continue to manageyour SRDF or MirrorView/S replication sessions after the restore.

Mixed backend configuration with SRDF and EMC MirrorView/S is not supported. Ina mixed backend configuration, Symmetrix is the boot storage device, which prohibitsuse of MirrorView/S. MirrorView/S requires VNX for block to be the boot storagedevice.

SRDF/Automated Replication (SRDF/AR) does not support SRDF/A devices.

R1 EFD, R2 EFD and BCV EFD devices are supported through FAST Storage Group.These devices must be added to the FAST Storage Group to be used on VNX forFile systems.

SRDF provides limited support for VNX FileMover. Using VNX FileMover providesmore information.

SRDF provides limited support for MPFS. Contact the local EMC sales organizationfor information about using MPFS with SRDF.

EMC E-Lab Interoperability Navigator on page 78 provides information about productinteroperability. System requirements on page 12 identifies the basic hardware andsoftware requirements.

User interface choices

This document describes how to configure SRDF/A by using the command line interface(CLI).You cannot use other VNX management applications to configure SRDF/A.


Introduction

Related information

VNX disaster recovery that is SRDF/A related, but is beyond the scope of this document, isincluded in:

VNX Command Line Interface Reference for File

Celerra Network Server Error Messages Guide

Online VNX for File man pages

Problem Resolution Roadmap for VNX

Using VNX FileMover

Using MirrorView/Synchronous with VNX for File for Disaster Recovery

Using TimeFinder/FS, NearCopy, and FarCopy on VNX

Using SRDF/S with VNX for Disaster Recovery

Other related EMC publications, include:

Solutions Enabler Symmetrix SRDF Family CLI Product Guide

Symmetrix Remote Data Facility (SRDF) Product Guide

EMC Business Continuity for Oracle Database 11g — Enabled by EMC Celerra usingDNFS and NFS Proven Solution Guide

EMC VNX documentation on the EMC Online Support website

The complete set of EMC VNX series customer publications is available on the EMCOnline Support website. To search for technical documentation, go tohttp://Support.EMC.com. After logging in to the website, click the VNX Support by Productpage to locate information for the specific feature required.

VNX wizards

Unisphere software provides wizards for performing setup and configuration tasks. TheUnisphere online help provides more details on the wizards.

Related information 15

Introduction



Introduction

2

Concepts

SRDF/A is an extended-distance asynchronous replication facility.

SRDF/A provides an economical and high-performance replicationconfiguration for business continuity, enabling greater service levels thanobtained with traditional asynchronous architectures. It is the optimalextended-distance replication option when service-level requirementsdictate that economics and application performance are more critical thanzero data exposure.

Benefits of SRDF/A include:

Extended-distance data replication that supports longer distances thanSymmetrix Remote Data Facility/Synchronous (SRDF/S).

Potential performance increase over SRDF/S because source hostactivity is decoupled from remote copy activity.

Efficient link utilization that results in lower link-bandwidth requirementsbecause less data is transferred and the data is handled fewer times.By exploiting the locality of reference writes within an application, datathat is updated multiple times in the same cycle is sent across theSRDF/A links only once, and does not have to be duplicated within theglobal memory of the sending control unit to preserve data consistency.The benefit of this approach might vary among applications.

Minimal data loss potential because the destination side lags behindthe source by minutes. Data could be out of date by twice the delta settime period or longer, depending on the write activity, and how fast thedata can be transmitted across the link.

Failover and failback capability between the source and destinationsites. Destination volumes can be used if the source is unavailable.

Facilities to invoke failover and restore operations manually.

SRDF/A is based on the Symmetrix Remote Data Facility (SRDF)technology for the EMC Symmetrix environment and is a part of the SRDFreplication software family that offers high-availability configurations.SRDF/A recovery capability completely relies on the SRDF functionality.


Topics include:

SRDF on page 19 SRDF/A on page 22 SRDF/A delta sets on page 23 Differences between SRDF/A and SRDF/S on page 25 Comparison of VNX for file high-availability and replication products

on page 26 Planning considerations on page 28 SRDF/A task overview on page 34 Overview of the sample configuration on page 39


Concepts

SRDF

SRDF is a replication technology that allows two or more Symmetrix systems to maintain amirror of data at multiple, remote locations.

SRDF supports two types of configurations:

active/active’ — Bidirectional configuration with two production sites, each acting as thestandby for the other. Each VNX has production and standby Data Movers. If one sitefails, the other site takes over and serves the clients of both sites.

active/passive — Unidirectional setup where one VNX for file, with its attached storagesystem, serves as the source (production) file server and another VNX for file, with itsattached storage, serves as the destination (backup).This configuration provides failovercapabilities in the event that the source site is unavailable.

Communication between Control Stations

In an SRDF configuration, the Control Stations associated with the source and destinationVNX for file can communicate by using the IP data network.

SRDF 19

Concepts

Figure 1 on page 20 shows a basic VNX-with-SRDF configuration that illustrates the meansof communication.

PS0 PS1 PS2 PS3 PS4 SMB0 SMB1

SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15


SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15

SourceR1

TargetR2

Site A source Site B target

VNXhost

VNXhost

Symmetrix Symmetrix

VNX-000033

IP

data network

connection

Figure 1. Basic VNX-with-SRDF configuration

SRDF and logical volumes

To implement the VNX with an SRDF configuration, the standard VNX for file configurationis modified.

Each logical volume defined in the VNX volume database consists of two physical volumes:one on the source Symmetrix and one on the destination Symmetrix system. For example,a typical Symmetrix volume, logical volume 001, consists of 001-R1 (the primary, or source,disk) and 001-R2 (the destination, or remote, disk).

The R2 devices mirror the data from the source R1 volume by using SRDF. The R2 devicesare transparent to the source VNX host and become write-accessible only when they areactivated through a manual failover operation. However, if a source (R1) volume fails duringnormal operation, SRDF automatically uses the R2 volume instead of R1 and continuesnormal operation.


Concepts

Figure 2 on page 21 provides a simplified view of a sample logical volume configuration.Note that Symmetrix volume IDs do not have to be identical (mapping 001 to 001 as shownin this view).


SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15


SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15

Dedicated SRDF link

SourceR1

001-R1

TargetR2

001-R2


VNXhost

VNXhost

Symmetrix Symmetrix

VNX-000034

Logical volume 001

Figure 2. VNX with SRDF logical volume

Note: In the VNX for file system setup phase, your local EMC Symmetrix system configurationRepresentative configures the volumes on the attached Symmetrix system (source) and the remoteSymmetrix system (destination). In addition, SRDF software and EMC communications hardware areinstalled to enable the Symmetrix systems to communicate with each other. Local EMC personnel canensure that the volumes are established on the dedicated SRDF link.

VNX support for SRDF includes:

Complete disaster recovery without data loss — The VNX for file supports SRDF/S, whichrepresents SRDF in synchronous mode. SRDF/S is a limited-distance replication facilityestablished between a pair of VNX/Symmetrix systems at a source site and a destinationsite. SRDF/S provides a synchronized, realtime remote mirror of data at more than onelocation.

To support a wide variety of business continuance requirements while maintaining a fulldisaster recovery environment employing Symmetrix storage, you can also use SRDF/Swith the TimeFinder/FS NearCopy feature. TimeFinder/FS NearCopy enables you tocreate and manage snapshots of a VNX for file file system on to dedicated Symmetrix

SRDF 21

Concepts

BCVs. To create the NearCopy snapshot, the source volumes can be destination (R2)volumes, operating in SRDF/S mode. Using SRDF/S with VNX for Disaster Recoveryand Using TimeFinder/FS, NearCopy, and FarCopy with VNX provide more information.

Extended-distance replication — The VNX supports SRDF/A, which represents SRDFin asynchronous mode. SRDF/A is an extended-distance replication facility establishedbetween a pair of VNX for file/Symmetrix DMX systems at a source site and a destinationsite. SRDF/A provides a restartable, point-in-time remote mirror of data lagging not farbehind the source.

VNX also supports the use of TimeFinder/FS FarCopy with SRDF in adaptive copywrite-pending mode, where you create the FarCopy snapshot by using R2 volumes asthe source volume.

Contact your EMC sales organization for more information about these and other availableconfiguration options.

SRDF/A

An SRDF/A configuration connects a local VNX/Symmetrix DMX system pair with a remoteVNX/Symmetrix DMX system pair.This connection can be made over longer distances thanthe SRDF/Synchronous (SRDF/S) product. After the VNX is configured, users can continueto access the SRDF-protected VNX file systems should the local (source) VNX, the SymmetrixDMX system, or both become unavailable.

This document describes how to use SRDF/A with VNX in an active/passive configuration.In an active/passive configuration, data is replicated between two attached Symmetrix DMXsystems connected through SRDF/A links. Normal mirroring between the Symmetrix DMXsystems is carried out, while the destination (target) VNX remains on standby. Whilefunctioning as a standby, the destination VNX is powered up and its Control Station is fullyoperational. The destination VNX configuration provides complete hardware redundancyfor the source VNX.

Note:

For the purposes of SRDF/A configuration, it is assumed that the local volumes and SRDF-protectedvolumes have their own, dedicated Data Movers. To avoid failover issues, SRDF-protected DataMovers should contain only SRDF volumes, not local (STD, BCV) volumes. Also, ensure that thefile systems do not span multiple storage systems.

The VNX continually monitors its internal hardware status and, if properly configured, initiates aCallHome event to report any problems that could require attention by local EMC personnel.


Concepts

SRDF/A delta sets

SRDF/A processes asynchronous-mode host writes from the source (R1) to the destination(R2) by using predetermined cycles of operation called delta sets.

Delta-set logical cycles, which include capture, transmit or receive, and restore, providedependent write consistency. Dependent write consistency refers to the maintenance of aconsistent replica of data between the source and destination, achieved by processing andpreserving all writes to the destination in ordered (sequential) numbered sets.

The following steps summarize the data processing in the delta set cycles:

1. Capture cycle N: An SRDF/A delta set (active cycle N) begins on the source (R1) tocapture new writes, overwriting any duplicate tracks intended for transfer over the SRDF/Alink. This cycle is active for a predetermined time period by default, using a cycle time of30 seconds.Your local EMC Customer Support Representative can help you configurethis on the Symmetrix DMX system.

2. Transmit cycle N-1: After the predetermined cycle time is reached, the delta-set datamoves to the transmit cycle N-1, during which the delta-set data collected on the sourceis transmitted to the destination (R2) Symmetrix DMX system.This data is inactive duringthe N-1 cycle. A new cycle N starts to collect new writes in preparation for the nextdelta-set transfer.

3. Receive cycle N-1: During the N-1 cycle, the delta-set data is collected at the destination(R2) Symmetrix DMX system. When all N-1 cycle data is successfully transmitted by thesource, and received by the destination, and the minimum cycle time elapses, the delta-setdata moves to the next cycle.

4. Restore cycle N-2: During this cycle, writes are restored to the R2 destination (markedas device write pendings or destaged to disk). When all writes associated with delta setN-2 are committed to the R2 destination, the R1 source and R2 destination are consideredto be in a consistent-pair state. A cycle switch occurs only after these conditions are met.

Figure 3 on page 24 illustrates how data is processed in delta set cycles.

A cycle switch occurs only after all these conditions are met:

1. The minimum cycle time elapses.

2. The N-1 cycle finishes transferring data, and the data is fully received at the destinationsite.

3. The N-2 cycle finishes restoring data at the destination.

SRDF/A delta sets 23

Concepts

Note: The cycle time is elongated if the write transfer or destaging exceeds the set cycle time. A newcycle cannot begin until all three conditions are met. It is possible a switch could take longer, forexample, 20 minutes or more, depending on how quickly the data travels across the link.


SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15


SB

0

SB

1

SB

2

SB

3

SB

4

SB

5

SB

6

SB

7

SB

8

SB

9

SB

10

SB

11

SB

12

SB

13

SB

14

SB

15


VNXhost

VNXhost

Symmetrix Symmetrix

SRDF/A device pairactive sessions

SRDF/A delta set begins

Dedicated SRDF link

1. Capture writes using default cycle time of 30 sec (active)

2. Transmit writes to R2 (inactive)

4. Writes restored to R2 target device (active)

3. Receive writes on R2 (inactive)

VNX-000032

SourceR1

device

TargetR2

device

Cycle

N-2

Cycle

N-1

Cycle

N-1

Cycle

N

I/O

Figure 3. SRDF/A active/passive configuration featuring delta sets (cycles) to transfer data

Important resource considerations

Over time, the SRDF/A facility might incur additional load due to increased levels of writeactivity. An example is increased write activity introduced by new applications or growthto existing applications.

During periods of intense write activity, completion of the delta-set transfer could exceed20 minutes, depending on how fast data can travel across the link.

Depending on how often you see extended-length transfers (cycle times), you might needto increase the bandwidth capacity between sites. Local EMC personnel can help youdetermine the appropriate amount of bandwidth for your configuration.You can monitorthe SRDF/A average cycle time and the time R2 is behind R1 by running a SYMCLI querycommand for your SRDF/A device group (for example, /nas/symcli/bin/symrdf -g<device_group> query -rdfa. A sample device group is 1R1_3).


Concepts

Differences between SRDF/A and SRDF/S

Table 2 on page 25 summarizes the key differences between SRDF/A and SRDF/S on VNXsystems with Symmetrix storage system.

Table 2. Summary of differences between SRDF/A and SRDF/S

SRDF/SSRDF/A

Works with Symmetrix 3xxx, 5xxx, 8xxx, or DMXsystems.

Requires Symmetrix DMX systems or later versions.

Complete disaster recovery configuration for systemswithin relatively short distances.

Extended-distance replication facility.

Provides a complete disaster recovery configurationwithout data loss.

Provides manual failover and failback of backendstorage systems and servers with minimal data loss.

Provides I/O-level consistency.Provides system-wide points of consistency.

Each write I/O to the source is synchronously repli-cated to the destination Symmetrix system.The hostis acknowledged of the write after the destinationSymmetrix system receives and CRC-checks thedata.

Each write I/O to the source is immediately commit-ted on the source image only and sent to the desti-nation in predefined timed cycles (delta sets). Thehost is acknowledged of the write before the destina-tion Symmetrix system receives the write.

Each I/O is sent to the destination Symmetrix systembefore an acknowledgment is sent to the host.

Locality of reference provides more efficient use ofnetwork bandwidth. With data sent as cycles of de-pendent write-consistent data, less data needs totransfer to the destination. If the same track is writtento more than once within an active set, SRDF/Asends the update over the link once.

Supports active/passive and active/active’ configura-tions.

Currently supports active/passive configurations only.

Can be implemented with TimeFinder/FS NearCopyand FarCopy (adaptive copy disk or disk-pendingmode).

Not implemented with configurations employingTimeFinder/FS NearCopy or FarCopy. NearCopy isnot applicable because SRDF/A volumes cannot besource volumes for TimeFinder/FS NearCopy filesystem snapshots. In a configuration employingFarCopy, adaptive copy disk mode or disk-pendingmode can be used instead of SRDF/A.

Appendix A provides illustrations of various business continuance and high-availabilityconfigurations that EMC supports with the VNX. Using SRDF/S with VNX for DisasterRecovery provides information about SRDF/S, which is the synchronous version of SRDF.

Differences between SRDF/A and SRDF/S 25

Concepts

Comparison of VNX for file high-availability and replication products

Table 3 on page 26 lists and compares the different VNX for file product options for disasterrecovery, high availability, and file system replication or copying. The local EMC salesorganization can provide information about the other configuration options.

Note: EMC recommends that all parts of a VNX file system use the same type of disk storage and bestored on a single storage system. A file system spanning more than one storage system increasesthe chance of data loss or data unavailability. Managing Volumes and File Systems with VNX AutomaticVolume Management and Managing Volumes and File Systems for VNX Manually provide moreinformation.

Table 3. Comparison of VNX for file high-availability and replication products

RestrictionsDescriptionStorage platformProduct

Cannot be used in the sameVNX for file configuration asSRDF. Cannot be used withSymmetrix-based productssuch as TimeFinder/FS.Cannot be used with the Au-tomatic File System Exten-sion feature.

Performs LUN (volume)cloning, not file systemcloning; remote volumes areaccessible only after afailover.

With MirrorView/S, the VNXfor file does not see the mir-rored LUNs at the destinationsite until a failover is activat-ed. This is different fromSRDF, in which the remotemirrors (R2 volumes) of theControl Station LUNs arevisible to the Data Movers.

Using attached VNX for fileand VNX for block backendpairs, performs synchronized,remote mirroring to providefull disaster recovery withoutdata loss at a limited dis-tance. Using MirrorView/Syn-

chronous with VNX for File

for Disaster Recovery pro-vides more information aboutMirrorView/S. RPQ processcontrols the access to thisdocument.

VNX for block only, CX series(CX700/600/500/400, CX3-40, or CX3-80)

MirrorView/Synchronous

Note: This product is current-ly available through the RPQprocess. Contact your localEMC Sales Representativeor EMC Service Provider formore information.

Cannot be used in the sameVNX for file configuration asMirrorView/S. Cannot beused with the Automatic FileSystem Extension feature.

Performs volume cloning, notfile system cloning; remotevolumes are only accessibleafter a failover.

Using attached VNX for fileand Symmetrix pairs, per-forms synchronized, remotereplication to provide full dis-aster recovery without dataloss at a limited distance.Using SRDF/S with VNX for

Disaster Recoveryprovidesmore information aboutSRDF/S.

Symmetrix only (3xxx, 5xxx,8xxx, or DMX series)

SRDF/Synchronous


Concepts

Table 3. Comparison of VNX for file high-availability and replicationproducts (continued)


Cannot be used in the sameVNX for file configuration asMirrorView/S. Cannot beused with the Automatic FileSystem Extension feature.

Performs volume cloning, notfile system cloning; remotevolumes are only accessibleafter a failover.

Using attached VNX for fileand Symmetrix pairs, per-forms asynchronous, point-in-time replication at an ex-tended distance.This techni-cal module provides more in-formation about SRDF/A.

Symmetrix only, DMX seriesSRDF/Asynchronous

A single file system shouldoccupy an entire STD vol-ume. TimeFinder/FS doesnot perform volume cloning.

Using a business continu-ance configuration with Sym-metrix business continuancevolumes (BCVs), provideslocal file system cloning.

Symmetrix onlyTimeFinder/FS

These products do not per-form volume cloning.NearCopy is limited to 200km. FarCopy supports extend-ed distances.

NearCopy relies on SRDF/Sand FarCopy relies on SRDFadaptive copy disk or disk-pending mode to manage filesystem snapshots.

TimeFinder/FS NearCopyand FarCopy do not workwith Automatic File SystemExtension.

Using a business continu-ance configuration with Sym-metrix BCVs, provides re-mote file system cloning,creating a point-in-time copy(snapshot) of a VNX produc-tion file system.

A remote file system can bemounted and used duringnormal operation (that is, it isgenerally accessible).

Using TimeFinder/FS,

NearCopy, and FarCopy on

VNX provides more informa-tion about TimeFinder/FSproducts.

Symmetrix onlyTimeFinder/FS NearCopyand FarCopy

Comparison of VNX for file high-availability and replication products 27

Concepts

Table 3. Comparison of VNX for file high-availability and replicationproducts (continued)


For TimeFinder/FS,NearCopy, or FarCopy, abusiness continuance volume(BCV) cannot be a source ora destination file system forreplication.You can replicatethe underlying source filesystem, but you cannot repli-cate the BCV.

You cannot use theTimeFinder/FS -Restore op-tion for a replicated sourcefile system. Replication isunaware of any changes be-cause these changes occurat the volume level.

Produces a read-only, point-in-time copy of a source filesystem and periodically up-dates this copy, making itconsistent with the source filesystem. The read-only copycan be used by a Data Moverin the same VNX cabinet, orby a Data Mover at a remotesite for content distribution,backup, and application test-ing. Using VNX Replicator

provides more information.

VNX for file-supported stor-age (one Symmetrix or VNXfor block pair)

VNX Replicator

Not intended to be a mirror,disaster recovery, or high-availability tool. Because it ispartially derived from realtimePFS data, a checkpoint couldbecome inaccessible (notreadable) if the PFS be-comes inaccessible. Onlycheckpoints and a PFSsaved to a tape or to an alter-nate storage location can beused to provide disaster re-covery.

In version 5.6, SnapSuresupports 112 checkpoints, 96in 5.5, 64 in 5.4, and 32 priorto 5.4.

On a VNX system, providesread-only, point-in-time logi-cal copies, also known ascheckpoints, of a productionfile system.

VNX-supported storageSnapSure

Planning considerations

Before configuring SRDF/A with the VNX, consider the following planning information:

Symmetrix system configuration on page 29

VNX volume and Data Mover decisions (flexibility) on page 30

VNX Data Mover configuration checklist on page 31


Concepts

Consideration when using applications that require transactional consistency on page32

Consideration when using applications that can switch to the NFS copy from the R2without a restart on page 33

Upgrading the VNX SRDF environment on page 34

Note: Examine the configuration in terms of local volumes against the intended SRDF-protectedvolumes. Examples are STD and BCV, not mirrored at the destination. Because SRDF/A does notsupport partial failovers, and all file systems associated with the SRDF-protected Data Movers areeligible for failover, ensure that different Data Movers support local volumes against SRDF volumes.Associate the local volumes with locally protected Data Movers, and associate the SRDF volumeswith dedicated SRDF-protected Data Movers. To avoid SRDF failover issues, do not mount localvolumes on a Data Mover that also mounts SRDF volumes. Chapter 4 provides a description of thefailover and restore issues for SRDF/A.

Symmetrix system configuration

The SRDF/A configuration setup tasks assume that the volumes on Symmetrix DMX systemsare configured in conformance with SRDF requirements, and the SRDF link is operational,and in asynchronous mode.Your local EMC Customer Support Representative can configurethe volumes on the source and destination Symmetrix systems, and install SRDF softwareand communications hardware to enable the Symmetrix systems to communicate with eachother.

The Solutions Enabler Symmetrix SRDF family documentation on the EMC Online Supportwebsite, which includes the Symmetrix Remote Data Facility (SRDF) Product Guide, providesdetails about SRDF requirements for Symmetrix systems.

Adding Symmetrix storage devices to the existing system configuration

After initial system configuration, if you want to add Symmetrix storage devices to theconfiguration, use the following procedure:

1. Run the server_devconfig -create -scsi -all command to search for newly added Symmetrixdevices and create new logical disk volumes.

2. Depending on the SRDF configuration, run the /nas/sbin/nas_rdf -init command on thesource and target Control Stations in the following manner:

• If the SRDF configuration is active/passive, run the /nas/sbin/nas_rdf -init commandfirst on the source and then on the target.

• If the SRDF configuration is active/active’, run the /nas/sbin/nas_rdf -init commandfirst on the source Control Station, then on the target Control Station and then againon the source Control Station.

Planning considerations 29

Concepts


In the active/active’ configuration, each Symmetrix system is partitioned into primary andremote volumes. There are two communication links, each connecting the primary volumeswith their remote counterparts on the remote Symmetrix system. Because of this, when yourun the /nas/sbin/nas_rdf -init command on the source Control Station for the first time, thedevice configuration database and R1 device groups on the primary side are updated, butthe device configuration database and R2 device groups on the target side are not updated.When you run the /nas/sbin/nas_rdf -init command on the target Control Station, the deviceconfiguration database for the other direction is updated. The R2 device groups are alsoupdated based on the device configuration database of the source side. The R2 devicegroups on the source Symmetrix are updated based on the target side device configurationdatabase when you run the /nas/sbin/nas_rdf -init command for the third time.

VNX volume and Data Mover decisions (flexibility)

A typical SRDF/A VNX active/passive configuration provides a full backup of the source site,Data Mover for Data Mover. However, you can choose which volumes and Data Movers toprotect with remote mirroring. For example, you can remotely mirror some volumes andData Movers while others are only locally protected.You do not have to remotely protect allvolumes and Data Movers.

When planning the Data Mover configuration:

For every source (production) Data Mover that you choose to protect with a remoteSRDF/A standby Data Mover, you must provide a dedicated standby Data Mover at thedestination site. There must be a one-to-one relationship between a source Data Moverthat you choose to protect, and a dedicated remote standby Data Mover at the destinationsite.

If the source Data Mover with a remote SRDF/A standby Data Mover also has a localstandby Data Mover, then that local standby must have a remote SRDF/A standby DataMover at the destination site. This prevents issues with failover.

An SRDF/A standby Data Mover at the destination can be paired only with one sourceData Mover.

The network configuration of the SRDF/A standby Data Mover must be a superset of thenetwork configuration of the source Data Mover.

The network configuration of the SRDF/A standby Data Mover must be able to accessthe R2 data volumes corresponding to the primary source Data Mover.

Local Data Movers can be configured to have a local standby Data Mover, separate fromthe SRDF-protected Data Movers.

Data Mover configurations possible at the local and remote locations in active/passiveSRDF/A environments include:

On the local VNX:

• Local production Data Mover paired with SRDF/A remote standby Data Mover


Concepts

• Local standby Data Mover paired with SRDF/A remote standby Data Mover

• Local production Data Mover (non-SRDF/A)

• Local standby Data Mover (non-SRDF/A)

On the remote (destination) VNX:

• SRDF/A standby Data Mover

• Local production Data Mover (non-SRDF/A)

• Local standby Data Mover (non-SRDF/A)

VNX Data Mover configuration checklist

Before performing any VNX procedures to establish SRDF/A, to ensure proper Data Moverconfiguration:

List which Data Movers to designate as SRDF/A primary and SRDF/A standby DataMovers. This is a one-to-one failover relationship. A Data Mover can be an SRDF/Astandby for only one primary Data Mover, and an SRDF/A primary Data Mover can haveonly one SRDF/A standby. In the initialization procedure, you designate and assign thesefailover relationships. Note that by default, Data Movers are referred to as servers andare named server_n, starting with server_2.

Do not attempt to configure a local Data Mover as an SRDF/A standby if that Data Moveris currently configured as a local standby. For an NS series gateway server active/passiveconfiguration with two Data Movers established for high availability, remove the defaultlocal standby status of the destination VNX server’s local Data Mover (for example,server_3) before configuring that Data Mover as an SRDF/A standby. Configure the DataMover as an SRDF/A standby during the initialization process on the destination VNXserver, for example:

server_standby server_2 -delete mover=server_3

server_setup server_3 -type nas

In the CNS-14 (14-Data Mover) cabinet configuration shown in this technical module,there is no default local standby, so you can proceed with an SRDF/A standby assignmentduring the initialization process. Initialize the destination VNX on page 49 provides moreinformation about initialization:

• Ensure that the Data Movers on the source and destination VNX systems are of asimilar model type, that is, the local SRDF/A Data Mover and its corresponding remotestandby Data Mover should be the same model or a supported superset. In addition,the source and destination cabinets must be a similar model type. For example, theyshould both be NS series gateway cabinets or CNS-14 cabinets, but not a mix of NSseries gateway and CNS-14 cabinets.


Concepts

• Consider IP data-network connectivity issues when planning the Data Moverassignments.

• Ensure that network interfaces for the SRDF/A primary and SRDF/A standby DataMovers are identical, and the same set of network clients can access the SRDF/Aprimary and the SRDF/A standby Data Movers.

• Ensure that each local standby Data Mover, providing local standby coverage for anSRDF-protected Data Mover, has a corresponding remote SRDF/A standby. This isa requirement for active/passive configurations.

• Evaluate the destination site’s infrastructure: Subnet addresses as well as availabilityof NIS/DNS servers in the correct UNIX domain, WINS/PDC/BDC/DC in the correctWindows domain, and NTmigrate or usermapper hosts.The CIFS environment requiresmore preparation to set up an SRDF configuration, due to higher demands on itsinfrastructure than the NFS environment. For example, authentication is handled bythe infrastructure against client OS. For the CIFS environment, perform mappingsbetween usernames/groups and UIDs/GIDs.

Note: If the VNX environment is configured with SRDF protection, and you plan to create checkpointsof a production file system that resides on an SRDF volume, ensure that the entire SnapSure volume(the SavVol, which stores the checkpoints) resides in the same pool of SRDF volumes used to createthe production file system. Otherwise, if any part of the SavVol is stored on a local standard (STD)volume rather than on an SRDF volume, the checkpoints are not failed over and not recoverable inthe SRDF-failback process.

Consideration when using applications that require transactionalconsistency

When using applications such as Oracle, which require transactional consistency, failure touse the forcedirectio NFS client mount option could result in incorrect data on SRDF orTimeFinder copies.

The use of client-side NFS write buffering can result in missing data in the point-in-timecopies made by SRDF or TimeFinder. To ensure that any point-in-time copies containconsistent data, ensure that you use the correct client-side NFS mount options to disableNFS client-side buffering and caching for any application that requires transactionalconsistency, such as Oracle or any other DBMS. The EMC Business Continuity for OracleDatabase 11g — Enabled by EMC Celerra using DNFS and NFS Proven Solution Guide,available on http://powerlink.emc.com, provides a summary of all best practice NFS mountoptions for Oracle. In addition to the NFS mount options discussed in this document, Solarisand HP-UX users should include the “forcedirectio” mount option for Oracle data volumesto avoid write buffering. Do not use forcedirectio on NFS volumes that contain executablesor other non-DBMS data. Linux users should include the init.ora option “filesystemio_options= DIRECTIO” to avoid NFS client-side write buffering.


Concepts

http://powerlink.emc.com

Consideration when using applications that can switch to the NFS copyfrom the R2 without a restart

Applications such as Oracle can switch to the NFS copy from the R2 without a restart.Because of this, crash recovery does not happen, and data could become corrupt due tomissing dependent writes. Some of the reasons why this happens are:

Upon failover, Site B or Site C Data Movers will go online using the same IP and MACaddress as the R1.

NFS mount options “hard” and “nointr” are recommended for Oracle and other suchapplications. These two options cause an NFS client to hold IO and wait indefinitely ifconnectivity is lost to the NFS server.

If the Site A and Site B/Site C VNX gateway systems are connected to the same networksegment, NFS clients will automatically find the R2 server without an application restart.

In an unplanned failover, Site B/Site C could be at an earlier point-in-time than the SiteA. For example, the SRDF link was not working first, and Site A still had write I/Os.Then,failover was done without propagating the data from Site A due to the unavailable SRDFlink.

To resolve this issue, a warning message appears, which reminds users to restart theirapplications. This warning message appears if the RDF link is partitioned during the SRDFactivation, which could mean Site B/Site C is behind Site A.

The warning message in nas_rdf –activate when RDF link is in a partitioned state for possibledata corruption:

!!!WARNING!!!Some or all devices in the SRDF device group are in a partitioned state!If you continue with the failover operation, it may result in data loss!EMC recommends that you:

Fix the partitioned state and ensure that the device group is synchronized,or is in a consistent state before continuing with the failover operation.

Or

In a true Disaster Recovery scenario, where siteA is unavailable andsynchronizing operation is not possible, continue with the failover operationand resume or restart your application by using the destination data.

If the partitioned state is in a recovery leg of 3 sites configuration,this message can be ignored, but it still needs to be solved in order tokeep the 3 sites configuration effective.

Do you wish to continue? [yes or no]:

Mitigating factors for this issue are:


Concepts

It is unusual for the customer to use the same layer-2 network segment for Site A andSite B/Site C. However, in a SRDF/Star environment, it may be more commonplacebetween Site A and Site B.

Non-Transactional, Generic “write a file, read a file” style NFS access should be minimallyexposed.

Upgrading the VNX SRDF environment

Local EMC personnel can help you determine the upgrade option that best suits yourenvironment, and perform the procedures to upgrade the VNX systems in an SRDFenvironment to version 5.6 software for SRDF/A.

Because you have remotely protected data, the procedure enables you to restore from thedestination volumes as a backout option, if needed.

To upgrade SRDF/A (or SRDF/S):

1. Halt all source Data Movers and shut down the Control Station.

2. Ensure that the source and destination Symmetrix DMX VNX volumes are synchronized.

3. Halt the SRDF links to ensure that the appropriate device groups are suspended.

4. Restart the source Control Station and Data Movers.

5. Perform the upgrade on the source site.

6. Test or verify the upgrade.

7. Resume the SRDF links. For SRDF/A, ensure that the devices in the appropriate devicegroups are in a Consistent state.

You can now upgrade the destination by using steps 1 through 7.You must upgrade thedestination VNX immediately after you upgrade the source VNX.

Note: The appropriate device group state for SRDF/A is Consistent. If it is not, consult the local EMCpersonnel or local EMC Service Provider to ensure that the SRDF/A device groups are establishedand that the devices in those groups are in a Consistent state.

SRDF/A task overview

Table 4 on page 35 provides an overview of the basic SRDF/A tasks to establishactive/passive SRDF/A on a source and destination VNX and their associated commands.When the activate and restore tasks are run, automatic, internal SRDF health checks areperformed before activating a failover and before restoring source and destination VNX


Concepts

systems, respectively. Adding the –nocheck option to the activate and restore commandsallows you to skip this health check. For example:

# /nas/sbin/nas_rdf –activate –nocheck

or

# /nas/sbin/nas_rdf –restore -nocheck

Table 5 on page 39 provides details of the health checks run by each command. Prior toperforming any of these tasks, review the requirements summarized in Symmetrix systemconfiguration on page 29 to ensure that the Symmetrix DMX systems and VNX Data Moversare configured correctly. In general, the local EMC Customer Support Representative isavailable to help you set up and manage the SRDF/A configuration. Chapter 4 describeshow to gather information and resolve problems associated with SRDF/A configurations.

Table 4. SRDF/A task overview

DescriptionCommand usedTask

Establishes trusted communication be-tween source and destination VNX sys-tems as a prerequisite to SRDF/A initial-ization. Must be performed on both VNXsystems, using the same passphrase(6–15 characters).

The nas_cel command replaces one ofthe nas_rdf -init commands from theprevious versions.

From each VNX, using nasadmin, log inas root:

nas_cel-create<cel_name>

-ip<ip>-passphrase

<passphrase>

Preinitialize the relationship be-tween the source and destinationVNX.

SRDF/A task overview 35

Concepts

Table 4. SRDF/A task overview (continued)


If all SRDF volumes are in a Consistentstate, enables the designated destinationVNX to provide full file system accessand functionality if a site failure occurs.

Configures the Control Station to useSRDF/A.

Identifies the remote destination VNXpaired with the source VNX.

Identifies the volume mapping on theSymmetrix DMX system, which maps R1volumes to their R2 counterparts.

Establishes the Data Mover relationshipsfrom the production Data Mover to thestandby.

Runs the SRDF session state check be-fore initialization.

Runs the SRDF standby Data Moverconfiguration check and the Symmetrixdevice state check after initialization.

Runs the device group configurationcheck and the Data Mover mirrored de-vice accessibility check for both R1 andR2 device groups after initialization.

From source and destination, as root:

/nas/sbin/nas_rdf

-init

Initialize an SRDF/A relationshipbetween attached VNX/Sym-metrix pairs.


Concepts



Performs a test failover or manualfailover. For example, a source VNX at-tached to a source Symmetrix systembecomes unavailable. After the failover,users have access to the same file sys-tems using the same network addressesas they did on the source, provided theyhave network access to the remote VNXsite.

Sets each R1 volume on the source VNX(the one failing over) as read-only. Thisonly occurs if the source Symmetrix sys-tem is available.

Sets each R2 volume on the remoteSymmetrix as read/write.

Enables each SRDF/A standby DataMover on the remote Symmetrix to ac-quire the following characteristics of itssource counterpart:

Network identity: IP and MAC ad-dresses of all network interfacecards (NICs) in the failed DataMover.

Service identity: Network File Sys-tem/ Common Internet File Service(NFS/CIFS) characteristics of theexported file system controlled bythe failed Data Mover.

Runs the SRDF standby Data Moverconfiguration check, the SRDF sessionstate check, and the Symmetrix devicestate check before activating a failover.

Runs the device group configurationcheck and the Data Mover mirrored de-vice accessibility check for R2 devicegroups before activating a failover.

From destination, as root:

/nas/sbin/nas_rdf

-activate

Activate an SRDF/A failover froma source VNX to a destinationVNX.

SRDF/A task overview 37

Concepts



Typically scheduled by and performedunder the guidance of your local EMCCustomer Support Representative toensure continuity between the Symmetrixsystems. Restoration of a source VNXinvolves a complete check of the Sym-metrix system and SRDF/A and verifica-tion of full connectivity to the restored filesystems on the source VNX.

Copies data from R2 volumes to thecorresponding R1 volume on the sourceSymmetrix system.

Restarts SRDF standby Data Movers intostandby mode.

Write-disables R2 volumes from the DataMovers.

Synchronizes R2 to R1.

Resumes mirroring of the R1 devices.

Restarts each Data Mover on the sourceVNX and reacquires the IP addressesand file system control from the SRDFstandby Data Movers.

Runs the SRDF session state check, theSymmetrix device state check, and theSRDF restored state check beforerestoring a destination VNX.

Runs the device group configurationcheck and the Data Mover mirrored de-vice accessibility check for R2 devicegroups before restoring a destinationVNX.

Runs the Symmetrix device state checkbefore restoring a source VNX.

Runs the device group configurationcheck and the Data Mover mirrored de-vice accessibility check for R1 devicegroups before restoring a source VNX.

From destination, as root:

/nas/sbin/nas_rdf

-restore

Restore a source VNX after afailover.


Concepts

Table 5. Health check details

DescriptionHealth check

Lists all slot IDs of SRDF source Data Movers on the sourceside, and compares the list with the list of all slot IDs of SRDFstandby Data Movers on the destination side. If a slot from thesource side is not an RDF standby Data Mover on the destina-tion side, a warning message with the slot ID is displayed.

SRDF standby Data Moverconfiguration check

Checks if all devices are synchronized. If they are not synchro-nized, and if they are in various states, a warning message isdisplayed. If none of them is synchronized, but all of them arein the same state, for example, all of them are consistent orpartitioned, and so on, no warning message is displayed.Whenthe nas_rdf –init command is run, if all the devices are parti-tioned, a warning message is displayed.

SRDF session state check

Lists all mirrored devices of an RDF group on backend (symrdflist) and compares the list with the list of devices in a devicegroup created by using the symdg (symrdf –g <device group>query) command. If there is a missing device in the devicegroup, a warning message is displayed.

Device group configurationcheck

Lists all devices in a device group, and compares the list withthe list of devices obtained by using the server_devconfig–probe –scsi –all command. If there is a device in the devicegroup that is not present in the output of the probe command,a warning message is displayed. Some of control volumes donot have TID/LUN depending on their configuration, and theseare ignored in the check.

Data Mover mirrored deviceaccessibility check

Checks if there is a degraded or failed state device on the backend by using the symdev list –service_state notnormal com-mand. If there is a degraded or failed state device, a warningmessage is displayed. All degraded or failed devices on theSymmetrix, including the devices that are not used for VNX,are listed in the message.

Symmetrix device statecheck

Checks and displays an error message if the source volumesare R1 and ReadWrite, and if the destination volumes are R2and ReadOnly. It does this by using the symdg show command.

SRDF restored state check

Overview of the sample configuration

To illustrate these steps using a sample configuration, they are performed on the VNXsystems as follows:

cs100_src serves as the source VNX. It resides in the production data center.

Overview of the sample configuration 39

Concepts

cs110_dst serves as the destination VNX. It resides in a remote disaster-recovery datacenter.

Note: In the example used for illustrating the configuration steps, cs100_src serves as the source VNXthat resides in the production data center, and cs110_dst serves as the destination VNX that residesin a remote disaster-recovery data center.


Concepts

3

Configuring

The tasks to configure SRDF/A are:

Preinitialize the configuration on page 42 Initialize the configuration (active/passive) on page 45 Activate a failover (active/passive) on page 56 Restore the source VNX on page 69


Preinitialize the configuration

As part of establishing an SRDF/A configuration, the local EMC Service Provider preinitializesthe VNX systems. Preinitialization establishes a trusted communication between the sourceand destination VNX systems.

Prerequisites

Preinitialization must be performed on both the VNX systems.

The source and destination Control Station system times must be within 10 minutesof each other.

Preinitialization must be performed by using the same 6–15 character passphrasefor both the VNX systems. An example is nasadmin.

To preinitialize, the user must log in to the VNX as nasadmin.

The nas_cel command must be run with the -create option as nasadmin on both theVNX systems. Systems using pre-5.6 versions of the software used the nas_rdf -initcommand for preinitialization.Versions 5.6 and later use the nas_cel -create command.

The preinitialization tasks are performed only once, after which the servers becomeready for the SRDF/A initialization procedures.

The VNX systems can be set up and in production prior to initialization.

Preinitialization is a prerequisite to the SRDF/A initialization.

You can preinitialize the SRDF/A configuration by using IPv4 or IPv6 IP addresses.Whiledoing this, make sure that the corresponding destination Control Station uses the sameIPv4/IPv6 network protocol as the source Control Station. An IPv4 Control Station cannotconnect to an IPv6 Control Station. Similarly, an IPv6 Control Station cannot connect toan IPv4 Control Station. The Control Stations on both sides must use the same versionof the protocol.

Verify the preinitialization once after preinitializing from the source side and then afterpreinitializing from the destination side.

Note: Before performing any of the following tasks, review the requirements summarized in Planningconsiderations on page 28 to ensure that the Symmetrix systems and VNX Data Movers areconfigured correctly. In general, the local EMC Customer Support Representative is available tohelp you set up and manage any aspect of the SRDF/A configuration.

To preinitialize the configuration:

1. Preinitialize from the source (first) VNX on page 43

2. Preinitialize from the destination (second) VNX on page 43

3. Verify the preinitialization on page 44


Configuring

Preinitialize from the source (first) VNX

ActionStep

Log in to the source VNX (cs100_src) as nasadmin.1.

Preinitialize the connection from the source (first) VNX to the destination (second) VNX in a VNX/Symmetrixconfiguration for SRDF/A by using this command syntax:

2.

$ nas_cel -create <cel_name> -ip <ip> -passphrase <passphrase>

where:

<cel_name> = name of the destination VNX

<ip> = IP address of the destination Control Station in slot 0

<passphrase> = 6–15 character password

Example:

To preinitialize the connection from cs100_src to cs110_dst with IP address 192.168.97.141 and passphrasenasadmin, type:

$ nas_cel -create cs110_dst -ip 192.168.97.141 -passphrase nasadmin

Output:

operation in progress (not interruptible)...id = 1name = cs110_dstowner = 0device =channel =net_path = 192.168.97.141celerra_id = 0001901005570354passphrase = nasadmin

Exit by typing:3.

$ exit

Output:

exit

Preinitialize from the destination (second) VNX

ActionStep

Log in to the destination VNX (cs100_dst) as nasadmin.1.

Preinitialize the configuration 43

Configuring

ActionStep

Preinitialize the connection from the destination (second) VNX to the source (first) VNX in a VNX/Symmetrixconfiguration for SRDF/A by using this command syntax:

2.

$ nas_cel -create <cel_name> -ip <ip> -passphrase <passphrase>

where:

<cel_name> = name of the source VNX

<ip> = IP address of the remote Control Station in slot 0

<passphrase> = 6–15 character password

Example:

To preinitialize the connection from cs110_dst to cs100_src with IP address 192.168.97.140 and passphrasenasadmin, type:

$ nas_cel -create cs100_src -ip 192.168.97.140 -passphrase nasadmin

operation in progress (not interruptible)...id = 0name = cs100_srcowner = 0device =channel =net_path = 192.168.97.140celerra_id = 000190100582034Dpassphrase = nasadmin

Exit by typing:3.

$ exit

Output:

exit

Verify the preinitialization

ActionStep


Verify preinitialization of cs100_src and cs110_dst by typing:2.

$ nas_cel -list

Output:

id name owner mount_dev channel net_path CMU0 cs100_src 0 192.168.97.140 000190100582034D1 cs110_dst 0 192.168.97.141 0001901005570354


Configuring

Note:

The id is always 0 for the system from which you are running the command.

You do not have to run nas_cel -list as root (or nas_cel -info).You must run nas_cel as root forcreate, modify, update, or delete operations.

Initialize the configuration (active/passive)

Initializing an active/passive configuration establishes one VNX to serve as the source(production) file server and another to serve as the destination.

Prerequisites

VNX software must be installed on the source and destination VNX systems.

SRDF/A link must be operational between the source and destination VNX systems.

The requirements summarized in Symmetrix system configuration on page 29 mustbe met to ensure that the Symmetrix systems and VNX Data Movers are configuredcorrectly.

All SRDF/A volumes must be in a consistent state.Your local EMC Customer SupportRepresentative can help ensure volume consistency.

To initialize an active/passive configuration:

Initialize the source VNX on page 45

Initialize the destination VNX on page 49

Verify SRDF/A on the source VNX (active/passive) on page 53

Initialize the source VNX

In an active/passive configuration, establish the source VNX (active site) first.

ActionStep


Initialize the configuration (active/passive) 45

Configuring

ActionStep

Verify the slots on the source VNX by typing:2.

# /nas/sbin/getreason

Output:

10 - slot_0 primary control station5 - slot_2 contacted5 - slot_3 contacted5 - slot_4 contacted5 - slot_5 contacted

List the Data Movers (servers) on the source VNX by typing:3.

# /nas/bin/nas_server -list

Output:

id type acl slot groupID state name1 1 1000 2 0 server_22 4 1000 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5

Get detailed information about the Data Movers on the source VNX by typing:4.

# nas_server -info -all

Output:

id = 1name = server_2acl = 1000, owner=nasadmin, ID=201type = nasslot = 2member_of =standby = server_3, policy=autostatus :defined = enabledactual = online, active

id = 2name = server_3acl = 1000, owner=nasadmin, ID=201type = standbyslot = 3member_of =standbyfor= server_2status :defined = enabledactual = online, ready


Configuring

ActionStep

id = 3name = server_4acl = 1000, owner=nasadmin, ID=201type = nasslot = 4member_of =standby = server_5, policy=autostatus :defined = enabledactual = online, ready


Switch (su) to root by typing:5.

$ su

Output:

Password:

Start the active/passive SRDF/A initialization process on the source VNX by typing:6.

# /nas/sbin/nas_rdf -init

Output:

Discover local storage devices ...Discovering storage (may take several minutes)doneStart R2 dos client...doneStart R2 nas client...doneContact cs110_dst... is alive


Configuring

ActionStep

At the prompt, assign the remote standby Data Mover on the destination VNX for each source (primary) DataMover on the source VNX. For each source Data Mover on cs100_src, type the slot number of its correspondingstandby on cs110_dst.

7.

Note: The actual standby designation occurs later in the procedure.To determine which Data Movers to designateas standbys, see VNX Data Mover configuration checklist on page 31. Data Movers are referred to as servers,and named server_n, starting with server_2. In the example, server_2 (source Data Mover) and server_3 (localstandby Data Mover) have remote SRDF/A standby Data Movers, that is, local server_2, slot 2, fails over to theremote server in slot 2, and local server_3, slot 3, fails over to the remote server in slot 3. Also, server_4 andserver_5 are local Data Movers not protected by SRDF (not configured with SRDF/A standby Data Movers).

Example:

Please create an rdf standby for each server listedserver server_2 in slot 2, remote standby in slot [2] (or none): 2server_2 : done

id = 1name = server_2acl = 1000, owner=nasadmin, ID=201type = nasslot = 2member_of =standby = server_3, policy=autoRDFstandby= slot=2status :defined = enabledactual = online, activeserver server_3 in slot 3, remote standby in slot [3] (or none): 3server_3 : doneid = 2name = server_3acl = 1000, owner=nasadmin, ID=201type = standbyslot = 3member_of =standbyfor= server_2RDFstandby= slot=3status :defined = enabledactual = online, ready

server server_4 in slot 4, remote standby in slot [4] (or none):noneserver server_5 in slot 5, remote standby in slot [5] (or none):none

The slot [ 2 3 ] on the remote cs110_dst system must be an rdfstandby Data Mover(s). Currently, the slot [ 2 3 ] is not configuredas an rdf standby Data Mover(s). Please run the nas_rdf -initcommand on the remote cs110_dst system.


Configuring

Initialize the destination VNX

ActionStep

Log in to the destination VNX (cs110_dst) as nasadmin.1.

Verify the slots on the destination VNX by typing:2.

# /nas/sbin/getreason

Output:


List the Data Movers (servers) on the destination VNX by typing:3.

# /nas/bin/nas_server -list

Output:


Note: The output indicates that server_2 and server_3 are free to be configured as SRDF/A standbys for thesource VNX (cs100_src), because they are not local standbys.

Get detailed information about the destination Data Movers on the destination VNX by typing:4.

# nas_server -info -all

Output:

id = 1name = server_2acl = 1000, owner=nasadmin, ID=201type = nasslot = 2member_of =standby =status :defined = enabledactual = online, ready


Configuring

ActionStep





$ su

Output:

Password:

Initialize the destination VNX for active/passive SRDF/A by typing:6.

# /nas/sbin/nas_rdf -init

Output:

Discover local storage devices ...Discovering storage (may take several minutes)done

Start R2 dos client...doneStart R2 nas client...done

Contact cs100_src ... is alive


Configuring

ActionStep

At the prompt, create the RDF administrative account (rdfadmin) on the Control Station.

Note: Use the rdfadmin account to manage the destination SRDF/A standby Data Movers on cs110_dst thatprovide the failover capability for cs100_src. The RDF site passphrase must consist of 6–15 characters, andbe the same for the source and destination VNX systems.

Example:

Create a new login account to manage the RDF site CELERRACaution: For an active-active configuration, avoid using the sameUID that was used for the rdfadmin account on the other side.New login username and UID (example: rdfadmin:500): rdfadmin:500doneNew UNIX password:BAD PASSWORD: it is based on a dictionary wordRetype new UNIX password:Changing password for user rdfadmin.passwd: all authentication tokens updated successfully.doneoperation in progress (not interruptible)...id = 1name = cs100_srcowner = 500device = /dev/ndj1channel = rdev=/dev/ndg, off_MB=391; wdev=/dev/nda, off_MB=391net_path = 192.168.97.140celerra_id = 000190100582034Dpassphrase = rdfadminDiscover remote storage devices ...doneThe following servers have been detected on the system:id type acl slot groupID state name1 1 1000 2 0 server_22 1 1000 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5

Note: The IDs of the Data Movers in the id column are used in the next step.

7.


Configuring

ActionStep

At the prompt, designate the Data Movers on the destination VNX as SRDF/A standby Data Movers for thesource VNX. To designate the standby Data Movers, type the ID number (not the slot number) of a single DataMover or ID numbers for multiple servers, separated by spaces.

8.

Note: For failover capability, each standby Data Mover previously assigned to a source (primary) Data Moveron cs100_src must be designated as a standby on cs110_dst. Review the Data Mover list created in VNX DataMover configuration checklist on page 31 to determine which servers to designate as standbys. Ensure thatyou review the checklist if you have an NS series gateway configuration with two Data Movers, and a defaultlocal standby Data Mover. During the source VNX initialization process, the term "slots" identifies the remoteSRDF/A standby Data Mover. During the destination VNX initialization process, the term "server_ids" identifiesthe SRDF/A standby Data Mover.

CAUTION Before you set up an SRDF standby Data Mover at the destination site, check the DataMover configurations at the source and destination sites. Verify that the SRDF standby Data Mover isin the appropriate slot. Verify that the hardware, including network interface adapters, is the same forthe source and SRDF standby Data Movers for failover purposes. Also, ensure that the destinationData Movers do not have any local standbys defined, as provided in the default installation, for example,server_5 is, by default, a standby for server_2, server_3, and server_4.

Example:

Please enter the id(s) of the server(s) you wish to reserve(separated by spaces) or "none" for no servers.Select server(s) to use as standby: 1 2id = 1name = server_2acl = 2000, owner=rdfadmin, ID=500type = standbyslot = 2member_of =standbyfor=status :defined = enabledactual = boot_level=0

id = 2name = server_3acl = 2000, owner=rdfadmin, ID=500type = standbyslot = 3member_of =standbyfor=status :defined = enabledactual = boot_level=0operation in progress (not interruptible)...


Configuring

ActionStep

id = 1name = cs100_srcowner = 500device = /dev/ndj1channel = rdev=/dev/ndg, off_MB=391; wdev=/dev/nda, off_MB=391net_path = 192.168.97.140celerra_id = 000190100582034Dpassphrase = rdfadminPlease create a rdf standby for each server listedserver server_5 in slot 5, remote standby in slot [5] (or none):noneserver server_4 in slot 4, remote standby in slot [4] (or none):none

Note: If you use the nas_server -list command to list the available Data Movers on the destination VNX(cs110_dst), only server_4 and server_5 appear. Data Movers server_2 and server_3 are unavailable to thenasadmin account because they are now managed by the rdfadmin account.

Exit root by typing:9.

# exit

Output:

exit

Note: The initialization process is complete and the active/passive configuration is established.The destinationVNX is ready to provide full file system access and functionality if a source site failure occurs.

Important: If the IP address of the Control Station changes after the initialization process begins, rerunthe /nas/sbin/nas_rdf -init command to accommodate the change. If you change any hostnames or IPaddresses and want to accommodate the change before you run the initialization process, edit andupdate the /etc/hosts file and ensure that each host can resolve its node name.

Verify SRDF/A on the source VNX (active/passive)

After initialization, run VNX CLI commands, informational Solutions Enabler Symmetrix CLI(SYMCLI) commands, or both to verify SRDF/A on the source VNX.

ActionStep



Configuring

ActionStep

List the file systems on the source VNX by typing:2.

$ nas_fs -list

Output

id inuse type acl volume name server1 n 1 0 74 root_fs_12 y 1 0 76 root_fs_2 13 y 1 0 78 root_fs_3 24 y 1 0 80 root_fs_4 35 y 1 0 82 root_fs_5 46 n 1 0 84 root_fs_67 n 1 0 86 root_fs_78 n 1 0 88 root_fs_89 n 1 0 90 root_fs_910 n 1 0 92 root_fs_1011 n 1 0 94 root_fs_1112 n 1 0 96 root_fs_1213 n 1 0 98 root_fs_1314 n 1 0 100 root_fs_1415 n 1 0 102 root_fs_1516 y 1 0 104 root_fs_common 2,1,317 n 5 0 137 root_fs_ufslog18 n 5 0 140 root_fs_d319 n 5 0 141 root_fs_d420 n 5 0 142 root_fs_d521 n 5 0 143 root_fs_d622 n 1 0 156 fs123 y 1 0 157 fs2 124 n 1 0 161 fs8k25 n 1 0 162 fs32k26 n 1 0 163 fs64k27 n 1 0 164 ufs128 n 5 0 165 root_panic_reserve29 n 1 0 229 ufs1_snap131 y 1 0 233 fs1a 132 y 1 0 234 fst1 135 y 1 0 292 fs5 3


Configuring

ActionStep

List the device groups on the source VNX by typing:3.

$ /nas/symcli/bin/symdg list

Output

D E V I C E G R O U P SNumber of

Name Type Valid Symmetrix ID Devs GKs BCVs VDEVs TGTs1R1_3 RDF1 Yes 000190100582 70 0 0 0 01REG REGULAR Yes 000190100582 48 0 64 0 0

Note:

In the sample configuration, these device groups represent the following: 1R1_3 representsthe SRDF/A-protected R1 data volumes and 1REG represents the local (non-SRDF/A)volumes. The device groups in the configuration might have different names.

Use only Solutions Enabler Symmetrix SYMCLI informational commands on the VNXdevice group; do not invoke any Solutions Enabler Symmetrix SYMCLI action commandsusing the Control Station host component, laptop service processor, or EMC ControlCen-ter. The Solutions Enabler Symmetrix SRDF family documentation on the EMC OnlineSupport website provides more information.

View key SRDF/A information for the device group representing the R1 data volumes (for example, 1R1_3) bytyping:

4.

$ /nas/symcli/bin/symrdf -g 1R1_3 query -rdfa

Note: Ensure that you provide the appropriate device group name with the command. Information you canmonitor is highlighted in the output. Also, the output uses “... ? to indicate a continuation of the device entries;not all entries are listed.

Output

Device Group (DG) Name : 1R1_3DG's Type : RDF1DG's Symmetrix ID : 000190100582RDFA Session Number : 2RDFA Cycle Number : 139RDFA Session Status : ActiveRDFA Minimum Cycle Time : 00:00:30RDFA Avg Cycle Time : 00:00:30Duration of Last cycle : 00:00:30RDFA Session Priority : 33Tracks not Committed to the R2 Side: 36Time that R2 is behind R1 : 00:00:36RDFA R1 Side Percent Cache In Use : 0RDFA R2 Side Percent Cache In Use : 0Transmit Idle Time : 00:00:00Source (R1) View Target (R2) View MODES


Configuring



ActionStep

--------------------------- -------------------- ---- --------ST LI ST

Standard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDAC STATE--------------------------- -- ------------------- ---- ---------d1 034D RW 0 0 RW 034D WD 0 0 A... Consistentd2 034E RW 0 0 RW 034E WD 0 0 A... Consistentd3 034F RW 0 0 RW 034F WD 0 0 A... Consistentd4 0350 RW 0 0 RW 0350 WD 0 0 A... Consistentd5 0351 RW 0 0 RW 0351 WD 0 0 A... Consistentd6 0352 RW 0 0 RW 0352 WD 0 0 A... Consistentd7 035A RW 0 0 RW 035A WD 0 0 A... Consistentd8 035B RW 0 0 RW 035B WD 0 0 A... Consistentd9 035C RW 0 0 RW 035C WD 0 0 A... Consistentd10 035D RW 0 0 RW 035D WD 0 0 A... Consistent...d138 0397 RW 0 0 RW 0397 WD 0 0 A... Consistentd139 0398 RW 0 0 RW 0398 WD 0 0 A... Consistentd140 0399 RW 0 0 RW 0399 WD 0 0 A... Consistent

Total -------- ----- ------ ------Track(s) 0 0 0 0MB(s) 0.0 0.0 0.0 0.0Legend for MODES:

M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopyD(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp offC(onsistency State): X = Enabled, . = Disabled, - = N/A

Note: Check the session status, average cycle time, tracks not committed to R2, time R2 is behind R1 (in sec-onds), as well as the mode (A for asynchronous) and RDF pair state (Consistent).

Activate a failover (active/passive)

Activating a failover enables the destination VNX to assume the source (primary) role in theactive/passive SRDF/A configuration.

Prerequisites

You might decide to perform a planned (graceful) failover for testing purposes, whichinvolves preparation before the failover is activated, or you might need to activate anunplanned failover in response to a disaster scenario.

In an unplanned failover scenario, assume that the source VNX attached to a sourceSymmetrix system is unavailable, and requires a failover to the destination VNX.Regardless of the planned or unplanned failover scenario, you always activate a failoverby running the /nas/sbin/nas_rdf -activate command from the destination VNX.


Configuring

To activate an active/passive SRDF/A failover:

1. Prepare for a graceful failover on page 57

2. Activate a failover from the destination VNX on page 59

3. Verify SRDF/A after failover activation on page 63

4. Ensure access after failover on page 69

CAUTION For sites with redundant Control Stations, ensure that all SRDF managementcommands including /nas/sbin/nas_rdf -init, -activate, and -restore are run from the primaryControl Station located in slot 0 (CS0). Ensure that CS1 is powered off at both sites beforeyou run any -activate or -restore commands. When the CS1 shutdown process completes,type /nas/sbin/getreason and check the output to verify the shutdown. The output shouldcontain the line 0 - slot_1 powered off.

Prepare for a graceful failover

ActionStep

Log in to the source VNX (cs100_src) as nasadmin and switch (su) to root.1.

Halt all Data Movers and restart the source Control Station (CS0) for the source VNX by typing:2.

# /nas/sbin/nas_halt now

Output:

************************** WARNING! ****************************You are about to HALT this Celerra including all of its ControlStations and Data Movers.DATA will be UNAVAILABLE when the system is halted.Note that this command does *not* halt the storage array.

Note: As an alternative to nas_halt, you can run server_cpu ALL -halt now &, to halt all the Data Movers.However, this requires an explicit restart of the source Control Station (CS0) with -reboot -f -n. For an NS600,server_cpu is the recommended command.

Activate a failover (active/passive) 57

Configuring

ActionStep

At the prompt, confirm the halting by typing:3.

yes

Example:

ARE YOU SURE YOU WANT TO CONTINUE? [ yes or no ] : yes

Sending the halt signal to the Master Control Daemon...: DoneFeb 7 18:39:35 cs100_src EMCServer: nas_mcd: Check and halt otherCS...: DoneFeb 7 18:40:00 cs100_src get_datamover_status: Data Mover server_2:COMMAND doesnt match.Feb 7 18:40:00 cs100_src get_datamover_status: Data Mover server_5:COMMAND doesnt match.Feb 7 18:40:01 cs100_src get_datamover_status: Data Mover server_3:COMMAND doesnt match.Feb 7 18:40:01 cs100_src get_datamover_status: Data Mover server_4:COMMAND doesnt match.Feb 7 18:40:12 cs100_src setup_enclosure: Executing -dhcpd stop op-tionFeb 7 18:40:12 cs100_src snmptrapd[4743]: Stopping snmptrapdFeb 7 18:40:12 cs100_src EV_AGENT[7847]: Signal TERM receivedFeb 7 18:40:12 cs100_src EV_AGENT[7847]: Agent is going downFeb 7 18:40:24 cs100_src DHCPDMON: Starting DHCPD on CS 0Feb 7 18:40:26 cs100_src setup_enclosure: Executing -dhcpd startoptionFeb 7 18:40:26 cs100_src dhcpd: Internet Software Consortium DHCPServer V3.0pl1Feb 7 18:40:26 cs100_src dhcpd: Copyright 1995-2001 Internet SoftwareConsortium.Feb 7 18:40:26 cs100_src dhcpd: All rights reserved.Feb 7 18:40:26 cs100_src dhcpd: For info, please visithttp://www.isc.org/products/DHCPFeb 7 18:40:26 cs100_src dhcpd: Wrote 0 deleted host decls to leasesfile.Feb 7 18:40:26 cs100_src dhcpd: Wrote 0 new dynamic host decls toleases file.Feb 7 18:40:26 cs100_src dhcpd: Wrote 0 leases to leases file.Feb 7 18:40:26 cs100_src dhcpd: Listening onLPF/eth2/00:00:f0:9c:1f:d5/128.221.253.0/24Feb 7 18:40:26 cs100_src dhcpd: Sending onLPF/eth2/00:00:f0:9c:1f:d5/128.221.253.0/24Feb 7 18:40:26 cs100_src dhcpd: Listening onLPF/eth0/00:00:f0:9d:00:e2/128.221.252.0/24Feb 7 18:40:26 cs100_src dhcpd: Sending onLPF/eth0/00:00:f0:9d:00:e2/128.221.252.0/24Feb 7 18:40:26 cs100_src dhcpd: Sending on Socket/fallback/fallback-netFeb 7 18:40:40 cs100_src mcd_helper: : Failed to umount /nas (0)Feb 7 18:40:40 cs100_src EMCServer: nas_mcd: Failed to gracefullyshutdown MCD and haltservers. Forcing halt and reboot...Feb 7 18:40:40 cs100_src EMCServer: nas_mcd: Halting all servers...


Configuring

Activate a failover from the destination VNX

Note: In a true disaster scenario where the source Symmetrix DMX system is unavailable, contactyour local EMC Customer Support Representative to coordinate restoration activities.

ActionStep

Log in to the destination VNX (cs110_dst) as nasadmin.

Note: The activation must always be performed from the destination VNX.

1.

List the device groups on the destination VNX by typing:2.

$ /nas/symcli/bin/symdg list

Output:

D E V I C E G R O U P SNumber of

Name Type Valid Symmetrix ID Devs GKs BCVs VDEVs TGTs1R1_4 RDF1 Yes 000190100557 6 0 0 0 01REG REGULAR Yes 000190100557 0 0 62 0 01R2_500_3 RDF2 Yes 000190100557 70 0 0 0 0

Note: In the sample cs110_dst configuration, these device groups represent the following: 1R1_4 representsthe R1 volumes of the destination Control Station, 1REG represents the local (non-SRDF/A) volumes, and1R2_500_3 represents the R2 data volumes.The device groups in the configuration might have different names.These steps apply to both types of failover scenarios (graceful or disaster recovery). However, the sample outputis based on a graceful failover activation.

Query the destination’s SRDF/A information for the device group representing the R2 data volumes (on cs110_dst,1R2_500_3) by typing:

3.

$ /nas/symcli/bin/symrdf -g 1R2_500_3 query -rdfa

Note: The output uses "..." to indicate a continuation of the device entries; not all device entries are listed.

Output:

Device Group (DG) Name : 1R2_500_3DG's Type : RDF2DG's Symmetrix ID : 000190100557RDFA Session Number : 2RDFA Cycle Number : 41


Configuring

ActionStep

RDFA Session Status : ActiveRDFA Minimum Cycle Time : 00:00:30RDFA Avg Cycle Time : 00:00:30Duration of Last cycle : 00:00:30RDFA Session Priority : 33Tracks not Committed to the R2 Side : 1276Time that R2 is behind R1 : 00:00:32RDFA R1 Side Percent Cache In Use : 0RDFA R2 Side Percent Cache In Use : 0Transmit Idle Time : 00:00:00

Target (R2) View Source (R1) View MODES-------------------------- -------------------- ---- --------

ST LI STStandard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDAC STATE-------------------------------- -- -------------- ---- --------DEV001 034D WD 0 0 RW 034D RW 0 0 A... ConsistentDEV002 034E WD 0 0 RW 034E RW 0 0 A... ConsistentDEV003 034F WD 0 0 RW 034F RW 0 0 A... ConsistentDEV004 0350 WD 0 0 RW 0350 RW 0 0 A... ConsistentDEV005 0351 WD 0 0 RW 0351 RW 0 0 A... ConsistentDEV006 0352 WD 0 0 RW 0352 RW 0 0 A... ConsistentDEV007 035A WD 0 0 RW 035A RW 0 0 A... ConsistentDEV008 035B WD 0 0 RW 035B RW 0 0 A... ConsistentDEV009 035C WD 0 0 RW 035C RW 0 0 A... ConsistentDEV010 035D WD 0 0 RW 035D RW 0 0 A... Consistent...DEV067 0396 WD 0 0 RW 0396 RW 0 0 A... ConsistentDEV068 0397 WD 0 0 RW 0397 RW 0 0 A... ConsistentDEV069 0398 WD 0 0 RW 0398 RW 0 0 A... ConsistentDEV070 0399 WD 0 0 RW 0399 RW 0 0 A... ConsistentTotal -------- -------- -------- --------Track(s) 0 0 0 0MB(s) 0.0 0.0 0.0 0.0

Legend for MODES:M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopyD(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp offC(onsistency State): X = Enabled, . = Disabled, - = N/A

Note: The output uses "..." to indicate a continuation of the device entries; not all device entries are listed.

Switch (su) to rdfadmin by typing:4.

$ su - rdfadmin

Output:

Password:


Configuring

ActionStep


$ su

Output:

Password:

Activate the failover by typing:6.

# /nas/sbin/nas_rdf -activate

CAUTION For sites with redundant Control Stations, the /nas/sbin/nas_rdf -activate command must berun from the primary Control Station located in slot 0 (CS0). Ensure that CS1 is powered off at both sitesbefore you run the nas_rdf -activate command, and CS1 remains powered off for the duration of theactivate and subsequent restore operations.

Note: Ensure that all Data Movers at the source are halted before proceeding. At the destination, do not shutdown or restart any Data Movers, that is, SRDF-protected or non-SRDF Data Movers, while the nas_rdf -activateor nas_rdf - restore command is running. This might interrupt the communication of VNX with the backend andcause the command to fail.

At the prompt, if this is a true disaster scenario, ensure that you have powered off the source VNX:

Is remote site cs100_src completely shut down (power OFF)?

Note: This prompt serves as a reminder to shut down the source VNX in a disaster scenario or for failovertesting. The nas_rdf -activate command will automatically shut down the source VNX if the command detectsthat the source is still operational.

7.

At the next prompt, confirm the SRDF/A activation by typing:8.

yes

Example:

Do you wish to continue [yes or no]: yesSuccessfully pinged (Remotely) Symmetrix ID: 000190100582

Note: The ping success message appears if the Symmetrix DMX system attached to the source VNX is suc-cessfully contacted, indicating it is operational and ready on the SRDF/A link. Activating the destination VNXwrite-enables the destination volumes and write-disables the source volumes, as long as SRDF/A and the sourceSymmetrix system are operational.


Configuring

ActionStep

An RDF 'Failover' operation execution is in progress for device group'1R2_500_3'.Please wait...

Write Disable device(s) on SA at source (R1)..............Done.Suspend RDF link(s).......................................Started.Suspend RDF link(s).......................................Done.Suspend RDF link(s).......................................Started.Suspend RDF link(s).......................................Done.Read/Write Enable device(s) on RA at target (R2)..........Done.Suspend RDF link(s).......................................Started.Suspend RDF link(s).......................................Done.

The RDF 'Failover' operation successfully executed for device group'1R2_500_3'.

Note: A file system check is executed on the R2 control volumes of the source’s Control Station file systems.

Waiting for nbs clients to die ... done/net/500 /etc/auto.500 -t 1,ro/dev/ndj1: recovering journal/dev/ndj1: clean, 11591/231360 files, 204230/461860 blocksfsck 1.26 (3-Feb-2002)Waiting for nbs clients to die ... doneid type acl slot groupID state name1 1 1000 2 0 server_22 4 1000 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5

Note: The SRDF/A standby Data Movers now become active.

server_2 :server_2 : going offlinerdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...done

doneserver_3 :server_3 : going offlinerdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...done

doneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured


Configuring

ActionStep


# exit

exit

Note: The failover is complete.

Note: In a true disaster scenario where the source Symmetrix DMX system is unavailable, contactyour local EMC Customer Support Representative to coordinate restoration activities.

Verify SRDF/A after failover activation

ActionStep

Log in to the destination VNX (cs110_dst) as rdfadmin.1.

List the available Data Movers at the destination by typing:2.

$ /nas/bin/nas_server -list

id type acl slot groupID state name1 1 0 2 0 server_22 4 0 3 0 server_3

Note:

This command shows the Data Mover server table with the ID, type, access control level(ACL) value, slot number, group ID, state, and name of the Data Mover.

You can run VNX CLI commands, informational SYMCLI commands, or both to verify theSRDF/A configuration on the write-enabled destination VNX.


Configuring

ActionStep

List and manage the file systems mounted for the Data Movers now active at the destination by typing:3.

$ /nas/bin/server_mount ALL

server_2 :root_fs_2 on / uxfs,perm,rwroot_fs_common on /.etc_common uxfs,perm,rofs2 on /fs2 uxfs,perm,rwfs1a on /fs1a uxfs,perm,rwfst1 on /fst1 uxfs,perm,rw

server_3 :root_fs_3 on / uxfs,perm,rw,<unmounted>root_fs_common on /.etc_common uxfs,perm,ro,<unmounted>

Note: The standby Data Movers activated at the destination acquire the IP and MAC addresses, file systems,and export tables of their counterparts on the source and have read/write access to all the R2 volumes on thedestination.

List interface configuration information for Data Mover server_2 by typing:4.

$ /nas/bin/server_ifconfig server_2 -all

Output:

server_2 :cge1 protocol=IP device=cge1

inet=192.168.97.147 netmask=255.255.255.0 broadcast=192.168.97.255

UP, ethernet, mtu=1500, vlan=0, macaddr=0:60:16:9:32:71loop protocol=IP device=loop

inet=127.0.0.1 netmask=255.0.0.0 broadcast=127.255.255.255UP, loopback, mtu=32768, vlan=0, macaddr=0:0:0:0:0:0 netname=local-

hostel31 protocol=IP device=mge1


UP, ethernet, mtu=1500, vlan=0, macaddr=0:60:16:5:5d:23 netname=lo-calhostel30 protocol=IP device=mge0


UP, ethernet, mtu=1500, vlan=0, macaddr=0:60:16:5:5d:24 netname=lo-calhost


Configuring

ActionStep

Display disk information after the failover activation by typing:5.

$ nas_disk -list

id inuse sizeMB storageID-devID type name servers1 y 11619 000190300281-002C STD root_disk 1,22 y 11619 000190300281-002D STD root_ldisk 1,23 y 2077 000190300281-002E STD d3 1,24 y 2077 000190300281-002F STD d4 1,25 y 2077 000190300281-0030 STD d5 1,26 y 31874 000190300281-0031 STD d6 1,27 n 22999 000190300281-0083 STD d7 1,28 n 22999 000190300281-0084 STD d8 1,29 n 22999 000190300281-0085 STD d9 1,210 n 22999 000190300281-0086 STD d10 1,211 y 22999 000190300281-0087 STD d11 1,212 n 22999 000190300281-0088 STD d12 1,213 n 22999 000190300281-0089 STD d13 1,214 y 22999 000190300281-008A STD d14 1,215 y 22999 000190300281-008B STD d15 1,216 y 22999 000190300281-008C STD d16 1,217 y 22999 000190300281-008D STD d17 1,218 y 22999 000190300281-008E STD d18 1,219 n 22999 000190300281-008F STD d19 1,220 n 22999 000190300281-0090 STD d20 1,2


Configuring

ActionStep

21 n 22999 000190300281-0091 STD d21 1,222 n 22999 000190300281-0092 STD d22 1,223 n 22999 000190300281-0093 STD d23 1,224 n 22999 000190300281-0094 STD d24 1,225 n 22999 000190300281-0095 STD d25 1,226 n 22999 000190300281-0096 STD d26 1,227 n 22999 000190300281-00AB BCV rootd27 1,228 n 22999 000190300281-00AC BCV rootd28 1,229 n 22999 000190300281-00AD BCV rootd29 1,230 n 22999 000190300281-00AE BCV rootd30 1,231 n 22999 000190300281-00AF BCV rootd31 1,232 y 61424 000190300281-00B6 ATA d32 1,233 y 61424 000190300281-00B7 ATA d33 1,234 y 61424 000190300281-00B8 ATA d34 1,235 y 61424 000190300281-00B9 ATA d35 1,236 y 61424 000190300281-00BA ATA d36 1,237 y 61424 000190300281-00BB ATA d37 1,238 y 61424 000190300281-00BC ATA d38 1,239 y 61424 000190300281-00BD ATA d39 1,240 y 61424 000190300281-00BE ATA d40 1,241 y 61424 000190300281-00BF ATA d41 1,242 y 61424 000190300281-00C0 ATA d42 1,243 y 61424 000190300281-00C1 ATA d43 1,244 n 61424 000190300281-00C2 ATA d44 1,245 y 61424 000190300281-00C3 ATA d45 1,246 y 61424 000190300281-00C4 ATA d46 1,247 y 61424 000190300281-00C5 BCVA rootd47 1,248 y 61424 000190300281-00C6 BCVA rootd48 1,249 y 61424 000190300281-00C7 BCVA rootd49 1,250 y 61424 000190300281-00C8 BCVA rootd50 1,251 n 61424 000190300281-00C9 BCVA rootd51 1,252 n 61424 000190300281-00CA R1ATA d52 1,253 n 61424 000190300281-00CB R1ATA d53 1,254 y 61424 000190300281-00CC R1ATA d54 1,255 y 61424 000190300281-00CD R1ATA d55 1,256 y 61424 000190300281-00CE R1ATA d56 1,257 n 61424 000190300281-00D4 R1BCA rootd57 1,258 n 61424 000190300281-00D5 R1BCA rootd58 1,259 n 61424 000190300281-00D6 R1BCA rootd59 1,260 n 61424 000190300281-00D7 R1BCA rootd60 1,261 n 61424 000190300281-00D8 R1BCA rootd61 1,2

Display information for the Data Movers and the file systems at the destination after the failover activation, includingthe available disk space for all Data Movers or file systems and how much of the total capacity of a file systemhas been used by typing:

6.

$ /nas/bin/server_df ALL


Configuring

ActionStep

Output:

[server_2 :server_2 :Filesystem kbytes used avail capacity Mounted onfst1 28798424 600 28797824 0% /fst1fs1a 9224 600 8624 7% /fs1afs2 460787024 600 460786424 0% /fs2root_fs_common 13624 5264 8360 39% /.etc_commonroot_fs_2 114592 776 113816 1% /

server_3 :Error 2: server_3 : No such file or directoryno error message in response

Query the key SRDF/A information now existing for the destination device group with the data volumes (oncs110_dst, 1R2_500_3) by using this command syntax:

7.

$ /nas/symcli/bin/symrdf -g <group> query -rdfa

Where:

<group> = name of the device group

Note: Use only Solutions Enabler Symmetrix SYMCLI informational commands on the VNX device group; donot invoke any Solutions Enabler Symmetrix SYMCLI action commands using the Control Station host component,laptop service processor, or EMC ControlCenter. The Solutions Enabler Symmetrix SRDF family documentationon the EMC Online Support website provides more information.

Note: The session state is now Inactive, the mode is A for asynchronous, and the RDF pair state is now FailedOver. Also note the output uses “... ? to indicate a continuation of the device entries, because not all device entriesare listed.

Example:

$ /nas/symcli/bin/symrdf -g 1R2_500_3 query -rdfa

Output:

Device Group (DG) Name : 1R2_500_3DG's Type : RDF2DG's Symmetrix ID : 000190100557


Configuring


ActionStep

RDFA Session Number : 2RDFA Cycle Number : 0RDFA Session Status : Inactive RDFA Minimum CycleTime : 00:00:30RDFA Avg Cycle Time : 00:00:00Duration of Last cycle :00:00:00RDFA Session Priority : 33Tracks not Committed to the R2 Side: 0 Timethat R2 is behind R1 : 00:00:00RDFA R1 Side Percent Cache In Use : 0 RDFA R2Side Percent Cache In Use : 0Transmit Idle Time : 00:00:00

Target (R2) View Source (R1) View MODES-------------------------- -------------------- ----- --------

ST LI STStandard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDAC STATE-------------------------- -- -------------------- ---- --------DEV001 034D RW 20 0 NR 034D WD 0 0 A... Failed OverDEV002 034E RW 2048 0 NR 034E WD 0 0 A... Failed OverDEV003 034F RW 0 0 NR 034F WD 0 0 A... Failed OverDEV004 0350 RW 0 0 NR 0350 WD 0 0 A... Failed OverDEV005 0351 RW 1204 0 NR 0351 WD 0 0 A... Failed OverDEV006 0352 RW 0 0 NR 0352 WD 0 0 A... Failed OverDEV007 035A RW 0 0 NR 035A WD 0 0 A... Failed OverDEV008 035B RW 0 0 NR 035B WD 0 0 A... Failed OverDEV009 035C RW 0 0 NR 035C WD 0 0 A... Failed OverDEV010 035D RW 0 0 NR 035D WD 0 0 A... Failed Over...DEV060 038F RW 0 0 NR 038F WD 0 0 A... Failed OverDEV061 0390 RW 0 0 NR 0390 WD 0 0 A... Failed OverDEV062 0391 RW 0 0 NR 0391 WD 0 0 A... Failed OverDEV063 0392 RW 0 0 NR 0392 WD 0 0 A... Failed OverDEV064 0393 RW 0 0 NR 0393 WD 0 0 A... Failed OverDEV065 0394 RW 2 0 NR 0394 WD 0 0 A... Failed OverDEV066 0395 RW 2 0 NR 0395 WD 0 0 A... Failed OverDEV067 0396 RW 0 0 NR 0396 WD 0 0 A... Failed OverDEV068 0397 RW 0 0 NR 0397 WD 0 0 A... Failed OverDEV069 0398 RW 0 0 NR 0398 WD 0 0 A... Failed OverDEV070 0399 RW 0 0 NR 0399 WD 0 0 A... Failed OverTotal ------ ----- ------ ----Track(s) 3280 0 0 0MB(s) 102.5 0.0 0.0 0.0

Legend for MODES:

M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopyD(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp offC(onsistency State): X = Enabled, . = Disabled, - = N/A


Configuring

Ensure access after failover

If you have not accounted for different IP subnets at the source and destination sites, performthese steps after a failover to ensure that users have access to the same file systems, usingthe same network addresses as they did on the source-site VNX, provided they have networkaccess to the destination-site VNX.

Note: You can perform these steps either manually or by creating and running a script. If you performthese steps at the destination site after activation, ensure that you perform them at the source siteafter the /nas/sbin/nas_rdf -restore command completes. These steps are required at the source toreturn everything to the original configuration. Restore the source VNX on page 69 provides informationabout the restore procedures.

ActionStep

Halt CIFS.1.

Set IP addresses and default routes.2.

Adjust services such as WINs, DNS, NIS, and NTP.3.

Restart CIFS.4.

Restore the source VNX

A restore of the source VNX is a planned, scheduled event performed by or under theguidance of your local EMC Customer Support Representative to ensure continuity betweenthe Symmetrix DMX systems.

To restore the source VNX:

1. Prepare for the restore on page 70

2. Restore from the destination on page 70

CAUTION For sites with redundant Control Stations, ensure that all SRDF managementcommands including nas_rdf -init, -activate, and -restore are run from the primary Control Stationlocated in slot 0 (CS0). Always ensure that CS1 is powered off at both sites before you run any-activate or -restore commands. When the CS1 shutdown process completes, type/nas/sbin/getreason and check the output to verify the shutdown.The output should contain theline 0 - slot_1 powered off. Because this is a planned event, ensure that you keep CS1 poweredoff for the duration of the event.


Configuring

Prepare for the restore

Note: Perform these steps under the guidance of your local EMC Customer Support Representative.

ActionStep

Verify proper SRDF/A operations by performing a complete system check of the Symmetrix system and SRDF/A.

CAUTION

If this is a true disaster scenario, keep the source VNX (cs100_src) powered offuntil you are instructed to power it up by your local EMC Customer SupportRepresentative.Your local EMC Customer Support Representative ensuresproper operation of the Symmetrix system attached to the source VNX.

Proceed only after confirmation from your EMC Customer Support Representative.

1.

Power up the source VNX, ensuring that the source Control Station restarts, and perform a restart by typing:2.

# reboot -f -n

Note: You can now perform the restore from the destination VNX.

Restore from the destination

The restoration process occurs in two phases:

The device-group update phase provides network clients continued access to thedestination VNX while the destination Symmetrix system updates the source Symmetrixsystem. The length of the update phase is based on the amount of data changed sincethe destination system was activated.

The device-group failback phase is typically short in duration, usually under 20 minutes,and network clients are suspended from file system access while the file systems fullysynchronize from the destination to the source Symmetrix system.

ActionStep

Log in to the destination VNX (cs110_dst) as rdfadmin and switch (su) to root.1.


Configuring

ActionStep

Start the restore of the source VNX by typing:2.

# /nas/sbin/nas_rdf -restore

CAUTION

For sites with redundant Control Stations, ensure that the nas_rdf -restore commandis run from the primary Control Station located in slot 0 (CS0). Always ensure thatCS1 is powered off at both sites before you run the - restore command. When theCS1 shutdown process completes, type /nas/sbin/getreason and check the outputto verify the shutdown. The output should contain the line 0 - slot_1 powered off.Because this is a planned event, ensure that you keep CS1 powered off for theduration of the event.

Do not shut down or restart any Data Movers (SRDF-protected or non-SRDF DataMovers) while the nas_rdf -restore command is running. This might interrupt thecommunication of VNX with the storage system and cause the command to fail.

Proceed only after your EMC Customer Support Representative has verified thatthe source Symmetrix DMX system and SRDF/A link are operational.

At the prompt, to continue restoration and begin the device-group update phase, type:3.

yes

Example:

Is remote site cs100_src ready for Storage restoration?Do you wish to continue [yes or no]: yesContact cs100_src ... is alive

At the next prompt, to continue the restore process, type:4.

yes

Example:

Restore will now reboot the source side control station.Do you wish to continue? [yes or no]: yes

CAUTION The source Symmetrix system must be operational and all volumes must be ready on the link.

Note: The system verifies that the Symmetrix devices are in proper condition to be restored at the source site.Note that the output uses “... ? to indicate a continuation of the device entries, because not all device entries arelisted.


Restore the source VNX 71

Configuring

ActionStep

Target (R2) View Source (R1) View MODES----------------------------- ------------------ --- --------

ST LI STStandard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDA STATE----------------------------- -- ----------------- --- --------DEV001 034D RW 20 0 NR 034D WD 0 0 A.. Failed OverDEV002 034E RW 2048 0 NR 034E WD 0 0 A.. Failed OverDEV003 034F RW 0 0 NR 034F WD 0 0 A.. Failed OverDEV004 0350 RW 0 0 NR 0350 WD 0 0 A.. Failed OverDEV005 0351 RW 1204 0 NR 0351 WD 0 0 A.. Failed OverDEV006 0352 RW 0 0 NR 0352 WD 0 0 A.. Failed OverDEV007 035A RW 0 0 NR 035A WD 0 0 A.. Failed OverDEV008 035B RW 0 0 NR 035B WD 0 0 A.. Failed OverDEV009 035C RW 0 0 NR 035C WD 0 0 A.. Failed OverDEV010 035D RW 0 0 NR 035D WD 0 0 A.. Failed OverDEV011 035E RW 0 0 NR 035E WD 0 0 A.. Failed OverDEV012 035F RW 0 0 NR 035F WD 0 0 A.. Failed Over...DEV065 0394 RW 2 0 NR 0394 WD 0 0 A.. Failed OverDEV066 0395 RW 2 0 NR 0395 WD 0 0 A.. Failed OverDEV067 0396 RW 0 0 NR 0396 WD 0 0 A.. Failed OverDEV068 0397 RW 0 0 NR 0397 WD 0 0 A.. Failed OverDEV069 0398 RW 0 0 NR 0398 WD 0 0 A.. Failed OverDEV070 0399 RW 0 0 NR 0399 WD 0 0 A.. Failed Over

Total ------ ------ ------- ------Track(s) 3280 0 0 0MB(s) 102.5 0.0 0.0 0.0Legend for MODES:M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopyD(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp off

Note:

The update of the source Symmetrix system now begins by automatically setting theSRDF device group mode to SYNC (synchronous) for the update and failback.

The file systems and shares are still available to the network clients during the updatephase. Under certain conditions, volumes might not be in the proper state to fail back, inwhich case the restore command exits. If this occurs, Chapter 4 can provide informationabout the errors that might occur. If necessary, contact your local EMC Customer SupportRepresentative.


Configuring

ActionStep

+++++ Setting RDF group 1R2_500_3 to SYNC mode.An RDF 'Update R1' operation execution is in progress for devicegroup '1R2_500_3'. Please wait...Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Devices: 036B-037C in (0557,03).......................... Merged.Devices: 034D-0352, 035A-036A in (0557,03)............... Merged.Devices: 037D-038E in (0557,03).......................... Merged.Devices: 038F-0399 in (0557,03).......................... Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Started.Resume RDF link(s)........................................Done.The RDF 'Update R1' operation successfully initiated for device group'1R2_500_3'.

Note: The device-group update phase is now complete. The next step begins the device-group failback phase.

At the next prompt, to begin the network restoration phase, type:5.

yes

Note: Ensure that the source CS0 is operational and on the data network.

Example:

Is remote site cs100_src ready for Network restoration?Do you wish to continue [yes or no]: yes

Note: The SRDF/A standby Data Movers are halted and the destination file systems and shares become unavail-able to network clients while the Symmetrix DMX systems (source and destination) fully synchronize.

server_2 : doneserver_3 : doneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured/dev/ndj1: clean, 11595/231360 files, 204256/461860 blocksfsck 1.26 (3-Feb-2002)Waiting for nbs clients to die ... doneWaiting for nbs clients to die ... done/net/500 /etc/auto.500 -t 0,rw,syncWaiting for 1R2_500_3 access ...done

Note: The R2 devices on the destination are set to read-only while the Symmetrix systems fully synchronize.

Note: The failback operation now occurs.


Configuring

ActionStep

An RDF 'Failback' operation execution is in progress for device group'1R2_500_3'.Please wait...

Write Disable device(s) on RA at target (R2)..............Done.Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.

Devices: 036B-037C in (0557,03).......................... Merged.

Devices: 034D-0352, 035A-036A in (0557,03)............... Merged.

Devices: 038F-0399 in (0557,03).......................... Merged.

Devices: 037D-038E in (0557,03).......................... Merged.

Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Started.

Resume RDF link(s)........................................Done.Read/Write Enable device(s) on SA at source (R1)..........Done.

The RDF 'Failback' operation successfully executed for device group'1R2_500_3'.

Waiting for 1R2_500_3 sync ....done

Note: The device-group failback phase is complete.

At the prompt, set the device group to run in SRDF/A mode by typing:6.

yes

Example:

Starting restore on remote site cs100_src ...Waiting for nbs clients to start ... doneWaiting for nbs clients to start ... done

Suspend RDF link(s).......................................Done.server_2 :server_2 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...donedoneserver_3 :server_3 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...donedone


Configuring

ActionStep

server_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured

Resume RDF link(s)........................................Started.

Resume RDF link(s)........................................Done.If the RDF device groups were setup to operate in ASYNCHRONOUS (SRDF/A ) mode, now wouldbe a good time to set it back to that mode.

Would you like to set device group 1R2_500_3 to ASYNC Mode ? [yesor no]: yes

An RDF Set 'Asynchronous Mode' operation execution is in progressfor device group'1R2_500_3'. Please wait...

The RDF Set 'Asynchronous Mode' operation successfully executed fordevice group'1R2_500_3'.

Starting Services on remote site cs100_src ...done


# exit

Output:

exit

Exit rdfadmin by typing:8.

$ exit

Output:

logout

Note: The restoration is complete.

Note: After the restore process completes, you should be able to log in to the source VNX (cs100_src),and manage it directly from the source nasadmin account. Chapter 4 provides information about whatto do if an error occurs during the restore process, or if you encounter problems with the restored VNX.If necessary, contact your local EMC Customer Support Representative.


Configuring


Configuring

4

Troubleshooting

As part of an effort to continuously improve and enhance the performanceand capabilities of its product lines, EMC periodically releases new versionsof its hardware and software. Therefore, some functions described in thisdocument may not be supported by all versions of the software or hardwarecurrently in use. For the most up-to-date information on product features,refer to your product release notes.

If a product does not function properly or does not function as describedin this document, contact your EMC Customer Support Representative.

Problem Resolution Roadmap for VNX contains additional informationabout using the EMC Online Support website and resolving problems.

Topics included are:

EMC E-Lab Interoperability Navigator on page 78 Known problems and limitations on page 78 Error messages on page 98 EMC Training and Professional Services on page 99



EMC E-Lab Interoperability Navigator

The EMC E-Lab Interoperability Navigator is a searchable, web-based application thatprovides access to EMC interoperability support matrices. It is available on the EMC OnlineSupport website at http://Support.EMC.com. After logging in, locate the applicable Supportby Product page, find Tools, and click E-Lab Interoperability Navigator.

Known problems and limitations

This section provides information on how to:

Retrieve information from log files on page 78

Resolve initialization failures on page 79

Resolve activation failures on page 84

Resolve restore failures on page 87

Resolve Data Mover failure after failover activation on page 95

Handle additional error situations on page 98

Retrieve information from log files

Normally, system messages are reported to the system log files. To retrieve informationfrom log files:

Check the system log (sys_log) by using the server_log command

Check the command error log (cmd_log.err) for message information

To retrieve substantial SRDF/A logging information:

Use the /nas/tools/collect_support_materials script, which collects data, such as disasterrecovery information, from the following log files:

• /nas/log/dr_log.al

• /nas/log/dr_log.al.rll

• /nas/log/dr_log.al.err

• /nas/log/dr_log.al.trace*

• /nas/log/symapi.log*

These log files can also be viewed individually.


Troubleshooting


To monitor these logs while the nas_rdf command is running, check the file in the /tmpdirectory. After the command completes, the logs appear in the /nas/log directory.

To gather more data after a failure, such as a failed restore, access the following sources:

Disaster recovery (dr*) files — Provide state changes, as well as other key informationalmessages

The symapi.log file — Logs storage-related errors

Resolve initialization failures

This section provides a sample failed initialization scenario in which two destination DataMovers (server_2 and server_3), intended to serve as SRDF/A standbys for two sourceproduction Data Movers, already have a local standby Data Mover (server_5). This resultsin an invalid configuration, that is, Data Movers serving as SRDF/A standbys cannot havea local standby Data Mover.

This section includes:

Example 1 for initialization failure on page 79

Resolution for initialization failure example 1 on page 81

Example 2 for initialization failure on page 82

Example 1 for initialization failure


[root@cs110_dst nasadmin]# /nas/sbin/nas_rdf -init

Discover local storage devices ...

Discovering storage (may take several minutes)done

Start R2 dos client ...doneStart R2 nas client ...done


Please create a new login account to manage RDF site cs100_src

New login: rdfadminNew password:BAD PASSWORD: it is based on a dictionary wordRetype new password:Changing password for user rdfadminpasswd: all authentication tokens updated successfullydone

Known problems and limitations 79

Troubleshooting


Please enter the passphrase for RDF site cs100_src:

Passphrase:rdfadmin

Retype passphrase:rdfadmin

operation in progress (not interruptible)...id = 1name = cs100_srcowner = 500device = /dev/ndj1channel = rdev=/dev/ndg, off_MB=391; wdev=/dev/nda, off_MB=391net_path = 192.168.97.140celerra_id = 000190100582034Dpassphrase = rdfadminDiscover remote storage devices ...done

The following servers have been detected on the system (cs110_dst):id type acl slot groupID state name1 1 1000 2 0 server_22 1 1000 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5Please enter the id(s) of the server(s) you wish to reserve(separated by spaces) or "none" for no servers.Select server(s) to use as standby:1 2

server_2 : Error 4031: server_2 : server_2 has a standbyserver: server_5server_3 : Error 4031: server_3 : server_3 has a standbyserver: server_5

operation in progress (not interruptible)...id = 1name = cs100_srcowner = 500device = /dev/ndj1channel = rdev=/dev/ndg, off_MB=391; wdev=/dev/nda, off_MB=391net_path = 192.168.97.140celerra_id = 000190100582034Dpassphrase = rdfadminPlease create a rdf standby for each server listedserver server_2 in slot 2, remote standby in slot [2] (or none):noneserver server_3 in slot 3, remote standby in slot [3] (or none):noneserver server_4 in slot 4, remote standby in slot [4] (or none):noneserver server_5 in slot 5, remote standby in slot [5] (or none):none


Troubleshooting

Resolution for initialization failure example 1

ActionStep

List and verify the servers by typing:1.

# nas_server -list

Output:


Delete the local standby relationship by typing:2.

# server_standby server_2 -delete mover=server_5

Output:

server_2 : done

Delete.....by typing:3.

[root@cs110_dst nasadmin]# /nas/sbin/getreason

Output:



[root@cs110_dst nasadmin]# /nas/sbin/getreason

Output:


Delete the.....by typing:5.

[root@cs110_dst nasadmin]# /nas/bin/nas_server -list

Output:



Troubleshooting

ActionStep


[root@cs110_dst nasadmin]# /nas/bin/nas_server -info -all

Output:






The following example highlights the error occurred when a file system is mounted on alocal Data Mover, intended to serve as an RDF standby. The example shows new promptsfor the user to change the configuration on another window before proceeding with theinitialization.


Troubleshooting


[root@cs110_dst nasadmin]# /nas/sbin/nas_rdf -init

Discover local storage devices ...

Discovering storage (may take several minutes)done

Please create a rdf standby for each server listedserver server_2 in slot 2, remote standby in slot [2] (or none): 2Error 3122: server_2 : filesystem is unreachable: rl64kServer server_2 has local file system mounted.Please unmount those file system in another window and try again.

Do you wish to continue? [yes or no]:********************$ nas_server -i server_2id = 1name = server_2acl = 1000, owner=nasadmin, ID=201type = nasslot = 2member_of =standby = server_3, policy=autoRDFstandby= slot=2status :

defined= enabledactual = online, active

$ server_mount server_2 rl64k /rl64kserver_2 : done

Warning 17716815751: server_2 :has a standby server: rdf, filesystem:rl64k is local, will not be able to failover[nasadmin@cs100_src ~]$ server_df server_2server_2 :Filesystem kbytes used avail capacity Mounted onrl64k 230393504 124697440 105696064 54% /rl64mc1 230393504 45333944 185059560 20% /mc1mc2 460787024 327825496 132961528 71% /mc2mc1a_ckpt1 230393504 410408 229983096 0% /mc1a_ckpt1mc1a 230393504 410424 229983080 0% /mc1arl32krdf 230393504 696 230392808 0% /caddata/wdc1/32k

root_fs_common 13624 5288 8336 9% /.etc_commonroot_fs_2 231944 6152 225792 3% /


Troubleshooting

Resolve activation failures

This section provides a sample failed activation scenario in which a local file system ismounted on an SRDF-protected standby Data Mover. The error conditions are illustratedand the corrective commands are listed after the error.

This section includes:

Example for activation failure on page 84

Resolution for activation failure on page 85

Example for activation failure

Example of activation failure

[root@cs110_dst rdfadmin]# /nas/sbin/nas_rdf -activate

Is remote site cs100_src completely shut down (power OFF)?Do you wish to continue? [yes or no]: yes

Successfully pinged (Remotely) Symmetrix ID: 000190100582

An RDF 'Failover' operation execution is in progress for device group'1R2_500_3'.Please wait...

Write Disable device(s) on SA at source (R1)..............Done.Suspend RDF link(s).......................................Done.Read/Write Enable device(s) on RA at target (R2)..........Done.

The RDF 'Failover' operation successfully executed fordevice group '1R2_500_3'.

Waiting for nbs clients to die ... done/net/500 /etc/auto.500 -t 1,ro/dev/ndj1: recovering journal/dev/ndj1: clean, 11587/231360 files, 204164/461860 blocksfsck 1.26 (3-Feb-2002)Waiting for nbs clients to die ... doneid type acl slot groupID state name1 1 1000 2 0 server_22 4 1000 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5

server_2 :server_2 : going offlinerdf : going activereplace in progress ...failedfailover activity complete


Troubleshooting

Example of activation failure

replace_storage:replace_volume: volume is unreachabled141,d142,d143,d144,d145,d146,d147,d148

server_3 :server_3 : going offlinerdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...donedoneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configuredAn RDF 'Update R1' operation execution is in progress for device'DEV001' in group '1R2_500_3'. Please wait...

Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Device: 034D in (0557,03)................................ Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Started.Resume RDF link(s)........................................Done.

The RDF 'Update R1' operation successfully initiated for device'DEV001' in group '1R2_500_3'.

Resolution for activation failure

ActionStep

List and verify the servers by typing:1.

# nas_server -list

Output:

id type acl slot groupID state name1 1 1000 2 2 server_2.faulted.rdf2 4 0 3 0 server_33 1 1000 4 0 server_44 4 1000 5 0 server_5

Unmount all non-SRDF (local) file systems from the Data Mover that failed to activate (in this case, server_2)by typing:

2.

[root@cs110_dst rdfadmin] # server_umount server_2.faulted.rdf - perm fs5

Output:

server_2.faulted.rdf : done


Troubleshooting

ActionStep

Manually activate SRDF for the Data Mover that originally failed by typing:3.

[root@cs110_dst rdfadmin]# server_standby server_2.faulted.rdf - activate rdf

Output:

server_2.faulted.rdf :server_2.faulted.rdf : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...donedone

Verify the configuration by typing:4.

[root@cs110_dst rdfadmin]# /nas/sbin/getreason

Output:



[root@cs110_dst rdfadmin] # /nas/bin/nas_server -list

Output:



[root@cs110_dst rdfadmin]# /nas/bin/server_mount ALL

Output:

server_2 :root_fs_2 on / uxfs,perm,rwroot_fs_common on /.etc_common uxfs,perm,rofs2 on /fs2 uxfs,perm,rwfs1a on /fs1a uxfs,perm,rwfst1 on /fst1 uxfs,perm,rwserver_3 :root_fs_3 on / uxfs,perm,rw,<unmounted>root_fs_common on /.etc_common uxfs,perm,ro,<unmounted>server_4 :root_fs_4 on / uxfs,perm,rwroot_fs_common on /.etc_common uxfs,perm,roserver_5 :root_fs_5 on / uxfs,perm,rw,<unmounted>root_fs_common on /.etc_common uxfs,perm,ro,<unmounted>


Troubleshooting

Resolve restore failures

This section provides sample restore failure scenarios.

Note: The source-side Control Station 0 must be operational for the restore, and VNX for file serviceson the source site start only after the restore process successfully completes. A Waiting for DisasterRecovery to complete message appears in the /var/log/messages and the source remains in that stateuntil the restore completes. This change, which involves use of a DR lock, ensures correct sequentialoperation, thereby ensuring that the source-site services come up correctly under RDF control andthat no user commands run until the source site is completely restored. Error messages on page 98provides more information about the associated errors.

Topics included are:

Example 1 for restoration failure on page 87

Resolution for restoration failure example 1 on page 91

Example 2 for restoration failure (NS series gateway) on page 92


Example 3 for restoration failure (database lock error) on page 93


Example 1 for restoration failure

This example shows an active/passive restore operation started by root from rdfadmin oncs110_dst using the nas_rdf -restore command. The output shows the events leading up tothe error message on the destination VNX, followed by output from the source after takingcorrective action.

Example 1 restore failure

[root@cs110_dst rdfadmin]# /nas/sbin/nas_rdf -restore

Is remote site cs100_src ready for Storage restoration?Do you wish to continue? [yes or no]: yes

Contact cs100_src ...Unable to contact node cs100_src at 192.168.96.58.Do you wish to continue? [yes or no]: yes



Troubleshooting


Target (R2) View Source (R1) View MODES-------------------------------- ------------------ ---- --------

ST LI STStandard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDA STATE-------------------------------- -- -------- ------- ---- --------DEV001 0000 RW 12 0 NR 0000 WD 0 0 A.. Failed OverDEV002 0001 RW 4096 0 NR 0001 WD 0 0 A.. Failed OverDEV003 000D RW 1 0 NR 000D WD 0 0 A.. Failed OverDEV004 000E RW 1 0 NR 000E WD 0 0 A.. Failed OverDEV005 000F RW 0 0 NR 000F WD 0 0 A.. Failed OverDEV006 0010 RW 0 0 NR 0010 WD 0 0 A.. Failed OverDEV007 0011 RW 0 0 NR 0011 WD 0 0 A.. Failed OverDEV008 0012 RW 0 0 NR 0012 WD 0 0 A.. Failed OverDEV009 0013 RW 0 0 NR 0013 WD 0 0 A.. Failed OverDEV010 0014 RW 0 0 NR 0014 WD 0 0 A.. Failed OverDEV011 0015 RW 1 0 NR 0015 WD 0 0 A.. Failed OverDEV012 0016 RW 1 0 NR 0016 WD 0 0 A.. Failed Over...DEV161 02D3 RW 0 0 NR 0253 WD 0 0 A.. Failed OverDEV162 02D7 RW 0 0 NR 0257 WD 0 0 A.. Failed OverDEV163 0003 RW 0 0 NR 0003 WD 0 0 A.. Failed OverDEV164 0004 RW 851 0 NR 0004 WD 0 0 A.. Failed OverDEV165 0005 RW 0 0 NR 0005 WD 0 0 A.. Failed Over

Total -------- --------- ------ ------Track(s) 4965 0 0 0MB(s) 155.2 0.0 0.0 0.0

Legend for MODES:

M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopy

D(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp off

+++++ Setting RDF group 1R2_500_11 to SYNC mode.An RDF 'Update R1' operation execution is in progress for device group'1R2_500_11'.Please wait...

Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Devices: 0000-0001 ...................................... Merged.Devices: 0003-0005 ...................................... Merged.Devices: 000D-0015 ...................................... Merged.Devices: 0016-001E ...................................... Merged.Devices: 001F-0027 ...................................... Merged.


Troubleshooting


Devices: 0028-0030 ...................................... Merged.Devices: 0031-0039 ...................................... Merged.Devices: 003A-0042 ...................................... Merged.Devices: 0043-004B ...................................... Merged.Devices: 004C-0054 ...................................... Merged.Devices: 0055-005D ...................................... Merged.Devices: 005E-0066 ...................................... Merged.Devices: 0067-006F ...................................... Merged.Devices: 0070-0078 ...................................... Merged.Devices: 0079-0081 ...................................... Merged.Devices: 0082-008A ...................................... Merged.Devices: 008B-008C ...................................... Merged.Devices: 01DB-01E1 ...................................... Merged.Devices: 01E2-01E8 ...................................... Merged.Devices: 01E9-01EF ...................................... Merged.Devices: 01F0-01F6 ...................................... Merged.Devices: 01F7-01FD ...................................... Merged.Devices: 01FE-0204 ...................................... Merged.Devices: 0205-020B ...................................... Merged.Devices: 020C-0212 ...................................... Merged.Devices: 0213-0219 ...................................... Merged.Devices: 021A-0220 ...................................... Merged.Devices: 0221-0227 ...................................... Merged.Devices: 0228-022E ...................................... Merged.Devices: 022F-0235 ...................................... Merged.Devices: 0236-023C ...................................... Merged.Devices: 023D-0243 ...................................... Merged.Devices: 0244-024A ...................................... Merged.Devices: 024B-0251 ...................................... Merged.Devices: 0252-0258 ...................................... Merged.Devices: 0259-025A ...................................... Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Done.

The RDF 'Update R1' operation successfully initiated for device group'1R2_500_11'.

Is remote site cs100_src ready for Network restoration?Do you wish to continue? [yes or no]: yes

server_2 : doneserver_3 : doneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured/dev/sdj1: clean, 11464/231360 files, 164742/461860 blocksfsck 1.26 (3-Feb-2002)/net/500 /etc/auto.500 -t 0,rw,syncWaiting for 1R2_500_11 access ...done


Troubleshooting


An RDF 'Failback' operation execution is in progress for device group'1R2_500_11'.Please wait...

Write Disable device(s) on RA at target (R2)..............Done.Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Devices: 0000-0001 ...................................... Merged.Devices: 0003-0005 ...................................... Merged.Devices: 000D-0015 ...................................... Merged.Devices: 0016-001E ...................................... Merged.Devices: 001F-0027 ...................................... Merged.Devices: 0028-0030 ...................................... Merged.Devices: 0031-0039 ...................................... Merged.Devices: 003A-0042 ...................................... Merged.Devices: 0043-004B ...................................... Merged.Devices: 004C-0054 ...................................... Merged.Devices: 0055-005D ...................................... Merged.Devices: 005E-0066 ...................................... Merged.Devices: 0067-006F ...................................... Merged.Devices: 0070-0078 ...................................... Merged.Devices: 0079-0081 ...................................... Merged.Devices: 0082-008A ...................................... Merged.Devices: 008B-008C ...................................... Merged.Devices: 01DB-01E1 ...................................... Merged.Devices: 01E2-01E8 ...................................... Merged.Devices: 01E9-01EF ...................................... Merged.Devices: 01F0-01F6 ...................................... Merged.Devices: 01F7-01FD ...................................... Merged.Devices: 01FE-0204 ...................................... Merged.Devices: 0205-020B ...................................... Merged.Devices: 020C-0212 ...................................... Merged.Devices: 0213-0219 ...................................... Merged.Devices: 021A-0220 ...................................... Merged.Devices: 0221-0227 ...................................... Merged.Devices: 0228-022E ...................................... Merged.Devices: 022F-0235 ...................................... Merged.Devices: 0236-023C ...................................... Merged.Devices: 023D-0243 ...................................... Merged.Devices: 0244-024A ...................................... Merged.Devices: 024B-0251 ...................................... Merged.Devices: 0252-0258 ...................................... Merged.Devices: 0259-025A ...................................... Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Done.Read/Write Enable device(s) on SA at source (R1)..........Done.


Waiting for 1R2_500_11 sync ...done

Starting restore on remote site cs100_src ...failed----------------------------------------------------------------Please execute /nasmcd/sbin/nas_rdf -restore on remote site cs100_src----------------------------------------------------------------[root@cs100_src rdfadmin]#


Troubleshooting

Note: The failure occurs after the failback operation is executed for the device group, when the restoreis set to begin on the source VNX.

Resolution for restoration failure example 1

ActionStep


[root@cs100_src nasadmin]# /nasmcd/sbin/nas_rdf -restore

server_2 : rdf : reboot in progress ............server_2 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...done

doneserver_3 : rdf : reboot in progress ............server_3 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...done

doneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured

If the RDF device groups were setup to operate in ASYNCHRONOUS (SRDF/A ) mode, now wouldbe a good time to set it back to that mode.

Would you like to set device group 1R1_11 to ASYNC Mode ? [yes orno]: yes

An RDF Set 'Asynchronous Mode' operation execution is in progressfor device group'1R1_11'. Please wait...

The RDF Set 'Asynchronous Mode' operation successfully executedfor device group '1R1_11'.

If the RDF device groups were setup to operate in ASYNCHRONOUS (SRDF/A ) mode, now wouldbe a good time to set it back to that mode.

Would you like to set device group 1R1_12 to ASYNC Mode ? [yes orno]: no

Starting Services ...done[root@cs100_src nasadmin]#

Exit as root. This concludes the device-group failback phase.2.


Troubleshooting

ActionStep

Log in to and manage the source VNX directly from the nasadmin account on the source VNX (cs100_src).If the restoration is still unsuccessful, gather SRDF/A logging information using the script /nas/tools/collect_sup-port_materials.

3.

Example 2 for restoration failure (NS series gateway)

The following example on an NS600G or NS700G shows a failed restore operation at thedestination.


[root@Celerra2 nasadmin]# /nas/sbin/nas_rdf –restore

...Starting restore on remote site Celerra1 ...Waiting for nbs clients to start ... WARNING: Timed outWaiting for nbs clients to start ... doneCRITICAL FAULT:Unable to mount /nas/dosStarting Services on remote site Celerra1 ...done

Note: “..." indicates not all lines of the restore output are shown.


ActionStep

Stop the services at the source as root by typing:1.

[root@Celerra1 nasadmin] # /sbin/service nas stop

Perform a restore at the source as root by typing:2.

[root@Celerra1 nasadmin] # /nasmcd/sbin/nas_rdf –restore

Output:

...Waiting for nbs clients to start ... doneWaiting for nbs clients to start ... doneSuspend RDF link(s).......................................Done.server_2 :replace in progress ...donecommit in progress (not interruptible)...donedoneserver_3 :Error 4003: server_3 : standby is not configuredResume RDFlink(s)........................................Done.Starting Services ...done


Troubleshooting

Example 3 for restoration failure (database lock error)

The following example shows a restore error that occurs when a server fails to acquire thedatabase lock.The restore completes with the error, but resolving the error involves runningthe server_standby command at the source for the server involved in the lock contention.The error is highlighted.

Example 3 restore error

[root@cs0_dst rdfadmin]# /nas/sbin/nas_rdf –restore

Is remote site cs100_src ready for Storage restoration?Do you wish to continue? [yes or no]: yes


Target (R2) View Source (R1) View MODES-------------------------------- -------------------- ---- --------

ST LI STStandard A N ALogical T R1 Inv R2 Inv K T R1 Inv R2 Inv RDF PairDevice Dev E Tracks Tracks S Dev E Tracks Tracks MDA STATE-------------------------------- -- ----------------- --- --------DEV001 08F2 RW 1 0 RW 37CD WD 0 0 C.D R1 UpdatedDEV002 08F3 RW 58 0 RW 37CE WD 0 0 C.D R1 UpdatedDEV003 08FA RW 0 0 RW 37D3 WD 0 0 C.D R1 UpdatedDEV004 08FB RW 0 0 RW 37D4 WD 0 0 C.D R1 UpdatedDEV005 08FC RW 12 0 RW 37D5 WD 0 0 C.D R1 UpdatedDEV006 08FD RW 0 0 RW 37D6 WD 0 0 C.D R1 UpdatedDEV007 092C RW 3546 0 RW 0629 WD 0 0 C.D R1 UpdatedDEV008 0930 RW 2562 0 RW 062D WD 0 0 C.D R1 UpdatedDEV009 06F5 RW 0 0 RW 0631 WD 0 0 C.D R1 UpdatedTotal -------- ----- ------ -----Track(s) 6179 0 0 0MB(s) 193.1 0.0 0.0 0.0

Legend for MODES:

M(ode of Operation): A = Async, S = Sync, E = Semi-sync, C = AdaptiveCopyD(omino) : X = Enabled, . = DisabledA(daptive Copy) : D = Disk Mode, W = WP Mode, . = ACp off

An RDF 'Update R1' operation execution is in progress for device group'1R2_500_4'.Please wait...

Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Devices: 37CD-37CE ...................................... Merged.Devices: 37D3-37D6 ...................................... Merged.Devices: 0629-0634 ...................................... Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Started.Resume RDF link(s)........................................Done.

The RDF 'Update R1' operation successfully initiated for device group'1R2_500_4'.


Troubleshooting


Is remote site cs0_src ready for Network restoration?Do you wish to continue? [yes or no]: yes

server_2 : doneserver_3 : doneserver_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured/dev/ndj1: clean, 10308/231360 files, 175874/461860 blocksfsck 1.26 (3-Feb-2002)Waiting for nbs clients to die ... doneWaiting for nbs clients to die ... done/net/500 /etc/auto.500 -t 0,rw,syncWaiting for 1R2_500_4 access ...done

An RDF 'Failback' operation execution is in progress for device group'1R2_500_4'.Please wait...Write Disable device(s) on RA at target (R2)..............Done.Suspend RDF link(s).......................................Done.Merge device track tables between source and target.......Started.Devices: 37CD-37CE ...................................... Merged.Devices: 37D3-37D6 ...................................... Merged.Devices: 0629-0634 ...................................... Merged.Merge device track tables between source and target.......Done.Resume RDF link(s)........................................Started.Resume RDF link(s)........................................Done.Read/Write Enable device(s) on SA at source (R1)..........Done.


Waiting for 1R2_500_4 sync ...done

Starting restore on remote site cs0_src ...Waiting for nbs clients to start ... doneWaiting for nbs clients to start ... doneserver_2 :Error 2201: server_2 : unable to acquire lock(s), try laterserver_3 :server_3 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...donedone

server_4 :Error 4003: server_4 : standby is not configuredserver_5 :Error 4003: server_5 : standby is not configured


Troubleshooting


If the RDF device groups were setup to operate in ASYNCHRONOUS ( SRDF/A )mode,now would be a good time to set it back to that mode.

Would you like to set device group 1R2_500_4 to ASYNC Mode ? [yes or no]:yes

Starting Services on remote site cs0_src ...[root@cs0_dst rdfadmin]# exit

[rdfadmin@cs0_dst rdfadmin]$ exit

[nasadmin@cs0_dst nasadmin]$ exit


Action

Run the server_standby command on the source VNX for the server that had the lock contention (in this example, server_2).

Example:

[nasadmin@cs0_src nasadmin]$ server_standby server_2 -restore rdf

Output

server_2 :server_2 : going standbyrdf : going activereplace in progress ...donefailover activity completecommit in progress (not interruptible)...done

doneserver_2 : Nil

Resolve Data Mover failure after failover activation

If a Data Mover develops hardware issues after activate, you can replace the affected DataMover and update the hardware information. To update the hardware information, you mustrun the setup_slot command first as nasadmin switching (su) to root and then as rdfadminswitching (su) to root.

ActionStep

Log in to the destination VNX (cs110_dst) as nasadmin and switch (su) to root.1.


Troubleshooting

ActionStep

Initialize the Data Mover by using this command syntax:2.

[root@cs110_dst nasadmin]# /nas/sbin/setup_slot -init <x>

where:

<x> = Slot number of the new Data Mover

Example:

To initialize the Data Mover for slot 2, type:

[root@cs110_dst nasadmin]# /nas/sbin/setup_slot -init 2

Initializing server in slot 2 as server_2 ...Starting PXE service...:doneReboot server in slot 2, waiting..... 0 0 0 0 0 0 1 1 1 3 3 3 3 33 3 4 (154 secs)Stopping PXE service...:donePing server in slot 2 on primary interface ...okPing server in slot 2 on backup interface ...okDiscover disks attached to server in slot 2 ...Discovering storage (may take several minutes)server_2 : doneserver_2 : doneserver_2 : doneserver_2 : doneSynchronize date+time on server in slot 2 ...server_2 : Mon Aug 17 12:11:05 EDT 2009server_2 :

Processor = Intel Pentium 4Processor speed (MHz) = 2800Total main memory (MB) = 4093Mother board = CMB-SledgehammerBus speed (MHz) = 800Bios Version = 03.80Post Version = Rev. 01.59

server_2 : reboot in progress 0.0.0.0.0.0.0.0.1.1.3.3.3.3.3.4.doneChecking to make sure slot 2 is ready........ 5 5 (63 secs)

Completed setup of server in slot 2 as server_2This Data Mover (also referred to as Blade) is a MirrorView or RDFstandby Data Mover, log in to the system as rdfadmin and switch(su) to root, and, regardless of straight (for example, server 2to server 2) or criss-cross (for example, server 2 to server 3)configuration, use the CLI command with the same slot id:/nas/sbin/setup_slot -i 2

[root@cs110_dst nasadmin]#


[root@cs110_dst nasadmin]# exitexit

Exit nasadmin by typing:4.

[nasadmin@cs110_dst ~]$ exit

logout


Troubleshooting

ActionStep

Log in to the destination VNX (cs110_dst) as rdfadmin and switch (su) to root.5.

Initialize the Data Mover by using this command syntax:6.

[root@cs110_dst rdfadmin]# /nas/sbin/setup_slot -init <x>

where:

<x> = Slot number of the new Data Mover

Example:

To initialize the Data Mover for slot 2, type:

[root@cs110_dst rdfadmin]#

/nas/sbin/setup_slot -init 2

The script will update only hardware related configuration suchas the MAC addresses for the internal network and then reboot theData Mover (also referred to as Blade).

server_2 : reboot in progress 0.0.0.0.0.0.0.0.1.1.3.3.3.3.3.4.doneChecking to make sure slot 2 is ready........ 5 5 (64 secs)Completed setup of server in slot 2 as server_2

[root@cs110_dst rdfadmin]#

CAUTION Ensure that you run the setup_slot command first as nasadmin switching (su) to root andthen as rdfadmin switching (su) to root.

If you run the command as rdfadmin before running it as nasadmin, you will get the following error message:

setup_slot has not been run as nasadmin before running it as rdfad-min user on this Data Mover (also referred to as Blade) The scriptwill exit without changing the state of the system or rebootingit. Please do the following to set up this Data Mover correctly:1. Initialize the new Data Mover for the nasadmin database bylogging in to the system as nasadmin, switching (su) to root,andusing the CLI command:/nas/sbin/setup_slot -init 22. Initialize the new Data Mover for the rdfadmin database bylogging in to the system as rdfadmin user, switching (su) to root,and using the CLI command:/nas/sbin/setup_slot -init 2


[root@cs110_dst rdfadmin]# exit

exit

Exit rdfadmin by typing:8.

[rdfadmin@cs110_dst ~]$ exit

logout


Troubleshooting

Handle additional error situations

If you shut down or restart any Data Movers (SRDF-protected or non-SRDF Data Movers)at the destination while the /nas/sbin/nas_rdf -activate or the nas/sbin/nas_rdf -restorecommand is running, the Control Station does not find a path to the storage system.Withthe communication between VNX and the backend interrupted, the command fails.Respond by doing the following:

1. Rerun the /nas/sbin/nas_rdf -activate or the /nas/sbin/nas_rdf -restore command afterthe Data Mover is operational.

2. Do not shut down or restart any Data Movers at the destination while these commandsare running.

When you run the -init command, it changes the ACL for the local Data Movers to 1000.This prevents DR administrators from inadvertently accessing the local Data Movers inthe failed over state. However, this also prevents Global Users in Unisphere fromaccessing this Data Mover in the normal state.

To resolve this problem, the ACL for the local Data Movers is no longer changed whenyou run the -init command. Instead, the ACL is changed to 1111 during failover whenyou run the -activate command. This prevents DR administrators from accessing theData Movers after failover and allows Global Users to access them in the normal state.

During failback, when you run the -restore command, the ACL is changed to 0. If youinitially use ACL 1111 for the local Data Movers on the source side, ensure that youchange the ACL from 0 back to 1111 after failback. Alternately, you can change the ACLfor the local Data Movers on the source side to some other value, for example, 1000, toavoid this manual change.

For some applications, IP Address configuration must be considered carefully. Otherwise,this could lead to the data becoming corrupt. Consideration when using applications thatcan switch to the NFS copy from the R2 without a restart on page 33 provides moreinformation.

When using applications such as Oracle, ensure that you use the correct client-side NFSmount options. Otherwise, this could result in incorrect data on SRDF or TimeFindercopies. Consideration when using applications that require transactional consistency onpage 32 provides more information.

Error messages

All event, alert, and status messages provide detailed information and recommended actionsto help you troubleshoot the situation.

To view message details, use any of these methods:

Unisphere software:


Troubleshooting

Right-click an event, alert, or status message and select to view Event Details, AlertDetails, or Status Details.

•

CLI:

• Type nas_message -info <MessageID>, where <MessageID> is the messageidentification number.

Celerra Error Messages Guide:

• Use this guide to locate information about messages that are in the earlier-releasemessage format.

EMC Online Support website:

• Use the text from the error message's brief description or the message's ID to searchthe Knowledgebase on the EMC Online Support website. After logging in to EMCOnline Support, locate the applicable Support by Product page, and search for theerror message.

EMC Training and Professional Services

EMC Customer Education courses help you learn how EMC storage products work togetherwithin your environment to maximize your entire infrastructure investment. EMC CustomerEducation features online and hands-on training in state-of-the-art labs conveniently locatedthroughout the world. EMC customer training courses are developed and delivered by EMCexperts. Go to the EMC Online Support website at http://Support.EMC.com for course andregistration information.

EMC Professional Services can help you implement your system efficiently. Consultantsevaluate your business, IT processes, and technology, and recommend ways that you canleverage your information for the most benefit. From business plan to implementation, youget the experience and expertise that you need without straining your IT staff or hiring andtraining new personnel. Contact your EMC Customer Support Representative for moreinformation.

EMC Training and Professional Services 99

Troubleshooting




Troubleshooting

Appendix A

Portfolio of High-AvailabilityOptions

This appendix illustrates the VNX SRDF high-availability configurationoptions.Figure 4 on page 102 shows a configuration featuring active/passiveSRDF in either asynchronous mode (SRDF/A) or synchronous mode(SRDF/S).

Figure 4 on page 102, Figure 5 on page 102, Figure 6 on page 102, Figure7 on page 103, Figure 8 on page 103, Figure 9 on page 104, and Figure 10on page 104show disaster recovery and business continuance configurationsfeaturing SRDF/S only, SRDF links, or both, with TimeFinder/FS NearCopy,TimeFinder/FS FarCopy, or both, with adaptive copy mode (adaptive copydisk or disk-pending mode).

Note: Prior to the introduction of SRDF/A, adaptive copy mode was referred to asasynchronous mode SRDF.


For more information on the configuration that best fits your businessneeds, contact your local EMC sales organization.

VNXR1 datavolumes

VNXR2 datavolumes

Dedicated SRDF links(synchronous or

asynchronous mode)

PFS

VNX

VNX

Symmetrix

Symmetrix

DR mirror ofPFS

Production site

DR recovery siteSRDF

SRDF

SRDF

SRDF

VNX DR overSRDF/S

VNX-000038

Figure 4. VNX replication and recovery with active/passive synchronous

SRDF (SRDF/S) or asynchronous SRDF (SRDF/A)

VNXR1 datavolumes

VNXR2 datavolumes

Dedicated SRDF links(synchronous mode only)

PFS

VNX

VNX

Symmetrix

Symmetrix

DR mirror ofPFS

Production site

DR recovery +business

continuance siteSRDF

SRDF

SRDF

SRDF

VNX DR overSRDF/S

VNX-000039

VNXlocal BCVvolumes

NearCopy

snapshot of

PFS

Figure 5. VNX disaster recovery active/passive SRDF/S only with

TimeFinder/FS NearCopy


Portfolio of High-Availability Options

VNXR1 datavolumes

VNXR2 datavolumes

Dedicated SRDF links(synchronous mode only)

PFS

PFS

VNXR2 datavolumes

VNX

VNX

Symmetrix

Symmetrix

DR mirror ofPFS

DR mirror ofPFS

VNXR1 datavolumes

Production + DRrecovery site

Production + DRrecovery site

SRDF

SRDF

SRDF

SRDF

VNX DR overSRDF/S

VNX-000040

Figure 6. VNX disaster recovery active/active’ SRDF/S only

VNXR1 BCVvolumes

VNXR2 BCVvolumes

BCV snapshot ofPFS

PFS

VNX

VNX

Symmetrix

Symmetrix

Imported FS

Production site

Businesscontinuancerecovery site

SRDF

SRDF

SRDF

SRDF

Adaptivecopy

VNX-000041

VNXlocal STDvolumes

VNXlocal STDvolumes

FarCopysnapshot of

PFS

Dedicated SRDF links(adaptive copy

write-pending mode)

Figure 7. VNX business continuance with TimeFinder/FS FarCopy (version

5.1)

103


VNXR1 BCVvolumes

VNXR2 BCVvolumes

Dedicated SRDF links(adaptive copy

write-pending mode)BCV snapshot of

PFS

PFS

VNX

VNX

Symmetrix

Symmetrix

Imported FS

Production site

Businesscontinuancerecovery site

SRDF

SRDF

SRDF

SRDF

Adaptive copy

VNX-000042

VNXlocal STDvolumes

VNXlocal STDvolumes

FarCopysnapshot of

PFS

BCV snapshot ofPFS

IP WAN

Control Station IP

Control Station IP

Figure 8. VNX business continuance with TimeFinder/FS FarCopy (version

5.3)

VNXR1 BCVvolumes

VNXR1 BCVvolumes

VNXR2 BCVvolumes

VNXR2 BCVvolumes

Dedicated SRDF Links

(adaptive copy

write-pending mode)

FarCopy

snapshot of

PFS

VNX

VNX

Symmetrix

VNXSymmetrix

Symmetrix

Imported FS

Imported FS

PFS

Business

continuance

recovery site 1

Business

continuance

recovery

site 2

SRDF

SRDF

SRDF

SRDF

SRDF

SRDF

VNX-000043

VNXlocal STDvolumes

VNXlocal STDvolumes

VNXlocal STDvolumes

FarCopy

snapshot of

PFS

BCV snapshot of

PFS

BCV snapshot of

PFS

IP WAN

Control Station IP

Control

Station IPControl

Station IP

Production site

VNX FarCopy over

adaptive copy

VNX FarCopy over

adaptive copy

Figure 9. VNX business continuance with redundant FarCopy sites



VNXR1 BCVvolumes

VNXR2 BCVvolumes

VNXR2 BCVvolumes

PFS

PFS

Imported FS

VNX

VNX

Symmetrix

VNXSymmetrix

Symmetrix

Imported FS

Production site 1

Production site 2

Business

continuance

recovery site

SRDF(adaptive copy

write-pending mode)

SRDF(adaptive copy

write-pending mode)

VNX-000044

VNXlocal STDvolumes

VNXlocal STDvolumes

VNXlocal STDvolumes

VNXlocal STDvolumes

FarCopy

snapshot of

PFS

FarCopy

snapshot of

PFS

IP WANControl Station IP

Control Station IP

Control Station IP

VNX FarCopy over

adaptive copy

BCV snapshot of

PFS

BCV snapshot of

PFS

VNXR1 BCVvolumes

VNX FarCopy over

adaptive copy

Figure 10. VNX business continuance using TimeFinder/FS FarCopy with

many sites

105




Glossary

A

active/active’%∂π $,"é 2¿¥¥¨ªπ∞øé 1¨¥∂ª¨ #®ª® %®™∞≥∞ª¿ (21#%é) ∂π $,",∞ππ∂π5∞¨æ/2¿µ™Øπ∂µ∂º∫™∂µ≠∞Æºπ®ª∞∂µ∫, ® ©∞´∞π¨™ª∞∂µ®≥ ™∂µ≠∞Æºπ®ª∞∂µ æ∞ªØ ªæ∂ ∑π∂´º™ª∞∂µ ∫∞ª¨∫, ¨®™Ø ®™ª∞µÆ ®∫ ªØ¨∫ª®µ´©¿ ≠∂π ªØ¨ ∂ªØ¨π. $®™Ø 5-7 ≠∂π ≠∞≥¨ Ø®∫ ©∂ªØ ∑π∂´º™ª∞∂µ ®µ´ ∫ª®µ´©¿ #®ª® ,∂Ω¨π∫. (≠ ∂µ¨∫∞ª¨ ≠®∞≥∫, ªØ¨ ∂ªØ¨π ∫∞ª¨ ª®≤¨∫ ∂Ω¨π ®µ´ ∫¨πΩ¨∫ ªØ¨ ™≥∞¨µª∫ ∂≠ ©∂ªØ ∫∞ª¨∫. %∂π 21#%, ¨®™Ø 2¿¥¥¨ªπ∞ø∫¿∫ª¨¥ ∞∫ ∑®πª∞ª∞∂µ¨´ ∞µª∂ ∫∂ºπ™¨ (∑π∂´º™ª∞∂µ) ®µ´ π¨¥∂ª¨ ´¨∫ª∞µ®ª∞∂µ Ω∂≥º¥¨∫. %∂π,∞ππ∂π5∞¨æ/2,¨®™Ø5-7 ≠∂π ©≥∂™≤ ∞∫ ™∂µ≠∞Æºπ¨´ ª∂ Ø®Ω¨ ∫∂ºπ™¨ ®µ´ ´¨∫ª∞µ®ª∞∂µ +4-∫ ®µ´ ® ™∂µ∫∞∫ª¨µ™¿ Æπ∂º∑.

active/passive%∂π 21#% ∂π ,∞ππ∂π5∞¨æ/2 ™∂µ≠∞Æºπ®ª∞∂µ∫, ® ºµ∞´∞π¨™ª∞∂µ®≥ ∫¨ªº∑ æØ¨π¨ ∂µ¨ 5-7 ≠∂π ≠∞≥¨, æ∞ªØ∞ª∫ ®ªª®™Ø¨´ ∫¿∫ª¨¥, ∫¨πΩ¨∫ ®∫ ªØ¨ ∫∂ºπ™¨ (∑π∂´º™ª∞∂µ) ≠∞≥¨ ∫¨πΩ¨π ®µ´ ®µ∂ªØ¨π 5-7 ≠∂π ≠∞≥¨, æ∞ªØ∞ª∫ ®ªª®™Ø¨´ ∫ª∂π®Æ¨, ∫¨πΩ¨∫ ®∫ ªØ¨ ´¨∫ª∞µ®ª∞∂µ (©®™≤º∑). 3Ø∞∫ ™∂µ≠∞Æºπ®ª∞∂µ ∑π∂Ω∞´¨∫ ≠®∞≥∂Ω¨π™®∑®©∞≥∞ª∞¨∫ ∞µ ªØ¨ ¨Ω¨µª ªØ®ª ªØ¨ ∫∂ºπ™¨ ∫∞ª¨ ∞∫ ºµ®Ω®∞≥®©≥¨. µ 21#% ™∂µ≠∞Æºπ®ª∞∂µ π¨∏º∞π¨∫2¿¥¥¨ªπ∞ø ∫¿∫ª¨¥∫ ®∫ ©®™≤¨µ´ ∫ª∂π®Æ¨. ,∞ππ∂π5∞¨æ/2 ™∂µ≠∞Æºπ®ª∞∂µ π¨∏º∞π¨∫ ∫∑¨™∞≠∞™5-7 ≠∂π ©≥∂™≤ ∫¨π∞¨∫ ∫¿∫ª¨¥∫ ®∫ ©®™≤¨µ´ ∫ª∂π®Æ¨.

adaptive copy disk-pending mode21#%¥∂´¨ ∂≠ ∂∑¨π®ª∞∂µ ∞µæØ∞™Øæπ∞ª¨ ª®∫≤∫ ®™™º¥º≥®ª¨ ∞µ Æ≥∂©®≥ ¥¨¥∂π¿ ∂µ ªØ¨ ≥∂™®≥ ∫¿∫ª¨¥©¨≠∂π¨ ©¨∞µÆ ∫¨µª ª∂ ªØ¨ π¨¥∂ª¨ ∫¿∫ª¨¥. 3Ø∞∫ ¥∂´¨ ®≥≥∂æ∫ ªØ¨ ∑π∞¥®π¿ ®µ´ ∫¨™∂µ´®π¿ Ω∂≥º¥¨∫ª∂ ©¨ ¥∂π¨ ªØ®µ ∂µ¨ (/. ∂ºª ∂≠ ∫¿µ™Øπ∂µ∞¡®ª∞∂µ. 3Ø¨ ¥®ø∞¥º¥ µº¥©¨π ∂≠ (/.∫ ªØ®ª ™®µ ©¨ ∂ºª∂≠ ∫¿™Øπ∂µ∞¡®ª∞∂µ ∞∫ ´¨≠∞µ¨´ º∫∞µÆ ® ¥®ø∞¥º¥ ∫≤¨æ Ω®≥º¨.

C

Common Internet File System (CIFS)%∞≥¨-∫Ø®π∞µÆ ∑π∂ª∂™∂≥ ©®∫¨´ ∂µ ªØ¨ ,∞™π∂∫∂≠ª 2¨πΩ¨π ,¨∫∫®Æ¨ !≥∂™≤ (2,!). (ª ®≥≥∂æ∫ º∫¨π∫ ª∂∫Ø®π¨ ≠∞≥¨ ∫¿∫ª¨¥∫ ∂Ω¨π ªØ¨ (µª¨πµ¨ª ®µ´ ∞µªπ®µ¨ª∫.

D

delta set%∂π 21#%/ ∫¿µ™Øπ∂µ∂º∫ (21#%/ ), ® ∑π¨´¨ª¨π¥∞µ¨´ ™¿™≥¨ ∂≠ ∂∑¨π®ª∞∂µ º∫¨´ ≠∂π ®∫¿µ™Øπ∂µ∂º∫Ø∂∫ª æπ∞ª¨∫ ≠π∂¥ ® ∫∂ºπ™¨ ª∂ ® ´¨∫ª∞µ®ª∞∂µ. $®™Ø ´¨≥ª® ∫¨ª ™∂µª®∞µ∫ Æπ∂º∑∫ ∂≠ (/.∫ ≠∂π ∑π∂™¨∫∫∞µÆ.3Ø¨ ∂π´¨π∞µÆ ∂≠ ªØ¨∫¨ (/.∫ ∞∫ ¥®µ®Æ¨´ ≠∂π ™∂µ∫∞∫ª¨µ™¿. %∂π 5-7 1¨∑≥∞™®ª∂π, ® ∫¨ª ™∂µª®∞µ∫ ªØ¨


©≥∂™≤ ¥∂´∞≠∞™®ª∞∂µ∫ ¥®´¨ ª∂ ªØ¨ ∫∂ºπ™¨ ≠∞≥¨ ∫¿∫ª¨¥ ªØ®ª 5-7 1¨∑≥∞™®ª∂π º∫¨∫ ª∂ º∑´®ª¨ ªØ¨´¨∫ª∞µ®ª∞∂µ ≠∞≥¨ ∫¿∫ª¨¥ (π¨®´-∂µ≥¿, ∑∂∞µª-∞µ-ª∞¥¨, ™∂µ∫∞∫ª¨µª π¨∑≥∞™® ∂≠ ªØ¨ ∫∂ºπ™¨ ≠∞≥¨ ∫¿∫ª¨¥).3Ø¨ ¥∞µ∞¥º¥ ´¨≥ª®-∫¨ª ∫∞¡¨ ∞∫ 128 ,!.

dependent write consistency%∂π 21#%/ , ªØ¨¥®∞µª¨µ®µ™¨ ∂≠ ® ™∂µ∫∞∫ª¨µª ∑∂∞µª-∞µ-ª∞¥¨ π¨∑≥∞™® ∂≠ ´®ª® ©¨ªæ¨¨µ ® ∫∂ºπ™¨ ®µ´´¨∫ª∞µ®ª∞∂µ ªØπ∂ºÆØ ªØ¨ ∑π∂™¨∫∫∞µÆ ®µ´ ∑π¨∫¨πΩ®ª∞∂µ ∂≠ ®≥≥ æπ∞ª¨∫ ª∂ ªØ¨ ´¨∫ª∞µ®ª∞∂µ ∞µ ∂π´¨π¨´,µº¥©¨π¨´ ´¨≥ª® ∫¨ª∫.

destination VNX for file3¨π¥ ≠∂π ªØ¨ π¨¥∂ª¨ (∫¨™∂µ´®π¿) 5-7 ≠∂π ≠∞≥¨ ∞µ ®µ 21#% ∂π ,∞ππ∂π5∞¨æ/2 ™∂µ≠∞Æºπ®ª∞∂µ. 3Ø¨´¨∫ª∞µ®ª∞∂µ 5-7 ≠∂π ≠∞≥¨ ∞∫ ª¿∑∞™®≥≥¿ ªØ¨ ∫ª®µ´©¿ ∫∞´¨ ∂≠ ® ´∞∫®∫ª¨π π¨™∂Ω¨π¿ ™∂µ≠∞Æºπ®ª∞∂µ.2¿¥¥¨ªπ∞ø ™∂µ≠∞Æºπ®ª∞∂µ∫ ∂≠ª¨µ ™®≥≥ ªØ¨ ´¨∫ª∞µ®ª∞∂µ 5-7 ≠∂π ≠∞≥¨: ªØ¨ ª®πÆ¨ª 5-7 ≠∂π ≠∞≥¨.

L

local mirror2¿¥¥¨ªπ∞ø ∫Ø®´∂æ Ω∂≥º¥¨ ªØ®ª ∞∫ ® ™∂¥∑≥¨ª¨ π¨∑≥∞™® ∂≠ ® ∑π∂´º™ª∞∂µ Ω∂≥º¥¨ æ∞ªØ∞µ ªØ¨ ∫®¥¨∫ª∂π®Æ¨ ºµ∞ª. (≠ ªØ¨ ∑π∂´º™ª∞∂µ Ω∂≥º¥¨ ©¨™∂¥¨∫ ºµ®Ω®∞≥®©≥¨, (/. ™∂µª∞µº¨∫ ª∂ º∫¨ ªØ¨ ≥∂™®≥¥∞ππ∂πªπ®µ∫∑®π¨µª ª∂ ªØ¨ Ø∂∫ª.

See also R1 volume.

M

metavolume.µ ®5-7 ≠∂π %∞≥¨, ® ™∂µ™®ª¨µ®ª∞∂µ ∂≠ Ω∂≥º¥¨∫,æØ∞™Ø ™®µ ™∂µ∫∞∫ª ∂≠ ´∞∫≤, ∫≥∞™¨, ∂π ∫ªπ∞∑¨ Ω∂≥º¥¨∫. ≥∫∂ ™®≥≥¨´ ® Ø¿∑¨πΩ∂≥º¥¨ ∂π Ø¿∑¨π. $Ω¨π¿ ≠∞≥¨ ∫¿∫ª¨¥ ¥º∫ª ©¨ ™π¨®ª¨´ ∂µ ª∂∑ ∂≠ ® ºµ∞∏º¨¥¨ª®Ω∂≥º¥¨.

See also disk volume, slice volume, stripe volume, and volume.

MirrorView Synchronous (MirrorView/S)2∂≠ªæ®π¨ ®∑∑≥∞™®ª∞∂µ ªØ®ª ∫¿µ™Øπ∂µ∂º∫≥¿¥®∞µª®∞µ∫ ™∂∑∞¨∫ ∂≠ ∑π∂´º™ª∞∂µ ∞¥®Æ¨∫ (∫∂ºπ™¨ +4-∫)®ª ® ∫¨∑®π®ª¨ ≥∂™®ª∞∂µ ª∂ ∑π∂Ω∞´¨ ´∞∫®∫ª¨π π¨™∂Ω¨π¿ ™®∑®©∞≥∞ª¿. 3Ø¨ ™∂∑∞¨´ ∞¥®Æ¨∫ ®π¨ ™∂µª∞µº∂º∫≥¿º∑´®ª¨´ ª∂ ©¨ ™∂µ∫∞∫ª¨µª æ∞ªØ ªØ¨ ∫∂ºπ™¨, ®µ´ ∑π∂Ω∞´¨ ªØ¨ ®©∞≥∞ª¿ ≠∂π ® ∫ª®µ´©¿ 5-7 ≠∂π ≠∞≥¨ ª∂ª®≤¨ ∂Ω¨π ≠∂π ® ≠®∞≥¨´ 5-7 ≠∂π ≠∞≥¨ ∞µ ªØ¨ ¨Ω¨µª ∂≠ ® ´∞∫®∫ª¨π ∂µ ªØ¨ ∑π∂´º™ª∞∂µ ∫∞ª¨. 2¿µ™Øπ∂µ∂º∫π¨¥∂ª¨ ¥∞ππ∂π∫ (∫∂ºπ™¨ ®µ´ ´¨∫ª∞µ®ª∞∂µ +4-∫) π¨¥®∞µ ∞µ ∫¿µ™Øπ∂µ∞¡®ª∞∂µ æ∞ªØ ¨®™Ø ∂ªØ¨π ≠∂π¨Ω¨π¿ (/.. ,∞ππ∂π5∞¨æ/2 π¨∏º∞π¨∫ 5-7 ≠∂π ©≥∂™≤ ©®™≤¨µ´ ∫ª∂π®Æ¨.

Multi-Path File System (MPFS)5-7 ≠∂π ≠∞≥¨ ≠¨®ªºπ¨ ªØ®ª ®≥≥∂æ∫ Ø¨ª¨π∂Æ¨µ¨∂º∫ ∫¨πΩ¨π∫ æ∞ªØ ,/%2 ∫∂≠ªæ®π¨ ª∂ ™∂µ™ºππ¨µª≥¿®™™¨∫∫, ´∞π¨™ª≥¿ ∂Ω¨π %∞©π¨ "Ø®µµ¨≥ ∂π ∞2"2( ™Ø®µµ¨≥∫, ∫Ø®π¨´ ´®ª® ∫ª∂π¨´ ∂µ ® $," 2¿¥¥¨ªπ∞ø∂π 5-7 ≠∂π ©≥∂™≤ ∫ª∂π®Æ¨ ∫¿∫ª¨¥.,/%2 ®´´∫ ® ≥∞ÆØªæ¨∞ÆØª ∑π∂ª∂™∂≥ ™®≥≥¨´ %∞≥¨,®∑∑∞µÆ /π∂ª∂™∂≥(%,/) ªØ®ª ™∂µªπ∂≥∫ ¥¨ª®´®ª® ∂∑¨π®ª∞∂µ∫.

P

Production File System (PFS)/π∂´º™ª∞∂µ %∞≥¨ 2¿∫ª¨¥ ∂µ5-7 ≠∂π %∞≥¨. /%2 ∞∫ ©º∞≥ª ∂µ 2¿¥¥¨ªπ∞ø Ω∂≥º¥¨∫ ∂π 5-7 ≠∂π !≥∂™≤+4-∫ ®µ´ ¥∂ºµª¨´ ∂µ ® #®ª® ,∂Ω¨π ∞µ ªØ¨ 5-7 ≠∂π %∞≥¨.


Glossary

R

R1 volume21#% ª¨π¥ ´¨µ∂ª∞µÆ ªØ¨ ∫∂ºπ™¨ (∑π∞¥®π¿) 2¿¥¥¨ªπ∞ø Ω∂≥º¥¨.

See also local mirror.

R2 volume21#% ª¨π¥ ´¨µ∂ª∞µÆ ªØ¨ ´¨∫ª∞µ®ª∞∂µ (∫¨™∂µ´®π¿) 2¿¥¥¨ªπ∞ø Ω∂≥º¥¨.

See also remote mirror.

remote mirror%∂π 21#%, ® π¨¥∂ª¨ ¥∞ππ∂π ∞∫ ® 2¿¥¥¨ªπ∞ø ∫Ø®´∂æ Ω∂≥º¥¨ ∑Ø¿∫∞™®≥≥¿ ≥∂™®ª¨´ ∞µ ® π¨¥∂ª¨2¿¥¥¨ªπ∞ø ∫¿∫ª¨¥. 4∫∞µÆ ªØ¨ $," 21#% ª¨™Øµ∂≥∂Æ¿, ªØ¨ π¨¥∂ª¨ ∫¿∫ª¨¥ ∞∫ ±∂∞µ¨´ ª∂ ® ≥∂™®≥∫¿∫ª¨¥ æ∞ªØ ® ≥∂™®≥ ¥∞ππ∂π. (≠ ªØ¨ ≥∂™®≥ ¥∞ππ∂π ©¨™∂¥¨∫ ºµ®Ω®∞≥®©≥¨, ªØ¨ π¨¥∂ª¨ ¥∞ππ∂π ∞∫®™™¨∫∫∞©≥¨. %∂π ,∞ππ∂π5∞¨æ/2, ® π¨¥∂ª¨ ¥∞ππ∂π ∞∫ ® +4-¥∞ππ∂π¨´ ∂µ ® ´∞≠≠¨π¨µª 5-7 ≠∂π ©≥∂™≤.$®™Ø π¨¥∂ª¨ ¥∞ππ∂π ™∂µª®∞µ∫ ® ∑®πª∞™º≥®π ∫∂ºπ™¨ +4- (∑π∞¥®π¿ ∞¥®Æ¨) ®µ´ ∞ª∫ ¨∏º∞Ω®≥¨µª´¨∫ª∞µ®ª∞∂µ +4- (∫¨™∂µ´®π¿ ∞¥®Æ¨). (≠ ªØ¨ ∫∂ºπ™¨ ∫∞ª¨ ∫¿∫ª¨¥ ≠®∞≥∫, ªØ¨ ´¨∫ª∞µ®ª∞∂µ +4- ∞µ ªØ¨¥∞ππ∂π ™®µ ©¨ ∑π∂¥∂ª¨´ ª∂ ª®≤¨ ∂Ω¨π, ªØº∫ ®≥≥∂æ∞µÆ ®™™¨∫∫ ª∂ ´®ª® ®ª ® π¨¥∂ª¨ ≥∂™®ª∞∂µ.

See also R2 volume.

S

SnapSure.µ ªØ¨ 5-7 ≠∂π ≠∞≥¨, ® ≠¨®ªºπ¨ ªØ®ª ∑π∂Ω∞´¨∫ π¨®´-∂µ≥¿, ∑∂∞µª-∞µ-ª∞¥¨ ™∂∑∞¨∫, ®≥∫∂ ≤µ∂æµ ®∫™Ø¨™≤∑∂∞µª∫, ∂≠ ® ≠∞≥¨ ∫¿∫ª¨¥.

SRDF/Asynchronous (SRDF/A)21#% ¨øª¨µ´¨´-´∞∫ª®µ™¨ π¨∑≥∞™®ª∞∂µ ≠®™∞≥∞ª¿ ∑π∂Ω∞´∞µÆ ® π¨∫ª®πª®©≥¨, ∑∂∞µª-∞µ-ª∞¥¨ π¨¥∂ª¨ π¨∑≥∞™®ªØ®ª ≥®Æ∫ µ∂ª ≠®π ©¨Ø∞µ´ ªØ¨ ∫∂ºπ™¨. 4∫∞µÆ 21#%/ æ∞ªØ 5-7 ≠∂π ≠∞≥¨ ∑π∂Ω∞´¨∫ ´¨∑¨µ´¨µª æπ∞ª¨™∂µ∫∞∫ª¨µ™¿ ≠∂π Ø∂∫ª æπ∞ª¨∫ ≠π∂¥ ® ∫∂ºπ™¨ 5-7 ≠∂π ≠∞≥¨/2¿¥¥¨ªπ∞ø #,7 ∫¿∫ª¨¥ ∑®∞π ª∂ ®´¨∫ª∞µ®ª∞∂µ5-7 ≠∂π ≠∞≥¨/2¿¥¥¨ªπ∞ø#,7 ∫¿∫ª¨¥∑®∞π ªØπ∂ºÆØ ∑π¨´¨ª¨π¥∞µ¨´ ª∞¥¨ ™¿™≥¨∫ (´¨≥ª®∫¨ª∫).

SRDF/Synchronous (SRDF/S)21#% ™∂¥∑≥¨ª¨ ´∞∫®∫ª¨π π¨™∂Ω¨π¿ ™∂µ≠∞Æºπ®ª∞∂µ ∂∑ª∞∂µ ªØ®ª ∑π∂Ω∞´¨∫ ∫¿µ™Øπ∂µ∞¡¨´, π¨®≥-ª∞¥¨¥∞ππ∂π∞µÆ ∂≠ ≠∞≥¨ ∫¿∫ª¨¥ ´®ª® ©¨ªæ¨¨µ ªØ¨ ∫∂ºπ™¨ 2¿¥¥¨ªπ∞ø ∫¿∫ª¨¥ ®µ´ ∂µ¨ ∂π ¥∂π¨ π¨¥∂ª¨2¿¥¥¨ªπ∞ø ∫¿∫ª¨¥∫ ®ª ® ≥∞¥∞ª¨´ ´∞∫ª®µ™¨ (º∑ ª∂ 200 ≤¥). 21#%/2 ™®µ ∞µ™≥º´¨ $,"3∞¥¨%∞µ´¨πé/%2 -¨®π"∂∑¿ ∞µ ªØ¨ ™∂µ≠∞Æºπ®ª∞∂µ (®™ª∞Ω¨/∑®∫∫∞Ω¨; ®™ª∞Ω¨/®™ª∞Ω¨).

Symmetrix Remote Data Facility (SRDF)$," ª¨™Øµ∂≥∂Æ¿ ªØ®ª ®≥≥∂æ∫ ªæ∂ ∂π ¥∂π¨ 2¿¥¥¨ªπ∞ø ∫¿∫ª¨¥∫ ª∂ ¥®∞µª®∞µ ® π¨¥∂ª¨ ¥∞ππ∂π ∂≠´®ª® ∞µ¥∂π¨ ªØ®µ ∂µ¨ ≥∂™®ª∞∂µ. 3Ø¨ ∫¿∫ª¨¥∫ ™®µ ©¨ ≥∂™®ª¨´æ∞ªØ∞µ ªØ¨ ∫®¥¨ ≠®™∞≥∞ª¿, ∞µ ® ™®¥∑º∫,∂π Øºµ´π¨´∫ ∂≠ ¥∞≥¨∫ ®∑®πª º∫∞µÆ ≠∞©π¨ ∂π ´¨´∞™®ª¨´ Ø∞ÆØ-∫∑¨¨´ ™∞π™º∞ª∫. 3Ø¨ 21#% ≠®¥∞≥¿ ∂≠π¨∑≥∞™®ª∞∂µ ∫∂≠ªæ®π¨ ∂≠≠¨π∫ Ω®π∞∂º∫ ≥¨Ω¨≥∫ ∂≠ Ø∞ÆØ-®Ω®∞≥®©∞≥∞ª¿ ™∂µ≠∞Æºπ®ª∞∂µ∫, ∫º™Ø ®∫21#%/2¿µ™Øπ∂µ∂º∫ (21#%/2) ®µ´ 21#%/ ∫¿µ™Øπ∂µ∂º∫ (21#%/ ).


Glossary

T

TimeFinder/FS!º∫∞µ¨∫∫ ™∂µª∞µº®µ™¨ ™∂µ≠∞Æºπ®ª∞∂µ ®≥≥∂æ∞µÆ ™º∫ª∂¥¨π∫ ª∂ º∫¨ 2¿¥¥¨ªπ∞ø ©º∫∞µ¨∫∫ ™∂µª∞µº®µ™¨Ω∂≥º¥¨∫ (!"5∫) ª∂ ∑π∂Ω∞´¨ ® ≥∂™®≥ ∂π π¨¥∂ª¨ ∑∂∞µª-∞µ-ª∞¥¨ ™∂∑¿ ∂≠ 5-7 ≠∂π ≠∞≥¨.


Glossary

Index

A

activating SRDF/A failover 37, 59active/passive

configuration 19, 45failover

activating 59initializing 22, 45restoring 69SRDF/A use of 22

B

business continuance configurations 26

C

capture cycle 23cautions 52, 57, 69, 71CLI 13CNS-14 configuration 31command

nas_rdf -init 47SYMCLI 13

commandsfs_copy restriction 26nas_rdf 40, 42

comparison of related VNX for file features 26configuring

active/passive 22, 45Data Movers 30

Control Stationcommunication 19preinitialization 42

control volumes 20CS0 requirement 13cycle time 23

cycles of SRDF/A 23

D

Data Moverschecklist for 31configuring 30destination configuration 52mirroring 30source configuration 48

delta setdescription 23

dependent write consistencydescription 23

destination Celerra Network Server 59destination VNX

initializing 50running restore operation on 70

device groupsquerying 59, 67

differences between NS series gateway and CNS-1431differences between SRDF/A and SRDF/S 25

E

editing the/etc/hosts file 53EMC E-Lab Navigator 78error messages 79, 84, 91, 92, 93, 98

database lock error 93local file system on SRDF Data Mover 84local standby for SRDF standbys 79restore on NS series gateway 92


error messages (continued)starting restore...failed 91

F

failover 12, 37, 38, 56, 59initiating active/passive 37, 56, 59initiating SRDF/A 37restoring VNX after a 38

FileMover 14fs_copy command, no support for 26

G

graceful failover 57

H

halting Data Movers 34, 57health check 34, 39high-availability and replication products

Celerra Replicator (V2) 28MirrorView/Synchronous 26restrictions 26SnapSure 28SRDF/Asynchronous 27SRDF/Synchronous 26storage platform 26TimeFinder/FS 27TimeFinder/FS NearCopy and FarCopy 27

HTTP communication 51

I

initializingactive/passive 22, 45active/passive failover 37, 56SRDF relationship 36, 40, 42

IP subnets 69

L

license 12limits

nas_cel passphrase 42logical volumes 20login account 51

M

mapping Symmetrix volumes 21messages, error 98

mirroring Data Movers 30MPFS 14

N

nas_rdf -activate 37nas_rdf -init 36, 40, 42nas_rdf -restore 38NS series gateway configuration

Data Movers 31error messages 92

P

passphrase 42, 51for nas_cel preinitialization 42

physical volumes 20postactivation script 69preinitializing MirrorView/S relationship 42

R

receive cycle 23Replicator 14restore cycle 23restoring

active/passive 69steps on destination 71VNX post-failover 38

S

SnapSure 32SnapSure, restriction 13source VNX

initializing 47restoring 70

SRDFconfiguration types 19

SRDF/Acomparison to SRDF/S 25overview 22

starting restore...failed message 91Symmetrix DMX systems 12Symmetrix system configuration 21Symmetrix volume IDs mapping Symmetrix volumes21synchronous mode (SRDF/S) 25

T

transmit cycle 23


Index

troubleshooting 77

U

Unisphere software 13upgrade options 34

user interface 14

V

verifyingoperation after activation 63

VNX FileMover 14volumes

configuring 30


Index


Index

Documents

VNX SRDF_A