RecoverPoint Deployment with Brocade

EMC RecoverPoint 3.3 Deploying with Connectrix AP-7600B & PB-48K-AP4-18 Technical Notes 1

This document describes how to deploy EMC RecoverPoint with the EMC Connectrix AP-7600B and PB-48K-AP4-18 Storage Application Platforms to deliver a complete data replication solution without host-based agents.

Topics include:

◆ Revision history .................................................................................... 2◆ Overview of deployment .................................................................... 3◆ Useful Connectrix Control Processor commands.......................... 19◆ RecoverPoint deployment procedure.............................................. 20◆ RecoverPoint splitter agent life cycle .............................................. 32◆ Software and Firmware Upgrades................................................... 34◆ Migrating from multi-VI to frame redirection ............................... 40◆ Troubleshooting .................................................................................. 47

EMC® RecoverPoint 3.3Deploying with

Connectrix AP-7600B and PB-48K-AP4-18

Technical NotesP/N 300-010-644

REV A03

August 16, 2010

2 EMC RecoverPoint 3.3 Deploying with Connectrix AP-7600B & PB-48K-AP4-18 Technical Notes

Revision history

Revision historyTable 1 on page 2 shows the revision history for this document.

Table 1 Revision history

Revision Date Description

A03 August 2010 • Port 7777 must be open for log collection• added new section that describes required

Fabric Design• added best practices for switch front-panel ports,

and ISL requirements• corrected, organized, removed duplications, and

added much material to prerequisites for deployment. Topics include:solution qualifier approvalhardwarefirmwarehost requirements best practices

• updated maintenance procedures for updating FOS and SAS firmware.

• added SAN Health diagnostics procedures to deployment, upgrade, migration, and troubleshooting procedures.

• Added prerequisites concerning McData switches, Multi-VI mode, and supported Brocade switches

• Added non-disruptive upgrade of FOS only or SAS only.

3

Introduction

EMC RecoverPoint 3.3 Deploying with Connectrix AP-7600B & PB-48K-AP4-18 Technical Notes

IntroductionRecoverPoint supports deployment with the following EMC® Connectrix® application platforms:

◆ Connectrix AP-7600B stand-alone switch

◆ Connectrix PB-48K-AP4-18 blade (which slots into a Connectrix ED-48000B director for full functionality)

RecoverPoint deployment involves the installation of a RecoverPoint splitter agent on each Connectrix device. In RecoverPoint terms, this RecoverPoint splitter agent serves as a splitter, which is responsible for ensuring that all data routed to the switch by host applications (and that is to be replicated) is forwarded in parallel both to the SAN-attached storage and to a RecoverPoint appliance (RPA), from where it is replicated to target-side storage.

This document is intended primarily for IT professionals who are responsible for deploying RecoverPoint in a storage application services (SAS)-equipped Connectrix B-Series environment, and who are experienced in the areas of SAN management and topology.

Related documents Use the release of any of the following documents, available in the Documentation Library on http://Powerlink.EMC.com, that matches your installed RecoverPoint version:

◆ EMC Connectrix B Series Fabric OS Release Notes

◆ EMC Connectrix B Series AP-7600B Hardware Reference Manual

◆ EMC Connectrix B Series Fabric OS Administrator’s Guide

◆ EMC Connectrix B Series Fabric OS Command Reference Manual

◆ EMC RecoverPoint and RecoverPoint/SE Release Notes

◆ EMC RecoverPoint Administrator’s Guide

◆ EMC RecoverPoint CLI Reference Guide

◆ EMC RecoverPoint Deployment Manager Product Guide

Overview of deploymentThe new generation of EMC Connectrix B-Series storage virtualization solutions is delivered both as the AP-7600B stand-alone

http://Powerlink.EMC.com



Overview of deployment

switch, and as the PB-48K-AP4-18 blade, which slots into a Connectrix EC-48000B director. Both offer 16 Fibre Channel ports.

The management of these two platforms is divided between the following components:

◆ Control processor (CP). Runs the Fabric OS and handles layer 2 communications.

◆ Blade processor (BP). Runs the storage application services (SAS) and manages the virtualization layer. The RecoverPoint splitter agent is installed on the Blade Processor.

The Control Processor and Blade Processor have unique IP addresses. You can use the ipaddrshow switch CLI command to display the IP address for each.

Example To display the IP addresses for Control Processor and Blade Processor:

1. Log in to Control Processor as admin user.

2. Run ipaddrshow command:

Remote-AP7600B:admin> ipaddrshow

The following output is returned (when working on the AP-7600B platform):

SWITCHEthernet IP Address: xxx.xxx.xxx.xxx (=Control Processor)Ethernet Subnetmask: 255.255.255.0Fibre Channel IP Address: noneFibre Channel Subnetmask: noneGateway IP Address: xxx.xxx.xxx.1DHCP: Offeth0: xxx.xxx.xxx.xxx/xxx (=Blade Processor)eth1: none/noneGateway: xxx.xxx.xxx.1Remote-AP7600B:admin>

Common terms for the deployment

It may be helpful to refer to Figure 2 on page 8 while reading the following paragraphs.

◆ Data path controller. Both the AP-7600B and PB-48K-AP4-18 have two data path controllers (DPCs). The data path controller handles most I/O operations without further CPU involvement. You can associate a set of virtual initiators and virtual targets with each data path controller.

5



Note: In frame redirection mode, both data path controllers are utilized. Otherwise, only one data path controller is utilized.

◆ Host initiator. Each HBA on a host machine can be a host initiator (HI). Each host initiator is represented by a single virtual initiator on each data path controller that it uses.

◆ Virtual initiator. When working in multi-VI mode, a single virtual initiator (VI) is created to correspond with each host (HBA) initiator. In frame redirection mode, if the host is bound to targets on both data path controllers, a separate virtual initiator is created for that host for each data path controller. The WWN that is set for the virtual initiator is different from the WWN of the host initiator to which it corresponds. When working in multi-VI mode, LUN masking exposes the storage ports to the virtual initiator, rather than to the host initiator. In frame redirection, LUN masking does not change from that which is already in place for the host HBA.

◆ System virtual initiator — RecoverPoint splitter agent creates a special System virtual initiator, to support communication between the splitter and the RPAs.

◆ Virtual target. The virtual target (VT) is a representation on the switch of the actual storage target to which the host initiator directs it I/O. Its WWN is different, however, from the WWN of the storage target (as presented in “Virtual target WWNs” on page 11).

Note: The point at which the RecoverPoint splitter agent creates a virtual target depends on whether you are working in multi-VI or frame redirection mode. These modes are described in “Multi-VI and frame redirection modes” on page 6.

The RecoverPoint splitter agent creates the virtual target as follows:

• In multi-VI mode, upon first binding a host initiator to the virtual initiator to which the storage target is exposed. The host initiator then sends its I/O to the virtual target, rather than to the storage target.

• In frame redirection mode, upon first binding a host initiator to the storage target.



◆ Appliance virtual target. The appliance virtual target (AVT) is a special virtual target that ensures that the RPA has access to the storage targets for groups that use reservations, as described in “Reservations” on page 16.

◆ Storage target. The storage target is a physical port on the storage array, to which the data written by the host initiator is ultimately directed.

Multi-VI and frame redirection modes

The RecoverPoint splitter agent on AP7600B or PB-48K-AP4-18 platforms can be deployed in either frame redirection or multi-VI mode.

Multi-VI mode will impact the existing fabric in the following ways:

◆ LUN mapping must direct the LUNs to the virtual initiators instead of the host initiators.

◆ Zoning must be completely reorganized.

RecoverPoint can use the frame redirection capabilities of the AP-7600B and PB-48K-AP4-18 to alleviate the need for much of the reconfiguration.

When frame redirection is enabled, the name server deliberately gives “misleading” answers to FC ID queries that are generated by the host and target. As a result, host initiators send writes that are directed to the storage targets to the virtual targets, and storage targets see writes that come from the virtual initiators as if they are coming from the host initiators themselves.

Figure 1 on page 7 shows an environment in which the storage target with WWN W1 and FC ID F1 is exposing LUNs to the host initiator with WWN W2 and FC ID F2.

7



Figure 1 frame redirection communications

When deploying RecoverPoint in multi-VI mode, the host initiator must be zoned with the virtual target (with WWN W3 and FC ID F3) that represents the physical target, and the target must be zoned with and reconfigured to expose LUNs to the virtual initiator (with WWN W4 and FC ID F4) that represents the host initiator.

When working in frame redirection mode, however, the zone between the initiator and the target remains, but the RecoverPoint splitter agent automatically creates an additional frame redirection zone between the host initiator, virtual initiator, virtual target, and storage target. Then, whenever the initiator sends a query to the name server requesting the corresponding FC ID of target W1, it receives the FC ID F3, therefore directing its writes to the virtual target instead of to the actual target. Further, whenever the target requests the corresponding FC ID of initiator W2, it receives the FC ID F4, therefore treating writes from the virtual initiator as if they are received from the host initiator.



Basic configurations Figure 2 shows a virtual initiator that has access to three storage target devices. As a result, three virtual targets are created.

Figure 2 Single data path controller with virtual initiator access to multiple storage targets in multi-VI mode

Figure 3 shows an example, when using multi-VI binding, of multiple (in this case, two) virtual initiators with access to the same storage target devices.

9



Figure 3 Host initiators accessing the same storage targets (multi-VI binding)

Multiple virtual initiators can access the same storage target. While only one virtual target is created for that storage target, a different set of LUNs, from among the LUNs that belong to the storage target, can be exposed to each initiator. For example, the DMX storage port can expose LUNs (0,1,2) to one of the virtual initiators and LUNs (3,4,5) to the other virtual initiator (in multi-VI mode). As a result, the corresponding virtual target will expose LUNs (0,1,2) to one of the host initiators and LUNs (3,4,5) to the other host initiator, according to the HI-VI matching.



In this configuration, if the same set of storage devices is exposed to a single virtual initiator through multiple storage targets (that is, multipathing), a virtual target is created for each path.

Using dual DPCs in frame redirection mode

When working in frame redirection mode, the agent utilizes both data path controllers (DPCs) by alternating the data path controller on which it creates each new virtual target. Since virtual initiators and virtual targets that reside on different data path controllers cannot interact, then if a single initiator is bound to targets on both data path controllers, it will be necessary to represent that initiator with a separate virtual initiator on each data path controller. A target, however, is always represented by a single virtual target.

For example, if initiator I1 is bound to target T1 and initiator I2 is bound to Target T2, the RecoverPoint splitter agent creates VI1 and VT1 on DPC 0 and VI2 and VT2 on DPC 1. If I1 is bound also to T2, the agent must create another virtual initiator for I1 on DPC 1—since VI1, that resides on DPC 0 cannot interact with VT2 on DPC 1. When enabling reservation support for a group, the agent will create an AVT for each virtual initiator that represents a host that has access to a volume that supports reservations, up to one AVT per initiator per data path controller.

Although the total number of virtual entities may be higher when utilizing two data path controllers, the number of virtual entities per data path controller is always lower than when using a single data path controller. Hence, the possible scalability in this mode is always higher (since the limit is per data path controller). The number of used ITLs is not affected by the utilization of both data path controllers, but the maximum possible number of ITLs doubles, as does the number of virtual initiators.

To determine the number of virtual entities on each data path controller, use the 4-bit DPC number in the WWN to determine the DPC number of each virtual target (as described under “Virtual target WWNs”). Then, use the binding scheme to determine the number of virtual initiators for each data path controller. Note that an additional virtual initiator (the “System virtual initiator”) will be created on DPC 0.

Note: Dual data path controller utilization is not supported for multi-VI mode.

11



Virtual target WWNs Figure 4 shows the format for the virtual target PWWN.

Figure 4 virtual target WWN format

For example, a virtual target created on DPC 0, where the WWN of the corresponding storage target is 0x1234567890123456, and the hash of that WWN is 0xabcd, would be:

0x70012482abcd0456

These WWNs are displayed by the get_virtual_targets CLI command, and are used to define front-end zoning on the switch, when working in multi-VI mode.

Even though the WWNs for virtual targets are different from the WWNs of the corresponding storage targets, the virtual targets borrow their SCSI personality from the storage targets (except, in some cases, for the LUN size, which RecoverPoint may “fake”). Therefore, the host initiator sees the virtual targets as the same type of storage as the actual storage target, including vendor, product, and UID. As a result, any multipathing software installed on the host works with the virtual targets as if they are the actual physical storage targets.

Splitter identification The RecoverPoint splitter agent behaves as a standard splitter in the RecoverPoint Management Application or CLI. The OS type for the splitter is designated as Brocade AP. The splitter name will be set during agent installation, which is presented in Step 1 on page 25 in “RecoverPoint deployment procedure”.

The format of the splitter name is as follows:

SW_name-SW_IPaddress

where:

SW_name is the name of the Blade Processor that you specified during agent installation; it can be obtained by running hostname on the Blade Processor.



SW_IPaddress is the IP address of the Blade Processor (at time of agent installation); it can be obtained by running ifconfig on the Blade Processor.

To determine the exact version of the splitter agent, run the following command on the Blade Processor:

> cat /thirdparty/recoverpoint/tweak/tweak.params.version

LUN configuration Note: This section is relevant only when working in multi-VI mode; LUN configuration changes are not needed in frame redirection mode.

LUNs are provisioned (or exposed) to initiators according to the LUN configuration. When deploying RecoverPoint on the Connectrix AP-7600B or PB-48K-AP4-18, and working in multi-VI mode, the LUN configuration must be directed at the virtual initiators, rather than the host initiators.

In most cases, an initiator must log in to a storage array before the storage allows LUN configuration related to that initiator. As part of the boot process, the RecoverPoint splitter agent performs SAN discovery, during which it logs in to all the storage targets from all virtual initiators. Therefore, once the RecoverPoint splitter agent is running, and you have created virtual initiators (through the process of binding the host and virtual initiators) and the Connectrix zone (see Table 2 on page 13), it should be possible to set the LUN configuration for the virtual initiators.

The best practice for changing the LUN configuration is:

1. Set HI-VI binding, through which the WWNs are set for the virtual initiators.

2. Create zones (virtual initiator-target zones) based on previous host-storage zones.

3. Modify device masking by substituting the host initiator WWN with the virtual initiator WWN (to which that host initiator is now bound).

Zoning The following types of Fibre Channel entities should be zoned when deploying RecoverPoint on the Connectrix AP-7600B or PB-48K-AP4-18:

13



◆ Host initiators

◆ Physical storage targets

◆ Virtual initiators

◆ System virtual initiator

◆ Virtual targets

◆ RPA initiator ports

◆ RPA target ports

◆ Appliance virtual targets

Table 2 lists the minimal zoning that you will need to define during deployment of RecoverPoint, as instructed in “RecoverPoint deployment procedure” on page 20. Note that working in frame redirection mode requires somewhat different zoning than working in multi-VI mode. The zones required when working in multi-VI and frame redirection modes are pictured in Figure 5 on page 14 and Figure 6 on page 15, respectively.

You will create and populate each of these zones according to the instructions in the “RecoverPoint deployment procedure” on page 20.

Table 2 Required zoning

Zone Members Multi-VI

frame redirect

ion

Front-end zone host initiator, virtual targets

Connectrix zone virtual initiator, physical storage targets

Host-storage zonea physical host initiator, physical storage targets. Both host and storage ports must be zoned using pWWNs, not nWWNs. pWWN zoning is the only type that is supported with Frame Redirection.

RPA front-end zoneb

RPA initiator ports, appliance virtual targets

RPA target zone virtual initiator, system virtual initiator, RPA target portsc

RPA initiator zone RPA initiator ports, physical storage targetsc



The following images are examples of zoning for multi-VI and frame redirection modes, without reservation support:

Figure 5 Required zoning for multi-VI, without reservation support

a. In “RecoverPoint deployment procedure” on page 20, it is assumed that this zone already exists prior to deployment.

b. This zone is needed only when working with reservations. “Reservations” on page 16 provides details. It is not shown in Figure 5.

c. When using RPAs with two dual-ported QLE24xx 4-Gb HBAs or QLE-2564 8-GB HBAs, each RPA port can serve both as an initiator and as a target.

15



Figure 6 Required zoning for frame redirection, without reservation support

It is possible to split these zones further, as needed. For example, when working in multi-VI mode, the front-end zone can be divided into several zones, where each contains a single host initiator and the relevant virtual targets. When working in frame redirection mode, the same can be done to the host-storage zone.

FOS/SAS compatibility check

The RecoverPoint Brocade splitter agent functions correctly only with a specific version of the fabric operating system (FOS) and storage



application services (SAS). RecoverPoint release notes list the exact version of the fabric operating system and storage application services that are compatible with the specific version of RecoverPoint. Not using the exact supported combination of fabric operating system and storage application services may result in splitter failure or unexpected behavior. To prevent this, RecoverPoint verifies that the fabric operating system and the storage application services are compatible with the installed splitter agent at the following times:

◆ During installation of the splitter agent. If the fabric operating system and storage application services version are not compatible with those of the splitter agent, the correct versions are displayed on screen and the installation is aborted.

◆ When the splitter agent is started. If a mismatch is found, the splitter agent will crash.

If in either of these cases a mismatch is detected, the system cannot be restarted until the mismatch is corrected.

Reservations Since reservation commands (both SCSI-2 and SCSI-3) are forwarded to the target storage devices, the storage devices are reserved, but by a virtual initiator, rather than by a host.

When replicating with RecoverPoint, the RPAs must be able to send I/O to these storage devices, even if they are reserved by some other initiator. The only way for the RPA to reach the target storage is through a virtual target, and from there (to the target storage) by way of a virtual initiator. The RPA cannot, however, use an existing virtual target to access the storage target, since each virtual target can use more than one virtual initiator to access the backend device, and each RPA port accessing the virtual target must be bound to a single virtual initiator.

Therefore, the RPA uses a special virtual target, called an appliance virtual target (AVT) to enable it to access the storage target. Whenever you specify (in the RecoverPoint Management Application or CLI) that a group side supports reservations, the RecoverPoint splitter agent creates one AVT for each virtual initiator that represents a host initiator that has access to volumes within this group. For every LUN that the initiator may reserve, a corresponding LUN that points to the same physical storage LUN is created on the AVT.

Because a single initiator may access LUNs on more than one storage target, the LUNs on the AVT may represent as many different storage targets as the number of storage targets on which the corresponding

17



initiator can reserve LUNs. Also, all LUNs on the AVT are created in pass-through mode, so that I/Os sent to the AVT LUNs are not split (which would cause the I/Os to be sent back to the RPA), but rather, are written directly to the storage target. Access to the AVT is limited so that only RPA ports can access the LUNs on the AVT.

When replicating a group that supports reservations, if the regular path to the target storage device returns a “Reservation conflict” error, the RPA then checks other potential paths, including the AVTs. Since the RPA cannot determine which host initiator is reserving the device, all existing AVTs are tried until one is found that allows access to the target storage.

A list of 500 pWWNs is produced during installation of the RecoverPoint splitter agent, and is placed in the file /thirdparty/recoverpoint/init_host/scimitar_wwns_list.txt on the Blade Processor module. The WWNs are unique for the switch. They are available for subsequent use by AVTs.

The RPA port must have access to the virtual target, so zoning must be set correctly according to the specification listed in “Zoning” on page 12. All of the WWNs from the scimitar_wwns_list.txt file should be zoned as AVTs.

Scalability limitations

The limitations of Connectrix AP-7600B or PB-48K-AP4-18 resources when deployed with RecoverPoint are published in the release notes for your release of RecoverPoint. Limits are presented for the following parameters:

◆ Host initiator-target nexuses

◆ Total virtual entities, including virtual initiators, virtual targets, AVTs, and the system virtual initiator

◆ Replicated LUNs

◆ Total LUNs

◆ ITLs

It is important to review these limitations when planning the deployment of RecoverPoint with your Connectrix AP-7600B or PB-48K-AP4-18.

Computing ITLs Total number of ITLs is the sum of host ITLs and AVT ITLs, where:

Host ITLs = number of LUNs to which a host initiator has access

AVT ITLs = Host ITLs (for which LUNs support reservations)



Example 1: For example if, when using one data path controller, the configuration includes:

◆ 4 RPA initiator ports

◆ 2 host initiators

◆ With access to 1000 LUNs

◆ 700 of the LUNs support reservations

Then the total number of ITLs is as follows:

ITLs = [(2 host initiators) x (1000 LUNs)] + [(2 host initiators) x (700 host ITLs for which LUNs support reservations)]

= 2000 + 1400

= 3400

Example 2: If, however, using frame redirection, you are able to use both data path controllers, then ITLs are calculated separately for each data path controller. Hence, if the configuration in the first example is changed, as follows:

◆ 500 (of the 1000 LUNs in the first example) are exposed to the hosts through DPC 0, and the remaining 500 are exposed through DPC 1

◆ All 500 of the hosts in DPC 0 support reservations; 200 of the hosts in DPC 1 support reservations

Then the total number of ITLs is as follows:

ITLs = {ITLs for DPC 0} + {ITLs for DPC 1}

= {[(2 host initiators) x (500 LUNs)] + (1000 host ITLs for which LUNs support reservations)} + {[(2 host initiators) x (500 LUNs)] + (400 host ITLs for which LUNs support reservations)}

= {1000 + 1000} + {1000 + 400}

= 3400

Note: The system displays warnings whenever your system approaches or exceeds scalability limits for ITLs, virtual entities, and total LUNs. Realtime ITL usage can be monitored in the Management Application GUI, from the Splitters tab for the System Monitoring component.

19

Useful Connectrix Control Processor commands


Useful Connectrix Control Processor commandsThe following commands are available through the Control Processor CLI:

◆ nsshow

Shows items currently in the switch name server. For each target, shows its PWWN, NWWN, and symbolic name. Also shows virtual initiators (“Brocade virtual Initiator-slot#,DPC#”) and virtual targets (“Brocade virtual target-slot#,DPC#”).

◆ switchshow

Shows switch information, including port information (WWN connected to the port, port status).

◆ cfgshow

Shows zoning configuration, including active and passive configuration and the zones included in each.

◆ firmwareshow

Shows the version number for the fabric operating system (FOS) installed on the Control Processor and the storage application services (SAS) installed on the Blade Processor.

◆ ipaddrshow

Shows the IP address of the Control Processor module and Blade Processor module.

◆ ipaddrset

Sets the IP address of the Control Processor module and Blade Processor module.

◆ firmwaredownload

Used to install the fabric operating system (FOS) on the Control Processor and the storage application services (SAS) on the Blade Processor.

Note: When using this command to upgrade the storage application services (SAS) version, virtualized switch traffic (that is, all traffic between virtual initiators and virtual targets) is disrupted.


RecoverPoint deployment procedure

◆ supportshow

Outputs a range of diagnostic information about the switch configuration and status. It is generally recommended to direct this output to a file.

◆ diagDisablePost

Disables Power On Self Test when switch is rebooted. The mode state is saved in flash memory and POST remains disabled until it is enabled using diagEnablePost. Diagnostic POST should be disabled before upgrading fabric operating system (FOS) or storage application services (SAS) firmware.

The EMC Connectrix B Series Fabric OS Command Reference Manual provides detailed information about these, and all other, switch CLI commands.

RecoverPoint deployment procedureThis section outlines the procedure for deploying RecoverPoint with the Connectrix AP-7600B or PB-48K-AP4-18 and Brocade splitter.

Prerequisites RecoverPoint deployment with the Connectrix B-Series application platforms requires the following items.

Hardwarerequirements

◆ One of the following per fabric (minimum two fabrics per site):

• Connectrix AP7600B Application Platform (intelligent switch)

• Connectrix ED48000B with PB-48K-AP4-18 intelligent blade

• Brocade DCX-4S with PB-48K-AP4-18 intelligent blade

Multi-VI and FrameRedirection modes

◆ Previously, RecoverPoint implementations supported both Multi-VI and Frame Redirection mode. Currently, only Frame Redirection mode is supported for new implementations. Existing customers using Multi-VI mode are currently supported.

Firmwarerequirements

◆ RecoverPoint splitter agent must be compatible with the version of RecoverPoint. Unless instructed otherwise, the RecoverPoint and the RecoverPoint splitter agent version number must be identical.

◆ The SAS version must be compatible with the RecoverPoint splitter agent version. Consult the EMC Support Matrix for compatible versions.

21



◆ The FOS version must be compatible with the SAS version. Consult the EMC Support Matrix for compatible versions.

Fabric design Hosts that are exposed to replicated volumes are bound to the intelligent fabric splitter. As a result, all I/Os between the bound host initiator and the storage target are routed via the intelligent switch or blade. In consequence:

◆ Disruption of the intelligent switch or blade operation distrusts the host-to-storage path of the bound initiator-target pair. Host-to-storage multipathing is required to survive a splitter failure.

◆ Fibre Channel front-panel ports on intelligent switches and blades do not share resources with the splitter I/O traffic. Connecting devices to the Fibre Channel front-panel ports is supported.

◆ Maintenance on the splitter may disrupt operation of the Fibre Channel front-panel ports and the devices connected to it. The best practice is, therefore, not to connect any devices to the front-panel ports. If the front-panel ports must be used, it is strongly recommended to connect only devices that participate in replication. In order of preference, the following devices may be connected: 1) RPAs, 2) bound host initiators, and 3) bound storage targets.

◆ If replication I/O traffic flows over interswitch links (ISLs), plan for sufficient ISL bandwidth. The minimum requirements for connecting the AP7600 to an existing fabric is 6 ISLs, three connected to ports 0–7, and three connected to ports 8–15.

Fabric requirements ◆ Previously, RecoverPoint implementations supported both Brocade and McData switches in the fabric. Currently, only Brocade switches are supported in new deployments. McData switches are not supported in the fabric, even if no initiators or targets are connected to them.

Existing customers with McData switches in the fabric should consult EMC Knowledge base article emc244711.



◆ When using Frame Redirection mode, every switch in the fabric must be able to support Frame Redirection, even if no initiators or targets are connected to the switch. The Fabric Operating System (FOS) version of each switch in the fabric should be 6.1.0e or higher.

The following Brocade switches do not support Frame Redirection mode and cannot be included in a fabric replicating with the RecoverPoint Brocade splitter:

Table 3 Brocade switches not supported when using RecoverPoint Brocade splitter

Switch type Switch Model

1 Brocade 1000 Switches

2, 6 Brocade 2800 Switch

3 Brocade 2100, 2400 Switches

4 Brocade 20x0, 2010, 2040, 2050 Switches

5 Brocade 22x0, 2210, 2240, 2250 Switches

7 Brocade 2000 Switch


10 Brocade 12000 Director



17 Brocade 3800VL


21 Brocade 24000 Director





23



◆ The best practice is that the FOS version of all switches in the fabric should be identical. If you cannot use the same FOS on all switches, consult the EMC Support Matrix for compatible versions.

Network requirements ◆ Port 7777 (hlr_kbox) must be open between all RecoverPoint appliances and all intelligent Brocade switches and blades in the fabric, otherwise log collection may fail.

Host requirements: ◆ Verify that your host configuration is supported. Check the EMC Support Matrix.

◆ Host-to-storage multipathing is required.

CAUTION!Brocade splitter binding and unbinding causes the FC ID of affected HBAs to change. HP-UX hosts without agile addressing and AIX hosts without dynamic tracking cannot automatically rediscover the paths to storage if the FC ID has changed. To discover the new paths, it may be necessary to reconfigure the hosts, rescan the SAN, or take other action to have the hosts recognize the new paths to storage. Host downtime may be required.

For more information about FC ID changes with AIX hosts, refer to the following:

• EMC Host Connectivity Guide for IBM AIX

(powerlink.emc.com, Support > Technical Documentation and Advisories > Installation/Configuration > Host Connectivity/HBAs > HBAs Installation > Configuration)

• EMC knowledge base solutions emc91523 and emc115725

• Deploying RecoverPoint with AIX Hosts Technical Notes

(powerlink.emc.com, Support > Technical Documentation and Advisories > Software ~P–R~ Documentation > RecoverPoint > Technical Notes/Troubleshooting)

For more information about FC ID changes with HP-UX hosts, refer to the following:

• EMC Host Connectivity Guide for HP-UX

powerlink.emc.com

powerlink.emc.com




• EMC Knowledge base solution emc199817.

Approved SolutionsQualifier

Any Brocade splitter installation requires a solutions qualifier approved by EMC.

◆ A Brocade SAN Health report must be submitted with the application for solutions qualifier approval. For instructions, refer to “SAN Health Report” on page 48.

◆ The application for solutions qualifier approval is located at http://gig.corp.emc.com/ under Business Continuity > Optional Presales Qualifiers > RecoverPoint and Business Continuity > Mandatory Postsales Qualifiers > RecoverPoint.

Preparing for installation

To prepare for deploying RecoverPoint in a Connectrix AP-7600B or PB-48K-AP4-18 environment:

1. Ensure that the host-storage zones exist, and that LUN masking from the targets to the initiators is defined.

If working in multi-VI mode, these zones will be replaced later with front-end zones.

2. Verify that every host can access every presented LUN through all available paths.

Installing RecoverPoint appliances

1. Physically connect the RPAs to the Connectrix devices.

2. Create the RPA initiator zone on each fabric, and add all RPA initiators and relevant storage targets on the fabric to that zone.

3. Install RecoverPoint on the RPAs. The EMC RecoverPoint Deployment Manager Product Guide provides RPA installation instructions.

Installing FOS and SAS

The fabric operating system (FOS) and storage application services (SAS) combination must be supported by the version of RecoverPoint to which you are upgrading. Refer to the EMC Support Matrix for your version of RecoverPoint for supported storage application services (SAS) and fabric operating system (FOS) combinations.

1. On the Control Processor, run the command:

http://gig.corp.emc.com

http://gig.corp.emc.com/

powerlink.emc.com

25



> diagDisablePost

Disable Diagnostic Power-On Self-Test (POST) before upgrading either fabric operating system (FOS) or storage application services (SAS) firmware. Failing to disable POSTs may cause the switch to go into an error state when upgrading firmware.

2. Upgrade the fabric operating system (FOS) and storage application services (SAS) using the following procedure.

CAUTION!Upgrade the fabric operating system (FOS) and storage application services (SAS) on all switches and blades, one fabric at a time to avoid risking downtime.

a. To upgrade the FOS, on the Control Processor, use the firmwaredownload -c command. The -c option disables the FOS/SAS compatibility check.

b. To upgrade the SAS, on the Control Processor, use the firmwaredownload command.

If the upgrade of the SAS fails because of the compatibility check, retry with the firmwaredownload -c command.

At conclusion of the firmware upgrade, the hardware reboots automatically.

3. If you wish to enable Diagnostic Power-On Self-Tests (optional), on the Control Processor, run the command:

> diagEnablePost

4. It is recommended to clear all port statistics. On the Control Processor, run the following command:

> slotstatsclear

Installing the splitter agent

After installing fabric operating system (FOS) and storage application services (SAS), install the splitter agent. Use the following procedure.

1. Install the RecoverPoint splitter agent:

a. Download the RecoverPoint splitter agent installation file from Powerlink at:





and copy the installation file onto the Blade Processor.

To do so:

1. Log in to Blade Processor as root. 2. Use ftp to download the RecoverPoint splitter agent binary

file to the /tmp directory:> cd /tmp> ftp ip_address ftp> binftp> get agent_binary_fileftp> bye

b. Run the installation package.

At the Blade Processor, use the following commands:

> cd /tmp> chmod +x agent_binary_file> ./agent_binary_file

This extracts the RecoverPoint splitter agent files under /thirdparty/recoverpoint.

c. When prompted, enter the hostname for the splitter agent.

The _ip address will automatically be appended to the name you enter here. The name with appended _ip address is the name of the splitter as it is displayed by the RecoverPoint Management Application.

2. If working in multi-VI mode, create the RPA target zone and RPA front-end zone as follows:

a. On the Blade Processor, find the WWN for the system virtual initiator, which is labeled “system VI WWN” in the file:

/thirdparty/recoverpoint/init_host/scimitar_wwns_list.txt

b. Create the RPA target zone, and add the system virtual initiator and RPA targets to it. This enables the RPA to locate the Connectrix-based splitter.

c. Create the RPA front-end zone, and add all RPA initiators and appliance virtual targets (AVTs) to the zone. The AVT WWNs are listed in:

/thirdparty/recoverpoint/init_host/scimitar_wwns_list.txt

27



3. If working in frame redirection mode, create the RPA target zone and RPA front-end zone as follows:

a. On the Blade Processor, run the script:

> /thirdparty/recoverpoint/install/zoning_script.sh

The script does the following:

– The script creates the RPA target zone (hostname_FR_RPA_Target_Zone) and the RPA front-end zone (hostname_RPA_Front_End_Zone).

– Adds the system virtual initiator, and all possible virtual initiator WWNs to the RPA target zone

– Adds all appliance virtual targets (AVTs) to the RPA front-end zone

– Adds RPA ports to both zones. Instead of using the script, you can create those zones manually, using the WWNs list and zone creation examples located in:

/thirdparty/recoverpoint/init_host/scimitar_wwns_ list.txt

b. Add or remove RPA pWWNs to or from RPA zones as follows.

If you are using initiator-target separation, remove RPA initiator ports from the RPA target zone and RPA target ports from the RPA initiator zone.

If multiple RPA clusters are connected to the same fabric, all RPA ports of all clusters will be added to the zoning scripts. Remove any RPA ports from the created RPA zones that should not be included.

c. Add both zones to the effective configuration, and re-enable the configuration.

4. Reboot the Blade Processor. Log in to the Blade Processor and run the following command:

> reboot

When the Blade Processor comes up, the RecoverPoint splitter agent should be activated. You can verify that it is activated by using the kdrv status command (“Running the RecoverPoint splitter agent” on page 32).



Configuring RecoverPoint and

splitters

At this point, RecoverPoint appliances have been installed; fabric operating system (FOS) and storage application services (SAS) and splitters agents were installed. The next step is to add splitters and create bindings.

1. Run the rescan_san command with parameter volumes=full to rescan all volumes in the SAN, including those that have been changed. This action may take several minutes to complete.

2. Add all splitters.

3. Use the following instructions to create bindings.

CAUTION!Brocade splitter binding and unbinding causes the FC ID of affected HBAs to change. HP-UX hosts without agile addressing and AIX hosts without dynamic tracking cannot automatically rediscover the paths to storage if the FC ID is changed. To discover the new paths, it may be necessary to reconfigure the hosts, rescan the SAN, or take other action to have the hosts recognize the new paths to storage. Host downtime may be required.











powerlink.emc.com

powerlink.emc.com

powerlink.emc.com

29



CAUTION!Create bindings one fabric at a time to avoid risking downtime. After adding bindings to a fabric, ensure that the hosts still have access to targets.

When working in frame redirection mode, it is required to keep both initiator and target available on the fabric.

Create bindings for one splitter at a time:

a. If working in Multi-VI mode, continue at Step 4 on page 29.

b. If working in frame redirection mode, skip to Step 10 on page 31

4. This and the following steps (through Step 9 on page 30) apply only when working in multi-VI mode.

Use the bind_host_initiators CLI command (multi-VI bindings cannot be created in the RecoverPoint GUI) to configure host binding to virtual initiators, as in the following example:

s1-bos> bind_host_initiators site=s1-bos splitter=myhostname_172.16.0.17 host_initiators=10000000c92ab3bdInitiator binding(s) successfully added.

The command returns the WWN of the virtual initiator that is bound to the host initiator.

Note: The default value for the frame_redirect parameter is “no”; when working in multi-VI mode, it should not be changed.

Alternatively, you can specify the virtual initiator that is to be bound to the host initiator; for example, so that the virtual initiator has the same WWN as the virtual initiator to which the host initiator was previously bound. To do so, add the following parameter to the bind_host_initiators command:

virtual_initiator=60012482xxxxxxxx

The WWN for the virtual initiator must always begin with the 60012482 prefix, whether it is returned by the system or specified by the user. The remainder of this WWN, following the 60012482 prefix, can be any value.



Verify that virtual targets are created on the Blade Processor for each available target storage port.

5. Add the virtual initiator WWNs that you obtained during the binding process to the RPA target zone.

6. Create the Connectrix zone, and add all virtual initiators and physical storage targets to the zone.

7. Define storage LUN masking for the virtual initiators according to the host binding scheme.

For example, if host HI1 is bound to VI1, duplicate the LUN configuration defined for HI1 to VI1.

8. Run rescan_san from the RecoverPoint CLI, and then use the get_virtual_targets CLI command to obtain the list of virtual targets accessible to each host initiator.

The following presents an example of the get_virtual_targets CLI command, run in interactive mode, together with the output generated:

s1-bos> get_virtual_targetsEnter the site names1-bosEnter Brocade splitter namemyhostname_172.16.0.12Host initiators: (default is all)1) 10000000c92ab3bdSelect, or press 'ENTER':----------------------------------------------------Host Initiator Virtual Target Original Port----------------------------------------------------10000000c92ab3bd 70012482ff711a43 5006016210601a43 70012482e7b41a43 5006016a10601a43

Create the front-end zone, and add the host initiators and virtual targets to the zone. Alternatively, you can split the front-end zones further, as desired. For example, you can create zones where each contains a single host initiator and the relevant virtual targets.

9. Remove the host-storage zones, since host-storage connectivity is now handled by front-end zones.

Skip to Step 11 on page 31.

31



10. This step applies only when working in frame redirection mode.

Use the bind_host_initiators CLI command to configure host binding to storage targets, as in the following example:

s1-bos> bind_host_initiators site=s1-bos splitter=sabre_172.16.0.17 host_initiators=10000000c92ab3bd target=500611111111 frame_redirect=yes

Initiator binding(s) successfully added.

Alternatively, you can add frame redirection bindings in the RecoverPoint Management Application, by using Brocade bindings, under Splitter Properties.

Whether you use the CLI or the Management Application, this command creates the virtual initiators and virtual targets that correspond to the host initiators and storage targets involved, and that enable the frame redirection mechanism to direct the data frames from the initiator to the target through the virtual entities. Each initiator should be bound to all targets that expose protected LUNs to it. You can bind multiple initiators to multiple targets.

A special zone configuration, r_e_d_i_r_c__fg, is created. This zone configuration contains the red_______base zone. The other zones in the zone configuration (whose names start with lsan_red…) contain physical host initiator and physical storage target pWWNs; and the corresponding virtual initiators and virtual targets. Each lsan_red… zone corresponds to a frame redirection binding. Under normal circumstances, r_e_d_i_r_c__fg should be managed only through RecoverPoint and should not be altered manually.

11. After adding bindings to the fabric, ensure that the hosts still have access to targets.

Repeat from Step 3 on page 28 to Step 10 on page 31 for each splitter.

12. Create consistency groups and configure RecoverPoint replication policies Refer to the EMC RecoverPoint Administrator’s Guide, chapter “Starting Replication,” sections “Creating new consistency groups” and “Configuring Replication Policies.”

13. Restart replication.


RecoverPoint splitter agent life cycle

CAUTION!Enabling the consistency groups will trigger a full sweep. If fast first-time initialization is enabled, the last complete and consistent image on the storage LUN will become inconsistent soon after the full sweep starts. However, if fast first-time initialization is disabled, the last complete and consistent image will be preserved as long as sufficient space is available in the replica journal.

Enable the consistency groups.

RecoverPoint splitter agent life cycleThis section presents information specific to the RecoverPoint splitter agent, which is installed on the Blade Processor module.

Running the RecoverPoint splitter agent

The following commands are useful when running the RecoverPoint splitter agent on the Blade Processor module on the Connectrix AP-7600B or PB-48K-AP4-18:

◆ To start the agent’s high-level process, assuming the low-level process is already running:

/thirdparty/recoverpoint/install/kdrv start_high

Note: This command starts the high-level process only. It is not recommended to use the kdrv start command to manually start the low-level process. For an explanation of the high-level and low-level processes, see “When the RecoverPoint splitter agent is stopped or crashes” on page 32.

◆ To check agent status:

/thirdparty/recoverpoint/install/kdrv status

◆ To stop the agent:

/thirdparty/recoverpoint/install/kdrv stop

When the RecoverPoint splitter agent is stopped or crashes

As noted earlier in this section, the RecoverPoint splitter agent runs separate “low-level” and “high-level” processes.

If the low-level process crashes, the splitter attempts to recover from the crash by importing the virtual entities as non-disruptively as

33

RecoverPoint splitter agent life cycle


possible. Recovering from a crash should not impact the I/O path from the hosts unless the platform itself crashes internally. Except in the following conditions, if the attempt to recover non-disruptively fails, the Blade Processor reboots automatically:

◆ The agent was stopped manually, using the kdrv stop command, to allow maintenance or upgrade of the switch.

◆ The agent detected and aborted a “reboot cycle”, in which the agent repeatedly crashed upon initialization after reboots.

The cause of this problem should be corrected before the switch is manually rebooted.

If the high-level process crashes, while the low-level process is functioning, the high-level process is automatically reactivated, without affecting the host data path. The high-level process is not reactivated if:

◆ The low level process was stopped (either manually or by an error), and the switch has not yet rebooted.

◆ The agent detects a “restart cycle”, in which the high-level process repeatedly crashes upon initialization. In this case, the high-level process will remain down, while the low-level process may continue functioning. If the cause of the problem is fixed without a switch reboot, the high-level process can be reactivated manually using the kdrv start_high command.

Upgrading the RecoverPoint splitter agent

The procedure for upgrading a RecoverPoint splitter agent is presented on page 38.

Uninstalling the RecoverPoint splitter agent

To uninstall the RecoverPoint splitter agent, on the Blade Processor, run the following script:

/thirdpart/recoverpoint/install/uninstall.sh

This script stops the splitter agent if necessary and removes all RecoverPoint files from the Blade Processor. The script also prompts the user about keeping splitter logs and persistent information before uninstalling. Logs and persistent information are kept on the Blade Processor in the directory /thirdparty/backup_logs.


Software and Firmware Upgrades

Software and Firmware UpgradesBefore performing any upgrade procedure, it is best practice to run the Brocade SAN Health Diagnostics Capture. For instructions, refer to “SAN Health Report” on page 48.

Upgrading to RecoverPoint 3.3 in the Brocade environment is only supported from RecoverPoint 3.2. To upgrade earlier versions to RecoverPoint 3.2, refer to EMC RecoverPoint 3.2 and earlier Deploying with Connectrix AP-7600B and PB-48K-AP4-18 Technical Notes and EMC RecoverPoint Deployment Manager Product Guide.

In most cases, upgrading RecoverPoint from release 3.2 to release 3.3 in the Brocade environment also requires upgrading fabric operating system (FOS) and storage application services (SAS).

◆ For migration from multi-VI to frame redirection mode, follow the instructions in “Migrating from multi-VI to frame redirection” on page 40.

◆ For non-disruptive upgrade, follow the instructions in “Non-disruptive upgrade of RecoverPoint from 3.2” on page 34.

Non-disruptive upgrade of RecoverPoint from 3.2

Starting with RecoverPoint 3.1, RPAs are backward compatible with the previous version of the RecoverPoint splitter agent. As a result, RPAs can be upgraded first; the upgraded RPAs will not conflict with the pre-upgrade splitter agent, and the upgrade can be performed without disrupting service.

The following are true during a non-disruptive upgrade:

◆ Replication will continue as normal; that is, splitter settings are not lost; resynchronization is not required; RecoverPoint journal will be preserved.

◆ All splitter and RPA persistent data will be preserved; therefore, there is no need to unbind and bind the host to the target during the upgrade.

◆ During the upgrade, only one host path will be available at a time. Host operation and configuration will not be affected, due to multipath capabilities of the host.

◆ FC IDs are not changed during the upgrade procedure. AIX and HP-UX hosts do not need to be rebooted, and SAN rescanning is not required.

35



Go to the appropriate procedure:

◆ “Non-disruptive upgrade of RecoverPoint with FOS or SAS upgrade” on page 35

◆ “Non-disruptive upgrade of RecoverPoint without FOS and SAS upgrade” on page 38

Non-disruptive upgrade of RecoverPoint with FOS or SAS upgrade

Use the following procedure to upgrade RecoverPoint and the fabric operating system or storage application services. Also use this procedure if updating both the fabric operating system and the storage application services.

1. If upgrading RecoverPoint, upgrade RecoverPoint on all RPAs now.

Refer to the EMC RecoverPoint Deployment Manager Product Guide, section “Upgrade RPA Software.”

CAUTION!Perform the following procedure one fabric at a time, to avoid host downtime.

2. Stop the RecoverPoint splitter agent. Run the following command from the Blade Processor:

>/thirdparty/recoverpoint/install/kdrv stop

Even though the splitter agent has been stopped, the host path and the splitter are still functioning.

3. Save the splitter settings to an off-switch location. These settings will be used to restore the splitter configuration.

To save the files:

a. Run:

>/thirdparty/recoverpoint/install/save_settings.sh

This saves all the splitter settings to the /thirdparty/hostname_config.tar.gz backup file.

b. Use ftp to copy this backup file to a remote location:



> cd /thirdparty/> ftp ip_address ftp> binftp> put hostname_config.tar.gzftp> bye

4. Save the switch configuration to an off-switch location using the configupload command.


> diagDisablePost

Disable Diagnostic Power-On Self-Test (POST) before upgrading either fabric operating system (FOS) or storage application services (SAS) firmware. Failing to disable POSTs may cause the switch to into an error state when upgrading firmware.

CAUTION!Perform Step 6 on page 36 to Step 11 on page 38 one fabric at a time to avoid host downtime. This procedure causes hosts to lose connectivity with the storage and makes the splitter unavailable on this fabric. If the hosts have multipath capabilities, they can remain on-line by using the other fabric.

6. Upgrade storage application services (SAS), fabric operating system (FOS), or both:




7. Clean up the previous splitter agent installation:

If /thirdparty/recoverpoint/ directory exists, the old agent files are still on disk, but the agent is not registered with the new storage application services (SAS).

• To ensure a clean splitter agent installation, stop the old agent and delete the /thirdparty/recoverpoint directory. Run the following commands on the Blade Processor:

> /thirdparty/recoverpoint/install/kdrv stop

37



> rm -rf /thirdparty/recoverpoint/

8. Install the new RecoverPoint splitter agent:




To do so:




Note: Do not use the –u parameter when re-installing the RecoverPoint splitter agent.




c. When prompted, enter the same hostname for the splitter agent as was used before the upgrade.


9. Restore to the switch the files that you saved in Step 3, as follows:

a. Copy the hostname_config.tar.gz file to the /thirdparty/ folder on the Blade Processor.




b. On the Blade Processor, run:

/thirdparty/recoverpoint/install/load_settings.sh

The previous configuration of the splitter should now be restored.


> reboot

11. After the Blade Processor reboot is completed, verify that hosts have access to storage through both paths and that the splitter is available in RecoverPoint.


> diagEnablePost

13. Repeat from Step 2 on page 35 for each fabric.

Non-disruptive upgrade of RecoverPoint without FOS and SAS upgrade

Use the following procedure to upgrade RecoverPoint when upgrading fabric operating system and storage application services is not required:


Refer to the EMC RecoverPoint 3.1 Installation Guide, section “Upgrading from release 3.1 and higher” or EMC RecoverPoint 3.2 Installation Guide, section “Upgrading from release 3.1 and later.”

CAUTION!Perform the following procedure one fabric at a time, to avoid host downtime.



Even though the splitter agent has been stopped, the host path and the splitter are still functioning.

3. Upgrade to the new version of the RecoverPoint splitter agent.

39



a. Download the RecoverPoint splitter agent installation file from Powerlink at;



To do so:



b. Run the installation package. At the Blade Processor, use the following command:

> cd /tmp> chmod +x agent_binary_file> ./agent_binary_file -u

CAUTION!Failure to add the -u parameter results in the loss of all current splitter settings and cause a full sweep.

c. If prompted, enter the same hostname for the splitter as was used before the upgrade.


4. Reboot the Blade Processor.

CAUTION!Rebooting the Blade Processor causes the bound host initiators to lose connectivity with the storage and makes the splitter unavailable. If the hosts have multipath capabilities, they can remain on-line by using the other fabric.

Log in to the Blade Processor and run the following command:



Migrating from multi-VI to frame redirection

> reboot


Migrating from multi-VI to frame redirectionMulti-VI mode is supported in RecoverPoint since release 2.4 SP2. Frame redirection mode is supported in RecoverPoint release 3.0 and later. It is recommended to migrate from multi-VI mode to frame redirection mode, whenever possible.

Before migrating from multi-VI to frame redirection mode, it is best practice to run the Brocade SAN Health Diagnostics Tool. For instructions, refer to “SAN Health Report” on page 48.

If performing disruptive upgrade, follow the procedure “Disruptive upgrade from RecoverPoint 2.4x and 3.0x to version 3.1x” in the EMC RecoverPoint 3.2 and earlier Connectrix AP-7600B and PB-48K-AP4-18 Technical Notes, which includes instructions for migrating. Otherwise, use the following instructions to migrate from multi-VI mode to frame redirection mode.

Migration from multi-VI to frame redirection mode will impact the system in the following ways:

◆ Consistency groups will be disabled, replication will stop, and journals are lost for the duration of the migration.

◆ Full sweep will be required after the migration.

◆ The last complete pre-migration image of each consistency group will be saved on the replica LUNs at least until the full sweep starts. During the full sweep, the last pre-migration image may be overwritten, leaving the system without a replica until the full sweep is completed.

◆ Bindings will need to be removed and recreated (one fabric at a time).

◆ Hosts are expected to maintain connectivity through multipathing throughout the migration procedure.

1. Use Plink or ssh to save the current settings to a file. You use the bindings section in this file for reference:

a. From a UNIX host:

$ ssh site_management_IP -l admin save_settings -n > save_settings1.txt

41



b. Windows host:

> plink -ssh site_management_IP -l admin -pw password save_settings -n > save_settings1.txt

To determine the command was successfully executed, open save_settings1.txt.

2. Disable all consistency groups.

CAUTION!Clicking Yes stops all replication, deletes journals, and causes a full sweep once the group is enabled again.

3. Recreate host-storage zones.

4. Use the following instructions to remove bindings.

CAUTION!Brocade splitter binding and unbinding causes the FC ID of affected HBAs to change. HP-UX hosts without agile addressing and AIX hosts without dynamic tracking cannot automatically rediscover the paths to storage if the FC ID is changed. To discover the new paths, it may be necessary to reconfigure the hosts, rescan the SAN, or take other action to have the hosts recognize the new paths to storage. Host downtime may be required.








powerlink.emc.com

powerlink.emc.com






CAUTION!The bindings should not all be removed simultaneously, to avoid risking downtime. After removing bindings from a fabric, ensure that the hosts still have access to targets.

Remove bindings for one splitter at a time:

a. Use the unbind_host_initiators CLI command to remove all multi-VI bindings, as in the following example:

s1-bos> unbind_host_initiators site=s1-bos splitter=sabre_172.16.0.17 host_initiators=10000000c92ab3bd

b. Delete the front-end and Connectrix zones, and remove virtual initiator WWNs from the RPA target zone.

c. Define storage LUN masking to the initiators, according to the host binding scheme (undo Step 7 on page 30).

d. Ensure that the hosts still have access to targets. Verify I/O capability on all paths.

At this point you should be able to access the physical storage target directly from the physical host initiator.

Repeat Step 4 on page 41 for each splitter.

Upgrading RecoverPoint, FOS, SAS, and splitter agent while migrating


Refer to the EMC RecoverPoint Deployment Manager 1.0 Product Guide, section “Upgrade RPA Software.”



If upgrading splitter agent without upgrading FOS or SAS, skip to Step 6 on page 43.

powerlink.emc.com

43



3. Save the switch configuration to an off-switch location using the configupload command.


> diagDisablePost

Disable Diagnostic Power-On Self-Test (POST) before upgrading either fabric operating system (FOS) or storage application services (SAS) firmware. Failing to disable POSTs may cause the switch to into an error state when upgrading firmware.

5. Upgrade storage application services (SAS), fabric operating system (FOS), or both:

CAUTION!Perform the FOS upgrade one fabric at a time, to reduce the risk of host downtime.




6. Clean up the previous splitter agent installation:

If /thirdparty/recoverpoint/ directory exists, the old agent files are still on disk, but the agent is not registered with the new storage application services (SAS).

• To ensure a clean splitter agent installation, stop the old agent and delete the /thirdparty/recoverpoint directory. Run the following commands on the Blade Processor:

> /thirdparty/recoverpoint/install/kdrv stop> rm -rf /thirdparty/recoverpoint/

7. Install the new RecoverPoint splitter agent:







To do so:




Note: Do not use the upgrade parameter when re-installing the RecoverPoint splitter agent.




c. When prompted, enter the hostname for the splitter agent.



> reboot


> diagEnablePost


11. Deploy RecoverPoint in frame redirection mode, according to the instructions in “Configuring RecoverPoint and splitters” on page 28.

45

Hardware replacement



Adding or replacing server node HBA

This procedure describes how to add or a replace an HBA or a host at a replica (non-production) side in the Brocade splitter environment. This procedure, if followed carefully, maintains data consistency without triggering a full/volume sweep on RecoverPoint consistency groups.

Prerequisites ◆ This procedure applies to RecoverPoint 3.2 and later only.

◆ This procedure only applies to Frame Redirection mode, not to Multi-VI mode.

◆ The relevant splitters should be operational.

◆ Volumes should be attached to the relevant splitter.

Procedure This procedure can be done on multiple initiator/target combinations in parallel or one by one.

1. If replacing an HBA or a host, use the instructions in this step to remove the old HBA or host:

a. Make sure image access is disabled.

b. Remove the zone between the HBA to be replaced and the storage target.

c. Remove the bindings between the HBA to be replaced and the storage target.

d. If you are replacing the entire host, repeat steps b and c for the other HBA(s).

e. Replace the HBA or the host.

2. Physically cable the new HBA(s) to the switch(es). HBA(s) must be logged in to the fabric(s), but must not have access to any replicated LUN. Zoning or LUN masking can be used to ensure that the host cannot access replicated LUNs.

3. Use the add_safe_bindings command from RecoverPoint CLI to add safe binding between the new HBA and the storage target. For example, the following command will add a safe binding between the host initiator 10000000c9668eb6 and the storage target 5006016239a01beb:



> add_safe_bindings site=Brocade_Right splitter=sabre_172.16.12.42 host_initiators=10000000c9668eb6 target=5006016239a01beb

For details about the add_safe_bindings command, refer to the RecoverPoint CLI Reference Guide.

4. Make the necessary changes to zoning and/or LUN masking to expose the LUNs to the host.

5. Wait 5 minutes to allow the splitter agent to automatically rescan the SAN.

6. Run the rescan_san command from the RecoverPoint CLI to refresh the RecoverPoint SAN view.

7. Run the get_safe_bindings_itls command from the RecoverPoint CLI to list the LUNs that are visible through a specific Brocade splitter safe binding.

For example, the following output means that Initiator 10000000c9668eb6 can access LUN 1 (non RecoverPoint LUN) and LUN 3 (Remote LUN on Replication set RSet1) on target 5006016239a01beb.

Site: Brocade_right: Splitter: sabre_172.16.12.42: Safe bindings: 10000000c9668eb6 => 5006016239a01beb: Visible LUNs: 1: Group Name: N/A UID: N/A 3: Group Name: [g, remote, RSet 1] UID: 60,06,01,60,f5,c3,1d,00,7e,83,f8,0d,14,0e,de,11

8. Ensure that all safe bindings appear correctly in the get_safe_bindings_itls output. If the bindings are not correct, use the CLI command remove_safe_bindings. This command is not available after the safe bindings are approved. For details about remove_safe_bindings, refer to the CLI Reference.

47

Troubleshooting


9. Ensure that all replicated LUNs appear in the get_safe_bindings_itls output. Exposing additional replicated LUNs to the host initiator after the safe bindings are approved will cause a volume sweep. If an expected LUN is missing:

a. Verify the zoning between the relevant host initiator and the storage target port.

b. Verify the LUN mapping/masking configuration for the relevant host initiator on the storage array.

c. Repeat rescan_san (see step 6).

10. Once all replicated LUNs appear in the get_safe_bindings_itls output, use the approve_safe_bindings command from RecoverPoint CLI to complete the process. For example, the following will complete the process for the binding created in Step 3 on page 45:

> approve_safe_bindings site=Brocade_Right splitter=sabre_172.16.12.42 host_initiators=10000000c9668eb6 target=5006016239a01beb

Once the save binding is approved, it becomes a regular binding, it will no longer appear in the get_safe_bindings_itls output, and the remove_safe_bindings command cannot be used to remove them.

11. Update the host's SAN view. The LUNs should now be visible from the host.

TroubleshootingAn important part of troubleshooting RecoverPoint issues in the Brocade environment is collecting the correct logs to allow EMC Customer Service to isolate, identify, and analyze the issues. The following sections provide instructions for log collection at customer sites according to the presenting issue. Collect the logs as soon after the event as possible. Three use cases are presented here:

◆ Splitter crashes and switch reboots.................................................. 48◆ Binding and host connectivity failures............................................ 49◆ I/O performance issues ..................................................................... 50


Troubleshooting

In addition, specific instructions are given for the following symptoms and procedures:

◆ Root file system is full ....................................................................... 51◆ Host initiators cannot write to virtual target ................................. 51◆ Replacing a faulty intelligent module............................................. 52◆ Replacing a failed host HBA ............................................................ 52◆ Adding hosts at the remote site ....................................................... 52◆ Using analysis pack ........................................................................... 52

SAN Health Report As part of troubleshooting, the best practice is to run the Brocade SAN Health Diagnostics Capture, a tool provided by Brocade that discovers and analyzes the SAN configuration. The tools detects problems in the following areas:

◆ zoning

◆ Frame Redirection

◆ unsupported switches and blades

◆ unsupported Fabric Operating System (FOS) and Storage Application Services (SAS)

◆ port errors

◆ Interswitch Link (ISL) connectivity

To run the SAN Health Diagnostics Capture, go to www.brocade.com and select Services & Support > Drivers & Downloads > SAN Health. Click Download SAN Health Diagnostics and follow instructions on screen.

After you email the SAN data to Brocade, it can take up to 48 hours to get a SAN Health report.

Splitter crashes and switch reboots

If you experience an unexpected splitter crash or switch reboot, do the following to gather the needed information.

1. In the RecoverPoint Management Application, go the System menu, and select Collect System Information.

Use the wizard to collect logs from all RecoverPoint appliances and from all splitters.

2. On the Control Processor of the switch running the splitter agent, run the following command, and when prompted, provide an FTP location to which to upload the logs:

http://www.brocade.com

49

Troubleshooting


> supportsave

3. On the Control Processor of the edge switches where hosts and targets are connected, run the following command, and when prompted, provide an FTP location to which to upload the logs:

> supportsave

4. Provide a time line and information regarding any changes in

• Fabric

• Ports

• Configuration

prior to the event.

Binding and host connectivity failures

If you experience binding or host connectivity failures, do the following as soon as possible and before you unbind:



2. On the Control Processor of the switch running the splitter agent, run the following command, and when prompted, provide an FTP location to which to upload the logs:

> supportsave

3. On the Control Processor of the edge switches where hosts and targets are connected, run the following command and, and when prompted, provide an FTP location to which to upload the logs:

> supportsave

4. If there are McData switches in the fabric, do the following to collect the required logs:

• If using the Brocade Enterprise Fabric Connectivity Manager (EFCM) interface, at the switch Element Manager, select Maintenance tab > Data collection. Select the log files and save them to the local disk.

• If using the Enterprise Fabric Connectivity Manager (EFCM) Basic interface, select Maintenance tab > System files > Data Collection Retrieval, select the log files, and save them to the local machine.


Troubleshooting

5. Use the emcgrab (for UNIX) or emcreports (for Windows) tools to collect relevant host data.

6. Run Brocade’s SAN Health tool and save the output. For instructions, refer to “SAN Health Report” on page 48.

7. Record host and HBA pWWN details, HBA driver, and firmware.

8. Record storage (target) pWWN details.

9. Provide a topology diagram of the fabric: host nodes, storage nodes, switches, and RecoverPoint appliances.

10. Provide a time line and information regarding any changes in

• Fabric

• Ports

• Configuration

prior to the event.

I/O performance issues

If you experience issues with I/O performance, do the following to gather the needed information.

1. On the Control Processor of all switches in the fabric, run the following command, and when prompted, provide an FTP location to which to upload the logs:

> supportsave

2. If there are McData switches in the fabric, do the following to collect the required logs:

• If using the Brocade Enterprise Fabric Connectivity Manager (EFCM) interface, at the switch Element Manager, select Maintenance tab > Data collection. Select the log files and save them to the local disk.

• If using the Enterprise Fabric Connectivity Manager (EFCM) Basic interface, select Maintenance tab > System files > Data Collection Retrieval, select the log files, and save them to the local machine.

3. Run Brocade’s SAN Health tool and save the output. For instructions, refer to “SAN Health Report” on page 48


51

Troubleshooting



5. Use the emcgrab (for UNIX) or emcreports (for Windows) tools to collect relevant host data.

6. From the RecoverPoint command-line interface, run the commands:

> detect_bottlenecks> export_statistics

and save the output.

7. Contact the vendor of your storage array and request an analysis of storage performance.

8. Record host node HBA pWWN details, drivers, firmware details.

9. Record storage pWWN details.

10. Provide a topology diagram of the fabric: host nodes, storage nodes, switches, and RecoverPoint appliances.

11. Provide a time line of the events leading up to the degradation of system performance.

Instructions for specific symptoms

Root file system is full When the root (/) file system on the Blade Processor module is full, various problems may occur, such as an inability to run the RecoverPoint splitter agent.

Beginning with RecoverPoint 3.2, you can monitor disk usage of the Blade Processor module in the Management Application GUI, from the Splitters tab of the System Monitoring component.

Use the df command to check disk usage, and remove files from the file system. The core files under /thirdparty/recoverpoint/log are usually the cause for this problem. Unless they are needed for bug analysis, remove the core files.

Host initiators cannotwrite to virtual target

If host initiators are able to see the virtual target and the LUNs, but fail in writing to the storage, it is most likely a LUN-masking problem in the target storage device.

Check the LUN configuration.


Troubleshooting

Replacing a faultyintelligent module

For information about replacing a faulty intelligent module, contact EMC Customer Service.

Replacing a failedhost HBA

For information about replacing a failed host HBA in an environment that uses splitters installed on Connectrix AP-7600B switches or PB-48K-AP4-18AP-7600B blades, contact EMC Customer Service.

Adding hosts at theremote site

For information about adding hosts on a remote “cold recovery site”, contact EMC Customer Service.

Using analysis pack The analysis pack is a set of scripts that is installed as part of the RecoverPoint splitter agent installation package. These scripts enable on-site analysis of splitter logs, and extraction of important diagnostic information about current SAN view and SAN view changes.

To execute the analysis scripts, run the following command on the Blade Processor:

/thirdparty/recoverpoint/install/analysis_pack/log_analysis.sh

The following parameters are valid:

You must also choose at least one of the following:

To display on-screen syntax help, run the command without parameters:

/thirdparty/recoverpoint/install/analysis_pack/log_analysis.sh

Parameter Description

v Runs the view analysis, to display the latest san view.

c Runs the changes analysis, to display the latest changes in SAN view (added and removed paths).

i Runs the host ITL analysis, and displays and count ITLs per host.

Parameter Description

r Parse recovered logs (as created after a splitter crash).

l Parse latest logs (host.log.00)

53

Troubleshooting


For example, if you want to see the SAN view as the splitter sees it, run the command with the following parameters:

/thirdparty/recoverpoint/install/analysis_pack/log_analysis.sh –vl

The output of this command will contain the following:

◆ Virtual Initiator information — For each virtual initiator:

• Port ID

• Corresponding initiator WWN

• virtual initiator WWN

• Whether it is in frame redirection mode

In this example, there is one frame redirection virtual initiator and one system virtual initiator. There is always a system virtual initiator, and its port ID is always 0.

VI information extracted from file: /thirdparty/recoverpoint/log/ram_log/host.log.00

ID Bound Initiator VI WWN FR==================================================0 N/A (System VI) 0x60012482c6de1000 -1 0x5001248206ce4bc8 0x60012482c6de1001 +

◆ SAN view — It is grouped by device GUID, and for each GUID it shows the relevant paths. The port number displayed is the port ID from the virtual initiator list. It also shows the size of the volume. Note that for the last device, there are 2 paths using the same GUID; both use LUN 1, since VI 1 can access the same volume through two different storage ports.

If all zones are configured correctly, the Host SAN view should match the view of its corresponding Virtual Initiator.

This portion of the output is shown on the following page.

############################################################################# Parsing results from file /thirdparty/recoverpoint/log/ram_log/host.log.00#########################################################################

############################################################################# View analysis#########################################################################

Device information

# WWN LUN Port Size (BLK) Size (MB) GUID==========================================================================================


Troubleshooting

1 0x5001248205ae4bc8 0 1 2097152 1024 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,50,54,32,32,32)

2 0x5001248205ae4bc8 1 1 4194304 2048 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,50,55,32,32,32)

3 0x5001248205ae4bc8 2 1 6291456 3072 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,50,56,32,32,32)

4 0x5001248205ae4bc8 3 1 8388608 4096 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,50,57,32,32,32)

5 0x5001248205ae4bc8 4 1 10485760 5120 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,48,32,32,32)

6 0x5001248205ae4bc8 5 1 12582912 6144 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,49,32,32,32)

7 0x5001248205ae4bc8 6 1 14680064 7168 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,50,32,32,32)

8 0x5001248205ae4bc8 7 1 16777216 8192 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,51,32,32,32)

9 0x5001248205ae4bc8 8 1 18874368 9216 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,52,32,32,32)

10 0x5001248205ae4bc8 9 1 20971520 10240 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,53,32,32,32)

11 0x5001248205ae4bc8 10 1 23068672 11264 SHARK(73,66,77,32,32,32,32,32,50,49,48,53,32,32,32,32,32,32,32,32,32,32,32,32,50,53,48,51,54,32,32,32)

12 0x50060e8014427506 1 1 6291840 3072 LIGHTNING(72,73,84,65,67,72,73,32,82,53,48,49,52,50,55,53,48,48,50,66)

0x50060e8014427526 1 1

55

Troubleshooting


Copyright © August 12, 2010 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.All other trademarks used herein are the property of their respective owners.

Documents

RecoverPoint Deployment with Brocade