Tech

nica

l Whi

te P

aper

Data Mirroring Alternatives for Network Storage Pools

TECHNICALWHITE PAPER

2 Fujitsu Softek

INTRODUCTION

This white paper discusses various data mirroring alternativesavailable to Fujitsu Softek customers using network storagepools. It suggests a preferred approach to a given data accessrequirement by considering the basic capabilities and limitationsof each option. Starting with locally mirrored copies, the dis-cussion entertains stretching these mirrors over increasinglylonger distances. Availability, performance and economic

aspects influencing the choice between real-time (synchronous)and store-and-forward (asynchronous) mirroring techniquesare also presented.

Throughout the document, the unique attributes of FujitsuSofteks host- and storage-independent mirroring selection willbe explored, as will the compelling reduction in total cost ofownership (TCO) and unprecedented flexibility attainable fromadvanced storage virtualization.

TABLE OF CONTENTS

Introduction page 2

Why Mirror? page 3

Economics of Managing Multiple Copies page 4

Mirroring at Its Simplest page 4

Distributed Data Replication page 5

What Does Fujitsu Softek Do Differently? page 6

Conclusion page 11

Fujitsu Softek 3

Basic Failure Protection

Mirrored copies are typically used to shield critical applicationsfrom equipment and operator errors. The extra copies act ashot-standby images for automatic fail-over mechanisms so thatthe users never sense a glitch. Examples of these conditionsinclude:

Disk failures

Subsystem failures

Path or link failures

To achieve application transparent switchover, the mirroredcopies must be kept in lock-step with the original online data.Generally that implies real-time synchronous mirroringmoreon this later.

Planned OutagesWith good planning and well-rehearsed procedures, mirroredcopies can also insulate users from planned equipment outages.Manually triggering the automatic fail-over mechanisms usedfor failure protection will just as easily divert the workload tostandby resources during:

Maintenance activities

Upgrades

Disaster Recovery

In disaster recovery scenarios, where multiple failures havecatastrophically impacted business operations, users mayexperience some downtime. The objective of data mirroringunder these conditions is to contain the downtime to a fewminutes or a few hours, as part of an expedited restoration plan.

Data MigrationMirroring facilities are very powerful substitutes for server-centricfile transfer and backup/restore methods traditionally used tomigrate data from one array or one location to another. Theonline image is copied in the background to the new destination.Once a convenient point of synchronization is reached, the copymay be disassociated from the original image and treated as astand-alone resource. The older equipment may be decom-missioned once the new environment is fully verified.

Local Access of Distributed Data

Perhaps the most recent facet of IT operations to benefit fromdata mirroring services are those striving to place largely distrib-uted data in close proximity to their users. Certainly the Internetboom has brought this paradigm into focus with the emphasison locally cached copies. The most common model relies on asingle writer whose updates are mirrored to multiple sites.

Known Good Working ImagesThe underlying assumption in all the above scenarios is thenotion that mirrored copies represent known good workingimages. This usually means not only selecting a few files to bemirrored, but also ensuring that all the objects (operating system,registry entries, application data structures, scripts, instructions,etc.) that make up the known good working set are replicated.Sometimes mirroring techniques are combined with snapshottechnologies to achieve these objectives. Softek Virtualizationoffers snapshot options that periodically freeze the mirroredimage at a known good point in time such as prior to a softwareor service pack upgrade.

WHY MIRROR?

There was a time when data was mirrored for only one reasonto protect against a disk failure. Later, it

became evident that having more than one copy of data was beneficial in many other ways. Today, with

the relatively low cost of online disks, mirrored data volumes are becoming an attractive method to ensure

continuous or nearly continuous data access in any of the following scenarios:

4 Fujitsu Softek

Economics of Managing Multiple Copies

Despite the broad applicability of data mirroring and the inexpen-sive nature of disks, the high cost of managing and maintainingupdated mirrored copies often precludes their use. Justificationhas been further hampered by the differences in mirrored storageadministration from one vendor to the next and the excessivecosts involved in many of the premium-priced products.

It is for this reason that Fujitsu Softek approaches data mirror-ing as an intrinsic property of uniform storage management fornetwork pools, rather than as a point product tied to a specificOS or a given disk array. At the same time Fujitsu Softek rec-ognizes that familiar ways die hard, and so the alternativespresented seek to leverage known operating procedures andlegacy product features.

Mirroring for the MassesThe design point and pricing structure for Softek Virtualizationencourages mirroring for the masses. No longer does the ITstaff have to do without these services to live within their budget.Softek Virtualization has proven that customers can do muchmore with less, by using a common, secure, centralized manage-ment approach that encompasses the mix of brands and supplierstypical to many organizations. The affordability message extendsto remote mirroring as well.

Distribute Access, Centralize Management,Fuel Transactions

Many Fujitsu Softek customers share the opinion that the bestway to achieve processing scalability with adequate and deter-ministic response time is to fan out multiple instances of data

where it is most needed. They see data mirroring as fuel forthe engine needed to meet the ever-increasing transactiondemands driving corporate revenues. These critical transac-tion-oriented workloads must be fed by subsystems capable ofhigher and higher input/output operations per second (IOPS).Such predictable performance levels are achievable in largescale only by allowing concurrent I/O threads to run undis-turbed against separately mirrored and cached copies of data.Network storage pools are the proven way to accomplishthese objectives.

Mirroring at Its Simplest

An easy way to achieve a mirrored copy is to have applicationswrite the same information twice, once for each of two disks. Inpractice, host-based operating system utilities, volume managersand RAID controllers as well as external RAID subsystemsrelieve applications of this duty (Figure 1). All of these techniquesmay be used in a Fujitsu Softek configuration. The virtual volumesserved by the network storage pools simply appear as well-behaved logical disks, but offer a great deal more flexibility.

Generally, the paths traveled by the mirrored I/O streams behavevery similarly, so the time to complete both I/Os is roughly thesame. Reads are taken from a pre-defined preferred patha relatively arbitrary decision given similar response timecharacteristics.

Host-Based Mirroring HBA-Based Mirroring External Mirroring Shared Storage Array

Application Server

SCSI/Fibre HBA

Application Server

SCSI/Fibre HBA

External RAID

Application Server

RAID HBA

Application Server

Softek Virtualization

Figure 1: Different approaches to simple local mirroring

Fujitsu Softek 5

Stretching the SeparationIntuitively one could obtain remote copies simply by stretchingone of the paths to reach a distant storage pool. SCSI and FibreChannel extenders do just that. They are inserted into the linkbetween the initiator system and the target device as nearlytransparent cable extenders. Some of these units passivelyamplify the signal to overcome transmission losses incurred overthe longer distances. Other products encapsulate or translate theI/O stream into an intermediate protocol better suited for widearea connections. These cable stretchers fulfill the generalrequirement for long distance connectivity but have no realsense for the end-to-end state of remotely mirrored information;especially when transmission or mirroring failures occur. Onlythe originating (initiator) end can determine if the remote copyis up-to-date or stale. Therefore, directly accessing the remotecopy could lead to unexpected and incorrect results (Figure 2).

Distributed Data Replication

To better ascertain the state of a remote copy, mirroring intelli-gence can be placed at both ends of the connection. In thisway, additional coordination between the originator and thedestination allows one to determine if the copy is synchronizedor if it has fallen behind. The approach is sometimes referredto as data replication. Typically the replicated I/O travels overa completely separate path from the local copy with potentiallyvery different performance characteristics.

Usually the most direct path is co-located with the originatingsubsystem and considered primary, while the lengthier indirectpath with longer transmission and handling latencies is definedas secondary (Figure 3).

Primary Secondary

Mirror Initiator

Cable Stretcher

Figure 2: Cable extenders to increase separationan incomplete solution

Secondary

Originating Mirror ProcessRemote Mirror Process

Primary

Indirect ConnectionLAN/WAN etc.

Figure 3: Distributed data replication between cooperating subsystems

6 Fujitsu Softek

The distributed mirroring processes take responsibility forkeeping the second copy updated. At the same time they hidethe underlying replication activities from the application. Bothends collaborate to inform each other of the remote copy state.The added coordination allows system administrators to betterunderstand the state, and hence validity of the secondary copybefore granting users access to them.

Conventional replication techniques over the past years havetaken either a host-centric or array-centric tactic.

Host-Centric Remote Replication

In the host-centric implementations, the originating mirroringprocess runs in the same server as the application. The remotedestination is usually another application server. The primaryand secondary storage devices may be different. Host-basedreplication is manageable when only a pair of hosts are involved.However, there are obvious downsides when multiple hostsrequire replication services:

The administration burden is proportional to the number ofconnections and different host operating systems. Thoughnot a problem with one or two servers, it becomes a definiteheadache with 2030 hosts, and a complete nightmare in alarge server farm such as those found at Internet ServiceProviders.

The replication processes steal host resources (CPU cycles,memory, I/O bandwidth, I/O slots) from the resident applica-tions potentially degrading their performance.

Array-Centric Remote Replication

The array-centric alternatives move replication off-board ontocooperating intelligent arrays. While these offload the hosts andcan replicate on behalf of multiple servers, they suffer three flaws:

Most work only with their own proprietary disks

The primary and secondary disk controllers must be of thesame proprietary make and model

Many rely on costly external equipment to convert diskchannel protocols into the handshake of common WANsand LANs.

Because of the cost and vendor-lock issues, this type of mirroringsolution has been confined largely to very wealthy consumers.

What Does Fujitsu Softek Do Differently?Fujitsu Softek considers data mirroring and replication as funda-mental services of network storage virtualization. The advanceddesign integrates local and remote mirroring functions intoaffordable network storage control nodes known as storagedomain servers (SDS). In so doing, the disadvantages of host-and array-centric alternatives are eliminated while retaining allof their benefits. SDS nodes collaborate not only to consolidatethe management of the underlying physical storage devices,but also to securely coordinate various replication activities,over short and long distances.

Uniqueness

The data mirroring solutions delivered by Softek Virtualizationenjoy the following unique combination of attributes: Affordable

Behave as natural extensions of network storage pools

Encompass disk arrays and JBOD from different suppliers

Support virtual volumes of arbitrary size

Allow some volumes to be replicated to one site and adifferent set to another

Support dynamic substitution of destinations, transparentto the applications

Resilient to network failures

Stretch over indefinitely long distances

Run over common Internet Protocol (IP) WANs and LANswithout conversion

Can operate directly over campus and extended SANs(metropolitan area networks)

Offload replication activities from hosts

Operate bi-directionally (either site can act as both initiatorand target)

Note: Familiar host- and array-centric products may continueto be used for specific roles in network storage pools along-side Fujitsu Softeks advanced mirroring features.

New Upgrade And Disaster Recovery ChoicesThe ability to transcend disk array boundaries creates newopportunities for low-impact data migration and cost-effectivedisaster recovery.

Softek Virtualization

For example, many organizations shy away from the hardship ofreplacing disk subsystem suppliers regardless of how dissatis-fied they are. Most prefer to prolong the relationship rather thanface lengthy downtime and uncertain upgrade results. Usingthe cross-vendor mirroring facilities offered by Fujitsu Softek, thenew equipment can be completely staged with live data in thebackground, while normal processing goes on. After due testing,the individual applications can be redirected to the replacementarrays by simply removing the original arrays from the mirroredvolume relationship. The users never experience downtime.

Other customers long for disaster recovery capabilities availablefrom premium-priced disk subsystems but know theyll neverhave the budget to execute such a plan. They can now obtainthese contingency services at a fraction of the cost using theirchoice of storage arrays at either end of the link without com-

promising their business continuance objectives. In fact, someorganizations use intelligent arrays at their primary site andrelegate relatively inexpensive JBODs (just-a-bunch-of-disks)for the remote recovery role.

Tradeoffs

With several alternatives for replicating data across remote sites,the final choice is largely dictated by tradeoffs between fourvariables:

Distance

Response time

Delivery guarantees

Networking costs

Contingency Site

VIRTUALIZATION

Network Storage Pooling

Switch

SANSDSSDS SDSSDS

Centralconsole

V IRTUALIZATION

DELL

SUN

IBM

JBOD

RAID

EMC

HP

HDS

Compaq

Network Storage Pooling

Production Site

UNIX Windows NetWare LinuxMainframe Mac OS

Virtualtape

Switch

SAN

LAN / MAN / WAN

Remote Mirroring

SDSSDS SDSSDS

EIDE

SSA

SCSI

Figure 4. Remote mirroring between network storage pools

Fujitsu Softek 7

8 Fujitsu Softek

Softek Virtualization

Disaster recovery and business continuance plans, for example,count on rapidly restoring operations from unexpected cata-strophes. Ideally they would maintain up-to-the-minute copiesof critical online volumes at regional contingency sites. Lesstime critical data mining and application testing scenarios mayhave less rigorous replication needs.

Asynchronous IP Mirroring OptionThe most convenient approach for widely separated locations isour Asynchronous IP Mirroring option, or AIM. AIM operatesover conventional Local Area Networks (LANs) and Wide AreaNetworks (WANs) using the popular Internet Protocol (TCP/IP).The term asynchronous describes the store and forwardnature of AIMs behavior. It also hints that a best effort will bemade to keep the copy up-to-date, but long distances and slowconnections will result in variable lag between the source andtarget. With very high-speed networks and short separation,AIM can minimize the differences between the two sites.

Synchronous Mirroring OptionCertain circumstances make it essential to keep the remotecopy in lock-step with the primary image. This calls for SoftekVirtualizations synchronous Network Mirroring option and isgenerally viable with separations up to 100 kilometers. Syn-chronous mirroring typically takes place over dedicated high-bandwidth optical routes using the Fibre Channel protocol.Generally, the recurring cost of such links is substantially higherthan the shared IP connections used for AIM.

All of these product features operate independent of the hostswho generate the data stream and the underlying storage deviceswhere their data is housed. They allow organizations to centrallyand securely manage remote mirroring services in a uniformmanner, despite variations in operating systems and disk arrays.They also amortize the investment in the replication infrastruc-ture over a pool of assets.

P1

P2

Switch

SAN

SDSSDSPrimary Data Center

S1

S2

Switch

SAN

SDSSDSSecondary Data Center

1

6

2

3 4

5

1. Initial I/O issued by the application2. SDS writes to local disk*3. SDS relays remote request*4. Remote write performed by secondary SDS*5. Remote copy confirmed6. I/O completion signaled

*In reality, steps 2 and 3 occur simultaneously and step 4 may be cached at the remote end.

Figure 5: Synchronous mirroring sequence

Guaranteed Delivery vs. Response Time

The distinction between synchronous and asynchronous mir-roring is better characterized as a choice between guaranteeddelivery and acceptable response time. Certainly ones instinctsdemand guaranteed delivery, but the laws of physics andeconomics sometimes overrule. Distance, performance andbudget are very influential elements in the final decision.

As the desired separation between the secondary copy and thereplication origin grows, so too do the delay and cost factors.Similarly, lower speed (less costly) connections to the secondarycopy can slow down the updates. Different applications and usershave different tolerances for delay. Many transaction-orientedusers find greater than two-second turnaround unusable, whileother batch users may regularly put up with 10-second waits.

Extending Synchronous MirrorsSynchronous behavior guarantees delivery by having the replica-tion service await acknowledgement that the second copy hasbeen updated before reporting I/O completion to the application(Figure 5).

Asynchronous Replication

The asynchronous method anticipates various network-relateddelays in the transmission to, and acknowledgement from theremote destination. Rather than hold up applications and slowdown user response time to unacceptable levels, AIM signalsI/O complete immediately after the write has completed over theprimary path (Figure 6). Of course, there is no timely guaranteethat the secondary update was posted, thus the tradeoff betweenresponse time and guaranteed delivery.

P1

P2

Switch

SAN

SDSSDSPrimary Data Center

S1

S2

Switch

SAN

SDSSDSSecondary Data Center

3

2

5

1. Initial I/O issued by the application2. SDS writes to local disk3. I/O completion signaled (dont wait on remote acknowledgement)4. SDS relays remote request5. Remote write performed by secondary SDS*6. Remote copy confirmed

*Step 5 may be postponed when the remote write is cached on the secondary SDS.

1

4

6

Figure 6: Asynchronous mirroring sequence

Fujitsu Softek 9

10 Fujitsu Softek

Softek Virtualization

At any one time the secondary may not have all the latestupdates, but the remote copy maintains an ordered time-sequenced image of past I/Os.

Note the differences between the asynchronous sequence inFigure 6, and the synchronous behavior described in Figure 5.

Elastic buffers are maintained on the transmit and receiveends to ensure smooth operation despite network anomalies.

How Close Can They Keep Up?

The fundamental difference between synchronous and asyn-chronous mirrors is illustrated by comparing the contents of asynchronous and an asynchronous copy relative to the sourcevolume over a period of time (Figure 7).The synchronous mirror remains in lock-step with the source overeach successive time period. However on the asynchronousmirror, viewed over the same period, writes that occurred on thesource at T0 may not appear on the mirror until T2 indicatingthat the asynchronous mirror lags behind the source. The lagwill vary depending on the bandwidth of the interconnectingnetwork and the SDS processing resources available to transmitand receive data. In the case shown in Figure 7, the asynchro-nous mirror doesnt catch up until T5 assuming no additionalwrites occur after T3.

Network Considerations When Extending MirrorsAs one might suspect, different network considerations comeinto play when operating synchronously and asynchronously.

The inter-SDS Fibre Channel links employed by synchronousNetwork Mirrors can take several forms depending on distancerequirements. A few guidelines are presented below:

Maintain a minimum of 500 meters or lessnative FibreChannel point-to-point connections between SDS HostBus Adapters

Maintain a minimum of 10 kilometers or lessuse point-to-point long wave laser connections between Fibre Channelswitches

Maintain a minimum of 100 kilometers or lessuse opticalextenders (including Dense Wave Division Multiplexers) tostretch the inter-switch link

Maintain a minimum of greater than 100 kilometerswhileachievable through IP-based Fibre Channel extenders, thelonger separation and protocol conversions introduce latenciesthat exclude them from many real time transaction-orientedapplications. WAN extenders may be more appropriate for lesstime sensitive situations with potentially lower update rates.

T0

Write #1

Source

SynchronousMirror

AsynchronousMirror

T1 T2 T3 T4

Write #2 Write #3 Write #4

T5

Figure 7: Updates with synchronous and asynchronous mirroring

Right of WayPublic right-of-way issues must also be taken into accountanytime the Fibre Channel connection leaves a private campusand traverses a public road. Under these circumstances, darkfiber suppliers must be brought into the picture. These companieshave been granted special privileges to tunnel underneath streets,railroads and even rivers. Customers then rent or lease theoptical infrastructure for their dedicated use.

IP Multiplies the Selection

The choice of inter-SDS mirroring links is much larger withAsynchronous IP Mirroring. Essentially any network interfacecard (NIC) capable of running under IP can be used as thenative SDS connection. These include the broad range of built-indial-up serial ports and Ethernet LAN cards, as well as the moresophisticated and powerful Gigabit Ethernet and ATM NICs.

The right-of-way concerns no longer dominate the choice ofnetwork providers. Instead, network provisioning is much morestraightforward, dovetailing in with the other IP services alreadyin use by the corporation. Downstream routers, virtual privatenetworks and encryption services can all be leveraged to enhancethe privacy and security of the remote mirroring transmissions.

Resiliency and Elasticity

By now everyone has become accustomed to the occasionalnetwork meltdown or the unexpected backhoe incident wherevital communications lines are accidentally severed. For synchro-nous mirroring, the link failures trigger logging at the originatingSDS. The logs are used to incrementally refresh the remotecopy when the link is restored.

AIMs elastic properties are even better suited to gracefully dealwith network anomalies. When the network slows to a creep,AIM patiently collects I/Os at the local SDS node, only to streamthem out in order as more bandwidth becomes available. In somecases, the network variations are a regular part of the dayheavy contention during peak business hours, interleaved withoff periods and very fast transmissions. Again, AIM ebbs andflows to match the network service level.

Conclusion

Softek Virtualization offers a wide range of network storagepooling alternatives capable of addressing the most commonmirroring and data replication requirements over short andlong distances. The compelling economics also make possiblethe use of these advanced facilities in fresh new opportunities,bringing both the acquisition and recurring investment in storagevirtualization within the reach of the masses.

Each mirroring option can be configured to circumvent failuresor simply to provide convenient data access at remote locations.The underlying technology combines the best features ofconventional replication techniques, without the typical lock-into a specific disk array or host. And unlike any products on themarket today, the replication services are supported on a choiceof cost effective and open hardware/software platforms using acentral and uniform management GUI.

For more information on data mirroring and network storagepools visit the Fujitsu Softek website at softek.fujitsu.com.

Fujitsu Softek 11

About Fujitsu Software Technology Corporation(Fujitsu Softek)Fujitsu Softek provides comprehensive end-to-end softwaresolutions to simplify todays complex data storage environment.Its unique vendor- and platform-independent solutions givecustomers the freedom and flexibility to proactively managetheir growing enterprises. Leading the industry in data backup,disaster recovery, migration, replication, SAN management,storage virtualization, storage resource management and qualityof service monitoring, Fujitsu Softek is backed by the financialstrength of Fujitsu Limited. With more than 300 employeesresiding in 30 countries, Fujitsu Softek maintains 24/7 supportcenters worldwide.

For more information, visit the Fujitsu Softek website atsoftek.fujitsu.com

FUJITSU SOFTEKWORLDWIDE HEADQUARTERS (USA)FUJITSU SOFTWARE TECHNOLOGY CORPORATION1250 East Arques Avenue, M/S 317Sunnyvale, CA 94085(NA and EU) 1 877 887 4562408 746 7638408 737 5900 faxEmail cs@softek.fujitsu.comCANADAFUJITSU SOFTWARE TECHNOLOGY (CANADA) LTD.450 686 2455450 686 0239 fax

EUROPEAN HEADQUARTERS (UK AND IRELAND)FUJITSU SOFTWARE TECHNOLOGY UK. LTD.16 Bartley Wood Business Park, Bartley WayHook, Hampshire RG27 9UZ+ 44 1256 386200+ 44 1256 386290 fax

FRANCE, SPAIN & PORTUGALFUJITSU SOFTWARE TECHNOLOGY SAS+ 33 1 44 10 41 00+ 33 1 44 10 41 08 fax

ITALYFUJITSU SOFTWARE TECHNOLOGYFiliale Italiana+ 39 02 241 0141+ 39 02 241 01408 fax

GERMANY, AUSTRIA & SWITZERLANDFUJITSU SOFTWARETECHNOLOGY (DEUTSCHLAND) GmbH+ 49 89 49058 100+ 49 89 49058 225 fax

BELGIUM & LUXEMBOURGFUJITSU SOFTWARE TECHNOLOGY+ 32 2 7150300+ 32 2 7209897 fax

THE NETHERLANDSFUJITSU SOFTWARE TECHNOLOGY+ 31 (0)346 244700+ 31 (0)346 244710 fax

Email Europe at info@softek.fujitsu.com

2002 Fujitsu Software Technology Corporation. Softek Virtualization is a trademark of Fujitsu Software Technology Corporation. The information in this document may be superseded by subsequent documents.

Contents of this whitepaper provided by DataCore Software Corporation.

All rights reserved. Printed in the U.S.A. SM0040-01 [v1.4/1M] 04/02

Softek Virtualization