9/11 - Respond andRecover: A Case Study

Joy Heath-PorterSidley Austin Brown & Wood LLP

LawNet 2002August 21, 2002

Who are we?

• Large, multi-office, multi-national firm

• 14 offices; six countries

• 1,500 lawyers; 3,000 total personnel

• Two offices in New York before September 11; 600 person office in the North Tower of the World Trade Center

Status on September 10

• Involved in one of the largest law firm mergers ever

• Two networks with different hubs

• Two different e-mail and document management systems

• Two different telephone systems

Status on September 10(cont’d.)

• Same financial system, but on different versions with different database engine

• Still in the process of consolidating technology

• Had just moved equipment to WTC previous weekend

Brief Chronology

• 7:50 a.m. Tuesday - how we found out; first steps

• Mid-morning Tuesday - disaster recovery steering group assembled; began to map out recovery strategy

• Noon Tuesday - offsite storage provider contacted; arrangements made to drive tapes to Chicago

Brief Chronology (cont’d.)

• Throughout Tuesday - made sure phones were manned, so that staff calling in would have a “live” voice; determined status of our people; identified and arranged for recovery resources; Chicago and New York offices “closed,” but many staff members were working

• C.O.B. Tuesday - system recovery priorities established; servers identified for recovery

Brief Chronology (cont’d.)

• Tuesday night - 800 dial-in information numbers established and published; information made available via the Firm’s Web site; temporary space identified

• Wednesday morning - all offices contacted for spare equipment; arrangements made for rental equipment; PC vendor contacted for purchase of equipment; cabling began

Brief Chronology (cont’d.)

• Wednesday noon - Tapes arrived; restoration process began

• Wednesday p.m. - voice mail accounts on alternate voice system began to be created for lawyers

• Wednesday night- new e-mail server built; account creation in process; Internet mail redirected (brownwoodlaw.com)

Brief Chronology (cont’d.)

• Thursday a.m. - all lawyers had e-mail accounts and mail was flowing; equipment was shipped out from other Firm offices

• Thursday p.m. - Internet café established in alternate NY sight; image build began for rental equipment

• Thursday night - secure extranet site established for technology information

Brief Chronology (cont’d.)

• Friday a.m. - document management was available to NY users

• Friday noon - equipment began arriving from other offices and from vendor

• Friday p.m. - financial systems restoration was complete and consolidation began; image for rentals and new PCs finalized

Brief Chronology (cont’d.)

• Saturday a.m. - Rental equipment began arriving; imaging began; teams from Chicago began driving to New York transporting equipment not available in New York (cell phones, card keys, etc.)

• Throughout weekend - Phones, network equipment, printers, desks, PCs rolled out in temporary space

Six Days Later

• Monday 10:00 a.m. - all New York personnel assembled in a mid-town hotel for a briefing

• Monday noon - all New York personnel reported for work in temporary space

Today

• All servers returned to New York

• All technology migrated to firm standards

• New York office moved into new, permanent home over July 4 weekend

Key Aspects of Disaster Recovery Approach

• “Maniacal” approach to tape backups - daily offsite storage of tapes; offsite location outside metropolitan area

• Focus on running our daily operation to perfection

• Standardized hardware (network, PCs, phones) with standard images

Key Aspects of Disaster Recovery Approach (cont’d.)

• Emphasis on documentation and standard operating procedures

• Commitment to serving our lawyers the way our lawyers serve the Firm's clients

Lessons Learned/Affirmed

• You cannot plan for a plane hitting your building - accept the fact that you will have to improvise to some degree in virtually any disaster

• You cannot have too many friends - clients, vendors, co-counsel, even “competitors”

• Practice, practice, practice - staff need to be able to perform basic disaster recovery tasks "in their sleep"

Lessons Learned/Affirmed(cont’d.)

• If you have multiple offices, leverage them

• The fact of back-ups is not enough - they must be quality-checked, verified, and viable

• Efficiencies of centralization must be weighed against risks from a disaster recovery perspective

Lessons Learned/Affirmed(cont’d.)

• Recruit for character and creativity, as well as technical expertise

• Understanding the relative priority of your critical applications cannot be overstated - in a disaster situation, restoring contact information may be more important than restoring documents

Lessons Learned/Affirmed(cont’d.)

• Web access to your systems is not a luxury; it is a necessity

• Stress with principals the importance of using firm-standard repositories - if documents are in three different places, restoring access becomes much more difficult

Lessons Learned/Affirmed(cont’d.)

• Think about disaster recovery when making a buy/build decision

• Smart, dedicated people ultimately make it happen

How does theStory End?

• We were never “down”

• Clients were impressed

• Our most important assets - our smart and dedicated people - were not lost

• The loss of people is the only disaster truly to fear - if you are prepared, everything else can be recovered or replaced