Log Shipping Best Practices

Database Log Shipping on Microsoft SQL Server

1


Applies to:

Any SAP Product with ABAP stack working with SQL Server 2005.

Summary

This document describes best practices in setting up, monitoring and administering Database LogShipping process which provides data redundancy and increases system availability. It focuses on thecommon scenarios for disaster recovery, physical data corruption, protection against user errors and highavailability.

Author(s): Alexander KosolapovCompany: SAP AGCreated on: 24 May 2007Last revision: 10 October 2007


2

Table of ContentOverview of High Availability solutions in Microsoft SQL Server ______________ 4Log Shipping fundamental concepts _____________________________________ 4

Overview _______________________________________________________________ 4Technical implementation of Log Shipping____________________________________ 4Fail over ________________________________________________________________ 4Fail back________________________________________________________________ 5Backup directory (backup folder)____________________________________________ 5Copy directory (copy folder)________________________________________________ 5STANDBY mode of a database______________________________________________ 5

Technical background of STANDBY mode_____________________________________________ 5

Influence of the Log shipping on a Backup Strategy ____________________________ 6Considerations about database corruption____________________________________ 6Considerations about user errors ___________________________________________ 6Log Shipping as a part of disaster recovery plan _______________________________ 6Evaluating data loss after failover ___________________________________________ 7

Log Shipping in SQL 2005 ______________________________________________ 7Prerequisites ____________________________________________________________ 7

SQL Server prerequisites __________________________________________________________ 7SAP prerequisites ________________________________________________________________ 7

Setting up Log Shipping ___________________________________________________ 8Preparations ____________________________________________________________ 8Configuring Log Shipping__________________________________________________ 9Preparing SAP System to failover __________________________________________ 10Failover scenarios_______________________________________________________ 12

Disaster recovery scenario ________________________________________________________ 12Partial database loss scenario _____________________________________________________ 13DB corruption scenario ___________________________________________________________ 13User error scenario ______________________________________________________________ 14Maintenance scenario____________________________________________________________ 15Completing failover ______________________________________________________________ 16General post-failover actions ______________________________________________________ 16

Fail back_______________________________________________________________ 17Preparing the system to fail back ___________________________________________________ 17Failing back____________________________________________________________________ 17

Monitoring and reporting for Log shipping ___________________________________ 18Suspend Log Shipping ___________________________________________________ 18Resume Log Shipping____________________________________________________ 19Remove Log Shipping permanently_________________________________________ 19


3

Special attention is required by following actions _____________________________ 19Adjusting Windows system timer ___________________________________________________ 19Adding data and log files to the Primary DB___________________________________________ 20SQL Jobs _____________________________________________________________________ 20Modifying the SAP DEFAULT profile ________________________________________________ 20

Frequently asked questions ___________________________________________ 21Appendix ___________________________________________________________ 22

SAP Notes, referred in this document _______________________________________ 22


4

Overview of High Availability solutions in Microsoft SQLServer

SQL Server 2000 supported two approaches to increasing DB service availability: High Availability Clusterand DB Log Shipping.

With SQL Server 2005 a new functionality was introduced additionally, the Database Mirroring.

The current document focuses only on the DB Log Shipping.

For more information on using SQL Server in a Windows Cluster (MSCS) environment, please refer tohttp://www.microsoft.com/technet/prodtechnol/sql/2000/deploy/hasog01.mspx.

For more information on using SQL Server DB Mirroring, please refer to FAQhttp://www.microsoft.com/technet/prodtechnol/sql/2005/dbmirfaq.mspx and to SAP Note 965908.

Log Shipping fundamental concepts

OverviewLog Shipping allows you to automatically send transaction log backup files (also referred to as TRN files)from a primary database on a primary server instance to one or more secondary databases onseparate secondary server instances.For the sake of simplicity the scenario with only one secondary database is considered in this document.

An optional third server instance, known as a monitor server records a history and a status of log backupand restore operations.

Advantages of Log Shipping are as follows:- There is no limitation for a distance between the primary and secondary servers.- The Log Shipping mechanism is tolerant to delays in transferring of transaction log backup files.- In contrast to DB cluster or DB mirroring, a problem in the Log shipping itself (e.g. lost connection)

doesnt case a fault of the productive database.- No special expensive hardware is required

Disadvantages:- A switch to the secondary server must be done manually.- The databases are not at sync permanently, some data loss is possible.

Technical implementation of Log ShippingLog Shipping process consists of 4 scheduled jobs:

1. Backup log job, running on the Primary Server

2. Copy job, running on the Secondary server

3. Restore job, running on the Secondary server

4. Monitoring and alerting job, running on the Monitoring server (optional).

All the jobs call the executable sqllogship.exe. The sqllogship application performs a backup, copy orrestore operation and associated clean-up tasks for a log shipping configuration. The operation isperformed on a specific instance of Microsoft SQL Server 2005 for a specific database.

Fail overFailover is a capability of a service to switch over to a redundant or standby computer server in case of afailure on the productive server.


5

Log Shipping does not possess a capability to fail over automatically from the primary server to thesecondary server. If the primary database becomes unavailable, a manual intervention of a systemadministrator is necessary to restore DB service. He must recover the rest of transaction log backups andbring the database to the online state.

Fail backFail back is a process of moving the service back to the original hardware after it is ready to run again. Inother words, it is a process, reverse to a fail over.

Like a fail over, the fail back in a log shipping scenario must be done manually by the system administrator.

Backup directory (backup folder)It is a network share, where transaction log backup files are stored temporarily.

Copy directory (copy folder)It is a directory, to where the transaction log backup files are copied before they are restored. Usually it islocated on the Secondary server.

STANDBY mode of a databaseThis mode will allow read-only operations to be performed on the database and additionally furthertransaction logs can be applied to the DB. A DB can get to STANDBY mode only after a restore databaseor restore log operation.

Technical background of STANDBY mode

A transaction log backup file may contain information about not yet committed transaction. Whether thetransaction will be committed or rolled back, is not yet known. That information will only come with the nexttransaction log backup file. Thus, any full recovery of a DB (RESTORE DATABASE WITHRECOVERY) consists of 2 phases:

? redo of all modifications? undo of all uncommitted transactions.

After the undo phase the DB will be in a transactional consistent state, but wont accept any furthertransaction log because some modifications are lost.

One can skip the undo phase by means of the option WITH NORECOVERY of the command RESTORE.However, as the DB is in a transactional inconsistent state, SQL Server doesnt allow using it even forreads only.

When RESTORE WITH STANDBY is used, SQL Server performs the undo phase but saves also thestate before undo in a so-called standby file or Transaction Undo File (extension TUF). This file is amandatory argument of the option STANDBY.

As the result, the database is transactional consistent and can be read. There are two possible transitionsfrom STANDBY state.

? The DB administrator can recover it completely and make it read/write with the commandRESTORE LOG WITH RECOVERY. A possibility to apply any further transaction logbackup will be lost forever.

? The next transaction log can be applied to the DB.


6

Influence of the Log shipping on a Backup StrategyAs a matter of fact, the Log shipping dramatically changes a common backup strategy. When LogShipping is implemented, its Backup log job must be one and only who is doing transaction log backups.As the job saves the backup files to a disk, they cannot be saved to tapes with SQL command BACKUPLOG. Neither SAP transaction DB13 nor any other SQL Backup software may be used for that.

Thats why generic backup software for file backups (e.g. NTBACKUP) has to be used to save transactionlog backup files from a disk to durable media.

Database backups can be done as usual, directly to tapes with SQL command BACKUP DATABASE.

Considerations about database corruptionThroughout the current document the concept of database corruption means a state of a database whichcauses impossibility to access at least part of data because it is lost or distorted and the state cannot berepaired without data loss.

Physical data corruptions are caused exclusively with hardware reasons or erroneous program coding onoperating system level and in DBMS code itself. Note however, that the latter is only a theoreticalpossibility which has not been observed as of SQL Server version 7.0 (1998) in combination with SAPproducts.

Note, data corruption cannot be caused with applications, using DB service. Data corruptions cannot beprecisely tracked in time and often they pass unnoticed over substantial time periods of many days orweeks. For more information about this topic please refer to SAP Note 327494.

Since there is no direct exchange of data pages between a primary and a secondary server, physical DBcorruptions cannot be transferred to the secondary server. Thus, Log Shipping provides a better protectionagainst DB corruptions. This protection is not absolute however. A hardware fault still may corrupt thetransaction log file itself. But such a corruption wont pass unnoticed as it will be recognized on thesecondary server during restore.

Considerations about user errorsUser errors include erroneous actions committed on DB level by DB administrators as well as onapplication level by functional users and errors in applications (e.g. ABAP reports). In this document weconsider only user errors that cause deleting or overwriting of needful data. Also inserting of nonsensedata into the database can be considered as a user error. We limit to the cases of such errors that cannotbe recovered with application means.

As a rule, user errors become apparent very fast, within minutes or few hours and can be preciselytracked.

As a rule, the DB must be stopped and restored to the point in time shortly before the time of the usererror. A common restore causes long system downtime of several hours and loss of all correct datamodifications done after that point in time.

Log Shipping allows intentional mistiming of the Secondary DB against the Primary DB by several minutesor hours. This is controlled by Log delay period option. Thus, with this option set to e.g.120 min, theSecondary DB at 10:00 has the same state as the Primary DB had at 08:00. This gives the DBadministrator time to react to user errors and rapidly switch to the secondary DB avoiding long downtime.

Log Shipping as a part of disaster recovery planSince the Secondary DB contains redundant data that is physically separate from the Primary DB, it cancontinue servicing even after a total damage of the Primary server along with primary data storage. Thatswhy Log Shipping can be used as a part of a disaster recovery plan.


7

Evaluating data loss after failoverSince Primary and Secondary database are not permanently synchronized, some committed data may getlost after failover. This depends on fail over scenario (see Failover scenarios, page 12) and on afrequency of backups.

Log Shipping in SQL 2005

PrerequisitesTo set up Log Shipping for an SAP Solution the following prerequisites should be met:

SQL Server prerequisites

1. SQL Server of the same version and edition must be installed on the Secondary server. If ServicePack 2 is installed on Primary SQL Server 2005, the Secondary server must have it too. Thisrequirement comes from the fact that as of MS SQL 2005 SP2 the data type VARDECIMAL canbe compressed, which is not supported in earlier builds.

2. Primary and Secondary servers must have the same server-wide collation:SQL_Latin1_General_CP850_BIN2

3. If you want to use an optional Monitoring Server, SQL Server software must be installed on thatserver.

4. The SQL Server services on Primary and Secondary servers must be running under a domainuser.

5. A network share with sufficient space for transaction log backups must exist. The share must beaccessible (read/write) for both Primary and Secondary SQL Server service users. By reliabilityreasons the share must not reside on the same hardware as the Primary server, though it istechnically possible.

6. The database must be set to the Full or Bulk-logged recovery model. Under no circumstancesmay the recovery model be switched to Simple, even for a short time.

7. SQL Server Agent service must be up and running permanently on Primary, Secondary andMonitoring servers.

8. Primary and Secondary servers must belong to the same Windows domain or there must exist atrusted relationship between the domain of the Primary server and the domain of the Secondaryserver.

9. If the Secondary database must be in STANDBY mode (see STANDBY mode of a database,page 5), install Cumulative update package 3 for SQL Server 2005 Service Pack 2. Otherwise anerror, described in http://support.microsoft.com/kb/940126/en-us may occur.

10. The internal servername on the SQL Server must not be null, that means the SQL statement

select @@servername

must return the local servername on the Primary and Secondary server.

The database format of SQL Server 2005 is independent from the server platform. Thus, its possibleto have 64-bit Primary server and 32-bit Secondary server and vice versa.

SAP prerequisites

There is no mechanism in SAP kernel supporting DB Log shipping, thus there are no SAP parameterscontrolling that. All actions must be done manually. Thats why there is no special requirement to SAPRelease, kernel patch level, SAP Support Package level.


8

Setting up Log ShippingIn this document we only consider a scenario of failing over and back the database service. If both SAPCentral Instance (CI) and DB server coexist on the same hardware, special measures should beadditionally taken to failover the SAP CI to another hardware server. For more information, refer to SAPNote 804078 Replicated Enqueue Configuration on Windows.

If all prerequisites above are met, the Log Shipping can be set up without downtime in the productionsystem.

Before getting started with log shipping, the following questions must be clarified:

? Do I need the Secondary DB to be readable?There are 2 options for the Secondary DB: to be in Loading state or in Standby/Read only state.In the loading state the database is not accessible at all. There is even no way to check thedatabase options (what files it consists of, their size etc.). In the Standby/Read only state onecan execute SELECT commands from DB tables, execute consistency checks. It can also beused for business reporting (with some precautions). However, an active use of the Secondary DBprevents restoring the transaction log files. Thats why there is an option for the Standby modewhich is called "Disconnect users from the secondary database when restoring backups". Whenactivated, the restore job has higher priority and cancels all established connections. Ifdeactivated, the restore job fails if there is a connection to the secondary DB. Transaction logbackups will accumulate until there are no user connections to the database.

? Decide about a time lag of the Secondary DB against the Primary DB (see Considerations aboutuser errors, page 6).

Preparations1. Make sure no transaction log backups for the DB are carried out until the Log Shippingconfiguration is finished. If transaction log backups are scheduled as SQL job or using a third party backupsoftware, they must be deactivated.

2. Back up the complete database (full database backup). There is no necessity to stop SAP forthat, although lower DB performance during the backup must be taken into account. Any kind of backupmedia (disk, tape) is suitable.

3. Restore the backup onto the secondary server. When using Management Studio, choose the optionLeave the database non-operational, and do not roll back the uncommitted transactions.Additional transaction logs can be restored. (RESTORE WITH NORECOVERY). See (2) on the figurebelow. When using T-SQL command, add the option WITH NORECOVERY to the option list of theRESTORE command.

During the restore there is a possibility to change data file destination folders. This is necessary if thesecondary server doesnt have a drive letter which existed on the Primary server (see the page Options).


9

Note, in the column Restore As you have to specify a full path to the files (1). All folders in the path mustexist, but the files itself must not.

Configuring Log ShippingThe most comfortable way to configure Log Shipping is SQL Server Management Studio.

1. Start SQL Server Management Studio, connect to the productive SQL Server with a user havingsysadmin privileges.

2. Right click the database you want to use as your Primary database, and then click Properties.3. Under Select a page, click Transaction Log Shipping.4. Select the Enable this as a primary database in a log shipping configuration check box.5. Under Transaction log backups, click Backup Settings.6. In the Network path to the backup folder box, type the network path to the share you created for the

transaction log backup folder. Leave the box If the backup folder is located on the primary server,type the local path to the backup folder empty.

7. Configure the Delete files older than: parameter. It is recommended to set the value to at least 3days. The maximum value is limited only through the space provided on the backup folder for TRNfiles.

8. Configure Alert if no backup occurs with desired parameters.9. It is recommended to leave the default name under Job name.10. Click Schedule and adjust the SQL Server Agent schedule. Set up the desired interval in the

group Daily frequency, field Occurs every.11. Click OK.12. Click OK.

1

2


10

13. Under Secondary server instances and databases, click Add.14. Click Connect and connect to the Secondary SQL Server. It is strongly recommended to use

Windows authentication because some problems were reported by using SQL authentication.15. In the Secondary Database box, choose the database restored during the preparation.16. On the Initialize Secondary database tab, choose No, the secondary database is initialized.17. On the Copy Files tab, in the Destination folder for copied files box, type the path of the folder into

which the transaction logs backups should be copied. Note, this folder is located on the secondaryserver.

18. Configure the Delete copied files after parameter. It is recommended to set the value to at least 3days. The maximum value is limited only through the space provided on the copy folder.

19. Click Schedule and adjust the SQL Server Agent schedule as needed. This schedule shouldapproximate the backup schedule with some shift necessary for a transaction log backup to complete.For instance, if you scheduled the backup job starting at 0:00 each 15 minutes, it makes sense to startthe Copy job at 0:05 each 15 minutes.

20. On the Restore Transaction Log tab, under Database state when restoring backups, choose theNo recovery mode or Standby mode option (see STANDBY mode of a database on page 5).Refer to SQL Server prerequisites page 7 point 9 in the latter case.

21. If you want to delay the restore process on the secondary server, type in the desired time lag underDelay restoring backups at least (see Considerations about user errors on page 6).

22. Choose an alert threshold under Alert if no restore occurs within23. Click Schedule and then adjust the SQL Server Agent schedule as needed. This schedule should

approximate the copy schedule with some shift necessary for a copy job to complete. For instance, ifyou scheduled the Copy job starting at 0:05 each 15 minutes, it makes sense to start the Restore jobat 0:10 each 15 minutes.

24. Click OK.

On the same screen an optional monitor server can be configured. Once the monitor server has beenconfigured it cannot be changed without removing log shipping first.

25. Under Monitor server instance, select the Use a monitor server instance check box, and then clickSettings.

26. Click Connect and connect to the instance of SQL Server that you want to use as your Monitorserver. It is recommended to use Windows authentication. The user must be assigned to thesysadmin fixed server role.

27. Under Monitor connections, choose the connection method to be used by the backup, copy, andrestore jobs to connect to the monitor server.

28. Under History retention, choose the length of time you want to retain a record of your log shippinghistory. It is recommended to retain the history for at least 2 weeks.

29. Click OK.30. On the Database Properties dialog box, click OK to begin the configuration process.In case of errors or warnings press Report and save the report to a file. If you cannot solve the problemby your own, create a problem message at SAP attaching the error report file.

Preparing SAP System to failover1. Create the procedure which fixes SAP logins. As a matter of fact, all security-relevant

information is stored in the master DB and thus is not transferred to the Secondary serverautomatically with Log Shipping. Thats why SQL Server logins and users must be created on theSecondary server manually. The easiest way to do so is to make use of the stored proceduresp_check_sap_login from SAP note 610640. Create this stored procedure in the masterdatabase on the Primary and Secondary servers.

2. Create a copy of the DEFAULT.PFL profile which will be used for failover. As a fact, during afailover the DB server name changes and SAP system must be aware of the change. FileDEFAULT.PFL in folder :\usr\sap\\sys\profile contains one obligatoryand one optional parameter that specify the database server name.


11

? SAPDBHOST is the obligatory parameter. Its the hostname of the database server. Sincethe hostname is changed after failover, this parameter must be corrected appropriately.

? dbs/mss/server is an optional parameter. If not set explicitly, its value is copied fromSAPDBHOST. This parameter is mandatory if SQL Server is installed as a namedinstance, i.e., network hostname is not equal to SQL Server instance name.

The recommended procedure is as follows:

? Copy the actual DEFAULT.PFL file to DEFAULT_PRIMARY.PFL? Copy the actual DEFAULT.PFL file to DEFAULT_ SECONDARY.PFL? In the DEFAULT_SECONDARY.PFL substitute with

for the parameter SAPDBHOST.

? If the parameter dbs/mss/server is explicitly set, in the DEFAULT_SECONDARY.PFLsubstitute with . This optionaccepts prefixes and port number that may be necessary when the Secondary server is anamed instance. For instance,

dbs/mss/server = tcp:QASDB\TST,1433

where tcp: is a protocol specifier,

QUASDB\PRD is the SQL Server named instance name,

1433 is the port number the SQL Server is listening to.

Refer to SAP note 208632 for more information.

3. Windows environment variables. Standalone SAP programs (i.e. tp, R3trans, saplicense)require the environment variable MSSQL_SERVER to be correctly specified with SQL Servername (see SAP note 98678). Since after a failover the name has changed, the variable must becorrected too. This action is not as important as it is necessary only for technical transactions.SAP System itself doesnt read the environment variables and can start even with wrong values.

On Windows 2003 Server you can prepare a script, setting Windows environment variables on thecentral instance and all application servers. The command is

SETX MSSQL_SERVER For example:

SETX MSSQL_SERVER QASDB\TST,1433The script has to be executed before failover under user \adm.

On Windows 2000 the action needs to me done manually as follows:

- Log on as user \adm.

- right click on My computer, choose Properties

- Go to tab Advanced

- Press button Environment variables

- Choose the variable MSSQL_SERVER in the User variables for adm and press Edit.

- Enter the name of the Secondary SQL server, e.g. QASDB\TST,1433.

- Press OK until you leave the application.


12

Failover scenariosBelow only a failover of DB server will be discussed. If SAP Central Instance (SAP CI) coexists withDB instance on the same hardware, a lot of further actions must be prepared for failover. All they areout of scope of this document.

Below we consider the following scenarios of failover:1. The Primary DB server is physically damaged or transaction log file is lost (Disaster recovery

scenario, page 12).2. At least one of data files is lost, but the Primary server itself is in good order and all transaction log

files (.LDF) are intact (Partial database loss scenario, page 13).3. Database server is functioning, but an irrecoverable database corruption has been detected (DB

corruption scenario, page 13).4. Some irrecoverable data have been deleted or distorted as a result of a user error (User error

scenario, page 14).5. Primary server must be stopped for hardware or software maintenance (Maintenance scenario,

page 15).

The crucial parameters for assessing data loss are:? time interval between 2 consecutive runs of the Backup job on the Primary server. This will

be referred below as backup interval.? Applying of TRN files can be additionally delayed by restore delay.

Disaster recovery scenario

In this case the Primary DB server suddenly suffers damage or completely destroyed along with datastorage. For SAP application servers it looks like it has just disappeared.

The same scenario is also applicable in case of damage or a loss of the transaction log file.

Data loss is calculated as time elapsed since the last backup and is less then backup interval.

To fail over to the Secondary server the following actions must be done:1. Stop the SAP System.2. Log on to the Secondary server with SQL Management Studio3. Start the Copy Job. It can be found under -> SQL Server Agent -> Jobs.

By default, its name is LSCopy__. Right click the job nameand choose Start job at step.

4. When the Copy job finished, start the Restore job. By default, its name isLSRestore__. Right click the job name and choose Start jobat step.

5. Suspend Log shipping as described in Suspend Log Shipping on page 18.6. If the restore delay is not used, proceed with step 9.7. Identify the last TRN file used for restore: right click the Restore job, choose View history,

expand the most recent history record. The step logs are displayed in reverse chronologicalorder. Find the topmost step containing the message

Restored log backup file. Secondary DB: '', File:\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'

Note its name.8. Find in the Copy folder the TRN files with timestamps greater then the last restored TRN file

and restore those files manually with a series of commands like follows:RESTORE LOG FROM DISK = '\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'WITH NORECOVERY

Note, the TRN files must be processed in strict chronological order, otherwise an error isthrown. Should it happen, find the correct TRN file and proceed with the restore.

9. Proceed with the section Completing failover on page 16.In this scenario a simplified fail back (see Fail back on page 17) is not possible.


13

Partial database loss scenario

In this case the Primary server cannot access at least one of data files. As the result, the DB is instate Suspect and cannot be opened. The SAP System cant run. However, if all transaction log files areavailable and not damaged, there is a chance to failover without loss of committed data.

To fail over to the Secondary server without data loss the following actions must be done:1. Suspend Log shipping as described in Suspend Log Shipping on page 18.2. Stop the SAP System.3. Back up the active transaction log on your primary server with option NO_TRUNCATE. That

option allows making a log backup even if the database is unavailable. This last log is alsocalled tail log. The command is as follows:

USE masterBACKUP LOG TO DISK ='\\backup_share\BackupFolder\_YYYYMMDDHHMM_tail.trn'WITH NO_TRUNCATE

If the command finishes successfully, there will be no data loss, proceed with step 4.Otherwise the situation meets not this scenario, rather Disaster recovery scenario, becausethe transaction log is damaged or lost. Please proceed with step 2 of Disaster recoveryscenario on page 12.

4. Copy the tail log backup from the Backup folder to the Copy folder manually.5. Log on to the Secondary server with SQL Management Studio6. Start the Copy Job. It can be found under -> SQL Server Agent -> Jobs.


7. When the copy job is finished, start the Restore job. By default, its name isLSRestore__. Right click the job name and choose Start jobat step.

8. If the restore delay is not used, proceed with step 11.9. Identify the most recent TRN file used for restore: right click the Restore job, choose View

history and expand the recent history record. The step logs are displayed in a reversechronological order. Find the topmost step containing the message

Restored log backup file. Secondary DB: '', File:'\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'.

Note its name.10. Find in the Copy folder TRN files with timestamps greater then the last restored TRN file and

restore those files in sequence with a series of commands like follows:RESTORE LOG FROM DISK = '\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'WITH NORECOVERY


11. Restore the tail log TRN file with the commandRESTORE LOG FROM DISK ='\\copy_share\CopyFolder\_YYYYMMDDHHMMSS_tail.trn'WITH NORECOVERY

12. Proceed with the section Completing failover on page 16.In this scenario a simplified fail back is not possible (see Fail back on page 17).

DB corruption scenario

For more information about DB corruptions please refer to SAP note 142731.

In this case the Primary DB appears to be corrupted after a regular check or because some SQLstatements fail causing an ABAP short dump in the SAP system. The DB server itself is functioning andnone of the database files is lost.


14

If you faced a DB corruption, there is a probability that the transaction log has been corrupted too. In thiscase the Restore job will be failing with errors like

During redoing of a logged operation in database 'SID', an erroroccurred at log record ID (xxx:xxx:zzz).

If this is the case, the database can only be restored to the point of a corruption in the transaction log andthus, some committed transactions will be lost.

If the restore operation finishes without errors, no data loss happens after a failover.

To fail over to the Secondary server the following actions must be done:

1. Inform end users about an urgent SAP System shutdown and stop the SAP system.2. Log on to the Primary DB server with SQL Management Studio3. Start the Log shipping Backup job. It can be found under -> SQL Server

Agent -> Jobs. By default, its name is LSBackup_. Right click the job name andchoose Start job at step.

4. Suspend Log shipping as described in Suspend Log Shipping on page 18.5. Log on to the Secondary server with SQL Management Studio6. Start the Copy Job. It can be found under -> SQL Server Agent -> Jobs.


7. When the copy job is finished, start the Restore job. By default, its name isLSRestore__. Right click the job name and choose Start jobat step. Check job history to identify failures during restore. If an error suggesting to acorrupted transaction log occurs, proceed with Completing failover on page 16. In this casetake some data loss into account.

8. If the restore delay is not used, proceed with step 11.9. Identify the last TRN file used for restore: right click the Restore job, choose View history,

expand the recent history record. The step logs are displayed in reverse chronological order.Find the topmost step containing the message

Restored log backup file. Secondary DB: '', File:\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'.

Note its name.10. Find in the Copy folder TRN files with greater timestamps and restore those files manually

with a series of commands like follows:RESTORE LOG FROM DISK = '\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'WITH NORECOVERY

Note, the TRN files must be processed in sequence in a strict chronological order, otherwisean error is thrown. Should it happen, find the correct TRN file and proceed with the restore.

11. Proceed with the section Completing failover on page 16.In this scenario a simplified fail back (see Fail back on page 17) is not possible.

User error scenario

In this case the hardware is in good order and the DB is physically consistent, but the database isunusable for business applications as it contains logical inconsistencies, or some data was deleted or a lotof false data was inserted.

The most effective solution in this case is to bring the database to the state as it had been before the usererror was committed. The consequence of point in time restore is an inevitable loss of all modificationsthat were done after that point in time.

In this scenario a simplified fail back (see Fail back on page 17) is not possible.

To fail over to the Secondary server without data loss the following actions must be done:1. Immediately suspend Log shipping as described in Suspend Log Shipping on page 18 and

note the time when it was done. This crucial point will be referred as LS_STOP below.2. Inform end users about an urgent SAP System shutdown and stop the system.


15

3. Identify the point in time when the user error was committed. If it cannot be identified precisely,take the earliest time it might have happened. It will be referred as TIME_ERROR. ConvertTIME_ERROR to UTC time, lets call it TIME_ERROR_UTC.

4. Let us call the difference between LS_STOP and TIME_ERROR as reaction time. If thereaction time is greater then the restore delay, the user error has already been applied to theSecondary DB. The error must be corrected by means of restore the DB from regular backups.Otherwise proceed with the next steps.

5. Log on to the Secondary server with SQL Management Studio6. Identify the last TRN file used for restore: right click the Restore job, choose View history,

expand the recent history record. The step logs are displayed in reverse chronological order.Find the topmost step containing the message

Restored log backup file. Secondary DB: '', File:\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'.

Note its name.7. Find in the Copy folder TRN files with greater timestamps, but not exceeding

TIME_ERROR_UTC and restore those files manually with a series of commands like followsRESTORE LOG FROM DISK = '\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'WITH NORECOVERY, STOPAT = ''

where should be 1 minute before TIME_ERROR. Adhere to the full date andtime format YYYY-MM-DD HH:MM:SS, e.g. '2007-05-24 09:59:59'.Note, the TRN files must be processed in strict chronological order, otherwise an error isthrown. Should it happen, find the correct TRN file and proceed with the restore.

8. Proceed with the section Completing failover on page 16

Maintenance scenario

In this case the Primary server must be stopped for a substantial period of time while the SAP systemmust go on running. In this scenario no committed data is lost.

To fail over to the Secondary server without data loss the following actions must be done:1. Suspend Log shipping as described in Suspend Log Shipping on page 18.2. Inform end users about an SAP System shutdown and stop the system.3. Close all connections to database. You can check for them in Management Studio

under Management -> Activity monitor.4. Back up the active transaction log on your primary server with option NORECOVERY. That

leaves the Primary DB in restoring state, where it is unavailable, but is prepared to simplifiedfail back (see Fail back on page 17). The command is as follows:

USE masterBACKUP LOG TO DISK ='\\backup_share\BackupFolder\_YYYYMMDDHHMM_tail.trn'WITH NORECOVERY

5. Copy the tail transaction log backup file from the Backup folder to the Copy folder.6. Log on to the Secondary server with SQL Management Studio7. Start the Copy Job. It can be found under -> SQL Server Agent -> Jobs.


8. When the copy job is finished, start the Restore job. By default, its name isLSRestore__. Right click the job name and choose Start jobat step.

9. If the restore delay is not used, proceed with step 12.10. Identify the most recent TRN file used for restore: right click the Restore job, choose View

history, expand the recent history record. The step logs are displayed in a reversechronological order. Find the topmost step containing the message

Restored log backup file. Secondary DB: '', File:\CopyFolder\_YYYYMMDDHHMMSS.trn'.

Note its name.


16

11. Find in the Copy folder TRN files with timestamps greater then the last restored TRN file andrestore those files in sequence with a series of commands like follows:

RESTORE LOG FROM DISK = '\\copy_share\CopyFolder\_YYYYMMDDHHMMSS.trn'WITH NORECOVERY


12. Restore the tail log TRN file with the commandRESTORE LOG FROM DISK ='\\copy_share\CopyFolder\_YYYYMMDDHHMMSS_tail.trn'WITH NORECOVERY

13. Proceed with the section Completing failover on this page below.

If all actions are done correctly, there is a possibility to fail back very fast without copying the databaseback to the Primary server (see Fail back below).

Completing failover

Independently from a failover scenario you have to execute the following actions to complete afailover process.

1. Issue the command:RESTORE LOG WITH RECOVERYThis brings the database online and it can be used with the SAP System.

2. Execute the procedure, fixing mappings between SAP SQL users and server logins:use master;exec sp_check_sap_login , , , repair;

Where is the current DB schema (see SAP note 98678), is the domain of standard SAP users adm and SAPService,repair is the keyword instructing to perform the correcting actions.

3. Copy the prepared SAP default profile file into DEFAULT.PFL in the folder:\usr\sap\\SYS\profile

copy DEFAULT_SECONDARY.PFL DEFAULT.PFLoverwriting the existing file.

4. Correct environment variable MSSQL_SERVER on each application server as per PrepareSAP System to failover, point 3 on page 11.

5. Start the SAP System.

General post-failover actions

Independently from a failover scenario, some actions must be done in SAP system if it had to run in afailover state for longer then one day.

1. Backup DB job must be scheduled anew. If SAP transaction DB13 was earlier used for that useDB13 again, taking into consideration possibly changed backup device names. If other DB backupsoftware was used, refer to its documentation.

2. For SAP Basis Release 610 and higher: create DB collector jobs. Execute the post processingstep "Creating the permanent stored procedures" described in SAP note 151603.

3. Reschedule SAP blocking locks collector job.a. In a system with installed DBACOCKPIT, start transaction DBACOCKPIT, on the

navigation pane expand Performance -> History, then double click Lock History.Press the button Turn collector job on which appears on the navigation frame.

b. In a system where DBACOCKPIT is not installed yet, start transaction ST04, go toDetailed Analysis, press Blocking lockstats button, then press Turn collector job onbutton.


17

Fail backFail back is a process of moving the database service back to the original Primary server. It can benecessary by various reasons, for example:? The original Secondary server is equipped with less powerful hardware and cannot permanently

withstand working load with satisfied quality of service.? The original Secondary server must be freed for another role, e.g. DB server for QAS or training

system

Fail back should be carried out after the original problem on the Primary server has been solved.There are 2 possibilities to fail back:

1. Simplified fail back without moving the database. This is only possible in case of a maintenancefail over. It can be done within a couple of minutes.

2. Regular fail back with moving the whole database. It needs to be executed in all other scenarios.As this fail back requires a full database backup and restore, it may take many hours to complete.The system is however not necessarily down during that time.

Preparing the system to fail back

Technically, preparing a fail back is identical to setting up Log Shipping where the original Secondaryserver plays the primary role and the original Primary server becomes the secondary one. Make sure LogShipping is suspended (i.e., all Log Shipping jobs are disabled) before you proceed.

1. If the original Primary server is lost and must be set up from scratch, refer to the chapter SQLServer prerequisites on page 7.

2. If the previous fail over was done as per Maintenance scenario (see page 15) and the originallyPrimary DB is in restoring state, proceed to step 4. Otherwise continue with 3.

3. Back up the original Secondary DB and restore it onto the originally Primary Server as describedin Preparations on page 8. This step may take a lot of time.

4. Configure Log Shipping as described in Configuring Log Shipping on page 9, with the onlydifference: now under Primary the originally Secondary server is meant and vice versa.

Note: use the same share for Backup folder that you used for the original log shipping.

On this stage there are two configured Log shipping processes. For further reference lets call themPrimary-to-Secondary and Secondary-to-Primary based on original roles of the servers.? The Primary-to-Secondary Log shipping is now suspended.? The Secondary-to-Primary one is active.

Failing back1. Fail over from the originally Secondary server to the originally Primary one as described in

Maintenance scenario on page 15, except of step Copy the prepared SAP default profileduring completing failover. On that step copy the original SAP default profile file intoDEFAULT.PFL in the folder :\usr\sap\\SYS\profile

copy DEFAULT_PRIMARY.PFL DEFAULT.PFLoverwriting the existing file.

2. Resume Primary-to-Secondary Log shipping as per Resume Log Shipping on page 19.


18

Monitoring and reporting for Log shippingIf the Monitor server is configured, an overall status of all parts of the log shipping process is available onit within one screen. If no Monitor server is installed, a DB administrator has to check status of a Primaryand a Secondary server separately. The procedure below is generic for Monitor, Primary and Secondaryservers, however the result will look differently. The table below shows what kind of information isavailable on each server.

Column Valid for Displayedon

Comment

Status Backup job PrimaryTime since last Backup Backup job Primary Elapsed time since the last Backup in minutesBackup Threshold Backup job Primary Configured threshold. When Time since last

Backup exceeds the value, an alert will be raised(if enabled).

Backup alert enabled Backup job Primary True/False.Time since last Copy Copy job Secondary Elapsed time (in minutes) since the last copy of

TRN files from the backup folder to the copy folder.Time since last Restore Restore job Secondary Elapsed time (in minutes) since the last restore of

TRN files on the Secondary server.Latency of Last File Restore job Secondary Latency between the last backup job and the last

restore job in minutes. When Time since lastBackup exceeds the value, an alert will be raised(if enabled).

Restore Threshold Restore job Secondary Configured threshold. When Time since lastRestore exceeds the value, an alert will be raised(if enabled).

Restore alert enabled Restore job Secondary True/False.Last Backup File Backup job Secondary File nameLast Copied File Restore job Secondary File nameLast Restored File Restore job Secondary File name

To call Log Shipping status using SQL Management Studio as of Support Pack 2 (SP2):1. Start SQL Management Studio, connect to a server.2. Right click on the server name and choose Reports -> Standard reports -> Transaction Log

Shipping Status.To call Log Shipping status with a Monitor server using SQL Management Studio earlier then SP2:

1. Start SQL Management Studio, connect to a server.2. Select the server instance in Object Explorer.3. In the Object Explorer Details page, display the list of available report types by clicking the

arrow next to the Reports button. If the Object Explorer Details page is not displayed, selectObject Explorer Details on the View menu.

4. Click Transaction Log Shipping Status.

Suspend Log Shipping? Disable the log shipping Backup job on the Primary server, if it still running.? Disable the Copy and Restore jobs on the Secondary server.

The path in the Object Explorer of SQL Management Studio: -> SQL Server Agent -> JobsRight click on the job then click Disable.? The job name convention for a backup job is LSBackup_? The job name convention is LS__, where can be

Copy or Restore.Note however, that the naming convention is not mandatory; another names may have been used.


19

Resume Log Shipping? Enable the log shipping Backup job on the Primary server.? Enable the Copy and Restore jobs on the Secondary server.

The path in the Object Explorer of SQL Management Studio: -> SQL Server Agent -> JobsRight click on the job then click Enable.? The job name convention for a backup job is LSBackup_? The job name convention is LS__, where can be

Copy or Restore.Note however, that the naming convention is not mandatory; another names may have been used.

Remove Log Shipping permanentlyTo remove log shipping? Right-click the Primary database and then click Properties.? Under Select a page, click Transaction Log Shipping.? Clear the Enable this as a primary database in a log shipping configuration flag. A confirmation

popup immediately appears:

? Confirm it with Yes.? Press OK, check the status popup window. In case of errors or warnings press Report, save it to

a file, otherwise close the window. If you cannot solve the problem by your own, create a problemmessage at SAP attaching the error report file

Special attention is required by following actions

Adjusting Windows system timer

The Log shipping backup job creates in the backup directory files with the following namingconvention:

_.trn,

where

is the Primary database name,

is the backup timestamp in UTC time on the Primary database server.UTC time is the world coordinated time which doesnt depend on time zone or daylight saving settings.By that reason the timestamp in the file name does not correspond to local time besides countries intime zone UTC+0 (Greenwich Mean Time). Despite some inconvenience, the naming conventionmakes sense since the primary and the secondary databases may be in different time zones and/oruse different daylight savings settings.

Thats why it is safe to change the current time zone or daylight saving setting on both Primary andSecondary servers, no follow-up actions are required.


20

The copy and restore jobs rely on correct timestamps in the file names. Thats why the primary andsecondary servers must have synchronized system clocks. Either Windows Time Service in a domain or anetwork time protocol should be used for that.

If system clocks must be adjusted, it is recommended to suspend log shipping and resume it afterwards.

Adding data and log files to the Primary DBAfter a file is added to the Primary DB, the appropriate file definition is created as a log record andtransferred with the next TRN file. The restore job on the Secondary server tries to create the additionalfile under the same path as on the Primary server.If this path doesnt exist, the restore job fails with the error

*** Error: Could not apply log backup file 'h:\ls_copy\SID_20070926163724.trn' tosecondary database 'SID'.(Microsoft.SqlServer.Management.LogShipping) ****** Error: Directory lookup for the file "P:\SID\Data\SID_22.ndf" failed with theoperating system error 3(The system cannot find the path specified.).File 'SID_22' cannot be restored to 'P:\SID\Data\SID_22.ndf'.Use WITH MOVE to identify a valid location for the file.

As a matter of fact, the newly created file may be very large, several dozen or even hundred gigabytes. Ifthe path exists by accident, the file will be created successfully. If the appropriate disk doesnt haveenough space or it is undesirable to put the new file to that folder, the following actions must be done:

1. Disable the Restore job on the Secondary server.2. Add the new data file3. Start the Restore job manually, take the TRN filename from the Job history4. Let the Copy job run or start it manually.5. Restore the last TRN file manually with the command like follows:

RESTORE LOG FROMDISK='\'WITH MOVE ''TO '\.ndf',NORECOVERY

6. Enable the Restore job on the Secondary server.

If the described procedure was not adhered to and the restore job has failed already, the DB administratorcan still execute step 5 for the failing TRN file.

SQL JobsSome SAP Basis functions are implemented as scheduled SQL Jobs. They are listed in General post-failover actions on page 16. Customers also may define own SQL jobs using standard tools (SQL Studio)or third-party tools. All those jobs are stored in a standard database msdb, which is separate from .As a result, any change in the jobs will not be automatically shipped to the Secondary server and shouldbe rescheduled by necessity on the Secondary server manually.

Modifying the SAP DEFAULT profileAfter each modification of a default profile, the modification has to be repeated for the fileDEFAULT_SECONDARY.PFL, which is used during fail over. Otherwise, after a failover the SAP systemstarts with old parameters. Additionally, the new DEFAULT.PFL must be copied intoDEFAULT_PRIMARY.PFL.


21

Frequently asked questionsQ1. Can Log Shipping coexist with Microsoft Cluster Services (MSCS)?

A1. Yes. Those two approaches ideally supplement each other. While MSCS provides an instantautomatic failover, it cannot protect a DB against physical corruptions nor against user errors. Because ofa limitation on the distance between cluster nodes and single data storage, MSCS hardly can be used fordisaster recovery. All mentioned features are covered by Log Shipping.

Q2. Can Log Shipping coexist with SQL Server database mirroring?

A2. Yes. A detailed explanation is out of scope of the document. Please refer to Microsoft Books Online,article Database Mirroring and Log Shipping (http://msdn2.microsoft.com/library/en-us/53e98134-e274-4dfd-8b72-0cc0fd5c800e.aspx).

Q3. What is a file with extension TUF, found in the Copy directory?

A3. The TUF file is the Transaction Undo File, and is created when performing log shipping to a server inSTANDBY mode of a database on page 5.

Q4. How a normal DB backup and TL backup on durable media (backup tapes) should be organized? CanSAP transaction DB13 (DBACOCKPIT/DBA Planning Calendar) coexist with Log Shipping?

A4.The Primary database can be backed up with any tool, including DB13. However no transaction logbackups other then Log Shipping are allowed. Thats why the functionality of DB13 may not be used. Thisis also valid for any third party backup software as well as for manual transaction log backups.

Q5. After somebody unintentionally switched the DB to SIMPLE recovery mode and then immediatelyback to FULL, the Log shipping Restore Job is failing. How to fix it?

A5. The current Log Shipping must be removed and then set up again including full DB backup andrestore, because the transaction log backup chain is broken.

Q6. Because of error 9002 Transaction log full the DBA administrator backed up the log manually withthe option NO_LOG or TRUNCATE_ONLY. Afterwards the Log shipping Restore Job is failing. How to fixit?

A6. The current Log Shipping must be removed and then set up again including full DB backup andrestore. The correct procedure in case of error 9002 would be to start the Log Shipping Backup jobimmediately.

Q7. After somebody unintentionally deleted some TRN files from the copy folder, the Log shipping RestoreJob is failing. How to fix it?

A7. Check if the deleted files are still in the Backup folder. By default they are retained there for 3 days. Ifyes, copy them manually to the Copy folder. If the files have already been deleted, the current LogShipping must be removed and then set up again including full DB backup and restore.


22

Appendix

SAP Notes, referred in this document

Note Short text

98678 SQL Server Connection Issues

142731 DBCC checks of SQL server

151603 Copying an SQL Server database

208632 TCP/IP network protocol for MSSQL

610640 sp_check_sap_login

Database Log Shipping on Microsoft SQL ServerOverview of High Availability solutions in Microsoft SQL ServerLog Shipping fundamental conceptsOverviewTechnical implementation of Log ShippingFail overFail backBackup directory (backup folder)Copy directory (copy folder)STANDBY mode of a databaseTechnical background of STANDBY mode

Influence of the Log shipping on a Backup StrategyConsiderations about database corruptionConsiderations about user errorsLog Shipping as a part of disaster recovery planEvaluating data loss after failover

Log Shipping in SQL 2005PrerequisitesSQL Server prerequisitesSAP prerequisites

Setting up Log ShippingPreparationsConfiguring Log ShippingPreparing SAP System to failoverFailover scenariosDisaster recovery scenarioPartial database loss scenarioDB corruption scenarioUser error scenarioMaintenance scenarioCompleting failoverGeneral post-failover actions

Fail backPreparing the system to fail backFailing back

Monitoring and reporting for Log shippingSuspend Log ShippingResume Log ShippingRemove Log Shipping permanentlySpecial attention is required by following actionsAdjusting Windows system timerAdding data and log files to the Primary DBSQL JobsModifying the SAP DEFAULT profile

Frequently asked questionsAppendixSAP Notes, referred in this document

Documents

Log Shipping Best Practices