TADM53 SAP Netweaver SAP Web as DB Operation MS SQL Server

© SAP AG 2003

TADM53 SAP NetWeaver – SAP Web AS DB Operation (MS SQL Server)

© SAP AG 2003

THE BEST-RUN BUSINESSES RUN SAP

TADM53SAP NetWeaver –SAP Web AS DB Operation (MS SQL Server)

6.20 2003/Q3 50065335

© SAP AG 2003

Copyright 2003 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

All rights reserved.

Copyright

Trademarks: Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation.

IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix and Informix® Dynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries.

ORACLE® is a registered trademark of ORACLE Corporation. UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group. Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc.

HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

JAVA® is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One. SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies.

© SAP AG 2002

Technology Consultant SAP NetWeaver - SAP Web AS Implementation & Operation for [DB]

TADM10SAP NetWeaver -SAP Web AS Implementation & Operation I

10 days TADM12SAP NetWeaver -SAP Web AS Implementation & Operation II

10 daysIntermediate testing (no handout of certificate)

CertificationTechnology Consultant SAP NetWeaver - SAP Web AS Impl. & Oper. for ORACLE (2003)

CertificationTechnology Consultant SAPNetWeaver - SAP Web AS Impl. & Oper. for MS SQL Server (2003)

TADM51SAP NetWeaver -SAP Web AS DB Operation (ORACLE)

5 days

TADM53SAP NetWeaver -SAP Web AS DB Operation(MS SQL Server)

5 days

TADM56SAP NetWeaver -SAP Web AS DB Operation (DB2 UDB)

5 days CertificationTechnology Consultant SAPNetWeaver - SAP Web AS Impl. & Oper. for DB2 UDB (2003)

Attendance ProjectTeam Training ADM515 (Database Administration SAP DB)

CertificationTechnology Consultant SAP NetWeaver - SAP Web AS Impl. & Oper. for SAP DB (2003)

3 days

© SAP AG 2002

Course Prerequisites

Essential

TADM10 (SAP NetWeaver – SAP Web AS Implementation & Operation I)

TADM12 (SAP NetWeaver – SAP Web AS Implementation & Operation II)

Experience in the administration of MS SQL Server databases

© SAP AG 2002

Target Group

??

Target Group

Technology Consultants who are responsible for implementing and administering SAP systems

Duration

5 Days

© SAP AG TADM53 1-1

© SAP AG 2002

Course goals

Course objectives

Course content

Contents:

Course Overview


© SAP AG 2003

At the conclusion of this course, you will be able to:

Course Goals

Consolidate your knowledge of database administration with reference to CCMS database tools in SAP systems and MS SQL Server tools

Put effective data backup strategies into practice

Recognize and resolve performance bottlenecks

Directly apply SAP knowledge you have gained in the previous courses, as well as what you have learned in this course, when you start your work as a junior consultant

Furthermore you got some first insight into the principles of identifying expensive SQL statements.


© SAP AG 2003

Course Overview

Section I Database Administration MS SQL Server

Section II SQL Cache Analysis (MS SQL Server)

Section III Certification Description

Course Content: TADM53 (SAP NetWeaver - SAP Web AS DB Operation (MS SQL Server))

Preface

This SAP Consultant Education course contains different individual courses (sections), each of which deal with a separate topic.

Each individual course (section) is subdivided into different units.


© SAP AG 2003

Content: Database Administration MS SQL Server

1 Introduction

2 SQL Server Architecture

3 How the SAP System uses SQL Server

4 Performance Monitoring and Tuning

5 Database Backup

6 Database Restore

7 Regular Maintenance and Error Handling


© SAP AG 2003

Content: SQL Cache Analysis (MS SQL Server)

1 Introduction and Technical Background2 Introduction to the SAP Stored Procedure Statistic3 How the SAP System uses SQL Server4 Analyzing the Stored Procedure Statistic5 Identify Coding6 Optimizer Statistics and Explain Plan7 Workflow and Reporting8 Index Utilization and Operators9 Cost Evaluation

10 Creating an Index11 Similar Statements12 View Processing13 Joins14 Pooled and Clustered Tables15 Expensive Statements and SAP Table Buffers


© SAP AG 2003

Content: Certification Description

1 Certification Description

This SAP Consultant Education course contains different individual courses (sections), each of which deal with a separate topic.

Each individual course (section) is subdivided into different units.


© SAP AG 2003

<leave this slide blank>

Section I - ADM520

Diese Seite wird von Andrea für euch noch erstell!

FS310 Inkasso/Exkasso


© SAP AG 2003

SECTION I

ADM520Database Administration MS SQL Server

SAP Web AS 6.20 2003/Q2 50062304


© SAP AG 2002

This course is intended for the following audiences:

Database administrators

SAP system administrators

Project team members

Duration: 3 days

Target Audience

User notes

These training materials are not a teach-yourself program. They complement the explanations provided by your course instructor. Space is provided on each page for you to note down additional information.

There may not be sufficient time during the course to complete all the exercises. The exercises provide additional examples that are covered during the course. You can also work through these examples in your own time to increase your understanding of the topics.


© SAP AG 2003

Required Knowledge:

SAPTEC –SAP NetWeaver: Fundamentals of the Application Platform

Working knowledge of the MS SQL Server 2000 database and the operating system Windows NT or Windows 2000

Recommended Knowledge:

For the administration of SAP systems the course ADM100 – SAP Web AS Administration I is necessary

Course Prerequisites


© SAP AG 2003

SQL Server Architecture

1 Introduction

2 SQL Server Architecture3 How the SAP System uses SQL Server


5 Database Backup

6 Database Restore



© SAP AG 2003

ContentsArchitecture

Database Server and InstancesAuthenticationDatabases and filesDatabase objects

LoggingLocking

ObjectivesAt the end of this unit, you will be able to:

Describe the main features of the SQL Server architecture

SQL Server Architecture


© SAP AG 2003

SQL Server

Windows server

Database 2

Database Server

Database 1

Database server, a processrunning the database system

The term "database server" is often used to describe the physical computer on which the database runs.

For the purpose of this explanation of SQL Server architecture, the term "database server" will refer to the operating system process on Windows that represents the active, programmed side of the database system. All management of the database system is performed through and by the server, and all communication with the database systems goes through the server.

When the database server is running, the database system is up and you can connect to the databases. When the database server is not running, the databases are down.


© SAP AG 2003

Services

SQL Server Enterprise Manager - [Console P

Console Window Help

Destination

R3DUMPO

Action View Tools

Console Root

Microsoft SQL Servers

SQL Server Group

sapprod [Windows NT]

Distributed Transaction Co

SQL Server Agent }Services viewed in Windows

Distributed Transaction .. Coordinates Transac… Manual Local SystemMSSQLSERVER Started Automatic PRDadmMSSQLServerAdHelper Manual Local SystemSQLSERVERAGENT Started Automatic PRDadm

Services viewed in the EnterpriseManager

Under Windows, a program can be started as a service. A service runs in the background and has no graphical user interface. To automatically or manually start and stop services at system startup, use the Control Panel Services.

Microsoft SQL Server has the following services: MSSQLServer: Database server SQLServerAgent: Automatic task execution and SQL Server alert handling Distributed Transaction Coordinator: distributes transactions over multiple SQL Servers MSSQLServerAdHelper: Adds and removes objects used to register instances of SQL Server and ensures that the Windows account under which an SQL Server service is running has permissions to update all of the Active Directory objects for the instance, as well as any replication publications and databases for that instance.

To start and stop these services, use the SQL Service Manager or the Enterprise Manager. The service MSSQLServer can also be started remotely from another computer. Starting SQL Server is the same as starting the service MSSQLServer.

The SQLServerAgent service must be started automatically together with MSSQLServer.


© SAP AG 2003

Trace Flags

Trace Flags can be set using

Command Prompt

Startup parameters in the Enterprise Manager

SQL Server can be started and stopped manually using the command prompt by typing: net start mssqlserver or sqlservr, or net start SQLServerAgent or by running SQLSERVR.EXE.

By specifying the –T, SQL Server should be started with a specified trace flag in effect. Trace flags are used to start servers with nonstandard behavior. This way a better understanding of SQL Server or even a change in the way SQL Server will react on specific conditions is provided.

SQL Server can also be started with trace flags using the Startup dialog box in the Enterprise Manager.


© SAP AG 2003

Instances of SQL Server

Windows server

SQL Server default instancesapprod

SQL Server named instancesapprod\SQLServer

MSSQLSERVRSQLSERVERAGENT

MSSQL$sapprod\SQLServerSQLAgent$sapprod\SQLServer

msdbmaster tempdb model

msdbmaster tempdb model

SQL Server 2000 supports multiple instances of the SQL Server database engine running concurrently on the same computer. Each instance of the SQL Server database engine has its own set of system and user databases that are not shared between instances.

There are two types of instances of SQL Server: Default Instances: The default instance is identified by the name of the computer on which the instance is running. When a client program such as the Query Analyzer specifies only the computer name in its request to connect to SQL Server, a connection to the default instance of the database engine on that computer is established. There can only be one default instance on any computer, the default instance can be of any version of SQL Server.

Named Instances: All instances of the database engine other than the default instance are identified by an instance name specified during installation of the instance. Clients must provide both the computer name and the instance name of any named instance to which they are attempting to connect. The computer name and instance name are specified in the format computer_name\instance_name.

There can be multiple named instances running on a computer. Only database engines from version SQL Server 2000 can operate as a named instance.


© SAP AG 2003

Workstation A

ODBC / OLE DB / DB Library

Named pipes

SQL Server

IPX/SPX TCP/IP

Client X

Workstation B

ODBC / OLE DB / DB Library

TCP/IP sockets

Named pipes TCP/IP sockets

ODS (Open Data Services)

Client Y

Windows server

TCP/IP

Client/Server

A database server is an application that runs as a system process on a computer. Client tools (for example, Query Analyzer) communicate with the database server through network protocols such as TCP/IP or NW Link IPX/SPX Compatible Transport, the native protocol of Novell NetWare networks.

The network protocol used by the database server determines the method used for inter-process communication, for example, named pipes or TCP/IP sockets. These methods are installed as net libraries (DLLs) using the program SQL Setup. Only one net library is active on a client. To configure the library, use the program Client Network Utility.

Several application programming interfaces (APIs) are available. For example, OLE DB and DBLib. Other client applications can use the ODBC interface.


© SAP AG 2003

Process...

Windows server

Thread 1 Thread NThread 2 Fiber 1 2 .. n

Processes and Threads

A process is a program that is currently being executed. Under Windows, to keep processes separate from each other and prevent interference, each process has its own address space. The process is also the owner of all the program resources, such as file handles and access tokens.

Windows 2000 allows a process to manage a 4GB range of linear addresses. An application process is able to control the lower portion of this space (either 2GB or 3GB). The upper portion is managed by system code for that particular process.

The basic unit of activity in Windows is the thread. Each process has at least one thread, and may have multiple threads. Windows assigns time slices on physical processors to threads. All threads can run concurrently. Since there is no address space switch involved, switching between threads within the same process is much faster than a conventional process switch in operating systems that run without the thread concept.

SQL Server has an internal layer that implements an environment similar to an operating system for scheduling and synchronizing concurrent tasks without having to call the Windows kernel. This internal layer can schedule fibers as effectively as it works with threads. A fiber is a unit of execution. Fibers run in the context of the threads that schedule them. Each thread can schedule multiple fibers.

The server configuration lightweight pooling parameter controls whether SQL Server uses threads or fibers. The default is 0, in which case SQL Server schedules a thread per concurrent user command. If lightweight pooling is set to 1, then SQL Server uses fibers instead of threads. For more information on the lightweight pooling parameter, please refer to the section SQL Server Configuration in the Unit 6: Regular Maintenance and Error Handling.


© SAP AG 2003

Workstation A

OLE DB

Named pipes

IPX/SPX TCP/IP

Clientprocess X

Workstation B

OLE DB

TCP/IP sockets

Clientprocess Y

Named pipes TCP/IP sockets

ODS (Open Data Services)

SQLSERVR process...Thread 1 Thread 2 Thread 3 Thread N

Windows server

TCP/IP

Processes and Threads (2)

SQL Server runs in an operating system process called SQLSERVR. This operating system process contains threads for the operating system and threads for clients logged on to the server.

Every connection between a client and SQL Server uses one of these threads. SQL Server maintains a pool of either threads or fibers for user connections. The maximum size of this pool is controlled by the max worker threads server configuration parameter. One thread can serve several connections. For more information, please refer to the section SQL Server Configuration in the Unit 6: Regular Maintenance and Error Handling.


© SAP AG 2003

Lazy Writer

Lock Manager

Log Writer

Checkpoint Manager

Background Task

Special Threads

The Lazy Writer thread scans the data cache to write dirty pages to disk. The Lazy Writer thread sleeps for an interval of time. When it is restarted, it checks the size of the free buffer list. If the free buffer list is below a certain point, the Lazy Writer thread scans the buffer cache to write dirty pages to disk. Most of the work writing dirty pages is done by the user threads and the Lazy Writer thread typically finds little to do.

The Lock Manager dynamically adjusts the resources it uses for larger databases, eliminating the need to adjust the locks server configuration parameter manually. Details on the locking mechanism are explained in later slides.

The Log Writer records are written asynchronously by a log writer thread, except when: A commit forces all pending records for a transaction to disk A checkpoint forces all pending records for all transactions to disk

The Checkpoint thread scans the buffer cache for dirty pages and writes to disk any buffer page that are marked as dirty. Checkpoints typically find few dirty pages to write to disk because most dirty pages get written to disk by the worker threads or Lazy Writer thread in the period between two checkpoints.

The Background task checks every 30 minutes if the database or the transaction log files can be shrunk, in case the database option autoshrink is set. It also starts every 5 seconds and checks on 20 data pages if they contain ghost records. These are records which are deleted logically but not physically.


© SAP AG 2003

Security Modes

SQL Server supports security modes for authenticationSQL Server security

Trusted security

Mixed security

SQL Server must verify that the login id supplied on each connection request is authorized to access the database server. This process is called authentication.authentication.

SQL Server supports three security modes for authentication: SQL Server security: a connection to SQL Server is established through a SQL Server login and password, e.g. using login id sa

Trusted security: a connection to SQL Server is established using the Windows user account. When a user connects to SQL Server through a Windows user account, SQL Server verifies that the account name and password were validated when the user logged on to the operating system. The SQL Server client then requests a trusted connection to SQL Server. The properties of a trusted connection include the Windows group and user accounts of the client that opened the connection. A member of the SQL Server sysadmin fixed server role, for example login id sa, must first specify to SQL Server all the Windows accounts or groups that can connect to SQL Server.

Mixed security: Mixed security supports both, SQL Server security and trusted security.


© SAP AG 2003

Databases

Files

msdbmaster tempdb

.ldf.mdf

.ldf.mdf

.ldf.mdf

model

.ldf.mdf

Databases and Files

The system databases are: master holds all the system level information for a SQL Server system. It stores all login accounts and all system configuration parameters.

model is used as the template for all databases created on SQL Server. msdb is used by SQL Server Agent for scheduling alerts and jobs. tempdb holds all temporary tables and temporary stored procedures for all users connected to SQL Server. It also fills any other temporary storage needs such as work tables generated by SQL Server. Every time SQL Server is started, tempdb is re-initialized. The initialization is recorded as “clearing tempdb” in the error log.

Northwind and pubs are sample databases that are provided as learning tools. SQL Server maps a database over at least two operating system files. Data and log information are never contained in the same file, and individual files are used only by one database.

The primary file (.mdf) contains the startup information and system tables for the database and is used to store data. Every database has one primary file. Secondary files (.ndf) hold data that does not fit in the primary data file. Databases do not require any secondary data files if the primary file is large enough to hold all the data in the database. Transaction log files (.ldf) contain the log information used to recover the database. There must be at least one log file for each database.


© SAP AG 2003

System tables andother user tables

Transaction log Log files

Files and Filegroups

RAID systems

RAID 5

RAID 1

PRIMARY

Objects Filegroups

Data files are assigned to one filegroup. A single table can use space in several different files within the same filegroup.

The PRIMARY filegroup is created when SQL Server is installed.


© SAP AG 2003

Volume

Database files

Log files

+ ....

: Free files

+ : ...

File

(Autogrow ONLY for out-of-space failure)

Automatic Growth

Database files only grow automatically in a filegroup if the option autogrow is set and no further space is available on any of the database files in the filegroup.

Filegroups use a proportional fill strategy across all the files within each filegroup. As data is written to the filegroup, SQL Server writes an amount proportional to the free space in the file to each file within the filegroup, rather than writing all the data to the first file until full and then writing to the next file. For example, if File1 has 100 MB free and File2 has 200 MB free, one extent is allocated from File1, two extents from File2, and so on. This way both files become full at about the same time and simple striping is achieved.

Transaction log files, however, cannot be part of a filegroup; they are separate from one another. As the transaction log grows, the first log file fills, then the second, and so on, using a fill-and-go strategy rather than a proportional fill strategy. Therefore, when a log file is added, it may not be used by the transaction log until the other files have been filled first.


© SAP AG 2003

11

22

33

Page File

Row 1

Row 2

Row 3

Tables X

8 Pages = 1 Extent

Page 8

Row 1

Row 2

Page 17

Row 3

0 1 2 3 4 5 6 7

8 9 10 11 12 13 14 15

16 17 18 19 20 21 22 23

Extents

Space Allocation

The page is the unit of data storage in SQL Server. Pages are 8 KB in size. SQL Server allocates pages to objects and reuses space freed up by deleted rows. These operations are internal to the system and use data structures not visible to users.

Extents are the basic unit in which space is allocated to tables and indexes. An extent consists of 8 contiguous pages, or 64KB. A new table or index allocates first pages from mixed extents. That means extents can contain pages from different objects. Extents are called uniform, when a table or index allocates all eight pages.

Log files do not contain pages, they contain a series of log records, allocated on virtual log files.


© SAP AG 2003

Space Allocation (2)

First extent on each file8 contiguous pages = 1 extent (64 KB)

File header Contains information about the attributes of the file

PFS Page allocation / free space availability (in %)Page Free Space

(S)GAM Usage of extents (free, mixed, full)(Secondary) Global Allocation Map

BCM Changes of extents when in Bulked_logged recovery modeBulk Changed Map

DCM Changes of extents since last database backupDifferential Changed Map

Boot page Contains information about the attributes of the databaseexists only in the primary data file

File Bootheader PFS GAM SGAM BCM DCM page ...

0 1 2 3 6 7 8

File Header is a special page that contains information about the file. Page Free Space (PFS) pages record whether an individual page has been allocated, and the amount of space

free on each page. Each PFS page covers 8000 pages. For each page, the PFS has a bitmap recording whether the page is empty, 1-50% full, 51-80% full, 81-95% full, or 96-100% full. Once an extent has been allocated to an object, SQL Server uses the PFS pages to record which pages in the extent are allocated or free, and how much free space is available for use. This information is used for allocating a new page or finding a page with free space available for a newly inserted row.

Global Allocation Map (GAM) pages record which extents have been allocated. Each GAM covers 64000 extents, or nearly 4 GB of data. The GAM has one bit for each extent it covers. If the bit is 1, the extent is free; if the bit is 0, the extent is allocated.

Shared Global Allocation Map (SGAM) pages record which extents are currently used as mixed extents and have at least one unused page. Each SGAM covers 64000 extents, or nearly 4 GB of data. The SGAM has one bit for each extent it covers. If the bit is 1, the extent is being used as a mixed extent and has free pages; if the bit is 0, the extent is not being used as a mixed extent, or it is a mixed extent whose pages are all in use.

Index Allocation Map (IAM) pages map the extents in a database file used by a heap or index. Each heap or index has one or more IAM pages recording all the extents allocated to the object. A heap or index has at least one IAM for each file on which it has extents. A heap or index may have more than one IAM for a file if the range of the extents for the heap or index on that file exceeds the range that an IAM can record.

Bulk Changed Map (BCM) pages record the extents which have been changed by bulk operations such as SELECT INTO and CREATE INDEX since the last transaction log backup. If the bit is 1, the extent has been changed by a bulk operation, if the bit is 0, the extent has not been changed. BCM pages are only relevant when the recovery model of the database is set to bulk_logged. See also chapter 5 Database Recovery.

Differential Changed Map (DCM) pages record which extent has changed since the last execution of BACKUP DATABASE. Each DCM covers 64000 extents, or nearly 4 GB of data. If the bit for one extent is 1, the extent has been changed since the last BACKUP DATABASE, if the bit is 0, the extent has not been changed. Read more about the usage of the DCM in Unit 4: Database Backup.

The Boot page is the ninth page in the database file, that is, the first page of the second extent. It is stored in the primary database file and in the first transaction log file. The boot page contains attributes of the database. It records for example attributes which are needed for an automatic recovery. See also Unit 5: Database Restore.


© SAP AG 2003

Free Space

Owner: prdadmDate created: 02/22/2002 05:55:54Size: 65.000,00 MBSpace available: 15.664,99 MBDatabase options: normalNumber of users: 2

Database

PRD

Last database backup: 10/22/2002 06:00:54 Last differential backup: NoneLast transaction log backup: 10/22/2002 10:00:54Maintenance plans: None

Maintenance

Space Allocated

Data:

PRDdata1: 20000 MB 15655,59 MB 4344,31 MB

PRDData2: 20000 MB 15888,19 MB 4111,81 MB

PRDdata3: 20000 MB 15823,69 MB 4176,31 MB

Transaction Log space: 4999,99 MB 73,53 MB 4926,46

Total Used Free

The log and data usage of a database can be displayed using the taskpad view in the Microsoft Management Console of the Enterprise Manager. The space used for each data file and the transaction log is shown.

In the first section the size of the database is shown as the size of the data and the transaction log. The space available shows the space available for logging information and for data.

The information returned by transact-SQL command DBCC SQLPERF(LOGSPACE) can be used to monitor the amount of log space used and indicates when to back up the transaction log.

System stored procedure sp_spaceused computes the amount of disk space used for data and indexes. When updateusage is specified, SQL Server scans the data pages in the database and makes any necessary corrections to the sysindexes table regarding the storage space used by each table. There are some situations, for example, after an index is dropped, when the sysindexes information for the table may not be current. This process can take some time to run on large tables or databases. The command DBCC UPDATEUSAGE can be run separately.


© SAP AG 2003

§Name xyz abc

Name xyz abc

Logical Database Objects

System Representation

Physical Memory

Tables Views Constraints Stored IndexesProcedures

System Tables

Pages Pages Pages

Database Objects

An SQL Server database consists of tables that contain data and other objects such as views, indexes, and stored procedures defined to support the activities performed on the data.

These database objects are stored on pages in physical memory, along with other data objects defined in the system tables.


© SAP AG 2003

IndexPagesindid >= 2Level 1

Intermediate nodes

Level 0Leaf nodes

Heidelberg,GermanyL.A.,USAAtlanta,USA

London,United King.Manchester,United King.Paris,France

Berlin,GermanyWashington,USAWalldorf,Germany

New York,USA

Page pointer

Rid (row identifier):- File ID,- Page number,- Row number

DataPagesindid = 0

New YorkParisWalldorfWashington

nonclusteredkey values rid

AtlantaBerlinHeidelbergL.A.LondonManchester rid

AtlantaNew York

Level nRoot

Non-clustered Index (on a table with no clustered index)

Indexes are used to speed up searching for records in tables. Indexes can be created for a frequently used search field or a combination of search fields.

SQL Server has two types of indexes: Non-clustered index Clustered index

A non-clustered index is a B tree, which is searched from the highest level to the data pages. A search through the index requires the following page accesses: One page for each index level, and one access on each data page(s).

This graphic displays data and index pages in a table. The search fields are named for City and Country and contain a non-clustered index in the field City. With a non-clustered index, index and data pages are divided, so that you can create as many non-clustered indexes for a table as needed.

The location of a data row is contained in each leaf level of the non-clustered index and is identified by a record identifier (RID) comprised of the file number, page number, and slot number of the row.


© SAP AG 2003

DataPages

Level 1(indid=1)

Level 0(indid=0) France,

Paris,EuropeGermany,Berlin,EuropeGermanyHeidelberg,Europe

Germany,Walldorf,EuropeUnited King.,London,EuropeUnited King.,Manchester,Europe

USA,Atlanta,North AmericaUSA,L.A.,North AmericaUSA,New York,North America

Page pointer

USA,Washington,North America

clusteredkey

IndexPages

France,ParisGermany,Walldorf.USA,AtlantaUSA,Washington

key

Clustered Index

A clustered index dictates the physical storage order of the data in the table. That means a table can contain only one clustered index. All inserts made fit in the ordering sequence of the clustered index key.

A clustered index is organized as a B tree. Each page in a clustered index holds a page header followed by index rows. Each clustered index row contains a key value and a pointer to either a page or a data row. Each page in a clustered index is called an index node. The top node of the B tree is called the root node. The bottom nodes in the clustered index are called the leaf nodes. In a clustered index, the data pages make up the leaf nodes. Index levels between the root and the leaves are known as intermediate levels.

For a clustered index, root points to the top of the clustered index. SQL Server navigates down the clustered index to find the row corresponding to a clustered index key. To find a range of keys, SQL Server navigates through the clustered index to find the starting key value in the range, and then scans through the data pages. To find the first page in the chain of data pages, SQL Server follows the leftmost pointers from the leaf node of the clustered index.


© SAP AG 2003

Index PagesIndexPages

New York

Paris

Walldorf

Washington

USA,New YorkFrance,ParisGermany,WalldorfUSA,Washington

clustered keyvalues

DataPages

Level 1 (indid 1)

Level 0 (indid 0)leaf nodes

Clustered Indexroot Index

Pages

Atlanta

Berlin

Heidelberg

L.A.

London

Manchester

USA,AtlantaGermany,BerlinGermany,HeidelbergUSA,L.A.United King.,LondonUnited King.,Manchester

Non-clustered Index

search via clustered index key (country, city)

Atlanta

New York

...

Non-clustered Index with a Clustered Index

clustered keyvalues

If a table has a clustered index and a clustering key, the leaf nodes of all non-clustered indexes use the clustering key as the row locator rather than the physical record identifier (RID). If a table does not have a clustered index, non-clustered indexes continue to use the RID to point to the data pages.

In both cases, the row locator is stable. When a leaf node of a clustered index is split (data page), the non-clustered indexes do not need to be updated because the row locators are still valid. If a table does not have a clustered index, page splits do not occur.


© SAP AG 2003

Cache

Files

SQL command

detail stats.

BEGIN TRANUPDATE

SQL Server Query Analyzer

Data Log

Logging

A database transaction is a series of SQL commands that are completed, or not executed at all. A transaction is started with the command BEGIN TRAN. All commands within one transaction are handled as one atomic command by SQL Server .

Each SQL Server database has a transaction log that records data modifications made in the database. The log records the start and end of every transaction and associates each modification with a transaction. SQL Server stores the information in the log to either redo (roll forward) or undo (roll back) the data modifications that make up a transaction. Each record in the log is identified by a unique log sequence number (LSN).


© SAP AG 2003

Cache

Files

. . . . .

detail stats.

SQL command


COMMIT TRAN

Data Log

Commit

A transaction is ended with the command COMMIT TRAN or ROLLBACK TRAN. SQL Server then writes all logging information from this transaction to the log file and sends a message to the application program confirming the COMMIT TRAN. This ensures that all confirmed changes from SQL Server are logged onto a physical disk. Records to be changed are first stored in a cache. Sometimes the records to be changed are only up to date in the cache.


© SAP AG 2003

Cache

Files

. . . . .

detail stats.

SQL command


CHECKPOINT

Data Log

Checkpoints

During a checkpoint, all dirty data and log pages in the cache are written to the files. There are 2 kinds of checkpoints:

Manual checkpoint (transact-SQL command Checkpoint) Automatic checkpoint

SQL Server configuration parameter recovery interval controls when SQL Server issues a checkpoint in each database. Checkpoints are done on a per database basis. The recovery interval sets the maximum number of minutes per database that SQL Server needs to recover databases. The default is 0, indicating automatic configuration by SQL Server. This means a recovery time of less than one minute and a checkpoint approximately every one minute for each database.

When a database is set to use the simple recovery model, logging information from the log is erased at each automatic checkpoint. This option is useful for databases that require many changes, and have minimum security requirements, for example database msdb and tempdb. If this mode is set, the restore cannot apply the transaction logs written later than the checkpoint. Perform a full backup immediately after disabling this option.


© SAP AG 2003

detail stats. ISQL/w

Begin TransactionUpdate MARA set MATNR = ‘4711’ where MATNR= ‘0000’Insert into KUNR values (‘9087’, ‘Microsoft’, ‘Redmond’)Select * from KUNR where KNR = ‘5679’

LSN LSN LSN LSN LSN LSN LSN301 302 303 304 305 306 307

Begin Tran1 Update Begin Tran2 Update Commit Checkpoint Begin Tran3 MONI MARA Tran1Tran1 Tran2

MinLSN303

Boot page

Transaction log


detail stats.SQL Server Query Analyzer

Begin TransactionUpdate MONI set KEY = ‘4711’ where KEY= ‘0000’;Commit

Checkpoints: Logical Sequence Number

During a checkpoint, the log sequence number (LSN) of the first log record at which a system-wide recovery must start is written to the database boot page. This LSN is called the Minimum Recovery LSN (MinLSN) and is the lowest of the three following values: The LSN of the checkpoint The LSN of the oldest recorded dirty data page The LSN of the start of the oldest active transaction

The portion of the log file from the MinLSN to the end of the log is called the active portion of the log. This is the portion of the log required for a full recovery of the database. No part of the active log can ever be truncated. All log truncation must be done from the parts of the log before the MinLSN.

The automatic recovery process is explained in detail in Unit 5: Database Restore.


© SAP AG 2003

detail stats.

Selectdetail stats.

Selectdetail stats.

Selectdetail stats.

Updatedetail stats.

Update

....ExclusiveShared

Lock Modes

An RDBMS processes a large number of transactions simultaneously. A lock synchronizes simultaneous accesses to an object. When a transaction accesses an object, the object is temporarily locked to prevent other transactions from accessing it simultaneously.

The type of lock determines which operations from other transactions can be executed on the locked object. Types of locks are: Shared (S) Used for operations that do not change or update data (read-only operations), such as a SELECT statement.

Exclusive (X) Used for data-modification operations, such as UPDATE, INSERT, or DELETE. This type of lock ensures that multiple updates cannot be made to the same resource at the same time.

Update (U) Used on resources that can be updated. This type of lock prevents a common form of deadlock that occurs when multiple sessions are reading, locking, and potentially updating resources later.

Intent (I) Used to establish a lock hierarchy.

Schema (Sch) Used when an operation dependent on the schema of a table is being executed. The two types of schema lock are: Schema stability (Sch-S), Schema modification (Sch-M)


© SAP AG 2003

Row

Table

.... 8 pages

Data rows

Index rows Key range

Database (single user)

Extent

Page

Lock Resources

SQL Server has multi-granular locking that allows different types of resources to be locked by a transaction. These resources are locked at a level appropriate to the task by using a dynamic locking strategy to determine the most cost-effective locks.

SQL Server can lock the following resources, which are listed in order of increasing granularity: RID: Row identifier, used to individually lock a single row within a table

KEY: Row lock within an index, used to protect key ranges in serializable transactions PAG: 8KB data or index page EXT: Extent, contiguous group of eight data or index pages TAB: Entire table, including all data and indexes DB: Database

The finer the lock granularity, the more locks are needed. For example, when accessing a table with 100,000 pages, you can use 100,000 page locks or only one table lock. More locks require more administration time.

If the lock granularity is coarse, other transactions must wait longer until the lock is released. To display locks currently in use, use the stored procedure sp_lock or use the Current Activity View in the Enterprise Manager.


© SAP AG 2003

Fine-grain locks

Coarse-grain locks

Exclusive (X)

Intent exclusive (IX)

Lock escalation( large number of rows or pages -> table lock )

Minimize cost of locking

Table

Transaction 1

RowPage

Row

Exclusive (X)

11

22

33

44

55

Lock Escalation

Intent exclusive (IX)

SQL Server may dynamically escalate or deescalate the granularity or type of locks. Lock escalation is the process of converting many fine-grain locks into fewer coarse-grain locks, thus reducing system overhead.

In this example, a transaction requests rows from a table for update purpose. SQL Server automatically acquires locks on those rows affected (1) and places higher level intent locks on the pages (2) or index, that contain those rows. The table which contains the rows also receives an intent exclusive lock (3). When the number of locks held by the transaction exceeds a threshold, SQL Server attempts to change the intent lock on the page to a stronger lock, e.g. an intent exclusive would change to an exclusive lock (4). After acquiring the stronger lock, all row level locks held by the transaction on the page are released, reducing lock overhead (5).

SQL Server may choose to use both row and page locking for the same query, for example, by placing page locks on the index (if enough contiguous keys in a non-clustered index node are selected to satisfy the query) and row locks on the data. This reduces the likelihood of lock escalation.

SQL Server rarely needs to escalate locks; the query optimizer usually chooses the correct lock granularity at the time the execution plan is compiled.

Lock escalation thresholds are determined dynamically by SQL Server and do not require configuration.


© SAP AG 2003

Transaction Isolation LevelTransaction Isolation Level (1)

T1

T2

X=5

update X

X=7

select X

rollback

X=7

X=5X=7

T1

T2

select X,Y

X=7Y=2

update X, delete Y

select X,Y

X=4X=4X=7Y=2

X=7,Y=2 X=4,Y unknown

T1

T2

select <10

insert X, update Y, Z

Y=7 select <10 X=5,Y=8, Z=3

Dirty read

Non-repeatableread

Phantom

X=5Y=8Z=3

X=5Y=8Z=3

Y=7Z=12

Y=7Z=12

changing command

reading command

X=7 data read

Several users running concurrent transactions can cause inconsistencies in the data read by other users. The following situations may occur: Dirty read: Transaction T1 updates data X. Another transaction T2 then selects X before T1

performs a COMMIT. T1 then performs a ROLLBACK. So T2 has read a value for X that never existed in the database as consistent (committed) data.

Non-repeatable read: Transaction T1 selects data X, Y. Another transaction T2 then updates X and deletes Y and commits. T1 then selects X, Y again. It reads a modified value for X and discovers that Y does not exist.

Phantom data: Transaction T1 selects all the data that satisfies the condition < 10. Only X is returned. Transaction T2 then creates data Z and updates Y so as to satisfy the condition < 10. T2 commits. T1 then again selects all the data < 10. Now X, Y, and Z are returned. So new data appeared (phantom data).


© SAP AG 2003

Transaction Isolation LevelTransaction Isolation Level (2)

Read uncommitted

selectcommit

Read committed

select commit

Serializable select commit

Repeatable read

select commit

key range lock

lock lock

lock lock

lock hold lock

hold lock

Data without lock

Another processholds a sharedlock on this data

Another processholds an exclusivelock on this data

Data is read

Select waits untillock is released

The isolation level determines to what degree one transaction is isolated from other transactions. A lower isolation level increases concurrency but at the expense of data correctness. A higher isolation level ensures that data is correct, but can negatively affect concurrency. The isolation level set by an application determines the locking behavior used by SQL Server.

To define the isolation level for a connection to the database, use the transact-SQL command SET TRANSACTION ISOLATION LEVEL.

SQL-92 defines four isolation levels, all of which are supported by SQL Server: Read uncommitted accepts all dirty reads, non-repeatable reads and phantoms. No shared locks are issued and no exclusive locks are honored.

Read committed avoids dirty reads. To achieve this shared locks are held while data is being read. But the data can be changed before the end of the transaction, resulting in non-repeatable reads or phantom data. This option is the SQL Server default.

Repeatable read avoids both dirty and non-repeatable reads. It sets locks on all data used in a query and prevents other users from updating the data. Phantom data can occur.

Serializable avoids all dirty reads, non-repeatable reads and phantoms. It sets a range lock on the data range selected. Other users cannot update or insert rows into that data range until the transaction is complete.


© SAP AG 2003

Now you are able to:

Describe the main features of the SQL Server architecture

Explain the different authentication modes

Understand the different index types

Name the system databases and their main functions

Summary of this Unit


© SAP AG 2003

Further Documentation

Inside Microsoft Sql Server 2000Microsoft Press, ISBN: 0-7356-0998-5


© SAP AG 2003

How the SAP System uses SQL Server

1 Introduction

3 How the SAP System uses SQL Server4 Performance Monitoring and Tuning

5 Database Backup

6 Database Restore




© SAP AG 2003

ContentsClient/Server architectureDatabase access and securityDatabases used in the SAP System


Describe the architectural specifics under SQL Server and the SAP SystemName the databases used in the SAP environment and explain theirfunctionality

How the SAP System uses SQL Server


© SAP AG 2003

SAPWork-

ProzessSAPWork-

Prozess

SAPWork-

Prozess

SAPApplicationServer

SAP Dispatcher

SAPInstance

SAPWork-

Prozess

SAP GUISAP GUI

PresentationServer

SAP GUISAP GUI

DatabaseServer SQL Server

Databases

SAP Instance

Architecture of the SAP System (1)

The SAP System has a 3-tier architecture that includes the following: Presentation server Application server Database server

The presentation server consists of a graphical user interface on the front end. Each SAP System user runs an SAP GUI on a dedicated front-end computer.

The application server is used primarily for data processing. An SAP instance that runs on an application server combines special SAP services such as dispatching requests. The dispatcher, a special process on each SAP instance, communicates with the front end SAP GUIs and distributes work requests to the work processes. The application logic runs in the work processes, which are connected to the database server.

The database server is the computer where the database management system (DBMS) is located. The DBMS (SQL Server) controls the databases and the database objects that perform all the database activities.

Note that an SAP GUI on a presentation server does not connect directly with the database; only the work processes do this.


© SAP AG 2003

SAPWork-

Prozess

SAPApplicationServer SAP

Dispatcher SAPWork-

Prozess

SAP GUISAP GUI

PresentationServer

SAP GUISAP GUI

SQL Server

Databases

Architecture of the SAP System (2)

SAP InstanceSAP Instance

SAPWork-

Prozess

DatabaseServer


© SAP AG 2003

SAP InstanceSAPApplicationServer

DatabaseServer SQL Server

Databases

SAP Work Process

Database Access Agent

R/3 DB IF

DBSL IF

R/3 Database Interface

SQL Server accesses objects stored in databases on the database server and accepts all requests from the SAP System through the database interface from the database access agent.

The database access agent in the work processes handles database requests, and consists of several subcomponents. One of these subcomponent is the database vendor-independent R/3 database interface (R/3 DB IF), which handles accesses to table and dictionary buffers.

The database access agent also provides the Database SQL Library Interface (DBSL IF), which is database vendor-specific. All components on the SAP System side of this interface are independent of the database used. On the database side of the interface, only system components provided by the database vendor are used.

With SQL Server, components on the database side of the DBSL IF are implemented using a Microsoft product. The main task of the DBSL IF is the mapping of ABAP Open SQL statements to the database vendor-specific SQL language.


© SAP AG 2003


DatabaseServer

SQL Server

TCP/IP sockets

ODS (Open data services)

OLEDB

TCP/IP sockets

TCP/IP

Database Connections

SAP Work-Prozess

SAP Work-Prozess

Each SAP work process is connected to the database server through several connections that are used for executing database commands. The DBSL IF is implemented using the Microsoft OLE DB.

If the SAP work processes have to connect to a computer other than the database server, the communication protocol will be TCP/IP, otherwise Named Pipes is chosen.

Open Data Services (ODS) is the component that manages all packets coming from the server net libraries, for example TCP/IP sockets. The database server processes all requests passed to it by ODS and returns the results to the client.


© SAP AG 2003

Client Network Utility

SAPApplicationServer TCP/IP sockets

DatabaseServer

TCP/IP

SAPCentralServer

Named Pipes

Named Pipes

SQL Server

Named PipesTCP/IP sockets

SQL Server

...SAP Work

Process...SAP Work

Process

SAP Work

Process

SAP Work

Process

In the SAP environment the network protocols should be activated as follows: SAP Application Server connecting over the network: On every SAP Application Server the default network protocol must be TCP/IP. In the client network utility, set TCP/IP at the top of the list.

Cluster server with one node running SAP connecting to an SQL Server running on another node: TCP/IP should be activated on the node where the SAP system runs.

Central system connecting within the same machine: If the central database server is also running the SAP system the protocol must be set to Named Pipes. Named Pipes is more efficient than TCP/IP on the local system, because the data transfer occurs in main memory only.

You can activate a protocol using the SQL Server Client Network Utility for each client, and the Server Network Utility for the server. Only one protocol can be active on any one SAP Application Server. To start the SQL Server Client Network Utility choose Programs/Microsoft SQL Server, and then click Client Network Utility.


© SAP AG 2003


SAP Work process

SQL Server

DatabaseServer

01234567 01234567

READ COMMITTED

READ UNCOMMITTED

0 Consistent transactions1 Stored proc.

Create/Select single2..N Dirty read selects

Database Connections to SAP Work Processes

SAP Work

Process

SAP Work

Process

Several connections are used for each SAP work process, with the following isolation levels: Connection 0: Committed read connection, used for consistent transactions involving inserts, deletes, updates, and select for update or database cursor usage.

Connection 1: Uncommitted read connection, used for all DDL transactions, creating stored procedures, and to execute single selects and dirty read cursors.

Connection 2...N: Uncommitted read connection, used for dirty read selects. A maximum number of 40 connections are established for each work process, other than connections 0 and connection 1. These connections are opened as needed but only closed when shutting down the instance or restarting the work process. If the select is nested in too many surrounding selects (more than 40), cursors are used in connection 1.

Each connection to the database uses approximately 50KB of memory.


© SAP AG 2003

SAP Process Monitor

0AWAITING COM.09.345734.9930sleepi.419011

0AWAITING COM.40123512.9020KERBE..sleepi.39

0AWAITING COM.69096745.2340KERBE..sleepi.63

0AWAITING COM.6.0127.90132.9010KERBE..sleepi.14

0SELECT8.3455.99888.3450KERBE..runn.87

0AWAITING COM.9.4509.023620.6010KERBE..sleepi.907901

0AWAITING COM.040155.3310sleepi.43

0AWAITING COM.568558090sleepi.59

0AWAITING COM.1.50990178.4010sleepi.67

0AWAITING COM.7.9031.390501.2900sleepi.891809

0AWAITING COM.121509.4570SAPLTHBEARLY..sleepi.42

0AWAITING COM.9045690.3460SAPLTHBEARLY..sleepi.78

0AWAITING COM.4561.234801.5700SAPLTHBEARLY..sleepi.56

0AWAITING COM.2.0488.85648.8900SAPLTHBEARLY..sleepi.125465sapprd

To call the SAP Process Monitor, choose Tools → Administration → Monitor → Performance → ∆αταβασε → (ST04) Activity. Choose button Detail analysis menu then button SQL processes. By default, the output is sorted by the CPU time and only connections used by the SAP System are shown. To display the SQL processes sorted by the host process id (pid), choose button Group by/Raw Display or F8.

The Appl. Server column displays the host name of the database client. For all connections established by the SAP System, this is the name of the SAP application server to which the SAP user is connected.

The host pid shows the process id of the SAP work process. Each SAP work process opens a number of database connections, each of which is identified by its SQL Server process id (spid) and labeled by the Application program name R3<T><nn>(<mm>)<type>: <T>: Work process identifier (D: Dialog, B: Background, S: Spool, U: Update, E: Enqueue, 2: Update2, ‘ ‘:external tool such as saplicense or tp)

<nn>: Work process number <mm>: Number of connection <type>: Connection context controlled by the DBIF - comm rd: Committed read - sp create, single select: DDL, creation of stored procedures and single selects - unc rd: Dirty read selects

Note: The Application program name is not shown in the default layout. Choose another layout to hide or display other columns.

To display the command that is currently being executed for a connection, double-click a row. Choose Goto → All DB in the SAP menu to display connections to all databases run on the database server.


© SAP AG 2003

SQL Server Process Monitor

Process Info 97 itemsProce. User Database Status Open Tran Command Application

1 system no database cont background 0 LAZY WRITER3 system master sleeping 0 SIGNAL HANDLER

10 sapprod\SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D00(1)sp create, single select OLEDB11 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D01(1)sp create, single select OLEDB12 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D00(0)comm rd OLEDB13 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D00(2)unc rd OLEDB15 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D01(2)unc rd OLEDB16 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D02(2)unc rd OLEDB4 system no database cont background 0 LOCK MONITOR

20 sapprod\SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3B11(1)sp create, single select OLEDB21 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3D02(1)sp create, single select OLEDB22 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3S12(0)comm rd OLEDB23 sapprod\ SAPServicePRD PRD sleeping 0 AWAITING COMMAND R3S12(2)unc rd OLEDB2 system no database cont background 0 LOG WRITER5 system master background 0 TASK MANAGER

You can also display the SQL processes by choosing Enterprise Manager → Management → Current Activity → Process Info. The Process Info view displays all SQL processes sorted by their login id. A colored globe indicates an active process. All inactive processes are marked by gray globes.

The information displayed is read from the table sysprocesses, which contains information about the client and system processes running on SQL Server. Table sysprocesses is logically found in database master. It is built dynamically each time it is read, and therefore does not exist physically.


© SAP AG 2003


SAP Instance


SAP Work Process

DBSL IF

Sharedbuffers

select * from T100where ...

endselect.

Data Select from ABAP: Table Buffers

This section explains how the SAP system accesses the database when executing ABAP programs. In ABAP, you access the database by using Open SQL commands. The ABAP program and its Open SQL commands are independent of the database system.

An Open SQL command is converted into a standard form and is passed to the Database Access Agent. The Database Access Agent checks whether the accessed table is buffered in an SAP table buffer. If the table is buffered, the data is retrieved from the SAP buffers and results are supplied without accessing the database.


© SAP AG 2003


SAP Instance

SAP Work Process


DBSL IF

create Y2R6GH78J676ABC0000TBTCO

select * from TBTCOwhere ...endselect.

Data Select from ABAP: PREPARE

SAP Work Process


DBSL IF

SELECT OBJECT, TABNAME, INDNAME, TGORDER, FCT, EXECMODE, SEVERITY, GDATE, GUSER FROM TBATG WHERE FCT IN (@P1, @P2, @P3, @P4, ...)

Permanent Stored Procedures

Direct SQL Statements

select * from TBATG into table I_TBATG for all

entries … where ...endselect.

If the requested data is not found in the SAP buffers, the DBSL IF translates the Open SQL command to one or more stored procedures or to a direct SQL statement: Stored procedures are reusable collections of SQL commands, compiled in a single execution plan. Permanent stored procedures are stored in the SAP database and are not deleted after restarting SQL Server or the SAP instance. The DBSL IF creates a unique stored procedure name for each Open SQL command.

Direct statements come from dynamic Open SQL statements, for example FOR ALL ENTRIES. Direct SQL statements are created directly on the SAP database without creating a stored procedure in prior.

When an Open SQL statement is send to the database access agent, the operation is called the PREPARE operation. The following steps are performed: The statement is analyzed by the database access agent and its origin is classified. The open SQL statement is converted into a native SQL statement. For permanent stored procedures a unique stored procedure name is created by the DBSL agent. A set of parameters is generated from the WHERE condition for the statement text. A stored procedure text is generated consisting of the stored procedure name, the parameters and the text. The stored procedure is created on the database.

For direct SQL statements the native SQL statement is executed on the database and cached in the procedure cache.


© SAP AG 2003


Data Select from ABAP: OPEN and FETCH

SAP Instance

SAP Work Process


DBSL IF

executeY2R6GH78J676ABC0000TBTCO

select * from TBTCOwhere ...endselect.

SAP Work Process


DBSL IF

Sp_executesql(SELECT OBJECT, TABNAME, INDNAME, TGORDER, FCT, EXECMODE, SEVERITY, GDATE, GUSER FROM TBATG WHERE FCT IN (@P1, @P2, @P3, @P4, ...))


Direct SQL Statements

select * from TBATG into table I_TBATG for all

entries … where ...endselect.

When a permanent stored procedure is executed, the following operations are performed: The DBSL agent passes the command to execute a stored procedure including the parameters to SQL Server.

The SQL statements in the stored procedure are compiled and optimized and an execution plan is created. If the stored procedure already exists and an execution plan is still in procedure cache, the execution plan is reused.

SQL Server executes the stored procedure. Direct SQL statements are executed using system stored procedure sp_execute in the OLE DB interface. This stored procedure executes a transact-SQL statement that can be reused many times, or that has been built dynamically. The transact-SQL statement can contain embedded parameters.

The execution of the stored procedure or the direct SQL statement starts with the OPEN operation. The group of records returned when a stored procedure or a direct SQL statement is executed is called a result set. While the stored procedure or direct SQL statement is executed, the result set is transferred from SQL Server to the requesting SAP work process. This is called the FETCH operation.


© SAP AG 2003


YR...

exec YR...

in cache

SAP Instance

SAP Work Process


Storedprocedure

namecache

DBSL IF

exec Y2R6...

exec Y2R6...

In namecache

?

yes no

Stored Procedure Name Cache

In sysobjects

?

no yes

create Y2R6…exec Y2R6...

Before creating a permanent stored procedure, the DBSL IF checks if the unique name already exists in a stored procedure name cache. If a stored procedure name is not found in the stored procedure name cache, the DBSL IF checks whether it exists in the sysobjects table. If it is not found in the sysobjects table it is created on the database before it is executed. If the stored procedure is found in the name cache, it is executed directly.

The names of permanent stored procedures are stored in a cache defined by the SAP instance profile parameter dbs/mss/pn_cache_size. This value gives the number of permanent stored procedure names in the name cache. The default value is 10000 for kernel releases 6.20; for later kernel releases the default value is 20000.

Stored procedure names are stored physically in the sysobjects table, stored procedure texts are stored physically in the syscomments table in the SAP database.


© SAP AG 2003

<SID>

Y2R10000000BB1H1911DD03LY2A000001D9C7VK3725RSCROSS10i0o5nsY2E000011F2CBC50301REPOLOADY2R65000168C73K0943ALPERFDBi11o18nsY2C00000453CBC21832DBSYNSEQY2A0000032FCAUK4623SAPLSEUwCOMPONENTut01i6o1nsY2R10000000C5GG0516TBTCPY2K00002B2DCBC21130DOKCLUi5o9nsY2R20000001C5GG1657TST01. . .

Stored Procedure Names

Naming conventions for permanent stored procedures: Y2<ModuleType><StatementId><Timestamp><ModuleName><Suffix> <ModuleType> identifies the type of module the statement comes from, for example, A for ABAP report, C for a C-program, R for a table name

<StatementId> is usually the same as the statement id in the SAP System; statement Ids uniquely identify an SQL-DML statement in the SAP environment

<Timestamp> usually specifies the generation time of an object, for example, the generation time of an ABAP report

<ModuleName> is derived from the module name given in the statement id, for example the table name or report name

<Suffix> is optional and indicates for example single selects (ut01 means up to one row)


© SAP AG 2003

Presentationserver

SAP Applicationserver

Databaseserver

MSSQLserver

<DOMAIN>\SAP_<SID>_GlobalAdmin

pubs

<SID>sapusr

§

Windows Authentication

SAP System

SAP GUI

SAP user: .....PW: .....

Windows user SAP user Login Id Server role

Example: Joe SAPUSR <DOMAIN>\ System SAP_<SID>_ Administrator,GlobalAdmin default DB <SID>

Authori- Logon at Access to Logon at Access tozation: workstation SAP objects SQL Server DB objects

From Release 4.5A, the SAP System uses trusted connections when running with SQL Server exclusively. With this method, the SQL Server login id sapr3 is not used for SAP work process connections. The Windows user running the SAP service (SAPService<SID>) connects to the database server.

Access to SQL Server is controlled by the Windows NT account or group, which is checked when logging on to the operating system on the application server. When SAP work processes connect to SQL Server, they request a Windows trusted connection to SQL Server. Windows does not open a trusted connection unless the SAP application server has successfully logged on using a valid Windows account. In this case SQL Server does not have to check the account. SQL Server gets the user account information from the trusted connection properties and matches them against the Windows accounts defined as valid SQL Server logins. If SQL Server finds a match, it accepts the connection.

Refer to the SAP Security Guide Volume II Microsoft SQL Server under Windows NT or Windows 2000 for detailed information.


© SAP AG 2003

msdbmaster tempdb

.ldf.mdf

.ldf.mdf

.ldf.mdf

<SID>

.ldf.mdf

Databases in an SAP SQL Server System

Databases

Files

During SQL Server installation, databases master, tempdb, and msdb are created. master: Records all of the system level information for a SQL Server system, that is, all login accounts, system configuration settings, existence of all other databases and the location of the primary files.

msdb: Used by SQL Server Agent for scheduling alerts and jobs, e.g. regular backups. tempdb: Contains all temporary tables and is re-created every time SQL Server is started. Temporary tables are automatically dropped at disconnect, and no connections are active when the system is shut down.

In addition to the standard databases created in every SQL Server installation, a database is created for the SAP System. This database receives a 3-letter identification (for example, PRD, TST).

In this course, the SAP database is referred to as <SID>.


© SAP AG 2003

SAP Database <SID>

Files

File System <drive>:\<SID>DATA1\<SID>DATA1.mdf

<drive>:\<SID>DATA2\<SID>DATA2.ndf

<drive>:\<SID>DATAn\<SID>DATAn.ndf

<drive>:\<SID>LOG1\<SID>LOG1.ldf

<drive>:\<SID>LOGm\<SID>LOGm.ldf

PRIMARYFilegroup

...

<drive>:\<SID>DATA3\<SID>DATA3.ndf

...

<SID>DATA1<SID>DATA2

<SID>DATA3

<SID>DATAn

<SID>LOG1

<SID>LOGm

...

SAP Database Files

Each database has two logical parts: data (data files) and transaction log (log files). When SQL Server and the SAP System are installed, the data files of the <SID> database are created in the directories <drive>:\<SID>DATA1\<SID>DATA1.mdf and <drive>:\<SID>DATAn\<SID>DATAn.ndf, where n is the number of the file. The data files may reside on different physical drives. SAP recommends storing the data files using RAID5. The standard installation creates 3 data files. This makes it easier to expand the database. See Chapter 6 ‘Regular Maintenance and Error Handling'.

The transaction log file is created in the directories <drive>:\<SID>LOG1\<SID>LOG1.ldf and <drive>:\<SID>LOGm\<SID>LOGm.ldf, where m is the number of the file. The log files must be mirrored. Hardware mirroring using RAID1 is strongly recommended. The standard installation creates one log file.

After a standard installation, all SAP data files reside in the special filegroup PRIMARY. Stored procedure sp_helpfile returns the physical names and attributes of files associated with the current database. Use this stored procedure to determine the names of files attached to one database.


© SAP AG 2003


Explain the architecture of SQL Server and the SAP System

Name the different databases used under SAP, and explain their function

Describe the authentication modes used under SAP

Explain how ABAP SQL statements are executed on the SAP database

Summary


© SAP AG 2003

Books Online

SAP Installation Guide

SAP Database Administration with Microsoft SQL Server 2000SAP Press, ISBN: 1-59229-005-1

SAP Security Guide Volume II Microsoft SQL Server under Windows NT or Windows 2000



© SAP AG 2003

Unit Actions

Exercises?

Solutions


Exercises

Unit: How the SAP System uses SQL Server Topic: Database Access and Security

At the conclusion of this exercise, you will be able to:

• Change the password of Windows user <Domain name>\SAPService<SID>

• Understand the way the SAP System opens connections to the database and executes SQL statements on the database

For security reasons the customer wants to change the password of the user who runs the SAP System.

The customer observes a high activity in one SAP work process and wants to find out, what this SAP work process executes on the database.

1-1 Changing the password of Windows user <Domain name>\SAPService<SID>

1-1-1 Change the password of Windows user <Domain name>\SAPService<SID>. Stop the SAP System using the SAP Microsoft Management Console and stop Windows service SAP<SID>_<instance number>. Restart the service and the SAP System. What do you observe?

1-1-2 Change the password of Windows user <Domain name>\SAPService<SID> back to the old password and restart the service and the SAP System. What happens?

1-1-3 How do you correctly change the password of Windows user <Domain name>\SAPService<SID>? What steps do you have to perform?

1-2 Uncommitted read connections to the database

1-2-1 Kill all uncommitted read connections from SAP work process 0, which have been opened to SQL Server. How do you determine the SQL Server Process IDs (spid)? Which statement or tool do you use to kill SQL Server connections?

1-2-2 Run program ZADM520NEST using transaction SE38. Observe the SQL processes and their activities using transaction ST05. Find out which SAP work process runs program ZADM520NEST.

1-2-3 Did the SAP work process open new connections to the database? What is the maximum number of uncommitted read connections?


Solutions

Unit: How the SAP System uses SQL Server Topic: Database Access and Security

1-1 Changing the password of Windows user <Domain name>\SAPService<SID>

1-1-1 Stop the SAP System. To change the password of <Domain name>\SAPService<SID>, use the Windows User Manager under Start → Programs → Administrative Tools → Computer Management. Under folder Local Users and Groups select folder Users and mark user SAPService<SID>, click the right-mouse button and choose Set password. Stop the SAP System using the SAP System Management Console and stop service SAP<SID>_<instance number> using the Computer Management tool. When restarting the service error message Could not start the SAP<SID>_<instance number> service on Local Computer. Error 1069: The service did not start due to a Logon Failure. occurs. The service cannot be restarted because the authorization is not correct. Windows authorization is checked for service SAP<SID>_<instance number>, but not when the work process is started. The login attempt of the work process connections to SQL Server fails.

1-1-2 Change the password back using the Windows User Manager. Start the service SAP<SID>_<instance number> again. Start the SAP System.

1-1-3 Stop the SAP System and service SAP<SID>_<instance number>. Change the password of user <Domain name>\SAPService<SID>. Specify the new password in the service used for the SAP System. Select service SAP<SID>_<instance number> in the Computer Management tool, click the right mouse button and select Properties. In the Log On tab change the password. Start the service and the SAP System again.


1-2 Uncommitted read connections to the database

1-2-1 To display the connections each SAP work process has opened to the database, call monitor Database Performance Analysis: SQL Server Database Overview (Transaction ST04). Choose Detail Analysis Menu → SQL processes. Choose layout /4ADM520 to sort the connections by their host process IDs (Host PID). Check only processes opened by the SAP System, i.e. processes where the login ID <Domain name>\SAPService<SID> is displayed in column SQL Server Login. The column Application program shows names starting with: R3D00(x), where D is the work process type (dialog), 00 the number of the SAP work process and x is the number of the connection. The uncommitted read connections are identified by unc rd OLEDB in the program name. The spid is displayed in column SPID. Alternatively, use the SAP Process Overview (transaction SM50) to display the Host PID belonging to SAP work process 0. To kill a SQL Server connection use transact-SQL statement KILL or choose the Enterprise Manager, mark the connection in the SQL Server Process Monitor, click the right-mouse button and choose Kill Process.

1-1-2 In a new window, call transaction ST05 and start a SQL trace. Call transaction SE38, and execute program ZADM520NEST. Call the SAP Process Monitor, use transaction code ST04, and choose Detail Analysis Menu → SQL processes. To group the output, select Group by/Raw display. To find out which SAP work process runs ZADM520NEST, look for column Report.

1-1-3 To find out the connection numbers used by the work process: Display the trace by choosing button Display trace in transaction ST05. Choose Trace List → Display Extended Trace List, and look for program ZADM520NEST in the column Program. In the comment in column Statement the stored procedure name and the connection number are displayed. The connection number is also displayed in the Database Process Monitor in the column Application program as R3D00 (X). ZADM520NEST executes nested selects on database table DD32S. For each nested select, a new uncommitted read connection to the database is opened. A maximum of 40 dirty read connections will be opened.


© SAP AG 2003

Performance Monitoring and Tuning

1 Introduction

5 Database Backup

6 Database Restore



4 Performance Monitoring and Tuning3 How the SAP System uses SQL Server


© SAP AG 2003

ContentsDatabase configuration

Cache size tuning / CPU utilization / Parameter tuning / Disk layout

Application problemsLockwaits / Expensive statements


Monitor the performance of SQL ServerIdentify the source of database performance problemsFind solutions to the performance problems detected

Performance Monitoring and Tuning


© SAP AG 2003

Performance problemsdue to:

Poor database configuration

Inefficient applications

Poor hardware configuration

1 2

Poor configuration

Sources of Database Performance Problems

Poor database performance normally results from problems with database configuration, application coding problems, or hardware capacity. This section explains how to identify and correct these problems.

The first step is normally to tune the database and hardware configuration. In the Database Monitor, you: Check the general database configuration Use the database tools to verify the efficiency of the application coding

Note: Tuning inefficient application coding may allow you to avoid buying new hardware. Changing the database configuration may affect the amount of hardware required. This unit therefore also explains how to check if your current hardware configuration allows the necessary database configuration changes.


© SAP AG 2003

Disk layout

Parameter settings

Data cache

Poor hardwareconfiguration

Mainmemory

CPU

Disks


Poor configuration

Database Configuration

This section discusses the following areas of database configuration: Data cache SQL Server configuration parameters Disk layout

To detect data cache problems, the cache hit ratio is the main indicator. An improper separation of several logical database portions and other frequently accessed files may lead to disk hot spots, which results in high disk response times.


© SAP AG 2003

Database

RDBMSData cache

Procedure cache

parsed SQLstatements andexecution plan

MemoryArea

SAP Work Process

SELECT * FROM MARA

WHERE ...

Tables Indexes Procedure Texts

MARAMARA_1

MARA_0

Y2R0000064B69RC5522MARA

SQL connections

create proc

Cache Usage

An Open SQL statement coming from an ABAP program is executed as a stored procedure or as a direct statement in SQL Server.

For each table, there is a set of common statements such as INSERT, UPDATE and DELETE operations. A permanent stored procedure exists for each of these statements. These stored procedures can be reused.

To shorten access times, the execution plan of a stored procedure is stored in the procedure cache. SQL Server automatically allocates the necessary space from the available memory, and dynamically adjusts the size of the procedure cache.

For a detailed description of this stored procedure mechanism, see Unit 2: How the SAP System uses SQL Server.


© SAP AG 2003

Database

RDBMSData cache

Procedure cache

parsed SQLstatements andexecution plan

MemoryArea

SAP Work Process

SELECT * FROM MARA

WHERE ...


MARAMARA_1

MARA_0

Y2R0000064B69RC5522MARA

SQL connections

load proc

Statement Execution (1)

If the stored procedure plan does not exist in the procedure cache, the procedure text in the database is used to generate the execution plan.


© SAP AG 2003

Database

RDBMSData cache

Procedure cache

MemoryArea

SAP Work Process

SELECT * FROM MARA

WHERE ...


MARAMARA_1

MARA_0

Y2R0000064B69RC5522MARA

SQL connections

exec proc

1

Statement Execution (2)

22

The execution plan determines which indexes are to be used for table access (step 1). Once the execution plan of the stored procedure is in the procedure cache, the procedure can be executed.

The required parts of the used indexes and tables are transported from the disk into the data cache for further processing (step 2). It is helpful if the pages required are already stored in the data cache from previous use.

The Query Processor selects the data required, and returns it through the database interface to the SAP work process.


© SAP AG 2003

1 > 500

345

Cache Usage Analysis

1 > 6h

2 > 98

To display the most important performance parameters of the database, call Transaction ST04, or choose Tools → Administration → Monitor → Performance → Database → Activity. An analysis is only meaningful if the database has been running for several hours with a typical workload. To ensure a significant database workload, we recommend a minimum of 500 CPU busy seconds. Note: The default values displayed in section Server Engine are relative values. To display the absolute values, press button Absolute values. Check the values in (1).

The cache hit ratio (2), which is the main performance indicator for the data cache, shows the average percentage of requested data pages found in the cache. This is the average value since startup. The value should always be above 98 (even during heavy workload). If it is significantly below 98, the data cache could be too small. To check the history of these values, use Transaction ST04 and choose Detail analysis menu → Performance database. A snapshot is collected every 2 hours.

The current size of the data cache (3) is displayed along with its target server memory, which is the total amount of dynamic memory the server can consume since server start (4). Memory setting (5) shows the memory allocation strategy used, and shows the following: FIXED: SQL Server has a constant amount of memory allocated, which is set by SQL Server configuration parameters min server memory (MB) = max server memory (MB).

RANGE: SQL Server dynamically allocates memory between min server memory (MB) < > max server memory (MB).

AUTO: SQL Server dynamically allocates memory between 4 MB and 2 PB, which is set by min server memory (MB) = 0 and max server memory (MB) = 2147483647.

FIXED-AWE: SQL Server has a constant amount of memory allocated, which is set by min server memory (MB) = max server memory (MB). In addition the address windowing extension functionality of Windows 2000 is enabled.

A detailed description of the memory parameters is found in the following slides.


© SAP AG 2003

Memory Settings

Windows Startup Parameters/3GB

/pae

SQL Server Memory Parametersmax server memory

min server memory

set working set size

awe enabled

Size of physical and virtual memory

Managing SQL Server memory involves consideration of the Windows startup parameters /3GB /pae

and the SQL Server configuration parameters max server memory min server memory set working set size awe enabled

How memory should be configured for SQL Server further depends on: the coexistence of other SAP components on one physical machine (i.e. either a Central Instance

or an Update Instance on the same machine as SQL Server runs) the amount of available RAM and virtual memory

Recommended memory settings are detailed in SAP note 327494.


© SAP AG 2003

Windows Startup Parameters

Memory Address Space

0

1 GB

2 GB

3 GB

4 GB

to allow one process the use of more than 4GBPhysical memory

to allow 1GB system address space

…


0

1 GB

2 GB

3 GB

4 GB

…

boot.ini

/pae

boot.ini

/3gb

2-3 GB user process space

and

16 GB64 GB

Standard Windows 2000 Server provides support for up to 4 processors and 4 GB of RAM. Windows 2000 Advanced Server supports up to 8 processors in an Symmetric Multi Processing (SMP) system and, with the Address Windowing Extensions (AWE), can support up to 8 GB of memory. Windows 2000 Datacenter supports up to 32-processor SMP systems and a maximum physical memory of 64 GB.

AWE lets applications such as SQL Server Enterprise Edition use addresses above 4 GB to cache data in memory. AWE maps up to 64 GB of physical memory into a 32-bit (4 GB) virtual address space in 4 KB pages. AWE extends the Windows page tables from 20 bits to 24 bits. The extra four bits enable the use of 36-bit physical addresses. To enable the Physical Address Extension (PAE) add /pae in the boot.ini file.

The /3GB startup parameter is a switch informing the operating system to use only 1 GB of memory for system addresses which then permits an increase in the amount of user process space from the default of 2 GB to 3 GB. This startup parameter is only available with the Advanced Server or Datacenter Windows 2000 products. This feature is enabled by adding /3GB in the boot.ini file.

When the /3GB switch is used in conjunction with the /PAE switch, the operating system does not use any memory in excess of 16 GB.


© SAP AG 2003

SQL Server Memory Parameters


max server memory (MB)min server memory (MB)

used to dynamically adjust memory that SQL Server needs

awe enabled = 1

Used to access up to 8 GB of memory on Windows 2000 Advanced Server and 64 GB on Windows 2000 Data Center


SQL Servermemory

SQL Servermemory


sp_configure‘max server memory (MB)’, ‘2048' reconfigure with override


sp_configure‘awe enabled’, ‘1' reconfigure with override

SQL Server is able to dynamically adjust the amount of memory that it needs for optimal performance. With the use of the max server memory and min server memory parameter settings the default behavior of automatic memory adjustment can be overridden. This can help in situations where the SAP System runs as a combined instance on the same machine doing substantial work that make it difficult for SQL Server to dynamically allocate memory properly.

Parameter awe enabled allows SQL Server to be able to use the AWE (Address Windowing Extension) feature of Windows 2000. By default, SQL Server can use a maximum of 3 GB of RAM. With Windows 2000, applications can use the AWE API to address more RAM. In Windows 2000 Advanced Server up to 8 GB of RAM can be used, in Windows 2000 Datacenter Server up to 64 GB of RAM can be used. Using AWE in SQL Server implies the following: SQL Server no longer dynamically manages the size of the address space. All memory is acquired at startup of SQL Server until shut down. Memory pages using AWE will not be paged out.

When the parameter awe enabled is used, SQL Server memory must be set to a fix size. Additionally the set working set size must be disabled, or the possibility exists that some network problems may occur when under high stress. Windows must be enabled to run AWE by the boot.ini startup parameter /pae.

In order to enable AWE use by SQL Server, SQL Server must be run by an account with the permission "Lock Page in Memory". Additionally, there must be at least 3GB of free memory available on the computer or SQL Server will not run in AWE mode.

When the awe enabled parameter has been set a message is recorded in the SQL Server errorlog "Address Windowing Extension enabled".

The SQL Server parameter set working set size is used to reserve physical memory space for SQL Server that is equal to the server memory setting. Setting set working set size to 1 means that Windows will not swap out SQL Server pages. If the values of max and min server memory are different, then the set working set size value should be set to 0. Even though set working set size can be set = 1 when memory has been fixed it is still recommended to set it equal to zero.

SQL Server parameters are set using stored procedure sp_configure. See Books Online for more details.


© SAP AG 2003

CPU Utilization Analysis

1

2

3

Transaction ST04 also shows the CPU usage of SQL Server. The header section of the ST04 main screen displays the number of CPUs installed in the computer where SQL Server is running, as well as the number of CPUs that can be used by SQL Server (1). Available CPUs are determined using a bit mask given in the SQL Server parameter affinity mask.

In the section Server Engine, the percentage of CPU seconds used by SQL Server is displayed in the field CPU busy (2). The field CPU idle (3) displays the percentage of seconds where the CPUs that were available for SQL Server were not used. The value IO busy, which is part of CPU busy, shows the percentage of seconds that was used for IO operations issued by SQL Server.

To display absolute values, press button Absolute values. Note: Each CPU counts separately, that is, if 3 CPUs are used in 2 seconds, CPU busy = 6. If 1 CPU was used and 2 were unused in 2 seconds, CPU idle = 4.

Usually, the numbers displayed have accumulated since SQL Server startup. These numbers are not very informative as the large amount of idle time at night covers up potential bottlenecks during peak times. Therefore, you should use the Reset and Since Reset functions to get the times during a representative time frame. The amount of elapsed time since the last Reset is also displayed. Initially and after a Refresh, this shows how much time has elapsed since SQL Server started.

The value for CPU busy (2) should always be approximately < 70%, that is, CPU idle > CPU busy / 2.

To check the history of the CPU utilization, use Transaction ST04 and choose Detail analysis menu → Performance database.


© SAP AG 2003

Poor SQL statements?

Cache hit ratio> 98%?

2 * CPU idle> CPU busy?

OS paging? All CPUs availablefor SQL Server?

Tune poorstatements

Increase SQL Server

memoryand set

memory mode

Increase server main

memoryAdd CPU(s)

to server

IncreaseCPUs for

SQL Server

Yes Yes

YesYes

No No

NoNo

?

Yes

No

Cache and CPU Tuning

+

Before tuning memory or CPU for SQL Server, you should make sure that poor SQL statements are already tuned since they can significantly affect memory or CPU utilization. (How to detect and tune poor SQL statements is discussed later in this unit).

Before increasing SQL Server memory, you must check if there is sufficient main memory available. Operating system paging is a sign of insufficient main memory; especially on the database server. Call Transaction ST06 and choose Detail analysis menu → Previous hours → Memory. The amount of page in per hour should not exceed 20% of the available physical memory. For optimal performance, the value should be 0.

The SQL Server memory parameters should be set as described in SAP note 327494. Before increasing the number of CPUs on the server, check the usage of the CPUs for other applications that could be moved to other servers (for example, SAP work processes). Also check the CPU utilization history of the Windows processes. Use transaction ST06, choose Detail analysis menu, then Top CPU processes. Alternatively, use the Windows Task-Manager.


© SAP AG 2003

Other SQL Server Parameter Settings

1show advanced options0user connections0user options

0set working set size0recovery interval (min)1priority boost0open objects8192network packet size (B)0min server memory (MB)1024min memory per query (KB)28media retention255max worker thread2147483647max server memory (MB)1max degree of parallelism0locks0lightweight pooling0index create memory (KB)0fill factor (%)0default language5cost threshold for parallelism0c2 audit mode0awe enabled0allow updates0affinity maskValueParameter name

SQL Server configuration parameters can be checked using transaction ST04 → Detail Analysis Menu → SQL Server Parameters.

A value 0 for SQL Server configuration parameters locks, open objects, index create memory (KB), and user connections indicates that memory is dynamically allocated for these objects as required. The amount of memory available for the data cache decreases if the number of these objects increases.

SQL Server can execute queries using more than one CPU. This is called parallel execution. SQL Server executes queries using only one CPU until the estimated execution time is below the value (in seconds) of Cost threshold for parallelism. A value of 0 for cost threshold for parallelism means that no queries are executed in parallel. This parameter is ignored if only 1 CPU is available to SQL Server or if max degree of parallelism is set to 1. The value max degree of parallelism shows the number of CPUs SQL Server can use for parallel statement execution (0 = all available CPUs, 1 = no parallel executions).

See Books Online for a detailed description on all SQL Server configuration parameters and SAP note 327494 for current recommendations.


© SAP AG 2003

min. 1 GB min. 20 GB

<SID>data files

SQL Servertempdb

paging file(s) <SID>Transaction log

1-4 GB min. 300 MB

Suggested disk requirements

Optimal Disk Layout (Data Separation)

Disk I/O is the most time-consuming operation for the database system. Therefore, you can significantly improve performance by: Using fast disks and I/O controllers Physically separating files with high I/O activity so that they can operate concurrently

Maximum throughput can be achieved by placing the following four types of files on separate physical disks (or disk systems/controllers): paging file(s) SQL Server tempdb file(s) <SID> transaction log file(s) <SID> data files

To check the hardware resources refer to the SAP Installation Guide found under http://services.sap.com/instguides. Above the minimum requirements for a small SAP system installation (application server and database server) are given. Depending on the amount of data involved, the requirements will change.


© SAP AG 2003

RAID 5

RAID 0

RAID 1

RAID 0+1(= RAID 10)

RAID Technology

RAID technology provides two basic functions: Higher I/O performance by striping data over multiple physical disks, and therefore achieving an even I/O distribution over all disks

Fault tolerance by mirroring disks or adding parity information RAID levels 2, 3, and 4 are not as efficient and are not commonly used. For more information about these levels, refer to SQL Server Books Online.

RAID 0 is called disk striping. All read/write operations are split into slices (usually 16-128 KB) that are spread across all disks in the RAID array. There is no redundancy added. Read and write performance is improved by equally distributing workload on all participating disks. This level provides the highest performance of all RAID levels.

In RAID 1, every physical disk is mirrored. All write operations are written to the original disk and to the disk mirror. Write performance may be a little lower as both disks’ writes must be synchronized. With mirroring, one disk may fail without data loss.

In RAID 0+1 (also called RAID 10), every disk in the disk stripe set is mirrored. It provides nearly the same performance as RAID 0 but adds redundancy at the cost of doubling the number of disks.

RAID 5 combines disk striping with parity. One physical disk is added to hold the parity information. The parity information is also striped along all disks. Every write operation needs 2 physical reads and 2 physical writes (read data + parity and write new data + new parity). It offers less fault tolerance, as only one disk in the whole array may fail without data loss. The total disk storage is not available to store data. There needs to be enough free space available for parity information to equal the size of the smallest disk in the array.


© SAP AG 2003

RAID 0+1

RAID 0

(RAID 0+1 forhigh availability)

RAID 0

(RAID 0+1 forhigh availability)

RAID 5

<SID>data files

(or RAID 0+1)

Optimal Disk Layout (Using RAID Technology)

min. 1 GB min. 20 GB1-4 GB min. 300 MB

Suggested disk requirements

SQL Servertempdb

paging file(s)

<SID>Transaction log

<SID>Transaction log

The paging files and the SQL Server tempdb files are temporary data files that do not require redundancy to ensure data security. For high availability, RAID 0+1 should be used. Since these files are write-intensive, they should not be placed on RAID 5 systems. If they are small enough to fit on one disk, RAID technology is not required.

Transaction log files must be placed on hardware mirrored disks (RAID 1) to ensure data security. If more than one disk is used for transaction log files, additional striping should be used (RAID 0+1).

Data files must be placed in a RAID 5 system to ensure data security (RAID 0+1 would be faster but more expensive).


© SAP AG 2003

12

Analyzing Disk I/O Problems: SQL Server I/O Statistics

1 > 101

SQL Server collects an I/O statistic for the data and log files for all the databases in the SQL Server. You can use these I/O statistics to compare the activity for each data file, and to determine whether activity is evenly distributed over the data files.

The I/O statistics are included in the SAP SQL Server database monitor ST04 → Detail Analysis menu → IO per File. You can choose between tempdb and the SAP database.

The column 'ms/IO' displays the average wait time for one operation in ms and is calculated from the output of SQL Server system function fn_virtualfilestats. For acceptable performance 'ms/IO' should be below 10ms for all data files (1). For the transaction log files the average wait time is much smaller than for data files (2).

fn_virtualfilestats counts the statistics since SQL Server start. For more information, see SQL Server Books Online and SAP note 521750.


© SAP AG 2003

Disk queue length up to

45 >> 4(2 disks)

100% Disk time

Analyzing Disk I/O Problems: Performance Monitor

Disk I/O performance can be measured using the Performance Monitor (from Start, choose Settings → Control Panel Administrative Tools → Performance Monitor). Note: Windows does not show the physical disks in the RAID system. Therefore, you first need to divide the performance counters by the number of physical disks. Then, for RAID 1 (and RAID 0+1), multiply the number of I/O write requests by 2. For RAID 5, multiply the writes by 4 to get the number of physical disk I/O requests.

When ”% Disk Time = 100” you must check I/O performance. The disk queue length (read + write queue) per physical disk should not significantly exceed 2, otherwise there is an I/O bottleneck.

To get disk performance data, enable the disk performance object using command diskperf -y (a Windows reboot is required afterwards).

For long term monitoring, use the Performance Monitor to create a log file with snapshot data from the specified update intervals.

For more information about using the Performance Monitor, see SAP note 110529 and Windows Online Help.


© SAP AG 2003

Slow RAIDidentified?

Avg. disk queuelength > 2 * phys.

Disks in RAID

Check all logical Disks with thePerformance

Monitor

?

No

Yes

Change hardwareconfiguration

I/O System Tuning

SQL Server I/O statsshow avg. wait time for

one disk > 10 ms

Yes

Yes

To find the peak throughput for a whole RAID system or the whole I/O bus, measure disk I/O performance on all physical disks.

If the SQL Server I/O statistics show an average wait time of more than 10 ms during peak workload, check the I/O system using the Windows performance monitor. The average disk queue length per physical disk should not significantly exceed 2.

If you detect an I/O bottleneck, ask your hardware partner to check your hardware configuration.


© SAP AG 2003

Disk layout

Parameter settings

Data cache

Hardware configuration

Main memory

CPU

Disks


Poor configuration

Cache hit ratio

SQL Server CPU utilization

High I/O times

Operating system paging CPU

utilization

Disk response

times> 98%

2 * idle > busy(SQL Server)

Select 1 row via prim. key< 10 ms 2 * idle > busy

(total)

Wait queueand low

transfer rate

Page in< 100 MB / h

Database Configuration: Summary


© SAP AG 2003

LockwaitsPoorly

qualifiedstatements

Poor configuration

Poor databaseconfiguration



Too manyunnecessarystatements


Application Problems

Application problems can be classified into three groups: Lockwaits

These are caused by long running transactions that are holding locks, which are blocking other applications that want to get the same locks.

Too many unnecessary statements Poor coding causes many small statements to be read in a loop or results in data being read twice.

Poorly qualified statements, where... Insufficient selection criteria is given The database optimizer has no efficient access path to the requested data (for example, in the case of missing indexes) The database optimizer does not find the best access path (due to an inappropriate execution plan)


© SAP AG 2003

If many users are waiting for a specific lock, they may block all other users.

Frontend users

SAP Application Server

SAP Work processes

Database

datarow

Exclusive Lockwaits

In this example, a large number of SAP user requests are channeled into a smaller number of SAP work processes on the application server.

A user holding a lock occupies an SAP work process. Other users trying to apply the same lock will have to wait and at the same time occupy their own SAP work process.

As the number of lockwaits increases, fewer and fewer SAP user requests can be processed by the available SAP work processes.

In the worst case (lockwaits = number of SAP work processes), a small number of users can cause the entire SAP System to freeze.


© SAP AG 2003

Time

AcquiresMARA Lock

A long periodof processing CommitUpdate

MARA

Working...WAITING!Update MARA

RequestsMARA Lock

AcquiresMARA Lock Commit

Working...WAITING!Update MARA

RequestsMARA Lock

AcquiresMARA Lock

WAITING ...Update MARA

RequestsMARA Lock

Lockedby:

WP 1 WP 2 WP 3

Lockwait Situations

MARA

SAPWork-

Process 2

SAPWork-

Process 3

SAPWork-

Process 4

SAPWork-

Process 3

When updates are programmed, the time that the program holds a lock should be kept as short as possible.

This diagram illustrates how the wait time to acquire a lock increases when several work processes are waiting to acquire a lock on the same object, and a long time is needed to process the update.

In order to update table MARA, work process 1 acquires a lock. Then, work process 2 requests a lock on the same object, but has to wait for work process 1 to release its lock. Work process 1 takes a long time processing before it performs a commit and releases the lock.

While work process 2 is waiting to acquire the lock, a lock request from work process 3 is pending. Work process 3 has to wait until work process 2 performs a commit. Work process 3 has to wait even longer than work process 2 because it has to wait for work process 1 and 2 to complete their commit.


© SAP AG 2003

Monitoring Lockwaits (1)

ANJAEARLYW

EARLYW

ANJA

SAPPRODSAPPROD

SAPPRODSAPPRODSAPPROD

SAPPROD

SAPPROD

SAPPROD

Exclusive lockwait situations due to database locks are shown in the SAP Database Monitor (Call Transaction ST04 and choose Detail analysis menu → Exclusive Lockwaits).

The Host PID is the process ID of an SAP work process. The owner of the lock is shown in a blue line and holds the status granted shown by a green signal in the first column. All blocked processes are shown below and have the status waiting indicated by the red signal.


© SAP AG 2003

SQL Server Enterprise Manager - [Console Root\Microsoft SQL Servers\SQL Server Group\207.105.67.113 [Windows NT]\Management\Cu... x_Console Window Help x_Action View Tools

Console RootMicrosoft SQL Servers

SQL Server Group

SAPPROD [Windows NT]Databases

51 Items

Data Transformation Services

ManagementSQL Server AgentBackupCurrent Activity - 14.12.2002 11:35:10

Process InfoLocks / Process IDLocks / Object

Database Maintenance PlansSQL Server Logs

SecuritySupport Services

spid 17 spid 18 spid 19 spid 20 spid 21 spid 22

spid 23 spid 24 spid 25 spid 26 spid 27 spid 109[blocked by 179]

spid 29

spid 50 spid 52 spid 53 spid 55 spid 56

spid 57 spid 58 spid 59 spid 60 spid 61 spid 62 spid 63

spid 64 spid 65 spid 66 spid 67 spid 68 spid 69 spid 70

spid 178

spid 16

Monitoring Lockwaits (2)

Replication

spid 1 spid 10 spid 11 spid 13 spid 14 spid 15

spid 177[blocking]

spid 179[blocked by 177]



You can also view the current lockwait situation using the Enterprise Manager. On the database server, choose Management → Current Activity → Locks/Process ID. All the SQL process IDs are displayed.

Blocking is written under the icons representing the SQL process IDs that have a blocking lock holder.

Blocked By <n> is written under the icons that have a lock waiter that is waiting for a lock held by a SQL process ID. Blocked SQL processes are additionally marked with a red square in their icon.

The head of a locking chain is marked with a red exclamation mark (spid 177 in the above example).


© SAP AG 2003

Database

RDBMSData Cache

Procedure Cache

MemoryArea

SAP Work Process

Tables Indexes Procedure PlansY2R0000064B69RC5522MARA

SQL Connections

WHILE ...SELECT * FROM MARAWHERE ...

ENDWHILE.

SQL Trace

Too Many Statements

MARAMARA_1

MARA_0

In this example, the same SELECT statement may be executed several times. If the conditions in the WHERE clause do not change, the same data is read several times from the database.

To find this kind of duplication, you can run an SQL Trace (Transaction ST05). This logs the communication between the SAP work processes and the database for a particular group of SAP users.


© SAP AG 2003

SQL Trace

SELECT WHERE „SPRSL“ = @P000 AND „ARBGB“ = @P001 /* R3:



OPENFETCHOPENFETCHOPENFETCH

To start, stop, and display the SQL Trace, call Transaction ST05 or choose Tools → Administration → Monitor → Traces → Performance Trace.

To see the time stamps and program names, select button Extended List. All statements that were sent from the SAP work processes to the database are displayed. Identical statements are probably caused by one statement being executed repeatedly. To directly access the corresponding ABAP program line, choose button Display Call Position in ABAP Program.

The column Records shows the number of read or manipulated rows. The column Return code (RC) shows the result value of the database operation. The duration (execution time) of the statement is shown in column Duration. The time is specified as: seconds.milliseconds.microseconds

The column Statement shows the statement executed on the database which is either the contents of the stored procedure or a direct statement. In case the content of a stored procedure is shown, the name of the stored procedure is appended to the statement.


© SAP AG 2003

Poorly Qualified Statements (1)

SELECT WHERE „MANDT“ = @P000 AND „BRGEW“ = @P001 /* R3:PREPAREOPENFETCHFETCHFETCHFETCH

0

0

MARAMARAMARAMARAMARAMARA

139.271147.068

2.809145290

1.794

11

High execution time + returning few records

The runtime (duration) of the statement is highlighted in red if it exceeds a given threshold value of 100.000 microseconds. If a statement returns few rows and has a long execution time, the statement must be improved.

An inefficient statement can have several consequences: The database is kept busy processing many data blocks CPU load is increased on the database server An SAP work process is blocked by the report and there are high wait times for other processes Several pages are displaced from the database buffer The cache hit rate for other SQL statements suffers Performance can be harmed system-wide by expensive statements


© SAP AG 2003

Database

RDBMSData cache

MemoryArea

SAP Work Process

Tables IndexesMARA

MARA_1

MARA_0

SQL connections

SELECT * FROM MARAWHERE MANDT = 001

AND BISMT IN (...)

Optimizer

Explain:Execution path

Used indexTable definition

...

Statistics

Poorly Qualified Statements (2)

The Database Optimizer is a cost-based optimizer that minimizes the number of pages to be read. It estimates the costs for each index used and compares these to the cost of a table scan for each table to be accessed.

For this cost estimation, the Optimizer uses statistical information about the data distribution, which is stored for each index in a statistics page.

There are some common reasons for long execution times: No index exists and therefore the whole table is read sequentially in a full table scan The index used is not very selective, so a large range has to be read in an index range scan The Optimizer re-uses the execution plan which keeps an unsuitable strategy The statement is badly formulated and selects much more data than is actually necessary


© SAP AG 2003

Information from SAP Data Dictionary:• Table fields• Indexes and index fields

Getting More Information

Jump to ABAP source:• Show ABAP statement

Explanation of Query:• Which index is used• Access method used• JOIN sequence

To determine why a statement runs slowly, you can request more details from the SQL Trace. Place the cursor on the desired line and choose: Button Display Details to show: - The complete statement - The name of the stored procedure used if a permanent stored procedure was created - If a database cursor was used for the execution - The parameter values and types used

Button DDIC Information to show: - The table structure according to the SAP Data Dictionary (field definitions) - The indexes defined on the table(s) used, as written in the SAP Data Dictionary

Button Explain to show: - The Optimizer decisions (for the time of the explanation). That is, the indexes used and the

JOIN sequence Button Display Call Position in ABAP Program to show:

The related ABAP statement (if executed from ABAP)


© SAP AG 2003

Strategy chosen by the Optimizer

Table and index name

Column values used for Seek

Filter conditions applied afterwards

Explain tree nodes

Understanding Explain SQL

The output of the optimizer explanation consists of the complete statement followed by the explain tree. In the explain tree, the root node shows the statement type (for example, SELECT). All other nodes are shown as PLAN_ROW. Except the root node, each node represents a working step necessary for the statement execution. The execution works from the bottom to the top, to produce the results for the statement. Everything belonging to the same node has the same indentation level.

Every PLAN_ROW node shows the corresponding step below the estimated execution costs for processing. It also shows the estimated number of rows returned and the estimated cost for I/O and CPU resources (these numbers have a virtual unit and are only used for comparing different execution plans).

Above the PLAN_ROW identifier, you can see the strategy used for this step (for example, Filter and Clustered Index Seek) followed by the parameters for the strategy (for example, filter values, index names, index fields, operators, and values).

In this example, two steps are required to process the SELECT statement (2 * PLAN_ROW): First, a Clustered Index Seek on index MARA~0 of table MARA is used at the leaf level to search the index field MANDT for the value ”000”. The output is sorted by the clustered index key. The estimated number of rows returned from this step is 211.

The results of this step are then fed into the next step, which filters the rows with the condition BISMT = ”2” OR BISMT = ”1”. The estimated number of rows returned by this step is 1. This is also the result of the whole statement (parent node is the SELECT).


© SAP AG 2003

2

7

Getting Table and Index Information

12

11

6 5 14 13

To display detailed information on a table and its indexes, use Transaction DB02 and choose Detail analysis, then enter the table name.

Total rows (1) displays the current number of rows in the table. Row modification counter (2) shows the number of changed rows since the last update statistics. An update statistics operation resets this counter to 0.

If Auto stats (3) is set and the database option Auto update statistics is also set, SQL Server automatically performs an update statistics when a certain percentage of the table rows have been changed (according to the row modification counter).

Index depth (4) shows the longest path from the root page to a leaf page of the index. A clustered index always has one level less than the same index created as a nonclustered index because the data pages are stored as the lowest level of the index. The index height gives the number of pages to be read for an index seek operation in the index. If a clustered index exists, the accesses through the nonclustered indexes are followed by a clustered index access, because the cluster key is stored in the nonclustered indexes instead of the row ID.

Last update statistics (5) displays the timestamp of the most recent update statistics operation. Index columns (6) displays the list of the index fields. If the database option Auto create statistics is set, column statistics (7) are created automatically by SQL Server. These automatically created histograms have a name starting with _WA_Sys_.


© SAP AG 2003

Histogram of the first index field

Total number of rows in table

Date of last update for these statistics

Density of given columns combination

Index Statistics

To display the statistical information stored in the SAP System for each index, call Transaction DB02 and choose Button Detailed analysis, then enter the table name. In the screen displayed, select the requested index and choose button Show statistics.

Only a contiguous list of columns of each index is usable for a selection where the first column is included. The index shown here can only be used if JOBNAME or JOBNAME and JOBCOUNT are in the WHERE clause. For JOBCOUNT alone it is not useful. The All density value is calculated by dividing 1 by the number of different values or value combinations for the field or fields. The index is best if the combined useable columns have a very low density and only a few duplicates exist for a given combination.

The section below shows a list of samples taken for the values of the first index column (histogram). The total number of samples taken is displayed in column Steps in the top line. The histogram provides statistical information about the frequency of any single value in the table. Statistics are comprised in the following information: RANGE_HI_KEY: Upper bound value of a histogram range (step). RANGE_ROWS: Number of rows from the sample that fall within the range, excluding the upper bound.

EQ_ROWS: Number of rows from the sample that are equal in value to the upper bound of the histogram step.

DISTINCT_RANGE_ROWS: Number of distinct values within a range, excluding the upper bound.

AVG_RANGE_ROWS: Average number of duplicate values within a histogram step, excluding the upper bound (RANGE_ROWS / DISTINCT_RANGE_ROWS for DISTINCT_RANGE_ROWS > 0).

The value RANGE_ROWS is equal to AVG_RANGE_ROWS multiplied by DISTINCT_RANGE_ROWS.


© SAP AG 2003

SAP WorkProcess

SQL connections

exec proc

Database systemSAP InstanceStored procedurename cache

Y2R10000000BB1H1911DD03LY2A000001D9C7VK3725RSCROSS1Y2E000011F2CBC50301REPOLOADY2R65000168C73K0943ALPERFDBY2C00000453CBC21832DBSYNSEQY2A0000032FCAUK4623SAPLSEUwY2R10000000C5GG0516TBTCPY2K00002B2DCBC21130DOKCLUi5

Stats info

Sp stats

Switch On / Off

Readstats

Finding Expensive Statements

The stored procedure name cache of each SAP instance provides some additional space for storing statistical information about the execution of stored procedures, such as the: Duration of the slowest and average execution (in ms) Number of executions (since the statistics were turned on) Number of rows returned during the fastest, slowest, and average running execution Parameter values for the execution that returned the maximum number of rows Name of the stored procedure

To view the stored procedure statistics, call Transaction ST04 and choose Detail analysis menu → SAP stats on SPs. By default, the stored procedure statistics is switched on (SAP instance profile parameter dbs/mss/stats_on = 1). The statistics can be displayed for a single application server or for all the servers. You can check the contents of a specific stored procedure or display a list of the selected stored procedures.


© SAP AG 2003

Stored Procedure Name Cache Statistics

Two times are measured for every stored procedure execution; the duration without fetches and the duration with fetches, which may be significantly higher if a larger number of rows are returned.

The number of pages read or written is not displayed as this information is not known at the SAP System level.

To check which procedures are responsible for the most database load, sort by column ∑ ms(+fetch). The top procedures are either slow or executed very often.

To find the slow statements that are targets for optimization, sort by column ∑ Max ms(+fetch). Statements used for changes, such as UPDATE, DELETE, INSERT, and SELECT FOR UPDATE, may have a high maximum time due to lockwait situations. Also, the execution times may vary because of different parameter values. Therefore, you should sort by column Avg. ms (+fetch). The top statements are the statements that are slow on every execution.

You can select a line and choose Button SQL statement to get the procedure text (the statements executed within the stored procedure) and the parameter list of the call that returned the largest number of rows (if available). You can then use the function Explain SQL to display a new explain plan with actual parameters. You can also check the Explain SP of the precompiled procedure in the SQL Server procedure cache to see if the stored execution plan is identical.


© SAP AG 2003

Poorstatement

DDICinfo

SQLExplain

Whereused list

Create or change secondary index

in DDIC + DB

Updatestatistics

Re-code

Yes

Yes

YesNo

No

No

Re-code byspecifying anoptimizer hint

Yes

Statistics page

Switch onauto updstats

Yes

No

How to Tune an Expensive SQL Statement

Inefficientcoding?

Is there asuitableindex?

GoodOptimizerdecision?

Index statistics

up to date?

Autoupdatestats on?

In this example, we assume that no suitable index exists and that the fields in the WHERE clause are MANDT and BISMT: SELECT * FROM MARA WHERE MANDT = xxx AND BISMT = yyy

If you were to tune this expensive statement, you might consider creating a secondary index. However, creating a secondary index does not solve all problems, in fact it may have its own consequences. For example, if you increase the number of indexes on a table, the duration of INSERTS, DELETES and UPDATES also increases.

Not every field has to be contained in the index. If you were to use the Explain SQL function for this example, you could see that the clustered index on MANDT and MATNR was used, but it was not very efficient. This means that the MANDT field was not helpful and should not have been selected for the index. Since there was only one row returned, the BISMT field is probably the selective one.

Now you must determine how to create a useful index.


© SAP AG 2003

A good indexIs used in many statementsIs used oftenContains only selective fieldsIs small and possibly covered

How to Create a Useful Index

A good index should be used in many statements. If an index is used often, the database request time and the database load are reduced. A selective field is a field that has:

Many different values in the real data in the table A small number of data rows with identical values, compared to the total number of rows within the table

If an index is small, many index rows fit on one index page. This is especially useful for index row scans that may be triggered, for example, by comparison operators.

A covered index contains all selected fields plus all the fields in the WHERE clause. It prevents accesses to the data pages.


© SAP AG 2003

Determine Selective Field Combinations

184239184239100%

100% selectivity is fine

To obtain the current selectivity of each combination of columns for a table, call Transaction DB02 and choose Detail analysis → Table name → Selectivity. You can select any columns that can be indexed, other than image or text fields.

From the screen displayed you can check the: Number of distinct entries for your selection Total number of rows in the table Selectivity (in percent)

If you enter a combination of fields, you cannot tell if only one of the given fields is selective or if the combination gives a high selectivity. Therefore, you should always check the sub selections.

A more accurate (but more costly) way to get the selectivity of field combinations is to use the density function. The result is weighted according to the number of occurrences for each distinct value.


© SAP AG 2003

Poorstatement

DDICinfo

SQLExplain

Whereused list



Inefficientcoding?


in DDIC + DB

Updatestatistics

Re-code

Yes

Yes

YesNo

Index statistics

up to date?

No

No


Yes

Statistics page

Autoupdatestats on?


Yes

No

How to Tune an Expensive SQL Statement (2)

In this example, we assume that a suitable index exists and that the optimizer has chosen this index, as shown by SQL Explain.

This means you must check the coding to find the poor statement. It is possible to jump to the ABAP coding directly from the Stored Procedure Name Cache.


© SAP AG 2003

SELECT * FROM zlips WHERE werks = int_lips-werksAND lgort = int_lips-lgort.

CHECK zlips-matnr IN sl_matnr....

ENDSELECT.

SELECT * FROM zlips WHERE werks = int_lips-werksAND lgort = int_lips-lgortAND matnr IN sl_matnr.

...ENDSELECT.

Example of an Expensive SELECT (in ABAP)

Good

Poor

In the first example, many data rows will be read unnecessarily from the database and filtered subsequently on the SAP System level.

The second example already gives all filter conditions to the database and therefore returns only the rows really needed to the SAP System. Finding the data on the database can also be faster because an additional field can be used in an index seek.


© SAP AG 2003

How to tune an expensive SQL statement (3)

PoorStatement

SQLExplain

Whereused list



Inefficientcoding?


in DDIC + DB

Updatestatistics

Re-code

Yes

Yes

YesNo

Index statistics

up to date?

No

No


Yes

Statistics page

Autoupdatestats on?


Yes

No

DDICInfo

In this example, the optimizer chooses an index which was appropriate when the statement was executed with different parameters. In this case the optimizer has to be notified to recompile the statement or to choose a specific index. This can be done by giving optimizer hints.


© SAP AG 2003

Recompilation and Optimizer Hints

RDBMS Memoryarea

SAP Work Process

SQL connections

SELECT * FROM ZMARAWHERE MANDT = 001

Optimizer

Procedure cache

execution plan:clustered index seekSAP Work

Process

SELECT * FROM ZMARAWHERE MANDT = 002

Execution plan is reused.High execution time because of inappropriate strategy

The SQL Server determines the execution plan with the first calling of a stored procedure. All further calls use the same execution plan until the stored procedure is compiled again (for example by the automatic update statistics). When first executing a stored procedure, the query optimizer decides to read the data following a certain strategy. When executing the stored procedure a second time, the strategy might not be appropriate anymore because the stored procedure uses different values now. A recompilation of the stored procedure forces the optimizer to create a new execution plan using a different strategy.

Since R/3 4.6A, optimizer hints in the ABAP for SQL Server (see SAP Notes 129385 and 133381) have been introduced. Appending the following hint to a SELECT in the ABAP has the effect that the SQL Server creates a new execution plan every time the SELECT is executed:

%_hints mssqlnt '&REPARSE&' This makes sense when the selectivity of an individual or a few SELECTs is extremely different.


© SAP AG 2003

An expensive SQL statement needs many reads and can cause system-wide performance problemsAnalyzing an expensive SQL statement

Statistical records → high database request time

SQL Trace → statements with long response times

Stored procedure name cache statistics → high execution times

Process overview → report and table name

Where used list → table

Tuning an expensive SQL statementCreate/change an index

Check if Auto update statistics is on

Understand the DB Optimizer

Rewrite poor coding

Use optimizer hints

Expensive SQL Statements: Summary

An expensive SQL statement needs many reads and can cause system-wide performance problems. When you analyze an expensive SQL statement, you must check the:

Statistical records for the high database request time SQL Trace for the statements with long response times Stored procedure name cache statistics for high execution times Process overview for the report and table name Where used list for the table

To tune an expensive SQL statement, SAP recommends that you: Create or change an index Use Automatic update statistics (DB option + index option) and Auto create statistics (DB option) Understand the DB Optimizer Rewrite poor coding Use optimizer hints


© SAP AG 2003

LockwaitsPoorly

qualifiedstatements

Poor configuration

Poor databaseconfiguration



Exclusivelockwaits

Procedure stats, SQL Trace

Procedure stats, SQL Trace

Too manyunnecessarystatements


Application Problems: Summary

Application problems typically involve either lock contention or inefficient SQL coding. SQL problems are normally caused by statements that are being executed more frequently than necessary or individual statements that are very expensive.

A statement can be very expensive because of poor WHERE clauses, missing indexes, or poor optimizer decisions due to outdated statistics.

If you determine that both the database configuration and the application have been adequately tuned and you still have performance problems, SAP recommends that you consult your hardware partner about additional hardware. This course does not cover a hardware analysis.


© SAP AG 2003


Analyze the performance of your SQL Server database system running with the SAP SystemIdentify the source of database performance problemsImprove database performance



© SAP AG 2003

Books Online

SAP Installation Guide

SAP Database Administration with Microsoft SQL Server 2000SAP Press, ISBN: 1-59229-005-1

SAP Notes

Windows Online Help



© SAP AG 2003

Unit Actions

Exercises?

Solutions


Exercises

Unit: Performance Monitoring and Tuning Topic: Application Problems


• Work with the SQL trace tool (transaction ST05)

• Detect an expensive SQL statement, analyze its cause and find an appropriate solution

The customer detects a slow running program and decides to trace the program using the SQL trace tool (transaction ST05). He further detects a slow running statement, which he analyzes and tunes.

1-1 SQL Trace

1-1-1 Report ZADM520MARA1 selects a record from table ZADM520MARA, which is a copy of master material table MARA. Call ST05 to turn on the trace, execute report ZADM520MARA1 (in transaction SE38). In ST05, turn off the trace, and display the trace results. Look at the access times to table ZADM520MARA. Check for the slow statement for the OPEN operation. Which index was used for the slow statement?

1-1-2 Apply the appropriate change to speed up the execution. Repeat the steps above and check the results.


Solutions

Unit: Performance Monitoring and Tuning Topic: Application Problems

1-1 SQL Trace

1-1-1 Display the trace by choosing button Display Trace in transaction ST05. Position your cursor on the slow statement, and choose Explain. The clustered index ZADM520MARA~0 is used. SEEK was only performed for field MANDT. For BISMT, the program read the entire table. Performance could be improved by creating a secondary index on column BISMT. Use transaction SE11. Display table ZADM520MARA. Choose Button Indexes .., and enter an ID for the new index. Trace the program again and check if the new index is used.


© SAP AG 2003

Database Backup

1 Introduction

6 Database Restore



5 Database Backup4 Performance Monitoring and Tuning



© SAP AG 2003

ContentsBackup TypesPerforming BackupsVerifying BackupsBackup Strategies


Describe and perform the different backup types which are supported by SQL Server using the SAP Planning Calendar and SQL Server toolsExplain the technical implementations of the described backup types and name the involved system tablesVerify the backupsDefine a backup strategy that is suitable for your company

Database Backup


© SAP AG 2003

To prevent data loss, a valid backup is necessary

Data loss

Logical errors(Such as a corruption)

External factors(Such as fire or water

damage)

Physical errors(Such as a media failure)

User errors(Such as a deleted table)

Importance of Database Backups

External factors, physical errors, and logical errors can all lead to data loss and system downtime. An effective backup strategy and recovery plan is essential in minimizing system downtime and data loss.

To ensure the availability of your SAP System, the backup strategy developed by your database administrator must be carefully tested.


© SAP AG 2003

Database Backup

Transaction Log Backup

Differential Backup

Complete System Backup

<SID>

<SID>

<SID>

Backup Types

msdbmaster

Before a database backup and restore strategy can be defined, the different backup types of SQL Server are explained and described in detail.

SQL Server provides different backup types, which are supported by SAP: Database backup Transaction log backup Differential backup

In addition to the above backup types SQL Server provides filegroup and file backups, which are not used in the SAP environment.

In addition to backups performed by SQL Server, the Windows operating system provides a backup tool to save files in the file system on a backup medium.


© SAP AG 2003

Copy all used data pages to the backup media

DataAndre Vasser Hil Frenzen

X0 X1 X2 X3 X4 X5 X6 X7

Kojak MagnumDerrick MarpleX8 X9 X10 X11 X12 X13 X14 X15

Landis Wolf Wang Kerber Kania Thomas MerdesX16 X17 X18 X19 X20 X21 X22 X23

Mozart Bach Strauss Wagner Beeth.X24 X25 X26 X27 X28 X29 X30 X31

11

Copy all used log pages to the backup media

Set the timestamp of the backup to the time when the backup has finished

Transaction Logbegin update begin insert commit chkp insert rollbackLSN0 LSN1 LSN2 LSN3 LSN4 LSN5 LSN6 LSN7

deleteLSN8 LSN9 LSN10 LSN11 LSN12 LSN13 LSN14 LSN15

LSN16 LSN17 LSN18 LSN19 LSN20 LSN21 LSN22 LSN23


22

33

Database Backup

In a database backup, all user-defined objects, system tables and data are copied to a backup medium. The database backup is done while the database is online and thus has a slight performance impact.

Therefore the database should be backed up at times when the workload is minimal. In addition to backing up the data pages, the transaction log is stored. Restoring the database from this backup means that the status of the time when the backup of the database ended is reached. This backup includes transactions that were performed during the backup. However, some operations are not allowed during a database backup: Creating or deleting database files Shrinking either the database (automatically or manually) or the database files

The transaction log is not truncated when you perform a full database backup.


© SAP AG 2003


begin1 update begin2 insert commit2 chkp insert begin3LSN0 LSN1 LSN2 LSN3 LSN4 LSN5 LSN6 LSN7

delete dump commit1 chkp insert insert delete deleteLSN8 LSN9 LSN10 LSN11 LSN12 LSN13 LSN14 LSN15

insertLSN16 LSN17 LSN18 LSN19 LSN20 LSN21 LSN22 LSN23


11

Transaction Log

Truncate the inactive portion of the transaction log



insertLSN16 LSN17 LSN18 LSN19 LSN20 LSN21 LSN22 LSN23


22

Transaction Log

Transaction Log Backup

When the transaction log is backed up, all used pages from the transaction log are backed up (1). Next the transaction log is truncated, that is, the inactive part is deleted (2).

By creating transaction log backups, a database can be restored to any point in time contained within the sequence of transaction logs, right up to the point of failure. When creating a transaction log backup, the starting point of the backup is where the previous transaction log backup ended (if a transaction log backup has been created). In this example with the log sequence number 0, because no transaction log backup has been created since the full database backup. A subsequent transaction log backup would start with log sequence number 17. Although part of the transaction log is backed up with a database backup this is not taken into account for the transaction log backup.

The transaction log cannot be backed up: If the recovery model is set to simple. Read more about the recovery models in the following slides.

Remember that a transaction log backup can only be created after a preceding full database or a differential backup.


© SAP AG 2003

Transaction Log

Start of logical log

unused

End of logical log

Virtual Log 1 Virtual Log 2 Virtual Log 3 Virtual Log 4

MinLSN

Determine the MinLSN before theTransaction Log is truncated

11

Transaction Log

Start of logical log

unused

End of logical log

Virtual Log 1 Virtual Log 2 Virtual Log 3 Virtual Log 4

22Truncate the log before the start of the Virtual Log containing the MinLSN

Truncating the Transaction Log

Virtual Log 5

Virtual Log 5

If log records were never deleted from the transaction log, the log would keep growing until it filled all the available space on the disks holding the log files. Therefore old log records that are no longer needed for recovering or restoring a database must be deleted to make space for new log records. The process of deleting these log records is called truncating the log.

The active part of the transaction log can never be truncated. The active part of the log is needed to recover the database at any time. It must always be present in the database. In case the server fails the database must be recovered when the server is restarted. The record at the start of the active portion of the log is identified by the minimum recovery log sequence number (MinLSN). See also Unit 1: SQL Server Architecture.

Transaction logs are divided internally into sections called virtual log files. Virtual log files are the unit of truncation. When a transaction log is truncated, all log records before the start of the virtual log file containing the MinLSN are deleted.

Truncating does not mean physically deleting. It simply marks the virtual log file as inactive. When the unused space at the end of the last virtual log is used up, the logging of the transactions starts again at the beginning of the first virtual log.


© SAP AG 2003

Recovery Model

Transaction Log

Transaction Log

BCM

303

Begin Tran2

Insert MARA ….

304

Create indexPage x

Tran1

305

Update A002 set ...

Tran2

306

Commit

Tran1

307

Delete A001Where ...

Tran2

308

Checkpoint

302

Begin Tran1

Create indexZMARA_1 ...

301

InsertMARD …

Tran3

Full

Bulked_logged

303

Begin Tran2

Insert MARA ….

304 305

Update A002 set ...

Tran2

306

Commit

Tran1

307


Tran2

308

Checkpoint

302

Begin Tran1


301

InsertMARD …

Tran3

303

Begin Tran2

Insert MARA ….

304 305

Update A002 set ...

Tran2

306

Commit

Tran1

307


Tran2

308

Checkpoint

302

Begin Tran1


301

InsertMARD …

Tran3

Simple

Extent x

Transaction Log

In SQL Server you can use predefined recovery model to support planning your backup and recovery strategy. Recovery Models offer different levels of system performance and data security requirements.

SQL Server provides the following three recovery models: Full: Full Recovery provides the ability to recover the database to the point of failure or to a specific point in time. All operations, including bulk operations such as CREATE INDEX, and bulk loading data, are fully logged in the transaction log. Every page of the newly created index tree gets written into the transaction log. When creating a clustered index the data pages will also be written into the log. The transaction log is truncated only by a transaction log backup.

Bulk logged: The Bulk Logged Recovery model provides protection against media failure combined with the best performance and minimal log space usage. Extents changed by bulk load operations such as CREATE INDEX are marked in the Bulk Changed Map (BCM), but not logged. All other database operations are fully logged. When the transaction log is backed up, both the transaction log data and the marked extents are backed up. Log backups generated can only be completely restored again.

Simple: Simple Recovery requires the least administration. In this model, data is recoverable only to the most recent full database or differential backup. Transaction log backups are not used, and minimal transaction log space is used. Bulk inserts and the creation of new indexes are not logged. A log truncation occurs every time a checkpoint is processed.

You can set the recovery models for each database using the SQL Server Enterprise Manager or the ALTER DATABASE statement. See Books Online for more details.

Check to see which model is active, using the DATABASEPROPERTYEX function, the SAP Database Allocation Monitor (transaction code DB02) or the SQL Server Enterprise Manager. In a productive SAP System, it is recommended that the Full Recovery Model is set for the SAP database. Only in exceptional cases, the recovery model full can be changed to bulk logged.


© SAP AG 2003

Data

Copy all modified extents since the last full database backup to the backup media

22


Set the timestamp of the backup to the time when the backup has finished

begin update begin insert commit chkp insert rollbackLSN0 LSN1 LSN2 LSN3 LSN4 LSN5 LSN6 LSN7

deleteLSN8 LSN9 LSN10 LSN11 LSN12 LSN13 LSN14 LSN15



33

44

Differential Database Backup

Read the DCM pages to determinewhich extents have been changedsince the last full database backup

11DCMDCMDCM

Transaction Log

Transaction Log

Andre Vasser Schu X0 X1 X2 X3 X4 X5 X6

Kojak Rex Derrick MarpleX7 X8 X9 X10 X11 X12 X13

Dilg Wolf Wang Kerber Kania ThomX14 X15 X16 X17 X18 X19 X20

A database backup can be supplemented by differential database backups that record only the changes made to the database after the last full database backup. Each subsequent differential database backup records all the changes made to a database after the last full database backup. If a database backup is created, followed by multiple differential database backups, only the database backup and the last differential database backup created must be restored.

Differential backups are used primarily in heavily used systems where a failed database must be brought up as quickly as possible. Differential backups are smaller than full database backups, so a restore from a differential backup takes less time, because only the extents that contain modifications are restored. These extents are recorded in the Differential Changed Map (DCM), which has been introduced in Unit 1: SQL Server Architecture.

During a differential backup, SQL Server backs up the transaction log to produce a consistent database when the database is restored.


© SAP AG 2003

Directory FilesX:\<SID>DATA1 Primary data fileX:\<SID>DATA2 Secondary data fileX:\<SID>DATA3 Secondary data file

Y:\<SID>LOG1 Transaction log file

Z:\Tempdb data and log files of the tempdb

C:\Program Files\Microsoft SQL Server\8.0\COM application programming interfaces Tools tools such as Books Online

C:\Program Files\Microsoft SQL Server\MSSQL\Backup Default Backup directoryBinn MS SQL Server executablesData System and sample database filesInstall Installation scripts and logsJobs Temporary job output filesLog Errorlogs and JoblogsRepldata Working directory for replication tasksTrace Trace files created by the SQLProfilerUpgrade Files used for upgrade

D:\usr\sap\<SID> SAP executablestrans Transport directory

D:\WINNT Windows System directory

Copy all SAP and SQL Server files to the backup media11

Create a document containingthe file structure22

File System Structure

File System Backup

Perform a complete system backup of the SAP file tree, the operating system files, as well as the database data files, the transaction log files, the SQL Server executables and system database files onto the backup media.

To ensure that the correct actions are performed in case of a system crash, you should create a document containing the file structure of the whole system. This document must be available and understood by the person who performs the database restore.


© SAP AG 2003

regular

Mon Tue Wed Thu Fri Sat

Tue Wed Thu Fri Sat Sun

Mon Tue Wed Thu Fri Sat Sun

Mon

SunTue


Tue

unplanned

MonthlyBackupCycle

SAP Planning Calendar DB13 or DB13C

How to Perform a Backup

Enterprise Manager

Query Analyzer

There are three ways to perform database backups : Using the SAP Planning Calendar (transaction code DB13) or the central SAP Planning Calendar (transaction code DB13C)

On the database level, using the Enterprise Manager tool, select: Tools → Backup Database. On the database level, using the transact-SQL statement BACKUP.

Other backup tools can also be used, but are not covered in this course. Course ADM100 provides a description of how to use the SAP Planning Calendar. Do NOT use Windows Backup to perform backups of the SAP database. Windows Backup should be used to perform a complete offline system backup.


© SAP AG 2003

Backup Devices

Backup on local or network disk device

Parallel backup onmore than one tape

Single backup on tapeR3DUMP0

R3DUMP0R3DUMP1R3DUMP2

Backup with third partysoftware using named pipes

Third party software

WindowsBackup

TAPE

TAPE

DISK

PIPE

Database X

You can use several different backup devices for a SQL Server backup: One or more tape backup devices (for example, devices R3DUMP0, R3DUMP1, …; created during the installation of the SAP System)

Disk device (local or network) Named pipe device, used by third party software

For small SAP databases (up to around 60 GB), backups should be performed on a single tape device, for example a DLT tape.

Larger SAP databases can be backed up using one device and several tapes, written sequentially. If more than one tape device is available (directly connected to the computer), tapes can be written in parallel onto these devices. A parallel backup is recommended if the time frame is small.

Backup to tape, whether to a single tape, or to multiple tapes using the sequential or parallel backup procedure, is supported by the SAP Planning Calendar (Transaction DB13).

To manage the backup of the SAP database into a disk backup device, use SQL Server Backup. Afterwards, use Windows Backup to back up the disk dump files onto a backup media. For this type of database backup, use Windows Backup to restore the database backup file on disk, then use the database restore facility with the backup file. The advantage of this method is that the backup to disk is quite fast and another computer may then be used to perform the Windows Backup.

Third party software can also be used to perform backups. Data is transferred using named pipes. Third party software can also be used to backup to tape devices located on a different computer than the one where SQL Server is installed.


© SAP AG 2003

Mon 19 Wed 21 Thu 22 Fri 23 Sat 24

Tue 27 Wed 28 Thu 29 Fri 30 Sat 31 Sun 01

Mon 02 Tue 03 Wed 04 Thu 05 Fri 06 Sat 07 Sun 08

Sun 25Tue 20

success

0:00 DB 9:00 Log

Mon 12 Wed 14 Thu 15 Fri 16 Sat 17 Sun 18Tue 13success 0:00

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log

success success 2:00 error

Mon 26

success success success

success

Job Scheduling Engine,SQLServerAgent

Logs (msdb Tables, SAP Tables)

SAP Layer

Database Layer

DB Interface

SAP Planning Calendar - Concept

success

When creating a backup job using the SAP Planning Calendar (transaction code DB13 or DB13C) the database interface creates a SQL command. This command defines and schedules a job using the SQL Servers Job Scheduling Engine. These jobs call stored procedures sap_backup_databases, sap_backup_diff_databases and sap_backup_log to back up the databases and the transaction log.

A job created by the SAP Planning Calendar can be defined to run immediately or on a recurring schedule, for example, once a week. Once you have created a job, you can view the job definition by double-clicking the job name or by choosing button Parameters. If the requirements of a task change, you can modify the task by double-clicking its name and choosing Edit. If the task is no longer needed, simply delete it by choosing button Delete.

Each scheduled job to be executed by SQL Server Agent is stored in table sysjobs on database msdb. Their names start with SAP CCMS and can be viewed using the SQL Enterprise Manager.


© SAP AG 2003

CCMS Planning Calendar

Planning Goto Listing Help System

MON TUE WEDWED THUTHU FRIFRI SATSAT SUNSUN



MON TUETUE WEDWED THUTHU FRIFRI SATSAT SUNSUN

Scheduling Backups

New Plan for Tue 17.12.-Start time 18:00:00 Period: 1 Action

Full database backup Differential database backup Transaction log backup Check database consistency

Database to be dumped

mastermsdbPRD

Database(s) selection

Select Backup devices

R3DUMP0 TapeR3DUMP1 TapeR3DUMP2 Tape

Backup device selectionBACKUP

Transactionlog

BACKUP

Transactionlog

BACKUP

Data files

BACKUP

Data files

The SAP Planning Calendar supports the following backup types out of the SAP System: Full database backup Differential database backup Transaction log backup

These actions can be scheduled to run on their own; once or repeatedly. Alternatively, they can be scheduled to run in combination with other tasks by selecting a predefined Planning pattern. A Planning pattern specifies a combination of tasks that are executed at predefined intervals and can be scheduled together. The Calendar offers the Planning pattern: 5 workdays, DB backup + hourly log backup (8am - 6pm) + Monthly Check DB

To schedule the pattern, you simply need to choose it and specify a time. The SAP System automatically assigns values to the parameters required for the execution.

To schedule a single task double-click the day on which you want a backup scheduled. In the selection screens, mark: Backup type required Database(s) you want to back up One (or more) backup device

For detailed information on how to use the SAP Planning Calendar, see the SAP Online Help.


© SAP AG 2003

Media Names

DatabaseDatabase

R = SAP DBR = SAP DB

M = msdbM = msdb

S = masterS = master

I = combination SAP DB includedI = combination SAP DB included

TypeType

D = databaseD = database

L = trans. logL = trans. log

TapecountTapecount

Device namesDevice names

NumberNumber

01 - 31 = day of the month

01 - 31 = day of the month+ = differential+ = differential P = parallelP = parallel

S = sequentialS = sequential

C = combination SAP DB not includedC = combination SAP DB not included

The media on which the backup is performed is identified by a label. This label identifies the contents by indicating the type and day of the backup and is given automatically when scheduling backups using the SAP Planning Calendar. The media name is put together as follows:

The first character shows the type of data on the media: R SAP database S master Database M msdb database I More than one database is backed up on the tape and the SAP database is included C More than one database is backed up on the tape and the SAP database is not included

The second character shows the type of backup: D Full database backup, L Transaction log backup + Differential database backup

The next two characters (numbers) show the day of the month. The last character on the label indicates whether it is a parallel or a sequential backup


© SAP AG 2003

Media Names (2)

ID15S = One tape of a sequential SAP, master, and msdb (I) database backup (D) on day 15 of the month

RD05P = One tape of a parallel (P) SAP (R) database backup (D) on day 5 of the month

ID15S

Stick name labelon tape cartridge

Name all tapes used for backups. Type in a tape name when creating an SQL Server backup task and stick a label with the same name on the tape medium.

The first tape label in this example has the name ID15S. Following the naming convention outlined, previously, this label denotes an SAP database, master, and msdb full database backup, performed on the 15th day of the month.

The second example is RD05P. This is an SAP database backup on the tape. The backup was performed on day 05 of the month. The character P indicates a parallel backup was performed.


© SAP AG 2003

Database and Log Backup Options

For the last Backup onlyFor the first Backup only

BackupDatabase

BackupTransactionLog

Log

msdb

master

<SID>

For a full database backup, use the following options: Unload tape Rewinds the tape after the backup, set this option for a database backup Initialize tape Existing backups are overwritten, if the expiration period is over, and the tape name corresponds to the day the backup is performed.

Verify backup Checks backup is to see if all data on tape is readable Expiration period Set to 27 days

For transaction log backups, set the following options as indicated: Unload tape Set for the last transaction-log backup of the day stored on this tape Initialize tape Set for the first transaction-log backup on the tape when the tape is reused Verify backup Set for all backups Expiration period Set to 27 days

When using the option Format tape, the media name and all data on the tape are overwritten, no matter what the media retention or expiration period. Use Format tape only when new tapes are used.

The option Initialize tape allows you to overwrite backups on a tape if the expiration period is past. The media name is not overwritten and can be used for identifying the tape.

The option Verify executes transact-SQL command RESTORE VERIFYONLY on the chosen backup medium after having performed the backup. We recommend that you set this option for each backup performed. It checks that the backup set is complete and that all volumes are readable. However, RESTORE VERIFYONLY does not attempt to verify the structure of the data contained in the backup volumes. If the backup is valid, SQL Server returns the message: ”The backup set is valid.” This message will be displayed in the pop-up window after double-clicking on the task. If it is not valid, you should repeat the backup on a new medium.


© SAP AG 2003

SQL Server Backup - PRD

General Options

x

Database:Name:Description:

PRD

PRD full database backup

PRD Backup Database

Backup

Destination

Overwrite

Schedule

Database - CompleteDatabase - DifferentialTransaction logFile and filegroup:

Backup To: Tape Disk

\\ \Tape0Add...

Remove...

Contents...

Append to mediaOverwrite existing media

Schedule:

OK Cancel Help

SQL Server Backup - PRD x

General Options

Backup

Verifying the backup will read the entire backup and check formedia integrity. Checking the identity and expiration of themedia prevents accidental overwrites.

Verify backup upon completionEject tape after backup

Media Set Labels:Initializing tape or disk media set erases the previous contentsof the media and labels the media set with a name anddescription.

Remove inactive entries from transaction logCheck media set name and backup set expiration

Media set name:

Remove inactive entries from transaction logRemove inactive entries from transaction logCheck media set name and backup set expirationCheck media set name and backup set expiration

Media set name:Media set name:

Backup set will expire:

RD25S

After:On:

daysFri 01/24/2003

Initialize and label mediaMedia set name:Media set description

RD25S

OK Cancel Help

27

Database Backup with Enterprise Manager

To back up the database using the Enterprise Manager, choose Tools → Backup Database. Select the database to back up, and select Database - Complete. Select the destination and specify the option to overwrite the existing media.

In the Option folder, different options can be set. When the backup is completed, you should verify the media integrity of the backup. An expiration period must be set to prevent the media to be overwritten within 27 days. A media name must be specified. Use the SAP naming conventions explained earlier in this chapter.

You can execute the backup immediately, or you can schedule it for execution by choosing Schedule and specifying the appropriate information in the Schedule Backup dialog box that is displayed.

The database can be backed up using the Transact SQL command BACKUP DATABASE which is described in detail in the SQL Server Books Online.


© SAP AG 2003

SQL Server Backup - PRD

General Options

x

Database:Name:Description:

PRD

PRD transaction log backup

PRD Transaction LogBackup

Destination

Overwrite

Schedule

Database - CompleteDatabase - DifferentialTransaction logFile and filegroup:

Backup To: Tape Disk

\\ \Tape0Add...

Remove...

Contents...

Append to mediaOverwrite existing media

Schedule:

OK Cancel Help

SQL Server Backup - PRD x

General Options

Backup

Verifying the backup will read the entire backup and check formedia integrity. Checking the identity and expiration of themedia prevents accidental overwrites.

Verify backup upon completionEject tape after backup

Media Set Labels:Initializing tape or disk media set erases the previous contentsof the media and labels the media set with a name anddescription.

Remove inactive entries from transaction logCheck media set name and backup set expiration

Media set name:

Remove inactive entries from transaction logRemove inactive entries from transaction logCheck media set name and backup set expirationCheck media set name and backup set expiration

Media set name:Media set name:

Backup set will expire:

RL25S

After:On:

daysFri 01/24/2003

Initialize and label mediaMedia set name:Media set description

RL25S

OK Cancel Help

27

Log Backup with Enterprise Manager

To back up the transaction log using the Enterprise Manager, choose Tools → Backup Database. Select the database you want to back up, then select Transaction log. Select the destination. Specify whether you want to overwrite an existing medium or whether you want to append the backup to the backup sets already on the medium.

In the Option folder different options can be set. When the backup is completed, you should verify the media integrity of the backup. An expiration period must be set to prevent the media from being overwritten within 27 days. A media name must be specified. The inactive portion of the transaction log should be removed. Use the SAP naming conventions explained earlier in this chapter.

You can execute the backup immediately, or you can schedule it for execution by choosing Schedule and specifying the appropriate information in the Schedule Backup dialog box that is displayed.

The database can be backed up using the Transact SQL command BACKUP LOG which is described in detail in the SQL Server Books Online.


© SAP AG 2003

Tape Options

When the SQLServerAgent service is started, it queries the system tables in the msdb database to determine what jobs to start. If a backup job is scheduled, SQL Server Agent will pass the backup command to SQL Server. If the tape device is not ready at the specified time, SQL Server will normally cancel the job and write an error to the error log. The following timeout values are available to overcome such a problem: Wait indefinitely: Wait indefinitely for SQL Server to respond. Try once then quit: Try once and then time out when waiting for SQL Server to respond. Try for minute(s): Specify the time (in minutes) to try before timing out when waiting for SQL Server to respond.

We recommend trying for 10 minutes and then send a timeout message.


© SAP AG 2003

File System Backup using the Windows Backup Tool

Distributed Transaction .. Coordinates Transac… Manual Local SystemMSSQLSERVER Automatic PRDadmMSSQLServerAdHelper Manual Local SystemSQLSERVERAGENT Automatic PRDadmSAPPRD_00 Automatic SAPServicePRDSAPOSCOL Automatic SAPServicePRD

The entire system, including all Windows, SAP and SQL Server files must be backed up regularly using the Windows Backup utility. The backup of these files is necessary for a restore operation when the disk on which SQL Server and SAP executables are located crashes. In addition, it serves as an additional backup that may play a vital role in dealing with emergency situations where other routine backups have been damaged. Choose Start → Programs → Accessories → System Tools → Backup to open the Windows Backup tool. For detailed information see the Windows Online Help.

Windows, SAP and SQL Server files need to be backed up: After installing a Windows or SQL Server Service pack Before special actions such as an SAP Upgrade

Windows Backup Tool only backs up closed files. Therefore the services MSSQLServer, SQLSERVERAGENT, SAP<SID>_<InstanceNo> and the SAPOSCol must be stopped.

To ensure that the whole system is backed up, you should install an additional Windows system (Help Windows), which is only used when backing up the system. For information on how to proceed, refer to the Windows documentation.You must ensure that you do not back up the local registry. If an additional Windows is not available, you must also back up the local registry.

Never use Windows backup for regular database backups that are part of a predefined backup and restore strategy. When you restore such a Windows backup, the database and transaction log files are restored to the state they were at the time you ran the Windows Backup. No subsequent differential or log backups can be applied. Therefore you can not do a point-in-time database recovery. This means that the changes you made to the database since the last Windows Backup are lost and cannot be redone. Instead, always use SQL Server-based backups for regular database and log backups.


© SAP AG 2003

Backup time

Restore time

High availability

Training

Acquisition costs

Administrative workload

Long Short

No Yes

Long Short

Short Long

Low High

Low High

?

?

?

?

?

?

Backup Requirements and Costs

Requirements. Depending on your situation, you may need one, or a combination of several, advanced backup scenarios. When you define your backup scenario, you must consider: How long it takes to perform a backup The maximum time required for a complete database restore To what extent your backup strategy provides additional security in case of a hardware failure

Costs. Your backup scenario will also depend on: How much training the administrator requires The amount of database administration required for implementation and production operation The acquisition costs

Now that you are familiar with the different backup types supported by SAP and Microsoft tools, we can introduce various advanced backup scenarios. The following slides introduce various advanced backup scenarios supported by SAP tools, ranked according to the above criteria using a ”smiley matrix.” This ranking is intended as a rough guideline only.


© SAP AG 2003

DB13: Full daily Database Backupof master, msdb and <SID> to one tape

ID01S RL01S

<SID>

Database Backup Transaction Log Backup

<SID><SID>

DB13: Transaction Log Backup of <SID> to one tape each hour

Single DB and Transaction Log Backups

Training

Backup timeRestore timeHigh availability


Acquisition costs

msdbmaster

<SID>

This scenario is the reference point for ranking the scenarios. A full database backup of the SAP database should be performed daily on a permanent storage media, for example, a data tape. This backup type records the complete state of the data in the database at the time the backup operation completes. The master database and the msdb database must also be backed up daily.

The transaction log of the SAP database must also be backed up in order to restore the database to a specific point in time. The transaction log contains information on database changes, which is used for roll forward and roll back operations. SAP recommends scheduling log backups every 30-60 minutes. The number of transaction log backups depends on the size of the transaction log (normally 1.5 GB), and the transaction rate of your system.

Transaction log backups are much smaller in size than database backups, as they only contain the changes since the last transaction log backup. Therefore, a transaction log backup has a low impact on system performance, if performed during normal operation.

A transaction log must be backed up before the transaction log file is full. If the transaction log is full, no further database transactions can be performed. See Unit 6: Regular Maintenance and Error Handling on a solution on how to handle a full transaction log. After the transaction log backup, all log information which is not active is truncated out of the transaction log file.

The databases and transaction log backups must not be written to the same backup device. SAP recommends this strategy because it meets the needs of most customers.


© SAP AG 2003

DB13: Full daily Database Backupof master, msdb and <SID> to more than one tape

Database Backup Transaction Log Backup

ID01P

ID01P

ID01P

RL01S

RL01S

DB13: Transaction Log Backup of <SID> to more than one tape each hour

Parallel Tape Support

Training



Acquisition costs

msdbmaster

<SID>

<SID><SID>

<SID>

To reduce the backup time, as well as the time required for restoring the database and the transaction log, transaction DB13 supports the parallel use of multiple devices. This strategy can also be used if all databases or the volume of all transactions logs to be backed up will not fit on one single tape and an automatic or manual device changing is not possible. A disadvantage may be that if one device cannot be read, the whole backup is not restorable. Therefore we recommend that you always verify the structure of the backup on the device as explained in previous slides.


© SAP AG 2003

“Slow” 2nd stepWindows Backup

Fast 1st step DB13: Full daily Database Backupof master, msdb and <SID> to DISK_DEV1

<SID>

DISK_DEV2

DISK_DEV1

<SID><SID>

Fast 1st step DB13: Transaction Log Backup of<SID> to DISK_DEV2 each hour

RL01S ID01S

Two-Step Disk Backup

“Slow” 2nd stepWindows Backup

Training



Acquisition costs

msdbmaster

<SID>

A two-step disk backup is performed as follows: First, a backup is made to disk. Typically, this is faster than a backup to tape. Second, the files are backed up from disk to tape. The advantages of this method are that it: Reduces backup time Reduces restore time (if the required files are still available on disk) Backups can be saved to remote disks, through shared folders

First, additional backup devices on disk must be created for the database backups and the transaction log backups. Enough disk space to hold these backups must be available. Note that the size of a database backup corresponds roughly to the used size of the database because only used pages are copied to the backup media.

Second, the files are copied to tape using Windows Backup or another third-party tool. If the backups on disk are still available when a restore is necessary, the restore time is reduced. If the backups are no longer available, the restore will need to be performed from tape.

The advantage of this strategy is that backups that are performed to a remote disk are physically separated from the server. In case of fire or flood, the backup on the remote disk may still be available.


© SAP AG 2003

Sun Mon Fri

<SID>

DB13: Full weekly Database Backup of master, msdb and <SID> to one tape

ID01S

<SID>

R+02S

DB13: Differential daily Database Backup of <SID> to one tape

RL02S

...

<SID><SID>

DB13: Transaction Log Backupof <SID> to one tapeeach hour

Supplementary Differential Backups

Training



Acquisition costs

msdbmaster

<SID>

Using a full database, differential database, and transaction log backups together can reduce the time needed to restore a database back to any point in time after the database backup was created. Additionally, creating both differential database and transaction log backups can increase the robustness of backup procedures in the event that either a transaction log backup or differential database backup becomes unavailable, for example, due to media failure. Differential backups reduces the runtime of the backup itself because during the backup procedure the DCM page is read to determine which extents have changed.

Typically combined database, differential database, and transaction log backups involve creating database backups at longer intervals, differential database backups at medium intervals, and transaction log backups at shorter intervals. For example, create database backups weekly, differential database backups daily, and transaction log backups hourly.


© SAP AG 2003

Standby serverProduction server

DISK_DEV1

<SID> <SID>

RL02SID02S

<SID><SID>

<SID><SID>

DISK_DEV2

Daily Windows Backup ofDISK_DEV2 to tape

Daily Windows Backup ofDISK_DEV1 to tape

DB13: Full Database Backup of<SID>, master and msdbto tape

Full Database Backupof master, msdb and <SID> to DISK_DEV1 once

DB13: Transaction Log Backup of<SID> to DISK_DEV2 each hour

Hot-Standby Server

Training


Admin. workload

Acquisition costs

msdbmaster

<SID>

msdbmaster

<SID>

A standby server is a second server that can be brought online in the event that the production server fails. The standby server contains a copy of the databases on the production server, including the system databases. This copy is maintained by initially backing up the databases on the production server and restoring them once on the standby server. Periodically, transaction log backups from the databases on the production server are applied to the standby server to ensure that the standby server is synchronized with the production server. If the production server fails, the standby database can be opened, and can take on the role of the production database.

The transaction log backups from the production server are applied on the standby database. This allows a delayed restore on the standby server in case of a possible user error. On the standby server, Windows backups to tape are performed.

NOTE: Certain structural changes on the production database are not automatically replicated on the standby database. In this case, the recovery is stopped, and the database administrator must resolve the situation manually. The creation and deletion of datafiles on the primary server are not automatically replicated on the standby server.

This scenario is highly dependent on the implementation, and you must therefore plan the environment in detail. See Books Online for a detailed description.


© SAP AG 2003

CCMS Database Backup History (DB12)

Database backups

!! ii??

Explorer

Monitoring Backups

Mon Tue Wed Thu Fri Sat

Tue Wed Thu Fri Sat Sun


Mon

SunTue


Tue

MonthlyBackupCycle

SAP Planning Calendar (DB13 or DB13C)

Enterprise Manager

Event Viewer

You must check regularly whether the backup tasks were completed successfully. Always check that: The most recent backup has run successfully All the backups in the backup cycle are being executed according to the schedule. Gaps in a backup sequence can have serious consequences in a subsequent attempt to restore the database.

The backup and restore process works successfully and enables you to restore your database to a correct and consistent state.

There are different ways of monitoring single backups of databases and transaction logs: Use the SAP Planning Calendar (transaction DB13 or DB13C) If the SAP System is unavailable, use the Enterprise Manager and the Event Viewer to check backup task execution.

To display the SQL Server error log, use the Explorer or the Enterprise Manager. Using the Explorer, select the drive and the MSSQL\LOG\ path. The current error log is in file ERRORLOG. All recent error logs can be found in files ERRORLOG.1 to ERRORLOG.6. By default SQL Server maintains up to seven error logs. The number of error logs can be configured in the Enterprise Manager.

To display the SQL Server Agent error log, use the Explorer or the Enterprise Manager. Using the Explorer, select the drive and the MSSQL\LOG\ path. The extension of each archived error log indicates the age of the error log. For example, an extension of .1 indicates the newest archived error log.

The backup cycle must be checked using the SAP Planning Calendar (transaction DB13) or the CCMS Database Backup History (transaction DB12).


© SAP AG 2003

Job Scheduling EngineSQL ServerAgent

Database LayerDB Interface

Mon 19 Wed 21 Thu 22 Fri 23 Sat 24

Tue 27 Wed 28 Thu 29 Fri 30

Mon 02 Tue 03 Wed 04 Thu 05 Fri 06

Sun 25Tue 20success

0:00 DB 9:00 Log


0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log


Mon 26


success

SAP Layer

SQL task information************** Task information ******************Jobname: SAP CCMS Log Backup of PRDTaskname: step1Type: TSQlDB-Name: PRDLastrun: 20021118 20000Nextrun: 20021125 20000 Created: Nov 14 20025:18PMCommand: sap_backup_log @r3db="EW1",@

@bdev="R3DUMP0",@expdays=28Description: SAP CCMS JOB

*************Task history information **********For run: 20021118 020000Status: 1 (success)Message: 4029Severity: 10Duration: 0 hours, 56 minutes, 29 secondsLast message: DBCC execution completed. If DBCC printed

error messages, contact your system admin

msdb tables

Monitoring using the SAP Planning Calendar

You can check whether your most recent database and transaction log backups are usable in the SAP Planning Calendar. It is best to check the backup immediately after the backup has finished, before you remove the tape.

You should check whether the transaction log backups are completed successfully on a daily basis. Omitting such a check could have serious consequences. In the event of a hardware crash and a subsequent restore operation, this may result in the loss of several hours of data.

The status of a finished task is marked red if the task failed, or green if the task was completed successfully. To display more information on a completed task, select the task and double-click. Each task not run successfully should be checked immediately.

The primary attributes of a backup job scheduled using the SAP Planning Calendar are: Jobname: The name of the job. Backups scheduled using the SAP Planning Calendar always begin with SAP CCMS [Log | Database ] of <database list> [timestamp]. The timestamp specifies the scheduling time and is always unique.

Job Step: A job step is an action that the job takes on a database or a server. The job step is defined by the transact-SQL statement (TSQL) sap_backup_databases, sap_backup_diff_databases or sap_backup_log.

The task history section returns information on the job and the job step execution. This information is read from msdb table sysjobsteps, which contains the information for each step in a job to be executed. Table sysjobhistory contains information about the execution of scheduled jobs.

If the option verify has been chosen, the runtime of the scheduled is longer even though the backup itself is already finished. The verification of the backup is done after the backup is completed and belongs to the job, but not to the backup.


© SAP AG 2003

Monitoring using the Database Backup History

Checking individual backups is not sufficient to ensure that your backup strategy is being implemented successfully. You should also periodically check all of your database backups together to make sure that your strategy is being adhered to and there are no gaps in the sequence.

In the SAP System, the Database Backup History (transaction DB12) offers an overview of all database backups and all transaction log backups scheduled in the SAP Planning Calendar.

Choose Backup History and then All Backups. A result list appears showing technical details about all backups that have been executed for the last 30 days (default). Select a backup and choose History for additional details.

If a backup failed diagnose exactly what happened. To do so, it is necessary to also look at the SQL Server error logs.

The Backup Device List displays all the defined backup devices.


© SAP AG 2003

Name Enabled Runnable Scheduled Status Last Run Status Next Run Date

SAP CCMS Blocking Lockstats Yes Yes Yes not running succeeded (10/30/20 10/30/2002 10:05:00 AMSAP CCMS Full DB Backup of PRD, master Yes Yes Yes not running failed 11/03/1998 10:00:00 PMSAP CCMS Full DB Backup of PRD, master Yes Yes Yes not running succeeded (10/28/20 11/04/2002 10:00:00 PMSAP CCMS Full DB Backup of PRD, master Yes Yes Yes not running succeeded (10/29/20 11/05/2002 10:00:00 PMSAP CCMS Full DB Backup of PRD, master Yes Yes Yes not running succeeded (10/30/20 11/06/2002 10:00:00 PMSAP CCMS Full DB Backup of PRD, master Yes Yes Yes not running succeeded (10/31/20 11/07/2002 10:00:00 PMSAP CCMS Log Backup of PRD Yes Yes Yes not running failed 11/03/2002 10:00:00 AMSAP CCMS Log Backup of PRD Yes Yes Yes not running succeeded (10/27/20 11/03/2002 11:00:00 AMSAP CCMS Log Backup of PRD Yes Yes Yes not running succeeded (10/27/20 11/03/2002 12:00:00 AMSAP CCMS Log Backup of PRD Yes Yes Yes not running succeeded (10/27/20 11/03/2002 01:00:00 PMSAP CCMS Log Backup of PRD Yes Yes Yes running failed 11/03/2002 02:00:00 PMSAP CCMS Log Backup of PRD Yes Yes Yes not running Unknown 11/03/2002 03:00:00 PMSAP CCMS Log Backup of PRD Yes Yes Yes not running succeeded (10/27/20 11/03/2002 04:00:00 PM

X

X

X

Verification Using SQL Enterprise Manager

To display the job history in the Enterprise Manager, select the folder Management, SQL Server Agent, item Jobs.

All the jobs scheduled from the SAP System begin with 'SAP‘. For example, 'SAP CCMS DB backup of <Databases>'. Jobs whose status is ‘failed' should be analyzed in detail. Double-click the job to retrieve more information. Do NOT use the Enterprise Manager to change the attributes of jobs coming from the SAP System. They should always be modified using the SAP Planning Calendar.

SQL Server keeps the history data of backups and jobs in system tables in database msdb. Some system tables were mentioned in previous slides. Tables that contain information about backups include: backupfile: contains one row for each database and log backup made backupset: contains one row for each backup set


© SAP AG 2003

Backup Errors - Additional Information

If a backup error occurs, you can find additional information in the Event Viewer and in the database error log. Further notification in case of a failure of the scheduled job can be defined using the Enterprise Manager.

Select Start → Settings → Control Panel → Administrative Tools → Event Viewer to open the Event Viewer.


© SAP AG 2003


Describe and perform the different backup types which are supported by SQL Server Explain the technical implementations of the described backup types and name the involved system tablesDefine a backup strategy that is suitable for your company and implement the scenarioVerify the backups



© SAP AG 2003

ADM100 mySAP Technology Administration

SQL Server Books Online

SAP Online Help

Windows Online Help



© SAP AG 2003

Exercises?

Solutions

Unit Actions


Exercises

Unit: Database Backup Topic: Performing Backups


• Perform and verify database and transaction log backups using the SAP Planning Calendar, the SQL Server Enterprise Manager and transact-SQL commands

The customer chooses a suitable backup strategy that meets the requirements for his company and implements the backup strategy using the SAP Planning Calendar and SQL Server tools.

1-1 Back up the database using the SAP Calendar. Perform a full database backup using the SAP Planning Calendar, as described in Unit 4: Database Backup.

1-1-1 Call the SAP Planning Calendar. Schedule a full database backup of all three databases recurring weekly at 10.00 p.m. to backup device R3DUMP4. Which options do you have to set?

1-1-2 Schedule a Transaction Log Backup for the next day at 12.00 p.m. to backup device R3DUMP5. Which options do you have to set?

1-1-3 Check the parameters of your scheduled tasks.

1-2 Back up the database using the SQL Enterprise Manager. Perform an immediate full database backup of databases test and master and backup the transaction log of database test using the SQL Enterprise Manager.

1-2-1 Open the SQL Enterprise Manager. On your local SQL Server perform a scheduled immediate full database backup of databases test and master to backup devices R3DUMP1 and R3DUMP2. What are the naming conventions for the media set name, assuming that test is the SAP database? What other options do you set?

1-2-2 Perform a scheduled immediate backup of the transaction log of database test to R3DUMP3. What options will you set? What are the naming conventions for the volume labels assuming that test is the SAP database?

1-2-3 Check the backups.


1-3 Back up the database using the Query Analyzer. Make an immediate database backup of msdb using Query Analyzer commands. If necessary, check Books Online on the correct syntax on the BACKUP command.

1-3-1 Open a Query Analyzer window. Enter the correct transact-SQL command to back up database msdb to backup devices R3DUMP1 and R3DUMP2 and execute it. Append the backup to the existing backups of databases test and master on the two backup devices.

1-3-2 Back up the transaction log of msdb to R3DUMP5 using the correct transact-SQL command. Are you able to perform a backup? If not, what error message is displayed?


Solutions

Unit: Database Backup Topic: Performing Backups

1-1 Back up the database using the SAP Calendar. Perform a full database backup using the SAP Planning Calendar, as described in Unit 4: Database Backup.

1-1-1 Call the SAP Planning Calendar under Tools → CCMS → DB Administration → Planning Calendar → DB13 – Local (transaction code DB13). Double-click today’s date. In the dialog box, enter Database Backup and the correct time (10:00 p.m.). In the next dialog box, databases master, msdb and DEV must be selected. In the Backup Device Selection dialog box select backup device R3DUMP4. Set the expiration period to 27 days. You need to set the following options: initialize, unload tape, and verify.

1-1-2 Call the SAP Planning Calendar under Tools → CCMS → DB Administration → Planning Calendar → DB13 – Local (transaction code DB13). Double-click tomorrow’s date. In the dialog box, enter Transaction Log Backup and the correct time (12:00 p.m.). In the Backup Device Selection dialog box, select backup device R3DUMP4. Set the expiration period to 27 days. Also set nounload, initialize device, and verify since it is the first backup you are performing on R3DUMP5.

1-1-3 To check the parameters, select the scheduled task and choose Parameters.

1-2 Back up the database using the SQL Enterprise Manager. Perform an immediate full database backup of databases test and master and backup the transaction log of database test using the SQL Enterprise Manager.

1-2-1 Open the SQL Enterprise Manager. Choose Tools → Backup Database. Select the database (test) from the database list, type a name and a description, and select Database - complete. In the destination list, select both backup devices R3DUMP1 and R3DUMP2. If only one backup device is displayed in the list, add a second device by choosing button Add. Choose radio button Overwrite existing media. Choose Schedule. In the Edit Schedule window select One Time to run the backup immediately. Choose OK when you have specified all the information for the backup.


Under tab Options, set flag Initialize and label media and enter the media set name (ID<dd>P, where <dd> is the day of the month.) and set the backup to expire after 27 days. Do the same for database master, but make sure that you set the Append to media option. This ensures that the backup will be appended to the devices. In the Option folder, you can check the media set name and expiration period.

1-2-2 Open the SQL Enterprise Manager. Choose Tools → Backup Database. Select the database (test) and Transaction Log. Select backup device R3DUMP3. Schedule the backup for immediate execution and specify a name for the task. Under tab Options, select Initialize and label media and Expires after 27 days. Remove inactive entries from the transaction log. The volume name for the test transaction log backup is RL<dd>S, where <dd> stands for the day of the month.

1-2-3 Check the run status, content and volume label of the scheduled backups by choosing Management → SQL Server Agent → Jobs. If a backup was completed successfully, the SQL Server error log will show a message similar to this: Database backed up: Database: test, creation date(time): 2003/02/18(18:28:24), pages dumped: 125, first LSN: 7:164:1, last LSN: 7:168:1, number of dump devices: 2, device information: (FILE=1, TYPE=DISK, MEDIANAME='ID18P': {'R3DUMP1', 'R3DUMP2'}).

1-3 Back up the database using the Query Analyzer. Make an immediate database backup of msdb using Query Analyzer commands. If necessary, check Books Online on the correct syntax on the BACKUP command.

1-3-1 In the Enterprise Manager, choose Tools → SQL Server Query Analyzer. Alternatively you can directly open a Query Analyzer window. Issue the following command: BACKUP DATABASE msdb to R3DUMP1, R3DUMP2 WITH NOINIT, MEDIANAME = ‘ID<dd>P’ where MEDIANAME is the name used in exercise 1-2-1. If the backup is successful, the following message will be returned: Processed 1456 pages for database 'msdb', file 'MSDBData' on file 3. Processed 1 pages for database 'msdb', file 'MSDBLog' on file 3. BACKUP DATABASE successfully processed 1457 pages in 8.370 seconds (1.425 MB/sec).


1-3-2 Issue the following command: BACKUP LOG msdb TO R3DUMP5 It returns with message: Server: Msg 4208, Level 16, State 1, Line 1 The statement BACKUP LOG is not allowed while the recovery model is SIMPLE. Use BACKUP DATABASE or change the recovery model using ALTER DATABASE. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP LOG is terminating abnormally.


© SAP AG 2003

Database Restore

1 Introduction





5 Database Backup

6 Database Restore


© SAP AG 2003

ContentsRestore strategy and scenariosRestore methodsChecking restore


Perform a database restore using SQL Server toolsDescribe the different restore methodsCheck the restore

Database Restore


© SAP AG 2003



damage)


Database restore without loss of data

Restoring the database to a specific point in time

Database restore in a specific time window(for various scenarios)

Document your procedures and escalation plans

Procedure andescalation plan

Goals of a Database Restore Strategy


If data is lost due to external factors, such as fire or water damage to your hardware, or physical errors, such as a hardware failure, you must restore the database up to the point in time when the database crashed. If a full restore is possible and the backup strategy is set up properly, only data from uncommitted transactions before the error will be lost.

If data is lost due to logical errors, such as an unintentionally deleted table, you must restore the database up to a point in time shortly before the error occurred.

To ensure that the correct actions are performed for each of the different scenarios, create a document containing organizational descriptions of procedures and an escalation plan. This document must be available and understood by the person who performs the database restore.


© SAP AG 2003

AvailabilityONLINETarget

Actual

OFFLINE

Replace hardwareand set up system

Problemanalysis

Applytransaction

logs

Automaticrecovery

Time

Restoredatabase

Principles of a Database Restore

Applydifferentialbackups

An appropriate backup and restore strategy ensures minimum down time when the SAP System becomes unavailable due to an error. SAP recommends that you plan these strategies in advance and test them in your test system.

When you plan your backup strategy, consider how long you can afford to shut down the SAP System in different scenarios. To calculate the total time required for a full database restore, include the following times required for: Analyzing the error Replacing the required hardware, and setting up the operating system and file systems Restoring the database from the database backup If differential backups were made, apply the last differential database backup Performing a forward recovery from the backed up transaction logs Performing an automatic recovery at server startup

The maximum downtime of a productive SAP System is the time from when the error is noted to the time when the administrator finishes checking the restored system.

The database is restored as follows: The content of the database backup is copied from the backup medium to the database (Restore). If differential backups were made, the last differential database backup is copied. All transaction log files are imported from the backup medium to the database. The automatic recovery rolls back uncommitted transactions and rolls forward committed transactions.


© SAP AG 2003

Step 1User 1

User 2

Begin Tran1

Begin Tran2

x1

x2

Data Cache

Step 2User 1

User 2

Commit Tran1

Action written totransaction log

dirty

SQL Server goes down due to power outage before checkpoint occurs Step 3

dirty

x1

x2

<SID>LOG1<SID>DATA1

Begin Trans1Begin Trans2Trans1: updateTrans2: insertCommit Trans1

SQL Server

SQL Server

Automatic Recovery (1)

SQL Server stops processing for example, if an operator restarts the server while users are connected and working in databases or in case of a sudden power outage. An automatic recovery is then carried out when SQL Server is restarted. This ensures that all transactions committed before system failure are physically entered in the database because the corresponding modified data pages might not have been flushed to the data files yet. All other partially completed and thus uncommitted transactions are rolled back. This principle is explained in the example shown.

Two transactions, TRAN1 and TRAN2, are started by two different users. The transactions are recorded in the transaction log as BEGIN TRAN1 and BEGIN TRAN2. Both transactions change pages in the data cache, and these pages are marked dirty.

In the second step, user 1 ends the transaction with a COMMIT TRAN1. SQL Server writes all log information on Transaction TRAN1 in the transaction log. COMMIT confirms that all the affected log records in the cache are written to the transaction log.

A checkpoint causes SQL Server to write the data pages marked dirty to the database. In this example however, SQL Server and the Windows Server go down before the checkpoint, due a power outage.


© SAP AG 2003

Step 4

Automatic recovery

Roll backWritten TRAN1

to database <SID>DATA1

Begin Trans1Begin Trans2Trans1: updateTrans2: insertCommit Trans1

Date Source Message

03/01/21 08:54:07.43 spid1 Recovering of database PRD (7) is 0% complete (approximately 21 more seconds)03/01/21 08:54:07.44 spid1 Recovering of database PRD (7) is 100% complete (approximately 0 more seconds)03/01/21 08:54:07.45 spid1 Recovery is checkpointing database PRD (7).03/01/21 08:54:07.46 spid1 Recovering database ‘PRD’03/01/21 08:54:07.47 spid1 Starting up database tempdb03/01/21 08:54:07.48 spid1 Analysis of database tempdb is 100% complete.03/01/21 08:54:07.43 spid1 Recovery completed

Automatic Recovery (2)

In the fourth step, the Windows Server and SQL Server are started again. Whenever SQL Server is started, the automatic recovery is processed in each SQL Server database: The LSN of the last checkpoint is read from the database boot block along with the Minimum Recovery LSN.

The transaction log is scanned from the Minimum Recovery LSN to the end of the log. All committed dirty pages are rolled forward by redoing the logical operation recorded in the log record.

SQL Server then scans backward through the log file rolling back all uncompleted transactions by applying the opposite of the logical operation recorded in the log records.

This type of recovery is also done by a RESTORE statement. After restoring the copy of the database or log, the RESTORE statement also has to ensure that all committed log records are rolled forward and all uncompleted transactions are rolled back.

Check the result of an automatic recovery in the SQL Server error log. SQL Server provides configuration parameter Recovery interval (min) for automatic recovery. It defines the maximum time in minutes needed for an automatic recovery per database. SQL Server uses this option to determine when a checkpoint must be released. The default is 0, indicating automatic configuration by SQL Server. In practice, this means a recovery time of less than one minute and a checkpoint approximately every one minute for each database.


© SAP AG 2003


damage)


Possible

error

situations

Restore Scenarios

Hardware Move(to a newer hardware)



Errors requiring a database restore are caused by one of the following: Hardware failure, for example when a disk crashes and affects the data or transaction log files A logical error resulting in a database corruption User error, such as the accidental deletion of a table A disaster that destroys existing hardware A move to new hardware involving the whole database system

If any of the above situations occur, use the restore procedures described in the following slides. After a power failure, the database does not have to be restored. When power is cut off, the system shuts down abruptly and all current transactions are aborted. SQL Server has an automatic recovery mechanism to deal with such a situation. This mechanism is explained in later slides.


© SAP AG 2003

<SID>LOG1

PRIMARY


<SID>DATA1 <SID>LOG1

DirectoryC:\TempdbC:\Mssql\C:\usr\sap\C:\WINNT

Restore due to Media Failure


The procedure required to bring the database back to its original state differs depending on how files are distributed on the system and which systems or disks are affected by the failure. Assuming that data distribution follows the recommendations in the SAP Installation Guide and as described in course ADM100, then the system files should be distributed across 3 volumes as follows: Volume 1 Log files of the SAP database Volume 2 Data files of the SAP database Volume 3 SAP executables, SQL Server executables, Windows executables

The procedure used for a database restore depends on which volume crashed due to media failure.


© SAP AG 2003

Back up recent transaction log

Replace disk in RAID system

Restore SAP database

Restore transaction logs

Check restore operation

<SID>

<SID>

<SID>


SAP Data Volume Crash

PRIMARY


<SID>DATA1

Apply differential backups

<SID>

The SAP data files reside in a RAID5 system for increased data security. If a single disk fails, it must be replaced and the RAID5 system must be synchronized. Database operation is not interrupted. However, if the whole RAID5 system crashes, the SAP System comes to a standstill. An error is written to the SQL Server error log. In this case the database must be restored.

The following steps must be taken to restore the database after an SAP data volume crash: Immediately back up the current transaction log before shutting down SQL Server Replace disks in the RAID5 system; the disks should be formatted and assigned the same drive letter as the old ones

Restore the database by applying the most current database backup If differential database backups were made after the last full database backup, the most current differential database backup can be applied.

Restore the transaction log by applying all transaction logs created since the most current database backup or the most current differential backup, then restore the transaction log created immediately after the volume crashed

Check the restore These steps must be described in detail in a separate document explaining your procedure and escalation plan. Note: If the disks on which the database resides are damaged, work performed after the last transaction log backup will be lost. Back up the most current transaction log. Do this before shutting down SQL Server. The current transaction log can only be backed up if the disks holding the transaction log and the executables disk are undamaged.


© SAP AG 2003

Backing up the current transaction log

SQL Server Query Analyzer - [Query - sapprod.master.sa - (untitled) - #1 New query] x_File Edit View Query Window Help

DB: PRD

x_

BACKUP LOG PRD to R3DUMP0 WITH NO_TRUNCATE, FORMAT

If the disks on which the SAP database resides crashes, it is important to immediately back up the current transaction log, otherwise the work carried out since the last transaction log backup will be permanently lost.

The current transaction log can only be backed up, if the disks on which the transaction log resides is undamaged and the disk on which the SQL Server executables reside is undamaged.

To back up the current transaction log, proceed as follows: Insert a new tape into the tape device Open the Query Analyser and enter the following transact-SQL statement:

BACKUP LOG <SID> to <backup_device> WITH NO_TRUNCATE, FORMAT Execute the statement


© SAP AG 2003

Replace RAID system




<SID>

<SID>


<SID>LOG1 <SID>LOG1

SAP Log Volume Crash


<SID>

The transaction log files of the SAP database normally reside in a RAID1 system for reasons of increased security. If one disk crashes, the mirrored disk can be used to bring the database up again. If both disks crash, the SAP database would be marked suspect and a restore is necessary.

If the transaction log is not available anymore, the current transaction log cannot be backed up. The data between the last transaction log backup and the crash of the RAID1 system is therefore lost.

If the SAP log volume crashes, proceed as follows: Replace disks in the RAID1 system. The disks must be formatted and assigned the same drive letters as the old ones

Restore the database by applying the most current database backup If differential database backups were made after the last full database backup, the most current differential database backup can be applied.

Restore the transaction log by applying all transaction logs created after the most current database backup or the most current differential backup

Check the restore These steps must be described in detail in a separate document explaining your procedure and escalation plan.


© SAP AG 2003

Reload lost files from latest Windows Backup

Reboot Primary Windows

Restore msdb (and master) Database

Check Restore Operation

Procedure andescalation Plan

Replace disks and installauxiliary Windows

Executable Volume Crash

msdbmaster

C:\TempdbC:\Mssql\C:\usr\sap\C:\WINNT

Directory

If the disk with all other files except the SAP data and transaction log files crashes, the Windows operating system, SAP executables, SQL Server executables, and the msdb and master databases are lost. The operating system including the SAP System and SQL Server cannot start.

If the executable volume crashes, proceed as follows: Replace disks. The disks must be formatted and assigned the same drive letters as the old ones. Install an auxiliary Windows operating system. Restore all lost files using the latest Windows backup. In this scenario, the SAP data and transaction log files are not affected as they reside on different volumes. Do not reload these files from the Windows backup.

Reboot Windows with the restored (primary) Windows system. Restore the msdb database using the latest backup You may also have to restore the master database if its contents were changed, for example, when SQL Server configuration parameters were changed



© SAP AG 2003

RAID 1 crash :Log files lost

<SID> suspect

Replace crashed disk(s), synchronize RAID

SQL Restore of database and transaction logs

Automatic recovery

OK Some data lost! OK OK OK

Windows Restore of EXEs;not log file(s)!

RAID 5 crash:Data files lost<SID> suspect

System crash: Data + EXEs lostSQL Server down

Exe disk crash: EXEs lost

SQL Server down

One diskcrash

Shut down SQL Server

SQL Restore of msdb (and master)

Back up log <SID> with

no_truncate

Restore due to Media Failure - Summary

If a hard disk containing a database data file in a RAID5 system crashes, the situation is not serious unless the whole RAID5 system goes down, in which case the data files in the file system are lost and must be restored.

When both RAID1 hard disks containing the transaction log files crash, a whole database restore must also be performed. In this type of situation, you are likely to experience some data loss because the current transaction log cannot be backed up.

If the volume containing the executables of SQL Server, the SAP System, and Windows crashes, only these executables must be restored using Windows Restore. To bring the job history to a current state, you should perform an SQL Server restore of database msdb. The content of the master database rarely changes. If after a Windows Restore the content of the master database is not current, perform a database restore of the master database.

After checking the restore, restart SQL Server. SQL Server then executes an automatic recovery to roll back uncommitted transactions.


© SAP AG 2003

<SID>

Restore due to Logical Error


When a database corruption is detected, the problem must be diagnosed further to determine the extent and cause of the damage. The optimal method of restoring the physical consistency depends on the following considerations: Type of object affected, for example an index or table Location of the object, for example in database <SID> Extent of the damage

Open a problem message using the Online Service System (OSS) and have a support engineer analyze the error to find the most effective solution.


© SAP AG 2003

Back up recent transaction log





SAP Database Corruption

<SID>

<SID>

<SID>


<SID>

To repair a corruption on the SAP database, a database restore may be necessary. In this scenario, you do not need to recreate the RAID5 system, since the data files stored there are not damaged.

Perform the following steps: Immediately back up the current transaction log before shutting down SQL Server Restore the database by applying the most current database backup which is successfully checked for not containing corruptions

If differential database backups were made after the last full database backup, the most current differential database backup can be applied.

Restore the transaction log by applying all transaction logs written since the most current database backup or differential backup



© SAP AG 2003

<SID>

Restore due to User Error


A user might accidentally delete a table, or import a faulty transport. Depending on what has happened, there are different ways of solving the problem.

To cancel incorrect changes such as the accidental deletion of a table, perform a point in time recovery.


© SAP AG 2003

Time

Transaction log backup

08:0009:00

10:00

Database backups

Sunday Monday Tuesday Wednesday Thursday Friday

Go back to the state as recorded on Wednesday at 09:30 a.m.

Point In Time Recovery - Principles

A point in time recovery enables you to reset a database to a specific date and time. Use this method when you need to return the database to an earlier state, due to user error.

During a point in time recovery, all transactions completed after a given point in time are rolled back.

Point in time recovery can only take place while the transaction logs are being restored, not during a database restore.

In this example, we can assume that the transaction log was backed up every hour, at 8.00 a.m., 9.00 a.m., and at 10.00 a.m. In order to return to Wednesday's 9.30 a.m. state, you need the transaction log backups from Wednesday 9.00 a.m. and 10.00 a.m., since these contain the transactions executed before 10.00 a.m. All transactions that were not completed by the specified time are rolled back. You can only recover data up to a point in time where there was a checkpoint in operation on the database. A checkpoint can also be set explicitly by a user through the transact-SQL statement CHECKPOINT. For details, see Unit 1: SQL Server Architecture.


© SAP AG 2003

Restore due to External Factors / Hardware Move

<SID>LOG1

PRIMARY


<SID>DATA1 <SID>LOG1


Y:\<SID>LOG1

X:\<SID>DATA1X:\<SID>DATA2X:\<SID>DATA3

Hardware Move(to a newer hardware)


damage)

If the system crashes due to external factors, the whole file system is lost and has to be set up newly. The same procedure has to be followed when your SAP environment moves to a new hardware. Set up all hardware including - RAID systems - Tape device

Install Windows Restore SQL Server and the SAP System


© SAP AG 2003

Reload executables from latest Windows Backup

Reboot primary Windows

Restore msdb (and master) database



Replace hardware and install auxiliary Windows

Complete System Setup



<SID>

<SID>


<SID>

msdbmaster


Y:\<SID>LOG1

X:\<SID>DATA1X:\<SID>DATA2X:\<SID>DATA3

If your whole system crashes, proceed as follows : Replace hardware It is best to format and assign the disks to the same drive letters as the old ones.

Install an auxiliary Windows operating system Restore all executables using the latest Windows backup SQL Server restore operations recreate lost files. Do not reload these files from the Windows Backup.

Reboot Windows with the restored (primary) Windows system and restart SQL Server The <SID> database will be marked suspect.

Restore the master and msdb database using the latest backup Restore the SAP database and apply all proceeding differential backups (if available) and transaction logs from the latest backups



© SAP AG 2003

Restore master database



Reload master files from latest Windows backupRebuild master database

Master Database Not available


master

If the database master is corrupted, you may need to restore the database master. However, you do not need to recreate the data files, as they are not damaged. In this scenario SQL Server cannot start, as all the necessary information it reads is from the master database. You must therefore restore the database master from the latest Windows backup to create a functioning master database.

Perform the following steps: Restore the master.mdf and mastlog.ldf files from the latest Windows backup Restore the master database from the latest SQL Server backup to get the current state of the master database

Check the restore If a current Windows backup of database master is not available, master can be rebuilt using the Rebuild master utility. When master has been rebuilt, a current backup of master must be restored to create the SAP database, the backup devices, and SQL Server logins. Using Rebuild Master is not the recommended strategy for creating the database master, and with a proper backup strategy it is unnecessary.

Before you rebuild your database master, consult a SQL Server support engineer. These steps must be described in detail in a separate document explaining your procedure and escalation plan.


© SAP AG 2003

Restore from backup history

Restore from device

Restore to a point in time

Restore Methods

Restoring a database backup returns the database to the same state it was in when the backup was created. Restoring a transaction log backup reapplies all completed transactions that are in the transaction log backup to the database. SQL Server reads through the transaction log, rolling forward all the transactions on the transaction log. When SQL Server reaches the end of the transaction log, it has recreated the exact state of the database at the time the backup operation started. The restore operation then rolls back all transactions that were incomplete when the backup operation started.

The next section introduces the three different restore methods and their tools: Restore from history Restore from device Restore to a point in time


© SAP AG 2003

Restore database

General Options

x

Restore as Database: PRD

Restore: Database Filegroups or file From device

Restore Type Backup set date Size Restore from Backup set na

OK Cancel Help

Parameters

Show backups for database:First backup to restore:

Point in time restore

PRD

12/1/03 01:00: 47 AM -

......

12/1/03 01:00... 960640 Kb //./Tape012/1/03 08:01... 9408 Kb //./Tape012/1/03 09:00... 256 Kb //./Tape0

PropertiesPropertiesProperties

Restore database

General Options

x

Eject tapes (if any) after restoring each backupPrompt before restoring each backupForce restore over existing database

Original file name Restore as

OK Cancel Help

Recovery completion state

Leave database operational. No additional logs can be restored.

Leave database nonoperational, but able to restore additional transaction logs.

Leave database read-only and able to restore additional transaction logs.

F:\PRDLOG1\PRDLOG1.ldfE:\PRDDATA2\PRDDATA2.ndfE:\PRDDATA3\PRDDATA3.ndfE:\PRDDATA1\PRDDATA1.mdf

Restore database files as:

F:\PRDLOG1\PRDLOG1.ldfE:\PRD4DATA2\PRDDATA2.ndfE:\PRDDATA3\PRDDATA3.ndfE:\PRDDATA1\PRDDATA1.mdf

Undo file: f:\Program Files\Microsoft SQL Server\MSSQL\BUndo file: f:\Program Files\Microsoft SQL Server\MSSQL\B ......

Restore From Backup History

SQL Server stores log entries for each backup in msdb tables backupset and backupfile. For a restore from backup history using these msdb records, proceed as follows: In the Enterprise Manager, choose Tools → Database Restore. In the list, the latest database backup and subsequent transaction log backups are already selected. Select the following options: - Eject tapes (if any) after restoring each backup - Prompt before restoring each backup - If one tape device is used, you have to switch tapes after

restoring each backup. - Force restore over existing database - If the database files exist, and you want to override the

existing database, for example after a restore due to a logical error. Do not select this option after a restore due to media failure.

- Leave database operational; No additional transaction logs can be restored - If all transaction logs are restored during this procedure.

Confirm your entries During a database restore, SQL Server automatically recreates the database and its associated files, by copying data from the backup into the database. To change original file locations, enter a new name under Restore as. The default is the original file name.

The advantage of working with the method Restore from backup history is that you do not have to find the latest backup.

Note: The system administrator restoring the database backup must be the only person currently using the database to be restored.


© SAP AG 2003

PRD

PRD

RL04S

ID04S

11

22

22Contents...

Media Set Labels:SQL Server Enterprise Manager x

Restore of database ‚PRD‘ completed successfully.

Ok

ii

Contents...Restore x

The restore operation has completed and the next restore operation isabout to proceed. To continue the restore process, press OK. Otherwise,press Cancel.

Backup set to restore:

Media Name:

12/4/02 11:14:09 AM -

Media Description:

OK Cancel

RL04S

Changing Tapes

If one tape device is used, make sure the right tape is in the tape device. Identify the tape by its tape label. If you are unsure about its content, issue a RESTORE HEADERONLY FROM <device> or use the Enterprise Manager to display the media header.

After a database backup is restored, a window appears asking you to insert the new tape to continue with the restore (1). Note that this window only appears if option Prompt before restoring each backup has been set. Switch the tape and confirm (2). Once the restore is complete, a confirmation appears.


© SAP AG 2003

Restore Database

General Options

x

Restore Database: PRD|

Backup: Database Filegroups or files From device

Database - CompleteDatabase - DifferentialTransaction logFile and Filegroup:

Parameters

Device

Select Device

Device 1 View Contents ...

Restore backup set

Read backup set information and add to backup history

OK Cancel Help

Choose Restore Device x

When the backup is restored, SQL Server will attempt to restore from thedevices listed below.

Backup set:

Add...

Media verification option

Only restore from media with the following name:

OK Cancel

Disk Tape

ID04S

Restore from: Device name

\\.\Tape0 Edit

Remove

Remove All

Media Name: ID04S

Restore From Device

If the msdb is not available, the backup history is lost. In this case, you must restore the database from the backup device or restore the msdb first, to use restore from history.

When using Restore from device, in the Enterprise Manager select option From device, then select the device. The header of the chosen backup device, for example R3DUMP0 for the tape device is then read. To display what has been stored on this device, choose View Contents. Select the type of backup you want to restore (Database Complete, Database Differential, Transaction log, File or Filegroup), then confirm your entries. When the database backup has been successfully imported, use the same procedure to import the transaction log backups in the order in which they were backed up.

To check a media name before performing the restore, specify the option Media verification. Alternatively, you can recreate the backup history by choosing option Read backup set information and add to backup history. This way the backup header is read from devices specified by you, and is added to the backup history tables in the msdb database. Only one backup set can be added at a time on one backup media. After adding all the necessary backup sets, you can use the method Restore from history.

The advantage of using the method From Device is that you can restore the database even after losing the backup history.


© SAP AG 2003

SQL Server Query Analyzer - [Query - sapprod.master.sa - (untitled) - #1 New query] x_File Edit View Query Window Help

DB: PRD

x_

RESTORE DATABASE PRD FROM R3DUMP0 WITH FILE = 1,

MEDIANAME = ‚ID04S‘, RECOVERY

RESTORE LOG PRD FROM R3DUMP0 WITH FILE = 1,

MEDIANAME = ‚RL04S‘, NORECOVERY

RESTORE DATABASE PRD FILE = ‘PRDDATA1’ FROM R3DUMP0

WITH MEDIANAME = ‘RF04S’

Restore From Device Using the Query Analyzer

You can restore a database from a device using the Query Analyzer and transact-SQL statement RESTORE DATABASE. This statement includes the following options: FROM: Specifies the backup devices from which to restore the backup. If the FROM clause is not specified, a restore is not performed; instead, the database is recovered.

FILE: When specified in the WITH clause, FILE identifies the backup set to be restored. For example, a file number of 1 indicates the first backup set on the backup medium and a file number of 2 indicates the second backup set. When specified in the file list clause, this option identifies the file or filegroup to be restored.

MEDIANAME: Specifies the media name for the entire backup set. If provided, the media name must match the media name on the backup volume; otherwise, the restore operation terminates. If no media name is given, the check for a matching media name is not performed.

NORECOVERY|RECOVERY: Instructs the restore operation (not) to roll back any uncommitted transactions. Option NORECOVERY must be specified if another transaction log has to be applied. Option RECOVERY is the default.

REPLACE: Instructs SQL Server to create the specified database and its related files even if another database already exists with the same name. In this case, the existing database is deleted. When the REPLACE option is not specified, a safety check occurs, which prevents overwriting a different database by accident.

For details, see SQL Server Books Online.


© SAP AG 2003

Restore database

General Options

x



OK Cancel Help

Parameters



PRD

12/1/03 01:00: 47 AM -

......




12/1/03 08:30: 47 AM -

Contents...Media Set Labels:Point In Time Restore x

Point in time restore will halt the restoration of transactionlog entries recorded after the specified date and time.

Date:

Time: 08:30 AM

Fri 12/ 1/2003

OK Cancel

Point In Time Recovery

To perform a point in time recovery, you must first restore the database backup and then define the time to which you want to recover your database. The transaction logs are then applied only up till the specified point in time.

Use the Enterprise Manager. Choose Tools → Restore Database. Select the option Point in Time Restore. In the dialog box, enter the time to which you wish to restore and confirm your entry.

Point in time recovery can also be executed through the transact-SQL statement RESTORE LOG and the option STOPAT.

Point in time recovery is not supported with the bulk-logged recovery model. Bulk-logged recovery model only allows the database to be recovered to the end of a transaction log backup when the log backup contains bulk changes.


© SAP AG 2003

Restore Strategies

Restore from single DB and transaction log backups

Restore from a parallel DB backup

Restore from a two-step disk backup

Restore from supplementary differential backups

The method you choose to restore your SAP database depends on the type of error and on your chosen backup strategy. However, all restore scenarios start with a restore of the latest database backup.

The next section explains restore strategies, which depend on the different backup strategies introduced in Unit 4: Database Backup.


© SAP AG 2003

Full backup Full backup

Sun, 1 Mon, 2 Tue, 3 Wed, 4 Thur, 5 Fri, 6

Full backup

ID03S

RL04S

Log backups Log backups

11

22

Restore from Single DB and TL Backups

Log backups

To restore an SAP database from a single database backup and subsequent transaction log backups, perform the following steps: If possible, back up the current transaction log to a new separate disk backup device, using the option NO_TRUNCATE.

Set the SAP database to single-user mode. Restore the most current database backup (1) and all subsequent transaction log backups (2) by choosing one of the restore methods: - Restore from history or - Restore from device.

Set the option Leave the database non-operational but able to restore additional transaction logs. Apply the current transaction log that you recently backed up. Set the option Leave the database operational no additional transaction logs can be restored. If this option is set, no more transaction logs can be applied.

The database is not recovered until the current transaction log has been applied. This prevents any transactions from being partially rolled back. Transactions can only be rolled back at the end of the last restore operation, when the current log has been applied.

Check the restore.


© SAP AG 2003

Full backup Differential


Differential

ID01S

R+03S

RL04S

Log backups Log backups Log backups

11

22

33

Restore From Differential Backups

To restore an SAP database using supplementary differential backups, perform the following steps: If possible backup the current transaction log by using the NO_TRUNCATE option to a new separate disk backup device.

Set the SAP database to single-user mode. Restore the last database backup created (1). Restore the last differential backup created since the database backup was created (2). Apply all transaction log backups, in sequence, created after the last differential backup was created, finishing with the transaction log backup created in the first step (3).

Perform the restore using one of the restore methods: - Restore from history or - Restore from device.

Check the restore.


© SAP AG 2003

Restore database

General Options

x



OK Cancel Help

Parameters



PRD

12/1/03 01:00: 47 AM -

......




Backup Set Properties

General

x

Name: PRD DB backup

Description:

Change...

OK Cancel

Type: Database

Size: 10976244 Kb

Start date: 12/01/03 1:00:47 AM

Finish date: 12/01/03 2:55:19 AM

Media Name: ID01S

Media Description:

Restore From://./Tape0//./Tape1

Restore from a Parallel Backup

To restore an SAP database from a parallel database backup and subsequent transaction log backups, perform the following steps: If possible, back up the current transaction log to a new separate disk backup device, by using the option NO_TRUNCATE.

Set the SAP database to single-user mode. Perform a restore as explained in the previous slide using one of the restore methods: - Restore from history or - Restore from device.

If you perform a restore from history, select Properties to find out which backup devices to restore from.

Put the tapes into the tape devices. Check the restore.


© SAP AG 2003

DISK_DEV2


Log backups Log backups Log backups

Full backup Full backup Full backup

DISK_DEV1

11

22

Restore from a Two-Step Disk Backup

To restore an SAP database from a two-step disk backup, perform the following steps: If possible, back up the current transaction log to a new separate disk backup device, using the option NO_TRUNCATE.

Set the SAP database to single-user mode. If the disk where the last database backup has been performed has also crashed, provide additional disk space for applying the latest Windows backup. Perform a restore as explained in the previous slides by using one of the restore methods: - Restore from history or - Restore from device.

Restore the latest database backup (1) and apply all subsequent transaction log backups including the one performed in the first step (2).

Check the restore.


© SAP AG 2003

Checking the Restore

The backup and restore history is stored in the msdb database. To find the history, choose Administration → CCMS → DB Administration → Backup Logs then button Restoration History.

If time permits, run some DBCC consistency checks on the newly restored database. Note: These checks are not absolutely required, as a consistency check should have been executed each month and should have guaranteed that the backup on the backup media is consistent.


© SAP AG 2003


Describe different restore strategiesDefine an appropriate restore strategy based on the backup strategy chosen by your company Perform a restore using SQL Server tools and identify the restore method you should use Check your restore

Summary


© SAP AG 2003

SAP Installation GuideSQL Server Books OnlineADM100 mySAP Technology Administration



© SAP AG 2003

Unit Actions

Exercises?

Solutions


Exercises

Unit: Database Restore Topic: Restore Methods


• Restore a database under SQL Server

• Detect a hardware problem and determine the correct restore method and strategy to be used to bring the database back to its original state

The customer detects a hardware problem, which makes it necessary to restore the SAP database.

1-1 Restore a user database after a media failure

1-1-1 In this exercise you will restore the test database using the backups you created in the previous exercise. Make sure that you have a valid database and transaction log backup of the test database. Execute stored procedure crash_me in the master database. Afterwards, check whether the test database is available.

1-1-2 Check the file system to see whether the data files TestData1.mdf, TestData2.ndf, and TestLog1.ldf are located in the directory F:\Mssql7db\MSSQL\Data\. Which step do you have to perform immediately before you start with the database restore?

1-1-3 Restore the test database by choosing one of the restore methods introduced in Unit 5: Database Restore. Which method do you choose? Which options do you set in the Restore Database window?

1-1-4 Check the restore operation.


Solutions

Unit: Database Restore Topic: Restore Methods

1-1 Restore a user database after a media failure

1-1-1 Close all connections to the test database. Open a Query Analyzer window, and execute crash_me in the master database. The stored procedure returns message: output -------- NULL (1 row(s) affected)

1-1-2 Use the Explorer to check whether the files are available. Normally, you should back up the current transaction log immediately before you start to restore the database. To do this use Transact SQL statement: backup log test to R3DUMP3 with no_truncate The following error occurs: Server: Msg 911, Level 16, State 1, Line 1 Could not locate entry in sysdatabases for database 'test'. No entry found with that name. Make sure that the name is entered correctly. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP LOG is terminating abnormally. When the primary database file is not available anymore, as in this particular case, you cannot back up the current transaction log and you experience some data loss.


1-1-3 To restore the test database, choose Tools → Restore Database … in the Enterprise Manager. Select the database. The backup history should show one database backup and one succeeding transaction log backup. Leave them all marked. Under Options, select: Prompt before restoring each backup Force restore over existing database Leave database operational. No additional logs can be restored A correct restore operation returns the message: Restore of database ‘test‘ completed successfully. Execute transact-SQL statement on database test select * from stores

1-1-4 Call the Backup Monitor under Tools → CCMS → DB Administration → DB12 – Backup Logs (transaction code DB12), and check the restoration history. Open the Query Analyzer and perform DBCC CHECKDB (test).


© SAP AG 2003

Regular Maintenance and Error Handling

1 Introduction




5 Database Backup

6 Database Restore

7 Regular Maintenance andError Handling


© SAP AG 2003

ContentsError and Alert handlingSpace managementData consistencySQL Server parameter maintenance


Handle common errorsMonitor your SAP System with the help of the CCMS Alert MonitorMonitor database and transaction log growth and extend the databasePerform regular administrative tasks

Regular Maintenance and Error Handling


© SAP AG 2003

CCMS Alert Monitor

It is vital to regularly monitor the availability, performance, correctness and security of your SAP System. The SAP Basis System contains a various set of tools for monitoring and performance analysis. These tools include a central monitoring architecture which performs the continual system monitoring of all hardware and software components, which is implemented in the CCMS Alert Monitor (transaction RZ20). In the SAP Easy Access menu choose Tools -> CCMS -> Monitoring -> Alert-Monitor.

SAP provides a pre-configured collection of monitors which are integrated in the CCMS Alert Monitor and which can be used immediately after the installation of your SAP System. Each collection combines an area of interest such as the database monitor.

The use, layout and maintenance of the CCSM Alert Monitor has been introduced in course ADM100 myTechnology Administration.


© SAP AG 2003

Alerts in the MMC

The CCMS Alert Monitor can also be viewed in the Microsoft Management Console on each SAP Application Server.


© SAP AG 2003

Categories of Information

Space Management

Performance

Backup / Restore

R/3 Consistency

Health

Information of alerts for SQL Server in the monitoring tree are grouped in the following main categories: Space Management Performance Backup / Restore R/3 Consistency Health

Alerts for the SQL Server indicate when defined threshold values have been exceeded. Existing threshold values reflect SAP recommendations, however they can be adjusted to meet individual system requirements.

The categories Performance and Backup / Restore are not handled in this chapter as they were already discussed in earlier chapters.


© SAP AG 2003

Space Management

Autogrow option

Current sizes

Free space

Status information and alerts related to the disk space on the database server are displayed for the <SID> and the tempdb database. For each physical database file the autogrow option, the current sizes and the free space is displayed.

The values are refreshed every 8 hours. If you double click on the values, the Database Allocation Monitor (transaction DB02) is displayed.


© SAP AG 2003

Monitoring Database Growth

2

13

56

748 9 10

11

Database growth must be monitored regularly using the Database Allocation Monitor (Transaction DB02). As the database grows, you may consider adding more disk space.

The main screen of the Database Allocation Monitor shows the amount of space used by the SAP database (1) and its transaction log (2).

Button DB space history (3) displays a history of the growth of the database. Performance data collected during the job COLLECTOR_FOR_PERFORMANCE_MONITOR (report RSCOLL00) is stored to table MONI once a day and evaluated for this display.

The Files section lists the database and log files used. Ensure that the log files and database files reside on separate disks (4). For data files, the PRIMARY filegroup is displayed (5). The column Growth displays the increment by which the file can grow (6). This increment should be at least 100 MB and the free space on the disk, displayed in column Free disk (7), should be large enough to hold at least two increments. The initial size of the file is displayed under Size(MB) (8). The used portion of this file is displayed under Used (MB) (9). A file can grow to an unrestricted or to a limited size. This information is displayed under Limit (10).

The last section displays the number and sizes of tables, indexes and stored procedures. Note that the reserved size for tables and indexes is shown.

If your database grows fast, you might want to find out which table is responsible for the growth. Choose button Space Statistics (11) and choose Top n growing tables.


© SAP AG 2003

Monitoring Database Growth (2)

DEV

Space Allocated

Data:

DEVdata1: 3000 MB 2999,99 MB

DEVdata2: 3000 MB 2999,99 MB

DEVdata3: 3000 MB 2999,99 MB


Total Used Free

SQL Server uses a proportional fill strategy across all files within filegroup PRIMARY, writing an amount of data proportional to the free space in the file (see Unit 1: SQL Server Architecture). As a result all the files in a filegroup tend to be filled at approximately the same time. This strategy ensures that I/O operations are distributed more evenly over the database files thus preventing hotspots on specific files and partitions.

To display the allocated size of a file, use the task pad view in the Enterprise Manager. When SQL Server is unable to allocate space for a database object, the following message appears:

Server: Msg 1105, Level 17, State 2, Procedure <procedure>, Line 10 Could not allocate space for object '<object>' in database '<SID>' because the 'PRIMARY' filegroup is full.

When this occurs, the current transaction aborts. SAP System activity cannot continue. You should monitor database growth regularly and expand the database early enough to prevent this type of error.


© SAP AG 2003

DEV

Space Allocated

Data:

DEVdata1: 3000 MB 19999,99 MB

DEVdata2: 3000 MB 19999,99 MB

DEVdata3: 3000 MB 19999,99 MB


Total Used Free

Automatic File Growth

Expand first file by 100 MB

If the database is full, the proportional fill mechanism is no longer used. SQL Server will then automatically expand one file at a time, provided that the file is set to grow automatically. If enough space is available on the volume used for the SAP data files, the file is expanded by one increment as specified in autogrow.

Insert operations fill the expanded file first, before using another file. When using only one volume, this mechanism does not affect performance. If more than one volume is used, insert operations will not be distributed equally across the volumes. Any performance improvement that would have been achieved by distributing insert operations equally across all volumes is therefore lost.

Each file can have a maximum size specified. If a maximum size is not specified, the file can continue to grow until it has used all available space on the volume.

Ensure that the free space on the volume is at least twice the increment, by which the file grows. If less space is available, you should consider purchasing additional disk space.

Note: Files should normally be expanded manually, and only grow automatically in unusual situations.


© SAP AG 2003

Extending the Database

Database must be extended

Can volume be extended?

Add new disk to volume

Add newvolume

Move one old fileto new volume

Yes No

+

Expand files andset all files to autogrow

PRIMARY

<SID>DATA1


Expand files on old and new volumeand set all files

to autogrow

PRIMARY

<SID>DATA1

PRIMARY

<SID>DATA1

<SID>DATA2

SAP recommends that you set the database files to grow automatically. Normally, you should expand files manually, and allow them to grow automatically only in exceptional situations, for example if the database administrator is unable to react before more space is needed. If files are expanded automatically in production operation, system performance can be impacted.

Once disk space is exhausted, you must extend the database. If the volume containing the data files can be extended, you can simply add a new disk to the volume. Increase the size of all files equally and leave them set to autogrow.

Alternatively, you can extend the database by adding a new volume. Procedures for doing this are explained in the following slides.

Information about the database files is stored in tables sysfiles and sysfiles1. In addition to your backup activities, keep a record of this information about the location of files on paper. You may need the information if a failure occurs and a whole system restore is required. Stored procedure sp_helpfile returns the physical names and attributes of files associated with the current database.


© SAP AG 2003

Expanding a File Using the Enterprise Manager

SQL Server Enterprise Manager x_Console Window Help

x_Action View Tools


SQL Server Group

sapprd [Windows NT]

PRDmastermodelmsdbNorthwindpubstempdb

Databases

Console Root\Microsoft SQL Server\SQL Server Group\sapprd [Windows NT]\Databases

7 Items

PRD master model msdb

northwind pubs tempdb

Data TransformationManagementSecuritySupport Services

PRD Properties x

Name: PRD

Database files

File groupSpace Allocated (MB)LocationFile namef:\PRDDATA1\PRD4D... 4000 PRIMARYf:\PRDDATA2\PRDD... 4000 PRIMARYf:\ PRDDATA3\PRDD... 4000 PRIMARY

...

...

...

PRDDATA1PRDDATA2PRDDATA3

File properties:

File growth

Cancel HelpOK ApplyApply

In megabytes: 100

By percent: 10

Maximum file size

Unrestricted filegrowth

Restrict filegrowth (MB): 2001

Automatically grow file

.

General Transaction Log Options Permissions

If the volume can be extended, data files on the larger volume must be expanded manually. To expand a file, use the Enterprise Manager. There you can specify the increase in space allocated to a particular file. If all other files have no free space left, all the insert operations will use the new space on the expanded file. You should expand all files equally and use almost all the disk space.

Keep extra space on the disk in reserve, and ensure that files are set to grow automatically by an increment of at least 100 MB. If you don't want to extend the database by the whole new disk space at one time, expand one file first and the next file when the next expansion is necessary.

Use these procedures only when all files are located on one volume. Alternatively, you can use the transact-SQL statement ALTER DATABASE to expand a file. See Books Online for details.


© SAP AG 2003

Extending the Database on a New Volume

SAP DB data fiIesRAID5 <SID>DATA3

PRIMARY

PRIMARY

RAID5 SAP DB data fiIes

11

22

22

<SID>DATA3

<SID>DATA1 <SID>DATA2

SAP recommends that you store the SAP database data files in a RAID5 or RAID0+1 system. A RAID system can grow up to a limited size. If a database grows beyond this limit and the RAID system cannot be further expanded, a new RAID array must be set up.

If you use an additional volume to extend your database, one file on the existing volume must be moved to the new volume (1) to free up space on the old volume. Expand all files on both volumes (2), or create new files if there is only one file per volume. Leave some reserve space to let the files grow automatically. SQL Server continues to distribute the insert operations proportionally over all new space, and the IO load is equally distributed across the two volumes.


© SAP AG 2003

Moving a Database File

detail stats.SQL Server Query Analyzer - [Query - sappod.master.sa - (untitled)]

sp_detach_db ‘PRD’, ‘true’

detail stats.C:\WINNT\System32\cmd.exeMicrosoft(R) Windows NT (TM)(C) Copyright 19985-1996 Microsoft Corp.

C:\> mv F:\PRDDATA2\PRDDATA2.NDF G:\PRDDATA2\PRDDATA2.NDF

msdbmaster

<SID>ID01S


sp_helpfile


sp_attach_db PRD’,‘F:\PRDDATA1\PRDDATA1.MDF’, ‘G:\PRDDATA2\PRDDATA2.NDF’

11

22

33

55

44

A database file can be moved to a different volume in order to distribute I/O operations evenly over the existing volume. To move a database file to a new volume: Ensure that a recent backup of the database is available and verified. (1) Detach the database. (2) Move one database file to the new volume. (3) Attach the database specifying the new location of the moved file. (4) Check the database files. (5)

Detaching a database removes the database from SQL Server, but leaves the data and transaction log files in the database intact. Use stored procedure sp_detach_db to detach a database from the server. Specify true as second option in the stored procedure. This prevents UPDATE STATISTICS from running on all tables on the database before detaching. Note: The database must not be used while it is being detached from the server, that is, there must be no connection open to the database.

Move the file to a new volume and reattach it using stored procedure sp_attach_db. When attaching a database, specify the physical location of the primary file and the file being moved, along with the new location. Because the primary file contains the information needed to find the other files comprising the database, you need only specify the location of the file that has changed location.

Check the files by running the stored procedure sp_helpfile, which returns the physical names and attributes of files associated with the current database.

See also the SAP Online Help and Books Online on how to move a database file to a new disk.


© SAP AG 2003

Adding a File Using the Enterprise Manager

SQL Server Enterprise Manager x_Console Window Help

x_Action View Tools


SQL Server Group

sapprd [Windows NT]

PRDmastermodelmsdbNorthwindpubstempdb

Databases

Console Root\Microsoft SQL Server\SQL Server Group\sapprd [Windows NT]\Databases

7 Items

PRD master model msdb

northwind pubs tempdb

Data TransformationManagementSecuritySupport Services

PRD Properties x

Name: PRD

Database files

File groupSpace Allocated (MB)LocationFile namef:\PRDDATA1\PRDD... 2000 PRIMARYf:\PRDDATA2\PRDD... 2000 PRIMARYf:\PRDDATA3\PRDD... 2000 PRIMARYf:\PRDDATA4\PRDD… 2000 PRIMARY

...

...

...

PRDDATA1PRDDATA2PRDDATA3PRDDATA4

File properties:

File growth

Cancel HelpOK ApplyApply

In megabytes: 100

By percent: 10

Maximum file size

Unrestricted filegrowth

Restrict filegrowth (MB): 2001

Automatically grow file

.

General Transaction Log Options Permissions

You can also use additional data and transaction log files to extend a database. When a file is added, the file is immediately available for use by the database.

Use the Enterprise Manager to add a new file. Specify the size of the file, the maximum size to which the file should grow, the increment by which the file is to grow each time (default = 10 %), and the filegroup to which the file belongs. In the SAP environment, filegroup PRIMARY is used. When you add a new file, use the SAP naming conventions.

When adding a new data file, keep extra space on the disk in reserve, and set the files to grow automatically in increments of at least 100 MB.

You can also use the transact-SQL statement ALTER DATABASE. See Books Online on more details.


© SAP AG 2003

1

32

Monitoring Transaction Log Growth

The transaction log grows if data in the database is changed. As the transaction log grows, the first log file fills, then the second, and so on, using a fill-and-go strategy.

In the SAP environment, there is usually only one log file. The size of this log file must be determined carefully. Watch the growth of the transaction log using the Database Allocation Monitor (Transaction DB02). Observe how the transaction log fills during exceptional workload, for example during Batch Input or an upgrade. You can monitor this at ST04 → Detailed Analysis Menu → Performance History → folder DB space. These values are refreshed every 2 hours. Alternatively choose DB02 → DB space history to display one value per day.

Use command DBCC SQLPERF(logspace) to display the amount of space used by the transaction log.

The Database Allocation Monitor displays the following: File name Location of file Space allocated for the transaction log files Log size (1) Log free size (1)

If automatic growth is configured for this log file, column Growth (2) displays the increment by which the transaction log can grow. The increment should be 50% of the file size, and the free space on the disk should be enough to handle at least one increment.

Column Free disk (3) displays the amount of free space on the disk.


© SAP AG 2003

Backup



insert update begin4 rollback chkp insert delete dropLSN16 LSN17 LSN18 LSN19 LSN20 LSN21 LSN22 LSN23

update begin5 commit3 insert delete chkp insert commit4LSN24 LSN25 LSN26 LSN27 LSN28 LSN29 LSN30 LSN31

<SID>

SQLServer

RL02A

Msg 9002,The log file for

database ‘<SID>' is full

11

22

33

Transaction Log Full

If the transaction log is full, SQL Server writes the following error message to the error log file and interrupts the current transaction: Server: Msg 9002, Level 17, State 2, Procedure <procedure>, Line 12 The log file for database ‘<SID>' is full. Back up the transaction log for the database to free up some log space

A similar message is written to the SAP System log. If you see this error message, back up the transaction log immediately using the SAP Planning Calendar (Transaction DB13). Use the same tape for this extra backup as used for all regularly planned transaction log backups. Append the backup to the existing backups on the tape. Do not specify the option Initialize. Remember that a transaction log backup truncates the inactive portion of the log and makes space for new transaction logging. See also Unit 4: Database Backup.

Do not shut down SQL Server or the SAP System, if the transaction log is full. If you shut down SQL Server, log entries will be lost.

With a good backup strategy, errors like this should not occur. However, the transaction log fills up quickly during periods of an exceptionally high workload, for example during Batch Input or an upgrade. Ensure that at least 1.5 GB space is available in the transaction log.


© SAP AG 2003

R/3 Consistency

Tables, Views and Indexes missing in DB

In this category objects that are defined in the SAP ABAP/4 dictionary but not on the database are displayed.

The values are refreshed once a day.


© SAP AG 2003

Database Allocation Monitor: ABAP/4 Dictionary

Objects of the ABAP/4 Dictionary which were not found in the database are displayed in the Database Allocation Monitor after choosing button Checks → Database ↔ ABAP/4 Dictionary.

Indexes that exist in the SAP ABAP/4 dictionary but not on the database can be created by positioning the cursor on the index and click button Create on DB.


© SAP AG 2003

Health

SQL Server configuration parameters

SQL Server and Windows Build Numbers

Errors in the SQL Server Error Log

The health of your SAP System can be judged by monitoring the effectiveness of parameter settings and watching out for common errors.

The most important alerts to be checked regularly are the correct settings of the SQL Server configuration parameters, the correct versions of SQL Server build numbers and errors in the SQL Server Error Log.


© SAP AG 2003

SQL Server Configuration

detail stats.SQL Server Query Analyzer - [Query - sappod.master.sa -]

name minimum maximum config_value run_value

------------- -------- ----------- ------------ ---------

affinity mask -2147483648 2147483647 0 0

allow updates 0 1 1 1

awe enabled 0 1 0 0

c2 audit mode 0 1 0 0

cost thresholdfor parallelism 0 32767 5 5

cursor threshold -1 2147483647 -1 -1

default full-text language 0 2147483647 1033 1033

default language 0 9999 0 0

fill factor (%) 0 100 0 0

index creatememory (KB) 704 2147483647 0 0

Lightweightpooling 0 1 0 0

locks 5000 2147483647 0 0

The CCMS Alert Monitor shows a red alert when SQL Server Parameters have been set, but are not active.

To change the configuration parameters select the server, than select Action → Properties to display the Properties window.

A list of SQL Server configuration parameters can also be retrieved by running sp_configure in the Query Analyzer. Columns minimum and maximum display the acceptable range of values. Column config_value contains the active value. Column run_value contains the value that is currently set. After changing a configuration parameter using sp_configure, use command reconfigure with override to update the currently configured value of a configuration option. Because some configuration options require a server stop and restart to update the currently running value, reconfigure does not always update the currently running value. The option with override disables the configuration value checking.

For dynamic configuration parameters, changes take effect immediately. For static configuration parameters, changes take effect after you shut down and restart SQL Server. Before restarting SQL Server, shut down the SAP System.


© SAP AG 2003

Error in parameter configurationError in parameter configuration

initconfig: Error 2 (The system cannot find the file specified.)

initconfig: Error 2 (The system cannot find the file specified.)

...

Open a problem message

Books OnlineSQLServer

SQL Server does not Start

Check forerror

messages

SAP Notes

If SQL Server does not start, check the SQL Server error log at file system level. To get a detailed description of the error message, use Books Online or the SAP Notes.

Because SQL Sever sets its resources dynamically, an incorrect setting of SQL Server configuration parameters sometimes causes SQL Server not to start. To start SQL Server with a minimal configuration, use the option -f in the sqlservr command. Reconfigure the configuration options using the sp_configure system stored procedure or the Enterprise Manager.

If the master database is not available, for example due to a corrupt master file, SQL Server cannot start because it is unable to read the configuration information. An error starting with Initconfig appears in the SQL Server error log.

If the problem cannot be solved, create a problem message immediately. Enter the SQL Server error message.


© SAP AG 2003

msdbmaster

Applying Service Packs

SAP ApplicationServer

1122

55

33

44

SQLServer

Database Server

install the ServicePack on the Database

Server and all SAPApplication Servers

download the current Service Pack

follow instructions in the readme file

stop the SAP System and SQL Server

perform a backup if no current backup exists

66

perform a backupand check the installation

msdbmaster

README.txtREADME.txt

SQL Server and Windows build data is displayed in the first two nodes under Startup parameters in the CCMS Alert Monitor.

The current service pack of SQL Server is always released by SAP. SAP strongly recommends that the current service pack is installed immediately when it is available. Explicit exceptions are mentioned in SAP Notes.

The exact SQL Server version appears in the first line of the SQL error log. The version can also be found out using the SQL command select @@version.

The installation of a service pack is described in SAP note 417089: Download the current service pack from the Microsoft Web Server or a CD (1). A service pack always contains a README file from Microsoft. This file contains detailed installation instructions (2).

Stop the SAP System and SQL Server (3). SAP recommends that you perform an Windows backup of all server files after each configuration change. The easiest way of running a Windows backup is to reboot it from a second Windows installed in parallel (4). Before installing the service pack, make sure that a current backup of the master and msdb model databases exist. Installing a service pack modifies the master and msdb databases, making them incompatible with older versions of SQL Server. These backups are required if you decide to reinstall SQL Server without the service pack.

Install the service pack by opening the SETUP.BAT file (5). Install the service pack on the SAP application server in the same way as it is installed on the database server.

Perform a Windows backup, back up the master and msdb databases and check the installation (6).


© SAP AG 2003

Check forerror messages

in:

SAP System Log

SQL Server Agent error log in path X:\Microsoft

SQL Server\MSSQL\LOG

SQL Server Agent error log in path X:\Microsoft

SQL Server\MSSQL\LOG

SQL Server error log in path X:\Microsoft SQL Server\MSSQL\LOG

SQL Server error log in path X:\Microsoft SQL Server\MSSQL\LOG

dev-trace files

Event Viewer

!! ii??

Error Handling

ABAP runtime errors

When a problem occurs, begin by checking the following: SAP System log Use Transaction SM21.

SQL Server error log Use the Database Monitor (Transaction ST04) or the Enterprise Manager. Alternatively, you can directly edit the log files located in directory X:\Microsoft SQL Server\MSSQL\LOG.

SQL Server Agent error log Use the Database Monitor (Transaction ST04) or the Enterprise Manager. Alternatively, you can directly edit the log files located in directory X:\Microsoft SQL Server\MSSQL\LOG.

Developer trace files (Transaction ST11) display all messages written by the SAP work processes. To limit the messages displayed on the database or the database interface, choose Display Components.

ABAP runtime errors Use transaction ST22.

Event Viewer Choose Start → Settings → Χοντρολ Πανελ → Administrative Tools → Event Viewer.

Search for related SAP Notes in SAPNet.


© SAP AG 2003

You may encounter the following SQL Server errors:

Database file full (Error 1105)

Transaction log full (Error 9002)

Consistency errors (Errors 601, 605, 644, 823, 2511, 2513, 2540, 8928,

8944, 8952, 8976, …)

Resource Problems (Error 845,…)

Performance Problems (Error 8645, …)

Deadlock (Error 1205, …)

Locking problems (Error 601, …)

Network errors (Error 11, …)

Some Errors that may Occur

Errors 1105 and 9002 indicate that either a data file or the log file has run out of space. What to do in such situations, has been discussed in previous slides.

SQL Server as a Relational Database Management System is responsible for the physical consistency of the database. There is a physical inconsistency (database corruption) if there are errors in internal structures (page pointer, allocation ...) in the database. You can suspect this if there are frequent short dumps in the SAP System, in particular if the following SQL Server error messages appear: SQL Error 601, 605, 644, 823, 2511, 8928, 8944, 8952, 8976. See SAP note 142731 for details.

See SAP note 429561 for details on SQL Server error 845. When SQL Server is low on memory a timeout may occur while waiting for memory resources to execute a query. In this case SQL Server error 8645 is shown in the SQL Server error log and an ABAP/4 short dump occurs. See SAP note 206694 for details.

SQL Server error 1205 indicates a deadlocks on a certain table. Known deadlocks are listed according to the table affected in SAP note 565710.

SQL Server error 601 can have two different causes: either database inconsistencies or a conflict between two database processes that were executed at the same time. See SAP note 565708.

Short Dumps of the type DBIF_RSQL_SQL_ERROR, as evidenced in ST22, may be observed in a badly configured SQL Server system. The details of the short dumps will indicate SQL error 11 and/or SQL error 0. See SAP note 392892 for details.


© SAP AG 2003

Database Consistency Check

Mon 19 Wed 21 Thu 22 Fri 23

Tue 27 Wed 28 Thu 29 Fri 30

Mon 02 Tue 03 Wed 04 Thu 05 Fri 06

Tue 20success

0:00 DB 9:00 Log


0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB 9:00 Log

0:00 DB9:00 Log


Mon 26


successsuccess


DBCC CHECKDB (PRD) WITH NO_INFOMSGS

Hardware problems, such as a defective hard disk controller, may cause inconsistencies in the database. These inconsistencies may only be detected when a table or index affected by the inconsistency is first accessed, or during a consistency check. To prevent problem escalation and possible loss of data, you must identify errors early.

The Database Consistency Checker (DBCC) is a system administration tool that verifies pointers internal to a database and its structure. DBCC commands typically lock user tables, indexes, and system tables whenever they are run. In addition, they are very I/O intensive and should not be run during normal operation.

Run periodic checks to ensure the physical consistency of your data. Use the SAP Planning Calendar to schedule a database consistency check at least once a month during a period of low workload, for example on the weekend. This job executes transact-SQL statement DBCC CHECKDB WITH NO_INFOMSGS on the SAP database.

Check the run status of the consistency checks using the SAP Planning Calendar. If the job is highlighted in red, check for further errors in the SQL Server error log. A file which reports all errors found is written to X:\Program Files\Microsoft SQL Server\MSSQL\CCMS_CHECK_DB_HIST_<year>.txt.

If the run status is green, no inconsistencies were found. The SQL Server error log displays messages: DBCC CHECKDB (<SID>) started at ... with Logfile: X:\ Program Files\Microsoft SQL Server\ MSSQL\ CCMS_CHECK_DB_HIST<year>.TXT.

DBCC CHECKDB (<SID>) executed by <user> found 0 errors and repaired 0 errors.


© SAP AG 2003

Database Allocation Monitor: Consistency Check

To perform a database consistency check, you can also use the Database Allocation Monitor (Transaction DB02). Choose button DBCC checkdb. Do not run a consistency check on the database while production work is carried out in the system.


© SAP AG 2003

Database Allocation Monitor: Consistency Check on a Table

The Database Allocation Monitor (Transaction DB02) also enables you to perform a consistency check on a single table. Choose button Detailed Analysis and specify the table to be checked. In the following screen, choose button Check Table.

A DBCC CHECKTABLE WITH NO_INFOMSGS is executed on the database for the specified table. This checks the integrity of the data, index, text, ntext, and image pages for the specified table.


© SAP AG 2003

Database Inconsistencies

DBCC CHECKDB(PRD) executed by <user> found X errors and repaired 0 errors...

DBCC CHECKDB(PRD) executed by <user> found X errors and repaired 0 errors...



Books Online

Error log messages

Job 'SAP CCMS Check Database PRD [20021213174935]' : Step 1, 'Step1' : Began Executing 12/13/02 6:49:31 PM...

Job 'SAP CCMS Check Database PRD [20021213174935]' : Step 1, 'Step1' : Began Executing 12/13/02 6:49:31 PM...

SAP Notes

If an inconsistency has been detected during the DBCC run, you must take immediate action to prevent further error escalation and possible data loss. Check the output of the DBCC check immediately after each run.

If an inconsistency is detected, the job scheduled with the SAP Planning Calendar is highlighted in red. The SQL Server error log shows the following message: DBCC CHECKDB(<SID>) executed by <user> found X errors and repaired 0 errors

Log file X:\ Program Files\Microsoft SQL Server\MSSQL\CCMS_CHECK_DB_HIST<year>.TXT reports all inconsistencies found.

If an error occurs, create a problem message immediately. Enter the full error text found in the log files in the problem description. In most cases, you can correct the inconsistency without data loss. Depending on the type and location, you may be able to correct the error by removing an index on a table, or by restoring the database.

For a detailed description of errors, see Books Online or SAP Notes.


© SAP AG 2003


...


Books Online

SQLServer

Error 1105: Could not allocate space...

Database <SID>cannot be opened

Error 8651: Could notperform the requested operation because ...

No Connection to SQL Server

SAP Notes

If the SAP System cannot connect to SQL Server, the SAP Service Manager displays a grayed-out dispatcher symbol. Check the developer trace files located in directory X:\usr\sap\<SID>\DVEBMGS<instanceno.>\work for an error description, as follows: A suspect SAP database. In this case error Error 945: Database <SID> cannot be opened because some of the files could not be activated appears. Similar errors occur in the SQL Server error log and the Event Log. See Unit 5: Database Restore.

A suspect SAP database. Error 1105: Could not allocate space for object <object> in database <SID> because the PRIMARY filegroup is full occurs. The status column in table sysdatabases is set to suspect if SQL Server is unable to complete recovery on a database because the disk drive no longer has free space. Execute stored procedure sp_resetstatus <SID>, provide additional disk space and restart SQL Server. See SAP note 81692 for details.

Not enough memory. Error 8651: Could not perform the requested operation because the minimum query memory is not available occurs. Change SQL server configuration parameters as explained in Unit 3: Performance Monitoring and Tuning.

If the problem cannot be solved, create a problem message immediately. Enter the error message shown in the developer trace file. Books Online or the SAP Notes may contain a detailed description of the error. SAP Note 98678 discusses connection problems.


© SAP AG 2003



Check for errors and solve common problems

Monitor the growth of the database and the transaction log

Extend the database:

Add a disk to an existing volume

Add a new volume

Expand data files

Perform regular administrative tasks


© SAP AG 2003


ADM100 myTechnologyAdministration

SQL Server Books Online

SAP Notes

Best Practices CCMS Monitoring for mySAP


© SAP AG 2003

Unit Actions

Exercises?

Solutions


Exercises

Unit: Regular Maintenance and Error Handling Topic: Space Management / Database Extension


• Monitor the growth of the database and transaction log

• Handle a full transaction log

• Extent a database on a new volume

The customer detects that the transaction log of his SAP database fills up rapidly and monitors its growth. As soon as the transaction log is full, he has to resolve the situation.

The SAP database runs out of space on the volume it resides on. The customer has to extent the database on a new volume.

1-1 Transaction log full

1-1-1 Open a Query Analyzer window and select the test database. Execute the stored procedure fill_log. This stored procedure will fill the transaction log by adding rows to the stores table. Monitor the growth of the transaction log. What error message is displayed when the transaction log is full? Where else can you see the error message?

1-1-2 Why did the transaction log not grow dynamically? Check the properties of the transaction log and its file.

1-1-3 What can be done to resolve the situation? Are the transaction log file properties set up correctly? If not, change them.

1-1-4 Display the log used space again, and check the correctness of the transaction log backup.

1-2 Database full, and extending the database on a new volume

1-2-1 Set the recovery model to simple on the test database. Open a Query Analyzer window and select the test database. Enter the command fill_db and execute it. This stored procedure will fill the database by adding rows into database table stores. Monitor the growth of the database. Execute fill_db again and check the growth. What do you observe?

1-2-2 What error message is displayed when the database is full?

1-2-3 Extend the test database on a new volume by 4 MB. Choose C:\temp as the new location. Which steps do you have to perform? How can you extend the database and equally distribute the load across the two volumes?


Solutions

Unit: Regular Maintenance and Error Handling Topic: Space Management / Database Extension

1-1 Transaction log full

1-1-1 Monitor the growth of the transaction log of the test database during the execution of stored procedure fill_log. In the Enterprise Manager expand the server and the Databases folder, then mark database test. Select the Tools → Taskpad to display the space usage of database test. Selecting the refresh button or the F5 key refreshes the display. The Query Analyzer displays the following message: Server: Msg 9002, Level 17, State 2, Procedure fill_log, Line 12 The log file for database 'test' is full. Back up the transaction log for the database to free up some log space. Use command DBCC SQLPERF(LOGSPACE) to display the log used space. It should display a rate of about 90%. Error message 9002 is displayed in the SQL Server error log and in the Event Viewer.

1-1-2 The properties of the test database can be displayed with the Enterprise Manager. Select database test, right mouse-click and choose Properties. In the Transaction Log Tab, the log file is displayed, with its allocated size and the file properties. The transaction log file should be set to grow automatically. But the file growth is restricted. Alternatively, you can use the SAP Database Allocation Monitor under Tools → Administration → Monitor → Performance → Database → DB02 Table/Indexes (transaction DB02) to display the properties of all the databases on the database server. Choose Other DB → DB list and select the test database. The column Limit displays the maximum growth.

1-1-3 Back up the transaction log of the test database. Use either the Enterprise Manager or the Query Analyzer. Make sure to append the backup to the existing backups on backup medium R3DUMP3. See the exercises in Unit4: Database Backup. The transaction log file of the SAP database should grow by at least 100 MB and there should be enough space left on the disk, on which it resides. Let the transaction log file of the test database grow to at least 3 MB. Change the properties in the Enterprise Manager.

1-1-4 Use command DBCC SQLPERF(LOGSPACE) to display the log used space.


1-2 Database full, and extending the database on a new volume

1-2-1 Go to the Properties window of the test database and set the recovery model to simple. Choose OK. The transaction log of the test database will now be truncated every time a checkpoint occurs. Monitor the growth of database test during the execution of stored procedure fill_db. In the Enterprise Manager, expand the server and the Databases folder, then mark database test. Choose View → taskpad to display the space usage of database test. Selecting the refresh button or the F5 key refreshes the display. After a few executions of fill_db you see that the first data file is expanded automatically. All insert operations will now take place in this first data file. As soon as this data file is full the second file is expanded and insert operations take place in this file. The proportional fill strategy is no longer used.

1-2-2 When the database is full, the Query Analyzer displays the following message: Server: Msg 1105, Level 17, State 2, Procedure fill_db, Line 15 Could not allocate space for object 'testtable' in database 'test' because the 'PRIMARY' filegroup is full. The SQL Server error log shows the same error message. Transaction DB02 shows the sizes of the data files and the used sizes of 4MB each.

1-2-3 The goal of extending the database is to distribute future insert operations equally across the two volumes. Therefore the database has to be extended on both volumes. Perform the following steps: Close all connections to database test and detach the test database from the server, using the transact-SQL command sp_detach_db ‘test’, ‘true’ Move database data file F:\mssql7db\MSSQL\data\TestData2.ndf to C:\temp. Re-attach database test using transact-SQL command sp_attach_db ‘test’, ‘F:\mssql7db\MSSQL\\data\TestData1.mdf’, ‘C:\temp\TestData2.ndf’ Execute sp_helpfile in the test database to check the result. Extend both data files by 2 MB each. Use the Enterprise Manager or transact-SQL commands. Make sure that you change the maximum file size the data files are allowed to grow as well. Execute the stored procedure fill_db again to observe how the database allocated new space proportionally on both files.


© SAP AG 2003


Section II - EWB20




© SAP AG 2003

SECTION II

EWB20Microsoft SQL Server SQL Cache Analysis

SAP R/3 Release 4.0 2002/Q2 Material Number: 5005 6776


© SAP AG 2002

Expensive SQL statements affect all users

The SAP Stored Procedure Statistics can be used to:Find and analyze expensive SQL statements

Identify performance issues that are not related to hardware configuration or SQL Server parameter settings

Performance issues must be reported to the appropriate people

Introduction

Database problems are often caused by expensive statements. An expensive SQL statement does not only affect the user running the ABAP program containing the statement. It affects other users as well, since shared database resources are heavily used by expensive statements.

An analysis of the SAP Stored Procedure Statistics shows up expensive statements. Further the statements can be analyzed and important information for the improvement is obtained.

With the SAP Stored Procedure Statistics, you can identify performance issues that are not related to hardware configurations or SQL Server parameter settings. This course does not cover hardware, and it only covers a few parameter modifications.

Performance issues must be reported to the appropriate people or groups, such as: Developers (coding, table, and index design) Application Designers (customizing and user requirements) SAP (standard coding, standard index, and table design)

Although specific transactions can be analyzed using other monitoring tools, such as SQL Trace, these tools are normally used to find specific problems that affect a small percentage of users. These other monitoring tools are not covered in this course.


© SAP AG 2002

SQL Server architecture basics

SAP Stored Procedure Name Cache and Statistics

SQL statement analysis

Workflow and reporting

Overview

This workshop begins with a review of the SQL Server architecture basics, and then moves on to analyzing SQL statements in the SAP Stored Procedure Name Cache.

Several optimization possibilities are discussed, such as: Creating, changing, and dropping indexes Changing database view definitions Changing the access to pooled and clustered tables Modifying Matchcodes

This workshop provides templates for recording and reporting expensive SQL statements. These templates are to be used by the: System administrators who are monitoring the system Developers who design the data model and develop the programs Application department that customizes the application Users who are processing the reports


© SAP AG 2002

MS SQL Server 7.0 architecture review

Processing of SQL statements

Contents:

Introduction and Technical Background

Once you have completed this unit you will be able to: Describe how SQL statements are processed by SQL Server


© SAP AG 2002

Introduction and Technical Background -Objectives

Describe how SQL Server processes an SQL statement

At the end of this unit you will be able to:


© SAP AG 2002

MS SQL Server Process

Memory area

Database files

MS SQLServer

msdb tempdb <SID>master

R/3 work processes

Data cacheData cache Locks,Locks,Connections,Connections,

Open objects,Open objects,

Procedure cache Procedure cache Temp. objectsTemp. objects

….….

SQL processes: ThreadsBackground TaskBackground Task

Lock ManagerLock ManagerLazy WriterLazy WriterLog WriterLog Writer Generic RefresherGeneric Refresher

Alert EngineAlert Engine

Checkpoint ManagerCheckpoint Manager

1:n

Database connections: Threads

Data files

Log files

sp c

rea

sp c

rea

sel s

ngl

sel s

ngl

com

m rd

com

m rd

unc

rdun

c rd

unc

rdun

c rd

....

...

MS SQL Server: Overview

Microsoft SQL Server is a Relational Database Management System (RDBMS). SQL Server consists of: Database server, or Microsoft SQL Server Databases: More than one database is administered by a database server. A SQL Server database consists of at least two files, one log file and one data file.

The SQL Server memory area contains data structures that allocate memory in SQL Server.The main types of objects allocated in this area are the data cache, the procedure cache, the log cache and memory space for temporary objects. This memory area is defined using parameters “min server memory (MB)” and “max server memory (MB)”.

The SQL Server is the operating system process on Windows NT called SQLSERVR which represents the active, programmed side of the database system. All management of the database system is performed through and by the server and all communication with the database systems goes through the server.

The basic unit of activity in Windows NT is the thread. Each process can have multiple threads but has at least one. All threads can run concurrently. Each connection between a client and SQL Server uses one of these threads. The number of threads is defined using parameter „max worker threads“. One thread serves several connections if the number of connections is higher than the number of threads (thread pooling).


© SAP AG 2002

Special Threads

Lazy Writer

Log Writer

Lock Manager

Checkpoint Manager

Background Task

A certain number of buffers in the data cache needs to be free for immediate use. If this number of free buffers is below a certain limit, the Lazy Writer thread (which is started periodically) writes dirty (changed) pages to disk and puts the buffers to the free list. User threads check this free buffer list also before reading data from disk into the buffer, and write dirty pages to disk if the free buffer list is too small.

Special log caches are used to store transaction log entries until a transaction is committed. When the commit is executed, the Log Writer thread flushes the log caches to disk.

Checkpoints occur if explicitely issued, if the recovery interval needs to be assured, and during shutdown. The Checkpoint Manager scans the data and log cache for dirty pages and writes to disk any page that are marked as dirty. Checkpoints typically find few dirty pages to write to disk because most dirty pages get written to disk by the user threads or Lazy Writer thread in the period between two checkpoints.

The Lock Manager administrates the lock resources. The Background Task checks every 30 minutes if the database or the transaction log files can be shrunk, in case the database option autoshrink is set. It also starts every 5 seconds and checks on 20 data pages if they contain ghost records. These are records which are logically deleted but not yet physically.


© SAP AG 2002

Applicationserver

DBIF

R/3 Work process

R/3 Work process

SQL Server

Databaseserver

01234567.. 01234567..

READ COMMITTED

READ UNCOMMITTED

0 Consistent transactions1 DDL transactions2 Select single3..N Dirty read selects

dbs/oledb/add_procs = X-> X+3 connections(default = 8)

Database Connections to R/3 Work Processes

Several connections are used for each R/3 work process, with the following isolation levels: Connection 0: Committed read connection, used for consistent transactions involving inserts, deletes, updates, and select for update or database cursor usage. (No auto-commit mode.)

Connection 1: Uncommitted read connection, used for all DDL transactions and for creating stored procedures. (Works in auto-commit mode.)

Connection 2: Uncommitted read connection, used to execute single selects and dirty read cursors. (Works in auto-commit mode.)

Connection 3...N: Uncommitted read connection, used for dirty read selects. (Works in auto-commit mode.)

The number of connections established for each work process, other than connections 0-2, is determined by the R/3 instance profile parameter dbs/oledb/add_procs. These connections are opened as needed but only closed when shutting down the instance or restarting the work process. A select using this connection occupies the connection during the execution and it is impossible to issue another command on this connection. Thus a new connection is needed for each nested select. If the select is nested in too many surrounding selects (more than parameter dbs/oledb/add_procs), cursors are used in connection 2. There is a minimum of 5 and a default recommendation of 8 dirty read connections.

Each connection to the database uses approximately 60KB of memory.


© SAP AG 2002

Memory Area

Files

Transaction

DataLog

SQL Server Query Analyzer - [Query - sapprod.PRD.sapr3 - (untitled) - #1 New query] x_File Edit View Query Window Help

DB: PRD

x_

begin tranupdate MARA set MATNR = ‘XX-CB589’ where MATNR = ‘XXCA589’commit tran

. . . . . .

11

2 3

Logging and Checkpoints

A database transaction is a series of SQL commands that are either completed, or not executed at all. A transaction is started with the commands BEGIN TRAN and ends with COMMIT or ROLLBACK. All commands within one transaction are handled as one atomic command.

When a transaction is executed all changes made to the database are recorded in the transaction log. When a record is either inserted, updated or deleted, a new log record is created. The Undo and Redo information for all transactions is stored in the log file, and kept until the next backup.

The changes to the record and their log record are first sent to pages in the data cache and the log cache (1).

The SQL command COMMIT ends a transaction. The Log Writer writes all logging information from the log cache to the log file (2). Then the SQL Server sends a message to the application program confirming the COMMIT. This ensures that a log of all confirmed changes from SQL Server is logged on a physical disk.

When a checkpoint occurs all dirty pages of the data cache are written to the data and log files (3). Using the server configuration parameter „recovery interval“, the number of automatic checkpoints can be set. The parameter recovery interval defines the maximum number of minutes the SQL Server needs to restore the database in case of failure.


© SAP AG 2002

ABAP Database Statements and Database SQL Library Interface

ABAP (Open SQL) statement

Native SQL statement

Stored Procedure Name & Parameters

select * from MARA where MATNR = ‘123456’.endselect.

select * from MARA where MATNR = ‘123456’

YG0000064B69RC5522MARA ‘123456’

Stored Procedure Text select * from MARA where MATNR = @P000

An Open SQL statement in an ABAP program is always executed as a Stored Procedure in SQL Server. Stored Procedures consist of:

- a name - a stored procedure text (which is mainly the native SQL statement text) - a set of parameters which specify field values in the native SQL statement

All this is generated by the Database SQL Library Interface (DBSL IF). In addition SQL Server creates an execution plan, which is stored in the procedure cache.

Once the DBSL interface has generated the native SQL statement, the values for the fields in the where clause are replaced by parameters. As a consequesnce, a stored procedure can be reused also with another set of parameters. The stored procedure is created in the R/3 database or in the tempdb database.

Only the stored procedure name and the parameters are transferred to SQL Server when the stored procedure is executed.


© SAP AG 2002

R/3 Work process

SELECT * FROM MARA

WHERE ...

DBSL IF

Database

RDBMSData Cache

Procedure Cache

MemoryArea

SQL Connections

Thread 3Thread 2

Thread 1

Indexes

YG0000064B69RC5522MARA

Stored Procedures

MARA~1

Create YG0000064B69RC5522MARA

MARATables Indexes

YG0000064B69RC5522MARAStored Procedures

MARA~1

1a

Appl. Server

SAP Stored ProcedureName Cache

1b

Statement Processing (Prepare Step)

During the first execution of an SQL statement in a newly generated ABAP/4 program, a new stored procedure is generated and stored in the database. The Database SQL Library Interface (DBSL IF) generates the stored procedure name and the native SQL statement text, which is passed to SQL Server, where a stored procedure is created (1a). The stored procedure name is also transferred to the SAP Stored Procedure Name Cache (1b). Once the stored procedure name is in the SAP SP Name Cache, stored procedure name and text are not generated before the execution.

Stored procedures are reusable SQL commands, compiled in an execution plan in the procedure cache. The DBSL IF creates a unique stored procedure name for the Open SQL command.

There are two kinds of stored procedures created by the DBSL IF: Permanent stored procedures are stored in the R/3 database and are not deleted after restarting SQL Server or the R/3 instance.

Temporary stored procedures are stored in database tempdb and are deleted when the last process that used them terminates, for example when the R/3 instance is shut down, after a restart of a work process, or after restarting SQL Server. Temporary stored procedures come from dynamic Open SQL statements, for example FOR ALL ENTRIES.

In fact there is a set of common statements (and stored procedures) for each table. These statements are reused in several programs and are only created once.


© SAP AG 2002

R/3 Work process

SELECT * FROM MARA

WHERE ...

DBSL IF

Database

RDBMSData Cache

Procedure Cache

MemoryArea

SQL Connections

Thread 3Thread 2

Thread 1

Indexes


Stored Procedures

MARA~1

Exec YG0000064B69RC5522MARA

MARATables Indexes


MARA~1

Appl. Server


44

2

3

Statement Processing (Execution Step)

The stored procedure is then forwarded to SQL Server for execution.

When the stored procedure is executed, SQL Server first looks through the procedure cache to see if there is an existing execution plan for this SQL statement. SQL Server reuses any existing plan it finds, saving the overhead of recompiling the SQL statement. If there is no existing execution plan, SQL Server generates a new execution plan for the query (2).

Once the execution plan of the stored procedure is in the procedure cache, the procedure can be executed. The execution plan determines which indexes will be used for the table accesses (3).

SQL Server starts to transfer the needed parts of the used indexes and tables are transported from the disk into the data cache for further processing (4).

It is much faster if the pages needed are already in the data cache from previous usage. The statement execution is slowed down by a factor of around 100 if disk access is necessary.


© SAP AG 2002

R/3 Work process

SELECT * FROM MARA

WHERE ...

DBSL IF

Database

RDBMSData Cache

Procedure Cache

MemoryArea

SQL Connections

Thread 3Thread 2

Thread 1

Indexes


Stored Procedures

MARA~1

Fetch

MARATables Indexes


MARA~1

Appl. Server


5

Statement Processing (Fetch Step)

The transfer of data from the data cache back to the work process is done in fetches (5).

Via the database interface the work process is supplied with data via a buffer. The size of this buffer is 40kByte. One fetch can transfer exactly the amount of data which fits in this buffer.

If the result contains more rows than this buffer can hold, more fetches are done, until all rows are transferred to the work process.


© SAP AG 2002

YR...

exec YR...

in cache

R/3 Work Process

DBSL IF

create Y7R6…exec Y7R6...

exec Y7R6...Temporary

storedprocedure

namecache

In namecache

no yes

R/3 Application Server

Permanentstored

procedurenamecache

Stored Procedure Name Cache

The size of the Permanent Stored Procedure Name Cache is defined by R/3 instance profile parameter dbs/oledb/pn_cache_size (number of procedure names, that can be stored). Only names of permanent stored procedures are stored here.

The size of the Temprorary Stored Procedure Name Cache is defined by the R/3 instance profile parameter dbs/oledb/tsp_cache_size. The names and bodies of temporary stored procedures as well as arguments of both stored procedure types are stored here.

Before creating a permanent stored procedure the Database SQL Library Interface (DBSL IF) checks if the unique stored procedure name already exists in the permanent stored procedure name cache. If the stored procedure name is not found in the name cache, it is created on database <SID> before it is executed. If the stored procedure name is found in the name cache, it is executed directly.

For temporary stored procedures the DBSL IF checks if the stored procedure body (stored procedure text) exists in the temporary stored procedure name cache. If the stored procedure body is not found in the name cache, it is created on database tempdb before it is executed. If the stored procedure body is found in the name cache, it is executed directly.

The stored procedure name caches of each R/3 application server provides some additional space to store statistical information about the execution of stored procedures. These are explained in detail in Unit 2: Introduction to the Stored Procedure Name Cache.


© SAP AG 2002

tempdb

##Y7DRESDENGAL0047000027100934##Y7DRESDENGAL0047700528106935##Y7DRESDENGAL0047000027100934##Y7DRESDENGAL0047700528106935##Y7DRESDENGAL0047000027100934##Y7PARISGAL0047700765528107036##Y7PARISGAL0047000027580100939##Y7PARISGAL0047700528106000936...

<SID>

Y7R61000068BKI3757MONIY7R200000015BKI3757MONIY7A0000051787LG0732SDB1FMSSY7A000000B9B822D0010SAPLSTAMY7C0000094197624847DBSYFMSSY7R400000015BK998757MARAY7A0000015BKI89H930489MARAY7A000000016BKI89H93347MARD...

Stored Procedure Names

Naming convention of permanent stored procedures: Y7<ModuleCharacter><StatementId><Timestamp><ModuleName> <ModuleCharacter> identifies the type of module the statement comes from, for example, A for ABAP report

<StatementId> is usually the same as the statement id in the R/3 System <Timestamp> usually specifies the generation time of an object, for example, the generation time of an ABAP report

<ModuleName> is derived from the module name given in the statement id, for example the table name or report name

Naming conventions for temporary stored procedures: ##Y7<Hostname><SID><ProcessId><Counter><Timestamp> <Hostname> is the name of the application server <SID> identifies the R/3 database <ProcessId> displays the process id by which it is created <Counter> indicates an application-specific counter which is incremented <Timestamp> usually specifies the generation time of the ABAP report


© SAP AG 2002

Summary

Describe the SQL Server architecture

Describe how SQL Server processes an SQL statement

You are now able to:

Now you are able to: Describe the most important parts of the SQL Server architecture with R/3. Describe how SQL statements are processed by SQL Server.


© SAP AG 2002

Introduction to the SAP Stored Procedure Statistics

Expensive Statements

Database Performance Monitor (Transaction ST04)

Contents:


© SAP AG 2002

Introduction to the SAP Stored Procedure Statistics - Objectives

Explain the impact of an expensive statement

Explain how the SAP Stored Procedure Name Cache works and how statistics are collected

Explain the information displayed in the SAP Stored Procedure Statistics


Once you have completed this unit you will be able to: Explain the impact of an expensive statement Explain how the Stored Procedure Name Cache works and how statistics are collected Explain the information displayed in the Stored Procedure Statistics


© SAP AG 2002

R/3 Work process

Exec YG0000064B69RC5522MARA

Data

RDBMSData Cache

Procedure Cache

Thread 3Thread 2

Thread 1

DataData

SQL connections

Memory Consumption

Consumptionof CPU

resources

Physical IOLoad

Expensive SELECT Statements

The important resources on the database server that are limited are the CPU capacity, the amount of memory, and the number of physical I/O operations that can be performed per time interval.

An R/3 system shows agood performance, if frequently accessed data are buffered in the data cache. The size of the data cache is limited by the amount of memory. While threads are processing the requests, CPU resources are used also.

Improper or unnecessary use of the data cache results in displacements of other data pages, and affects overall system performance. A high amount of displacements of data pages that are accessed frequently results in a high amount of disk I/O activitay and will decrease disk performance.

When a statement (stored procedure) is submitted for example from an R/3 work process, data pages, which are used to execute this query have to be located in the data cache. The amount of data cache, which is used by a statement execution, depends on: the number of rows in the cache which have to be analyzed to find the requested rows

The execution of a statement, which causes heavy load on the database, usually takes also a high amount of time. Siince SQL Server does not provide statistics about resource consumption per statement, the execution time is used to estimate the load of the statements.


© SAP AG 2002

Definitions

Data Cache Hit Ratio:The hit ratio for data pages in the data cache

Request Buffer Pages:Total number of data pages accessed by statements

SQL batches:Number of SQL batches (statements) submitted from R/3 work processes via the database interface

Physical Reads:Total number of data pages read from disk

The following performance indicators are displayed in Database Performance Monitor (ST04): Data Cache Hit Ratio is the cache hit ratio for the data cache. This ratio is the percentage of data pages already found in the cache of the total number of pages read.

Procedure Cache Hit Ratio is the cache hit ratio for the procedure cache. Request Buffer Pages is the total number of requested pages in the data cache by statements. SQL batches is the total number of statements transmitted via the database interface to SQL Server.

Physical Reads are the total number of pages read from disk. All values are counted since SQL Server startup.


© SAP AG 2002

Performance Indicators

Database Performance Analysis: SQL Database Overview

DB analysis Goto Monitor System Hilfe x_

Refresh Reset Since reset Totals per second Detail analysis menu

Microsoft SQL Server 7.00 - 7.00.699 (Intel X86) SAP release 40BWindows NT 4.0 (Build 1381: Service Pack 4, RC 1.132)Database PRD Time of analysis 12/07/1999 05:56:13DB Server prdserv DB startup 12/06/1999 11:43:32CPUs available for SQL Server 4 of 4 Pysical memory MB 1,023Trace flagMemory UsageCurrent memory KBMaximum memory KBMemory settingProcedure cache KB

hit ratio %

316,304 865,896

AUTO26,231

99

Total SQL connectionsR/3 connections

Free buffersData cache size KB

hit ratio %

4443

125277,045

99

Total data size MBFree space MB

9,0003,843

Total log size MBFree space MB

1,4641,405

Space Usage

Server Engine/Elapsed secs 61,962 ( 17:12:42 )CPU busy secCPU idle sec

IO busy sec

6,053241,795

47

SQL RequestsSQL batchesRead ahead pagesRequest buffer pages

readswrites

199,37422,764

1,664,44132,278

101

Full table/index scansIndex range scansIndex searchesProbe scansLazy write

16197,813

522,889118,493

0

1 > 6h

1 > 500

2 > 95

Physical readswriteserrors

16,0816,093

0

3

To display the most important performance indicators of the database, call Transaction ST04, or choose Tools → Administration → Monitor → Performance → Database → Activity. For a successful analysis, the database must be running at least for several hours with a typical R/3 workload. To ensure a significant database workload, we also recommend a minimum of 500 CPU busy seconds (1).

The data cache hit ratio (2), which is the main performance indicator for the data cache, shows the ratio of requested data pages found in the cache on average. This gives a snapshot of a short period of time before the analysis. The value should always be above 95% (even during heavy workload). To check the history of these values, use Transaction ST04 and choose Detail analysis menu → Performance database. A snapshot is collected every 2 hours.

The procedure cache hit ratio (3), which is the main performance indicator for the procedure cache, shows the ratio of requested pages and found pages in the procedure cache on average.As for the data cache this gives a snapshot of a short period of time before the analysis. The value should always be above 95%. A history is also available in the performance database.

SQL Server dynamically adjusts the size of the data and the procedure cache. If both cache hit ratios are very small, it might be an indication that SQL Server does not have enough memory. Then it is recommended ton increase SQL Server configuration parameters max server memory (MB) and min server memory (MB).


© SAP AG 2002

A data cache hit ratio of 95% or greater does not mean the cache is large enoughA frequently executed expensive SELECT statement can increase the data cache hit ratio because it significantly increases the number of successful page reads

ExpensiveSELECT on table VBBE

Other

Tables

Data Cache

VBBE

Examine the Data Cache Hit Ratio

The hit rate of the data cache depends on many factors. A hit rate of 95% or higher does not necessarily mean that the data cache is large enough.

Consider the following situation: One expensive SELECT statements is executed often. The data pages of the accessed tables reside in the data cache and are always read from the data cache. Other, less expensive statements may read from the disk but do not harm the data cache hit ratio because only a few pages have to be read from the physical disk. This type of statement increases the overall data hit ratio.If this expensive statement is tuned and reads less data, the data cache hit ratio may even drop.

The data pages accessed for such an expensive statement use a large fraction of the data cache. Therefore, the size of the data cache that can be used for all the other tables is significantly reduced. Disk accesses are performed to read data pages for statements to the other tables, which could otherwise be read from the data cache.

If the expensive SELECT statement is tuned, significantly fewer data pages are read from the cache. Data pages used by other statements can now be buffered in the data cache. This can even cause the data cache hit ratio to drop. However, the space available for all tables increases and the performance increases for all statements as there are fewer disk accesses.


© SAP AG 2002

Stored Procedure Statistics

Stored Procedure Statistics is a SAP tool

Stored Procedure Statistics information is located nearby the Stored Procedure Name Cache on each applicationserver

Statistical values are collected since R/3 instance startup

The SQL Server performance indicators can be used to analyze the overall status of a SQL Server. In addition to these performance indicators provided by SQL Server, SAP provides a tool which collects and displays statistics based on stored procedures, this means related to statements.

Stored Procedure Statistics is a data collection about stored procedures based on execution times. These statistics are collected on each application server, even if more than one instance is running on one server.


© SAP AG 2002

How Statistics Collection Works?

SQL connections

Database system

Application Server

Stored procedurename caches

Y720000001613C5941TBTCO

Y7R00000005BKH4024TSP01

##SAPPRODPRD0023805727##SAPPRODPRD1100160746

Statistical info

Switch On / OffStatisticsCollection Display

Statistics

Thread 3Thread 2

Thread 1

R/3 Work processR/3

Work processR/3 Work process

The stored procedure name cache of each R/3 application server provides some additional space for storing statistical information.The statistics collection must explicitly be turned on to gather statistical information about the stored procedures.

Call the Database Performance Monitor (Transaction ST04). Choose Detail analysis menu → SAP stats on SPs. On a central system the name cache statistics are displayed. Otherwise click on button Name cache stats to display the name cache statistics of the server you are logged on. To display the name cache statistics of all servers, press the radio button all servers before Name cache stats.

In the first row a message reads Statistics are not being collected or Statistics are being collected if the statistics collection is switched off or on. Choose Set statistics → Set statistics → On/Off to switch it on/off. You must wait until at least 3 hours have passed before evaluating the collected statistics.

Using R/3 instance profile parameter dbs/oledb/stats_on, the statistics collection can be turned on automatically, when the instance is started. The value must then be set to 1, which is recommended.

Up to R/3 Release 4.6A, the external program msstats.exe is used to monitor the stored procedure name cache. From R/3 Release 4.6A, this function is implemented in the R/3 Kernel. msstats.exe can be run in a command prompt window if no interface has been implemented in R/3.


© SAP AG 2002

Name Cache Information

12

3

The first line shows the R/3 instance name (1), if the statistics collection is switched on (2) and when the name cache has been reset last (3).

The following information is displayed for the permanent / temporary stored procedures: Number of Slots: Slots available for stored procedure names. This is defined by the R/3 instance profile parameter dbs/oledb/pn_cache_size / dbs/oledb/tsp_cache_size.

Used Slots:Used slots for stored procedure names. Slot Replacements: Shows the replacements of the names in the cache after the cache was full. The replacements are accumulated since the instance startup or the statistics last reset , whichever is the most recent.

The following additional information is stored in the temporary stored procedure name cache: Number of slots: Stored procedure bodies and arguments. (dbs/oledb/tsp_cache_size * 2 slots) TSP bodies: The number of bodies for temporary stored procedures which are currently kept in the cache.

PSP/TSP arguments: The number of arguments of permanent/temporary stored procedures which are currently kept in the cache.

Total used slots: The number of currently used slots for PSP/TSP arguments and TSP bodies.


© SAP AG 2002

Calling the Stored Procedure Statistics

12

To display the Stored Procedure Statistics, use Transaction ST04 → Detail Analysis → SAP Stats on SPs (1). On an R/3 system with only one application server the statistics about the usage of the permanent and temporary stored procedures names (Name Cache Information) is displayed. On an R/3 system with more than one application server an additional selection screen lets you choose between the Name Cache Information and the Stored Procedure Statistics on this server and on all servers.

To display statistics on stored procedures, press SP Statistics. A selection screen lets you enter selection criteria for the stored procedures. Be sure the button Show dependency is deselected because of a longer runtime of the analysis. This

option is not important at this point. Leave the screen blank (it can be filled also) and execute (2).


© SAP AG 2002

Stored Procedure Statistics

Stored procedure statistics

Stored procedure name

Download System Hilfe x_

Object Total timeStatement Number call Averag Max Min Tot runtm&fet

Y7G00000E6E8BIL1335D010L 1 UPDATE D010L 1211 812131 631 843 31 916878 930 904 40 Y7E000019048BIL1321D010LINF 1 UPDATE D010LINF 8 45879 5343 7896 4324 78909 9098 9904 8907 Y7A000009CD83IF0519SAPLSMPI 3 DELETE SDBAC 207 12131 58 430 31 16878 82 704 40 Y7R6100006877GC3752D342L 2 SELECT D342L 6 456 456 456 456 566 566 566 566 Y7R6100006877GC3755D345T 1 SELECT D345T1 1290 465 465 465 465 500 500 500 500Y7R400000017BPC3746RSMPTEXTS 1 SELECT SINGLE. 1 450 450 450 450 490 490 490 490 Y7I000033BC8BIL1228DDNTTut01 3 INSERT DDNTT 1 411 411 411 411 411 411 411 411Y7R6180008878RM3303EUDB 2 SELECT EUDB1 1 456 456 456 30 566 566 566 42 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 1 456 456 456 15 966 566 566 42 ##Y7SAPPRODPRD0032900067173545 1 SELECT MARA 1 856 456 456 34 1566 566 566 42 ##Y7SAPPRODPRD0032900068174567 1 SELECT MARC 1 3456 456 456 38 566 566 566 42 ##Y7SAPPRODPRD0032900065174125 1 SELECT MARD 1 750 456 456 80 576 566 566 42 ##Y7SAPPRODPRD0032900019183545 2 SELECT MARA 1 650 456 456 30 566 566 566 42 Y7A0000010081GD0019SAPLSTUW 1 SELECT D342L 1 356 456 456 30 368 566 566 42 Y7E000012B18BIL1321D010SINFut01 1 SELECT D010SINF 1899 56 456 456 30 66 566 566 42

Avg Max Min

Sp entriesNumber of callsCursor used

601545

7

SummaryRuntime without fetch (ms)

MaximumMinimumAverageTotal

789631

510827

Runtime without fetch (ms)


990440

9321.637

Rows returned


1.0421

551.212

Total runtime of maximum ms call 884528 Total rows from maximum ms call 1094420

No.

Two sections are shown in the stored procedure statistics. A list of the selected stored procedures in the name cache with their number of calls, total, minimum, maximum and average runtimes

A summary section at the bottom of the screen. Two times are measured for every stored procedure execution:

duration without fetches duration with fetches

In addition the duration and the nuber of fetched rows of the call with the longest runtime is stored. The number of pages read or written is not displayed as this information is not known at the R/3 level.

The display is discussed in more detail in the following slides.


© SAP AG 2002

Stored Procedure List




Object Total timeStatement Number call Averag Max Min Tot runtm&fet

Y7G00000E6E8BIL1335D010L 1 UPDATE D010L 1211 812131 631 843 31 916878 930 904 40 Y7E000019048BIL1321D010LINF 1 UPDATE D010LINF 8 45879 5343 7896 4324 78909 9098 9904 8907 Y7A000009CD83IF0519SAPLSMPI 3 DELETE SDBAC 207 12131 58 430 31 16878 82 704 40 Y7R6100006877GC3752D342L 2 SELECT D342L 6 456 456 456 456 566 566 566 566 Y7R6100006877GC3755D345T 1 SELECT D345T1 1290 465 465 465 465 500 500 500 500Y7R400000017BPC3746RSMPTEXTS 1 SELECT SINGLE. 1 450 450 450 450 490 490 490 490 Y7I000033BC8BIL1228DDNTTut01 3 INSERT DDNTT 1 411 411 411 411 411 411 411 411Y7R6180008878RM3303EUDB 2 SELECT EUDB1 1 456 456 456 30 566 566 566 42 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 1 456 456 456 15 966 566 566 42 ##Y7SAPPRODPRD0032900067173545 1 SELECT MARA 1 856 456 456 34 1566 566 566 42 ##Y7SAPPRODPRD0032900068174567 1 SELECT MARC 1 3456 456 456 38 566 566 566 42 ##Y7SAPPRODPRD0032900065174125 1 SELECT MARD 1 750 456 456 80 576 566 566 42 ##Y7SAPPRODPRD0032900019183545 2 SELECT MARA 1 650 456 456 30 566 566 566 42 Y7A0000010081GD0019SAPLSTUW 1 SELECT D342L 1 356 456 456 30 368 566 566 42 Y7E000012B18BIL1321D010SINFut01 1 SELECT D010SINF 1899 56 456 456 30 66 566 566 42

Avg Max MinNo.

In the list of stored procedures the following information is displayed: stored procedure name: name of temporary or permanent stored procedure No: the number of application servers where this stored procedure is executed Statement Type: the type of Open SQL statement which is executed, e.g. SELECT, UPDATE, INSERT, DELETE, SELECT FOR UPDATE, SELECT SINGLE

Object: the object name(s) the stored procedure accesses Number call: the number of calls of this stored procedure since the statistics are switched on [Total | Avg | Max | Min] time: the [total | average | maximum | maximum ] runtime of this stored procedure without fetch

[Total | Avg | Max | Min] runtime&fetch: the [total | average | maximum | maximum ] runtime of this stored procedure with fetch

[Total | Avg | Max | Min] rows: the [total | average | maximum | maximum ] rows returned of this stored procedure

SP cursor: displays ‚C‘/‚N‘ if a database cursor was used / not used during the longest call (Max time)

Row count: number of rows returned by this stored procedure during the execution of the longest call (Max time)

Rf ms: runtime with fetch during the execution of the longest call (Max time)


© SAP AG 2002

Summary



Sp entriesNumber of callsCursor used

190041545

67

SummaryRuntime without fetch (ms)


78960

510890427

Runtime with fetch (ms)


99040

9321947637

Rows returned


1.0421

551.212

Total runtime of maximum ms call 884528 Total rows from maximum ms call 1094420

Total Runtime with fetch =

Database Load

In the summary list of stored procedures the following information is displayed: SP Entries: number of stored procedures for which statistics are collected. Number of Calls: the number of calls for all stored procedures for which statistics are collected Cursor used: the number of stored procedures for which statistics are collected and which used a cursor during the longest execution

For the Runtime without fetch (ms): the [total | average | maximum | maximum ] runtime without fetch of all stored procedures which are displayed

For the Runtime with fetch (ms): the [total | average | maximum | maximum ] runtime with fetch of all stored procedures which are displayed

For the Rows returned: the [total | average | maximum | maximum ] rows returned of all stored procedures which are displayed

Total Runtime of maximum ms call: total runtime of all stored procedures for which statistics are collected during the execution of the longest call (Max time)

Total Rows from maximum ms call: total number of rows returned by all stored procedures for which statistics are collected during the execution of the longest call (Max time)

Since the logical reads are not collected in the stored procedure name cache we consider the total runtime with fetch as a unit for the total database load. Note that this load is only measured since statistics collection is switched on.


© SAP AG 2002

Name Cache InformationReplacements are set to 0TSP / PSP arguments are deleted

Stored Procedure StatisticsAll statistical information about execution times is is removed

Reset of Statistics

A reset of the statistics is done via ST04, SAP stats on SPs, Name Cache Stats (only if more than one server is available), and then in the menu Set Statistics -> Set statistics ON/OFF.

The reset will set the replacements and the arguments in the Name Cache Information. For the Stopred Procedure Statistics all measured values for the stored procedures are deleted.

A switch on/off does not reset the statistics. If the last R/3 instance on an application server is stopped, the statistics are also reset.


© SAP AG 2002

Explain how the Stored Procedure Name Cache works and how statistics are collected

Explain the information displayed in the Stored Procedure Name Cache / Statistics



Now you are able to: Explain the impact of an expensive statement Use the Database Performance Monitor to analyze the Stored Procedure Name Cache Statistics Explain how the Stored Procedure Name Cache works and how statistics are collected Explain the information displayed in the Stored Procedure Name Cache Statistics


© SAP AG 2002

Analyzing the Stored Procedure Statistics


Recognizing expensive statements in the Stored Procedure Statistics

Contents:


© SAP AG 2002

Determine if and why a statement is expensive

Using the Stored Procedure Statistics

Understanding the time measurement


Analyzing the Stored Procedure Statistics -Objectives

This unit describes how to: Recognize expensive SQL statements in the Stored Procedure Statistics Understand the time measurement Use the Summary List

Once you have completed this unit, you will be able to use the analysis tools to determine if and why statements are expensive.


© SAP AG 2002

SELECT MANDT, VBELN, OBJNR FROM ZVBAK WHERE MANDT = 001 AND ERNAM = ERNIE

Search area(the data that is

searched through for the requested data)

Records specified by the WHERE-clause

MANDT VBELN OBJNR ERNAM

001 000013245 654875454 ERNIE 001 000013246 697935435 FRED 001 000013247 673225747 JAMES 001 000013248 216356543 GUSS001 000013249 687535435 ERNIE001 000013250 258674802 KURT001 000113245 983231365 MAX 001 000113251 522112486 BERT001 000113252 325348654 BERT002 000013249 241567685 HUGO002 000013250 968351332 FRED002 000013251 874352321 MAX 002 000013252 544546546 GUSS

Selection ListFROM-clause

WHERE-clause

Expensive SQL Statements

A SELECT statement is divided into a selection list, which specifies the columns which should be retrieved from the table, the FROM-clause, which specifies from which table(s) data should be selected and the WHERE-clause, which determines whether a row should be included into the result set.

The search area is the set of data (records) that are searched to evaluate the query. This set is not explicitly specified in the query. It is determined by the optimizer when the statement is evaluated. The search area has to be searched to find the requested data in the WHERE-clause of the statement.


© SAP AG 2002

Expensive SQL Statements - Classification

Type 1: The access path is unsuitableSymptom: A lot of data is read but only a few records are returned

Type 2: The access path is suitable Symptom: A lot of data is read and many records are returned

An expensive SQL statement reads a lot of data, i.e. the search area is large. The goal of tuning expensive SQL statements is to reduce the search area as much as possible so that less data has to be read.

When an expensive SQL statement is found, classify the statement first, to find the possible tuning methods.

There are two different types of statements: Type 1: Statements search through a lot of data for only a few requested records. Here the access path is unsuitable.

Type 2: Statements request a lot of records. Here the access path is suitable.


© SAP AG 2002

Type 1: The access path is unsuitableSymptom: A lot of data is read but only a few records are returned

SELECT MANDT, VBELN, OBJNR FROM ZVBAK WHERE MANDT = 001 AND ERNAM = ERNIE



Records specified by the WHERE clause



Expensive SQL Statement - Type 1

In this example, the application requests the records of table ZVBAK where MANDT (the client) equals 001 and the field ERNAM (the name of person who created the record or object) is 'ERNIE'. Many records are read, although only two are requested. Therefore, this statement is of type 1. The access path is unsuitable to retrieve the result set.

To accelerate this type of statement, you can: Change the ABAP code, for example specify additional fields in the WHERE clause so that an existing index is chosen by the optimizer, read the data from a different table where an appropriate index exists, or avoid reading the data.

Create a new index Extend an existing index

Caution: The performance of other statements may suffer if an index is dropped or extended.


© SAP AG 2002

SELECT MANDT, VBELN, OBJNR FROM ZVBAK WHERE MANDT = 001



Records specified by the WHERE clause



Type 2: The access path is suitable Symptom: A lot of data is read and many records are returned

Expensive SQL Statement - Type 2

In this example, the application requests all records of table ZVBAK where MANDT = 001. Because the database returns all the records that are read, this statement is of type 2. The access path is suitable to return the requested data.

A statement that returns many records cannot be accelerated by using an index. To accelerate this type of statement, you can: Tune the ABAP report Tune the business process Optimize the user input


© SAP AG 2002

Database server

Application server

runtime without fetch = 0runtime with fetch = 01

Memoryarea Data Cache

Thread 1

ProcessSQLSERVR

runtime without fetch = 5000 ms

2runtime with fetch = 9000 ms4

3R/3 Work Process

Exec YG0000064B69RC552ZVBAK

Time Measurement

Before analyzing statements in the Stored Procedure Statistics, the measurement of the runtimes has to be explained in detail.

If a stored procedure is executed the time measurement of the stored procedure execution starts when the R/3 work process sends the execution command to the database server (1). When the execution on the database server is successfully started and the first rows of the result set are placed in the data cache, the database server returns a confirmation back to the application server and the runtime without fetch ends. Next an IO buffer of the size of 32 KB is created on the application server and formatted to retrieve the results from the database server (3). The database server sends the results to the application servers IO buffer, until the result set is transferred completely. If the whole result set is sent back, the runtime with fetch ends (4). Then the DBSL cursor and the IO buffer is removed. This is the case if 0, 1 or more than 1 rows are returned. Between two fetches data in the IO buffer are transferred to the R/3 user buffer.

Note that the runtime measured includes also the time spend transferring data over the network.

DBSL cursors are data structures that are maintained within the DBSL, i.e. these are client-side cursors and should not be mixed up with server-side cursors. If a server-side cursor was used, this is indicated in the ST05 trace of SQL statements as well as in the Stored Procedure Statistics (column SP cursor: A server-side cursor shows the entry ‘Y’, a client-side cursor shows the entry ‘N’). Usually the execution of server-side cursors is slower than with client-side cursors.


© SAP AG 2002

Runtime per Row is high (Type 1)

Pages readRequested record

average runtime per row is high (>= 10 ms)

Application server

Memoryarea

Data pages

Statements for which the average runtime per row is high

Have high consumption of system resources during execution

Have a large search area

The statements that have a high average runtime with fetch per row usually have a large search area. These statements use an unsuitable access path.

If there are lock wait situations when the table is accessed, the runtime is also high, but in this special case the access path must not be unsuitable necessarily.

These statements may be improved by reducing the search area to a minimum.


© SAP AG 2002

Statements with a high number of maximum rows per execution may use suboptimal logic

Example:ABAP coding:

SELECT * FROM MARA.CHECK MARA-MATNR = 12345.... ENDSELECT.

High number of Rows (Type 2)

Pages readRequested record

Application server

Memoryarea

Data pages

Statements with a high maximum number of rows are expensive but use a suitable access path. They may be coded inefficiently.

In this example, the complete table MARA is read from the database to the application server when only one record is required.


© SAP AG 2002

Find expensive statements

Check total database loadSummary section, Runtime with fetch (ms), Total value

Check load of single SP’s (statements)Sort the SP statistics for Total Runtime with fetch, check the highest values

Calculate the relative load of a single stored procedure Calculate the ratio of Runtime with fetch (ms), Total value and Total runtime with fetch of a single SPstatement is expensive and must be investigated, if the relative load is higher than 5%

Before you analyze the Stored Procedure Statistics, check if the statistics are running long enough and if the database has processed enough statements to make the analysis meaningful. See Unit 2: Introduction to the Stored Procedure Name Cache for details.

Then call the Stored Procedure Statistics in the Database Monitor (Transaction ST04). Mark the radio button All servers (not available on a central system) and click the button SP Statistics. Leave all fields in the filter screen free and click execute. In section Summary, box Runtime with fetch (ms), line Total the database workload is displayed (in ms).

The list is sorted by the column Total runtm & fetch. The value in this column represents the load of the statement executed by the stored procedure. The statements at the top of the list cause the highest consumption of database resources.


© SAP AG 2002

Restrict the list output to expensive statementsCalculate 5% of the total database load in msGo back to the selection screen, and enter this value into the field Total runtime including fetch >= and execute

Restrict your analysis to SELECT Statements (see column Statement type)

Check if the statement is type 1 or 2Check the column Total rows, divide Total runtime with fetch by Total rows for each statement. If the runtime per row is higher than 10 ms, the statement is of type 1 (unsuitable access path).


Use the following criteria to decide which statements to analyze for unsuitable access paths: The total runtime (Total runtm&fetch) is more then 5 % of the total database load. Restrict analysis to SELECT statements. Total runtm&fetch / Total rows ≥ 10 ms (average runtime per row is high)

If the difference between the Total Runtime and the Total Runtime & Fetch is very high, it might be an indication that either a lot of rows are returned or the network time is very high.


© SAP AG 2002

Display the statement of a stored procedure




Object Total timeStatement Number call Averag Max Min Tot runtm&fet Avg Max MinNo.Y7G00000E6E8BIL1335D010L 1 SELECT D010L 1211 812131 631 843 31 916878 930 904 40 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 87 45879 5343 7896 4324 78909 9098 9904 8907 Y7A000009CD83IF0519SAPLSMPI 3 SELECT MARC 2070 12131 58 430 31 16878 82 704 40 Y7R6100006877GC3752D342L 2 SELECT D342L 6 456 456 456 456 566 566 566 566 Y7R6100006877GC3755D345T 1 DELETE D345T1 1290 465 465 465 465 500 500 500 500Y7R400000017BPC3746RSMPTEXTS 1 SELECT SINGLE. 1 450 450 450 450 490 490 490 490 Y7I000033BC8BIL1228DDNTTut01 3 INSERT DDNTT 1 411 411 411 411 411 411 411 411Y7R6180008878RM3303EUDB 2 SELECT EUDB1 1 456 456 456 30 566 566 566 42 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 1 456 456 456 15 966 566 566 42 ##Y7SAPPRODPRD0032900067173545 1 SELECT MARA 1 856 456 456 34 1566 566 566 42 ##Y7SAPPRODPRD0032900068174567 1 SELECT MARC 1 3456 456 456 38 566 566 566 42 ##Y7SAPPRODPRD0032900065174125 1 SELECT MARD 1 750 456 456 80 576 566 566 42 ##Y7SAPPRODPRD0032900019183545 2 SELECT MARA 1 650 456 456 30 566 566 566 42 Y7A0000010081GD0019SAPLSTUW 1 SELECT D342L 1 356 456 456 30 368 566 566 42 Y7E000012B18BIL1321D010SINFut01 1 SELECT D010SINF 1899 56 456 456 30 66 566 566 42

1

2

Once a expensive stored procedure is found in the Stored Procedure Statistics, the statement text and the parameters executed by the stored procedure can be displayed. See Chapter Introduction and Technical Background for details.

Mark the stored procedure (1), and press the Single detail button (2).


© SAP AG 2002

Display the Statement Summary List (1)




Object Total timeStatement Number call Averag Max Min Tot runtm&fet Avg Max MinNo.

Y7G00000E6E8BIL1335D010L 1 SELECT D010L 1211 812131 631 843 31 916878 930 904 40 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 87 45879 5343 7896 4324 78909 9098 9904 8907 Y7A000009CD83IF0519SAPLSMPI 3 SELECT MARC 2070 12131 58 430 31 16878 82 704 40 Y7R6100006877GC3752D342L 2 SELECT D342L 6 456 456 456 456 566 566 566 566 Y7R6100006877GC3755D345T 1 DELETE D345T1 1290 465 465 465 465 500 500 500 500Y7R400000017BPC3746RSMPTEXTS 1 SELECT SINGLE. 1 450 450 450 450 490 490 490 490 Y7I000033BC8BIL1228DDNTTut01 3 INSERT DDNTT 1 411 411 411 411 411 411 411 411Y7R6180008878RM3303EUDB 2 SELECT EUDB1 1 456 456 456 30 566 566 566 42 ##Y7SAPPRODPRD0032900066174555 1 SELECT MARA 1 456 456 456 15 966 566 566 42 ##Y7SAPPRODPRD0032900067173545 1 SELECT MARA 1 856 456 456 34 1566 566 566 42 ##Y7SAPPRODPRD0032900068174567 1 SELECT MARC 1 3456 456 456 38 566 566 566 42 ##Y7SAPPRODPRD0032900065174125 1 SELECT MARD 1 750 456 456 80 576 566 566 42 ##Y7SAPPRODPRD0032900019183545 2 SELECT MARA 1 650 456 456 30 566 566 566 42 Y7A0000010081GD0019SAPLSTUW 1 SELECT D342L 1 356 456 456 30 368 566 566 42 Y7E000012B18BIL1321D010SINFut01 1 SELECT D010SINF 1899 56 456 456 30 66 566 566 42

1

2

3

Now that the expensive statements are found in the Stored Procedure Statistics, they have to be analysed in detail to find out why they re expensive. Mark the statements you want to analyse, e.g. the top 5 in the Stored Procedure Statistics after sorting after the Total Runtime (1). To display a single statement, double-click the statement.

Click the icon Summary List (2). A pop-up window appears with two checkboxes. Mark both checkboxes and press execute (3). The summary list appears.

Use the statement summary list to document your findings. Download the list and store it in a Word or Access Document. This is explained in chapter Workflow and Reporting.


© SAP AG 2002

-------------------------------------------------------------------------------------------------------------------------------------------| stored procedure name | stmt type | object | n_call| total_ms | avg_ms | max_ms | min_ms | rf_total_ms| rf_avg_ms| rf_max_ms|-------------------------------------------------------------------------------------------------------------------------------------------| Y7A000005A296UF3751SAPLZFPC | | SAPLZFPC| 68 | 49.050.706| 721.333| 1175672| 54.875| 49.051.388 | 721.343 | 1175.688 || Y7I00003447974L1335DDNTF | | DDNTF | 5.989 | 60.385| 99| 1.767| 15| 8.343.964 | 4.296 | 33.906 || ...-------------------------------------------------------------------------------------------------------------------------------------------| Summary for 5 entries: | 6.060 | 54.994.999| 1321068| 4003360| 15| 73.196.369 | 3305.331 | 7202.094 |-------------------------------------------------------------------------------------------------------------------------------------------

Stored procedure details:-------------------------------------------------------------------------------------------------------------------------------------------| stored procedure name | stmt type | object | n_call| total_ms | avg_ms | max_ms | min_ms | rf_total_ms| rf_avg_ms| rf_max_ms|-------------------------------------------------------------------------------------------------------------------------------------------| Y7A000005A296UF3751SAPLZFPC | | SAPLZFPC| 68 | 49.050.706| 721.333| 1175672| 54.875| 49.051.388 | 721.343 | 1175.688 |-------------------------------------------------------------------------------------------------------------------------------------------

name: Y7A000005A296UF3751SAPLZFPC object: SAPLZFPC type: ABAP timestamp: 19990630153751parameter: @P000 varchar(3)

@P001 varchar(4)@P002 varchar(25)@P003 varchar(25)

Statement:SELECT "MANDT" AS c,"BUKRS" AS c,"BELNR" AS c,"GJAHR" AS c,"BLART" AS c, ...FROM "BKPF"WHERE "MANDT" = @P000 AND "GJAHR" = @P001 AND "BKTXT" BETWEEN @P002 AND @P003

Explanation:SELECT "MANDT" AS c,"BUKRS" AS c,"BELNR" AS c,"GJAHR" AS c,"BLART" AS c, ...FROM "BKPF" WHERE "MANDT" = ' ' AND "GJAHR" = ' ' AND "BKTXT" BETWEEN ' ' AND ' 'SELECT|--Clustered Index Seek(OBJECT:([P01].[dbo].[BKPF].[BKPF~0]), SEEK:([BKPF].[MANDT]=[@1]), WHERE:([BKPF].[BKTXT]>=[@

3] AND [BKPF].[GJAHR]=[@2] AND [BKPF].[BKTXT]<=[@4]) ORDERED)PLAN_ROWest.rows: 1 est.io: 0.01 est.cpu: 0.00

Table statistics-------------------------------------------------------------------------------------------------------------------------------------------| table name | reserved KB | data KB | index KB | unused KB | total rows | min row sz | max row sz | row mod ct | schema ver |-------------------------------------------------------------------------------------------------------------------------------------------| BKPF | 2.204.288 | 1144.496 | 1101.544 | 41.752- | 2.788.281 | 38 | 560 | 6.986 | 8 |-------------------------------------------------------------------------------------------------------------------------------------------

Statement Summary List (2)

The first part of the statement summary list displays the statistical information from the marked stored procedures.

The next part shows the name of the stored procedure, the object it accesses or the object from which it is executed, the type of the stored procedure and the timestamp of the creation of the stored procedure on the database.

Next the execution plan of the stored procedure is displayed. This execution plan is displayed in the procedure cache on SQL Server. The details of the execution plan is explained in Unit 5: Explain the Execution Plan.


© SAP AG 2002

Index details-------------------------------------------------------------------------------------------------------------------------------------------| index name | Auto| IDXhi| Updated | index description | index keys -------------------------------------------------------------------------------------------------------------------------------------------| BKPF~0 | ON | 3 | Jul 21 1999| clustered, unique, primary key located on PRIMARY | MANDT, BUKRS, BELNR, GJAHR | BKPF~1 | ON | 4 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, BUKRS, BSTAT, XBLNR | BKPF~2 | ON | 4 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, BUKRS, BSTAT, BUDAT | BKPF~3 | ON | 3 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, BUKRS, BSTAT, BLART | BKPF~4 | ON | 4 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, AWTYP, AWKEY, AWSYS | BKPF~5 | ON | 4 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, BUKRS, CPUDT, BSTAT | BKPF~6 | ON | 4 | Jul 21 1999| nonclustered located on PRIMARY | MANDT, BUKRS, BLDAT, BSTAT | _WA_Sys_TCODE_73A521EA | ON | 0 | Jul 21 1999| nonclustered,statistics,auto create located on PRIMARY | TCODE | _WA_Sys_BLART_73A521EA | ON | 0 | Jul 21 1999| nonclustered,statistics,auto create located on PRIMARY | BLART | _WA_Sys_BUDAT_73A521EA | ON | 0 | Jul 21 1999| nonclustered,statistics,auto create located on PRIMARY | BUDAT | _WA_Sys_GJAHR_73A521EA | ON | 0 | Jul 21 1999| nonclustered,statistics,auto create located on PRIMARY | GJAHR | ...|-------------------------------------------------------------------------------------------------------------------------------------------

Statistics for INDEX 'BKPF~0'.Updated Rows Rows Sampled Steps Density Average key length==================== =========== ============ =========== ==================== ====================Jul 21 1999 10:00PM 2788281 52249 299 0.0 0.0All density Columns=============================

1.0 MANDT1.3333334E-2 MANDT, BUKRS3.6171872E-7 MANDT, BUKRS, BELNR3.5946874E-7 MANDT, BUKRS, BELNR, GJAHR

...

Statement Summary List (3)


© SAP AG 2002

Recognize an expensive statements

Analyze the Stored Procedure Statistics

Understand the time measurement




Recognize expensive statements Analyze the Stored Procedure Statistics Use the Explain function Use the ABAP Dictionary


© SAP AG 2002

Where-used lists

Contents:

Identify Coding


© SAP AG 2002

Identify the lines of code where a statement is issued

Find programs using the Global Work Process Monitor and SQL Processes Monitor

Describe the differences between ABAP OPEN SQL statements and SQL statements


Identify Coding - Objectives

Once you have completed this unit you will be able to: Identify the lines of code where a statement is issued Find programs using the Global Work Process Monitor and the SQL Processes Monitor Describe the differences between ABAP OPEN SQL statements and SQL statements


© SAP AG 2002

Yes

No

Identified SQL

statementIn

Where-usedlist

Use Transaction SM66to find the process and

program used

Use the SQL Processes Monitorto verify that the expensive SQL statement is processed

Search the Where-used

listWorkflow

communicationsprocess

Identify statement

in code

Roadmap for Finding SQL Statements in Programs

Use this roadmap to find the code responsible for an expensive SQL statement.


© SAP AG 2002

Where-used list

Identify Coding

Program listing

Stored ProcedureSingle detail

Where-Used List

Once a Stored Procedure running an expensive SQL statement is found in the Stored Procedure Statistics, the ABAP program needs to be analyzed.

To find the SQL statement and the table accessed by the stored procedure, use the “Single detail” button.

To find the ABAP program, use the ABAP Dictionary (Transaction SE12). To display a list of all the programs using a specific table, use Transaction SE12. Enter the name of

the table, and choose Utilities Where-used list. To display only the relevant locations, you can click on Search Area in the following action and

specify select in the field for ABAP key words and a search string that specifies the statement more closely, for example a certain table field that is specified in the where clause, or, as in this example, the string order by if the database statement contains an order by.

You can schedule the Where used list as a background job and view the generated spool list later on, or you can start the search online. If you start the search online you can navigate to the source code by a double click. However, for frequently used tables the where used list may take a while to complete.


© SAP AG 2002

Use the ABAP Editor to display the program attributes

The program attributes contain the user name of who created and last changed a program

Identify CodingFind the Program Developer

To find the program developer, use ABAP Editor (Transaction SE38). Enter the program name, and choose Display Goto Attributes.

If a program was created and last modified by SAP, search for OSS Notes containing the program or table name using the search terms:

<Table> OR <Program> OR <Transaction> AND Performance OR Index If a program was created or modified by one of your developers, contact the developer about

optimization possibilities. Use the templates provided for the communication process.


© SAP AG 2002

Statements not in the Where-used List

A statement cannot be found in the Where-used list, for example, if:

The code is generated at program runtime

The SQL statement is from an Native SQL program line

Important fields in the WHERE clause are not specified because internal tables used are empty

Use the Global Work Process Monitor and the SQL Processes Monitor instead

A statement cannot be found in the Where-used list, for example, if: The code is generated at program runtime The SQL statement is from an Native SQL program line Important fields in the WHERE clause are not specified because internal tables used are

empty Use the Global Work Process Monitor (Transaction SM66) and the SQL Processes Monitor to find

the code. If the total runtime is very high, the probability that the statement is executed while the system is monitored is high.


© SAP AG 2002

Using the Global Work Process Monitor

To monitor if the table for which the expensive statement is issued is currently accessed, use the Global Work Process Monitor (Transaction SM66). The Global Work Process Monitor displays the user that issues the statement, the report, and the transaction that is processed.

To display only the work processes that are currently accessing a specific table, choose Select process, and enter the table name in the dialog box displayed.

To display the table and program name, choose Settings and deselect Display only abbreviated information, avoid RFC.

The Global Work Process Monitor displays only the table for which a SQL statement is issued. To verify that the expensive statement you are looking for is processed, use the SQL Processes Monitor.


© SAP AG 2002

Using the SQL Processes Monitor

To display the SQL Processes Monitor, use the Database Monitor (Transaction ST04) and choose Detailed analysis menu SQL Processes. All SQL processes are displayed, representing the connections between the R/3 work processes and SQL Server.

The monitor connects the PID of the work process (coloumn Host pid) to the SQL statement that is processed using one of the connections (SQL processes).

For a better overview select in the Menu Process Monitor Grouped output. The SQL Processes (connections) are grouped by the work process PID.

One of the connections to the work process shows up “runnable” in the coloumn “Status” and a for example “SELECT#” in the coloumn “SQL command”. This shows if the statement you are looking for is currently processed.

If the statement currently processed is the expensive statement you are looking for, the report displayed in the Global Work Process Monitor must be analyzed.

NOTE: The expensive statement you are looking for can be used in several programs.


© SAP AG 2002

Differences in ABAP Open SQL and SQL Statements

ABAP Open SQL statement is different than the SQL statement in the Shared SQL Area

This happens, for example, when you use:Internal tables (IN ITAB)

The command FOR ALL ENTRIES

Projection views

The ABAP OPEN SQL statement may not resemble the SQL statements displayed in the Shared SQL Area. This happens, for example, when: Internal tables are used to build the WHERE clause The ABAP command FOR ALL ENTRIES is used in the OPEN SQL statement Data is selected from projection views. If this occurs, the SELECT statement to the view is converted to a SELECT statement to the underlying table.


© SAP AG 2002

Selectionscreen

ABAP OPEN SQL:

SELECT * FROM BKPFWHERE BUKRS IN ITAB_1AND BELNR IN ITAB_2AND GJAHR IN ITAB_3AND BLART IN ITAB_4AND BLDAT IN ITAB_5AND BUDAT IN ITAB_6.ENDSELECT.

SQL statement:

SELECT … FROM"BKPF"

WHERE "MANDT" = @P000 AND "BUKRS" = @P001AND ( ( "BELNR" BETWEEN @P002 AND @P003

OR "BELNR" BETWEEN @P004 AND @P005 ) OR "BELNR" IN ( @P006 , @P007 , @P008 ) OR "BELNR" >= @P009 OR NOT "BELNR" = @P010 )

Multiple range selection

Statements using Internal Tables

Internal tables, also called ranges tables, can be used to build the WHERE clause of an SQL statement. The internal table contains values and operators for the WHERE clause. Multiple values and operators for one table field are possible (ITAB_1 .. ITAB_6 in the example).

If one of these internal tables does not contain any values, the corresponding field will not be specified in the WHERE clause of the SQL statement, although it appears in the WHERE clause of the ABAP OPEN SQL statement.

In this example, the WHERE clause of a statement to table BKPF is built by ranges tables. Only the ranges tables for the field BUKRS and BELNR are filled. Field BUKRS is specified with an EQUAL condition and field BELNR is specified with two BETWEEN conditions, an IN condition, a GREATER OR EQUAL condition, and a NOT condition linked with the OR operator. All the internal tables for the other fields in the OPEN SQL statement are empty and do not appear in the SQL statement.


© SAP AG 2002

Internal Table Handling: FOR ALL ENTRIES

Open SQL statements using FOR ALL ENTRIES:SELECT * FROM DBTAB FOR ALL ENTRIES IN ITAB

WHERE BUKRS = 'X' AND BELNR = ITAB-BELNR.

are resolved by the SQL Server database interface as single executions of:

SELECT … FROM "DBTAB"

WHERE "MANDT" = @P000 AND "BUKRS" = @P001 AND "BELNR" = @P002

for every entry in the internal table ITAB.

OPEN SQL statements using the command FOR ALL ENTRIES for an internal table generate a set of SQL statements. For each entry in the internal table ITAB the Stored Procedure is executed once.

Generally this leads to frequently called temporary Stored Procedures. If you want to find frequently called SQL statements / Stored Procedures in ABAP programs, look

for OPEN SQL statements using the command FOR ALL ENTRIES.


© SAP AG 2002

SQL Statements to Projection Views

SQL call to table KBED

Fields selectedfor statement

Selection criteriafor access

Identify Coding

View fields

This example shows an expensive statement to table KBED, which was found in the Stored Procedure Statistics. Using the Where-used list for table KBED and reviewing all ABAP programs, no program could be found that contained this statement. The fields in the SELECT clause are a subset of the fields of table KBED. No statement was issued in any of the programs where these fields were specified in the SELECT clause.

When there is a subset of the table fields specified in the SELECT clause of a statement, check if any projection views are associated with the table. If a statement to a projection view is processed, the database interface converts the OPEN SQL statement to the projection view to a SQL statement to the underlying table. In the SELECT clause of the SQL statement, only the fields contained in the projection view are specified.

To check if a projection view with the fields of the SELECT clause exists, use the Where-used list for the table in views. Display the views, and compare the view fields with the SELECT clause.

The statement to the view is only converted to a statement to the table if the view is defined in the ABAP Dictionary as a projection view and not as a database view.

In this example, the SELECT clause of the SQL statement to table KBED matched the view fields of KBED_AVCHK, which is defined as a projection view. A Where-used list for the projection view KBED_AVCHK showed the program that caused the expensive SELECT statement.


© SAP AG 2002

The ABAP report RSSTATUS displays the application area for tables, transactions, programs, or data elementsContact the development department for the application area

Report RSSTATUS

Find the Application Area for a Statement

If you cannot find the statement, use the ABAP report RSSTATUS. This report shows the application area for tables, transactions, programs, or data elements.

Contact the development department of the application area, and analyze the statement with the developer responsible for this area.


© SAP AG 2002


Identify the lines of code where a statement is issued

Find programs using the Global Work Process Monitor and SQL Processes Monitor

Describe the differences between ABAP OPEN SQL statements and SQL statements



© SAP AG 2002

Optimizer Statistics and Explain Plan

Index Statistics

Column Statistics

Explain Plan

Contents:


© SAP AG 2002

Describe the recommended update statistics strategy for the R/3 database

Explain the statistical information stored for each index

Explain what an explain plan is

At the end of this unit, you will be able to:

Optimizer Statistics and Explain Plan - Objectives

Once you have completed this unit you will be able to: Describe the recommend update statistics strategy for the R/3 database Explain the statistical information stored for each index Explain what an explain plan is


© SAP AG 2002

Update Optimizer Statistics

Why?The optimizer uses index statistics to find the most efficient execution plan for a SQL statement / Stored Procedure

Statistics must be up to date

How?Automatic create and update index and column statistics

The SQL Server query processor uses statistical information to determine the optimal strategy for query processing. The costs of a query (SQL statement / Stored Procedure) are evaluated.

If the table statistics change significantly, another access path may be chosen, for example: another index is chosen for execution another access operation is chosen (full table scan, index scan)

Therefore current statistical information is necessary to enable SQL Server to find the correct access path.

With SQL Server 7.0 index statistics are created and updated automatically. This is the default setting for SQL Server with R/3.


© SAP AG 2002

Settings for Automatic Statistics

Database options (Database Properties)Auto create statistics

Auto update statistics

Individual settings for tablesAutostats ON / OFF (sp_autostats TABLENAME)

The settings for automatic create and update statistics are set individually for each database. These settings are maintained in the database options.

Database Options can be displayed using R/3 in the Database Monitor (DB02). Changes to the settings can be made using the Enterprise Manager or a Query Analyzer window using the system stored procedure sp_dboption.

For some tables in the R/3 database index statistics are not useful, and therefore statistics creation and update is switched off for these particular tables. Reasons for not maintaining table statistics include:

The tables are changed frequently. Hence, the table statistics cannot be up-to-date. The access path to the data in the table is determined by the application logic. Only one specific

access path is desired that is supported and chosen. Maintaining table statistics would be an overhead.

The tables for which no statistics are maintained are: SAP update tables (VBDATA, VBMOD, VBHDR) Pooled tables Clustered tables ….


© SAP AG 2002

Tables and Indexes Monitor (1)

The index statistics are displayed using the Database Tables and Indexes Monitor (DB02). In this case press the button Detail analysis in the section Tables. Enter a table name. The screen consists of two sections, space allocation section and index section.

Space allocation: Reserved: Sum of Data, Index and Unused Data size: Space allocated by data pages Index size: Space allocated by index pages Unused: Pages allocated but not used Total rows: Number of data rows stored in the table Minimum row size / Maximum row size: minimum / maximum number of bytes of a data row Row modification counter: Rows modified after the last update statistics Statistics schema version: Number of update statistics runs (4-bit counter)

Menu: Showcontig: Display fragmentation info (DBCC SHOWCONTIG) Check Table: Run a consistency check (DBCC CHECKTABLE) Update Usage: Update space allocation data in system tables (DBCC UPDATEUSAGE)


© SAP AG 2002

Tables and Indexes Monitor (2)

In the index section information for each index and column histogram is displayed: Index name: Name of the index on database level AutoUpd: sp_autostats settings for the table IdxHeight: Number of levels in the B*tree of the index Updated on: Date and time of last update statistics Index description: Technical index properties Index keys: Fields used by the index

Menu: Show Statistics: Select an index and the statistics page for this index is shown. Update Statistics: Used to manually update statistics. Selectivity: Distinct value combinations of a given set of fields Density: (number of nonfrequent step values / number of distinct nonfrequent step values) /

number of rows


© SAP AG 2002

Statistical Information (Index)

Overview

Selectivity

Histogram

In the table statistics the following information is stored in 3 sections: Overview

updated: date and time when the statistics were last updated rows: the total number of rows stored in the table rows sampled: the number of rows used to build the statistics steps: number of values in the histogram density: selectivity of the index average key length: average length of a row

Selectivity Selectivity values for the fields in the index in the order they occur in the index.

Histogram Sample values for the first field of the index. This is very often the MANDT field in the R/3 environment.


© SAP AG 2002

Overview

Selectivity

Histogram

Statistical Information (Column Histogram)

If the optimizer needs additional information about the values stored in a single field of a table, a column histogram is created. Histograms are used by the optimizer to determine whether a value occurs often in a field or not. This is important if values are not distributed evenly over the rows.

The name of these column histograms always begins with the string ‘_WA_SYS_’, followed by the field name, and finally a hexcode number.

The overview information is identical with the regular index statistics. In the next section the selectivity of the field is displayed in combination with the fields of the

primary key. This information can be also used by the administrator to decide if and how further indexes should be created.

The histogram values show values stored in the field, their frequency represent the frequency of the values in the tables column.


© SAP AG 2002

Manual Update of Index Statistics

Using transaction DB02, statistics can be updated manually for each table or index. From the detailed table info press Update statistics. In the popup you can select several options: On all indexes / On one specific index: you can select each index or column histogram separately, or update the statistics of all indexes

of the table in one run. Full scan / Sample scan with a given percentage of rows: Enter the percentage of rows to be scanned when the statistics are build. A sample scan with 100

% is equivalent to a Full scan. Warning: Update statistics on a table with more than 100 MB on all indexes takes a while and

consumes a lot of system resources! Manual update statistics is necessary only in rare cases.


© SAP AG 2002

Explain plan

Command

Statement

Explain Plan

The optimizer creates an execution plan for every stored procedure that is stored in the procedure cache. This execution plan is compiled and can be compared with a program that is executed. This execution plan consists of one or more operators, for example a Clustered Index Seek. The Explain plan shows what this execution plan does when executed. For most statements submitted from an R/3 work process this execution plan is relatively simple. Usually only one operator is needed, that is one table is accessed per statement, and only one index is used.

The optimizer uses the index statistics to find the execution plan with the lowest calculated costs. This is shown in the PLAN ROW section (line below the PLAN ROW line) in the explain plan. One PLAN ROW section exists for every operator in the explain plan. The optimizer calculates a forecast, how many rows, I/O cost and CPU cost is expected for an operator of a statement / stored procedure.

See also chapter 8 Cost Evaluation for further details. With a Query Analyzer window this output can be obtained using the SET SHOWPLAN_ALL ON command. For further details see Books Online.


© SAP AG 2002


Explain why index statistics are necessary

Explain what index / column statistics are

Display the Explain Plan for a stored procedure



© SAP AG 2001

Interdepartmental workflow and communication

Contents:

Workflow and Reporting

Once you have completed this unit you will be able to: Record and report your analysis results


© SAP AG 2001

Workflow and Reporting - Objectives

Record and report your analysis results



© SAP AG 2001

Development or application department

System administratorUsers

Performance view Functional view

Information needsSystem resources

Table, index, and query design

Analyze queriesOptimize coding,

indexes, andcustomizing

Optimize input

Index designrecommendations

Workflow Overview

The system administrator monitors the performance of the R/3 System, and can see what impact expensive SQL statements actually have on the system. If the system is well tuned and properly sized, however, the administrator can only analyze the situation, not solve it.

Users have an information need that has to be met. Users have a functional view of the business processes. That is, the information they require must be available at a certain frequency and at certain times. This requires a certain amount of system resources.

Developers have both functional and performance views. They provide the programs that meet the information needs of the users and they are responsible for the data model design.


© SAP AG 2001

Table statistics

Developer

Administrator

IndexesZVBAK~0ZVBAK~1

Update statistics

Create

Index design

Communication

A cost based optimizer uses the table statistics and the values in the WHERE clause to determine which index is used

System Monitoring

Administrator and Developer Responsibilities

We strongly recommend that you nominate a single person to be responsible for monitoring and solving problems with expensive SQL statements.

The database administrator must ensure that the optimizer statistics are up-to-date and performed with sufficient accuracy.

The program developer is responsible for the code of the expensive statement and the table and index design. Therefore, the developer must be contacted if the code has to be changed or if the table and index design have to be changed.

The creation of an index locks the table. Therefore, indexes should be created, that is transported into the production system, by the database administrator, because he or she knows best when the creation of an index does not disrupt the work of online users.


© SAP AG 2001

Log the expensive statements and your efforts!

Statement Documentation and Logging

It is important that you log the expensive SQL statements you found. Copy the expensive statement into the Excel spreadsheet or Access database provided and note important information like: the person responsible the report where the statement originates from important performance information - statement summary list out of the ST04, SP statistics.

possible solutions actions you performed people who were contacted and are involved in solving the problem

For large installations you can easily find many expensive statements that have to be improved. If you do not log your work, it may be lost.


© SAP AG 2001

R

OSSOpens an OSS Call if necessary

System administrator

Log the statements and send a summary to IT management if requested

Expensive Statements

Development or application department

Contact the responsible developer

Tuning the ABAP code is preferred over other tuning methods.

Administrator Responsibilities

Workflow Overview

If an expensive statement is found, the administrator must contact the developer of the statement. If the query originates from a standard SAP report, and there is no OSS Note that solves the problem, an OSS Call should be opened.

An expensive statement must be tuned. To do this, contact the developer of the statement and explain the possible tuning methods you found. Changing the ABAP code or application customizing is the preferred method of tuning statements because other technical tuning methods, such as creating indexes, may impair the performance of other statements.

The system administrator can suggest other tuning methods, which have to be evaluated by the developer of the data model. If the code of the statement cannot be changed, the developer must determine which technical tuning methods should be used, for example creating an index.

After the expensive statement is solved, you have to monitor the result. Check if the statement is not expensive anymore. If the index, table, or view design has been changed, check also if other statements have become expensive.

A summary of the expensive SQL statements can be sent to the IT Management periodically if required.


© SAP AG 2001

OSS Call Template

If the expensive statement originates from a standard SAP report, check the OSS Notes

If an appropriate Note does not exist, open an OSS call using the OSS Call Template

If the expensive statement originates from a standard SAP report, check the OSS Notes for possible solutions. Use the search terms <TABLE> or <Program> together with "performance" or "index”.

If an appropriate Note does not exist, open an OSS call using the OSS Call Template. NOTE: Do not open an OSS call:

For transactions such as SART or SE16, because user input and customizing are usually responsible for slow response times

If user input is responsible for the expensive statement, such as for a field not specified in a selection screen (example:Matchcodes)

For sapdb* programs called from your own reports. These programs are logical databases that may be inefficiently used in your coding.

If the statement is caused by inaccurate or old optimizer statistics If an index is missing (check DB02)


© SAP AG 2001

Users

System administrator

Discuss a possible solution with the users who issue the statement

If User Input is responsible for the expensive Statement contact the Users

Workflow Overview

The administrator can contact the users to see if the statement is really required and to find out when the statement needs to be processed. For example, an expensive statement is less critical if it can be processed in background or during periods of low system activity.

Also, users might be able to optimize their input so that the statement is no longer expensive anymore, for example, when Matchcodes or selection screens are used inefficiently.


© SAP AG 2001

Roles: - Coordination of problem resolution- Establish contacts between parties- Documentation of problem status - Support level agreement definition

UsersApplication Department(s)

Coordinator at Customer

Roles: - Train users - Solve problems in customizing- Redesign business process

Development

Roles: - Monitoring and analysis of expensive SQL statements

- Solve technical problems with DB- Find ABAP Source of problems- Find Users responsible for expensive input- Assist in SQL problem resolution- Document expensive statements- Open SAP problem messages for problems in SAP standard

- Monitor successSystem administrator

Roles: - Solve problems in in-house developments- Change coding- Design indexes to support in-house coding- Monitor own developments inproductive environment

- Verification of solutions

Responsibilities Overview

For the monitoring and resolution of expensive SQL statements, different tasks have to be fulfilled. The system administrator monitors the system for expensive SQL statements, finds the source of the

problem, which could be either the code or the user responsible for input that caused the problems. He solves technical problems, for example outdated optimizer statistics, assists the developers in problem resolution, and opens problem messages to SAP. The system administrator also monitors the success of the statement tuning and documents the expensive statements.

If the system is administered by an outsourcing partner we recommend that a coordinator at the customer site that owns the R/3 System coordinates and documents problem resolution. He establishes contacts between the two parties, outsourcing partner and the customer, if necessary. Also support level agreements are defined by the coordinator representing the customer.

If the system is run by an in-house department the two roles 'Coordinator' and 'System Administrator' can be performed by one person.

The development department is responsible for solving problems with in-house solutions. This involves tuning the coding as well as designing indexes to support in-house coding. If the developer is not familiar with designing indexes, this may be done in close co-operation with the system administrator. We recommend that the developers have a user in the production system to monitor their own programs and verify the success of their solutions for expensive SQL statements.

The application department is responsible for end user training, solving problems in customizing settings and redesigning the business process if necessary.


© SAP AG 2001

Record and report expensive SQL statements to the application or development department




© SAP AG 2002

Index Utilization and Operators

Indexes

Index structure

Operators

Contents:


© SAP AG 2002

What an index is

How an Index is structured

How data is accessed using an index

What operators are

At the end of this unit you will be able to describe:

Index Utilization and Operators - Objectives

Once you have completed this unit, you will know: Types of indexes available in SQL Server What an index is How an index is structured How data is accessed using an index How the optimizer chooses an index


© SAP AG 2002

Indexes

An index can accelerate data access

Different types of indexes are:Primary index (regularly a clustered index)

Secondary index (regurlarly a nonclustered index)

Indexes support data searches in database tables. In the R/3 environment, there are two types of indexes (as defined in the R/3 Data Dictionary): Primary index Secondary index

All R/3 tables have a primary index, which consists of the key fields that the developer must define when creating an R/3 table. An index can be defined as unique if a maximum of one record exists for any combination of fields in the index. The primary index is unique if the key fields must identify each table record uniquely.

For queries where the primary index is not helpful to search the result set of rows, for example, because none of the primary index fields were specified in the WHERE clause, the query processor would scan through the whole table. To avoid this, you can define secondary indexes to significantly reduce the amount of data records searched to determine the result set.

When activated, R/3 indexes are created in the SQL Server database as 2 types of database indexes: Clustered index (the R/3 primary index is usually realized as a clustered index) Nonclustered index (the R/3 secondary indexes are usually realized as nonclustered indexes).


© SAP AG 2002

Index B* tree

Leaf node/Data pages

Intermediatelevel

Root node

Index leaf level= data pages

MANDT VBELN ERDAT ERZET ERNAM ... …001 0000123 04051999 085123 ERNIE ...001 0000124 04051999 085125 PETER ...001 0000125 04051999 085129 HELLA …001 0000126 04051999 085133 HEINZ ...001 0000127 04051999 085139 JACKIE…...002 0000546 01021998 210136 HUGO ......

Root node and Intermediate levels containpointers to the next index level

MANDT VBELN...001 0000123002 0000546003 0000789…

Clustered Index Structure

A clustered index consists of a root page, several levels of intermediate level pages, and finally the leaf level pages.

Leaf level pages of a clustered index contain the whole data row of the table, or in other words, they are the data pages itself. The next higher (intermediate and root) level pages contain one index row per leaf level page combined with a pointer to the next underlying level, and so on, until the index rows to the next underlying level can be stored in one single page, the root page. As a consequence, the number of levels depends on the number of rows in the table and on the size of the index fields.

The rows of a clustered index in all levels are sorted by the indexed fields. Finally the data rows on the data pages (leaf level) are stored in sorted order. This means, only one clustered index can exist on a table, because the data rows on the leaf level can only be sorted in the order of one combination of index fields.

When a query is executed and a data row is searched using a clustered index, the Root page is read first, followed by the appropriate intermediate level pages using the pointers to the next underlying level. Finally the data row is found in the leaf level pages.

When you search for a single row using the clustered index, the number of pages that have to be read equals the number of levels from the root page to the leaf level (=data) page.

In the R/3 environment the number of fields in the clustered index depends on the application demands of the unique constraint, which is defined on the clustered index.


© SAP AG 2002

Index B* tree

Leaf level

Intermediatelevel

Root node

Index leaf levelpoints to clustered index key

MANDT VBELN …001 0092342

001 0048526 001 0000021 001 0002345 001 0101876 ...002 0008926 ...


MANDT VBELN...001 0000123002 0000546003 0000789…

MANDT ERDAT ERNAM...001 04051999 HELLA002 04051999 JACKIE003 04051999 PETER…

Nonclustered Index Structure

A nonclustered index consists also of a root page, intermediate level pages, and the leaf level pages. Leaf level pages of a nonclustered index contain a row locator. This row locator is a clustered index

key of the data row, if a clustered index exists on the table. Like for the clustered index, the index tree consists of intermediate level pages containing index rows and pointers to the next underlying index level. On top of the index tree the root page is located.

The rows of a nonclustered index in all levels are sorted by the indexed fields. When a query is executed and a data row is searched using a nonclustered index, the Root page is

read first, followed by the appropriate intermediate level pages using the pointers to the next underlying level. Finally the clustered index key of the row is found in the leaf level pages. With these key fields the row is searched via the clustered index. This database operation is called a bookmark lookup.

When you search for a single row using the clustered index, the number of pages that have to be read equals the number of levels from the root page to the leaf level, and additionally the pages which have to be read in the clustered index.

Nonclustered indexes can have a unique constraint, but usually in the R/3 environment this is not used.

The number of fields in the nonclustered indexes (secondary indexes in the R/3 Data Dictionary) should be as low as possible. Further the combination of fields should be as selective as possible.

If a nonclustered index contains many fields of the table, the index size is nearly as big as the table itself. As a consequence, this index is not useful.


© SAP AG 2002

Index B* tree

Leaf level

Intermediatelevel

Root node

Index leaf levelpoints to a Row ID

Row ID…...…......


MANDT VBELN...001 0000123002 0000546003 0000789…

MANDT ERDAT ERNAM...001 04051999 HELLA002 04051999 JACKIE003 04051999 PETER…

Nonclustered Index Structure of a Heap Table

In the case no clustered index exists, the row locator is the row identifier of the data row. When a query is executed and a data row is searched using a nonclustered index on a heap table, the

Root page is read first, followed by the appropriate intermediate level pages using the pointers to the next underlying level. Finally the data row is found by the row identifier. This database operation is also called a bookmark lookup.

When you search for a single row using the clustered index, the number of pages that have to be read equals the number of levels from the root page to the leaf level, and additionally the pages which have to be read to find the row locator.


© SAP AG 2002

Frequently used operators for data access:Clustered Index Seek / Scan (Insert / Update / Delete)

Index Seek / Scan (Insert / Update / Delete)

Bookmark Lookup

Table Scan (Insert / Update / Delete)

Other operators:Nested Loop, Merge Join, Hash Match

Filter

Concatenation

Constant Scan / Compute Scalar

Stream Aggregate (sum, avg, count, …), Sort

Top

Operators used by the Query Processor

Operators used by the query optimizer are of two types: Operators which are used for data access directly Operators which combine the result sets of data access operators in a more complicate access path

Clustered Index Seeks and Index Seeks use the corresponding index types, a Bookmark Lookup uses a row locator to find the corresponding data page for a nonclustered index page, and a Table Scan uses simply no index to access all data of a table.

Nested Loop, Merge Join and Hash Match are used for statements which access multiple tables and are explained in the chapters Views and Joins.

A Filter is used to reduce a given result set according to a filter condition. A Concatenation is used for example to combine the result sets of several or conditions in the where clause.

The Stream Aggregate operator is used to compute aggregate functions, the Sort operator sorts a result set by a given condition.

The Top operator cuts the result set after a given number of rows is returned.


© SAP AG 2002

SELECT * FROM ZVBAK WHERE MANDT = '001' AND VBELN = '0000123'

Clustered Index(Primary index)

MANDT VBELN001 0000121001 0000122001 0000123001 0000124001 0000125001 0000126001 0000127001 0000128001 0000129002 0000001002 0000002002 0000123010 0000004

MANDT VBELN ERDAT ERZET ERNAM ... ...001 0000123 04051999 085123 ERNIE ...001 0000124 01021998 210136 HUGO ...002 0000123 13061999 102145 JACKIE ......

~ 3 pages accessed

Clustered Index Seek

The rows of a clustered index and finally the data rows are sorted by the index fields. Operations using this index try to use the following mechanism to search data rows: Search the first data row using the index tree which matches the where clause Start scanning the data rows in the leaf level of the index, as long as the given values in the WHERE clause match the values in the key fields.

Due to the unique constraint of the index a maximum of one row can be returned, if all all key fields of the clustered index are specified in the WHERE clause of the SQL statement.

If not all index fields are specified, a Clustered Index Seek or Clustered Index Scan can be performed.

A Clustered Index Seek searches only once a data row via the index and scans subsequent data rows through the data pages, until the fields in the WHERE clause match the clustered index fields. This means, a Clustered Index Seek is performed, if the fields in the WHERE clause match the clustered index fields in the order of the fields in the index, no matter how many fields are specified, for example: SELECT * FROM ZVBAK WHERE MANDT = ‘001’ AND VBELN = ‘00000123’ (1 row returned)

SELECT * FROM ZVBAK WHERE MANDT = ‘001’ (all rows in client ‘001’ returned)


© SAP AG 2002

Clustered Index(Primary index)

SELECT * FROM ZVBAP WHERE MANDT = '001' AND VBPOS = '00120'

MANDT VBELN VBPOS001 0000121 00010001 0000122 00010001 0000123 00120001 0000124 00010001 0000125 00010001 0000126 00010001 0000127 00010001 0000128 00120001 0000129 00010002 0000001 00010002 0000002 00010002 0000003 00010010 0000004 00010

MANDT VBELN VBPOS ERDAT ERZET ... ...001 0000128 00010 04051999 085123 ...001 0000128 00120 01021998 210136 ......

MANDT VBELN VBPOS ERDAT ERZET ... ...001 0000123 00010 04051999 085123 ...001 0000123 00120 01021998 210136 ......

Clustered Index Scan

A Clustered Index Scan is performed, if the fields in the WHERE clause are not in the order of the fields in the index, this means one field in the order of the index fields is missing. The query processor has to scan several regions of the index pages to find the result set. In this case for each VBELN a row with VBPOS = ‘00120’ has to be searched. This can lead to a large amount of I/O to find all rows of the result set.

In this example table ZVBAP has a clustered index with the fields MANDT, VBELN and VBPOS. This index is used to find the row WHERE MANDT = '001' and VBPOS = '00120'. All index pages with different VBELN’s have to be read and in addition the pages where the row with the corresponding VBPOS is located.

If many rows are returned by the statement, it is likely that a Clustered Index Seek using the MANDT field only is performed (an operation similar to a table scan if only one value for MANDT exists). All Data rows are scanned then with a more effective I/O operations, and the result set is selected via a Filter condition or operator.


© SAP AG 2002

SELECT * FROM ZVBAK WHERE MANDT = '001' AND OBJNR = '0000000000000000000020' ANDAUFNR = '000000000004'

MANDT OBJNR AUFNR...001 0000000000000000000010 00000000000001...001 0000000000000000000011 00000000000001...001 0000000000000000000020 00000000000004

MANDT OBJNR AUFNR Primary Index key001 0000000000000000000019 00000000000001001 0000000000000000000019 00000000000002001 0000000000000000000019 00000000000003001 0000000000000000000019 00000000000004001 0000000000000000000019 00000000000005001 0000000000000000000020 00000000000001001 0000000000000000000020 00000000000002001 0000000000000000000020 00000000000003001 0000000000000000000020 00000000000004 001 0000000099001 0000000000000000000020 00000000000005001 0000000000000000000021 00000000000001

+ Bookmark Lookupoperator

3 + 3 pages accessed

Index Seek

If a nonclustered index is used to find the result set specified by the WHERE clause of the statement, the operators used can be Index Seek or Index Scan. To find the data rows, a two step process is performed: The first step is to find the rows in the leaf level pages of the nonclustered index pages, which are part of the result set.

The second step is called a Bookmark Lookup. For each row found in the index the corresponding data row is accessed via the row locator, which is usually a clustered index key in the R/3 environment.

As for the Clustered Index Seek / Scan, an Index Seek / Scan can be performed. This depends again on the order of the fields in the index and the WHERE clause.

The different structure of a clustered and nonclustered index cause also different costs when a certain fraction of data is read. To access data in a range of key values via a clustered index is much faster than via a nonclustered index, because the Bookmark Lookup operation is expensive. Therefore it can be observed, that the query optimizer switches to a access patch using the clustered index, even if the whole data for one client has to be read.


© SAP AG 2002

SELECT * FROM EXTERNALTABLE WHERE FIELD1 = '000815'

No clustered index definedAll rows are read

Table EXTERNALTABLE

Nonclustered Index

FIELD1 FIELD2 FIELD3 FIELD4 ... …010 000000122 0003 000815 ...

...

FIELD1 FIELD2 FIELD3 FIELD4 ... ...005 000000122 0003 004711 ......

FIELD1 FIELD2 FIELD3 FIELD4 ... ...002 00009658 0012 000123 ...002 00009874 0007 000815 ......

FIELD1 FIELD2 FIELD3 FIELD4 ... ...001 000000122 0013 001234 ......

Table Scan

If a Table Scan operator is used all data pages of the table are read sequentially but no index pages are read.

This access path is efficient if a large fraction of the table rows are requested by the query. However, in the example above only few table rows are requested. In this case a full table scan is very inefficient.


© SAP AG 2002

Efficiency of Access Paths / Operators

The whole explain plan has to be investigatedIf index statistics are existing, always the path with the lowest costs is chosen

For complicate access paths the costs can not be calculated easily

Generally the acces path is efficient, if the data access operators can use selective fields in the indexes

The SQL Server query optimizer will switch to “rules” only in the case NO statistics are defined at all

The decision, if an access plan is suitable or not, usually is simple in the case an index is missing for the fields in the where clause. Sometimes it is helpful to cross-check the data model of the application to see, if simply other tables can be accessed to get the same information faster. For example see SAPnote 185530.

In the case the result set consists of a lot of rows, for example in the case of OPEN SQL “in itab” statements, it is often quite hard to understand the decision of the optimizer.


© SAP AG 2002


What an index is

How an index is structured

How data is accessed using an index

Explain the action of some Operators for data access

Now you are able to describe:

Now you know: What an index is How an index is structured How data is accessed using an index


© SAP AG 2002

Cost Evaluation

The different factors that influence estimated cost

At the end of this unit you will be able to describe:

Once you have completed this unit, you will know: The different factors that influence the costs estimated for an access path by the SQL Server

optimizer


© SAP AG 2002

Cost Evaluation

Query AnalysisDetermination of search arguments

Index SelectionIndex statistics

Index cost

During the first phase of query optimization, the query optimzier analyzes the where clause of the statement. During this phase the optimizer evaluates, if a part of where the clause can be used as a search argument or not: useful clauses (search arguments): NAME = “BERT”, SALARY > 25000, NAME LIKE ‘PET%’ not useful: NAME <> “JOHN”, NAME LIKE “%ER”, SALARY !> 25000

The second phase is the Index Selection. The optimizer determines, whether an useful access path exists to support the search arguments. The expected number of rows is calculated from the index statistics, and additionally the costs for I/O and CPU. Finally the execution plan is created and stored.


© SAP AG 2002

Index statistics

Cost based optimizer

Estimated Costs

est.rows: number of rowsest.io: amount of logical I/O

Exact costs for the execution can not be calculated but estimated

SELECT * FROM ZVBAK WHERE MANDT = @P000 AND AUFNR = @P001

SELECT Index Seek(OBJECT:([EW4].[dbo].[ZVBAK].[ZVBAK~0]),PLAN_ROWest.rows: 1 est.io: 0.01 est.cpu: 0.00

SELECT Clustered Index Seek(OBJECT:([EW4].[dbo].[ZVBAK].[ZVBAK~0]),PLAN_ROWest.rows: 1 est.io: 0.47 est.cpu: 0.00

SELECT Bookmark Lookup(BOOKMARK:([Bmk1000]), OBJECT:([EW4].[dbPLAN_ROWest.rows: 1 est.io: 0.01 est.cpu: 0.00

Index Seek(OBJECT:([EW4].[dbo].[ZVBAK].[ZVBAK~ZB])PLAN_ROWest.rows: 1 est.io: 0.01 est.cpu: 0.00

Query Optimizer

The main factors that are considered by the query optimizer to calculate the estimated costs for an access path are explained in this chapter.

During the first part of the index selection, the estimated number of rows returned by the query, is calculated using the index statistics, as well as column histograms.

The cost calculation is based on the estimated number of returned rows (est.rows) and results in two values: est.io: Estimated costs of logical I/O (access to the data cache, not disk I/O) est.cpu: estimated CPU costs

Mainly the costs for logical I/O are considered when the access path is selected.


© SAP AG 2002

Select * from ZVBAP where VBELN = @P000 and POSNR = @P001

1 row, 2 logical readsest.rows: 1, est.io = 0,0063285 (≈0,01)

Costs for a Clustered Index Seek (1)

The OPEN SQL statement shown in this example is a full qualified access to table ZVBAP. Values are specified for VBELN, POSNR. The key fields of the primary index are MANDT, VBELN, POSNR.

The database interface adds the field MANDT the fields to the where clause of the SQL statement. The explain function in the SP statistics shows values for these fields in the first line. The values are stored in the Stored Procedure Name Cache and are collected for the execution with the longest runtime.

The explain plan is displayed in the lower part. A Clustered Index Seek is used as the data access operator. In the same line (OBJECT section) the table and index accessed is shown, as well as the fields which are included in the seek operation.

The query optimizer calculates from the statistics that 1 row will be returned, which is the (correct) estimate.

As the index height (levels of the index tree) is 2, two pages have to be accessed to obtain all information, the index page and the data page.

To access 1 row of a table using a Clustered Index Seek causes logical I/O costs of 0.01, or more exactly 0.0063. This number does not depend on the levels of the index tree, until the number of levels is small.


© SAP AG 2002

Select * from ZVBAP where VBELN in (@P000, @P001, @P002)

15 rows, 9 logical readsest.rows: 19, est.io = 0,0070692 (≈0,01)

Costs for a Clustered Index Seek (2)

In the case of the ABAP statement in this example, the WHERE clause consists of an IN (A, B, C) for the field VBELN.

The first 2 fields of the clustered index are specified, so again a Clustered Index Seek is performed to access data (not shown in the slide). Now several rows can be returned for each combination of values for MANDT and VBELN with different POSNR values.

The explain plan consists of several operators, the most of them are used to show how VBELN values are calculated and sorted for the access.

The main operator is a Nested Loop. For each value in the IN clause a Clustered index Seek is performed. The estimated number of returned rows for all 3 seeks is given in the PLAN ROW section of the Nested Loop operator. In this case 19 rows are estimated.

The estimated logical I/O is again ~0.01. If the query is executed, 15 rows are returned, and 9 logical reads are used (pages are accessed in the data cache). This information is obtained running the stored procedure call (exec ...) with the parameters in a Query Analyzer window and the STATISTICS I/O connection property is set to ON.


© SAP AG 2002

Select * from ZVBAK where OBJNR = @P000 and AUFNR = @P001

1 row, 5 logical readsest.rows: 1, est.io = 0,01265 (≈0,01)

Costs for an Index Seek (1)

The ABAP statement in this example uses the field OBJNR and AUFNR of the secondary index ZVBAK~ZB. All fields of the index are: MANDT, OBJNR, AUFNR.

Usually a secondary index is not defined with an unique constraint, so even in the case all fields are specified, it is not sure that number of rows returned is 1.

The fields in the where clause match the order of the fields in the index, so an Index Seek is used to access the data. The Bookmark Lookup operator is shown also in the plan.

To access the requested data, 3 pages out of the nonclustered index ZVBAK~ZB (index height 3), and 2 pages of the clustered index/data pages need to be accessed to return the requested data.

To return this one row via a nonclustered index, 5 pages have to be read, and the costs calculated by the optimizer are 0.012. This is a factor of two compared to 0.006 for one row using a clustered index.


© SAP AG 2002

Select * from ZVBAK where OBJNR = @P000

5 rows, 17 logical readsest.rows: 5, est.io = 0,031412 (≈0,03)

Costs for an Index Seek (2)

In the ABAP statement the field OBJNR is specified. Again the secondary index ZVBAK~ZB is used, an Index Seek is performed. The estimated number of rows is 5, the estimated I/O costs are around 0.03.

The number of logical reads is less than 5 times 5 = 25, the reason is that the index pages are accessed only once to find the leaf level pages of the nonclustered index. The access to the data pages via the clustered index must be done 5 times anyway.

The PLAN_ROW part of the Bookmark Lookup shows the total number of rows estimated, coming from the Index Seek.


© SAP AG 2002

Select * from ZVBAK where VPRGR = ‘X’ / VPRGR = ‘ ‘

Select using VPRGR = X9990 rows, 705 logical readsest.rows: 9966, est.io = 0,4685 (≈0,47)

Select using VPRGR = 10 rows, 705 logical readsest.rows: 9966, est.io = 0,4685 (≈0,47)

Nonuniform Data Distribution

The ABAP statement has the field VPGRP in the where clause, which is not in an index of the table ZVBAK.

For 9990 of 10000 rows the value for the field VPRGR is set to X, in the remaining rows it is empty. The access path is suitable, if we search for the rows with VPGRP = X. If a secondary index would exist on the field VPGRP, the optimal access path would be via this secondary index for rows with empty VPGRP.

The DBSL interface generates a stored procedure for this statement, which is permanent, this means the access path will not be changed as long as it is stored in the procedure cache. If the procedure is called the first time with the value X for VPGRP, the execution plan will no change.

Even if a secondary index exists on the field and the stored procedure is generated with the Clustered Index Seek, this execution plan will be chosen.

Execution plans are stored in the procedure cache. From time to time an execution plan is thrown out of the cache to store other plans. In this case a new plan will be generated. This helps in the most cases to generate new plans for stored procedures where the data distribution has changed.

In the case that nonuniform data distribution is a problem for a query, the R/3 instance profile parameter dbs/oledb/max_duration can be set. See also the corresponding SAPnote.

If these solutions do not help for any reason, please open a problem message for such problems in component BC-DB-MSS


© SAP AG 2002

Table Scans and Clustered Indexes

Tables with Clustered IndexClustered Index leaf level pages = data pages

Clustered Index Seek (whole table / all rows in one client) is performed instead of Table Scan

Typical in R/3, where the primary key is a clustered index on the database level

Tables without Clustered IndexAll indexes are nonclustered

Table Scans are used

The query processor uses the Table Scan operator only in the case, that no clustered index exists. For nearly all tables in the R/3 database a clustered index exists, and therefore instead of a Table Scan a Clustered Index Seek with MANDT = client is used.


© SAP AG 2002

Appendix: R/3 Parameters that Control SQL Statements

RSDB/MAX_BLOCKING_FACTORThe SQL Server port of the database interface does not perform OR’s or UNION’s when a FOR ALL ENTRIES statement is processed.

This parameter is not really performance critical.

DBS/OLEDB/PREFER_IN_ITAB_OPTControls the processing of “in itab” statements.

This parameter is performance relevant.

“For all entries” statements are transformed in a set of single statements by the SQL Server database interface library. The number of single statements, which are submitted in one batch, is determined by the R/3 instance profile parameter “rsdb/max_blocking_factor”.


© SAP AG 2002

Missing indexes

Guidelines for creating an index

Contents:

Creating an Index

Once you have completed this unit, you will be able to: Determine if an index could be created Determine if a statement is expensive by analyzing the Stored Procedure Statistics


© SAP AG 2002

Creating an Index - Objectives

Determine if an index could be created

Determine if an SQL statement is expensive



© SAP AG 2002

Before you create an index, check for missing indexes

Missing Indexes

Before you create an index, check for missing indexes using Transaction DB02 >>DB analysis >> missing indexes.

Indexes are declared in the ABAP Dictionary and are created in the database. If an index does not exist in either the dictionary or the database, it is missing. If the index does not exist on the database, queries for which the index is intended will be

inefficient and slow. If an exisiting database index does not exist in the ABAP Dictionary this may lead to confusion,

for example, when creating an index or analyzing interference between indexes. When do missing indexes occur?

Indexes created in the ABAP Dictionary must be activated and generated before they are available on the database. If the indexes are not activated and generated, they are missing.


© SAP AG 2002

Before you create an index check the table statistics

Check table statistics

Before you create a new index check the table statistics of the existing index for the following criteria:

Are the statistics up to date What is the table size now and how large was the table at the time of the last update statistics


© SAP AG 2002

Rules for Creating Indexes

Use only selective fields if possible

Use as few fields as possible

Use selective fields at the beginning if possible

Do not create an unselective index

Create as few indexes per table as possible

Do not create an index that can be used unintentionally

Do not change an index created by SAP

When you create an index, use only selective fields if possible. Use as few fields as possible. Every field that is included in the index costs storage space and makes

the index access less efficient. Selective fields should appear at the beginning of an index if possible. Do not create an unselective index. Never change an SAP created index, unless you are requested to do so by SAP. Check if the index really supports the query for which it was created.


© SAP AG 2002

Selectivity Analysis

Which Index Should Be Created?

To find out the selectivity of an index, use:Transaction DB05

Or

Query Analyzer:select Field1,Field2,Field3,count(*) from Table group by Field1,Field2,Field3 having count(*) > 100order by count(*);

This statement shows which combinations of Field1, Field2, and Field3 occur more than a hundred times

Selectivity analysis is expensive!

Before you create an index, you must know how selective the index would be. To display the distribution of records for possible field and value combinations of the index, use

Transaction DB05. You can also use a Query Analyzer to determine the index selectivity. The SQL statement given

above will return the combinations that occur more than a hundred times. Note: The selectivity analysis using either Transaction DB05 or Query Analyzer is expensive.


© SAP AG 2002


Number of Records returned

Index:Field1Field2Field3

SELECT FROM TABLE WHERE FIELD1 = ‘A’AND FIELD2 = ‘B’AND FIELD3 = ‘C’

How many records will be returned by the Index scan?

Num

ber o

f diff

eren

t co

mbi

natio

nsHistogram of combinations

Access via an index is cheap if only a few records are read and returned using the index. To ensure that there are no combinations of values that would return many columns a histogram can be calculated. The table contents are scanned for the different possible combinations of the indexed fields. For each combination the number of records that are returned is recorded. In the histogram the number of different combinations that return a certain number of records is recorded.


© SAP AG 2002


Number of Recordsreturned


Num

ber o

f diff

eren

t co

mbi

natio

ns

A Selective Index An index that is not selective for most combinations of indexed fields


An index that is selective for most combinations of indexed fields

Num

ber o

f diff

eren

t co

mbi

natio

ns

Num

ber o

f diff

eren

t co

mbi

natio

ns

Histogram of combinations

The histogram on the left hand side shows an index that is selective for all of the possible field combinations.

If the index is selective for most of the combinations of indexed fields a histogram similar to the one displayed in the middle will result. Most of the combination return only a few records and some combinations of fields exist that would return a lot of records. In this case you must check if a query could be issued by the application with the combinations of values for which a lot of records would be returned. This would cause an expensive unselective index range scan. Only the developers of the table and the programs that are selecting data from the table can decide if this could happen.

For example: A spool table exists where all the print requests are stored. All requests have a status flag STATUS, that can be set to either ‘printed’ or ‘not printed’. The majority of the spool requests are usually in the status ‘printed’ and only a few are ‘not printed’. The histogram for an index with the fields PRINTER and STATUS will show some combinations for which only a few records would be returned by the index, that are the not yet printed print requests to a certain printer. Other combinations would return many records, these are all the print requests that are already printed on a printer. Only if it can be assured by the application and table logic, that the already printed print requests are not selected from the spool table using the flag STATUS the index could be created to accelerate the search of the not yet printed print requests.

An index that usually returns many records will show a histogram similar to the histogram as displayed on the right hand side. Such an index should not be created.


© SAP AG 2002

In this example, there are 18,607 different combinations for all of the fields.Compare this number to the number of table rows.


Transaction DB05 displays the selectivity of a field combination. Use this information to determine if an index should be created.

The number of Distinct values shows you how many different combinations of fields exist. This number should be close to the number of table rows for the combination of all of the key fields of the index.

In this example, there are: 4,506 different values for field USPOB 7,172 possible combinations of USPOB and GJAHR 9,195 possible combinations of USPOB, GJAHR, and WRTTP 18,607 possible combinations of all the fields. 635, 336 table rows

When you compare the possible combination of all the fields to the number of table rows, you can determine that this index would return 30 rows on average.


© SAP AG 2002

If the number of distinct values does not increase for a field, the field is unselective.


If the number of distinct values does not increase for a field, the field should not be included in the index.


© SAP AG 2002

In this example, 16,587 combinations of all the fields would return 1 to 10 rows.


This column display how many field combinations would return 1 to 10 rows. For a selective index, these numbers are close to the number listed in the column Distinct values.


© SAP AG 2002

In this example, 8 combinations of all the fields would return up to100,000 rows. This index could cause an unselective index range scan.


In this example, 8 combinations of all the index fields have a hit set of between 10,001 and 100,000 records. If one of these combinations is specified, an expensive access via this index will occur.

In general, it is not effective if an index returns many rows. To determine if you should create an index, even if unselective field combinations exist, you must

know the program logic when this table is accessed.


© SAP AG 2002

Query Analyzer

To display the combinations of index fields for which more than a hundred records exist, use the following SQL statement: select Field1,Field2,Field3,count(*) from Table group by Field1,Field2,Field3 having count(*) > 100 order by count(*);

In this example, 1084 combinations of the fields TABNAME and AS4LOCAL exist with more than hundred records.

The combination that returns the most records is the combination of the fields where TABNAME = FUSS_MB and the field AS4LOCAL = A. This combination occurs 741 times.


© SAP AG 2002

Never change an SAP Index without approval and advisement of SAP

Check for missing indexes with DB02 and recreate them

Run Update statistics

Change the code so that existing indexes can be used

Changing an index, so that it can be utilized for queries that used it before as well as the expensive queries

Create a new index to best satisfy the query

Preferential Order of Possible Optimizations

Check for missing indexes with DB02 and recreate them Change the code so that existing indexes can be used Changing an index, so that it can be utilized for queries that used it before as well as the expensive queries

Create a new index to best satisfy the query


© SAP AG 2002

Determine if an index should be created

Determine if a statement is expensive



Now you are able to: Determine if an index should be created Determine if a statement is expensive by analyzing the Stored Procedure Statistics


© SAP AG 2002

Similar Statements

Find expensive similar statements in the Stored Procedure Statistics


Once you have completed this unit you will be able to: Find expensive similar statements in the Stored Procedure Statistics


© SAP AG 2002

Similar StatementsUse the same access path, have the same or a similar WHERE clause

Can be accelerated by using one tuning method

Might not be individually expensive, but the sum of similar statements can be expensive

Stored ProceduresFor each OPEN SQL statement in the ABAP coding a stored procedure is generated

The same statement or similar statements can occur in several stored procedures

Temporary stored procedures are generated on each application server and always have different names

Stored Procedures and Similar Statements

Similar statements use the same access path. They may have the same or a similar WHERE clause. Using one tuning method, such as creating, changing, or dropping an index could help to accelerate

the similar statements. Stored Procedure Statistics are collected for each stored procedure name separately. A stored

procedure is generated for each ABAP OPEN SQL statement, independent of the statement in the stored procedure. If a statement is used often in ABAP reports, for each occurrence a stored procedure name is created and therefore statistics are collected separately.

Each of the similar statements might not be expensive, but the sum of similar statements can be expensive.


© SAP AG 2002


Detection NOT straight forward in the SP statistics!

Stored Procedure Statistics Display:

Do not deselect show dependency in the selection screen to display the database table used by the query (warning: time consuming!)

Enter a value of 0,1% of the total runtime with fetch for all stored procedures in the “total runtime including fetch >=“ in the selection screen

Sort the output by column object for identical entries and compare the statements for similar WHERE clauses.

For temporary stored procedures the field object is never filled

There is no straight forward method to detect similar statements in the Stored Procedure Sstatistics. A possibility to search for similar statements executed stored procedures is to display the database

table (column object) in the Stored Procedure Statistics. This is activated by the button show dependency in the selection screen for SP statistics. The search for the database table (object) is time consuming!

To limit the time for a search, enter a value of 0,1% of the total runtime with fetch (sum for all stored procedures) into the field “total runtime including fetch >=”.

Sort the output by the column object, and check the statements for tables that occur often. To determine if the statements found are similar, compare the SQL statements. Add the Runtime with fetch for all similar statements. If the sum of all statements is expensive, treat

these statements as a single expensive statement.


© SAP AG 2002

Possible Optimizations

Use the same methods as for single expensive statements

Be aware, that optimization is necessary at different locations in ABAP sources

To optimize similar statements, you can: Use the optimization possibilities for non-similar statements Change ABAP code at each location where a similar statement occurs


© SAP AG 2002

Find similar statements in the Stored Procedure Statistics



Now you are able to: Find similar statements in the Stored Procedure Statistics


© SAP AG 2001

Explain how views are processed by SQL Server

Analyze expensive statements to views


View Processing

Once you have completed this unit, you will be able to: Explain how views are processed by SQL Server Analyze expensive statements to views


© SAP AG 2001

T1.K1 T1.F1 T1.F2

T1K1 F1 F2

T2K1 F3 F4

Database view

T2.F3 T2.F4

Applicationtransaction

Database interface

Views

A view is a logical object. Data is stored in the tables on which the view is defined.

The data is read from the appropriate tables during runtime.

A view is a logical object. That is, the data of a view is stored only in the tables on which the view is defined. From these tables data of a view is read during runtime.

The structure of a view is defined by the tables and fields it contains and a JOIN condition. A view can be used to summarize data that is distributed among several tables (database view). A view can also be used to reduce the number of fields read from a table, keeping interfaces to a

minimum (projection view).


© SAP AG 2001

Nested Loop:

Get next row from outer table A using the where clauseGet all rows in inner table B using join conditions and the where clause -> output

Merge Join (on sorted tables):

Get next rows from tables A and B using join conditions and where clause If the row of table A or B shows a “lower” value for the joined fields, get rows from this table until the join condition is met -> output

Hash Join (on tables without a useful index):

Build up hash buckets using hash functions for each row of tables A and B using join conditions (and where clause)Scan for matches -> output

View Processing

The database has different strategies to process statements to a view: Nested Loops Merge Join (on sorted tables) Hash Join

The access path with the lowest estimated costs is used. The Nested Loops strategy is used very often to execute queries to a view in the R/3 environment, because indexes exist on the tables. At least the field MANDT is used from the clustered index.

For the Nested Loops strategy, the costs of the statement execution depend critically on the number of rows read on the outer table. The number of iterations of the loop equals the number of rows returned from the outer table.

In the case that many rows are returned by the statement, the optimizer will switch to the Merge Join or Hash Join strategy.


© SAP AG 2001

Nested Loop

Evaluation of the strategy: The estimated number of rows for the queries to ZVBAP and ZVBAK are evaluated from the table statistics. - Statement 1 on table A: select * from ZVBAP where MANDT =, CUOBJ = - Statement 2 on table B: select * from ZVBAK where MANDT = , VBELN = - (NOTE: MANDT is generally inserted in the where clause, not as a join condition)

Result of the cost evaluation: - Statement 1: estimated number of rows is estimated from the statistics to be 5. - Statement 2: estimated number of rows = 1 for each row out of ZVBAP one row in ZVBAK,

because the unique constraint of the clustered index covers the fields MANDT and VBELN. If the number of estimated rows from the outer table is higher than a certain percentage, the query processor will probably switch to the merge join or hash join strategy. Execution plan: - For each row in ZVBAP search for rows in ZVBAK and return them.


© SAP AG 2001

MANDT VBELN ...

001 0000120001 0000121001 0000122001 0000123001 0000124001 0000125

...

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000123 ...…

001 0000123 ...001 0000456 ...

ZVBUK ZVBAK ZVBAP

MANDT VBELN CUOBJ...001 0000121 0000001 ...001 0000122 0000002...

001 0000123 0000011...001 0000124 0000012...001 0000125 0000013...001 0000126 0000014......

SELECT * FROM ZVIVEDA2 WHERE MANDT = '001' AND CUOBJ > '11'

Data read for the Merge Join

Merge Join Nested Loop

Merge Join:

1. All appropriate data is read from the joined tables and sorted (if necessary) according to the join condition

2. The rows of the tables are merged

Merge Join

For a merge join the steps of processing a view are as follows: The appropriate data of the different tables for the view is read and, if necessary, sorted

according to the join conditions. The data of the different tables is merged, that is the data rows and fields that belong to the view

according to its definition are returned. If no selective join conditions exist on a joined view, a merge join is more efficient than a nested loop. However, for most SAP Views the join conditions are very selective. Therefore usually the SQL Server optimizer chooses a nested loop for these type of statements, only in very special cases like this a merge join is used.

In this example a merge join is performed for tables ZVBUK and ZVBAK. For both tables (ZVBUK and ZVBAK) all records for MANDT = '001' are read. When the join condiditon (field VBELN) is checked, simply first two fields of the primary index

are comparedfor each row. In this case the fields MANDT and VBELN are the first fields in the clustered index and therefore the rows from the tables are alredy sorted when read.

The rows for both tables are merged now. For each combination of values MANDT and VBELN, which occur in both tables, the fields defined in the view are returned.

A merge join over ZVBUK and ZVBAK can be combined in the access path with a nested loop over ZVBAP.


© SAP AG 2001

Hash Join:

Hash Join operations are only efficient if a major fraction of each of the view tables has to be read to satisfy the query

View Processing

A hash join operation is only efficient if the majority of the different table records are requested, this is not normally the case for R/3 operations.


© SAP AG 2001

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

MANDT VBELN CUOBJ ...001 0000121 1 ...001 0000122 2 ...001 0000123 3 ...001 0000124 4 ...001 0000125 5 ...001 0000126 6 ...

...

001 1000815 12345 ......

SELECT * FROM ZVIVEDA WHERE MANDT = '001' AND CUOBJ = '12345'

ZVBAP ZVBAK ZVBUK

Join conditions on fields MANDT, VBELN for all tables

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

Correct Access order

Importance of the Table Access Order for a Nested Loop

In this example the view ZVIVEDA is created on three tables ZVBAP, ZVBAK and ZVBUK with join conditions on the fields MANDT and VBELN. That means, only records of these tables are eligible for the view if records with the same values for these fields exist in all three tables.

The query selects all records from the view where the fields MANDT = '001' and CUOBJ = '12345', which are both read from table ZVBAP.

Only for the records that match the where condition on the outer table, the records from inner tables are read. For finding the records on inner tables the join conditions can be used. That means in this example, for table ZVBAK the records are read where MANDT = '001' and VBELN = '1000815'. Further fields specified from ZVBAK would have been used as well.

Because of the join conditions on table ZVBUK, the record where MANDT = '001' and VBELN = '1000815' is read as well. In total 3 records are read in total.

Because on the inner tables of a view the rows are often found by the join conditions, an appropriate index must exist on the join conditions to find the records efficiently.


© SAP AG 2001

SELECT * FROM ZVIVEDA WHERE MANDT = '001' AND CUOBJ = '12345'

ZVBAPZVBUK

Join conditions on fields MANDT, VBELN for all tables

MANDT VBELN CUOBJ ...001 0000121 1 ...001 0000122 2 ...001 0000123 3 ...001 0000124 4 ...001 0000125 5 ...001 0000126 6 ...

...

001 1000815 12345 ......

ZVBAK

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

Importance of the Table Access Order for a Nested Loop

Incorrect Access order

In this example the view ZVIVEDA is processed with a different sequence of table accesses. First table ZVBAK is read. On this table the field CUOBJ does not exist. Therefore, all the records where MANDT = '001' are selected. For each of these records the appropriate records in the inner tables are read. To find these records the join conditions can be used, that is the records are searched with the criteria MANDT = '001' and VBELN = '0000121' then the record MANDT = '001' and VBELN = '0000122' etc. All the records of the inner tables where the search criteria of the outer table matches are found. However, when the last table VBAP is accessed, the condition CUOBJ = '12345' is checked and the database finds that the record is not valid for the select statements where condition in most of the cases.

For views the order of table access paths is important for the statement costs. Thus, the statement costs for views are calculated for different orders of table access. Also, costs for other view access strategies are evaluated, like hash join and sort merge join.


© SAP AG 2001

ZVBAP~0--------------------MANDTVBELN

Execution Plan for a View Statement

Table Field Table FieldVBAK MANDT = ZVBUK MANDTVBAK VBELN = ZVBUK VBELN VBAP MANDT = ZVBUK MANDT VBAP VBELN = ZVBUK VBELN

ZVBUK~0--------------------MANDTVBELN

ZVBAK~0--------------------MANDTVBELN

Indexes

View field Table Field name MANDT ZVBAK MANDTVBELN ZVBAK VBELNPOSNR ZVBAP POSNRCUOBJ ZVBAP CUOBJ

Join ConditionsView Fields

In this example the select statement selects all records from view ZVIVEDA where MANDT = @P000 and CUOBJ = @P001.

The optimizer decides to perform two nested nested loops for the statement, where the ‘outer table’ of the outer nested loop operator is the result of the inner nested loop operator: In the inner nested loop operator first table ZVBAP is read with a Clustered Index Seek, because

there is no suitable index for the field CUOBJ. This is the outer table of the join. For all records found in ZVBAP and according to the join conditions, records from ZVBAK are selected.

In the outer nested loop operator for each row of the result set of the inner nested loop records are selected from table ZVBUK.

In this example the transitivity of the join conditions can be used to read the field MANDT from table ZVBAP although in the view definition this field is defined to be read from ZVBAK.

The cost based optimizer is in general able to resolve the join conditions and make use of the transitivity of the join conditions.


© SAP AG 2001

Never change an SAP View without approval and advisement of SAP

Check the staistics for the tables

Change the code so that an existing index of the outer table of the join can be used

Change the view definition

Create an index to support access to the outer table of the join

Change the code to a nested select instead of a view


Before you analyze the expensive statement in detail check the statistics for all tables of the view. Change the code so that an existing index of the outer table of the view can be used Create an index to support access to the outer table of the view Change the code to a nested select instead of a view


© SAP AG 2001


Analyze expensive statements to views


Now you are able to: Analyze expensive statements to views


© SAP AG 2002

Joins

Explain how joins are processed by SQL Server

How joins are structured in the ABAP code

Analyze expensive statements to joins


Once you have completed this unit, you will be able to: Explain how joins are processed by SQL Server Explain how joins are structured in the ABAP code Analyze expensive statements to joins

For ABAP Joins no explain exists in ST05 Trace for SQL Server!


© SAP AG 2002

SQL Statements for Joins

Selection Conditions

Join Fields

Joined Tables and Join Conditions

SELECT T_00 ."VBELN",T_00 ."AUFNR",T_01 ."POSNR”

FROM ( ( "ZVBAK" T_00 INNER JOIN "ZVBAP" T_01 ON T_01 ."MANDT" = @P000 AND T_00."VBELN" = T_01 ."VBELN" ) INNER JOIN "ZVBUK" T_02 ON T_02 ."MANDT" = @P001 AND T_00 ."VBELN" = T_02 ."VBELN" )

WHERE ( T_00 ."MANDT" = @P002 AND T_00 ."VBELN" = @P003)

You can identify Joins in the Stored Procedure Statistics by statements that contain field descriptors of type "T_00.”.

The first part of the statement defines the join fields that are read from the different joined tables that are defined in the from clause, as well as the join conditions. You can identify the join conditions by the comparison of two different table fields, for example T_00.”VBELN" = T_02.”VBELN". The WHERE clause are the selection conditions. The selection conditions are the conditions that can limit the number of rows returned from the outer table of the join.

Joins are processed similar to joined views.


© SAP AG 2002

Execution plan of a Join

Joins are processed similar to joined views on the database. In this example the optimizer chooses nested loop operators in the access path. The access sequence is not necessarily the sequence of tables in the from clause. However, from the logic of joins in the ABAP, usually the best access sequence is the same as the sequence of tables in the from clause.

When you analyze a join statement: Check if the table is read first in the access sequence, that returns the smallest number of rows. This is the outer table of the join.

Check if for each table the correct index is used. Fields that can be used on index are for the outer table the selection conditions and for the inner tables the selection condition and the fields from join conditions to tables that were accessed before.

In this example for table ZVBAK the fields MANDT, VBELN and AUFNR are read unsing a Clustered index Seek where the VBELN matches the input parameter. For table ZVBUK the fields MANDT and VBELN are read if the join condition on VBELN is matched. Finally from ZVBUK MANDT and VBELN are also read if the join condition is met.


© SAP AG 2002

ABAP Statements for Joins

Join Conditions

SelectionConditions

Join Fields

Joined Tables

Internal Table Fields

To find a join statement in the ABAP coding use the where used list for the first table, in this example ZVBAK, the key word ‘select’ and search string ‘join’.

select f~vbeln f~aufnr g~posnr into (beln, aufnr, posnr) from ( zvbak as f inner join zvbap as g

on f~vbeln = g~vbeln ) inner join zvbuk as h on f~vbeln = h~vbeln

where f~vbeln = '0000000050'.

endselect.

You can identify join statements in the ABAP code by the ABAP command 'inner join'. In a join you should always take care that you can access each table in the join statement by a suitable index. The database can use indexes for either the fields specified in the selection conditions or the join conditions. Normally you should use the table as the first table of the join that most limits the number of records. All subsequently accessed tables should have a selective join condition to one of the tables accessed above and an index on the join condition.

To find the ABAP code for a join, start a where used list for the first table in the statement and the key word ‘select’ and the additional search string ‘join’.

There also exists the possibility to use outer joins in the ABAP.


© SAP AG 2002

Never change an SAP Join without approval and advisement of SAP

Check the statistics for the tables

Change the table access sequence

Change the join conditions to conditions that are supported by an index

Create a suitable index for the outer table of the join


Check if the database options Auto create statistics and Auto update statistics are set (use DB02). Futher check the table details, if austostats is switched on or off for the table and if this is the SAP standard setting.

Change the table access sequence in the ABAP code Change the join conditions to conditions that are supported by an index Create a suitable index for the outer table of the join


© SAP AG 2002

Analyze expensive statements to joins



Now you are able to: Analyze expensive statements to views


© SAP AG 2002

Pooled and Clustered Tables

Describe pooled and clustered tables

Analyze expensive statements to pooled and clustered tables

At the end of this unit, you will be able to:

Once you have completed this unit, you will be able to: Describe pooled and clustered tables Analyze expensive statements to pooled and clustered tables


© SAP AG 2002

Roadmap for analyzing expensive SQL-Statements for Pools and Clusters

Buffering possible?

No

Pool or Cluster?Yes

Table Call Statistics: Table with many fetches?

Yes

Find Table/report name in SM66or using the

Repository Information System

Check coding with development or SAP Code Change possible?

No

See Roadmap:Find ABAP/4

coding

Use this roadmap for the analysis of expensive statements to pool and clustered tables.


© SAP AG 2002

Pooled and Clustered Tables

DefinitionSeveral R/3 tables can be stored in one database table, which iseither a clustered or a pooled table

The data fields of pooled or clustered tables cannot be indexed.They do not exist as separate physical columns in the database.

The data in the tables can be processed with Open SQL but not with Native SQL

Table pools, also referred to as pools, and table clusters, also referred to as clusters, are managed as objects in the ABAP Dictionary.

Several logical pooled or clustered tables are combined to form a table pool or table cluster. That is, the data from several different tables is stored in one common table in the database.

Data fields cannot be indexed because they do not exist as separate columns in the database. The data in the tables can be processed with Open SQL but not with Native SQL.


© SAP AG 2002

Table_A

1 2

Key Data

Table pool

TABNAME VARDATAVARKEYTable_B

Key Data

DATALN

Table_ATable_B

Pooled Tables

CharacteristicsSeveral pooled tables can be stored in a table pool

The pooled tables may have different primary key fields

The primary key of a pooled table is stored in column VARKEY in the table pool

All the data columns are stored in column VARDATA of the table pool

Pooled tables are logical tables. They exist as tables in the ABAP Dictionary but not as separate tables in the database.

Several pooled tables are stored together in one transparent database table, the table pool. The pooled tables may have different primary key fields. The different key fields of the pooled tables are combined to a single field in the table pool (VARKEY).

All of the non-key fields, the data fields of the pooled table, are also combined to form a single column in the table pool (VARDATA).


© SAP AG 2002

Transparent tableDDIC TABLENAME Key1 Key2 Key3 Data1 Data2 ...

DB Table Key1 Key2 Key3 Data1 Data2 ...

Pooled tableDDIC TABLENAME Key1 Key2 Key3 Data1 Data2 ...

DB Table pool TABNAME VARKEY DATALN VARDATA

How the tables are represented in the database

Pooled Tables

Transparent tables are stored with a one to one relation in the database. That is, for every R/3 table field, a table field exists in the database.

For pooled tables, only the table pool exists in the database. The key fields of the pooled table are stored in column VARKEY. The key fields for the database table are columns TABNAME and VARKEY. All data fields are stored together in column VARDATA.

The table fields of the pooled tables only exist in the ABAP Dictionary. The database interface converts the logical R/3 pool table rows to the database table rows from the table pool.


© SAP AG 2002

Pooled and Transparent Tables in the Dictionary

Pooled tables have key fields and data fields like transparent tables. However, these fields are only defined and visible in the ABAP Dictionary, not on the database.

In this example, pooled table A004 has a similar structure as transparent table A068. The only difference is the representation in the database.

The ABAP Dictionary also displays the name of the pool where the pooled tables are stored. In this example, table A004 is stored in table pool KAPOL.


© SAP AG 2002

ABAP Statement:SELECT * FROM A004 WHERE KAPPL = XKAPPL AND KSCHL = XKSCHLAND VKORG = KOMK-VKORGAND VTWEG = KOMK-VTWEGAND MATNR = KOMP-PMATN

Stored Procedure Statistics:SELECT "TABNAME" , "VARKEY" , "DATALN" , "VARDATA" FROM "KAPOL" WHERE "TABNAME" = @P000 AND "VARKEY“ LIKE @P001

Statements to Pooled Tables

For pooled tables, the SELECT statement in the ABAP coding is different than the SELECT statement sent to the database. Accesses to pooled tables look like statements to transparent tables in the ABAP coding. However, the statement looks different in the database. In the database, the table pool is accessed instead of the pooled table.

If you select data from a pooled table, all the key fields up to a selective field must be specified for a quick data retrieval.

In this example, all fields up to the very selective material number (MATNR) are specified in the ABAP coding. The database interface converts this statement to a select statement to the table pool KAPOL.

Since not all pool table key fields are specified, the statement contains a LIKE clause for the key field VARKEY of the table pool.

From the statement in the Stored Procedure Cache, you cannot see which pooled table is accessed or which fields were specified.


© SAP AG 2002

Analysis Path

Shared SQL AreaSELECT "TABNAME" , "VARKEY" , "DATALN" , "VARDATA“ FROM "KAPOL" WHERE "TABNAME" = @P000 AND "VARKEY“ LIKE @P001

Table Call StatisticsDB activity

Open Fetch Rows Table

...KAPOL A004 1.234 45.987 54.968.732KAPOL A005 5.663 6.897 56.654...

Total 1.345.643 1.587.554 75.897.654

Where used list

A004Object name:

Logical databasesScreensPrograms

System-Wide Work Process Overview

Server Typ Report Action Table

P22589 DIA ZVALST61 sequential read KAPOL

For statements that read many rows, use the Where used list

For statements that use an inefficient access path, use the Work Process Overview

To display expensive statements to pooled tables, use the Table Call Statistics (Transaction ST10), and choose All tables and All servers.

For tables that belong to a pool, the pool name is written in front of the table name. Analysis path for a statement that reads a lot of data:

The column DB activity displays how many fetches were performed and how many rows retrieved. An expensive statement may perform a lot of fetches. For tables inside the pool that have a lot of fetches, use the Where used list in the ABAP dictionary, and look for the ABAP code responsible for the access.

Analysis path for a statement that does not read a lot of data but is processed inefficiently: If the expensive statement is not reading many records but is processed inefficiently, you cannot identify the pooled table from the Table Call Statistics. Therefore, use the system-wide work process overview (Transaction SM66) to look for accesses to the table pool and the underlying ABAP report. In the ABAP reports you find, look for accesses to all the tables inside the pool. Because pool tables are always accessed using the fields TABNAME and VARKEY, you cannot distinguish between different statements. Therefore, it is not efficient to analyze these statements using the SQL Processes Monitor.

If you are interested in all tables that are stored in a table pool use the Repository Info System. Use Transaction SE12 Repository info system - Other objects - Pool/cluster tables - <Pool table> - Where-used list - tables.


© SAP AG 2002

CharacteristicsA table cluster stores several clustered tables that have primary key fields in commonThe data contained in one clustered table is logically dependent on the data in the other clustered tablesThe data that belongs to one key combination is stored in column VARDATA

Table_A

1 2

Key DataTable_B

1 2

Key Data

Table cluster

1 2

Key DATA

TABLE_A

PAGENO

TABLE_B

Clustered Tables

Clustered tables are logical tables. They exist as tables in the ABAP Dictionary, but not as separate tables in the database.


© SAP AG 2002

Statements to Clustered Tables

ABAP Statement:SELECT * FROM BSEG WHERE BELNR = XBELNRAND GJAHR = XGJAHR ...

Stored Procedure Statistics:SELECT "MANDT" , "BUKRS" , "BELNR" , "GJAHR" , "PAGENO", "TIMESTMP" , "PAGELG" , "VARDATA" FROM "RFBLG" WHERE "MANDT" = @P000 AND "BELNR" = @P001 AND "GJAHR" = @P002...

For clustered tables, the select statement in the ABAP is different than the select statement in Stored Procedure Cache. In the ABAP coding, the access to a clustered table looks like the access to a transparent table. In the Stored procedure Cache, however, the table cluster is accessed.

In this example, table BSEG is accessed in the ABAP coding and the access to the cluster RFBLG is displayed in the stored procedure text.

Unlike pool tables, different accesses to clustered tables can look different. For example, a different number of primary key fields can be specified or different ranges of some fields can be specified. For an expensive statement, some key fields before the first selective field are often not specified. The information about the primary key fields specified often enables you to find the code.


© SAP AG 2002

Analysis Path

SP statisticsSELECT * FROM “RFBLG“ WHERE “MANDT“ = @P000 AND “BELNR“ = @P001Table Call Statistics

DB activityOpen Fetch Rows

Table

...RFBLG BSEC 1.234 45.987 54.968.732RFBLG BSED 5.663 6.897 56.654...

Total 1.345.643 1.587.554 75.897.654

Where used list

BSECObject name:

Logical databasesScreensPrograms

System-Wide Work Process Overview

Server Typ Report Action Table

P22589 DIA ZVALST61 sequential read RFBLG

For statements that read many rows, use the Where used list

For statements that use an inefficient access path, use the Repository Information system and the Where used list or the Work Process Overview

Repository Information system

Table Short description

BSEC One-time account data document segment BSED Bill of exchange fields document segmentBSEG Accounting document segment BSES Document control data BSET Tax data document segment

Analysis path for a statement that reads a lot of data: The column DB activity displays how many fetches were performed and how many rows retrieved. An expensive statement may perform a lot of fetches. For tables inside the cluster that have a lot of fetches, use the Where used list in the ABAP dictionary, and look for the ABAP code responsible for the access.

Analysis path for a statement that does not read a lot of data but is processed inefficiently: If the expensive statement is not reading many records but is processed inefficiently, you cannot identify the clustered table from the Table Call Statistics. Unlike pool tables, table clusters usually contain only a few tables. If you cannot find any heavily accessed clustered tables, use the Where used list to check the select statements to all the tables inside the table cluster that is causing the expensive select statement. Use the Repository Information System to find the tables inside the cluster and perform a where used list search for all the tables inside the cluster.

To find the tables inside the cluster use Transaction SE12 Repository info system - Other objects - Pool/cluster tables - <Cluster table> - Where-used list - tables.

You can also use the system-wide work process overview to look for accesses to the table cluster and the underlying ABAP report. In the ABAP reports you find, look for accesses to all the tables inside the cluster.


© SAP AG 2002

A Common Mistake for Accessing Clustered Tables

No selective key field is specified

Only selective fields inside VARDATA are specified. Therefore, no selective index access possible

Example:

Statements to clustered table VBFA: VBELN is specified but not VBELV

To find the preceding document number, use tables:VBAP For ordersLIPS For deliveries VBRP For billingThe preceding document number is the field VGBEL

If no selective key field is specified, data cannot be retrieved efficiently from a table cluster. That is, if only selective fields inside VARDATA are specified in the ABAP coding, no selective index access is possible.

For example, for queries to clustered table VBFA (document flow), often the preceding document number (VBELV) is not specified. The preceding document number is the only selective field in the key of the table cluster.

If you require the preceding document number and you know the subsequent document number (VBELN) do not use the clustered table VBFA. Instead, use other tables, such as VBAP, in case the subsequent document is an order LIPS, in case the subsequent document is a delivery VBRP, in case the subsequent document is a bill


© SAP AG 2002

ABAP code:select * from VBFA where VBELN = VALUE⇒ No selective key field of

the cluster is specified

A Common Mistake for Accessing Clustered Tables

In this example, you see that table VBFA contains the subsequent document number VBELN. As shown in the ABAP Dictionary, the table is included in the cluster VBFCL. The key of the cluster does not contain field VBELN. Therefore, a Clustered Index Seek is performed on the field MANDT, this means all rows of the specified client are read.

With the following ABAP open SQL statement, all preceding document numbers should be found for the subsequent document number that is specified:

select * from VBFA where VBELN = VALUE

However, this is not the intended use of table VBFA.


© SAP AG 2002

Do not change an SAP Object without approval and advisement of SAP

Pool Tables:Specify more key fields of pooled table for access in ABAP code

Buffer the pooled table if possible

Pull table out of Pool, make it transparent and add index

Cluster tables: Specify necessary cluster key fields in ABAP code

Investigate alternative tables to access the same data

Pull table out of Cluster and make it transparent and add index on fields


Pool Tables: Specify more key fields of pooled table for access Buffer Pooled table Pull table out of Pool and make it transparent and add index on fields

Cluster tables: Specify necessary cluster key fields in code When no selective cluster key field can be specified, an alternative table should be investigated

to access the data Pull table out of Cluster and make it transparent and add index on fields


© SAP AG 2002

Analyze statements to pooled and clustered tables




© SAP AG 2002

Expensive Statements and SAP Table Buffers

Identify incorrectly buffered tables in the Stored Procedure Statistics


Once you have completed this unit, you will be able to: Identify incorrectly buffered tables in the Stored Procedure Cache


© SAP AG 2002

Buffered Tables

Reasons for expensive SQL statements to buffered tablesTables are buffered incorrectly

Tables are frequently changed

Tables are single key buffered but sequentially read

Table is bigger than the SAP table buffer

The SAP table buffers are too small

This causes a large amount of data to be read from the database

To reduce the load on the database, tables can be buffered on application servers. Therefore, only few accesses to buffered tables should be detected in a properly tuned system.

In the following cases expensive SELECT statements can occur: Changes on generic key buffered tables:

- If a record in a generic region is changed, the complete generic region is invalidated. If a record contained in this region is accessed later, the complete generic region is read from the database, and not only the records requested by this OPEN SQL statement. This means frequent changes to generically buffered tables may cause expensive SQL statements, especially if the generic regions are large.

Sequential access to tables in the single key buffer: - If single keys are accessed frequently, an expensive statement can be avoided by using generic

or full buffering for the table, if possible. The SAP buffers are too small / one or more tables are to big for buffering

- Swaps in the table buffers can be detected. Buffered data have to be reloaded because they are swapped out during the load of another table. Check the size of the buffered region of the tables.


© SAP AG 2002

ABAP:

SELECT … FROM "A117" WHERE

KAPPL = … AND KSCHL = … ANDVKORG = … ANDVTWEG = … ANDKUNNR = … ANDMATNR = … ANDDATBI = …

Database:


"MANDT" = @P000ORDER BY <Primary key>

SQL Statement for Buffered Transparent Table

If you find a SQL statement on a full or generic buffered table that is executed very often, check the table buffering. To check the SAP Table Buffers for swaps, use Transaction ST02. Check also if there are a lot of changes compared to reads.

SELECT statements containing an ORDER BY clause for the primary key can be caused by filling the table buffer.

To find out the appropriate buffering type, check the technical settings of the table (use Transaction SE13).

On the other hand, if you find an expensive SQL statement on a transparent table that is executed

very often check if: The table can be buffered The table is buffered but the table buffer is too small The table buffering type can be changed to support the expensive statement


© SAP AG 2002

ABAP:


KAPPL = … AND KSCHL = … ANDVKORG = … ANDVTWEG = … ANDMATNR = … ANDDATBI = …

Database:

SELECT"TABNAME" , "VARKEY" , "DATALN”, "VARDATA”

FROM "KAPOL"

WHERE "TABNAME" = @P000 AND "VARKEY" LIKE @P001

ORDER BY "TABNAME" , "VARKEY"

SQL Statement for Buffered Pool Table

If you find an expensive SQL statement on a table pool that is executed very often, check the table buffering. To check the SAP Table Buffers for buffer swaps, use Transaction ST02. Check also if there are a lot of changes compared to reads.

You must identify the table that is read from the table pool and the related report. To do this, use the methods described in the unit Pooled and Clustered Tables.

For a frequently issued SQL statement check if: The table can be buffered The table is buffered but the table buffer is too small The table buffering type can be changed to support the expensive statement

SELECT statements containing an ORDER BY clause for the table fields TABNAME and VARKEY can be caused by filling the table buffer.

To find out the appropriate buffering type, check the technical settings of the table (use Transaction SE13).


© SAP AG 2002


1. Determine the appropriate buffering type

2. Enlarge SAP table Buffers

To optimize accesses to tables you should: Determine the appropriate buffering type. These types are:

- Not buffered - Single Record - Generic - Full buffered

Check if the buffers need to be enlarged


© SAP AG 2002

Identify stored procedures which access incorrectly buffered tables in the Stored Procedure Cache




© SAP AG 2003


Section III – Certification Description




© SAP AG 2003

SECTION III

Certification Description

SAP Web AS 6.20 2003/Q2 50062304


SAP Consultant Certification Technology Consultant SAP NetWeaver - SAP Web AS Implementation & Operation for MS SQL Server (2003) Software components: SAP Web AS 6.20, MS SQL Server 2000

Certification duration: 1 hour Number of certification questions: 40 multiple choice questions Required certifications for participation in this certification test: None (please note, that the certificate Technology Consultant SAP NetWeaver - SAP Web AS Implementation & Operation for MS SQL Server (2003) is only handed out, if the certification participant suceeded in the intermediate test based on SAP Web AS 6.20 (certification 2003, certification booking code C_TADM12_03) and this certification test. Courses for certification preparation: TADM53 SAP NetWeaver - SAP Web AS DB Operation (MS SQL Server)

Please note that you are not allowed to use any reference materials during the certification test (no access to online documentation or to any SAP system).

The certification test Technology Consultant SAP NetWeaver - SAP Web AS Implementation & Operation for MS SQL Server (2003) verifies the knowledge in the area of the SAP NetWeaver for the consultant profile SAP Web AS Implementation & Operation in the area of the MS SQL Server.

This certificate proves that the candidate has a basic understanding within this consultant profile based on the MS SQL Server, and can implement this knowledge practically in projects. This certification test only includes questions about the administration of the MS SQL Server within a SAP installation (see topics below). Questions about the non database dependant part of the consultant profile are covered in the intermediate test (certification booking code C_TADM12_03). Please note, that the certificate is only handed out, if the certification participant suceeded in both tests (see above).

The certification test consists of questions from the areas specified below:

Topic Areas

1. Device Configuration (+++)

Data and storage architecture using databases, files, and indexes

Files of a Web AS database

File size management

2. Restore and Recovery (++)

Steps of a restore

Reasons for a restore and resulting restore procedures

Point in time recovery


3. Troubleshooting (++)

Checking database error messages

Handle exceptional situations on SQL Server

Client/Server communication

Detecting poorly qualified SQL statements

Index statistics

Lock wait situations

4. Administration (+++)

Windows services, processes, and threads

Database connections of Web AS

SAP dictionary objects - database objects

Update statistics

Memory management

Maintaining SQL Server configuration parameters and database options

Authentication

Locking

SAP statistics on stored procedures

Transaction logging

5. Backup (++)

Backup types

Backup strategies

Tape management and tape options

Backups with CCMS and SQL Server tools

Amount of questions by topic (as percentage of test):

+ = 1 - 10%

++ = 11 - 20%

+++ = over 20%

Documents

TADM53 SAP Netweaver SAP Web as DB Operation MS SQL Server