The SAP Transaction Model - Know Your Applications

8/6/2019 The SAP Transaction Model - Know Your Applications

http://slidepdf.com/reader/full/the-sap-transaction-model-know-your-applications 1/34

SYSTEMATIC THOUGHT LEADERSHIP FOR INNOVATIVE BUSINESS

The SAP

Transac t ion Model :

K now Your

App l ica t ions

Shel Finkelstein, Rainer Brendle, Dean Jacobs,Manfred Hirsch and Ulrich Marquard

SAP Labs{firstname.lastname}@sap.com



© SAP ACM SIGMOD Conference, June, 2008 2

Out l ine

SAP Applications and Data

Examples, Requirements, Properties

SAP Data Management Architecture

Application Architecture

Database Execution Model

SAP Scalability and Performance

Application Scalability

Benchmarks




SAP Appl ic at ions and Dat a




SAP Cust omer Appl ic at ion Requi rement s

SAP ERP

On a laptop

SAP for Consumer

Produc ts 5000+ concurrently

active usersSAP SCM

4.5 million characteristiccombinations & 256 GBmemory in live cache

SAP for Retai l

1.3+ million salesorder items per daySA P

NetWeaver BI

Currently productive:10 TB database(planned 60 TB)

SAP ERP - HCM

Payroll calculation for 500,000employees and retirees in 4 hours

SAP for Ut i l i t ies

25 million business partners 85 million service and salesorders per year

SAP NetWeaver

Porta l

100,000 concurrent users




Solut ion Areas and Data Ac c ess

Order Processing

Enter order with line items; schedule delivery; handle changes

Read mostly; most updates are inserts; most updates don’t conflict

Supply Chain Management (SCM)

Planning for procurement, sales and demand, production, distribution; control

and management

Read mostly, most updates are inserts; most updates don’t conflict

Business processes handle impacts of updates

Integrated Product and Process Engineering (IPPE)

Master data that provide high integration of engineering processes and

manufacturing processes Read mostly; most updates don’t conflict

Collaboration, but changes usually involve independent subtrees




Data Acc ess Pat t erns for App l ica t ions

Applications may be conversational, with multiple conversational userinteraction steps in a transaction

Most business data is read-only

Update operations are insert-mostly

Most database transactions are short

For all updates, conflicts are rare in practice

Potential hotspots (e.g., inventory stock; sequence numbers that must beconsecutive for legal reasons) must be addressed




Dat a Management Pr inc ip les

Technical: Maintain integrity of Business Data across apps

Multiple solution areas use the same Business Objects

Legal: Handle most updates as inserts of new data

Support for Governance, Risk and Compliance (GRC) and auditing

Operational: Run on multiple DBMS products

Don’t depend on special properties of any DBMS

DB must not be a bottleneck

Programming Model: Message-based consistency

Like normalization and serializability, transactions should only be used whereappropriate

Use transactions within a deployment unit; avoid distributed commit acrossdeployment units




SAP Dat a Management Arc hi t ec t ure




A p p l i c a t i o n S e r v e r DispatcherGateway

Shared

Memory

and

Buffers

Bas ic Applic a t ion Server Arch i t ec t ure

DialogHandler

User User User User

DialogHandler

DialogHandler

DialogHandler

UserUser

Database






Lock Manager

The ProblemMany applications have multiple screens with long user think times

Maintaining open database connections would hurt performance

Implicit Pessimistic and Optimistic Concurrency Control have disadvantages

E.g., scalability; rollback after user completes work

The Solution A business-level logical lock manager (Enqueue Manager) outside the database

Business-level locking is very flexible approach

No danger of lock escalation or index locking in the database

Locks are explicitly requested; a “contract” between applications, using framework

Lock types: shared, intention, exclusive

Rather than getting blocked, applications get errors that they can handle

Can request that individual locks be held after commit




Database

A p p l i c a t i o n

S e r v e r 2

A p p l i c a t i o n S e r v e r 1

DialogHandler

DialogHandler

DialogHandler

Dispatcher Dispatcher

LockManager

User User User

Message

Server

LockTable

SharedMemory

1

2

34

56

St eps in Tw o-St age Ex ec ut i on

Gateway




Asynchronous Updat es

The packaging of rollback-free queued updates into a bundle allows them to beexecuted after the transaction completes (write behind)

Doing so allows the user to continue on to other work sooner

The locks are released after the updates have been performed so subsequent writeoperations never get stale data

Appropriate when subsequent operations go on to other work, so they never readstale data

The updates are performed by an Update Handler, which reduces the number ofhandlers that do writes

The updates from several transactions may be delayed indefinitely until acoordinator picks them up and performs them all together

Example: A background process that updates statistical data




Screen

SAVE COMMIT WORK

Screen Screen Screen

Three-St age Ex ec ut ion Model

Dialog Stage Update Stage Write Stage

DB COMMIT

Read from database

Acquire logical locks

No writes

Rollback possible

Read update operations fromdatabase and perform them indeadlock-free order

Single message databasetransaction

No rollbacks

Release locks uponcompletion

Write descriptors ofupdate operations tothe database

No rollbacks




Database Manager

Database

A p p l i c a

t i o n S e r v e r 2

A p p l i c a

t i o n S e r v e r 1

Dialog

Handler

Dialog

Handler

Update

Handler

Update

Handler

Dispatcher Dispatcher

Lock

Manager

Application data Data to be updated

MessageServer

Lock

Table

1

2 3

4

56

St eps in Three-St age Ex ec ut ion

7

Gateway

User User User




Comp ar ison of the Models

Two-StageUpdates always visible immediately

after transaction completes

Lower database load for singletransaction

Synchronous execution of updateshelps to build process chains

Three-StageUser can continue sooner, since return

to user as soon as update descriptor isin DB

Lower database load when multiple

transactions are boxcarred togetherShorter locking times and/or log for

sequence number assignment andother hotspot operations




What Does SAP Use Dat abase For?

Store/Retrieve data

Transaction commit

Internal locks are held very briefly

Canonical ordering of locked resources avoids deadlocks

Rollbacks for internal DB reasons only (can retry)

Operational utilities, including disaster recovery

Indexing

Internal indexes

But external indexes are maintained in main memory, kept consistent by post-commit processing




Hot Spot s: Sequence Numbers

Many ERP and Financial applications require that sequence numbers beassigned to transactions or documents

Number allocation requirements depend on legal requirements

Weaker requirements permit implementations that scale better

Strictest guarantee: chronological assignment with no gaps

Number assignment must occur inside the transaction and be rolled back incase of abort

Forces serialized access to the next available number, which reducesconcurrency

Essential to hold the (database) lock for as short a time as possible

Two-stage model: lock held during updates and writes

Three-stage model: lock held during updates but not writes

Weaker guarantee: Gaps must be logged for auditing

Three-stage model: pending updates provide a log




Hot Spots: Aggregates

Applications often must update shared aggregate data,as well as exclusive data

Order Processing focuses on a specific Order

Processing an Order also affects aggregate data

Total expense for Contract associated with that Order Stock remaining for Products sold in that Order

Transaction execution strategy is designed for scalability

Acquire exclusive logical lock on specific Order

After Commit, perform updates on Order

After Commit, perform (commutative) delta update on aggregatedata

SAP also can perform merge update, e.g., for collaborativeapplications such as Integrated Product and Process Engineering




Review ing SAP Transact ion Ex ec ut ion St ages

During transaction Logically lock data that needs to be changed

Read Business Data into an application context

Before commit, perform updates only on the application context

After commit Stage 1: Application does logical consistency checks, then queues

update descriptor

Due to logical locks, there can be no conflicts with concurrent transactions

Stage 2: Transaction’s queued update descriptor is written to DB insingle message transaction (with boxcarring)

Return to user after its single message transaction completes

Stage 3: Updates to Business Data in DB are performed

Updates are executed in canonical order to avoid deadlocks

Delta updates to aggregated executed last, with coarsest aggregationsexecuted last of all




Sc alabi l i ty and Per formanc e




Performance typically defined by two metrics

Response time - speed of task completion (latency)

System throughput - amount of work done in a givenamount of time

Good performance if metrics match customer

needs or Service Level AgreementsMost important Key Performance Indicators (KPIs)

CPU time consumed

Peak memory used

Disk space

Network load

Proven and predictable scalability

Precondition for translating business requirements intohardware configurations (sizing) with linear growth

Minimize load on central resources (e.g., DB)

No system wait times, no bottlenecks

What Ex act ly Is Good Per formanc e?

Application ServerApplication Server(s)

Database

Web Server(s)

ResponseTime = t[i]

ResponseTime

t[1]

t[2]

t[3]

t[4]

t[5]

t[6]

t[i]




Ver t i ca l & Hor izon ta l Sca lab i l i t y

(Cross-Tier Sc al ing a nd Scal eOut )

From one tier (laptop demo) tomulti-tier systems

Presentation More than 150K very active

users (fat transactions)Internet More than tens of thousands of

hits /sec 10 servers at one of our largest

customersApplication and Integration Over 150 app servers

connected to DBDatabase More than 100 CPUs; more

than 10 TB DB size Scalability through SMP

database server Scalability through parallel

partitioned databases








0. Logon 10. Choose customer order

1. Main screen 11. Change delivery info for order

2. Create customer order 12. Posts goods issue (schedule delivery)

3. Enter order info 13. List orders

4. Enter details for 5 items 14. Choose set of orders

5. Save 15. Create invoice

6. Create a delivery 16. Save

7. Enter delivery info 17. End

8. Save 18. Confirm logoff

9. Display customer order

Steps 2 to 16 are repeated n times; each 15 step run takes at least 150 seconds due to 10 secondthink time between steps.

Business aspect:One run (steps 2 to 16) corresponds to the selling of 5 line items.

User Interact ion St eps –SAP SD Standard Appl icat ion Benc hm ark




2005

Proven Sc alabi l i t y Using SD Benchm ark s

SD Benchmark, Two-Tier Internet Architecture: Scale with Hardware

1996 1997 1998 1999 2000 2001 2002 2003 2004

http://www.sap.com/benchmark

21,000

6001,410 1,708

3,0004,100

7,8008,000

13,000

20,000

2006

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

23,456

30,000

35,400

2007 2008

Number of SD

Benchmark users






Key F igures for th e 168,300 User

Three-Tier SD Benc hm ark (2005)

8,392 DB transactions per second (commits)

323 MB written to disk per second on average

611 GB data written in one hour (~ 170 MB per second on average)

1,157,000 network packets per second on average, with packet size of 512 bytes

469,091 SQL statements per second

2,240 GB database disk space

5,632 SAP (as opposed to DB) transactions / second

14,081 Screen changes / second

Database used a 32-processors SMP server

R/3 was running on 712 processors

16,896,670 fully business processed order line items per hour ( 4,694 per second )




Conclus ion

Data and transaction management should be based on applications andtheir data utilization

Traditional DBMS are not ideally suited for this

SAP has had its own external data and transaction layer since 1985

– This works; DB hasn’t been a bottleneck for us

Get the programming model right for your apps Explicit locking for applications, based on framework

Use distributed commit only within a deployment unit

– Across deployment units, use message/process-based (loose) consistency

Know your applications

One size hasn’t fit all, for SAP, for other companies, and for our customers




Furt her Informat ion

Publ ic Referenc es

SAP Sta ndard Appl icat ion Benc hm arks

SAP St andard Benc hmark Cer t i f i ca t ion 2005021

Rudiger Buck-Emden, “The SAP R/3 System: AClient/Server Technology”, Addison-Wesley, 1996

Horst Keller, “The Official ABAP Reference”, SAP Press,2005

Gerhard Knolmayer, “Supply Chain Management Based onSAP Systems: Processes and Architecture”, Springer, 2002

Burkhard Neidecker-Lutz, Frying your infrastructure: Are

transactions really useful in a distributed realtimeenterprise system?, HPTS 2005

[H-Store] Mike Stonebraker, The End of an ArchitecturalEra (It's Time for a Complete Rewrite), VLDB 2007




Furt her Informat ion

SAP Serv ice Market p lac e (SAP c ustom ers)

ht tp: / /serv ice.sap.com/benchmark

ht tp: / /serv ice.sap.com/per formance

http: / /service.sap.com/siz ing

ht tp: / /serv ice.sap.com/quicks iz ing

http://service.sap.com/benchmark

http://service.sap.com/performance

http://service.sap.com/sizing

http://service.sap.com/quicksizing

http://service.sap.com/quicksizing

http://service.sap.com/sizing

http://service.sap.com/performance

http://service.sap.com/benchmark




Copyr ight 2008 SAP AG

Al l r ights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changedwithout prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are

trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed

are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document

contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy,

and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, orother items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of

merchantability, fitness for a particular purpose, or non-infringement.

SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitationshall not apply in cases of intent or gross negligence.

The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in thesematerials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch

SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.

Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.

SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie diedazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namen

von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu

Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen

Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmtenGeschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien . SAP garantiert nicht die

Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder

ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck

sowie für die Gewährleistung der Nichtverletzung geltenden Rechts.

SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung

dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.

Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks

zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über

Internetseiten Dritter ab.

Alle Rechte vorbehalten.




The SAP Transac t ion Model :

Know Your App li ca t i ons

Shel Finkelstein, Rainer Brendle, Dean Jacobs,Manfred Hirsch and Ulrich Marquard

SAP Labs

{firstname.lastname}@sap.com



Documents

The SAP Transaction Model - Know Your Applications