0102 - Foundation - Persistence in HANA

SAP HANAPersistence

© 2012 SAP AG. All rights reserved. 2

Agenda

Persistence

Data Management

Log Management

Persistence


HANA

PersistenceIn-Memory Data Is Regularly Saved to Disk

Data:SQL data and undo log informationAdditional HANA information,such as modeling dataKept in-memory to ensure maximum performanceWrite process is asynchronously

Log:Information about data changes (redo log)Directly saved to persistent storage when transaction is committedCyclical overwrite (only after backup)

Savepoint:Changed data and undo log is written from memory to persistent storageAutomaticAt least every 5 minutes (customizable)

Memory

Data

Persistent Storage

Regular automatic savepoints

Information aboutdata changes

LogVolume

DataVolumes


Distributed HANA database even on a single host with shared nothing concept

Standby without own persistence

PersistenceHANA Database Landscape

DataDisks

LogDisks

DataDisks

LogDisks

DataDisks

LogDisks

DataDisks

LogDisks Data

Disks

LogDisks

Name Server (active)Index Server (standby)

Name ServerIndex Server



Statistic Server

Host

Host

Host

Standby Host


PersistenceRegular Saving of In-Memory Data to Disk, Restart

The SAP in-memory database holds the bulk of its data in memory for maximum performance, but still uses persistent storage to provide a fallback in case of failure. The log is capturing all changes by database transactions (redo logs)Data and undo log information (part of data) are automatically saved to disk at regular savepointsThe log is also saved to disk continuously and synchronously after each COMMIT of a database transaction (waiting for end of disk write operation)After a power failure, the database can be restarted like a disk-based database:

System is normally restarted („lazy“ reloading of tables to keep the restart time short: tables with preload flag + subsequently requested tables are loaded first)System returns to its last consistent state (by replaying the redo log since the last savepoint)

Savepoint:Data & undo log is written

to disk (data area)

1Continously and after each COMMIT,redo log is written to disk (log area)

2Power failure

3

Time


PersistenceShadow Memory Concept – Details

Mem

ory

Writ

e lo

ad

TimeTime

Dis

k

DATA

Change of Page 4711

No change of Page 4711

Instead a new Page 4711’

Two versions of one Page

Savepoint

DATA‘Page4711

Page4711’ Wait for IO-requests to finish

Write anchor page

Acquire lock to prevent modification of pages

Determine log position

Remember open transactions

Copy modified pages and trigger write

Increase savepoint version

Release lock

Write changed pages in parallel

very

sho

rt

Savepoint Phases

Reserved page 4711’

Savepoint is similar to Copy on write

Data Management


Data Volumes

Data ManagementData Volumes

Data Volumes are located in file systems or RAW devices

Per instance one data volumeGrowing until disk or LUN is fullLogical Volume Manger (LVM) or similar needed on OS level to extend the file systems or dedicated partition/LUNSize reduction possible with dedicated commandGrowing with number of data volumes currently not available (unlike MaxDB) Different page sizes (4k, 16k, ...16M) which are arranged in superblocks of 64M

Typical sizing of the disks for data volumes

3 - 4 times main memory of HANA node

superblocks


Data ManagementStartup Process

ROW StoreLoaded completely into memory during startup and has to stay thereSecondary indexes are created during load

Columnar StoreColumns loaded “lazy” on demand during startupEnsures early availability (slightly lesser performance for a short time after startup)

Important factors for startupRemaining Log to be rolled forwardI/O performance of data and log disksSeparate log, data and backup disk areas not only logically, but also physically for the best performance

Log Management


Log ManagementRedo & Undo

Redo Log Information is written into Online Log

Undo Log Information stays in data area until COMMIT Undo Log is then deleted for committed transaction, if not still needed for other parallel running uncommitted transactions (MVCC).

Note: Until a first data backup is created, HANA log management still is operating in Log mode OVERWRITE (ensures restart ability), even if the parameter LOGMODE is already switched to NORMAL

DataDisks

LogDisks


Log ManagementLog Modes

4 Log modes are implementedOlder, pre-release style Log writing is simulated

NO – Log writing is completely switched off

LEGACY– Log is kept only since the last data backup– Older log is purged asynchronously after data backup– No Log Backup possible

Additional Log modesOVERWRITE

– No backup necessaryNORMAL

– Backup is automatically performed– Only log which is backed up will be freed– This is the preferred Log mode


Log ManagementLog-Buffers in Memory and Log-Segments on Disk

Log writingLog is written to Log-Buffers in MemoryIf a Log-Buffer becomes full OR a commit entry is written, the Log-Buffer will be written to the assigned Log VolumeThe log is finally written into Log-Segments where multiple Log-Buffers can be combined intoSegements are large (often 1MB)

Log Volume==

Log Disk

Fix size log buffer (1MB)








Memory

Disk


Log ManagementLog-Buffers on Disk in Log-Segments

Log writingOn the Log volume a lot of log files as Log segments (1 GB) can be foundCircular process of writing until all files system (LUN) space is consumed with not backed up log informationLog backup saves log information out of the log segments into log backup files; the log segment files are not deleted in file system (Log mode = normal)Backed up log segments files are defined as empty and re-consumed for new log segments (no reallocation on file system needed)

Typical sizing of the area for Log volume

one times main memory of HANA node

Log Volume==

Log Disk

Log Segment


© 2012 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the United States and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries.

Oracle and Java are registered trademarks of Oracle and its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc.

HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.

IOS is a registered trademark of Cisco Systems Inc.

RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc.

INTERMEC is a registered trademark of Intermec Technologies Corporation.

Wi-Fi is a registered trademark of Wi-Fi Alliance.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Motorola is a registered trademark of Motorola Trademark Holdings LLC.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

Documents

0102 - Foundation - Persistence in HANA