Logging in Flash-based Database Systems

Logging in Flash-based Database Systems

Lu Zeping2009-06-27

2

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

3

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

4

Characteristics of flash memory

• No in-place update

× Out – place update

5

Characteristics of flash memory

• No mechanical latency

MediaRandom-to-Sequential Ratio

Read workload Write workload

Magnetic Disk 4.3-12.3 4.5-10.0

Flash 1.1-1.2 2.4-14.2

6

Characteristics of flash memory

• Asymmetric speed of read/write

Media Read Write Erase

Magnetic disk

12.7ms(2KB)

13.7ms(2KB)

N/A

Flash 80μs(2KB)

200μs(2KB)

1.5ms(128KB)

7

The flash memory organization

Device : 1G

Block : 1024

Page : 64

2K Bytes 64 Bytes

Page Register

8

The flash memory organization

Device : 1G

Block : 1024

Page : 64

2K Bytes 64 Bytes

Page Register

9

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

10

Design of IPL

• IPL ： In-Page Logging

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

DatabaseBuffer

in-memorydata page(8KB)

in-memorylog sector (512B)

11

Design of IPL

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

Flash Memory

log area (8KB): 16 sectors

Erase unit (Block): 128KB

15 data pages (8KB each)

….….….

……

……

12

IPL Write

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

DatabaseBuffer

in-memorydata page(8KB) in-memory

log sector (512B)

Flash Memory

log area (8KB): 16 sectors

Erase unit (Block): 128KB

15 data pages (8KB each)

….….….

……

……

13

IPL Read

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

DatabaseBuffer

in-memorydata page(8KB) in-memory

log sector (512B)

Flash Memory

log area (8KB): 16 sectors

Erase unit (Block): 128KB

15 data pages (8KB each)

….….….

……

……

14

IPL Read

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

DatabaseBuffer

in-memorydata page(8KB) in-memory

log sector (512B)

Flash Memory

log area (8KB): 16 sectors

Erase unit (Block): 128KB

15 data pages (8KB each)

….….….

……

……

15

IPL Merge

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

Flash Memory

….….….

….….….

……

……

……

……

….

Merge

….

16

IPL Merge

S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

Flash Memory

….….….

….….….

……

……

……

……

….

Merge

….

17

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

18

SSD

• SSD = Flash memory + FTL （ Flash Translation Layer ）

19

To be used in……

• Accessing data in secondary storage is the main source of bottlenecks in database processing.

• Database table, index, log, temporary table and rollback segment could be created on physically separate storage devices .(SSD+ HDD)

• Log records are always appended to the end of log.

21

TPC-B benchmark performance

B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.

We used TPC-B to stress-test the log storage part of the commercial database server.

Due to the group commit mechanism, the average size of a single log write was slightly different between the two storage media.

22

TPC-B benchmark performance

B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.

In the case of SSD, the bottleneck in transaction throughput is the processing speed of CPU , while it was not in the case of disk.

23

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

USB flash memory advantages

• USB ports are widely available.

• USB bus bandwidth is good for logging.

• The hot-plug capability allows easy replacement of drives.

• The price of an individual drive is low.Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 24

25

Flash devices and disk performance

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

26

Four key challenges

• Efficiently exploiting an array of flash drives.

• Coping with large variance of write latencies.

• Efficient recovery processing.

• Combining USB flash drives with disks for better logging and recovery performance.

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

27

Exploiting an array of flash drivesWhat’s wrong with the conventional array

organization?

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

In-memory log buffer

……Devices ： N

the stripe unit size is S

Data at address A are found on the k-th device, where k = A/S mod N.⌊ ⌋

Unconventional array design

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.28

In-memory log buffer

Request queue

Worker Worker Worker……

LSN ： Log Sequence Number

29

Coping with outlier Writes

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

• Sequential writes have large variance

•Outlier prediction is difficult.

31

Outlier Detection and Hiding

Worker

In-memory log buffer

Worker

Measure the average sequential write latencies for different request sizes for a device.

Longer than twice the average latency?

Reissue the same request to a different ready device.

Non-decreasing LSN?

Outlier hiding area

32

Efficient recovery processing

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

Two main steps of recovery processing(i) Locating the log record corresponding to the LSN in the last checkpoint.

(ii) Scanning the log sequentially until the crash point.

33

Flash devices and disk performance

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

34

FlashLogging design

Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

Request queue

Worker Worker Worker……

In-memory log buffer

36

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

37

Several recovery technologies Log Group

commitPTLog Shadow

pageTxFlash

Update Pattern

In-Place In-Place Out-Place Out-Place Out-Place

Concurrency Granularity

Record Record Page Segment/file

Page

Atomicity Type

Write atomicity

Write atomicity

Propagate atomicity

Propagate atomicity

Propagate atomicity

Commit Overhead

High Low Low High Low

Rollback Overhead

High High None None High

Logging I/O Type

Synchronous Synchronous Synchronous None None

GC overhead

None None Low Low High

38

Outline

• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging

• Future work– Recovery processing– Experiments of PTLog

39

Experiments of PTLog

• Berkeley DB

• Linux Nandsim nand_base.c 、 nand_bbt.c 、 nand_ids.c 、 nand_ecc.c 、 nandsim.c 、 diskonchip.c 、 cafe_nand.c 、 s3c2410.c

40

Thank you ！Q&A ？