Logging in Flash-based Database Systems
Lu Zeping2009-06-27
2
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
3
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
4
Characteristics of flash memory
• No in-place update
× Out – place update
5
Characteristics of flash memory
• No mechanical latency
MediaRandom-to-Sequential Ratio
Read workload Write workload
Magnetic Disk 4.3-12.3 4.5-10.0
Flash 1.1-1.2 2.4-14.2
6
Characteristics of flash memory
• Asymmetric speed of read/write
Media Read Write Erase
Magnetic disk
12.7ms(2KB)
13.7ms(2KB)
N/A
Flash 80μs(2KB)
200μs(2KB)
1.5ms(128KB)
7
The flash memory organization
Device : 1G
Block : 1024
Page : 64
2K Bytes 64 Bytes
Page Register
8
The flash memory organization
Device : 1G
Block : 1024
Page : 64
2K Bytes 64 Bytes
Page Register
9
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
10
Design of IPL
• IPL : In-Page Logging
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
DatabaseBuffer
in-memorydata page(8KB)
in-memorylog sector (512B)
11
Design of IPL
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
Flash Memory
log area (8KB): 16 sectors
Erase unit (Block): 128KB
15 data pages (8KB each)
….….….
……
……
12
IPL Write
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
DatabaseBuffer
in-memorydata page(8KB) in-memory
log sector (512B)
Flash Memory
log area (8KB): 16 sectors
Erase unit (Block): 128KB
15 data pages (8KB each)
….….….
……
……
13
IPL Read
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
DatabaseBuffer
in-memorydata page(8KB) in-memory
log sector (512B)
Flash Memory
log area (8KB): 16 sectors
Erase unit (Block): 128KB
15 data pages (8KB each)
….….….
……
……
14
IPL Read
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
DatabaseBuffer
in-memorydata page(8KB) in-memory
log sector (512B)
Flash Memory
log area (8KB): 16 sectors
Erase unit (Block): 128KB
15 data pages (8KB each)
….….….
……
……
15
IPL Merge
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
Flash Memory
….….….
….….….
……
……
……
……
….
Merge
….
16
IPL Merge
S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007
Flash Memory
….….….
….….….
……
……
……
……
….
Merge
….
17
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
18
SSD
• SSD = Flash memory + FTL ( Flash Translation Layer )
19
To be used in……
• Accessing data in secondary storage is the main source of bottlenecks in database processing.
• Database table, index, log, temporary table and rollback segment could be created on physically separate storage devices .(SSD+ HDD)
• Log records are always appended to the end of log.
21
TPC-B benchmark performance
B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.
We used TPC-B to stress-test the log storage part of the commercial database server.
Due to the group commit mechanism, the average size of a single log write was slightly different between the two storage media.
22
TPC-B benchmark performance
B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.
In the case of SSD, the bottleneck in transaction throughput is the processing speed of CPU , while it was not in the case of disk.
23
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
USB flash memory advantages
• USB ports are widely available.
• USB bus bandwidth is good for logging.
• The hot-plug capability allows easy replacement of drives.
• The price of an individual drive is low.
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 24
25
Flash devices and disk performance
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
26
Four key challenges
• Efficiently exploiting an array of flash drives.
• Coping with large variance of write latencies.
• Efficient recovery processing.
• Combining USB flash drives with disks for better logging and recovery performance.
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
27
Exploiting an array of flash drives
What’s wrong with the conventional array organization?
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
In-memory log buffer
……Devices : N
the stripe unit size is S
Data at address A are found on the k-th device, where k = A/S mod N.⌊ ⌋
Unconventional array design
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.28
In-memory log buffer
Request queue
Worker Worker Worker……
LSN : Log Sequence Number
29
Coping with outlier Writes
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
• Sequential writes have large variance
•Outlier prediction is difficult.
31
Outlier Detection and Hiding
Worker
In-memory log buffer
Worker
Measure the average sequential write latencies for different request sizes for a device.
Longer than twice the average latency?
Reissue the same request to a different ready device.
Non-decreasing LSN?
Outlier hiding area
32
Efficient recovery processing
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
Two main steps of recovery processing(i) Locating the log record corresponding to the LSN in the last checkpoint.
(ii) Scanning the log sequentially until the crash point.
33
Flash devices and disk performance
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
34
FlashLogging design
Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.
Request queue
Worker Worker Worker……
In-memory log buffer
36
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
37
Several recovery technologies Log Group
commitPTLog Shadow
pageTxFlash
Update Pattern
In-Place In-Place Out-Place Out-Place Out-Place
Concurrency Granularity
Record Record Page Segment/file
Page
Atomicity Type
Write atomicity
Write atomicity
Propagate atomicity
Propagate atomicity
Propagate atomicity
Commit Overhead
High Low Low High Low
Rollback Overhead
High High None None High
Logging I/O Type
Synchronous Synchronous Synchronous None None
GC overhead
None None Low Low High
38
Outline
• Characteristics of flash memory• Log records in different kinds of Flash devices – In Flash chips - IPL– In SSD– In USB – FlashLogging
• Future work– Recovery processing– Experiments of PTLog
39
Experiments of PTLog
• Berkeley DB
• Linux Nandsim nand_base.c 、 nand_bbt.c 、 nand_ids.c 、 nand_ecc.c 、 nandsim.c 、 diskonchip.c 、 cafe_nand.c 、 s3c2410.c
40
Thank you !
Q&A ?