Upload
ailish
View
81
Download
0
Embed Size (px)
DESCRIPTION
The Design and Implementation of a Log-Structured File System. Mendel Rosenblum and John K. Ousterhout. Contents. Overview Motivation Design and implementation of LFS Cleaning policy Evaluation of real implementation Concluding comments. Overview. Goal 전체 디스크 사용 효율을 높임 Method - PowerPoint PPT Presentation
Citation preview
Log-Structured File System
The Design and Implementation ofa Log-Structured File System
Mendel Rosenblumand John K. Ousterhout
Log-Structured File System
Contents
OverviewMotivationDesign and implementation of LFSCleaning policyEvaluation of real implementationConcluding comments
Log-Structured File System
Overview
Goal전체 디스크 사용 효율을 높임
Method작은 랜덤 쓰기 작업 -> 하나의 큰 순차적인 쓰기 작업디스크의 로그 구조 : 모든 쓰기 작업은 “ appended”
Key issue지움 정책의 효율성
Log-Structured File System
Motivation
Technology trendsCPU 와 디스크의 발전 속도 차이가 점점 증가
전송율은 적게 증가됨엑세스 타임에 대한 향상은 적음
메인 메모리의 크기는 지수적으로 (exponentially) 증가큰 파일 캐쉬 : 많은 읽기 요청을 수용
File system workloadOffice and engineering applications : 작은 파일 워크로드
작은 랜덤 디스크 I/O 를 유발LFS 는 작은 파일 워크로드에 초점을 둠
Log-Structured File System
Motivation
Problems of other file systems넓게 펼쳐진 (spreading) 정보
매우 많은 작은 단위의 엑세스를 유발Ex. dir entry, inode, data block
쓰기 동기화 문제메타데이터는 일관성을 위해 동기화되도록 쓰여져야 함많은 작은 파일 워크로드에 대해서 , 디스크 소통은 동기화된 메타데이터 쓰기작업에 제한적
Log-Structured File System
Contents
OverviewMotivationDesign and implementation of LFSCleaning policyEvaluation of real implementation
Log-Structured File System
Logical structure of file
Indexed structure : same as Unix FFS
metadata
inode
block ptrblock ptr
block ptrblock ptrblock ptr
…
data
block
index
block
dir entry
file name inode number
Unix FFS에서 inode
의 위치 고정
Log-Structured File System
Physical layout in disk
Example of creating 2 files in different directories
inodes data blocks cylinder group
LFS 에서 inode 위치는 고정되지
않음
Log-Structured File System
Segments
Segment : unit of writing and cleaning
512KB ~ 1024KB
segment 0 segment 1 … segment n
Disk : consists of segments + checkpoint region
…
checkpoint region
Segment summary block Contains each block’s identity : <inode number, offset> Used to check validness of each block Modified times for each block
Log-Structured File System
Free space management
Threading and CopyingSprite LFS 는 threading 과 copying 을 같이 사용
segment -> in-placelive data -> out-of-placeLeave the
live data in place
Copy live data out of the log
Log-Structured File System
Additional structures
Inode map<Physical location>
Segment usage table
<Bytes of valid data, last modified time>
checkpoint region
inode map block ptrs
inode0inode1
…
checkpoint region
segment usage table ptrs
segment0segment1
…
checkpoint region
의 위치는 고정
Log-Structured File System
Operations
Read a blockInode map block ptr -> inode map block -> inode -> data block
Write a blockData block, inode, inode map block, segment usage table block
Update inode map table ptr, segment summary block, segment usage table
메모리에 있는현재 세그먼트
used not used
In memory
Same as FFS
Log-Structured File System
Crash recovery
Checkpoint주기적으로 혹은 사용자의 요구시 , inode map table ptrs, segment usage table ptrs 에 씀Consistent state : 메모리에 남겨진 수정된 데이터가 없음
Roll-forward만약 crash 가 발생하면 ,
가장 최근의 checkpoint 의 쓰여진 로그를 살펴봄checkpoints
crash
roll-forward
Log-Structured File System
Contents
OverviewMotivationDesign and implementation of LFSCleaning policyEvaluation of real implementation
Log-Structured File System
Cleaning policy
CleaningRead segments -> collect valid data -> write segments : 소거 (clean) 세그먼트가 발생
4 problemswhen?how many segments?Segment selection policy - most fragmentedBlock redistribution policy
files in the same directoryaging sort : 최종 수정 시간으로 정렬
Major concern
Log-Structured File System
Measurement :write cost
Write cost새로운 데이터 쓰기의 바이트에 대한 디스크 활동 평균 총 시간 [ 다수의 모든 범위 쓰기 작업 ]
UNIX FFS : seek/rotational timeLFS : cleaning overhead
Ex. write cost 10.0 : 90% time is wasted
Ideal case : 1.0 ( 모든 범위의 활용률 의미 )
Log-Structured File System
Write cost of LFS
No seek/rotational time in LFS쓰기 비용은 소거 중 “복사된 총 데이터”에 의해 결정
Goal : 소거된 세그먼트에 valid 데이터를 감소
u : 소거된 세그먼트의 활용률
Log-Structured File System
Tradeoff : cost & utilization
LFS 에서 cost-performance 와 utilization 과 tradeoff 관계
Bimodal segment distribution
Log-Structured File System
Simulation based research
Simulator디스크는 4-KB files 로 채움엑세스 패턴의 발생
Uniform : randomHot-and-cold : 90% writes to 10% “hot” files, 10% writes to 90% “cold” files
Log-Structured File System
Simulated policy
Segment selectionGreedy : 최소 사용된 세그먼트 선택
Block redistributionNo redistribution : used in random workloadAge sorting : used in hot-and-cold workload
Age : last modified time of file, 파일의 모든 블록이 같은 age 를 가짐
Log-Structured File System
First result
locality 혹은 “ better” 재분배(redistribution) 가 “ worse” 성능의 결과를 보임
FFS improved
FFS
---- : hot-and-cold (age sorting) ___ : uniform
Logging, delayed write, disk request sorting
Log-Structured File System
Analysis
Hot segments are more frequently cleaned
hot-and-cold 에서 소거된 세그먼트의 활용이 uniform 보다 높음
Log-Structured File System
Cost-benefit selection policy
Segment selection
Rationalecold segment 는 더 천천히 invalid 블록을 발생cold segment 의 수정된 블록은 더 많은 “값(value)” 을 가짐
1 : a cost to read segment
u : write back the live data
Log-Structured File System
Result
Log-Structured File System
Contents
OverviewMotivationDesign and implementation of LFSCleaning policyEvaluation of real implementation
Log-Structured File System
Implementation study
Implementation complexityFFS 와 대부분 같음
But, FFS can reuse codes
Sprite network operating system 의 구현Installed in 5 different disk partitions used by about 30 user
LFS FFS
Cleaner
Recovery code
Allocation bitmaps, layout policiesfsck code
Log-Structured File System
Micro-benchmarks
Small file workload, no cleaning happened (best case performance)
create/delete case 는 대략 10 배 정도 FFS
보다 빠름
expectation
of performance improvement with faster
processor FFS is
disk- bound : 85% utilized
(Cf. LFS : 17%)
Log-Structured File System
Micro-benchmarks
Large file workload, no cleaning happened
100MB file, write & read performance (5 phases are run in sequence)
New write creating file
Overwrite to existing file
Log-Structured File System
Long term usage statistics
Collected over a 4-month period About 70% of bandwidth utilized (write cost 1.2~1.6 : bandwidth 63~83%)
Segment utilization of /user6 partition Large number of fully utilized and totally empty segments
Log-Structured File System
Critics on LFS
LFS 의 성능 향상 이득은 최상인가 ?메타데이터 집중 워크로드에서 뛰어남읽기 / 쓰기의 일반적인 I/O 성능은 Sun-FFS 와 비슷하거나 적음
LFS 읽기 성능은 일반적으로 FFS 보다 적음지움 (cleaning) 에 대한 오버헤드는 성능을 저하시킴Sun-FFS 구현 비용은 LFS 보다 훨씬 적음