Upload
alize
View
58
Download
0
Embed Size (px)
DESCRIPTION
Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems. Emery Berger University of Massachusetts Amherst. Advanced File Systems. Motivation & Brief review Journaling Log-structured file system. New Challenges. Servers have special needs! Large hard-disk partitions - PowerPoint PPT Presentation
Citation preview
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Emery BergerUniversity of Massachusetts Amherst
Operating SystemsCMPSCI 377
Lecture 23: Advanced File Systems
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 2
Advanced File Systems
Motivation & Brief review Journaling Log-structured file system
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 3
New Challenges
Servers have special needs! Large hard-disk partitions Quick crash recovery High-performance I/O Storing thousands of files, TB of
data None supported well by ext2
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 4
Review: Blocks & Fragmentation
Logical block – smallest unit of storage that can be allocated by file system
Internal fragmentation – occurs when file does not fill block completely Example: file = 10K, block = 8K: wastes
6K External fragmentation – occurs
when logical blocks that make up file are scattered over the disk Causes poor performance
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 5
Review: Organization
Extent – contiguous blocks: Triple: (file offset, starting block
number, length) file offset – offset of extent's first block from
file start starting block number – first block in extent length –number of blocks in extent
Meta data – file system's internal data structures Time stamps, ownership, size &
locations on disk, etc.
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 6
Extents
Triple:(offset, start, length)
Contiguous blocks
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 7
Review: Inodes
inode – stores information about file e.g., permissions, types, # of links to
file pointers to file data blocks (direct
pointers) pointers to direct pointers (indirect
pointers) each has unique inode number directory = special file: contains
pointers to other files
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 8
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 9
Metadata Updates
Lots of meta data being updated Scattered on disk = slow, and non-
atomic
Example: Creating a new file Modifies inodes, free lists, directory
entries What happens if interrupted?
e.g., power outage, crash Interrupted meta-data write
) file system in inconsistent state
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 10
Repairing File System Inconsistency
UNIX solution: fsck = “file-system check” Analogous utilities for Windows, etc. Detect and repair structural integrity
problems Scan file system’s entire meta data
Looks for inconsistencies & fixes them
Problems? S…l…o…w… Might not succeed
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 11
Journaling File Systems
Solve problem by maintaining journal Log all transactions to disk Updates to the disk committed
atomically Example: Creating a new file
Log modifications to inodes Log changes to free lists, directory
entries Now begin update When finished, write “committed” into
journal
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 12
Recovery in JFS
Power outage: Some updates fully committed to file
system No problem
Not yet fully committed File system reads journal, replays
transaction Advantages over fsck:
Much quicker than scan of disk Guarantees file system always consistent But: does not guarantee zero data loss
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 13
Advanced File Systems
Brief review Journaling Log-structured file system
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 14
LFS: Beyond JFS
Insight:While logging updates, why not logthe data too?
“Log-structured file system” Preserves data integrity Provides improved performance,
too!
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 15
Sprite LFS[Rosenblum & Ousterhout]
Outperforms then-current Unix file systems by an order of magnitude for small-file writes
Matches or exceeds Unix performance for reads and large writes
Even with overhead Can use 70% of disk bandwidth for writing Unix file systems typically can use only 5-
10%
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 16
Technology Trends
Disk technology improving rapidly But: capacity rather than performance
Increasing RAM ) cache more effective Disk traffic dominated by writes
Logs: write new information sequentially Increase write performance,
eliminating seeks
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 17
Existing File Systems
Exercise worst-case given trends! Information spread around disk
Many small accesses Synchronous writes
Application performance can be bottlenecked by disk performance
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 18
Log-Structured File Systems
Approach: improve write performance by combining into single sequential writes Converts small random writes into
large sequential transfers Use file cache as write buffer
But where does meta data go?
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 19
Structure of LFS (1/3)
Some index structures used to retrieve information Inodes not fixed: inode map used
to maintain location of each inode Fixed checkpoint region on each
disk: identifies locations of inode map blocks
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 20
Structure of LFS (2/3) Disk layout (compared with Unix FFS)
Disk
Disk
Log
Inode
Directory
Data
Inode map
Sprite LFS
Unix FFS
dir1
dir2
file1
file2
dir1
dir2
file1
file2
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 21
Structure of LFS (3/3)
Segments Important to maintain large free
extents for writing new data Divide disk into large fixed-size
extents called segments Segments always written
sequentially Before segment is rewritten,
all live data must be copied out (“cleaned”)
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 22
Crash Recovery (1/2)
Checkpoints Position in log at which all of file
system structures are consistent and complete
Checkpoint region written on special fixed position:
Periodic intervals When the file system unmounted or
shut down
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 23
Crash Recovery (2/2)
Roll-forward After crashes, LFS scans through
log segments that were written after last checkpoint
Use information in segment summary blocks
When summary block indicates presence of new inode, LFS updates inode map
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 24
LFS Wrap-Up
Extends journaling to data Sequential writes in segments
Fast! Periodic clean-up
Periodic checkpoints
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 25
Summary
Traditional filesystems – integrity problems
Solved by journaling Log-structured further improves
speed by optimizing write performance