Part IIIPart IIIStorage Management

Chapter 10: File System InterfaceStorage Management

Chapter 10: File-System Interface

1Fall 2010

Files

A file is a named collection of related information that is recorded on secondary storage.The operating systems maps this logical storage g y g gunit to the physical view of information storage.A file may have the following characteristics

File AttributesFile OperationsFile OperationsFile TypesFile StructuresFile StructuresInternal Files

2

File AttributesFile Name: The symbolic name is perhaps the only h d bl fil ibhuman readable file attribute.Identifier: A unique number assigned to each file f id ifi ifor identification purpose.File Type: Some systems recognize various file

Wi d i d ltypes. Windows is a good example.File Location: A pointer to a device to find a file.File Size: The current size of a file, or the maximum allowed size.File Protection: This is for access-control.File Date, Time, Owner, etc.

3

File Operations: 1/2pA file can be considered as an abstract data type h h d d i ithat has data and accompanying operations.

Creating a fileWriting a fileReading a filegRepositioning within a fileDeleting a fileDeleting a fileTruncating a fileOther operations (e g appending a file renaming aOther operations (e.g., appending a file, renaming a file)

4

File Operations: 2/2File Operations: 2/2disk

system-wideopen-file table file

processopen-file table

file index

file pointerfile open count

disk locationone file

access right

5

File StructureFile Structure

S t t ifi fil t th tSome systems support specific file types that have special file structures. For example, files that contain binary executables.An operating system becomes more complex when more file types (i.e., file structures) are yp ( )supported. In general, the number of supported file typesIn general, the number of supported file types is kept to minimum.

6

File Access MethodsFile Access MethodsAccess method: how a file be usedAccess method: how a file be used.There are three popular ones:

Sequential access method for sequential filesDirect access method for direct filesIndexed access method for indexed files.

7

Sequential Access MethodSequential Access MethodWith the sequential access method, a file isWith the sequential access method, a file is processed in order, one record after the other.If p is the file pointer, the next record to be p p ,accessed is either p+1 (forward) or p-1 (i.e., backward).

end of filecurrent record

beginning end of filenext record

8rewind

read/write

Direct Access MethodDirect Access MethodA file is made up of fixed length logical recordsA file is made up of fixed-length logical records.The direct access method uses a record number to id tif h d F l d 0 iidentify each record. For example, read rec 0, write rec 100, seek rec 75, etc.Some systems may use a key field to access a record (e.g., read rec “Age=24” or write rec “Name=Dow”). This is usually achieved with hashing.Since records can be accessed in random order, direct access is also referred to as random access.Direct access method can simulate sequential access.

9

Direct access method can simulate sequential access.

Indexed Access MethodIndexed Access MethodWith the indexed access method a file is sorted inWith the indexed access method, a file is sorted in ascending order based on a number of keys.E h di k bl k t i b f fi dEach disk block may contain a number of fixed-length logical records.An index table stores the keys of the first block in each block.We can search the index table to locate the block that contains the desired record. Then, search the block to find the desired record.This is exactly a one-level B-, B+ or B* tree.

10

This is exactly a one level B , B or B tree.Multi-level index access method is also possible.

data file

last name logical rec #

index table

Adams

Arthur

Ashcroft, … Asher, … Atkinsg

Ashcroft

Smith, …. Sweeny, … Swell, …

Smith

index tables are stored in physical memory

11

in physical memorywhen file is open

Directory Structure: 1/2Directory Structure: 1/2

A large volume disk may be partitioned intoA large volume disk may be partitioned into partitions, or mini disks, or volumes.Each partition contains information about filesEach partition contains information about files within it. This information is stored in entries of a device directory or volume table of content (VTOC)device directory or volume table of content (VTOC).The device directory, or directory for short, stores the name location size type access method etc ofthe name, location, size, type, access method, etc of each file.Operations perform on directory: search for a fileOperations perform on directory: search for a file, create a file, delete a file, rename a file, traverse the file system, etc.

12

y ,

Directory Structure: 2/2Directory Structure: 2/2

Th fi l d di tThere are five commonly used directory structures:

Single-Level DirectoryTwo-Level DirectoryyTree-Structure DirectoriesAcyclic Graph DirectoriesAcyclic-Graph DirectoriesGeneral Graph Directories

13

Single-Level Directory

All files are contained in the same directory.It is difficult to maintain file name uniqueness.CP/M-80 and early version of MS-DOS use ythis directory structure.

14

T L l Di t 1/2Two-Level Directory: 1/2This is an extension of the single-level directoryThis is an extension of the single-level directory for multi-user system. Each user has his/her user file directory TheEach user has his/her user file directory. The system’s master file directory is searched for the user directory when a user job starts.y jEarly CP/M-80 multi-user systems use this structure.

15

T L l Di t 2/2Two-Level Directory: 2/2To locate a file path name is used ForTo locate a file, path name is used. For example, /user2/a is the file a of user 2.Different systems use different path namesDifferent systems use different path names. For example, under MS-DOS it is C:\user2\a.\ \

The directory of a special user, say user 0, may contain all system files.y y

16

Tree Structured DirectoryTree-Structured DirectoryEach directory or subdirectory contains filesEach directory or subdirectory contains files and subdirectories, and forms a tree.Directories are special filesDirectories are special files.

17/bin/mail/prog/spell

A li G h Di t 1/2Acyclic-Graph Directory: 1/2

Thi f di iThis type of directories allows a file/directory to be shared by multiplebe shared by multiple directories.This is different from twoThis is different from two copies of the same file or directory.directory.An acyclic-graph directory is more flexibledirectory is more flexible than a simple tree structure. However, it is

file count is shared by directoriesdict and spell

18

,more complex.

Acyclic-Graph Directory: 2/2y ySince a file have multiple absolute path names, how do we calculate file system statistics or do backup?we calculate file system statistics or do backup? Would the same file be counted multiple times?How do we delete a file?How do we delete a file?

If sharing is implemented with symbolic links, we only delete the link if we have a list of links to the o y de ete t e we ave a st o s to t efile. The file is removed when the list is empty.Or, we remove the file and keep the links. When the O , we e ove e e d eep e s W e efile is accessed again, a message is given and the link is removed.Or, we can maintain a reference count for each shared file. The file is removed when the count is

19zero.

General Graph Directory: 1/2General Graph Directory: 1/2

It is easy to traverse theIt is easy to traverse the directories of a tree or an acyclic directory system.acyclic directory system.However, if links are added arbitrarily theadded arbitrarily, the directory graph becomes arbitrary and mayarbitrary and may contain cycles.H d h f

a cycle

How do we search for a file?

20

General Graph Directory: 2/2General Graph Directory: 2/2How do we delete a file? We can use reference count!How do we delete a file? We can use reference count!

In a cycle, due to self-reference, the reference count may be non-zero even when it is no longercount may be non-zero even when it is no longer possible to refer to a file or directory.Thus garbage collection may needed A garbageThus, garbage collection may needed. A garbage collector traverses the directory and marks files and directories that can be accessed.A second round removes those inaccessible items.

To avoid this time-consuming task a system canTo avoid this time-consuming task, a system can check if a cycle may occur when a link is made. How? You should know!

21

File Sharing

When a file is shared by multiple users, how i i ?can we ensure its consistency?

If multiple users are writing to the file, should all of the writers be allowed to write? Or, should the operating system protect the , p g y puser actions from each other?This is the file consistency semantics.This is the file consistency semantics.

22

File Consistency SemanticsFile Consistency SemanticsConsistency semantics is a characterization ofConsistency semantics is a characterization of the system that specifies the semantics of multiple users accessing a shared file i l lsimultaneously.

Consistency semantics is an important criterion for evaluating any file system that supports filefor evaluating any file system that supports file sharing.There are three commonly used semanticsThere are three commonly used semantics

Unix semanticsSession SemanticsSession SemanticsImmutable-Shared-Files Semantics

A file session consists all file accesses between23

A file session consists all file accesses between open() and close().

Unix SemanticsUnix Semantics

W it t fil b i iblWrites to an open file by a user are visible immediately to other users who have the file

t th tiopen at the same time.All users share the file pointer. Thus, advancing the file pointer by one user affects all sharing users.A file has a single image that interleaves all accesses, regardless of their origin.g gFile access contention may cause delays.

24

Session SemanticsWrites to an open file by a user are not visible immediately to other users who have the same fileimmediately to other users who have the same file open simultaneously.Once a file is closed the changes made to it are visibleOnce a file is closed, the changes made to it are visible only in sessions started later.Already open instances of the file are not affected byAlready-open instances of the file are not affected by these changes.

A file may be associated temporarily with severalA file may be associated temporarily with several and possible different images at the same time.Multiple users are allowed to perform both readMultiple users are allowed to perform both read and write concurrently on their image of the file without delay

25

without delay.The Andrew File System (AFS) uses this semantics.

Immutable Shared Files SemanticsImmutable-Shared-Files SemanticsOnce a file is declared as shared by its creator itOnce a file is declared as shared by its creator, it cannot be modified.An immutable file has two important properties:An immutable file has two important properties:

Its name may not be usedIts content may not be alteredIts content may not be altered

Thus, the name of an immutable file indicates that the contents of the file is fixed a constant ratherthe contents of the file is fixed – a constant rather than a variable.The implementation of these semantics in aThe implementation of these semantics in a distributed system is simple, since sharing is disciplined (i e read-only)

26

disciplined (i.e., read only).

File ProtectionFile Protection

We can keep files safe from physical damage (i.e., reliability) and improper access (i.e., protection).Reliability is generally provided by backup.The need for file protection is a direct result of theThe need for file protection is a direct result of the ability to access files. Access control may be a complete protection byAccess control may be a complete protection by denying access. Or, the access may be controlled.

27

File Protection: Types of AccessFile Protection: Types of Access

Access control may be implemented by limitingAccess control may be implemented by limiting the types of file access that can be made.The types of access may beThe types of access may be

Read: read from the fileW i i i h filWrite: write or rewrite the fileExecute: load the file into memory and

iexecute itAppend: write new info at the end of a fileDelete: delete a fileList: list the name and attributes of the file

28

File Protection: Access Control: 1/4File Protection: Access Control: 1/4The most commonly used approach is to makeThe most commonly used approach is to make the access dependent on the identity of the user.Each file and directory is associated with anEach file and directory is associated with an access matrix specifying the user name and the types of permitted accesstypes of permitted access.When a user makes a request to access a file or

di t hi /h id tit i da directory, his/her identity is compared against the information stored in the access

t imatrix.

29

File Protection: Access Control: 2/4File Protection: Access Control: 2/4

A M t i

File 1 File 2 File 3 File 4 A t 1 A t 2

Access Matrix

OwnR

OwnR

InquiryC dit

File 1 File 2 File 3 File 4 Account 1 Account 2

U A RW

RW

Credit

Own Inquiry Inquiry

User A

ROwn

RW W

RInquiry

debitInquiryCreditUser B

R ROwn

RInquiry

debitUser C

30W W

File Protection: Access Control: 3/4File Protection: Access Control: 3/4A

OB CFile 1 Own

RW

R RW

File 1Access-control Lists

In practice, the access matrix is sparse.

BOwn

RW

C

RFile 2

The matrix can be decomposed into

W

AOwn

BFile 3 pcolumns (files), yielding access-control lists (ACL)

OwnRW W

( )However, this list can be very long!

B

R

COwn

RFile 4

31

very long!R RW

File Protection: Access Control: 4/4File Protection: Access Control: 4/4Decomposition by File 1 File 3 p yrows (users) yields capability tickets.E h h

File 1Own

RW

File 3Own

RW

User A

Each user has a number of tickets for file/directory access.

File 1

R

File 2Own

RUser B File 3 File 4

Ry

These tickets may be authorized to loan or b i t th

R RW W

R

be given to other users.All tickets may be held and managed by the

File 1

R

File 2

RUser C File 4

OwnR and managed by the

OS for better protection.

W W

Capability Lists32

Capab ty sts

Th E dThe End

33