22
CS323-650 Operating Systems _________________________________________________________________________________ Computer Science, Prince of Songkla University 1 Chapter 4 File Systems We have three essential requirements for long-term information storage: 1. It must be possible to storage a very large amount of information. 2. The information must survive the termination of the process using it. 3. Multiple processes must be able to access the information concurrently. The usual solution to all these problems is to store information on disks and other external media in units called files. Processes can then read them and write new ones if need be. Information stored in files must be persistent, that is, not be affected by process creation and termination. A file should only disappear when its owner explicitly removes it. Files are managed by the operating system. How they are structured, named, accessed, used, protected, and implemented are major topics in operating system design. As a whole, that part of the operating system dealing with files is known as the file system and is the subject of this chapter. 4.1 FILES 4.1.1 File Naming When a process creates a file, it gives the file a name. When the process terminates, the file continues to exist and can be accessed by other processes using its name. The exact rules for file naming vary somewhat from system to system. Some file systems distinguish between upper case letters and lower case letters, whereas others do not. UNIX falls in the first category; MS-DOS falls in the second. Many operating systems support two-part file names, with the two parts separated by a period, as in prog.c. The part following the period is called the file extension and usually indicates something about the file. In MS-DOS, for example, file names are 1 to 8 characters, plus an optional extension of 1 to 3 characters. In UNIX, the size of the extension, if any, is up to the user, and a file may even have two or more extensions, as in prog.c.Z. 4.1.2 File Structure Files can be structured in any of several ways. Three common possibilities are depicted in Figure 4.1.

CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

Embed Size (px)

Citation preview

Page 1: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

1

Chapter 4 File Systems

We have three essential requirements for long-term information storage:1. It must be possible to storage a very large amount of information.2. The information must survive the termination of the process using it.3. Multiple processes must be able to access the information concurrently.The usual solution to all these problems is to store information on disks and other external media inunits called files. Processes can then read them and write new ones if need be. Information stored infiles must be persistent, that is, not be affected by process creation and termination. A file shouldonly disappear when its owner explicitly removes it.

Files are managed by the operating system. How they are structured, named, accessed,used, protected, and implemented are major topics in operating system design. As a whole, that partof the operating system dealing with files is known as the file system and is the subject of thischapter.

4.1 FILES4.1.1 File Naming

When a process creates a file, it gives the file a name. When the process terminates, the filecontinues to exist and can be accessed by other processes using its name.

The exact rules for file naming vary somewhat from system to system. Some file systemsdistinguish between upper case letters and lower case letters, whereas others do not. UNIX falls inthe first category; MS-DOS falls in the second.

Many operating systems support two-part file names, with the two parts separated by aperiod, as in prog.c. The part following the period is called the file extension and usually indicatessomething about the file. In MS-DOS, for example, file names are 1 to 8 characters, plus an optionalextension of 1 to 3 characters. In UNIX, the size of the extension, if any, is up to the user, and a filemay even have two or more extensions, as in prog.c.Z.

4.1.2 File Structure Files can be structured in any of several ways. Three common possibilities are depicted inFigure 4.1.

Page 2: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

2

1 Byte 1 Record

Ant Fox Pig

Cat Cow Dog Goat Lion Owl Pony Rat Worm

Hen Ibis Lamb (a) (b) (c) Figure 4.1. Three kinds of files. (a) Byte sequence. (b) Record sequence. (c) Tree

Having the operating system regard files as nothing more than byte sequences provides themaximum flexibility. UNIX and DOS use structure (a). CP/M use structure (b) with 128-characterrecord. Nowadays, the idea of a file as a sequence of fixed length records is pretty much gone,although it was once the norm. On the large mainframe computers use structure (c) in somecommercial data processing. In (c) search and update records are handled by operating system.

4.1.3 File TypesMany operating systems support several types of files. UNIX and MS-DOS, for example,

have regular files and directories. UNIX also has character and block special files. Regular files arethe ones that contain user information. Directories are system files for maintaining the structure of thefile system. Character special files are related to input/output and used to model serial I/O devicessuch as terminals, printers , and networks. Block special files are used to model disks.

Regular files are generally either ASCII (text) files or binary files. ASCII files consist of linesof text. In UNIX line is terminated by carriage return. In MS-DOS line is terminated by carriage returnand line feed. Other files are binary files, which just means that they are not ASCII files.

4.1.4 File AccessEarly operating systems provided only one kind of file access: sequential access, a process

could read all the bytes or records in a file in order, starting at the beginning, but could not skiparound and read them out of order. When disks come into use, it became possible to read the bytesor records of a file out of order. Files whose bytes or records can be read in any order are calledrandom access files.

4.1.5 File Attributes

Page 3: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

3

Every file has a name and its data. In addition, all operating systems associate otherinformation with each file, for example, the date and time the file was created and the file’s size. Wewill call these extra items the file’s attributes.

4.1.6 File OperationsDifferent systems provide different operations to allow storage and retrieval. Below is the

most common system calls relating to files.1. create2. delete3. open4. close5. read6. write7. append8. seek9. get attributes10. set attributes11. rename12. link13. unlink

4.2 DIRECTORIESTo keep track of files, file systems normally have directories, which, in many systems, are

themselves file.• Single-Level Directory all files are contained in the same directory e.g.

directory name and attributes …

file file

Since all files are in the same directory, they must have unique names.

• Two-Level Directory

Page 4: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

4

master file directory user1 user 2 user 3

user file directory

files

• Tree-Structured Directorye.g. MS-DOS

root

• Acyclic-Graph DirectoryA tree structure prohibits the sharing of files or directories. An acyclic graph allows

directories to have shared subdirectories and files. The same file or directory may be in two differentdirectories. An acyclic graph is a graph with no cycles.

root dict spell

list all w count count words list

list rade w7

For example, in UNIX , a link can be used to create share files (both hardlink and softlink (or symboliclink)).

• General Graph Directory graph can have cycle, for example

Page 5: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

5

root avi tc jim

text mail count book book mail unhex hyp

avi count unhex hex

Directory Operations- create- delete- opendir- closedir- readdir- rename- link- unlink

4.3 FILE SYSTEM IMPLEMENTATIONNow it is time to turn from the user’s view of the file system to the implementor’s view.

Implementors are interested in how files and directories are stored, how disk space is managed, andhow to make everything work efficiently and reliably.

4.3.1 Implementing FilesProbably the most important issue in implementing file storage is keeping track of which

disk blocks go with which file. Various methods are used in different operating systems. In thissection, we will examine a few of them.

Contiguous AllocationThe simplest allocation scheme is to store each file as a contiguous block of data on the

disk. For example, on a disk with 1K blocks, a 50K file would be allocated 50 consecutive blocks.Advantage:- it is simple to implement because keeping track of where a file’s blocks are is reduced to

remembering one number, the disk address of the first block.- the performance is excellent, just read the next consecutive block or can read the entire file in a

single operationDisadvantage :

Page 6: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

6

- it is not feasible unless the maximum file size is known at the time the file is created- problem of fragmentation on the disk, compaction of the disk is usually prohibitively expensive

Linked List AllocationThe second method for storing files is to keep each one as a linked list of disk blocks, as

shown below:File A

File File File File Fileblock block block block block0 1 2 3 4

physical 4 7 2 10 12block

File B

File File File Fileblock block block block0 1 2 3

physical 6 3 11 14block

Figure 4.2 Storing a file as a linked list of disk blocks

The first word of each block is used as a pointer to the next one. The rest of the block is fordata. No space is lost to disk fragmentation (except for internal fragmentation in the last block). Also,it is sufficient for the directory entry to merely store the disk address of the first block. The rest can befound starting there. On the other hand, although reading a file sequentially is straightforward,random access is extremely slow.

Linked List Allocation Using an IndexThe disadvantage of the linked list allocation can be eliminated by taking the pointer word

from each disk block and putting it in a table or index in memory. Figure 4.3 shows what the tablelooks like for the example of figure 4.2. In both figures, we have two files. File A uses disk blocks 4, 7,2, 10, and 12, in that order, and file B uses disk blocks 6, 3, 11, and 14, in that order.

Although the chain must still be followed to find a given offset within the file, the chain isentirely in memory, so it can be followed without making any disk references. Like the previous

Page 7: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

7

method, it is sufficient for the directory entry to keep a single integer (the starting block number) andstill be able to locate all the blocks, no matter how large the file is. MS-DOS uses this method for diskallocation.physical block

012 103 114 7 File A start here56 3 File B start here7 28910 1211 1412131415

Figure 4.3 Linked list allocation using a table in main memory

The primary disadvantage of this method is that the entire table must be in memory all thetime to make it work. Although MS-DOS uses this mechanism, it avoids huge tables by using largeblocks (up to 32K) on large disks.

I-nodesThe last method for keeping track of which blocks belong to which file is to associate with

each file a little table called an i-node (index-node), which lists the attributes and disk addresses ofthe file’s blocks, as shown in figure 4.4.

Page 8: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

8

I-nodeattributes

single indirect block doubledisk indirectaddresses block

triple indirect block

Figure 4.4 An i-node

4.3.2 Implementing DirectoriesDirectories in MS-DOS

Figure 4.5 shows an MS-DOS directory entry.bytes 8 3 1 10 2 2 2 4 file name size

extension attributes reserved time date first block numberFigure 4.5 The MS-DOS directory entry

In MS-DOS, directories may contain other directories, leading to a hierarchical file system.

Directories in UNIXThe directory structure traditionally used in UNIX is extremely simple, as shown in figure 4.6.

Each entry contains just a file name and its i-node number. All the information about the type, size,times, ownership, and disk blocks is contained in the i-node.

Page 9: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

9

bytes 2 14 file name

i-node numberFigure 4.6 A UNIX directory entry

When a file is opened, the file system must take the file name supplied and locate its diskblocks. Let us consider how the path name /usr/ast/mbox is looked up. First the file system locatesthe root directory. In UNIX its i-node is located at a fixed place on the disk.

Then it looks up the first component of the path, usr, in the root directory to find the i-nodenumber of the file /usr. Locating an i-node from its number is straightforward, since each one has afixed location on the disk. From this i-node, the system locates the directory for /usr and looks up thenext component, ast, in it. When it has found the entry for ast, it has the i-node for the directory/usr/ast. From this i-node it can find the directory itself and look up mbox. The i-node for this file isthen read into memory and kept there until the file is closed. The lookup process is illustrated inFigure 4.7.

block 132 i-node 26 block 406i-node 6 is /usr is for is /usr/ast

root directory is for /usr directory /usr/ast directory 1 . mode 6 . mode 26 . 1 .. size 1 .. size 6 .. 4 bin times 19 dick times 64 grants 7 dev 132 30 erik 406 92 books 14 lib 51 jim 60 mbox 9 etc 26 ast 81 minix 6 usr 45 bal 17 src

8 tmplooking up i-node 6 i-node 26usr yields says that /usr/ast says that /usr/ast/mboxi-node 6 /usr is in is i-node /usr/ast is in is i-node block 132 26 block 406 60

Figure 4.7 The steps in looking up /usr/ast/mboxExample, MINIX file system

The following figure shows a 360K floppy disk with 127 i-nodes and a 1K block size. Largerfile systems, or those with more or fewer i-nodes or a different block size, will have the same sixcomponents in the same order, but their relative sizes may be different.

Page 10: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

10

boot block super block I-nodes one disk block ...

I-node bit map Zone bit map Datanumber of nodesnumber of zonesnumber of i-node bit map blocks

Present number of zone bit map blockson disk first data zoneand in log 2 (zone size/block size)memory maximum file size

magic numberpointer to i-node bit map block

Present ...in memory pointer to zone bit map blockbut not ...on super-block’s device numberdisk point to i-node of mounted file system

point to i-node mounted ontime of last updateread-only flag/dirty flag

The MINIX super block16 bitsmode File type and RWX bitsuid identifies the user who owns the filefile size number of bytes in the filetime of last modification in seconds, since Jan. 1, 1970

32 bytes links | gid directories listing this i-node/owner’s groupzone 0 number... zone numbers for the first 7 data zoneszone 6 numberindirect only used for files larger than 7 zonesdouble indirect

The MINIX i-node (Note! MINIX version 1 size = 32 bytes but version 2.0 size = 64 bytes)

Page 11: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

11

Given the block size and the number of i-nodes, it is easy to calculate the size of the i-nodebit map and the number of blocks of i-nodes. For example, for a 1K block, each block of the bitmap has 1K bytes (8K bits), and thus can keep track of the status of up to 8191 i-nodes (i-node 0always contains zeros and is effectively unused). For 10,000 i-nodes, two bit map blocks areneeded. Since i-nodes are 32 bytes, a 1K block hold up to 32 i-nodes. With 127 usable i-nodes, 4disk blocks are needed to contain them all.

Disk storage can be allocated in units (zones) of 1,2,4,8, or in general 2n blocks. The zonebit map keeps track of free storage in zones, not blocks. For the standard 360K floppy disk MINIXdistribution, the zone and block size are the same (1K), so for a first approximation a zone is thesame as a block on these devices. Note that the number of blocks per zone is not stored in thesuper-block, as it is never needed. All that is needed is the base 2 logarithm of the zone to blockratio, which is used as the shift count to convert zones to blocks and vice versa. For example, with8 blocks per zone, log 2 8 = 3, so to find the zone containing block 128 we shift 128 right 3 bits toget zone 16. Zone 0 is the boot block, but the zone bit map includes only the data zones. The ideabehind zones is to help ensure that disk blocks that belong to the same file are located on thesame cylinder, to improve performance when the file is read sequentially.

Before a disk can be used as a MINIX file system, it must be given the structure of layoutabove. The utility program mkfs has been provided to build file systems. This program can becalled by a command likemkfs /dev/fd1 360. The only complication is what happens when a mounted file system is encountered. To seehow that works, we must look at how mounting is done. When the user types the command/etc/mount /dev/fd1 /useron the terminal, the file system contained on floppy disk 1 mounted on top of /user in the root filesystem. The file systems before and after mounting are shown below:

root file system unmounted file system after mounted / / /

/lib /bin /user /bal /jim /ast /lib /bin /user

/ast/f1 /ast/f2 /user/bal /jim /ast

/user/ast/f1 /user/ast/f2

Page 12: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

12

struct inode {unshort i_mode;uid i_uid;file_pos i_size;real_time i_modtime;gid i_gid;links i_nlinks;zone_nr i_zone[NR_ZONE_NUMS];

/* the following items are not present on the disk. */dev_nr i_dev; /* which device is the i-node on */inode_nr i_num; /* i-node number on its (minor) device */short int i_count; /* # item i-node used; 0 means slot is free */char i_dirt; /* CLEAN or DIRTY */char i_pipe; /* set to I_PIPE if pipe */char i_mount; /* THIS BIT SET IF FILE MOUNTED ON */char i_seek; /* set on LSEEK, cleared on READ/WRITE */

}

The key to the whole mount business is a flag set in the i-node of /user after a successfulmount. This flag (i_mount) indicates that the i-node is mounted on. We see that super-blocks inmemory contain two fields related to mounted file systems. The first of these, the “i-node ofmounted file system”, is set to point to the root i-node of the newly mounted file system. Thesecond, the “i-node mounted on” , is set to point to the i-node mounted 0n, in this case, the i-nodefor /user. When a path such as /user/ast/f2 is being looked up, the file system will see a flag in the i-node for /user and realize that it must continue searching at the root i-node of the file systemmounted on /user. The question is: “How does it find this root i-node?” The answer isstraightforward. The system searches all the super-blocks in memory until it finds the one whose i-node mounted on field points to /user. This must be the super-block for the file system mounted on/user. Once it has the mounted file system. Now the file system can continue searching. In thisexample, it looks for ast in the root directory of floppy disk 1.

Example, DOS file system reserved sector FAT copy FAT [ ... copy FAT] root directory data

boot sector ... ... ...sector 0 1 . . .

Page 13: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

13

boot sector layoutaddress contents type00H jump to boot routine (E9xxx or EBxx90) 3 bytes

03H manufacturer’s name and version number 8 bytes 0BH bytes per sector 1 word 0DH sectors per cluster1 1 byte 0EH number of reserved sectors2 1 word 10H number of FATs 1 byte 11H number of entries in root directory 1 word 13H number of sectors in volume 1 word 15H media descriptor 3 1 byte 16H number of sectors per FAT 1 word 18H sectors per track 1 word 1AH number of read/write heads 1 word 1CH number of hidden sectors 1 word 1EH-1FFH BOOT ROUTINE

Note!1. The number of sectors comprising a cluster depends on the storage medium:device sectors per clusterfloppy disk 1 FAT12

The following table shows the clustering used by DOS version 4.0 for volumes larger than 32megabytes and up to 2 gigabytes.

volume size (M) 128 256 512 1024 2048cluster size 2K 4K 8K 16K 32Ksectors per cluster 4 8 16 32 64

2. Several reserved sectors may follow the boot sector. These can contain additional bootstrap code.3. e.g. code F0H 3.5” disk drive (2 sides, 80 tracks, 18 sectors per track) F8H hard disk (varies)

Page 14: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

14

The FAT layoutThe size of individual entries in the FAT under DOS 1.0 and 2.0 is 12 bits. For DOS 3.0 and

up, the size of an entry in the FAT depends on the number of clusters. If a volume has more than,4096 clusters, then each FAT entry is 16 bits; otherwise each FAT entry is 12 bits.

A 12-bit FAT permits control of 4,096 clusters, which corresponds to 4 sectors per cluster,providing a total of 8 megabytes. Althrough this amount could be expanded by adding more sectorsto a cluster, such an expansion isn’t recommended. Therefore, you’ll find only 16-bit FATs on newerhard drives of 20 megabytes and up, thus allowing the 65,536 maximum of addressable clusters.

The total number of sectors in the volume can be found starting at location 13H in bootsector. Divide this number by the number of sectors per cluster to obtain the number of clusters inthe volume.

The first two entries of the FAT are reserved and aren’t related to the cluster assignment.They are media descriptors, which is also stored in address 15H of the boot sector. Cluster# FAT

0 media descriptor12 match with first cluster of data3 match with second cluster of data4 . . .56

... ...

FAT 12 FAT 16code code meaning000H 0000H cluster is availableFF0H-FF6H FFF0H-FFF6H reserved clusterFF7H FFF7H cluster damaged, not usedFF8H-FFFH FFF8H-FFFFH last file clusterxxxH xxxxH next file cluster

DOS is designed so that several identical copies of the FAT can be kept on a volume. TheDOS CHKDSK command test the various FATs to see if they are identical. If the primary FAT isdamaged, CHKDSK replaces the damaged primary FAT with another FAT.

Page 15: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

15

The Root DirectoryThe root directory of a volume immediately follows the last copy of the FAT. This root

directory (like all subdirectories) consists of 32-bytes entries, in which information about individualfiles, subdirectories, and volume label names can be stored. The maximun number of entries in theroot directory, and therefore its size, is stored at address 11H in boot sector.

Directory entry layoutaddress contents type00H filename (blanks padded with spaces)1 8 bytes08H file extension (blanks padded with spaces) 3 bytes0BH file attribute2 1 byte0CH reserved 10 bytes16H time of last change3 1 word18H date of last change4 1 word1AH first cluster of file 1 word1CH file size 2 words

Note!1. the first byte of the directory entry ,

code meaning00H last directory entryE5H file deleted2EH current directory .

2. attributebit position 7 6 5 4 3 2 1 0

0 1 = write protected 0 = read/write enable1 1 = hidden file (invisible to dir)2 1 = system file3 1 = volume name4 1 = subdirectory5 1 = archive bit (every time a file is created or modified, this bit is set to 1. if a program is used to

backup, BACKUP command, this file, the archive bit is reset to 0.)6 reserved7 reserved

Page 16: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

16

3. time15-11 10-5 4-0 bit position hour minute seconds in 2-second increments (e.g. 13 = 26 seconds)

4. date15-9 8-5 4-0 bit positionyear month dayrelativeto 1980

4.3.3 Disk Space ManagementFile are normally stored on disk, so management of disk space is a major concern to file

system designers. Storing a file as a contiguous sequence of bytes has the obvious problem that if afile grows, it will probably have to be moved on the disk. For this reason, nearly all file systems chopfiles up into fixed-size blocks that need not be adjacent.

Block SizeOnce it has been decided to store files in fixed-size blocks, the question arises of how big

the block should be. Given the way disks are organized, the sector, the track and the cylinder areobvious candidates for the unit of allocation. Having a large allocation unit, such as a cylinder,means that every file, even a 1-byte file, ties up an entire cylinder. On the other hand, using a smallallocation unit means that each file will consist of many blocks. Reading each block normally requiresa seek and a rotational delay, so reading a file consisting of many small blocks will be slow.

Keeping Track of Free BlocksOnce a block size has been chosen, the next issue is how to keep track of free blocks. Two

methods are widely used, as shown in Figure 4.8.

Page 17: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

17

free disk blocks : 16, 17, 1842 230 86 1001101101101100136 162 234 0110110111110111210 612 897 101011011011011097 342 422 011011011011101141 214 140 111011101110111163 160 223 110110101000111121 664 223 000011101101011148 216 160 1011101101101111262 320 126 1100100011101111

310 180 142 0111011101110111516 482 141 1101111101110111

A 1K disk block can hold 256 A bit map 32-bit disk block numbers

(a) (b) Figure 4.8 (a) Storing the free list on a linked list. (b) A bit map

4.3.4 File System ReliabilityBackups

File systems on floppy disk can be backed up by just copying the entire floppy disk to ablank one. File systems on small winchester disks can be backed up by dumping the entire disk tomagnetic tape. Current technologies include 150M cartridge tapes, and 8G Exabyte or DAT tapes.For large winchesters (e.g., 10 GB), backing up the entire drive on tape is awkward and timeconsuming. One strategy that is easy to implement but wastes half the storage is to provide eachcomputer with two drives instead of one. Both drives are divided into two halves: data and backup.Each night the data portion of drive 0 is copied to the backup portion of drive 1, and vice versa. Disk 0 Disk 1

Backup of Backup of Data1 Data 0 Data 0 Data 1

Page 18: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

18

An alternative to dumping the entire file system every day is to make incremental dumps.The simplest form of incremental dumping is to make a complete dump periodically, say weekly ormonthly, and to make a daily dump of only those files that have been modified since the last fulldump. A better scheme is to dump only those files that have changed since they were last dumped.

File System ConsistencyMany file systems read blocks, modify them, and write them out later. If the system crashes

before all the modified blocks have been written out, the file system can be left in an inconsistentstate. To deal with the problem of inconsistent file systems, most computers have a utility programthat checks file system consistency. It can be run whenever the system is booted, especially after acrash. Two kinds of consistency checks can be made: blocks and files. To check for blockconsistency, the program builds two tables, each one containing a counter for each block, initially setto 0. The counters in the first table keep track of how many times each block is present in a file; thecounters in the second table record how often each block is present in the free list (or the bit map offree blocks).

If the file system is consistent, each block will have a 1 either in the first table or in thesecond table, as illustrated in Figure 4.9

block number0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 blocks in use0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 1 free blocks

(a) consistent0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 blocks in use0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 free blocks

(b) missing block0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 blocks in use0 0 1 0 2 0 0 0 0 1 1 0 0 0 1 1 free blocks

(c) duplicate block in free list0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 151 1 0 1 0 2 1 1 1 0 0 1 1 1 0 0 blocks in use0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 1 free blocks

(d) duplicate data blockFigure 4.9 File system states

Page 19: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

19

In (b), block 2 does not occur in either table. While missing blocks do no real harm, they dowaste space and thus reduce the capacity of the disk. The solution to missing blocks isstraightforward: the file system checker just adds them to the free list.

In (c), block number 4 occurs twice in the free list (for bit map it is impossible). The solutionhere is also simple: rebuild the free list.

In (d), the worst thing that the same data block is present in two or more files. Theappropriate action for the file system checker to take is to allocate a free block, copy the contents ofblock 5 into it, and insert the copy into one of the files.

In addition to checking to see that each block is properly accounted for, the file systemchecker also checks the directory system. It too, uses a table of counters, but these are per file,rather than per block. For more information see text book.

4.3.5 File System PerformanceAccess to disk is much slower than access to memory. Reading a memory word typically

takes tens of nanoseconds. Reading a block from hard disk may take fifty microseconds. The mostcommon technique used to reduce disk accesses is the block cache or buffer cache. Variousalgorithms can be used to manage the cache, but a common one is to check all read requests to seeif the needed block is in the cache. If it is, the read request can be satisfied without a disk access. Ifthe block is not in the cache, it is first read into the cache, and then copied to wherever it is needed.Subsequent requests for the same block can be satisfied from the cache. When a block has to beloaded into a full cache, some block has to be removed and rewritten to the disk if it has beenmodified since being brought in. This situation is very much like paging, and all the usuall pagingalgorithms such as FIFO, second chance, and LRU, are applicable.

Caches in which all modified blocks are written back to the disk immediately are calledwrite-through caches. The MS-DOS use this method. UNIX will collect all the characters in the cacheand write the block out once every 30 seconds, or whenever the block is removed from the cache(nonwrite-through caches). However UNIX has a system call, SYNC, which forces all the modifiedblocks out onto the disk immediately. When UNIX is started up, a program, usually called update, isstarted up in the background to sit in an endless loop issuing SYNC calls, sleeping for 30 secbetween calls. As a result, no more than 30 seconds of work is lost due to a crash.Note! Caching is not the only way to increase the performance of a file system. See text book formore methods.

Page 20: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

20

4.4 SECURITYFile systems often contain information that is highly valuable to their users. Protecting this

information against unauthorized usage is therefore a major concern of all file systems. These issueapply equally well to timesharing systems as to networks of personal computers connected to sharedservers via local area networks.

Who want to break security and why? Famous Security Flaws, Trojan horse, Logic bomb, Worm,Viruses

see text book pp. 434-441

User AuthenticationMany protection schemes are based on the assumption that the system knows the identity

of each user. The problem of identifying users when they log in is called user authentication. Forexample, password, physical identification, etc.

4.5 PROTECTION MECHANISMS4.5.1 Protection Domains

A computer system contains many “object” that need to be protected. These objects can behardware (e.g. CPUs, memory segments, disk drives, or printers), or they can be software (e.g.,processes, files, data base, or semaphores). In order to discuss different protection mechanism, it isuseful to introduce the concept of a domain. A domain is a set of (object, rights) pairs. Each pairspecifies an object and some subset of the operations that can be performed on it. A right in thiscontext means permission to perform one of the operations. Figure 4.10 shows three domains,showing the objects in each domain and the rights [Read, Write, eXecute] available on each object.

domain 1 domain 2 domain 3 file3[R] file6[RWX] file4[RWX] printer1[W] file5[RW] plotter2[W]

Figure 4.10 Three protection domains.

file1[R]file2[RW]

Page 21: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

21

An important question is how the system keeps track of which object belongs to whichdomain. Conceptually, at least, one can envision a large matrix, with the rows being the domains andthe columns being the objects. Each box lists the rights, if any, that the domain contains for theobject. The matrix for Figure 4.10 is shown in Figure 4.11.

objectdomain file1 file2 file3 file4 file5 file6 printer1 plotter21 R RW2 R RWX RW W3 RWX W W

Figure 4.11 A protection matrix

4.5.2 Access Control ListsIn practice, actually storing the matrix of Figure 4.11 is rarely done because it is large and

sparse. Two methods that are practical, however, are storing the matrix by rows (capability lists ) orby columns (access control list) and then storing only the nonempty elements.

Let us now assume that we have four users (i.e., uids) Jan, Els, Jelle, and Maaike, whobelong to groups system, staff, student, and student, respectively. Suppose that some files have thefollowing Access Control List (ACL):file0: (Jan, *, RWX)file1: (Jan, system, RWX)file2 : (Jan,*,RW-), (Els, staff, RW-), (Maaike,*,RW-)file3 : (*,student, R--)file4: (Jelle,*,---),(*,student,R--)Note! * means all uids or gids

For UNIX, it provides three bits, rwx, per file for the three domains : owner, the owner’sgroup, and others. This scheme is just the ACL, but compressed to 9 bits. It is clearly less generalthan a full-blown ACL system in practice it is adequate, and its implementation is much simpler andcheaper.

4.5.3 CapabilitiesThe other way of slicing up the matrix is by rows. Associated with each process is a list of

objects that may be accessed, along with an indication of which operations are permitted on each, inother words, its domain. This list is called a capability list or C-lists, and the individual items on it arecalled capabilities.

Page 22: CS323-650 Operating Systems 1 Chapter 4 File …staff.cs.psu.ac.th/iew/cs323-650/chapter4.pdf1 Chapter 4 File Systems We have three essential requirements for long-term information

CS323-650 Operating Systems_________________________________________________________________________________

Computer Science, Prince of Songkla University

22

For example, the capability list for domain 2 in figure 4.11 is as follow:

# Type rights object0 file R-- pointer to file31 file RWX pointer to file42 file RW- pointer to file53 pointer -W- pointer to printer1

4.6 Distributed File SystemsA distributed system is a collection of loosely coupled machines (processors do not share

memory or a clock) interconnected by a communication network. From the point of view of a specificmachine in a distributed system, the rest of the machines and their respective resources are remote,whereas the machine’s own resources are refereed to as local.

In this section we will illustrate only the Sun NFS distributed file systems. See practice9.doc.client server

system calls interface

VFS interface VFS interface

other types UNIX file NFS client NFS server UNIX file of file systems systems systems RPC/XDR RPC/XDR