Chapter 5 File Management

Chapter 5Chapter 5 File Management File Management

FILE SYSTEM

ContentsContents

File and file system overviewFile and file system overview

Logical organization of fileLogical organization of file

File directories and file sharing File directories and file sharing

Physical organization of file (secondary storage Physical organization of file (secondary storage

management)management)

FILE SYSTEM

5.1 File and File System Overview5.1 File and File System Overview

Basic Concept (pp501) Basic Concept (pp501)

٭ FieldField

٭ RecordRecord

٭ FileFile

٭ DatabaseDatabase

٭ File Management SystemFile Management System

٭ Database Management SystemDatabase Management System

FILE SYSTEM

FieldField

A field is the basic element of data A field is the basic element of data

Field may be fixed length or variable lengthField may be fixed length or variable length

Attribution of field include name, length, type, Attribution of field include name, length, type,

default…default…

FILE SYSTEM

RecordRecord

A record is a collection of related fields that can be A record is a collection of related fields that can be

treated as a unit by some application programs.treated as a unit by some application programs.

A record may be fixed length or variable length.A record may be fixed length or variable length.

Key field: can be seen as the identify of the record.Key field: can be seen as the identify of the record.

FILE SYSTEM

FileFile

A file is a collection of similar records or a A file is a collection of similar records or a

streaming of byte.streaming of byte.

Access control restrictions may apply at file or Access control restrictions may apply at file or

record or field.record or field.

FILE SYSTEM

File AttributionFile Attribution

NameName

LengthLength

Access controlAccess control

Create time , modify timeCreate time , modify time

Physical addressPhysical address

Where they are?Where they are?

FILE SYSTEM

DatabaseDatabase

Database is a collection of related data. Database is a collection of related data.

The database itself consists of one or more types of files.The database itself consists of one or more types of files.

FILE SYSTEM

Database OperationsDatabase Operations

Retrieve_allRetrieve_all

Retrieve_oneRetrieve_one

Retrieve_nextRetrieve_next

Retrieve_previousRetrieve_previous

Insert_oneInsert_one

Delete_oneDelete_one

Update_oneUpdate_one

Retrieve_fewRetrieve_few

FILE SYSTEM

File Management Systems (p503)File Management Systems (p503)

A set of system software to manage files of the system A set of system software to manage files of the system

and provides services to users and applications in the and provides services to users and applications in the

use of files.use of files.

FILE SYSTEM

File Management SystemsFile Management Systems

A separate file management systems may provide for A separate file management systems may provide for

some special storage systems.some special storage systems.

Mostly, the entire file management system is considered Mostly, the entire file management system is considered

part of the operating system.part of the operating system.

FILE SYSTEM

Database Management SystemsDatabase Management Systems

It can be independent of operation system.It can be independent of operation system.

It can have an independent file system or use file systems of It can have an independent file system or use file systems of

operation system.operation system.

Database file is referred as structure file.Database file is referred as structure file.

FILE SYSTEM

Objectives for a File System pp503Objectives for a File System pp503

To meet(To meet( 满足满足 ) the management requirements of the u) the management requirements of the u

ser.ser.

To guarantee the data in file are valid.To guarantee the data in file are valid.

To optimize performance of the file access.To optimize performance of the file access.

To support for a variety of storage device type.To support for a variety of storage device type.

FILE SYSTEM

Objectives for a File SystemObjectives for a File System

To eliminate the potential for lost data.To eliminate the potential for lost data.

To provide a standardized set of I/O interface.To provide a standardized set of I/O interface.

To provide I/O support for multiple users.To provide I/O support for multiple users.

FILE SYSTEM

Minimum RequirementMinimum Requirement

CreateCreate 、、 delete and update file.delete and update file.

Each user may have controlled access to other users’ files.Each user may have controlled access to other users’ files.

Each user may control what type of accesses are allowed to Each user may control what type of accesses are allowed to

the users’ files.the users’ files.

Each user should be able to restructure the user’s files in a Each user should be able to restructure the user’s files in a

form appropriate to the problem.form appropriate to the problem.

FILE SYSTEM

Minimum RequirementMinimum Requirement

Each user should be able to move data between files.Each user should be able to move data between files.

Each user should be able to backup and recover the useEach user should be able to backup and recover the use

r’s files in case of damage.r’s files in case of damage.

To access the user’s files by a symbolic(To access the user’s files by a symbolic( 符号符号 ) name.) name.

FILE SYSTEM

Example : FAT Example : FAT

BOOT sectionBOOT section FDTFDT （（ file directory tablefile directory table ）） FAT1/FAT2FAT1/FAT2 （（ file allocation table)file allocation table) DATA sectionDATA section

FILE SYSTEM

磁盘盘区划分表磁盘盘区划分表

区名内容软盘占扇区数扇区号

保留区引导记录与磁盘参数表

1 0

控制区 FAT1 文件分配表

2 1 － 2

FAT2 2 3 － 4

FDT 文件目录表

7 5 － 11

文件区文件内容余下部分 ≥ 12

FILE SYSTEM

操作流程操作流程

文件名文件名磁盘目录表磁盘目录表FDTFDT

磁盘参数表磁盘参数表

文件位置分配文件位置分配表表 FATFAT

磁盘扇区磁盘扇区定位定位

扇区物理扇区物理操作操作

磁盘基数表磁盘基数表

FILE SYSTEM

File Operations File Operations

Read/writeRead/write LseekLseek LstateLstate UnlinkUnlink OpenOpen CloseClose …………

FILE SYSTEM

File Management Function pp505File Management Function pp505

Identify and locate the selected file: LookupIdentify and locate the selected file: Lookup （查（查找）找） the directories of the file systems to get the lthe directories of the file systems to get the l

ocation of the file.(ocation of the file.( 按名存取按名存取 ))

In sharing file systems, accessing control should In sharing file systems, accessing control should

be verified.be verified.

FILE SYSTEM

File Management FunctionFile Management Function

To map the logical unit (record/byte) to physical unit To map the logical unit (record/byte) to physical unit

(block) .(block) .

To allocate free block for files.To allocate free block for files.

To manage the free block of secondary storage.To manage the free block of secondary storage.

FILE SYSTEM

Elements of File Management Elements of File Management

FILE SYSTEM

File System Architecture(p504)File System Architecture(p504)

FILE SYSTEM

Device DriversDevice Drivers

Lowest level.Lowest level.

Communicates directly with peripheral devices(Communicates directly with peripheral devices( 外外设设 ).).

Responsible for starting I/O operations on a device.Responsible for starting I/O operations on a device.

Processes the completion of an I/O request.Processes the completion of an I/O request.

FILE SYSTEM

Basic File SystemBasic File System

Physical I/O.Physical I/O.

Deals with exchanging blocks of data.Deals with exchanging blocks of data.

Concerned with the placement of blocks between Concerned with the placement of blocks between disk-cash and memory.disk-cash and memory.

FILE SYSTEM

Basic I/O SupervisorBasic I/O Supervisor

Responsible for file I/O initiation and termination.Responsible for file I/O initiation and termination.

Control structures (e.g.SDT) are maintained.Control structures (e.g.SDT) are maintained.

Concerned with scheduling access to optimize Concerned with scheduling access to optimize performance.performance.

Part of the operating system.Part of the operating system.

FILE SYSTEM

Logical I/OLogical I/O

Enables users and applications to access records.Enables users and applications to access records.

Provides general-purpose record I/O capability.Provides general-purpose record I/O capability.

FILE SYSTEM

Access MethodAccess Method

Reflect different file structures.Reflect different file structures.

Different ways to store and process data.Different ways to store and process data.

FILE SYSTEM

Criteria for File Organization(p507)Criteria for File Organization(p507)

Rapid AccessRapid Access

٭ Needed when accessing a single record.Needed when accessing a single record.

٭ Not needed for batch mode.Not needed for batch mode. Ease of updateEase of update

٭ File on CD-ROM will not be updated, so this is not File on CD-ROM will not be updated, so this is not a concern.a concern.

FILE SYSTEM

Criteria for File OrganizationCriteria for File Organization

Economy of storageEconomy of storage

٭ Should be minimum redundancy in the data.Should be minimum redundancy in the data.

٭ Redundancy can be used to speed access such as an Redundancy can be used to speed access such as an index.index.

Simple maintenance.Simple maintenance.

Reliability.Reliability.

FILE SYSTEM

5.2 Logical Organization of File5.2 Logical Organization of File

The pile.The pile.

The sequential file.The sequential file.

The indexed sequential file.The indexed sequential file.

The indexed file.The indexed file.

The direct, or hashed file.The direct, or hashed file.

FILE SYSTEM

The PileThe Pile

٭ Data are collected in the order they arrive.Data are collected in the order they arrive.

٭ Purpose is to accumulate a mass of data and save Purpose is to accumulate a mass of data and save it.it.

٭ Records may have different fields.Records may have different fields.

٭ No structure.No structure.

٭ Record access is by exhaustive search.Record access is by exhaustive search.

FILE SYSTEM

The PileThe Pile

FILE SYSTEM

The Sequential FileThe Sequential File

٭ Fixed format used for records so records are the Fixed format used for records so records are the same length.same length.

٭ All fields the same (order and length).All fields the same (order and length).

٭ Field names and lengths are attributes of the file.Field names and lengths are attributes of the file.

٭ One field is the key field.One field is the key field.

▪ Uniquely identifies the record.Uniquely identifies the record.

▪ Records are stored in key sequence.Records are stored in key sequence.

FILE SYSTEM


Is the only type that is easily stored on tape as well as disk.Is the only type that is easily stored on tape as well as disk.

Poor performance.Poor performance.

New records are placed in a log file or transaction file.New records are placed in a log file or transaction file.

Batch update is performed to merge the log file with the Batch update is performed to merge the log file with the master file.master file.

FILE SYSTEM


FILE SYSTEM

Indexed Sequential FileIndexed Sequential File

Index provides a lookup(Index provides a lookup( 查找查找 ) capability to quickly reach t) capability to quickly reach the vicinityhe vicinity （邻近）（邻近） of the desired record.of the desired record.

٭ Contains key field and a pointer to the main file.Contains key field and a pointer to the main file.

٭ Indexed is searched to find highest key value that is equal Indexed is searched to find highest key value that is equal or less than the desired key value.or less than the desired key value.

٭ Search continues in the main file at the location indicated Search continues in the main file at the location indicated by the pointer.by the pointer.

FILE SYSTEM

Comparison of Sequential and Indexed SequentialComparison of Sequential and Indexed Sequential

٭ Example: a file contains 1 million recordsExample: a file contains 1 million records

٭ On average 500,000 accesses are required to find a On average 500,000 accesses are required to find a record in a sequential filerecord in a sequential file

٭ If an index contains 1000 entries, it will take on If an index contains 1000 entries, it will take on average 500 accesses to find the key, followed by 500 average 500 accesses to find the key, followed by 500 accesses in the main file. Now on average it is 1000 accesses in the main file. Now on average it is 1000 accesses.accesses.

FILE SYSTEM

Indexed Sequential File (Insert)Indexed Sequential File (Insert)

٭ New records are added to an overflow file.New records are added to an overflow file.

٭ Record in main file that precedes it is updated to Record in main file that precedes it is updated to contain a pointer to the new record.contain a pointer to the new record.

٭ The overflow is merged with the main file during a The overflow is merged with the main file during a batch update.batch update.

٭ Efficiency is not high , and only one index.Efficiency is not high , and only one index.

FILE SYSTEM

Indexed Sequential FileIndexed Sequential File

FILE SYSTEM

Indexed FileIndexed File

Uses multiple indexes for different key fields.Uses multiple indexes for different key fields.

May contain an exhaustiveMay contain an exhaustive （耗尽）（耗尽） index that contaiindex that contains one entry for every record in the main file.ns one entry for every record in the main file.

May contain a partial index.May contain a partial index.

The index is a sequential file with fixed records.The index is a sequential file with fixed records.

FILE SYSTEM

Indexed FileIndexed File

FILE SYSTEM

Hashed FileHashed File

Directly access a block at a known addressDirectly access a block at a known address

Key field required for each recordKey field required for each record

FILE SYSTEM

5.3 File Directories (table 12.2)5.3 File Directories (table 12.2)

Contains information about filesContains information about files

٭ AttributesAttributes

٭ LocationLocation

٭ OwnershipOwnership

Directory itself is a Directory itself is a filefile owned by the operating system owned by the operating system

Provides mapping between file names and the files Provides mapping between file names and the files themselvesthemselves

FILE SYSTEM

Simple Structure for a DirectorySimple Structure for a Directory

It may only include the key elements: name,address,size It may only include the key elements: name,address,size and organization.and organization.

So most of the files’ directory can cash to memory to So most of the files’ directory can cash to memory to improve the search speed.improve the search speed.

FILE SYSTEM

Basic OperationsBasic Operations

SearchSearch （（ lookuplookup ）） Create fileCreate file Delete fileDelete file List directoryList directory

FILE SYSTEM

Type of DirectoryType of Directory

Single-levelSingle-level

Two-levelTwo-level

Multi-levelMulti-level

FILE SYSTEM

Single LevelSingle Level

List of entries, one for each file.List of entries, one for each file.

Sequential file with the name of the file serving as thSequential file with the name of the file serving as the key.e key.

Provides no help in organizing the files.Provides no help in organizing the files.

Forces user to be careful not to use the same name foForces user to be careful not to use the same name for two different files.(r two different files.( 不能重名不能重名 ))

FILE SYSTEM

Two LevelTwo Level

One directory for each user and a master directoryOne directory for each user and a master directory（主目录）（主目录） ..

Master directory contains entry for each user.Master directory contains entry for each user.

٭ Provides address and access control informationProvides address and access control information

Each user directory is a simple list of files for that useEach user directory is a simple list of files for that user.r.

Still provides no help in structuring collections of files.Still provides no help in structuring collections of files.

FILE SYSTEM

Multi LevelMulti Level

Master directory with user directories underneath it.Master directory with user directories underneath it.

Each user directory may have subdirectories and Each user directory may have subdirectories and files as entries.files as entries.

How can lookup a file in multi-level?How can lookup a file in multi-level?

FILE SYSTEM

FILE SYSTEM

FILE SYSTEM

PathnamePathname

Files can be located by following a path from the root, oFiles can be located by following a path from the root, or master, directory down various branchesr master, directory down various branches

٭ This is the pathname for the fileThis is the pathname for the file

Can have several files with the same file name as long asCan have several files with the same file name as long as(( 只要只要 ) they have unique path names) they have unique path names

Working Directory.Working Directory.

FILE SYSTEM

Work DirectoryWork Directory

FILE SYSTEM

InodeInode

Most of information storage into inode, and a directory eMost of information storage into inode, and a directory entry has a pointer to the relative inode.ntry has a pointer to the relative inode.

Using inode can improve the accessing speed for files.Using inode can improve the accessing speed for files.

FILE SYSTEM

File SharingFile Sharing

In multiuser system, allow files to be shared among useIn multiuser system, allow files to be shared among users.rs.

Two issues.Two issues.

٭ Access rightsAccess rights

٭ Management of simultaneous accessManagement of simultaneous access

FILE SYSTEM

Access RightsAccess Rights

Users or groups: e.g.owner, specific user or groups, allUsers or groups: e.g.owner, specific user or groups, all

None,knowledge,execution,reading, appending, None,knowledge,execution,reading, appending, updating, changing protection, deletion.updating, changing protection, deletion.

FILE SYSTEM

Access RightsAccess Rights

OwnersOwners

٭ Has all rights previously listedHas all rights previously listed

٭ May grant rights to others using the following classes of May grant rights to others using the following classes of usersusers

▪ Specific userSpecific user

▪ User groupsUser groups

▪ All for public filesAll for public files

FILE SYSTEM

Modify RightModify Right

FILE SYSTEM

Simultaneous AccessSimultaneous Access

User may lock entire file when it is to be updatedUser may lock entire file when it is to be updated

User may lock the individual records during the User may lock the individual records during the updateupdate

Mutual exclusion and deadlock are issues for Mutual exclusion and deadlock are issues for shared accessshared access

FILE SYSTEM

5.4 Physical Organization5.4 Physical Organization

The organization of the file data on secondary The organization of the file data on secondary storage.storage.

FILE SYSTEM

Record BlockingRecord Blocking

The logical unit of file is byte or record, and the physicThe logical unit of file is byte or record, and the physical unit of file is block.al unit of file is block.

Block type:Block type:

٭ Fixed blockingFixed blocking

٭ Variable-length spanned blockingVariable-length spanned blocking

٭ Variable-length unspanned blockingVariable-length unspanned blocking

FILE SYSTEM

Record BlockingRecord Blocking

Choose fixed length or variable lengthChoose fixed length or variable length

٭ Fixed is higher efficiency of I/O transfer ,memory Fixed is higher efficiency of I/O transfer ,memory allocation ….allocation ….

How to choose the size of blockHow to choose the size of block

٭ Large blocking is efficiency (because of locality) for Large blocking is efficiency (because of locality) for transfer, but needs more disk-cash transfer, but needs more disk-cash

FILE SYSTEM

Fixed Blocking (Fixed-Length Records)Fixed Blocking (Fixed-Length Records)

FILE SYSTEM

Variable Blocking: SpannedVariable Blocking: Spanned

Variable-length records, spannedVariable-length records, spanned

FILE SYSTEM

Variable Blocking UnspannedVariable Blocking Unspanned

Variable-length records , unspannedVariable-length records , unspanned

FILE SYSTEM

ConclusionConclusion

Variable length is efficient of storage but is difficult Variable length is efficient of storage but is difficult to implement.to implement.

FILE SYSTEM

Secondary Storage ManagementSecondary Storage Management

Space must be allocated to filesSpace must be allocated to files

Must keep track of the space available for allocationMust keep track of the space available for allocation

FILE SYSTEM

Portion SizePortion Size

Variable-size or fixed-sizeVariable-size or fixed-size

variable: complexity but efficiencyvariable: complexity but efficiency

Large-size or small-sizeLarge-size or small-size

large: efficiency for transferlarge: efficiency for transfer

FILE SYSTEM

Pre-AllocationPre-Allocation

Need the maximum size for the file at the time of Need the maximum size for the file at the time of creationcreation

Difficult to reliably estimate the maximum potential Difficult to reliably estimate the maximum potential size of the filesize of the file

Tend to overestimated file size so as not to run out Tend to overestimated file size so as not to run out of spaceof space

FILE SYSTEM

Methods of File AllocationMethods of File Allocation

Contiguous allocationContiguous allocation

٭ Single set of blocks is allocated to a file at the time of Single set of blocks is allocated to a file at the time of creationcreation

٭ Only a single entry in the file allocation tableOnly a single entry in the file allocation table

▪ Starting block and length of the fileStarting block and length of the file

External fragmentation will occurExternal fragmentation will occur

FILE SYSTEM

FILE SYSTEM

FILE SYSTEM


Chained allocationChained allocation

٭ Allocation on basis of individual blockAllocation on basis of individual block

٭ Each block contains a pointer to the next block in Each block contains a pointer to the next block in the chainthe chain

٭ Only single entry in the file allocation tableOnly single entry in the file allocation table

▪ Starting block and length of fileStarting block and length of file No external fragmentationNo external fragmentation Best for sequential filesBest for sequential files No accommodation of the principle of localityNo accommodation of the principle of locality

FILE SYSTEM

FILE SYSTEM

FILE SYSTEM


Indexed allocationIndexed allocation

٭ File allocation table contains a separate one-level File allocation table contains a separate one-level index for each fileindex for each file

٭ The index has one entry for each portion The index has one entry for each portion allocated to the fileallocated to the file

٭ The file allocation table contains block number The file allocation table contains block number for the indexfor the index

FILE SYSTEM

FILE SYSTEM

FILE SYSTEM

UNIX File ManagementUNIX File Management

Types of filesTypes of files

٭ OrdinaryOrdinary

٭ DirectoryDirectory

٭ SpecialSpecial

٭ NamedNamed

FILE SYSTEM

Documents

Chapter 5 File Management