Files & File System Structure Most users work mostly with the file system structure and are...

Files & File System Structure•Most users work mostly with the file system structure and are constantly performing file-related operations such as creating, reading, writing/modifying, deleting.

•Operating system concepts and principles and Unix/Linux internals

•Unix/Linux in this book deals with local file systems.•Unix/Linux network file system (NFS) in Ch. 14.

Files & File System Structure

• What is a file?A file is a sequence of bytes.

• NIC, disk drive, kybd, printer, file, directory, input/oupt devices are treated as files in Unix/Linux.

Files & File System Structure

• Unix/Linux supports 6 types of files:

1. Simple/ordinary file

2. Directory

3. Symbolic (soft) link

4. Special file (device)

5. Named pipe (FIFO)

6. Socket

Files & File System Structure• Simple/Ordinary Files• Are used to store information & data on

secondary storage, e.g. disk.• Can contain a source program (in C, C++,

Java, html, etc.)• Can contain an executable program

(compiler, database tools, desktop publishing tools, graphing software, postcript code, pictures, audio, graphics, etc.)

Files & File System Structure

• Unix/Linux treats NO difference to a file’s contents. (Simply a sequence of bytes)

Files & File System Structure• Unix/Linux does NOT impose any naming

conventions.

• .exe may be given to a executable pgm.

• .doc may be given to a document file.

• C compiler requires that C source programs have a .c extension.

Table 7.1 Commonly Used Extensions for Some Applications

Files & File System Structure• Directory (folder, drawer, cabinet)• Contains the names of other files and/or

directories. (Sub-directories)• Unix file names and directory are lists of "link"

structures, each of which contains one filename and one inode number.

• Structure of a directory entry:

Files & File System Structure

• inode (index node)• Inode number is 4 bytes long.• Inode is an index value for an array on the disk.• Every unique file has a unique inode number.

Files & File System Structure• Link File

• A link file is a symbolic link to a existing file. (symbolic link is a soft link.)

• A link file points to an existing file and allows you to rename an existing file without duplicating its contents.

$ mkdir -p /tmp/one$ echo "test_a" >/tmp/one/a $ echo "test_b" >/tmp/one/b $ cd /tmp/one $ ls -l -rw-r--r-- 1 user group 7 Jan 01 10:01 a -rw-r--r-- 1 user group 7 Jan 01 10:01 b $ cd /tmp $ ln -s /tmp/one three $ ls -l /tmp/three

lrwxrwxrwx 1 user group 12 Jul 22 10:02 /tmp/three -> /tmp/one $ ls -l /tmp/three/ -rw-r--r-- 1 user group 7 Jan 01 10:01 a -rw-r--r-- 1 user group 7 Jan 01 10:01 b $ cd three $ ls -l -rw-r--r-- 1 user group 7 Jan 01 10:01 a -rw-r--r-- 1 user group 7 Jan 01 10:01 b $ cat a test_a $ cat /tmp/one/a test_a $ echo "test_c" >/tmp/one/a $ cat /tmp/one/a test_c $ cat /tmp/three/a test_c

Files & File System Structure• Special File (Device)• A means of accessing hardware devices

(kybd, disk, printer, CD-ROM, DVD, etc.)• Special files are divided into 2 types:

1. Character special file (kybd)

2. Block special file (disk)• Special files are stored in /dev directory

fd0 (floppy drive 0); hda (hard drive a);

lp0 (line printer 0); tty0 (teletype terminal 0)

Files & File System Structure• Named Pipe (FIFO)

• Interprocess communication, IPC, mechanisms enables process to communication with each other is the key to client-server software paradigm.

• Three commonly used:Pips, named pipes (FIFO), sockets.

Files & File System Structure• A pipe is an area in the kernel memory (buffer)

that allows two processes to communicate with each other. (They are related to each other.)

• FIFO is a file (of named pipe type) that allows two processes to communicate with each other; but the processes do not have to be related to each other. (Ch. 12 Redirection & Piping)

• A socket is a data structure in the kernel memory that can be used by processes on different computer to communicate with each other. (Computer can be on a network (intranet) or on the Internet. (such as TCP/IP)

Files & File System Structure

• File System Organization Hierarchical (parent-child relationship) Starts with one main directory, /, root directory.

A typical LINUX file system structure

Files & File System Structure• Home and Present Working Directories• /home/login directory e.g. /home/faculty/tengc• Using the tilde (~) character to specify your home

directory• e.g. cd ~• present working directory (pwd); (current directory)• The pwd is denoted . (pronounced dot)• The parent of the pwd id denoted .. (dot dot)

Files & File System Structure• Pathname:Absolute and Relative• A file/directory is specified by a pathname.

• Pathname can be specified in 3 ways:1. Starting with the root directory

2. Starting with the present working directory

3. Starting with the user’s home directory

1: Absolute

2 & 3: Relative

Files & File System Structure• A typical Unix/Linux system has several

disks and/or partitions.

• Disks/Partitions can be mounted on the same file system structure, allowing their access as directories and not as named drives A:, B:, C:, had, etc.

Files & File System Structure• Every Linux system contains a set of standard

files and directories organized according to the File System Standard (FSSTND) proposed in 1994.

Files & File System Structure• Root Directory ( / ) The top of the file system hierachy Contains some standard files and directories

Files & File System Structure• /bin (bindary directory) Contains binary (executable) images of most

essential LINUX commands for SAs and users. All files are either executable files or symbolic

links to executable files in some other directories. (bash, cat, chmod, cp, date, echo, kill, ln, ls,mkdir, vi, more, mv, ps, pwd, rm, etc.)

Contains commands for system restoration. (tar, gzip, gunzip, zcat, etc.)

Contains networking commands. (domainname, nisdomainname, hostname, netstat, ping, etc.)

Files & File System Structure• /boot Contains all files needed to boot the Linux

system, including the binary image of the Linux kernel.

Kernel file name is vmlinux (or vmlinuz) followed by the version and release information.

e.g. /boot/vmlinux-2.2.5

Files & File System Structure• /dev (Device Directory)• Contains files corresponding to the devices

connected to the computer.• Character special files & block special files• Character special files: for devices perform

character-oriented I/O, such as kybd.• Block special files: for devices perform I/O in

terms of blocks or chunk of bytes, such as disk drive.

Files & File System Structure• /etc (host specific files and directories) Contains system configuration files. Does NOT contain binaries. Primarily for the use of SA only. An average user has read permission for most of

these files. e.g. passwd, syslog.conf, inetd.conf, services,

hosts.allow, hosts.deny, resolv.conf, snmp, exports, etc.

Files & File System Structure• /home (users’ home directories) The setup differs from host to host.

• /lib (Contains a collection of related object image files for a given language in a single file called an archive.)

• e.g. contains libraries for C, C++, FORTRAN such as /lib/libc.so.*, libm.so.*, etc.

Files & File System Structure• /lost+foundContains files on the system not

connected to any directory.e.g. files found by fsck (file system check),

superblocks.mostly for SA

Files & File System Structure• /mnt Primarily used by SA to mount file systems

temporarily by using the mount command. e.g. Contains the cdrom, disk, dvd, mount

points. Access the files on a CD-ROM/DVD as files

under the /mnt/cdrom or /mnt/dvd.

Files & File System Structure• /opt

• This directory is used to install add-on software package.

/opt/package_name/bin/opt/package_name/mane.g. /opt/omni/bin/xomni

Files & File System Structure• /procContains process and system information

• /rootHome directory of root account

• /tmpContains temporary files.

Files & File System Structure• /sbin, /usr/sbin, /usr/local/sbin Contains SA tools, utilities, & root-only

commands. e.g. getty, init, update, mkswap, swapon,

swapoff, halt, reboot, shutdown, fdisk, fsck, fsck.ext3, mkfs, mksfs.ext3, ifconfig, route, etc.

Files & File System Structure• /usr• Contains read-only data that can be shared

between various hosts.• Largest section in Linux file system

Files & File System Structure• Main Subdirectories in /usr (continued on next slide)

Files & File System Structure• Main Subdirectories in /usr (continued from previous slide)

Files & File System Structure• /var Contains the variable data (data that keeps

changing while the system is running.) e.g. /var/log, /var/spool/mail, /var/yp

Files & File System Structure• /etc/passwd One line per user Has 7 fields, separated by colonsJsmith:x:1000:1000:Joe Smith, Room 218, (631)-344-1234; /home/jsmith:/usr/bin/csh

1: login name (user name)

2: passwd (* to deactivate; x: shadow passwd; encrypted password information)

3: user ID

4: principle group ID

5: user information field, comma-separated

6: home deirctory

7: login shell

UID: 0-65535; 0 is super-user; 1-99 reserved

GID: 0-65535; 0-99 reserved

Files & File System StructureNavigating The File Structure• Determine The Absolute Pathname for Your Home Directory $ echo $HOME

/home/faculty/tengc $ pwd

/home/faculty/tengc• Browsing The File System cd; cd /usr; cd ~; cd ~teng ls; ls /home/faculty/tengc; ls ~; ls /opt/omni/*; ls $HOME

• Tilde Expension• $ echo ~

/home/faculty/tengc• $ echo ~chenb

/home/faculty/chenb

Files & File System Structure• Creating Files & Directories Use various editors (pico, vi, emacs, nano, etc)

to create plain text files. Use rm to remove files. Use mkdir to create directories. Use rmdir to remove directories.

Files & File System Structure• Determine File Attributes• $ ls –l

drwx------ 2 jdoe1 student 512 Jun 19 12:06 mail

-rwxr--r-- 1 jdoe1 student 163 May 30 16:09 file1

drwxr----- 4 jdoe1 student 512 May 30 14:24 files

-rwxr-xr-x 1 jdoe1 student 321 May 30 14:36 script

brw-rw-rw- 2 bin bin 2, 64 Dec 8 20:41 fd0

lrwxrwxrwx 1 user group 12 Jul 22 10:02 /tmp/three -> /tmp/one/two

Files & File System StructureSummary of the Output of the ls -l Command (fields are listed left to right)

Files & File System Structure• Determine The Type of A File’s Contents• $ file file.c file /dev/{wd0a,hda} file.c: C program text file: ELF 32-bit LSB executable, Intel 80386,

version 1 (SYSV), dynamically linked (uses shared libs), stripped

/dev/wd0a: block special (0/0) /dev/hda: block special (3/0)

Files & File System Structure• inode attributes of a file stored in a data structure on the disk

• i-list A list (array) of inode on a disk

• Inode table The Linux kernel maintains a table of inode in the main

memory.• An inode is allocated from the inode table when a file is

opened.• The i-list and inod table are indexed by a file’s inode

number.

Files & File System Structure• The inode number is used to index the inode table,

allowing access to the attributes of a open file.• When a file’s attributes change, the inode in the main

memory is updated; the disk copies of inode are updated at fixed interval.

• For the files that are not open, their inodes reside on the disk.

Files & File System Structure• Contents of an inode

Link Count: the number of differnet names the file has.File Mode: the file was opened for read, write, etc.User ID: ID of the owner of the file….….Access Permission: who can access the file for what type of operationFile’s Location on Disk: FLD is specified by a number of direct and indirect pointers to disk blocks containing file data.

Files & File System Structure• Physical& Logical organization of data on a disk.

Files & File System Structure

Files & File System Structure• Logical view of a disk drive—an array of disk blocks

Files & File System Structure• Relationship between the file lab1.c in a directory and its contents on disk

Files & File System Structure

• $ ls –i /bin/bash $ 130575 /bin/bash• $ ls –i /bin/cat $ 130590 /bin/cat

Files & File System Structure• Standard Files & File Descriptors• When an application needs to open an I/O operation, it

must first open the file and then issue the file operation (read, write, seek, etc.)

• Linux automatically opens 3 files for every I/O command. 1. Standard input (stdin) files, 2. Standard output (stdout) files, 3. Standard error (stderr) files.

By default: stdin: kybd; stdout & stderr: screen

Files & File System StructureRedirection operators:

< for input redirection

> For output & error redirection

File Descriptor: (a small integer associated with open file)

0: stdin; 1: stdout; 2:stderr

Files & File System StructureRedirection Operators

Files & File System Structure• End of File (eof) Marker• The value of eof marker is not a character; it is usually a

small negative integer such as –1.

• The <Ctrl-D> on a new line is end of file of standard input.

e.g. cat a_big_file (to display on a monitor)<Ctrl-D> terminates display.

e.g. /bin/csh<Ctrl-D> exit csh