Introduction to Linux OS

AUBG ICoSCIS TeamProf. Volin Karagiozov

March, 09 – 10, 2013 SWU, Blagoevgrad

Outline• Input/output redirection, pipes• Filters, wild cards• File system and access modes• Protecting files and directories• Sharing files• The power of the “command line”

Filters

• The grep program searches a file or files for lines that have a certain pattern. The syntax is:

grep pattern file(s)• The name "grep" derives from the ed (a UNIX line editor)

command g/re/p which means "globally search for a regular expression and print all lines containing it."

• A regular expression is either some plain text (a word, for example) and/or special characters used for pattern matching. When you learn more about regular expressions, you can use them to specify complex patterns of text.

More commands …

• grep to Find a String

$ cat memo01$ grep homework memo01$ grep –v homework memo01

• wc (word count) command – print the numbers of bytes, words and lines in file

$ wc memo01 $ wc –w memo01$ wc –c memo01 $ wc –l memo01

• head to look at the top of the file

$ head memo01$ head –1 memo01$ head –3 memo01

• tail to look at the end of the file

$ tail memo01$ tail –1 memo01$ tail –4 memo01

More commands …

• sort to display a File in Order$ sort mem01

• uniq to remove adjacent duplicated lines in a fileCopy from my home directory the file named test:

cp /home/vkaragiozov/cos231/test .$ cat test$ uniq test

What about to use first sort command and after it uniq? sort test | uniq Alternative solution – look at sort command options …. sort –u test

One more solution: awk ’ a[$0]++ == 0 ’ test

Using Wildcards - practice

Let’s create quickly the files we need:$ touch chap10 chap2 chap5 cold chap1a.old chap3.old

chap6 haha chap1b chap4 chap7 oldjunkAnd try the following commands:

$ ls chap?$ ls chap??$ ls chap[5-8]$ ls chap*$ ls *old$ ls *a*a*

Exercise: Manipulating files

In this exercise, you'll create, rename and delete filesGo to home directory. Enter cdCopy distant file to working directory. Enter cp /etc/passwd myfileCreate new directory. Enter mkdir tempList working directory. Enter ls -FMove file to new directory. Enter mv myfile tempChange working directory. Enter cd tempCopy file to working directory. Enter cp myfile myfile.twoList filenames with wildcard. Enter ls -l myfile*Remove files. Enter rm myfile*Go up to parent directory. Enter cd ..Remove directory. Enter rmdir tempVerify that directory was removed. Enter ls -F

Input/Output Redirection – practice

$ cd; cat /etc/passwd > passwd$ cat passwd$ who > users$ date > today $ ls$ cat users $ cat today

The >> operator – practice

$ cat today > today_users$ cat users >> today_users$ cat today_users

Exercise: Redirecting input/output

• In the following exercises you'll redirect output, create a simple pipe, and use filters to modify output.

Redirect output to a file. Enter who > usersSort output of a command. Enter who | sortAppend sorted output to a file. Enter who | sort >> usersDisplay output to screen. Enter more users or less usersDisplay long output to screen. Enter ls -l /bin | more or ls -l /bin |

less

What did we learn so far?

• Special Characters & Quoting Characters • Using whereis and which to Find Utilities• Using diff to Compare Two Files• Compressing the files• File and Directory Wildcards• Creating Directories• Removing Files and Directories• Standard Input and Standard Output• Input/Output and Standard Error Redirection• Pipes and Filters

Output from ls -al

Protecting and Sharing Files

• Directory Access PermissionsA directory's access permissions help to control access to the files in it. These affect the overall ability to use files and subdirectories in the directory. (Once you have access to a directory, the ability to read or modify the contents of specific files is controlled by the file access permissions

• File Access PermissionsThe access permissions on a file control what can be done to the file's contents. The access permissions on the directory where the file is kept control whether the file can be renamed or removed.

chmod commandDirectory Access Permissions:• To keep yourself from accidentally removing files

(or adding or renaming files) in a directory, use chmod 555 dirname. To do the same, but also deny other users any access, use chmod 500 dirname.

• To protect the files in a directory and all its subdirectories from everyone else on your system - but still be able to do anything you want to do there - use chmod 700 dirname.

• To let other people on the system see what's in a directory - and read or edit the files if the file permissions let them - but not rename, remove, or add files - use chmod 755 dirname.

• To let people in your UNIX group add, delete, and rename files in a directory of yours - and read or edit other people's files if the file permissions let them - use chmod 775 dirname.

• To give full access to everyone on the system, use chmod 777 dirname.

File Access Permissions• To make a private file that only you

can edit, use chmod 600 filename. To protect it from accidental editing, use chmod 400 filename.

• To edit a file yourself, and let everyone else on the system read it without editing, use chmod 644 filename.

• To let you and all members of your UNIX group edit a file, but keep any other user from reading or editing it, use chmod 660 filename.

• To let nongroup users read but not edit the file, use chmod 664 filename.

• To let anyone read or edit the file, use chmod 666 filename.

Filesystem Permission Bits

Changing Permission Bits to an Octal Number

chmod commandDirectory Access Permissions:• To keep yourself from accidentally removing

files (or adding or renaming files) in a directory, use chmod 555 dirname. To do the same, but also deny other users any access, use chmod 500 dirname.

• To protect the files in a directory and all its subdirectories from everyone else on your system - but still be able to do anything you want to do there - use chmod 700 dirname.

• To let other people on the system see what's in a directory - and read or edit the files if the file permissions let them - but not rename, remove, or add files - use chmod 755 dirname.

• To let people in your UNIX group add, delete, and rename files in a directory of yours - and read or edit other people's files if the file permissions let them - use chmod 775 dirname.

• To give full access to everyone on the system, use chmod 777 dirname.

chmod 555 dirname - r-x r-x r-xchmod 500 dirname - r-x --- ---

chmod 700 dirname - rwx --- ---

chmod 755 dirname - rwx r-x r-x

chmod 775 dirname - rwx rwx r-x

chmod 777 dirname - rwx rwx rwx

chmod commandFile Access Permissions• To make a private file that only you

can edit, use chmod 600 filename. To protect it from accidental editing, use chmod 400 filename.

• To edit a file yourself, and let everyone else on the system read it without editing, use chmod 644 filename.

• To let you and all members of your UNIX group edit a file, but keep any other user from reading or editing it, use chmod 660 filename.

• To let nongroup users read but not edit the file, use chmod 664 filename.

• To let anyone read or edit the file, use chmod 666 filename.

chmod 600 filename - rw- --- ---chmod 400 filename - r-- --- ---

chmod 644 filename - rw- r-- r--

chmod 660 filename - rw- rw- ---

chmod 664 filename - rw- rw- r--

chmod 666 filename - rw- rw- rw-

Changing Permission Bits (letter mode)

• A combination of the letters `ugoa' controls which users‘ access to the file will be changed: the user who owns it (u), other users in the file's group (g), other users not in the file's group (o), or all users (a). If none of these are given, the effect is as if `a' were given.

• The operator `+' causes the permissions selected to be added to the existing permissions of each file; `-' causes them to be removed; and `=' causes them to be the only permissions that the file has.

Changing Permission Bits (letter mode)

• Examples:chmod 600 filename is equivalent to chmod u=rw filename - rw- --- ---

chmod g+r filename - rw- r-- ---chmod o+rx filename - rw- r-- r-xchmod o-x filename - rw- r-- r-- chmod u=r,g-r filename - r -- --- r--chmod -r filename - --- --- ---

chmod a=r filename is equivalent to chmod =r filename - r -- r-- r--

Protecting Files with the Sticky Bit

• UNIX directory access permissions say that if a user has write permission on a directory, he/she can rename or remove files there, even files that don't belong to her. Many newer versions of UNIX have a way to stop that. The owner of a directory can set its sticky bit (mode 1000). The only people who can rename or remove any file in that directory are the file's owner, the directory's owner, and the superuser.

Protecting Files with the Sticky Bit

Here's an example: the user jerry makes a world-writable directory and sets the sticky bit (shown as t here):

jerry$ mkdir share jerry$ chmod 1777 share jerry$ ls -ld share drwxrwxrwt 2 jerry ora 32 Nov 19 10:31 share

• Other people create files in it. When jennifer tries to remove a file that belongs to ellie, she can't:jennifer$ ls -l total 2

-rw- r--r-- 1 ellie ora 120 Nov 19 11:32 data.ellie

-rw-r--r-- 1 jennifer ora 3421 Nov 19 15:34 data.jennifer

-rw-r--r-- 1 peter ora 728 Nov 20 12:29 data.peter

jennifer$ rm data.ellie data.ellie: 644 mode ? y rm: data.ellie not removed. Permission denied

A UNIX Filesystem Tree

Links

• In addition to moving and copying files, UNIX systems also allow you to link them - to have two filenames, perhaps in different directories or even on different filesystems, that point to the same file. We talk about why you'd want to do that, the difference between "hard" and "soft" links, how to create links.

UNIX provides two different kinds of links:– Hard links– Symbolic links (also called soft links or symlinks)

Hard links• With a hard link, two filenames (i.e., two directory

entries) point to the same inode and the same set of data blocks. All UNIX versions support hard links. They have two important limitations:

1. a hard link can't cross a filesystem (i.e., both filenames must be in the same filesystem), and

2. you can't create a hard link to a directory (i.e., a directory can only have one name).

• They have two important advantages: 1. the link and the original file are absolutely and always

identical, and 2. the extra link takes no disk space (except an occasional

extra disk block in the directory file).

Symbolic links• With a symbolic link, there really are two different files.

One file contains the actual data; the other file just contains the name of the first file and serves as a "pointer." We call the pointer the link. The system knows that whenever it opens a link, it should read the contents of the link, and then access the file that really holds the data you want. All Berkeley UNIX systems and System V.4 support symbolic links. Symbolic links are infinitely more flexible than hard links.

1. They can cross filesystems, or even computer systems (if you are using NFS or RFS).

2. You can make a symbolic link to a directory. A symbolic link has its own inode and takes a small amount of disk space to store.

Creating and Removing Links

• The ln command creates both hard and soft (symbolic) links. Only UNIX versions with symbolic links have the -s option, though:

$ ln filename linkname To create a hard link $ ln -s filename linkname To create a symbolic link

Existing file Link to the file

Creating and Removing LinksExample:$ ln colors.1 colors1.hard.link$ ln –s colors.1 colors1.symb.link$ ls –li

i – option to list the inodes

$ls -litotal 01406826 -rw-r--r-- 2 volin users 0 Feb 3 23:42 colors.11406826 -rw-r--r-- 2 volin users 0 Feb 3 23:42 colors.1.hard.link1406827 lrwxrwxrwx 1 volin users 8 Feb 3 23:42 colors.1.symb.link -> colors.1

Hard and symbolic links

chmod and symbolic links

• chmod never changes the permissions of symbolic links; the chmod system call cannot change their permissions. This is not a problem since the permissions of symbolic links are never used.

What did we learn recently?

• Protecting and Sharing Files• Directory Access Permissions• File Access Permissions• chmod command• Protecting Files with the Sticky Bit• Links – hard and symbolic links• Creating and removing links

Output from ls -al

Changing the ownership of files chown command

chown [options] newowner files Change the ownership of one or more files to newowner. (Note: The BSD version lets you change the group as well.) newowner is either a user ID number or a login name located in /etc/passwd.

Options-h change the owner on symbolic links. Normally, chown acts

on the file referenced by a symbolic link, not on the link itself.-R Recursively descend through the directory, including

subdirectories and symbolic links, resetting the ownership ID.

Example:$ chown user_name test$ chown user_name.group_name test

Changing the group ownership of files chgrp command

chgrp [options] newgroup files Change the ownership of one or more files to newgroup. newgroup is either a group ID number or a group name located in /etc/group. You must own the file or be a privileged user to succeed with this command.

Options-h Change the group on symbolic links. Normally, chgrp acts on the

file referenced by a symbolic link, not on the link itself.-R Recursively descend through the directory, including

subdirectories and symbolic links, setting the specified group ID as it proceeds.

Some important files• /etc/passwd – contains information from the

password data base about the registered users

velimiram:x:593:100:Velimira Metodieva,010013258:/home/velimiram:/bin/bash

milenm:x:594:100:Milen Milev,010010041:/home/milenm:/bin/bash

gerganan:x:595:100:Gergana Natcheva,010011606:/home/gerganan:/bin/bash

ivans:x:577:100:Ivan Stoyanov,020008700:/home/ivans:/bin/bash

waiting:x:596:100:Wai Ting,010013337:/home/waiting:/bin/bash

petjan:x:575:0:Petja Nakova,010012574:/home/petjan:/bin/bash

Some important files• /etc/group – contains the names and ID of the

groups and the names of the users in each groupDisplay these files on your terminals (they are more than one page long).

kmem:x:9:

wheel:x:10:root

mail:x:12:mail

news:x:13:news

uucp:x:14:uucp

man:x:15:

games:x:20:

gopher:x:30:

dip:x:40:

ftp:x:50:

nobody:x:99:

users:x:100:peter,eddie,yavor,test,dimitars,nadejdai,martink,hristoh,rossen

Check your UID and group(s)id command

id [ -a ] List user and group IDs; list all groups with -a.

Example:$ iduid=578(volin) gid=100(users) groups=100(users)

or

$ id user_name

Translate utility - tr

tr [options] [string1 [string2]] Copy standard input to standard output, performing

substitution of characters from string1 to string2 or deletion of characters in string1.

Options-d Delete characters in string1 from output.-s Squeeze out repeated output characters in

string2.

Translate utility - trExamples

Change uppercase to lowercase in a file:$ cat file | tr ’[A-Z]’ ’[a-z]’

Turn spaces into newlines (ASCII code 012):$ tr ' ' '\012' < file

Strip blank lines from file and save in new.file (or use \011 to change successive tabs into one tab):

$ cat file | tr -s ”” ”/012” > new.file

Delete colons from file; save result in new.file:

$ cat > fileThis : is : a : file : with : semicolons :: :::

We ::: will :: use ::: tr : utility ::: to remove semicolons.

Ctrl+D$ tr -d : < file > new.file

The tee Utilitytee [options] [files]

Duplicate the standard input; send one copy to standard output and another copy to files.

Options-a Append output to files.-i Ignore all interrupts.

ExamplesDisplay a who listing on the screen, and store it in two files:

$ who | tee userlist ttylist $ ls –la$ cat userlist ttylist

cut utilitycut - remove sections from each line of files cut [OPTION]... [FILE]... Print selected parts of lines from each FILE to standard output.Options:-d, --delimiter=DELIM use DELIM instead of TAB for field delimiter-f, --fields=LIST select only these fields;-c, --characters=LIST select only these characters

Examples• Extract usernames and real names from /etc/passwd:

cut -d: -f1,5 /etc/passwd • Find out who is logged on, but list only login names:

who | cut -d" " -f1 • Cut characters in the fourth column of file, and paste

them back as the first column in the same file:cut -c4 file | paste - file

paste command paste [options] files Merge corresponding lines of one or more files into vertical columns, separated

by a tab. Options-

Replace a filename with the standard input.-d'char'

Separate columns with char instead of a tab. char can be any regular character or the following escape sequences:

\n Newline\t Tab\ Backslash\0 Empty String

Examples• Create a three-column file from files x, y, and z:

paste x y z > file • List users in two columns:

who | paste - -

Running a Program in the Background

Running a command as a background process is most often done to free a terminal when you know the command will take a long time to run.

To run a command in the background, add the "&" character at the end of the command line before you press the [RETURN] key. The shell will then assign and display a process ID number for the command:

$ nroff -ms chap1 > chap1.out & [1] 29890$

(The nroff program formats documents. It's a good example because text formatting usually takes a while, so users often do it in the background. See your UNIX documentation for details on nroff.)

Job ControlOn many systems, the shells have another feature called job control. You can

use the suspend character (usually [CTRL-Z] to suspend a program running in the foreground. The program will pause and you'll get a new shell prompt. You can then do anything else you like, including putting the suspended program into the background using the bg command. The fg command will bring a background process to the foreground.

For example, you might start sort running on a big file, and, after a minute, want to send email. You stop sort, then put it in the background. The shell prints a message, then another shell prompt. You send mail while sort runs.

$ sort hugefile1 hugefile2 > sorted ...time goes by... [CTRL-Z] Stopped $ bg [1] sort hugefile1 hugefile2 > sorted & $ mail eduardo@nacional.cl …… etc.

Checking on a Processps command

If a background process takes too long, or you change your mind and want to stop a process, you can check the status of the process and even cancel it.

When you enter the single-word command ps, you can see how long a process has been running. The output of ps also tells you the process ID of the background process and the terminal from which it was run.

$ ps PID TTY TIME COMMAND 8048 020 0:12 sh 8699 020 0:02 ps $

In its basic form, ps lists the following:Process ID (PID) A unique number assigned by UNIX to the process.Terminal line (TTY) The terminal number from which the process was started.Run time (TIME) The amount of computer time (in minutes and seconds)

that the process has used.Command (COMMAND) The name of the process.

ps - command

EXAMPLESTo see every process on the system using standard syntax: ps -eTo see every process on the system using BSD syntax: ps axTo see every process except those running as root

(real & effective ID) ps -U root -u root -NTo see every process with a user-defined format:

ps -eo pid,tt,user,fname,tmout,f,wchanOdd display with AIX field descriptors:

ps -o "%u : %U : %p : %a"Print only the process IDs of syslogd:

ps -C syslogd -o pid=

Controlling processes

• For every process that is created the UNIX operating system stores information including

• its real UID, GID and its effective UID and GID • the code and variables used by the process (its address

map) • the status of the process • its priority • its parent process Try ps –ax command

Canceling a Process

The kill command aborts a process. The command's format is:kill PID(s)

kill terminates the designated process IDs (shown under the PID heading in the ps listing). If you do not know the process ID, do a ps first to display the status of your processes.

In the following example, the "sleep n" command simply causes a process to "go to sleep" for n number of seconds. We enter two commands, sleep and who, on the same line, as a background process.

Example$ (sleep 60; who) & [1] 21087 $ ps PID TTY TIME COMMAND 20055 4 0:10 sh 21087 4 0:01 sh 21088 4 0:00 sleep 21089 4 0:02 ps $ kill 21088 Terminated $ tom tty2 Aug 30 11:27 grace tty4 Aug 30 12:24 tim tty5 Aug 30 07:52

dale tty7 Aug 30 14:34

We decided that 60 seconds was too long a time to wait for the output of who. The ps listing showed that sleep had the process ID number 21088, so we used this PID to kill the sleep process. You should see a message like "terminated" or "killed"; if you don't, use another ps command to be sure the process has been killed.The who command is executed immediately, since it is no longer waiting on sleep; it lists the users logged into the system.

Problem checklist

• The process didn't die when I told it to. Some processes can be hard to kill. If a normal kill of these

processes is not working, enter "kill -9 PID". This is a sure kill and can destroy almost anything, including the shell that is interpreting it.

In addition, if you've run an interpreted program (like a shell script), you may not be able to kill all dependent processes by killing the interpreter process that got it all started; you may need to kill them individually. However, killing a process that is feeding data into a pipe will generally kill any processes receiving that data.