Week 9 Agenda

Week 9 Agenda Unix Commandsdu, dfwrite, talk, mesgsort, uniq, grepwctar I/O redirection, pipe, tee

Week 9 - Mar 14, 2005

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

du

report the file usage

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

df

report the number of free disk blocks and inodes available on all mounted filesystems

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

write

Send a message to another user

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

talk

talk to another user

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

mesg

Control write access to your terminal

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

sort [options][files]

sorts the lines of the named files

sort - arranges lines in ASCII order allows relative or absolute pathnamesprocesses either filename arguments or standard inputchanges only the output not the fileallows a list of filenamesallows output to be piped or redirectedUNIX Command

Week 9 - Mar 14, 2005

ex. sort file1 file2 file3merges all the files into its workspacesorts the workspacedisplays the results

there are many options to control what characters or fields are used to arrange the lines..... UNIX Command - sort

Week 9 - Mar 14, 2005

Sorting numbers by magnitude -nUNIX Command - sort

cat numberssort numberssort n numbers 66 erasers5 pens10 rulers10 rulers5 pens66 erasers5 pens10 rulers66 erasers

Week 9 - Mar 14, 2005

Sort ignoring leading blanks -b (+0b)

UNIX Command - sortNote: phobos works differently than zenit

cat mystuffsort mystuffsort -b mystuff pens rulerserasers rulers penseraserserasers pens rulers

Week 9 - Mar 14, 2005

-b is used as a simple sort option to ignore leading blanks in Linux and some older systems

Newer commercial versions only use a special field option +0b

Warning!

Week 9 - Mar 14, 2005

Sort ignoring case -f(ie folding upper & lowercase together)UNIX Command - sortNote: phobos works differently than zenit

cat myfriendssort myfriendssort -f myfriendsZoro ALLANaladdinALLANZoro aladdinaladdinALLANZoro

Week 9 - Mar 14, 2005

Sort ignoring punctuation -d

UNIX Command - sortNote: phobos works differently than zenit

cat friends1sort friends1sort -d friends1aladdin!Zoro!Zoro

aladdin!Zoroaladdin

Week 9 - Mar 14, 2005

Combining sort optionsSort ignoring punctuation & case -dfUNIX Command - sortNote: phobos works differently than zenit

sort friends1sort -d friends1sort -df friends1!Zoro

aladdin!Zoroaladdinaladdin

!Zoro

Week 9 - Mar 14, 2005

In your zenit account, if you specify sort,By default, it will ignore the leading space and punctuationWarning!

Week 9 - Mar 14, 2005

Sort in reverse order -r

UNIX Command - sort

sort numberssort n numberssort rn numbers 10 rulers5 pens66 erasers5 pens10 rulers66 erasers66 erasers10 rulers5 perns

Week 9 - Mar 14, 2005

Sort Options -o put output in file sort numbers -o numberscat numbers10 rulers5 pens66 erasersUNIX Command - sort

Week 9 - Mar 14, 2005

Sorting based on fields within the line

for sort any sequence of tabs or spaces is the start of a new field

sort +n means skip n fields then sort based on the next characters on the line UNIX Command - sort

Week 9 - Mar 14, 2005

Sort Options Sorting based on fields within the lineex. ls -l produces 9 fields

-r-xr-x- - - 1 zoro staff30 Mar 26 19:49 coloursdrwxr-xr-x 4 aladdin sys 512 Apr 1 17:41 comfindfield1 f2 f3 f4 f5 f6 f7 f8 f9UNIX Command - sort

Week 9 - Mar 14, 2005

Sorting based on fields within the lineTo sort by owner ls -l | sort +2(skip 2 fields, start on field 3)

drwxr-xr-x 4 aladdinsys 512 Apr 1 17:41 comfind-r-xr-x- - - 1 zoro staff30 Mar 26 19:49 coloursfield1 f2 f3 f4 f5 f6 f7 f8 f9UNIX Command - sort

Week 9 - Mar 14, 2005

UNIX Command

Week 9 - Mar 14, 2005

Name:

What it does

uniq [options] [file1][file2]

displays lines of a file that are unique, removing duplicate adjacent lines

uniq - displays lines of a file that are unique allows relative and absolute pathname arguments processes input from file specified on command line or from stdin sends output to stdout unless you specify output file on command line changes only the output not the fileUNIX Command - uniq

Week 9 - Mar 14, 2005

UNIX Command - uniqNote: it only removes duplicate adjacent lines

cat vacationuniq vacationboringboringexpensivenofunvery boringvery boringvery very boringexpensiveboringexpensivenofunvery boringvery very boringexpensive

Week 9 - Mar 14, 2005

The file must be sorted first....uniq onlyremoves any lines that exactly match the preceding line!Warning!

Week 9 - Mar 14, 2005

uniq -c -- counts the number of duplicates for each line.UNIX Command - uniq

cat vacationuniq c vacationboringboringexpensivenofunvery boringvery boringvery very boringexpensive2 boring1 expensive1 nofun2 very boring1 very very boring 1 expensive

Week 9 - Mar 14, 2005

uniq -d -- displays one copy of only the duplicated lines.UNIX Command - uniq

cat vacationuniq d vacationboringboringexpensivenofunvery boringvery boringvery very boringexpensiveboringvery boring

Week 9 - Mar 14, 2005

uniq u -- displays only the lines that are unique.UNIX Command - uniq

cat vacationuniq u vacationboringboringexpensivenofunvery boringvery boringvery very boringexpensiveexpensivenofunvery very boringexpensive

Week 9 - Mar 14, 2005

grep - searches line by line for a pattern in one or more files allows relative and absolute pathname arguments processes input from file(s) specified on command line or from stdinsends output to stdout unless you specify output file on command linechanges only the output not the fileUNIX Command- grep

Week 9 - Mar 14, 2005

grep - searches line by line for a pattern in one or more filesex. to search for lines with the R.E. boring in the file diary...UNIX Command- grep

cat diarygrep boring diary boringexpensivevery boringnofunboringvery very boring boringvery boringboringvery very boring

Week 9 - Mar 14, 2005

grep - searches line by line for a pattern in one or more files grep pattern file(s)grep boring diary

search for what? boring is the pattern to search for....UNIX Command- grep

Week 9 - Mar 14, 2005

grep - searches line by line for a pattern in one or more files grep pattern file(s)grep boring diary

search for where? diary is the file to search in....UNIX Command- grep

Week 9 - Mar 14, 2005

grep options-c counts the number of lines with matches-i ignore case-l lists only the name of each file that contains one or more matches -nnumber - precedes each line with the line number in the file-vreverse - output all lines where pattern is NOT foundUNIX Command- grep

Week 9 - Mar 14, 2005

WarningWhen search for a pattern, some characters have special meaning^ - Beginning of a line$ - End of a lineUNIX Command- grep

Week 9 - Mar 14, 2005

wc word count, displays number of lines, words and bytes in a file or standard input

wc -c only displays the number of bytes

wc -w only displays the word counts

wc -l only displays the line countsUNIX Command- wc

Week 9 - Mar 14, 2005

tar a utility to archive files

Dont need before optionDo you remember the options you used?

UNIX Command- tartar xzvf apache_1.3.31.tar.gz

Week 9 - Mar 14, 2005

Redirection and PipingMost processes in UNIX:send their output to the terminal screenget their input from the keyboardsend any error messages to the terminal

Week 9 - Mar 14, 2005

Redirection and PipingMost processes in UNIX:send their output to the terminal screenget their input from the keyboardsend any error messages to the terminalDo you remember what a process is?

Week 9 - Mar 14, 2005

A process is the execution of a command by the UNIX system.UNIX assigns a unique process identification (PID) number at the creation of each process. As long as a process is in existence, it keeps the same PID number.

Week 9 - Mar 14, 2005

The ps commanddisplays the processes currently running on the controlling terminalwith the -e option displays all processes running on the systemwith the f option displays a full listing of the processes

Week 9 - Mar 14, 2005

The ps commandps with no options displays just the processes running on the controlling terminal.....

Week 9 - Mar 14, 2005

ps -eps e displays all the processes running on the system...

Week 9 - Mar 14, 2005

ps -fps f displays a full listing...

Week 9 - Mar 14, 2005

ps -efps ef displays a full listing... of all processes

Week 9 - Mar 14, 2005

RedirectionMost processes in UNIX:send their output to the terminal screenget their input from the keyboardsend any error messages to the terminal

Week 9 - Mar 14, 2005

RedirectionUNIX allows us to redirect where processes send their output whereprocesses get their input and where processes send any error messages.

Week 9 - Mar 14, 2005

Week 9 - Mar 14, 2005

I/O Channel 0 - standard input (stdin)I/O Channel 1- standard output (stdout)I/O Channel 2 - standard error (stderr)These channels are called file descriptors

Week 9 - Mar 14, 2005

stdinstdoutstderr

Week 9 - Mar 14, 2005

I/O Channel 0 stdin -default is the terminal keyboardI/O Channel 1- stdout - default is the terminal screenI/O Channel 2- stderr - default is the terminal screenThese channels are called file descriptorsLets learn to redirectstdout first

Week 9 - Mar 14, 2005

Standard Input, Standard Output, Standard Error.....not a piece of hardwarenot a keyboard, a screen or a card in your computerActually virtual files...opened by the kernel for every command Remember

Week 9 - Mar 14, 2005

Redirecting the output of a command to a fileUse the greater than symbol >this redirects the standard output or channel 1 of a command this is the same as 1> but you dont need to type the 1 - it is understood.

Week 9 - Mar 14, 2005

Redirecting the output of a command to a fileThe Rulesa command always comes before the > symbola filename always comes after the > symbolcommand > filealways a command before the >always a file name after the >

Week 9 - Mar 14, 2005

Redirecting the output of a command to a fileThe output of this command is displayed on the screen by defaultNothing is displayed on the screen the output of the ps command is redirected to the file psfileIssue the command ps > psfile

Week 9 - Mar 14, 2005

Redirecting the output of a command to a fileNow, look at the contents of the file named psfile

Week 9 - Mar 14, 2005

Now, look at the contents of the file named psfileQuestion:What other UNIX commands would display the contents of the file called psfile on the screen?

Week 9 - Mar 14, 2005

Now, look at the contents of the file named psfile1. more psfile2. head psfile3. tail psfile

Try this at home! Learn the difference between these commands...

Week 9 - Mar 14, 2005

Warning!If the file already exists using the > symbol will overwrite its contents.

Use the >> symbols to append the output of a command to an existing file!

Week 9 - Mar 14, 2005

We can also redirect the output of one command to be input to another commandRedirection and Piping

Week 9 - Mar 14, 2005

Piping - redirecting the output from a command to a commandUse the pipe symbol to connect commandsThe pipe character is a broken vertical line on the keyboardMost fonts display it as a single line that looks like this: | command1 | command2

Week 9 - Mar 14, 2005

Using pipesQ. How many files are in my current directory?

ls | wc -w

Week 9 - Mar 14, 2005

Using pipesls | wc -wThe output of the ls command is a list of file namesThe wc command takes the output of the ls command as its input and displays a count of the words

Week 9 - Mar 14, 2005

Using Pipesls | wc w > file_countThe wc command sends its output to a file called file_countThe spaces before and after a pipe are optional(you will find your commands easier to read if you use them)The spaces before and after a file redirection symbol (>, >>) are also optional

Week 9 - Mar 14, 2005

Using PipesUNIX commands were designed to do one thing very wellWe use a series of pipes between simple UNIX commands to create complex commands linesMuch of the power and flexibility of the UNIX command line is built upon pipelines of commands!

Week 9 - Mar 14, 2005

Always-Remember-and-Never-Forget...RULE #1 Use file redirection symbols such as > and >> before a filenameRULE #2 Use a pipe before a command

Week 9 - Mar 14, 2005

Always-Remember-and-Never-Forget...RULE #1 ...RULE #2 ...

Week 9 - Mar 14, 2005

Always-Remember-and-Never-Forget...RULE #3Dont use pipes when you dont need them!example:cat psfile |moremore psfileWRONG!CORRECT!

Week 9 - Mar 14, 2005

Always-Remember-and-Never-Forget...RULE #3Dont use pipes when you dont need them!example:cat psfile |head -5head -5 psfileWRONG!CORRECT!

Week 9 - Mar 14, 2005

Skill Testing Question #1On the command line what comes AFTER a > or >> symbol?

Week 9 - Mar 14, 2005

Skill Testing Question #2On the command line what comes AFTER a | symbol?

Week 9 - Mar 14, 2005

Skill Testing Question #3When should you NOT use file redirection OR piping?

Week 9 - Mar 14, 2005

I/O Channel 0 stdin -default is the terminal keyboardI/O Channel 1- stdout - default is the terminal screenI/O Channel 2- stderr - default is the terminal screenWeve done this!Now lets do this!Redirection and Piping

Week 9 - Mar 14, 2005

Redirecting the input of a command by default commands get their input from the terminal keyboardyou can redirect stdin so that it comes from a file or from another command

Week 9 - Mar 14, 2005

Redirecting the input of a command Use the less than symbol tempcat < tempcat < temp > temp2

Week 9 - Mar 14, 2005

Redirecting the input of a commandAre these the same?cat < temp cat temp

Are these the same?cat temp2cp temp temp2

Week 9 - Mar 14, 2005

The < symbol redirects the input of a command to come from a file Q. What if you want the input of a command to come from another command?

A. When you place a command after a pipe you are redirecting its input to come from a command instead of from the keyboard

Week 9 - Mar 14, 2005

I/O Channel 0 stdin -default is the terminal keyboardI/O Channel 1- stdout - default is the terminal screenI/O Channel 2- stderr - default is the terminal screenWeve done this!Weve done this!Redirection and PipingLets do this!

Week 9 - Mar 14, 2005

Error RedirectionThe standard error of a command can be redirected to a file using the 2> symbol

example:cat 2> error.log

Week 9 - Mar 14, 2005

Error Redirectionexample:cat 2> error.logThis would overwrite the file error.log

cat 2>> error.log This would append the output of the cat command to the file error.log

Week 9 - Mar 14, 2005

2> redirects the Standard Error from a command to a fileQ. What if you want to redirect stderr to another command?A. You can use the |& symbols to connect the standard output and standard error of the command before the pipe to the command after the pipe.x

(not on the test!)

Week 9 - Mar 14, 2005

A Special Place of UNIX/Linux /dev/null Can be called big bucket Can be called black hole

Week 9 - Mar 14, 2005

Use > or >> to redirect output to a fileUse a | to redirect output to a commandQ. What if you want to do both?(this is on the test!)A. The tee command

Week 9 - Mar 14, 2005

The tee command duplicates the standard output of a command you can then redirect one copy of the output to a file and one copy of the output to a command

example:ls l | tee listing | more

the filename you are sending one copy of the output of the ls l command tothe command you are sending one copy of the output of the ls l command to

Week 9 - Mar 14, 2005

The tee command duplicates the standard output of a command Example:ls l | tee listing | more

if this file exists it will be overwrittenTo append to an existing fileuse tee a filename

ls l | tee a listing | more

Week 9 - Mar 14, 2005

Never use > within the tee command! ls l | tee > listings | more

or

ls l | tee >> listings | moreWRONG!!!!

Week 9 - Mar 14, 2005

Lets review some commands then practice redirection with them...sortuniqgrep

Week 9 - Mar 14, 2005

Some Special symbols when you grep a pattern ^ - beginning of a line $ - end of a line

Week 9 - Mar 14, 2005

How do you learn a new command?Learn its purposeLearn its syntax including common optionsLearn where it gets input and where it sends its output

Week 9 - Mar 14, 2005

sort, grep and uniq belong to a group of UNIX utilities called filtersFilters are utilities that:read from stdinwrite to stdoutdo not change the contents of the original file

Week 9 - Mar 14, 2005

Other UNIX utilities that are filtersheadtailwctee

Week 9 - Mar 14, 2005

Commands that are not filters include:cd chmodcprmdateecho ls mvpwdwho....Warning!You can not put these commands between pipes!

Moral of the Story:

ALWAYS be careful about what you put in a pipe!

Week 9 - Mar 14, 2005

Documents

Week 9 Agenda