Principle of Shell Scripts

8/8/2019 Principle of Shell Scripts

1/36

Principle of Script

Defining the Shell Type

To make a ksh script (which is a ksh program) crate a new file with a starting line like:

#!/usr/bin/kshIt is important that the path to the ksh is propper and that the line doesn not have morethan 32 characters. The shell from which you are starting the script will find this line andand hand the whole script over to to ksh. Without this line the script would be interpretedby the same typ of shell as the one, from which it was started. But since the syntax isdifferent for all shells, it is necessary to define the shell with that line.

Four Types of Lines

A script has four types of lines: The shell defining line at the top, empty lines,commentary lines starting with a # and command lines. See the following top of a script

as an example for these types of lines:

#!/usr/bin/ksh

# Commentary......

file=/path/fileif [[ $file = $1 ]];then

commandfi

Start and End of Script

The script starts at the first line and ends either when it encounters an "exit" or the lastline. All "#" lines are ignored.

Start and End of Command

A command starts with the first word on a line or if it's the second command on a linewith the first word after a";'.A command ends either at the end of the line or whith a ";". So one can put severalcommands onto one line:

print -n "Name: "; read name; print ""

One can continue commands over more than one line with a "\" immediately followed bya newline sign which is made be the return key:

grep filename | sort -u | awk '{print $4}' | \uniq -c >> /longpath/file


2/36

Name and Permissions of Script File

The script mus not have a name which is identical to a unix command: So the script mustNOT be called "test"!After saveing the file give it the execute permissions with: chmod 700 filename.

Variables

Filling in

When filling into a variable then one uses just it's name: state="US" and no blanks. Thereis no difference between strings and numbers:price=50.

Using

When using a variable one needs to put a $ sign in front of it:print $state $price.

Arrays

Set and use an array like:

arrname[1]=4 To fill in

print ${arraname[1]} To print out

${arrname[*]} Get all elements

${#arrname[*]} Get the number of elements

Declaration

There are happily no declarations of variables needed in ksh. One cannot have decimalsonly integers.

Branching

if then fi

if [[ $value -eq 7 ]];thenprint "$value is 7"

fior:

if [[ $value -eq 7 ]]then

print "$value is 7"fior:


3/36

if [[ $value -eq 7 ]];then print "$value is 7";fi

if then else fi

if [[ $name = "John" ]];then

print "Your welcome, ${name}."elseprint "Good bye, ${name}!"

fi

if then elif then else fi

if [[ $name = "John" ]];thenprint "Your welcome, ${name}."

elif [[ $name = "Hanna" ]];thenprint "Hello, ${name}, who are you?"

elseprint "Good bye, ${name}!"

fi

case esac

case $var injohn|fred) print $invitation;;martin) print $declination;;*) print "Wrong name...";;

esac

Loopingwhile do done

while [[ $count -gt 0 ]];doprint "\$count is $count"(( count -= 1 ))

done

until do done

until [[ $answer = "yes" ]];doprint -n "Please enter \"yes\": "read answerprint ""

done

for var in list do done


4/36

for foo in $(ls);doif [[ -d $foo ]];then

print "$foo is a directory"else

print "$foo is not a directory"fi

done

continue...break

One can skip the rest of a loop and directly go to the next iteration with: "continue".

while read linedo

if [[ $line = *.gz ]];thencontinue

elseprint $line

fidone

One can also prematurely leave a loop with: "break".

while read line;doif [[ $line = *!(.c) ]];then

breakelse

print $linefi

done

Command Line Arguments

(Officially they are called "positional parameters")

The number of command line arguments is stored in $# so one can checkfor arguments with:

if [[ $# -eq 0 ]];thenprint "No Arguments"exit

fi

The single Arguments are stored in $1, ....$n and all are in $* as one string. Thearguments cannotdirectly be modified but one can reset the hole commandline for another part of theprogram.If we need a first argument $first for the rest of the program we do:

if [[ $1 != $first ]];then


5/36

set $first $*fi

One can iterate over the command line arguments with the help of the shift command.Shift indirectly removes the first argument.

until [[ $# -qe 0 ]];do# commands ....shift

done

One can also iterate with the for loop, the default with for is $*:

for arg;doprint $arg

done

The program name is stored in $0 but it contains the path also!

Comparisons

To compare strings one uses "=" for equal and "!=" for not equal.To compare numbers one uses "-eq" for equal "-ne" for not equal as well as "-gt" forgreater thanand "-lt" for less than.

if [[ $name = "John" ]];then# commands....

fiif [[ $size -eq 1000 ]];then

# commands....fi

With "&&" for "AND" and "||" for "OR" one can combine statements:

if [[ $price -lt 1000 || $name = "Hanna" ]];then# commands....

fiif [[ $name = "Fred" && $city = "Denver" ]];then

# commands....fi

Variable Manipulations

Removing something from a variable


6/36

Variables that contain a path can very easily be stripped of it: ${name##*/} gives you justthe filename.Or if one wants the path: ${name%/*}. % takes it away from the left and # from the right.%% and ## take the longest possibility while % and # just take the shortest one.

Replacing a variable if it does not yet exits

If we wanted $foo or if not set 4 then: ${foo:-4} but it still remains unset. To change thatwe use:${foo:=4}

Exiting and stating something if variable is not set

This is very important if our program relays on a certain vaiable: ${foo:?"foo not set!"}

Just check for the variable

${foo:+1} gives one if $foo is set, otherwise nothing.

Ksh Regular Expressions

Ksh has it's own regular expressions.Use an * for any string. So to get all the files ending it .c use *.c.A single character is represented with a ?. So all the files starting with any sign followedbye 44.f can be fetched by: ?44.f.

Especially in ksh there are quantifiers for whole patterns:

?(pattern) matches zero or one times the pattern.*(pattern) matches any time the pattern.+(pattern) matches one or more time the pattern.@(pattern) matches one time the pattern.!(pattern) matches string without the pattern.

So one can question a string in a variable like: if [[ $var = fo@(?4*67).c ]];then ...

Functions

Description

A function (= procedure) must be defined before it is called, because ksh is interpreted atrun time.


7/36

It knows all the variables from the calling shell except the commandline arguments. Buthas it'sown command line arguments so that one can call it with different values from differentplaces inthe script. It has an exit status but cannot return a value like a c funcition can.

Making a Function

One can make one in either of the following two ways:

function foo {# commands...

}

foo(){# commands...

}

Calling the Function

To call it just put it's name in the script: foo. To give it arguments do: foo arg1 arg2 ...The arguments are there in the form of$1...$n and $* for all at once like in the maincode.And the main $1 is not influenced bye the $1 of a particular function.

Return

The return statement exits the function imediately with the specified return value as anexit status.

Data Redirection

General

Data redirection is done with the follwoing signs: "> >> <


8/36

Standard Error Redirection

To redirect the error output of a command do: command 2> file

To discard the error alltogether do: command 2>/dev/null

To put the error to the same location as the normal output do: command 2>&1

File into Command

If a program needs a file for input over standard input do: command < file

Combine Input and Output Redirection

command < infile > outfile

command < infile > outfile 2>/dev/null

Commands into Program ( Here Document )

Every unix command can take it's commands from a text like listing with:

command


9/36

Read Input from User and from Files

Read in a Variable

From a user we read with: read var. Then the users can type something in. One shouldfirst print something like:print -n "Enter your favorite haircolor: ";read var; print "". The-n suppresses the newline sign.

Read into a File Line for Line

To get each line of a file into a variable iteratively do:

{ while read myline;do# process $myline

done } < filename

To catch the output of a pipeline each line at a time in a variable use:

last | sort | {while read myline;do

# commandsdone }

Special Variables

$#Number of arguments on commandline.

$? Exit status of last command.$$ Process id of current program.$! Process id of last backgroundjob or background function.$0 Program name including the path if started from another directory.$1..n Commandline arguments, each at a time.$* All commandline arguments in one string.

Action on Success or on Failure of a

CommandIf one wants to do a thing only if a command succeded then: command1 && command2.If the second command has to be performed only if the first one failed, then: command1 ||command2.

Trivial Calculations


10/36

Simpe calculations are done with either a "let" in front of it or within (( ... )). One canincrement a variable within the (( )) without a "$": (( a+=1 )) orlet a+=1.

Numerical Calculations using "bc"

For bigger caluculations one uses "bc" like: $result=$(print "n=1;for(i=1;i


11/36

"perl"

Perl is a much richer programming language then ksh, but still one can do perl commandsfrom within a ksh script. This might touch Randal, but it's true. Let's say you want to

remove all ^M from a file, then take perl for one line in your ksh script:

perl -i -ep 's/\015//g' filename.

Perl can do an infinite amount of things in many different ways. For anything bigger useperl instead of a shell script.


12/36

1. Handling Command Line Arguments

Why is it necessary to write something about command line arguments? The concept isvery easy and clear: if you enter the following command

$ ls -l *.txt

the command "ls" is executed with the command line flag "-l" and all files in the

current directory ending with ".txt" as arguments.

Still many shell scripts do not accept command line arguments the way we are used to(and came to like) from other standard commands. Some shell programmers do not evenbother implementing command line argument parsing, often aggravating the script's users

with other strange calling conventions.

For examples on how to name command line flags to be consistent with existing UNIXcommands see the tableFrequent option names.

Here are some examples ofbadcoding practices.

Setting environment variables for script input that could be specified on thecommand line.

One example:

:# AUTORUN must be specified by the userif [ "$AUTORUN" != yes ]then

echo "Do you really want to run this script?"echo "Enter ^D to quit:"if read answerthen

echo "o.k, starting up memhog daemon"else

echo "terminating"exit 0

fifi# start of script...

Consider the script's user who might ponder "What was the name of this variable?FORCERUN? AUTOSTART? AUTO_RUN? or AUTORUN?"
http://www.shelldorado.com/goodcoding/cmdargs.html#flagnameshttp://www.shelldorado.com/goodcoding/cmdargs.html#flagnames


13/36

Don't get me wrong, environment variables do have their place and can make lifeeasier for the user. A much better way to solve the autorun option would be toimplement a command line flag, i.e. "-f" for "force non-interactive execution".

Positional parameters.

Example:

:# process - process input file

ConfigFile="$1"InputFile="$2"OutputFile="$3"

# Read config fileget_defaults "$ConfigFile"

# Do the processingprocess_input < "$InputFile" > "$OutputFile"

This script expects exactly three parameters in exactly this order: the name of aconfiguration file with default settings, the name of an input file, and the name ofan output file. The script could be called with the following parameters:

$ process defaults.cf important.dat output.dat

It then reads the configuration file "defaults.cf", processes the input file

"important.dat" and then writes (possibly overwriting) the output file

"output.dat". Now see what happens if you call it like this:

$ process output.dat defaults.cf important.dat

Now the script tries to read the output file "output.dat" as configuration file. If

the user is lucky the script will terminate at this point, before it tries to overwritehis data file "important.dat" it will be using as the output file!

This script would have been better with the following usage:

$ process -c default.cf -o output.dat file.dat

The command line option "-c" precedes the default file, the output file is

specified with the "-o" option, and every other argument is taken to be the input

file name.

Our goal are shellscripts, that use "standard" command line flags and options. We willdevelop a shell script code fragment that handles command line options well. You maythen use this template in your shell scripts and modify it to fit your needs.


14/36

Consider the following command line:

$ fgrep -v -i -f excludes.list *.c *.h

This command line consists of a command("fgrep") with threeflags "-v", "-i" and "-f".One flag takes an argument ("excludes.list"). After the command line flags multiple filenames ("*.c", "*.h") may follow. At this point we do not know how many file names thatmay be; the shell will expand the file name patterns (or "wildcards") to a list of actual filenames before calling the command "fgrep". The command itself does not have to dealwith wildcards.

What happens if there is no file matching the pattern "*.c" in the current directory? In thiscase the shell will pass the parameter unchanged to the program.

If we wanted to handle command lines like the above, we must be prepared to handle

command line flags (i.e. "-v", "-i") command line flags with arguments (i.e. "-f file") multiple file names following the flags

The shell sets some environment variables according to the command line argumentsspecified:

$0

The name the script was invoked with. This may be a basename without directorycomponent, or a path name. This variable is not changed with subsequent shift

commands.

$1, $2,

$3, ...The first, second, third, ... command line argument, respectively. The argumentmay contain whitespace if the argument was quoted, i.e. "two words".

$# Number of command line arguments, not counting the invocation name $0

$@"$@" is replaced with all command line arguments, enclosed in quotes, i.e. "one",

"two three", "four". Whitespace within an argument is preserved.

$*

$* is replaced with all command line arguments. Whitespace is notpreserved, i.e.

"one", "two three", "four" would be changed to "one", "two", "three", "four".This variable is not used very often, "$@" is the normal case, because it leaves the

arguments unchanged.

The following code segment loops through all command line arguments, and prints them:

:# cmdtest - print command line arguments

while [ $# -gt 0 ]do


15/36

echo "$1"shift

done

The environment variable $# is automatically set to the number of command linearguments. If the script was called with the following command line:

$ cmdtest one "two three" four

$# would have the value "3" for the arguments: "one", "two three", and "four". "two

three" count as one argument, because they are enclosed within quotes.

The shift command "shifts" all command line arguments one position to the left. The

leftmost argument is lost. The following table lists the values of$# and the command line

arguments during the iterations of the while loop:

$# remaining arguments comments

3$1 = "one"$2 = "two three"$3 = "four"

start of the command

2$1 = "two three"$2 = "four"

after the first shift

1 $1 = "four" after the second shift

0 end of the while loop

Now that we can loop through the argument list, we can set script variables depending on

command line flags:

vflag=offwhile [ $# -gt 0 ]do case "$1" in

-v) vflag=on;;esacshift

done

The command line option -v will now result in the variable vflag to be set to "on". Wecan then use this variable throughout the script.

Now let's improve this code fragment to handle file names. It would be nice if the scriptwould handle all command line flags, but leave the file names alone. This way we coulduse the shell variable $@ with the remaining command line arguments later on, i.e.

# ...


16/36

grep $searchstring "$@"

and be sure that it only contains file names. But how do we recognize file names fromcommand line switches? That's easy: files do not start with a dash "-" (at least not yet...):

vflag=offwhile [ $# -gt 0 ]do

case "$1" in-v) vflag=on;;-*)

echo >&2 "usage: $0 [-v] [file ...]"exit 1;;

*) break;; # terminate while loopesacshift

done

This example prints a short usage message and terminates if an unknown command lineflag starting with a dash was specified. If the current argument does not start with a dash(and therefore probably is a file name), the while loop is terminated with the break

statement, leaving the file name in the variable "$1".

Now we just need a switch for command line flags with arguments, i.e. "-f filename".This is also pretty straight forward:

vflag=offfilename=

while [ $# -gt 0 ]do

case "$1" in-v) vflag=on;;-f) filename="$2"; shift;;-*) echo >&2 \

"usage: $0 [-v] [-f file] [file ...]"exit 1;;


done

If the argument $1 is "-f", the next argument ($2) should be the file name. We now

handled two arguments ("-f" and the filename), but the shift after the case construct will

only "consume" one argument. This is the reason why we execute an initial shift after

saving the filename in the variable filename. This shift removes the "-f" flag, while the

second (after the case construct) removes the filename argument.


17/36

We still have a problem handling file names starting with a dash ("-"), but that's aproblem every standard unix command interpreting command line switches has. It iscommonly solved by inventing a special command line option named "--" meaning "endof the option list".

If you for example had a file named "-f", it could not be removed using the command "rm-f", because "-f" is a valid command line option. Instead you can use "rm -- -f". Thedouble dash "--" means "end of command line flags", and the following "-f" is theninterpreted as a file name.

Note:You can also remove a file named "-f" using the command "rm ./-f"

The following (recommended) command line handling code is a good way to solve thisproblem:

vflag=offfilename=while [ $# -gt 0 ]do

case "$1" in-v) vflag=on;;-f) filename="$2"; shift;;--) shift; break;;-*)

echo >&2 \"usage: $0 [-v] [-f file] [file ...]"exit 1;;


done# all command line switches are processed,# "$@" contains all file names

The drawback of this command line handling is that it needs whitespace between theoption character and an argument, ("-f file" works, but "-ffile" fails), and that multipleoption characters cannot be written behind one switch character, ("-v -l" works, but "-vl"does not).

Portability:This method works with all shells derived from the Bourne Shell, i.e. sh, ksh,ksh93, bash, pdksh, zsh.

Using "getopt"


18/36

Now this script processes its command line arguments like any standard UNIXcommand, with one exception. Multiple command line flags may be combined withstandard commands, i.e. "ls -l -a -i" may be written as "ls -lai". This is not that easy tohandle from inside of our shell script, but fortunately there is a command that does thework for us: getopt(1).

The following test shows us, how getopt rewrites the command line arguments "-vl -f

file one two three":

$ getopt f:vl -vl -ffile one two three

produces the output

-v -l -f file -- one two three

These are the command line flags we would have liked to get! The flags "-vl" areseparated into two flags "-v" and "-l". The command line options are separated from thefile named by a "--" argument.

How did getopt know, that "-f" needed a second argument, but "-v" and "-l" did not? Thefirst argument to getopt describes, what options are acceptable, and if they havearguments. An option character followed by a colon (":") means that the option expectsan argument.

Now we are ready to let getopt rewrite the command line arguments for us. Since getoptwrites the rewritten arguments to standard output, we use

set -- `getopt f:vl "$@"`

to set the arguments. `getopt ...` means "the output of the command getopt", and "set -- "sets the command line arguments to the result of this output. In our example

set -- `getopt f:vl -vl -ffile one two three`

is replaced withset -- -v -l -f file -- one two three

which results in the command line arguments-v -l -f file -- one two three

These arguments can easily be processed by the script we developed above.

Now we include getopt within our script:

vflag=offfilename=set -- `getopt vf: "$@"`[ $# -lt 1 ] && exit 1 # getopt failedwhile [ $# -gt 0 ]do

case "$1" in-v) vflag=on;;


19/36

-f) filename="$2"; shift;;--) shift; break;;-*)

echo >&2 \"usage: $0 [-v] [-f file] file ..."exit 1;;


done# all command line switches are processed,# "$@" contains all file names

The first version of this document contained the line

set -- `getopt vf: "$@"` || exit 1

This commands do not work with all shells, because the set command doesn't always

return an error code if getopt fails. The line assumes, that getopt sets its return value ifthe command line arguments are wrong (which is almost certainly the case) and that setreturns an error code if the command substitution (that executes getopt) fails. This is notalways true.

Why didn't we use getopt in the first place? There is one drawback with the use of getopt:it removes whitespace within arguments. The command line

one "two three" four

(three command line arguments) is rewritten asone two three four

(four arguments). Don't use the getopt command if the arguments may contain

whitespace characters.

Newer shells (Korn Shell, BASH) have the build-in getopts command, which does not

have this problem. This command is described in the following section.

Portability:The getopt command is part of almost any UNIX system.

Using "getopts"

On newer shells, the getopts command is built-in. Do not confuse it with the oldergetopt(without the trailing "s") command. getopts strongly resembles the C library functiongetopt(3).

Below is a typical example of how getopts is used:


20/36

vflag=offfilename=while getopts vf: optdo

case "$opt" inv) vflag=on;;

f) filename="$OPTARG";;\?) # unknown flag

echo >&2 \"usage: $0 [-v] [-f filename] [file ...]"exit 1;;

esacdoneshift `expr $OPTIND - 1`

Portability:The getopts command is an internal command of newer shells. As a rule of thumball systems that have theKSHhave shells (including theBourne Shellsh) that

include a built-in getopts command.

Frequent option names

The following table should help you find good names for your command line flags. Lookat the second column (Meaning), and see if you find a rough description of yourcommand line option there. If you i.e. are searching for the name of on option to appendto a file, you could use the "-a" flag.

Flag Meaning UNIX examples

-a

append, i.e. output to a file

show/process all files, ...

tee -als -a

-c

count something

command string

grep -c

sh -ccommand

-d

directory

specify a delimiter

cpio -d

cut -ddelimiter

-e

expand something, i.e. tabs to spaces

execute command

pr -exterm -e/bin/ksh

-f read input from a file force some condition (i.e. no prompts, non-interactive

fgrep -ffilerm -f


21/36

execution)

specify field number

cut -ffieldnumber

-h

print a help message

print a headerNote:-t fortitle may be more appropriate.

pr -hheader

-i

ignore the case of characters Turn on interactive mode

Specify input option

grep -irm -i

-l

long output format list file names line count

login name

ls -l, ps -l, who-lgrep -l

wc -lrlogin -lname

-L follow symbolical links cpio -L, ls -L

-n

non-interactive mode

numeric processing

rsh -nsort -n

-o output option, i.e. output file name cc -o, sort -o

-p

process id

process path

ps -p pidmkdir -p

-q

quickmode

quiet mode

finger -q, who-q

-r

process directories recursivelyNote: the flag-R would be better for this purpose.

process something in the reverse order

specify root directory

rm -rsort -r, ls -r

-R process directories recursivelychmod -Rls -R

-s

be silent about errorsNote: such an option is unnecessary, because the user can makethe program silent by redirecting standard output and standard

error to /dev/null.

cat -slp -s


22/36

-t specify tab character sort -ttabchar

-u

Produce unique output

process data unbuffered

sort -ucat -u

-v

print verbose output, the opposite of-q

reverse the functionality

cpio -v, tar -vgrep -v

-w

specify width wide output format

work with words

pr -w, sdiff -wps -wwc -w

-x exclude something

-y

answeryes to all questions (effectively making thecommand non-interactive)

Note: The flag-fmay be better for this purpose.

fsck -y,shutdown -y

Now you know the standard option names, on to "standard" UNIX commands that do notuse them.

dd - disk dump

dd if=infile of=outfile bs=10k

The syntax of this command probably is older than UNIX itself. One majordisadvantage is that argument names and file names are written together withoutwhitespace, i.e. if=mydoc*.txt. The shell will take "if=" as part of the file name,

and cannot expand the wildcards "mydoc*.txt".

find - find files

find / -name '*.txt' -print

With this command option names have more than one character. This makes themmore memorable and more readable. If only all commands would be like this!

And if only -print was a default option!

By the way, did you know that the command line

$ ls -bart -simpson -is -cool

is a valid usage for the SOLARISls command?


23/36

2. Temporary files and signal handling

Temporary files are frequently used in shell scripts. In a typical shell script often somedata is processed, and the results are written to a scratch file, the new data is processed in

another way, and eventually the scratch file is removed.

So why write an article about this topic?

Often shell script programmers use temporary files in their scripts, and remove them atthe end of the program. This simple and straight forward approach works well as long asa user does not interrupt the script using a signal (i.e. by pressing ^C or DEL). In this casethe script doesn't have a chance to remove its temporary files before closing.

This article shows how to intercept interrupts from shell scripts.

One example:

:# viman - start "vi" on a manual page

Tmp=/tmp/viman

man "$@" | col -b | uniq > $Tmpvi $Tmprm -f $Tmp

This script passes its command line arguments on to the man command, and writes theresult to a temporary file /tmp/viman. Before starting vi on the file, all control

characters are removed ("col -b"), and duplicate or empty lines are removed ("uniq").

Aftervi terminates, the file is removed.

This simple script has two drawbacks.

Consider what happens if two people call this script, one after the other. The first one hashis manual page written to /tmp/viman. Shortly after that the second one has his manual

page written to the same file, overwriting the contents of the first manual page. Now thefirst user gets the wrong manual page in the vi editor, and terminates. His instance of the

script removes the file /tmp/viman, and with a little bad luck the first user at the sametime now has an empty file within the vi.

The solution to this problem is clear: each user needs to have a unique temporary file, buthow to do it? We could try to create the temporary file in the directory $HOME. Each user

is (normally) guaranteed to have a unique HOME directory. But even then the user mayoverwrite the file if he has a windowing system (like OpenWindows or the Common


24/36

Desktop Environment(CDE)) and is logged in more than once with the same HOME

directory.

Steve Bourne (the creator of the Bourne Shell) suggests in The UNIX system to use theunique process identifier (PID) of the shell script as part of the file name. Since the

process id of the script is always available via the environment variable $$, we couldrewrite the script as follows:

:# viman - start "vi" with a manual page

Tmp=/tmp/vm$$

man "$@" | col -b | uniq > $Tmpvi $Tmprm -f $Tmp

This small change solves the problem.

But one problem remains: what happens to the temporary file, if the script is terminatedwith a signal? In this case, the temporary file may is not removed, because the last line ofthe script is never reached!

You may think: "Who cares about files clogging up the /tmp directory? The directorygets cleaned up automatically anyway!" On the other hand you are reading this text tobecome a better shell programmer, and could be excited to come to know there is an easyway to "trap" signals from a shell script.

The general syntax for the trap command is:

trap [ command]signal[signal... ]

Signals may be specified using numbers (0 to 31), "0" being a pseudo-signal meaning"program termination". The Korn shell also understands names for the signal, i.e. HUP for

HANGUP signal, TERM for the SIGTERM signal etc. Newerkill commands display a

list of signal names if called with the flag -l. The following table lists the most commonsignals along with their KSH names:

Number KSHname

Comments

0 EXIT This number does not correspond to a real signal,but the corresponding trap is executed before scripttermination.

1 HUP hangup

2 INT The interrupt signal typically is generated using the


25/36

DEL or the ^C key

3 QUIT The quit signal is typically generated using the^[ key. It is used like the INT signal but explicitlyrequests a core dump.

9 KILL cannot be caught or ignored

10 BUS bus error

11 SEGV segmentation violation

13 PIPE generated if there is a pipeline without reader toterminate the writing process(es)

15 TERM generated to terminate the process gracefully

16 USR1 user defined signal 1

17 USR2 user defined signal 2

- DEBUG KSH only: This is no signal, but the correspondingtrap code is executed before each statement of thescript.

A simple example would be:

trap "rm -f $Tmp" 0 1 2 3 15

This means: execute the command "rm -f $Tmp" if the script terminates ("signal" 0), or

after receiving any of the signals 1 (HANGUP), 2 (QUIT), 3 (INTR), or 15 (TERM).Actually, a good shell script should handle all these signals.

Only one refinement has to be made before we can present The Canonical Way ToHandle Temporary Files. Suppose we use the following line in our script:

trap "rm -f $Tmp" 0 1 2 3 15

If somebody sends the SIGTERM signal to our script (i.e. by entering "kill -15scriptpid"), the following would happen:

1. The script would trap the signal 15, and execute the command "rm -f $Tmp",

thus removing the temporary file.2. Then it would continue with the next script command. This could cause strange

results, because the (probably needed) temporary file $Tmp is gone. Another point

is that somebody explicitly tried to terminate the script, a fact it deliberatelyignores.

3. Just before the script exits the trap for signal "0" is always performed, resulting ina second attempt to remove $Tmp. This will result in unwanted error messages

(although in this case it will do no harm).


26/36

A better (and the recommended) way to handle the signals is as follows:

trap 'rm -f "$Tmp" >/dev/null 2>&1' 0trap "exit 2" 1 2 3 15

The first trap ensures that the temporary file $Tmp is removed at the end of the script

execution. Possible error messages are simply discarded.

The second trap causes our script to terminate after receiving one of the specified signals.Before the script terminates, the trap for "signal" 0 is executed, effectively removing thetemporary file.

Our original script, now rewritten to handle signals and use unique temporary files looksas follows:

:# viman - start "vi" with a manual page

Tmp="${TMPDIR:=/tmp}/vm$$"

# Assure the file is removed at program termination# or after we received a signal:trap 'rm -f $Tmp >/dev/null 2>&1' 0trap "exit 2" 1 2 3 13 15

EXINIT="set ignorecase nowrapscan readonly"export EXINIT

man "$@" | col -b | uniq > "$Tmp" || exit

[ -s "$Tmp" ] || exit 0 # file is emptyhead -1 < "$Tmp" |

grep 'No.*entry' && exit 0 # no manual page

${EDITOR:-vi} "$Tmp"

Handling signals requires a bit more overhead; perhaps overkill for simple scripts likethis one but definitely worthwhile for complex scripts.

Stop simultaneous execution of the same scriptLevel: Advanced Submitted by:[email protected] URL: none

My_pid=$$print $My_pid >> temp.fileread input_pid < temp.fileif [[ "$input_pid" == "$My_pid" ]] then

(allow the script to continue)else
mailto:[email protected]:[email protected]:[email protected]


27/36

(exit the script, My_pid was not 1st)fiThis guarantees that the first instance of the script submitted will runand any other occurances of the same script will exit. Remove temp.filebefore the script ends and I suggest using a Trap. Better than trying todeal with ps/grep and it make a nice little function.

Backup Log files without renaming and interrupting service

Level: Advanced Submitted by: [email protected] URL: none

Problem: Periodically your log files become too large and need to bebackedup and compressed. Renaming the file with mv will mess up any links tothe file and could disrupt your servers using the logs. If your serversare expected to be running 24X7 renaming is not an option.

Solution:

suffix=`date +%Y%m%d%H%M%S`.bak

newFileName=${YourLogName.log}.${suffix}cp -p YourLogName.log $newFileNamecp /dev/null YourLogName.logcompress $newFileName

A copy of your log is made and compressed and the log file is initialized(emptied) and ready for more messages. Processing can continue withoutinterruption. I suppose there is a possibility of a few messages beingdropped (what we don't see won't be missed) , so do this during times ofslow usage with a crontab entry.

saving stdout, stderr and both into 3 separate files

Level: Advanced Submitted by:[email protected] URL: none

# Sometimes it is useful to not only know what has gone to stdout andstderrbut also where they occurred with respect to each other:# Allow stderr to go to err.txt, stdout to out.txt and both to mix.txt#((./program 2>&1 1>&3 | tee ~/err.txt) 3>&1 1>&2| tee ~/out.txt) > ~/mix.txt 2>&1

"comment out" code blocksLevel: Script Programmer Submitted by: ??? URL: none

One line of shell code can be "commented out" using the"#" character. Sometimes however it would be nice to "commentout"more than one line of code, like the C "/* */" comments.

One way to comment out multiple lines is this:: '
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


28/36

,,,,,,'

After the ":" command (that returns "true") the restof the codeis a large string constant enclosed within 'single quotes'.

Of course this works only if the code "commented-out" doesnot contain single quotes.

cleaning up tmp files

Level: Script Programmer Submitted by: [email protected] URL: none

I've seen too many scripts using massive number of tmp files, which iswrong in its self.But not only that, people tend to clean them up one at a time in afashionsuch as

if [ -a ${tmpFile} ]; thenrm ${tmpFile};fi

This, when you use up to 10 or even 5 tmp files gets nasty. A quickerwayof cleaningup such tmp files is to use a simple loop, I even perfer to use array'swhich areavailible in Korn shell. Here is an example.

clean(){tmpfiles[0]=${temp1}

tmpfiles[1]=${temp2}

for file in ${tmpfiles[*]}do

if [ -a ${file} ]; thenrm ${file}

fidone

This way, as you accumulate more and more tmp files, you need only to addone line to get it cleaned up.

cleaning up tmp files (2)

Level: Script ProgrammerSubmitted by:???

URL: none

Another way to clean up multiple temporaryfiles is to create them within a subdirectory, i.e.

TmpBase=${TMPDIR:=/tmp}/myscript.$$mkdir "$TmpBase" || exit 1 # create directory
mailto:[email protected]:[email protected]:[email protected]


29/36

chmod 700 "$TmpBase" || exit 1 # restrict access

# Remove all temporary files after program termination# or at receiption of a signal:trap 'rm -rf "$TmpBase" >/dev/null 2>&1' 0trap "exit 2" 1 2 3 15

# The following files will be remove automatically:input=$TmpBase/inputoutput=$TmpBase/output#...

Convert "relative" in "absolute" path name

Level: Script Programmer Submitted by: ??? URL: none

In shell scripts it is often necessary to convert arelative path name, i.e. "../usr/../lib/somefile" to

an absolute path name starting with a slash "/", i.e."/lib/somefile". The following code fragment does exactly this:

D=`dirname "$relpath"`B=`basename "$relpath"`abspath="`cd \"$D\" 2>/dev/null && pwd ||echo \"$D\"`/$B"

Positioning the cursor from within shell scripts


[This tip was first published within the SHELLdorado Newsletter 1/99]

For some shell scripts it would be desirable, if the scriptcould position the cursor to arbitrary (row, column) pairs(i.e. to display a status line, ...)

The following shell function uses the "tput" command tomove the cursor to the specified (row, column) position:

# move cursor to row $1, col $2 (both starting with zero)# usage: writeyx message rowno colnowriteyx () {

tput cup $2 $3echo "$1"

}

Example usage:

clear # clear the screenwriteyx "This is a centered message" 11 26writeyx "press any key to continue..." 22 0read dummy

The "tput" comm!and looks up the escape command sequence for


30/36

a feature needed for the current terminal. You can use itfor other terminal related things, too:

tput smso # "start mode shift out": usually# reverse

echo "This is printed reverse"

tput rmso # "reset mode shift out"

All available capability names are listed on the terminfo(5)manual page.

Portability:The "tput" command is available with the "terminfo"terminal information database

Setting default values for variables


In shell scripts it's often useful toprovide default values for script variables, i.e.

if [ -z "$Host" ]then

Host=`uname -n`fi

For this kind of assignment the shellhas a shorthand:

: ${Host:=`uname -n`}

This means: if the variable "Host" isnot already set, execute the command"uname -n" and set the variable tothe returned value.

Getting a file into "memory"


Sometimes its convienient to have a fileread into memory to work on it. Theform that you take to accomplish this isan array data structure. In ksh88 themaximum is 1024 elements, however, onsome of the more modern versions you cango much higher.

To do this the following can be done:

#/usr/bin/ksh
mailto:[email protected]:[email protected]


31/36

typeset -i cnt=0

while read linedomyarray[$cnt]=$line((cnt = cnt + 1))

done < myfile# end of file---------

Now, if I want to access any line of thatfile, I simply use:

${[]}

echo ${myarray[4]}

This is useful for parsing, or for interactiveuse of of the file's contents. I haveall !of the lines of the file available in

the array, and I can move around in them,select the ones I want.

Look at the following example:

#!/usr/bin/ksh

typeset -i cnt=0

while read linedomyarray[$cnt]=$line((cnt = cnt + 1))

done < myfile

PS3="Select a number: "select linefromfile in ${myarray[@]}doecho $linefromfile

done# end of file------------

There are many other uses for this techique.Dynamic menusnumeric error message referencegetting mulitple specific lines of a filein a single pass

Have fun.

Find user's name


The full name of each user is available in the /etc/passwd file. If youwould like to use the full name in your script instead of $LOGNAME,


32/36

which simply returns the user's login name, you can declare the followingvariable in your script:

fullname=`grep $LOGNAME /etc/passwd | cut -f 5 -d :`

If you only want the first name, you would declare this variable:

firstname=`grep $LOGNAME /etc/passwd | cut -f 5 -d : | cut -f 1 -d ""`

Find user's name (2)


Since the full name is the 5th columnof the file /etc/passwd, it's easyto look up the full name for a

login name like "joe":

awk -F: '$1 == name {print $5}' name=joe /etc/passwd

The option "-F" tells awk to use ":" als fieldseparator (instead of whitespace).

Using "here-documents" instead of multiple "echo"


Multiple "echo" commands may be replaced by a"here-document".

This makes the script faster and easier to read.

Example:

echo "Please enter your choice:"echo "1 - list current directory"echo "2 - list current users"echo "3 - log off"

may be replaced with

cat Using "here-documents" instead of multiple "echo" (2)


33/36

Level: ScriptProgrammer

Submitted by:mailto:[email protected]

URL:http://www.geocities.com/bijoytg_

# you can also turn multi-echos into a single echo

echo "Welcome to Foo.Bar v0.8=======================Press enter to continue...

";

To find idle users


w | gawk 'BEGIN { FIELDWIDTHS = "9 11 13 10 8 7 7 14" }NR > 2 {

idle = $5sub(/^ */, "", idle)

if ( idle == "" )idle = 0if (idle ~ /:/) {

split(idle, t, ":")idle = t[1] * 60 + t[2] #Converts idle time into seconds

}if (idle ~ /days/)

idle *= 24*60*60print $1, $2, idle

}'

Using ksh builtins instead of external commands


Submitted by: [email protected]

URL:http://www.community.net/~atomik

Many times, scripters will use external commands like basename, dirnameandtr because they don't realize they can instead use ksh builtins.

An added bonus is the builtins are faster and require less systemresourcesbecause no sub-process is spawned.

basename replacement:---------------------

$ fullfile="/some/dir/file.txt"# replaced: file=$(basename $fullfile)$ file=${fullfile##/*}$ echo $filefile.txt

dirname replacement:--------------------
mailto:[email protected]://www.geocities.com/bijoytg_http://www.geocities.com/bijoytg_mailto:[email protected]:[email protected]://www.community.net/~atomikmailto:[email protected]://www.geocities.com/bijoytg_mailto:[email protected]:[email protected]://www.community.net/~atomik


34/36

$ fullfile="/some/dir/file.txt"# replaced: dir=$(dirname $fullfile)$ dir=${fullfile%/*}$ echo $dir/some/dir

tr replacements:----------------

$ word="MiXeD"# replaced: word=$(echo $word | tr [A-Z] [a-z])$ typeset -l word$ echo $wordmixed

# replaced: word=$(echo $word | tr [a-z] [A-Z])$ typeset -u word$ echo $wordMIXED

KSH build-in networking functions

Level: Script Programmer Submitted by: ? URL: none

[Note: the following examples will work only with standardksh implementations. They will not work with the Linux KornShell pdksh.]

Most Korn Shells (/bin/ksh) have sparsely documented, build-innetworking functions.

Example:

$ date=

$ read date < /dev/tcp/127.0.0.1/13$ echo $dateWed Feb 10 00:45:39 MET 1999

This command opens a TCP connection to the IP address 127.0.0.1(the local loopback IP address), and connects to the port "13"(daytime, see /etc/services). The current date and time isreturned, and assigned to the variable "date".

Note that the "/dev/tcp/*" directories do not have to exist;the file names are special to the Korn Shell and are interpre!tedby the shell internally. Only numerical ip addresses and port

numbers are supported; "read date < /dev/tcp/localhost/daytime"does not work.

"Normalize" input field separators


Script programmers sometimes have to process input that consistsof fields separated by whitespace, i.e.


35/36

field1 field 2 field3

This input has the disadvantage that it uses different combinationsof blank and TAB characters as input, and is hard to process using"cut" and "sort", because these commands expect

exactly onefield separator character.

The following "sed" line "normalizes" this inputreplacing eachsequence of two or more whitespace characters with exactly one character:

sed 's/ [ ][ ]*//g'

Substitute the five characters "" with a "real"TAB character(ASCII 9).

Further processing can be done using this characteras field separator.

To Reverse a File


Submitted by:[email protected]

URL: none

######## TO PRINT FILE IN REVERSE ORDER BY LINE #############3

if [ $# -ne 1 ]thenecho "Usage reverse_file "exit 1;

fi

######## By using for loop #############awk '{ line[NR] = $0 } END { for (i=NR; i>0; i=i-1)

print line[i] }' $1

Script debugging settings

Level: Script ProgrammerSubmitted by: ??? URL: none

Most shell script programmers know the command

set -vx

to print each script command before execution. Sometimesthe following flags are useful, too:

set -e # terminate the script at first errorset -u # unset variables are fatal errors


36/36

Documents

Principle of Shell Scripts