Unix - Advanced Shell Scripting_1 Day

1

IBM Global Business Services

© Copyright IBM Corporation 2010

Advanced Shell ScriptingAdvanced Shell Scripting

2



Session objectives

By the end of this session, you should be able to:

Execute shell script

Use kill, basename, exec, eval, expr, and let commands

Create and use Arrays

Use functions and nested functions

Create library files

3



Course content (Day 3)

Module 1: Customizing shell environment

Module 2: Advanced shell scripting

Module 3: Arrays

Module 4: Functions



Unix

Module 1: Customizing Shell Environment

5



Unix Shell: Shell variables

There are two types of shell variables, namely, predefined shell variables and user-defined shell variables.

The built-in variables reside in the environment right from the time the user logs in.

The user-defined variables created during a log-in session, exists in the environment till the user logs out.

When the user creates a sub-shell, the variables are not inherited to the sub-shell.

If the variables have to be inherited to the sub-shells, then the user needs to issue the command:$ export <var1> <var2> <var3> …

6



Unix Shell: Shell variables (continued)

A shell variable can be removed from the environment with the following command:unset <var1> <var2> <var3> …

A shell variable can be made read only by issuing the following command:readonly <var1> <var2> <var3> …

A read-only shell variable can neither be removed from the environment nor its value can be changed.

7



Unix Shell: Shell variables: Built-in variables

PATH – This stores a list of absolute pathname of directories delimited by :, which are searched from left to right by the shell, when user issues an external command.

MAIL – This stores the absolute pathname of the file which stores the user’s mails.

MAILCHECK – This stores the number of seconds in the interval of which the mail daemon process checks for the arrival of new mails in the user’s mailbox.

8



Unix Shell: Shell variables: Built-in variables (continued)

HOME – This stores the absolute pathname of user’s home directory.

PS1 – This stores the user’s primary prompt.

PS2 – This stores the user’s secondary prompt.

LOGNAME – This stores the user’s login name.

PWD – This stores the user’s working directory.

To display the content of a variable, we have to issue the following command: echo ${<var_name>}

9



Unix Shell: Shell variables: Commands displaying variables

env – This displays the list of built-in shell variables and their values.

set – This displays the list of built-in shell variables and user-defined variables with their values and user-defined function definitions, in bash environment.

set – This displays the list of built-in shell variables and user defined variables with their values in korn shell environment.

10



Unix Shell: Customizing user’s shell environment

User’s shell environment can be customized by changing the default values of the built-in variables.

Examples:

$ echo $PS1

$ PS1=“Hello “

$ echo $PS1

Hello

$ PATH=$PATH:$HOME

$ echo $PATH

/usr/kerberos/bin:/usr/bin:/bin:/home/manoj/bin: /home/manoj

Explanation:

The shell displays value stored in variable PS1 as primary prompt for the shell.

We can append any desired directory to PATH variable, as indicated in the code example.

11



Unix Shell: Customizing user’s shell environment (continued)

The shell’s start-up scripts in the Unix environment are:

/etc/bashrc or /etc/environment – universal start-up script. The commands in this script would affect all the user’s shell environment. This script can be edited only by the super user.

$HOME/.bashrc or $HOME/.kshrc– bash / korn specific script, which can be customized by any user

$HOME/.bash_profile or $HOME/.profile – bash or korn specific script, where the environment variables are set

These scripts are run by the shell in the order given, when the user logs in.

12




Aliases – Aliases are short-names for the commands. A long command may be required to be executed very often. The solution for this is to create an alias.

Syntax:$ alias myfind='find $HOME -name "*.sh" -print 2> /dev/null'

The above alias would reside in the memory as long as the user is logged in. During the next log in, this alias myfind won’t be available. We can make this alias available in all login sessions by adding this alias command in the start-up script, $HOME/.bashrc (in bash) or $HOME/.kshrc (in korn shell) .

13




Example:

$ set -o noclobber

$ cat > animals

-bash: animals: cannot overwrite existing file

$ set +o noclobber

$ cat > animals

lion

tiger

giraffee

leopard

14




Example:

$ set -o noclobber

$ cat > animals

-bash: animals: cannot overwrite existing file

$ cat >| animals

jackal

fox

donkey

The above example shows that we can override the noclobber setting by succeeding > symbol with a | symbol.

15



Unix Shell: Set commands

The Set commands to customize shell environment are the following:

The setting ‘set –o noclobber’ can be unset by issuing the following command:set +o noclobber

$ set -o ignoreeofWhen the user presses <ctrl-d> key on his login prompt, the user is logged off. Issuing the above command ensures that the user is NOT logged off if he presses <ctrl-d> on his login prompt.

$ set +o ignoreeofThe above command when issued, would unset the ‘ignoreeof’ setting.

16



Unix Shell: Set commands (continued)

The Set commands to customize shell environment are the following:

$ set -o viWhen the above command is issued, command line is enabled with vi editing capabilities.

$ set +o viWhen the above command is issued, command line vi editing capability is disabled.

17



Unix Shell: Tips in customizing user’s shell environment

The customizations made during a particular login session would exist only for that session.

To make the customizations permanent, we need to do these customizations in shell’s start-up script, .bash_profile and .bashrc in Unix.

Changes to PATH variable or any other variable could be done in .profile in AIX or .bash_profile in Unix.

Aliases, functions, and set commands could be defined in .bashrc in Unix.



Unix

Module 2: Advanced Shell Scripting

19



Module agenda

Shell and sub shell

Execute a shell script in background

- basename command

- exec command

- eval command

- expr command

- let command

20



Shell and sub shell

Let us assume that we have a shell script called createVars.sh, which is as below:

planet=Mercuryflower=Lotus

When we execute the above shell script in one of the below methods, the shell script is executed in sub shell’s environment.

$ bash dispSysInfo.sh$ ./dispSysInfo.sh$ dispSysInfo.sh

As the script is executed in sub shell, the variables that are created in the script are not available to the current shell.

21



Shell and sub shell (continued)

If the variables are to be created in current shell, then we need to execute the script in the current shell’s environment.

For executing the script in the current shell’s environment, we need to execute the script with an internal command known as . (dot) command or source command.

Syntax:

$ . createVars.sh (or)$ source createVars.sh

When the script is executed in the current shell’s environment, the variables created in the script are also available in the current shell.

22




When we specify a set of commands within a pair of parenthesis, then these commands are executed in sub shell’s environment.

Let us consider the following example:

$ (cd cppdir; pwd; )/home/murali/cppdir$ pwd/home/murali

As the cd command is executed in the sub shell’s environment, the working directory is changed to cppdir in the sub shell’s environment. As the pwd command is also executed in the same sub shell’s environment, it displays the working directory as cppdir.

23




But once both the commands are executed, the control is returned back to the login shell.

The login shell is unaffected by the change of directory made in the sub shell.

Hence, pwd command when issued in the $ prompt displays the user’s home directory.

24




When we specify a set of commands within a pair of curly braces, then these commands are executed in the current shell’s environment.


$ { cd cppdir; pwd; }/home/murali/cppdir$ pwd/home/murali/cppdir

In the above example, as the cd command and the pwd command are executed in the current shell, the working directory is changed to /home/murali/cppdir in the current shell. Hence, the above output.

25



Execute a shell script in background

A process which is not user-interactive, does not need the terminal, and hence could be executed in the background.

When a shell script is non user-interactive (for example, it may be a script which reads information from system, and updates a log-file) it could be run in background.

Example: Let us consider we have a shell script as below:

while true # mybgscript.shdo

df >> myDiskStatus.textdate +%d-%h-%Y:%H:%M%S >> myDiskStatus.text

sleep 60done

26



Execute a shell script in background (continued)

To execute the script mybgscript.sh in the background, we need to execute it in the following way:

$ sh mybgscript.sh &[1] 7636

The kernel allocates a pid for the script, and the script is run in the background. The shell displays the pid and the job-ID of the background process.

The job command also displays information about user-initiated background jobs and suspended jobs.

$ jobs[1]+ Running sh mybgscript.sh &

27




We can control a background process by sending a signal to it using the kill command.

Syntax of kill command:

kill –s <signal> <pid>kill –s <signal> %<job_id>

For instance, to suspend a background process with a job-id 1, we could issue the following command:

$ kill -s TSTP %1[1]+ Stopped sh mybgscript.sh

28




Suppose the user x has initiated a script called as script1.sh, to execute in background, which periodically updates a logfile, with the file system status.

Suppose that the user x logs out of his session after some time, forgetting about the background process he started. Now, what would happen to this background process?

Before the login shell terminates, it sends HUP signal to all its child processes, to terminate them.

Hence, the script script1.sh is also terminated abruptly.

If the script script1.sh has to execute without any interruption even after the user x logs out, then the script has to be initiated in the background using nohup command, which is illustrated in the next slide.

29




$ nohup sh mybgscript.sh &[1] 9343$ jobs

[1]+ Running nohup sh mybgscript.sh &

Now, even if the user x logs out, the script would continue to execute uninterruptedly.

In this case, before the shell terminates, the shell puts a request to the init process to take parentage of the background child process.

Once init process takes up the parentage, the login shell terminates.

30



The basename command

Syntax:

basename <absolute_path> (or)basename <filename> ‘<suffix>’

When a “/” separated pathname is specified with basename command, then the command extracts the filename specified after the rightmost “/”, and displays it on the standard output.

When a filename is specified as first argument and a quoted suffix is specified as the second argument, basename command removes suffix from the first argument, and returns the resultant prefix.

31



The basename command: Example

$ basename unixnotes.text '.text‘unixnotes$ basename /opt/freeware/binbin

32



The exec command

The exec command, when used with input redirection operator, <, loads the specified disk file into the buffer associated with the standard input device, keyboard; it also sets temporarily the standard input device as the disk file.

Syntax:

exec < employee.data

Now, read command reads one line from the buffer and assigns to the variable specified, rather than issuing input interrupt.

If we are giving the above statement in a script, and if in the same script, subsequently, we want to take input from the user, then we need to re-assign the standard input file as the keyboard. This could be done as follows:

exec < /dev/tty

33



The eval command

The eval command parses the command line arguments twice.

Let us understand the purpose of eval command with the help of the following example:

$ hb=planet$ planet=mercury$ eval echo \$$hbmercury

During the first parse, special meaning of first $ is not interpreted as it is preceded by escape character, “\”; and hence, shell gets the value of $hb, namely planet.

During the second parse, the “\” character is ignored, and hence $planet is evaluated to mercury; hence the result.

34



The eval command: Example

Let us consider another example:

$ prompt1="Enter emp number“$ prompt2="Enter emp name“$ i=1$ eval echo \$prompt$iEnter emp number$ i=2$ eval echo \$prompt$iEnter emp name

35



The expr command

We can perform arithmetic operations as well as string operations with expr.

Following are the arithmetic operators supported by expr:

+ : It performs addition

- : It performs subtraction

* : It performs multiplication

/ : It performs division

% : It performs modulus operation

36



The expr command (continued)

String length:

$ echo ${#str}10$ expr length $str10$ expr $str : '.*‘10

37




Length of matching substring at the beginning of string:

$ str=abcABC123ABCabc$ expr match $str 'abc[A-Z]*.2‘8$ expr $str : 'abc[A-Z]*.2‘8

Index:

$ str=IBM_Values$ expr index $str "Val“5

38




Substring extraction:

$ str="Much ado about nothing“$ echo ${str:5}ado about nothing$ echo ${str:5:3}ado$ str="IBM_Values“$ expr substr $str 5 3Val

39



The let command

The let command allows us to perform arithmetic operations.

Syntax:

let var=value (or)let var=<arithmetic_expression>

The let command of bash supports the following operators:

- + This is for addition.

- - This is for subtraction.

- * This is for multiplication.

- / This is for division.

- % This is for modulus.

- ** This is for exponentiation.

40



The let command (continued)

The let command of bash supports the following operators:

- ++ This is for prefix and postfix increment operator.

- -- This is for prefix and postfix decrement operator.

The following are the arithmetic operators supported:

- +=

- -=

- *=

- /=

- %=

Limitation of let command:

- We can not perform floating point arithmetic with let command.

41



Questions and answers

rbtolentino

This is a new slide.

42



Knowledge check

1. When we specify a set of commands within a pair of parenthesis, these commands are executed in the current shell’s environment. True or false?

2. What command allows us to perform arithmetic operations?

3. What command parses the command line arguments twice?



Unix

Module 3: Arrays

44



Module agenda

Definition

Array size

Adding Array elements

45



Definition

An array is a collection of similar elements, stored in contiguous memory area.

Shell supports only single dimensional arrays.

An array is defined as follows:

$ planets=(Mercury Venus Earth Mars)

Elements in the array can be listed as follows:

$ echo ${planets[@]}Mercury Venus Earth Mars

46



Array size

Array elements have subscripts starting from 0. To access the second element of the array, we could issue the command as follows:

$ echo ${planets[1]}Venus

To find the number of elements in the array or array size, we can issue the following command:

$ echo ${#planets[@]}4$ echo ${#planets[0]}7

47



Adding Array elements

We can add some more elements to the array, as follows:

$ planets[4]=Jupiter$ planets[5]=Saturn

Now, elements in the array are:

$ echo ${planets[@]}Mercury Venus Earth Mars Jupiter Saturn Uranus

Size of the array is:

$ echo ${#planets[@]}7

48




49



Knowledge check

1. What is an Array?

2. How will you access the 5th element of the array kingdoms?

3. How can you add the element Finertia to the array of size 9, kingdoms?



Unix

Module 4: Functions

51



Module agenda

Function

Library file

Passing parameters to a function

Functions in all log-in sessions

Nested functions

52



Function

A function is an independent named block of code that could be called by any application.

A function executes in its own runtime environment.

A function is defined as follows:

$ myfun()> {> echo "Hello world“> }$ myfunHello world

53



Library file

A library file can be created as follows:

$ cat function.shmyadd(){

echo "scale=2;$1+$2;"|bc}mymul(){

echo "scale=2;$1*$2;"|bc}

54



Library file (continued)

The library file function.sh can be included in any script application which needs to use those functions.

Let us look at an example:

$ cat mymain.shif [ $# -ne 2 ]; then

echo "Usage: $0 <num1> <num2>“; exit 1fi. function.shecho "Sum of $1 and $2 is $(myadd $1 $2)“echo "Product of $1 and $2 is $(mymul $1 $2)"

55



Passing parameters to a function

Each of the parameters that passed through a function call should be separated by a space.

Inside the function, these parameters are accessible in the built-in variables of the function, namely $1, $2, $3, and so on.

$# stores the number of arguments and $* and $@ stores list of arguments passed to function.

56



Functions in all log-in sessions

If the user wants to have a set of functions to be available in all of his or her login sessions, the user can specify these functions in the start-up script of the shell, $HOME/.bashrc

For example, let us suppose the user adds these functions in the file $HOME/.bashrc:

$ cat .bashrc# .bashrc# Source global definitionsif [ -f /etc/bashrc ]; then

. /etc/bashrcfi

57



Functions in all log-in sessions (continued)

$ cat .bashrc # output continued from previous slide…# User specific aliases and functions

mychmod() { # to add execute permission for any shellchmod u+x $1 # script, to user, the user needs to specify} # as: mychmod <filename>lpdfile() { # this function displays the latest modified filels –t | head -1

}

58



Nested functions

A function can be called from another function. This is known as nesting of functions.


greater () {if [ $1 -gt $2 ]; then

echo $1 is greater than $2;elif [ $2 -gt $1 ]; thenecho $2 is greater than $1;

else echo Both $1 and $2 are equal...;fi;

fi }

59



Nested functions (continued)

display_gr () {echo "The status of comparison of $1 and $2 is as follows...";greater $1 $2

}$ display_gr 38 38

The status of comparison of 38 and 38 is as follows:

Both 38 and 38 are equal...

$ display_gr -40 0

The status of comparison of -40 and 0 is as follows:

0 is greater than -40

60




61



Knowledge check

1. What is a function?

2. What should be done to have a set of functions available in all login sessions?

3. What is a nested function?



Unix

Exercises

63



Exercise 1

1. Write a script that finds a new file created, every minute, in any of the subdirectories of your home directory, and writes that information in a logfile called fileCreated.data. This script should be executed in the background.

2. What would you do if you don’t want the script to be terminated even after you log out?

3. Write a script that would accept a pattern and a set of filenames as command line arguments; the script should tell the user as to how many times the pattern occurs in each of these files. Write appropriate code to check that the user has entered the right number of arguments and give appropriate usage message. Put this code in a function.

64



Exercise 2

4. Write a shell script to accept employee number, emp name, job, doj, basic pay from the user. This process has to be done repeatedly wherein before terminating every iteration, it will ask the user if he or she wants to enter another emp details, and based on the user response take decision about continuing the iteration.

Check the following conditions:

- The emp number should not be null; it should be unique.

- The emp name should not be null, and it should not contain any other characters other than alphabets, white spaces and . (dot) characters.

- The job should not be null.

- For doj, accept date, month, and year as separate inputs, check for the validity in the doj inputs, and concatenate dd, mm and yy with '/' character

- The basic pay input should contain only floating point values. Basic pay should not be less than 6000.00.

65



Thank you for attendingThank you for attending

Documents

Unix - Advanced Shell Scripting_1 Day