1
IBM Global Business Services
© Copyright IBM Corporation 2010
Advanced Shell ScriptingAdvanced Shell Scripting
2
IBM Global Business Services
© Copyright IBM Corporation 2010
Session objectives
By the end of this session, you should be able to:
Execute shell script
Use kill, basename, exec, eval, expr, and let commands
Create and use Arrays
Use functions and nested functions
Create library files
3
IBM Global Business Services
© Copyright IBM Corporation 2010
Course content (Day 3)
Module 1: Customizing shell environment
Module 2: Advanced shell scripting
Module 3: Arrays
Module 4: Functions
© Copyright IBM Corporation 2010
IBM Global Business Services
Unix
Module 1: Customizing Shell Environment
5
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Shell variables
There are two types of shell variables, namely, predefined shell variables and user-defined shell variables.
The built-in variables reside in the environment right from the time the user logs in.
The user-defined variables created during a log-in session, exists in the environment till the user logs out.
When the user creates a sub-shell, the variables are not inherited to the sub-shell.
If the variables have to be inherited to the sub-shells, then the user needs to issue the command:$ export <var1> <var2> <var3> …
6
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Shell variables (continued)
A shell variable can be removed from the environment with the following command:unset <var1> <var2> <var3> …
A shell variable can be made read only by issuing the following command:readonly <var1> <var2> <var3> …
A read-only shell variable can neither be removed from the environment nor its value can be changed.
7
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Shell variables: Built-in variables
PATH – This stores a list of absolute pathname of directories delimited by :, which are searched from left to right by the shell, when user issues an external command.
MAIL – This stores the absolute pathname of the file which stores the user’s mails.
MAILCHECK – This stores the number of seconds in the interval of which the mail daemon process checks for the arrival of new mails in the user’s mailbox.
8
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Shell variables: Built-in variables (continued)
HOME – This stores the absolute pathname of user’s home directory.
PS1 – This stores the user’s primary prompt.
PS2 – This stores the user’s secondary prompt.
LOGNAME – This stores the user’s login name.
PWD – This stores the user’s working directory.
To display the content of a variable, we have to issue the following command: echo ${<var_name>}
9
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Shell variables: Commands displaying variables
env – This displays the list of built-in shell variables and their values.
set – This displays the list of built-in shell variables and user-defined variables with their values and user-defined function definitions, in bash environment.
set – This displays the list of built-in shell variables and user defined variables with their values in korn shell environment.
10
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Customizing user’s shell environment
User’s shell environment can be customized by changing the default values of the built-in variables.
Examples:
$ echo $PS1
$ PS1=“Hello “
$ echo $PS1
Hello
$ PATH=$PATH:$HOME
$ echo $PATH
/usr/kerberos/bin:/usr/bin:/bin:/home/manoj/bin: /home/manoj
Explanation:
The shell displays value stored in variable PS1 as primary prompt for the shell.
We can append any desired directory to PATH variable, as indicated in the code example.
11
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Customizing user’s shell environment (continued)
The shell’s start-up scripts in the Unix environment are:
/etc/bashrc or /etc/environment – universal start-up script. The commands in this script would affect all the user’s shell environment. This script can be edited only by the super user.
$HOME/.bashrc or $HOME/.kshrc– bash / korn specific script, which can be customized by any user
$HOME/.bash_profile or $HOME/.profile – bash or korn specific script, where the environment variables are set
These scripts are run by the shell in the order given, when the user logs in.
12
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Customizing user’s shell environment (continued)
Aliases – Aliases are short-names for the commands. A long command may be required to be executed very often. The solution for this is to create an alias.
Syntax:$ alias myfind='find $HOME -name "*.sh" -print 2> /dev/null'
The above alias would reside in the memory as long as the user is logged in. During the next log in, this alias myfind won’t be available. We can make this alias available in all login sessions by adding this alias command in the start-up script, $HOME/.bashrc (in bash) or $HOME/.kshrc (in korn shell) .
13
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Customizing user’s shell environment (continued)
Example:
$ set -o noclobber
$ cat > animals
-bash: animals: cannot overwrite existing file
$ set +o noclobber
$ cat > animals
lion
tiger
giraffee
leopard
14
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Customizing user’s shell environment (continued)
Example:
$ set -o noclobber
$ cat > animals
-bash: animals: cannot overwrite existing file
$ cat >| animals
jackal
fox
donkey
The above example shows that we can override the noclobber setting by succeeding > symbol with a | symbol.
15
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Set commands
The Set commands to customize shell environment are the following:
The setting ‘set –o noclobber’ can be unset by issuing the following command:set +o noclobber
$ set -o ignoreeofWhen the user presses <ctrl-d> key on his login prompt, the user is logged off. Issuing the above command ensures that the user is NOT logged off if he presses <ctrl-d> on his login prompt.
$ set +o ignoreeofThe above command when issued, would unset the ‘ignoreeof’ setting.
16
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Set commands (continued)
The Set commands to customize shell environment are the following:
$ set -o viWhen the above command is issued, command line is enabled with vi editing capabilities.
$ set +o viWhen the above command is issued, command line vi editing capability is disabled.
17
IBM Global Business Services
© Copyright IBM Corporation 2010
Unix Shell: Tips in customizing user’s shell environment
The customizations made during a particular login session would exist only for that session.
To make the customizations permanent, we need to do these customizations in shell’s start-up script, .bash_profile and .bashrc in Unix.
Changes to PATH variable or any other variable could be done in .profile in AIX or .bash_profile in Unix.
Aliases, functions, and set commands could be defined in .bashrc in Unix.
© Copyright IBM Corporation 2010
IBM Global Business Services
Unix
Module 2: Advanced Shell Scripting
19
IBM Global Business Services
© Copyright IBM Corporation 2010
Module agenda
Shell and sub shell
Execute a shell script in background
- basename command
- exec command
- eval command
- expr command
- let command
20
IBM Global Business Services
© Copyright IBM Corporation 2010
Shell and sub shell
Let us assume that we have a shell script called createVars.sh, which is as below:
planet=Mercuryflower=Lotus
When we execute the above shell script in one of the below methods, the shell script is executed in sub shell’s environment.
$ bash dispSysInfo.sh$ ./dispSysInfo.sh$ dispSysInfo.sh
As the script is executed in sub shell, the variables that are created in the script are not available to the current shell.
21
IBM Global Business Services
© Copyright IBM Corporation 2010
Shell and sub shell (continued)
If the variables are to be created in current shell, then we need to execute the script in the current shell’s environment.
For executing the script in the current shell’s environment, we need to execute the script with an internal command known as . (dot) command or source command.
Syntax:
$ . createVars.sh (or)$ source createVars.sh
When the script is executed in the current shell’s environment, the variables created in the script are also available in the current shell.
22
IBM Global Business Services
© Copyright IBM Corporation 2010
Shell and sub shell (continued)
When we specify a set of commands within a pair of parenthesis, then these commands are executed in sub shell’s environment.
Let us consider the following example:
$ (cd cppdir; pwd; )/home/murali/cppdir$ pwd/home/murali
As the cd command is executed in the sub shell’s environment, the working directory is changed to cppdir in the sub shell’s environment. As the pwd command is also executed in the same sub shell’s environment, it displays the working directory as cppdir.
23
IBM Global Business Services
© Copyright IBM Corporation 2010
Shell and sub shell (continued)
But once both the commands are executed, the control is returned back to the login shell.
The login shell is unaffected by the change of directory made in the sub shell.
Hence, pwd command when issued in the $ prompt displays the user’s home directory.
24
IBM Global Business Services
© Copyright IBM Corporation 2010
Shell and sub shell (continued)
When we specify a set of commands within a pair of curly braces, then these commands are executed in the current shell’s environment.
Let us consider the following example:
$ { cd cppdir; pwd; }/home/murali/cppdir$ pwd/home/murali/cppdir
In the above example, as the cd command and the pwd command are executed in the current shell, the working directory is changed to /home/murali/cppdir in the current shell. Hence, the above output.
25
IBM Global Business Services
© Copyright IBM Corporation 2010
Execute a shell script in background
A process which is not user-interactive, does not need the terminal, and hence could be executed in the background.
When a shell script is non user-interactive (for example, it may be a script which reads information from system, and updates a log-file) it could be run in background.
Example: Let us consider we have a shell script as below:
while true # mybgscript.shdo
df >> myDiskStatus.textdate +%d-%h-%Y:%H:%M%S >> myDiskStatus.text
sleep 60done
26
IBM Global Business Services
© Copyright IBM Corporation 2010
Execute a shell script in background (continued)
To execute the script mybgscript.sh in the background, we need to execute it in the following way:
$ sh mybgscript.sh &[1] 7636
The kernel allocates a pid for the script, and the script is run in the background. The shell displays the pid and the job-ID of the background process.
The job command also displays information about user-initiated background jobs and suspended jobs.
$ jobs[1]+ Running sh mybgscript.sh &
27
IBM Global Business Services
© Copyright IBM Corporation 2010
Execute a shell script in background (continued)
We can control a background process by sending a signal to it using the kill command.
Syntax of kill command:
kill –s <signal> <pid>kill –s <signal> %<job_id>
For instance, to suspend a background process with a job-id 1, we could issue the following command:
$ kill -s TSTP %1[1]+ Stopped sh mybgscript.sh
28
IBM Global Business Services
© Copyright IBM Corporation 2010
Execute a shell script in background (continued)
Suppose the user x has initiated a script called as script1.sh, to execute in background, which periodically updates a logfile, with the file system status.
Suppose that the user x logs out of his session after some time, forgetting about the background process he started. Now, what would happen to this background process?
Before the login shell terminates, it sends HUP signal to all its child processes, to terminate them.
Hence, the script script1.sh is also terminated abruptly.
If the script script1.sh has to execute without any interruption even after the user x logs out, then the script has to be initiated in the background using nohup command, which is illustrated in the next slide.
29
IBM Global Business Services
© Copyright IBM Corporation 2010
Execute a shell script in background (continued)
$ nohup sh mybgscript.sh &[1] 9343$ jobs
[1]+ Running nohup sh mybgscript.sh &
Now, even if the user x logs out, the script would continue to execute uninterruptedly.
In this case, before the shell terminates, the shell puts a request to the init process to take parentage of the background child process.
Once init process takes up the parentage, the login shell terminates.
30
IBM Global Business Services
© Copyright IBM Corporation 2010
The basename command
Syntax:
basename <absolute_path> (or)basename <filename> ‘<suffix>’
When a “/” separated pathname is specified with basename command, then the command extracts the filename specified after the rightmost “/”, and displays it on the standard output.
When a filename is specified as first argument and a quoted suffix is specified as the second argument, basename command removes suffix from the first argument, and returns the resultant prefix.
31
IBM Global Business Services
© Copyright IBM Corporation 2010
The basename command: Example
$ basename unixnotes.text '.text‘unixnotes$ basename /opt/freeware/binbin
32
IBM Global Business Services
© Copyright IBM Corporation 2010
The exec command
The exec command, when used with input redirection operator, <, loads the specified disk file into the buffer associated with the standard input device, keyboard; it also sets temporarily the standard input device as the disk file.
Syntax:
exec < employee.data
Now, read command reads one line from the buffer and assigns to the variable specified, rather than issuing input interrupt.
If we are giving the above statement in a script, and if in the same script, subsequently, we want to take input from the user, then we need to re-assign the standard input file as the keyboard. This could be done as follows:
exec < /dev/tty
33
IBM Global Business Services
© Copyright IBM Corporation 2010
The eval command
The eval command parses the command line arguments twice.
Let us understand the purpose of eval command with the help of the following example:
$ hb=planet$ planet=mercury$ eval echo \$$hbmercury
During the first parse, special meaning of first $ is not interpreted as it is preceded by escape character, “\”; and hence, shell gets the value of $hb, namely planet.
During the second parse, the “\” character is ignored, and hence $planet is evaluated to mercury; hence the result.
34
IBM Global Business Services
© Copyright IBM Corporation 2010
The eval command: Example
Let us consider another example:
$ prompt1="Enter emp number“$ prompt2="Enter emp name“$ i=1$ eval echo \$prompt$iEnter emp number$ i=2$ eval echo \$prompt$iEnter emp name
35
IBM Global Business Services
© Copyright IBM Corporation 2010
The expr command
We can perform arithmetic operations as well as string operations with expr.
Following are the arithmetic operators supported by expr:
+ : It performs addition
- : It performs subtraction
* : It performs multiplication
/ : It performs division
% : It performs modulus operation
36
IBM Global Business Services
© Copyright IBM Corporation 2010
The expr command (continued)
String length:
$ echo ${#str}10$ expr length $str10$ expr $str : '.*‘10
37
IBM Global Business Services
© Copyright IBM Corporation 2010
The expr command (continued)
Length of matching substring at the beginning of string:
$ str=abcABC123ABCabc$ expr match $str 'abc[A-Z]*.2‘8$ expr $str : 'abc[A-Z]*.2‘8
Index:
$ str=IBM_Values$ expr index $str "Val“5
38
IBM Global Business Services
© Copyright IBM Corporation 2010
The expr command (continued)
Substring extraction:
$ str="Much ado about nothing“$ echo ${str:5}ado about nothing$ echo ${str:5:3}ado$ str="IBM_Values“$ expr substr $str 5 3Val
39
IBM Global Business Services
© Copyright IBM Corporation 2010
The let command
The let command allows us to perform arithmetic operations.
Syntax:
let var=value (or)let var=<arithmetic_expression>
The let command of bash supports the following operators:
- + This is for addition.
- - This is for subtraction.
- * This is for multiplication.
- / This is for division.
- % This is for modulus.
- ** This is for exponentiation.
40
IBM Global Business Services
© Copyright IBM Corporation 2010
The let command (continued)
The let command of bash supports the following operators:
- ++ This is for prefix and postfix increment operator.
- -- This is for prefix and postfix decrement operator.
The following are the arithmetic operators supported:
- +=
- -=
- *=
- /=
- %=
Limitation of let command:
- We can not perform floating point arithmetic with let command.
41
IBM Global Business Services
© Copyright IBM Corporation 2010
Questions and answers
42
IBM Global Business Services
© Copyright IBM Corporation 2010
Knowledge check
1. When we specify a set of commands within a pair of parenthesis, these commands are executed in the current shell’s environment. True or false?
2. What command allows us to perform arithmetic operations?
3. What command parses the command line arguments twice?
© Copyright IBM Corporation 2010
IBM Global Business Services
Unix
Module 3: Arrays
44
IBM Global Business Services
© Copyright IBM Corporation 2010
Module agenda
Definition
Array size
Adding Array elements
45
IBM Global Business Services
© Copyright IBM Corporation 2010
Definition
An array is a collection of similar elements, stored in contiguous memory area.
Shell supports only single dimensional arrays.
An array is defined as follows:
$ planets=(Mercury Venus Earth Mars)
Elements in the array can be listed as follows:
$ echo ${planets[@]}Mercury Venus Earth Mars
46
IBM Global Business Services
© Copyright IBM Corporation 2010
Array size
Array elements have subscripts starting from 0. To access the second element of the array, we could issue the command as follows:
$ echo ${planets[1]}Venus
To find the number of elements in the array or array size, we can issue the following command:
$ echo ${#planets[@]}4$ echo ${#planets[0]}7
47
IBM Global Business Services
© Copyright IBM Corporation 2010
Adding Array elements
We can add some more elements to the array, as follows:
$ planets[4]=Jupiter$ planets[5]=Saturn
Now, elements in the array are:
$ echo ${planets[@]}Mercury Venus Earth Mars Jupiter Saturn Uranus
Size of the array is:
$ echo ${#planets[@]}7
48
IBM Global Business Services
© Copyright IBM Corporation 2010
Questions and answers
49
IBM Global Business Services
© Copyright IBM Corporation 2010
Knowledge check
1. What is an Array?
2. How will you access the 5th element of the array kingdoms?
3. How can you add the element Finertia to the array of size 9, kingdoms?
© Copyright IBM Corporation 2010
IBM Global Business Services
Unix
Module 4: Functions
51
IBM Global Business Services
© Copyright IBM Corporation 2010
Module agenda
Function
Library file
Passing parameters to a function
Functions in all log-in sessions
Nested functions
52
IBM Global Business Services
© Copyright IBM Corporation 2010
Function
A function is an independent named block of code that could be called by any application.
A function executes in its own runtime environment.
A function is defined as follows:
$ myfun()> {> echo "Hello world“> }$ myfunHello world
53
IBM Global Business Services
© Copyright IBM Corporation 2010
Library file
A library file can be created as follows:
$ cat function.shmyadd(){
echo "scale=2;$1+$2;"|bc}mymul(){
echo "scale=2;$1*$2;"|bc}
54
IBM Global Business Services
© Copyright IBM Corporation 2010
Library file (continued)
The library file function.sh can be included in any script application which needs to use those functions.
Let us look at an example:
$ cat mymain.shif [ $# -ne 2 ]; then
echo "Usage: $0 <num1> <num2>“; exit 1fi. function.shecho "Sum of $1 and $2 is $(myadd $1 $2)“echo "Product of $1 and $2 is $(mymul $1 $2)"
55
IBM Global Business Services
© Copyright IBM Corporation 2010
Passing parameters to a function
Each of the parameters that passed through a function call should be separated by a space.
Inside the function, these parameters are accessible in the built-in variables of the function, namely $1, $2, $3, and so on.
$# stores the number of arguments and $* and $@ stores list of arguments passed to function.
56
IBM Global Business Services
© Copyright IBM Corporation 2010
Functions in all log-in sessions
If the user wants to have a set of functions to be available in all of his or her login sessions, the user can specify these functions in the start-up script of the shell, $HOME/.bashrc
For example, let us suppose the user adds these functions in the file $HOME/.bashrc:
$ cat .bashrc# .bashrc# Source global definitionsif [ -f /etc/bashrc ]; then
. /etc/bashrcfi
57
IBM Global Business Services
© Copyright IBM Corporation 2010
Functions in all log-in sessions (continued)
$ cat .bashrc # output continued from previous slide…# User specific aliases and functions
mychmod() { # to add execute permission for any shellchmod u+x $1 # script, to user, the user needs to specify} # as: mychmod <filename>lpdfile() { # this function displays the latest modified filels –t | head -1
}
58
IBM Global Business Services
© Copyright IBM Corporation 2010
Nested functions
A function can be called from another function. This is known as nesting of functions.
Let us consider the following example:
greater () {if [ $1 -gt $2 ]; then
echo $1 is greater than $2;elif [ $2 -gt $1 ]; thenecho $2 is greater than $1;
else echo Both $1 and $2 are equal...;fi;
fi }
59
IBM Global Business Services
© Copyright IBM Corporation 2010
Nested functions (continued)
display_gr () {echo "The status of comparison of $1 and $2 is as follows...";greater $1 $2
}$ display_gr 38 38
The status of comparison of 38 and 38 is as follows:
Both 38 and 38 are equal...
$ display_gr -40 0
The status of comparison of -40 and 0 is as follows:
0 is greater than -40
60
IBM Global Business Services
© Copyright IBM Corporation 2010
Questions and answers
61
IBM Global Business Services
© Copyright IBM Corporation 2010
Knowledge check
1. What is a function?
2. What should be done to have a set of functions available in all login sessions?
3. What is a nested function?
© Copyright IBM Corporation 2010
IBM Global Business Services
Unix
Exercises
63
IBM Global Business Services
© Copyright IBM Corporation 2010
Exercise 1
1. Write a script that finds a new file created, every minute, in any of the subdirectories of your home directory, and writes that information in a logfile called fileCreated.data. This script should be executed in the background.
2. What would you do if you don’t want the script to be terminated even after you log out?
3. Write a script that would accept a pattern and a set of filenames as command line arguments; the script should tell the user as to how many times the pattern occurs in each of these files. Write appropriate code to check that the user has entered the right number of arguments and give appropriate usage message. Put this code in a function.
64
IBM Global Business Services
© Copyright IBM Corporation 2010
Exercise 2
4. Write a shell script to accept employee number, emp name, job, doj, basic pay from the user. This process has to be done repeatedly wherein before terminating every iteration, it will ask the user if he or she wants to enter another emp details, and based on the user response take decision about continuing the iteration.
Check the following conditions:
- The emp number should not be null; it should be unique.
- The emp name should not be null, and it should not contain any other characters other than alphabets, white spaces and . (dot) characters.
- The job should not be null.
- For doj, accept date, month, and year as separate inputs, check for the validity in the doj inputs, and concatenate dd, mm and yy with '/' character
- The basic pay input should contain only floating point values. Basic pay should not be less than 6000.00.
65
IBM Global Business Services
© Copyright IBM Corporation 2010
Thank you for attendingThank you for attending