Intermediate shell scripting in BASH - folk.uio.nofolk.uio.no/buzh/bash/old/fall2012.pdf · Intermediate shell scripting in BASH ... be used to execute the script: /bin/bash ... Advanced

Intermediate shell scripting in BASHResearch Computing Services course week, fall 2012

Lecturer:Andreas Buzh Skau Head Engineer, RCS

[email protected]://folk.uio.no/buzh/bash/

mailto:[email protected]

11. april 2011Ny Powerpoint mal 20112

What is a shell script?

● The shell is a computer program that presents you with the command line interface used on linux/unix systems.

● A script is a list of commands that you ask the shell to perform, like a theater director giving a script to his cast of actors.

● An actor reads, interprets and executes the script according to the director's instructions. The shell does the same with your scripts.


cd /tmpecho "some text" > some-filels -l /tmp/some-fileps auxw | grep flowers

● In addition to just listing commands sequentially like this, shells provide us with the ability to use variables, tests, loops and other methods found in common programming languages like C, Python, Fortran etc

● Bash scripts are plain text files which can be executed either by making the file executable:

chmod +x myscript.sh and then running the command:

./myscript.sh or by passing the filename to bash itself like this:

bash myscript.sh”

● If the script is chmod +x and in your $PATH, you can type it's name anywhere, and it will run.


● To make scripts in Linux, you should add a “hashbang” to the first line of the file. The # is called a hash, and the ! Is a “bang”. Then; the absolute path to the interpreter which will be used to execute the script: /bin/bash

● Scripts are executed line by line

● This is what a simple script looks like, in it's entirety:

#!/bin/bashecho Hello World


When assigning variables, we simply write

variable=value

● Note! There is no space before or after the = sign.

● To get the contents of the variable we precede the name of the variable with a dollar sign

$variable

● Before the shell executes a line, it will expand $variable to the actual value contained within. This means that in this case, $variable is translated to “value” before execution.

●

●

●

●

Variables:


Variables

mypath=/tmp/buzh/stuffmyfilename=filenamemyfile=$mypath/$myfilenamemytext='Howdy partner!'mynumber=23myarray=(apple banana cucumber daffodil)

mkdir -p $mypathecho “$mytext” > $myfileecho $mynumber >> $myfilemv $myfile $myfile.${myarray[3]}for ((i=0; i<=$mynumber; i++)); do echo $i; done

Usage:


Tests

One of the most useful things in scripting are tests. They make up much of the logic of our scripts, checking whether some condition is true or false and then executing the appropriate commands.

if <condition>; then <commands>; fi

if [ -e "$myfile" ]then echo $mytextelse echo "$myfile does not exist!"fi

[ is actually a program called "test". You can find it's documentation with the command "man test". In this example we use "-e" to check for the existence of a file. It can also do arithmetic tests, string comparisons etc.

You can nest if/then tests, and don't forget about elif:

if [ $mynumber = 23 ]; then echo $mytextelif [ $mynumber = 24 ]; then echo "hello, this can't be right?" if [ $(date +%A) = Sunday ]; then export mynumber=23 echo mynumber is now 23 again else echo Try again next sunday, $(date -d 'next sunday' “+%D %h”)fielse echo "This would be executed if my number is neither 23 nor 24"fi

The program 'date' used above is also a gem of usefulness. See 'man date'.


● If the return value of a test condition is 0, the test is logically true. If it is non-zero, it's logically false. This is the typical UNIX way of determining success or failure of a command. Consider this:

ls /unlikely && echo YEAH || echo NOPE

if (ls /unlikely); then echo YEAH; else echo NOPE; fi

If the path /unlikely exists, ls will exit with a return code 0. If it does not exist, it will return a non-zero value. As you might know, the && command willread the exit status of the previous command and then execute the next command if the exit status was 0, a logical “and”. Conversely || will only execute the next command if the exit status is not 0, a logical “or”.

The “if” command above does the same thing!

The “for” loop:

for <item> in <list>; do <commands>; done

The first argument in this example (<item>) is the name of a variable, to which each value contained in the list you supplied will be assigned. You take an arbitrary word (“fruit”), and through these commands it is used as a variable.

Example:

for fruit in apple banana orange pear;do echo $fruit is a fruit!done

Make lists of anything: for stuff in $(someprogram | grep things); do..Work with lists in files: for line in $(cat /path/to/somefile); do..

You can also write loops more like in traditional C, using arithmetic expansion (more on that later).Example:

for ((i=0; $i<4; i++)); do if [ $i = 0 ]; then echo "The first number is $i"; else echo "then comes $i.." if [ $i = 4 ]; then echo "...and that was all."; fi fi; done

Note the way the variable i is adressed in the first line. i= is an assignment, $i is a reference to the value contained in the variable and lastly, because we are using (( double parenthesis )) we can increase the value of the integer i with 1 by writing i++

Same stuff, different wrappings:for i in {0..4}; do....for i in 0 1 2 3 4; do....for i in $(seq 0 4); do....

The “while” loop:

Syntax: while <condition>; do <commands>; done

Example:

myvar=0mylimit=10

while (( myvar < mylimit ))do echo "My Variable is $myvar, let's add to it" (( myvar += 1 ))done

echo "This will echo only after the while loop has finished"

Functions:

Functions in bash are much like functions in any other programming language.You declare it, then call it with the parameters you want

Note that since bash reads the scripts line by line from the top, you must declare any functions before you use them - or else bash won't recognize the command when you try to call it!

myfunction (){ echo You just called my function}# To invoke this function, we call it as it were any normal command:myfunction

# Or assign the output (“You just called my function”) to a varmytext=$(myfunction)


We've now seen some basic building blocks of a bash shell script. Next, let's see take a step back and look at how bash actually interprets our commands and scripts.

echo Hello, $USER, how are you today?

After bash reads this line, it's "expanded" before execution. So before the command is actually executed, it is rewritten, or expanded, like this:

echo Hello, buzh, how are you today?

..if your username is buzh. The command is executed after this expansion has taken place. It's bash that does the parsing of the variables, the program "echo" has no knowledge of the variable $USER, only it's value

To inspect what will actually be executed, the “echo” command is a good tool:

echo echo Hello, $USER, how are you today?

Expansion also happens for other input with special meaning, not just variables.

echo “Hello, $USER. 2 times 4 equals $(echo “2*4” | bc)”echo This host has $(grep -c bash /etc/passwd) bash users

Bash will try to execute all the little sub-tasks first, then fill in the main command with the results, before executing the expanded command.

In the above examples we use the $sign to expand the result of anothercommand. The paranthesis aroundthe bc and grep commands abovestarts a “subshell”, another instanceof bash, and the prepending $ meansthat the output of this is treated muchlike you would treat a variable.


(Some) Types of expansion:

Curly brace:for n in {0..10}; do echo I can count to $n; done→ for n in 0 1 2 3 4 5 6 7 8 9 10; do....

Command substitution:mycomputer="$(cat /etc/hostname)"→ mycomputer=abel.uio.no

Arithmetic expansion:Fifteen=$(( (10 / 2) + 10 ))→ Fifteen=15

Filename expansion:echo /etc/pa*→ /etc/pam.conf /etc/pam.d /etc/papersize /etc/passwd /etc/passwd-

Special Variables:

$0 - the name of the parent, eg "scriptname" or "bash" $1 to $n - the parameter given on the n'th position “$*” - all the parameters passed as a string - REMEMBER TO QUOTE THIS “$@” - same as $*, but each word quoted - must also be quoted! $# the number of parameters passed, e.g. n from above safari (){ if [ $# = 3 ]; then # we got 3 parameters echo First I saw a $1 echo Then I saw a $2 echo Finally a wild $3 appeared else # the if condition was false, so $# must be something other than 3 echo "You failed to pass three animals to the safari function" exit 1 fi}

We invoke this like so:

safari cow sheep gnu

These variables work for scripts too, not just functions!

Further reading:

http://www.tldp.org/LDP/abs/html/abs-guide.html

Advanced Bash Scripting Guide – This is a great reference, littered with examples and pretty much the definitive guide to bash scripting

These slides:

http://folk.uio.no/buzh/bash

..also a more wordy txt/html version of this course (a.k.a. the unreadable slides from last time)

The most important thing:

Study your repetitive tasks, and try to automate them!

http://www.tldp.org/LDP/abs/html/abs-guide.html

http://folk.uio.no/buzh/bash

Debugging and troubleshooting#!/bin/bash

myletter=xecho Hello $USERecho In /tmp there are these files with the letter "$myletter" in them:echo $(ls /tmp | grep $myletter)

buzh@stridselg ~ $ bash test.sh Hello buzhIn /tmp there are these files with the letter x in them:xmms_buzh.0

buzh@stridselg ~ $ bash -x test.sh + myletter=x+ echo Hello buzhHello buzh+ echo In /tmp there are these files with the letter x in them:In /tmp there are these files with the letter x in them:++ ls /tmp++ grep x+ echo xmms_buzh.0xmms_buzh.0

#!/bin/bash

func (){ myvar=$(echo $1 | sed s/Hello/Goodbye/) echo $myvar}

greeting=Helloecho $greetinggreeting=$(func $greeting)echo $greeting

This outputs:

HelloGoodbye


while true; do if (ping -c2 vin.uio.no &> /dev/null); then echo We are online at $(date) else echo Uh.. seems down at $(date); fi sleep 30; done

RESULTS=~/myresults

for file in /mydata/*.dat; do TODAY=$(date +%d%m%y) # %d = day, %m month and so on. “date” is very handy! INFILE=$(basename $file) # Basename is a program that parses full file paths and # returns the filename only. Try it on $0 OUTFILE=$RESULTS/$file.$TODAY.log # Because of expansion, OUTFILE will become one long text string # e.g. /mydata/sampledata12345.dat.120412.log mycoolprogram $INFILE &> $OUTFILEdone

Documents

Intermediate shell scripting in BASH - folk.uio.nofolk.uio.no/buzh/bash/old/fall2012.pdf · Intermediate shell scripting in BASH ... be used to execute the script: /bin/bash ... Advanced