Case Study #2

Bourne Again Shell (bash)an interpreted language

What is a Shell?

A program that takes keyboard commands and issues them to the operating system to

be executed on behalf of the user

Several types, e.g., C-Shell (csh, tcsh), Bourne Shell (sh), Bourne Again Shell (bash)

With a graphical interface, terminal emulators are used to issue commands, e.g., terminal,

xterm, konsole, gnome-terminal.

The files in a Unix system are organized hierarchically as directory tree called the file system

the root of the file system is the directory “/”

when a user logs in, the operating system sets the current directory to the user’s home directory, a sub-

tree of the file system where the user can freely operate on files

in the labs the home directory has the form/net/areas/homes/up2018xxxx

to move between directories we use the “cd” command with an absolute or a relative path

an absolute path gives the location of a file with respect to root directory, thus beginning always with “/”

$ cd /net/areas/homes/up2018xxx/IC

a relative path gives the location of a file with respect to the current directory

$ cd ../PI/

the symbols “.” and “..” stand for, respectively, the current directory and the parent directory

command “pwd” can be used to verify which is the current directory

$ pwd

command “tree” can be used to get a graphical representation of the file system

under the current directory

$ cd .. $ tree

we can use the command “ls” or “ls -l” to get extra information about files and directories

$ ls -l PI ; cd PI ; ls -l teste.c

the “-l” option provides extra information on the files, namely, permissions, size, owner, creation date

to find out the type of a given file we can use the command “file”

$ file teste.c teste.c: ASCII c program text

$ gcc teste.c -o teste $ file teste

teste: 64-bit executable x86_64

we can create new directories with the command “mkdir” and new (empty) files with the command “touch” or with

an editor like “emacs”

$ cd PI ; mkdir praticas ; cd praticas$ touch folha2.txt or $ emacs folha2.txt

to remove files and directories we can use the commands “rm” and “rmdir” (the latter only if the directory is empty)

$ cd .. ; rm -rf praticas (be very carefull!)or

$ rm -rf ./* ; cd .. ; rmdir praticas

we can change the permissions associated with files and directories using command “chmod”

chmod a+rwx filechmod ugo-x file

or in the octal notation

chmod 755 file (typical for executables)chmod 644 file (typical for text files)

the shell keeps a record of the commands we execute in a session

$ history

we can see the contents of files using multiple commands

$ cat teste.c$ more teste.c$ less teste.c$ head -n 100 teste.c$ tail -n 100 teste.c

to find files or directories with certain characteristics we can use the powerful command “find”

$ find . -name *.txt -print$ find . -type d -size +1k -print$ find . -name *.o -exec rm -f {}\;

also the command “locate”

$ locate fcup

to find out which version of a program we are running we can use the “which” command

$ which gcc ; which emacs ; which find

to identify differences between files we use “diff”

$ diff teste1.c teste2.c

to find text patterns within files we can use the “grep” and “egrep” commands

$ grep int test.c$ grep \”*\” teste.c$ grep \;$ teste.c

to select only certain parts of the input text we use “cut”

$ cut -d ‘=‘ -f 1 teste.c $ grep = teste.c | cut -d ‘=‘ -f 1

we can find/replace certain text patterns in files using the “sed” command

sed 's/Marco/Paulo/' file.txt(only first instance)sed 's/Marco/Paulo/4' file.txt(only 4th instance)sed ’s/Marco/Paulo/g' file.txt(all instances in line)sed ’s/^[ \t]*//‘ file.txt(remove spaces at beginning of each line)

we may see which processes are running in the computer with “ps”

$ ps (user processes only)$ ps -aux (all processes)

we can execute a command and still be able to use the shell by doing

$ emacs &

the processes thus created can be listed with the command “jobs”

$ jobs

each process has an unique system identifier that can be used to send it signals, namely the “commit suicide” signal

$ kill %1 (the number in the list given by jobs)

or

$ kill -9 23893 (the PID given by ps)

shell commands can be composed sequentially

$ cd PI ; gcc teste.c -o teste ; ./teste

or through pipes that link the output of the first to the input of the second

$ cat teste.c | grep int

$ ps -aux | grep ^up2018xxxx

we may redirect the input and/or output of commands using the “<“ and “>” operators, respectively

$ cat file1.txt file2.txt > all.txt

$ gcc calcPi.c -o calcPi$ ./calcPi > digitosPi.txt

$ gcc calcMedia.c -o calcMedia$ ./calcMedia < dados.txt

SHELL SCRIPTING

the following material is mostly based on the excellent online tutorial by Ryan Chadwick

https://ryanstutorials.net/bash-scripting-tutorial/

it is possible to include multiple shell commands in a file, with a little extra syntax, to

perform more complex processing

such a shell program is commonly called a (shell) script

shell scripts are a fundamental tool for the software developer and, even more, to the

system/network administrator

they allow many tedious tasks to be programmed once and automated

we will now see how to write such scripts

your very first Bash script

#!/bin/bash

echo Hello World!

write it in emacs, save to myscript.sh

the sequence #! in the beginning is known as the ‘shebang’

after it comes the path to the interpreter that will be used to process the remainder of the file

you can run it as follows

$ ./myscript.sh

you may need to change the file permissions

$ chmod 755 myscript.sh$ ./myscript.sh

scripts can have command line arguments

these can be accessed through the variables $1, $2, …

this script makes a copy of the file $1 into a file named $2

#!/bin/bashcp $1 $2# check that $1 and $2 are similarls -l $1ls -l $2

there are several special variables

$0 — the name of the script$1 to $9 — the first 9 arguments to the script$# — the number of arguments passed to the script$@ — all the arguments passed to the script$? — the exit status of the most recently run process$$ — the process ID of the script$USER — the username of the user running the script$HOSTNAME — the name of the machine running the script$SECONDS — the number of seconds since script started$RANDOM — different random number each reference $LINENO — current line number in the script

we can also define variables to hold temporary values

#!/bin/bashmyvariable=Helloanothervar=Fredecho $myvariable $anothervarechosampledir=/etcls $sampledir

we can assign constants and the values in other variables to variables

var1=3var2=“Hello!”var3=/etc/passwdvar4=$var3var5=$HOSTNAME

but also, the results of commands

var=$( ls /etc | wc -l )

note: no space on either side of the equals ( = ) sign !!

we can interactively ask users for input

#!/bin/bashecho Hello, who am I talking to?read varnameecho Nice to meet you $varname

we can format the request

#!/bin/bashread -p 'Username: ' uservarread -sp 'Password: ' passvarechoecho $uservar we have your login details

each process in a Unix system has a PID and associated input, output and error devices

STDIN — /dev/stdinSTDOUT — /dev/stdoutSTDERR — /dev/stderr

the STDIN, STDOUT and STDERR can be redirected with the operators |, > and <

cmd1|cmd2 — connect STDOUT of cmd1 to STDIN of cmd2cmd < file — file is used as STDIN of cmdcmd > file — file is used as STDOUT of cmd

using stdin to feed data to a bash script that cuts the 2 first fields of each line and sorts the resulting list of values

#!/bin/bash# basic sales reportecho Summary of the sales data:echo ==========================echocat /dev/stdin | cut -d ' ' -f 2,3 | sort

this can be executed as

$ ./summary.sh < sales_data.txt

or $ cat sales_data.txt | ./summary.sh

$ cat sales_data.txtFred apples 20 November 4Susy oranges 5 November 7Mark watermelons 12 November 10Terry peaches 7 November 15

$ cat sales_data.txt | ./summaryor$ ./summary < sales_data.txt

Summary of the sales data:==========================apples 20oranges 5peaches 7watermelons 12

we can also do arithmetic using let to assign the result to a variable

#!/bin/bashlet a=5+4echo $a # 9let "a = 5 + 4"echo $a # 9let a++echo $a # 10let "a = 4 * 5"echo $a # 20let "a = $1 + 30"echo $a # 30 + first argument

or using expr that evaluates and prints the result immediately

#!/bin/bashexpr 5 + 4 #9expr “5 + 4” #5 + 4 expr 5+4 # 5+4expr 5 \* $1 # $1 * 5expr 11 % 2 # 1a=$( expr 10 - 3 )echo $a # 7

sometimes it is very useful to know the size of a variable, i.e., the number of characters it

requires to be represented

#!/bin/basha='Hello World'echo ${#a} # 11b=4953echo ${#b} # 4

bash also features basic “conditional statements”

#!/bin/bash

if [ $1 -gt 100 ]then

echo Hey that\'s a large number.fi

#!/bin/bashif [ $1 -eq 0 ]thenecho You have a zero

elif [ $1 -gt 0 ]thenecho You have a positive

elseecho You have a negative

fi

full form of “conditional statement”

! EXPRESSION (expression is false)-n STRING (length of STRING is greater than zero)

-z STRING (the lengh of STRING is zero)STRING1 = STRING2STRING1 != STRING2

INTEGER1 -eq INTEGER2INTEGER1 -gt INTEGER2INTEGER1 -lt INTEGER2

-d FILE (FILE exists and is a directory)-e FILE (FILE exists)

-r FILE (FILE exists and is readable)-s FILE (FILE exists and is not empty)-w FILE (FILE exists and is writable)

-x FILE (FILE exists and is executable)

#!/bin/bashif [ -r $1 ] && [ -s $1 ] thenecho This file is useful.

fi

#!/bin/bashif [ $USER == ‘bob' ] || [ $USER == ‘andy' ]thenecho User can be trusted.

elseecho User can not be trusted.

fi

another example

#!/bin/bashcase $1 instart)echo starting;;stop)echo stoping;;restart)echo restarting;;*)echo don\'t know;;esac

basic “case statement”

#!/bin/bashcounter=1while [ $counter -le 10 ]do

echo $counter((counter++))

doneecho all done

basic “while loop”

#!/bin/bashcounter=1until [ $counter -eq 10 ]do

echo $counter((counter++))

doneecho all done

basic “until loop”

#!/bin/bash

until [ git pull &> /dev/null ]do echo Trying git server ... sleep 1done

echo Repository is pulled.

another example

#!/bin/bashnames='Stan Kyle Cartman'for name in $namesdo

echo $namedoneecho all done

basic “for loop”

#!/bin/bashfor value in {1..5}do

echo $valuedoneecho all done

“for loop” with range

#!/bin/bashfor file in $1/*.htmldo

cp $file $1/$( basename -s .html $file ).php

done

example: copy html files to php files with same name

#!/bin/bashfor file in $1/*do

if [ ! -r $file ]thenecho $file not readable 1>&2continue

ficp $file $1/backup/

done

example: backing up a set of readable files

#!/bin/bashfor file in $1/*do

used=$( df $1 | tail -1 | awk '{print $5}' | sed 's/%//' )if [ $used -gt 90 ]then

echo Low disk space 1>&2break

ficp $file $1/backup/

done

example: backing up a set of files if space is scarce

#!/bin/bashprint_something () {

echo Hello I am a function}print_somethingprint_something

functions

#!/bin/bashprint_something () {

echo Hello $1}print_something Marsprint_something Jupiter

passing arguments to functions

#!/bin/bashprint_something () {

echo Hello $1return 5

}print_something Marsprint_something Jupiterecho The return value is $?

returning values from functions