BI296: Linux and Shell Programming Lecture 04: Bash …cbb.sjtu.edu.cn/course/bi296/lecture/lec04.pdf · BI296: Linux and Shell Programming Lecture 04: Bash Scripting Maoying,Wu

BI296: Linux and Shell Programming

Lecture 04: Bash Scripting

Maoying,[email protected]

Dept. of Bioinformatics & BiostatisticsShanghai Jiao Tong University

Spring, 2018

Maoying Wu (CBB) BI296-Lec04 Spring, 2018 1 / 48

Lecture Outline

Bash Programming (BASH脚本编程)Bash: Introuction（BASH发展历史与相关概念）Types of Variables (变量声明与定义)Conditional Statements and Flow Control（条件结构和流程控制）Command Line: Arguments (命令行)BASH Functions (函数的定义)Built-in Variables and Functions (内置变量与函数)

Applications of BASH（BASH应用案例）


The first BASH scripts

greeting.sh

#!/bin/bash# This is the first bash implementation.# Scriptname: greetings.sh# Written by: Ricky Woo ([email protected])echo -e "What’s your name and your age: "read name ageecho -e "I’m $name, and I’m $age years old."read

Run the script file

# grant the executable permissionchmod u+x greeting.sh# run the script./greeting.shbash greeting.sh


Interpretation of BASH scripts

greeting.sh

#!/bin/bash# This is the first bash implementation.# Scriptname: greetings.sh# Written by: Ricky Woo ([email protected])echo -e "What’s your name and your age: "read name ageecho -e "I’m $name, and I’m $age years old."read

Interpretation#!/bin/bash: Shebang to indicate the command to interpret thefollowing scripts.#: Comments (注释).$name: Variable substitution (变量替换).read: Obtain the variable value interactively (交互式变量赋值).


Shells: Types and Descriptions

The default shell for each user is defined in /etc/passwd.

Described in the file /etc/shells.sh: Bourne Shell, light-weighted shell (UNIX)ksh: Korn shell, superset of shcsh: Berkeley UNIX C-shelltcsh: Enhanced csh.bash: Bourne Again SHell，sh+csh


BASH Advantages and Disadvantages

Bourne Again SHellIn memory of Stephen Bournede facto standard for shell scriptingsh-compatibleDerive some useful features from ksh as well as csh

When not to useResource-intensive tasks (sorting, hashing, recursion, ...)Heavy-duty math operations (floating-point arithmetic)Precision calculations (use c++/fortran instead)Cross-platform portability required (c/java instead)


BASH Advantages and Disadvantages

Bourne Again SHellIn memory of Stephen Bournede facto standard for shell scriptingsh-compatibleDerive some useful features from ksh as well as csh

When not to useResource-intensive tasks (sorting, hashing, recursion, ...)Heavy-duty math operations (floating-point arithmetic)Precision calculations (use c++/fortran instead)Cross-platform portability required (c/java instead)


Variables: Declaration (变量声明)

declare: syntax (语法)declare [-aAfFgilrtux] [-p] [name[=value] ...]

-a/-A: indexed/associative array (索引/关联数组);

-f/-F: function body/name (函数实体/函数名);

-i: integer (整数)

-l/-u: lowercase/uppercase (小写/大写字母)

-r: read-only (只读);

-g: global (全局变量)

declare: examples (示例)

# ‘i‘: integerdeclare -i int# ‘a‘: arrarydeclare -a arr# ‘A‘: associative arraydeclare -A Arr# ‘r‘: read-onlydeclare -r ro# ‘f‘: functiondeclare -f func


Variables: Definitions (变量定义)

name=value

Variable Names (变量命名)name=/^[A-Za-z_][A-Za-z0-9_]*$/

Use interpretable variable names.

Variable Assignment (变量赋值)Space is forbidden besides = operator.Note: Use double quotes to enclose the value.

Examples (变量定义的例子)

# ‘i‘ is an integer, 2declare -i i="8/3"echo ${i}


Built-in Variables (内置变量)

Variables Description

PATH the directories of executable commands (可执行命令路径).

MANPATH the directories of manuals (手册目录).

FS the field separator, default ” ”.

PS1, PS2, PS3 the prompt (提示符).

PWD, OLDPWD the current/previous working directory (工作路径).

SHELL the default shell for current user (当前shell).

USER, $USERNAME, $LOGNAME the current user name (当前用户).

HOSTNAME the MACHINE name (主机名).

HOME the home directory for the current user (用户家目录).

$? the return code of the last command issued (退出码).


Variable Manipulation

FILEPATH=/path/to/my/output.lisecho $FILEPATHecho ${FILEPATH%t*} # non-greedy tail-truncationecho ${FILEPATH%%t*} # greedy tail-truncationecho ${FILEPATH#*/} # non-greedy head-truncationecho ${FILEPATH##*/} # greedy head-truncationunset a; b=${a-5}; echo $b; unset bunset a; b=${a=5}; echo $b; unset bunset a; b=${a+5}; echo $b; unset ba=3; b=${a-5}; echo $b; unset ba=3; b=${a=5}; echo $b; unset ba=3; b=${a+5}; echo $b; unset becho ${t:?undefined}; t=5; echo ${t:?undefined}dna="ACCTAGGACG"; echo ${dna:3:4} # substringdna2=‘echo $dna | rev‘; echo ${dna2} # reverse


Exercise: Variables

1 Which of the following variable names are not legal? If not, tellwhy.

3x5, x=1, x+, wt-5, _xyz, #tt, CMT, echo, if, $test

2 Output the results.

#!/bin/bashnum1=3;num2=5sum=$num1+$num2; echo ${sum} # sum?declare -i sum=$num1+$num2; echo ${sum} # sum?diff=$num2-$num1; echo ${diff} # diff?declare -i diff=$num2-$num1; echo ${diff} # diff?prod=$num1*$num2; echo ${prod} # product?declare -i prod=$num1*$num2; echo ${prod} # product?div=$num2/$num1; echo ${div} # division?declare -i div=$num2/$num1; echo ${div} # division?


Next we will talk about ...

1 bash: an introduction

2 flow controlconditional statementsif statementloop statement

3 mathematical computinginteger computationfloat computation

4 command linepositional parameterscommand-line options

5 functions


Testing Files (文件的检验)

Expression Description[ -a FILE ] TRUE if FILE exists (存在性).[ -b FILE ] TRUE if FILE is a block device file (块设备).[ -c FILE ] TRUE if FILE is character device file (字符设备).[ -d FILE ] TRUE if FILE is a directory (目录).[ -e FILE ] TRUE if FILE exists (存在).[ -f FILE ] TRUE if FILE is a regular file (普通文件).[ -g FILE ] TRUE if FILE SGID bit is set (SGID).[ -h FILE ] TRUE if FILE is a symbolic link (符号链接).[ -k FILE ] TRUE if FILE sticky bit is set (粘附位).[ -p FILE ] TRUE if FILE is a named pipe (FIFO,管道文件).[ -r FILE ] TRUE if FILE is readable (可读).[ -s FILE ] TRUE if FILE size is greater than 0 (非空).[ -t FD ] TRUE if FILE DESCRIPTOR FD refers to a terminal (终端).[ -u FILE ] TRUE if FILE SUID bit is set (SUID).[ -w FILE ] TRUE if FILE is writable (可写).[ -x FILE ] TRUE if FILE is executable (可执行).[ FILE1 -nt FILE2 ] TRUE if FILE1 is newer than FILE2 (新).[ FILE1 -ot FILE2 ] TRUE if FILE1 is older than FILE2 (旧).[ FILE1 -ef FILE2 ] TRUE if FILE1 and FILE2 refers to the same (同).


Test of strings and numbers (字符串和数值的检验)

Expression Description[ -z STRING ] TRUE if STRING is of zero-length (空串).[ -n STRING ] TRUE if STRING is non-null (非空).[ STRING ] TRUE if STRING is of non-zero length (非空).[ STRING1 == STRING2 ] TRUE if two strings are equal (相等).[ STRING1 != STRING2 ] TRUE if two strings are not equal (不等).[ STRING1 \< STRING2 ] TRUE if STRING1 is lexically less than STRING2.[ STRING1 \> STRING2 ] TRUE if STRING1 is lexically greater than STRING2.[ ARG1 op ARG2 ] Arithmetic binary comparison (数值比较).

op=(-eq | -ne | -lt | -le | -ge | -gt)

[ ! EXPR ] TRUE if EXPR is FALSE (逻辑反).[ (EXPR)] Returns the value of EXPR.[ EXPR1 -a EXPR2 ] Logical AND (逻辑与).[ EXPR1 -o EXPR2 ] Logical OR (逻辑或).







5 functions


Conditional Statement: if

Syntax

if [ cond_statement1 ]; thendo_something

elif [ cond_statement2 ]; thendo_other_thing

elsedo_else_thing

fi

Example

#!/bin/bashread -p "Please input a score: " scoreif [ $score -gt 90 ]; thenecho "You got A"

elif [ $score -gt 75 ]; thenecho "You got B"

elif [ $score -gt 60 ]; thenecho "You got C"

elseecho "You failed"

fi


Three Conditional Expressions (条件表达式)

[ EXPR ] [[ EXPR ]] test EXPR

Word splitting Yes No Yes

Pathname expansion Yes No Yes

Pattern globs No Yes No

work splitting: using double quotes

var="split word";[ $var == "split word" ]; echo $? # ERROR[ "$var" == "split word" ]; echo $? # WORK[[ $var == "split word" ]]; echo $? # WORK[[ "$var" == "split word" ]]; echo $? # WORKtest $var == "split word"; echo $? # ERRORtest "$var" == "split word"; echo $? # WORK

pathname expansion (路径扩展)

[ "./lec4.tex" -ef ./lec4.te* ]; echo $?[[ "./lec4.tex" -ef ./lec4.te* ]]; echo $?test "./lec4.tex" -ef ./lec4.te*; echo $?

Pattern globbing (模式匹配)

[ "expression" == expr* ]; echo $?[[ "expression" == expr* ]]; echo $?test "expression" == expr*; echo $?


The Conditional Statements (条件语句)

if..then..elif..then..else..fi

if test "$(whoami)" == "root"; thenecho "You are using a privileged account"exit 1

fi

Equivalent && and ||

test "$(whoami)" != "root" && (echo "you are using a non-privilegedaccount"; exit 1)

test "$(whoami)" == "root" || echo "Please verify that you have thesuper-user privilege."


case...in...esac construct

Syntax

case var inval1)statement1

;;;val2)statement2

;;;

*)statement3

;;;esac

Examples

#!/bin/bashecho "Enter your favorite color: "read colorcase $color inr*|R*)echo "Your favorite color is red."

;;;b*|B*)echo "Your favorite color is blue."

;;;g*|G*)echo "Your favorite color is green."

;;;

*)echo "The color you entered is invalid."

;;;esac


select: creating menu (选择菜单)

#!/bin/bashPS3="Choose your favorite dish (q to exit): "select dish in "Roast Duck" "Mapo Tofu" "Sweet and Sour Ribs"docase $dish inR*)echo "You may be from Beijing."

;;M*)echo "You may be from Sichuan."

;;S*)echo "You may be from Shanghai."

;;q)break

;;

*)echo "Let me guess."

;;esacbreak

done







5 functions


while loop

Syntax

while conditiondodo_something

done

Example

#!/bin/bashIFS=":"while read f1 f2 f3 f4 f5 f6 f7doif [ "$f7" == "/sbin/nologin" ]; thenecho "$f1 cannot login to the system."

fidone < /etc/passwd


General for loop

Syntax

for var in val-listdodo_something

done

Examples

#!/bin/bash# for1.shdeclare -i jfor i in ‘seq 20‘doj=$i%5if [ $j -eq 0 ]; thencontinue

fiecho $i

done


C-style for loop

Syntax

for ((start-condition;end-condition;loop-action))dodo_somethingdone

Examples

#!/bin/bashdeclare -i jfor ((i=1;i<=20;i++))doj=$i%5if [ $j -eq 0 ]; thencontinue

fiecho $i

done


Exercise

1 Guess what the following scripts do.

#!/bin/bash# space.sh: A very simple test for checking disk space.space=‘df -h | awk ’{print $5}’ | grep % | grep -v Use| sort -n | tail -1 | cut -d "%" -f1‘alertvalue="80"if [ "$space" -ge "$alertvalue" ]; thenecho "At least one disk is nearly full!"

elseecho "Disk space normal"

fi







5 functions


Four integer computing (整数)

1 declare -i2 (())3 expr4 let

Example

# declaredeclare -i intint=13%5echo $int# (())x=3((x++))echo $x# exprexpr 3 * 5# letlet "x = x * 3"echo $x







5 functions


floating computing (浮点数)

The command bc does help.

example

echo "3.3*4" | bc # 13.2echo "2.1ˆ4" | bc # 19.4echo "scale=2; (2.1ˆ2)ˆ2" | bc # 19.44echo "scale=1; (2.1ˆ2)ˆ2" | bc # 19.3echo "scale=2; 1.5*100/2" | bc # 75.00# computing with functionecho "sqrt(2)" | bcecho "sqrt(2)" | bc -lecho "a(1)" | bc -lecho "scale=3; 4*a(1)" | bc


expr in string operation (字符串)

# return the string lengthexpr length "DNA transcription"# return the substringexpr substr "DNA transcription" 5 13# return the indexexpr index "DNA transcription" ’c’# regex matchexpr account.doc : docexpr account.doc : acc# regex matchexpr 1234bcdf : ".*"# capture by regexexpr abcdefgh : ’...$...$..’# capture by regexexpr account.doc : ’$.*$.doc’







5 functions


Positional Parameters (位置参数)

$* treats all positional parameters as a whole string.$@ treats all positional parameters as an array.$0 is the script file itself.$1,$2,... is the first, second, ... positional parameters.$#: number of positional parameters.

Examples

#!/bin/bashfor i in {1..$#}; doecho ${!i}

done


Exercise

1 Print all the positional arguments, one per line.2 Compute the product of all the positional arguments.3 If your arguments contain some float numbers, what to do then?







5 functions


options (选项)

./test.sh -a -b -c

short options without argument (无参数短选项).

./test.sh -abc

short options as above (同上).

./test.sh -a arg -b -c

short options, where -a arg needs argument, while -b and -cdo not need argument (有/无参数短选项).

./test.sh --a-long=arg --b-long

long options where --a-long=arg requires argument, while--b-long does not require argument (有/无参数长选项).


getopts

getopts does not support long options.

Example

#!/bin/bash# testoptions.sh# ./testoptions.sh -a something -bc other-stuffwhile getopts "a:bc" arg # colon indicates ‘a’ requires argumentdocase $arg ina)echo "a’s arg: $OPTARG" ;;

b)echo "b" ;;

c)echo "c" ;;

?)echo "unknown argument"; exit 1 ;;

esacdoneecho "The argument is at $OPTIND"


getopt: supports long options

#!/bin/bash# testlongopts.sh# ./testlongopts.sh -a arg1 ’arg2’ --c-long ’wow!*\?’ -cmore -b " very

long "TEMP=‘getopt -o ab:c:: --long a-long,b-long:,c-long:: \

-n ’testlongopts.sh’ -- "$@"‘if [ $? != 0 ]; then echo "Terminating..." >&2 ; exit 1 ; fi# ‘set’ will reorder the parametereval set -- "$TEMP"while true ; docase "$1" in-a|--a-long) echo "Option a"; shift ;;-b|--b-long) echo "Option b, argument \‘$2’" ; shift 2 ;;-c|--c-long)case "$2" in"") echo "Option c, no argument"; shift 2 ;;

*) echo "Option c, argument \‘$2’" ; shift 2 ;;esac ;;

--) shift ; break ;;

*) echo "Internal error!" ; exit 1 ;;esac

doneecho "Remaining arguments: "for arg do echo ’--> ’"\‘$arg’" ; done


Exercise

1 As a simple exercise, write a shell script that prints the file type foreach file passed as an argument to the script. Here is an example:

$ ./ftypes.sh ftypes.sh .emacs .bashrc public_htmlftypes.sh: Bourne-Again shell script text executable.emacs: Lisp/Scheme program text.bashrc: ASCII English text, with very long linespublic_html: symbolic link to ‘www’


Functions: Definition

Function defintion (函数定义)

# function func_name() { commands; }function dir() {target=${1:-.}if [ -e $target ]; thenecho "List Directories in $target: ";ls -l $target | awk ’/ˆd/{print $NF}’;

elseecho "$target not exists."return -1

fi}

Function call (函数调用)

#!/bin/bashread -p "Input the directory name: " dirnamedir $dirname


Variable Scopes (变量作用域)

Local Variables (局部变量): inside a functionGlobal Variables (全局变量): declare -g name=val

Environmental Variables (环境变量): export name=val

Examples (示例)

#!/bin/bashfunction1() {local func1var=20echo "Within function1, \$func1var = $func1var."function2

}function2 () {echo "Within function2, \$func1var = $func1var."

}function1echo "Outside function, \$func1var = $func1var."exit 0


A little project: prime finder (1)

#!/bin/bash# SCRIPT: primefactors.sh# USAGE: primefactors.sh <Positive Integer># PURPOSE: Produces prime factors of a given number##################################################### Argument checking####################################################if [ $# -ne 1 ]; thenecho "Usage: $0 <Positive Integer>"exit 1

fiexpr $1 + 1 &>/dev/nullif [ $? -ne 0 ]; thenecho "Sorry, you supplied a non-numerical value."exit 1

fi[ $1 -lt 2 ] && echo "Value < 2 are not prime numbers" && exit 1num=$1



######################################### Functions######################################### script to find prime numberprimenumber() {primenum=$1for ((cnt2=2;$((cnt2*cnt2))<=$primenum; cnt2++)); do[ $((primenum%cnt2)) -eq 0 ] && return 1

donereturn 0

}primefind() {primenumber $1 && echo "$1" && exit 0for ((cnt1=$2;cnt1<=$1;$cnt1++)); doprimenumber $cnt1 && factorcheck $1 $cnt1 && break

done}



factorcheck(){prime=$2newnum=$1remainder=$((newnum%prime))

if [ $remainder -eq 0 ]; thenprintf "%dx" $primenewnum=$((newnum/prime))primefind $newnum 2return

elselet prime++primefind $newnum $prime

fi}



################################################ main###############################################echo -n "Prime Factor of $1: "primefind $num 2print "\b\n"


Exercise

Explain why the final output is blank.#!/bin/bashOUTPUT="name1 ip ip status" # normally output of another command with

multi line output

if [ -z "$OUTPUT" ]then

echo "Status WARN: No messages from Hacker"exit $STATE_WARNING

elseecho "$OUTPUT"|while read NAME IP1 IP2 STATUSdo

if [ "$STATUS" != "Optimal" ]then

echo "CRIT: $NAME - $STATUS"echo $((++XCODE))

elseecho "OK: $NAME - $STATUS"

fidone

fi

echo $XCODE


Indexed Arrays (索引数组)

declare -a arrayname

Only 1-dim array is allowedArray Assignment

1 array=(val1 val2 val3 ... valN)2 array=([0]=val0 [1]=val1 ... [N]=valN)3 array[0]=val0; array[1]=val1; ...

Size: ${#array[@]}, ${#array[*]}

echo ${array[2]}

echo ${array[@]:3:2}


Associative Arrays (关联数组)

declare -A Arrayname

Array Definition1 declare -A Array=([unix]=1 [windows]=2 [mac]=3)2 declare -A Array[unix]=1; Array[windows]=2; Array

[mac]=3

Iterate over all keys and values:

for key in "${!Array[@]}"; do echo $key, ${Array[$key]}; done


Summary

Run script file: shebang (#!/bin/bash)Variable declaration (declare), assignment (=) and destroy(unset)Conditional statement ([ ], test, [[ ]])if...then...fi statementfor...do...done statementcase...in...esac statementwhile...do...done statementFunction definition (func(){...})Positional parameters ($0, $1,$*, $@)getopts statement


Documents

BI296: Linux and Shell Programming Lecture 04: Bash …cbb.sjtu.edu.cn/course/bi296/lecture/lec04.pdf · BI296: Linux and Shell Programming Lecture 04: Bash Scripting Maoying,Wu