1. BASH Scripting aanndd RREEGGEEXX bbaassiiccssFFrreeddLLUUGGHHooww ttoo ddeemmyyssttiiffyy BBAASSHH aanndd RREEGGEEXXPeter LarsenSeptember 27, 2014 Sr. Solutions Architect, Red Hat

2. AAggeennddaa Introduction to BASH Basics with exercises A little more advanced Introduction to REGEX Understanding the Shellshock bugSep 27, 2014 2 3. BBAASSHH IInnttrroodduuccttiioonn Shell what is it and who cares? Current directory, umask Exit codes Functions Built in commands Environment Variables Traps, Options and Aliases sh, ksh, csh and the rest of the family How does it work? Command Expansion Command ExecutionSep 27, 2014 3 4. CCoommmmaanndd EExxppaannssiioonn Variable assignments and redirections are saved for laterprocessing Words are expanded. In the result, the first word is takento be the command and the rest are arguments Redirections are executed Text after = are expanded/substitudedSep 27, 2014 4 5. CCoommmmaanndd EExxeeccuuttiioonn If no / in command shell searches for command If function, calls function If shell built-in, execute Search $PATH To execute Create subshell Run command in subshell If not found, return exit code 127Sep 27, 2014 5 6. BBAASSHH -- BBaassiiccss Hello World Variables Passing arguments to a script Arrays Basic Operations Basic String Operations Decision Making Loops Shell FunctionsSep 27, 2014 6 7. HHeelllloo WWoorrlldd Anything after # is ignored Blank lines are ignored Start all bash scripts with #!/bin/bash To execute a bash script, set execution bit or run:bash[scriptname] Always provide path to script or place it in $HOME/binSep 27, 2014 7 8. HHeelllloo WWoorrlldd --EExxaammppllee Create a script that writes Hello, World! on the screen The print command in bash is called echo it's aninternal bash command.#!/bin/bashecho "Hello, World!"Sep 27, 2014 8 9. VVaarriiaabblleess Variables are created when assigned Syntax: VAR=VALUE Note: No spaces before/after the = Case sensitive one word, can contain _ but not other special characters Read variables by adding $ in front of it or use ${var} Useto escape special characters like $ Preserve white-spaces with Assign results of commands to variables with ` (back-ticks) or $(command)Sep 27, 2014 9 10. VVaarriiaabblleess EExxaammpplleessPRICE_PER_APPLE=5MyFirstLetters=ABCgreeting='Hello world!'PRICE_PER_APPLE=5echo "The price of an Apple today is: $HK $PRICE_PER_APPLE"MyFirstLetters=ABCecho "The first 10 letters in the alphabet are: ${MyFirstLetters}DEFGHIJ"greeting='Hello world!'echo $greeting now with spaces: "$greeting"FILELIST=`ls`FileWithTimeStamp=/tmp/my-dir/file_$(/bin/date +%Y-%m-%d).txtSep 27, 2014 10 11. VVaarriiaabblleess -- EExxeerrcciissee#!/bin/bash# Change this codeBIRTHDATE=NonePresents=NoneBIRTHDAY=None# Testing code - do not change itif [ "$BIRTHDATE" == "Aug 11 1967" ] ; then Create 3 variables in the sample code: String (BIRTHDATE) contain the text 11 1967echo BIRTHDATE is correct, it is $BIRTHDATE Integer (PRESENTS) else contain the number 10echo "BIRTHDATE is incorrect - please retry" Complex (BIRTHDAY) fi contain the weekday of $BIRTHDATE(Friday)if [ $Presents == 10 ] ; thenecho I have received $Presents presentselse Hint: use the 'date' command to get the weekday from adatedate -d $date1 +%Aecho "Presents is incorrect - please retry"fiif [ "$BIRTHDAY" == "Friday" ] ; thenecho I was born on a $BIRTHDAYelseecho "BIRTHDAY is incorrect - please retry"fiSep 27, 2014 11 12. VVaarriiaabblleess -- SSoolluuttiioonn#!/bin/bash# Change this codeBIRTHDATE=" BIRTHDATE="Aug 11 1967"Sep 27 2014"Presents=10BIRTHDAY=$(Presents=date -d "$BIRTHDATE" +%A)# Testing code - do 10not change itif [ "$BIRTHDATE" == "Sep 27 2014" ] ; then echo BIRTHDAY=$(BIRTHDATE is correct, date it -d is "$$BIRTHDATE" BIRTHDATE+%A)elseecho "BIRTHDATE is incorrect - please retry"fiif [ $Presents == 10 ] ; thenecho I have received $Presents presentselseecho "Presents is incorrect - please retry"fiif [ "$BIRTHDAY" == "Friday" ] ; thenecho I was born on a $BIRTHDAYelseecho "BIRTHDAY is incorrect - please retry"Sep 27, 2014 12fi[fredlug@fredlug class]$ bash ./var-solution.shBIRTHDATE is correct, it is Aug 11 1967I have received 10 presentsI was born on a Friday 13. Passing aarrgguummeennttss ttoo aa ssccrriipptt Arguments are passed to a script when it's run Arguments are given after the command line with spaces between them Refer to arguments inside the script with: $1 first argument $2 second argument Etc. $0 is the script name $# number of arguments $@ all parameters space delimitedSep 27, 2014 13 14. AArrgguummeennttss -- EExxaammpplleess ./my_shopping.sh apple 5 banana 8 Fruit Basket 15 $ echo $3 banana $ echo A $5 costs just $6 A Fruit Basket costs just 15 $ echo $# 6Sep 27, 2014 14 15. AArrrraayyss Several values in the same variable name Created with space separated values in ( ) Total array values: ${#arrayname[@]} Use ${array[index]} to refer to values Note index numbers start at 0 (not 1).Sep 27, 2014 15 16. AArrrraayy -- EExxaammpplleess my_array=( apple banana Fruit Basket orange ) new_array[2]=apricot $ echo ${#my_array[@]} 4 $ echo ${my_array[3]} orange $ my_array[4]=carrot $ echo ${#my_array[@]} 5 $ echo ${my_array[${#my_array[@]}-1]} carrotSep 27, 2014 16 17. AArrrraayy -- EExxeerrcciissee Create a bash script Define array NAMES with 3 entries: John, Eric and Jessica#!/bin/bashNAMES=( John Eric Jessica ) Define array NUMBERS with 3 entries: 1, 2, 3 Define variable NumberOfNames containing the number ofnames in the NAMES array using $# special variable# write your code hereNUMBERS=(1 2 3)NumberOfNames=${#NAMES[@]}second_name=${NAMES[1]}echo NumberofNames is: $NumberOfNamesecho second_name is: $second_name Define variable second_name that contains the second name inthe NAMES array Print the content of NumberOfNames and second_name[fredlug@fredlug class]$ bash ./array.shNumberofNames is: 3second_name is: EricSep 27, 2014 17 18. BBaassiicc OOppeerraattiioonnss Use $((expression)) Addition: a + b Subtraction: a b Multiplication: a * b Division: a / b Modulo: a % b (integer remainder of a divided with b) Exponentitation: a ** b (a to the power of b)Sep 27, 2014 18 19. BBaassiicc OOppeerraattiioonnss -- EExxeerrcciissee Given COST_PINEAPPLE=50 COST_BANANA=4 COST_WATERMELON=23 COST_BASKET=1#!/bin/bashCOST_PINEAPPLE=50COST_BANANA=4COST_WATERMELON=23COST_BASKET=1TOTAL=$(( $COST_BASKET + ( $COST_PINEAPPLE * 1 ) + ( $COST_BANANA * 2 ) + ( $COST_WATERMELON * 3 ) ))echo Total is: $TOTAL Calculate TOTAL of a fruit basket containing 1 pinapple, 2bananas and 3 watermelons Print the content of TOTAL$ bash ./operations.shTotal is: 128Sep 27, 2014 19 20. BBaassiicc SSttrriinngg OOppeerraattiioonnss STRING=this is a string String length: ${#STRING} 16 Numerical position of character:expr index $STRING a 9 Substring: ${STRING:$POS:$LEN) POS=1, LEN=3 his ${STRING:12} ring # from pos and to the end of var Substring replacement: ${STRING[@]/string/text} this is a text Substring replace ALL: ${STRING[@]//is/xx}thxx xx a string Delete all occurrences: ${STRING[@]// a /}this is string Replace first occurrence: ${STRING[@]/#this/that/} Replace last occurrence: ${STRING[@]/%string/text}Sep 27, 2014 20 21. SSttrriinnggss -- EExxeerrcciissee#!/bin/bashBUFFET="Life is like a snowball. The important thing is finding wet snow and a really long hill." Given BUFFET="Life is like a snowball. The important thing isfinding wet snow and a really long hill." Create ISAY variable with the following changes:ISAY="$BUFFET"ISAY=${ISAY[@]/snow/foot}echo First: $ISAYISAY=${ISAY[@]/snow/}echo Second: $ISAY First occurence of 'snow' with 'foot'$ bash ./string2.shFirst: Life Delete is like a football. The important Second: Life is like second a football. occurence The important of snowthing is finding wet snow and a really long hill.thing is finding wet and a really long hill.Third: Life is like a football. The important thing is getting wet and a really long hill.Fourth: Life is like a football. The important thing is getting wet Replace 'finding' with 'getting' Delete all characters following 'wet' Print ISAYISAY=${ISAY[@]/finding/getting}echo Third: $ISAYPOS=`expr index "$ISAY" "w"`ISAY=${ISAY:0:POS+3}echo Fourth: $ISAYSep 27, 2014 21 22. DDeecciissiioonn MMaakkiinngg If [ expression ]; thencode the true partelsecode the false partfi Else can be replace with elif if followed by another if Case $variable incondition1)command ...;;condition2)command ...;;esacmycase=1case $mycase in1) echo "You selected bash";;2) echo "You selected perl";;3) echo "You selected python";;4) echo "You selected c++";;5) exitesacSep 27, 2014 22 23. EExxpprreessssiioonnss Can be combined with ! (not), && (and) and || (or) Conditional expressions should use [[ ]] (double) Nummeric Comparisons $a -lt $b $a < $b $a -gt $b $a > $b $a -le $b $a = $b $a -eq $b $a == $b $a -ne $b $a != $b String Comparisons $a = $b or $a == $b $a != $b -z $a a is emptySep 27, 2014 23 24. DDeecciissiioonn mmaakkiinngg -- EExxeerrcciissee Change variables to make expressions true#!/bin/bash# change these variablesNUMBER=10APPLES=12KING=GEORGE# modify above variables# to make all decisions below TRUEif [ $NUMBER -gt 15 ] ; thenecho 1fiif [ $NUMBER -eq $APPLES ] ; thenecho 2fiif [[ ($APPLES -eq 12) || ($KING = "LUIS") ]] ; thenecho 3fiif [[ $(($NUMBER + $APPLES)) -le 32 ]] ; thenSep 27, 2014 24echo 4fiNUMBER=16APPLES=16KING=LUIS 25. LLooooppss for loopfor arg in [list]docommand(s) ....done while loopwhile [ condition ]docommand(s) ...done until loopuntil [ condition ]docommand(s) ...done break - skip iteration continue - do next loopnowSep 27, 2014 25 26. LLoooopp EExxaammpplleess# Prints out 0,1,2,3,4COUNT=0while [ $COUNT -ge 0 ]; do# loop on array memberNAMES=(Joe Jenny Sara Tony)for N in ${NAMES[@]} ; doecho Value of COUNT is: $COUNTCOUNT=$((COUNT+1))if [ $COUNT -ge 5 ] ; thenecho My name is $Ndonebreak# loop on command fioutput resultsfor f in $( ls *.sh /etc/localtime ) ; dodone# Prints out only odd numbers - 1,3,5,7,9COUNT=0while [ $COUNT -lt 10 ]; doecho "File is: $f"doneCOUNT=4while [ $COUNT -gt 0 ]; doecho Value of count is: $COUNTCOUNT=$(($COUNT - 1))doneCOUNT=1until [ $COUNT -gt 5 ]; doecho Value of count is: $COUNTCOUNT=$(($COUNT + 1))doneCOUNT=$((COUNT+1))# Check if COUNT is evenif [ $(($COUNT % 2)) = 0 ] ; thencontinuefiecho $COUNTSep 27, 2014 26done 27. LLoooopp EExxeerrcciissee#!/bin/bashNUMBERS=(951 402 984 651 360 69 408 319 601 485 980 507 725 547 544 615 83 165 141 501 263)for num in ${NUMBERS[@]}do NUMBERS=(951 402 984 651 360 69 408 319 601 485980 507 725 547 544 615 83 165 141 501 263) Print all even numbers in order of array Do not print anything after 547if [ $num == 547 ]; thenbreakfiMOD=$(( $num % 2 ))if [ $MOD == 0 ]; thenecho $numfidone$ bash ./loops.sh402984360408980Sep 27, 2014 27 28. SShheellll FFuunnccttiioonnssfunction function_B { Sub-routine that implements echo Function set B.of commands andoperations.}function function_A {echo $1 Can take parameters}function adder { Useful for repeated tasks}# FUNCTION CALLS# Pass parameter to function Afunction_A "Function A." # Function A.function_B # Function B.# Pass two parameters to function adderadder 12 56 # 68 function_name {command ....}echo $(($1 + $2))Sep 27, 2014 28 29. FFuunnccttiioonnss -- EExxeerrssiizzee#!/bin/bashfunction ENGLISH_CALC { Write a function ENGLISH_CALC which process thefollowing:NUM1=$1 ; OPTXT=$2 ; NUM2=$3case $OPTXT inplus) OP='+' ;;minus) OP='-' ;;times) OP='*' ;;*)echo Bad operator $OPTXT;;esacecho $NUM1 "$OP" $NUM2 = $(($NUM1 $OP $NUM2)) ENGLISH_CALC 3 plus 5 ENGLISH_CALC 5 minus 1 ENGLISH_CALC 4 times 6 The function prints the results as 3 + 5 = 8, 5 1 = 4 etc.}ENGLISH_CALC 3 plus 5ENGLISH_CALC 5 minus 1ENGLISH_CALC 4 times 6Sep 27, 2014 29 30. BBAASSHH AAddvvaanncceedd Special Variables Bash trap command File testing There's a lot more features this is not comprehensive $man bash is your friendSep 27, 2014 30 31. SSppeecciiaall VVaarriiaabblleess* $* = $1 $2 $3 ...... $ Process ID of shell@ $@ $1 $2 $3 ..... ! Process ID of most recentbackground process# Number of parameters 0 Name of shell or program beingexecuted? Exit status _ Aboslute path of shell or command- Current option flags (shopt)Sep 27, 2014 31 32. BBaasshh ttrraapp ccoommmmaanndd trap executes a script automatically when a signal isreceived $ trap program sigspec List all signals with trap -l Great for catching a HUP or INT to clean up temporaryfiles etc before exitingSep 27, 2014 32 33. FFiillee tteessttiinngg Used as a condition to set actions based on file attributes Exists, readable, writable etc. File1 older/newer than File2 Commonly used in if statements [ ] [[ ]] etc.Sep 27, 2014 33 34. FFiillee TTeessttiinngg ooppttiioonnss -f regular file exists -d directory exists -h symbolic link exists -r file is readable -w file is writable file1 -nt file2: file1 newerthan file2 file1 -ot file3: file1 olderthan file2 file1 -ef file2: file1 and file2refers to same inodeSep 27, 2014 34 35. RReegguullaarr EExxpprreessssiioonnss Characters/Strings Character Classes and Bracket Expressions Anchoring Backslash and special expressions Repetition Concatenation, Alternation, PrecedenceSep 27, 2014 35 36. DDeemmoo ffiillee Create a file resolv.conf with the following contents Make sure grep is aliased to:grep color=auto; generated by /sbin/dhclient-script ^$[](){}-?*.+:_search brq.com mylab.brq.com lab.eng.brq.com world.comnameserver 12.14.255.7nameserver 14.14.255.6 37. CChhaarraacctteerrss//SSttrriinnggss Simple Character strings are matched as you wouldexpectSep 27, 2014 37 38. Character Classes aanndd BBrraacckkeett EExxpprreessssiioonnss [ ] is a set of characters that matches. A string matches if itmatches any of the characters in the set. A ^ inside the [ ]means do not patch Predefined sets like [[:alnum:]] [[:digit:]] exists to makewriting easier Decimal point (.) matches any single characterSep 27, 2014 38 39. EExxaammpplleessSep 27, 2014 39 40. AAnncchhoorriinngg Locks the search pattern to a specific position ^ beginning of line $ end of lineSep 27, 2014 40 41. Backslash aanndd ssppeecciiaall eexxpprreessssiioonnss Backslashes can prefix specialfunctions < = Start of word > = End of word b = beginning of word B = not b w = word W = not wordSep 27, 2014 41 42. RReeppeettiittiioonn * repeats 0 or more times + repeats 1 or more times ? repeats 0 or 1 time {5} repeats 5 times {2,3} repeats 2 or 3 timesSep 27, 2014 42 43. Concatenation, AAlltteerrnnaattiioonn,, PPrreecceeddeennccee Concatenation: sequence of characters (literal/special) Alternation: Separate different patterns with | Precedence: Parentheses, Repetition, Concatenation,Alternation Use ( ) to group things together for later referenceSep 27, 2014 43 44. Al Concatenation/Altteerraattiioonn eexxaammpplleeSep 27, 2014 44 45. #shellshock tthhee ffaammoouuss BBAASSHH bbuugg So what is it? CVE-2014-6271 Try this at your command prompt:x='() { :;}; echo vulnerable' bash -c "echo test" Does it print vulnerable? If so, you need to update yourBASH right away. $ rpm -q bashShould report version 4.2.45-5.4 if not yum update now.Sep 27, 2014 45 46. ##sshheellllsshhoocckk -- hhooww A little known hack allows functions to be treated asvariables $ function foo { echo "hi mom"; }$ export -f foo$ bash -c 'foo' # Spawn nested shell, call 'foo'hi mom Great Blog:http://lcamtuf.blogspot.com/2014/09/quick-notes-about-bash-bug-Sep 27, 2014 46 47. ##sshheellllsshhoocckk hhooww ccoonnttiinnuueedd $ foo='() { echo "hi mom"; }' bash -c 'foo'hi mom Let's break it downfoo='() { echo hi mom;}'magic property () {is executed before the command is runexecute bash running foo Since env variables are used by httpd, dhcpd and other daemons, itpotentially allows them to run code by simply setting a value in avariable.Sep 27, 2014 47 48. TThhaannkkss ffoorr ccoommiinnggQQuueessttiioonnss??Sep 27, 2014 48