Upload
joella-lewis
View
212
Download
0
Embed Size (px)
Citation preview
Review• Please turn in your homework and practicals• sed
Today
• Regular Expressions– Again!– Review• grep & sed• Again! (The Revenge!)• awk
The Guy Who Draws XKCD• I think I have a Bash problem. What follows is an actual
command from my history.• cat /usr/share/dict/words | fgrep -v "'" | perl -ne
'chomp($_); @b=split(//,$_); print join("", sort(@b))." ".$_."\n";' | tee lookup.txt | perl -pe 's/^([^ ]+) .*/\1/g' | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | uniq -c | sort -nr | egrep "^[^0-9]+2 " | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | perl -pe 's/[ 0-9]//g' | xargs -i grep {} lookup.txt | perl -pe 's/[^ ]+ //g' | tail -n2
• It’s just so hard to bite the bullet, admit that the problem has grown in scope, and move it to its own Perl/Python script. (P.S. The Guinness Book is wrong. “Conservationalists” is not a real word.)
Quick Searching
• The grep utility is used for matching• Printing to STDOUT – an ‘eyeball’ test or
redirection to another file• grep ‘static string’ file >> redirection• grep ‘metacharacters’ file• And grep starts at the first line, at the beginning
of the row, reads across the row, and as soon as it matches its regex, prints
• Doesn’t modify, only needs to match once
Replacement
• The sed utility is used for ‘substitutions’• Can either execute once (by default), or across
all entries (globally)• Can also print• sed ‘s/from/to/g’ file• sed -n ‘/regex/p’ file
awk is Awkward
• Final regex utility is awk• For use when your data is evenly formatted• Teams.txt last two lines– Seattle Mariners– There might be some guys called the Cubs too– No even formatting
• First few lines– City Teamname– Even formatting
Example - /etc/passwd
• smithj:x:561:561:Joe Smith:/home/smithj:/bin/bash• smith:*:100:100:8A-74(office):/home/smith:/usr/bin/sh• jsmith:x:1001:1000:John Smith:/home/jsmith:/bin/sh
• Between each entry is a :• Username:Password:UID:GID:Name:Homedir:Shell• Empty entries are still “bounded” (ie, :: indicates one
empty value)
awk Examples
• awk -F ‘:’ ‘{ print $1 }’ /etc/passwd• awk - command• -F ‘:’ - -F sets our ‘delimiter’ to the : character• ‘{ print $1 }’ - action• /etc/passwd - file to use• Prints the first column in the /etc/passwd file– Numbering starts at 1; 0 is the whole file– /etc/passwd is ‘delimited’ by : and empty values still
have a : around them
Advanced awk
• I will not quiz you on this• The more powerful awk commands are in
scripts– #!/bin/bash– awk ‘\– BEGIN { print “File\tOwner” } \– { print $0, “\t”, $3} \– END { print “ – DONE –” } \– ‘
Case Study
• I used awk yesterday• Something close to • awk -F ‘,’ ‘{ print $1\tprint $7}’ example.csv
| sort \• | uniq -c | sort -nr• Counting audit findings so that we could see
where to concentrate
Unfortunately
• I had to reimage my laptop over the weekend• This wiped out our vm on my system• I have basic commands, but they’re not
guaranteed to work• There may be one question on awk on the
final
Finally
• I have yet to see my son play T-ball• I will be “sick” a week from today (Wednesday,
June 4th) so I can see his last game
• All late/resubmitted work is due June 9th
• Final is Monday, June 16th during normal class time
• Just under 50 questions, similar to quizzes
Own Study
• Regex’s• Grep, sed and awk