bash_day3

8/13/2019 bash_day3

1/8

Operating on Files (3)

pastemerge lines of files

joinbasic database functionality (join tables)

file1.txt

line1

line2line3

line4

f i l e 2 . t x t

a

bc

d

file3.txt

1

23

4

p as te - d .- f il e ?. t xt

p as te f il e [ 1 2] . t xt > j1 . t xtp as te f il e [ 1 3] . t xt > j2 . t xt

j oi n j ? . t xt

Tobias Neckel: Scripting with Bash and Python

Compact Course @ Max-Planck, October 07 - 18, 2013 53

8/13/2019 bash_day3

2/8

find

Tool to retrieve recursively(!) files and directories depending onsearch criteria

Perform operations on the files found

In connection with other bash tools perform tasks such as

Find all files not older than two days Find all .txt files containing the phrase Hello World! Find all .bak files and remove them

General syntax:find [path...] [expression]

An expression can be

Options, such as --maxdepth n Tests, such as -amin n(last access less than n minutes ago) Actions, such as -delete

If path is not specified, assume ./



8/13/2019 bash_day3

3/8

find (2)

wget h t t p : / / www5.i n. tum . de / l e h r e / vo r l e su n g e n / \progc ourse / bashpython / st o ry . tgzt a r x z f s t o r y . t g z

Examples

# ! / b i n / b a s h

f in d - m ax de pt h 2 - na me " * . t x t "f in d ./ - am in -500 - i na me " * . T X T "

f in d - at im e +1 - na me " * . t x t "

f in d - p a th " * f o l d e r 1 * "

f in d - si ze + 40 0 c - na me " f i l e * . * " - ls

f in d - n a me " * . t x t " - printf " % h , % f \ n "

f in d - n a me " f i l e ? ? . t x t " - exec g rep - H " B i l b o " {} \;

f in d - r e ge x

.*[^0-9][0-4].txt

f in d - n a me " * . t x t " -delete # t a ke c a r e ! !

Parameter -regextypeto determine type of regular expression



8/13/2019 bash_day3

4/8

Executing Commands

For executing a single command, it first has to be found! Either path is provided, or command is in $PATH

which, whereisand type: find out about program type and path

For commands with more than one line:

echo " T h i s c o m m a n d p r i n t s a l o t o f u s e l e s s t e x t \

w h i c h d o e s n o t f i t i n t o o n e l i n e "

multiple commands

One way to execute several commands: connecting IO via pipes

cmd1; cmd2; ...executing several commands

cmd1 && cmd2 && ...execute following commands if previoussuccessful

cmd1 || cmd2 || ...execute following commands if previous fails



8/13/2019 bash_day3

5/8

awk - by Aho, Weinberger and Kernighan

awk is the most powerful bash tool

awk {}

or {}has to be specified (or both)

Options awk -f applies commands from

commandfileto the contents of file

-v var=valuedefines a variable to be available within theexecuted commands

-F set the field-separator to be available within theexecuted commands

Internal variables

NFNumber of fields in current line NRcurrent row

nn-th field of the current line

echo -e " 1 ; 2 ; 3 \ n 4 ; 5 ; 6 " | awk -F

;

1 > 3 { p ri nt

2 +

3 }

echo -e " 1 ; 2 \ n 4 ; 5 " | awk

{ l ine =

0 } END { print line }



8/13/2019 bash_day3

6/8

awk - Patterns

/regular expression/ relational expressionOperators: , =, !=, ==, ~, !~

Example: 1 ~ /regex/(matching a regex)

pattern && pattern

pattern || pattern

pattern ? pattern : pattern(conditional assignment)

(pattern)

! pattern

pattern1, pattern2(specifying a range)

awk

lengt h >10

input.txt

awk

2 >

1

input.txt

awk

1 != prev { print ; p rev =

1 }

input.txt



8/13/2019 bash_day3

7/8

awk - Commands

Operators

++, --Increment and decrement

+, -, *, /, %Basic arithmetics

^Exponentiation

, ...Relational operators

||, &&Logical or/and

?:C conditional expression

=, +=, *=, ...Assignment operations

Control statements

if, for, while, ...awk IS a programming language

awk

2 >

1 { print

3 +

4 }

i n p u t . t x t

awk

{ if (

1 >

2 ) p ri nt " f o o " ; else print " b a r " }

input.txt



8/13/2019 bash_day3

8/8

awk - More Commands

I/O statements

printprints the whole line print expr-listprints the list of expressions

print ... >> "file"prints the result to a file

print ... | commandprints the result to a pipe

nextjumps to the next line

printfC-type print command

...

Numeric functions

sin(expr), cos(expr), sqrt(expr), log(expr), ...

Further functionality string functions (sort, substrings, matching, ...)

Time functions

Bit manipulation

User functions



Documents

bash_day3