Lecture #7: bash & a bit of Linux

7/27/2019 Lecture #7: bash & a bit of Linux

1/21

High performance computing

for non-programmers

Lecture #7:bash & a bit of Linux

Glib Ivashkevych

junior researcher, NSC KIPT


2/21

Linux shell

text processingcut- get sections of line

useful options: -f -d -s

paste- merge lines

useful options: -d

sort- sort linesuseful options: -V -k -f

(seeman sort for more)


3/21

Linux shell

text processinguniq- get or skip repeated lines

useful options: -c -i -f

wc- count words, lines and characters

useful options: -l -L -w

join- join lines by common fielduseful options: -a -j -1 -2 -e

(seeman join for more)


4/21

Linux shell

text processingtr- translate/delete characters

useful options: -c

tee- read from input, write to outputand files

useful options: -adate, cal- date and calendar


5/21

Linux shell

text processinggrep- print matching lines

useful options: -c -i -l -m -v

(and many more, see man grep)

sed- stream editor

Swiss army knife later

awk- scanning and processing

Swiss army knife later


6/21

Linux shell

search for filesfind- search for files

useful options: -name -type -exec

(and many more, see man find)

ack- search for files with some

patternuseful options: -c -A -B -C

which- show full path of command

useful options: -a


7/21

Linux shell

redirectionsResults of program execution are sent to

standard output

Errors

standard error

Input

standard input

In Unix everything is a file. So, we canchange I/O directions.


8/21

Linux shell

redirectionsStandard output redirection:

>or >> (1>, actually)

Standard error redirection:

2>or 2>>

Both:

&>

Standard input redirection:


9/21

Linux shell

redirectionsTo throw smth. away:

/dev/null

Trick:named pipes

mkfifo /some/path/pipe

Trick:process substitution

command1 >(command2) 2> >(command3)


10/21

Linux shell

pipes

Pass STDOUT of one command to STDIN of

another:command1 | command2


11/21

Linux shell

expansionsBrace expansion:

{a,b,c,...} {start..end[..incr]}

a b c ...

Tilde expansion:

~/foo - directory fooof home directory

~name/foo - directory fooof names homedirectory


12/21

Linux shell

expansionsParameters expansion:

$VARNAME ${VARNAME}

Command substitution:

$(command) $(command)

Arithmetic expansion:

$((expr))


13/21

Linux shell

expansionsProcess substitution:

(command)

Word splitting:

everything not inside is splitted

Filename expansion:

* ? [ (pattern matching here)


14/21

Linux shell

quoting... - strong quoting

no special meaning of characters

... - weak quoting

no special meaning, except $,\,`

\ - escape

prevent special meaning of next character


15/21

Linux shell

environment variablesName/value pairsto influence behavior of software

Some are:

$PATH $HOME $LD_LIBRARY_PATH $USER $PS1Process local, inherited from parent process

To create one:

export $ENVVARNAME=valueTo get:

env or echo


16/21

Linux shell

configset command

~/.bash_profile- is executed at each login

~/.bashrc- is executed at each subshell start

~/.bash_logout- is executed at logout


17/21

Linux

going furtherLinux is

multiuser

multitasking

POSIX-compliant

open-source

secure

flexible

OS (kernel)


18/21

Linux

open source

Linux is most amazing success

in OSS movement

GPL license

Huge ecosystem


19/21

Linux

open source

GPL:Linux, GNU coreutils

BSD:BSD, Darwin

Apache: Android, Apache

Python, Ruby, git

nginx, MySQL

PostgreSQL, MongoDB


20/21

Linux

usage

HPC: 96% of Top-500

Web:not just Linux, but OSS

Embedded

Mobile: Android


21/21

Questions?

Documents

Lecture #7: bash & a bit of Linux