chapter 2

processes, pipes and signals

• A process is a running program• On multi-CPU machines, processes execute

simultaneously on separate CPUs.• On single-CPU machines, processes “appear” to run

simultaneously through time-sharing.• Most processes execute independent of one another;

sharing nothing.• multitasking - one purpose with multiple threads of

execution – implemented with cloning (fork()) and threads

process state machine

RUN

WAIT

IO Blocking

waiting for CPU time

running on CPU

blocking because ofIO request

time out

IO complete

IO request

taking its turn

fork():

• fork() is called with no arguments and returns a result code

fds

$pid = fork();if ($pid>0) { # parent} else { # child $ppid = getppid();}

$pid = fork();if ($pid>0) { # parent} else { # child $ppid = getppid();}

parent child

parent runsthis code

child runsthis code

shared

copydata

can potentially be usedto share information

signal one another

in child, $pidnow has newvalue

same program in both processes

fork() (2):

$pid = fork();

in parent, this isthe child PID

in child, this is 0; to get parent PID childexecutes $pid = getppid();

NOTE: In case of error, fork() returns undef and $! contains the appropriate error message

process group:

• All processes that fork() from the same ancestor belong to the same process group. The group PID is usually the parent process PID.

• You can fork() from a child and produce grandchildren

• You can fork() repeatedly and produce siblings

$processid = getpgrp([$pid]);

if $pid is not specified then getthe group for the current process.

sharing

• Each member of group shares whatever filehandles were open when their common ancestor forked.

• Shared filehandles can be used to exchange information between processes in the same group.

• Processes in the same group share STDIN, STDOUT and STDERR.

sharing files:

• fork() is called with no arguments and returns a result code

a shared file/filehandle

RW RW

parent childshared

copydata

book calls this “shar[ing] the current values”, p36

Simple program:

#!/usr/bin/perl# file: fork.pl

print “PID = $$\n”; # $$ is current process PIDmy $child = fork();die “Can’t fork: $!” unless defined $child;if ( $child > 0 ) { # parent since $child != 0 print “parent PID = $$, child PID = $child\n”;} else { # child since $child == 0 my $ppid = getppid(); print “child PID = $$, parent PID = $ppid\n”;}

defined() is a functionthat tests if a variablehas a value

which print statement prints out first? on Windows? on Linux?

Executing other programs:

• Executes a command from inside a perl program; the program blocks until the command completes and then continues executing at the next line

• $result == 0 if all went well; -1 if command couldn’t start or command failed. Look at $? for the error.

• None of the command’s STDOUT output is returned to the program but is sent rather to whatever STDOUT is currently writing to. This is relevant since you often want to process the STDOUT output that is generated by something called from a perl program.

$result = system(“command and arguments”);

system():

• echoes “hello, world!” to

terminal

• sends “hello, world!”

to the file fileout

#!/usr/bin/perl# system.plsystem(“echo ‘hello, world!”);

$ ./system.plhello, world!$

#!/usr/bin/perl# system.plsystem(“echo ‘hello, world!”);

$ ./system.pl > fileout$ cat fileouthello, world!$

system() 2:

#!/usr/bin/perl# system.pl$result = system(“echo ‘hello, world!”);printf $result . “\n”;$result = system(“cat nofile”);$err = $? >> 8;printf $result . “\n”;printf “error: $err\n”;

$ ./system.plhello, world!0cat: nofile: no such file or directory256error: 1$

$result == 0 if ok

error code comes as mostsignificant byte of two byteword; shift right 8 bits to seeactual value.

$result != 0 if commandfails

exec():

• With exec() you actually replace the existing program with a new one using the same PID.

• All data and “knowledge” about its past self are lost to the process; it becomes the program being execed.

exec() 2:

• use fork() and exec() to make one program startup and run another

fds

$pid = fork();if ($pid>0) { # parent} else { # child $err = exec(“command and args”);}

$pid = fork();if ($pid>0) { # parent} else { # child $err = exec(“command and args”);}

parent child

parent runsthis code

shared

entire processreplaced with“command andargs”

child runs this code

exec() 3:

#!/usr/bin/perl

my $child = fork();if ( $child == 0 ) { open(STDOUT,”>log.txt”) or die “open() error: $!”; exec (“ls –l”); die “exec error: $!”; }exit 1; # parent exits

the parent process exits immediately

we only execute this only if exec() fails

redirect STDOUT outputto log.txt

the newly execed programinherits my STDOUT

Problem: IS STDOUT redirectedin the parent process?

pipes:

• Used for communication between two processes running on the same host.

• The processes may be related (parent-child-sibling) or unrelated.

• The above opens a pipe between (from) the ls command and (to) the INFILE handle of this program.

• The above opens a pipe between (from) this program (to) the OUTFILE handle of the wc command.

open(INFILE, “ls –la | “);

open (OUTFILE, “ | wc -l”);

pipes 2:

$wc = IO::Filehandle->open(“ | wc –lw”) or die “Can’t open: $!”;

Using pipes (simple example):

#!/usr/bin/perl# fle: whos_there.pl

use strict;my %who; # accumulate login count

open(WHOFH, “who |”) or die “can’t open who: $!”;while ( <WHOFH> ) { next unless /^(\S+)/; $who{$1}++;}foreach (sort {$who{$b} <=> $who{$a}} keys %who) { printf “%10s %d\n”,$_, $who{$_};}close WHOFH or die “can’t close who: $!”;

non-whitespace tokenanchored at start of line

$1 == value of match found inside ( )

fancy way to sorta list of keys by hashvalue in ascending order

can’t use redirect sincewho is a command

read about sort() at perldoc perlfunc use $_ if foreach has no specifiedvariable

read perldoc perlvar about exit codes

backtick (`…`):

• is the same as $arguments = “-lart”;$ls_output = `ls $arguments`;

$arguments = “-lart”;open(INFILE, “ls $arguments |”);while (<INFILE>) { $ls_output .= $_;}

all stdout output readinto a single scalar

concatenate

backtick and stderr:

• ` … ` returns only STDOUT output; what about STDERR output?

• The above delivers both STDOUT and STDERR output to $ls_output.

$ls_output = `ls 2>&1`;

pipe():

• This opens a pipe; one can read from READHANDLE and write to WRITEHANDLE.

• One uncommon usage is to open a pipe (in fact, two) just before forking a child. The pipe and all handles are shared by both processes. Close handles to leave two unidirectional pipes.

$result = pipe(READHANDLE,WRITEHANDLE);

parent child

pipeRW

WR

pipeRW

WR

close close

pipe (in detail):

WRITER

READER

WRITER

READER

parent child

READER

WRITER

pipe

pipe

before fork()

after fork()

pipe() 2:

child1parent

pipeWRITER

child2

pipeWRITER READER

fibonacci()factorial()print while <READER>;

only one pipe

two child processes

single READER handle in parent

single WRITER handle shared by two siblings

facfib.pl:#!/usr/bin/perl# file: facfib.pl

use strict;my $arg = shift || 10;

pipe(READER,WRITER) or die “Can’t open pipe: $!”);

if ( fork() == 0 ) {# child 1 close READER; select WRITER; $| = 1; factorial($arg); exit 0;}if ( fork() == 0 ) {# child 2 close READER; select WRITER; $| = 1; fibonacci($arg); exit 0;}

at this point defaultoutput is WRITER

facfib.pl 2:

sub factorial { my $target = shift; for (my $result = 1, ,y $i = 1; $i <= $target; $i++) { print “factorial($i) => “ , $result *= $i , “\n”; }}

sub fibonacci { my $target = shift; my ($a,$b) = (1,0); for (my $i = 1; $i <= $target; $i++) { my $c = $a + $b; print “fibonacci($i) => $c\n”; ($a,$b) = ($b,$c); }}

list assignment

default outputis still WRITER

pipe() as an alternative to open():

• The problem with

• is that you can only read from the program and you are constrained by reading in a loop inside your program.

• What if you want to read from and write to the program?

• What if you want to program to go off on its own to do its own thing while you are busy doing something else?

open(READER,”myprog.pl $args |”);

Example 1:

• almost identical to

open(READER,”myprog.pl $args |”);

pipe(READER,WRITER) or die “pipe no good: $!”;my $child = fork();die “can’t fork: $!” unless defined $child;if ( $child == 0 ) { close READER; open(STDOUT, “>&WRITER”); exec myprog.pl $args; die “exec failed: $!”;}close WRITER;…

at this point myprog.pl is executed in the process spaceoriginally intended for the childprocess; it inherits the STDOUThandle, which points to WRITER.

myprog.pl doesn’texecute in parallelin the open() example

bidirectional pipes:

• pipes are created unidirectional;

• is illegal

• For true bidirectional communication we use socketpair() which is introduced in Chapter 4

open(FH, “| $cmd |”);

Is a filehandle a pipe or what?

• perl offers a file testing mechanism to determine many things about a file

Test Description

-p Is the filehandle a pipe?

-t Is the filehandle opened on a terminal?

-S Is the filehandle a socket?

examples:

• Test if we have closed STDIN for batch mode.

• Test if a handle is a pipe.

if ( -t STDIN ) { printf “STDIN not closed yet.\n”;}

print “Not a pipe\n” if ! –p FILEHANDLE;

PIPE Error:

• Reading: no problem

Suppose you are reading at one end of a pipe and your childis writing at the other end.

Now suppose the child closes its end of the pipe or even exits?

parent child

pipeRW

WR

now close the pipeor exit the program

when next you read,EOF will happen

PIPE Error:

• Writing: a problem

Suppose you are writing at one end of a pipe and your parentis reading at the other end.

Now suppose the parent closes its end of the pipe or even exits?

parent child

pipeRW

WR

when you try to writeyou get the Broken piperror

now close the pipeor exit

program example: #!/usr/bin/perl#file: write_ten.pluse strict;open(PIPE,”| read_three.pl”) or die “Can’t open pipe: $!”;select PIPE; $| = 1; select STDOUT;my $count = 0;for ( 1..10) { warn “Writing line $_\n”; print PIPE “This is line number $_\n”; $count++; sleep 1; # or select(undef,undef,undef,0.5)}close PIPE or die “Can’t close PIPE: $!”;print “Write $count lines of text\n”;

#!/usr/bin/perl# file: read_three.pl

use strict;for (1..3) { last unless defined ($line = <>); warn “read_three got: $line”}

after the 3rd write, the read_three.plprogram terminates and closes the PIPE. The output is

program output:

Writing line 1Read_three got: This is line number 1Writing line 2Read_three got: This is line number 2Writing line 3Read_three got: This is line number 3Writing line 4Broken pipe

tried to write to a pipe closed by read_three.pl;no line was successfully passed to the read_three.pl

this error is called aPipe Exception

PIPE Exception:

• A PIPE Exception results in a PIPE signal being delivered to the writing process.

• Processes that receive the PIPE signal terminate by default.

• This also happens when writing across a network to a remote program that has closed its end of the communication channel or exited.

• To deal programmatically with a PIPE signal you mist write a signal handler.

Signals:

• A signal is a message sent to your program by the OS when something “important” occurs.

• Processes can signal each other.

Examples: - the program asks a stdlib routine to divide by zero - a user tries to interrupt the program (other than ^C). - a sub-process terminated but it is not the end of the world

Examples: - when user hits ^C the shell sends a signal to the program - a process sends signals to itself - a user sends the USR1 signal to a program (using the a shell to send the signal) to have the program reread its configuration file w/o recycling.

Common Signals:

• There are 19 standard signals; each has associated with it a small integer and a name.

HUP 1 A Hangup detected

INT 2 A Interrupt from kybd

QUIT 3 A Quit from kybd

ILL 4 A Illegal instruction

ABRT 6 C abort

FPE 8 C Floating pt exception

KILL 9 AF Termination signal

USR1 10 A User defined #1

SEGV 11 C Invalid memory ref

USR2 12 A User defined #2

PIPE 13 A Write to pipe; no reader

ALRM 14 A Timer signal

TERM 15 A Termination signal

CHLD 17 B Child terminated

CONT 18 E Continue if stopped

STOP 19 DF Stop process

TSTP 20 D Stop typed at tty

TTIN 21 D tty input: background

TTOU 22 D tty output: background

Name Value

Notes:A: Default is terminate processB: Default is ignore signalC: Default is terminate process and dump coreD: Default is stop processE: Default is resume processF: Signal can not be caught or ignored

Notes Comment

Further Signal Comments:

• We won’t use all these signals (italics)• HUP happens when a program is running in a terminal that is killed.• INT is send when someone hits ^C.• TERM and KILL used by one process (eg, shell) to terminate

another. ^C sends TERM but you can disable the default behaviour by writing a TERM signal handler.

• PIPE is sent when a process writes to a closed (remote end) pipe or socket.

• ALRM used with alarm() to send a prearranged signal to yourself.• CHLD sent to parent when child process status changes (exit, stop,

continue).• STOP suspends the current process; resumed by CONT.• TSTP suspends current process from tty (^Z); resumed by CONT

and can be caught by your own signal handler.

Catching Signals:

• %SIG holds references all signal handlers (functions).

• You catch a signal yourself by assigning a value to any signal except $SIG{KILL} and $SIG{TERM}. This value references a small function.

$SIG{INT} references the current/default INT signal handler

Example:

#!/usr/bin/perl#file: interrupt.pl

use strict;my $interruptions = 0;$SIG{INT} = \&handle_interruptions;

while ($interruptions < 3) { print “I am sleeping.\n”; sleep(5); # blocks for 5 seconds}

sub &handle_interruptions { $interruptions++; warn “Don’t interrupt me. You’ve already interrupted \ me ${interruptions}x.”;}

replaces defaultbehaviour (terminate)with pedantic counter

you need the {} because you can’t write$interruptionsx.

this is a real/namedsubroutine. You can alsouse an anonymous subroutine (see next slide).

Anonymous Subroutine:

$SIG{INT} = sub {$interruptions++; warn “Don’t interrupt me. You’ve already interrupted \ me ${interruptions}x.”; };

Special Cases:

• Perl recognizes two special values for an interrupt handler – DEFAULT and IGNORE.

• DEFAULT restores the default behaviour• IGNORE tells perl to ignore this signal if received.

Handler Reuse:

$SIG{TERM} = $SIG{HUP} = $SIG{INT} = \&handler;

sub &handler { my $sig = shift; warn “handling a $sig signal.\n”;}

The signal number is obviously passedautomatically to any signal handler as its first argument.

PIPE Exceptions 1:

#!/usr/bin/perl#file: write_ten_ph.pluse strict;$SIG{PIPE} = sub { undef $ok; }open(PIPE,”| read_three.pl”) or die “Can’t open pipe: $!”;select PIPE; $| = 1; select STDOUT;my $count = 0;for ( $_ = 1; $ok && $_ <= 10; $_++) { warn “Writing line $_\n”; print PIPE “This is line number $_\n”; $count++; sleep 1; # or select(undef,undef,undef,0.5)}close PIPE or die “Can’t close PIPE: $!”;print “Write $count lines of text\n”;

Now when read_three.plcloses, instead ofterminating write_ten_ph.plthe loop terminates gracefully (no Broken pipemessage).

PIPE Exceptions 2:#!/usr/bin/perl#file: write_ten_i.pluse strict;$SIG{PIPE} = ‘IGNORE’;open(PIPE,”| read_three.pl”) or die “Can’t open pipe: $!”;select PIPE; $| = 1; select STDOUT;my $count = 0;for ( 1..10) { warn “Writing line $_\n”; if (print PIPE “This is line number $_\n”) { $count++; } else { warn “An error occurred during writing: $!”; last; # break out of loop } sleep 1; # or select(undef,undef,undef,0.5)}close PIPE or die “Can’t close PIPE: $!”;print “Write $count lines of text\n”;

Now when read_three.plcloses, instead ofterminating write_ten_i.plthe loop terminates gracefully (we see Broken pipemessage only because weask to) by checking if theprint() function failed.

Exceptions 3:

• What if we wanted to handle the Broken pipe error separately from all other errors print() could return?

use Errno ‘:POSIX’;. . .unless (print PIPE “This is line number $_\n”) { last if $! == EPIPE; # break out of loop if Broken pipe die “IO error: $!”; # all other possible errors}

numeric value only available ifwe import Errno ‘:POSIX’ orErrno qw(EPIPE)

Sending Signals:

• From a process:

• Special kill signals:

$count = kill($signal, @processes);

you can use signal numbersor symbol names

you can send the signal to a listof processes but only if you have aright to do so – same user privileges orroot can kill anything.

number of processes successfully signaled

$count = kill(0,@processes);

returns the number of processes from list which could receive a signal but no signal is sent

$count = kill(-n,@processes);

kill treats abs(n) as a process group ID andelivers signal to all processes in the group

Sending signals 2:

• Commiting suicide:

kill INT => $$ ; # same as kill(‘INT’,$$)

Caveats:

• What if you are doing something important or executing a system() or other system command when a signal arrives?– keep it simple ($g_var = 1;) and handle the heavy lifting back in

the main program– don’t do IO inside a handler (strip out warn() in production)– ok to call die() or exit() in a handler since you are exiting anyway

but NOT on a Windows machine– In 2001 Windows only implemented a minimal set of signal

handlers – CHLD was not implemented, for example.

Timing out slow system calls:

• If a signal occurs while perl is executing a slow system call, the handler is executed and the system call is restarted at the point it left off.

• Some slow system calls don’t restart however; sleep() for example.

$slept = sleep([$seconds]);

sleeps $seconds or until signal received;sleeps forever if no argument

returns how many seconds it

actually slept

What if Automatic Restart is no good:

my $timed_out = 0;$SIG{ALRM} = sub { $times_out = 1; };

printf STDERR “type your passwd;alarm(5);my $passwd = <STDIN>;alarm(0);

printf STDERR “You timed out \n” if $timed_out;

handles ALRM signal

sets ALRM signal to be sent in 5 seconds

if you type nothing, ALRM isreceived after 5 secondsturns timer off;

important or another slow system call could timeout

the logic is that if nothing is typed in 5 seconds the program moveson; perhaps to a prompt insisting on a password. In reality the <>call is restarted and the program hangs until you enter a password.

We need another solution.

Timing out keyboard input:print STDERR “type your password: “;my $passwd = eval { local $SIG{ALRM} = sub {die “timeout\n”;}; alarm(5); return <STDIN>; }alarm(0);print STDERR “you timed out\n” if $@ =~ /timeout/;

An eval{…} block is like an anonymous program.

When die() executes inside an eval{} block it only kills the eval{} block which then returns undef and the program continues on the line after the eval{} block.

If you enter a password before the timer expires then the eval{} block returns its value.

When an eval{} block dies, $@ is set to the last string error message.

same logic as beforebut this time the ALRMhandler calls die() contains pattern

‘timeout’