BIT 4206 -Network Programming

1 | N e t w o r k P r o g r a m m i n g

P.O. Box 342-01000 Thika Email: [email protected]

Web: www.mku.ac.ke

DEPARTMENT OF INFORMATION

TECHNOLOGY

COURSE CODE: BIT 4206

COURSE TITLE : NETWORK PROGRAMMING

Instructional Manual for BBIT – Distance Learning

Prepared by Mr. Paul Mutinda

E-mail: [email protected]

http://www.mku.ac.ke/�

mailto:[email protected]�


Table of Contents INTRODUCTION UNIX TECHNOLOGIES .............................................................................................. 5

Primary Objectives of OS: ........................................................................................................................ 5

What is Unix ? ........................................................................................................................................... 6

Unix Architecture: ..................................................................................................................................... 7

The Kernel ............................................................................................................................................. 8

The Shell: ............................................................................................................................................. 10

UNIX SYSTEM CALLS ............................................................................................................................ 16

System Call Interface .............................................................................................................................. 16

Library calls and system calls .................................................................................................................. 17

Categories of System Calls in Unix .......................................................................................................... 23

PROCESS AND INTERPROCESS COMMUNICATION ..................................................................................... 24

What is process? ..................................................................................................................................... 24

The Process Model .................................................................................................................................. 24

Interrupts ................................................................................................................................................ 26

Process operations (System Calls) .......................................................................................................... 30

Inter-Process Communication ................................................................................................................ 35

Shared Memory .................................................................................................................................. 36

Message Passing IPC ............................................................................................................................ 39

Process Synchronization ......................................................................................................................... 40

POSIX THREADS PROGRAMMING ............................................................................................................... 43

Pthreads Overview .................................................................................................................................. 43

Pthreads Overview .................................................................................................................................. 45

Designing Threaded Programs ................................................................................................................ 48

The Pthreads API ..................................................................................................................................... 51

Thread Management .............................................................................................................................. 53

Creating and Terminating Threads ..................................................................................................... 53


Passing Arguments to Threads ............................................................................................................ 58

Joining and Detaching Threads ........................................................................................................... 60

Example: Pthread Joining ....................................................................................................................... 62

Stack Management ............................................................................................................................. 64

Miscellaneous Routines ...................................................................................................................... 67

Mutex Variables ...................................................................................................................................... 68

Overview ............................................................................................................................................. 68

Creating and Destroying Mutexes ...................................................................................................... 69

Locking and Unlocking Mutexes ......................................................................................................... 70

Example: Using Mutexes ..................................................................................................................... 71

Condition Variables ................................................................................................................................. 75

Overview ............................................................................................................................................. 75

Creating and Destroying Condition Variables ..................................................................................... 76

Waiting and Signaling on Condition Variables .................................................................................... 77

SOCKET PROGRAMMING ............................................................................................................................ 82

Introduction ................................................................................................................................................ 82

Prerequisites for socket programming ............................................................................................... 82

How sockets work ................................................................................................................................... 83

Socket characteristics ............................................................................................................................. 85

How socket characteristics are determined ........................................................................................... 86

Socket address structure ........................................................................................................................ 87

Socket address family ............................................................................................................................. 88

AF_INET address family ....................................................................................................................... 88

AF_INET6 address family......................................................................................................................... 89

AF_UNIX address family ...................................................................................................................... 90

AF_UNIX_CCSID address family .......................................................................................................... 91


Socket type .............................................................................................................................................. 92

Socket protocols...................................................................................................................................... 93

BASIC SOCKET DESIGN ................................................................................................................................ 95

Creating a connection-oriented socket................................................................................................... 95

Creating a connectionless socket .......................................................................................................... 103

Designing applications with address families ....................................................................................... 112

Using AF_INET address family ....................................................................................................... 112

Using AF_INET6 address family ......................................................................................................... 112

Using AF_UNIX address family .......................................................................................................... 112

Using AF_UNIX_CCSID address family .............................................................................................. 122

SOCKET APPLICATION DESIGNS: EXAMPLES ............................................................................................. 131

Examples: Connection-oriented designs ............................................................................................... 131

Example: Using the spawn() API to create child processes .................................................................. 136

Example: Enabling the worker job to receive a data buffer: ................................................................ 142

Example: Server program used for sendmsg() and recvmsg(): ............................................................. 145

Example: Worker program used for sendmsg() and recvmsg(): ........................................................... 149

Examples: Using multiple accept() APIs to handle incoming requests ................................................. 152


INTRODUCTION UNIX TECHNOLOGIES

Operating systems Basics An operating system is a suite of programs that control a Computer-system.

A Computer system consists of one or more CPUs, device controllers, shared memory connected

through a common bus.

Every piece of hardware is different, but the OS abstracts this disparity to give the user a unified

interface that is easy to use.

Virtual Machine Abstraction

Primary Objectives of OS:

-Making a computer system convenient to use: by hiding the details of the hardware resources

from the user provides him/her with a convenient interface.

-Resource management

Each OS component (e.g. file systems, virtual memory, networking, scheduling):

What’s the hardware interface? (Physical reality)

What’s the application interface? (Nicer abstraction)

Systems software

Systems software help create abstraction and a uniform interface for programs to use when

requesting for OS services. Examples of systems software:

assemblers

compilers

linkers


The dichotomy of programming

Application programming: targeted toward developing systems to support the end-user.

Systems programming: targeted toward developing systems to support the programmer.

Recently, this boundary has become fuzzy.

Building a web browser, such as Google Chrome might once have been considered

application programming.

However, nowadays developing such applications requires attention to system details

such as resources and efficiency (e.g., Google Chrome is multi-threaded).

Systems programming

Historically, systems programming meant programming the system (i.e., building

compilers, shells, loaders, and so on).

However, nowadays, systems programming has come to mean programming with the

system (i.e., making system calls, managing threads, and so on).

Systems programming requires a greater awareness of issues of hardware and efficiency

than application programming.

It also requires knowledge of low level languages that provides direct access to and

control of system resources; this leads us to UNIX and C

What is Unix ? The UNIX operating system is a set of programs that act as a link between the computer and the

user.

The computer programs that allocate the system resources and coordinate all the details of the

computer's internals is called the operating system or kernel.

Users communicate with the kernel through a program known as the shell. The shell is a

command line interpreter; it translates commands entered by the user and converts them into a

language that is understood by the kernel.

Unix was originally developed in 1969 by a group of AT&T employees at Bell Labs,

including Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna.

There are various Unix variants available in the market. Solaris Unix, AIX, UP Unix and

BSD are few examples. Linux is also a flavour of Unix which is freely available.


Several people can use a UNIX computer at the same time; hence UNIX is called a

multiuser system.

A user can also run multiple programs at the same time; hence UNIX is called

multitasking.

Unix Architecture: Here is a basic block diagram of a UNIX system:

The main concept that unites all versions of UNIX is the following four basics:

• Kernel: The kernel is the heart of the operating system. It interacts with hardware and

most of the tasks like memory management, tash scheduling and file management.

• Shell: The shell is the utility that processes your requests. When you type in a command

at your terminal, the shell interprets the command and calls the program that you want.

The shell uses standard syntax for all commands. C Shell, Bourne Shell and Korn Shell

are most famous shells which are available with most of the Unix variants.

• Commands and Utilities: There are various command and utilities which you would use

in your day to day activities. cp, mv, cat and grep etc. are few examples of commands


and utilities. There are over 250 standard commands plus numerous others provided

through 3rd party software. All the commands come along with various optional options.

• Files and Directories: All data in UNIX is organized into files. All files are organized

into directories. These directories are organized into a tree-like structure called the file

system.

UNIX has evolved over the past thirty years from its conceptual stage into a powerful and

effective operating system and the credit goes to the fundamental structure of UNIX, which is

very robust due to its layered approach comprising of the:

• Kernel

• Shell

• Utilities and applications

One of the most powerful and attractive features of UNIX is its file system, which manages data,

stored on the computer's secondary storage devices. The file system facilitates organizing stored

information in a very logical way, and manipulating it as and when required.

Fig: UNIX Architecture

The Kernel

> Core of the UNIX system

> Interacts directly with the hardware

> Insulates other parts of UNIX from hardware

> Performs all low level functions


> Parts of kernel deals with I/O devices, called Device Drivers

> All programs and applications interact with the kernel

Fig: UNIX Kernel

The kernel is the Core of the UNIX system, which controls the system hardware and performs

various low level functions. Viewing the operating system as a set of layers, the kernel usually

refers to the operating system itself.

However, the kernel does not directly deal with the user. Instead, it starts a separate program,

called shell, which actually interacts with the user and interprets the commands. The utilities and

applications add special capabilities to the operating system. Kernel functions

• Memory management

• Process scheduling

• File management and security

• Interrupt handling and error reporting

• Input / Output services

• Date and Time services

• System accounting


The Kernel:

• Controls the execution of processes

• Allows creation, termination, suspension and communication of process.

• Enforces the security scheme and prevents unauthorized access to stored information like

files and directories.

• Handles all low level input and output services.

• Handles the interrupts and system errors.

• Schedules the Central Processor and allocates system resources to each user of the

system.

If the UNIX operating system is to be loaded on different hardware platforms, the kernel has to

be customized accordingly. All applications and utilities including the shell, interact with the kernel by invoking well

structured routines of kernel by means of messages called system calls.

The Shell:

• Is the command interpreter of UNIX

• Interface between the kernel and the user

• Provides powerful programming capabilities

The shell is the interface between a user and the operating system. When any user logs in, a copy

of the shell program is started up, displaying a prompt sign such as '$' (dollar), which informs

the user that the shell is waiting to accept a command. In UNIX, there are three types of shells:

1. The Bourne Shell

2. The C Shell

3. The Korn Shell

From the figures you can conceive the way; various parts and portions of UNIX are arranged

around the hardware. The core of UNIX is the kernel, which schedules jobs and manages data


storage, and it is very closely interfaced with hardware. The hardware functions according to the

machine instructions released ultimately by the kernel, based on the shell's interpretation of user

commands. Surrounding the kernel are parts of software of the shell relating to:

1. Execution of commands for piping and filtering actions.

2. Tools for carrying out foreground and background processing.

3. Utilities for configuring the hardware.

4. I/O redirection and command execution utilities. 5. Filename substitution.

The shell forms an interface between the user and the kernel. Outside the shell are the

different user-specific utilities of UNIX, numbering around 300, which enhance its capability.

Above this, are the user programs and application packages, which can facilitate data entry, data

modification, query, report generation, etc.


Figure: Shell functions

The shell is a program that collects and interprets the user commands, invokes the concerned

program file from memory and directs the kernel to execute them.

As the name shell suggests, the shell envelops the kernel. Externally, the UNIX operating system

is made up of two parts. One part is a large set of system programs, each one corresponding to a

command, and another part is the shell, which interprets, manages and coordinates the execution

of these programs. Most of the users are unaware of the various complexities of the operating

system or the hardware.


The user sees the shell command as one peculating down the UNIX system, and the kernel,

looking up to the user through the shell- for the next set of process. This relationship and

interface of the kernel with different users is shown in the figure below.

Every user, once accepted by the kernel as authorized, is issued a shell prompt. Each user has

his own shell (a separate and personal copy for each user) and the commands issued by him is

counted and numbered serially. A separate shell procedure is reserved for each user.

Figure: UNIX interaction with the users


Command Execution

Refer figure below, which illustrates the cycle of command execution. To begin with, the

kernel displays a shell prompt after authorized login. The shell waits for the input from

the user, decodes the command line and searches for the program. If the program is

found, the shell retrieves and submits it to the kernel

to execute the program. The kernel delivers the output. If the command is not found, it signals

the kernel to display command not found at the terminal. The shell accepts the kernel's reply,

and in both cases displays the next prompt. This cycle continues until the user with <CTRL>

D or logout terminates it. Once the shell encounters end of input, it instructs the kernel to log

out the user, and displays the login message again.

The system directories where the command file programs are stored are the /bin and

/usr/bin. On receipt of the command, which is nothing but the executable filenames in the

system directory, the shell locates them and transfers control to the kernel to execute.

These filenames can be verified by the command pwd and Is, after changing to the

directories Ibin and /usrlbin.

Shell Environment

A separate shell process is apportioned to each user as stated earlier. The separate shell once

created provides an environment within which the user operates. The shell environment consists of

the following to ensure proper interaction between the user and the system:

1. The name of the users, and a working directory (a directory allotted to the user by the


operating system).

2. A list of system directories to be searched to locate the commands given by the user.

3. The file access permission modes.

4. A record of users identification (user id) and group identification number (group id).

Foreground and Background Processing

The shell can execute commands in the foreground and also in the background. Foreground

commands execute sequentially (one command cannot commence until execution of the preceding

command has completed execution). Foreground commands like editing a program or processing a

text involves interaction with the terminal and the input. Background processing jobs related

commands are accompanied by & (ampersand) symbol. Jobs like compiling, printing can be carried

out as background jobs, since they do not require any interaction with the user. There may be

several background commands executed at a time. This truly makes UNIX a multitasking system.

Review Questions

1. List the main functions of the kernel, and shell, respectively.

2. Explain the relation between UNIX commands and UNIX system filenames.

3. Define and explain system calls in UNIX, and explain how UNIX is made portable across

various platforms.

4. Sketch the cyclic process of executions of commands issued from the terminal, and

various roles played by the shell, kernel and hardware.

5. What does the shell environment contain with reference to every specific user interacting

with UNIX.

6. What does background processing mean?

7. What do you mean by tools of UNIX, list the classification of tools and utilities?

8. Sketch the UNIX directory system listing the system directories and the functions of the files

under them.

9. Sketch and explain the conceptual structure of UNIX system software layers.

10. Explain with a sketch, how a common user looks upon his interaction with hardware.


UNIX SYSTEM CALLS

System Call Interface All operating systems provide service points through which programs request services form

the kernel.

A system call is a mechanism used by an application to request for services from the

kernel.

Processes can either be user or kernel initiated and can run either in user mode or kernel

mode.

Kernel mode

Kernel mode processes can:

Enable/Disable interrupts.

direct access to I/O devices.

Access to memory table.

Etc

User mode only has rights to user initiated processes.


Library calls and system calls The difference between system calls and library calls is that:

System calls are provided by the system and are executed in the system kernel.

They are entry points into the kernel and are therefore NOT linked into your

program. These are not portable calls.

Library calls include the ANSI C standard library and are therefore portable.

These functions are linked into your program. (Recall linking in programming)

Because system calls are part of the O/S, the program has to make a context switch to the

kernel when they are called and because of this, they have a high startup overhead. The

upside is that the time executing these routines is assigned to the OS and not the user

program.

Calling standard library functions does not involve system call interface. However a library

function may initiate a system call which in turn issues a system call e.g. a call to printf()

may invoke write() system call to do output.

A system call always involves using system call interface and using the kernel of OS.

Anatomy of a System call

General form of a system call is as follows:

[returned value] = system_call_name ( parameters ) ; (if any)

Where:

returned_value = nonnegative integer if OK, -1 if error(in this case error code is placed in

external variable errno)

U ser p rog ra m

Sy ste m ca ll in te rface

L ib rar ie s: s in () , c o s() , ato i() ,…

Syste m ca ll p ro ce dure s:w rite( ) , rea d(), for k () , . . .


Handling System call

General Syntax:

Example:


A system call Error

When a system call discovers and error, it returns -1 and stores the reason the called failed

in an external variable named "errno".

The "/usr/include/errno.h" file maps these error numbers to manifest constants, and it these

constants that you should use in your programs.

When a system call returns successfully, it returns something other than -1, but it does not

clear "errno".

"errno" only has meaning directly after a system call that returns an error.


Using a library function perror();

When you use system calls in your programs, you should check the value returned by those

system calls.

Furthermore, when a system call discovers an error, you should use the "perror()"

subroutine to print a diagnostic message on the standard error file that describes why the

system call failed.

The syntax for "perror()" is:

void perror(string)

char string;


File creat() system call

The prototype for the creat() system call is:

int creat(file_name, mode)

char *file_name;

int mode;

Where:

file_name is pointer to a null terminated character string that names the file.

mode defines the file's access permissions. - read, write etc.

Example 1: creat() system call

/* creat.c */

#include <stdio.h>

#include <sys/types.h> /* defines types used by sys/stat.h */

#include <sys/stat.h> /* defines S_IREAD & S_IWRITE */

int main()

{

int fd;

fd = creat("datafile.dat", S_IREAD | S_IWRITE);

if (fd == -1)

printf("Error in opening datafile.dat\n");

else {

printf("datafile.dat opened for read/write access\n");

printf("datafile.dat is currently empty\n");

}

close(fd);

exit (0);

}


Example 2: use of fork() and exec() to create a new directory /* newdir.c

create a new directory, called newdir, using fork() and

exec() */

#include <stdio.h>

int main()

{

int fd;

if ( fork() != 0)

wait ((int *) 0);

else {

execl ("/bin/mkdir", "mkdir", "newdir", (char *) NULL);

fprintf (stderr, "exec failed!\n");

exit (1);

}

/* now use newdir */

if ( (fd = open("newdir/foo.bar", O_RDWR|O_CREAT, 0644))== -1)

{

fprintf (stderr, "open failed!\n");

exit (2);

}

write (fd, "Hello, world\n", 14);

close (fd);

exit (0);

}


Categories of System Calls in Unix


PROCESS AND INTERPROCESS COMMUNICATION

What is process?

• The simplest definition of a process is that it is a program in execution on a computer.

• Another definition; a process the basic computational element in modern computer system.

• How does it differ from a program?

• A process consists of:

o Instructions to define the behaviour of process.

o The data operated on by the process and the results it produces

o A set of resources to provide an environment for execution.

o A status record to keep track of the progress of the process during execution

• Each process uses resources to execute programs on data on Von Neumann computer.

• During execution, the process requests different resources according to the particular

behaviour defined by the program

• Multiprogramming systems explicitly allow multiple processes to exist at any given time,

where only one is using the CPU at any given moment, while the remaining processes are

performing I/O or are waiting for resources.

Process Manager

• The process manager implements:

o The process abstraction by creating a model for the way the process uses the CPU

and any system resources.

o CPU sharing (scheduling), process synchronization mechanisms.

o Part of the Operating system’s protection strategy.

The Process Model

• The scheduler moves process between execution "states".A process’ state defines what it can

do.

• A process state consists of two elements:

o an address space (a subsection of the OS memory space where a process

executes) and

o A set of registers (including PC and stack pointer).

The states are (roughly)


o new (waiting to be run first time)

o running

o ready (runnable)

o blocked (waiting)

o completed

Figure: Process states

Process Control Block

PCB (also process descriptor) is a data block that describes the process to the operating

system.

With the PCB, the OS can manipulate all parts of a program's state.

PCB contents include

o Process execution state (what is the process currently doing?)

o Saved program counter (for processes not in running state). The counter indicates

the address of the next instruction to be executed for this process.

o Saved CPU register set (for processes not running) The registers vary in number and

type, depending on the computer architecture. They include accumulators, index

registers, stack pointers, and general-purpose registers, plus any condition-code

information. Along with the program counter, this state information must be saved

when an interrupt occurs, to allow the process to be continued correctly afterward

o Scheduling information (what priority, reason for sleep, etc.)

o Memory management book-keeping (where is the processes memory?) .This


information may include such information as the value of the base and limit

registers, the page tables, or the segment tables, depending on the memory system

used by the operating system.

o Accounting information. This information includes the amount of CPU and real

time used, time limits, account numbers, job or process numbers, and so on.

o I/O status information: The information includes the list of I/O devices allocated to

this process, a list of open files, and so on.

o user id – the process owner.

A PCB is needed because the OS does not always "pay attention" to a process so from the

PCB, the OS can determine what the process is doing, and how it should respond to some

event.

It also needed to allow OS to organize processes for example: disk I/O

Process switching (context switch)

CPU switches between a running process and one that is runnable (i.e. on the ready

queue).

When this switching is so frequent that hardly is there any execution of the process it

becomes an overhead called a context switch overhead.

A process will switch because of an interrupt or a trap.

What’s the difference?

Interrupts

An interrupt is an “unplanned” function call to a system routine (aka, the interrupt

handler)

Unlike a normal function call, the interrupted thread cannot anticipate the control

transfer or prepare for it in any way

Control is later returned to the main thread at the interrupted instruction

Running process may be leaving up CPU for two reasons

o time slice has expired (clock interrupt) => PCB goes on ready queue

o process makes a blocking call => PCB goes on specified queue and state is set

to blocked


Types Interrupts

Several classes of interrupts exist:

o Program interrupts(trap), generated by a program such as an overflow,

division by zero etc.

o Clock Interrupts, generated by the CPU clock.

o I/O interrupt, generated by I/O device

o H/W failure, caused by a failure e.g. power, or parity error.

Traps

An interrupt is an exceptional event that is automatically handled by the interrupt handler.

In the case of an overflow, memory addressing violation, and the use of privileged instruction

in user mode, the handler will abort the program

These types of interrupts are called traps

All traps are going to be considered synchronous interrupts since they are generated by the

processor itself when it encounters an error inside the code stream.

I/O Interrupts

This type of interrupt occurs when a device sends a signal to inform the CPU that an I/O

operation has been completed.

An I/O flag is used to handle this type of interrupt

When an I/O interrupt occurs, the Program State of the running program is saved so that it

can be restarted from the same point after the interrupt has been handled.

Timer Interrupt


What if a program has an infinite loop?

We can add a time register, set to a specific value before a program stops, which is

decremented with each clock tick

When the timer reaches zero, the Timer Interrupt bit (TI) is set to “1”, indicating that a

timer interrupt has occurred and transferring control to the interrupt handler

Prevents a program from monopolizing the CPU

Interrupt Cycle

1. Suspend execution of current program

2. Save context

3. Set PC to start address of interrupt handler routine

4. Process interrupt

5. Restore context and continue interrupted program

Multiple interrupts

Disable interrupts

o Processor will ignore further interrupts whilst processing one interrupt.

They remain pending and are checked after first interrupt has been processed

Interrupts handled in sequence as they occur, however they can be prioritized:

Low priority interrupts can be interrupted by higher priority interrupts

When higher priority interrupt has been processed, processor returns to previous interrupt.

Context switch process

Steps:

1. Save the context of the processor, incl. PC and other registers.

2. Update the PCB of the currently running process. Includes changing the state of the

process, other fields and reason for changing the state.

3. Move the PCB of this process to the appropriate queue.

4. Select another process for execution.

5. Update the PCB of the process selected, incl. changing the state of this process to running.

6. Update memory mgt data structures. depends on how address translation is managed.

7. Restore the context of the processor to that which existed at the time the selected process

was switched out.


Execution of a new program from within the current program (with the return to the old

program)

int system(const char *string) ;

Syntax

main()

{

int status ;

. . . .

status = system( “new_prog par1 par2”) ;

Analysis of status

. . . .

}

Shell process (starting your program in foreground)

Current program The sam e process

New program . . . . system(name); . . . .

. . . . . . . . exit(0) ;

Returned status


Process operations (System Calls)

They are:

o Process creation :(fork(),

o Process Execution : exec(),

o Process wait (Blocked) : wait()

o Process Termination : exit ()

Fork()

A process is created using fork() system call.

This call makes an exact replica of process state except for PID.

The forked process is the “child” while the creator of the process is the parent.

UNIX implements through the fork() and exec() system calls an elegant two-step

mechanism for process creation and execution.

fork() is used to create the image of a process using the one of an existing one.

Accepts command line params (with progname “X”)

shell process (as a parent)

Switching to prog. “X”

child processfork()

execl(“X”, … )

Prog. “X”

exit()

wait()shell process is blocked

A new shell prompt appears


exec is used to execute a program by overwriting that image with the program's one.

Process creation

Once a parent creates a child process, a number of execution possibilities exist:

o The parent may immediately enter a wait state for the child to finish.

o The parent could immediately terminate;

o Both may continue to execute.

If the parent happens to terminate before the child has returned its value, then the child will

become a zombie process and may be listed as such in the process status list!

The init process(pid = 1) is the parent of all processes.it is responsible for bringing up a

Unix system after the kernel has been bootstrapped.

Initially, init duplicates (forks) several times and each child process replaces its code

(execs) with the code of the program they are running.

All orphaned child processes becomes a child of init when the parent process dies.

Example

A call to fork() of the form:

#include <sys/types.h>

pid_t childpid;

...

childpid = fork(); /* returns child's pid in the parent, 0 in

the child */

...

On creating new process:

The two processes obviously have two different process ids(pid).

In a C program process ids are conveniently represented by variables of pid_t type,

the type being defined in the sys/types.h header.

In UNIX the PCB of a process contains the id of the process's parent, - as parent id

(ppid) the pid of the process that called fork(),

Child process id is :pid.


Example 2:simpfork.c

main()

{

int i;

printf("simpfork: pid = %d\n", getpid());

i = fork();

printf("Did a fork. It returned %d. getpid = %d. getppid

= %d\n", i, getpid(), getppid());

}

When it is run, the following happens:

o UNIX> simpfork

simpfork: pid = 914 Did a fork. It returned 915. getpid = 914. getppid = 381

Did a fork. It returned 0. getpid = 915. getppid = 914

o UNIX>

When simpfork is executed, it has a pid of 914. Next it calls fork() creating a duplicate

process with a pid of 915. The parent gains control of the CPU, and returns from fork()

with a return value of the 915 -- this is the child's pid. It prints out this return value, its own

pid, and the pid of csh, which is still 381. Then it exits. Next, the child gets the CPU and

returns from fork() with a value of 0. It prints out that value, its pid, and the pid of the

parent

getpid() and getppid()

In order to know the pid of the child system call named getpid() is provided for this

purpose, and

another one, named getppid() is used to ask the system about the parent's id.

Both functions take no arguments and return the requested value in pid_t type, or -1 in case

of failure.


wait() system call

UNIX defines several sophisticated inter-process communication (IPC) mechanisms, the

simplest of which is a parent's ability to test the termination status of its children.

A synchronization mechanism is provided via the wait() system call, that allows a parent to

sleep until one of its children exits, and then get its exit status.

When wait() is called, the process is suspended until one of its child processes exits, and

then the call returns with the exit status of the child process.

State diagram of unix process

Process termination


Once a process executes its final instruction, a call to exit() is made.

The operating system then flushes any I/O buffers.

All resources such as physical and virtual memory, I/O buffers, and open files are

deallocated and returned to the operating system.

A parent may terminate execution of children processes by calling abort().

Example : exit()

Note: since wait() returns the exit status multiplied by 256 (contained in the upper 8 bits), the status

value is shifted right 8 bits (divided by 256) to obtain the correct value.

Example 2; Process system call- execlp( ) /* myshell.c This program is a simple command interpreter that uses execlp() to execute commands typed in by the user. */ #include <stdio.h> #define EVER ;; int main() { int process; char line[81]; for (EVER) {


fprintf(stderr, "cmd: "); if ( gets (line) == (char *) NULL) /* blank line input */ exit (0); /* create a new process */ process = fork (); if (process > 0) /* parent */ wait((int *) 0); /* null pointer - return value not saved */ else if (process == 0) /* child */ { /* execute program */ execlp (line, line, (char *) NULL); /* some problem if exec returns */ fprintf (stderr, "Can't execute %s\n", line); exit (1); } else if ( process == -1) /* can't create a new process */ { fprintf (stderr, "Can't fork!\n"); exit (2); } } }

Inter-Process Communication Mechanisms for IPC

Two basic IPC mechanism:

o Shared memory

o Message passing

Shared memory requires that these processes share a common buffer pool.

The code for implementing the buffer can be written explicitly by the application programmer.

Another way to achieve the same effect is for the operating system to provide the means for

cooperating processes to communicate with each other via an inter process communication

(IPC) facility.

IPC provides a mechanism to allow processes to communicate and to synchronize their actions

without sharing the same address space.

IPC is particularly useful in a distributed environment where the communicating processes

may reside on different computers connected with a network. An example is a chat program

used on the World Wide Web.


Shared Memory

Shared memory offers an extremely fast way of communicating; any data written by one

process to a shared memory region can be read immediately by any other process that has

mapped that region into its address space.

To obtain synchronization, however, shared memory must be used in conjunction with another

Interprocess-communication mechanism.

Steps in creating and using a shared memory

One of the processes creates a shared segment. Other processes get ID of the created

segment.

Each process, using the same key, attaches to itself the created shared segment.

Now each process may write and read data into the shared segment.

To avoid possible race condition, processes should coordinate using shared memory.

Each process detaches the segment (after finishing).

Normally, UNIX prevents a user process from accessing any data in memory belonging to another

process.

System calls for shared memory -shmget()

shmget()- creates a shm segment

Creating a shared segment or getting its ID

Address space of

P1

Ptr1 mapping

Shared memory segment

Process P1 Process P2

Address space of

P2

Ptr2


The key argument is an access value associated with the semaphore ID.

The size argument is the size in bytes of the requested shared memory.

The flag argument specifies the initial access permissions and creation control flags.

When the call succeeds, it returns the shared memory segment ID. This call is also used to

get the ID of an existing shared segment (from a process requesting sharing of some

existing memory portion).

System calls for Attaching to shared memory - Shmat()

Suppose process 1, a server, uses shmget() to request a shared memory segment

successfully. That shared memory segment exists somewhere in the memory, but is not yet

part of the address space of process 1 (shown with dashed line below). Similarly, if process

N um eric key o f type in t

S ize of the se gm en t in b y tes , in t

in t

id = shm ge t ( ke y, s ize , f la g ) ;

Shared m e m ory is allo ca ted bu t N OT m apped …

IPC_C RE AT | 06 6 6 To crea te n ew segm en t w ith rw -rw -rw- O r 0 t o g et an ID o f ex isting segm en t

Process A Process B

pointer

SharedSegment

OS Kernel Key

123

System wide information

pointer


2 requests the same shared memory segment with the same key value, process 2 will be

granted the right to use the shared memory segment; but it is not yet part of the address

space of process 2. To make a requested shared memory segment part of the address space

of a process, use shmat().

Attaching (mapping) the existing memory segment (getting its pointer).

shmat() returns a pointer address, to the head of the shared segment associated with a valid

shared memory id from shmget().

shmdt() detaches the shared memory segment located at the address indicated by addr

shmdt ( ptr ) ;

Where ptr- is result of shmat()

Features of shared memory

Efficiency (no intermediate copying of data as in pipes)

Random access (a sequential byte stream in pipes)

Many-to-many mechanism of IPC: many processes may attach the same segment (pipes

provide one-to-one mechanism of IPC)

No synchronization is provided (pipes provide a synchronization) – Disadvantage.

NB: Pipes are used extensively in message passing

From shmget() Address to map at (typically

0, let system choose)

Starting address of shared segment

ptr = shmat ( id, addr, flag ) ;

Now shared memory is mapped

Typically 0, or SHM_RDONLY


Process A Process B

#include …

#define MSIZ 27

main()

{ char c ; int shmid ;

key_t key = 123 ;

char *shm, *s ;

if ( (shmid = shmget( key,MSIZ,IPC_CREAT | 0666) )

< 0 )

{error…; exit(1) ; }

if ( (shm = shmat( shmid, NULL, 0)) == (char *) –1 )

(error … ; exit(1) ; }

/*Put data into shared memory */

s = shm ;

for (c = ‘a’ ; c <= ‘z’ ; c++ )

*s++ = c ;

*s = ‘\0’ ;

/*wait until process B changes the first character */

while(*shm != ‘*’ ) sleep( 1 );

shmdt( shm ) ;

exit( 0 ) ;

}

#include …

#define MSIZ 27

main()

{ char c ; int shmid ;

key_t key = 123 ;

char *shm, *s ;

if ( (shmid = shmget( key,0) ) < 0 )

{error … ; exit(1) ; }

if ( (shm = shmat( shmid, NULL, 0)) == (char *) –1 )

(error … ; exit(1) ; }

/* get from shared memory */

for (s=shm ; *s != ‘\0’ ; s++ )

putchar( *s ) ; / * to display */

putchar( ‘\n’) ;

/* Change the first char in segment */

*shm = ‘*’ ;

shmdt ( shm ) ; /* detach */

exit( 0 ) ;

}

File 11_5.c File 11_6.c

Message Passing IPC

The function of a message system is to allow processes to communicate with one another

without the need to resort to shared data. An IPC facility provides at least the two operations:

send( dest, &message)

receive( source, &message).

If processes P and Q want to communicate, they must send messages to and receive messages

from each other; a communication link must exist between them.


IPC local – takes place in the same machine.

IPC Remote - takes place in the different machines.

Process Synchronization Since processes frequently needs to communicate with other processes therefore, there is a need for

a well Structured communication, in order to avoid synchronization problems such as Race

Condition (shared memory)

In operating systems, processes that are working together share some common storage

(main memory, file etc.) that each process can read and write. When two or more

Host

Process A Process B

OS kernel

request reply

OS kernel

Network

Host 1

Process A Process B

OS kernel

Host 2

request

Client

Server

reply


processes are reading or writing some shared data and the final result depends on who

runs precisely when, are called race conditions.

A race condition is a flaw in a process whereby the output and/or result of the process is

unexpectedly and critically dependent on the sequence or timing of other events.

Concurrently executing threads that share data need to synchronize their operations and

processing in order to avoid race condition on shared data. Only one ‘customer’ thread at a

time should be allowed to examine and update the shared variable.

Race Conditions

Race conditions are also possible in Operating Systems. If the ready queue is implemented

as a linked list and if the ready queue is being manipulated during the handling of an

interrupt, then interrupts must be disabled to prevent another interrupt before the first one

completes. If interrupts are not disabled than the linked list could become corrupt

Critical section problem

The critical section of a program is a fragment, which performs the access to a shared

resource (such as a common variable).

No two processes should enter their critical section at the same time, this causes what is

called race condition. A race condition occurs when multiple processes access and

manipulate the same data concurrently, and the outcome of the execution depends on the

particular order in which the access takes place.

To avoid this condition we need some form of synchronization .

The key to preventing trouble involving shared memory is find some way to prohibit more

than one process from reading and writing the shared data simultaneously. That part of the

program where the shared memory is accessed is called the Critical Section. To avoid race

conditions and flawed results, one must identify codes in Critical Sections in each thread.

The characteristic properties of the code that form a Critical Section are

Codes that reference one or more variables in a “read-update-write” fashion while any of

those variables is possibly being altered by another thread.

Codes that alter one or more variables that are possibly being referenced in “read-update-

write” fashion by another thread.

Codes use a data structure while any part of it is possibly being altered by another thread.

Codes alter any part of a data structure while it is possibly in use by another thread.


Here, the important point is that when one process is executing shared modifiable data in its

critical section, no other process is to be allowed to execute in its critical section. Thus, the

execution of critical sections by the processes is mutually exclusive in time

Mutual Exclusion Conditions

If we could arrange matters such that no two processes were ever in their critical sections

simultaneously, we could avoid race conditions. We need four conditions to hold to

have a good solution for the critical section problem (mutual exclusion).

No two processes may at the same moment inside their critical sections.

No assumptions are made about relative speeds of processes or number of CPUs.

No process should outside its critical section should block other processes.

No process should wait arbitrary long to enter its critical section.

Semaphores

A semaphore is an integer variables that restricts access to shared resources. Main purpose

is to synchronize the access of processes to shared resources (files, shared memory), by

enforcing mutual exclusion to the resource.

Monitors

A higher-level abstraction that provides a convenient and effective mechanism for process

synchronization

Monitors are embedded in some concurrent programming languages. Java has monitors.

Monitors enforce a style of programming where complex synchronization code doesn’t get

mixed with other code: it is separated and put in monitors.


POSIX THREADS PROGRAMMING

Pthreads Overview

What is a Thread?

• A thread is defined as an independent stream of instructions that can be scheduled to run as

such by the operating system.

• To the software developer, the concept of a "procedure" that runs independently from its

main program may best describe a thread.

• To go one step further, imagine a main program (a.out) that contains a number of

procedures. Then imagine all of these procedures being able to be scheduled to run

simultaneously and/or independently by the operating system. That would describe a

"multi-threaded" program.

• Before understanding a thread, one first needs to understand a UNIX process. A process is

created by the operating system, and requires a fair amount of "overhead". Processes

contain information about program resources and program execution state, including:

o Process ID, process group ID, user ID, and group ID

o Environment

o Working directory.

o Program instructions

o Registers

o Stack

o Heap

o File descriptors

o Signal actions

o Shared libraries

o Inter-process communication tools (such as message queues, pipes, semaphores, or

shared memory).


Figure: Unix process

Figure : Threads Within a Unix Process


• Threads use and exist within these process resources, yet are able to be scheduled by the

operating system and run as independent entities largely because they duplicate only the

bare essential resources that enable them to exist as executable code.

• This independent flow of control is accomplished because a thread maintains its own:

o Stack pointer

o Registers

o Scheduling properties (such as policy or priority)

o Set of pending and blocked signals

o Thread specific data.

• So, in summary, in the UNIX environment a thread:

o Exists within a process and uses the process resources

o Has its own independent flow of control as long as its parent process exists and the

OS supports it

o Duplicates only the essential resources it needs to be independently schedulable

o May share the process resources with other threads that act equally independently

(and dependently)

o Dies if the parent process dies - or something similar

o Is "lightweight" because most of the overhead has already been accomplished

through the creation of its process.

• Because threads within the same process share resources:

o Changes made by one thread to shared system resources (such as closing a file) will

be seen by all other threads.

o Two pointers having the same value point to the same data.

o Reading and writing to the same memory locations is possible, and therefore

requires explicit synchronization by the programmer.

Pthreads Overview What are Pthreads?

• Historically, hardware vendors have implemented their own proprietary versions of threads.

These implementations differed substantially from each other making it difficult for

programmers to develop portable threaded applications.


• In order to take full advantage of the capabilities provided by threads, a standardized

programming interface was required.

o For UNIX systems, this interface has been specified by the IEEE POSIX 1003.1c

standard (1995).

o Implementations adhering to this standard are referred to as POSIX threads, or

Pthreads.

o Most hardware vendors now offer Pthreads in addition to their proprietary API's.

• The POSIX standard has continued to evolve and undergo revisions, including the Pthreads

specification.

• Pthreads are defined as a set of C language programming types and procedure calls,

implemented with a pthread.h header/include file and a thread library - though this library

may be part of another library, such as libc, in some implementations.

Why Pthreads?

• The primary motivation for using Pthreads is to realize potential program performance

gains.

• When compared to the cost of creating and managing a process, a thread can be created

with much less operating system overhead. Managing threads requires fewer system

resources than managing processes.

For example, the following table compares timing results for the fork() subroutine and the

pthread_create() subroutine. Timings reflect 50,000 process/thread creations, were

performed with the time utility, and units are in seconds, no optimization flags.

Note: don't expect the system and user times to add up to real time, because these are SMP

systems with multiple CPUs working on the problem at the same time. At best, these are

approximations run on local machines, past and present.

Platform fork() pthread_create()

real user sys real user sys

Intel 2.6 GHz Xeon E5-2670 (16cpus/node) 8.1 0.1 2.9 0.9 0.2 0.3

Intel 2.8 GHz Xeon 5660 (12cpus/node) 4.4 0.4 4.3 0.7 0.2 0.5

AMD 2.3 GHz Opteron (16cpus/node) 12.5 1.0 12.5 1.2 0.2 1.3


AMD 2.4 GHz Opteron (8cpus/node) 17.6 2.2 15.7 1.4 0.3 1.3

IBM 4.0 GHz POWER6 (8cpus/node) 9.5 0.6 8.8 1.6 0.1 0.4

IBM 1.9 GHz POWER5 p5-575 (8cpus/node) 64.2 30.7 27.6 1.7 0.6 1.1

IBM 1.5 GHz POWER4 (8cpus/node) 104.5 48.6 47.2 2.1 1.0 1.5

INTEL 2.4 GHz Xeon (2 cpus/node) 54.9 1.5 20.8 1.6 0.7 0.9

INTEL 1.4 GHz Itanium2 (4 cpus/node) 54.5 1.1 22.2 2.0 1.2 0.6

• All threads within a process share the same address space. Inter-thread communication is

more efficient and in many cases, easier to use than inter-process communication.

• Threaded applications offer potential performance gains and practical advantages over non-

threaded applications in several other ways:

o Overlapping CPU work with I/O: For example, a program may have sections where

it is performing a long I/O operation. While one thread is waiting for an I/O system

call to complete, CPU intensive work can be performed by other threads.

o Priority/real-time scheduling: tasks which are more important can be scheduled to

supersede or interrupt lower priority tasks.

o Asynchronous event handling: tasks which service events of indeterminate

frequency and duration can be interleaved. For example, a web server can both

transfer data from previous requests and manage the arrival of new requests.

• The primary motivation for considering the use of Pthreads on an SMP architecture is to

achieve optimum performance. In particular, if an application is using MPI for on-node

communications, there is a potential that performance could be greatly improved by using

Pthreads for on-node data transfer instead.

• For example:

o MPI libraries usually implement on-node task communication via shared memory,

which involves at least one memory copy operation (process to process).

o For Pthreads there is no intermediate memory copy required because threads share

the same address space within a single process. There is no data transfer, per se. It

becomes more of a cache-to-CPU or memory-to-CPU bandwidth (worst case)

situation. These speeds are much higher.


Designing Threaded Programs Parallel Programming:

• On modern, multi-cpu machines, pthreads are ideally suited for parallel programming, and

whatever applies to parallel programming in general, applies to parallel pthreads programs.

• There are many considerations for designing parallel programs, such as:

o What type of parallel programming model to use?

o Problem partitioning

o Load balancing

o Communications

o Data dependencies

o Synchronization and race conditions

o Memory issues

o I/O issues

o Program complexity

o Programmer effort/costs/time

• In general though, in order for a program to take advantage of Pthreads, it must be able to

be organized into discrete, independent tasks which can execute concurrently. For example,

if routine1 and routine2 can be interchanged, interleaved and/or overlapped in real time,

they are candidates for threading.

• Programs having the following characteristics may be well suited for pthreads:

o Work that can be executed, or data that can be operated on, by multiple tasks

simultaneously


o Block for potentially long I/O waits

o Use many CPU cycles in some places but not others

o Must respond to asynchronous events

o Some work is more important than other work (priority interrupts)

• Pthreads can also be used for serial applications, to emulate parallel execution. A perfect

example is the typical web browser, which for most people, runs on a single cpu

desktop/laptop machine. Many things can "appear" to be happening at the same time.

• Several common models for threaded programs exist:

o Manager/worker: a single thread, the manager assigns work to other threads, the

workers. Typically, the manager handles all input and parcels out work to the other

tasks. At least two forms of the manager/worker model are common: static worker

pool and dynamic worker pool.

o Pipeline: a task is broken into a series of suboperations, each of which is handled in

series, but concurrently, by a different thread. An automobile assembly line best

describes this model.

o Peer: similar to the manager/worker model, but after the main thread creates other

threads, it participates in the work.

Shared Memory Model:

• All threads have access to the same global, shared memory

• Threads also have their own private data

• Programmers are responsible for synchronizing access (protecting) globally shared data.


Thread-safeness:

• Thread-safeness: in a nutshell, refers an application's ability to execute multiple threads

simultaneously without "clobbering" shared data or creating "race" conditions.

• For example, suppose that your application creates several threads, each of which makes a

call to the same library routine:

o This library routine accesses/modifies a global structure or location in memory.

o As each thread calls this routine it is possible that they may try to modify this global

structure/memory location at the same time.

o If the routine does not employ some sort of synchronization constructs to prevent

data corruption, then it is not thread-safe.


• The implication to users of external library routines is that if you aren't 100% certain the

routine is thread-safe, then you take your chances with problems that could arise.

• Recommendation: Be careful if your application uses libraries or other objects that don't

explicitly guarantee thread-safeness. When in doubt, assume that they are not thread-safe

until proven otherwise. This can be done by "serializing" the calls to the uncertain routine,

etc.

The Pthreads API • The original Pthreads API was defined in the ANSI/IEEE POSIX 1003.1 - 1995 standard.

The POSIX standard has continued to evolve and undergo revisions, including the Pthreads

specification.

• Copies of the standard can be purchased from IEEE or downloaded for free from other sites

online.

• The subroutines which comprise the Pthreads API can be informally grouped into four

major groups:

1. Thread management: Routines that work directly on threads - creating, detaching,

joining, etc. They also include functions to set/query thread attributes (joinable,

scheduling etc.)


2. Mutexes: Routines that deal with synchronization, called a "mutex", which is an

abbreviation for "mutual exclusion". Mutex functions provide for creating,

destroying, locking and unlocking mutexes. These are supplemented by mutex

attribute functions that set or modify attributes associated with mutexes.

3. Condition variables: Routines that address communications between threads that

share a mutex. Based upon programmer specified conditions. This group includes

functions to create, destroy, wait and signal based upon specified variable values.

Functions to set/query condition variable attributes are also included.

4. Synchronization: Routines that manage read/write locks and barriers.

• Naming conventions: All identifiers in the threads library begin with pthread_. Some

examples are shown below.

Routine Prefix Functional Group

pthread_ Threads themselves and miscellaneous subroutines

pthread_attr_ Thread attributes objects

pthread_mutex_ Mutexes

pthread_mutexattr_ Mutex attributes objects.

pthread_cond_ Condition variables

pthread_condattr_ Condition attributes objects

pthread_key_ Thread-specific data keys

pthread_rwlock_ Read/write locks

pthread_barrier_ Synchronization barriers

• The concept of opaque objects pervades the design of the API. The basic calls work to

create or modify opaque objects - the opaque objects can be modified by calls to attribute

functions, which deal with opaque attributes.

• The Pthreads API contains around 100 subroutines. This tutorial will focus on a subset of

these - specifically, those which are most likely to be immediately useful to the beginning

Pthreads programmer.


• For portability, the pthread.h header file should be included in each source file using the

Pthreads library.

• The current POSIX standard is defined only for the C language. Fortran programmers can

use wrappers around C function calls. Some Fortran compilers (like IBM AIX Fortran) may

provide a Fortram pthreads API.

Compiling Threaded Programs

• Several examples of compile commands used for pthreads codes are listed in the table

below.

Compiler / Platform Compiler Command Description

INTEL

Linux

icc -pthread C

icpc -pthread C++

PGI

Linux

pgcc -lpthread C

pgCC -lpthread C++

GNU

Linux, Blue Gene

gcc -pthread GNU C

g++ -pthread GNU C++

IBM

Blue Gene

bgxlc_r / bgcc_r C (ANSI / non-ANSI)

bgxlC_r, bgxlc++_r C++

Thread Management

Creating and Terminating Threads

Routines:

• pthread_create (thread,attr,start_routine,arg)

• pthread_exit (status)

• pthread_cancel (thread)

• pthread_attr_init (attr)

• pthread_attr_destroy (attr)


Creating Threads:

• Initially, your main() program comprises a single, default thread. All other threads must be

explicitly created by the programmer.

• pthread_create creates a new thread and makes it executable. This routine can be called any

number of times from anywhere within your code.

• pthread_create arguments:

o thread: An opaque, unique identifier for the new thread returned by the subroutine.

o attr: An opaque attribute object that may be used to set thread attributes. You can

specify a thread attributes object, or NULL for the default values.

o start_routine: the C routine that the thread will execute once it is created.

o arg: A single argument that may be passed to start_routine. It must be passed by

reference as a pointer cast of type void. NULL may be used if no argument is to be

passed.

• The maximum number of threads that may be created by a process is implementation

dependent.

• Once created, threads are peers, and may create other threads. There is no implied hierarchy

or dependency between threads.

Thread Attributes:


• By default, a thread is created with certain attributes. Some of these attributes can be

changed by the programmer via the thread attribute object.

• pthread_attr_init and pthread_attr_destroy are used to initialize/destroy the thread attribute

object.

• Other routines are then used to query/set specific attributes in the thread attribute object.

Attributes include:

o Detached or joinable state

o Scheduling inheritance

o Scheduling policy

o Scheduling parameters

o Scheduling contention scope

o Stack size

o Stack address

o Stack guard (overflow) size

• Some of these attributes will be discussed later.

Thread Binding and Scheduling:

Question: After a thread has been created, how do you know a)when it will be scheduled to run

by the operating system, and b)which processor/core it will run on?

• The Pthreads API provides several routines that may be used to specify how threads are

scheduled for execution. For example, threads can be scheduled to run FIFO (first-in first-

out), RR (round-robin) or OTHER (operating system determines). It also provides the

ability to set a thread's scheduling priority value.

• The Pthreads API does not provide routines for binding threads to specific cpus/cores.

However, local implementations may include this functionality - such as providing the non-

standard pthread_setaffinity_np routine. Note that "_np" in the name stands for "non-

portable".

• Also, the local operating system may provide a way to do this. For example, Linux provides

the sched_setaffinity routine.

Terminating Threads & pthread_exit():

• There are several ways in which a thread may be terminated:

https://computing.llnl.gov/tutorials/pthreads/man/sched_setaffinity.txt�


o The thread returns normally from its starting routine. It's work is done.

o The thread makes a call to the pthread_exit subroutine - whether its work is done or

not.

o The thread is canceled by another thread via the pthread_cancel routine.

o The entire process is terminated due to making a call to either the exec() or exit()

o If main() finishes first, without calling pthread_exit explicitly itself

• The pthread_exit() routine allows the programmer to specify an optional termination status

parameter. This optional parameter is typically returned to threads "joining" the terminated

thread (covered later).

• In subroutines that execute to completion normally, you can often dispense with calling

pthread_exit() - unless, of course, you want to pass the optional status code back.

• Cleanup: the pthread_exit() routine does not close files; any files opened inside the thread

will remain open after the thread is terminated.

• Discussion on calling pthread_exit() from main():

o There is a definite problem if main() finishes before the threads it spawned if you

don't call pthread_exit() explicitly. All of the threads it created will terminate

because main() is done and no longer exists to support the threads.

o By having main() explicitly call pthread_exit() as the last thing it does, main() will

block and be kept alive to support the threads it created until they are done.

Example: Pthread Creation and Termination

• This simple example code creates 5 threads with the pthread_create() routine. Each thread

prints a "Hello World!" message, and then terminates with a call to pthread_exit().

Example Code - Pthread Creation and Termination

#include <pthread.h>


#include <stdio.h>

#define NUM_THREADS 5

void *PrintHello(void *threadid)

{

long tid;

tid = (long)threadid;

printf("Hello World! It's me, thread #%ld!\n", tid);

pthread_exit(NULL);

}

int main (int argc, char *argv[])

{

pthread_t threads[NUM_THREADS];

int rc;

long t;

for(t=0; t<NUM_THREADS; t++){

printf("In main: creating thread %ld\n", t);

rc = pthread_create(&threads[t], NULL, PrintHello, (void

*)t);

if (rc){

printf("ERROR; return code from pthread_create() is

%d\n", rc);

exit(-1);

}

}

/* Last thing that main() should do */

pthread_exit(NULL);

}


Passing Arguments to Threads

• The pthread_create() routine permits the programmer to pass one argument to the thread

start routine. For cases where multiple arguments must be passed, this limitation is easily

overcome by creating a structure which contains all of the arguments, and then passing a

pointer to that structure in the pthread_create() routine.

• All arguments must be passed by reference and cast to (void *).

Question: How can you safely pass data to newly created threads, given their non-

deterministic start-up and scheduling?

Example 1 - Thread Argument Passing

This code fragment demonstrates how to pass a simple integer to each thread. The

calling thread uses a unique data structure for each thread, insuring that each

thread's argument remains intact throughout the program.

long *taskids[NUM_THREADS];

for(t=0; t<NUM_THREADS; t++)

{

taskids[t] = (long *) malloc(sizeof(long));

*taskids[t] = t;

printf("Creating thread %ld\n", t);


*) taskids[t]);

...

}


Example 2 - Thread Argument Passing

This example shows how to setup/pass multiple arguments via a structure. Each

thread receives a unique instance of the structure.

struct thread_data{

int thread_id;

int sum;

char *message;

};

struct thread_data thread_data_array[NUM_THREADS];

void *PrintHello(void *threadarg)

{

struct thread_data *my_data;

...

my_data = (struct thread_data *) threadarg;

taskid = my_data->thread_id;

sum = my_data->sum;

hello_msg = my_data->message;

...

}


{

...

thread_data_array[t].thread_id = t;

thread_data_array[t].sum = sum;

thread_data_array[t].message = messages[t];

rc = pthread_create(&threads[t], NULL, PrintHello,

(void *) &thread_data_array[t]);

...

}

Example 3 - Thread Argument Passing (Incorrect)

This example performs argument passing incorrectly. It passes the address of


variable t, which is shared memory space and visible to all threads. As the loop

iterates, the value of this memory location changes, possibly before the created

threads can access it.

int rc;

long t;

for(t=0; t<NUM_THREADS; t++)

{

printf("Creating thread %ld\n", t);


*) &t);

...

}

Joining and Detaching Threads

Routines:

• pthread_join (threadid,status)

• pthread_detach (threadid)

• pthread_attr_setdetachstate (attr,detachstate)

• pthread_attr_getdetachstate (attr,detachstate)

Joining:

• "Joining" is one way to accomplish synchronization between threads. For example:


• The pthread_join() subroutine blocks the calling thread until the specified threadid thread

terminates.

• The programmer is able to obtain the target thread's termination return status if it was

specified in the target thread's call to pthread_exit().

• A joining thread can match one pthread_join() call. It is a logical error to attempt multiple

joins on the same thread.

• Two other synchronization methods, mutexes and condition variables, will be discussed

later.

Joinable or Not?

• When a thread is created, one of its attributes defines whether it is joinable or detached.

Only threads that are created as joinable can be joined. If a thread is created as detached, it

can never be joined.

• The final draft of the POSIX standard specifies that threads should be created as joinable.

• To explicitly create a thread as joinable or detached, the attr argument in the

pthread_create() routine is used. The typical 4 step process is:

1. Declare a pthread attribute variable of the pthread_attr_t data type

2. Initialize the attribute variable with pthread_attr_init()

3. Set the attribute detached status with pthread_attr_setdetachstate()

4. When done, free library resources used by the attribute with pthread_attr_destroy()

Detaching:

• The pthread_detach() routine can be used to explicitly detach a thread even though it was

created as joinable.

• There is no converse routine.

Recommendations:

• If a thread requires joining, consider explicitly creating it as joinable. This provides

portability as not all implementations may create threads as joinable by default.


• If you know in advance that a thread will never need to join with another thread, consider

creating it in a detached state. Some system resources may be able to be freed.

Example: Pthread Joining

Example Code - Pthread Joining

This example demonstrates how to "wait" for thread completions by using the Pthread

join routine. Since some implementations of Pthreads may not create threads in a

joinable state, the threads in this example are explicitly created in a joinable state so that

they can be joined later.


#include <stdio.h>

#include <stdlib.h>

#include <math.h>


void *BusyWork(void *t)

{

int i;

long tid;

double result=0.0;

tid = (long)t;

printf("Thread %ld starting...\n",tid);

for (i=0; i<1000000; i++)

{

result = result + sin(i) * tan(i);

}

printf("Thread %ld done. Result = %e\n",tid, result);

pthread_exit((void*) t);

}



{

pthread_t thread[NUM_THREADS];

pthread_attr_t attr;

int rc;

long t;

void *status;

/* Initialize and set thread detached attribute */

pthread_attr_init(&attr);

pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);

for(t=0; t<NUM_THREADS; t++) {

printf("Main: creating thread %ld\n", t);

rc = pthread_create(&thread[t], &attr, BusyWork, (void

*)t);

if (rc) {

printf("ERROR; return code from pthread_create()

is %d\n", rc);

exit(-1);

}

}

/* Free attribute and wait for the other threads */

pthread_attr_destroy(&attr);

for(t=0; t<NUM_THREADS; t++) {

rc = pthread_join(thread[t], &status);

if (rc) {

printf("ERROR; return code from pthread_join()

is %d\n", rc);

exit(-1);

}

printf("Main: completed join with thread %ld having a status


of %ld\n",t,(long)status);

}

printf("Main: program completed. Exiting.\n");

pthread_exit(NULL);

}

Stack Management

Routines:

• pthread_attr_getstacksize (attr, stacksize)

• pthread_attr_setstacksize (attr, stacksize)

• pthread_attr_getstackaddr (attr, stackaddr)

• pthread_attr_setstackaddr (attr, stackaddr)

Preventing Stack Problems:

• The POSIX standard does not dictate the size of a thread's stack. This is implementation

dependent and varies.

• Exceeding the default stack limit is often very easy to do, with the usual results: program

termination and/or corrupted data.

• Safe and portable programs do not depend upon the default stack limit, but instead,

explicitly allocate enough stack for each thread by using the pthread_attr_setstacksize

routine.

• The pthread_attr_getstackaddr and pthread_attr_setstackaddr routines can be used by

applications in an environment where the stack for a thread must be placed in some

particular region of memory.

Some Practical Examples at LC:

• Default thread stack size varies greatly. The maximum size that can be obtained also varies

greatly, and may depend upon the number of threads per node.

• Both past and present architectures are shown to demonstrate the wide variation in default

thread stack size.


Node

Architecture

#CPUs Memory (GB) Default Size

(bytes)

Intel Xeon E5-2670 16 32 2,097,152

Intel Xeon 5660 12 24 2,097,152

AMD Opteron 8 16 2,097,152

Intel IA64 4 8 33,554,432

Intel IA32 2 4 2,097,152

IBM Power5 8 32 196,608

IBM Power4 8 16 196,608

IBM Power3 16 16 98,304

Example: Stack Management

Example Code - Stack Management

This example demonstrates how to query and set a thread's stack size.


#include <stdio.h>

#define NTHREADS 4

#define N 1000

#define MEGEXTRA 1000000


void *dowork(void *threadid)

{

double A[N][N];

int i,j;

long tid;

size_t mystacksize;


tid = (long)threadid;

pthread_attr_getstacksize (&attr, &mystacksize);

printf("Thread %ld: stack size = %li bytes \n", tid,

mystacksize);

for (i=0; i<N; i++)

for (j=0; j<N; j++)

A[i][j] = ((i*j)/3.452) + (N-i);

pthread_exit(NULL);

}

int main(int argc, char *argv[])

{

pthread_t threads[NTHREADS];

size_t stacksize;

int rc;

long t;


pthread_attr_getstacksize (&attr, &stacksize);

printf("Default stack size = %li\n", stacksize);

stacksize = sizeof(double)*N*N+MEGEXTRA;

printf("Amount of stack needed per thread = %li\n",stacksize);

pthread_attr_setstacksize (&attr, stacksize);

printf("Creating threads with stack size = %li

bytes\n",stacksize);

for(t=0; t<NTHREADS; t++){

rc = pthread_create(&threads[t], &attr, dowork, (void *)t);

if (rc){

printf("ERROR; return code from pthread_create() is %d\n", rc);

exit(-1);

}


}

printf("Created %ld threads.\n", t);

pthread_exit(NULL);

}

Miscellaneous Routines

• pthread_self ()

• pthread_equal (thread1,thread2)

• pthread_self returns the unique, system assigned thread ID of the calling thread.

• pthread_equal compares two thread IDs. If the two IDs are different 0 is returned, otherwise

a non-zero value is returned.

• Note that for both of these routines, the thread identifier objects are opaque and can not be

easily inspected. Because thread IDs are opaque objects, the C language equivalence

operator == should not be used to compare two thread IDs against each other, or to compare

a single thread ID against another value.

pthread_once (once_control, init_routine)

• pthread_once executes the init_routine exactly once in a process. The first call to this

routine by any thread in the process executes the given init_routine, without parameters.

Any subsequent call will have no effect.

• The init_routine routine is typically an initialization routine.

• The once_control parameter is a synchronization control structure that requires initialization

prior to calling pthread_once. For example:

pthread_once_t once_control = PTHREAD_ONCE_INIT;


Mutex Variables

Overview

• Mutex is an abbreviation for "mutual exclusion". Mutex variables are one of the primary

means of implementing thread synchronization and for protecting shared data when

multiple writes occur.

• A mutex variable acts like a "lock" protecting access to a shared data resource. The basic

concept of a mutex as used in Pthreads is that only one thread can lock (or own) a mutex

variable at any given time. Thus, even if several threads try to lock a mutex only one thread

will be successful. No other thread can own that mutex until the owning thread unlocks that

mutex. Threads must "take turns" accessing protected data.

• Mutexes can be used to prevent "race" conditions. An example of a race condition

involving a bank transaction is shown below:

Thread 1 Thread 2 Balance

Read balance: $1000 $1000

Read balance: $1000 $1000

Deposit $200 $1000

Deposit $200 $1000

Update balance $1000+$200 $1200

Update balance $1000+$200 $1200

• In the above example, a mutex should be used to lock the "Balance" while a thread is using

this shared data resource.

• Very often the action performed by a thread owning a mutex is the updating of global

variables. This is a safe way to ensure that when several threads update the same variable,

the final value is the same as what it would be if only one thread performed the update. The

variables being updated belong to a "critical section".

• A typical sequence in the use of a mutex is as follows:

o Create and initialize a mutex variable

o Several threads attempt to lock the mutex

o Only one succeeds and that thread owns the mutex


o The owner thread performs some set of actions

o The owner unlocks the mutex

o Another thread acquires the mutex and repeats the process

o Finally the mutex is destroyed

• When several threads compete for a mutex, the losers block at that call - an unblocking call

is available with "trylock" instead of the "lock" call.

• When protecting shared data, it is the programmer's responsibility to make sure every

thread that needs to use a mutex does so. For example, if 4 threads are updating the same

data, but only one uses a mutex, the data can still be corrupted.

Creating and Destroying Mutexes

Routines:

• pthread_mutex_init (mutex,attr)

• pthread_mutex_destroy (mutex)

• pthread_mutexattr_init (attr)

• pthread_mutexattr_destroy (attr)

Usage:

• Mutex variables must be declared with type pthread_mutex_t, and must be initialized before

they can be used. There are two ways to initialize a mutex variable:

1. Statically, when it is declared. For example:

pthread_mutex_t mymutex = PTHREAD_MUTEX_INITIALIZER;

2. Dynamically, with the pthread_mutex_init() routine. This method permits setting

mutex object attributes, attr.

The mutex is initially unlocked.

• The attr object is used to establish properties for the mutex object, and must be of type

pthread_mutexattr_t if used (may be specified as NULL to accept defaults). The Pthreads

standard defines three optional mutex attributes:

o Protocol: Specifies the protocol used to prevent priority inversions for a mutex.

o Prioceiling: Specifies the priority ceiling of a mutex.

o Process-shared: Specifies the process sharing of a mutex.

Note that not all implementations may provide the three optional mutex attributes.

The pthread_mutexattr_init() and pthread_mutexattr_destroy() routines are used to create

and destroy mutex attribute objects respectively.


pthread_mutex_destroy() should be used to free a mutex object which is no longer needed.

Locking and Unlocking Mutexes

Routines:

• pthread_mutex_lock (mutex)

• pthread_mutex_trylock (mutex)

• pthread_mutex_unlock (mutex)

Usage:

• The pthread_mutex_lock() routine is used by a thread to acquire a lock on the specified

mutex variable. If the mutex is already locked by another thread, this call will block the

calling thread until the mutex is unlocked.

• pthread_mutex_trylock() will attempt to lock a mutex. However, if the mutex is already

locked, the routine will return immediately with a "busy" error code. This routine may be

useful in preventing deadlock conditions, as in a priority-inversion situation.

• pthread_mutex_unlock() will unlock a mutex if called by the owning thread. Calling this

routine is required after a thread has completed its use of protected data if other threads are

to acquire the mutex for their work with the protected data. An error will be returned if:

o If the mutex was already unlocked

o If the mutex is owned by another thread

• There is nothing "magical" about mutexes...in fact they are akin to a "gentlemen's

agreement" between participating threads. It is up to the code writer to insure that the

necessary threads all make the the mutex lock and unlock calls correctly. The following

scenario demonstrates a logical error:

• Thread 1 Thread 2 Thread 3

• Lock Lock

• A = 2 A = A+1 A = A*B

• Unlock Unlock

Question: When more than one thread is waiting for a locked mutex, which thread will be

granted the lock first after it is released?


Example: Using Mutexes

Example Code - Using Mutexes

This example program illustrates the use of mutex variables in a threads program

that performs a dot product. The main data is made available to all threads

through a globally accessible structure. Each thread works on a different part of

the data. The main thread waits for all the threads to complete their computations,

and then it prints the resulting sum.


#include <stdio.h>

#include <stdlib.h>

/*

The following structure contains the necessary information

to allow the function "dotprod" to access its input data

and place its output into the structure.

*/

typedef struct

{

double *a;

double *b;

double sum;

int veclen;

} DOTDATA;

/* Define globally accessible variables and a mutex */

#define NUMTHRDS 4

#define VECLEN 100

DOTDATA dotstr;

pthread_t callThd[NUMTHRDS];


pthread_mutex_t mutexsum;

/*

The function dotprod is activated when the thread is

created.

All input to this routine is obtained from a structure

of type DOTDATA and all output from this function is

written into

this structure. The benefit of this approach is apparent

for the multi-threaded program: when a thread is created we

pass a single argument to the activated function -

typically this argument is a thread number. All the other

information required by the function is accessed from the

globally accessible structure.

*/

void *dotprod(void *arg)

{

/* Define and use local variables for convenience */

int i, start, end, len ;

long offset;

double mysum, *x, *y;

offset = (long)arg;

len = dotstr.veclen;

start = offset*len;

end = start + len;

x = dotstr.a;

y = dotstr.b;

/*

Perform the dot product and assign result

to the appropriate variable in the structure.

*/


mysum = 0;

for (i=start; i<end ; i++)

{

mysum += (x[i] * y[i]);

}

/*

Lock a mutex prior to updating the value in the shared

structure, and unlock it upon updating.

*/

pthread_mutex_lock (&mutexsum);

dotstr.sum += mysum;

pthread_mutex_unlock (&mutexsum);

pthread_exit((void*) 0);

}

/*

The main program creates threads which do all the work and then

print out result upon completion. Before creating the threads,

the input data is created. Since all threads update a shared structure,

we need a mutex for mutual exclusion. The main thread needs to wait for

all threads to complete, it waits for each one of the threads. We specify

a thread attribute value that allow the main thread to join with the

threads it creates. Note also that we free up handles when they are

no longer needed.

*/


{

long i;

double *a, *b;

void *status;



/* Assign storage and initialize values */

a = (double*) malloc (NUMTHRDS*VECLEN*sizeof(double));

b = (double*) malloc (NUMTHRDS*VECLEN*sizeof(double));

for (i=0; i<VECLEN*NUMTHRDS; i++)

{

a[i]=1.0;

b[i]=a[i];

}

dotstr.veclen = VECLEN;

dotstr.a = a;

dotstr.b = b;

dotstr.sum=0;

pthread_mutex_init(&mutexsum, NULL);

/* Create threads to perform the dotproduct */


pthread_attr_setdetachstate(&attr,

PTHREAD_CREATE_JOINABLE);

for(i=0; i<NUMTHRDS; i++)

{

/*

Each thread works on a different set of data.

The offset is specified by 'i'. The size of

the data for each thread is indicated by VECLEN.

*/

pthread_create(&callThd[i], &attr, dotprod, (void

*)i);

}



/* Wait on the other threads */

for(i=0; i<NUMTHRDS; i++)

{

pthread_join(callThd[i], &status);

}

/* After joining, print out the results and cleanup */

printf ("Sum = %f \n", dotstr.sum);

free (a);

free (b);

pthread_mutex_destroy(&mutexsum);

pthread_exit(NULL);

}

Condition Variables

Overview

• Condition variables provide yet another way for threads to synchronize. While mutexes

implement synchronization by controlling thread access to data, condition variables allow

threads to synchronize based upon the actual value of data.

• Without condition variables, the programmer would need to have threads continually

polling (possibly in a critical section), to check if the condition is met. This can be very

resource consuming since the thread would be continuously busy in this activity. A

condition variable is a way to achieve the same goal without polling.

• A condition variable is always used in conjunction with a mutex lock.

• A representative sequence for using condition variables is shown below.

Main Thread

o Declare and initialize global data/variables which require synchronization (such

as "count")

o Declare and initialize a condition variable object


o Declare and initialize an associated mutex

o Create threads A and B to do work

Thread A

o Do work up to the point where a

certain condition must occur (such

as "count" must reach a specified

value)

o Lock associated mutex and check

value of a global variable

o Call pthread_cond_wait() to

perform a blocking wait for signal

from Thread-B. Note that a call to

pthread_cond_wait() automatically

and atomically unlocks the

associated mutex variable so that it

can be used by Thread-B.

o When signalled, wake up. Mutex is

automatically and atomically

locked.

o Explicitly unlock mutex

o Continue

Thread B

o Do work

o Lock associated mutex

o Change the value of the global

variable that Thread-A is waiting

upon.

o Check value of the global Thread-A

wait variable. If it fulfills the

desired condition, signal Thread-A.

o Unlock mutex.

o Continue

Main Thread

Join / Continue

Creating and Destroying Condition Variables

Routines:

• pthread_cond_init (condition,attr)

• pthread_cond_destroy (condition)

• pthread_condattr_init (attr)

• pthread_condattr_destroy (attr)

Usage:


• Condition variables must be declared with type pthread_cond_t, and must be initialized

before they can be used. There are two ways to initialize a condition variable:

1. Statically, when it is declared. For example:

pthread_cond_t myconvar = PTHREAD_COND_INITIALIZER;

2. Dynamically, with the pthread_cond_init() routine. The ID of the created condition

variable is returned to the calling thread through the condition parameter. This

method permits setting condition variable object attributes, attr.

• The optional attr object is used to set condition variable attributes. There is only one

attribute defined for condition variables: process-shared, which allows the condition

variable to be seen by threads in other processes. The attribute object, if used, must be of

type pthread_condattr_t (may be specified as NULL to accept defaults).

Note that not all implementations may provide the process-shared attribute.

• The pthread_condattr_init() and pthread_condattr_destroy() routines are used to create and

destroy condition variable attribute objects.

• pthread_cond_destroy() should be used to free a condition variable that is no longer needed.

Waiting and Signaling on Condition Variables

Routines:

• pthread_cond_wait (condition,mutex)

• pthread_cond_signal (condition)

• pthread_cond_broadcast (condition)

Usage:

• pthread_cond_wait() blocks the calling thread until the specified condition is signalled. This

routine should be called while mutex is locked, and it will automatically release the mutex

while it waits. After signal is received and thread is awakened, mutex will be automatically

locked for use by the thread. The programmer is then responsible for unlocking mutex when

the thread is finished with it.

• The pthread_cond_signal() routine is used to signal (or wake up) another thread which is

waiting on the condition variable. It should be called after mutex is locked, and must unlock

mutex in order for pthread_cond_wait() routine to complete.

• The pthread_cond_broadcast() routine should be used instead of pthread_cond_signal() if

more than one thread is in a blocking wait state.


• It is a logical error to call pthread_cond_signal() before calling pthread_cond_wait().

NB: Proper locking and unlocking of the associated mutex variable is essential when using these

routines. For example:

• Failing to lock the mutex before calling pthread_cond_wait() may cause it NOT to

block.

• Failing to unlock the mutex after calling pthread_cond_signal() may not allow a

matching pthread_cond_wait() routine to complete (it will remain blocked).

Example: Using Condition Variables

Example Code - Using Condition Variables

This simple example code demonstrates the use of several Pthread condition

variable routines. The main routine creates three threads. Two of the threads

perform work and update a "count" variable. The third thread waits until the

count variable reaches a specified value.


#include <stdio.h>

#include <stdlib.h>


#define TCOUNT 10

#define COUNT_LIMIT 12

int count = 0;

int thread_ids[3] = {0,1,2};

pthread_mutex_t count_mutex;

pthread_cond_t count_threshold_cv;

void *inc_count(void *t)

{

int i;

long my_id = (long)t;


for (i=0; i<TCOUNT; i++) {

pthread_mutex_lock(&count_mutex);

count++;

/*

Check the value of count and signal waiting thread when

condition is

reached. Note that this occurs while mutex is locked.

*/

if (count == COUNT_LIMIT) {

pthread_cond_signal(&count_threshold_cv);

printf("inc_count(): thread %ld, count = %d

Threshold reached.\n",

my_id, count);

}

printf("inc_count(): thread %ld, count = %d, unlocking

mutex\n",

my_id, count);

pthread_mutex_unlock(&count_mutex);

/* Do some "work" so threads can alternate on mutex

lock */

sleep(1);

}

pthread_exit(NULL);

}

void *watch_count(void *t)

{

long my_id = (long)t;


printf("Starting watch_count(): thread %ld\n", my_id);

/*

Lock mutex and wait for signal. Note that the

pthread_cond_wait routine will automatically and

atomically unlock mutex while it waits.

Also, note that if COUNT_LIMIT is reached before this

routine is run by the waiting thread, the loop will be

skipped to prevent pthread_cond_wait from never

returning.

*/

pthread_mutex_lock(&count_mutex);

while (count<COUNT_LIMIT) {

pthread_cond_wait(&count_threshold_cv, &count_mutex);

printf("watch_count(): thread %ld Condition signal

received.\n", my_id);

count += 125;

printf("watch_count(): thread %ld count now = %d.\n",

my_id, count);

}

pthread_mutex_unlock(&count_mutex);

pthread_exit(NULL);

}


{

int i, rc;

long t1=1, t2=2, t3=3;

pthread_t threads[3];


/* Initialize mutex and condition variable objects */


pthread_mutex_init(&count_mutex, NULL);

pthread_cond_init (&count_threshold_cv, NULL);

/* For portability, explicitly create threads in a

joinable state */


pthread_attr_setdetachstate(&attr,

PTHREAD_CREATE_JOINABLE);

pthread_create(&threads[0], &attr, watch_count, (void

*)t1);

pthread_create(&threads[1], &attr, inc_count, (void

*)t2);

pthread_create(&threads[2], &attr, inc_count, (void

*)t3);

/* Wait for all threads to complete */

for (i=0; i<NUM_THREADS; i++) {

pthread_join(threads[i], NULL);

}

printf ("Main(): Waited on %d threads. Done.\n",

NUM_THREADS);

/* Clean up and exit */


pthread_mutex_destroy(&count_mutex);

pthread_cond_destroy(&count_threshold_cv);

pthread_exit(NULL);

}


SOCKET PROGRAMMING

Introduction A socket is a communications connection point (endpoint) that you can name and address in a network.

Socket programming shows how to use socket APIs to establish communication links between remote

and local processes.

The processes that use a socket can reside on the same system or different systems on different

networks. Sockets are useful for both stand-alone and network applications. Sockets allow you to

exchange information between processes on the same machine or across a network, distribute work to

the most efficient machine, and they easily allow access to centralized data.

Socket application program interfaces (APIs) are the network standard for TCP/IP. A wide range of

operating systems support socket APIs. i5/OS® sockets support multiple transport and networking

protocols. Socket system functions and the socket network functions are threadsafe.

NB: The Java language also supports a socket programming interface.

Prerequisites for socket programming

Before writing socket applications, you must complete these steps to meet the requirements for compiler,

AF_INET and AF_INET6 address families, Secure Sockets Layer (SSL) APIs, and Global Secure Toolkit

(GSKit) APIs.

Compiler requirements

1. Install QSYSINC library. This library provides necessary header files that are needed when compiling

socket applications.

2. Install the ILE C licensed program (5761-WDS option 51).

Requirements for AF_INET and AF_INET6 address families

In addition to the compiler requirements, you must complete these tasks:

1. Plan TCP/IP setup.

2. Install TCP/IP.

3. Configure TCP/IP for the first time.

4. Configure IPv6 for TCP/IP if you plan to write applications that use the AF_INET6 address family.

Requirements for Secure Sockets Layer (SSL) APIs and Global Secure Toolkit (GSKit) APIs

In addition to the requirements for compiler, AF_INET address families, and AF_INET6 address families,

you must complete the following tasks to work with secure sockets:

1. Install and configure Digital Certificate Manager licensed program (5761–SS1 Option 34). See Digital


Certificate Manager (DCM) in the information center for details.

2. If you want to use SSL with the cryptographic hardware, you need to install and configure the 2058

Cryptographic Accelerator, or the 4758 Cryptographic Coprocessor, or the 4764 Cryptographic

Coprocessor. The 2058 Cryptographic Accelerator allows you to offload SSL cryptographic processing

from the operating system to the card. The 4758 Cryptographic Coprocessor can be used for SSL

cryptographic processing; however, unlike the 2058, this card provides more cryptographic functions,

like encrypting and decrypting keys. The 4764 Cryptographic Coprocessor is a better version of the 4758

Cryptographic Coprocessor. See Cryptography for complete descriptions of the 2058 Cryptographic

Accelerator, 4758 Cryptographic Coprocessor, and 4764 Cryptographic Coprocessor.

How sockets work

Sockets are commonly used for client and server interaction. Typical system configuration places the server

on one machine, with the clients on other machines. The clients connect to the server, exchange information,

and then disconnect.

A socket has a typical flow of events. In a connection-oriented client-to-server model, the socket on the

server process waits for requests from a client. To do this, the server first establishes (binds) an address that

clients can use to find the server. When the address is established, the server waits for clients to request a

service. The client-to-server data exchange takes place when a client connects to the server through a socket.

The server performs the client's request and sends the reply back to the client.

The following figure shows the typical flow of events (and the sequence of issued APIs) for a

connection-oriented socket session. An explanation of each event follows the figure.


This is a typical flow of events for a connection-oriented socket:

1. The socket() API creates an endpoint for communications and returns a socket descriptor that

represents the endpoint.

2. When an application has a socket descriptor, it can bind a unique name to the socket. Servers must bind

a name to be accessible from the network.

3. The listen() API indicates a willingness to accept client connection requests. When a listen() API is

issued for a socket, that socket cannot actively initiate connection requests. The listen() API is issued

after a socket is allocated with a socket() API and the bind() API binds a name to the socket. A listen()

API must be issued before an accept() API is issued.

4. The client application uses a connect() API on a stream socket to establish a connection to the server.

5. The server application uses the accept() API to accept a client connection request. The server must issue

the bind() and listen() APIs successfully before it can issue an accept() API.

6. When a connection is established between stream sockets (between client and server), you can use any

of the socket API data transfer APIs. Clients and servers have many data transfer APIs from which to

choose, such as send(), recv(), read(), write(), and others.


7. When a server or client wants to stop operations, it issues a close() API to release any system

resources acquired by the socket.

Note: The socket APIs are located in the communications model between the application layer and the

transport layer. The socket APIs are not a layer in the communication model. Socket APIs allow

applications to interact with the transport or networking layers of the typical communications model.

The arrows in the following figure show the position of a socket, and the communication layer that

the socket provides.

Typically, a network configuration does not allow connections between a secure internal network and a less

secure external network. However, you can enable sockets to communicate with server programs that run on

a system outside a firewall (a very secure host).

Sockets are also a part of IBM's AnyNet® implementation for the Multiprotocol Transport Networking

(MPTN) architecture. MPTN architecture provides the ability to operate a transport network over additional

transport networks and to connect application programs across transport networks of different types.

Socket characteristics

Sockets share some common characteristics.

• A socket is represented by an integer. That integer is called a socket descriptor.

• A socket exists as long as the process maintains an open link to the socket.

• You can name a socket and use it to communicate with other sockets in a communication domain.


• Sockets perform the communication when the server accepts connections from them, or when it

exchanges messages with them.

• You can create sockets in pairs (only for sockets in the AF_UNIX address family).

The connection that a socket provides can be connection-oriented or connectionless.

• Connection-oriented communication implies that a connection is established, and a dialog between

the programs follows. The program that provides the service (the server program) establishes the

available socket that is enabled to accept incoming connection requests. Optionally, the server can

assign a name to the service that it supplies, which allows clients to identify where to obtain and

how to connect to that service. The client of the service (the client program) must request the service

of the server program. The client does this by connecting to the distinct name or to the attributes

associated with the distinct name that the server program has designated. It is similar to dialing a

telephone number (an identifier) and making a connection with another party that is offering a

service (for example, a plumber). When the receiver of the call (the server, in this example, a

plumber) answers the telephone, the connection is established. The plumber verifies that you have

reached the correct party, and the connection remains active as long as both parties require it.

• Connectionless communication implies that no connection is established, over which a dialog or

data transfer can take place. Instead, the server program designates a name that identifies where to

reach it (much like a post-office box). If you send a letter to a post office box, you cannot be

absolutely sure that the receiver got the letter. You might need to wait for a response to your letter.

There is no active, real-time connection, in which data is exchanged.

How socket characteristics are determined

When an application creates a socket with the socket() API, it must identify the socket by specifying these

parameters:

• The socket address family determines the format of the address structure for the socket. This topic

contains examples of each address family's address structure.

• The socket type determines the form of communication for the socket.

• The socket protocol determines the supported protocols that the socket uses.

These parameters or characteristics define the socket application and how it interoperates with other socket

applications. Depending on the address family of a socket, you have different choices for the socket type

and protocol. The following table shows the corresponding address family and its associated socket type and

protocols:


Table : Summary of socket characteristics

Address family Socket type Socket protocol AF_UNIX SOCK_STREAM N/A

SOCK_DGRAM N/A

AF_INET SOCK_STREAM TCP

SOCK_DGRAM UDP

SOCK_RAW IP, ICMP

AF_INET6 SOCK_STREAM TCP

SOCK_DGRAM UDP

SOCK_RAW IP6, ICMP6 AF_UNIX_CCSID SOCK_STREAM N/A

SOCK_DGRAM N/A

In addition to these socket characteristics or parameters, constant values are defined in network routines and

header files that are shipped with the QSYSINC library. For descriptions of header files, see the individual

APIs. Each API lists its appropriate header file in the usage section of the API description.

Socket network routines allow socket applications to obtain information from the Domain Name System

(DNS), host, protocol, service, and network files.

Socket address structure

Sockets use the sockaddr address structure to pass and receive addresses. This structure does not require the

socket API to recognize the addressing format.

Table . Address structure

Address structure field Definitionsa_len This field contains the length of the address for UNIX 98

specifications. Note: The sa_len field is provided only for BSD 4.4 compatibility. It is not necessary to use this field even for BSD 4.4/UNIX 98 compatibility. The field is ignored on

sa_family This field defines the address family. This value is specified

for the address family on the socket() call. sa_data This field contains 14 bytes that are reserved to hold the address

itself. Note: The sa_data length of 14 bytes is a placeholder for the

address. The address can exceed this length. The structure is generic

because it does not define the format of the address. The format of

the address is defined by the type of transport, which a socket is

created for. Each of the transport providers define the exact format

for its specific addressing requirements in a similar address


sockaddr_storage This field declares storage for any address family address. This

structure is large enough and aligned for any protocol-specific

structure. It can then be cast as sockaddr structure for use on the

APIs. The ss_family field of the sockaddr_storage always aligns

with the family field of any protocol-specific structure.

Socket address family

The address family parameter (address_family) on a socket() API determines the format of the address

structure to be used on socket APIs.

Address family protocols provide the network transportation of application data from one application to

another (or from one process to another within the same system). The application specifies the network

transport provider on the protocol parameter of the socket.

AF_INET address family This address family provides interprocess communication between processes that run on the same system or

on different systems.

Addresses for AF_INET sockets are IP addresses and port numbers. You can specify an IP address for an

AF_INET socket either as an IP address (such as 130.99.128.1) or in its 32–bit form (X'82638001').

For a socket application that uses the Internet Protocol version 4 (IPv4), the AF_INET address family uses

the sockaddr_in address structure. When you use _XOPEN_SOURCE macro, the AF_INET address

structure changes to be compatible with BSD 4.4/ UNIX 98 specifications. For the sockaddr_in address

structure, these differences are summarized in the table:

Table . Differences between BSD 4.3 and BSD 4.4/ UNIX 98 for sockaddr_in address structure

BSD 4.3 sockaddr_in address structure BSD 4.4/ UNIX 98 sockaddr_in address structurestruct sockaddr_in { short sin_family; u_short sin_port; struct in_addr sin_addr; char sin_zero[8]; };

struct sockaddr_in {

uint8_t sin_len; sa_family_t sin_family; u_short sin_port; struct in_addr sin_addr; char sin_zero[8]; };

Table 5. AF_INET address structure

Address structure field Definitionsin_len This field contains the length of the address for UNIX 98

specifications. Note: The sin_len field is provided only for BSD 4.4 compatibility. It is not necessary to use this field even for BSD 4.4/ UNIX 98 compatibility. The field is ignored on


sin_family This field contains the address family, which is always

AF_INET when TCP or User Datagram Protocol (UDP) is used.

sin_port This field contains the port number. sin_addr This field contains the IP address. sin_zero This field is reserved. Set this field to hexadecimal zeros.

AF_INET6 address family This address family provides support for the Internet Protocol version 6 (IPv6). AF_INET6 address family

uses a 128 bit (16 byte) address.

The basic architecture of these addresses includes 64 bits for a network number and another 64 bits for the

host number. You can specify AF_INET6 addresses as x:x:x:x:x:x:x:x, where the x's are the hexadecimal

values of eight 16-bit pieces of the address. For example, a valid address looks like this:

FEDC:BA98:7654:3210:FEDC:BA98:7654:3210.

For a socket application that uses TCP, User Datagram Protocol (UDP) or RAW, the AF_INET6 address

family uses the sockaddr_in6 address structure. This address structure changes if you use

_XOPEN_SOURCE macro to implement BSD 4.4/ UNIX 98 specifications. For the sockaddr_in6 address

structure, these differences are summarized in this table:

Table . Differences between BSD 4.3 and BSD 4.4/ UNIX 98 for sockaddr_in6 address structure

BSD 4.3 sockaddr_in6 address structure BSD 4.4/ UNIX 98 sockaddr_in6 address structurestruct sockaddr_in6 { sa_family_t sin6_family; in_port_t sin6_port; uint32_t sin6_flowinfo; struct in6_addr sin6_addr; uint32_t sin6_scope_id; };

struct sockaddr_in6 {

uint8_t sin6_len; sa_family_t sin6_family; in_port_t sin6_port; uint32_t sin6_flowinfo; struct in6_addr sin6_addr;

Table . AF_INET6 address structure

Address structure field Definitionsin6_len This field contains the length of the address for UNIX 98

specifications. Note: The sin6_len field is provided only for BSD 4.4 compatibility. It is not necessary to use this field even for BSD 4.4/ UNIX 98 compatibility. The field is ignored on

sin6_family This field specifies the AF_INET6 address family.


sin6_port This field contains the transport layer port. sin6_flowinfo This field contains two pieces of information: the traffic class

and the flow label. Note: Currently, this field is not supported and should be set to

sin6_addr This field specifies the IPv6 address. sin6_scope_id This field identifies a set of interfaces as appropriate for the

scope of the address carried in the sin6_addr field.

AF_UNIX address family This address family provides interprocess communication on the same system that uses the socket APIs. The

address is actually a path name to an entry in the file system.

You can create sockets in the root directory or any open file system but file systems such as QSYS or

QDOC. The program must bind an AF_UNIX, SOCK_DGRAM socket to a name to receive any datagrams

back. In addition, the program must explicitly remove the file system object with the unlink() API when the

socket is closed.

Sockets with the address family AF_UNIX use the sockaddr_un address structure. This address structure

changes if you use _XOPEN_SOURCE macro to implement BSD 4.4/ UNIX 98 specifications. For the

sockaddr_un address structure, these differences are summarized in the table:

Table . Differences between BSD 4.3 and BSD 4.4/ UNIX 98 for sockaddr_un address structure

BSD 4.3 sockaddr_un address structure BSD 4.4/ UNIX 98 sockaddr_un address structurestruct sockaddr_un { short sun_family; char

sun_path[126]; };

struct sockaddr_un {

uint8_t sun_len;

sa_family_t sun_family;

char sun_path[126];

Table . AF_UNIX address structure

Address structure field Definitionsun_len This field contains the length of the address for UNIX 98

specifications. Note: The sun_len field is provided only for BSD 4.4 compatibility. It is not necessary to use this field even for BSD 4.4/ UNIX 98 compatibility. The field is ignored on

sun_family This field contains the address family. sun_path This field contains the path name to an entry in the file system.

For the AF_UNIX address family, protocol specifications do not apply because protocol standards are not

involved. The communications mechanism that the two processes use is specific to the system.


AF_UNIX_CCSID address family The AF_UNIX_CCSID family is compatible with the AF_UNIX address family and has the same

limitations.

They both can be either connectionless or connection-oriented, and no external communication functions

connect the two processes. The difference is that sockets with the address family AF_UNIX_CCSID use the

sockaddr_unc address structure. This address structure is similar to sockaddr_un, but it allows path names in

UNICODE or any CCSID by using the Qlg_Path_Name_T format.

However, because an AF_UNIX socket might return the path name from an AF_UNIX_CCSID socket in an

AF_UNIX address structure, path size is limited. AF_UNIX supports only 126 characters, so

AF_UNIX_CCSID is also limited to 126 characters.

A user cannot exchange AF_UNIX and AF_UNIX_CCSID addresses on a single socket. When

AF_UNIX_CCSID is specified on the socket() call, all addresses must be sockaddr_unc on later API calls. struct sockaddr_unc {

short sunc_family; short sunc_format; char sunc_zero[12]; Qlg_Path_Name_T sunc_qlg; union { char unix[126]; wchar_t wide[126]; char* p_unix; wchar_t* p_wide; }

sunc_path

;

};


Table .AF_UNIX_CCSID address structure

Address structure field Definitionsunc_family This field contains the address family, which is always

AF_UNIX_CCSID.sunc_format This field contains two defined values for the format of the path

name: v SO_UNC_DEFAULT indicates a wide path name using the

current default CCSID for integrated file system path names. The

sunc_qlg field is ignored. v SO UNC USE QLG indicates that the sunc qlg field defines

sunc_zero This field is reserved. Set this field to hexadecimal zeros.

sunc_qlg This field specifies the path name format. sunc_path This field contains the path name. It is a maximum of 126

characters and can be single byte or double byte. It can be

contained within the sunc_path field or allocated separately and

pointed to by sunc_path. The format is determined by

Socket type

The second parameter on a socket call determines the socket type. Socket type provides the

type identification and characteristics of the connection that are enabled for data transportation

from one machine or process to another.

The system supports the following socket types:

1. Stream (SOCK_STREAM)

This type of socket is connection-oriented. Establish an end-to-end connection by using the bind(),

listen(), accept(), and connect() APIs. SOCK_STREAM sends data without errors or duplication,

and receives the data in the sending order. SOCK_STREAM builds flow control to avoid data

overruns. It does not impose record boundaries on the data. SOCK_STREAM considers the data to

be a stream of bytes. In the i5/OS implementation, you can use stream sockets over Transmission

Control Protocol (TCP), AF_UNIX, and AF_UNIX_CCSID. You can also use stream sockets to

communicate with systems outside a secure host (firewall).

2. Datagram (SOCK_DGRAM)

In Internet Protocol terminology, the basic unit of data transfer is a datagram. This is basically a

header followed by some data. The datagram socket is connectionless. It establishes no end-to-end

connection with the transport provider (protocol). The socket sends datagrams as independent


packets with no guarantee of delivery. You might lose or duplicate data. Datagrams might arrive

out of order. The size of the datagram is limited to the data size that you can send in a single

transaction. For some transport providers, each datagram can use a different route through the

network. You can issue a connect() API on this type of socket. However, on the connect() API,

you must specify the destination address that the program sends to and receives from. In the i5/OS

implementation, you can use datagram sockets over User Datagram Protocol (UDP), AF_UNIX,

and AF_UNIX_CCSID.

3. Raw (SOCK_RAW)

This type of socket allows direct access to lower-layer protocols, such as Internet Protocol (IPv4 or

IPv6) and Internet Control Message Protocol (ICMP or ICMP6). SOCK_RAW requires more

programming expertise because you manage the protocol header information used by the transport

provider. At this level, the transport provider can dictate the format of the data and the semantics

that are transport-provider specific.

Socket protocols

Socket protocols provide the network transportation of application data from one machine to

another (or from one process to another within the same machine).

The application specifies the transport provider on the protocol parameter of the socket()

API.

For the AF_INET address family, more than one transport provider is allowed. The protocols of

Systems Network Architecture (SNA) and TCP/IP can be active on the same listening socket at the

same time. The ALWANYNET (Allow ANYNET support) network attribute allows a customer to

select whether a transport other than TCP/IP can be used for AF_INET socket applications. This

network attribute can be either *YES or *NO. The default value is *NO.

For example, if the current status (the default status) is *NO, the use of AF_INET over an SNA

transport is not active. If AF_INET sockets are to be used over a TCP/IP transport only, the

ALWANYNET status should be set to *NO to improve CPU utilization.

Note: The ALWANYNET network attribute also affects APPC over TCP/IP.

The AF_INET and AF_INET6 sockets over TCP/IP can also specify a SOCK_RAW type, which

means that the socket communicates directly with the network layer known as Internet Protocol

(IP). The TCP or UDP transport providers normally communicate with this layer. When you use


SOCK_RAW sockets, the application program specifies any protocol between 0 and 255 (except

the TCP and UDP protocols). This protocol number then flows in the IP headers when machines

are communicating on the network. In fact, the application program is the transport provider,

because it must provide for all the transport services that UDP or TCP transports normally provide.

For the AF_UNIX and AF_UNIX_CCSID address families, a protocol specification is not really

meaningful because there are no protocol standards involved. The communications mechanism

between two processes on the same machine is specific to the machine.


BASIC SOCKET DESIGN

These examples below illustrate the most common types of socket programs that use the most

basic design, which can be a basis for more complex socket designs.

Creating a connection-oriented socket

These server and client examples illustrate the socket APIs written for a connection-oriented

protocol such as Transmission Control Protocol (TCP).

The following figure illustrates the client/server relationship of the sockets API for a connection-

oriented protocol.


Socket flow of events: Connection-oriented server

The following sequence of the socket calls provides a description of the figure. It also describes

the relationship between the server and client application in a connection-oriented design. Each

set of flows contains links to usage notes on specific APIs.

1. The socket() API returns a socket descriptor, which represents an endpoint. The statement also

identifies that the Internet Protocol version 6 address family (AF_INET6) with the TCP transport

(SOCK_STREAM) is used for this socket.

2. The setsockopt() API allows the local address to be reused when the server is restarted

before the required wait time expires.

3. After the socket descriptor is created, the bind() API gets a unique name for the socket. In

thisexample, the user sets the s6_addr to zero, which allows connections to be established from

any IPv4

or IPv6 client that specifies port 3005.

4. The listen() API allows the server to accept incoming client connections. In this example,

the backlog

is set to 10. This means that the system queues 10 incoming connection requests before the

system

starts rejecting the incoming requests.

5. The server uses the accept() API to accept an incoming connection request. The accept() call

blocks indefinitely, waiting for the incoming connection to arrive.

6. The select() API allows the process to wait for an event to occur and to wake up the process

when the event occurs. In this example, the system notifies the process only when data is

available to be read. A 30-second timeout is used on this select() call.

7. The recv() API receives data from the client application. In this example, the client sends

250 bytes of data. Thus, the SO_RCVLOWAT socket option can be used, which specifies

that recv() does not wake up until all 250 bytes of data have arrived.

8. The send() API echoes the data back to the client.

9. The close() API closes any open socket descriptors.

Socket flow of events: Connection-oriented client

The following sequence of APIs calls describes the relationship between the server and client

application in a connection-oriented design.


1. The socket() API returns a socket descriptor, which represents an endpoint. The statement

also identifies that the Internet Protocol version 6 address family (AF_INET6) with the TCP

transport


2. In the client example program, if the server string that was passed into the inet_pton() API

was not a valid IPv6 address string, then it is assumed to be the host name of the server. In

that case, use the getaddrinfo() API to retrieve the IP address of the server.

3. After the socket descriptor is received, the connect() API is used to establish a

connection to the server.

4. The send() API sends 250 bytes of data to the server.

5. The recv() API waits for the server to echo the 250 bytes of data back. In this example, the

server responds with the same 250 bytes that was just sent. In the client example, the 250

bytes of the data might arrive in separate packets, so the recv() API can be used over and

over until all 250 bytes have arrived.


Example: A connection-oriented server

This example shows how a connection-oriented server can be created.

You can use this example to create your own socket server application. A connection-oriented

server design is one of the most common models for socket applications. In a connection-oriented

design, the server application creates a socket to accept client requests.

/**************************************************************************/ /* This sample program provides a code for a connection-oriented server. */ /**************************************************************************/ /**************************************************************************/ /* Header files needed for this sample program . */ /**************************************************************************/ #include <stdio.h> #include <sys/time.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <sys/poll.h> /**************************************************************************/ /* Constants usedbythis program */ /**************************************************************************/ #define SERVER_PORT 12345 #define BUFFER_LENGTH 250 #define FALSE 0 void main() {


/***********************************************************************/ /* Variable and structure definitions. */ /***********************************************************************/ int sd=-1, sd2=-1; int rc, length, on=1; char buffer[BUFFER_LENGTH]; struct pollfd fds; nfds_t nfds = 1; int timeout; struct sockaddr_in6 serveraddr; /***********************************************************************/ /* A do/while(FALSE) loop is used to make error cleanup easier. The */ /* close() of each of the socket descriptors is only done once at the */ /* very endofthe program. */ /***********************************************************************/ do {

/********************************************************************/ /* The socket() function returns a socket descriptor, representing */ /* an endpoint. The statement also identifies that the INET6 */ /* (Internet Protocol version 6) address family with the TCP */ /* transport (SOCK_STREAM) will be used for this socket. */ /********************************************************************/ sd = socket(AF_INET6, SOCK_STREAM, 0); if (sd < 0) {

perror("socket() failed"); break;

} /********************************************************************/ /* The setsockopt() function is used to allow the local address to */ /* be reused when the server is restarted before the required wait */ /* time expires. */ /********************************************************************/ rc = setsockopt(sd, SOL_SOCKET, SO_REUSEADDR, (char *)&on, sizeof(on));

if (rc < 0) {

perror("setsockopt(SO_REUSEADDR) failed"); brea

k; }

/********************************************************************/ /* After the socket descriptor is created, a bind() function gets a */ /* unique name for the socket. In this example, the user sets the */ /* s6_addr to zero, which allows connections to be established from */ /* any client that specifies port 12345. */ /********************************************************************/ memset(&serveraddr, 0, sizeof(serveraddr)); serveraddr.sin6_family = AF_INET6; serveraddr.sin6_port = htons(SERVER_PORT); memcpy(&serveraddr.sin6_addr, &in6addr_any, sizeof(in6addr_any)); rc = bind(sd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)); if (rc < 0) {


perror("bind() failed"); brea

k; }

/********************************************************************/ /* The listen() function allows the server to accept incoming */ /* client connections. In this example, the backlog is set to 10. */ /* This means that the system will queue 10 incoming connection */ /* requests before the system starts rejecting the incoming */ /* requests. */ /********************************************************************/ rc = listen(sd, 10); if (rc< 0) {

perror("listen() failed"); brea

k; }

printf("Ready for client connect().\n");

/********************************************************************/ /* The server uses the accept() function to accept an incoming */ /* connection request. The accept() call will block indefinitely */ /* waiting for the incoming connection to arrive. */ /********************************************************************/ sd2 = accept(sd, NULL, NULL); if (sd2 < 0) {

perror("accept() failed"); brea

k; }

/********************************************************************/ /* The poll() function allows the process to wait for an event to */ /* occur and to wake up the process when the event occurs. In this */ /* example, the system notifies the process only when data is */ /* available to read. A 30 second timeout is used on this poll */ /* call. */ /********************************************************************/ timeout = 30000; memset(&fds, 0, sizeof(fds)); fds.fd = ds2; fds.events = POLLIN; fds.revents = 0;

rc = poll(&fds, nfds, timeout);

if (rc < 0) {

perror("poll() failed"); break; }

if (rc == 0) { printf("poll() timed out.\n"); break; }


/********************************************************************/ /* In this example we know that the client will send 250 bytes of */ /* data over. Knowing this, we can use the SO_RCVLOWAT socket */ /* option and specify that we don’t want our recv() to wake up until*/ /* all 250 bytesofdata have arrived. */ /********************************************************************/ length = BUFFER_LENGTH; rc = setsockopt(sd2, SOL_SOCKET, SO_RCVLOWAT,

(char *)&length, sizeof(length)); if (rc < 0) {

perror("setsockopt(SO_RCVLOWAT) failed"); break; }

/********************************************************************/ /* Receive that 250 bytes data from the client */ /********************************************************************/ rc = recv(sd2, buffer, sizeof(buffer), 0); if (rc < 0) {

perror("recv() failed"); break; }

printf("%d bytes of data were received\n", rc); if (rc == 0 ||

rc < sizeof(buffer)) { printf("The client closed the connection before all of the\n"); printf("data was sent\n"); break; }

/********************************************************************/ /* Echo the data backtothe client */ /********************************************************************/ rc = send(sd2, buffer, sizeof(buffer), 0); if (rc < 0) {

perror("send() failed"); break; }

/********************************************************************/ /* Program complete */ /********************************************************************/

} while (FALSE);

/***********************************************************************/ /* Close down any open socket descriptors */ /***********************************************************************/ if (sd != -1)

close(sd); if (sd2 != -1) close(sd2); }

Example: A connection-oriented client

This example shows how to create a socket client program to connect to a connection-oriented

server in a connection-oriented design.

The client of the service (the client program) must request the service of the server program.

You can use this example to write your own client application.


/**************************************************************************/ /* This sample program provides a code for a connection-oriented client. */ /**************************************************************************/

/**************************************************************************/ /* Header files needed for this sample program */ /**************************************************************************/ #include <stdio.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h>

/**************************************************************************/ /* Constants usedbythis program */ /**************************************************************************/ #define SERVER_PORT 3005 #define BUFFER_LENGTH 250 #define FALSE 0 #define SERVER_NAME "ServerHostName"

/* Pass in 1 parameter which is either the */ /* address or host name of the server, or */ /* set the server name in the #define */ /* SERVER_NAME. */ void main(int argc, char *argv[]) {

/***********************************************************************/ /* Variable and structure definitions. */ /***********************************************************************/ int sd=-1, rc, bytesReceived; char buffer[BUFFER_LENGTH]; char server[NETDB_MAX_HOST_NAME_LENGTH]; struct sockaddr_in6 serveraddr; struct addrinfo hints, *res;

/***********************************************************************/

/* A do/while(FALSE) loop is used to make error cleanup easier. The */ /* close() of the socket descriptor is only done once at the very end */ /*ofthe program. */ /***********************************************************************/ do {

/********************************************************************/ /* The socket() function returns a socket descriptor, representing */ /* an endpoint. The statement also identifies that the INET6 */ /* (Internet Protocol version 6) address family with the TCP */ /* transport (SOCK_STREAM) will be used for this socket. */ /********************************************************************/ sd = socket(AF_INET6, SOCK_STREAM, 0); if (sd < 0) { perror("socket() failed"); break; }

/********************************************************************/ /* If an argument was passed in, use this as the server, otherwise */ /* use the #define


that is located at the top of this program. */ /********************************************************************/ if (argc > 1)

strcpy(server, argv[1]); else strcpy(server, SERVER_NAME);

memset(&serveraddr, 0, sizeof(serveraddr)); serveraddr.sin6_family = AF_INET6; serveraddr.sin6_port = htons(SERVER_PORT); rc = inet_pton(AF_INET6, server, &serveraddr.sin6_addr.s6_addr);

if (rc != 1) { /*****************************************************************/ /* The server string that was passed into the inet_pton() */ /* function was not a hexidecimal colon IP address. It must */ /* therefore be the hostname of the server. Use the */ /* getaddrinfo() function to retrieve the IP address of the */ /* server. */ /*****************************************************************/ memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_INET6; hints.ai_flags = AI_V4MAPPED; rc = getaddrinfo(server, NULL, &hints, &res); if (rc != 0) {

printf("Host not found! (%s)\n", server); perror("getaddrinfo() failed\n"); break; }

memcpy(&serveraddr.sin6_addr, (&((struct sockaddr_in6 *)(res->ai_addr))->sin6_addr), sizeof(serveraddr.sin6_addr));

} freeaddrinfo(res);

/********************************************************************/ /* Use the connect() function to establish a connection to the */ /* server. */ /********************************************************************/ rc = connect(sd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)); if (rc < 0) {

perror("connect() failed"); break; }

/********************************************************************/ /* Send 250 bytesofa’stothe server */ /********************************************************************/ memset(buffer, ’a’, sizeof(buffer)); rc = send(sd, buffer, sizeof(buffer), 0); if (rc < 0) {


/********************************************************************/ /* In this example we know that the server is going to respond with */ /* the same 250 bytes that we just sent. Since we know that 250 */ /* bytes are going to be sent back to us, we can use the */ /* SO_RCVLOWAT socket option and then issue a single recv() and */ /* retrieve allofthe data. */ /* */ /* The use of SO_RCVLOWAT is already illustrated in the server */ /* side of this example, so we will do something different here. */ /* The 250 bytes of the data may arrive in separate packets, */ /* therefore we will issue recv() over and over again until all */ /* 250 bytes have arrived. */


/********************************************************************/ bytesReceived = 0; while (bytesReceived < BUFFER_LENGTH) {

rc = recv(sd, & buffer[bytesReceived], BUFFER_LENGTH - bytesReceived, 0);

if (rc < 0) {

perror("recv() failed"); break; } else if (rc == 0) {

printf("The server closed the connection\n"); break; }

/*****************************************************************/

/* Increment the number of bytes that have been received so far */ /*****************************************************************/ bytesReceived += rc;

} while (FALSE);

/***********************************************************************/ /* Close down any open socket descriptors */ /***********************************************************************/ if (sd != -1) close(sd); }

Creating a connectionless socket

Connectionless sockets do not establish a connection over which data is transferred. Instead,

the server application specifies its name where a client can send requests.

Connectionless sockets use User Datagram Protocol (UDP) instead of TCP/IP.

The following figure illustrates the client/server relationship of the socket APIs used in the

examples for a connectionless socket design.


Socket flow of events: Connectionless server

The following sequence of the socket calls provides a description of the figure and the

following example programs. It also describes the relationship between the server and client

application in a connectionless design. Each set of flows contains links to usage notes on

specific APIs. If you need more details on the use of a particular API, you can use these

links. The first example of a connectionless server uses the following sequence of API calls:

1. The socket() API returns a socket descriptor, which represents an endpoint. The

statement also identifies that the Internet Protocol version 6 address family (AF_INET6)

with the UDP transport (SOCK_DGRAM) is used for this socket.

2. After the socket descriptor is created, a bind() API gets a unique name for the socket. In

this example, the user sets the s6_addr to zero, which means that the UDP port of 3555 is

bound to all IPv4 and IPv6 addresses on the system.

3. The server uses the recvfrom() API to receive that data. The recvfrom() API waits

indefinitely for data to arrive.

4. The sendto() API echoes the data back to the client.

5. The close() API ends any open socket descriptors.


Socket flow of events: Connectionless client

The second example of a connectionless client uses the following sequence of API calls.

1. The socket() API returns a socket descriptor, which represents an endpoint. The

statement also identifies that the Internet Protocol version 6 address family

(AF_INET6) with the UDP transport

(SOCK_DGRAM) is used for this socket.

2. In the client example program, if the server string that was passed into the inet_pton()

API was not a valid IPv6 address string, then it is assumed to be the host name of the

server. In that case, use the getaddrinfo() API to retrieve the IP address of the server.

3. Use the sendto() API to send the data to the server.

4. Use the recvfrom() API to receive the data from the server.

5. The close() API ends any open socket descriptors.

gethostbyname()--Get Host Information for Host Name API



Example: A connectionless server

This example illustrates how to create a connectionless socket server program by using User Datagram

Protocol (UDP).

/**************************************************************************/ /* This sample program provides a code for a connectionless server. */ /**************************************************************************/ /**************************************************************************/ /* Header files needed for this sample program */ /**************************************************************************/ #include <stdio.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> /**************************************************************************/ /* Constants usedbythis program */ /**************************************************************************/ #define SERVER_PORT 3555 #define BUFFER_LENGTH 100

#define FALSE 0

void main() { /***********************************************************************/ /* Variable and structure definitions. */ /***********************************************************************/ int sd=-1, rc; char buffer[BUFFER_LENGTH]; struct sockaddr_in6 serveraddr; struct sockaddr_in6 clientaddr; int clientaddrlen = sizeof(clientaddr); /***********************************************************************/ /* A do/while(FALSE) loop is used to make error cleanup easier. The */ /* close() of each of the socket descriptors is only done once at the */ /* very endofthe program. */ /***********************************************************************/ do {

/********************************************************************/ /* The socket() function returns a socket descriptor, representing */ /* an endpoint. The statement also identifies that the INET6 */ /* (Internet Protocol version 6) address family with the UDP */ /* transport (SOCK_DGRAM) will be used for this socket. */ /********************************************************************/ sd = socket(AF_INET6, SOCK_DGRAM, 0); if (sd < 0) { perror("socket() failed"); break; } /********************************************************************/ /* After the socket descriptor is created, a bind() function gets a */ /* unique name for the socket. In this example, the user sets the */ /* s_addr to zero, which means that the UDP port of 3555 will be */ /* bound to all IP addresses on the system. */


/********************************************************************/ memset(&serveraddr, 0, sizeof(serveraddr)); serveraddr.sin6_family = AF_INET6; serveraddr.sin6_port = htons(SERVER_PORT); memcpy(&serveraddr.sin6_addr, &in6addr_any, sizeof(in6addr_any)); rc = bind(sd, (struct sockaddr *)&serveraddr, sizeof(serveraddr)); if (rc < 0) {

perror("bind() failed"); break; }

/********************************************************************/

/* The server uses the recvfrom() function to receive that data. */

/* The recvfrom() function waits indefinitely for data to arrive. */ /********************************************************************/

rc = recvfrom(sd, buffer, sizeof(buffer), 0, (struct sockaddr *)&clientaddr, &clientaddrlen); if (rc < 0) {

perror("recvfrom() failed"); break; }

printf("server received the following: <%s>\n", buffer);

inet_ntop(AF_INET6, &clientaddr.sin6_addr.s6_addr, buffer, sizeof(buffer));

printf("from port %d and address %s\n", ntohs(clientaddr.sin6_port), buffer);

/********************************************************************/ /* Echo the data backtothe client */ /********************************************************************/ rc = sendto(sd, buffer, sizeof(buffer), 0, (struct sockaddr *)&clientaddr, sizeof(clientaddr)); if (rc < 0) {

perror("sendto() failed"); break; }

/********************************************************************/ /* Program complete */ /********************************************************************/

} while (FALSE);

/***********************************************************************/ /* Close down any open socket descriptors */ /***********************************************************************/ if (sd != -1) close(sd); }

Example: A connectionless client

This example shows how to use User Datagram Protocol (UDP) to connect a connectionless socket client

program to a server.

/**************************************************************************/ /*

This sample program provides a code for a connectionless client. */

/**************************************************************************/

/**************************************************************************/


/* Header files needed for this sample program */ /**************************************************************************/ #include <stdio.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h>

/**************************************************************************/ /* Constants usedbythis program */ /**************************************************************************/ #define SERVER_PORT 3555 #define BUFFER_LENGTH 100 #define FALSE 0 #define SERVER_NAME "ServerHostName"

/* Pass in 1 parameter which is either the */

/* address or host name of the server, or */

/* set the server name in the #define */

/* SERVER_NAME */ void main(int argc, char *argv[]) {

/***********************************************************************/ /* Variable and structure definitions. */ /***********************************************************************/ int sd, rc; char server[NETDB_MAX_HOST_NAME_LENGTH]; char buffer[BUFFER_LENGTH]; struct sockaddr_in6 serveraddr; int serveraddrlen = sizeof(serveraddr); struct addrinfo hints, *res;

/***********************************************************************/ /* A do/while(FALSE) loop is used to make error cleanup easier. The */ /* close() of the socket descriptor is only done once at the very end */ /*ofthe program. */ /***********************************************************************/ do {

/********************************************************************/

/* The socket() function returns a socket descriptor, representing */

/* an endpoint. The statement also identifies that the INET6 */

/* (Internet Protocol) address family with the UDP transport */

/* (SOCK_DGRAM) will be used for this socket. */ /********************************************************************/

sd = socket(AF_INET6, SOCK_DGRAM, 0); if (sd < 0) { perror("socket() failed"); break; }

/********************************************************************/


/* If an argument was passed in, use this as the server, otherwise */ /* use the #define that is

located at the top of this program. */

/********************************************************************/

if (argc > 1) strcpy(server, argv[1]); else strcpy(server, SERVER_NAME);

memset(&serveraddr, 0, sizeof(serveraddr)); serveraddr.sin6_family = AF_INET6; serveraddr.sin6_port = htons(SERVER_PORT); rc = inet_pton(AF_INET6, server, &serveraddr.sin6_addr.s6_addr); if (rc != 1) {

/*****************************************************************/ /* The server string that was passed into the inet_pton() */ /* function was not a hexidecimal colon IP address. It must */ /* therefore be the hostname of the server. Use the */ /* getaddrinfo() function to retrieve the IP address of the */ /* server. */ /*****************************************************************/ memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_INET6; hints.ai_flags = AI_V4MAPPED; rc = getaddrinfo(server, NULL, &hints, &res); if (rc != 0) { printf("Host not found! (%s)", server); break;

}

memcpy(&serveraddr.sin6_addr, (&((struct sockaddr_in6 *)(res->ai_addr))->sin6_addr), sizeof(serveraddr.sin6_addr));

freeaddrinfo(res);

} /********************************************************************/

/* Initialize the data block that is going to be sent to the server */

/********************************************************************/

memset(buffer, 0, sizeof(buffer)); strcpy(buffer, "A CLIENT REQUEST");

/********************************************************************/

/* Use the sendto() function to send the data to the server. */

/********************************************************************/

rc = sendto(sd, buffer, sizeof(buffer), 0, (struct sockaddr *)&serveraddr, sizeof(serveraddr));

if (rc < 0) { perror("sendto() failed"); break; }


/********************************************************************/

/* Use the recvfrom() function to receive the data back from the */

/* server. */ /********************************************************************/

rc = recvfrom(sd, buffer, sizeof(buffer), 0, (struct sockaddr *)&serveraddr, & serveraddrlen); if

(rc < 0) { perror("recvfrom() failed"); break; }

printf("client received the following: <%s>\n", buffer); inet_ntop(AF_INET6,

&serveraddr.sin6_addr.s6_addr, buffer, sizeof(buffer)); printf("from port %d, from address %s\n",

ntohs(serveraddr_sin6_port), buffer);

/********************************************************************/

/* Program complete */ /********************************************************************/

} while (FALSE);

/***********************************************************************/

/* Close down any open socket descriptors */ /***********************************************************************/

if (sd != -1) close(sd);


Designing applications with address families

These scenarios illustrate how to design applications with each of the socket address families,

such as AF_INET address family, AF_INET6 address family, AF_UNIX address family, and

AF_UNIX_CCSID address family.

Using AF_INET address family

AF_INET address family sockets can be either connection-oriented (type SOCK_STREAM) or

connectionless (type SOCK_DGRAM). Connection-oriented AF_INET sockets use

Transmission Control Protocol (TCP) as the transport protocol. Connectionless AF_INET

sockets use User Datagram Protocol (UDP) as the transport protocol.

When you create an AF_INET domain socket, you specify AF_INET for the address family in

the socket program. AF_INET sockets can also use a type of SOCK_RAW. If this type is set,

the application connects directly to the IP layer and does not use either the TCP or UDP

transport.

Using AF_INET6 address family

AF_INET6 sockets provide support for Internet Protocol version 6 (IPv6) 128 bit (16 byte)

address structures. Programmers can write applications using the AF_INET6 address family

to accept client requests from either IPv4 or IPv6 nodes, or from IPv6 nodes only.

Like AF_INET sockets, AF_INET6 sockets can be either connection-oriented (type

SOCK_STREAM) or connectionless (type SOCK_DGRAM). Connection-oriented AF_INET6

sockets use TCP as the transport protocol. Connectionless AF_INET6 sockets use User

Datagram Protocol (UDP) as the transport protocol. When you create an AF_INET6 domain

socket, you specify AF_INET6 for the address family in the socket program. AF_INET6 sockets

can also use a type of SOCK_RAW. If this type is set, the application connects directly to the IP

layer and does not use either the TCP or UDP transport.

Using AF_UNIX address family

Sockets that use the AF_UNIX or AF_UNIX_CCSID address family can be connection-

oriented (type SOCK_STREAM) or connectionless (type SOCK_DGRAM).


Both types are reliable because there are no external communication functions

connecting the two processes.

UNIX domain datagram sockets act differently from UDP datagram sockets. With UDP

datagram sockets, the client program does not need to call the bind() API because the system

assigns an unused port number automatically. The server can then send a datagram back to that

port number. However, with UNIX domain datagram sockets, the system does not automatically

assign a path name for the client. Thus, all client programs using UNIX domain datagrams must

call the bind() API. The exact path name specified on the client's bind() is what is passed to the

server. Thus, if the client specifies a relative path name (that is, a path name that is not fully

qualified by starting with /), the server cannot send the client a datagram unless it is running

with the same current directory.

An example path name that an application might use for this address family is /tmp/myserver or

servers/thatserver. With servers/thatserver, you have a path name that is not fully qualified (no /

was specified). This means that the location of the entry in the file system hierarchy should be

determined relative to the current working directory.

Note: Path names in the file system are NLS-enabled.

The following figure illustrates the client/server relationship of the AF_UNIX address family.


Socket flow of events: Server application that uses AF_UNIX address family

The first example uses the following sequence of API calls:


also

identifies the UNIX address family with the stream transport (SOCK_STREAM) being used

for this

socket. You can also use the socketpair() API to initialize a UNIX socket.

AF_UNIX or AF_UNIX_CCSID are the only address families to support the socketpair()

API. The socketpair() API returns two socket descriptors that are unnamed and

connected.

2. After the socket descriptor is created, the bind() API gets a unique name for the socket.

The name space for UNIX domain sockets consists of path names. When a sockets program

calls the bind() API, an entry is created in the file system directory. If the path name already

exists, the bind() fails. Thus, a UNIX domain socket program should always call an unlink()

API to remove the directory entry when it ends.


3. The listen() allows the server to accept incoming client connections. In this example, the

backlog is set to 10. This means that the system queues 10 incoming connection requests

before the system starts rejecting the incoming requests.

4. The recv() API receives data from the client application. In this example, the client sends 250

bytes of data over. Thus, the SO_RCVLOWAT socket option can be used, which specifies

that recv() is not required to wake up until all 250 bytes of data have arrived.



7. The unlink() API removes the UNIX path name from the file system.

Socket flow of events: Client application that uses AF_UNIX address family

The second example uses the following sequence of API calls:


identifies the UNIX address family with the stream transport (SOCK_STREAM) being used for this


AF_UNIX or AF_UNIX_CCSID are the only address families to support the socketpair() API.

The socketpair() API returns two socket descriptors that are unnamed and connected.

2. After the socket descriptor is received, the connect() API is used to establish a connection to the

server.

3. The send() API sends 250 bytes of data that are specified in the server application with the

SO_RCVLOWAT socket option.

4. The recv() API loops until all 250 bytes of the data have arrived.


Example: Server application that uses AF_UNIX address family:

This example illustrates a sample server program that uses the AF_UNIX address family. The

AF_UNIX address family uses many of the same socket calls as other address families, except that it

uses the path name structure to identify the server application. /**************************************************************************/

/* This example program provides code for a server application that uses */

/* AF_UNIX address family */

/**************************************************************************/

/**************************************************************************/

/* Header files needed for this sample program */


/**************************************************************************/

#include <stdio.h>

#include <string.h>


#include <sys/socket.h>

#include <sys/un.h>

/**************************************************************************/

/* Constants usedbythis program */

/**************************************************************************/

#define SERVER_PATH "/tmp/server"

#define BUFFER_LENGTH 250

#define FALSE 0

void main() {

/***********************************************************************/

/* Variable and structure definitions. */

/***********************************************************************/

int sd=-1, sd2=-1;

int rc, length;

char buffer[BUFFER_LENGTH];

struct sockaddr_un serveraddr;

/***********************************************************************/

/* A do/while(FALSE) loop is used to make error cleanup easier. The */

/* close() of each of the socket descriptors is only done once at the */

/* very endofthe program. */

/***********************************************************************/

do

{

/********************************************************************/

/* The socket() function returns a socket descriptor, which represents */

/* an endpoint. The statement also identifies that the UNIX */

/* address family with the stream transport (SOCK_STREAM) will be */

/* used for this socket. */

/********************************************************************/

sd = socket(AF_UNIX, SOCK_STREAM, 0);

if (sd < 0)

{

perror("socket() failed"); break; }

/********************************************************************/

/* After the socket descriptor is created, a bind() function gets a */


/* unique name for the socket. */

/********************************************************************/

memset(&serveraddr, 0, sizeof(serveraddr));

serveraddr.sun_family = AF_UNIX;

strcpy(serveraddr.sun_path, SERVER_PATH);

rc = bind(sd, (struct sockaddr *)&serveraddr, SUN_LEN(&serveraddr));

if (rc < 0)

{

perror("bind() failed");

break; }

/********************************************************************/

/* The listen() function allows the server to accept incoming */

/* client connections. In this example, the backlog is set to 10. */

/* This means that the system will queue 10 incoming connection */

/* requests before the system starts rejecting the incoming */

/* requests. */

/********************************************************************/

rc = listen(sd, 10);

if (rc< 0) {

perror("listen() failed");

break; }


/********************************************************************/

/* The server uses the accept() function to accept an incoming */

/* connection request. The accept() call will block indefinitely */

/* waiting for the incoming connection to arrive. */

/********************************************************************/

sd2 = accept(sd, NULL, NULL); if (sd2 < 0) {

perror("accept() failed");

break; }

/********************************************************************/

/* In this example we know that the client will send 250 bytes of */

/* data over. Knowing this, we can use the SO_RCVLOWAT socket */

/* option and specify that we don’t want our recv() to wake up */

/* until all 250 bytes of data have arrived. */

/********************************************************************/

length = BUFFER_LENGTH; rc = setsockopt(sd2, SOL_SOCKET, SO_RCVLOWAT,



perror("setsockopt(SO_RCVLOWAT) failed");

break; }

/****************************************************/ /*

Receive that 250 bytes data from the client */ /***************************/

rc = recv(sd2, buffer, sizeof(buffer), 0); if (rc < 0) {

perror("recv() failed");

break; }


rc < sizeof(buffer)) {

printf("The client closed the connection before all of the\n");

printf("data was sent\n");

break; }

/********************************************************************/

/* Echo the data backtothe client */

/********************************************************************/

rc = send(sd2, buffer, sizeof(buffer), 0);

if (rc < 0)

{

perror("send() failed");

break; }

/********************************************************************/

/* Program complete */

/********************************************************************/

} while (FALSE);

Socket programming 33

/***********************************************************************/

/* Close down any open socket descriptors */

/***********************************************************************/


if (sd2 != -1) close(sd2);

/***********************************************************************/

/* Remove the UNIX path name from the file system */

/***********************************************************************/

unlink(SERVER_PATH);

}


Example: Client application that uses AF_UNIX address family:

This example shows a sample application that uses the AF_UNIX address family to create a

client connection to a server.

/******************************************************************

********/

/* This sample program provides code for a client application that

uses */

/* AF_UNIX address family */ /****************************************************************

**********/

/****************************************************************

**********/

/* Header files needed for this sample program */ /*******************************************************************

*******/ #include <stdio.h>

#include <string.h>



#include <sys/un.h>

/****************************************************************

**********/

/* Constants usedbythis program */ /*******************************************************************

*******/



#define FALSE 0


/* path name of the server as a UNICODE */

/* string, or set the server path in the */

/* #define SERVER_PATH which is a CCSID */

/* 500 string. */ void main(int argc, char

*argv[]) { /***********************************************************************/ /* Variable and structure definitions. */ /***********************************************************************/ int sd=-1, rc, bytesReceived; char buffer[BUFFER_LENGTH]; struct sockaddr_un serveraddr;

/***********************************************************************/


/* A do/while(FALSE) loop is used to make error cleanup easier. The */ /* close() of the socket descriptor is only done once at the very end */ /*ofthe program. */ /***********************************************************************/ do {

/********************************************************************/ /* The socket() function returns a socket descriptor, which represents */ /* an endpoint. The statement also identifies that the UNIX */ /* address family with the stream transport (SOCK_STREAM) will be */ /* used for this socket. */ /********************************************************************/ sd = socket(AF_UNIX, SOCK_STREAM, 0); if (sd < 0) {


/***************************************************************

*****/ /* If an argument was passed in, use this as the server,

otherwise */ /* use the #define that is located at the top of

this program. */

/***************************************************************

*****/ memset(&serveraddr, 0, sizeof(serveraddr));

serveraddr.sun_family = AF_UNIX; if (argc > 1) strcpy(serveraddr.sun_path, argv[1]);

else strcpy(serveraddr.sun_path, SERVER_PATH);

/***************************************************************

*****/

/* Use the connect() function to establish a connection to

the */

/* server. */ /******************************************************************

**/ rc = connect(sd, (struct sockaddr *)&serveraddr,

SUN_LEN(&serveraddr)); if (rc < 0) { perror("connect() failed"); break; }

/********************************************************************/ /* Send 250 bytesofa’stothe server */ /********************************************************************/ memset(buffer, ’a’, sizeof(buffer)); rc = send(sd, buffer, sizeof(buffer), 0);


if (rc < 0) {


/********************************************************************/

/* In this example we know that the server is going to respond with */

/* the same 250 bytes that we just sent. Since we know that 250 */

/* bytes are going to be sent back to us, we can use the */

/* SO_RCVLOWAT socket option and then issue a single recv() and */

/* retrieve allofthe data. */ /* */ /* The use of SO_RCVLOWAT is already illustrated in the

server */

/* side of this example, so we will do something different

here. */

/* The 250 bytes of the data may arrive in separate packets,

*/

/* therefore we will issue recv() over and over again until

all */

/* 250 bytes have arrived. */ /***************************************************************

*****/ bytesReceived = 0; while (bytesReceived < BUFFER_LENGTH)

{ rc = recv(sd, & buffer[bytesReceived],

BUFFER_LENGTH - bytesReceived, 0); if (rc < 0) {


break; } else if (rc == 0) {

printf("The server closed the connection\n"); break;

}

/*************************************************************

****/ /* Increment the number of bytes that have been

received so far */

/*************************************************************

****/ bytesReceived += rc; }

} while (FALSE);


/****************************************************************

*******/

/* Close down any open socket descriptors */ /******************************************************************

*****/ if (sd != -1) close(sd);

}

Using AF_UNIX_CCSID address family AF_UNIX_CCSID address family sockets have the same specifications as AF_UNIX address family

sockets. AF_UNIX_CCSID address family sockets can be connection-oriented or connectionless. They

can provide communication on the same system.

Before working with an AF_UNIX_CCSID socket application, you must be familiar with the

Qlg_Path_Name_T structure to determine the output format.

When working with an output address structure, such as one returned from accept(), getsockname(),

getpeername(), recvfrom(), and recvmsg(), the application must examine the socket address structure

(sockaddr_unc) to determine its format. The sunc_format and sunc_qlg fields determine the output

format of the path name. But sockets do not necessarily use the same values on output as the application

used on input addresses.

Socket flow of events: Server application that uses AF_UNIX_CCSID address family


The first example uses the following sequence of API calls:


identifies that the UNIX_CCSID address family with the stream transport (SOCK_STREAM) is used

for this socket. You can also use the socketpair() API to initialize a UNIX socket.




The name space for UNIX domain sockets consists of path names. When a sockets program calls the

bind() API, an entry is created in the file system directory. If the path name already exists, the bind()

fails. Thus, a UNIX domain socket program should always call an unlink() API to remove the

directory entry when it ends.

3. The listen() allows the server to accept incoming client connections. In this example, the backlog is

set to 10. This means that the system queues 10 incoming connection requests before the system starts

rejecting the incoming requests.

4. The server uses the accept() API to accept an incoming connection request. The accept() call blocks

indefinitely, waiting for the incoming connection to arrive.

5. The recv() API receives data from the client application. In this example, the client sends 250 bytes of

data over. Thus, the SO_RCVLOWAT socket option can be used, which specifies that recv() is not

required to wake up until all 250 bytes of data have arrived.



8. The unlink() API removes the UNIX path name from the file system.

Socket flow of events: Client application that uses AF_UNIX_CCSID address family



identifies that the UNIX address family with the stream transport (SOCK_STREAM) is used for this





server.

3. The send() API sends 250 bytes of data that are specified in the server application with the


SO_RCVLOWAT socket option.

4. The recv() API loops until all 250 bytes of the data have arrived.


Example: Server application that uses AF_UNIX_CCSID address family:

This example program shows a server application that uses the AF_UNIX_CCSID address family. /**************************************************************************/

/* This example program provides code for a server application that uses */

/* AF_UNIX address family */

/**************************************************************************/

/**************************************************************************/


/**************************************************************************/

#include <stdio.h>

#include <string.h>



#include <sys/un.h>

/**************************************************************************/


/**************************************************************************/



#define FALSE 0

void main() {

/***********************************************************************/


/***********************************************************************/

int sd=-1, sd2=-1;

int rc, length;


struct sockaddr_un serveraddr;

/***********************************************************************/


/* close() of each of the socket descriptors is only done once at the */

/* very endofthe program. */

/***********************************************************************/

do


{

/********************************************************************/


/* an endpoint. The statement also identifies that the UNIX */



/********************************************************************/

sd = socket(AF_UNIX, SOCK_STREAM, 0);

if (sd < 0)

{


/********************************************************************/

/* After the socket descriptor is created, a bind() function gets a */

/* unique name for the socket. */

/********************************************************************/


serveraddr.sun_family = AF_UNIX;

strcpy(serveraddr.sun_path, SERVER_PATH);

rc = bind(sd, (struct sockaddr *)&serveraddr, SUN_LEN(&serveraddr));

if (rc < 0)

{

perror("bind() failed");

break; }

/********************************************************************/

/* The listen() function allows the server to accept incoming */

/* client connections. In this example, the backlog is set to 10. */

/* This means that the system will queue 10 incoming connection */

/* requests before the system starts rejecting the incoming */

/* requests. */

/********************************************************************/

rc = listen(sd, 10);

if (rc< 0) {

perror("listen() failed");

break; }


/********************************************************************/

/* The server uses the accept() function to accept an incoming */

/* connection request. The accept() call will block indefinitely */

/* waiting for the incoming connection to arrive. */


/********************************************************************/

sd2 = accept(sd, NULL, NULL); if (sd2 < 0) {

perror("accept() failed");

break; }

/********************************************************************/

/* In this example we know that the client will send 250 bytes of */

/* data over. Knowing this, we can use the SO_RCVLOWAT socket */

/* option and specify that we don’t want our recv() to wake up */

/* until all 250 bytes of data have arrived. */

/********************************************************************/

length = BUFFER_LENGTH; rc = setsockopt(sd2, SOL_SOCKET, SO_RCVLOWAT,


perror("setsockopt(SO_RCVLOWAT) failed");

break; }

/****************************************************/ /*

Receive that 250 bytes data from the client */

/****************************************************/

rc = recv(sd2, buffer, sizeof(buffer), 0); if (rc < 0) {


break; }


rc < sizeof(buffer)) {

printf("The client closed the connection before all of the\n");

printf("data was sent\n");

break; }

/********************************************************************/

/* Echo the data backtothe client */

/********************************************************************/

rc = send(sd2, buffer, sizeof(buffer), 0);

if (rc < 0)

{


break;

}

/********************************************************************/

/* Program complete */

/********************************************************************/

} while (FALSE);

/***********************************************************************/



/***********************************************************************/


if (sd2 != -1) close(sd2);

/***********************************************************************/

/* Remove the UNIX path name from the file system */

/***********************************************************************/

unlink(SERVER_PATH);

}

Example: Client application that uses AF_UNIX_CCSID address family:

This example program shows a client application that uses the AF_UNIX_CCSID address family. /**************************************************************************/

/* This example program provides code for a client application for */

/* AF_UNIX_CCSID address family. */

/**************************************************************************/

/**************************************************************************/


/**************************************************************************/

#include <stdio.h>

#include <string.h>

#include <wcstr.h>



#include <sys/unc.h>

/**************************************************************************/


/**************************************************************************/



#define FALSE 0


/* path name of the server as a UNICODE */

/* string, or set the server path in the */

/* #define SERVER_PATH which is a CCSID */

/* 500 string. */

void main(int argc, char *argv[]) {

/***********************************************************************/


/***********************************************************************/


int sd=-1, rc, bytesReceived;


struct sockaddr_unc serveraddr;

/***********************************************************************/


/* close() of the socket descriptor is only done once at the very end */

/*ofthe program. */

/***********************************************************************/

do

{

/********************************************************************/


/* an endpoint. The statement also identifies that the UNIX_CCSID */



/********************************************************************/

sd = socket(AF_UNIX_CCSID, SOCK_STREAM, 0);

if (sd < 0)

{


/********************************************************************/

/* If an argument was passed in, use this as the server, otherwise */

/* use the #define that is located at the top of this program. */

/********************************************************************/


serveraddr.sunc_family = AF_UNIX_CCSID; if (argc > 1) {

/* The argument is a UNICODE path name. Use the default format */

serveraddr.sunc_format = SO_UNC_DEFAULT;

wcscpy(serveraddr.sunc_path.wide, (wchar_t *) argv[1]); }

else {

/* The local #define is CCSID 500. Set the Qlg_Path_Name to use */

/* the character format */

serveraddr.sunc_format = SO_UNC_USE_QLG;

serveraddr.sunc_qlg.CCSID = 500;

serveraddr.sunc_qlg.Path_Type = QLG_CHAR_SINGLE;

serveraddr.sunc_qlg.Path_Length = strlen(SERVER_PATH);

strcpy((char *)&serveraddr.sunc_path, SERVER_PATH); }

/********************************************************************/

/* Use the connect() function to establish a connection to the */


/* server. */

/********************************************************************/

rc = connect(sd, (struct sockaddr *)&serveraddr, sizeof(serveraddr));

if (rc < 0) {

perror("connect() failed");

break; }

/********************************************************************/

/* Send 250 bytesofa’stothe server */

/********************************************************************/

memset(buffer, ’a’, sizeof(buffer));

rc = send(sd, buffer, sizeof(buffer), 0);

if (rc < 0)

{


break; }

/********************************************************************/

/* In this example we know that the server is going to respond with */

/* the same 250 bytes that we just sent. Since we know that 250 */

/* bytes are going to be sent back to us, we can use the */

/* SO_RCVLOWAT socket option and then issue a single recv() and */

/* retrieve allofthe data. */

/* */

/* The use of SO_RCVLOWAT is already illustrated in the server */

/* side of this example, so we will do something different here. */

/* The 250 bytes of the data may arrive in separate packets, */

/* therefore we will issue recv() over and over again until all */

/* 250 bytes have arrived. */

/********************************************************************/

bytesReceived = 0;

while (bytesReceived < BUFFER_LENGTH) {

rc = recv(sd, & buffer[bytesReceived],

BUFFER_LENGTH - bytesReceived, 0);

if (rc < 0)

{

perror("recv() failed"); break;

}

else if (rc == 0)

{

printf("The server closed the connection\n"); break;


}

/*****************************************************************/

/* Increment the number of bytes that have been received so far */

/*****************************************************************/

bytesReceived += rc;

} while (FALSE);

/***********************************************************************/


/***********************************************************************/


}


SOCKET APPLICATION DESIGNS: EXAMPLES

These example programs illustrate the more advanced socket concepts. You can use these example

programs to create your own applications that complete a similar task.

With these examples, there are graphics and a listing of calls that illustrate the flow of events in each of

these applications. You can use the Xsockets tool interactively, try some of these APIs in these programs,

or you can make specific changes for your particular environment.

Examples: Connection-oriented designs

You can design a connection-oriented socket server on the system in a number of ways. These example

programs can be used to create your own connection-oriented designs.

While additional socket server designs are possible, the designs provided in these examples are the most

common.

Iterative server

In the iterative server example, a single server job handles all incoming connections and all data flows

with the client jobs. When the accept() API is completed, the server handles the entire transaction. This is

the easiest server to develop, but it does have a few problems. While the server is handling the request

from a given client, additional clients can be trying to get to the server. These requests fill the listen()

backlog and some of the them are rejected eventually.

Concurrent server

In the concurrent server designs, the system uses multiple jobs and threads to handle the incoming

connection requests. With a concurrent server there are typically multiple clients that connect to

the server at the same time.

For multiple concurrent clients in a network, it is recommended that you use the asynchronous I/O

socket APIs. These APIs provide the best performance in networks that have multiple concurrent

clients.

• spawn() server and spawn() worker

• The spawn() API is used to create a new job to handle each incoming request. After spawn() is

completed, the server can wait on the accept() API for the next incoming connection to be

received.

• The only problem with this server design is the performance overhead of creating a new job each

time a connection is received. You can avoid the performance overhead of the spawn() server


example by using prestarted jobs. Instead of creating a new job each time a connection is

received, the incoming connection is given to a job that is already active. All of the remaining

examples in this topic use prestarted jobs. sendmsg() server and recvmsg() worker The

sendmsg() and recvmsg() APIs are used to handle incoming connections. The server prestarts all

of the worker jobs when the server job first starts.

• Multiple accept() servers and multiple accept() workers

• For the previous APIs, the worker job does not get involved until after the server receives the

incoming connection request. When the multiple accept() APIs are used, each of the worker jobs

can be turned into an iterative server. The server job still calls the socket(), bind(), and listen()

APIs. When the listen() call is completed, the server creates each of the worker jobs and gives a

listening socket to each one of them. All of the worker jobs then call the accept() API. When a

client tries to connect to the server, only one accept() call is completed, and that worker handles

the connection.

Example: Writing an iterative server program

This example illustrates how to create a single server job that handles all incoming connections. When

the accept() API is completed, the server handles the entire transaction.

The figure illustrates how the server and client jobs interact when the system uses the iterative server

design.


Flow of socket events: Iterative server

The following sequence of the socket calls provides a description of the graphic. It also describes the

relationship between the server and worker applications. Each set of flows contains links to usage notes

on specific APIs. If you need more details on the use of a particular API, you can use these links. The

following sequence shows the API calls for the iterative server application:


identifies that the INET (Internet Protocol) address family with the TCP transport



3. The listen() allows the server to accept incoming client connections.




5. The recv() API receives data from the client application.



/**************************************************************************/ /* Application creates an iterative server design */ /**************************************************************************/ #include <stdio.h> #include <stdlib.h> #include <sys/socket.h> #include <netinet/in.h>

#define SERVER_PORT 12345

main (int argc, char

*argv[]) { int i, len, num, rc, on = 1; int listen_sd, accept_sd; char buffer[80]; struct sockaddr_in6 addr;

/***********************************************

**/ /* If an argument was specified, use it

to */ /* control the number of incoming

connections */

/***********************************************

**/ if (argc >= 2) num = atoi(argv[1]);

else num = 1;

/*************************************************/ /* Create an AF_INET6 stream socket to receive */ /* incoming connectionson */ /*************************************************/ listen_sd = socket(AF_INET6, SOCK_STREAM, 0); if (listen_sd < 0) {

perror("socket() failed"); exit(-

1); }

/***********************************************

**/ /* Allow socket descriptor to be

reuseable */

/***********************************************

**/ rc = setsockopt(listen_sd, SOL_SOCKET, SO_REUSEADDR, (char *)&on,

sizeof(on)); if (rc < 0) {


perror("setsockopt() failed"); close(listen_sd); exit(-

1); }

/*************************************************/ /* Bind the socket */ /*************************************************/ memset(&addr, 0, sizeof(addr)); addr.sin6_family = AF_INET6; memcpy(&addr.sin6_addr, &in6addr_any, sizeof(in6addr_any)); addr.sin6_port = htons(SERVER_PORT); rc = bind(listen_sd,

(struct sockaddr *)&addr,

sizeof(addr)); if (rc < 0) { perror("bind() failed"); close(listen_sd); exit(-1);

}

/*************************************************/ /* Set the listen back log */ /*************************************************/ rc = listen(listen_sd, 5); if (rc < 0) {

perror("listen() failed"); close(listen_sd); exit(-1); }

/*************************************************/

/* Inform the user that the server is ready */

/*************************************************/

printf("The server is ready\n");

/*************************************************/ /*

Go through the loop once for each connection */

/*************************************************/

for (i=0; i < num; i++) { /**********************************************/ /* Wait for an incoming connection */ /**********************************************/ printf("Interation: %d\n", i+1); printf(" waiting on accept()\n"); accept_sd = accept(listen_sd, NULL, NULL); if (accept_sd < 0) {

perror("accept() failed"); close(listen_sd); exit(-1);

}


printf(" accept completed successfully\n");

/**********************************************/

/* Receive a message from the client */

/**********************************************/

printf(" wait for client to send us a message\n");

rc = recv(accept_sd, buffer, sizeof(buffer), 0);

if (rc <= 0) { perror("recv() failed"); close(listen_sd); close(accept_sd); exit(-1); } printf("

<%s>\n", buffer);

/**********************************************/ /* Echo the data back to the client */ /**********************************************/ printf(" echo it back\n"); len = rc; rc = send(accept_sd, buffer, len, 0); if (rc <= 0) {

perror("send() failed"); close(listen_sd); close(accept_sd); exit(-1);

}

/**********************************************/

/* Close down the incoming connection */ /**********************************************/

close(accept_sd); }

/*************************************************/

/* Close down the listen socket

*

/ /*************************************************/

close(listen_sd); }

Example: Using the spawn() API to create child processes This example shows how a server program can use the spawn() API to create a child process that inherits

the socket descriptor from the parent.

The server job waits for an incoming connection, and then calls the spawn() API to create children jobs

to handle the incoming connection. The child process inherits the following attributes with the spawn()

API:

• The socket and file descriptors.


• The signal mask.

• The signal action vector.

• The environment variables.

The following figure illustrates how the server, worker, and client jobs interact when the spawn() server

design is used.

Flow of socket events: Server that uses spawn() to accept and process requests


relationship between the server and worker examples. Each set of flows contains links to usage notes on

specific APIs. If you need more details about the use of a particular API, you can use these links. The

first example uses the following socket calls to create a child process with the spawn() API call:



identifies that the INET (Internet Protocol) address family with the TCP transport






5. The spawn() API initializes the parameters for a work job to handle incoming requests. In

this example, the socket descriptor for the new connection is mapped over to descriptor 0 in

the child program.

6. In this example, the first close() API closes the listening socket descriptor. The second close()

call ends the accepted socket.

Socket flow of events: Worker job created by spawn()


1. After the spawn() API is called on the server, the recv() API receives the data from the

incoming connection.

2. The send() API echoes data back to the client.

3. The close() API ends the spawned worker job.

Example: Creating a server that uses spawn():

This example shows how to use the spawn() API to create a child process that inherits the socket

descriptor from the parent.

/**************************************************************************

/

/* Application creates an child process using spawn().

*

/ /**************************************************************************/

#include <stdio.h>

#include <stdlib.h>

#include

<sys/socket.h>

#include

<netinet/in.h>

#include <spawn.h>




*argv[]) { int i, num, pid, rc, on = 1; int listen_sd, accept_sd; int spawn_fdmap[1]; char *spawn_argv[1]; char *spawn_envp[1]; struct inheritance inherit; struct sockaddr_in6 addr;

/***********************************************

**/ /* If an argument was specified, use it

to */ /* control the number of incoming

connections */

/***********************************************

**/ if (argc >= 2) num = atoi(argv[1]);

else num = 1;


perror("socket() failed"); exit(-1); }

/*************************************************/

/* Allow socket descriptor to be reuseable */

/*************************************************/

rc = setsockopt(listen_sd, SOL_SOCKET, SO_REUSEADDR, (char *)&on,

sizeof(on)); if (rc < 0) { perror("setsockopt() failed"); close(listen_sd); exit(-1); }

/*************************************************/

/* Bind the socket */ /*************************************************/

memset(&addr, 0, sizeof(addr)); addr.sin6_family =

AF_INET6; addr.sin6_port = htons(SERVER_PORT);

memcpy(&addr.sin6_addr, &in6addr_any, sizeof(in6addr_any));

rc = bind(listen_sd, (struct sockaddr *)&addr, sizeof(addr)); if

(rc < 0) { perror("bind() failed"); close(listen_sd);


exit(-1); }



/*************************************************/


/*************************************************/


/*************************************************/ /*

Go through the loop once for each connection */

/*************************************************/

for (i=0; i < num; i++) { /**********************************************/ /* Wait for an incoming connection */ /**********************************************/ printf("Interation: %d\n", i+1);


printf(" waiting on accept()\n"); accept_sd

= accept(listen_sd, NULL, NULL); if

(accept_sd < 0) { perror("accept() failed"); close(listen_sd); exit(-1); } printf(" accept completed

successfully\n");

/**********************************************/

/* Initialize the spawn parameters */ /* */ /* The socket descriptor for the new */

/* connection is mapped over to descriptor 0 */

/*inthe child program. */ /**********************************************/

memset(&inherit, 0, sizeof(inherit));

spawn_argv[0] = NULL; spawn_envp[0] = NULL;

spawn_fdmap[0] = accept_sd;

/**********************************************/

/* Create the worker job */ /**********************************************/

printf(" creating worker job\n"); pid =

spawn("/QSYS.LIB/QGPL.LIB/WRKR1.PGM", 1,

spawn_fdmap, &inherit, spawn_argv,

spawn_envp); if (pid < 0) { perror("spawn() failed"); close(listen_sd); close(accept_sd); exit(-1); } printf(" spawn completed

successfully\n");

/**********************************************/

/* Close down the incoming connection since */

/* it has been given to a worker to handle */

/**********************************************/

close(accept_sd);

}


/*************************************************/

/* Close down the listen socket

*

/ /*************************************************/

close(listen_sd); }

Example: Enabling the worker job to receive a data buffer:

This example contains the code that enables the worker job to receive a data buffer from the client

job and echo it back. /************************************************************************

**/ /* Worker job that receives and echoes back a data buffer to a

client */

/************************************************************************

**/

#include <stdio.h>

#include <stdlib.h>



*argv[]) { int rc, len; int sockfd; char buffer[80];

/***********************************************

**/ /* The descriptor for the incoming

connection is */ /* passed to this worker job

as a descriptor 0. */

/***********************************************

**/ sockfd = 0;

/*************************************************/ /* Receive a message from the client */ /*************************************************/ printf("Wait for client to send us a message\n"); rc = recv(sockfd, buffer, sizeof(buffer), 0); if (rc <= 0) {

perror("recv() failed"); close(sockfd); exit(-1); }

printf("<%s>\n",

buffer);


/*************************************************/ /* Echo the data back to the client */ /*************************************************/ printf("Echo it back\n"); len = rc; rc = send(sockfd, buffer, len, 0); if (rc <= 0) {

perror("send() failed"); close(sockfd); exit(-1);

}

/***********************************************

**/

/* Close down the incoming connection */ /***********************************************

**/ close(sockfd);

}

Example: Passing descriptors between processes

These examples demonstrate how to design a server program using the sendmsg() and recvmsg() APIs to

handle incoming connections.

When the server starts, it creates a pool of worker jobs. These preallocated (spawned) worker jobs wait

until needed. When the client job connects to the server, the server gives the incoming connection to one

of the worker jobs.

The following figure illustrates how the server, worker, and client jobs interact when the system uses the

sendmsg() and recvmsg() server design.


Flow of socket events: Server that uses sendmsg() and recvmsg() APIs


relationship between the server and worker examples. The first example uses the following socket calls to

create a child process with the sendmsg() and recvmsg() API calls:


also identifies that the INET (Internet Protocol) address family with the TCP transport





4. The socketpair() API creates a pair of UNIX datagram sockets. A server can use the

socketpair() API to create a pair of AF_UNIX sockets.

5. The spawn() API initializes the parameters for a work job to handle incoming requests.

In this example, the child job created inherits the socket descriptor that was created by

the socketpair().



7. The sendmsg() API sends an incoming connection to one of the worker jobs. The child

process accepts the connection with therecvmsg() API. The child job is not active when the

server called sendmsg().

8. In this example, the first close() API closes the accepted socket. The second close() call

ends the listening socket.

Socket flow of events: Worker job that uses recvmsg()


1. After the server has accepted a connection and passed its socket descriptor to the worker job, the

recvmsg() API receives the descriptor. In this example, the recvmsg() API waits until the server

sends the descriptor.

2. The recv() API receives a message from the client.


4. The close() API ends the worker job.

Example: Server program used for sendmsg() and recvmsg():

This example shows how to use the sendmsg() API to create a pool of worker jobs.

/**************************************************************************/

/* Server example that uses sendmsg() to create worker jobs */ /**************************************************************************/ #include <stdio.h>

#include <stdlib.h> #include <sys/socket.h> #include <netinet/in.h> #include <spawn.h>


main (int argc, char *argv[]) { int i, num, pid, rc, on = 1; int listen_sd, accept_sd; int server_sd, worker_sd, pair_sd[2]; int spawn_fdmap[1]; char *spawn_argv[1];


char *spawn_envp[1]; struct inheritance inherit; struct msghdr msg; struct sockaddr_in6 addr;

/*************************************************/ /* If an argument was

specified, use it to */ /* control the number of incoming connections */

/*************************************************/ if (argc >= 2) num = atoi(argv[1]); else num = 1;



/*************************************************/ /* Allow socket

descriptor to be reuseable */

/*************************************************/ rc =

setsockopt(listen_sd, SOL_SOCKET, SO_REUSEADDR, (char *)&on, sizeof(on)); if (rc < 0) {

perror("setsockopt() failed"); close(listen_sd); exit(-1); }


(struct sockaddr *)&addr, sizeof(addr)); if (rc < 0) { perror("bind() failed");

close(listen_sd); exit(-1); }




/*************************************************/ /*

Create a pair of UNIX datagram sockets */

/*************************************************/ rc

= socketpair(AF_UNIX, SOCK_DGRAM, 0, pair_sd); if (rc

!= 0) { perror("socketpair() failed"); close(listen_sd); exit(-1); }

server_sd = pair_sd[0];

worker_sd = pair_sd[1];

/*************************************************/

/* Initialize parms before entering for loop */

/* */ /* The worker socket descriptor is mapped to */

/* descriptor 0 in the child program. */ /*************************************************/

memset(&inherit, 0, sizeof(inherit)); spawn_argv[0]

= NULL; spawn_envp[0] = NULL; spawn_fdmap[0] =

worker_sd;

/*************************************************/ /* Create each of the worker jobs */ /*************************************************/ printf("Creating worker jobs...\n"); for (i=0; i < num; i++) {

pid = spawn("/QSYS.LIB/QGPL.LIB/WRKR2.PGM", 1,


spawn_envp); if (pid < 0) { perror("spawn() failed");

close(listen_sd);

close(server_sd);

close(worker_sd); exit(-1); } printf(" Worker = %d\n", pid);

}

/*************************************************/

/* Close down the worker side of the socketpair */

/*************************************************/

close(worker_sd);

/*************************************************/



/*************************************************/


/*************************************************/

/* Go through the loop once for each connection */

/*************************************************/

for (i=0; i < num; i++) { /**********************************************/ /* Wait for an incoming connection */ /**********************************************/ printf("Interation: %d\n", i+1); printf(" waiting on accept()\n"); accept_sd = accept(listen_sd, NULL, NULL); if (accept_sd < 0) { perror("accept() failed"); close(listen_sd);

close(server_sd); exit(-1); }

printf(" accept completed successfully\n");

/**********************************************/

/* Initialize message header structure */

/**********************************************/

memset(&msg, 0, sizeof(msg));

/**********************************************/

/* We are not sending any data so we do not */

/* need to set either of the msg_iov fields. */

/* The memset of the message header structure */

/* will set the msg_iov pointer to NULL and */

/* it will set the msg_iovcnt field to 0. */

/**********************************************/

/**********************************************/

/* The only fields in the message header */

/* structure that need to be filled in are */

/* the msg_accrights fields. */ /**********************************************/

msg.msg_accrights = (char *)&accept_sd;

msg.msg_accrightslen = sizeof(accept_sd);


/**********************************************/

/* Give the incoming connection to one of the */

/* worker jobs. */ /* */ /* NOTE: We do not know which worker job will */

/* get this inbound connection. */

/**********************************************/

rc = sendmsg(server_sd, &msg, 0); if (rc < 0) { perror("sendmsg() failed"); close(listen_sd); close(accept_sd); close(server_sd); exit(-1); } printf(" sendmsg completed successfully\n");

/**********************************************/

/* Close down the incoming connection since */

/* it has been given to a worker to handle */

/**********************************************/

close(accept_sd); }

/*************************************************/ /*

Close down the server and listen sockets */

/*************************************************/

close(server_sd); close(listen_sd); }

Example: Worker program used for sendmsg() and recvmsg():

This example shows how to use the recvmsg() API client job to receive the worker jobs.

/**************************************************************************/

/* Worker job that uses the recvmsg to process client requests */ /**************************************************************************/

#include <stdio.h> #include <stdlib.h> #include <sys/socket.h>

main (int argc, char *argv[])

{ int rc, len; int worker_sd, pass_sd; char buffer[80]; struct iovec iov[1]; struct msghdr msg;

/*************************************************/

/* One of the socket descriptors that was */

/* returned by socketpair(), is passed to this */

/* worker job as descriptor 0. */


/*************************************************/

worker_sd = 0;

/*************************************************/ /*

Initialize message header structure */

/*************************************************/

memset(&msg, 0, sizeof(msg)); memset(iov, 0,

sizeof(iov));

/*************************************************/

/* The recvmsg() call will NOT block unless a */

/* non-zero length data buffer is specified */

/*************************************************/

iov[0].iov_base = buffer;

iov[0].iov_len = sizeof(buffer);

msg.msg_iov = iov;

msg.msg_iovlen = 1;

/*************************************************/

/* Fill in the msg_accrights fields so that we */

/* can receive the descriptor */ /*************************************************/

msg.msg_accrights = (char *)&pass_sd;

msg.msg_accrightslen = sizeof(pass_sd);

/*************************************************/


}

/* Wait for the descriptor to arrive */ /*************************************************/ printf("Waiting on recvmsg\n"); rc = recvmsg(worker_sd, &msg, 0); if (rc < 0) {

perror("recvmsg() failed"); close(worker_sd); exit(-1);

} else if (msg.msg_accrightslen <= 0)

{ printf("Descriptor was not received\n"); close(worker_sd); exit(-1);

} else

{ printf("Received descriptor = %d\n", pass_sd); }

/*************************************************/

/* Receive a message from the client */ /*************************************************/

printf("Wait for client to send us a message\n");

rc = recv(pass_sd, buffer, sizeof(buffer), 0); if

(rc <= 0) { perror("recv() failed"); close(worker_sd); close(pass_sd); exit(-1); }

printf("<%s>\n", buffer);

/*************************************************/ /* Echo the data back to the client */ /*************************************************/ printf("Echo it back\n"); len = rc; rc = send(pass_sd, buffer, len, 0); if (rc <= 0) {

perror("send() failed"); close(worker_sd); close(pass_sd); exit(-1);

}

/*************************************************/ /* Close down the descriptors */ /*************************************************/ close(worker_sd); close(pass_sd);


Examples: Using multiple accept() APIs to handle incoming requests These examples show how to design a server program that uses the multiple accept() model for handling

incoming connection requests.

When the multiple accept() server starts up, it does a socket(), bind(), and listen() as normal. It then

creates a pool of worker jobs and gives each worker job the listening socket. Each multiple accept()

worker then calls accept().

The following figure illustrates how the server, worker, and client jobs interact when the system uses the

multiple accept() server design.


Flow of socket events: Server that creates a pool of multiple accept() worker jobs

The following sequence of the socket calls provides a description of the figure. It also describes the

relationship between the server and worker examples. Each set of flows contains links to usage notes on

specific APIs. If you need more details about the use of a particular API, you can use these links. The first

example uses the following socket calls to create a child process:

1. The socket() API returns a socket descriptor, which represents an endpoint. The statement also identifies

that the INET (Internet Protocol) address family with the TCP transport (SOCK_STREAM) is used for

this socket.


3. The listen() API allows the server to accept incoming client connections.

4. The spawn() API creates each of the worker jobs.

5. In this example, the first close() API closes the listening.

Socket flow of events: Worker job that multiple accept()


1. After the server has spawned the worker jobs, the listen socket descriptor is passed to this worker

job as a command line parameter. The accept() API waits for an incoming client connection.

2. The recv() API receives a message from the client.


4. The close() API ends the worker job.

Example: Server program to create a pool of multiple accept() worker jobs:

This example shows how to use the multiple accept() model to create a pool of worker jobs.

/*****************************************************************************/

/* Server example creates a pool of worker jobs with multiple accept() calls */

/*****************************************************************************/

#include <stdio.h>

#include <stdlib.h>


#include <netinet/in.h>

#include <spawn.h>



{ int i, num, pid, rc, on = 1;


int listen_sd, accept_sd; int spawn_fdmap[1]; char *spawn_argv[1]; char *spawn_envp[1]; struct inheritance inherit; struct sockaddr_in6 addr;

/*************************************************/

/* If an argument was specified, use it to */

/* control the number of incoming connections */

/*************************************************/

if (argc >= 2) num = atoi(argv[1]);

else num = 1;



/*************************************************/ /*

Allow socket descriptor to be reuseable */

/*************************************************/ rc

= setsockopt(listen_sd, SOL_SOCKET, SO_REUSEADDR, (char *)&on,

sizeof(on)); if (rc < 0) { perror("setsockopt() failed"); close(listen_sd); exit(-1); }


(struct sockaddr *)&addr, sizeof(addr)); if

(rc < 0) { perror("bind() failed"); close(listen_sd); exit(-1); }

/*************************************************/


/* Set the listen back log */ /*************************************************/ rc = listen(listen_sd, 5); if (rc < 0) {


/*************************************************/

/* Initialize parameters before entering for loop */

/* */ /* The listen socket descriptor is mapped to */

/* descriptor 0 in the child program. */ /*************************************************/

memset(&inherit, 0, sizeof(inherit)); spawn_argv[0] =

NULL; spawn_envp[0] = NULL; spawn_fdmap[0] = listen_sd;

/*************************************************/

/* Create each of the worker jobs */ /*************************************************/

printf("Creating worker jobs...\n"); for (i=0; i < num; i++) {

pid = spawn("/QSYS.LIB/QGPL.LIB/WRKR3.PGM", 1,


spawn_envp); if (pid < 0) { perror("spawn() failed");

close(listen_sd); exit(-1); } printf(" Worker = %d\n", pid);

}

/*************************************************/


/*************************************************/


/*************************************************/

/* Close down the listening socket */ /*************************************************/

close(listen_sd); }

Example: Worker jobs for multiple accept():

This example shows how multiple accept() APIs receive the worker jobs and call the accept() server.


/**************************************************************************/ /*

Worker job uses multiple accept() to handle incoming client connections*/

/**************************************************************************/

#include <stdio.h> #include <stdlib.h> #include <sys/socket.h>


{ int rc, len; int listen_sd, accept_sd; char buffer[80];

/*************************************************/

/* The listen socket descriptor is passed to */

/* this worker job as a command line parameter */

/*************************************************/

listen_sd = 0;

/*************************************************/ /* Wait for an incoming connection */ /*************************************************/ printf("Waiting on accept()\n"); accept_sd = accept(listen_sd, NULL, NULL); if (accept_sd < 0) {

perror("accept() failed"); close(listen_sd);


}

exit(-1); } printf("Accept completed

successfully\n");

/*************************************************/

/* Receive a message from the client */ /*************************************************/

printf("Wait for client to send us a message\n");

rc = recv(accept_sd, buffer, sizeof(buffer), 0);

if (rc <= 0) { perror("recv() failed"); close(listen_sd); close(accept_sd); exit(-1); }

printf("<%s>\n", buffer);

/*************************************************/ /* Echo the data back to the client */ /*************************************************/ printf("Echo it back\n"); len = rc; rc = send(accept_sd, buffer, len, 0); if (rc <= 0) {

perror("send() failed"); close(listen_sd); close(accept_sd); exit(-1);

}

/*************************************************/ /* Close down the descriptors */ /*************************************************/ close(listen_sd); close(accept_sd);


Example: Generic client

This example contains the code for a common client job. The client job does a socket(), connect(), send(),

recv(), and close() operation.

The client job is not aware that the data buffer it sent and received is going to a worker job rather than the

server. If you want to create a client application that works whether the server uses the AF_INET address

family or AF_INET6 address family, use the IPv4 or IPv6 client example.

This client job works with each of these common connection-oriented server designs:

• An iterative server. See Example: Writing an iterative server program.

• A spawn server and worker. See Example: Using the spawn() API to create child processes.

• A sendmsg() server and rcvmsg() worker. See Example: Server program used for sendmsg()

and recvmsg().

• A multiple accept() design. See Example: Server program to create a pool of multiple accept()

worker jobs.

• A nonblocking I/O and select() design. See Example: Nonblocking I/O and select().

• A server that accepts connections from either an IPv4 or IPv6 client. See Example:

Accepting connections from both IPv6 and IPv4 clients.


Socket flow of events: Generic client

The following example program uses the following sequence of API calls

The socket() API returns a socket descriptor, which represents an endpoint. The statement also identifies

that the INET (Internet Protocol) address family with the TCP transport (SOCK_STREAM) is used for this

socket.


server.

2. The send() API sends the data buffer to the worker jobs.

3. The recv() API receives the data buffer from the worker jobs.


/**************************************************************************/ /* Generic client example is used with connection-oriented server designs */ /**************************************************************************/ #include <stdio.h> #include <stdlib.h> #include <sys/socket.h> #include <netinet/in.h>



{ int len, rc; int sockfd; char send_buf[80]; char recv_buf[80]; struct sockaddr_in6 addr;

/*************************************************/ /* Create an AF_INET6 stream socket */ /*************************************************/ sockfd = socket(AF_INET6, SOCK_STREAM, 0); if (sockfd < 0) {

perror("socket"); exit(-1); }

/*************************************************/ /* Initialize the socket address structure */ /*************************************************/ memset(&addr, 0, sizeof(addr)); addr.sin6_family = AF_INET6; memcpy(&addr.sin6_addr, &in6addr_any, sizeof(in6addr_any)); addr.sin6_port = htons(SERVER_PORT);


/*************************************************/

/* Connecttothe server */ /*************************************************/ rc

= connect(sockfd, (struct sockaddr *)&addr, sizeof(struct

sockaddr_in6)); if (rc < 0) { perror("connect"); close(sockfd); exit(-1); } printf("Connect

completed.\n");

/*************************************************/ /*

Enter data buffer that is to be sent


/*************************************************/

printf("Enter message to be sent:\n"); gets(send_buf);

/*************************************************/

/* Send data buffer to the worker job */ /*************************************************/

len = send(sockfd, send_buf, strlen(send_buf) + 1, 0);

if (len != strlen(send_buf) + 1) { perror("send"); close(sockfd); exit(-1); } printf("%d bytes sent\n", len);

/*************************************************/

/* Receive data buffer from the worker job */

/*************************************************/

len = recv(sockfd, recv_buf, sizeof(recv_buf), 0);

if (len != strlen(send_buf) + 1) { perror("recv"); close(sockfd); exit(-1); } printf("%d bytes received\n", len);

/*************************************************/

/* Close down the socket */ /*************************************************/

close(sockfd);

}

Documents

BIT 4206 -Network Programming