1 Interprocess Communications Natalia Khuri & Seetharam Samptur

1

Interprocess Communications

Natalia Khuri &

Seetharam Samptur

2

Outline

• Introduction to IPC • Computer networks• Protocols for remote operations • Remote Procedure Calls• Conclusion

3

What is IPC?

• IPC mechanisms allow processes to communicate and to synchronize their actions without sharing the same address space.

• IPC mechanism in a distributed system should make communication transparent.

4

Traditional IPC

• How do we pass information from one process to another?– Pipes, shared memory.

• How do we avoid race conditions and achieve mutual exclusion?– Mutexes, lock variables, semaphores.

• How do we schedule processes when dependencies are present?– Batch, interactive, real-time.

5

Distributed IPC

• How do we support communication over a network?

• How do we hide the distinction between local and remote communication?

• How do we deliver information with minimum latency, maximum throughput and minimum jitter?

• How do we shield one process from failures of another?

• How do we ensure security?

6

Distributed Communication

• All communication in distributed system is based on low-level message passing as offered by the underlying network.

• Process A builds a message in its own address space and executes a system call that causes the operating system to send the message over the network to process B.

7

What Do We Need?

• Good network infrastructure.• Efficient protocol stack.• Transport protocol to handle various types

of data.• Communication models.• Security.

8

Network Requirements

• Ability to transfer any data type:– Bulk, voice, video.

• Minimum latency, maximum throughput, minimum jitter.

• Error detection and correction:– Physical medium is never error-free.

• Different types of networks:– Local Area Networks (LAN)– Metropolitan Area Networks (MAN)– Wide Area Networks (WAN)

9

Asynchronous Transfer Mode

• Communication technology that uses high-bandwidth, low-delay transport technology, and multiplexing techniques.

• ITU standard for cell relay in which multiple service type (such as voice, video, or data) are conveyed in fixed-length.

• ATM networks are connection-oriented.• ATM Services:

– Constant/variable bit rate– Available/unspecified bit rate

10

Switching

• Packet Switching:– No predefined paths, e.g. traditional LAN.

• Circuit Switching:– Specific path set up for each connection, e.g.

telephone networks.

• ATM Switching:– Hybrid of packet and circuit switching.– Multiple connections over a predefined path.

11

ATM Network Operation

ATM devices:– ATM Switches.– ATM endpoints, e.g. workstations, video

coders/decoders.– User-Network Interface (UNI) & Network-Node

Interface (NNI).

12

ATM Virtual Connection• Virtual Channel:

– End-to-end connection

• Virtual Path:– Bundle of virtual channels that have local significance.

13

ATM Path SwitchingATM Switching:• Uses VCI & VPI from the cell.• Uses translation/routing table.

14

ATM Protocol Data Unit

Generic Flow Control Virtual Path Identifier

Virtual Path Identifier Virtual Channel Identifier

Virtual Channel Identifier

Virtual Channel Identifier Payload Type Cell Loss Priority

Header Error Control

48-byte Payload

Bit-8 Bit-4 Bit-1

5-byte Header

48-byte Payload

ATM Protocol Data Unit [CELL]

15

Why 48-byte Cells?

• Telephone quality audio is sampled at 8000 Hz using 8-bit samples.

• PTTs send a packet every 6 msec each containing 48 bytes of audio.– (1/8000) * 48 = (125 usec) * 48 = 6 msec

• Smaller header is required to assist in faster switching of packets in the network.

16

Protocol Organization

• Different agreements (protocols) are needed in message exchanges:– Number of volts per bits– Data type representation– Error detection and recovery

• The Open Systems Interconnection Reference (OSI) model identifies the standard rules that govern the format, contents, and meaning of the messages.

17

OSI Layers

• Physical Layer: – Interface to physical medium.

• Data Link Layer:– Reliable transit of data over the physical link.

• Network Layer:– Routing packets in the network.

• Transport Layer:– Consistent end-to-end delivery of the packet to

applications.

18

Mapping ATM to OSI

• Physical Layer:– medium dependent

transmissions.

• ATM layer: – Connection setup and

passing of the cells.

• ATM Adaptation Layer:– isolates higher layer

protocols from ATM processes.

Physical Layer

ATM

ATM Adaptation Layer

Upper Layer Applications

ATM Layers

19

Demultiplexing

• ISO OSI model allows multiplexing and demultiplexing in six of the seven layers.

• Faster in ATM:– Fewer layers in ATM stack.– Support for virtual circuits at MAC level.– Virtual channels expedite demultiplexing.

20

Classes of Communication Protocols

• Four classes of communication protocols:– Remote Operations (Client/Server,

Request/Response)– Bulk data management (File transfer protocol)– One-to-many communication

(Multi-/Broadcasting)– Continuous Media (Video/Audio)

• The diversity of communication requirements in a distributed system prohibits the use of a single transport protocol.

21

Remote Operations

• The most basic form of communication in distributed systems.– Process A (client process) sends a message to

process B asking it to do some work.– Process B (server process) carries out work and

returns a message with the result.

• Client and server can be the same process.

• Return message can merely be ACK.

22

Bulk Data Transfer

• Very large request or response messages.• Bulk data transfer can be viewed as a

special form of remote operation.– When data goes to the client, the operation

can be viewed as a read operation.– When data comes from the client, the

operation can be viewed as a write operation.

• Request/response protocols are very efficient.

23

One-to-Many Communication

• Services that use replication to tolerate failures.

• Large amounts of internal communication is needed to keep the replicas consistent.

• It is important that this communication reaches all replicas in a well-defined order.

• In practice, replication protocols are always multicast protocols.

24

Continuous Media

• Multimedia networks require low-latency and constant-rate data transport for continuous media and low-latency for “normal” data.

• ATM networks appear to be good at allowing the mixture of these data types.

25

Fundamental Properties of Protocols

• Message delivery:– At-least-once.– At-most-once.

• Feedback:– End-to-end application-level acknowledgement.– Error report on failures.

• Low end-to-end latency.• Support large request/response messages.

26

Communication Failures

• Networks exhibit failures that manifest themselves as lost packets.

• If not all packets are lost, these failures can be corrected using feedback:– Acknowledgements, timeouts.

• Communication protocols cannot be made completely reliable:– Client cannot distinguish a server that went

down from one that has become disconnected.

27

Unresponsive Server

Client Process Server Process

Request

Client Process Server Process

Request

Response

28

Amnesia Failure

• A processor suddenly stops (crashes) and forgets all its state.

• It resumes operations from the initial state (reboots).

• At-most-once message delivery cannot be achieved:– Aim for at-least-once message delivery.

29

Recovering From Failure

• Session management: – Unique session identifier.– Guaranteed at-most-once message delivery.

• End-to-end acknowledgement:– No dependence on protocol stack.– Positive feedback about delivery and response

can only be provided by an end-to-end application level acknowledgement.

30

At-most-once Delivery

• Session management is essential to provide at-most-once delivery.

• Delta-T protocol– Sessions are alive until packets are

exchanged and until a no-traffic timer expires.– New sessions are created by timeouts that

last for a period of ∆T.– ∆T = transmission delay + max. # of retrans.

+ time interval between retransmissions.

31

Delta-T ProtocolClient Server

∆T

Request

End of session

Delta -T protocol

32

Two-way Handshake with Piggybacking

• Positive feedback when no failure occurs. • Protocol:

– Client sends a request and starts a retransmission timer

– Server responds to the client before the retransmission timer expires. The response is also an acknowledgement for the request.

– Client sends the next request, which also acts as an acknowledgement for the prior response.

– If the client does not have any request to send, the client will acknowledge the last response before terminating.

33

Two-way Handshake with Piggybacking

Client Server

Request

Two-way handshake with piggybacking

Response/ACK

Request/ACK

Response/ACK

34

Optimized Two-way Handshaking

• Error report on failures.• Protocol:

Client sends a request and starts a timerWhile retransmission count is not zeroDo

If ACK receivedSend request header only

ElseSend request

Restart timerReduce retransmission count

End While

35

Optimized Two-Way Handshaking

Client Server

Request-0

Optimized Two-way handshake with piggybacking

ACK-0

Response

Seq #0Timer = sACK = False

Timer Expired Request-0 (R)

Seq #0Timer = sACK = TRUE

Timer ExpiredRequest-0

(R)

ACK-0

Seq #1Timer = sACK = False Request-0

36

Packet Blast Protocol

• Support large request and response messages.

• Fixed number of packets sent as a blast and acknowledged as one unit.

• Retransmit all packets in the blast if no acknowledgement received.

37

Packet Blast ProtocolClient Server

Request-0

Packet Blast Protocol

ACK-0

Blast 0Timer = s

Timer Expired Request-0 (R)

Request-1

ACK-1

38

Messages• Initially: hand-coded messages to send

requests and responses:– Hand-coding messages gets tiresome.– Need to worry about message formats.– Have to pack and unpack data from messages.– Have to decode and dispatch messages to

handlers.– Messages are often asynchronous.

• Messages are not a natural programming model (Birrell and Nelson, 1984):– Could encapsulate messaging into a library.– Invoke library routines to send a message.

39

Procedure Calls• Procedure calls - more natural way to

communicate:– Every language supports them.– Natural for programmers to use.

• Idea: Servers can export a set of procedures that can be called by client programs– Similar to module interfaces, class definitions, etc.– Clients just do a procedure call as it they were

directly linked with the server.– Under the covers, the procedure call is converted

into a message exchange with the server.

40

Implementing RPC• No architectural support for making

remote procedure calls.• Simulate it with available tools:

– local procedure calls and – sockets for network communication.

• This simulation makes remote procedure call a language level construct as opposed to sockets, which are an operating system level construct:– Compiler knows that remote procedure calls

need special code.

41

Implementing RPC

• Trick:– Create stub functions that make it

appear to the user that the call is really local.

– A stub function looks like the function that the user intends to call but really contains code for sending and receiving messages over a network.

42

Functional Steps in RPC

W. Richard Steven “UNIX Network Programming”

43

Benefits

• Procedure call interface.• Writing applications simplified:

– RPC hides all network code into stub functions.

– Application programmers don’t have to worry about details:• Sockets, port numbers, byte ordering.

• Presentation layer in OSI model.

44

Implementation Issues

• How do we make this invisible to the programmer?

• What are the semantics of parameter passing?

• How do we bind (locate, connect to) servers?• How do we support heterogeneity (OS,

architecture, language)?• How do we make it perform well?• How do we deal with failures?

– Client and server can fail independently.

45

RPC Messages

• RPC uses a message-based communication scheme to provide remote service.

• RPC messages are well structured (no longer just packets of data):– addressed to an RPC daemon listening to a

port on a remote system– contain an identifier of the function to

execute and the parameters to pass to that function.

46

RPC Model• A server defines the server’s interface

using an interface definition language (IDL)– The IDL specifies the names, parameters, and

types for all client-callable server procedures.• A stub compiler reads the IDL and

produces two stub procedures for client and server:– The server programmer implements the server

procedures and links them with the server-side stubs.

– The client programmer implements the client program and links it with the client-side stubs.

– The stubs are responsible for managing all details of the remote communication between client and server.

47

RPC Stubs• A client-side stub looks to the client as if it

were a callable server procedure.• A server-side stub looks to the server as if a

client called it.• The client program thinks it is calling the

server (In fact, it is calling the client stub).• The server program thinks it is called by

the client (In fact, it is called by the server stub).

• The stubs send messages to each other to make the RPC happen “transparently”.

48

Marshalling

• Marshalling is the packing of procedure parameters into a message packet.

• The RPC stubs call type-specific procedures to marshal (or unmarshall) the parameters to a call– The client stub marshals the parameters into a

message.– The server stub unmarshalls parameters from

the message and uses them to call the server procedure.

• On return:– The server stub marshals the return parameters.– The client stub unmarshalls return parameters

and returns them to the client program.

49

Marshalling Goals

• Linearize the data structures for transport in messages and reconstructing the original data structures at the other end.

• Convert data structures from the data representation on the calling process to that of the called process.

50

Linearizing• Pointers:

– Do not marshal the data pointed to, but generate a call-back handle.

– Avoid unnecessary copying of data across the interface.

– Multiple pointers to the same data will usually go unrecognized.

• Implicit typing:– only values are transmitted, not data types or

parameter info, e.g., Sun XDR.• Explicit typing:

– Type is transmitted with each value, e.g., ASN.1, XML.

51

Data Representation

• Use representation identical to that of one of the participating machines.

• Support multiple representations to avoid translations.

• Define a machine-independent representation of data– Sun’s XDR (eXternal Data Representation).

52

Semantics of RPC

• Local procedure calls fail only under extreme circumstances.

• RPCs can fail, or be duplicated and executed more than once due to common network errors.– Ensure at-most-once delivery by attaching to

each message a timestamp and require the server to keep history of the processed messages.

53

Where To Bind?

• Solution 1 (Birrell & Nelson’s 1984 proposal):– Centralized database that can locate a host that

provides a particular service.– Server sends message to central authority stating

its willingness to accept certain remote procedure calls (and sends port number).

– Clients then contact this authority when they need to locate a service.

• Solution 1 is problematic for Sun NFS:– identical file servers serve different file systems.

54

Where To Bind?

• Solution 2: Require client to know which host it needs to contact:– A server on that host maintains a

database of locally provided services.

55

Rendezvous

• Typically, an operating system provides a rendezvous daemon also known as Port Mapper on a fixed RPC port.

• A client sends a message, containing the name of the RPC to the rendezvous daemon requesting the port address of the required RPC.

• The port number is returned, and the RPC call may be sent to that port until the process terminates or the server crashes.

56

Transport Protocol

• Which one?– Some implementations may offer only

one (e.g. TCP).– Most support several:

• Allow programmer (or end user) to choose.

57

When Things Go Wrong• Local procedure calls do not fail:

– If they core dump, entire process dies.

• More opportunities for error with RPC:– Server could generate error.– Network problems.– Server crash.– Client might crush while server is still

executing code for it.

• Transparency breaks here:– Applications should be prepared to deal with

RPC failure.

58

When Things Go Wrong

• Local procedure call: exactly once.• Exactly once may be difficult to achieve

with RPC.• A remote procedure call may be called:

– 0 times: server crashed or server process died before executing server code.

– 1 time: everything worked well.– 1 or more: excess latency or lost reply from

server and client retransmission.

59

RPC Semantics

• Most RPC systems will offer either:– At-least-once semantics.– Or at-most-once semantics.

60

Programming With RPC

• Language support– Most programming languages have no

concept of remote procedure calls.– Language compilers will not generate client

and server stubs.

• Common solution:– Use a separate compiler to generate stubs

(pre-compiler).

61

Interface Definition Language

• Allow programmer to specify remote procedure interfaces (names, parameters, return values)

• Pre-compiler can use this to generate client and server stubs:– Marshaling code– Unmarshaling code– Network transport routines– Conform to defined interface– Similar to function prototypes

62

Writing an RPC Program

• Client code has to be modified:– Initialize RPC-related options.– Transport type.– Locate server/service.– Handle failure of remote procedure call.

• Server functions– Generally need little or no modification.

63

ONC RPC

• RPC for Unix System V, Linux, BSD:– ONC RPC (Open Network Computing) and Sun RPC.

• Sun provides a compiler that takes the definition of a remote procedure interface and generates the client and server stub functions.– This compiler is called rpcgen.

• Before running this compiler, the programmer has to provide the interface definition. – The interface definition contains the function

declarations, grouped by version numbers (to support older clients connecting to a newer server) and a unique program number.

64

Sun RPC

• Other components provided by Sun are XDR and a run-time library that implements the necessary protocols and socket routines to support RPC.

• All the programmer has to write is a client procedure, the server functions, and the RPC specification.

65

SUN RPC IDL

name.xprogram GETNAME {

version GET_VERS {long GET_ID(string<50>) = 1;string GET_ADDR(long) = 2;

} = 1; /* version */} = 0x31223456;

66

rpcgen

$ rpcgen name.x• produces:

– name.h– name_svc.c – name_clnt.c– [ name_xdr.c ]

• Function names are derived from IDL function names and version numbers

• Client gets pointer to result– Allows it to identify failed RPC (null return)

67

Server-side

• Start server– Server stub creates a socket and binds any

available local port to it.– Calls a function in the RPC library:

• svc_register to register program#, port #– contacts portmapper (rpcbind on SVR4):

• Name server• Keeps track of {program#,version#,protocol}

port# bindings– Server then listens and waits to accept

connections.

68

Client-side

• Client calls clnt_create with:– Name of server– Program #– Version #– Protocol#

• clnt_create contacts port mapper on that server to get the port for that interface– early binding – done once, not per every

procedure call.

69

Advantages

• No need to get a unique transport address (port):– With SUN RPC you need a unique program

number per server.• Greater portability.• Transport independent:

– Protocol can be selected at run-time.• Application does not have to deal with maintaining

message boundaries, fragmentation, reassembly.• Applications need to know only one transport

address:– Port mapper

• Function call model can be used instead of send/receive.

70

RPC Transparency

• One goal of RPC is to be as transparent as possible:– Make remote procedure calls look like local

procedure calls.– We have seen that binding breaks transparency.

• Failures – remote nodes/networks can fail in more ways than with local procedure calls:– Need extra support to handle failures well.

• Performance – remote communication is inherently slower than local communication.– If program is performance-sensitive, could be a

problem.

71

Conclusion

• Distributed IPC• ATM and ATM protocol stack • Transport layer

– Session management– Two-way handshaking– Blast protocol

• Communication models– Remote procedure calls

72

References

• Black U.D., Emerging Communications Technologies.

• Mullender, S.J. Distributed Systems, 2nd edition, 1993.

• Cisco ATM Internetworking.• Krzyzanowski P., Notes on Distributed Computing.• Voelker, G.M. “Principles of Operating Systems”,

lecture notes, 2004.• http://www.webopedia.com.