Distributed Computing

Distributed Computing

Chapter 04: Interprocess Communication

2

Chapter 3

Networking issues for distributed systems Types of network Network Principles

Protocol layers Internetworking Routing

Internet protocols Case Studies

Ethernet MobileLAN ATM

3

Objectives of the lecture

To study the general characteristics of interprocess communication and the particular characteristics of both datagram and stream communication in the Internet.

To be able to write Java applications that use the Internet protocols and Java serialization.

To be aware of the design issues for Request-Reply protocols and how collections of data objects may be represented in messages (RMI and language integration are left until Chapter 5).

To be able to use the Java API to IP multicast and to consider the main options for reliability and ordering in group communication.

4

Introduction The API for the Internet protocols External data representation and marshalling Client-Server communication Group communication Case study: interprocess communication in

UNIX Summary


5

Middleware layer TCP ( UDP) from a programmers point of view

TCP : two way stream UDP : datagram, message passing

Java interface, UNIX Socket marshalling and demarshalling

data form translation Client-Server communication

Request-Reply protocols Java RMI, RPC

Group communication

Introduction

6

Middleware layer (1)

Applications, services

Middlewarelayers

request-reply protocol

marshalling and external data representation

UDP and TCP

Thischapter

RMI and RPC

7

Middleware layer (2)

An adapted reference model for networked communication.

2-5

8

Middleware layer (3) A software layer that

masks the heterogeneity of systems provides a convenient programming abstraction provides protocols for providing general-purpose services

to more specific applications, e.g. naming, security, transaction, persistent storage and event notification

authentication protocols authorization protocols distributed commit protocols distributed locking protocols high-level communication protocols

remote procedure calls (RPC) remote method invocation (RMI)

9

Middleware (4) General structure of a distributed system as middleware.

1-22

10

Middleware and Openness

In an open middleware-based distributed system, the protocols used by each middleware layer should be the same, as well as the interfaces they offer to applications.

1.23

11

Middleware programming models Remote Calls

remote Procedure Calls (RPC) distributed objects and Remote Method

Invocation (RMI) eg. Java RMI

Common Object Request Broker Architecture (CORBA)

Other programming models remote event notification remote SQL access distributed transaction processing

12


UNIX Summary


13

Send and receiveSend and receive Synchronous and asynchronousSynchronous and asynchronous

a queue associated with message destination, Sending process add message to remote queue

Receiving process remove message from local queue Synchronous:

send and receive are blocking operations Asynchronous:

send is unblocking, receive could be blocking or unblocking

(receive notification by polling or interrupt)

The characteristics of interprocess communication (1)

14

Message destinations: Internet address + local port service name: names into server locations at run

time location independent identifiers, e.g. in Mach

Reliability validity: messages are guaranteed to be delivered

despite a reasonable number of packets being dropped or lost

Integrity: messages arrive uncorrupted and without duplication

Ordering the messages be delivered in sender order

The characteristics of interprocess communication (2)

15

Endpoint for communication between processes Both forms of communication (UDP and TCP ) use the socket

abstraction Originate from BSD Unix, be present in Linux, Windows NT and

Macintosh OS etc be bound to a local port (216 possible port number) and one of

the Internet address a process cannot share ports with other processes on the same

computer

Socket

message

agreed portany port socketsocket

Internet address = 138.37.88.249Internet address = 138.37.94.248

other ports

client server

16

App 3App 2App 1

Sockets The application level API to TCP &

UDP Allows users to write application

“protocol objects” which sit on top of TCP and UDP.

Execution of TCP/UDP/IP are in kernel space. Sockets provide the bridge to user space.

In Java provides 3 types of sockets DatagramSocket – for UDP ServerSocket – for TCP server Socket – endpoint of a TCP stream

IP

TCP UDP

socket socket socket

17

UDP datagrams are sent without acknowledgement or retries

Issues relating to datagram communication Message size: not bigger than 64k in size, otherwise

truncated on arrival blocking: non-blocking sends (message could be

discarded at destination if there is not a socket bound to the port ) and blocking receives (could be timeout)

Timeout: receiver set on socket Receive from any: not specify an origin for messages,

but could be set to receive from or send to particular remote port by socket connection operation

UDP datagram communicationUDP datagram communication

18

UDP datagram communication Failure model

omission failure: message be dropped due to checksum error or no buffer space at sender side or receiver side

ordering: message be delivered out of sender order

application maintains the reliability of UDP communication channel by itself (ACK)

19

DatagramPacket DatagramSocket

send and receive : transmit datagram between a pair of sockets

setSoTimeout : receive method will block for the time specified and then throw an InterruptedIOexception

connect: connect to a particular remote port and Internet address

Examples be acceptable to services that are liable to

occasional omission failures, e.g. DNS

Java API for UDP datagrams

20

Datagram Sockets Creating a datagram socket

DatagramSocket soc = new DatagramSocket(port-no.) Creating a datagram packect

DatagramPacket p = new DatagramPacket(byte[] data, int len, InetAddress dest, int dest-port);

Sending a packet soc.send( p )

Receiving a packet q = new DatagramPacket(buff, len);

soc.receive( q );

IP header: source-ip-address destination-ip-address protocol = 17 (udp)

UDP header: source port-no destination port-no

ApplicationData

21

UDP client sends a message to the server and gets a reply

import java.net.*;import java.io.*;public class UDPClient{ public static void main(String args[]){

// args give message contents and server hostnameDatagramSocket aSocket = null; try {

aSocket = new DatagramSocket(); byte [] m = args[0].getBytes();InetAddress aHost = InetAddress.getByName(args[1]);int serverPort = 6789; DatagramPacket request = new DatagramPacket(m, args[0].length(), aHost,

serverPort);aSocket.send(request); byte[] buffer = new byte[1000];DatagramPacket reply = new DatagramPacket(buffer, buffer.length);aSocket.receive(reply);System.out.println("Reply: " + new String(reply.getData()));

}catch (SocketException e){System.out.println("Socket: " + e.getMessage()); }catch (IOException e){System.out.println("IO: " + e.getMessage());}}finally {if(aSocket != null) aSocket.close();}

} }

22

UDP server repeatedly receives a request and sends it back to the client

import java.net.*;import java.io.*;public class UDPServer{

public static void main(String args[]){ DatagramSocket aSocket = null; try{ aSocket = new DatagramSocket(6789);

byte[] buffer = new byte[1000]; while(true){ DatagramPacket request = new DatagramPacket(buffer, buffer.length); aSocket.receive(request); DatagramPacket reply = new DatagramPacket(request.getData(),

request.getLength(), request.getAddress(), request.getPort()); aSocket.send(reply);}

}catch (SocketException e){System.out.println("Socket: " + e.getMessage()); }catch (IOException e) {System.out.println("IO: " + e.getMessage());}}finally {if(aSocket != null) aSocket.close();}

}}

23

The API to the TCP provide the abstraction of a stream of bytes to which data may be

written and from which data may be read Hidden network characteristics

message sizes (how much data to write/read) lost messages (ACK) flow control (match the speed) message duplication and ordering (identifier with each IP Packet) message destinations (establish connection)

Once connection is established, no need of address & ports While connection establishing: client/server

Client Connect request, Server Accept request, becomes Peers Client: creates stream socket, bind to a port, and makes a connect

request to server Server: creates a listening socket, bind to a server port, and waits

for clients

TCP stream communication (1)

24

issues related to stream communication Matching of data items: agree to the contents of the transmitted

data (int & double, write/ read, ) Blocking: send blocked until the data is written in the receiver’s

buffer, receive blocked until the data in the local buffer becomes available

Threads: server create a new thread for every connection failure model

integrity and validity have been achieved by checksum, sequence number, timeout and retransmission in TCP protocol

connection could be broken due to unknown failures Can’t distinguish between network failure and the destination

process failure Can’t tell whether its recent messages have been received or

not

TCP stream communication (2)

25

TCP byte stream sockets Server socket – waits for connections

soc = new ServerSocket( port-no ); Socket newsoc = soc.accept( );

Client socket – connects to server client = new Socket(server-addr, server-port);

When connected, get streams InputStream in = client.getInputStream( );

OutputStream out = client.getOutputStream( )

Server side

Serversocket

Client Side

clientsocket

1. Contact server

Newsocket

2. Create newStream socket

3. Establish stream communication

26

ServerSocket accept: listen for connect requests from clients

Socket constructor

not only create a socket associated with a local port, but also connect it to the specified remote computer and port number

getInputStream getOutputStream

Java API for TCP Streams

27

TCP client makes connection to server, sends request and receives reply

import java.net.*;import java.io.*;public class TCPClient {

public static void main (String args[]) {// arguments supply message and hostname of destinationSocket s = null; try{ int serverPort = 7896; s = new Socket(args[1], serverPort);

DataInputStream in = new DataInputStream( s.getInputStream());DataOutputStream out =

new DataOutputStream( s.getOutputStream());out.writeUTF(args[0]); // UTF is a string encoding see Sn 4.3String data = in.readUTF(); System.out.println("Received: "+ data) ;

}catch (UnknownHostException e){System.out.println("Sock:"+e.getMessage());

}catch (EOFException e){System.out.println("EOF:"+e.getMessage()); }catch (IOException e){System.out.println("IO:"+e.getMessage());}

}finally {if(s!=null) try {s.close();}catch (IOException e){System.out.println("close:"+e.getMessage());}} }}

28

TCP server makes a connection for each client and then echoes the client’s request (1)

import java.net.*;import java.io.*;public class TCPServer { public static void main (String args[]) {

try{int serverPort = 7896; ServerSocket listenSocket = new ServerSocket(serverPort);while(true) {

Socket clientSocket = listenSocket.accept();Connection c = new Connection(clientSocket);

}} catch(IOException e) {System.out.println("Listen :"+e.getMessage());}

}}

// this figure continues on the next slide

29

TCP Server (2)

class Connection extends Thread {DataInputStream in;DataOutputStream out;Socket clientSocket;public Connection (Socket aClientSocket) { try {

clientSocket = aClientSocket;in = new DataInputStream( clientSocket.getInputStream());out =new DataOutputStream( clientSocket.getOutputStream());this.start();

} catch(IOException e) {System.out.println("Connection:"+e.getMessage());}}public void run(){ try { // an echo server

String data = in.readUTF(); out.writeUTF(data);

} catch(EOFException e) {System.out.println("EOF:"+e.getMessage()); } catch(IOException e) {System.out.println("IO:"+e.getMessage());} } finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}}}

}

30


UNIX Summary


31

Why does the communication data need external data representation and marshalling?

Different data format on different computers, e.g., big-endian/little-endian integer order, ASCII (Unix) / Unicode character coding

How to enable any two computers to exchange data values?1. The values be converted to an agreed external format before

transmission and converted to the local form on receipt2. The values are transmitted in the sender’s format, together with an

indication of the format used, and the receipt converts the value if necessary

External data representation An agreed standard for the representation of data structures and

primitive values Marshalling (unmarshalling)

The process of taking a collection of data items and assembling them into a form suitable for transmission in a message

Usage: for data transmission or storing in files Three alternative approaches

CORBA’s common data representation / Java’s object serialization & XML

External data representation and marshalling introduction

32

Represent all of the data types that can be used as arguments and return values in remote invocations in CORBA

15 primitive types Short (16bit), long(32bit), unsigned short, unsigned long, float, char,

… Constructed types

Types that composed by several primitive types A message example The type of a data item is not given with the data

representation in message It is assumed that the sender and recipient have common

knowledge of the order and types of the data items in a message.

RMI and RPC are on the contrary

CORBA’s Common Data Representation (CDR)

33

CORBA CDR for constructed types

Type Representation

sequence length (unsigned long) followed by elements in order

string length (unsigned long) followed by characters in order (can also

can have wide characters)

array array elements in order (no length specified because it is fixed)

struct in the order of declaration of the components

enumerated unsigned long (the values are specified by the order declared)

union type tag followed by the selected member

34

CORBA CDR message

The flattened form represents a Person struct with value: {‘Smith’, ‘London’, 1934}

0–34–78–1112–1516–1920-2324–27

5"Smit""h___" 6"Lond""on__"1934

index in sequence of bytes 4 bytes

notes on representation

length of string

‘Smith’

length of string‘London’

unsigned long

Struct Person { string name; string place; long year;};

35

External data representation-The SUN XDR standard The entire XDR message consists of a sequence of

4-byte objects: example: ‘Hi’, ‘There’, 2001

2

“Hi--”5

“Ther”“e---”2001

sequence length‘Hi’

‘There’

Integer 2001

Must also define which end of each object is the most significant

The marshalled message of the example: 2 Hi 5 There 2001

36

Serialization (deserialization) The activity of flattening an object or a connected set of objects into

a serial form that is suitable for storing on the disk or transmitting in a message

Include information about the class of each object Handles: references to other objects are serialized as handles

Each object is written once only Example Make use of Java serialization

ObjectOutputStream.writeObject, ObjectInputStream.readObject The use of reflection

Reflection : The ability to enquire about the properties of a class, such as the names and types of its instance variables and methods

Reflection makes it possible to do serialization (deserialization) in a completely generic manner

Java object serialization

37

Indication of Java serialized form

The true serialized form contains additional type markers; h0 and h1 are handles

Serialized valuesPerson

31934

8-byte version number

int year

5 Smith

java.lang.Stringname:

6 London

h0

java.lang.Stringplace:h1

Explanation

class name, version number

number, type and name of instance variables

values of instance variables

Public class Person implements Serializable {private String name;private String place;private int year;public Person (String aName, String aPlace, int aYear){

name = aName;place = aPlace;year = aYear;

}// followed by methods for accessing the instance variables

}

Person p = new Person(“Smith”, “London”, 1934);

38

Representation of a remote object reference

Internet address port number time object number interface of remote object

32 bits 32 bits 32 bits 32 bits

•An identifier for a remote object that is valid throughout a distributed system

•Must ensure uniqueness over space and time

•Existence of remote object references

•Life of the remote object references

•Remote object references may not be used for relocated objects

39


UNIX Summary


40

Request-reply protocol - basics Client-server communication in terms of the

send and receive operations in the Java API for UDP datagrams.

Uses remote object references, identifiers for a remote object that is valid throughout a distributed system.

Communication primitives: doOperation. getRequest. sendReply.

41

The request-reply protocol

The request-reply protocol Normally synchronous May be reliable

Reply is ACK Asynchronous may be alternative

Client may retrieve the reply latter. Mostly implemented over UDP but not TCP

Acknowledgements are redundant since requests are followed by replies

Establishing a connection involves two extra pairs of messages in addition to the pair required for a request and a reply

Flow control is redundant for the majority of invocations, which pass only small arguments and results

42


Request

ServerClient

doOperation

(wait)

(continuation)

Replymessage

getRequest

executemethod

messageselect object

sendReply

public byte[] doOperation (RemoteObjectRef o, int methodId, byte[] arguments)sends a request message to the remote object and receives the reply. The arguments specify the remote object, the method to be invoked and the arguments of that method.

public byte[] getRequest ();acquires a client request via the server port.

public void sendReply (byte[] reply, InetAddress clientHost, int clientPort); sends the reply message reply to the client at its Internet address and port.

43

Request-reply message structure

messageType

requestId

objectReference

methodId

arguments

int (0=Request, 1= Reply)

int

RemoteObjectRef

int or Method

array of bytes

44

Request-reply message structure

Fields: 1. Message Type

Request or reply 2. A doOperation generates requestId for each message A server copies in the reply message 3 Remote object reference

marshalled 4. methodId

Message Identifier: requestId Identfier for the sender process, for example its port and internet

address

45


Failure model Problems:

Omission failures. No guarantee of delivery in order.

Timeout doOperation return exception when repeatedly issued requests

are all timeout Duplicate request messages:

filter out duplicates by requestID if the server has not yet sent the reply, transmit the reply after

finishing operation execution Lost Reply Messages

If the server has already sent the reply, execute the operation again to obtain the result. Note idempotent operation, e.g., add an element to a set, and a contrary example, append an item to a sequence

History History: server contains a record of reply messages that have

been transmitted to avoid re-execution of operations

46


RPC exchange protocol

Implement the request-reply protocol on TCP Costly, but no need for the request-reply protocol to deal with retransmission and filtering Successive requests and replies can use the same stream to reduce connection overhead

Name Messages sent byClient Server Client

R RequestRR Request Reply

RRA Request Reply Acknowledge reply

47

HTTP used by web browsers to make requests to web servers and to receive replies from them.

Client requests specify a URL that includes the DNS hostname of a web server, an optional port number and an identification of a resource.

HTTP specifies: Messages. Methods. Arguments. Results. Rules for representing data in messages.

HTTP: an example of a request – reply protocol

48

HTTP: an example of a request – reply protocol

Over TCP Each client-server interaction consists of the following steps

The client requests and the server accepts a connection at the default server port or at a port specified in the URL

The client sends a request message to the server The server sends a reply message to the client The connection is closed

Persistent connection (http1.1, RFC 2616) Connections that remain open over a series of request-reply

exchanges between client and server Closing

Marshalling Request and replies are marshalled into messages as ASCII text

string Resources are represented as byte sequences and may be

compressed MIME (Multipurpose Internet Mail Extensions) to send text,

images an so on. HTTP Methods

GET, HEAD, POST, PUT, DELETE, OPTIONS, TRACE

49

HTTP - methods GET: Requests the resource whose URL is given as

argument. HEAD: Similar to GET, but no data returned. POST: Can deal with data supplied with the request. PUT: data supplied in the request is stored with the

given URL as its identifier either as modification of the an existing resource or as a new resource.

DELETE: Server deletes resource identified by the URL. OPTIONS: Server supplies client with list of methods

that it allows. TRACE: Server sends the request back.

50

HTTP request / reply messages

GET //www.dcs.qmw.ac.uk/index.html HTTP/ 1.1

URL or pathnamemethod HTTP version headers message body

HTTP/1.1 200 OK resource data

HTTP version status code reason headers message body

Request/Reply: Header: for example acceptable content type The message body contains data associated with the

URL specified in the request

51


UNIX Summary


52

The usage of Multicast

Fault tolerance based on replicated services Client request are multicast to all the members of the group,

each of which performs an identical operation Finding the discovery servers in spontaneous networking

Multicast message can be used by servers and clients to locate available discovery services to register their interfaces or to look up the interfaces of other services

Better performance through replicated data Data are replicated to increase the performance of a service,

e.g., Web Cache. Each time the data changes, the new value is multicast to the processes managing the replicas

Propagation of event notification Multicast to a group may be used to notify processes when

something happens, e.g., the Jini system uses multicast to inform interested clients when new lookup services advertise their existence

53

IP Multicast – group communication

A multicast group is specified by a class D Internet address Built on top of IP Sender is unaware of the identities of the individual recipients Available only via UDP

The membership of a group is dynamic It is possible to send datagram to a multicast group without being a member

and join or leave the group at any time. IPv4

Multicast routers use the broadcast capability of the local network MTTL (time to live) - specify the number of routers a multicast message is

allowed to pass Multicast address allocation

Permanent group – exist even without members 224.0.0.1 to 224.0.0.255 Temporary group – the other addresses, set TTL to a small value

Failure model: due to UDP, so it is a unreliable multicast Java API to IP multicast

54

Multicast peer joins a group and sends and receives datagrams

import java.net.*;import java.io.*;public class MulticastPeer{

public static void main(String args[]){ // args give message contents & destination multicast group (e.g. "228.5.6.7")

MulticastSocket s =null; try {

InetAddress group = InetAddress.getByName(args[1]); s = new MulticastSocket(6789); s.joinGroup(group);

byte [] m = args[0].getBytes(); DatagramPacket messageOut =

new DatagramPacket(m, m.length, group, 6789); s.send(messageOut);

// this figure continued on the next slide

55

Multicast peers example… continued

// get messages from others in group byte[] buffer = new byte[1000];

for(int i=0; i< 3; i++) { DatagramPacket messageIn =

new DatagramPacket(buffer, buffer.length); s.receive(messageIn); System.out.println("Received:" + new String(messageIn.getData())); }

s.leaveGroup(group); }catch (SocketException e){System.out.println("Socket: " + e.getMessage());

}catch (IOException e){System.out.println("IO: " + e.getMessage());}}finally {if(s != null) s.close();}

} }

56

Reliability and ordering of multicast

Failures Router failure prevent all recipients beyond it receiving the

message Members of a group receive the same array of messages in

different orders Some examples of the effects of reliability and ordering

Fault tolerance based on replicated services if one of them misses a request, it will become inconsistent

with the others Finding the discovery servers in spontaneous networking

an occasional lost request is not an issue when locating a discovery server

Reliable multicast or unreliable multicast? According to application’s requirement

57


UNIX – FOR SELF STUDY Summary


58

UNIX socket

Datagram communication Datagram Socket Bind Sendto recvfrom

Stream communication stream socket , bind Accept Connect Write and read

59

Sockets used for datagrams

ServerAddress and ClientAddress are socket addresses

Sending a message Receiving a message

bind(s, ClientAddress)

sendto(s, "message", ServerAddress)

bind(s, ServerAddress)

amount = recvfrom(s, buffer, from)

s = socket(AF_INET, SOCK_DGRAM, 0)s = socket(AF_INET, SOCK_DGRAM, 0)

60

Sockets used for streams

Requesting a connection Listening and accepting a connection

bind(s, ServerAddress);listen(s,5);

sNew = accept(s, ClientAddress);

n = read(sNew, buffer, amount)

s = socket(AF_INET, SOCK_STREAM,0)

connect(s, ServerAddress)

write(s, "message", length)

s = socket(AF_INET, SOCK_STREAM,0)

ServerAddress and ClientAddress are socket addresses

61


UNIX Summary


62

Two alternative building blocks Datagram Socket: based on UDP, efficient but

suffer from failures Stream Socket: based on TCP, reliable but

expensive Marshalling

CORBA’s CDR and Java serialization Request-Reply protocol

Base on UDP or TCP Multicast

IP multicast is a simple multicast protocol

Summary

Documents

Distributed Computing