Communication essentials

04/22/23 Distributed Systems - Comp 655 1

Communication essentials• Communication patterns• Communication structure: OSI reference

model and TCP/IP coverage• Major middleware communication

services– RPC– RMI– http– I hate to wait …– Message passing– (skip) Streams


Communication patterns

Producer Consumer

query

response


Communication patterns with broker

Producer 2

query

response

Producer 1Consumer 2

Consumer 1

Consumer 3

Subscriptionservice

Client 2

Client 1Server 2

Server 1

Server 3

Requestbroker

queryresponse


Communication patterns -other variations

• Blocking or non-blocking• Connection-based or connectionless• Transient or persistent

Examples:• blocking transient query-response ==

RPC• non-blocking persistent

producer/consumer == message queuing


A note about blocking• Blocking communication blocks the

thread that makes the call• Other work may continue in other

threads in the same process• More about threads and processes

next week


Communication essentials• Communication patterns• Communication structure: OSI

reference model and TCP/IP coverage• Major middleware communication services

– RPC– RMI– http– I hate to wait …– Message passing


OSI Reference Model

2-1

TCP/IP

sending bitse.g. Ethernetconnectionless IPTCP, UDP

not usednot used

HTTP, FTP, …


Middleware Protocols

2-5RPCRMI


Where does http fit?• Originally, it looked like an

application-level protocol, where the application was fetching and viewing HTML pages

• As the Web has matured, it has been used increasingly as middleware.

• It’s the foundation for– Web applications– Web services



model and TCP/IP coverage• Major middleware communication services



RPC is all about• Allowing a client to make a

procedure call that is processed on a remote machine

• Without the client’s having to care (much)


Ordinary procedure callprocess

caller

callee(callee is atanh)

0.98 2.3


RPC: call the procedure across a network

clientprocess

serverprocess

network

caller callee

clientstub

serverstub

0.98

Atanh, 0.98message

0.98


RPC: call the procedure across a network

clientprocess

serverprocess

network

caller callee

clientstub

serverstub

2.3

OK, 2.3message

2.3


RPC Summaryclient

processserver

process

network

caller

clientstub

callee

serverstub

•client stub pretends to be the callee•server stub pretends to be the caller•caller and callee are written as if they were on the same machine•location transparency!!

(almost)


Warning: passing parameters and results

can be tricky


Passing Value Parameters (1)


Passing Value Parameters (2)

a) Original message (JILL, 5) on the Pentium (little endian)b) The message after receipt on the SPARC (big endian)c) The message after being inverted. The little numbers in

boxes indicate the address of each byte

Number is backwards String is backwards


Conclusion from the byte order problem

• You have to explicitly define your interfaces.

• This problem is one of the fundamental motivators for the use of Interface Definition Languages (IDL)

• Passing value parameters is a key component of access transparency.


Interface definitionInterface definitions are critical

factors in • Openness• Flexibility• Access transparency• Enabling proxy construction in

middleware


Activity – what can go wrong?

(with an RPC)Brainstorm things that can go wrong

when a program on machine A tries to make an RPC to a server on machine B

A BNetwork


Pick the top three• The instructor will pick some of the

failures you brainstormed• Multi-vote on the ones in whose

solutions you are most interested• You will use the top three vote-

getters in the next activity


Activity – what could you do about it?

Brainstorm approaches to dealing with the top three failure modes

A BNetwork


Communication patterns -reminder

Parameters:• Query-response or producer-consumer• Blocking or non-blocking• Connection-based or connection-less• Persistent or transientExamples:• blocking transient query-response == RPC• non-blocking persistent

producer/consumer == message queuing


Finding the server or peer• In all cases, the process that initiates a

communication has to find the process it wants to communicate with

• The address could be in– Source code (simplest, least flexible)– Configuration file– A network directory service (requires client

to know a name for a server) (e.g. DNS)– A series of directories (e.g. LDAP, then RMI

Registry)• Interaction with the directory is usually

blocking request/response






Focus on Remote Objects• Here, object ==

state + methods + interface• Usually, only the interface is

distributed• “Remote objects” refers to

approaches in which, for each object there is some server where all of it state resides


Distributed Objects

• Common organization of a remote object with client-side proxy.

2-16


Basic Java RMI

rmiregistry**

Server

Client1. Server registers its name

and remote object2. Client gets reference to

remote object from rmiregistry

3. Client calls server

Interface.class_Stub.class *

1.

2.

3.

Interface.class_Skel.class *

* = not needed if Client and Server are both Java 1.5 or newer** = RMI Registry can be a separate process or an object inside the Server process. If separate, must be on same machine.


Passing objects around …• In RMI, parameters and return

values can be object references– If the parameter refers to a local

object, do you copy the object or pass only a reference?

– If the parameter refers to a remote object, what do you pass?


Local, remote objects in Java RMI

• Local objects are passed by value• Remote objects by reference• Passing a remote object by

reference means passing a proxy by value


Parameter Passing in RMI


http• Communication patterns:

– Transient– Query/response

• But, http does NOT simulate a procedure call– Non-blocking (officially)

• Many clients choose to block• HTTP 1.1 is defined in RFC 2616• Depends on TCP for reliable

communication

http://tools.ietf.org/html/rfc2616


Steps in an HTTP interaction

1. Client requests TCP connection2. Client and server collaborate to

create a TCP connection3. Client sends an HTTP request4. Server sends a response5. TCP connection torn down

• In HTTP 1.1, connection can be kept alive


Anatomy of an HTTP request

POST /don/demo HTTP/1.1accept: */*accept-language: en-usreferer: http://localhost:8080/don/echoclient.htmcontent-type: application/x-www-form-urlencodedaccept-encoding: gzip, deflateuser-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; NovaPacs Viewer 6.0.197.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)host: localhost:8080content-length: 37connection: Keep-Alivecache-control: no-cache

name=007&mission=deep%20dark%20secret

methodrequest URI protocolheaders

content


Watch out for Norton (and others)

POST /don/demo HTTP/1.1accept: */*accept-language: en-us-------: ----:-----------:-----------------------content-type: application/x-www-form-urlencoded---------------: ----- -------user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; NovaPacs Viewer 6.0.197.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)host: localhost:8080content-length: 37connection: Keep-Alivecache-control: no-cache

name=007&mission=deep%20dark%20secret


Anatomy of an HTTP response

HTTP/1.1 200 OKX-Powered-By: Servlet/2.5Content-Type: text/xml;charset=utf-8Content-Length: 57Date: Sun, 16 Sep 2007 17:55:54 GMTServer: Sun Java System Application Server Platform Edition 9.0_01

<greeting>Hello, 007. You made a POST request.</greeting>

protocolstatuscode

headers

content

reason phrase

04/22/23 Distributed Systems - Comp 655 38 Mak

ing

an H

TTP

requ

est i

n Ja

va


Server side: HttpServlet


Servlet container

client

Container (Tomcat, Glassfish,WebLogic, WebSphere, etc)

your Servlet

DB

Servletbase classrequest

response

whatever


Smudging the transparency …

• Network-unfriendly code• Lots of clients can mean lots of

configuration management


Network-unfriendly code …• In OO, there are usually lots of fine-grained

classes, and lots of getters and setters

• Performance is not good if you do this over a network

Person person = new Person();person.setFirstName(fname);person.setMiddleInitial(initial);person.setLastName(lname);Address address = new Address();address.setStreet(street);address.setCity(city);address.setState(state);person.setAddress(address);…


Network-unfriendlyclient server

domainobject

domainobject


A more network-friendly way

• This is a recurring problem in distributed systems

• Remote façade and Data transfer object patterns provide the core of a solution– Define a façade class with methods that group

multiple fine-grained operations together– Define classes (DTOs) with nothing but getters

and setters for moving groups of data items around

– Keep the business logic in the original class(es) on the server side

http://www.martinfowler.com/eaaCatalog/dataTransferObject.html

http://www.martinfowler.com/eaaCatalog/dataTransferObject.html


Using Remote Façade + DTO1. Client creates a DTO

2. Client calls lots of setters on DTO3. Client calls façade with DTO as

parameter4. Service calls lots of getters on DTO,

setters (and other methods, as needed) on the domain object(s)

5. Façade creates DTO26. Façade calls setters on DTO27. Façade returns DTO2 to client8. Client calls lots of getters on DTO2


Using Remote Façade + DTO

client server

DTO DTOdomainobject

DTO2 DTO2

Façade

domainobject


Remote Façade + DTO code (flavor)

PersonTO pto = new PersonTO();pto.setFirstName(fname);pto.setMiddleInitial(initial);pto.setLastName(lname);pto.setStreet(street);pto.setCity(city);pto.setState(state);Facade facade = new Facade();ResultTO rto = facade.addPerson(pto);…


Remote Façade + DTO consequences

• Reduced network traffic• Reduced coupling between client and

domain model• Costs include

– More classes to deal with– Getters and setters called multiple times

(client DTO.get, server DTO.get + domain.set)• Risks include

– Temptation to put business logic in DTO– Temptation to skip domain model

development and build a DTO-processing system


More about client/domain coupling

• Refactoring is a good thing• Issues with refactoring a network

service– Refactored client needs to be

deployed in many places– Service administrators may not know

where all the clients are– There may be clients the service

developers know nothing about

http://en.wikipedia.org/wiki/Refactoring


A more refactoring-friendly way

• This is a recurring problem in distributed systems

• Patterns include– Migrate code at runtime

• e.g. applet• See Tanenbaum & van Steen, section 3.5,

if interested– Remote façade + DTO helps here, too


Remote façade + DTO (again)

client server

PersonTO PersonTO Person+ name+ addr

client server

PersonTO PersonTO Person+ name

Address

befo

reaf

ter

Façade

Façade


I hate to wait …


RMI is usually synchronous


Asynchronous RPC (1)

a) interconnection between client and server in a traditional RPC

b) The interaction using asynchronous RPC

2-12


Asynchronous RPC (2)

2-13






Looser coupling• In RPC and RMI, client and server

must be running at the same time• This coupling makes them

relatively sensitive to failures and remote-machine policies

• Message queuing enables more loosely-coupled communication


General Architecture of a Message-Queuing System (1)


Message-Queuing Primitives

Primitive MeaningPut Append a message to a specified queue

Get Block until the specified queue is nonempty, and remove the first message

Poll Check a specified queue for messages, and remove the first. Never block.

Notify Install a handler to be called when a message is put into the specified queue.


Consumer message queuing: email

sender’smachine

sender’semailserver

receiver’semailserver

receiver’smachine

host hostcommserver

commserver


Example: IBM MQSeries• General organization of IBM's MQSeries

message-queuing system.2-31


Message Brokers

• The general organization of a message broker in a message-queuing

• system.

2-30


Middleware communication summary

Common name PatternRPC Blocking, transient

request/responseRMI Blocking, transient

request/responsehttp (officially) non-blocking,

transient request/response

Asynchronous RPC

Non-blocking, transient request/response

Message queuing

Non-blocking, persistent producer/consumer


Bonus material


Implementing a Java RMI server and client

1. Define the remote interface2. Write a class that implements it3. Include that class in an application4. Compile5. Generate stub and skeleton with

rmic(not necessary with 1.5)

6. Start rmiregistry, if necessary


1. Define the Interfaceimport java.rmi.Remote;import java.rmi.RemoteException;

public interface HelloInterface extends Remote {String sayHello(String yourName) throws

RemoteException;String sayGoodbye(int howMany) throws

RemoteException;void ping() throws RemoteException;

}


2. Implement the interfacepublic class HelloServer extends UnicastRemoteObject implements HelloInterface {

HelloServer() throws RemoteException {super(Constants.serverPort);System.out.println(toString());

// RegisterRegistry reg =

LocateRegistry.getRegistry(Constants.rmiPort);reg.rebind(Constants.serverName, this);

}

…


2. Implement the interface, con’t

public void ping() throws RemoteException {System.out.println("I've been pinged");

}

public String sayGoodbye(int howMany) throws RemoteException {

StringBuffer sb = new StringBuffer();for(int i=0;i<howMany;i++) sb.append("goodbye ");return sb.toString();

}

public String sayHello(String yourName) throws RemoteException {

return "Hello, " + yourName;}


3. Include in an applicationpublic static void main(String[] args) throws

RemoteException {new HelloServer();

}


6. Start RMI Registry>> start rmiregistry


Implementation continued8. Assign the server a name9. Run the server, using that name10.Write a class that uses the

remote interface (by name)11.Compile12.Run13.Enjoy


8,9. Name the serverpublic class Constants {

public static final int rmiPort = 5555;public static final int serverPort = 43215;public static final String serverName =

"whatever";}


10. Write a clientpublic class HelloClient {

public static void main(String args[]) throws RemoteException, NotBoundException {

Registry reg = LocateRegistry.getRegistry(Constants.rmiPort);

HelloInterface hi = (HelloInterface) reg.lookup(Constants.serverName);

for( String name : args ) {System.out.println(hi.sayHello(name));

}System.out.println(hi.sayGoodbye(args.length));hi.ping();}

}


11. Run>> java HelloClient Alice Bob Carol DaveHello, AliceHello, BobHello, CarolHello, Davegoodbye goodbye goodbye goodbye

>>


RMI Resources• java.sun.com/j2se/1.5.0/docs/guide/

rmi/• java.sun.com/j2se/1.5.0/docs/guide/

rmi/faq.html(Java RMI and Object Serialization FAQ)

• java.sun.com/j2se/1.5.0/docs/tooldocs/windows/rmiregistry.html

Documents

Communication essentials