Department of Computer Science Courant Institute of Mathematical Sciences New York University

Filterfresh COOTS’98

NYU

Department of Computer Science

Courant Institute of Mathematical Sciences

New York University

Filterfresh: Hot Replication of

Java RMI Server Objects

Arash Baratloo, P. Emerald Chung, Yennun Huang,

Sampath Rangarajan, and Shalini Yajnik

Bell Laboratories

Lucent Technologies


NYUFilterfresh Goals

Support highly-available RMI services in presence of failures

Handle crash failures

Transparent failure masking

Easily integrate into Java RMI


NYURoadmap

Goals RMI Registry architecture & crash failures RMI architecture & crash failures Process group approach to fault tolerance Highly available registry service “Reverse lookup” for masking (state-less) servers failures Towards highly available servers Conclusions


NYURMI in a nutshell

Step 1: a server object registers with the RMI registry running on the local host

Steps 2-3: Clients get server’s remote reference by performing a lookup operation at a known registry

Step 4: Given a remote reference, clients invoke server’s methods through RMI

host A

RMI server

host B

RMI client

#2: lookup <name>

#4: RMI

RMI registry

#3: result <ref>#1: bind <name,ref>


NYULimitations of RMI Registry

Single point of failure Clients need to know a priori which registry to contact Does not allow multiple RMI servers to register under the

same service name Not suited for replicated highly-available RMI server objects

host A

RMI server

RMI registry

bind

host B

RMI client

lookup

RMI server

bind


NYURMI Architecture

The programmer writes the client and server application codes The RMI compiler (rmic) generates the client stub and server skeleton The RMI package implements the RRL and transport layers

Transparent masking of failures must occur below the stub/skeleton levels

client application

client stub

Transport layer

server application

server skeleton

remote referencelayer (RRL)

remote referencelayer (RRL)

program

rmicompiler

inherited


NYUA unified solution

Fault-tolerance based on process group approach Non-faulty processes form a logical group Members interact using a set of group primitives Group primitives are guaranteed to be reliable -- all or nothing Group primitives are guaranteed to be ordered Group members have a consistent view of other group members

Applications built on process groups view events in a synchronous fashion: The group view changes for all members as though it is

instantaneous -- synchronous Events (e.g, send & receive of multicasts) occur in a logical order,

within the same view Members have the same view of the group


NYUFortunately

Process group approach is Well studied Well defined protocols

Process group approach has been used in building general purpose fault-tolerant Middle-ware systems, such as Horus/Ensemble, Transis, etc. Services, such as FT directory and file servers OO systems, such as ISIS+ORBIX, Electra, Orca Java middle-ware systems such as iBus

Seems a good candidate for FT RMI services


NYUBasis for process groups

A GroupManager Class 100% Pure Java built on top of UDP/IP

Implements Group creation Join operation (with atomic state transfer) Leave operation Group multicast operation Failure detection and recovery

All events are reliable and totally ordered


NYUPerformance of group multicast

0

10

20

30

40

50

60

70

Tim

e (m

sec)

1 512 1024

Message size (bytes)

local RMI

remote RMI

multicast-1

multicast-2

multicast-4

multicast-8

PentiumPro 200, Linux 2.030, Fast Ethernet connected by a hub JDK1.1.1 Thread and object serialization influenced the performance?


NYURoadmap

Goals RMI Registry architecture & crash failures RMI architecture & crash failures Process group approach to fault tolerance Highly available registry service “Reverse lookup” for masking (state-less) servers failures Towards highly available servers Conclusions


NYUFT Registry architecture

Embedded a GroupManager class to ensure reliable ordered events Reliable and ordered group operations ensure consistent state Replicated registry service for high availability Supports dynamic joins w/state transfer Detects and removes failed registry servers

host A

registry

groupmanager

host B

registry

groupmanager

FT Registry FT Registry

process group


NYUBind operation

Bind operations are sent to every replica Reliable multicast ensures every replica receives the

event Ordered group operation ensures consistency even if a

new replica joins

host A

RMI server

bind

registry

groupmanager

host B

registry

groupmanager

group multicast


NYULookup operation

host A

RMI client

lookup

registry

groupmanager

host B

registry

groupmanager

answer

Lookup operations are handled locally Provides location transparency to clients

able to locate servers registered at unknown hosts no need to have a priori knowledge of server’s host


NYUPerformance of FT Registry

01020

30405060

7080

Tim

e (

ms

ec

)

bind lookup

Operation

RMI Registry (local)

RMI Registry (remote)

FT Registry-1

FT Registry-2

FT Registry-4

FT Registry-8

PentiumPro 200, Linux 2.030, Fast Ethernet connected by a hub JDK1.1.1


NYURMI & FT Registry

Supports multiple replicated servers to register under the same service name

Object references remain valid after the associated object has failed

FT Registry

client

RRLtransport

stub

server

RRLskel

server

RRLskel

transport

server

RRLskel


NYUIn the event of server failure

The failure is detected below the stub level, and ...

FT Registry

client

RRLtransport

stub

server

RRLskel

server

RRLskel

transport

server

RRLskel

Ouch!


NYU

Failure recovery forstate-less servers

A “reverse” lookup returns the name of a given wire connection The old connection is patched with a connection to a non-faulty server The operation is re-attempted Transparent to the client: illusion of a valid object reference

FT Registry

client

RRLtransport

stub

server

RRLskel

server

RRLskel

transport

reverse lookup


NYUFT server Architecture

Client has the illusion of a single server In reality, a group of servers process clients requests Operations are performed at each server, in the same

order for consistency Replicated servers for high availability

serverskel

transport

groupmagr

serverskel

groupmagr

serverskel

groupmagr

client

RRLtransport

stub

process group


NYUConclusions and future work

To provide high availability there is need for A reliable registry service A reliable RMI architecture

Showed suitability of process group approach by Transparently masking failures Easily integrated our services into Java RMI

Future work Complete work on general-purpose FT services Address nested RMI calls for replicated servers

Documents

Department of Computer Science Courant Institute of Mathematical Sciences New York University