Large-scale Deployment in P2P Experiments

Using the JXTA Distributed Framework

Gabriel Antoniu, Luc Bougé, Mathieu Jan & Sébastien Monnet

PARIS Research GroupIRISA / INRIA & University of Rennes I & ENS Cachan, France

Euro-Par 2004Pisa, 3rd September

2

How to reproduce and test P2P systems?

Peer-to-Peer (P2P) systems Very large-scale Volatile peers Heterogeneous architecture

Extremely difficult to reproduce and analyze!

Solutions used for testing P2P systems Simulation Emulation Tests on real testbeds

3

Solutions used for testing P2P prototypes

Simulation Emulation Experiments on real testbed

Reproducibility

Yes Limited No

Express the reality

Are we sure?

Yes Yes

Heterogeneity Limited Almost Yes

Deployment and

management

Easy Difficult Very difficult

Experiments on real testbed or via emulation

Crucial step when validating software!

4

Testing P2P systems: a difficult problem!

Papers on P2P systems Lack of experiments

Mostly simulation Real experiments using up to a few ten of physical

nodes Large-scale (thousands of peers) via emulation

The methodology for testing is not discussed Deployment? Volatility?

Technical infrastructures to support testing activities are needed!

Allow developers to focus on interesting problems

“The five commandments

of P2P experiments”

6

Commandment C1

Thou shalt easily specify the requested virtual network of peers

What is needed? Type of peers (application-specific) How they are interconnected Which communication protocols they should

use Where they should de deployed When they should be bootstrapped etc

7

Commandment C2

What is needed? Retrieve outputs of each peer

Log files, result files, etc Store them on the control node of the

experiment Conflicting names

Thou shalt enable designers of P2P prototypes to trace

the behavior of their system

8

Commandment C3

What is needed for grid deployment ? Hierarchical deployment on a federation of

distributed clusters Handling batch systems: PBS, Sun Grid

Engine, etc

Thou shalt efficiently deploy peers on a large number of physical

nodes

9

Commandment C4

What is needed? A peer should have the possibility to wait for

other peers Support complex tests consisting of several

stages

Thou shalt allow peers to synchronize between stages of a test

10

Commandment C5

What is needed? Artificially enforce various volatile behaviors

The testbed is assumed to be stable Simulate correlated failures, network

partitions Take into account the status of the peer to

play the role of the enemy

Thou shalt control the simulation of peers’ volatility

11

Example: deploying JXTA

JXTA: open-source framework for programming P2P applications

Specify a set of protocols Different type of peers

Edge peers Rendezvous peers Relay peers

Rdv

Rdv

Rdv

Rdv

Rdv

Edge

Edge

Edge

Edge

Edge

Edge

Edge

Edge

Rdv

Edge

Conducting JXTA-based experiments:the JXTA Distributed Framework (JDF)

13

The JXTA Distributed Framework (JDF)

A framework for automated testing of JXTA-based systems from a single node (control node)

Started by Sun Microsystems http://jdf.jxta.org

Hypothesis All the nodes must be “visible” by the control node

Requirements Java Virtual Machine Bourne shell ssh/rsh configured to run with no password on each node

Set of shell scripts Deploy, configure, run, log, analyze, kill, cleanup,

update, etc

14

Required JDF files to specify a distributed test

Network description file Defines the requested JXTA-based network Notion of peer profile

Set of Java classes extending the JDF framework

Behavior of each peer Analyze the results

Node file List of physical nodes Path of the JVM on each physical node

Package file List of jar files to deploy on each physical node

15

The JDF specification language (C1)

<network analyze-class=“prototype.test.Analyze”>

<profile name=“Rendezvous”><peer instances=“1”/><rdvs is-rdv=“true”/><transports> <tcp base-port=“13000”/> <transports/><bootstrap class=“prototype.test.Rendezvous”/>

</profile>

<profile name=“Edge”><peer instances=“3”/><rdvs is-rdv=“false”> <rdv profile=“Rendezvous”/>

<rdvs/><transports> <tcp base-port=“13000”/> <transports/><bootstrap class=“prototype.test.Edge”/>

</profile>

</network>

16

A basic Java test class inside JDF’s framework

public class Edge extends JxtaBootStrapper {public static void main(String[] args) {

Edge peer = new Edge(); peer.start(args); peer.stop(); }public void start(String[] args) { // Start the test on the local peer

super.startJxta(); p2pService = group.lookupService(...); p2pService.doSomething();

}public void stop() { // Stop the test on the local peer

p2pService.stopApp();super.stopJxta();

}protected void updateProperties() { // Store the local results

super.updateProperties(); setProperty(PROPERTY_TAG, property_result);}

17

Does JDF observe the 5 commandments?

JDF specification language (C1) Is it enough?

Collecting log and result files (C2)

And that’s all!

Our contribution to the JXTA Distributed Framework

(JDF)

19

Improving JDF: a more concise specification language (C1)

Original JDF requires 1 profile for each physical node

Not scalable!

Enhanced network description file Ability to specify that a single profile can be shared

by multiple physical nodes

<profile name=“RendezVous1” replicas=“1”/><profile name=“RendezVous2” replicas=“1”/><profile name=“Edge1” replicas=“9”/><profile name=“Edge2” replicas=“9”/>

20

Improving JDF: a more concise specification language (C1)

Original JDF requires 1 profile for each physical node

Not scalable!

Enhanced network description file Ability to specify that a single profile can be shared

by multiple physical nodes

<profile name=“RendezVous1” replicas=“1”/><profile name=“RendezVous2” replicas=“1”/><profile name=“Edge1” replicas=“99”/><profile name=“Edge2” replicas=“99”/>

21

Improving JDF: interacting with grid resource management systems (C2)

Original JDF CAN’T use clusters managed by batch systems

Many clusters are managed via a batch system

Improvement JDF now supports PBS and Globus The node file is dynamically created

Possible future extension How to deal with co-scheduling jobs across

multiple batch systems Beyond the scope of JDF-like tools

22

Improving JDF: controlling the simulation of volatility (C5)

High volatility Most significant feature of P2P systems

Improvement Basic support of failure injection in JDF

Pre-computed failure control file Global MTBF of the network peers

Possible future extensions More complex failure injection mechanisms

Correlated failures? Network partitions?

23

Using JDF with JXTA-based projects

Several projects JXTA’s benchmarks (part of JDF project) Deploy Meteor peers

Chord-like system above JXTA

Our JXTA-based service: JuxMem Data sharing service for grid computing environments Compromise between DSM systems and P2P systems Transparent access to mutable data via a global ID

Use of JDF in JuxMem Tests Benchmarks

24

Preliminary results: deployment, configuration and update

0

10

20

30

40

50

60

70

80

90

0 10 20 30 40 50 60

Number of physical nodes

Tim

e in

sec

onds

deployment (JXTA+JDF+JuxMem) configuration (JXTA peers) update (JuxMem)

JXTA and its dependancies =~ 4MB

25

Preliminary results: various volatility conditions

0

10

20

30

40

50

60

70

0 5 10 15 20 25 30

Time in minutes

Num

ber

of p

eers

rem

ainin

g

experimental results theoretical curve

Global MTBF of 1 minute

26

Conclusion

Testing P2P systems is difficult!

We state“The five commandments of P2P experiments”

We enhanced JDF to fulfill some commandments More precise and concise specification language (C1) Handling batch systems (C3) Basic control of the volatility conditions (C5) ?

Are we missing other commandments?

27

Ongoing work: fully observe the 5 commandments

Enhancement to the specification language (C1)

We can always do better, so …

Hierarchical deployment (C3) Like Ka-run/Taktuk (ID IMAG, France)

Distributed synchronization mechanism (C4) Support more complex tests

Improved volatility control (C5) Part of Sébastien Monnet’s PhD (IRISA, Rennes)