1

MSc ProjectMSc Project

Yin ChenYin Chen

Supervised by Dr Stuart AndersonSupervised by Dr Stuart Anderson

20032003

Grid Services MonitorGrid Services Monitor Long Term Monitoring of Grid Services Long Term Monitoring of Grid Services

Using Peer-to-Peer TechniquesUsing Peer-to-Peer Techniques

2

ContentContent

●● Requirements

● ● Architecture

●● Design & Implementation

● ● Implementation Outcome

3

Grid Services MonitorGrid Services Monitor

Requirements …

4

RequirementsRequirements

● ● Monitoring grid services behaviours

● ● Providing long-term historical information

●● Reporting to an end-user or a prediction model

● ● Helping make prediction of services behaviours

5

RequirementsRequirements

●● The system must be fault tolerant

●● Should allow nodes to join and leave dynamically

●● Must be able to scale with the grid

●● Monitoring data must be distributed

6

Architecture …

GridGrid ServicesServices MonitorMonitor

7

ArchitectureArchitecture

●● Peer-to-Peer architecture

●● Why P2P ?

-- Decentralized fashion

-- Much scalability

- - Better fault tolerance

● ● Traditional P2P system designed for file sharing and storing

● ● This project focuses on decentralized data querying and retrieval

8

System ArchitectureSystem Architecture

Monitor Sensor

Data Store

Data Store

Local Collector

Collector

Monitor Sensor

Data Store

Local Collector

Requester

Listener A Listener B

9

TopologyTopology

Peer

Group

A

Peer

Group

B

Peer

Group

C

10

Pull ModelPull Model

●● Data delivered by Pull

● ● Push model: the listener sends out notifications to other peers

● ● Advantages :Advantages :

- - Less network traffic: data deliver only when necessary

-- Has NO time synchronisation problem: collect data from resources at the same time

-- The requester determines query conditions, data type etc.,

making data operation easier

11

Response ModeResponse Mode

12

Query PlanningQuery Planning

Send Send

Union

Receive

Listener A Listener N

Requester

Local QueryLocal Query

Merge Query

Receive …

13

Design & Implementation …

Grid Services MonitorGrid Services Monitor

14

Design & ImplementationDesign & Implementation

●● Monitor Sensor

●● P2P Transport Mechanism

●● Decentralized Data Storage and Querying

15

Monitor SensorMonitor Sensor

●● Implement GT3 ServiceLifecycleMonitor

Create – when an instance of service is going to be created

Destroy – when the instance is going to be destroyed

PreCall – the service is going to be invoked

PostCall – service invocation has finished

PreSerializationCall – input parameters are going to be desterilized

PostSerializationCall – input parameters have been desterilized

●● Problem: can NOT report failure of the service instance

●● Solution: use timeout

16

P2P Transport MechanismP2P Transport Mechanism

●● Based on JXTA platform

●● Queries are scoped to one peer group

●● A requester multicasts a request message, all listeners response by sending back requested data

●● Controlling the messages not to flood the network

-- Data delivered by Pull

-- Reduce the size of messages: Local collector precooked data

-- Reduce the volume of messages

●● Timeout: to finish a data collection process

17

P2P Transport MechanismP2P Transport Mechanism

Monitor Sensor

ListenerA

Data Store

ListenerC

Requester

Data Store

Peer GroupLocal Collector

Collector

Output Pipe

Input PipeOutput Pipe

Input Pipe

Input PipeOutput Pipe

ListenerB

18

Data Storage & QueryingData Storage & Querying

●● Using XML file : light weight and platform independence●● Decentralized manner: Insert handled by Monitor Sensor of each Listener

Retrieve by Local Collector of each Listener Data join by Collector of Requester

●● Data operations: Insert : inserts a ChildNode at the root of log file

Query : XPath Join : adds PeerID as a part of primary key to avoid primary key replication problems

19

Implementation Outcome …

Grid Services MonitorGrid Services Monitor

20

Generating Monitoring DataGenerating Monitoring Data

<?xml version="1.0" encoding="ISO-8859-1" ?>

- <log>

  <service name="Weather Service" date="Aug 14, 2003" start="9:55:43 AM BST" end="9:55:59 AM BST" succeed="true" />

  <service name="Weather Service" date="Aug 14, 2003" start="9:55:43 AM BST" end="9:56:00 AM BST" succeed="true" />

  </log>

Gt3 Server keeps tracing and monitoring

Sample of log record

A Grid Service having a monitor sensor on it

21

Collecting Monitoring DataCollecting Monitoring Data

ListenerA

RequesterA> request

Finish calculating data

RequesterB> request

Finish calculating data

ListenerB

RequesterA> request

Finish calculating data

RequesterB> request

Finish calculating data

ListenerC

RequesterA> request

Finish calculating data

RequesterB> request

Finish calculating data

RequesterA

ListenerA>I got it

ListenerC>I got it

ListenerB>I got it

Finish collecting data

Data collected from 3 peers

RequesterB

ListenerB>I got it

ListenerC>I got it

Finish collecting data

Data collected from 2 peers

22

Displaying Data In Graphical LayoutDisplaying Data In Graphical Layout

Service name Number of responding peersKey

X Axis indicates the duration of the data

Y Axis indicates successful or total running times

Line of successful running times

Line of total running times

Marks on the line indicate the values

23

Displaying Data In Table LayoutDisplaying Data In Table Layout

Service name Number of responding peers

Success time

The date of period

Total running time

24

Comparing Different ServicesComparing Different Services

25

Grid Services MonitorGrid Services Monitor

Thanks !