UPPSALA DATABASE LABORATORY Managing Scientific Queries over Distributed Data in a Grid Environment Ruslan Fomkin

UPPSALA DATABASE LABORATORY

Managing Scientific Queries over Distributed Data in a Grid

Environment

Ruslan Fomkin

January 20, 2006 NGN workshop Uppsala

2

UU- IT - UDBL Ruslan Fomkin

Uppsala DataBase Laboratory (UDBL)

Supervisor• prof. T. Risch

Database research• How to make extensible middleware query

processing allowing scalable and application oriented search to different kinds of wrapped information sources

http://www.it.uu.se/research/group/udbl/


3


AMOS II

Virtual Mediator Database

Simulation Visualization Analysis

PatientMonitoring

GRID hist.Measurments

RelationalDatabases

Plug-ins

Wrappers

Queries and views

Queries

Data sources

Applications

Continuous

Queries


4


Ongoing Research at UDBL

Stream Queries on BlueGeneErik Zeitler, MSc

FEM DatabasesKjell Orsborn, PhD

Mediating Web ServicesManivasakan Sabesan, BSc

Semantic Web Queriesto Hidden WebJohan Petrini, MSc

Stream Data ManagerMilena Ivanova, PhD

UDBL

Expensive GRID Queries Ruslan Fomkin, MSc


5


Outline

Introduction The project Test application Developed framework Conclusion Future work


6


Scientific Applications, Grid and Databases

A lot of scientific data• Complex structure• Stored in files distributed in Grid

Scientific analyses can be represented as declarative queries• Complex queries with numerical computations• Long running or batch queries

Utilization of computational resources of Grid


7


Parallel Object Query System for Expensive Computations (POQSEC)

Query processor for scientific applications• high-level interface to specify the analyses• automatically generates execution plans and

evaluates them Requirements

• Scalable, efficient, flexible, transparent Properties

• Distributed and parallel


8


Layered Architecture of the System

POQSEC provides• scientific query management

Grid provides• computation management• file management

NorduGrid Middleware Application area provides

• computational libraries• data management libraries

ROOT library

POQSEC

Applicationlibraries

Grid

Data Clusters

User

ROOT NorduGrid


9


Our Test Application

From Particle Physics Analysis of collision events for presence of

Higgs particles Data produced by ATLAS simulation software

• stored in files • distributed in the Grid (e.g. NorduGrid)• managed by ROOT library


10


Object-Relational Schema of the Application Data

Event Particle

Lepton

Muon Electron Jet

particles1 n

PxMiss PyMissPx Py Pz

Kf

Ee

inheritancerelationship


11


General Query of the Analysis

Selection of those events that satisfy predicates containing numerical operations

SELECT ev FROM Event ev WHERE jetvetocut(ev) AND zvetocut(ev) AND topcut(ev) AND misseecuts(ev) AND leptoncuts(ev)AND threeleptoncut(ev);

Each predicate called cut in application area Predicates are defined as queries


12


Example of a predicate: Z-veto cut

Either event does not have a pair of opposite charged leptons

or invariant mass of the pair is not close to the mass of a Z particle

CREATE FUNCTION zvetocut(Event ev)-> Event AS SELECT evWHERE NOTANY(oppositeLeptons(ev)) OR abs(invMass(oppositeLeptons(ev)) - zMass)

>= minZMass;

CREATE FUNCTION oppositeLeptons (Event ev) -> bag of <Lepton, Lepton> AS

SELECT l1, l2 FROM Lepton l1, Lepton l2WHERE l1 = particles(ev) AND l2 = particles(ev) AND Kf(l1) = -Kf(l2);


13


Current Framework

Basic tool for utilizing NorduGrid through Advanced Resource Connector (ARC)

Submission mechanism• submit query • parallelize query to several subqueries• generate job scripts (one per subquery)

Babysitter functionality Data exchange mechanism through files


14


Client and Coordinator PartPOQSEC client personal

database with application schema

ROOT wrapper

Coordinator server receives queries creates jobs

Grid Meta-Database computational

resources data files

Babysitter

Coordinatorserver

Grid Meta-Database

SubmissionDatabase

Job queue

QueryCoordinator

Local Storage

ARCClient

Grid ClientNode

POQSECClient

Submission Database received

submissions created jobs

Babysitter interactions with

ARC


15


Query SubmissionQuery submission query file name

selection degree of

parallelism CPU time for

each job

Submission and its jobs saved in Submission Database

Created jobs added to Job queue Script files saved to Local Storage

Babysitter

Coordinatorserver

Grid Meta-Database

SubmissionDatabase

Job queue

QueryCoordinator

Local Storage

ARCClient

Grid ClientNode

POQSECClient

Coordinator server creates jobs same query partitions of data with equal size same CPU time provided by user corresponding job script files


16


Jobs Submission

Babysitter

Coordinatorserver

Grid Meta-Database

SubmissionDatabase

Job queue

QueryCoordinator

Local Storage

ARCClient

Grid ClientNode

POQSECClient

Babysitter Takes jobs

from Job queue Submits each

job to ARC client

Change status of submitted jobs in Submission DB

ARC GridManager

CEARC GridManager

CEARC client finds Computing Element submits job to corresponding ARC

Grid manager


17


Job Execution

ARC Grid Manager downloads input files submits job to Local Batch System

After some delay LBS starts Executor on allocated a CE node

Executor during execution execute given subquery accesses data through

ROOT wrapper saves result to files

on CE Storage

CE

CEStorage

Executor

wrapper

CE node

ARC GridManager

SE SE

LBS Queue


18


Downloading Result

Babysitter

Coordinatorserver

Grid Meta-Database

SubmissionDatabase

Job queue

QueryCoordinator

Local Storage

ARCClient

Grid ClientNode

POQSECClient

ARC GridManager

CE

CEStorage

ARC GridManager

CE

CEStorage

Babysitter polls ARC

client for jobs statuses

requests to download results for finished jobs

Results downloaded to Local StorageUser can retrieve result when all jobs are ready


19


Conclusion

We provide• declarative query interface for representation scientific

queries• parallel query execution in Grid

(generating scripts)• babysitter to keep track of job execution

Query parallelization is importantStandalone desktop Grid, one job Grid, four jobs

Response time 190 min 225 min 24 min

Requested CPU time - 200 min 20 min


20


Future work

Estimation time of executing query Dealing with underestimation of execution time Automatic making decision on degree of

parallelism and resource brokering• adaptive• based on current load and job statistics

Dealing with failures in Grid POOL wrapper


21


Thank you for attention! Your questions?

Documents

UPPSALA DATABASE LABORATORY Managing Scientific Queries over Distributed Data in a Grid Environment Ruslan Fomkin