View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Alice Workshop, Aug 21, 2008
1
Recent experiences in Grid-enabling legacy software codes
Dana Petcu
[email protected]://web.info.uvt.ro/~petcu
Prof. & Dir. of Computer Science Dep., West Univ. of Timisoara, www.uvt.roRes. & Dir. of research Institute e-Austria Timisoara, Romania, www.ieat.ro
Alice Workshop, Aug 21, 2008
2
Content
Context Romanian sites & Grid initiatives UVT & IeAT Grid-related activities Grid – where to?
Cases studies of Grid-enabling existing codes For Computational Grids (CFDs, DLs etc) For Service Grids (EOs, CASs etc)
Conclusions
Alice Workshop, Aug 21, 2008
3
RO sites in EGEE-II / Feb 08
Alice Workshop, Aug 21, 2008
4
RO sites in EGEE-III / 20 Aug 08
Locations (SEE-Grid monitoring):
Alice Workshop, Aug 21, 2008
5
UVT in SEE-Grid-Sci / 20 Aug 08
Alice Workshop, Aug 21, 2008
6
Current infrastructureUVT-CS Department’s cluster IeAT’s cluster
(28 CPUs, gLite 3) (30 CPUs, GT 4)
Alice Workshop, Aug 21, 2008
7
Recent history – UVT & IeAT team 2002
UVT’s cluster of 8 IBM PCs with Myrinet connection (2Gps) included in ASC facilities (NSF’s Astrophysics Simulation Collaboration)
2003 Test platform between UVT and RISC-Austria
2004 First prototypes of Grid-enabled codes (CFD + CAS) in
CompGrid project 2005
Started national collaborations MedioGrid (imag.proces.) http://mediogrid.utcluj.ro GridMOSI (optimization) http://www.gridmosi.ro
Alice Workshop, Aug 21, 2008
8
Recent history – UVT & IeAT team 2006:
Involved in FP6 SCIEnce – SymGrid (symb. Comp) http://www.symbolic-computation.org
Involved in national SINRED prj. Digital lib. & Grids Started Grid training and dissemination activities (ISPDC06 symp., GridCAD ws, GridTDT training, PhD programme)
2007: Participation at FP6 EGEE-II and SEE-Grid-II Accepted in RoGrid consortium Participation at national prj. PEGAF (workflow &Grid services)
2008: Participation at FP7 EGEE-3 and SEE-Grid-Sci Leading the ESA’s project: GiSHEO http://gisheo.info.uvt.ro
Alice Workshop, Aug 21, 2008
9
Application prototypes in the last year gLite oriented applications
DEMO/G – Distributed Evolutionary Multiobjective Optimization on Grid
GOC – portal for gLite jobs
GT4 oriented applications SymGrid-Services – Web and Grid services for
symbolic computations components of MedioGrid platform – Web and
Grid services for satellite image processing
Alice Workshop, Aug 21, 2008
10
Back to the international context (1)
European Commission programs: FP6 funds:
Infrastructure (e.g. EGEE-II) IST – Grid Unit F2
FP7 funds: E-Infrastructure (e.g. EGEE-3) ICT – Service Oriented
Architectures (Grid Unit -> Service-Oriented Archs. Unit)
The business pressure companies promoting Web
services & SOA E.g. Aug’08 GridToday newsletter -> On-demand Entreprise Daily news EU projects promoting SOA in
FP7-ICT coming from enterprise-driven consortiums like NESSI
Alice Workshop, Aug 21, 2008
11
Back to the international context (2) NSF’s Cyberinfrastructure Vision – March 07
Grid infrastructure = world-wide Cyber-utility populated by self-* cooperating services which interact as in a complex gigantic software ecosystem
European NGG3 expert group – Jan 2006 Grid services + Knowledge technologies = SOKU
Services are knowledge-assisted + Architecture comprises services + Utility: immediately useable service
Different Grids: Computational Grids Service Grids Instrument Grids …
New-related buzzwords: cloud computing, on demand computing (ODC) Service oriented computing (SOC) Service oriented infrastructures (SOI) Software as a Service (SaaS) …
Alice Workshop, Aug 21, 2008
12
Legacy codes & Grids
Legacy system = a mission-critical software system developed sometime in the past that has been around and has changed for a long time without undergoing systematic remedial actions
Modernization techniques are applicable to legacy systems:1. Redevelopment, from scratch, using a new hardware platform and modern
architectures tools and database; 2. Migration that moves the system to a new platform, while retaining the
original system data and functionalities3. Wrapping, which surrounds existing data, programs, applications and
interfaces with new interfaces
Grid-related solutions Migration towards Grids in order to increase their ability to solve bigger
problems by (“gridification” for Computational Grids) Wrapping as Grid services is a cost-effective and quality solution for evolving
them in order to meet new requirements (“wrapping” for Service Grids)
Alice Workshop, Aug 21, 2008
13
Gridification for Computational Grids1. Identify parallelism degree
2. Manage the issue related to the resources heterogeneity
3. Deal with intra-cluster and inter-cluster communications
4. Create interfaces for easy-to-use
Alice Workshop, Aug 21, 2008
14
Example for 1. Parallelism degree Embarrassing gridification in analogy with
embarassing parallelism – no communication Digital libraries [in our case for SINRED project, developments on
RoGrid infrastructure based on gLite] Multimedia conversions for large collections of
documents (e.g. PDF -> audio) – considerable requirements in what concerns response time & storage
Applying OCR techniques on large collections of documents in order to create indexes (keywords, references etc), extract images etc
Used available free convertors
Alice Workshop, Aug 21, 2008
15
Example for 2. Manage heterogeneity Stage 1 – 2003 parallel version of STHAMAS 3D
commercial CFD simulator for crystal growing
three-dimensional finite-volume code
based on a block-structured finite-volume Navier-Stokes solver Stage 2 – 2004 (CompGrid project)
adapt the parallel code to use heterogeneous computing resources improved efficiency through avoiding load balancing problems
Alice Workshop, Aug 21, 2008
16
Example for 3. Intra/Inter-cluster comm.DEMO/G (in GridMOSI project)Distributed Evolutionary Multiobjective
Optimization on Grid= implementation of evolutionary algorithms
for multiobjective optimization (EMOA) based on a layered structure First layer: sequential EMOA executed on
one processor Second layer: parallel EMOA based on a
colony of inter-communicating populations
Third layer: multiple colonies which evolves separately
Used for multiobjective multidisciplinary optimization (case study: airfoil design) Evaluation of objective functions is based
on numerical simulations; coupling between EMOA and other codes
e.g. CFD
Alice Workshop, Aug 21, 2008
17
Example for 4. Create interfaces easy-to-use
GOC Grid Operations Center for gLite 3 Modular Approach
Authentication Application
interface Demo/G,
STHAMAS, OCR etc
Grid Engine
Application Interface / DEMO/G
Alice Workshop, Aug 21, 2008
18
Wrapping codes for Service Grids Non-interactive legacy systems1. Example 1: Service for CFD
simulation
Interactive legacy systems*1. Example 2: Service for image
processing2. Example 3: Services for
symbolic computations3. Example 4: Service for an
expert tool
*systems where any transaction is completed by means of a dialogue between the user and the system
Alice Workshop, Aug 21, 2008
19
Example 1: service for CFD simulation Legacy code: GERRIS
open source free software library for the solution of the PDEs describing fluid flow
Inputs: a file describing the simulation
parameters; the number of processes used by
Gerris; the name(s) of the resulting file(s)
to be transferred to the client side.
Similar experiences with other codes E.g. for optimization DEME - library
for sequential, parallel & distributed metaheuristic algorithms
INPUT – SIMULATION PARAMETERS:
# The classical lid-driven cavity test case.
1 0 GfsSimulation GfsBox GfsGEdge {} {
Time { end = 300 } Refine 6
SourceDiffusion {} U 1e-3
SourceDiffusion {} V 1e-3
EventStop { istep = 10 } U 1e-4 DU
OutputScalarNorm { istep = 10 } du { v = DU }
OutputPPM { start = end } { v = Velocity }
{ convert -colors 256 ppm:- velocity.eps } }
GfsBox {
top = Boundary { BcDirichlet U 1 BcDirichlet V 0 }
bottom = Boundary { BcDirichlet U 0 BcDirichlet V 0 }
right = Boundary { BcDirichlet U 0 BcDirichlet V 0 }
left = Boundary { BcDirichlet U 0 BcDirichlet V 0 } }
OUTPUT:
Alice Workshop, Aug 21, 2008
20
Example 1: service for CFD simulationThe service exposes: Simple client interface:
setType(type) Set the execution type: use gerris2D, sequential (type=1) or in parallel (type=2), or gerris3D, sequential (type=3) or in parallel (type=4); by default type=1
exec(simFileName) Launch the simulation described in the simFile
getExample(type) Get an example of simulation file
getHelp() Get a user manual for problem description
getStatus() Get the status (Active, Done or Error) of the simulation; by default the status is Done
Alice Workshop, Aug 21, 2008
21
Example 2: Service for image processing A Grid service for remote image processing based on GIMP
GIMP accept command lines! despite its classical interactive mode
A distributed DB of images is available to a virtual organization (VO) as a shared facility.
A user apply a sequence of image processing standard procedures to extract a particular information from specific images
Usage in MedioGrid platform
Follows: ESA project GiSHEO! Combining ESA’s G-POD with MedioGrid platform services for educational purposes
The service exposes:
createServInstance <FactoryRef> <EPRFile>
startGIMPServer <EPRFile>
<DatabaseRef>
clientGIMPAction <EPRFile>
<GIMPcommand>
clientGetFile <EPRFile>
<FileName> <Destination>
Simple client interface
Alice Workshop, Aug 21, 2008
22
Example 3: Services for Symbolic Computations Computer Algebra Systems
(CAS) are wrapped Create an instance for each
client Gradually interpret
commands Examples CAS: GAP, Kant,
MuPad, Maple, Yacas, Singular, Pari, CoCoA, Macaulay etc
Allows interoperability between CASs
Only one part of SymGrid- (EU SCIEnce project) – another part: access Grid/Web/gLite services within CAS
Alice Workshop, Aug 21, 2008
23
Example 4: an expert tool for ODEs EpODE (1997) @ UVT EpODE (2008) @PEGAF prj.
Alice Workshop, Aug 21, 2008
24
Conclusions
The process of Grid-enabling legacy codes depends at least the followings: Grid type (e.g. Computational/Service etc) Code type (e.g. Interactive/Non-interactive, GUI oriented/or
not etc) Despite there are several existing tools for Grid-
enabling codes, those are useful for particular conditions, not for the general case
An automatized process for Grid-enabling any legacy code is far for being possible at this moment