43
Integration of Multidiscipline Applications Integration of Multidiscipline Applications in Grid-computing Environments in Grid-computing Environments NGUYEN G.T., J. BLACHON, C. PLUMEJEAUD PARA’02, Espoo, June 16th, 2002 « OPALE » PROJECT Conference on Applied Parallel Computing

Integration of Multidiscipline Applications in Grid-computing Environments

  • Upload
    chinara

  • View
    53

  • Download
    0

Embed Size (px)

DESCRIPTION

Conference on Applied Parallel Computing. Integration of Multidiscipline Applications in Grid-computing Environments. NGUYEN G.T., J. BLACHON, C. PLUMEJEAUD. « OPALE » PROJECT. PARA’02 , Espoo, June 16th, 2002. OPALE. • New INRIA project since January 1st, 2002. • Follow up SINUS project. - PowerPoint PPT Presentation

Citation preview

Page 1: Integration of Multidiscipline Applications in Grid-computing Environments

Integration of Multidiscipline Applications Integration of Multidiscipline Applications in Grid-computing Environmentsin Grid-computing Environments

NGUYEN G.T., J. BLACHON, C. PLUMEJEAUD

PARA’02, Espoo, June 16th, 2002

« OPALE » PROJECT

Conference on Applied Parallel Computing

Page 2: Integration of Multidiscipline Applications in Grid-computing Environments

• Topics

• Located Sophia-Antipolis & Grenoble

• Follow up SINUS project

• New INRIA project since January 1st, 2002

OPALEOPALE

NUMERIC OPTIMISATION (genetic, hybrid, …)

MODEL REDUCTION (hierarchic, multi-grids, …)

INTEGRATION PLATFORMS

Coupling, distribution, parallelism, grids, clusters, ...

APPLICATIONS : aerospace, electromagnetics, …

Page 3: Integration of Multidiscipline Applications in Grid-computing Environments

• APPLICATION DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 4: Integration of Multidiscipline Applications in Grid-computing Environments

• DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 5: Integration of Multidiscipline Applications in Grid-computing Environments

WHERE WE ARE TODAYWHERE WE ARE TODAY

• 1980 : one year CPU time

• 1992 : one month «  »

• 1997 : four days «  »

• 2002 : one hour «  »

• ASCI White (LLNL) : 8192 IBM SP procs

• ASCI Red (Sandia) : 9632 Intel procs

• ASCI Blue Mountain (LANL) : 6144 SGI procs

Bits and pieces….

Page 6: Integration of Multidiscipline Applications in Grid-computing Environments

TEST CASETEST CASEWING PROFILE OPTIMISATION

Page 7: Integration of Multidiscipline Applications in Grid-computing Environments

TEST CASETEST CASE

• SHOCK-WAVE INDUCED DRAG REDUCTION

• WING PROFILE OPTIMISATION (RAE2822)

• Euler eqns (0,84 Mach, i = 2°) + BCGA (100 gen.)

• 2D MESH : 14747 nodes, 29054 triangles

• 4.5 hours CPU time (SUN Micro SPARC 5, Solaris 2.5)

• 2.5 minutes CPU time (PC cluster 40 bi-procs PIII, Linux)

Page 8: Integration of Multidiscipline Applications in Grid-computing Environments

““CAST” INTEGRATION PLATFORMCAST” INTEGRATION PLATFORM

GOALS

• TEST CASES IMPLEMENTATION

• “DECISION” CORBA INTEGRATION PLATFORM

• DESIGN FUTURE HPCN OPTIMISATION PLATFORMS

COLLABORATIVE MULTI-DISCIPLINE OPTIMISATION

GENETIC & PARALLEL OPTIMISATION ALGORITHMS

CODE COUPLING FOR CFD, CSM SOLVERS & OPTIMISERS

COLLABORATIVE APPLICATIONS SPECIFICATION TOOL

Page 9: Integration of Multidiscipline Applications in Grid-computing Environments

The front stage….

Page 10: Integration of Multidiscipline Applications in Grid-computing Environments

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMUser interface

Page 11: Integration of Multidiscipline Applications in Grid-computing Environments

• DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 12: Integration of Multidiscipline Applications in Grid-computing Environments

• DISTRIBUTED : LAN, WAN, HSN...

• CODE-COUPLING FOR HETEROGENEOUS SOFTWARE

• COLLABORATIVE APPLICATIONS

• COMMON DEFINITION, CONFIGURATION, DEPLOYMENT, EXECUTION & MONITORING ENVIRONMENTS

• TARGET HARDWARE : NOW, COW, PC-clusters, grids, ...

• TARGET APPLICATIONS : multidiscipline engineering, ...

INTEGRATIONINTEGRATION PLATFORMSPLATFORMS

Distributed tasks interacting dynamically in controlled and formally provable way

What they are...

Page 13: Integration of Multidiscipline Applications in Grid-computing Environments

DISTRIBUTED SIMULATION

• MULTI-DISCIPLINE PROBLEM SOLVING ENVIRONMENTS

• HIGH-PERFORMANCE & TRANSPARENT DISTRIBUTION

• USING CURRENT COMMUNICATION STANDARDS

• USING CURRENT PROGRAMMING STANDARDS

• WEB LEVEL USER INTERFACES

• OPTIMIZED LOAD BALANCING & COMMUNICATION FLOW

What is required...

Page 14: Integration of Multidiscipline Applications in Grid-computing Environments

DESIGN ALTERNATIVESDESIGN ALTERNATIVES

• HARWARE & SOFTWARE ENVIRONMENTS

• EXISTING PLATFORMS

• LEGACY APPLICATION SOFTWARE

• PROBLEM REQUIREMENTS

Optimize specific pbs & solutions : ReMAP

System evolution & development : PARIS

Globus, Condor, NetSOLVE, Legion, ….

How to integrate them into new PSE ?

Page 15: Integration of Multidiscipline Applications in Grid-computing Environments

INRIA PROJECTS ALTERNATIVESINRIA PROJECTS ALTERNATIVES

• HARWARE & SOFTWARE ENVIRONMENTS

• OTHER EXISTING PLATFORMS

• LEGACY & NEW APPLICATION SOFTWARE

• PROBLEM REQUIREMENTS

Optimize specific pbs : ReMAP, ATHAPASCAN

System development : PARIS, OASIS

Globus, Condor, NetSOLVE, Legion, ….

How to integrate them into new PSE ?

Page 16: Integration of Multidiscipline Applications in Grid-computing Environments

• VISUAL PROGRAMMING

• COMPONENTS PROGRAMMING

• OBJECT-ORIENTED TECHNOLOGY

ADVANCES IN SOFTWAREADVANCES IN SOFTWARE

PROGRAMMING : C++, JAVA, C#, ...

APPLICATION MODELING : UML

REUSABILITY

MODULARITYINTEROPERABILITY

Page 17: Integration of Multidiscipline Applications in Grid-computing Environments

DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURE

• TRANSPARENT DISTRIBUTED OBJECT COMPUTING

• CORBA COMPLIANT

• SIMPLE SOFTWARE MODEL

• COMPONENTS PLUG-IN (e.g., optimizers, solvers)

- COMPONENTS

- CONNECTORS

Page 18: Integration of Multidiscipline Applications in Grid-computing Environments

DISTRIBUTED OBJECTS ARCHITECTUREDISTRIBUTED OBJECTS ARCHITECTURE

SOFTWARE COMPONENTS

• COMPONENTS ARE DISTRIBUTED OBJECTS

• WRAPPERS AUTOMATICALLY (?) GENERATED

• COMPONENTS ENCAPSULATE CODES

• DISTRIBUTED PLUG & PLAY

Page 19: Integration of Multidiscipline Applications in Grid-computing Environments

CAST PROTOTYPECAST PROTOTYPE

CAST OPTIMIZERS

CORBA

SOLVERS

Server Wrapper Wrapper

Modules Modules

Page 20: Integration of Multidiscipline Applications in Grid-computing Environments

SOFTWARE COMPONENTSSOFTWARE COMPONENTS

• BUSINESS COMPONENTS

LEGACY SOFTWARE

• OBJECT-ORIENTED COMPONENTS

• DISTRIBUTED OBJECTS COMPONENTS

• METACOMPUTING COMPONENTS ?

C++, PACKAGES, ...

Java RMI, EJB, CCM, ...

Page 21: Integration of Multidiscipline Applications in Grid-computing Environments

DISTRIBUTED OBJECT ARCHITECTUREDISTRIBUTED OBJECT ARCHITECTURESOFTWARE CONNECTORS

• CONNECTORS ARE SYNCHRONISATION CHANNELS

• SEVERAL PROTOCOLS

• CONNECTORS = DATA COMMUNICATION CHANNELS

- SYNCHRONOUS METHOD INVOCATION

- ASYNCHRONOUS EVENT BROADCAST

• COMPONENTS COMMUNICATE THROUGH SOFTWARE CONNECTORS

Page 22: Integration of Multidiscipline Applications in Grid-computing Environments

• APPLICATION DEVELOPER POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 23: Integration of Multidiscipline Applications in Grid-computing Environments

• SUPPORT FOR NEW APPROACHES

• // SOFTWARE LIBRARIES : MPI, PVM, SciLab //, ...

• PARALLEL and/or DISTRIBUTED HARDWARE

• SUPPORT SEVERAL DEGREES PARALLELISM

PARALLEL APPLICATIONSPARALLEL APPLICATIONS

DOMAIN DECOMPOSITION

GENETIC ALGORITHMS

GAME THEORY

HIERARCHIC MULTI-GRIDS

The good news….

Page 24: Integration of Multidiscipline Applications in Grid-computing Environments

• Lays the ground for GRID and METACOMPUTING

• PC & Multiprocs CLUSTERS : thousands GHz procs...

• HIGH-SPEED NETWORKS : ATM, FIBER OPTICS...

ADVANCES IN HARDWAREADVANCES IN HARDWARE

GLOBUS, LEGION

CONDOR, NETSOLVE

Gigabits/sec networks available (2.5, 10, …)

The best news….

Page 25: Integration of Multidiscipline Applications in Grid-computing Environments

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Linux Pentium III procs.)

Page 26: Integration of Multidiscipline Applications in Grid-computing Environments

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes (216 Pentium III procs.)

Page 27: Integration of Multidiscipline Applications in Grid-computing Environments

-300

-200

-100

0

100

200

300

-200-1000100200300400500600

Volet sortiBec sorti

Corps principal

AIRFOIL OPTIMISATIONAIRFOIL OPTIMISATION

Page 28: Integration of Multidiscipline Applications in Grid-computing Environments

AIRFOIL OPTIMISATIONAIRFOIL OPTIMISATION

Page 29: Integration of Multidiscipline Applications in Grid-computing Environments

The results...

Page 30: Integration of Multidiscipline Applications in Grid-computing Environments

CLUSTER COMPUTINGCLUSTER COMPUTINGPC-cluster at INRIA Rhône-Alpes

Multi-airfoil optimization : game theory + multi-grids hierarchic algo.

Page 31: Integration of Multidiscipline Applications in Grid-computing Environments

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM

NICE

RENNES

GRENOBLE

PC cluster

PC clustern CFD solvers

CAST

GA optimiser

PC clustersoftware

VTHD Gbits/s network

GRID computing...

July 2001...

Page 32: Integration of Multidiscipline Applications in Grid-computing Environments

Check for syntaxe of request

NSD

CORBA

Event channell,i1, i2, i3, ….

IRD

Algogen.idl

AlgoGeni1,i2, i3, …, in

CAST

CfdSolvercfd1

CfdSolver cfd2

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORMBehind the stage, again...

Page 33: Integration of Multidiscipline Applications in Grid-computing Environments

Event channel,i1, i2, i3, …, in

CfdSolverCfd1

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

i1

CfdSolverCfd2

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

i2

CfdSolverCfd3

ProcessorP0

ProcessorP1

ProcessorP3

ProcessorP2

i3

Genetic Algorithm

i1, i2 ,i3, …, in

Parallelized with MPI on p processors

Genetic algorithm based on selection, mutation, crossover

CORBA server implemented in C++

CORBA client implemented in C++

THREE LEVELS of PARALLELISMTHREE LEVELS of PARALLELISM

Page 34: Integration of Multidiscipline Applications in Grid-computing Environments

Ag2DWithCorba

0100200300400500600700800

1 2 3 4 5 6 7

Nb CfdSolvers

Tim

e (

s) SOPHIA

RENNES

GRENOBLE

CfdSolvers at Sophia, CAST at Grenoble

0 200 400 600 800 1000

Ag atGrenoble

Ag atSophia

Ag atRennes

Time (s)

6 Cfd

5 Cfd

4 Cfd

3 Cfd

2 Cfd

1 Cfd

* Curves quasi-parallels

=> same speed up, whatever the place.

* Join an horizontal asymptote:

time = 200 s

CAST DISTRIBUTED INTEGRATION PLATFORMCAST DISTRIBUTED INTEGRATION PLATFORM

The game : load balancing,...

Page 35: Integration of Multidiscipline Applications in Grid-computing Environments

• DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 36: Integration of Multidiscipline Applications in Grid-computing Environments

BCGA FUN

END

InitB HYBRID

PROCESS FORMULAEPROCESS FORMULAEMILNER ’S SCCS PROCESS ALGEBRA

InitBCGA:InitHybrid:BGGA:(TRUE:(END)+FALSE:(FUN:(TRUE:(HYBRID:

(TRUE: (=>InitHybrid)+FALSE:(=>FUN)))+FALSE:(=>BGGA))))

InitH

Page 37: Integration of Multidiscipline Applications in Grid-computing Environments

OPERATORSOPERATORS

• SYNCHRONIZATION

• PARALLEL EXECUTION

• SERIAL EXECUTION

• ITERATIONS

• COMPLEX EXPRESSIONS : process formulae

• CHOICE

IC simulation : several coupled models

Page 38: Integration of Multidiscipline Applications in Grid-computing Environments

STRONG POINTSSTRONG POINTS

• STRONG THEORETICAL FOUNDATIONS

• SPECIFICATION & VERIFICATION OF COMPLEX APPS

Process algebra for asynchronous systems

• FORMAL SPECIFICATION SYSTEM

• EASY TO USE

Intuitive interface : simple component modelNo theoretical background knowledge requiredTransparent distribution using CORBA

Milner ’s SCCS algebra

Page 39: Integration of Multidiscipline Applications in Grid-computing Environments

• DEVELOPER ’S POINT OF VIEW

• SOFTWARE ENGINEERING POINT OF VIEW

• USER POINT OF VIEW

• CONCLUSIONS

OVERVIEWOVERVIEW

• THEORETICAL POINT OF VIEW

Page 40: Integration of Multidiscipline Applications in Grid-computing Environments

• GRID COMPUTING

• DISTRIBUTED INTEGRATION PLATFORMS

• MULTIDISCIPLINE SIMULATION

TODAY ’S FUTURETODAY ’S FUTURE

e.g., DIGITAL DYNAMIC AIRCRAFT

CAST, JACO3, CCAT, ProACTIVE ...

GLOBUS, LEGION, CONDOR, ...

Page 41: Integration of Multidiscipline Applications in Grid-computing Environments

• DYNAMIC LOAD BALANCING & RESSOURCE ALLOC

• « COTS » PROGRAMMING

• METACOMPUTING

TOMORROW’S FUTURETOMORROW’S FUTURE

COMPONENTS OFF THE SHELF

POWER SUPPLY PARADIGM APPLIED TOCOMPUTING RESOURCES WORLDWIDE

Behind the stage, again...

OBSERVE, START, SUSPEND, RESUME, STOP, MIGRATE

REMOTE PROCESSES DYNAMICALLY

Page 42: Integration of Multidiscipline Applications in Grid-computing Environments

CONCLUSIONCONCLUSION

• INTEGRATION PLATFORMS PROVIDE

• GRID COMPUTING

• FULLY CORBA COMPLIANT

• ALSO ALLOWS CORBA & non-CORBA COMPONENTS

• SMOOTH TRANSITION FROM EXISTING CODE-COUPLING ENVIRONMENTS

DEFINE, CONFIGURE, DEPLOY, EXECUTE & MONITORCOLLABORATIVE APPLICATIONS

• ALLOWS SEQUENTIAL & PARALLEL COMPONENTS

Page 43: Integration of Multidiscipline Applications in Grid-computing Environments

DOCUMENTATIONDOCUMENTATION

• http://www.inrialpes.fr/opale

[email protected]

• http://cast.sourceforge.net/manuel