Upload
zachary-fowler
View
223
Download
2
Tags:
Embed Size (px)
Citation preview
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
www.enea.it
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
26 April 2005 Johon Cabot University
GRID Computing: esperienze realizzative ed elementi architetturali.
Outline• ENEA-GRID Infrastructure and Resources• Case studies: numerical simulations and support to
experimental facilities• Connection with other GRID projects• GRID paradigm & ENEA-GRID
GRID COMPUTINGReal Case GRID At ENEA
Ing. S. Migliori ENEA INFO [Servizio Centralizzato Informatica e Reti], Roma
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
The Grid: The Web on Steroidshttp://
http://
Web: Uniform access to HTML documents
Grid: Flexible, high-perf access to all significant resources
Ian Foster
On-demand creation of powerful virtual computing systems
Sensor nets
Data archives
Computers
Softwarecatalogs
Colleagues
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
TELEMICROSCOPY & GRID - BASED COMPUTINGTELEMICROSCOPY & GRID - BASED COMPUTINGREMOTE ACCESS FOR DATA ACQUISITION AND ANALYSISREMOTE ACCESS FOR DATA ACQUISITION AND ANALYSIS
NETWORKNETWORK
DATA ACQUISITIONDATA ACQUISITION DATA ANALYSISDATA ANALYSISADVANCEDADVANCEDCOMPUTERCOMPUTERGRAPHICSGRAPHICS
QuickTime™ and a YUV420 codec decompressor are needed to see this picture.
Cell Centered Data Base Cell Centered Data Base ““CCDB”CCDB”
ByByMark EllismanMark Ellisman
IMAGINGIMAGINGINSTRUMENTSINSTRUMENTS COMPUTATIONALCOMPUTATIONAL
RESOURCESRESOURCES
MULTI-MULTI-SCALESCALEDATA-DATA-BASESBASES
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Architecture --- > Function
Application
Fabric
Connectivity
Resource
Collective
User
Sensor netsData archives
Computers
Softwarecatalogs
Colleagues
?
Le Funzioni / Esigenze comandano sulla tecnologia
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Network Exponentials
• Network vs. computer performance– Computer speed doubles every 18
months– Network speed doubles every 9
months– Difference = order of magnitude
per 5 years
• 1986 to 2000– Computers: x 500– Networks: x 340,000
• 2001 to 2010– Computers: x 60– Networks: x 4000
Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan-2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and Perkins.
Ian Foster
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Performance Extrapolation
N=1
N=500
Sum
1 GFlop/s
1 TFlop/s
1 PFlop/s
100 MFlop/s
100 GFlop/s
100 TFlop/s
10 GFlop/s
10 TFlop/s
10 PFlop/s
PFlop/scomputer
TFlop/sTo enter the list
Jac. Dongarra SC2002
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Selected Major Grid ProjectsName URL & Sponsors Focus
Access Grid www.mcs.anl.gov/FL/accessgrid; DOE, NSF
Create & deploy group collaboration systems using commodity technologies
BlueGrid IBM Grid testbed linking IBM laboratories
DISCOM www.cs.sandia.gov/discomDOE Defense Programs
Create operational Grid providing access to resources at three U.S. DOE weapons laboratories
DOE Science Grid
sciencegrid.org
DOE Office of Science
Create operational Grid providing access to resources & applications at U.S. DOE science laboratories & partner universities
Earth System Grid (ESG)
earthsystemgrid.orgDOE Office of Science
Delivery and analysis of large climate model datasets for the climate research community
European Union (EU) DataGrid
eu-datagrid.org
European Union
Create & apply an operational grid for applications in high energy physics, environmental science, bioinformatics
g
g
g
g
g
g
New
New
Ian Foster
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Selected Major Grid ProjectsName URL/
SponsorFocus
TeraGrid teragrid.org
NSF
U.S. science infrastructure linking four major resource sites at 40 Gb/s
UK Grid Support Center
grid-support.ac.uk
U.K. eScience
Support center for Grid projects within the U.K.
Unicore BMBFT Technologies for remote access to supercomputers
g
g
New
New
Also many technology R&D projects: e.g., Condor, NetSolve, Ninf, NWS
See also www.gridforum.org
Ian Foster
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEAItalian Agency for New Tecnologies, Energy and Environment. 12 research centers in Italy A Central Computer and Network Service (INFO)6 Computer CentresCasaccia, Frascati, Bologna,Trisaia, Portici, Brindisi
Multiplatform resources for serial & parallel computation and graphical post-processing.
Others computer resources in ENEA:departments & individuals
Centrodi Brindisi
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA computational resources are distributed over WAN, connected by GARR, the Italian Academic & Research Network
ENEA Network connection
ENEA-GARR 9 PoP, 4-32 Mbps
BrindisiBolognaCasacciaFrascatiPorticiTrisaiaPalermoPisaRoma Sede
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA-GRID has been developed since 1998 and now it is an integrated infrastructure:
• offering a production quality, service oriented system
• covering most of the computational resources belonging to ENEA INFO Scientific Computation Service, connected in a Wide Area Network.
• providing a unified user environment and an homogenous access method for all ENEA researchers irrespective of their location
• implementing tools to facilitate the integration of department and individual resources and support to experimental facilities
ENEA GRID
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Services for research activity in ENEA
• Multiplatform Parallel systems:
• AIX, Linux (Alpha,x86,AMD64), IRIX, [Unicos (Cray)]
• Graphical simulation & code result post-processing:
• SGI + 3D immersive facilities
• Software resources:
• Commercial Codes: Fluent, Gambit, Abacus, Catia, Ansys
• Research codes: mcpn/x, eranos, fluka,…
• Elaboration environments: IDL, Matlab, Mathematica, SAS
• Windows applications
ENEA GRID Services
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID Computational resources
Portici, Brindisi & Trisaia resources increased in the framework of TELEGRID Project
Frascati(6), Portici(4),Trisaia(4), Brindisi(4)
5018Windows
Frascati(1), Trisaia(13)8021Mac OS X
Trisaia(4), Casaccia(2), Bologna(2)108Solaris
Frascati(8), Casaccia(4), Portici(1),Trisaia(8), Brindisi(1), Bologna(5)
4026IRIX
Casaccia10080Linux Alpha
Frascati(28), Casaccia(54), Portici(4),Trisaia(4), Brindisi(4)
20096Linux x86 32/64
Frascati(128), Bologna(8), Portici(18)1000154AIX
Gflops#cpu OS
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
CitrixMetafram
e
ICA
WEB(ICA)
power3.frascati.enea.it
boquad.bologna.enea.it
dafne.casaccia.enea.it
Kleos.portici.enea.it
GEANT
infocal.trisia.enea.it
……...brindisi.enea.it
Portici
Brindisi
CitrixNfuse
new
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
AFS Geographical cross platform & File System
AIX SGI SUN HP Win
Linux
Linux Alpha
MAC
LSF
User programs and commercial codes
LSF multi-cluster as integrator
XTERM
Graphical User Interface
Application Portal (Metaframe/Nfuse)
ICAWEB
Qu
ality
of
serv
ice
Mon
itori
ng
, A
uto
mati
c r
ecovery
Accau
nti
ng
Load Leveler RMS
ENEA GRID
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Software Conceptual Structure
AFS
Load Leveler LSF
Graphic User Interface LSF multi-cluster as integrator
Telnet
User programs & commercial code
ICAWEB
Cluster 1° ( Bologna)
AFS
Load Leveler LSF
Graphic User Interface LSF multi-cluster as integrator
Telnet
User programs & commercial code
ICAWEB
Cluster 3° (Frascati)
AFS
Load Leveler LSF
Graphic User Interface LSF multi-cluster as integrator
Telnet
User programs & commercial code
ICAWEB
Cluster 2° (Nord Rome)
AFS
Load Leveler LSF
Graphic User Interface LSF multi-cluster as integrator
Telnet
User programs & commercial code
ICAWEB
Cluster 4° (Trisaia )
LSF
Graphic User Interface
Telnet
User programs & commercial code
ICAWEB
Cluster 5° ( Portici)
File System
LSF
Graphic User Interface
ICAWEB
Cluster 6° ( Brindisi)
NEW
NEW
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Gestione generale di una specifica richiesta al sistema
InterfacciaGrafica
(RICHIESTA)LSF
Cluster Configuratio
n(STATICO)
Stato delle Risorse
(TEMPO REALE)
Politica delle code
(STATICO)•Accoda la richiesta
•Assegna la risorsa
•Lancia il comando sul calcolatore selezionato
DATI(SW/Prog.)
File Server
Client AFS
Cache locale
•Risolve la piattaforma•Verifica i diritti di accesso•Mantiene la coerenza dei dati
AFS
Data Base Server
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID Graphics and 3D resources(1)
C.R. Frascati C.R. Bologna
C.R. Casaccia
SGI systems are coupled with 3D immersive facilities together with 3D modelling and rendering software as CAVELIB, DV-Mockup, AVS
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID Graphics and 3D resources(2)
C.R. BrindisiC.R. Trisaia
C.R. Portici
SGI systems are coupled with 3D immersive facilities together with 3D modelling and rendering software as CAVELIB, DV-Mockup, AVS
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID ARCHITECTURE
A choice of mature components for reliability and easiness of support and maintenance:
Distributed File System: AFS
Job and resources manager: LSF Multicluster
Unified GUI access: Java and Citrix Technologies
Quality monitoring system: Patrol
Licence Servers
Integration with department and individual resources
Distributed File System: AFS for software and data
Licence pool sharing
Possible integration with other institutions
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
GRID usage, examples and some case studies
650 registered users
2 TB stored software/data out of 4 TB available in enea.it AFS cell
Examples and cases studies● Running FLUENT on ENEA GRID
● Support to department experimental research:
• Biotechnology
• Electron microscopy
• Nuclear fusion research
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
CitrixMetafram
e
ICA
WEB(ICA)
power3.frascati.enea.it
boquad.bologna.enea.it
dafne.casaccia.enea.it
Kleos.portici.enea.it
GEANT
infocal.trisia.enea.it
……...brindisi.enea.it
Portici
Brindisi
CitrixNfuse
new
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
CITRIX / NFUSE
ENEA GRID makes use of Citrix Metaframe to publish an application providing a unified GUI interface all the available resources
Application components:●Java (GUI)●shell scripts●EnginFrame
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
FLUENT (CFD) code batch submission
LSF Options-o filename.%J - output file-w “done(idjob)” - start after idjob-u e-mail userid - output by E-mail-b begintime - job start time….
A FLUENT case can be run from a dedicated GUI for submission
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Monitoring job status by xlsbatch (LSF)
xlsbatch informationJobidUsersJob statusQueuesSubmission hostRun hostSubmission timeSubmission command…..
Job status:
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Monitoring host status by xlsmon (LSF)
Status of sp3-1 host using xlsmon utility
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
3D CFD Simulation on Aircraft Axial Turbin Stage with Wake & 3D CFD Simulation on Aircraft Axial Turbin Stage with Wake & Vortex Shedding Analysis on Turbine Blades [FLUENT]Vortex Shedding Analysis on Turbine Blades [FLUENT]
Pressure traces on stator trailing edge
165000
170000
175000
180000
185000
190000
195000
pres
sure
(Pa)
isolated stator stage stator
Temperature & wakes interactions
Large Eddy Simulation (LES) provides high
resonable results on wake analysis and turbine stage performance.
Secondary flow prevision and 3D Path Lines analysis
Max, min and time average Cp for the vane
-2.5
-2
-1.5
-1
-0.5
0
0.5
-0.01 0 0.01 0.02 0.03 0.04
x (m)
Cp
Temperature contours on Stator blades & Cp experimental data comparison
Mesh: 1 million cellsCPU Time: 6000 hours
Elapsed Time: 525 hours on
12 IBM SP3 platforms
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Pressure losses
0
500000
1000000
1500000
2000000
2500000
3000000
0 0.2 0.4 0.6 0.8 1 1.2
velocity (m/s)
Pres
sure
jum
p (P
a)
Serie1
Caratteristica parabolica dello smorzatore con valvola aperta fino a 3 mm
Punto di progetto (apertura valvola)
CFD Analysis of a fluid dumper [FLUENTCFD Analysis of a fluid dumper [FLUENT]]
Vortex stagnationVortex stagnation
Secondary fluxSecondary flux
Primary FluxPrimary Flux
Mesh: 600,000 cells CPU Time: 960 hours
Elapsed Time: 130 hours on 8 Linux platforms
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
p-patm da 0 a 1.55 s.
0
10
20
30
40
50
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5time (s)
p-p
atm
Real P-Patm
linea-1
linea-2
linea-3
Blast Wave Analysis for VEGA Programm
DMA-Uniroma1 – AviogroupRome 2003
0.00E+00
5.00E+03
1.00E+04
1.50E+04
2.00E+04
2.50E+04
3.00E+04
3.50E+04
4.00E+04
0.00E+00 2.00E+01 4.00E+01 6.00E+01
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
La simulazione con Cover:il campo di Temperatura
Grid: 1,5 million cellsCPU Time: 5000hours
Elapsed Time: 672 hours on
6 SGI platforms
(onyx2ced )
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Uno sguardo verso il futuro: la parametrizzazione via WebUno sguardo verso il futuro: la parametrizzazione via Web
4.0
100.0V
ProfiloNACA xxx
STEPSDefinire la geometria e le condizioni al contorno.Generare la griglia di calcolo necessariaEseguire il calcoloRisultati del Post-processing (su Web o home/directory)
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
FLUENT performance on different platforms in ENEA GRID
Cpu time normalized to the case: Sp4-1, 2 CPUs
GHz#CPU
IRIX
0.48Onyx.
1.88bw
34C..03
Linux
1.38Pace
1.316Ostro
1.132Sp4-1
.37516Sp3-1
AIX
Scaling with #CPU & platform type
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
DB_1
CPUSENEA GRID
WEBICASSH
DNA Sequence system
( ABI Prism 3700)Trisaia
DB_3
DB_2
Electronic Microscope (Brindisi)
300 Kev (sept. 2004)
ENEA GRID and experimental facilities
Controlled Nuclear Fusion:
FTUFrascati Tokamak
UpgradeVideo Acquisition
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID: FTU video acquisition data
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Conceptual structure for TEMProgetto UTS Materiali
Electronic Microscope (Brindisi)
300 KeV (sept. 2004)
RemoteLocal
Operation
TEM Room
Local Operation
Remote ControlledWeb-cam
Remote User
Local DataAFS Data Integration
> 4 TByte
Mtaframe
AFS
Special Driver for TEM
Local GRID Cluster
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA-Geographic SUBMISSION PROCEDURE
Input FilesLIV1
List of pathInput File
LIV1
IDL “GOME” Program
Output FileLIV2
.jpg Image
Input FileLIV1
List of pathInput File
LIV1
IDL Program“GOME”
Output FileLIV2
.jpg Image Input File
LIV1
List of pathInput FileLIV1
Case-1
Case_2
Case n
WEB
User afs/home
ENEA GRID/DATAGRID Test case: GOME data analysis
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID/DATAGRID : GOME run cases in ENEA
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID/DATAGRID: GOME results on Web
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID
ANSALDO(GE)
ENEA(ISPRA)
ORNL(USA)
CRIOTEC(TO)
• DATA sharing• Applications in ENEA GRID• Sharing Working Sessions (Citrix)
ENEA IGNITOR/Pellet Collaboration Activity
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Collaborazioni
DMA ( Prof. ENRICO Scubba) Fluidodinamica e sviluppo di metodologie
dell’utilizzo di grid … Istituto Matemetica Castelnuovo (In fase di perfezionamento)
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA ENEA
Main activities & Main activities & resources for EGEEfor EGEE
•Integration of EGEE-GRID to ENEA-GRID;Integration of EGEE-GRID to ENEA-GRID;
•100 CPUs at 20% of time100 CPUs at 20% of time
DataGrid at
ESA\ESRIN
Globus
Based g
ateway
GigaBit-
Link
EGEE GRID
Globus Based gateway
Glo
bu
s B
ased
gate
way
ENEA-GRID
S-PACIGRID
Globus Based gateway
GRID.IT WP13 Chem-Grid
(MAT)
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Windows services Unix/Linux Services
CITRIX: ENEA GRID Web Access
http://www.afs.enea.it
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID & GRID PARADIGM“GRID concept: coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”
Foster & Kesselman, The Anatomy of the Grid, 2001
ENEA GRID infrastructure has been designed to provide a multisite resource sharing inside ENEA, both to optimize the utilization of Central Computer Service (ENEA INFO) resources and to integrate the other resources available inside the organization.
VO concept can not be fully developed inside an unique organization but similarities can be found with Project Groups sharing specific resources and ENEA GRID components can provide support to this (AFS groups, LSF resources).
There are cases where ENEA GRID infrastructure has been used in support to the collaboration with other institutions (+ other GRID projects)
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID architecture answers to many of the challenges posed by the “GRID Problem” : “unique authentication, authorization, resource access and resource discovery”, Foster & Kesselman 2001
ENEA GRID & GRID PARADIGM
The choice of mature components (LSF & Citrix - proprietary, AFS - opensource project since 2000) reduce the generality of ENEA GRID as a GRID model but improves greatly the reliability and the easiness of update & management: production quality infrastructure.
Sensor nets
Colleagues Data archives
ComputersSoftwarecatalogs
by Ian Foster
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID EXPERIENCE
Some consideration can be drawn from the analysis of the operation of ENEA GRID infrastructure in the last years.
User acceptance of GRID working mode requires a new cultural approach to the access to computational resources and depends strongly on the service quality:
● “Value” of the accessed resources
● Efficient networking and system faults monitoring
● Support of user customization for GRID environment
The administration of a distributed & integrated resource system is a technical, cultural and organization challenge in a diversified structure as ENEA.
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ITALIAN NATIONAL AGENCYFOR NEW TECNOLOGY, ENERGY
AND THE ENVIRONMENT
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID improvements in progress
Increasing resources: AIX +32 SP nodes/Frascati, Cluster 16 AMD 64bit/Casaccia, Cluster 8 G5/Trisaia, 2 CPU AIX/Brindisi (TELEGRID Project).
Extending Windows services.
AFS optimization.
Studies in collaboration with CASPUR to manage the replication of large amount of data over WAN with better performances than AFS.
Participation in CASPUR storage technology tracking activity (SAN systems and file systems testing and evaluation, AFS implementation for parallel systems [IPP Garching]=> DEISA Grid Project)
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
Connection with other GRID Projects
Each GRID project is characterized by its own architectural design and middleware.
The focus of the partecipation of ENEA INFO in other GRID projects is GRID interoperability
Access to ENEA GRID through other GRID models should be compatible with ENEA GRID architecture: gateway implementation.
Current GRID projects:
● EGEE [Datagrid]
● FIRB/Grid.it : WP13, Computational Chemistry (CHEMGRID), Perugia Univ., ENEA Casaccia, et al. ENEA activity starting July 2004 at Frascati
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA GRID and DATAGRID PROJECT
ENEA has partecipated (unfunded) in DATAGRID project (ended 3/2004) through the collaboration with ESA, ESRIN Laboratory in Frascati.
A gateway bewteen DATAGRID infrastructure and ENEA-GRID has been set-up at ESRIN enabling the submission of jobs from DATAGRID to ENEA-GRID
The gateway has been tested running the analysis of the data from the GOME satellite (Ozone level) using IDL codes which can be easily run in a multiplatform environment.
The gateway provided a limited integration between the different GRID environment: EGEE should a attain a full integration
GRID computing Frascati 14/10/2004 [email protected] www.telegrid.enea.it
ENEA-GRID and EGEE Project
EGEE (Enabling GRID for e-science in Europe) is a project in the 6th EU framework program, with the mission to deliver production level GRID services [www.eu-egee.org].
ENEA is one of the funded partners in the project, inside the Italian partecipation which is coordinated by INFN.
EGEE project is starting (April 2004); a kick-off Meeting has been hold in Cork, April 2004.
The installation of a small testing environment (7 Linux boxes) with standard EGEE middleware is currently in progress at ENEA Frascati.
The development of a gateway to ENEA-GRID will follow.