Embed Size (px)
Citation preview
www.consorzio-cometa.it
Consorzio COMETA - Progetto PI2S2UNIONE EUROPEA
NEMO Monte Carlo Application on the Grid
NEMO Monte Carlo NEMO Monte Carlo Application on the GridApplication on the Grid
R. CalcagnoR. Calcagno for the NEMO Collaborationfor the NEMO Collaboration
Grid Open Day at LnsGrid Open Day at LnsCatania 2008 January 23Catania 2008 January 23
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 2
The NEMO project
Shore stationShore stationShore stationShore station
Electro-optical cableElectro-optical cableElectro-optical cableElectro-optical cable
KmKm33 detector detectorKmKm33 detector detector
R&D for the design and construction of a km3 high energy neutrino detector
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 3
Neutrino astronomy: detector concept
neutrino
muon
Cherenkov light
~5000 PMTatmospheric muon
depth>3000m
Detection of Cherenkov light in transparent natural media (water or ice)
neutrino
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 4
Neutrino detection: signal and background
Search for muon tracks from neutrino interactions
Earth is used as shield to cut off the downgoing atmospheric µ
background
Upgoing tracks can unambigously be attributed to
neutrinos
The predicted neutrino fluxes are weak and it is necessary to take
into account the expected background noise.
One of the main sources of background are the atmospheric
muon generated by the interaction of cosmic rays with
the atmosphere.
Simulation of this background needs a huge number of
MonteCarlo events.
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 5
NEMO proposed km3 telescope
main EO cablemain Junction Box
secondary JB
“tower”
km3 size square array of detection structures ( “towers“ )
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 6
Code Chain
Corsika Music Km3 Modk40 Femsim Reco
Code chain for atmospheric muon simulation are:
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 7
Simulation
• Corsika code simulates the interaction of cosmic rays with the atmosphere and the propagation of the generated showers.Muons are propagated up to the sea level.
• Km3 simulates the light emission, the propagation in water,and the detection by the PMTs. In order to simulate the light radiation is necessary a parameterization which depends on the muon energy lost in the trace segment and on the optical water properties. PMT hits are simulated.
• Music simulates the muon propagation in the sea water.
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 8
Simulation
• FemSim simulates the working operation of the electronic devices
• Reco performs the reconstruction of the muon track direction
• Modk40 adds hits due to optical background that in salt water is due 40K
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 9
Simulation Codes Structure
Simulation codes are performed in sequential way: the output of the first code is the input of the second one, the output for the second one is the input of the
third one and so on
Moreover, many files are necessary to the code chain to
run (detector geometry file, input parameter files, tables …)
Simulation codesI n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
CORSIKA
MUSIC
KM3
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t RECO
MOD40
FEMSIM
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 10
Grid Application
Background Monte Carlosimulation needs a wide
statistic and big processing time, therefore GRID is the
ideal tool to simulate this huge amount of events.
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
CORSIKA
MUSIC
RECO
KM3
MOD40
FEMSIM
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 11
Grid Application
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
The developed application is a trivial parallel execution of simulation codes on the Grid.
...
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 12
Implementation
• The first of Code Chain is Corsika
O u t p u t
CORSIKA
Input file size < 10MB...
Inputs
EGSDAT4_.15 esegui_100.sh
corsika_flat.exe EGSDAT4_3.
EGSDAT4_1.QGSDAT01
EGSDAT4_.05 EGSDAT4_3.
EGSDAT4_.4 EGSDAT4_25.
NUCNUCCSinputs_v2_100
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 13
Upload on SE
A virtual directory is created on the storage element (SE)
The output file is downloaded
The upload of Corsika file is performed on a SE and is recorded on the LFC Catalog
CORSIKA
CORSIKA
SE
O u t p u t
WN
CORSIKA
The submitted job transfers the input files from the SE to the WN
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 14
Upload on SE: User
SE
SE
Catalog File
SEU I
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 15
Input for Music
( ~200 MB zipped file )
TestSite.dat
Input file size > 10MB
O u t p u t
MUSIC trasf_out_wn_100.sh
music-eloss-seawater.dat
music-cross-sections-seawater.datmusic-double-diff-water.dat
music-double-diff-seawater.dat
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 16
I/O Upload on the SE
The output Corsika and the Input file are uploaded on a SE and are recorded in the LFC Catalog
The job is submitted
The output file is downloaded
OUTPUT CORSIKA
WN
INPUT >10M
SE
WN
...MUSIC
...
CORSIKA ...
O u t p u t
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 17
Replicate file
• A file can be stored on multiple SE.
lcg-rep –-vo VO –d addressSE pathLFN
SE
SE
SE
SE
INPUT >10M
INPUT
(OUTPUT_
_CORSIKA)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 18
Job Description Language (JDL)The jdl is a high-level, user-oriented language for describing jobs and
aggregates of jobs such as Direct Acyclic Graphs (DAG)
A jdl file describes a job that can be run on the Grid
UI
CE
SE
Mass
Storage
Input
Sandbox
Output
Sandbox
WN WN WN WN WN WN
RB
Job execution
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 19
InputSandbox
The input file is transferred
from the local UI to the WN
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 20
OutputSandbox
The output for downloading
from the WN to the UI
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 21
Job requirements on resourse (CE)
Requirements
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 22
Job List Match
• This list shows the Computing Element that matching our job requirements.
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 23
no success
Polling
sleep 90 s
status
submit
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 24
no success
Polling
sleep 900 s
status
submit
outputsuccess
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 25
Input File
• The input for Corsika will be different for every chain performed in parallel way, changing the SEED parameter.
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 26
Step
User chooses:• the number of job to execute • a seed value as reference
To set up: • n files .jdl to submit on Corsika• n files .jdl to submit on Music• n files .jdl to submit on Km3• … … …• n files .jdl to submit on Reco
To observe
the sequence
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 27
Trivial Parallel Execution
CE
WN WN WN WN WN WN
CE
WN WN WN WN WN WN
RB
Every jdl file
works in independent
way the one from the
other on the destination
WN chosen by the CE
infnlns-ce-01.ct.pi2s2.it:2119
unipa-ce-01.pa.pi2s2.it:2119
unime-ce-01.me.pi2s2.it:2119
CE
WN WN WN WN WN WN
CORSIKA-1 CORSIKA-2
MUSIC-1
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 28
GRID implementation
Expert
jdl 1
jdl 2
jdl N
jdl 1
jdl 2
jdl N
jdl 1
jdl 2
jdl N
WN
WN
WN
WN
WN
WN
IN File 1
EXE File
IN File 1EXE File
IN File 1
EXE File
IN File 1
EXE File
IN File 1EXE File
IN File 1
EXE File
Output
Output
Output
Output
Output
Output
submit status get-output execute
CONTROLLER
submit status get-output execute
CONTROLLER
CORSIKA MUSIC KM3
...
...
...
...
......... .........
The codes need the use of libraries (CERN libraries, NAG libraries…)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 29
Why Grid ?
• Many CPU
• To ask specific requirements
• Redundance of the systems
• Many SE
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 30
Any Questions ?
G R
I D
