CHEP04 Interlaken, CH 26 September – 1 October 2004 adele rimoldi University of Pavia & INFN, Italy [email protected] The full detector simulation

CHEP04 Interlaken, CH 26 September – 1 October 2004

adele rimoldiUniversity of Pavia & INFN, Italy

[email protected]

The full detector simulation The full detector simulation for the ATLAS experiment: for the ATLAS experiment:

status and outlookstatus and outlook

A.Rimoldi, J. Boudreau, D. Costanzo A. Dell’Acqua, M. Gallas, A. Nairz, V.Tsulaia

mailto:[email protected]

Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 2

outlook

Simulation data flow

The ATLAS Simulation Project G4ATLAS the Geant4-based simulation for ATLAS The ATLAS Simulation from the Data Challenge and Physics Validation

perspective

The ATLAS Detector in GEANT4 and its Subdetectors the Inner Detector Simulation The ATLAS Calorimeters simulation The Muon System Simulation

The ATLAS Testbeam

The Detector Digitization

The Simulation Validation Preproduction and DC2 Memory usage @runtime Timing for different event samples

The Data Challenges in ATLAS

The Physics with DC2 and CTB as a feedback for Simulation

Conclusions


ATLAS Simulation data flow

Generator McTruth(Gen)HepMC

ROD EmulationAlgorithm

L1 Digitization

Particle Filter Simulation

PileUp

McTruth(Sim)HitsROD Input

Digits

McTruth(PileUp)

DigitizationRawDataObjects

ByteStreamConversionSvc

MergedHits

L1Digits

L2Result

EFResult

L1 Emulation(inc. L1 ROD)

L1Result

ROD Emulation

(passthru)

L2 SelectionAlgorithm

EF SelectionAlgorithm

ByteStream

Use

s R

aw

Da

taO

bje

cts

ATLAS


The ATLAS Simulation Project

Present Status

GEANT3-based simulation was operational for the last 10 years, now discontinued (mid 2004)

• It provided a simulation infrastructure used for Data Challenges(DC0,DC1), heavy ions, early testbeam and design optimization, experiment commissioning

GEANT4-based simulation developed in a full OO environment since 2000• Very detailed and up-to-date in all the previous items, in most cases more

accurate and performant used for DC2 the 2004 combined testbeam, the last before the first data-taking Heavy ions production Ready for early commissioning studies

A strategy for passage from GEANT3 to GEANT4 was successfully launched and followed in ATLAS end of last Year

• ATL-SOFT-2003-013 . Strategy for the transition from Geant3 to Geant4 in ATLAS. by:Barberis, D.; Polesello, G.; Rimoldi, A ; Geneva : CERN, 13 Nov 2003

Now the Geant4-based simulation is the main simulation engine in ATLAS


G4ATLAS: the GEANT4-based simulation for ATLAS

Features Completely written in C++ Extensively usage of dynamical loading and action on demand Completely embedded in the ATLAS ATHENA framework Success story in terms of

• open software • Multi programmers facility• Results: Performance and robustness optimal after a short ramp-up

Started as standalone exercise, then embedded in the ATLAS framework, now fully operational for experiment and testbeam purposes with the same software

POOL utilized for the I/O

Functionality Most functionality is there. Interactivity is provided Python scripting replacing the old macro-files structure

Developments backward compatibility always provided Not the end of the story: improvemnet foreseen in many fields (background treatment, visualization,

more interactivity, documentation for end users, etc.)

Validation process Multi-step process through Data Challenges -> see next slide Readiness for the next Physics Workshop in 2005 with new features and upgrades


The Atlas Simulation in GEANT4 from..

the Data Challenge viewpoint DC0 (since end 2001 tests of event productions Geant3 & Geant4) DC1(Phase I ->II)

Geant3 based• Validation samples (single particle, Et scans,Higgs) 740K ev• Single-particle production 30 million ev• Minimum-bias production 1.5 million ev• QCD di-jet production 5.2 million ev• Physics events requested by HLT groups 4.4 million ev• Pile-up • Data samples requests from end-user community

DC2 and following GEANT4 based

• large scale physics analysis, tests on computing model, test calibrations and alignment procedures

12 millions fully simulated events• And a grand total of 1 job crash !And a grand total of 1 job crash !

Distributed production

• 1M Z->ee events in 10K jobs and no failures (@NorduGrid)

the Physics Validation viewpoint Used since 2001 mainly for testbeam simulations and simple setups from 2004 for physics events analyses (Z->ee, , single particle ..)

• Growing users community (the only way of shaking down bugs…)• Comparisons with real data in the testbeams, different layouts


The ATLAS detector in Geant4

Four main subdetectors Inner detector - momentum

measurement of charged particles, electron ID

• high precision silicon trackers• straw tracker with TR capability

Calorimeters - measurement of particle energies

• EM LAr calorimeters (barrel & endcap)• Hadron Lar calorimeters (endcap)• Scintillating Tile hadron calorimeter

Muon spectrometer - muon identification and measurement

• High precision Drift Tubes for tracking, RPCs and TGCs for triggering

Magnet system - bending of charged particles for momentum measurements

5.2M volumes objects(G3 27M) 110K volume types (G3 23K)


The Inner Detector

Eta

Red: InitialBlack: Final

Eta


GeometryThree subdetectors components

• Pixel• SCT• TRT

Final / initial layout available, preliminary validation on hits content done

Still to do• Allow global movements of the Pixel• Increase the level of details

Detector responseTuned on test beam resultsHome-grown TR model

DigitizationAdapt hit reading for pile-upIntroduce the concept of time in the digitizationNoise for TRT

Digitization packages used by the Combined Test Beam Tuning with data to feed the Atlas Simulation

Eta

Pixel

SCT

Pixel+SCT



The ATLAS Calorimeters Simulation

4 subsystems Electromagnetic barrel (EMB) Hadronic end-cap (HEC) Forward calorimeter (FCal) Hadronic calorimeter (Tile)

Heavy tests/investigations for optimizing the physics in Geant4 the geometry for reducing memory consumption and cpu

time• Parameterization studies ongoing

Main software infrastructure issues: the detector description

no more hand-coded numbers, full GeoModel version available (library of geometrical primitives for describing detector geometries)

-> V.Tsulaia talk #279, tomorrow the versioning of the database constants

• LAr has already been switched to Oracle the inclusion of calibration hits

• do a careful accounting of where the energy goes in ATLAS


The Muon System Simulation

System composed by Four main subdetectors

• 2 precision chambers tracking detectors MDT, CSC

• 2 trigger chambers detectors RPC,TGC

Interleaved with the toroids structure • Feets and rails

The outermost detector of ATLAS• Services and cables passing through

Pileup & cavern background• Functionality for handling pileup in place

digitization time window for all technologies Current DC2 production: no cavern background yet, only minimum-bias Full background as in DC1 expected soon

CombinedTestBeam (Muon side)• Robustness of sim-digi chain demonstrated through tons of events produced

Comparisons with real data for all technologies• Now the fine-tuning stage


The ATLAS Testbeam

The 2004 CTB case All the components for a

complete ATLAS sector are being tested together on a beam line in Summer 2004 (Combined TestBeam Setup)

• @different beam energies• Magnetic field (2)• Ancillary detectors• Customized beam profile at

generation• Deep comparisons with real

data sample in each subdetector prototype

• Fine tuning• Full chain Simulation-

Digitization-Reconstruction done!.

• In production mode.

• Same software as for the full experiment


The Detector Digitization

The digitization procedure is disentangled from the simulation process proper and it can be started from pregenerated hits or in a full chain

Fully functional and "leak free" since months Each subdetector loaded on demand Digitization of GEANT4 hits done Hits I/O with Pool for all subdetectors

Expect extensive Validation work from the CTB by comparing with data All assumptions (resolutions, smearing, …) to be revisited

Pile up

Pile-up stays in ~1Gbyte of memory, takes 5mn/event (1GHz) and needs ~10Mbyte/event of disk Pythia used for the min-bias pile up events

• A set of 500K min bias events with the atlas tunings is used To do

• Optimize I/O use vs. memory

• Occupancy studies need to be done

• MC truth navigation, not supported for pile-up

• Lot of work still needed. Both for Validation and new Development


The Simulation Validation Preproduction and DC2

preliminary tests started at end 2003

comparisons with Geant3 using common event samples and about the same geometry.

Hits and digits for all the ATLAS subdetectors generated

jobs run in parallel using the LSF batch facility and Castor facility (at CERN and outside)we measured at different event/run phases the local peformance and memory usageGenerated samples

Single particle vs. E SUSY events H->4 leptons, Z->2leptons(e, mu,tau) di-jet minimum bias

Initial failure rate of ~10% for single particle jobs (30% in physics events), corrected patiently (geometry problems, G4 physics problems…)

Final failure rate is approximately 0% apart from AFS or Castor problems. All jobs go straightforward to the end


Watching the application @runtime

Configuration @ Process Size @runtime (MB)

G4 initialisation (application alone) 50 MB

building ATLAS envelopes 0.9 MB

building ID Geometry + SensitiveDetector 2.2 MB + 0.3 MB

building LArCalo 83.7 MB

building Tile Geometry + SensitiveDetector 0.6 MB + 0.3 MB

building Muon Geometry + SensitiveDetector 8.0 MB + 0.2 MB

geometry optimisation 30 MB

magnetic field 12 MB

loading of full (default) Physics Lists 49 MB

External Dependencies reading events from ROOT files 27 MB

declaring POOL in jobOptions (hit persistification)* ~40 MB

LAr hit calibration ~ 100 MB

using ORACLE database (under study now) ~ 100 MB

ID GeoModel @ initialisation (overhead) ~18 MB

Inspections @runtime allow to control the memory consumption control the possible memory leaks during data production evaluate pros / cons when a new feature is implemented

They are possible everywhere in the production flow, particulary useful at Begin of Run Begin of Event Begin of Step

And at EOR, EOE, EOS


G4 Timing: single particle and full events

2191.2200 H(130) 4l, ||<5 2200394420664.0

2265.5200 Z ee, ||<5 1890407790686.5

2546.1200 SUSY/SUGRA events,||<5 1950458290771.5

Configuration Event SizeCPU Time per Event

Full Detector (DC1 Layout) [NCU-s] [SI2k-s](2.4 GHz PIV) [s] [kB]

Heavy ions (Hijing) 3 full events produced

Configuration Event SizeCPU Time per Event(2.4 GHz PIV) [s]

±,||<3.2 ±, ||<3.2e±, ||<2.5 [kB]

Full Detector (DC1 Layout) 1.33 22.7 /40.4 / 10.654.0469.21

Muon System + Toroids

Tile Calorimeter

LAr Calorimeter

Inner Detector

Full Detector, no B-Field

.17

.16

.82

.11

.92

147.9 / 74.4 /2.738.8216.45

0.7 / 0.8 / .319.1310.91

5.8 /16.6 / .6149.5769.50

14.0 /10.2 / 5.1 .930.44

22.0 / 39.3 /10.245.0160.27

pT=50 GeV


The Physics with DC2 & CTB as a feedback to Simulation

Our goals Get physics community familiar with the new software, new persistency, new analysis

tools Use large produced samples to understand performance and tune new simulation Test algorithms on CTB data:

• Understand limitations of simulation• Understand key issues for reconstruction algorithms• Tune simulation parameters

Get to the end of the year with large well-understood samples of simulated data, stable and tested software chain

• Full simulation analyses (signal + background) for initial detector setup on key physics channels

Inject additional realism into simulation studies

Waiting for feedback from our users community


Conclusions

The Simulation Geant4-based was successfully tested and it has by now replaced the Geant3-based one

We extensively measured the performance and robustness of the new simulation with great success

We can use Parameterizations for further improving the simulation performance

Geant4 is the main engine for the simulation in ATLAS


Thanks

To all the core developersFor the robust, versatile and complete code provided

To the subdetector people For their prompt implementation of any new functionality provided

To the GEANT4 collaboration for their help in transforming this exercise into a success

Documents

CHEP04 Interlaken, CH 26 September – 1 October 2004 adele rimoldi University of Pavia & INFN, Italy [email protected] The full detector simulation