Future Software Needs HDGeant4 project status MC simulation on the OSG Exploiting cheap GPUs for PWA Richard Jones – University of Connecticut GlueX Collaboration

Future Software Needs

HDGeant4 project status

MC simulation on the OSG

Exploiting cheap GPUs for PWA

Richard Jones – University of Connecticut

GlueX Collaboration Meeting, Newport News, May 21-23, 2012


HDGeant4 project: work plan

1. Download and install a recent stable release of G4

2. Create a template from one of the standard examples

3. Design classes for geometry import from hdds

4. Implement geometry import from hdds

5. Make a set of scripts to generate some standard views of GlueX

6. Implement the particle gun event generator

7. Design classes for reading Monte Carlo events from hddm stream

8. Implement Monte Carlo event input

9. Design classes for internal cobrems generator

10. Implement and test the internal cobrems generator

11. Create and document a standard input control file for a few common scenarios

12. Set up mechanisms for verbose tracking output

13. Design classes for hits collection and output

14. Implement hits collection and output to hddm

15. Implement the history mechanism for generating useful truth information

16. Begin the process of porting hits code from hdgeant to hdgeant4

progressso far


HDGeant4 : first look with G4 vis


HDGeant4 : first look with G4 vis


HDGeant4 : upstream closeup

addCutawayPlane


HDGeant4 : more cutaway views








HDGeant4 : views of the pair spectrometer


HDGeant4 : views of the pair spectrometer

old asymmetricreadout designis still in thesimulationgeoemtry


HDGeant4 : next steps The geometry has accumulated defects since the

original implementation in hdds example: readout cables from CDC are readout cables from CDC are

completely outside of their mother volume!completely outside of their mother volume! Geant3 is largely blind to such defects, although the although the

results are affected by them.results are affected by them. Geant4 gets very worried by inconsistencies in the

geometry – and crashes. Next step: work through geometry defects and fix

them one by one – time consuming but essential.time consuming but essential. Remaining steps: 5 – 10 days of workRemaining steps: 5 – 10 days of work


MC simulation on the OSG

Total OSG capacity GlueX usage past and present GlueX in-kind contribution (UConn-OSG) Historical usage patterns: supply & demand


OSG Weekly Usage by all VO’s


Date range: 2009-9-1 - 2012-05-21

OSG Usage History by GlueX


Date range: 2009-9-1 - 2012-05-21

UConn Site Usage History by OSG


Date range: 2009-9-1 - 2012-05-21

OSG Usage History by Gluex

0

500000

1000000

1500000

2000000

2500000

3000000

3500000

4000000

4500000

wal

lclo

ck h

ours


OSG All-Hands Meeting – Gluex slide

Gluex VOGluex VO created 9/2009 Experiment is in construction phase until 2014construction phase until 2014 Usage increasing with demand for Monte Carlo

Growth has slowed as work increases to digest the results Task:Task: simulation of background QCD photoproduction (Pythia) PurposePurpose: develop cuts to suppress background, measure leakage from

minimum-bias events into signal sample after cuts, requires very large statistics MC samples, shared between analysis tasks.

Plans: saturate at the level 5-10M cpu.hr/yr until physics data collection begins ca. 2015.

Strategy: glideinWMS – support from OSG admins outstanding !

run periodrun period usageusage9/2009 – 9/2010 26.4 khr9/2010 – 9/2011 1.1 Mhr9/2011 – present 2.1 Mhr



The short story of what has been learned at UConnThe short story of what has been learned at UConnapplying gpu’s to the PWA study of b1,p final stateapplying gpu’s to the PWA study of b1,p final state

• Igor Senderovich• Jon Rifkin, UConn UITS



factor x100for amplitudes

NVIDIAGTX-580gamer gpu

$420 on Newegg$420 on Newegg

What about double precision?What about double precision?



Latest news: development at UConn of a new 5p PWA generatorbased on Matt Shepherd’s AmpTools package withhooks for GPU – considerable customization is required…considerable customization is required…

Question: Are there any hidden gotchas with amplitude generation in SP?Question: Are there any hidden gotchas with amplitude generation in SP?

ABS. ERROR REL. ERROR CPU AMPLITUDE GPU AMPLITUDE============ ========== ======================== ======================== 2.800000e-13 0.0000080 -2.298916e-08 2.625512e-08 -2.298916e-08 2.625484e-08 1.166190e-12 0.0000030 -3.518453e-07 -1.532152e-07 -3.518443e-07 -1.532146e-07 1.648423e-12 0.0001306 7.594444e-09 -1.008245e-08 7.593624e-09 -1.008102e-08 2.239863e-12 0.0005734 -3.334717e-09 -2.034811e-09 -3.336553e-09 -2.036094e-09 3.032227e-12 0.0000156 -1.841378e-07 -6.039711e-08 -1.841351e-07 -6.039573e-08 1.040433e-11 0.0000043 -2.394655e-06 1.663630e-07 -2.394661e-06 1.663715e-07 1.204159e-11 0.0000058 -1.260541e-06 1.662465e-06 -1.260529e-06 1.662466e-06 1.352122e-11 0.0000101 -1.337222e-06 -9.173907e-09 -1.337235e-06 -9.177625e-09 1.414214e-11 0.0000020 6.550594e-06 2.532435e-06 6.550580e-06 2.532437e-06 1.469193e-11 0.0000991 1.477365e-07 -1.291444e-08 1.477507e-07 -1.291821e-08

Event selection is essentially identical using either CPU (dp) or GPU (sp)

Documents

Future Software Needs HDGeant4 project status MC simulation on the OSG Exploiting cheap GPUs for PWA Richard Jones – University of Connecticut GlueX Collaboration