1

ParallelizationGeant4 simulation is an embarrassingly parallel computational problem – each event can possibly be treated independently

2

Why parallelizeMonte-Carlo simulation is a computationally demanding problem.- For good statistics need large number of primary events- Each primary may generate large numbers of secondaries- Complex and large detector setups – from the LHC to Earth or

space

Simply increasing the GHz count on CPUs has reached its limits, more practical (thermally and cost-wise) are multicore, multi-processor and cluster-architecturesJust another form of divide and conquer: if the problem is to large to solve it as a whole, sub-divide it.

3

How to parallelize?Simplest solution: run the same application on multiple cores/computers with different initial parameters in parallel.- Few to no adaptions of application necessary- Overhead because resource sharing is not used- Need additional tools to collect results

Intermediate solution: integrate task scheduling into application. It is still run as a complete process but is aware that it may share resource and need to collect results.- Modification of application necessary- Processes still generally run in their own memory space- Result collection implemented in application

4

How to parallelize?Multi-Threaded- Less overhead because resources are shared (only one process)- Need to take great care to not have race conditions (e.g. on thread

writes into memory another thread reads)- Bound to local machine

Multi-Threaded and Distributed- Requires extensive adaption of application- Message passing e.g. with ZeroMQ needed- Data is passed as message, which may not be replied to- Message are passed transparently over network or memory- Very powerful and scalable, nowadays often runs into I/O-bound

limits.

5

Application-level parallelizationCurrently available concurrency solutions in Geant4:Parallelize on a run level, i.e. run multiple full simulations with different random seeds, then combine results afterwards- Not really concurrent Geant4, but concurrently run Geant4- User needs to take care of result composition afterwards- Overhead from full Geant4 application instantiated for each

simulation (process vs. thread)- Nevertheless: straight-forward, minimal adaption of existing

simulation, useful for great variety of parallel architectures

//in application mainG4long seed = time(NULL); // get a “unique” seedCLHEP::HepRandom::setTheSeed(seed); //set this seed for the RNG

6

Process-level ParallelizationParallelize on an per-event level using ParGeant4- Replaces run manager with a parallel version which uses TOP-C

based messaging to launch slave processes- Marshaling of hit data has to be done by user using specific

comments, i.e. which data can be copied, how should it be copied.- ParGeant4 takes care of aggregating results of parallel processing

into sequential containers- Can parallelize on multiple cores and multiple machines- Each slave is essentially a complete Geant4 application in itself- No thread level parallelism

7

Parallelization HintsIf you do not want to have to think about sharing memory, marshaling data or where to modify your code: run parallel application.- A simple bash script can start up multiple simulations - Depending on how you persist your results they can often easily

be combined by pure concatenation.

Using ParGeant4 is more elaborate but also requires more code modification, i.e. there are more banana skins along the way.