Maria Grazia Pia, INFN Genova Geant4 Electromagnetic Validation (mostly electromagnetic, but also a bit of hadronic…) K. Amako, G.A.P. Cirrone, G. Cuttone,

Maria Grazia Pia, INFN Genova

Geant4 Electromagnetic Geant4 Electromagnetic ValidationValidation

(mostly electromagnetic, but also a bit of hadronic…)(mostly electromagnetic, but also a bit of hadronic…)

K. Amako, G.A.P. Cirrone, G. Cuttone, F. Di Rosa, S. Guatelli, V. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen, L. Pandola, S. Parlati,

M.G. Pia, G. Russo, T. Sasaki, L. Urban

et al.

Geant4 Workshop

Bordeaux, 7-10 November 2005


Geant4 Electromagnetic Geant4 Electromagnetic ValidationValidation

Ample variety of physics models in the Geant4 Toolkit– complementary and alternative

Electromagnetic physics – Standard, LowEnergy, Muon, Optical, Utilities (contains multiple scattering, energy loss!)

Hadronic physics– in some experimental cases EM and hadronic physics cannot be separated

Geant4 Physics BookGeant4 Physics Book– on-going project to document the performance of Geant4 physics against

experimental data and in relevant experimental application domains


Validation processValidation process

Geant4 test process– Physics packages are subject to unit and system testing– Verification, validation of single processes/models performed by Working Groups

Validation process– systematicsystematic: cover all models of a given process– comparison to experimental dataexperimental data and established reference databases– rigorous software processrigorous software process to guarantee quality and reliability

– statistical analysisstatistical analysis: quantitative mathematical evaluation

Goals– evaluate quantitatively the accuracy of Geant4 physics models– document their respective strength– provide guidance to users to select appropriate models in their

applications


Scope of this talkScope of this talk

Common project for Geant4 Electromagnetic Physics validation– systematicsystematic– spanning all all existing electromagnetic models– rigorous software processrigorous software process– performed in a transparenttransparent way (agreed methods & tools, code in CVS)– quantitative results through rigorous statistical analysisstatistical analysis– goal: produce publication-qualitypublication-quality results

Other on-going “private” activities not covered in this talk– no time to present everything…– but also don’t know what is going on…


Overview of recent validation Overview of recent validation activitiesactivities

Geant4 Physics Book: Electromagnetic Volume– comparison against the NIST databases– K. Amako, S. Guatelli, V. N. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen,

L. Pandola, S. Parlati, M. G. Pia, M. Piergentili, T. Sasaki, L. Urban

Comparison of Geant4 electromagnetic physics models against the NIST reference dataIEEE Trans. Nucl. Sci., Vol. 52, Issue 4, Aug. 2005, 910-918

Current Physics Book projects (preliminary results)– Bremsstrahlung final state– Atomic relaxation and PIXE + Radioactivity from composite materials– Bragg peak (EM + hadronic)

Other Geant4 validation activities– Validation of specific physics models done by each Geant4 Working Groups– LCG Simulation Validation Project: focus on hadronic physics– User own projects


NIST TestNIST TestPhoton Mass Attenuation Coefficient

Photon Partial Interaction Coefficient – related to the cross section of a specific photon

interaction process

Electron CSDA range and Stopping Power

Proton CSDA range and Stopping Power

CSDA range and Stopping Power

Elements Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb, U

(span the periodic element table)

Energy rangephoton 1 keV – 100 GeVelectron 10 keV – 1 GeV proton 1 keV – 10 GeV 1 keV – 1 GeV

Geant4 models: electrons and photonsStandard

Low Energy EEDL/EPDLLow Energy Penelope

Geant4 models: protons and Standard

Low Energy ICRU49Low Energy Ziegler 1977Low Energy Ziegler 1985Low Energy Ziegler 2000(Low Energy: free electron gas + parameterisations + Bethe-Bloch)

Simulation configuration reproducing NIST conditions (ionisation potential, fluctuations, production of secondaries etc.)


G.A.P Cirrone, S. Donadio, S. Guatelli, A. Mantero, B. Mascialino, S. Parlati, M.G. Pia, A. Pfeiffer, A. Ribon, P. Viarengo

“A Goodness-of-Fit Statistical Toolkit”IEEE- Transactions on Nuclear Science (2004), 51 (5): 2056-2063

Partly funded by ESA (SEPTIMESS Project)


H0: Geant4 simulation = NIST dataH1: Geant4 simulation ≠ NIST data

Statistical analysisStatistical analysis Goodness-of-Fit test (Statistical Toolkit)

GoF test(χ2 test)

Distance between Geant4 simulation

and NIST reference data

Test resultp-value

Geant4 simulation

results+

ReferenceData

The p-value represents the probability that the test statistics has a value at leastat least as extreme as the one observed, assuming the null hypothesis is true

0 0 ≤ p ≤ 1≤ p ≤ 1

p < 0.05p < 0.05 Geant4 simulation and NIST data differ significantly p > 0.05p > 0.05 Geant4 simulation and NIST data do not differ significantly

Alternative hypotheses under test:


Photon mass attenuation coefficientPhoton mass attenuation coefficientGeant4 models:

• Standard• Low Energy – EPDL• Low Energy – Penelope

Reference data: NIST - XCOM

Mass attenuation coefficient in Fe

Geant4 LowE Penelope Geant4 StandardGeant4 LowE EPDLNIST - XCOM

ResultsResults

All Geant4 models compatible with NIST

Best agreement: Geant4 LowE models

H0 REJECTION AREA

Monochromaticphoton beam (Io)

Transmitted photons (I)

0

ln1

I

I

d

p-value stability study

Experimental set-up


Compton interaction coefficient Compton interaction coefficient (cross (cross section)section)


Compton interaction coefficient in Ag

cc )(

AVN

A


H0 REJECTION AREA

Geant4 models:• Standard• Low Energy – EPDL• Low Energy – Penelope


ResultsResults


Best agreement: Geant4 LowE-EPDL


Photoelectric interaction coefficient (cross section)

Geant4 LowE PenelopeGeant4 LowE Penelope Geant4 StandardGeant4 StandardGeant4 LowE EPDLGeant4 LowE EPDLNIST - XCOMNIST - XCOM


Photoelectric interaction coefficient in Ge

phph )(

AVN

A

H0 REJECTION AREA




ResultsResults


Best agreement: Geant4 LowE models


Pair production interaction coefficient Pair production interaction coefficient (cross section)(cross section)


Pair production interaction coefficient in Au

pppp )(

AVN

A


H0 REJECTION AREA

p-va

lue

(pai

r pr

oduc

tion

inte

ract

ion

coef

fici

ent t

est)



ResultsResults

All Geant4 models compatible with NIST and equivalent


Rayleigh interaction coefficient (cross section)Rayleigh interaction coefficient (cross section)

ResultsResultsThe Geant4 Low Energy models look in disagreement with the reference data for some materials

Geant4 LowE Penelope Geant4 LowE EPDLNIST - XCOM

Rayleigh interaction coefficient in Be

rr )(

AVN

A

Be 0.99 1

Al 0.32 <0.05

Si 0.77 <0.05

Fe 1 <0.05

Ge <0.05 0.39

Ag 0.36 0.08

Cs <0.05 <0.05

Au <0.05 <0.05

Pb <0.05 <0.05

U <0.05 <0.05

EPDLXCOM

PenelopeXCOM

Geant4 models:• Low Energy – EPDL• Low Energy – Penelope• (no standard Rayleigh process)



Zaidi H., 2000, Comparative evaluation of photon cross section libraries for materials of interest in PET Monte Carlo simulation IEEE Transaction on Nuclear Science 47 2722-35

The disagreement is evident between 1 keV and 1 MeV photon energies

For what concerns the Geant4 Low Energy EPDL model, the effect observed derives from an intrinsic inconsistency between Rayleigh

cross section data in NIST-XCOM and the cross sections of EPDL97, on which the model

is based

Differences between EPDL97 and NIST-XCOM have already been highlighted in a paper by Zaidi,

which recommends the Livermore photon and electron data libraries as the most up-to-date and

accurate databases available for Monte Carlo modeling.

Rayleigh interaction coefficientRayleigh interaction coefficient

EPDL 97

NIST

Rayleigh interaction coefficient in Au

rr )(

AVN

A


Electron Stopping PowerElectron Stopping PowerExperimental set-up


H0 REJECTION AREA

)(1

SP dx

dE

Geant4 LowE Penelope Geant4 StandardGeant4 LowE LivermoreNIST - ESTAR

Electrons are generated with random direction at the center of the box and stop inside the box

Maximum step allowed in tracking particles was set about1/10 of the expected range

value, to ensure the accuracy of the calculation

Geant4 models:• Standard• Low Energy – EEDL• Low Energy – Penelope

Reference data: NIST – ESTAR (ICRU 37)

ResultsResults

All Geant4 models compatible with NISTand equivalent


Electron CSDA RangeElectron CSDA Range CSDA: particle range without energyloss fluctuations and multiple scattering

Geant4 LowE Penelope Geant4 StandardGeant4 LowE LivermoreNIST - ESTAR

CSDA range in U p-value stability study

H0 REJECTION AREA

Geant4 models:• Standard• Low Energy – EEDL• Low Energy – Penelope

Reference data: NIST – ESTAR (ICRU 37)

ResultsResults

All Geant4 models compatible with NISTand equivalent


Proton stopping power - rangeProton stopping power - rangeStopping power in Al

Geant4 LowE Ziegler 1985Geant4 LowE Ziegler 2000Geant4 StandardGeant4 LowE ICRU 49NIST - PSTAR

+

H0 REJECTION AREA

Stopping power: p-value stability study

H0 REJECTION AREA

CSDA range: p-value stability study

ResultsResultsZiegler parameterisations are as authoritative as ICRU 49 onesComparison rather than validation


stopping power and rangestopping power and range

H0 REJECTION AREA

Stopping power: p-value stability studyCSDA range in Si

Geant4 LowE Ziegler 1977 Geant4 StandardGeant4 LowE ICRU 49NIST - ASTAR

The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the main motivations for the developing of this package.

The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the

main motivations for developing this package


Quantitative process evaluationQuantitative process evaluation2 conference presentations with preliminary results (CHEP, NSS 2003)Final results presented at NSS 2004, October 2004

~ 10 months to put test infrastructure in place and get first results~ 1 year to get publication-quality results~ 3 months to write paper and produce final plots

Publication – submitted to IEEE TNS March 2005– accepted April 2005– published August 2005

Workers/authors– 13 authors– Barbara, Koichi, Sandra, Susanna + unnamed person did all the work – other people provided significant support to the project (funds, management,

infrastructure etc.)


BremsstrahlunBremsstrahlungg

3 sets of models availableStandard: G4eBremsstrahlungLow Energy EEDL: G4LowEnergyBremsstrahlungLow Energy Penelope: G4PenelopeBremsstrahlung

3 angular distributions: Tsai, 2BS, 2BN

Angular distribution of photons is strongly model-dependent

Penelope

Standard

Low Energy EEDL (default)

PenelopeLowE-EEDL TSAI (def)

LOWE-EEDL 2BSLOWE-EEDL 2BN

Angle (deg) Angle (deg)

Susanna GuatelliBarbara Mascialino

Luciano PandolaMG Pia


Reference dataReference data

The absolute Bremsstrahlung cross section can be tested

Transmitted energy spectrum at two different

emission angles for four materials (Al, Pb, W,

Ag)

Absolute yields are reported (=

photons/primary), though with an “odd” normalization

R. Ambrose et al., Nucl. Instr. Meth. B 56/57 (1991)

327


Relative comparison...Relative comparison...

Relative comparison (45 degree direction)Shapes of the spectra are in good agreement

Work in progress, will be published

Inte

nsi

ty/Z

(eV

/sr

keV

)

Photon energy (keV)

Photon energy (keV)

LowE-Penelope

Low E EEDL - TSAI

Inte

nsi

ty/Z

(eV

/sr

keV

)


Proton Bragg PeakProton Bragg Peak

Geant4 models: electromagneticGeant4 models: electromagneticStandardLow Energy ICRU 49Low Energy Ziegler 1977Low Energy Ziegler 1985Low Energy Ziegler 2000

Geant4 models: hadronicGeant4 models: hadronicPrecompound + default de-excitationPrecompound + GEM evaporationwith/without Fermi Break-upBinary Cascade

(including Precompound + de-excitation options as above)

Bertini CascadeBertini Cascade + Bertini elastic scattering (when available)ParameterisedGeant4 “educated guess” Medical Dosimetry Physics List

Reference data from CATANA (INFN-LNS Hadrontherapy Group)

Systematic test in progress

Lot of work…

Preliminary results

Pablo CirroneGiacomo Cuttone

Francesco Di RosaSusanna Guatelli

Barbara MascialinoMG Pia

Giorgio Russo


EM only – StandardEM only – Standard


EM only – ICRU49EM only – ICRU49ENTIRE

PEAK

Exp G4

S 2.89 2.43

T 3.26 3.83

GoF test CVM-AD

ENTIRE

PEAK

Exp G4

S 2.89 2.43

T 3.26 3.83

GoF test CVM-AD

LEFT TAILx<=30mm

Exp G4

S 9.77 11.89

T 2.66 3.16

GoF test CVM-AD

RIGHT TAIL

X>30mm

Exp G4

S 3.89 12.24

T 1.03 1.00

GoF test KS-AD


EM only – ICRU49 – GoF resultsEM only – ICRU49 – GoF resultsALL (N1=149 N2=67) CVM AD

Test statistics 0.112938 0.853737

p-value 0.525095 0.443831

LEFT TAIL x<=30mm

(N1=140 N2=61)

CVM AD


p-value 0.750593 0.606120

RIGHT TAIL X>30mm

(N1=9 N2=6)

KS AD


p-value 0.724871 0.469251


LowE + precompound defaultLowE + precompound defaultALL Exp G4

S 2.89

T 3.26

GoF test CVM-AD

LEFT TAILx<=30mm

Exp G4

S 9.77

T 2.66

GoF test AD

RIGHT TAIL

X>30mm

Exp G4

S 3.89

T 1.03

GoF test KS-AD


ICRU49 + precompound – GoF ICRU49 + precompound – GoF resultsresults

ENTIRE

PEAK

(N1=149 N2=66)

CVM AD


p-value 0.79 0.747452

LEFT TAIL

x<=30mm

(N1=140 N2=60)

CVM AD


p-value 0.97 0.978972

RIGHT TAIL

X>30mm

(N1=9 N2=6)

KS AD


p-value 0.73 0.413129

0.525095 0.443831

ICRU 49 only

0.750593 0.606120

0.724871 0.469251


Nuclear de-excitation alternative modelsNuclear de-excitation alternative models

Work in progress, more to come…


Studies of environmental radioactivity from rocks and sands at the Gran Sasso Laboratory

Geant4 can reproduce the result of a calibration with a 60Co source (in the presence of the sample) very well

detectorsample

source

simulation

datasimulation

data

Radioactive spectrumRadioactive spectrum

Lower part of the histogram is not meaningful

Anderson-Darling testAnderson-Darling test (for binned data)

Lower E peak– A2 = 0.45 – p-value = 0.80

Higher E peak– A2 = 1.05– p-value = 0.33

Both peaks– A2 = 0.80 – p-value = 0.48

Barbara MascialinoLuciano Pandola

MG Pia


BackscatteringTransmission

etc.Tests

1st set of tests set-up for NSS 2003

BUT

No time yet to bring them to publication

quality

Collaboration for systematic work

desirable

References in part from Standard EM tests


Backscattering coefficient – E=100keVBackscattering coefficient – E=100keV

G4 LowE

Lockwood et al. (1981)

Angle of incidence (withrespect to the normal to the sample surface) = 0°


CommentsComments

Feedback from users is a helpful contribution to Geant4 validation– but the systematicsystematic work needed to document Geant4 validity

quantitatively can only come from the Geant4 Collaboration

Better communication and collaboration is desirable– internal competition in the Geant4 Collaboration is not constructive– competition against Geant4 students and post-docs is not constructive– slow progress is due to limited womanpower

Dedicated validation workshops are necessary and fruitfulfruitful– not easy to organize (and expensive to attend)

Regular VRVS Validation meetings will be organized in 2006

The key to success for publication-quality results is a The key to success for publication-quality results is a rigorous software processrigorous software process


Geant4 Physics Validation Workshop, Genova, July 2005http://www.ge.infn.it/geant4/events/july2005/


ConclusionConclusion

More results available, no time to show them all…– multiple scattering?– atomic relaxation (A. Mantero, B. Mascialino, M.G. Pia, V. Zampichelli)– validation for specific use cases – etc.

Systematic, quantitative validation of Geant4 EM physics in progress– all available models– rigorous statistical analysis

A lot of work!– first paper published– several on-going projects– limited resources available

The validation work provides valuable feedback for the improvement of Geant4 physics models


Geant4 validation is not an easy Geant4 validation is not an easy job…job…

experimental data often exhibit large differences!


PublicationsPublications

It would be desirable to agree a publication plan for Geant4 physics validation

– electromagnetic + hadronic

IEEE Trans. Nucl. Sci.– major scientific journal about nuclear technologies and instrumentation– many Geant4-related papers published or currently in the review process– please consider publishing your results concerning Geant4– copyright compatible with Geant4 Physics Book

Documents

Maria Grazia Pia, INFN Genova Geant4 Electromagnetic Validation (mostly electromagnetic, but also a bit of hadronic…) K. Amako, G.A.P. Cirrone, G. Cuttone,