Lecture 11.1: Purpose, Uses, and Limits of Simulationodessa.phy.sdsmt.edu/~bai/zzteaching_2013Fall/PHYS... · Figure : FD data samples compared to predictions with and without oscillations

Lecture 11.1: Purpose, Uses, and Limits of

Simulation

Prof. Luke A. CorwinPHYS 733

South Dakota School of Mines & Technology

Nov. 8, 2013

L. Corwin, PHYS 733 (SDSM&T) Lec. 11.1: Simulation Intro. Nov. 8, 2013 1 / 32

Outline

Outline

1 Purpose

2 Uses

3 LimitsKnown KnownsKnown UnknownsUnknown Unknowns

4 References


Purpose

Reminder

From Reality to Data

Reality Reality↓ ↑

Interactions with MediumData Analysis

Simulation

↓Signal Particles

↓Signal in Detectors

Calibration↓Detector Output

↓ ↑Data Data


Purpose

Introduction

Definition

“the imitative representation of the functioning of one system orprocess by means of the functioning of another” [6]

Stages [5]

Simulate physical interactions and processes (e.g. beam)from theory and measurement

↓Simulate the geometry, materials, and response of theexperiment, including sub-detectors and electronics

↓Simulate detector response to particles

↓Simulated Data


Uses

Determining Purity, Efficiency, Optimal Cuts

In many particle physics analyses, we have some (often large)amount of events in our detector. We want to select onlythose of interest to us.

We have several parameters upon which we can placeconstraints (“cuts”) or apply more sophisticated procedures

We do not examine the data where our signal might befound. “Blinding” and is done to avoid inadvertent bias.

We simulate our analysis to determine optimal cuts, based onefficiency purity, or other factors

Efficiency =number of events selected in a given category

total number of events in that category in the data set

Purity =number of events selected in a given category

total number of events selected


Uses

Training and Validation Samples

If the simulation includes any random number generation,each simulated data set is unique due to statisticalfluctuations

If we optimize cuts on a particular simulated data set, thewise and standard practice is to measure the purity,efficiency, etc. on an independent simulated data set.

The set upon which we optimize our cuts is called the“training” set

The set we use to measure purity, efficiency, etc. is called the“validation” or “testing” set.


Uses

Figure : Measurement of νµ CC Inclusive Cross-Sectionσ(νµA→ µ−X) in the NOvA Near Detector on the Surface (NDOS)


Uses

Verifying Analysis Techniques

When we run a simulation, we know the true values of allparameters, such as the energy of the particles.

Our simulation takes the particles though all physicalinteractions and detector responses.

We can run this fake data through our analysis andreconstruction procedures to obtain the reconstructed energy.

Compare the true and reconstructed energy.

Their difference may be assigned as a systematic uncertainty.

If the difference is large enough, a better analysis technique isneeded.


Uses

Figure : Measurement of νµ CC Inclusive Cross-Section in NDOS.Events with tracks longer than 2m long are from νµ CC interactions.


Uses

Experimental Design

Before an experiment is built, all of the specifications need tobe chosen

This is commonly done though simulation

Simulation can estimate backgrounds or sensitivity to desiredparameters

This simulations are by necessity less detailed than thoseused in data analyses, but they are very valuable


Uses

LBNE Baseline

Baseline (km)500 1000 1500 2000 2500 3000

Fra

ctio

nC

Pδ

0.0

0.2

0.4

0.6

0.8

1.0

1.2

MHσ5Normal Hierarchy

) = 0.0913θ(22sin35 kT LAr, 5+5 yrs

BkgdτνNo

BkgdτνWith

Baseline (km)500 1000 1500 2000 2500 3000

Fra

ctio

nC

Pδ

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0 CPVσ3

Normal Hierarchy) = 0.0913θ(22sin

35 kT LAr, 5+5 yrs

BkgdτνNo

BkgdτνWith

Figure : The fraction of δcp values for which the mass hierarchy can bedetermined at the 5σ level or greater as a function of baseline (top)and the fraction of δcp values which CP violation can be determined atthe 3σ level or greater as a function of baseline (bottom).


Uses

Finding New Physics

The simulation initially does not include the new orunexpected phenomenon

If unexpected differences betweens simulation and data arefound, this could be an indication of new physics.

A thorough investigation of possible backgrounds andmistakes usually occurs.

The difference may not be new physics (e.g. OPERAsuperluminal neutrinos [3])

Occasionally, we do find something totally unexpected (e.g.Raymond Davis).


Uses

[GeV] )ν

( L[km] / E10

log0 1 2 3 4

Events

0

20

40

60µνAtmos. containedvertex

37.88 ktonyearsµν

[GeV] )µ

( L[km] / E10

log0 1 2 3 4

Events

0

10

20

30µAtmos. nonfiducial

µν

[GeV] )ν

( L[km] / E10

log0 1 2 3 4

Events

0

10

20

30 µνAtmos. containedvertex

µν

[GeV] )µ

( L[km] / E10

log0 1 2 3 4

Events

0

5

10

15

20

25+µAtmos. nonfiducial

µν

Neutrino Energy (GeV)

Even

ts / G

eV

0

100

200

300

400

500

600

0 2 4 6 8 10 12 14

dominated beamµν

POT20 10×10.71

µνcontainedvertex

µν

MINOS data

Best fit oscillations

No oscillations

NC background

Cosmicray muons


Eve

nts

/ G

eV

0

20

40

60

80

0 2 4 6 8 10 12 14

enhanced beamµ

ν

POT20 10×3.36

µνcontainedvertex

µν


Eve

nts

/ G

eV

0

5

10

15

20

25

30

0 5 10 15 20 25

dominated beamµν

µνcontainedvertex µν

Muon Energy (GeV)

Eve

nts

/ G

eV

0

200

400

600

800

0 2 4 6 9 15

dominated beamµν

µnonfiducial µν

Figure : FD data samples compared to predictions with and withoutoscillations. The top row shows the energy spectra of the beam samples,while the bottom row shows the L/E distributions for the atmospheric eventsamples. [1]


Uses

Measuring Values

Usually, we prepare all analysis tools using simulation and dataoutside of the region of interest before “unblinding” the region ofinterest.

Making a Measurment

Many analyses seek to measure the mass of a particle or aparticular physics parameter

We can simulate many different values of that parameter andcheck the resulting simulated data distribution against actualdata

The simplest method is to scan a across a phase space withsimulations (grid search)


Uses

) 2

eV

3| / (1

02

m∆ |

2.0

2.5

3.0

) 2 eV3| / (102

m∆ |2.0 2.5 3.0

disappearanceµνMINOS

modeµν POT 20

10× 10.71

modeµν POT 20

10× 3.36

37.88 ktyr Atmospheric

|2

m∆| = |2m∆|

68% C.L.

90% C.L.

Best fit


Limits

Limits

“The problem with simulations is that they always give youan answer.”1

The simulation is always wrong.

Donald Rumsfeld, the most quoted Sec. of Defense in Phyiscs [7]

...there are known knowns; there are things we know weknow. We also know there are known unknowns; that is to saywe know there are some things we do not know. But thereare also unknown unknowns – the ones we don’t know wedon’t know.

1If anyone can find the source of this quote, I would appreciate it.L. Corwin, PHYS 733 (SDSM&T) Lec. 11.1: Simulation Intro. Nov. 8, 2013 16 / 32

Limits Known Knowns

Computational and Statistical Limits

Precision of simulations is ultimately limited by availablecomputing resources

Each simulated event or interaction requires a certain amountof CPU time and memory.

N-body Simulation

In simulations of large numbers of gravitationally interactingbodies, such as dark and light matter “particles” constituting theUniverse or stars in a galaxy, the gravitational attraction of everyparticle by every other particle must be simulated for sufficientlysmall time steps. Ideally, one would simulate every elementaryparticle, but modern computers are not nearly powerful enoughfor that.


Limits Known Knowns

Millennium-XXL [2]

Simulation Scale

The ‘Millennium-XXL Simulation’ (MXXL) follows the nonlineargrowth of dark matter structure within a cubic region of 4.11 Gpc(3h−1Gpc) on a side. The dark matter distribution is representedby 67203 = 303, 464, 448, 000 particles . . . Its particle mass ismp = 8.456× 109 M�,

Computational Resources

1536 nodes × 2 quad-core processors with 24 GB of RAM

approximately 87 trillion force calculations to reach z = 0

28.5 TB of RAM

Run-time = 9.3 days (wall-clock) = 326 years in serial.

long-term storage space = about 100 TB


Limits Known Knowns

The mass densityfield in theMillennium-XXLfocusing on the mostmassive halo presentin the simulation atz=0. Each insetzooms by a factor of8 from the previousone; the side-lengthvaries from 4.1 Gpcdown to 8.1 Mpc. Allthese images areprojections of a thinslice through thesimulation ofthickness 8 Mpc.


Limits Known Unknowns

Uncertainty in Predictions

Often we are looking for phenomena that have been predictedbut not yet found

We can simulate a wide range of theoretical predictions

Until measurements of these phenomena are made, we oftenmust simulate a wide range of predictions without know ifany of them are accurate.



10 100 1000 10 -13

10 -12

10 -11

10 -10

10-9

10-8

10-7

10-6

10-5

10-4

m

scal

ar(p

b)

m (GeV)r

UKDMC

DAMA

Heidelberg

CDMS

Heidelberg- Genius (projected)

CDMS Soudan (projected)

Summary plotcompilingpredictions fordark matter intheconstrainedMSSM for thespin-independentelastic LSPscatteringcross sectionson protons [4].



Inconsistent Simulations

Often, several different simulation software packages areavailable to simulate a given phenomena (e.g. Fluka,GEANT4, and FLUGG for particle production andinteractions).

Within a given simulation there are often many differentconfigurations that all produce different predictions

In those cases, we can either decide to go with one simulationif we have solid grounds to conclude that it is correct.

More commonly, the difference between simulations isincluded as a systematic uncertainty.



Lack of Measurements

In some cases, we simply do not have data in relevant parameterregions, or the data have large uncertainties. We will take a lookat some examples in the final section of today’s class.


Limits Unknown Unknowns

Unknown Unknowns

Strictly speaking, for current unknown unknowns, this slidewould need to be blank by definition.

When an unknown unknown becomes known, it is called asurprise.



I’m not sure if this ever affected particle physics simulations, butit is too good a story not to tell2.

Pentium FDIV Bug

In 1994, Prof. Thomas Nicely was performing large-scalenumerical calculations involving prime numbers

“I soon discovered (on 4 October) that I was nowencountering a new error, a discrepancy in the long double(sum of the) floating-point reciprocals returned by the x87FPU. The results for the first trillion, as computed on thePentium-60, differed from the results obtained on a 486DX-33by an amount orders of magnitude in excess of that expectedfrom rounding or truncation error accumulation”

4195835.0/3145727.0 = 1.333 820 449 136 241 002 5 (Correct value)

4195835.0/3145727.0 = 1.333 739 068 902 037 589 4 (Flawed Pentium)

2http://www.trnicely.net/pentbug/pentbug.htmlL. Corwin, PHYS 733 (SDSM&T) Lec. 11.1: Simulation Intro. Nov. 8, 2013 25 / 32


Unexpected Backgrounds: Pioneer Anomaly3

The Pioneer 10 and 11 space probes motion was modeled andsimulated very precisely.

For many years, a persistent anomalous acceleration towardsthe sun was noted at (8.74± 1.33)× 10−10 m/s2

Many sources for this acceleration were proposed, frommodifications of General Relativity, to gas leaks, to radiationpressure.

The currently accepted solution required a detailed modelingof the heat absorbed, generated and emitted by thespacecraft, taken directly from the blue prints!

3PRL 108, 241101 (2012)L. Corwin, PHYS 733 (SDSM&T) Lec. 11.1: Simulation Intro. Nov. 8, 2013 26 / 32


OPERA and Superluminal Neutrinos4

Figure : OPERA detected neutrinos arriving at the far detector 60 nsfaster than c.

4http://kds.kek.jp/getFile.py/access?contribId=39&sessionId=

18&resId=0&materialId=slides&confId=9151L. Corwin, PHYS 733 (SDSM&T) Lec. 11.1: Simulation Intro. Nov. 8, 2013 27 / 32

http://kds.kek.jp/getFile.py/access?contribId=39&sessionId=18&resId=0&materialId=slides&confId=9151



Reaction

OPERA announced the result after months of trying to findan error.

Internet goes bonkers.

Every other experiment capable of this measurementprepares to make it.

Resolution

MINOS, T2K, etc. all find no deviation from c.

OPERA finds two problems:

“Faulty connection of the optical fibre to the Master Clockartificially increasing the neutrino anticipation by ∼ 74 ns.”

Internal Master Clock frequency off by ∆f/f = 1.24× 10−7

(124 ns/s) artificially decreasing the neutrino anticipation by∼ 15 ns.


References

References I

[1] P. Adamson et al.Measurement of Neutrino and Antineutrino Oscillations UsingBeam and Atmospheric Data in MINOS.Phys. Rev. Lett., 110:251801, 2013.

[2] R. Angulo, V. Springel, S. White, A. Jenkins, C. Baugh, et al.Scaling relations for galaxy clusters in the Millennium-XXLsimulation.2012.


References

References II

[3] M. Dracos.The neutrino velocity measurement by opera experiment.Presentation given at The XXV International Conference onNeutrino Physics and Astrophysics, Kyoto, Japan, June 8,2012, June 8, 2012.http://kds.kek.jp/getFile.py/access?contribId=

39&sessionId=18&resId=0&materialId=slides&confId=

9151.

[4] J. R. Ellis, A. Ferstl, and K. A. Olive.Reevaluation of the elastic scattering of supersymmetric darkmatter.Phys.Lett., B481:304–314, 2000.





References

References III

[5] T. Lindblad.Lecture 1a: Introduction to physics simulation.http://gluon.particle.kth.se/~fmi/kurs/

PhysicsSimulation/Lectures/01A/index.html.Accessed Nov. 6, 2013.

[6] Merriam-Webster.Simulation - definition and more from the freemerriam-webster dictionary.http:

//www.merriam-webster.com/dictionary/simulation.Accessed Nov. 6, 2013.


http://gluon.particle.kth.se/~fmi/kurs/PhysicsSimulation/Lectures/01A/index.html

http://gluon.particle.kth.se/~fmi/kurs/PhysicsSimulation/Lectures/01A/index.html

http://www.merriam-webster.com/dictionary/simulation

http://www.merriam-webster.com/dictionary/simulation

References

References IV

[7] D. H. Rumsfeld.Dod news briefing - secretary rumsfeld and gen. myers.News transcript, U.S. Department of Defense, Office of theAssistant Secretary of Defense (Public Affairs), 11:30 AMEST, Feb. 12, 2002.http://www.defense.gov/transcripts/transcript.aspx?

transcriptid=2636.


http://www.defense.gov/transcripts/transcript.aspx?transcriptid=2636

http://www.defense.gov/transcripts/transcript.aspx?transcriptid=2636

Documents

Lecture 11.1: Purpose, Uses, and Limits of Simulationodessa.phy.sdsmt.edu/~bai/zzteaching_2013Fall/PHYS... · Figure : FD data samples compared to predictions with and without oscillations