Internship report - eddata.fnal.goveddata.fnal.gov/lasso/summerstudents/papers/2017/Matteo-Mazzanti-paper.pdf · Internship report Matteo Mazzanti October 29, 2017 Abstract This document

Internship report

Matteo Mazzanti

October 29, 2017

Abstract

This document is a report on the work done at Fermilab by me (Matteo Mazzanti) during the summerschool organized by the university of Pisa between August and September 2017. I’ve been working withthe MINERvA collaboration, in particular with Minerba Betancourt and Patrick Stowell on the tuning ofMonte-Carlo simulation’s parameters for the resonant pion process generated using GENIE. The reportedwork was possible thanks to the stage offered by the Cultural Association of Italians at Fermilab.

1 Objective

The main scope of this internship is to reduce the biguncertainties from some of the model parameters inneutrino simulations that are used for the MINERvAexperiment.Up to now the model coming from Monte Carlo sim-ulation are not in agreement with experimental data;during the internship I’ve used different fitting tech-niques to constrain the values for some parametersfor the GENIE generator, these new values will beuseful for simulations in the DUNE experiment.

2 The MINERvA experiment

MINERvA is a neutrino experiment using a high-intensity beam (NuMI, Neutrino Main Injector beam-line) generated at Fermilab. The main scope of theexperiment is to study the interaction of neutrinowith five different nuclei; normally this has been doneusing an electron beam but never with a neutrinobeam.The neutrino energies used in MINERvA are in arange of 1-10 GeV.Using data from MINERvA experiment will be possi-ble to study neutrino oscillations and their change oftype but also to access various informations about thestructure of protons and neutrons probing the weakforce dynamics that affect neutrino-nucleon interac-tions.The MINERvA detector was finished and installed inMarch 2010, it is composed by:

• A veto wall upstream the detector to shieldagainst low energy hadrons and muons. This is

made by different layers of steel alternated withlayers of scintillators

• Between the veto wall and the main detector isplaced a cryogenic vessel filled with liquid helium

• The inner detector (ID) made by planes of scin-tillator strips mixed with the nuclear targets (C,Pb, Fe, H2O)

• A downstream electromagnetic calorimeter(ECAL) and an hadronic calorimeter (HCAL)

• An outer detector (OD) made by a frame of steelembedded with scintillator

Figure 1: The MINERvA detector

The MINERvA detector is situated nearly 2 metersin fron of the MINOS near detector inside of theNuMI (Neutrinos at the Main Injector) beamline.The NuMI beamline provides an intense νµ beam;protons are accelerated by the 120 GeV Main In-jector against a water-cooled graphite target. Theparticles created by the interaction of protons andcarbon atoms are kaons are pions, those are focused

1

using magnets in the required direction (NorthernMinnesota where the MINOS far detector is located).The neutrino flux is mostly coming from the pionswith momentum between 2 GeV and 60 GeV usingthe decay channel represented in Fig. 2 The MINOSdetector is used to analyse the muons produced in-side of the MINERvA detector.

W+

Figure 2: The creation of neutrinos at NuMIbeamline

3 The GENIE generator andNUISANCE

GENIE [1] is a Monte Carlo event generator that pro-vides a wide energy range for modelling neutrino (ofall flavours) interactions; from 100 MeV to some hun-dred GeV.Genie is based on ROOT and it was developed be-tween 2004 to 2007, version 2.0.0 of GENIE becameavailable on August 2007. The main reason of itsdevelopment was to have a ”canonical” Monte Carlogenerator for interaction of all flavours of neutrinoswith all nuclei. GENIE uses the Relativistic FermiGas (RFG) nuclear model for all process with someapproximations; for A>20 the 2-parameter Woods-Saxon density function is used and all isotopes of aparticular nucleus are assumed to have the same den-sity.The kinds of simulated interaction are three:

• Quasi-elastic scattering: (νµ+n→ µ−+p) mod-elled by the Llewellyn-Smith model [5]

Figure 3: Feynman diagram of Quasi Elastic process

• Resonant Pion: (νµ+n→ µ−∆+(π+ +n)) mod-elled by the Rein-Sehgal model [4]

Figure 4: Feynman diagram of Resonant Pionprocess

• Deep inelastic scattering: (νµ+p→ µ+X) mod-elled by the Aivazis, Olness and Tung model [2]

Figure 5: Feynman diagram of Deep Inelasticprocess

NUISANCE [3] has been created to compare neutrinogenerator’s simulations with real data or with simula-tions coming from other generators, it can work withdifferent Monte Carlo generators (GENIE, NEUT,NuWro, GiBUU and NUANCE). NUISANCE can beused to tune some parameters of the simulation, us-ing the ROOT’s minimizer libraries to obtain the besttuning option; it’s also possible to scan the parame-ters space systematically using ”throws”. Each com-bination of value for the various parameter create athrow, one can scan the space of one or more pa-rameters using a flat probability distribution for thegeneration of the random value for the parameter ora Gaussian probability distribution.

3.1 Events reweighing

Cross section reweighing modifies the neutrinointeraction probability; the neutrino event weight,ωevtσ is calculated as:

ωevtσ =

(dnσ′ν/dK

n)(

dnσν/dKn) (1)

2

(a) An example of likelihood scan using NUISANCE forthe two parameters Eν and Q2

QE

(b) The comparison of effects on EQEν for Nominal and

best fit values of MA parameter

Where:

•(dnσν/dK

n)

is the nominal differential cross sec-tion

•(dnσ′ν/dK

n)

is the differential cross section aftermodifying some physics parameter

For the scattering off nuclear targets the off-shellkinematic, used in the original simulation, has to berecreated before evaluating the cross sections; thiscan sometimes require time. Event reweighing, how-ever, remains a good way to analyse the effects on theMonte Carlo of changes in value for some parameters.This avoids to create a new Monte Carlo sample foreach different combination of values that would re-quire an enormous amount of time

4 GENIE models testing usingDUNE flux

GENIE permits to select different models for the in-teraction between neutrinos and nuclei, to do so theneutrino flux that will be used for the future DUNEexperiment has been used.The various models tested have been:

• Bodek Richie Fermi Gas model (the default forgenerating events)

• Effective spectral function model

• Local Fermi gas model

Moreover different test have been done on the hadrontransport simulation, on GENIE is possible to choosebetween 2 configurations:

• HA : An effective model which just approximateswhether a hadron should leave the nucleus or notgiven its initial momentum

• HN : A Monte-Carlo model where step each par-ticle forwards in tiny steps until it re-interactswith another part of the nucleus or leaves it.

The comparison have been done using the model forthe neutrino/antineutrino flux that will be used inthe future DUNE experiment.

4.1 The DUNE experiment

Using a proton accelerator a neutrino beam will beproduced at Fermilab trying to give it the directionto the Sanford Underground Research Facility, thebeam will be analysed using two different detectors,a near detector (at Fermilab) and a far detector (atSanford).

Figure 7: Schematic of how the DUNE experimentwill work

The main scopes of the experiment are:

• Study the CP violation for leptons using neu-trino oscillations (P

[νµ → νe

]6= P

[ν̄µ → ν̄e

])

3

• Study the hierarchy of neutrino masses

• Search of new kind of neutrinos

4.2 Comparison of Nuclear models

Comparison between different Nuclear models hasbeen done using only the CCQE component, for afirst check the distributions of El (Energy of thelepton) and Eν −El (the energy given to the nuclei)have been plotted.

0 1 2 3 4 5 6 7 8 9 10

N o

f eve

nts

0

50

100

150

200

250

310×

GENIE Bodek Richie FG model

GENIE Effective spectral function model

GENIE Local Fermi Gas model

El

0 1 2 3 4 5 6 7 8 9 10

0.5

1

RATIO def/SF

E [GeV]0 1 2 3 4 5 6 7 8 9 10

0.951

1.051.1

1.15RATIO def/LFG

(a) Distribution of the lepton energy for various nuclearmodels

0 0.5 1 1.5 2 2.5 3

N o

f eve

nts

0

100

200

300

400

500

600

700

800

900

310×

GENIE Bodek Richie FG model

GENIE Effective spectral function model

GENIE Local Fermi Gas model

0 0.5 1 1.5 2 2.5 3

1

2

3 RATIO def/SF

E [GeV]0 0.5 1 1.5 2 2.5 3

0.9

1

1.1

RATIO def/LFG

(b) Distribution of the energy lost in the nuclei forvarious nuclear models

Figure 8: Comparison for various Nuclear modelsand the default one

The simulations have been done for five millionsof interactions between the GENIE flux of neu-trino/antineutrino and the liquid Argon (the twoDUNE detector will use liquid Argon for detection);efficiency of the detector has been taken 100% forthis analysis because the main scope of it is to un-derstand which model is giving the biggest variationsfor the interactions. From Fig. 8 is possible to seethat for the various model the change between boththe distributions and the default model is nearly 5%.

4.3 Comparison of Hadronic trans-portation

The next comparison has been done between the HAand HN model for hadronic transport. From the neu-trino scattering protons and pions are produced thesecan then interact inside of the nuclei or leave it (seeFig.9) In this case a change of shape of the distribu-

Figure 9: Final state interaction particles

tion for the energy of pions or proton is expected dueto the difference in the mechanics of the two models.For these five millions of events have been generatedand the comparison has been done for the distribu-tion of the pions and protons energy and momentum.In this case all the kind of interactions have been gen-erated (CCEQ, DIS, and RES) and the previously de-fined comparison has been done between all of them.It is clear that this comparison is intended to com-pare the different in shape for the various model notfocusing on the differences in the number of events. Infact, one should expect that the percentage of eventsof each kind should converge to the same value forall simulations when going up with the number ofevents but still fluctuations should occur due to therandomization of the process.

4.3.1 Comparison for CCQE

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

100

200

300

400

500

600

700

800

900 GENIE Bodek Richie FG model HN

GENIE Bodek Richie FG model HA

Pion p

P [GeV}

N e

vent

s

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

10000

20000

30000

40000

50000

60000

70000

80000


GENIE Bodek Richie FG model HN

Proton p

N e

vent

s

P[GeV]

Figure 10: Comparison of the HN and HA hadronictransport models for pions and protons

4

4.3.2 Comparison for RES

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

5000

10000

15000

20000

25000

30000

35000

40000 GENIE Bodek Richie FG model HN


Pion p

P [GeV]

N e

vent

s

0 0.5 1 1.5 2 2.5 30

20

40

60

80

100

120

140

160

180

2003

10×



Proton p

N e

vent

s

P[GeV]


4.3.3 Comparison for DIS

0 1 2 3 4 5 6 7 80

50

100

150

200

250

310×



Pion p

P [GeV]

N e

vent

s

0 0.5 1 1.5 2 2.5 3 3.5 4 4.50

20

40

60

80

100

120

140

160

180

200

220

310×



Proton p

P[GeV]

N e

vent

s


5 GENIE parameters tuningusing NUISANCE

As it’s possible to see from 13 there have been somediscrepancies between Monte Carlo simulations andMINERvA data.

Figure 13: Discrepancies between Monte Carlosimulations and data for the hadronic energy

This can be solved tuning the Monte Carlo over datain order to reduce the uncertainties on some param-eters. To tune the Monte Carlo simulations overdata one can use the NUISANCE software. NUI-SANCE is using Migrad algorithm to search for thebest combination of values of the GENIE parame-ters to fit the various data distributions given (forexample Q2,pt,etc.). The scanned space will be anxn space where n is the number of parameters totune, for each combination of parameters the MonteCarlo is reweighed (as described in Section 3.1).The Migrad algorithm, however, has some drawbacks:

• Can fall in a local minima of the parametersspace.

• One needs to re-do the whole minimization whenadding new distributions.

From Fig. 14 is possible to see a systematization ofhow this process works.

(a) A single point in the parameters space

(b) Points with the relative observable’s distribution

(c) Effects of uncertainty on a configuration

(d) Finding the best fit for the parameters

Figure 14: Schematics of the parameter space fit.

Taking a parameter space one can obtain a observabledistribution from a specific configuration (Fig. 14a),taking another point inside the parameters space one

5

obtains a different distribution for the same observ-able (Fig. 14b).If there are uncertainties on the various parametersthe original point of the parameter space will bebroadened and one gets a diffrent distribution of thesame observable for each of the points inside of theuncertainty area (Fig. 14c).To scan a zone inside of the parameters space oneshould generate a Monte Carlo simulation for eachcombination, this it’s clearly unfeasible; NUISANCEis basically doing the opposite, finding the best fit forthe observable using a reweighing of the Monte Carlo(see Section 3.1). Once the best fit has been obtained(for one or more observables) it’s possible to extrap-olate the configuration of the parameters that betterdescribes the data (Fig. 14d).

5.1 Finding the dials to tune

Due to the big amount of parameters, a first test hasbeen done; for each of the GENIE parameter has beenplotted the variations of the simulations for valuesfrom −3σ to +3σ with a step of one sigma.This has been done for the following dials:

• MaCCRES: Axial mass for the CC Resonantpion

• MvCCRES: Vector mass for CC Resonant pion.

• NormCCRES: Normalization for CC Resonantpion.

• RvnCC1pi, RvnCC2pi, RvpCC1pi, RvpCC2pi:Respectively, Non resonant background for ν +n CC1π ,Non resonant background for ν + nCC2π,Non resonant background for ν+p CC1π,Non resonant background for ν + p CC2π

And for the following FSI (Final State Interaction)dials:

• FormZone: Formation Zone

• MFP pi, FrCEx pi, FrElas pi, FrInel pi,FrAbs pi, FrPiProd pi: Respectively, Mean FreePath for the pion, pion charge exchange, pionelastic reaction, pion inelastic reaction, pionabsorption, pion π-production

• MFP N, FrCEx N, FrElas N, FrInel N,FrAbs N, FrPiProd N: As for the previouspoint but for Neutrons

Due to the impossibility of reporting here all the plotsfor all the previous listed dials only two meaning-ful examples are shown here in Fig. 15 where it’spossible to distinguish between a dial with a big ef-fect (the MaCCRES) and a dial with a smaller effect(MFP N).

(MeV)πT50 100 150 200 250 300 350

/MeV

/nuc

leon

)2

(cm

π/d

Tσd

0

2

4

6

8

10

12

14

16

18

20

2242−10×

MaCCRES_-1.000000nominalDataMaCCRES_-2.000000MaCCRES_-3.000000MaCCRES_1.000000MaCCRES_2.000000MaCCRES_3.000000

MaCCRES

(a) The kinetic energy of the pion varying the Axialmass for the CC Resonant production

(MeV)πT50 100 150 200 250 300 350

/MeV

/nuc

leon

)2

(cm

π/d

Tσd

0

2

4

6

8

10

12

14

1642−10×

MFP_N_-1.000000nominalDataMFP_N_-2.000000MFP_N_-3.000000MFP_N_1.000000MFP_N_2.000000MFP_N_3.000000

MINERvA_CC1pip_XSec_1DTpi_nu_fluxcorr

MFP_N

(b) The kinetic energy of the pion varying the mean freepath of the neutron

Figure 15: Comparison between MaCCRES andMFP N, as it’s possible to see that the effects of thevariations on MaCCRES are bigger. The variations

are in units of sigma.

After checking all the previous dials the onesthat have the biggest impact on data are :MaCCRES, MinervaRW, RvnCC1pi, RvpCC1pi,RvbarnCC1pi (as RvnCC1pi but for the antineu-trino), RvbarpCC1pi (as RvpCC1pi but for the an-tineutrino), FrCEX pi, FrInel pi, FrAbs pi and Fr-PiProd pi.

5.2 The bayesian method

To have a better idea about the parameter’s spaceanother method can be used; the Bayesian method.The latter is working in the following way:

• Throw a random value for each parameters fol-lowing a probability distribution that can be flat

6

or gaussian

• For the random point in the parameter’s spacereweigh the Monte Carlo simulation and calcu-late the χ2 value

• Assign a weigh to the throw as w = e−χ2

This method can be considered a brute force way tofind the best fit for the parameters, instead of search-ing the minimum for a set of observables (as Minuitis doing) is scanning the space and calculating the χ2

for each point.Reweighing the various points with e−χ

2

for eachobservable it’s possible to have a better estimationabout which value of a parameter that observableprefers and an idea about the correlation betweentwo different dials.Using NUISANCE is possible to throw random pointin the parameter space following two distributions:

• Gaussian distribution: Assuming that the bestvalue for the dial should be near the one used asdefault (the center of the gaussian).

• Flat distribution: Not assuming any a-prioriknowledge about the dial value.

)σMaCCRES (3− 2− 1− 0 1 2 3

Arb

Uni

ts

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35posterior -0.56 +- 0.32prior 0.07 +- 1.71posterior2 -0.62 +- 0.35prior2 0.01 +- 1.0

MaCCRES throws

Figure 16: Differences between flat (light blue line)and Gaussian throws (pink line) without weights

(dashed lines) and after applying weights(respectively blue and red lines)

When using a flat distribution it’s possible to add apenalty term to avoid values that are going outsideof the physical region as:

w = e−χ2−v2 (2)

Where v is the variation in sigma units of the dial un-der examination; using this values that are far awayfrom the 0σ value are discouraged (as it’s happeningusing a Gaussian distribution for the random throws).

5.3 Scan of the parameters space

Using the flat throws has been possible to scan the pa-rameter space, a first test has been done evaluatingthe correlation between the MinervaRW and MaC-CRES dials as can be seen from Fig. 17 :

MINERvARW_NormCCRES3− 2− 1− 0 1 2 3

MaC

CR

ES

3−

2−

1−

0

1

2

3

posteriorEntries 72034Mean x 0.4089− Mean y 0.05979− RMS x 0.07737RMS y 0.5222

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007


posterior -0.41 +- 0.08

(a) The 2D plot of the space projection on MinervaRWand MaCCRES parameter space

MINERvARW_NormCCRES1− 0.9− 0.8− 0.7− 0.6− 0.5− 0.4− 0.3− 0.2− 0.1− 0

MaC

CR

ES

1−

0.5−

0

0.5

1


0

0.001

0.002

0.003

0.004

0.005

0.006posterior

Entries 72034Mean x 0.4081− Mean y 0.06266− RMS x 0.07685RMS y 0.4785

posterior -0.41 +- 0.08

(b) Zoom on the Gaussian peak created by the two dials

Figure 17: A flat scan of the parameter scan; afterprojecting on MaCCRES and MinervaRW and

applying the weights it’s possible to see that the twodials are not correlated

Adding more dials will increase the dimension of thedial’s space, because of that the projection on the 2Dspace created by the combination of two of them canbe done only when one has enough statistics (throwsin the n dimensional space); nevertheless even with amodest number of statistics it’s possible to extrapo-late some informations about it.As can be seen from Fig. 18 odd situations can hap-pen; in Fig. 18a it’s possible to see the uncertaintiesover MaCCRES dial are bigger than the ones overNormCCRES. From Fig. 18b is possible to see a niceexample of where Minuit minimization can fail, inthis case the zone with the maximum likelihood issplitting in two branches and can happen that onlyone of the two will be detected by Minuit.

7

)σMaCCRES (

3− 2− 1− 0 1 2 31−

0.5−

0

0.5

1

1.5

2Post Enu 2D


0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Post Enu 2DEntries 14700Mean x 0.9594− Mean y 0.04482− RMS x 1.677RMS y 0.5006

MINERvA_CC1pip_XSec_1DEnu_nu_2017_likelihood 2D

)σMaCCRES (

3− 2− 1− 0 1 2 3

)σN

orm

CC

RE

S(

1−

0.5−

0

0.5

1

1.5

2Post Q2 2D


0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

Post Q2 2DEntries 14700Mean x 0.6863− Mean y 0.3389− RMS x 0.6745RMS y 0.08768

MINERvA_CC1pip_XSec_1DQ2_nu_2017_likelihood 2D

)σMaCCRES (

3− 2− 1− 0 1 2 31−

0.5−

0

0.5

1

1.5

2Post Tpi 2D

Entries 14700Mean x 2.131− Mean y 0.08617RMS x 0.6348RMS y 0.3141

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

3−10×Post Tpi 2D


MINERvA_CC1pip_XSec_1DTpi_nu_2017_likelihood 2D

)σMaCCRES (

3− 2− 1− 0 1 2 3

)σN

orm

CC

RE

S(

1−

0.5−

0

0.5

1

1.5

2Post th 2D


0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

12−10×Post th 2D


MINERvA_CC1pip_XSec_1Dth_nu_2017_likelihood 2D

3− 2− 1− 0 1 2 31−

0.5−

0

0.5

1

1.5

2Post thmu 2D


0

0.5

1

1.5

2

2.5

3

3.5

4

6−10×Post thmu 2D


MINERvA_CC1pip_XSec_1Dthmu_nu_2017_likelihood 2D

MaCCRES (σ)

Nor

mC

CR

ES (σ

)

(a) Example of the projection for two uncorrelated dials(MaCCRES and NormCCRES).)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post Enu 2D


0

0.02

0.04

0.06

0.08

0.1

Post Enu 2DEntries 14700Mean x 0.5082− Mean y 1.692RMS x 0.351RMS y 1.518

MINERvA_CC1pi0_XSec_1DEnu_antinu_likelihood 2D

)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post Q2 2D


0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02Post Q2 2D


MINERvA_CC1pi0_XSec_1DQ2_antinu_likelihood 2D

)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post Tpi 2D


0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

Post Tpi 2DEntries 14700Mean x 0.4695− Mean y 0.745RMS x 0.2863RMS y 1.089

MINERvA_CC1pi0_XSec_1DTpi0_antinu_likelihood 2D

)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post th 2D


0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045Post th 2D


MINERvA_CC1pi0_XSec_1Dth_antinu_2016_likelihood 2D

)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post thmu 2D


0

0.002

0.004

0.006

0.008

0.01

0.012

Post thmu 2DEntries 14700Mean x 0.4242− Mean y 1.992RMS x 0.1386RMS y 1.041

MINERvA_CC1pi0_XSec_1Dthmu_antinu_likelihood 2D

)σNormCCRES (

1− 0.5− 0 0.5 1 1.5 2

)σN

onR

ES

(

2−

1−

0

1

2

3

4Post pmu 2D


0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

Post pmu 2DEntries 14700Mean x 0.4451− Mean y 1.676RMS x 0.3906RMS y 1.603

MINERvA_CC1pi0_XSec_1Dpmu_antinu 2D

(b) Example of a ”fork” in the parameter space.

Figure 18: A flat scan of the parameter space; afterprojecting on MaCCRES and MinervaRW and

applying the weights it’s possible to see that the twodials are not correlated

6 Obtaining the best fit usingthe Bayesian method

Due to the enormous dimension of the space, a firstvalidation of the Minuit fits has been done taking onlythree dials (MaCCRES, FrAbs pi and NonRES).These fits have been done on the following observ-ables:

• Enu : Energy of neutrino

• Q2 : Q2 of the interaction

• Tpi : Kinetic energy of the pion

• th : Angle theta

• thmu : Theta angle of the muon

The following plots show the distribution of thethrows after rescaling using χ2 as a weight, also foreach observable is plotted the best throw (dashedline); one should expect that, if the number of throwsis sufficient, the best throw should be located wherethe peak of the weighted distribution is.

6.1 MaCCRES

As is possible to see from Fig. 19, besides the Tpi dis-tribution, all others are compatible within one sigma.

)σMaCCRES (3− 2− 1− 0 1 2 3

Arb

Uni

ts

0

0.02

0.04

0.06

0.08

0.1

posterior Enu -0.15 +- 0.73

posterior Enu -0.15 +- 0.73posterior Q2 -0.37 +- 0.5posterior Tpi -0.9 +- 0.61posterior th -0.31 +- 0.7posterior thmu -0.26 +- 0.59

Enu :0.022χBest Q2 :-0.352χBest Tpi :-0.892χBest th :-0.232χBest thmu :-0.372χBest

Figure 19: MaCCRES dial best throws andreweighed distribution

6.2 NormCCRES

)σNormCCRES (1− 0.5− 0 0.5 1 1.5 2

Arb

Uni

ts

0

0.1

0.2

0.3

0.4

0.5

0.6

posterior Enu -0.08 +- 0.43posterior Q2 -0.34 +- 0.09posterior Tpi 0.05 +- 0.28posterior th -0.78 +- 0.14posterior thmu -0.48 +- 0.09

Enu :0.312χBest Q2 :-0.332χBest Tpi :0.142χBest th :-0.652χBest thmu :-0.492χBest


Figure 20: NormCCRES dial best throws andreweighed distribution

For the NormCCRES dial the distributions arecompatible in groups, the th, thmu and Q2 prefer asmaller value whilst Tpi and Enu prefer a positivevalue slightly different than the default one.

6.3 NonRES

)σNonRES (2− 1− 0 1 2 3 4

Arb

Uni

ts

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2posterior Enu -0.09 +- 0.69posterior Q2 0.2 +- 0.64posterior Tpi -0.03 +- 0.68posterior th -0.26 +- 0.6posterior thmu 0.32 +- 0.65

Enu :-0.122χBest Q2 :0.282χBest Tpi :-0.052χBest th :-0.192χBest thmu :0.352χBest


Figure 21: NonRES dial best throws and reweigheddistribution

All distributions in the NonRES case arecompatible within one sigma.

8

Dataset MaCCRES NormCCRES NonRESQ2 -0.43 +- 0.18 0.22 +- 0.05 -1.10 +- 0.08Tpi -0.86 +- 0.22 0.15 +- 0.07 -1.07 +- 0.08th -0.94 +- 0.07 0.10 +- 0.03 -1.06 +- 0.07thmu -0.29 +- 0.19 -0.02 +- 0.06 -1.11 +- 0.07

Dataset MaCCRES NormCCRES NonRESQ2 -0.37 +- 0.5 0.2 +- 0.64 -0.34 +- 0.09Tpi -0.9 +- 0.61 -0.03 +- 0.68 0.05 +- 0.28th -0.31 +- 0.7 -0.26 +- 0.6 -0.78 +- 0.14thmu -0.26 +- 0.59 0.32 +- 0.65 -0.48 +- 0.09

7 Conclusions

It can be meaningful to compare the values obtainedwith the Bayesian method with the ones obtainedthrough Minuit.From Minuit: From bayesian method: Whilst somevalues are compatible between the two method someother (in particular for the NonRES dial) are not,this can be due to the fact that the parameter spacehasn’t been properly scanned.Increasing the number of throws for the Bayesianmethod should reduce the statistic uncertainties andconverge to the results given by Minuit.In any case, taking a space of dimension d and re-questing at least 100 throws for each dimension (thatcan be considered the minimum) one should do, in to-tal, 100d throws. In the previous example the numberof throws has been 225000 for the 3 parameters space,this is obviously quite less than 1003; these ”low”statistics are mainly due to computational time.Can be interesting, however, to analyse the behaviourof the scan to understand if effectively one shouldthrow ∼ 100d times before having a proper scan ofthe space; to do so a naive measure can be defined asfollowing:

m =(X̄ −B)2

X̄2(3)

where :

• X̄ : is the mean of the throws distribution afterthe reweighting

• B : is the best throw obtained

We should expect that the χ2 (m) between themshould converge to 0 while increasing the number ofthrows. From Fig. 22 is clear that while nearly all dis-tributions converge at ∼ 100000 throws, th, Enu and

Figure 22: Evolution of the m value for MaCCRES,NormCCRES and NonRES parameters space

thmu do not. These are also the observables with thebiggest differences between the Minuit method andthe Bayesian one; this should suggest that going upwith the number of throws the two methods shouldconverge to the same value. However this puts a clearevidence on the Bayesian method, the latter has beendone for a parameter space of dimension 3, having todo the same for the whole space (more than 7 dimen-sional) it would be unfeasible requiring an enormousamount of computational time.

9

References

[1] C. Andreopoulos et al. The GENIE NeutrinoMonte Carlo Generator. Nucl. Instrum. Meth.,A614:87–104, 2010.

[2] F. I. Olness M. A. G. Aivazis and W.-K. Tung.Leptoproduction of heavy quarks. 1. General for-maism and kinematics of charged current andneutral current production processes. Phys. Rev.,1994.

[3] C. Wilkinson L. Pickering et. al. P. Stowell,C. Wret. Nuisance: a neutrino cross-section gen-erator tuning and comparison framework. https://arxiv.org/abs/1612.07393.

[4] D. Rein and L. M. Sehgal. Neutrino excitationof baryon resonances and single pion production.Ann. Phys., page vol. 133 p. 79.

[5] C. H. Llewellun Smith. Neutrino reactions at ac-celerator energies. Phys. Rept., page vol. 3 p. 261,1972.

10

https://arxiv.org/abs/1612.07393

https://arxiv.org/abs/1612.07393

Documents

Internship report - eddata.fnal.goveddata.fnal.gov/lasso/summerstudents/papers/2017/Matteo-Mazzanti-paper.pdf · Internship report Matteo Mazzanti October 29, 2017 Abstract This document