Sept 2003 PHYSTAT 1

Our treatment of systematic errors

Sept 2003 PHYSTAT 2

What is a systematic error?

“This is why people are so frightened of systematic errors, and most other textbooks avoid the subject altogether. You never know whether you have got them and can never be sure that you have not – like an insidious disease…

The good news, however, is that despite popular prejudices and superstitions, once you know what your systematic errors are, they can be handled with standard statistical methods.”

R. J. BarlowStatistics

Sept 2003 PHYSTAT 3

Imagine that two experimental groups have measured a quantity , with the results shown.

OK, what is the value of ?

This is very analogous to what happens in global analysis of PDF’s. But in the case of PDF’s the systematic differences are only visible through the PDF’s.

Sept 2003 PHYSTAT 4

We use 2 minimization with fitting of systematic errors.

Minimize 2 w. r. t. {a} optimal parameter values {a0}.

All this would be based on the assumption that

Di = Ti(a0) + i ri

drre

dP 2

2/2

For statistical errors define

error lstatistica :

valueal theoretic:

valuedata :

1

2

22

i

i

iN

i i

ii T

DTD

Ti = Ti(a1, a2, ..,, ad) a function of d theory parameters

(S. D.)

Sept 2003 PHYSTAT 5

Treatment of the normalization error

N

i i

ii TDfffa

12

2N

2

norm

NN

2 1),(

In scattering experiments there is an overall normalization uncertainty from uncertainty of the luminosity. We define

where fN = overall normalization factor

Minimize 2 w. r. t. both {a} and fN.

Sept 2003 PHYSTAT 6

A method for general systematic errors

j

K

jijiiii rraTD ˆ)(

10

K

jj

N

i i

j ijijis

TsD

1

2

12

2

2

Minimize 2 with respect to both shape parameters {a} and optimized systematic shifts {sj}.

quadratic penalty term

i : statistical error of Di

ij : set of systematic errors (j=1…K) of Di

Define

Sept 2003 PHYSTAT 7

K x K matrix K x 1 matrix

m

K

mjmjj

j

BAasss

1

1)0(2

)( implies 0

N

i i

ijiij

N

i i

ikijjkjk

TDBA

12

12

and

where

Because 2 depends quadratically on {sj} we can solve for the systematic shifts analytically

Sept 2003 PHYSTAT 8

Now let

sexperiment

022

global )(, )( asaa

and minimize w.r.t {a}.

The systematic shifts {sj} are continually optimized[ s s0(a) ]

Sept 2003 PHYSTAT 9

So, we have accounted for …• Statistical errors• Overall normalization uncertainty (by fitting {fN,e})• Other systematic errors (analytically)

We may make further refinements of the fit with weighting factors

Default : we and wN,e = 1

The spirit of global analysis is compromise – the PDF’s should fit all data sets satisfactorily.If the default leaves some experiments unsatisfied, we may be willing to reduce the quality of fit to some experiments in order to fit better another experiment. (We use this sparingly!)

e e

ee

eee

fwfawfa

2N,

2

,NN2

N2global

)1(}){},({}){},({

Sept 2003 PHYSTAT 10

How well does this fitting procedure work?

Quality

Sept 2003 PHYSTAT 11

Comparison of the CTEQ6M fit to the H1 data in separate x bins. The data points include optimized shifts for systematic errors. The error bars are statistical only.

Sept 2003 PHYSTAT 12

Comparison of the CTEQ6M fit to the ZEUS data in separate x bins. The data points include optimized shifts for systematic errors. The error bars are statistical only.

Sept 2003 PHYSTAT 13

Comparison of the CTEQ6M fit to the BCDMS and NMC data on p scattering. The data points include optimized shifts for systematic errors. The error bars are statistical only.

Sept 2003 PHYSTAT 14

Comparison of the CTEQ6M fit to the inclusive jet data. (a) D0 cross section versus pT for 5 rapidity bins; (b) CDF cross section for central rapidity.

Sept 2003 PHYSTAT 15

Closer comparison between CTEQ6M and the D0 jet data as fractional differences.

Sept 2003 PHYSTAT 16

How large are the optimized normalization factors?

Expt f N

BCDMS 0.976H1 (a) 1.010H1 (b) 0.988ZEUS 0.997NMC 1.011CCFR 1.020E605 0.950D0 0.974CDF 1.004

Sept 2003 PHYSTAT 17

We must always check that the systematic shifts are not unreasonably large.

j sj1 1.672 -0.673 -1.254 -0.445 0.006 -1.077 1.288 0.629 -0.40

10 0.21

j sj1 0.672 -0.813 -0.354 0.255 0.056 0.707 -0.318 1.059 0.61

10 0.2611 0.22

10 systematic shifts NMC data

11 systematic shifts ZEUS data

Sept 2003 PHYSTAT 18

Comparison to NMC F2iiiii TD /)( βs

without systematic shifts

Sept 2003 PHYSTAT 19

Comparison to ZEUS F2

iiiii TD /)( βs

without systematic shifts