Precision and AccuracyAgreement Indices in HSP

An Introduction to Rietveld Refinementusing

PANalytical X’Pert HighScore Plus v2.2d

Scott A Speakman, Ph.D.MIT Center for Materials Science and Engineering

speakman@mit.edu

There are two different questions:

How good is my fit?

How good is my answer?

One question is easier to answer in HSPOne question is more important ...

What are the chances these are the same questions?

How Good is My Fit?

• HSP gives you many numerical indicators of how good your Rietveld model has been refined– Rp: residual of least-squares refinement– wRp: weighted residual– GOF: goodness of fit

• During refinement, you see a plot of Rp as it improves (hopefully) during the refinement

• Numerically, you can consult Rp, wRp, and GOF indices to judge how good your fit is

Residuals

• Rp quantifies the difference between the observed and calculated data points on a point-by-point basis

• Rwp weights the residual so that higher intensity data points are more important than low intensity data points– the rationale for using Rwp: fitting the diffraction peaks is more

important than fitting the background– Rwp is unfavorable in some situations where the important

information is contained in the weakest peaks• distortions in perovskites• discerning between ordered and/or disordered system

• In general, you want Rp or Rwp to be less than 10%

io

icioP y

yyR

2/1

2

2

ioi

icioiwp yw

yywR

Accounting for Data Quality

• Rp and Rwp simply compare the calculated pattern to the data– you will never have a perfect fit because all data contains noise – HSP does not model noise

• Rexp evaluates the quality of the data– data with more noise or low peak intensities will have a larger Rexp

– theoretically, your Rwp can never be better than the Rexp

• GOF compares Rwp to Rexp

– a GOF=1 means the model is as good as possible– want GOF less than 4– two ways to improve GOF

• better model or noisier data• be careful your GOF isn’t good because your data is bad

2/1

2exp

ioi yw

PNR

2

exp

2

R

RGOF wp

How to find the agreement indices

• After refinement, the agreement indices are shown in the bottom status bar– this information disappears as soon as the mouse

shifts focus to something else• ie as soon as the mouse floats over something else• this makes this information very elusive- it disappears quickly

To Hunt Down the Agreement Indices

• Indices can be found in the Object Inspector pane for Global Parameters

• You can find help references in 9.Algorithms > +Rietveld Algorithm

Individual Phase Residuals

• Rp, Rwp, and GOF evaluate how well the entire Rietveld model fits the entire data set

• RBragg attempts to evaluate how well individual phases are fit– allows you to discern if

phase 1 is well fit and phase 2 is poorly fit

– Find RBragg in the Object Inspector for each individual phase

Other ways to evaluate your fit

• Visual Estimation

• Difference Pattern

• Equations can be fooled, so you must always use your eyes

Position [°2Theta] (Copper (Cu))

30 40 50 60 70 80 90

Counts

0

2500

10000

NIST640bSilicon 100.0 %

0

10

-10

20

-20

What do the agreement indices really tell us?

• Agreement indices evaluate how well the calculated pattern produced by the Rietveld model fits the experimental data– does not account for poor or incorrect data– the model may be wrong

• The only true evaluation of your results- does this make sense?– bond distances and angles– difference Fourier maps– what else is known about the sample

• complementary data

If the refinement produced significant changes in lattice parameter or atom positions, then check the new bond

distances and angles for reasonableness

• go to Distances and Angles list– Select the Phase from the drop-down

menu– Click Calculate

• get options by clicking on … button

Evaluating Precision- the Estimated Standard Deviation (ESD)

• The ESD is shown in the Deviation Column in the Refinement Control list OR in the Object Inspector for individual parameters

• ESD quantifies the amount that a parameter could change without changing the Rwp of the refinement– the amount of wobble or wiggle in that

parameter– the smallest the error in that

parameter could be if everything else is perfect

ESD is horribly misused in published literature

• ESD tells you how precisely the parameter is coupled to your refined data– a small ESD indicates that changes in the parameter will have a large

effect on the fit of the calculated pattern to your experimental data• we have a precise fit• we do not necessarily have a precise or accurate answer!• example: we may not have allowed another variable to refine appropriately

– a large ESD indicates that changes in the parameter does not significantly effect the goodness of the fitfit

• why?– our measurement is not sensitive to that parameter

» Example: O atoms refined alongside Pb atoms– our data is not good enough to refine that parameter– we have too many correlating variables

• ESD is not a standard deviation nor does it give you error bars• True standard deviation- refine data sets from several different

samples

Testing the Solution

• The numbers that HSP give you quantify how well the calculated pattern fits the experimental data, nothing more

• To test the solution– Probe for false minima

• false minima in finding the lowest possible residual• even the true minima might not be the correct answer• Change a parameter, then refine again

– do you achieve the same answer?– genetic fitting algorithms use this methodology to better avoid false

minima– may be added in HSP soon?

– Refine a second data set from the same sample– Refine a data set from a different sample of the same population

What is the Real Answer?

• Question the First: is the solution realistic, reasonable, and reproducible?

• Question the Second: does the answer agree with other complementary data?

• Question the Third: how much accuracy do I really need?– how much time and resources am I willing to invest to get

accuracy?

• The key for QA and routine analyses- consistency

Testing your procedures- particularly for QPA

• for QPA analysis, in particular, you can make sure that your procedure is valid by testing standard mixtures– it is always easier to get the right solution when you already

know the answer!

• make standard mixtures of the phases found in your sample– use the exact same procedure for preparing the sample,

collecting the data, and refining the model– test mixtures with a range of concentrations to evaluate the

ability to quantify a phase when it is present as the majority phase and when it is present as a trace phase

Spiking to evaluate QPA

• if you don’t have single phase analogues for all of your materials, then use one or two of your actual samples – do the QPA with original sample– add a known amount of a single phase specimen that is analogous to

one of the ingredients in your sample– repeat the QPA– add more of the standard specimen to your sample and repeat the QPA

again

• by making sure that the calculated amounts of the standard phase tracks properly with your standard additions, you can gain confidence in your refinement procedure

• by using several different levels of standard additions, you can avoid errors from amorphous content (expected or unexpected)