Embed Size (px)
Citation preview
June 28, 2007 1
Trend Data
Lynn TorbeckTorbeck and Assoc.Evanston, IL
June 28, 2007 2
Overview
OOT vs. OOSWhy trend?How to get startedTypes of trends with examplesOOT is relativeGraphical toolsTend limits
June 28, 2007 3
Why Trend Data?
Good business practice.Early warning of possible Out Of Specification (OOS) results.Gain process understanding.Minimize risk of potential failures of product in the market.Find the “gold in the hills” for process improvements.
June 28, 2007 4
Regulatory Basis for Trending
No specific regulation requirement211.180(e) Annual ReviewsFDA Form 483 for observationsEstablishment Inspection ReportsWarning lettersFDA presentations at conferences
June 28, 2007 5
OOS Guidance Footnote
“Although the subject of this document is OOS results, much of the guidance may be useful for examining results that are out of trend (OOT).”
How is OOT different than OOS?How is OOT the same as OOS?
June 28, 2007 6
Out Of Specification - OOS
OOS is the comparison of one result versus a predetermined specification criteria.OOS investigations focus on determining the truth about that one value.Is the OOS result confirmed or not?
June 28, 2007 7
Out Of Trend - OOT
OOT is the comparison of many historical data values versus time.OOT investigations focus on understanding non-random changes.Is the non-random change confirmed or not?
June 28, 2007 8
OOS Guidance
Taking into account the differences between OOS and OOT, the guidance does provide a framework for OOT investigations:ResponsibilitiesPhilosophical basisGeneral principles of investigations
June 28, 2007 9
1. How to get started
Select the variable to be studied: Potency Yield Impurities Hardness Bioburden
June 28, 2007 10
2. How to get started
Select a time period:At least one year if possible.More than two preferred.Do not go past a major change in the process. Use process knowledge to advantage.Use the reportable result, the value compared to the specifications.
June 28, 2007 11
3. How to get started
Enter the data into analysis software: Excel Minitab Sigma Plot JMP StatGraphics Northwest Analytical SAS
June 28, 2007 12
4. How to get started
Plot the data vs. time or lot sequence.Look for non-random changes over time.Determine if they are of practical importance.Statistical significance is insufficient.Do an impact and risk assessment.
June 28, 2007 13
What is Trending?
The several activities of: Collecting data, Recording it, Documenting it, Storing it, Monitoring it, Fitting models to it Evaluating it, and Reporting it.
June 28, 2007 14
What is a trend?
Any non-random pattern.
Short and long term patterns in data over time that are of practical importance.
June 28, 2007 15
Beneficial Trends
Desirable patterns in the data series.Examples: A move toward the target or center of
the specification. More consistent with less variation. Less likelihood of an OOS value. A benefit to SSQuIP.
June 28, 2007 16
Beneficial Trend
0
0.2
0.4
0.6
0.8
1
1.2
12/10/2002
6/28/2003 1/14/2004 8/1/2004 2/17/2005 9/5/2005 3/24/2006 10/10/2006
Date
mg/
mL
June 28, 2007 17
No Trend
Easier to define what a trend is not.Random dataNoiseStationary No ups, no downsNo cyclesNo outliers
500400300200100
104
103
102
101
100
99
98
97
Index
No
rma
l
June 28, 2007 18
Neutral or No Trend
Neither beneficial or adverseExamples: Results that are always the same. Stability data with a slope of zero. Data in a state of “statistical control”
on a control chart.
June 28, 2007 19
Process Control
Statistical Process Control, SPC Normal random data over time Due to common causes only
Engineering Process Control, EPC Estimate departures from target Feedback to control point Physical changes to the process
June 28, 2007 20
Adverse Trends
Undesirable patterns in the data series.Examples: A movement away from the target. Increased variability. Increased probability of OOS. An unexplained change to a beneficial
trend. A challenge to SSQuIP.
June 28, 2007 21
Out-of-Trend (OOT)
A change from an established pattern that has the potential of an adverse effect on SSQuIP or of becoming OOS.Must be large enough to be of practical significance.Statistical significance is insufficient to determine OOT.
June 28, 2007 22
Long Term Change
Not stationary around a fixed valueIncreasing or decreasing average.Apparently will continue to get worse (or better) unless action is taken.
10 20 30 40 50 60 70 80 90 100
98
99
100
101
102
103
104
105
106
107
IndexY
ield
s
Increasing Trend
.1 per step after 50
June 28, 2007 23
The Aberrant Outlier
Stationary and random but with one very large value that could be a statistical outlier.Generally assumed to be due to a “special cause.”
100908070605040302010
105
104
103
102
101
100
99
98
97
IndexY
ield
%
An outlier
Mu=100, Sigma=1.0
June 28, 2007 24
Shift in the Average
Here the mean has increased from 100 to 104 at sample 51.No other changes were made.Variability is the same. 100908070605040302010
106
101
96
IndexY
ield
Mean Shift
Mu=100 to 104 Sigma=1.0
June 28, 2007 25
Variation Change
This is stationary around a fixed mean of 100%.But, the standard deviation increased from 1.0 to 4.0.
10 20 30 40 50 60 70 80 90 100
90
100
110
IndexY
ield
%
Increasing Variability
Mu=100, Sigma=1.0, 2.0, 3.0 & 4.0
June 28, 2007 26
Cycles
A reoccurring cycle.Stationary about a fixed mean.The data are not independent.
10 20 30 40 50 60 70 80 90 100
96
97
98
99
100
101
102
103
104
IndexC
ycle
s
Cycles
June 28, 2007 27
Autocorrelated
Data are correlated with the previous data.Not stationary.Check different time lags, 1,2, ….
100908070605040302010
105
104
103
102
101
100
99
98
IndexA
uto
Co
rr
Autocorrelated
June 28, 2007 28
OOT is Relative
10 20 30 40 50 60 70 80 90 100
90
100
110
Index
Yie
ld %
Stationary White Noise
mu=100%, S=1%
June 28, 2007 29
OOT is Relative
The importance of a trend is its size relative to the specification criteria.A state of Statistical Control is desired but not necessary.A state of Engineering Control is necessary to meet specifications.Success is a marriage of the two.
June 28, 2007 30
A Little Humor (Very Little)
Lottery: A tax on the statistically-challenged.If you want three opinions, just ask two statisticians.Statistics means never having to say you're certain.http://www.keypress.com/x2815.xml
June 28, 2007 31
Trend Fitting
“The general process of representing the trend component of a time series.”A Dictionary of Statistical Terms. Marriott
Depends very much on the type of data and the subject matter being studied.Need to adapt the tools and techniques to our specific data and issues.
June 28, 2007 32
Tools of Trending
Summary statistics Averages, Medians Ranges, Standard Deviations, %RSD
Graphical plotsDistribution analysis - HistogramsOutlier determinationRegression analysis
June 28, 2007 33
Graphic Tools
Line Plots vs. time.Shewhart Control Charts.Histograms.Sector chart
June 28, 2007 34
Line Plots vs. Time
Response on the vertical axis.Time or batch # on the horizontal axis.Usually connect the data points with a line, but optional.
10 20 30 40 50 60 70 80 90 100
98
99
100
101
102
IndexY
ield
%
Stationary Time Series
Mu=100, Sigma=1.0
June 28, 2007 35
Control Chart
Add ‘natural process limits’ to the line plot. ± 3 A chart for the response.A chart for the variability.
0Subgroup 50 100
96.5
97.5
98.5
99.5
100.5
101.5
102.5
103.5
Ind
ivid
ual
Val
ue
Mean=100
UCL=103
LCL=97
0
1
2
3
4
Mov
ing
Ran
ge
R=1.128
UCL=3.686
LCL=0
I and MR Chart for Yield %
June 28, 2007 36
Control Chart Family
IndividualsAveragesMediansStandard deviationsRangesNumber of defectivesFraction defectivesDefects per unitsNumber of defects
June 28, 2007 37
Variation Change
A control chart will detect change in the variation.
0Subgroup 50 100
90
100
110
Ind
ivid
ual
Val
ue
1 1
11 1 1
1
11
1
1 11 11
1
1
1
1
11
1
1 1
1
Mean=100
UCL=103
LCL=97
0
5
10
Mov
ing
Ran
ge
111
1 111 11
11
11
11
1
11
11
1
1
11
1
1
11 1
1
1
111
1
R=1.128
UCL=3.686
LCL=0
I and MR Chart for Yield %
June 28, 2007 38
The Outlier
A control chart finds values outside the natural limits of the data.The value is larger than would be expected by chance alone.
0 50 100
96
97
98
99
100
101
102
103
104
105
106
Observation Number
Ind
ivid
ual
Val
ue
I Chart for Yield%
1
Mean=100
UCL=103
LCL=97
June 28, 2007 39
“Western Electric” Rules
1. One value outside 3 S limits.2. Nine values in a row on one side
of the average.3. Six values in a row all increasing
or decreasing.4. 14 values in a row alternating up
and down.
June 28, 2007 40
“Western Electric” Rules
5. Two of three values greater than 2 S from the average.
6. Four of five values greater than 1 S from the average.
7. 15 values in a row within 1 S of the average.
8. Eight values in a row greater than 1 S.
June 28, 2007 41
Histogram
Show the ‘shape’ of the distribution of data.In this case it is Normally distributed.
96 97 98 99 100 101 102 103 104
0
10
20
Yield %
Fre
qu
ency
June 28, 2007 42
The Outlier
The outlier is clearly seen in the histogram.
97 98 99 100 101 102 103 104 105 106
0
10
20
Yield%
Fre
qu
ency
Variation Change
June 28, 2007 43
Outlier Determination
Reference: USP 30 NF 25 Chapter <1010> “Analytical Data – Interpretation and
Treatment” Page 392 “Outlying Results” Appendix C: Examples of Outlier Tests
for Analytical Data.
June 28, 2007 44
Regression Analysis99% Prediction Interval
100000
120000
140000
160000
180000
200000
220000
0 5 10 15 20
Months
Va
lue
June 28, 2007 45
Trend Limits
Numeric (or non-numeric) criteria, that if exceeded, indicates that an out-of-trend change has occurred.Usually the ‘natural process’ variationAKA “Alert limits”Use Statistical Tolerance LimitsSee USP <1010> Appendix E
June 28, 2007 46
Here, Trend This
100 200 300
20
30
40
Index
Var
1
June 28, 2007 47
A New Engineering Chart
Brings together for the first time: Comparison to the specification limits in
place of the probability limits Divides the specification range into
equal zones in place of 1, 2, & 3 sigma areas
Uses cumulative scores
Pharmaceutical Technology, April 2005
June 28, 2007 48
The New “Sector Chart”SIALIC ACID EXAMPLE
Fail3.915 3.695 3.298 4.04 3.87 4.147 3.938 4.167 3.9 3.927 3.81 3.9 4.033 3.853 4.142 3.958 3.77
Sector Weight Low HighF 10D 2 4.1 4.2 2 2 2C 1 4 4.099 1 1B 0 3.9 3.999 0 0 0 0 0 0A 0 3.8 3.899 0 0 0A 0 3.7 3.799 0B 0 3.6 3.699 0C 1 3.5 3.599D 2 3.4 3.499F 10 10
Batch 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
June 28, 2007 49
The New “Sector Chart” Rules
The first batch tally takes the weight of the sector it is in.Subsequent batches have a cumulative tally of the previous tally plus the current sector weight.If the tally reaches a value of, say, 10, an alert is given.If the batch enters the A or B sectors, the tally is reset to zero.
June 28, 2007 50
The New “Sector Chart” Rules
Sectors A and B cover the center 50% of the specification range.Sector F is outside the current specification.Other weights can be set to fit the process and the degree of sensitivity needed.
June 28, 2007 51
Advantages of Sector Chart
No minimum sample size. Can start with one data point.No assumptions about the data at all.Identifies beneficial and adverse trends.Weights and tally total are selected by scientific and empirical knowledge.A decision is made with each new point.Alerts quickly if a problem exists.
June 28, 2007 52
Justification for Sector Chart
If the process is well inside the specification, it need not be in a state of statistical control.The focus is on OOT and SSQuIP not being out of “statistical” control.Sensitivity of the chart is adjustable.Can be use in parallel with other charts.
June 28, 2007 53
That’s All Folks
Summary Points:1. OOT is not OOS2. OOT is non-random changes over time3. OOT is a statistical and graphical issue4. OOT is relative. Statistical significance
is not sufficient.5. Trend limits = Natural Limits
June 28, 2007 54
References
Graphics: http://www.edwardtufte.com/tufte/ http://www.itl.nist.gov/div898/
handbook/eda/section3/eda34.htm
Statistics http://www.itl.nist.gov/div898/
handbook/index.htm
June 28, 2007 55
Software References
http://www.minitab.com/http://www.systat.com/products/sigmaplot/http://www.nwasoft.com/http://www.jmp.com/http://www.statgraphics.com/http://www.sas.com/
