Upload
mads-tandoc
View
126
Download
2
Tags:
Embed Size (px)
Citation preview
Assessment of the Analytical Phase of Selected Biochemical Tests at the UP-PGH Central Laboratory: Application of Six Sigma Metrics
to Internal and External Quality Control Data
Amado O. Tandoc III, MD Principal Author
Ariel M. Vergel De Dios, MD, FPSP Nelson T. Geraldino, MD, MSPH, FPSP Ma. Carmen L. Cagampan, MD, FPSP
Joselyn A. Mendoza, RMT Co-Authors
UNIVERSITY OF THE PHILIPPINES-PHILIPPINE GENERAL HOSPITAL DEPARTMENT OF LABORATORIES
1
Assessment of the Analytical Phase of Selected Biochemical Tests at the UP-PGH Central Laboratory: Application of Six Sigma Metrics to
Internal and External Quality Control DataA.O.Tandoc III, MD; A.M.Vergel De Dios,MD,FPSP; N.T.Geraldino,MD,MSPH,FPSP; M.C.L.Cagampan,MD,FPSP; J.A.Mendoza,RMT
UNIVERSITY OF THE PHILIPPINES-PHILIPPINE GENERAL HOSPITAL DEPARTMENT OF LABORATORIES
ABSTRACT. Ten (10) common biochemical tests (Glucose, BUN, Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol, Sodium, Potassium and Chloride) performed at the Chemistry Section of the UP-PGH Central Laboratory were analyzed by reviewing available paired Internal and External QC (EQAS, AQAS, PCQACL & NRLB) data in 2007 & 2008 and subjecting these data to Six Sigma principles. With a benchmark for world class quality of 6 sigma and acceptable process quality of 4-5 Sigma, only Uric Acid testing achieved an acceptable average Sigma metric at 4.69 using EQAS data. Creatinine, Total protein, Albumin and Potassium testing were within 3-4 Sigma. Glucose, Cholesterol, BUN, Sodium and Chloride testing all fall below 3-Sigma which is unacceptable quality. Using AQAS data, no analyte tested above 4-Sigma. Using PCQACL and NRLB data, only Potassium was within 4-5 Sigma.
INTRODUCTION
Medical laboratory work is composed of the technical activities that produce laboratory
results for patient care. Preanalytic (blood sample collection, receiving, accessioning), analytic
(testing, examinations, interpretation) and postanalytic activities (reporting results, archiving samples,
charging) transform a clinician’s order for a laboratory examination into the results used by the
clinician to diagnose and treat patients.
Central to every laboratory is its obligation to assure that the whole testing process is
accurate, reliable and ultimately, useful to the clinician. It is mandated by the Department of Health
(DOH) Administrative Order No. 2007-0027 Revised Rules and Regulations Governing the Licensure
and Regulation of Clinical Laboratories in the Philippines that every clinical laboratory shall be
organized to ensure the quality of laboratory results.1
Quality control mechanisms, both internal and external, are put in place just for this purpose.
Internal QC procedures are performed regularly through assaying QC materials for which the correct
result is known in advance in order to verify that a measurement procedure is performing as
expected.2 External quality control programs, in the form of proficiency testing, is one way by which
laboratory performance is evaluated against those of its peers that use the same instruments and
principles within the country and between countries. Such assessments allow a laboratory to verify
that its results are consistent with those of other laboratories using the same or similar methods for an
analyte.2
2
FIG. 1. THE SIX SIGMA PERFORMANCE GOAL. Six standard deviations of process variation should fit within the tolerance limits for the process.
FIG. 2. Two ways of deriving the Sigma Metric.
As will be demonstrated in the present study, a step further towards a more meaningful
interpretation of the data gleaned from both internal and external QC data may be achieved through
the use of the principles of Six Sigma.
Six Sigma is a process improvement
strategy that has been hailed as an “evolution
in quality management.”3 It is the universal
and objective way of quality measurement as
it measures the degree to which any process
deviates from its goal. How well the process
works is analyzed and measured as the first
step towards improving quality. It has been
implemented initially–and successfully–in
business and industry starting in the early
1990’s by companies such as Motorola and General Electric and espouses a performance goal that
“Six sigmas or 6 standard deviations of process variation should fit within the tolerance limits for the
process.”4
As a universal measure of process performance, any process can be evaluated in terms of a
Sigma Metric that describes how many “sigmas” fit within the tolerance limits. Of the two ways of
assessing performance, the measurement of variation is applicable in the laboratory setting. It makes
use of the information on precision and accuracy that laboratories derive from internal and external
quality control programs.5
The sigma-capability of the process tells us where and what level of improvement is
necessary. A high quality process strives to achieve a sigma value of 6. Processes of acceptable
quality have a sigma value of 4-5. Most processes are sigma 3, which connotes great opportunity for
improvement. As Sigma increases, process reliability improves, operating costs go down and
customer satisfaction increases.6,7,8
3
MEASURE OUTCOME MEASURE VARIATION
Inspect Outcomes & Count Defects
Calculate Defect Per Million (DPM)
Convert DPM to Sigma Metric
Measure Variation of a Process
Calculate SD and Process Capability
Convert Capability to Sigma Metric
In 2000, Nevalainen et al evaluated laboratory performance of quality indicators with the Six
Sigma Scale.8 This is the first application of Sigma metrics to laboratory data. In this study, the
preanalytic, analytic and postanalytic phase of laboratory operations in 3 clinical laboratories in the
U.S. were analyzed. Laboratory performance was expressed in parts-per-million which when
converted to Six Sigma using a standard table, demonstrated opportunities for significant
improvements across the total testing process encompassing all the phases. In 2006, Westgard and
Westgard, utilized Six Sigma principles in assessing the quality of the analytic phase of laboratory
testing of selected analytes in the US using data from proficiency testing surveys.9 For both studies,
the highest Sigma of US laboratories was between 4-5 sigma with most methods and processes
falling below this level.
Internet search revealed that no local study has been done on the application of Six Sigma in
clinical laboratories in the Philippines.
OBJECTIVES
The study aims to use the principles of Six Sigma to determine the quality of ten (10) selected
biochemical tests [Glucose, BUN, Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol,
Sodium, Potassium and Chloride] performed at the UP-PGH Central Laboratory Chemistry Section
using internal and external QC data. Specifically, the study aims to:
1. Review the performance of the UP-PGH Central Laboratory in terms of External Quality
Assurance Programs (EQAS, AQuAS, PCQACL, NRLB) that the laboratory has
participated in for the last two years;
2. Determine the quality of ten (10) selected tests [Glucose, BUN, Creatinine, Uric Acid,
Total Protein, Albumin, Cholesterol, Sodium, Potassium and Chloride] at the UP-PGH
Central Laboratory Chemistry Section by applying Six Sigma metrics to data gleaned
from the aforementioned proficiency testing programs and the laboratory’s own internal
QC data;
3. Make recommendations on how to improve the present quality of laboratory testing at the
UP-PGH; and to
4. Make recommendations on possible directions of further investigations using Six Sigma
principles.
4
FIG. 3. Formula for deriving the Sigma Metric for an analytic test.
Sigma = (TEa – bias)/CV
Where TEa = Total Allowable Error for the analytic test based on CLIA criteria
Bias = [(Laboratory mean – Group Mean)/Group Mean] x 100 from External QC
CV = Coefficient of Variation of the UP-PGH Central Laboratory Internal QC
METHODOLOGY
Data from Internal QC procedures and External Quality Assurance Programs* for the years
2007-2008 from the Section of Biochemistry of the UP-PGH Central Laboratory pertaining to ten
common (10) biochemical tests were collected. The analytes included are Glucose, Blood Urea
Nitrogen (BUN), Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol, Sodium, Potassium and
Chloride. Values of these analytes were expressed in SI units.
From the External QC results, data grouped according to same principle-and-machine are
chosen over data grouped only according to same principle. This is in order to decrease the bias that
may be imparted by a difference in the chemistry analyzer being used. Other sources of variability
such as environmental factors due to geographic location and climate, patient load per annum,
regularity of maintenance and calibration (under which conditions the analyzer is operated) were
beyond the scope of this study.
Where available, Internal and External Quality control data were matched as to month of
actual execution of QC procedures. This is in order to arrive at a true picture of QC performance at a
particular time. Only those months that have both Internal and External QC data were included in this
study. As this is a retrospective approach, Internal QC was performed by different medical
technologists over the study period.
The Sigma metric for each of the ten (10) analytes were computed based on the following
formula (Fig. 3) which takes into account both internal and external QC data.
__________________
*The UP-PGH Central Laboratory has been enrolled in two (2) international proficiency testing programs: 1) EQAS
(worldwide) and AQUAS (Asia). EQAS proficiency testing was performed every 2 weeks from November 2007 to April
2008. AQUAS has been performed twice yearly from 2001 to the present. The Central Laboratory has also participated in
two (2) local proficiency testing programs: 1) PCQACL (Philippine Council for Quality Assurance of Clinical Laboratories)
and 2) NRLB (National Reference Laboratory for Biochemistry). Both were done once during the year 2007.
5
TABLE 1. CLIA Criteria for Chemistry Analytes. Highlighted are the ones included in this study.
The most recent CLIA (Clinical Laboratory
Improvement Ammendments) Criteria for
Acceptable Performance in Proficiency Testing**
(Table 1) were used to determine the value of the
allowable total errors (TEa) for each analytic test.
The TEa represents the sum of all allowable
imprecision and bias of a test, i.e., the tolerance
specifications within which 6 Sigmas of standard
deviation must fall, in order to make the results of a
test measurement reliable.
The calculated value is the Sigma Metric
which will deem if the analytic process is of superior
or inferior quality. As earlier stated, the target goal
is a Sigma value of 6. Acceptable performance
ranges from Sigma 4 to 5.
RESULTS
External QC results (EQAS) from November
2007 to April 2008 were matched per month with
Internal QC data from the UP-PGH Central
Laboratory gathered from the same period of time.
Internal QC data were matched for June and
September 2007 AQuAS results. Results from local External QC surveys (PCQACL, NRLB)
conducted once in September 2007 were likewise matched with September 2007 Internal QC data.
Tables 2 through 11 show that the external
and
internal QC data are used to determine the Bias and
Coefficient of Variation (CV), respectively, in order
to arrive at the Sigma metric through the formula,
Sigma metric = (TEa – Bias)/CV. The TEa for
each analyte is taken from the CLIA requirements
(see Table 1).
6
Analyte PT Limit
Alanine aminotransferase (ALT, SGPT)
+/- 20%
Albumin +/- 10%
Alkaline phosphatase +/- 30%
Amylase +/- 30%
Aspartate aminotransferase (AST, SGOT)
+/- 20%
Bilirubin, total +/- 0.4 mg/dL or +/- 20% (greater)
Blood gas p02 +/- 3 SD
Blood gas pCO2 +/- 5 mm Hg or +/- 8% (greater)
Blood gas pH +/- 0.04
Calcium, total +/- 1.0 mg/dL
Chloride +/- 5%
Cholesterol, high dens. lipoprotein
+/- 30%
Cholesterol, total +/- 10%
Creatine kinase +/- 30%
Creatine kinase isoenzymes
MB elevated (present or absent) or Target value +/- 3 SD
Creatinine +/- 0.3 mg/dL or +/- 15% (greater)
Glucose +/- 6 mg/dL or +/- 10% (greater)
Iron, total +/- 20%
Lactate dehydrogenase (LDH)
+/- 20%
LDH isoenzymes LDH1/LDH2 (+ or -) or Target value +/- 30%
Magnesium +/- 25%
Potassium +/- 0.5 mmol/L (+/- 12%)
Sodium +/- 4 mmol/L (+/- 2.6%)
Total protein +/- 10%
Triglycerides +/- 25%
Urea Nitrogen +/- 2 mg/dL or +/- 9% (greater)
Uric acid +/- 17%
** There is no local equivalent as yet for the U.S. CLIA Criteria for Acceptable Performance in Proficiency Testing. Other sources of values for the TEa include the European Biologic Goals and Calculated Biologic Allowable Total Errors, Global Quality Requirements Consensus Draft, Desirable Specifications for Total Error, Imprecision and Bias Derived from Biologic Variation, et cetera.
TABLE 2. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR GLUCOSE.
GLUCOSE with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV
(NOVEMBER 2007)NO. 1 17.65 18.35 3.87 4.56 0.15 3.29 1.86 14.13 0.34 2.41 2.55NO. 2 19.45 23.61 17.58 4.56 0.15 3.29 NO SIGMA 14.13 0.34 2.41 NO SIGMA
(DECEMBER 2007) NO. 3 2.57 2.54 1.18 4.70 0.16 3.40 4.96 14.61 0.26 1.78 3.54NO. 4 12.50 12.98 3.70 4.70 0.16 3.40 1.85 14.61 0.26 1.78 3.54(JANUARY 2008) NO. 5 18.14 18.46 1.73 4.73 0.16 3.38 2.44 14.73 0.41 2.78 2.97NO. 6 12.68 13.13 3.43 4.73 0.16 3.38 1.94 14.73 0.41 2.78 2.36(FEBRUARY 2008) NO. 7 2.64 2.55 3.53 5.11 0.09 1.76 3.67 16.41 0.36 2.19 2.95 NO. 8 22.66 23.62 4.06 5.11 0.09 1.76 3.37 16.41 0.36 2.19 2.71(MARCH 2008) NO. 9 22.53 23.41 3.76 4.80 0.15 3.13 2.00 15.36 0.60 3.91 1.60 NO. 10 2.46 2.53 2.77 4.80 0.15 3.13 2.31 15.36 0.60 3.91 1.85 (APRIL 2008) NO. 11 18.2 18.31 0.60 4.75 0.28 5.89 1.59 15.21 0.79 5.19 1.81 NO. 12 13.21 12.90 2.40 4.75 0.28 5.89 1.29 15.21 0.79 5.19 1.46
AQUAS (JUNE 2007)LEVEL 1 5.34 5.61 4.90 5.07 0.16 3.16 1.62LEVEL 2 13.7 13.97 1.97 15.41 0.76 4.93 1.63(SEPTEMBER 2007) LEVEL 1 4.90 5.39 9.18 4.68 0.14 2.99 0.27LEVEL 2 12.93 13.97 7.48 14.31 0.33 2.31 1.09
PCQACL (SEPTEMBER 2007)NORMAL 5.88 5.70 3.16 4.68 0.14 2.99 2.29ABNORMAL 14.52 14.4 0.83 14.31 0.33 2.31 3.98
NRLB (SEPTEMBER 2007)
VIAL 1 4.42 4.48 7.92 4.68 0.14 2.99 0.70VIAL 2 13.86 14.70 5.71 14.31 0.33 2.31 1.86
For glucose, 6-sigma values range from as low as 0.27 (which is substandard) to 4.96 (which is acceptable). Two instances during the 2nd cycle of November 2007 for the EQAS have no calculated sigma value. This is because the bias calculated is greater than the TEa set by the CLIA for glucose. When this happens, a negative value is calculated for the sigma metric. Because the bias is greater than the total allowable error for the analyte, we cannot consider the calculated sigma metric as “within 6 standard deviations of the tolerance limits.” Out of 22 observations where the sigma metric can be calculated, 5 are above 3.00, which is not impressive by 6-sigma standards. If we average the sigma metric for glucose, we get 2.48 (EQAS), 1.15 (AQUAS), 3.14 (PCQACL) and 1.63 (NRLB) – all of which are unimpressive and below the acceptable standards.
7
TABLE 3. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR BLOOD UREA NITROGEN.
BLOOD UREA NITROGEN with TEa = + 9% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV
(NOVEMBER 2007)NO. 1 27.62 27.50 0.44 6.26 0.58 9.27 0.92 18.15 1.25 6.89 1.24NO. 2 30.71 36.00 14.69 6.26 0.58 9.27 NO SIGMA 18.15 1.25 6.89 NO SIGMA(DECEMBER 2007) NO. 3 4.13 3.62 14.09 5.37 0.67 12.48 NO SIGMA 16.51 1.11 6.72 NO SIGMANO. 4 18.32 19.60 6.53 5.37 0.67 12.48 0.20 16.51 1.11 6.72 0.37(JANUARY 2008) NO. 5 28.39 27.80 2.12 5.28 0.41 7.77 0.89 17.1 0.67 3.92 1.76NO. 6 19.08 19.40 1.65 5.28 0.41 7.77 0.95 17.1 0.67 3.92 1.88(FEBRUARY 2008) NO. 7 3.43 3.53 2.83 5.60 0.48 8.57 0.72 17.30 0.79 4.57 1.35NO. 8 34.75 35.20 1.28 5.60 0.48 8.57 0.90 17.30 0.79 4.57 1.69(MARCH 2008) NO. 9 35.75 35.40 0.99 5.47 0.66 12.07 0.66 16.56 1.17 7.07 1.13NO. 10 4.11 3.55 15.77 5.47 0.66 12.07 NO SIGMA 16.56 1.17 7.07 NO SIGMA(APRIL 2008) NO. 11 26.70 27.30 2.20 4.97 0.59 11.87 0.57 16.1 1.19 7.39 0.92NO. 12 18.95 18.90 0.26 4.97 0.59 11.87 0.74 16.1 1.19 7.39 1.18
AQUAS (JUNE 2007)LEVEL 1 6.07 7.14 15.00 6.01 0.56 9.32 NO SIGMALEVEL 2 21.78 20.35 7.02 18.18 1.01 5.56 0.36(SEPTEMBER 2007) LEVEL 1 6.07 7.14 15.00 5.57 0.78 14.00 NO SIGMALEVEL 2 18.21 19.99 8.93 17.12 1.24 7.24 0.01
PCQACL (SEPTEMBER 2007)NORMAL 7.99 7.29 9.6 5.57 0.78 14.00 NO SIGMAABNORMAL 23.25 20 16.25 17.12 1.24 7.24 NO SIGMA
NRLB (SEPTEMBER 2007)VIAL 1 5.46 6.00 9.00 5.57 0.78 14.00 0.00VIAL 2 17.08 16.70 2.28 17.12 1.24 7.24 0.93
For BUN, 6-sigma values range from as low as 0.00 to 1.88. In nine instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. Moreover, most of the CV’s of the internal QC data are very high (6-14), which will likewise decrease the calculated sigma metric. No sigma values are above 3. The average sigma metric for BUN is 1 (EQAS), 0.37 (AQUAS) and 0.47 (NRLB).
8
TABLE 4. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CREATININE.
CREATININE with TEa = + 15% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 256.00 260.00 1.54 189.36 8.75 4.62 2.91 540.96 24.93 4.61 2.92NO. 2 57.00 58.40 2.40 189.36 8.75 4.62 2.73 540.96 24.93 4.61 2.73(DECEMBER 2007) NO. 3 874.00 884.00 1.13 187.20 7.48 4.00 3.47 559.10 14.93 2.67 5.19NO. 4 434.00 484.00 10.35 187.20 7.48 4.00 1.17 559.10 14.93 2.67 1.75(JANUARY 2008) NO. 5 263.00 267.00 1.50 187.48 5.41 2.89 4.68 567.97 8.80 1.55 8.71NO. 6 484.00 465.00 4.09 187.48 5.41 2.89 3.78 567.97 8.80 1.55 7.04(FEBRUARY 2008) NO. 7 880.00 880.00 0.00 199.71 7.27 3.64 4.12 574.14 18.34 3.19 4.70NO. 8 53.00 59.10 10.32 199.71 7.27 3.64 1.29 574.14 18.34 3.19 1.46(MARCH 2008) NO. 9 53.00 57.10 7.18 200.23 11.21 5.60 1.40 563.93 20.11 3.57 2.19NO. 10 889.00 873.00 1.83 200.23 11.21 5.60 2.35 563.93 20.11 3.57 3.69(APRIL 2008) NO. 11 265.00 265.00 0.00 199.80 12.32 6.17 2.43 583.80 22.69 3.89 3.86NO. 12 476.00 460.00 0.83 199.80 12.32 6.17 2.30 583.80 22.69 3.89 3.64
AQUAS (JUNE 2007)LEVEL 1 123.76 114.92 7.69 188.86 19.40 10.27 0.71LEVEL 2 362.44 344.76 5.13 570.07 41.06 7.20 1.37(SEPTEMBER 2007) LEVEL 1 106.08 114.92 7.69 200.11 7.55 3.77 1.94LEVEL 2 335.92 344.76 2.56 578.54 32.31 5.58 2.23
PCQACL (SEPTEMBER 2007)NORMAL 109.00 115.00 5.22 200.11 7.55 3.77 2.59ABNORMAL 348.00 354.00 1.69 578.54 32.31 5.58 2.38
NRLB (SEPTEMBER 2007)VIAL 1 197.00 153.70 28.17 200.11 7.55 3.77 NO SIGMAVIAL 2 572.00 508.00 12.60 578.54 32.31 5.58 0.43
For Creatinine, 6-sigma values range from as low as 0.43 to 8.71 (even higher than 6-sigma). In two instances the sigma metric is greater than 6-sigma, in four instances between 4-5 sigma, and in five instances between 3-4 sigma. In only one instance (NRLB vial 1) is there no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. The findings suggest an excellent method performance for Creatinine since the bias and CV are relatively low to yield higher sigma values. However, it must be considered that the CLIA requirement for Creatinine is very high at + 15% total allowable error leaving
9
TABLE 5. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR URIC ACID.
much room for bias and inaccuracy. The average sigma metric for Creatinine is 3.35 (EQAS), 1.56 (AQUAS), and 2.49 (PCQACL).
URIC ACID with TEa = + 17% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 0.260 0.276 5.80 0.25 0.01 4.00 2.80 0.53 0.02 3.77 2.97NO. 2 0.160 0.193 17.10 0.25 0.01 4.00 NO SIGMA 0.53 0.02 3.77 NO SIGMA(DECEMBER 2007) NO. 3 0.560 0.553 1.27 0.28 0.01 3.57 4.41 0.56 0.01 1.79 8.81NO. 4 0.380 0.374 1.60 0.28 0.01 3.57 4.31 0.56 0.01 1.79 8.62(JANUARY 2008) NO. 5 0.280 0.277 1.08 0.28 0.02 7.14 2.23 0.56 0.01 1.79 8.91NO. 6 0.380 0.368 3.26 0.28 0.02 7.14 1.92 0.56 0.01 1.79 7.69(FEBRUARY 2008) NO. 7 0.560 0.549 2.00 0.29 0.01 3.45 4.35 0.56 0.10 18.04 0.83NO. 8 0.190 0.193 1.55 0.29 0.01 3.45 4.48 0.56 0.10 18.04 0.86(MARCH 2008) NO. 9 0.200 0.194 3.09 0.35 0.02 5.71 2.43 0.55 0.02 3.64 3.82NO. 10 0.570 0.552 3.26 0.35 0.02 5.71 2.40 0.55 0.02 3.64 3.78(APRIL 2008) NO. 11 0.280 0.281 0.36 0.28 0.01 3.57 4.66 0.55 0.01 1.82 9.15NO. 12 0.370 0.372 0.54 0.28 0.01 3.57 4.61 0.55 0.01 1.82 9.05
AQUAS (JUNE 2007)LEVEL 1 0.400 0.360 11.67 0.27 0.02 7.41 0.72LEVEL 2 0.600 0.550 8.60 0.56 0.02 3.57 2.35(SEPTEMBER 2007) LEVEL 1 0.370 0.340 6.90 0.29 0.01 3.45 2.93LEVEL 2 0.600 0.550 8.60 0.61 0.03 4.92 1.71
PCQACL (SEPTEMBER 2007)NORMAL 0.370 0.340 5.80 0.29 0.01 3.45 3.22ABNORMAL 0.590 0.540 9.26 0.61 0.03 4.92 1.57
NRLB (SEPTEMBER 2007)VIAL 1 0.260 0.280 8.61 0.29 0.01 3.45 2.43VIAL 2 0.620 0.550 13.62 0.61 0.03 4.92 0.69
For Uric Acid, 6-sigma values range from as 0.72 to 9.15 (greater than 6-sigma). In two instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. Again, the TEa set by CLIA for Uric Acid is generous at + 17%, accounting for
10
TABLE 6. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR TOTAL PROTEIN.
the high sigma values in six instances (7.69-9.15). In six instances the sigma metric is between 4-5-sigma. The average sigma metric for Uric Acid is 4.69 (EQAS), 1.93 (AQUAS), 2.40 (PCQACL) and 1.56 (NRLB).
TOTAL PROTEIN with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 70.00 68.21 2.62 65.92 2.12 3.22 2.29 43.52 1.39 3.19 2.31NO. 2 68.00 78.15 12.99 65.92 2.12 3.22 NO SIGMA 43.52 1.39 3.19 NO SIGMA(DECEMBER 2007) NO. 3 40.00 39.30 1.78 66.00 1.97 2.98 2.75 43.37 1.27 2.93 2.81NO. 4 56.00 58.40 4.11 66.00 1.97 2.98 1.97 43.37 1.27 2.93 2.01(JANUARY 2008) NO. 5 70.00 68.90 1.60 66.48 1.18 1.77 4.73 44.97 1.05 2.33 3.60NO. 6 59.00 58.80 0.38 66.48 1.18 1.77 5.44 44.97 1.05 2.33 4.14(FEBRUARY 2008) NO. 7 40.00 39.00 2.56 68.59 1.18 1.72 4.32 44.97 1.05 2.33 3.18NO. 8 78.00 78.00 0.00 68.59 1.18 1.72 5.81 44.97 1.05 2.33 4.28(MARCH 2008) NO. 9 77.00 78.36 1.74 66.63 1.47 2.21 3.75 43.77 1.38 3.15 2.62NO. 10 37.00 38.70 4.39 66.63 1.47 2.21 2.54 43.77 1.38 3.15 1.78(APRIL 2008) NO. 11 67.00 68.69 2.46 64.13 1.30 2.03 3.72 42.73 1.10 2.57 2.93NO. 12 57.00 59.00 3.39 64.13 1.30 2.03 3.26 42.73 1.10 2.57 2.57
AQUAS (JUNE 2007)LEVEL 1 63.00 62.00 1.61 66.79 2.69 4.03 2.08LEVEL 2 47.00 47.00 0.00 44.38 1.82 4.10 2.44(SEPTEMBER 2007) LEVEL 1 59.00 62.00 4.84 67.89 1.69 2.49 2.07LEVEL 2 45.00 47.00 4.26 45.46 1.10 2.42 2.37
PCQACL (SEPTEMBER 2007)NORMAL 60.00 61.40 2.28 67.89 1.69 2.49 3.10ABNORMAL 44.00 45.80 3.93 45.46 1.10 2.42 2.51
NRLB (SEPTEMBER 2007)VIAL 1 67.00 62.60 7.029 67.89 1.69 2.49 1.19VIAL 2 46.00 41.70 10.31 45.46 1.10 2.42 NO SIGMA
For Total Protein, 6-sigma values range from 1.19 to 5.81. In three instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa.
11
TABLE 7. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR ALBUMIN.
In two instances, the sigma value is greater than 5, in four instances, between 4-5 sigma, and in six instances, between 3-4 sigma. The average sigma metric for Total Protein is 3.31 (EQAS), 2.24 (AQUAS) and 2.81 (PCQACL).
ALBUMIN with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 41.00 41.15 0.36 38.08 1.22 3.20 3.01 25.56 0.71 2.78 3.47NO. 2 41.00 46.25 11.35 38.08 1.22 3.20 NO SIGMA 25.56 0.71 2.78 NO SIGMA(DECEMBER 2007) NO. 3 25.00 28.20 11.35 38.73 1.20 3.10 NO SIGMA 26.10 0.55 2.11 NO SIGMANO. 4 35.00 35.29 0.82 38.73 1.20 3.10 2.96 26.10 0.55 2.11 4.36(JANUARY 2008) NO. 5 41.00 41.22 0.53 38.38 0.86 2.24 4.22 25.79 0.62 2.40 3.94NO. 6 35.00 35.20 0.57 38.38 0.86 2.24 4.21 25.79 0.62 2.40 3.92(FEBRUARY 2008) NO. 7 46.00 46.50 1.08 38.60 1.30 3.37 2.65 25.90 0.84 3.24 2.75NO. 8 24.00 23.70 1.27 38.60 1.30 3.37 2.59 25.90 0.84 3.24 2.69(MARCH 2008) NO. 9 24.00 23.80 0.84 39.34 0.77 1.96 4.68 26.31 0.54 2.05 4.46NO. 10 46.00 46.41 0.88 39.34 0.77 1.96 4.66 26.31 0.54 2.05 4.44(APRIL 2008) NO. 11 43.00 41.54 3.51 37.07 0.59 1.59 4.07 24.57 0.82 3.32 1.95NO. 12 37.00 35.09 5.44 37.07 0.59 1.59 2.86 24.57 0.82 3.32 1.37
AQUAS (JUNE 2007)LEVEL 1 43.00 44.00 2.27 41.21 1.72 4.17 1.85LEVEL 2 27.00 29.00 6.90 28.34 1.56 5.50 0.56(SEPTEMBER 2007) LEVEL 1 39.00 42.00 7.14 39.11 0.99 2.53 1.13LEVEL 2 25.00 27.00 7.41 26.79 0.83 3.10 0.84
PCQACL (SEPTEMBER 2007)NORMAL 38.00 41.20 7.77 39.11 0.99 2.53 0.88ABNORMAL 26.00 26.90 3.35 26.79 0.83 3.10 2.15
NRLB (SEPTEMBER 2007)VIAL 1 46.00 44.30 3.84 39.11 0.99 2.53 2.43VIAL 2 30.00 27.80 7.91 26.79 0.83 3.10 0.67
For Albumin, 6-sigma values range from as low as 0.67 to 4.68. In four instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa.
12
TABLE 8. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CHOLESTEROL.
In nine instances, the sigma value is between 4-5, and in four instances, between 3-4 sigma. The average sigma metric for Albumin is 3.23 (EQAS), 1.10 (AQUAS), 1.52 (PCQACL) and 1.55 (NRLB).
CHOLESTEROL with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 6.87 6.78 1.33 6.25 0.17 2.72 3.19 2.55 0.06 2.35 3.69NO. 2 6.96 8.07 13.75 6.25 0.17 2.72 NO SIGMA 2.55 0.06 2.35 NO SIGMA(DECEMBER 2007) NO. 3 2.65 2.58 2.71 6.11 0.16 2.62 2.78 2.50 0.05 2.00 3.64NO. 4 5.22 5.71 8.58 6.11 0.16 2.62 0.54 2.50 0.05 2.00 0.71(JANUARY 2008) NO. 5 6.76 6.93 2.45 6.08 0.12 1.97 3.82 2.50 0.04 1.60 4.72NO. 6 5.43 5.38 0.93 6.08 0.12 1.97 4.60 2.50 0.04 1.60 5.67(FEBRUARY 2008) NO. 7 8.06 8.07 0.12 6.29 0.17 2.70 3.65 2.50 0.08 3.20 3.09NO. 8 2.45 2.58 5.04 6.29 0.17 2.70 1.84 2.50 0.08 3.20 1.55(MARCH 2008) NO. 9 2.64 2.56 3.13 6.38 0.15 2.35 2.92 2.54 0.06 2.36 2.91NO. 10 7.76 8.07 3.84 6.38 0.15 2.35 2.62 2.54 0.06 2.36 2.61(APRIL 2008) NO. 11 6.80 6.88 1.16 6.99 0.16 2.27 3.89 2.29 0.15 6.55 1.35NO. 12 5.36 5.37 0.19 6.99 0.16 2.27 4.34 2.29 0.15 6.55 1.50
AQUAS (JUNE 2007)LEVEL 1 3.99 4.79 9.11 3.20 0.24 7.50 0.12LEVEL 2 7.18 7.59 5.31 6.92 0.33 4.77 0.98(SEPTEMBER 2007) LEVEL 1 4.19 4.34 3.39 6.34 0.18 2.84 2.33LEVEL 2 7.38 7.93 6.93 2.57 0.05 1.95 1.58
NRLB (SEPTEMBER 2007)VIAL 1 6.08 6.20 1.94 6.34 0.18 2.84 2.84VIAL 2 2.52 2.70 6.67 2.57 0.05 1.95 1.71
*No values were submitted for Cholesterol for the PCQACL External Quality Control Survey for September 2007.
For Cholesterol, 6-sigma values range from as low as 0.12 to 5.67. In two instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. In four instances, the sigma value is between 4-5, and in seven instances, between 3-4 sigma. The average sigma metric for Cholesterol is 2.98 (EQAS), 1.25 (AQUAS), and 2.28 (NRLB).
13
TABLE 9. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR SODIUM.
SODIUM with TEa = + 2.6% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 155.00 147.40 5.16 148.80 4.50 3.02 NO SIGMA 126.68 3.39 2.68 NO SIGMANO. 2 143.00 157.40 9.15 148.80 4.50 3.02 NO SIGMA 126.68 3.39 2.68 NO SIGMA(DECEMBER 2007) NO. 3 110.00 110.80 0.72 148.87 2.79 1.87 1.00 126.00 2.07 1.64 1.14NO. 4 139.00 135.90 2.28 148.87 2.79 1.87 0.17 126.00 2.07 1.64 0.19(JANUARY 2008) NO. 5 157.00 149.70 4.88 149.55 2.98 1.99 NO SIGMA 127.10 1.78 1.40 NO SIGMANO. 6 140.00 136.40 2.64 149.55 2.98 1.99 NO SIGMA 127.10 1.78 1.40 NO SIGMA(FEBRUARY 2008) NO. 7 113.00 110.90 1.89 149.76 1.83 1.22 0.58 129.03 1.92 1.49 0.47NO. 8 174.00 160.50 8.41 149.76 1.83 1.22 NO SIGMA 129.03 1.92 1.49 NO SIGMA(MARCH 2008) NO. 9 168.00 158.90 5.73 148.60 2.55 1.72 NO SIGMA 129.03 3.59 2.78 NO SIGMANO. 10 110.00 110.70 0.63 148.60 2.55 1.72 1.15 129.03 3.59 2.78 0.71(APRIL 2008) NO. 11 156.00 149.20 4.56 149.00 2.42 1.62 NO SIGMA 128.57 2.97 2.31 NO SIGMANO. 12 142.00 138.00 2.90 149.00 2.42 1.62 NO SIGMA 128.57 2.97 2.31 NO SIGMA
AQUAS (JUNE 2007)LEVEL 1 143.00 141.00 1.42 148.52 5.38 3.62 0.33LEVEL 2 159.00 156.00 1.92 127.00 5.45 4.29 0.16(SEPTEMBER 2007) LEVEL 1 142.00 140.00 1.43 148.86 3.42 2.30 0.51LEVEL 2 158.00 156.00 1.28 126.50 6.01 4.75 0.28
PCQACL (SEPTEMBER 2007)NORMAL 142.00 137.00 3.65 148.86 3.42 2.30 NO SIGMAABNORMAL 159.00 158.00 0.63 126.50 6.01 4.75 0.41
NRLB (SEPTEMBER 2007)VIAL 1 144.00 144.30 0.21 148.86 3.42 2.30 1.04VIAL 2 125.00 124.70 0.24 126.50 6.01 4.75 0.50
For Sodium, 6-sigma values range from 0.17 to 1.15. Due to a very stringent CLIA requirement of + 2.6%, no sigma value was calculable in seventeen instances. The rest of the values
14
TABLE 10. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR POTASSIUM.
are between slightly above 0 to slightly above 1. No values reach at least 3-sigma. Taking into consideration the strict CLIA criterion, the findings for Sodium are suggestive of a very difficult quality assurance for this analyte, a very important chemical test in many a patient’s diagnostic work up. The average sigma metric is 0.68 (EQAS), 0.32 (AQUAS), 0.41 (PCQACL) and 0.77 (NRLB).
POTASSIUM with TEa = + 12% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 2.50 2.39 4.60 4.03 0.11 2.73 2.71 6.02 0.18 2.99 2.47NO. 2 1.20 1.24 3.23 4.03 0.11 2.73 3.21 6.02 0.18 2.99 2.93(DECEMBER 2007) NO. 3 6.00 5.96 0.67 4.00 0.08 2.00 5.66 5.97 0.10 1.68 6.76NO. 4 3.80 3.63 4.68 4.00 0.08 2.00 3.66 5.97 0.10 1.68 4.37(JANUARY 2008) NO. 5 2.60 2.39 8.79 4.04 0.07 1.73 1.85 6.31 0.08 1.27 2.53NO. 6 3.80 3.62 4.97 4.04 0.07 1.73 4.06 6.31 0.08 1.27 5.54(FEBRUARY 2008) NO. 7 6.20 5.56 11.51 4.08 0.08 1.96 0.25 5.98 0.10 1.67 0.29NO. 8 1.40 1.24 12.90 4.08 0.08 1.96 NO SIGMA 5.98 0.10 1.67 NO SIGMA(MARCH 2008) NO. 9 1.30 1.22 6.56 4.93 0.08 1.99 2.74 5.99 0.16 2.67 2.04NO. 10 6.00 5.91 1.52 4.93 0.08 1.99 5.28 5.99 0.16 2.67 3.92(APRIL 2008) NO. 11 2.50 2.36 5.93 4.07 0.05 1.23 4.94 5.99 0.08 1.34 4.54NO. 12 3.90 3.64 7.14 4.07 0.05 1.23 3.95 5.99 0.08 1.34 3.64
AQUAS (JUNE 2007)LEVEL 1 4.00 4.00 0.00 4.04 0.15 3.71 3.23LEVEL 2 6.10 6.10 0.00 6.01 0.28 4.66 2.58(SEPTEMBER 2007) LEVEL 1 4.00 3.90 2.56 4.02 0.10 2.49 3.79LEVEL 2 6.00 5.90 1.69 6.01 0.13 2.16 4.77
PCQACL (SEPTEMBER 2007)NORMAL 4.00 3.95 1.27 4.02 0.10 2.49 4.31ABNORMAL 6.00 6.05 0.83 6.01 0.13 2.16 5.17
NRLB (SEPTEMBER 2007)VIAL 1 3.90 3.90 0.00 4.02 0.10 2.49 4.82VIAL 2 6.00 5.90 1.70 6.01 0.13 2.16 4.77
15
TABLE 11. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CHLORIDE.
Potassium fares better with sigma values ranging from 0.25 to 6.76. In only two instances there is no calculated sigma because the bias is greater than the total allowable error. In one instance, the sigma is greater than 6 (6.76), in four instances, the sigma is between 5-6, in eight instances, the sigma is between 4-5 and in seven instances, between 3-4. The average sigma metric for Potassium is 3.52 (EQAS), 3.59 (AQUAS), 4.74 (PCQACL) and 4.80 (NRLB). Is this acceptable quality? We must consider that the CLIA requirement for Potassium is + 12%.
CHLORIDE with TEa = + 5% EXTERNAL QC DATA INTERNAL QC DATA
SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 119.00 109.30 8.87 110.60 4.26 3.85 NO SIGMA 93.96 3.12 3.32 NO SIGMANO. 2 111.00 109.30 1.56 110.60 4.26 3.85 0.89 93.96 3.12 3.32 1.04(DECEMBER 2007) NO. 3 92.00 88.45 4.01 110.30 2.25 2.04 0.48 93.40 1.89 2.02 0.49NO. 4 109.00 103.40 5.42 110.30 2.25 2.04 NO SIGMA 93.40 1.89 2.02 NO SIGMA(JANUARY 2008) NO. 5 120.00 110.40 8.70 108.79 2.21 2.03 NO SIGMA 92.83 1.56 1.68 NO SIGMANO. 6 110.00 104.00 5.77 108.79 2.21 2.03 NO SIGMA 92.83 1.56 1.68 NO SIGMA(FEBRUARY 2008) NO. 7 127.00 117.90 7.72 110.03 2.98 2.71 NO SIGMA 92.53 2.97 3.21 NO SIGMANO. 8 91.00 88.46 2.87 110.03 2.98 2.71 0.79 92.53 2.97 3.21 0.66(MARCH 2008) NO. 9 94.00 88.50 6.21 109.71 2.71 2.47 NO SIGMA 92.17 1.63 1.77 NO SIGMANO. 10 131.00 119.70 9.44 109.71 2.71 2.47 NO SIGMA 92.17 1.63 1.77 NO SIGMA(APRIL 2008) NO. 11 120.00 110.80 8.30 111.53 2.00 1.79 NO SIGMA 93.33 1.33 1.43 NO SIGMANO. 12 111.00 103.50 7.25 111.53 2.00 1.79 NO SIGMA 93.33 1.33 1.43 NO SIGMA
AQUAS (JUNE 2007)LEVEL 1 103.00 103.00 0.00 110.79 3.87 3.49 1.43LEVEL 2 121.00 121.00 0.00 94.69 4.01 4.23 1.18(SEPTEMBER 2007) LEVEL 1 103.00 101.00 1.98 110.75 2.89 2.61 1.16LEVEL 2 121.00 117.00 2.56 94.39 2.50 2.65 0.92
PCQACL (SEPTEMBER 2007)NORMAL 104.00 102.00 1.96 110.75 2.89 2.61 1.16ABNORMAL 121.00 121.00 0.00 94.39 2.50 2.65 1.89
NRLB (SEPTEMBER 2007)VIAL 1 106.00 101.10 4.85 110.75 2.89 2.61 0.06VIAL 2 93.00 87.30 6.53 94.39 2.50 2.65 NO SIGMA
16
For Chloride, 6-sigma values range from 0.06 to 1.89. In nineteen instances, there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. No sigma values are beyond 1.50, suggesting a poor method performance for Chloride. The average sigma value for Chloride is 0.73 (EQAS), 1.17 (AQUAS), 1.53 (PCQACL) and 0.06 (NRLB).
17
DISCUSSION
Physicians use laboratory work-ups in aid of diagnosis and management of patients.
Accuracy of laboratory results is therefore important in assuring and improving the quality of patient
care. The Philippine government is mandated to ensure accuracy and precision of laboratory
examinations to safeguard public health and safety.
Laboratories are thus obliged to regularly perform Internal Quality assurance procedures that
include use of control materials, calibration, result verification, application of Westgard Multirules
and statistical analysis, and equipment maintenance.
At the same time, laboratories are enjoined to participate in External Quality Assurance
Programs in which the quality of laboratory performance is assessed through the closeness of its
results to the pre-determined value or reference value generated by the other participants through peer
group analysis.
If the lab control results for a particular run are “in-control” and do not violate the Westgard
Multirules, is the laboratory performing as well as it should? If the lab satisfactorily passes External
Quality Assurance Programs, is the lab truly consistent and dependable? Corollary to these two
questions: are the results generated by an “in-control” lab reliable and, therefore, truly useful to the
clinician?
Six Sigma Metrics, as an objective and universal tool in measuring process performance,
redefines the concept of quality. Novel–and more stringent–standards are set for methods and
processes to ensure results that are clinically relevant and reliable. A process is deemed to be of
superior or inferior quality by calculating the “Sigma metric” with a standard formula that utilizes
both Internal and External QC data. The once separate (but essentially complimentary) concepts and
methods of Quality Assurance are combined, to yield data that is more reflective of true lab
performance at a particular time.
Table 12 shows a summary of the 6-sigma values calculated for each analyte using the EQAS
data to calculate the bias.
At 4.69 sigma, Uric Acid is at an acceptable Sigma process capability. Creatinine, Total
protein, Albumin and Potassium are within 3-4 sigma. Glucose, Cholesterol, BUN, Sodium and
Chloride all fall below 3-sigma which is unacceptable quality. We must consider of course
differences in the TEa set by the CLIA for each analyte (i.e., the TEa of Uric Acid is at a generous +
17% while the TEa of sodium is strict at + 2.6%).
18
TABLE 12. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING EQAS DATA.
SIX SIGMA PERFORMANCE USING EQAS DATA
EQAS GLU BUN CREA UA TP ALB CHOLE NA K CLNOV.2007
NO.1 1.86 0.92 2.91 2.8 2.29 3.01 3.19 NO SIGMA 2.71 NO SIGMA
2.55 1.24 2.92 2.97 2.31 3.47 3.69 NO SIGMA 2.47 NO SIGMA
NO.2 NO SIGMA NO SIGMA 2.73 NO SIGMA NO SIGMA NO SIGMA NO SIGMA NO SIGMA 3.21 0.89NO SIGMA NO SIGMA 2.73 NO SIGMA NO SIGMA NO SIGMA NO SIGMA NO SIGMA 2.93 1.04
DEC.2007NO.3 4.96 NO SIGMA 3.47 4.41 2.75 NO SIGMA 2.78 1 5.66 0.48
3.54 NO SIGMA 5.19 8.81 2.81 NO SIGMA 3.64 1.14 6.76 0.49NO.4 1.85 0.2 1.17 4.31 1.97 2.96 0.54 0.17 3.66 NO SIGMA
3.54 0.37 1.75 8.62 2.01 4.36 0.71 0.19 4.37 NO SIGMA
JAN.2008NO.5 2.44 0.89 4.68 2.23 4.73 4.22 3.82 NO SIGMA 1.85 NO SIGMA
2.97 1.76 8.71 8.91 3.6 3.94 4.72 NO SIGMA 2.53 NO SIGMA
NO.6 1.94 0.95 3.78 1.92 5.44 4.21 4.6 NO SIGMA 4.06 NO SIGMA
2.36 1.88 7.04 7.69 4.14 3.92 5.67 NO SIGMA 5.54 NO SIGMA
FEB.2008NO.7 3.67 0.72 4.12 4.35 4.32 2.65 3.65 0.58 0.25 NO SIGMA
2.95 1.35 4.7 0.83 3.18 2.75 3.09 0.47 0.29 NO SIGMA
NO.8 3.37 0.9 1.29 4.48 5.81 2.59 1.84 NO SIGMA NO SIGMA 0.792.71 1.69 1.46 0.86 4.28 2.69 1.55 NO SIGMA NO SIGMA 0.66
MAR.2008NO.9 2.00 0.66 1.4 2.43 3.75 4.68 2.92 NO SIGMA 2.74 NO SIGMA
1.60 1.13 2.19 3.82 2.62 4.46 2.91 NO SIGMA 2.04 NO SIGMA
NO.10 2.31 NO SIGMA 2.35 2.4 2.54 4.66 2.62 1.15 5.28 NO SIGMA
1.85 NO SIGMA 3.69 3.78 1.78 4.44 2.61 0.71 3.92 NO SIGMA
APR.2008NO.11 1.59 0.57 2.43 4.66 3.72 4.07 3.89 NO SIGMA 4.94 NO SIGMA
1.81 0.92 3.86 9.15 2.93 1.95 1.35 NO SIGMA 4.54 NO SIGMA
NO.12 1.29 0.74 2.3 4.61 3.26 2.86 4.34 NO SIGMA 3.95 NO SIGMA
1.46 1.18 3.64 9.05 2.57 1.37 1.50 NO SIGMA 3.64 NO SIGMA
AVERAGE 2.48 1.00 3.35 4.69 3.31 3.23 2.98 0.68 3.52 0.73
19
TABLE 13. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING AQAS DATA.
TABLE 14. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING PCQACL DATA.
TABLE 15. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING NRLB DATA.
Table 13 shows a summary of the 6-sigma values calculated for each analyte using the
AQUAS data to calculate the bias.
SIX SIGMA PERFORMANCE USING AQUAS DATA
AQUAS GLU BUN CREA UA TP ALB CHOLE NA K CLJUN.2007 LEVEL 1 1.62 NO SIGMA 0.71 0.72 2.08 1.85 0.12 0.33 3.23 1.43LEVEL 2 1.63 0.36 1.37 2.35 2.44 0.56 0.98 0.16 2.58 1.18
SEP.2007 LEVEL 1 0.27 NO SIGMA 1.94 2.93 2.07 1.13 2.33 0.51 3.79 1.16LEVEL 2 1.09 0.01 2.23 1.71 2.37 0.84 1.58 0.28 4.77 0.92
AVERAGE 1.15 0.37 1.56 1.93 2.24 1.10 1.25 0.32 3.59 1.17
The UP-PGH Central Laboratory’s performance using AQUAS data is unimpressive by the Sigma
standards with Sigma values below 1 (BUN and Sodium), the highest at 3.59 (Potassium) and the rest
between 1-3 sigma.
Tables 14 and 15 show a summary of the 6-sigma values calculated for each analyte using the
PCQACL and NRLB data, respectively to calculate the bias.
SIX SIGMA PERFORMANCE USING PCQACL DATA
PCQACL GLU BUN CREA UA TP ALB CHOLE NA K CLSEP.2007 NORMAL 2.29 NO SIGMA 2.59 3.22 3.1 0.88 NO RESULT NO SIGMA 4.31 1.16
ABNORMAL 3.98 NO SIGMA 2.38 1.57 2.51 2.15 NO RESULT 0.41 5.17 1.89AVERAGE 3.14 ---- 2.49 2.40 2.81 1.52 ---- 0.41 4.74 1.53
SIX SIGMA PERFORMANCE USING NRLB DATA
NRLB GLU BUN CREA UA TP ALB CHOLE NA K CLSEP.2007
VIAL 1 0.7 0 NO SIGMA 2.43 1.19 2.43 2.84 1.04 4.82 0.06VIAL 2 1.86 0.93 0.43 0.69 NO SIGMA 0.67 1.71 0.5 4.77 NO SIGMA
AVERAGE 1.63 0.47 0.43 1.56 1.19 1.55 2.28 0.77 4.80 0.06
Again, we note that the average performance of the analytes (except for Potassium) falls
below of the acceptable quality of 4-5 sigma and world class quality of 6-sigma using the two local
proficiency testing survey results.
20
Internal QC data generated by the UP-PGH laboratory itself in its daily operations is a major
contributor to its poor sigma performance. A review of Tables 2 to 11 will show that the Coefficients
of Variation for the analytes at various points in time and at various levels of analyte testing are high
and far from the ideal CV of 1.
Short of changing analyzers and/or reagents/control materials, which entails substantial cost
to the hospital, are there any realistic solutions to such a low Sigma performance in the UP-PGH
Central Laboratory?
Westgard urges that for a low quality performance such as that measured in our institution,
Internal QC procedures should be maximized, utilizing all of the Westgard Multirules in order to
detect laboratory errors and a minimum of 2-4 control measurements per run when using two control
materials is recommended. Our findings thus strengthen the obvious need to emphasize the
importance of regular and properly implemented QC practices. Participation in External Quality
Assurance Programs is likewise a relevant practice that is necessary in order to provide the laboratory
with an assessment of its performance compared with its peers.
Conscious efforts must be made to reduce the CV (taken from Internal QC data) and the Bias
(taken from External QC data) through more frequent instrument function checks, calibrations and
preventive maintenance. Other “non-statistical” Quality Control strategies should be employed and
these include: selection of appropriate standards, improving calibration procedures, standardizing
operator techniques, mechanizing manual steps in the process, and increasing operator training and
expertise. Moreover, with frequent assessment of laboratory performance using Six Sigma,
information on the performance of different methods and instruments will allow both laboratory
managers and manufacturers more informed selection and marketing decisions.
Local QC groups (e.g., PCQACL) should focus on formulating and promulgating a US CLIA
88 equivalent which will be adhered to by accrediting and licensing agencies in our country. This
may give even more meaningful interpretation of performance as the tolerance specifications that
shall be set for all analytes shall be applicable to and relevant in our setting. Future local studies on
Six Sigma principles may be made on the pre-analytic and post-analytic phases of laboratory testing.
Application of Six Sigma may be done with data gathered from other sections of the laboratory (e.g.,
Hematology, Immunology, Microbiology, et cetera). Prior to purchase and commercial use, laboratory
analyzers may be subjected to Six Sigma in order to determine its true analytic performance.
21
CONCLUSION
By telling us the level of quality of our laboratory performance, calculating the Sigma Metric is
the first step towards improving quality. Moreover, it emphasizes the need for more Quality Control
procedures and the importance of participating in External Quality Assurance Programs.
The analytic quality of ten (10) common biochemical tests (Glucose, BUN, creatinine, uric acid,
total protein, albumin, cholesterol, sodium, potassium and chloride) performed at the Chemistry
Section of the UP-PGH Central Laboratory was determined using Six Sigma principles. With a
benchmark for world class quality of 6-Sigma and acceptable process quality of 4-5 Sigma, only Uric
Acid testing achieved an acceptable Sigma metric at 4.69 using EQAS data. Creatinine, Total
protein, Albumin and Potassium testing are within 3-4 Sigma. Glucose, Cholesterol, BUN, Sodium
and Chloride testing all fall below 3-Sigma which is unacceptable quality. Using AQAS data, no
analyte tested above 4-Sigma. Using PCQACL and NRLB data, only Potassium is within 4-5 Sigma.
While the eventual Sigma value derived from the equation is dependent on how high or low the
tolerance specifications are (which are actually standards set by the US for their own use), the results
of this study do suggest significant room for improvement.
Several recommendations on how to improve the present quality of laboratory testing at the UP-
PGH were made as well as possible directions of further investigations using Six Sigma principles.
As a preliminary study, it is hoped that the conclusions from this research will draw more attention to
the issues of quality assurance in the country because the reliability and accuracy of the laboratory’s
output will impact heavily on patient diagnosis and management.
22
BIBLIOGRAPHY
1. Department of Health. Administrative Order No. 2007-0027. Revised Rules and Regulations Governing the Licensure and Regulation of Clinical Laboratories in the Philippines. Http://www.doh.ph.gov. 2007.
2. Westgard, James O., Ph.D.; Klee, George G., M.D., Ph.D. Quality Management. Henry’s Clinical Diagnosis and Management by Laboratory Methods, 21st Edition. 2007; 19: 485-523.
3. Westgard, J.O. Six Sigma Basics Process Improvement, Goals, and Measurements. Http://www.westgard.com.
4. Westgard, J.O. Six Sigma Quality Management and Desirable Laboratory Imprecision. Http://www.westgard.com.
5. Westgard, J.O. Six Sigma Basics: Outcome Measurement of Process Performance. Http://www .westgard.com.
6. Westgard, J.O. Six Sigma Quality Management and Requisite Laboratory QC. Http://www.westgard.com .
7. Westgard, J.O. AACC Expert Access: A Primer for Six Sigma. Http://www.westgard.com
8. Nevalainen, D.; Berte, L; Kraft, C; et al. Evaluating Laboratory Performance on Quality Indicators with the Six Sigma Scale. Arch Pathol Lab Med. 2000; 124:516-519.
9. Westgard, James O., Ph.D. and Westgard, Sten A., MS. The Quality of Laboratory Testing Today: An Assessment of Sigma Metrics for Analytic Quality Using Performance Data from Proficiency Testing Surveys and the CLIA Criteria for Acceptable Performance. American Journal for Clinical Pathology. 2006; 125.
10. Riebling, Nancy, M.B.B., M.S., M.T. (ASCP) and Tria, Laurel, M.S., S.C. (ASCP). Six Sigma Project Reduces Analytical Errors in Automated Lab. Http://www.mlo-online.com. 2005: 20-23.
11. Westgard, JO.; Basic Method Validation. Chapter 12: The Decision on Method Performance in Basic Method Validation. Madison, WI: Westgard QC, Inc., 1999: 125-134.
12. Westgard, JO, and Burnett, R.W.; Precision requirements for cost-effective operation of analytical processes. Clin Chem 1990;36:1629-1632.
13. Koch, D.D.; Oryall, J.J., Quam, E.F.; Feldbruegge, D.H.; Dowd, D.E.; Barry, P.L.; Westgard, J.O. Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system. Clin Chem 1990;36:230-233.
14. Westgard, J.O.; Sechafer, J.j.; Barry, Patricia L. Allowable Imprecision for Laboratory Tests Based on Clinical and Analytical Test Outcome Criteria. Clin Chem 1984;40:1909-1914.
15. Rej, R.; Jenny, R.W. How Good Are Clinical Laboratories? An Assessment of Current Performance. Clin Chem 1992;36:1210-1217.
23