Medical laboratory diagnostics and statistics

DOI 10.1007/s10182-007-0043-0

O R I G I N A L P A P E R

AStA (2007) 91: 399–406

Medical laboratory diagnostics and statistics

Christoph Berding · Wilhelm Kleider

Received: 31 January 2007 / Accepted: 9 September 2007 / Published online: 16 October 2007© Springer-Verlag 2007

Summary We review the statistical contributions to the development and productionof laboratory diagnostic assays. The tasks and facets of the role of a statistician indiagnostic industry are discussed.

Keywords Diagnostics · Industry · Consulting · Measurement · Assay technology

1 Introduction

The purpose of this note is twofold. Without attempting completeness we review typ-ical tasks in the field of diagnostic assay technology. In particular we focus on the roleof statistics for products and processes in research and development (R&D) and pro-duction. It becomes apparent that aside from the classical statistical work in clinicalstudies there are a number of statistical issues in biotechnology as well.

The second point we would like to address is the nature of the involvement ofstatisticians in typical industrial projects. In this context it will become evident thatdiagnostic industry needs long-term employed, in-house professional statisticianswith a broad background, not only in statistics but in the field of medical diagnos-tics as well. In fact statisticians in the diagnostic industry could expect a lifelongupgrading of their technical and statistical knowledge, and it is precisely this type ofopportunity to expand one’s knowledge that makes industrial statistics an exciting andintellectually rewarding endeavor.

The paper is organized as follows. After a brief introduction to the diagnosticindustry and Roche Diagnostics specifically we give a rough outline of related sta-

C. Berding (�) · W. KleiderDepartment of Biostatistics, Roche Diagnostics GmbH, Am Nonnenwald 2, 82377 Penzberg,Germanye-mail: [email protected]

1 3

400 C. Berding, W. Kleider

tistical issues. In particular we emphasize the importance of measurement relatedstatistics. The second half is devoted to some thoughts on statistical consulting. Weconclude with the description of two examples from the authors’ industrial practice.

1.1 In vitro diagnostic market and Roche Diagnostics

To distinguish laboratory and imaging diagnostics the technical terms in vitro and invivo diagnostics have been coined. To anticipate the economic background of the di-agnostic market a few numbers might be of interest: According to the WHO the worldhealth spending is about $2.5 trillion: 16% pharmaceutics, 1% diagnostics, and about83% other health care costs.

The current in vitro diagnostics (IVD) market, which is a small part of diag-nostics, has a volume of about 25 billion €, with a market share of Roche Diag-nostics of about 5 billion €. In fact Roche Diagnostics is one of the world leadersin IVD and offers a wide variety of products and services in all fields of medi-cal testing. According to the market requirements Roche Diagnostics is organizedinto four subdivisions: Roche Professional Diagnostics (RPD), Roche MolecularSystems (RMS), Roche Applied Science (RAS), and Diabetes Care (DC). WhileRAS has a strong focus on the investigation and provision of new biomolecu-lar methods and materials, RPD and RMS serve the IVD market with productsfor medical testing. RPD in particular directs its products and services to pri-vate labs, laboratory associations and central hospital laboratories, offering high-performance analysis systems to measure hundreds of different parameters in clinicalspecimens.

In total Roche Diagnostics has about 20 000 employees working at different sitesthroughout the world. At major sites with R&D and operations activities, statisticalworking groups have been established to support product development, productionand evaluation. Depending on the local needs each of these groups cover a broadrange of statistical services. In the next section we shall take a closer look at typicalactivities where statistics play an important role.

2 Processes in diagnostic assay technology and statistics

As usual in regulated industry diagnostics has well-defined processes for the devel-opment and operation of medical products. As an example, in the following we shallhave a brief look at three processes: the marker identification process, the standard-ization process, and the clinical trials process.

In research and development a marker identification process is in place to searchand identify biologically meaningful components, the markers. Depending on themedical question, i.e., the intended use, researchers conduct experiments and studiesto investigate the correlation between the clinical status (diagnosis, stratification) orthe therapeutic efficacy (prognosis, therapy prediction) and the corresponding markerresults. In fact the overall process from marker identification to marker validation isa very involved, multistep process requiring significant statistics contributions at allstages (Pepe 2001).

1 3

Medical laboratory diagnostics and statistics 401

Statistical expertise is required in designing the experiments and studies (Simon2006) as well as in study evaluations where the statistics used stretches from descrip-tive univariate statistics to the whole battery of multivariate classification methods(see, e.g., Hastie et al. 2001).

In operations a standardization process is in place to secure the accuracy of thediagnostic assay. For practical reasons an assay is based on different biologicalcomponents and operated on different measurement devices (Konnert and Berding2006). The standardization process requires statistical methods like linear and non-linear, parametric and non-parametric regression procedures, which allow for thecomparison of measurement methods throughout their measurement ranges (Cheng1999). The required analysis is often non-standard as specific variance situations,outliers, linearity issues and boundary effects need to be addressed. Above all thestatistical estimators and the related statistical information must be translated intomeaningful conclusions for an operations department. Additionally, in the standard-ization situation where measurement results are influenced by assay components,systems, operators, etc., it is evident that statistical models for variance analysisas well as statistical quality control play an important role (Mandel 1964, Searle1992). Finally it should also be mentioned that regulatory agencies require the dec-laration of the uncertainty of the routine diagnostic assay. As a consequence uncer-tainty has to be evaluated from the reference material of highest metrological levelthroughout the traceability chain to the calibrators used in the routine laboratory(GUM 1995).

As in pharmaceutical studies, the statistics support of clinical studies in diag-nostics addresses issues such as the definition of study purpose (intended use ofthe diagnostic assay), patient and control population, outcome (in diagnostic stud-ies the specification of a reference diagnostic or a gold standard), and the relevantdifference of the new method to the established clinical routine. On the other handthere are a number of important differences to the pharmaceutical trial like, e.g.,the specific technical assay performance issues. Moreover, due to the low preva-lence of many clinical conditions being diagnosed, studies are often planned andconducted retrospectively. Accordingly all kinds of bias, issues of sample selec-tion, and pre-analytical handling need to be considered carefully. For the tech-nical performance characteristics and the related statistics involved see for ex-ample the protocols provided by the Clinical and Laboratory Standards Institute(CLSI) protocols EP5A, EP17A, EP9A or the book by Haeckel (1992); more clin-ically related issues are discussed in the books by Pepe (2003) and Zhou et al.(2001).

However, central to all described processes and activities is the product, the diag-nostic assay, which is considered in more detail in the next section.

3 The diagnostic assay and statistics

The diagnostic assay is a biochemical test system allowing for a reliable and pre-cise determination of some biologically meaningful quantity, the analyte, within therelevant body fluids, e.g., serum, urine. It typically encompasses many components

1 3


and process steps which makes it very suitable for a design of experiment (DoE) ap-proach. To get a better understanding of assays, we give the main steps of a so-calledsandwich assay:

1. The medical specimen and an appropriate incubation buffer are pipetted into a re-action vessel coated with analyte-specific antibodies.

2. After binding of the analyte to the specific antibodies the remaining componentsof the serum are removed by a washing step.

3. Enzyme-labeled antibodies are added, which again bind specifically to the al-ready antibody-fixated analyte (sandwich principle).

4. A second washing step removes the excess of labeled antibodies.5. By adding a substrate an indicator reaction is started and the respective reaction

kinetic recorded.6. A net signal is derived from the kinetic curve, e.g., the amplitude or reaction rate

at defined time points.

From samples with known concentrations, i.e., the calibrators, a calibration curve isconstructed, which in turn allows for the determination of the unknown concentra-tion of a medical sample in question. It should be noted that modern immunologicalassays with a large measurement range exhibit a non-linear signal-to-concentrationrelation. This requires flexible and reliable techniques for non-linear regression (Se-ber and Wild 2003).

The detailing of the assay process highlights the important role of DoE in the iden-tification and validation of assay components like antibodies, buffers, concentrationof reactants, incubation temperature profiles, washing and mixing procedures, etc.Factorial as well as algorithmic, D, A, and G-optimal designs from early screening tooptimization and validation are necessary to fulfill all the needs. Examples for DoE inbiotechnology can be found in the book by Haaland (1989).

After completion of the assay development the assay’s performance character-istics have to be determined, and eventually in operations quality assured. Nat-urally most of the performance characteristics are evaluated as statistical estima-tors, which have to be determined in a reasonable design on the basis of theirstatistical properties. The most important performance characteristics are: preci-sion, accuracy with regard to a reference method, linearity, transport and calibra-tion stability, carry-over evasion, lower limit of blank and lower limit of detection.Most of these are regulated by protocols of the CLSI or company internal stan-dard operating procedures (SOP). Indeed statistics makes important contributionsto the SOPs, which are also subject to review by external bodies like, e.g., theFood and Drug Administration, Paul Ehrlich Institute, Landes Gewerbeanstalt andothers.

While the last two sections review some of the statistical topics of relevance in di-agnostic products and processes, Sect. 4 is devoted to the question: how is statisticalexpertise delivered to the benefit of products, projects, and processes? As a matter offact these points are of utmost importance for the impact of statistics and the receptionof the statisticians in an industrial project landscape.

1 3


4 Industrial statistical consulting

Statistical consulting in diagnostics is a dynamic process with many aspects. Herewe would like to focus on two issues: (1) statistical competence, and (2) consultingcompetence.

4.1 Statistics

It is evident that industrial statisticians should be comfortable with the major statisti-cal methods and concepts. However, the key ability is to choose the statistical conceptwhich is most appropriate for application in the particular case at hand. So on onehand the statistical consultant must have a sensitive ear to the client’s message, andon the other hand he or she must be open minded in their methodological approach.It may be simple and straightforward or a mathematically involved method, a paperand pencil or a numerically extensive solution, a Bayesian or Frequentist approach,etc. In the author’s view this key message is captured nicely in the book by van Belle(2002).

Sometimes new statistical methods are required in the development of innovativeproducts like multimarker or gene-array technologies. These fields require exper-tise not only in classical multivariate statistics such as multiple testing, discriminantanalysis, and logistic regression but also in the machine learning approaches suchas support vector machines, classification and regression trees, and artificial neuralnetworks. In situations with unfavorable variable-to-subject ratio, i.e., many observ-ables/few samples, simulation and bootstrap techniques play an important role (Efronand Tibshirani 1993). Due to the broadness and dynamical evolution of the statisticalfield, consultants have to expand their knowledge continuously.

4.2 Consulting

The process of statistical consulting has been described by a number of books (see,for example, Derr 2000; Cabrera and McDougall 2002). Hereafter we briefly summa-rize some key points and mainly comment and illustrate the issue by examples frompractice.

Looking at the consulting situation from a bird’s eye view, the following majorsteps can be identified: (1) The client’s problem is posed and analyzed. (2) An ap-propriate statistical model is developed and formulated. (3) The statistical solution isworked out, and (4) finally translated into the client’s solution. From this sequenceof steps it immediately transpires that the statistician needs communication skills toreally understand what the client’s problem is. Thereby a full understanding often re-quires much more than an understanding of the isolated problem alone: additionallythe statistician needs background information on the project such as medical issues,project resources (lab and personal capacity, budget, etc.), intellectual property (IP)issues, sample availability issues and so on. For all these reasons particularly for largeprojects the statistician should be a regular project team member.

Generally speaking the issues to be clarified in the client-statistician interaction arethe understanding of the mutual roles, the deliverables, the project conditions, e.g.,

1 3


timelines, the role of other project members, security or confidentiality constraints,limitations or requirements regarding the statistical methods, and mode and style ofcommunication. While at the beginning of the projects the key issues center aroundclarifying the objectives, design, randomization and sample size, in later phases is-sues of data collection, -coding, -management and quality assurance take over. Withinthe final phases of the discussion the focus turns to the exploration of the data, dataconsistency checks, statistical model building, inference, and conclusions. Finally theresults have to be interpreted and evaluated in view of other results available andprepared for communication to other users.

Although the situation is improving, two frequent problems should not be sweptunder the carpet: (1) In practice statisticians are not always involved from the verybeginning of a project. As a consequence data could be gathered inappropriatelyand research questions could not be answered. The term post mortem analysis hasbeen coined to describe this statistically unpleasant situation. (2) The easy access tospread-sheet programs has lead to large collections of unstructured data. As a conse-quence statistical analysis is often preceded by lengthy and painful data managementoperations.

5 Examples

Finally, here we would like to review two illustrative examples. The first exampleproject was completed by the development of software years ago, but due to its per-manent application in the routine there is a stream of new requirements and ideas tobe implemented as well. The second project is a rather typical statistical consultingexample, which has also led to a new software solution. In fact software serves asa trailblazer to the application of statistics in the field.

5.1 Calibration and calibration design

Due to inevitable alterations in assay chemistry and analyzer hardware componentsdiagnostic assays need to be calibrated. In calibration manufacturer-provided sampleswith known concentrations, i.e., the calibrators, are used to derive an often non-linearcalibration curve. A successful calibration has to fulfill a number of requirements:(1) The functional model describing the signal-to-concentration relation should beflexible enough to cover a wide range of different assays and assay calibration situa-tions. (2) The calibration procedure should balance between keeping the well-definedassay-specific curve shape and meeting the actual measured calibrator signals as closeas possible. (3) The fitting procedure of the calibration curve should allow for anappropriate treatment of the assay-specific variance situation as well as the consider-ation of assay-specific requirements, e.g., a closer fit at medical decision regions. (4)The calibration concept should provide the means to reduce the customers investmentin calibration, e.g., the number of calibrators, the frequency of calibrations, etc.

This project has been presented to the authors by R&D colleagues from assaydevelopment. The project team involves coworkers from marketing (assay speci-fications and customer requirements), assay development (technical feasibility, lab

1 3


experiments), operations (scale-up for production, standardization issues) and bio-statistics. In the course of the project a number of statistical techniques have beenapplied: model selection and evaluation, procedures for non-linear regression withand without weighting, and calibration design optimization. The latter task has beena project on its own, and as for a given calibration relation and variance situation anoptimal design has to be derived. In calibration design optimization the non-linearregression procedure as well as the specific assay performance constraints have tobe taken into account. Monte Carlo simulation which generates a ranked list of de-sign alternatives is the method of choice to solve this problem. The success of theapproach eventually leads to the development of a customized software for the R&Droutine.

5.2 Evaluation of analytical methods

The second example refers to a fundamental evaluation task in diagnostic assay tech-nology: the comparison of different analytical methods. More specifically in methodcomparison the degree of equivalence (or difference) between two measurementsystems executed on the same set of medical samples is analyzed and evaluated. Situ-ations where method comparison studies occur are the evaluation of a routine methodin relation to a designated reference method, or the evaluation of the homogeneityof different reagent lots used within a routine diagnostic method. From a statisticalpoint of view a linear model between both methods is anticipated and depending onthe situation a standard linear, orthogonal, or robust regression is executed. Withinthe routine typically the lab researchers focus on the intercept and slope of the esti-mated regression line and if these are within acceptable limits the two methods areconsidered equivalent.

In this context the following problem has been presented to the biostatistics groupby a product quality team. A reagent lot has been found outside the intercept/slopespecifications. As a consequence this production lot cannot be used, which eventu-ally leads to a substantial loss of time and money. However, from visual inspectionmembers of the quality team were not convinced about the out-of-specification clas-sification. Indeed the feeling of these colleagues could be confirmed by a carefulstatistical analysis. This analysis reveals that the individual deviations are well withinthe required limits and the reason for failing the acceptance criteria is a slight butirrelevant deviation from linearity.

The project statistician was able to demonstrate that an evaluation based on theconfidence intervals of a piece-wise, local, robust regression model provides a muchmore adequate and insightful analysis of the situation (Geistanger 2007). Finallya customized software solution for assay developers has been developed which allowsfor a convenient access for all R&D coworkers to this useful statistical tool.

6 Conclusion

Diagnostic industry offers a wide variety of statistical tasks and a challenging envi-ronment for statisticians. Statisticians who consider joining diagnostics should love to

1 3


solve real problems in an highly interactive environment and enjoy life-long learningin statistics as well as in medical diagnosis.

References

Cabrera, J., McDougall, A. (2002) Statistical consulting. Springer, Berlin Heidelberg New YorkCheng, C.L., Van Ness, J.W. (1999) Statistical regression with measurement error. Oxford University

Press, New YorkCLSI (2007) http://www.nccls.org. Accessed 1 Jan 2007Derr, J. (1999) Statistical consulting: a guide to effective communication. Duxbury, Belmont, CAEfron, B., Tibshirani, R.J. (1993) An introduction to the bootstrap. Chapman & Hall, LondonGeistanger, A. (2007) Local regression in method comparison. Manuscript in preparationGUM (1995) Guide to the expression of uncertainty in measurement. International Organization for

Standardization (ISO), GenevaHaaland, P.D. (1989) Experimental design in biotechnology. Dekker, New YorkHaeckel, R. (1992) Evaluation methods in laboratory medicine. VCH, WeinheimHastie, T., Tibshirani, R., Friedman, J. (2001) The elements of statistical learning: data mining, inference

and prediction. Springer Series in Statistics. Springer, Berlin Heidelberg New YorkKonnert, A., Berding, C. (2006) Uncertainty calculation for calibrators and controls of laboratory diag-

nostic assays. Clinical Chemistry and Laboratory Medicine 44, 1075–1082Mandel, J. (1964) The statistical analysis of experimental data. Dover, Mineola, NYPepe, M.S., Etzioni, R., Feng, Z., Potter J.D., Thompson, M., Thornquist, M., Winget, M., Yasui, Y.

(2001) Phases of biomarker development for early detection of cancer. Journal of the NationalCancer Institute 93, 1054–1061

Pepe, M.S. (2003) The statistical evaluation of medical tests for classification and prediction. OxfordUniversity Press, New York

Searle, S., Casella, G., McCulloch, C. (1992) Variance components. Wiley, New YorkSeber, G.A.F., Wild, C.J. (2003) Nonlinear regression. Wiley Series in Probability and Statistics. Wiley,

New Yorkvan Belle, G. (2002) Statistical rules of thumb. Wiley, New YorkSimon, S. (2006) Statistical evidence in medical trials: what do the data really tell us? Oxford University

Press, New YorkZhou, X.H., Obuchowski, N.A., McClish, D.K. (2002) Statistical methods in diagnostic medicine. Wiley,

New York

1 3

Documents

Medical laboratory diagnostics and statistics