Guidelines for Interlaboratory Testing Programs

R E P O R T FOR A N A L Y T I C A L C H E M I S T S

Guidelines for Interlaboratory Testing Programs RAYMOND H. PIERSON AND EDWARD A. FAY

U. S. Naval Ordnance Test Station, China Lake, California

Interlaboratory testing demands careful and extensive planning in order to achieve worth-while and satisfying results. This article, which was presented at the 135th National Meeting of the American Chemical Society at Boston, reviews the statistical and practical problems of planning and conducting an interlaboratory testing program. Possible benefits from such a program are enumerated, and possible pitfalls are pointed out. It is recommended that the planning and coordination of the program should be the responsibility of a single chairman. Qualifications for such a chairman are outlined and practical directions are given for his guidance in organizing the work. Methods of achieving adequate replication, randomization, and symmetry are suggested. Rejection of suspect divergent values at the operator level by a statistical criterion (Q test) is advocated. Attention is called to some statistical aspects of cooperative testing work, such as considerations of bias and confounding, that differ from those commonly encountered in a single laboratory.

TNTERLABORATORY tests, coopera-•*· tive test programs, or "round robins"—as the programs are frequently but rather loosely termed— have been and probably will continue to be extensively used by analytical chemists and others in technical fields. If an inexperienced group plunges into a round robin without competent guidance and good planning, the chances are great that the results will be disappointing. Indeed, the results from interlaboratory tests have turned out to be worthless on so many occasions that many people have formed the opinion that the methodology itself has no value. This paper is in defense of the. technique and advocates the use of cooperative testing programs, but only after considerable thought has been given to the

design of the experiment. The interlaboratory test is a good tool when properly applied, but it is an expensive procedure and subject to many pitfalls.

Both statistical and practical pitfalls are involved. Many good references provide statistical background and the fundamentals of statistical design applicable to cooperative testing (6, 7, 10, 19, 22), but no book has been written that deals exclusively with this subject and only a few good magazine articles (14,15,20) and pamphlets (2, 3) bear directly on the topic.

This article extracts statistical information from the standard references and attempts to organize it for the use of participants in interlaboratory programs, especially those who have not had much previ

ous experience. The paper includes concepts derived from experiences of the Joint Army-Navy-Air Force Panel on Analytical Chemistry of Solid Propellants and is intended as a practical guide for organizing and conducting round robins.

The key person in a cooperative testing program is its chairman. Rarely is it possible to find an ideal chairman with all the necessary background in analytical chemistry and statistical techniques. The main purpose of this paper is to provide guidance for the chairman who does not consider himself an expert at the task, but who, nevertheless, finds himself selected to carry out the project.

In the past a good deal of effort has been devoted toward developing standardized approaches to co-

VOL 31 , NO. 12, DECEMBER 1959 · 2 5 A

REPORT FOR ANALYTICAL CHEMISTS

R A Y M O N D HENRY PIERSON is a research chemist in the f ie ld of rocket propel lants a t the U. S. Naval Ordnance Test Stat ion, China Lake, Ca l i f . He was born in Chris-man, I I I . , November 24, 1897. From 1920 to 1943 he served as a chemist, pr inc ipal ly in the analyt ical phases, for the Ar thur R. Maas Chemical Co. , Union O i l Co. , G i l -more O i l Co. , Smith-Emery Co. , A l l i e d Plastics, and Adhere, Inc. In 1942 he received his A.B. f rom the University of Southern Ca l i fo r nia (magna cum laude) .

In 1943 he became Laboratory Superintendent a t the U. S. Naval Shipyard at Terminal Island, Cal i f . , and in 1947 joined the research staff a t the Naval Ordnance Test Stat ion.

He is interested in visible, ul trav io let , and infrared spectrophotometry and x-ray d i f f ract ion, absorpt ion, and fluorescence methods as appl ied to problems of ordnance chemistry. He is keenly interested in statist ical design and interpretat ion of experiments.

He is a member of the Amer ican Chemical Society, Amer ican Associat ion for the Advancement of Science, Scientif ic Research Society of Amer ican Society for Testing Mate rials, and is past chairman of the Joint Army-Navy-A i r Force Panel on Ana ly t i ca l Chemistry of Sol id Propellants.

2 6 A · ANALYTICAL CHEMISTRY

operative work. Because the field is so complex, this paper advocates a more flexible approach; it provides general guidelines and supplements these with several practical check lists.

Possible Benefits from Interlaboratory Testing

For an interlaboratory testing program to be successful, all participants must have a clear conception of the benefits to be derived from the program. Misunderstanding or vagueness regarding the possible benefits has frequently led to disappointment in the results.

What then can be gained by subjecting a method of analysis to an interlaboratory test? One important dividend is that each participant can see how well or how poorly his laboratory performed in comparison with a number of other laboratories engaged in similar analytical work. If his laboratory did well or about average, he gains confidence in his own work or that of his assistants. If his laboratory showed wide variability or a high divergence from the general averages, he knows that improvement is needed in techniques, equipment, or supervision at his establishment. These benefits may be had merely by examination of the raw data or graphs showing means and ranges and without benefit of more than elementary statistics.

As a second benefit, interlaboratory testing can sometimes be used for sharing the workload when it is desirable to compare a relatively large number of methods or to test a new procedure against several others. This is especially apropos when different instrumental approaches are to be compared, and the required equipment is spread among the various participating laboratories. A block diagram can be set up so that each laboratory tests one specific procedure against one or two others.

A third advantage of the interlaboratory test is that it provides a cross-sectional or unprejudiced estimate of the value of a proposed method. The originator of a

method often obtains better results with it than others are able to achieve. There may be several reasons for this difference in evaluation. A properly conducted interlaboratory test may reveal that the originator was too optimistic about the precision and accuracy readily attainable (some personal bias existed), or that considerable practice with the new method is required for good precision.

The originator of a method is likely also to be too optimistic regarding the time required for the analysis. A fourth advantage of the interlaboratory test is that it can provide a more realistic average for the time factor.

Fifth, when a number of methods are compared, the cooperative work will also show which methods the participants prefer with respect to over-all convenience, space requirements, availability or cost of equipment, type of operator required, and safety. These are factors that the originator may overlook or suppress.

Sixth, the program will show whether the descriptions of the methods are adequate or need improvement.

In addition to the specific advantages mentioned, it is frequently to be expected that carefully designed interlaboratory work will yield statistical evaluations of precision with respect to a number of factors such as laboratories, operators, days, and levels of ingredient. Similar evaluations with respect to accuracy may be obtainable if standard samples or standard reference methods of known accuracy are available.

Possible Pitfalls in Interlaboratory Testing

Failures in cooperative testing programs may be caused by the following major factors:

Benefits derivable from the program not fully understood

Chairman not fully qualified for the task and unaware of some of the requirements

Objectives not clearly stated and understood

Improper selection, preparation, or packaging of samples.

Inadequate written instructions from


the chairman to the participants Inadequate statistical design Inadequate statistical evaluation

The first of these hazards has been discussed in some degree in the preceding remarks. The following sections cover the other six hazards and offer a set of guidelines for the chairman.

Chairman's Qualifications and Responsibilities

Qualifications for a chairman which might be considered as minimum requirements are :

At least one university course in elementary probability theory and statistical methods

Familiarity with the t test for comparing two means

Familiarity with the F test for comparing several means or two variances

Ability to use a test for homogeneity of several variances, such as the M test (75, Tables 31 to 33)

Familiarity with a test (such as Dixon's Q test) for rejection of data

Ability to carry out relatively simple analysis of variance calculations

Acquaintance with most of the references of this paper, especially 1-3, H, and 20

If the chairman is not a statistician, he should avail himself of the consulting services of a statistician, preferably one in his own organization

In initiating an interlaboratory test, the chairman should assume the responsibility for concluding the project by the preparation and distribution of a complete, formal, written report. A check list for this report is helpful in avoiding omission of essential details. The report may contain all of the following i tems:

1. A title (and sometimes a serial number)

2. The report date 3. Security classification, if required 4. The name of the chairman and

his activity 5. Acknowledgment 6. A list of participating activities 7. A clear statement of the objec

tives in an itemized form 8. Description of the samples used,

including identification, form, and source

9. Description of test methods 10. A summary of the findings and

a statement about each of the ob

jectives, indicating how well the program fulfilled each objective

11. All the raw data and suitable collective or summary tables

12. Appropriate graphs—e.g., means, ranges of replicates or confidence limits for the various laboratories

13. The statistical evaluation of the findings

14. Comments from participants about samples, test methods, forms used for collecting the data, and statistical design

15. Recommendations, when appropriate

16. An appendix showing instructions issued and data forms used

17. A list of literature cited

I t is the chairman's responsibility to make sure t ha t the needed elements will be available for his report. The report for an interlabora tory testing program should include a number of elements usually omitted from journal publications. For example, articles for ANALYTICAL CHEMISTRY will con

tain either a table or a graph but not both when they show the same thing, and it is usually desirable to omit much of the detailed data . On the other hand, it is strongly recommended tha t for reports of interlaboratory tests items 11 and 12 always be included. Space in such reports is not a serious limitation. The part icipants often wish to see all the values sent to the chairman. They are entitled to this complete information and also should have the opportunity of examining all the statistical calculations and evaluations which the chairman has made. When the group meets for discussion of the results, it may be advantageous in some cases to use the tables and in others to use the graphs. I tem 14 has great value in improving the proposed method of analysis as a direct benefit of the cooperative work and item 16 is of value in the planning of future test programs.

Object ives

The objectives of the program should be defined and agreed upon by the part icipants in the early stages of planning, before samples are chosen and shipped. Guidance is available in references 2, S, 6, 14, 20, a n d # J .

EDWARD ALLEN FAY, born in Berkeley, Cal i f . , August 13, 1918, has been a mathematical statistician in the Statistics Branch, Research Department, U. S. Naval Ordnance Test Station, since 1950. He received his A.B. (mathematics) f rom the University of Cal i fornia at Berkeley in 1939 and his A . M . (mathematics) a\ Harvard University in 1941.

A f te r graduating, he served for one year each as a teaching assistant in mathematics at the University of Rochester and as an inspector of ordnance material at the Rochester Ordnance District. He then served in the Army for three years. From 1946 to 1950 he was a graduate student in mathematical statistics at the University of Ca l i fornia at Berkeley.

His chief area of interest is mathematical statistics, including combinatorial theory, stochastic processes, and techniques of sampl ing inspection.

He is a member of the American Mathematical Society, Mathematical Association of America, Institute of Mathematical Statistics, Association for Symbolic Logic, and the Scientific Research Society of Amer ica.

VOL. 31, NO. 12, DECEMBER 1959 · 2 7 A

Courtesy Esso Research & Engineering Co.

THE IDEAL PRESS FOR MAKING KBR

PELLETS FOR INFRARED SPECTROSCOPIC

ANALYSIS I Also for forming pellets for x-ray and I other types of spectroscopic analysis

9 2 0 Ton Capacity Hydraulic Press

I · Accurate alignment in op-I eration

• Guided moving platen • Sturdy 3 column construction • Even pressure distribution for

uniformly dense pellets • Self-contained 9 Bench mounted 9 Hand operated • 9 " χ 9 " platen area 9 2 2 " max. vertical opening,

adjustable

A multi-purpose unit with qu ick ly adjusted ve r t i ca l opening, by handwheel, permits many different set-ups in minimum time.

Used around the world for RESEARCH, CONTROL & TESTING.

I Available accessories include: Heat-I ing and Cooling Platens; Temp. Con-I trois, Auxiliary low pressure gauges; I Fast air closing; Extruding units; I Testing units, etc.

I Avai lable in 3 0 and 50 ton sizes.

Write for Bulletin.

LOOMIS ENGINEERING & MANUFACTURING CO. Dept. A , Route 4 6 , Ca ldwel l , N . J.

Circle No. 58 on Readers' Service Card



Objectives of various cooperative programs may differ a great deal. For example, Willits (21) and Wernimont (20) describe contrasting approaches that have widely different objectives. In one case reported by Willits, a number of laboratories made determinations on two types of samples representing two chemical species and two levels of the ingredient (element), using two basic procedures but with variations of their own choice (equivalent to many methods). The primary objective was to select the best method or methods from the ones examined. In a case cited by Wernimont, one sample was tested by eight laboratories using one basic procedure with no choice of variation of techniques or equipment, and using two operators in each laboratory, each operator performing two replications of the tests on each of three different days. In this example, the objectives were to measure the precision of the method for one common sample and to assess the laboratory, operator, and day effects. In the Joint Army-Navy-Air Force Analytical Panel work, the approaches given in both of the references cited above have been used—sometimes first the survey approach described by Willits, followed by the more closely controlled experiments advocated by Wernimont. At times, the design of a single round robin has combined features of both approaches.

Samples

Every chemist realizes that a test can be no better than the sampling. Yet, faulty samples have ruined many cooperative testing programs. Usually it is desirable that particle size distribution not constitute a variable. Hence, the santple should be uniform, and the particles sufficiently fine to have no influence on test results. Instructions may need to take into account the possibility of size segregations that might occur during shipment. They may need to warn the participants to use the sample as received (omitting some process, such as grinding, that would ordinarily be used), or to process the sample in some specified way before use—e.g., grinding

or drying. Packaging for materials that can either lose volatile ingredients or take on moisture during shipment is a critical item. Changes caused by the aging of samples must be considered and directions given that will eliminate such effects or minimize them.

Chairman's Instructions to Participants

The chairman should issue clear instructions covering such important details of interlaboratory testing as the following:

Methods. I t is the chairman's responsibility to see that all participants are supplied with correct descriptions of the method (s) to be tested. These descriptions must be uniform for all concerned and as well written as possible.

Practice Period. Frequently in interlaboratory testing a new method does not meet expectations with respect to precision as compared with older methods. I t also happens that sets of replicates by new procedures will need replacement (alternative) values when a test for rejection of data is applied. Lack of experience with the new method is responsible for some of the variability encountered. Hence, it is desirable to ask participants to practice with the new technique until some familiarity with it is achieved before making the determinations to be reported for the interlaboratory test. If the official samples are in short supply a practice sample should be used. Practical limitations of time and expense may determine the extent of the practice period according to the operator's judgment.

Moisture Determination. Frequently also it is essential that the chairman instruct each participant to make a moisture determination on his samples by a prescribed method, and to calculate values for other ingredients to a moisture-free basis.

Sample Weights. Sample weights for individual determinations have sometimes been set by chairmen at an exact number of grams (no variation permitted) in order to avoid errors of recording weighings and to simplify the work


of checking analytical calculations. These two advantages of uniform spécifie weights are overshadowed by two disadvantages, namely, the introduction of bias (either conscious or unconscious) and the loss of a device for detecting systematic error in the procedure under test (22, p. 40). In the authors' opinion, uniform weights should never be used in cooperative work. It is permissible in most' cases to suggest a small range or two small ranges, one being about double the other.

There are cases in which the range of weight must be prescribed by the test method itself. For example, in the determination of heat of explosion, density of loading— i.e., relationship of size of sample to size of the bomb—influences the \7alues obtained and must be held within narrow limits.

Data Forms. Forms for the collection of data should be designed, tested by use in the source laboratory, and distributed in triplicate to the participants by the chairman. One form can then be returned to him for his report, and the other two retained for use by the participating laboratory.

In addition to spaces for the recording of test data, these forms might also include provision for the noting of the following information:

Assignment of operators Operator's time Total elapsed time Deadline date for return of data to

chairman Remarks or suggestions on testing

procedure Departures from specified procedures

If the identities of participants are not to be coupled with their data, the chairman will assign code numbers to be used on data sheets and in his report.

Committee D-13 of the American Society for Testing Materials has recommended a standard (although somewhat flexible) form for data collection (2). In the analytical panel work, the forms have varied widely depending upon the type of problem and a standard form has not been feasible. Examples of forms used in the first twelve round robins of the Analytical Panel may be seen in a collective report by Pierson (17).

Rounding Off the Data. Good rules for rounding off data for reports are conveniently available (1, 7, 19, 22) and may be applied by the coordinator of a cooperative test program in the final stages of report preparation. Rules for data collection in cooperative testing are different! The chairman should specify in his instructions exactly how many digits to the right of the decimal he wishes the analysts to report for each test value obtained. He should set this at the level of the digit that represents the last place the analyst can estimate. If the rounding off is done prematurely —at the operator level—there is a high probability that the greater part of the measures of precision will be thrown away, and that efforts put into the work will be completely wasted. Early rounding off is then to be rigorously avoided in interlaboratory work until the statistical evaluations have been made ; it is much better to have too many decimal places than too few (22, p. 7) !

Table I is a simple illustration of the complete loss of statistical measures of precision when one decimal place is dropped prematurely. The values of 18 replicate moisture determinations, 6 for each of three laboratories, were estimated and reported to the third decimal place in the table as shown. Observe that if the data were rounded off to two decimals—the place to which the results are customarily reported when they are to be used only for calculating percentage values of other ingredients to a "dry basis"—every replicate would have exactly the same value, 0.02%, all the means would be 0.02%, standard deviations for all three laboratories would be zero, and no statistical evaluations would be possible. With the values shown to three decimals, precisions obtained by laboratories A and Β are equal and are better than that of laboratory C, and the three means are different. The statistical significance of these differences can be examined by appropriate tests.

Statistical Design

Simple vs. Complex Designs. In

1111111111111111111111111 Range 100 cm — Reads to

0.01 mm. Scale engraved in 0.5 mm direct on solid Ws inch

cross section bar.

CATHETOMETER

Reference Page 11 of the Ealing Catalog for complete description of this outstanding instrument. This is just one of the more than 300 instruments for the physical sciences offered exclusively by Ealing.

THE EALING C O R P O R A T I O N 35 University Road Cambridge 38 Massachusetts

a Baird Atomic Affiliate


VOL. 3 1 , NO. 12, DECEMBER 1959 · 2 9 A

OHAUS TRIPLE BEAM BALANCE Model 750-S

FOR WIDEST SELECTION, GREATEST CAPACITY SPECIFY OHAUS. THERE IS A MODEL TO FIT YOUR EVERY NEED.

NEW BOX END ft!AM High strength, die cast aluminum alloy, with ends cross braced.

SLIDING POISE with center indicating panel, insures rapid correct readings. Eliminates secondary beam oscttlations.

ANGLE-VIEW

BEAMS stainless steel relief etched graduations for easy reading.

END READING DEVICE pointed beam registers against graduated dial eliminating parallax error.

ATTACHMENT WEIGHTS

extend capacity to 2610 grams with this handy set.

List Price $5.00

WHITE FOR FREE BHOCHUBE

SCALE CORPORATION 1050 COMMERCE AVE. UNION, NEW JERSEY




research work, the use of experimental designs that will test a relatively large number of variables in a single-experiment framework will often yield high returns over simple or "classical" designs. In the authors' opinion, designs used for in-terlaboratory testing should be kept at a relatively simple level. When a number of different laboratories try to follow complex designs (containing too many objectives), some of the participants are likely to become confused and serious errors may be made.

Often the desire arises to test a large number of types of samples, each at more than one level of an ingredient, perhaps more than one ingredient, and each ingredient by several test methods. In one research laboratory with good statistical background a single complex design can be advantageous, but for a number of laboratories a better answer might be to split the work into more than one cooperative testing program.

Confounding. If a statistical design is such that it is not possible to estimate separately the effects of two (or more) factors, but only their combined effect, these factors are said to be confounded with each other. In other words, the design fails to provide the means for separating the effects of the confounded factors.

Sometimes confounding is purposely used (6). Sometimes it inadvertently enters into the design of experimental work and disrupts statistical evaluations. In cooperative test work, care should be taken to avoid such interference. Confounding of time with other variables can often be eliminated by careful randomization of these other variables (samples, methods, replications, etc.) with respect to time. Thus, instead of introducing time as an effect to be measured, one may use randomization to reduce the size of the experiment.

Confounding of operator and laboratory effects under the single heading "laboratory" has often been purposely done in the Analytical Panel programs to avoid the expense of multiple operators for each test within each laboratory. If it is desired to compare several methods

for more than one level of ingredient on more than one type of sample with adequate replication in a number of laboratories, adding an operator factor to the many already present may make the entire program unwieldy and prohibitively expensive. The operator factor is then considered the one that can be sacrificed (confounded with laboratory) with least damage to the results. A recent article by Youden (24) advocates simplicity of design within each laboratory and supports the plan of excluding operator as a factor.

Unintentional confounding of operators within the various laboratories engaged in a round robin is to be carefully avoided. As indicated by the statement under Data Forms, the chairman should regulate the assignment of operators within laboratories for each group of tests. For example, if two instrumental methods are being compared, the results would not be statistically valid if some laboratories used a single operator for both instruments while others used two operators. Operator would then be confounded with laboratory in the first case and with method in the second.

Randomization. I t is basic to good statistical evaluation that the design of the experiment include randomization, replication, and symmetry, all in the proper degree. The chief purpose of randomization is to prevent confounding of factors, particularly of time with other factors. Statistical evaluations may be invalid unless randomization is properly applied. The difficulty of getting all participants of a cooperative program to appreciate the importance of randomization provides some ammunition for those who would do away with interlabo-ratory testing altogether. It is a source of much disappointment to a chairman if some of the participants carry out all the instructions except those for randomization. It cannot be overemphasized, therefore, that the chairman should do everything he can to get the randomization idea across. Indeed, he should never leave the randomization up to the participants, but

$ 1 9 . 1 5 WEIGHTS

OHAUS

should supply each of them with a prescribed sequence and an earnest plea that it be followed exactly. The data form should reflect the desired sequence whenever possible, and should always be constructed in a manner that will make entries easy when the sequence is followed. Sometimes it is impossible to apply a complete randomization scheme, as, for example, when one chemist is the expert on one type of equipment while another man at the same activity is the logical choice for another method or piece of equipment. Then a partial or compromise randomization is all that is practical. Sometimes all the samples for two different operations can be weighed in a random fashion at the same period, thereby eliminating at least the possibility of time effects on the individual portions from the same sample.

Replication. Too much replication causes a waste of effort; too little fails to give the required sensitivity. Some guidance on replication is given in the American Society for Testing Materials publication D 1421-56T (3). When some practice has been had with a method and some knowledge exists as to the precision which it can show, then it may be found that only two or three replicates will be sufficient to provide the sensitivity desired in an interlaboratory test. It should be remembered, however, that programs conducted at this low level of replication are without benefit of a good means for detecting and rejecting faulty data at the operator level as described in the following section. In the work of the Analytical Chemistry Panel mentioned above, five or six replica-

• RATE METERS · AMPLIFIERS · MONITORING SYSTEMS »

Victoreen Model 770 Radcell

DOS1MLT T. F S · Ζ h

for gamma irradiation of solids, liquids, gases

continuous temperature control

200,000 r/hr or more* Uses for Victoreen's new Radcell high intensity gamma irradiator are virtually unlimited for research in the fields of chemistry, petroleum and biology. No specially shielded room is required. Exclusive turret loading principle assures operator safety. Source is surrounded by two metallurgical-ly bonded coils: one provides for flow of fluid samples around source, the other provides for flow of refrigerated or heated liquids for precise temperature control of sample. A-ei9A

*Us~'ng 1000 curie Cobalt 60 source. Shielding is adequate for 1,000,000 r/hr when loaded with sufficient Cesium 137.

Write for your copy of Form 3024-9 "Victoreen Radcell Gam-ma Irradiator"

The Victoreen Instrument Company 5806 Hough Avenue · Cleveland 3, Ohio

Export Department, 735 Liberty Street, New York 6, Ν. Y, Cable: TRILRUSH, New York

ÂN1MA I R R A D I A T O R S » P O W E R S U P P L I E S » Ε ̂ O . 7 - C M ET L RS Circle No. 87 on Readers' Service Card

WORLDS FIRST

NUCLEAR COMPANY

VOL. 31 , NO. 12, DECEMBER 1959 » 3 3 A

Table I. Illustration of Rounding Off Data

Laboratories A Β C

Replicates Moisture, %

1 0.018 0.024 0.024 2 0.018 0.024 0.023 3 0.017 0.023 0.020 4 0.017 0.023 0.019 5 0.017 0.023 0.018 6 0.016 0.022 0.016

Mean, x6 0.0172 0.0232 0.0200 Est . std.

dev., s 0.000753 0.000753 0.00303

π

Ο

Ο π Η η

ρ

•ν

ο

Ο 2 π Η η

!/ϊ

VI

Ο

> r η 3 </> 3 J>

η S Η χ

> 2 -ο r -π η 7.

2

r

ιτ m

(/! !/ϊ

C < π < 2 Η Π 3 (Si

Ό Ο !£ S

3 (/ι

Ο > 2 2 > 3 3 > Ο > -Ι ο 3 {/>

Τ) Ο £ π 3 Φ C TJ -ο Γη

71

> Ζ > Γ" ·< in

VICTOREEN

RAD CELL®

S9BSQ

REPORT FOR ANALYTICAL CHEMISTS The shell of the P-K Type (LV) Lab Blender can be charged from either side by the quick removal of either of two dust- and watertight covers.

YOU GET THESE 3 DIFFERENT MIXING ACTIONS.. .

LIQUID-SOLIDS TYPE - To disperse and blend, uniformly, liquids of any viscosity with dry solids, just insert the wire cage liquid-feed bar assembly. Handles liquids from minute quantities up to 40% of total weight. Liquid is introduced through hollow shaft of bar. Even critical formulations blended in 5 to 10 minutes. INTENSIFIER TYPE — To get intensive mix· ing of hard-to-blend dry solids, or to break up agglomerates, use wire cage bar assembly. Average blending time for dry formulations: 3 to 5 minutes. FOR GENTLE PRECISION BLENDING — Simply remove wire cage bar. Unique blending action will not cause attrition or break down even the most delicate crystals. Average blending time: 3 to 5 minutes.

This new P-K Type (LV) Lab Blender owes its versatility to the design of the removable, spring-loaded liquid-feed bar assembly. Liquids and solids are prevented from entering the bearing housings by an exclusive, patented seal. Standard (LV) Lab Blenders are available in 8 and 16 quart capacities, in transparent Lucite or stainless steel. Interior of shell and stainless steel bar assembly can be cleaned, clinically, in minutes. Operates on 115 ν AC.

Units can be shipped from stock. For complete technical information and prices, write to PK's Chemical and Process Equipment Division, 1012 Hanson Street, East Stroudsburg, Pennsylvania. IOQ

Patterson-Kelley All P-K Twin-Shell Blenders are patented

and the name registered Circle No. 124 on Readers' Service Card


tions were often found to be optimum.

Symmetry and Rejection of Data. •Complete symmetry is of great aid to the coordinator of a test program, that is, to the chairman or anyone who assists him in making a statistical evaluation of the data.

Symmetry with respect to the number of replications supplied by each laboratory is particularly important. Although there are statistical methods for providing fill-in values for missing or unacceptable ones, some loss of sensitivity results. I t is desirable, therefore, to have faulty data rejected and replaced with valid data before the results from each laboratory are forwarded to the chairman. Such rejection and replacement has been accomplished in the cooperative testing work of the Analytical Chemistry Panel by applying Dixon's Q test {8, 9) at the operator level. Symmetry without loss of sensitivity is thus achieved. This result cannot be had by applying the test for rejection after the data are in the hands of the chairman. It is then simply too late to obtain valid replacements unless the entire experiment is discarded and a complete new one performed. This may be true because of purely practical reasons: the same chemist may not be available, or the equipment may have been disassembled. But more importantly, the very fact that time has elapsed means that another variable can be present which was not present when the original values were obtained,

and randomization with respect to time is forever lost.

The panel considered a number of criteria (4, 7-9,19) for a posteriori rejection of data and chose Dixon's Q test as the most convenient and suitable tool. A partial table of Q values adapted from Dixon (9) is shown in Table II , which is adequate for most cooperative testing. A more extensive table is given by Crow, Davis, and Max-field (7, p. 252).

In evaluating analytical procedures, the Q test is used at a low value of œ (1%) so that the evaluation will not tend to be biased in their favor. In other words, a value is not rejected unless it is very far indeed from the other clustered values. Conditions should be in a state of statistical control during the course of replicate determinations, and a low value of oc corresponds to a strong presumption that they are in control, so that an outlying value must stand relatively far apart from the other values (in the absence of any other adverse evidence) in order to cast doubt on that presumption. The chairman prescribes the exact value of Q to be used.

One case in which a high oc (10 or 20%) might be applied is that of standardization of solutions where the best possible factor is desired. In this case, the physical limitations are well known (buret errors, temperature change effects, indicator sensitivities, etc.) and a large number of replications (10 or more) are made.

Table II. Partial Table of Ο Values0

Q -D - Ν

R where D = a suspected divergent value

Ν = the divergent value's nearest neighbor R — total range or difference between D and the value farthest from it

a This table is adapted from a more extensive table (9). The probability values for «χ are double those shown in the original table because the above table is for use in "two-tail" applications.

6 Probability of obtaining by chance alone a value higher than that of the table.

η (No. of Probability Level , * ,% 6

Replicates) _J 2 10 20

3 0.994 0.988 0.941 0.886 4 Q.926 0.889 0.765 0.679 5 0.821 0.780 0.642 0.557 6 0.740 0.698 0.560 0.482 7 0.680 0.637 0.507 0.434 8 0.634 0.590 0.468 0.399 9 0.598 0.555 0.437 0.370

10 0.568 0.527 0.412 0.349

For wider model choice choose

LABCONCO KJELDAHL

REPORT

9

A SIZE FOR EVERY LAB

Standard Labconco Kjel-dahl units are available in s izes from 6 to 48 heaters. In labs with a heavy work load, larger u n i t s m e a n l a r g e r " b a t c h e s " , with a big saving in time at every step in the determination.

HOODED UNITS

Here's the ideal solution to the problem of excessive heat. H e a t c o l l e c t s in hood, is removed by hood blower. Acid fumes are removed by c o n v e n t i o n a l blower and ejector. Hoods with glass sashes also are available.

VARIABLE HEAT CONTROLS

All electric m o d e l s are a v a i l a b l e with single heat or 3-heat switches . . . or full transformer c o n t r o l . Also available a r e d i g e s t i o n heater timers and thermo-water c o n t r o l f o r distillation. Fume ejection by water is optional, too, w h e r e air ejection is difficult.

Just Out — All new cata log on complete line of LABCONCO apparatus and equipment for agricultural and al l ied laborator ies. Wr i te today for your copy. No cost, no ob l igat ion.

LABORATORY CONSTRUCTION COMPANY 881 1 Prospect Avenue Kansas City, Missouri

WORLD'S LEADING MANUFACTURER OF KJELDAHL APPARATUS Circle No. 114 on Readers' Service Card

i In the work of the A n a l y t i c a l | P a n e l , the c h a i r m a n ' s i n s t r u c t i o n s j on use of t he (J test h a v e inc luded I s t a t e m e n t s s imi l a r to those in the I fol lowing p a r a g r a p h s .

| It is extremely important thai ι he specified niimhcr ( > of impartial replications be reported for each method tried by any laboratory. Absolutely no selection of data should take place except in ihe three cases described below.

1. In case any determination w lost, another should be math1 and reported m its place at the earliest possible time.

2. If a sound a priori reason exists for rejecting a value—e.g., loss of material during analytical operation—the value may be discarded and replaced

i by one obtained in a run free from | mishap. The rejected value should be ! omitted from the data sheets. In the j absence of a truly a priori reason, no I result is to be discarded. [ o. Space is provided on the data j sheets for an alternative value, when ! it is shown by a (J test that such a | value is needed. The Q test is to be | applied to all sots of replicates and the ί alternative determination made at once I if the value of () {'Xccvil- .._.. ; (which represents the ('<

probability level for Λ = ). Report the value of Q found for the original set of replicates whether an alternative value is needed or not. This test is the only basis to be used for any a posteriori rejection of data. Such data are not discarded by the analyst but are reported in proper time sequence with the replacement value shown as an alternative and the rejected value marked with an asterisk (*) . The footnote for the asterisk will be ' 'Rejected by the Q test." Report all replicates in the order in which they were obtained—i.e., do not sort them to read in sequence from highest to lowest or vice versa.

In the case of two extreme values (suspected deviates)—a high one and a low one, each of which is the same distance from the average value— choose for the Q test that extreme value which has the greater distance between itself and its nearest neighbor —i.e., the extreme value thai will yield the higher Q value.

Evaluation of the Data

A p a p e r by M c A r t h u r a n d o the r s \14) p r o v i d e s an excel lent d i s cus sion on the e v a l u a t i o n of c o o p e r a t ive testing; d a t a . It is difficult, h o w e v e r , t o cover all t h e c o n t i n gencies t h a t m a y a r i se in i n t e r l a b o -


FAST . . . ACCURATE polarographic analyses


with the L&Ν Electro-Chemograph & Polarotron When analyzing metals, organics or bio-chemicals with an L&N Electro-Chemograph and Polarotron, you get a c c u r a t e resul t s—quick resu l t s — because design and construction are u n m a t c h e d among p o l a r o g r a p h i c instruments. Users find that the Electro-Chemograph provides:

Fast determination—recorder balancing time for full scale travel is 1 second— assures delineation of current wave peaks in undamped operation. Minimum error—polarizing is accurate to Jf mv—values are read directly from the scale . . . no interpolation required. Flexible operation—reversible polarizing potential permits determinations of half-wave potentials. Four damping ranges, instead of three, allow optimum flexibility for accurate analyses. Polarotron Dropping Mercury Electrode gives:

Time saving operation — unmatched speed and convenience provided by tilting sample cell—stable (reference) half-cell already prepared for immediate use. Minimum mercury required — automatic level control . . . needs only 50 grams. For more detailed information, ask for Da ta Sheet E-94(3). Write Leeds & Northrup Company, 4906 Stenton Ave., Philadelphia 44, Pa., or check with your nearby L&N Field Office.

ra tory programs. In the case of the more complicated experimental designs, such issues as the appropriate way to combine da ta from various par ts of the experiment, when there is evidence of bias or of nonhomo-geneity of variances, or both, can sometimes lead to ra ther delicate questions of statistical theory. See, for example, the report of round robin 16 of the Analytical Chemistry Panel (18, Appendix F ) . A guide t h a i the authors have found helpful in questions of this sort is Cochran (5).

A kind of intuit ion—with a basis in scientific experience—is always helpful in looking a t statistical evaluations of data . If the evaluations appear to be radically different from what might be expected by the investigator, the statistical methods should be very critically examined. Some circumstances or experimental factors may have been overlooked or not properly weighted in applying the statistical analysis

—i.e., the statistical model chosen was inappropriate—or perhaps an error in the calculations may be discovered. On the other hand, statistical examination may prove tha t hunches or intuitions were unsound. An excellent example of this situation is dramatical ly presented by Youden (23).

Bias, Precision, and Accuracy. An understanding of bias and its sources is essential for part icipants in cooperative testing programs. Publications by Dorsey, Eisenhart , and Youden (11-13, p . 546f.; and 22, pp. 7, 40, 41) contain succinct information on this topic.

Eisenhar t (13) presents a clear picture of the relationships among bias, precision, and accuracy. Eisenhart ' s sketch reproduced here as Figure 1 shows the four possible cases. Following the illustration, Eisenhar t points out t h a t a completely unbiased procedure is not always to be preferred to a biased one. He says,

LEEDS Instruments

NORTHRUP Automatic Controls · Furnaces iiiiii


Figure 1. Interrelations of bias. Precision and accuracy


BETHLEHEM SEAL-CRIB for Glass Tubing

Suspend from ceiling Circls No. 19 on Readers' Service Card

• The Filter-Seal door keeps out all dust—even dust d e p o s i t e d b y ' 'a tmospheric breathing."

• Divided c o m p a r t m e n t s for easy classification.

• Holds tubing of all standard sizes and lengths.

• Saves space in the laboratory.

Write for Bulletin SC-54

BETHLEHEM Apparatus Company, Inc. HEILERTOWN, PENNSYLVANIA

T h e most c o m p l e t e select ion <^. ί of table m o d e l centr i fuges a n y w h e r e

A D A M S Safety-Head —3 models ^_x ι *__._ L. ' _ 5 models Safeguard Standard —5 models Junior —3 models ?~~\

icient centnfugation. All (except Junior AC model) are permanently lubricated and are ruggedly-built for long trouble-free life. The 52° angle of suspension on Safety-Head and Safeguard models results in 35% greater efficiency than conventional, horizontal, free-swinging centrifuges. These models are adaptable for micro and semi-micro work by substituting proper shields and tubes.

REPORT

"Indeed, a procedure with a small bias and a high precision can be more accurate than an unbiased procedure of low precision. It is important to realize this, for in practical life it is often far better to always be quite close to the true value than to deviate all over the place in individual cases but be strictly correct on the average. Consider carpentry: I sincerely doubt whether even the best of carpenters hit nails with absolutely no bias (up, down, right or left) on the average, but good carpenters surely don't miss the nail altogether very often, and are certainly to be preferred to an imprecise but well balanced novice who hits most every spot within 6 inches of the nail, with absolutely no bias in the long run, but rarely if ever hitting the nail itself. This we must remember: in practical life we rarely make a very large number of decisions of a given type—we can't wait to be right on the average—our decisions must stand up in individual cases as often as possible ! "

This example makes clear that the considerations of precision and freedom from bias may be in conflict, and tha t some single measure of accuracy is needed that takes both into account. Such a measure is the mean square error, defined as the long-run average of the square of the error. It can be shown to be equal to the variance plus the square of the bias. A sample estimate of the mean square error is available only if the true value or quaesitum, Qu, is known; then the estimate is

Ι Σ (*< - w

A source of bias frequently overlooked is revealed in the following quotation from Dorsey and Eisen-hart Ι Π , p. 2) or Eisenhart {12, p. 108).

"To say that an observer's results are influenced by his preconceived opinion does not in the least imply that those results were not obtained and published in entire good faith. It is merely a recognition of the fact that it seems more profitable to seek for error when a result seems to be erroneous, than when it seems to be approximately correct. Thus reasons are found for discarding or modifying results that do violence to the preconceived opinion, while those that accord with it go un-



Ask your local dealer to demonstrate these ADAMS models.

Clay-Adams N E W YORK 10

new...

FOR THERMAL SHOCK RESISTANCE,

CHEMICAL INERTNESS IN TUBING

Sand-surfaced, glazed or satin-surfaced and transparent . . . in al l normal lengths a n d d iamete rs . Homogeneous, transparent, free from chemical i m p u r i t i e s . U s e f u l t o 1 0 0 0 ° C .

FOR GUARDING THE REAL PURITY OF YOUR COMPOUNDS IN CRUCIBLES, RETORTS, MUFFLES, DISHES, TANKS, POTS, TRAYS

Non-absorbent, non-catalytic, non-porous, immune to ex t reme e l ec t r i ca l , t he rma l a n d chemica l cond i t i ons . M a n y sizes a n d t y p e s i n s t o c k . Prompt delivery for special sizes.

FOR OUTSTANDING ELECTRICAL PROPERTIES, STRENGTH, IMPERMEABILITY IN ELECTRIC IMMERSION HEATERS, BALL & SOCKET JOINTS, STANDARD TAPER JOINTS, GRADED SEALS

Corrosion resistant, impervious to all organic and inorganic chemicals at high temperature regardless of concentration (except strong caustics and hydrofluoric acids.) Immediate de l ivery on stock sizes. Custom items to order.

Write for our information bulletins. See our ad in Chemical Engineering Catalog.

®

Company

Name & Title

Street

City

THERMAL AMERICAN FUSED QUARTZ CO., INC.

18-20 Salem Street, Dover New Jersey


Please send technical data on

Zone_ State-

tested. An observer who thinks that he knows approximately what he should find labors under a severe handicap. His result is almost certain to err in such a direction as to approach the expected value."

It is possible for the average results from one laboratory to diverge greatly from the grand average of the other laboratories and yet lie nearer the quaesitum or true value than any of the averages of the other laboratories. Thus, some source of bias may be affecting the results of all the laboratories but one. Furthermore, if values are discarded because of preconceived opinion, the recorded data are likely to exhibit unwarranted precision.

Statistical and Practical Significance. The chairman and other participants in interlaboratory testing should realize that statistical significance and practical significance are not necessarily equivalent. In fact, four situations are possible as shown in Table III .

Cases 1 and 2 require no discus

sion. Case 3 arises when the reproducibility of the analytical procedure is excellent (low standard deviation), so that the statistical test measures as significant differences between factors (methods, properties of materials, laboratories, etc.) that are not of commercial importance. Case 4 arises when the precision of an analytical procedure is not good enough to detect differences that would be commercially important; the procedure itself needs improvement, or a larger number of replicates is needed.

Summary

Valuable and satisfying results may be obtained by interlaboratory testing but success is dependent upon very careful planning. A qualified chairman with a knowledge of statistical methods and evaluation techniques is a necessity for a successful program. Practical aspects of conducting the program include instructing the participants

Participants in the Interlaboratory Testing Programs of the Joint Army-Navy-Air Force Panel on

Analytical Cnemistry of Solid Propellants

Aerojet General Corp., Azusa, Calif. Aerojet General Corp., Sacramento, Calif. Allegany Ballistics Laboratory, Cumberland, Md. Atlantic Research Corp., Alexandria, Va. Atas Powder Co., Tamaqua, Pa. Badger Ordnance Works, Government Laboratory, Baraboo, Wis. Badger Ordnance Works, Liberty Powder Defense Corp., Baraboo, Wis. Ballistic Research Laboratories, Aberdeen Proving Ground, Aberdeen, Md. Canadian Armament Research and Development Establishment, Quebec, P.Q., Ca

(guest) Chemical Inspectorate Bishopton, Scotland, U.K. (guest) E. I. du Pont de Nemours & Co., Burnside Laboratory, Penns Grove, N. J . E. I. du Pont de Nemours & Co., Indiana Ordnance Works, Charleston, Ind. Frankford Arsenal, Philadelphia, Pa. Hercules Powder Co., Experiment Station, Wilmington, Del. Hercules Powder Co., Kenvil, N. J . Hercules Powder Co., Radford Arsenal, Radford, Va. Hercules Powder Co., Sunflower Ordnance Works, Lawrence, Kan. Jet Propulsion Laboratories, Pasadena, Calif. Ol in Mathieson Chemical Corp., East Alton, III . Phillips Petroleum Co., Bartlesville, Okla. Phillips Petroleum Co., McGregor, Tex. (now merged to form Astrodyne) Picatinny Arsenal, Dover, N. J . Radford Arsenal, Government Laboratory, Radford, Va. Redstone Arsenal, OML, MKL, Huntsville, A la. Rohm & Haas Co., Redstone Arsenal, Huntsville, A la. Sunflower Ordnance Works, Government Laboratory, Lawrence, Kan. Thiokol Chemical Corp., Elkton, Md. Thiokol Chemical Corp., Longhorn Ordnance Works, Marshall, Tex. Thiokol Chemical Corp., Redstone Arsenal, Huntsville, Ala. U. S. Naval Air Rocket Test Station, Lake Denmark, Dover, N. J . U. S. Naval Ordnance Test Station, China Lake, Calif. U. S. Naval Propellent Plant, Indian Head, Md.



NOTHING MEASURES UP TO. . .

VITREOSIL

Reliable, stable Ainsworth

KEYBOARD balances

for high precision ^ Four models, sensitivities of 1 /20 , 1 /40 , 1 /200, or 1/1000 mg.

ψ Keyboard operated weight carrier handles smaller weights while case is closed.

k P R O M P T D E L I V E R Y from your favorite laboratory supply house.

Made in U.S.A. by

W M . AINSWORSTH & SONS, INC. 2 1 5 1 L A W R E N C E S T R E E T D E N V E R 5 . C O L O R A D O

REPORT

so that objectives are clearly understood; adequate samples are used; suitable practice periods on new methods will be indicated; randomization, replication, and symmetry will be achieved ; and premature rounding-οίϊ of data will not occur. Symmetry and sensitivity will be promoted by the use of a suitable criterion—e.g., Q test—for rejection of data at the operator level. An efficient design of experiment that scrupulously avoids confounding of important factors will be used. This design will be kept as simple as possible, yet should provide answers to the important questions without waste of effort. I t is strongly recommended that evaluation of "operator" and "day" as factors be avoided in most instances. The chairman should conclude the program by issuing a detailed written report. I t will be well to keep in mind tha t some statistical problems encountered in cooperative testing are perhaps unique to that field and differ in some ways from problems of an apparently similar nature arising in research work in a single laboratory.

Acknowledgment The authors are indebted to

members of the Joint Army-Navy-Air Force Panel on Analytical Chemistry of Solid Propellants for encouragement in preparing this report based largely on panel experiences. The authors gratefully acknowledge permission to quote from references 9, 11-13.

L i te ra ture C i t e d (1} Am. Soc. Testing Materials, Phila

delphia, Pa., "A.S.T.M. Manual on Quality Control of Materials," 1951.

(2) Am. Soc. Testing Materials, Philadelphia, Pa., "Tentative Recommended Practice for Interlaboratory Testing of Textile Materials," A.S.T.M. Designation D 090-54T, June 1954.

(3) Am. Soc. Testing Materials, Philadelphia, Pa., "Tentative Recom-



Table III. Comparison of Statistical and Practical Significance

C ases Significance Statistically

significant Practically

significant

1

yes

yes

2

no

no

yes

no

4

no

yes

mended Practice for In ter labora tory Testing of Rubber and Rubber-Like Materials ." A.S.T.M. Designation I) 1421-56T, June 1956.

(4) Blaedel, W. J., Meloche, Υ. M., Ramsay, J. Α., ./. Chew. Educ. 28, 048 (1951).

(5) Cochran, W. 0 . , Biometrics 10, 101 (1954).

(()) Cochran, W. G., Cox, G. Μ., "Ex-])erimental Designs," 2nd éd., Wiley, New York, 1957.

( 7 ) Crow, E. L., Davis, F . A , Maxfield, M. W., "Statist ics M a n u a l / ' p. 102, NAVOR1) Kept . 3369, Γ . S. Naval Ordnance Test Station, China Lake, Calif. (1955).

(S) Dean, R. B., Dixon, W. J., A N A L . C H K M . 23, 080 ι 1951).

(9) Dixon, W. J., Λ Η Η . .Ι/αίΛ. S ia i . 22, ON (1951).

(10) Dixon, W. J., Massey, F . J , Jr . , " In t roduct ion to Statistical Analysis," 2nd éd., McGraw-Hil l , New York, 1957.

( 1 1 ) Dorsey, Ν. Ε., Trans. Am. Phil. Snc. 34, Π 1944).

(12) Dorsev, X. E., Eisenhart , Churchill, Sci. 'Monthly L X X V I I , 103 (1953).

(13) Eisenhart , Churchill , Photogram-metric Eng. X V I I I , 542 (1952).

(14) McAr thur , D . S., Baldeschwieler, E. L., White , W. H., Anderson, J . S., A N A L . C H E M . 26, 1012 (1954).

(15) Mandel, John, Lashof, T . W , A.S.T.M. Bull. 239, p. 53f. (July 1959).

(10) Pearson, E. S., Har t ley , H. O. (eds.) , "Biometr ika Tables for Stat isticians," Vol. 1, ]). 179, Universi ty Press, Cambridge, U. K., 1954.

(17) Pierson, R. H , U. S. Xaval Ordnance Test Stat ion R e p t . 1937, P a r t s 1-6, 1958. Par t 4 security classified "Confidential ."

( IS) Pierson, R. H., "Repor t of Round Robin Xo. 16 of the Joint Army-Xavy-Air Force Panel on Analytical Chemistry of Solid Propel lants ," l \ S. Xaval Ordnance Test Station, 195S.

(19) Villars, 1). S., "Statistical Design and Analysis of Exper iments for Development Research," pp . IS, 5S, Wm. C. Brown Co., Dubuque , Iowa, 1951.

(20) Wernimont , Gran t , A N A L . C H E M . 23, 1572 (1951).

(21) Willits, C. ()., Ibid., 23, 1565 (1951).

(22) Youden, W. J., "Statistical Methods for Chemists ," pp . 7, 16, 40, Wiley, Xew York, 1951.

(2o) Youden, W. J., Technical Xews Bulletin of the Xat ional Bureau of Standards , July, 1949.

(24) Youden, W. J., Ind. Eng. Chem. 50, 91 A (1958).

NEW Automatic Recording

Vacuum Balance by

AINSWORTH

Will weigh samples. . . • in air or inert gases • at atmospheric or reduced pressures • at room or higher temperatures • on balance pan or suspended below the balance in a furnace for

thermogravimetry or differential thermal weighing

Remote control, built-in weights

Recorder charts weight or weight and temperature against time

Semi Micro Analytical Capacity 100 g 200 g Sensitivity 1/100 mg 1/10 mg Range of automatic weight operation 4 0 0 mg 4 0 0 0 mg

Write for Bulletin 459

WM. AINSWORSTH & SONS, INC. 2 1 5 1 L A W R E N C E S T R E E T D E N V E R 5 , C O L O R A D O


VOL. 3 1 , N O . 12 , DECEMBER 1 9 5 9 · 4 9 A

Documents

Guidelines for Interlaboratory Testing Programs