STATISTICS DIGEST - ASQasq.org/statistics/2016/10/statistics/statistics-digest-february-2017.pdf• Social Media (LinkedIn) • Conference presentations (Fall Technical Conference,

Continued on page 3

STATISTICS DIGESTThe Newsletter of the ASQ Statistics Division

Chair’s Message ...........................1

Editor’s Corner ..............................4

YOUDEN ADDRESSUnderstanding Today’s Complex World ..........................5

MINI-PAPERAttribute Agreement Analysis (AAA): Calibrating the Human Gage! ..........................15

Design of Experiments ..................19

Statistical Process Control .............21

Statistics for Quality Improvement ...............................28

Stats 101 ....................................31

Testing and Evaluation .................32

Standards InSide-Out ...................34

FEATUREPredictive Analytics: Is Your Problem Collinearity, Sparsity, or Both? ...................................36

Upcoming Conference Calendar ...42

Statistics Division Committee Roster 2017 ..................................43

IN THIS ISSUE

Vol. 36, No. 1 February 2017

ASQ Statistics Division, welcome to 2017! My name is Richard N. McGrath and I am proud to serve as the Chair of the Division for 2017. I should start off by saying that nobody (besides my mother) calls me Richard. My nickname is Herb and has been since fifth grade. Long story but if you receive any correspondence from R. N. “Herb” McGrath, it is not from a registered nurse.

I am taking over from Teri Utlaut who has done a great job leading a group of dedicated volunteers. As she said in this column a year

ago, she had not served in a leadership role in the Statistics Division before becoming Chair-Elect. Her predecessor, Adam Pintar, served as Secretary before moving into the Chair-Elect role. My experience with Division leadership is a mixture of theirs. I was Secretary from 2010–2012 while I was also holding leadership roles in my local Toledo Section. I then stepped back for a few years before returning to division leadership as Chair-Elect last year.

I am a Professor of Applied Statistics at Bowling Green State University in Bowling Green, OH and have been serving as Associate Dean of the College of Business Administration since 2012. I have a B.S. in Industrial Engineering as well as a Ph.D. in Statistics, both from Penn State. In between degrees I worked as a quality engineer and quality manager with AT&T at locations in Pennsylvania, North Carolina, and New Jersey. I am married to Valerie and we have a son Max. If we also had a daughter, naturally she would have been named Min.

There is a lot going on with the Division and it takes a great group of volunteer leaders to make it happen. The structure of serving as Chair-Elect, then Chair, then Past-Chair has allowed me to learn a tremendous amount from Adam and Teri. I have big shoes to fill. But with the extensive list of dedicated volunteers (please see the page at the end of this digest), I am confident that we will accomplish a lot this year. We will continue this format of the Statistics Digest that provides more statistical information than traditional newsletter content. News, announcements, etc. will be published in our E-zines (monthly email announcements from ASQ) and on the division website, and discussions will continue on our blog. We will continue our popular webinar series with a plan to offer six this year with at least one in a language other than English. As we have for decades, we will be a co-sponsor of the Fall Technical Conference. The 61st annual FTC will be held in Philadelphia, PA on October 5–6. If you are interested in presenting, please see http://asq.org/conferences/fall-technical/2016/about/ and click on 2017 Call for Papers. We will also have a booth and sponsor speakers at the World Conference on Quality and Improvement held May 1–3 in Charlotte, NC.

Message from the Chairby Richard Herb McGrath

Richard Herb McGrath

http://asq.org/conferences/fall-technical/2016/about/

http://asq.org/conferences/fall-technical/2016/about/

2 ASQ Statistics DIVISION NEWSLETTER Vol. 36, No. 1, 2017 asq.org/statistics

Submission Guidelines

Mini-PaperInteresting topics pertaining to the field of statistics; should be understandable by non-statisticians with some statistical knowledge. Length: 1,500-4,000 words.

FeatureFocus should be on a statistical concept; can either be of a practical nature or a topic that would be of interest to practitioners who apply statistics. Length: 1,000-3,000 words.

General InformationAuthors should have a conceptual understanding of the topic and should be willing to answer questions relating to the article through the newsletter. Authors do not have to be members of the Statistics Division. Submissions may be made at any time to [email protected].

All articles will be reviewed. The editor reserves discretionary right in determination of which articles are published. Submissions should not be overly controversial. Confirmation of receipt will be provided within one week of receipt of the email. Authors will receive feedback within two months. Acceptance of articles does not imply any agreement that a given article will be published.

VisionThe ASQ Statistics Division promotes innovation and excellence in the application and evolution of statistics to improve quality and performance.

MissionThe ASQ Statistics Division supports members in fulfilling their professional needs and aspirations in the application of statistics and development of techniques to improve quality and performance.

Strategies1. Address core educational needs of members • Assessmemberneeds • Developa“base-levelknowledgeofstatistics”curriculum • Promotestatisticalengineering • Publishfeaturedarticles,specialpublications,andwebinars

2. Build community and increase awareness by using diverse and effective communications

• Webinars • Newsletters • BodyofKnowledge • Website • Blog • SocialMedia(LinkedIn) • Conferencepresentations(FallTechnicalConference,WCQI,etc.) • Shortcourses • Mailings

3. Foster leadership opportunities throughout our membership and recognize leaders

• Advertiseleadershipopportunities/positions • Invitationstoparticipateinupcomingactivities • Studentgrantsandscholarships • Awards(e.g.Youden,Nelson,Hunter,andBisgaard) • Recruit,retainandadvancemembers(e.g.,SeniorandFellowstatus)

4. Establish and Leverage Alliances • ASQSectionsandotherDivisions • Non-ASQ(e.g.ASA) • CQECertification • Standards • Outreach(professionalandsocial)

Updated October 19, 2013

Disclaimer

The technical content of material published in the ASQ Statistics Division Newsletter may not have been refereed to the same extent as the rigorous refereeing that is undergone for publication in Technometrics or J.Q.T. The objective of this newsletter is to be a forum for new ideas and to be open to differing points of view. The editor will strive to review all articles and to ask other statistics professionals to provide reviews of all content of this newsletter. We encourage readers with differing points of view to write to the editor and an opportunity to present their views via a letter to the editor. The views expressed in material published in this newsletter represents the views of the author of the material, and may or may not represent the official views of the Statistics Division of ASQ.

Vision, Mission, and Strategies of the ASQ Statistics Division

The Statistics Division was formed in 1979 and today it consists of both statisticians and others who practice statistics as part of their profession. The division has a rich history, with many thought leaders in the field contributing their time to develop materials, serve as members of the leadership council, or both. Would you like to be a part of the Statistics Divisions’ continuing history? Feel free to contact [email protected] for information or to see what volunteer opportunities are available. No statistical knowledge is required, but a passion for statistics is expected.

mailto:[email protected]

asq.org/statistics ASQ Statistics DIVISION NEWSLETTER Vol. 36, No. 1, 2017 3

Message from the ChairContinued from page 1

One of my priorities this year is to have the Division be more involved in Big Data Analytics. The number of positions with titles such as data analyst or data scientist is exploding. A quick search on Monster.com (no endorsement here) showed 1000+ positions advertised for both data analyst and data scientist and 441 for statistician. Our division is diverse in terms of occupation and educational background. I suspect that most of our members do not have statistician in their title, but are using statistics daily in their job or are interested in learning more about the discipline. For these reasons, I would like to start an interest group on the subject. Please let me know if you are interested in getting involved ([email protected]).

As I mentioned above, our division and its leadership are diverse in terms of occupation and educational background, but I would like to increase diversity in other aspects. Again, please let me know if you have any suggestions.

I look forward to a great 2017 for the Statistics Division and please feel free to contact me with any comments or suggestions, or if you would like to join the leadership team.

Past Chair’s Message by Theresa I. Utlaut

Therehasbeenalotofdiscussionabouttheyear2016withsomepeoplehavingmixedemotionsaboutit.Youmight have even heard people say they are relieved it is over. I won’t get into the political quagmire that is probably on everyone’s mind when discussing 2016. This publication is committed to being neutral on issues that might raise the blood pressure of our members.

For the Statistics Division, I think 2016 was a very good year. We met almost all of the goals we established in the business plan (only one was not met and that was due to extenuating circumstances that we could not control). Some of the goals that we did achieve during the year were free webinars for our members (six in English and one

in Spanish), continued and active involvement in the development of standards impacting Statistics, outreach at several events (FTC, WCQI, section meetings, and other division meetings), the publication of the book Statistical Roundtables (check it out on the ASQ website!), co-hosted certification exams, three publications of the Statistics Digest, and the distribution of thousands of dollars in awards and scholarships. Overall, it has been a busy and productive year!

Given all that was accomplished in 2016, I want to thank the Statistics Division leadership council. They are an awesome group of people who dedicate their time to serve our members, and I have enjoyed working with and getting to know each of them. It is also not possible without great members! We appreciate your feedback whether it is in person, through an email, or when you complete the VOC survey. I think Statisticians, as a group, are more willing to provide feedback than most other groups of people because they know that data are important. (Has there been a study on that??). We do take our member input and feedback seriously so please continue to provide it.

In October, the council met to make our plans for 2017. I know Herb has covered some of the plans in his Chair’s Message. The one I am really excited about is getting the Division involved in Big Data Analytics. There is demand for people with strong analytic skills and people with mixed skill sets that include statistics, computer science, and business. I do get nervous about the fact that some people seem to slap the title “Data Scientist” or “Data Analyst” after their names without even knowing what it means or assuming since they can enter data in a spreadsheet that they are qualified to use these titles. If we let everyone run-amuck, then I believe it will be disastrous so some guidance and perspective on this issue is important.

Ithinkthat2017willbeanexcitingyear.IlookforwardtoseeingwhattheNewYearbrings.

Theresa I. Utlaut


https://asq.org/quality-press/display-item?item=H1508&xvl=76119018

4 ASQ Statistics DIVISION NEWSLETTER Vol. 36, No. 1, 2017 asqstatdiv.org

Editor’s Cornerby Matt Barsalou

Hello and welcome to the February 2017 issue of Statistics Digest. I’d like to thank our outgoing Division Chair Theresa l. Utlaut for her service to the Division and I’d like to welcome out new Chair, Richard “Herb” McGrath.

InthisissuewehavetheYoudenAddress,“UnderstandingToday’sComplexWorld”byJoanneR.Wendelberger.TheYoudenAddressisnamedafterDr.JackYouden(1900–1971)andisgivenannuallyattheFallTechnicalConference by an individual who shows an outstanding ability to communicate and explain statistical tools and methods to others.

Our Mini-Paper is “Attribute Agreement Analysis (AAA): Calibrating the Human Gage!” by Jd Marevko, and the feature by Marco Reis is on the topic of collinearity and sparsity in SPC.

This issue’s DoE column is “Celebrating Statisticians; George E.P. Box” by Bradley Jones. Jack B. ReVelle’s Stats 101 column is a discussion of the Z-Score, and Mark Johnson provides us with “Some General Thoughts on Standards” in his Statistics InSide-Out column. This issue’s Testing and Evaluation column is “System Reliability as an Emergent Property” by guest author Harish Jose.

I’d also like to introduce our new SPC columnist, Donald J. Wheeler. His first column for Statistics Digest is “SPC and Statistics.” He is a consulting statistician who knew and worked with Dr. W. Edwards Deming for over 20 years. He graduated from the University of Texas with a Bachelor’s degree in Physics and Mathematics, and holds M.S. and Ph.D. Degrees in Statistics from Southern Methodist University. From 1970 to 1982 he taught in the Statistics Department at the University of Tennessee where he was an Associate Professor. He is a Fellow of both the American Statistical Association and the American Society for Quality, and was awarded the Deming Medal in 2010. He has conducted over 1100 seminars in seventeen countries on six continents. He is author or co-author of 34 books and over 275 articles.

Matt Barsalou

Empirical Root Cause Analysis Webinar 29 Mar 2017 14:00 to 15:00

This webinar will describe how to conduct an empirical root cause analysis. A brief mention of reasons for performing a root cause analysis will be given followed by typical tools. A case will be made for using empirical root cause analysis. The correct application of the scientific method will also be explained. This includes concepts such as Exploratory Data Analysis for hypothesis generation as well as the attributes of a good hypothesis. John Platt’s version of the scientific method will be explained using examples and these concepts will be combined into a Plan-Do-Check-Act process. The concept of empirical root cause analysis will be depict in the form of a graphic which will show the various steps to combine Exploratory Data Analysis, the scientific method, and Box’s iterative inductive deductive process into a Plan-Do-Check-Act like method. Examples will be presented to illustrate these concepts. Real world examples will include the analysis of a cracked cable coating, vibration sensor failures and a bushing and bracket failure. This final section of the presentation will provide advice on how both the customer and supplier can better prepare for a complaint. The view of both customers and suppliers will be represented including information the customer must provide with a claim as well as how a supplier can better prepare for a customer issue.

Web site: https://attendee.gotowebinar.com/register/6501478605683347969

https://attendee.gotowebinar.com/register/6501478605683347969

asqstatdiv.org ASQ Statistics DIVISION NEWSLETTER Vol. 36, No. 1, 2017 5

YOUDEN ADDRESSUnderstanding Today’s Complex Worldby Joanne R. WendelbergerLos Alamos National Laboratory

IntroductionIwillbeginmyaddresswithsomebackgroundonW.J.YoudenandwhyIchosemytitle,“UnderstandingToday’s Complex World.” I will then share my thoughts on the Process of Scientific Exploration and Discovery, focusing on three important concepts: sampling, error analysis, and statistical design of experiments. As we will see, the use of these concepts to address real problems has led to the evolution and advancement of statistical methods over time. Throughout my address, I will share some of my personal and professional experiences and insights.

Iamdeeplyhonoredtohavetheopportunitytopresentthe2016W.J.YoudenMemorialAddress.JustasW.J.Youdenrecognizedtheopportunitytousestatisticalmethodstounderstandcomplexphenomena in the 20th century, statisticians and quality professionals today have the opportunity

to contribute statistical knowledge and expertise to understanding today’s complex phenomena and the accompanying explosion of data.

BackgroundW.J.Youden,whowasoriginallyfromAustralia,hadabackgroundinphysicalchemistryandchemicalengineering.Hecameto the United States and joined the National Bureau of Standards in 1948. As described in the Complete Dictionary of Scientific Biography(2008),“Byhispublicationsandbyhisexample,Youdencontributedsubstantiallytotheachievementofobjectivityinexperimentationandtotheestablishmentofmoreexactstandardsfordrawingscientificconclusions.”ThroughoutYouden’swork,theenduringconceptsofstatisticalsampling,erroranalysis,andstatisticalexperimentdesignfeatureprominently.AmongYouden’smanytechnicalcontributionsaretheYoudenSquareexperimentdesign,workonanalysisofsystematicerrors,interlaboratorytesting,theYoudenPlot,theYoudenIndex,restrictedrandomization,andtheExtremeRankTestforOutliers.

Youden’sideashavepoppedupthroughoutmylifeasastatistician.Earlyinmycareer,IwasaskedtoserveasamemberandlaterchairoftheAmericanStatisticalAssociationW.J.YoudenAwardforInterlaboratoryTestingCommitteewhichrecognizesauthors of publications that make outstanding contributions to the design and/or analysis of interlaboratory tests or describe ingenious approaches to the planning and evaluation of data from such tests. This experience exposed me to the statistical issues and approaches used in the area of interlaboratory testing. Later, I had the opportunity to serve as an issue nominator and judge fortheAmericanSocietyforQuality’sYoudenPrize,whichisawardedtothebestexpositorypaperinTechnometrics, recognizing the importance of communication of technical ideas. These experiences instilled in me an appreciation for both sound technical approaches and clear communication.

According to the Complete Dictionary of Scientific Biography(2008),YoudenwroteamanuscriptonRisk,Choice,andPredictionin 1962, later published in 1974, that he said was, “for anyone … who wants to learn in a relatively painless way how the concept and techniques of Statistics can help us better understand today’s complex world,” and this inspired me to choose my title, “Understanding Today‘s Complex World.”

ComplexityIn many ways, our world is becoming increasingly complex. Complexity arises in diverse settings and has been defined in various ways. Often, complexity is defined in a manner that gives the impression that it is essentially impossible to model. The U.S. Army Operating Concept defines a complex environment to be “not only unknown, but unknowable and constantly changing.” According to Neil Johnson (2009), “Even among scientists, there is no unique definition of complexity,” and “Complexity Science can be seen as the study of the phenomena which emerge from a collection of interacting objects.” According to another perspective posted at en.wikiquote.org, “Understanding today’s complex world of the future is a little like having bees live in your head,” conveying a rather alarming situation.

Joanne R. Wendelberger

http://en.wikiquote.org


Despite the difficulties posed by complexity, even if it is not possible to completely capture all of the intricate details of an interacting system precisely, that should not cause us to throw up our hands in despair. Instead, we should proceed cautiously, and in the spirit of George Box (1979), acknowledge that, “All models are wrong, but some are useful.”

One might ask, “Where do Quality and Statistics fit in?” In the spirit of the 2016 Fall Technical Conference theme, “Statistics and Quality: Twin Pillars of Excellence,” statisticians and quality professionals can contribute to the understanding of complex problems and accelerate scientific discovery by using their knowledge of statistical methods and models to understand and disentangle different components of interacting systems.

For the remainder of this discussion, I will focus on complexity in Science and Technology and the potential for accelerating Scientific Discovery using the tools of the Statistics and Quality fields.

The process of scientific exploration and discoveryIn the process of scientific exploration and discovery, scientists seek improved knowledge over time, gathering data to confirm or refute proposed models. Statistics provides a rigorous framework for drawing inferences from scientific data, bringing together the key concepts of Sampling, Error Analysis, and Design of Experiments.

To draw valid conclusions, we need to understand the distributions from which data arises. Analysis of errors requires an understanding of measurement processes, associated errors, and how they will interact and propagate. Given that complex problems generally involve limited resources, statistical experiment design methodology is needed to provide structured techniques for efficiently selecting sets of experimental runs from the space of possible experimental settings.

Thefundamentalconceptsassociatedwithsampling,erroranalysis,anddesignofexperimentsusedinYouden’sdayhavelaida foundation for understanding data in a structured and principled manner. These fundamental concepts continue to play an important role today in understanding complex relationships and behavior. Statistical methods built on these concepts are evolving and advancing over time to address increasingly complex problems. These ideas will now be discussed in further detail, along with their evolution to address modern challenges.

SamplingSampling theory gives us a sound mathematical structure to understand and probe statistical populations. Many different types of sampling procedures have been proposed to address different types of sampling scenarios. Traditionally, attention has focused primarily on relatively simple sampling scenarios such as simple random sampling, stratified random sampling, and systematic sampling. Over the years, many other types of sampling have been studied and implemented.

In order to employ effective sampling approaches, it is import to understand the population to be sampled, the sampling objective, and practical constraints. Table 1 provides a list of selected sampling methods that have been developed to address different types of sampling situations. As we see here, many different sampling strategies have been developed and applied as increasingly complex problems have arisen, leading to new advances in the field of Statistics.

Youden’sworkonexperimentationandmeasurement(1962)includesanumberofexperimentsinvolvingweighing,includinganexperiment where he weighed pennies to determine the distribution of weight values and to compare the weights to a specification. This example reminds me of a practical application of weighing methods that I encountered in one of my first summer jobs, conducting inventory at Harley Davidson. At that time, an annual manual inventory was conducted to determine how many items were on hand of every type of part used in the assembly of motorcycles. Many items were simply counted by hand. For boxed items stacked on pallets, simple arithmetic calculations could be used to quickly determine the total number of items. For large quantities of small items, estimates of total quantities were determined by weighing both a small countable sample and the total quantity available, and then calculating an estimated total. Many years later, I read an article by Stefan Steiner and Jock McKay(2004)onscale-countingthatappearedinTechnometrics, providing a thorough statistical treatment of weighing methods that was the winner of the American Society for Quality’s Wilcoxon Prize for the best practical applications paper. In recent years,

Understanding Today’s Complex World


many time- consuming manual inventories have been replaced by automated inventory systems. Just as manual weighing methods improved upon hand counting, automated inventory systems bring a new level of accuracy in counting and timely transfer of information for multiple business needs.

Today’s problems often involve large and complex data. Observations may take the form of curves, spectra, or more general types of functions. Sometimes, a functional approach can be useful in understanding statistical problems. I was once asked to evaluate some data where an unusual measurement had been observed. An initial assessment based on a small sample led to the conclusion that, “This was a non- problem.” However, this result relied on several assumptions that were difficult to validate. Additional measurements were recommended, and the selection of units for measurement took into account historical functional data. This approach explored the space defined by the first couple principle components of the functional responses. The new samples led to the identification of an issue that was more prevalent than originally thought.

When problems become larger and more complex, it becomes increasingly important to develop and apply sampling, design, and analysis methods that can provide representative samples and effective analysis results. With the rise of computational models and sophisticated Bayesian estimation schemes, substantial growth has occurred in the use of techniques such as Latin Hypercube Samplingforchoosingrunsinstudiesoflargecomputationalmodels,asdescribedinMcKayetal(1979),andMarkovChainMonte Carlo methods for Bayesian estimation, as discussed in Gelman et al (1995).

Error AnalysisUnderstanding measurement error has long been a concern of both statisticians and metrologists. Extensive work in this area was conducted at the National Bureau of Standards, today known as the National Institute of Standards and Technology. Understanding data often requires careful analysis of the measurement process used to collect the data. As discussed by Vardeman et al (2010) in The American Statistician, failure to consider errors associated with the measurement process can result in misleading analyses, even for simple basic statistical methods such as comparisons and linear regression.

Despite its importance, measurement error is often simply ignored. The interaction of sources of physical variation with the data collection plan will ultimately determine what can be learned from the data. Unfortunately, the implications of measurement error are rarely given much emphasis in standard statistics courses.

While the field of Statistics focuses on uncertainty, the field of Metrology focuses on measurement. These two fields really go hand in hand, and both perspectives are important to understanding the different sources of variability present when analyzing measurements made on samples drawn from a specified population. Probability theory defines the concept of a distribution which can then be used to describe empirical variation and uncertainty. In an article in Journal of Quality Technology,


Simple Random Sampling

Stratified Random Sampling

Systematic Sampling

Subsampling

Judgement Sampling

Convenience Sampling

Biased Sampling

Latin Hypercube Sampling

Markov Chain Monte Carlo

Table 1: Selected Sampling Methods


Vardeman et al (2014) propose a simple thought process for understanding measurement error that begins by modeling a single measurand, then considers multiple measurements from a stable process or fixed population, and then progresses to analysis of data obtained from multiple measurement methods.

In the presence of measurement error, we aim to measure a variable x, but what we observe is actually the sum of the desired variable and an associated measurement error. The overall error includes an intrinsic variation in the underlying variable as well as an added source of variability associated with measurement error. This type of assessment can be extended to more complicated situations including multiple measurements and multiple measuring devices.

When carrying out statistical analyses, an important concept to understand is that sample variances are inherently much more variable than sample means. While working on my Ph.D. thesis with George Box, he instructed me to go look at how many samples would be needed to obtain a coefficient of variation of 5% for the variance of a normal distribution as shown in Wendelberger (2010). I was shocked to find that 801 samples were needed, and this made a huge impression on me. Analysis methods which rely on estimates of variability often ignore the inherent variability in sample variances.

Fortunately, in some situations, we can turn to transformations and apply a log transformation to stabilize variances as suggested byBartlettandKendall(1946).This ideawasextendedtomoregeneralpowertransformationsbyBoxandCox(1964),aresult that is widely used in diverse application areas. I recall seeing an article posted on the bulletin board in the University of Wisconsin- Madison Statistics Department in the early 1980’s showing that the Box and Cox paper was on a list of the 10 most- cited scientific references, truly an amazing feat, having impact not just in Statistics, but also in the broader scientific community.

Ultimately, when we are presented with uncertainty in measured data, this will have an impact not just on the analysis of the measured values themselves, but also on subsequent analyses, where the measured data is used as input to models, and the associated variation is transmitted to the resulting model outputs.

In recent years, increasing interest in different sources of uncertainty has led to the interdisciplinary field of Uncertainty Quantification which focuses on understanding uncertainty throughout the modeling process. As defined in National Research Council (2012), Uncertainty Quantification is:

“the process of quantifying uncertainties associated with model calculations of true, physical quantities of interest, with the goals of accounting for all sources of uncertainty and quantifying the contributions of specific sources to the overall uncertainty.”

Modern Statistics and Uncertainty Quantification draw on many statistical ideas that have evolved over the past century. In scientific modeling, analysis of outputs from computer models has become increasingly important. Statistical techniques such as Gaussian Process Modeling, first applied to computer experiments by Sacks et al (1989), are now in common use for approximating the behavior of complex computer codes and associated discrepancy of these models from observed data. Gaussian Process Modeling is essentially an evolution of classical response surface modeling, where Gaussian processes are used in place of the typical low order polynomials, and discrepancy functions build upon and extend the idea of residual analysis.

Design of ExperimentsStatistical experiment design methodology provides structured approaches for efficiently selecting sets of experimental runs from the space of possible experimental settings for both physical and computational experiments.

Latin Squares and Graeco- Latin SquaresMy first introduction to experiment design was a puzzle I received at an early age. The challenge provided by this puzzle was to arrange 16 colored shapes on a 4×4 grid such that each row and column contained each color and shape exactly once. I later learned that this puzzle was actually a 4×4 Graeco- Latin Square. A somewhat simpler arrangement is a Latin Square where a single characteristic, such as color, appears exactly once in each row and column. Figures 1 and 2 display a 4×4 Latin Square and



a 4×4 Graeco- Latin Square, which represent examples of experiment design layouts that can be used in designing experiments with a single 4-level variable or two 4-level variables.

Youden SquaresAnothertypeoflayoutcloselyrelatedtotheLatinSquareistheYoudenSquare,whichisactuallynotasquarebutarectangle.YoudenSquarescanbeusedtogenerateBalancedIncompleteBlockDesigns.AninterestingpropertyofaYoudenSquare(orLatin Rectangle) is that it can be converted to a Latin Square by adding a column.

Magic SquaresLatinSquares, Graeco-LatinSquares,andYoudenSquaresareall relatedtoMagicSquares,whichhaveexisted for severalhundred years. In a Magic Square, the entries are typically represented as numbers, and the row sums and column sums are all equal. In a Perfect Square, in addition to having all row and column sums equal, the diagonal sums are also equal. Figure 3 displays a Perfect Magic Square, where the row, column, and diagonal sums are all equal. I constructed this Magic Square during


Figure 2: Graeco- Latin Square—Each color and shape appears exactly once in each row and column

Figure 1: Latin Square—Each color appears exactly once in each row and column


a beach vacation, when I was thinking about a problem I was working on with an artist involving assessment of color maps used to assign colors to different values for scientific visualization. An experiment plan was needed that would assign different color maps to each subject in the study in order to evaluate the impact of the color maps on the subjects’ performance on tasks involving color perception.

Magic Squares have a long and interesting history. According to Block and Taveres (2009), Magic Squares first appeared in ancient Chinese literature over 4000 years ago, where a mythical emperor observed marks on a tortoise, with the counts being the same for each of the rows, columns, and diagonals. A Magic Square was reported in India in the first century A.D. German artist and printmaker, Albrecht Dürer, included a magic square in a 1514 engraving. In 1783, Swiss Mathematician and Physicist, Leonhard Euler introduced the Latin Square as an N×N grid, with the numbers 1 through N appearing exactly once in each row and column. Equivalently, these labels can be represented by colors, as in the puzzles described earlier.

SudokuToday’s popular Sudoku puzzles, commonly found in newspapers, magazines, puzzle books, online websites, and electronic games, represent an evolution of the design patterns introduced earlier. According to Block and Taveres (2009), American puzzle designer, Howard Garns, is credited with initiating the 9×9 grid puzzle with nine 3×3 sub- grids that was first published inapuzzlemagazineandcalledNumberPlace.ThegameNumberPlacewasrenamedSudokubyMakiKajiandpopularizedin Japan beginning in 1984. Note that the rapid and successful popularization of Sudoku is reminiscent of the Quality Revolution that took place after W. Edwards Deming brought statistical process control methods to Japan. Another puzzle gamecalledKakuroinvolvestheinsertionofthenumbers1–9intoablackandwhitepatternsothatrowsandcolumnsaddup to specified sums.

In recent years, statistical researchers have studied the mathematical properties of Sudoku puzzles and developed Sudoku- Based Experiment Designs. Xu, Haaland, and Qian (2001) have developed space- filling designs that achieve maximum uniformity in univariate and bivariate margins. Xu, Qian, and Lu (2016) have extended these ideas to produce space- filling designs for data pooling that include overlaps and provide uniformity in univariate and bivariate margins for a complete design, sub- designs, and overlaps. Advances in our understanding of design patterns support the development of new statistical designs to address increasingly complex problems.


Figure 3: Magic Square at the Beach (created and photographed by J. R. Wendelberger)— Each number occurs exactly once in each row, column, and diagonal


Experiment Designs for Diverse ProblemsAs we look back at how the field of statistical experiment design expanded beyond traditional two- way layouts motivated by agricultural experiments, we see the evolution of the ongoing search for balanced patterns to efficiently obtain information about multiple experimental factors. Table 2 lists a number of different types of statistical experiment designs that have been developed. A variety of types of experiment plans have been proposed to address diverse types of problems. Note, for example, the progression over time that has occurred with the development of Factorials and Fractional Factorials, as described by Box, Hunter, and Hunter (2005), and the recent work on Definitive Screening Designs by Jones and Nachtsheim (2011). The evolution of design strategies overtimewasalsonotedbyDennisLininhis(2010)FallTechnicalConferenceYoudenMemorialaddress.


Figure 4: Compressive Sensing

Table 2: Selected Experiment Design Approaches

Latin Square Definitive Screening

Graeco-Latin Square Adaptive Design

YoudenSquare Bayesian Design

Full Factorial Functional Response Surface

Fractional Factorial Composite Design

Plackett-Burman High Throughput Screening

Mixed Level Compressive Sensing

Orthogonal Array Covering Arrays

Near-Orthogonal Array Generic Algorithms

The use of mathematical arrays for designing experiments is certainly not limited to statisticians. In recent years, there has been great interest by mathematicians in compressive sensing techniques which attempt to find small systems of equations designed to extract specified information, often by invoking principles of linear algebra, optimization, and penalty constraints. Similarly, genetic algorithms inspired by biological systems have been used to generate novel approaches for constructing experimental designs. See for example, Hamada et al (2001) and Lin et al (2015).

Compressive SensingFigure 4 illustrates a problem involving weights of coins described by Bryan and Leise (2013) to illustrate the basic idea of compressive sensing. There are 7 coins, and one is suspected of being a different mass. Note that each column gives the binary representation of the index associated with each of the 7 coins. It turns out that the bad coin can be identified by weighing


3 times proscribed by the binary representation. Mathematically, this type of problem can be solved as a linear optimization problem subject to a constraint. Let Φx = b, where b is assumed to have at most k nonzero components, and then find x, such that min ||x||1 is achieved subject to ||Φx-b||2 ≤ ε.

Bryan and Leise (2013) describe a problem involving 7 coins, one of which is suspected to have an aberrant weight. By writing out the index assigned to each coin in binary, it is possible to develop a design that requires only weighing 3 times, corresponding to sum of the weights of the coins in the 3 rows, to detect a single aberrant weight coin.

Covering ArraysAnother area closely related to experiment design is the idea of covering arrays. Covering arrays are used in software engineering to test software that can have large numbers of configurations. A subset of configurations is selected for testing so that for any subset of t variables, all combinations of the settings of the tvariablesoccurinthetestplan.KleitmanandSpencer(1973)andaNIST website provide further information on covering rays.

Designing Experiments in PracticeMy personal fascination with patterns and arrangements continues to this day and has influenced and motivated much of my statistical work. When designing real- world experiments, there are often practical considerations and constraints that must be taken into account, as discussed in Wendelberger et al (2009). Often, the challenges of a practical design problem can motivate new advances. The need to generate a mixed level design for a physical experiment eventually led to the development of an algorithm for generating orthogonal and near- orthogonal arrays for main effects screening, described in Lekivetz (2015). This algorithm goes beyond traditional hand calculations and can be used for relatively large computer experiments. Each experiment seems to have its own nuances and opportunities for innovation. As an example, recall the importance of understanding error structure, and consider the common situation where the settings of the experimental variables cannot be obtained exactly. Approaches for addressing variability in controlled experimental variables and the associated uncertainty that can arise in designed experiments may be found in Wendelberger (2010, 2015). When developing designs in practice, specific departures from the usual assumptions require careful thought, and often some ingenuity, to determine appropriate types of designs and corresponding analyses.

Summary and ConclusionsIn summary, we see that our complex world creates opportunities for Statisticians and Quality Professionals to contribute to increasingly difficult analysis challenges. At the heart of solving complex problems, we see the key concepts of sampling, error analysis, and experiment design playing a prominent role. As new problems arise, sampling, design, and analysis methods can be built on these concepts to address the increasing complexity of diverse interdisciplinary challenges.

AcknowledgementsInclosing,IwouldliketothanktheYoudenCommitteeandtheFallTechnicalConferenceProgramCommitteeforinvitingmetogivethisyear’sYoudenaddress.I’dalsoliketorecognizetheimpactofmygraduateadvisor,GeorgeBox,andtheUniversityofWisconsin Statistics Department for providing me with my initial introduction and training in Statistics and Quality and all of my colleagues in the Statistical Sciences Group at Los Alamos National Laboratory who have influenced my career.

I’d also like to thank my family: my parents Don and Marie Roth (both trained as chemists), my husband Jim who is a statistician, my daughter Barbara, who recently defended her PhD thesis on statistical methods for FMRI, my daughter Laura, who is contemplating a career in Statistics, and my daughter Beth, who is working in the healthcare field, making a difference in people’s quality of life through the use of scientific studies supported by data and statistical analysis. In addition, my family has had a numberofmathematically-orientedpetsovertheyears,includingadognamedSpline,andgerbils,KakuroandSudoku,shownin Figure 5, who were named after the mathematical puzzle games known by the same names.



REFERENCESBartlett,M.S.,Kendall,D.G.,“TheStatisticalAnalysisofVariance-HeterogeneityandtheLogarithmicTransformation,”J. of the Royal

Statistical Society, Series B, 8, 128–150, 1946.Block, S. and Tavares, S., Before Sudoku, The World of Magic Squares, Oxford, 2009.Box, G. E. P., “Robustness in the Strategy of Scientific Model Building,” in Robustness in Statistics, ed. By R. L. Launer and G. N. Wilkinson,

1979.Box, G. E. P., Hunter, J. S., and Hunter, W. G., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building,

2nd. Ed., Wiley, 2005.Bryan,KurtandLeise,Tanya,“MakingDowithLess:AnIntroductiontoCompressedSensing,”SIAMReview,v.55,3,547–566,Sept.2013.Complete Dictionary of Scientific Biography, Charles Scribner’s Sons, 2008, http://www.encyclopedia.com/doc/1G2-2830904751.html Covering Array Tables, National Institute of Standards, http://math.nist.gov/coveringarrays/coveringarray.html#ks, accessed on 27oct2016.en.wikiquote.org, https://en.wikiquote.org/wiki/I_Think_We’re_All_Bozos_on_This_Bus Gelman, A., Carlin, J., Stern, H., and Rubin D., Bayesian Data Analysis, Chapman and Hall, 1995.Hamada, M.S., Martz, H.M., Reese, C.S. and Wilson, A.G., “Finding Near- Optimal Bayesian Experimental Designs via Genetic Algorithms,”

The American Statistician, 55, 175–181 (2001), DOI: 10.1002/qre.1591.Johnson, Neil F., “Chapter 1: Two’s company, three is complexity,” 2009. Simply complexity: A clear guide to complexity theory (PDF).

Oneworld Publications. p. 3. ISBN 978-1780740492.Jones, B. and Nachtsheim. C., “A Class of Three- Level Designs for Definitive Screening in the Presence of Second- Order Effects,” J. of Quality

Technology, 43, 1, 1–15, 2011.Kleitman,D.J.,andSpencer,J.,“Familiesofk-independentsets,”Discrete Math, 6, pp. 255. 262, 1973.Lekivetz, R., Sitter, R., Bingham, D., Hamada, M. S., Moore, L. M., Wendelberger, J. R., “On Algorithms for Obtaining Orthogonal and

Near- Orthogonal Arrays for Main Effects Screening,” Journal of Quality Technology, Vol. 47, No. 1, 2–13, 2015.


Figure 5: Sudoku and Kakuro—Sudoku (left) and Kakuro (right) are gerbils whose names were inspired by mathematical puzzles

http://www.encyclopedia.com/doc/1G2-2830904751.html

http://math.nist.gov/coveringarrays/coveringarray.html#ks

https://en.wikiquote.org/wiki/I_Think_We�re_All_Bozos_on_This_Bus

http://www.uvm.edu/rsenr/nr385se/readings/complexity.pdf



Lin C. D., Anderson- Cook C. M., Hamada M. S., Moore L. M., and Sitter R. R., “Using Genetic Algorithms to Design Experiments: A Review,” Qual. Reliab. Eng. Int., 31, 155–167, 2015, doi:

Lin,Dennis,K.J.,“YoudenSquaretoGeorgeBoxtoLatinHypercube:TheEvolutionofExperimentalDesign,”W.J.YoudenAddress,FallTechnical Conference, Birmingham, Alabama, 2010.

McKay,M.D.,Beckman,R.J.,andConover,W.J.,“AComparisonofThreeMethodsforSelectingValuesofInputVariablesintheAnalysisof Output from a Computer Code,” Technometrics, 21, 2, 239–245.

National Research Council, Assessing the Reliability of Complex Models, Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification, The National Academies Press, 2012.

Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P., “Design and Analysis of Computer Experiments,” Statistical Science, 4, 409–434, 1989.

SteinerS.H.andMacKayR.J.,“ScaleCounting,”Technometrics, 46, 348-354, 2004.U.S. Army Operating Concept, Win in a Complex World 2020–2040, http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/

MilitaryReview_20160430_art009.pdf Vardeman, S., Hamada, M., Burr, T., Morris, M., Wendelberger, J., Jobe, J., Moore, L., and Wu, H., “An Introduction to Statistical Issues and

Methods in Metrology for Physical Science and Engineering,” J. of Quality Technology, Vol. 46, No. 1, 2014. Vardeman, Stephen B., Wendelberger, Joanne R., Burr, Tom, Hamada, Michael S., Moore, Leslie M., Jobe, J. Marcus, Morris, Max D., and

Wu, Huaiqing, “Elementary Statistical Methods and Measurement Error,” The American Statistician, Vol. 64, No. 1, 46–51, 2010.Wendelberger, J., Moore, L.M., and Hamada, M.S., “Making Tradeoffs in Designing Scientific Experiments: A Case Study with Multi- level

Factors,” Quality Engineering, 21, 143–155, 2009.Wendelberger, J., “Variation in Controlled Experimental Variables,” Quality Technology and Quantitative Management, Vol. 12, No. 1, 29–40,

2015.Wendelberger, Joanne R., “Uncertainty in Designed Experiments,” Quality Engineering, Vol. 22, 88–102, 2010.Xu Xu, Ben Haaland, and Peter Qian, “Sudoku-Based Space- Filling Designs,” Biometrika, 98, 3, 711–720, 2001.Xu Xu, Peter Qian, and Qing Liu, “Samurai Sudoku- Based Space- Filling Designs for Data Pooling,” American Statistician, 70, 1, 1–8, 2016.Youden,W.J.,Experimentation and Measurement, NIST Special Publication 672, U.S. Department of Commerce, 1997. (originally appeared

as a Vistas of Science book in 1962)

About the AuthorDr. Joanne R. Wendelberger has been a member of the Statistical Sciences Group at Los Alamos National Laboratory since 1992. In 2016, she was promoted to a senior level Scientist position, after serving in multiple leadership roles as an R&D Manager for the Statistical Sciences Group and the Computer, Computational, and Statistical Sciences Division. She previously worked as a Statistical Consultant at the General Motors Research Laboratories. She received her Ph.D. in Statistics from the University of Wisconsin in 1991, working with Professor George Box. Her research interests include statistical experiment design and test planning, statistical bounding and uncertainty quantification, materials degradation modeling, sampling and analysis for large- scale computation and visualization, probabilistic computing, and education modeling. She is a Fellow of the American Statistical Association (ASA) and a Senior Member of the American Society for Quality. She has served as an Associate Editor for Technometrics and as an ASQ representative on the Technometrics Management Committee. She has served as Chair and Program Chair of the ASA Section on Physical & Engineering Sciences, President of the ASA Albuquerque Chapter, and as a member of several conference organization committees and awards committees.

http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_20160430_art009.pdf

http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_20160430_art009.pdf


MINI-PAPERAttribute Agreement Analysis (AAA): Calibrating the Human Gage! by Jd Marhevko, VP Quality, Lean & EHS Systems, Accuride CorporationMBB, ASQ Fellow, CMQ/OE, CSSBB, Past Chair ASQ QMD

As a closet statistician, I use data every day to make predictions and decisions. The accuracy of that data is key in increasing the odds of my predictions being valid or not and my decisions being correct. When we measure something or make an assessment of the goodness of an item, we need to be sure that our result is correct. If it is not correct, we run the chance of one of two risks:

1. Making an Alpha (α) error where we inadvertently reject a good unit or 2. Making a Beta (β) error where we inadvertently pass on a bad unit.

We can look at α errors as an, “Aw, darn.” The burden and loss is borne mostly by the business when units are either unnecessarily scrapped and/or reworked. While there are other losses associated with α errors, the bottom line is that the business took the hit. These costs are often passed on to the customer. β errors cause customer risk. These are “Bad” errors where the unit is indeed defective and it was unknowingly passed on directly to the customer. An analogy might be a jury giving an innocent verdict to a defendant who really is guilty thereby enabling that person to go back into the community. β is truly “Bad.”

It is the management team’s responsibility to ensure that people are enabled to make the right pass/fail decision every time. If that’s not feasible, then they need a system that minimizes the chances of those α and β errors. This is usually easier to do when a physical gage is used such as a caliper or micrometer. But how about when the gage is a human and the decision is based on their assessment? This relies on their opinions, experience, personality and a host of other human characteristics. Think about when a doctor uses a light to look into your ears, eyes and throat. They are making an assessment as to whether or not your condition is normal or not. In the manufacturing world, how can a management/engineering team objectively deal with all of those touchy feely things to get a consistent and repeatable result? Can they really calibrate a human gage?

The answer is yes. One tool commonly used to manage this process is called an Attributes Agreement Analysis (AAA). An AAA* looks for 100% agreement in THREE (3) aspects of the inspection process:

1. Within “Myself.” If I look at a unit and fail it, will I consistently fail it when I see it again in the future?2. Between “Me and a Peer.” Did I consistently pass or fail a unit the same way as my peer did?3. To the Standard. If I passed or failed a unit, was I correct to the standard?

Look at this example of 3 sets of tests between me and a peer shown in Table 1.

The first unit was good per the standard. I passed it both times. Therefore, I agreed with me (yay) and the standard (double yay!). My peer also inspected the same unit and passed it both times. The second unit was a failing standard. I was inconsistent when I passed it (improperly) the first time and then failed it (properly) the second time. My peer did better than me and properly failed the unit both times. On the 3rd unit, both me and my peer were consistent within ourselves and with each other. However, we were both wrong to the standard. We both would have unknowingly passed on a defective unit creating a beta error. We need to better manage this risk.

Table 1: Example of tests

Sample Standard Me#1 Me#2 AgreeMe AgreeStd Peer#1 Peer#21 Pass Pass Pass Yes Yes Pass Pass2 Fail Pass Fail No No Fail Fail3 Fail Pass Pass Yes No Pass Pass

*Note: It is not the intent of this article to explain the formulae or statistics behind the AAA but to explain its overall use and general application


MINI-PAPER

KeepinmindthatthisformofevaluationwouldalsobehelpfultothosetryingtomitigateriskintheworldsofISO9001:2015and IATF 16949:2016. Most people evaluate gaging system risk from the variables perspective only. The AAA enables more control when variables measurement just can’t be used.

An AAA gives a series of graphs to show how the operators perform in general. Ideally, the goal is to be in 100% agreement all of the time. There are generally two ways to calculate the AAA%.

1. Compare the percentages of “Statistical Agreement” between the appraisers and the standard.2. Adjust the above Statistical Agreement percentage by the amount of agreement that happens by chance. This is known

asKappa (Κ) statistics1.TheAutomotive IndustryActionGroup (AIAG) recommends aKappa level of0.75 asacceptable2.

It is common that attribute decisions in manufacturing systems are not “life or death” but more often of a cosmetic nature. Scratches, dings, color shade, pits/voids, etc. are not usually measured with devices but given a quick glance for “okay-ness” and then passed along. In this type of application, the “non-adjusted” format of the AAAs “Statistical Agreement” is more generally applied. When using statistical software, be sure to know which type of Statistical Agreement results are being given. Minitab’s Assistantinversion17usestheStatisticalAgreementformwithoutadjustingforKappa1. This is very helpful. (If the AAA evaluation is of a more serious nature, remember this when you are next at the doctor’s office or visually evaluating radiographs,consideradjustingtheAAAresultswiththeKappastatistics.)

Figure 1: AAA example one


MINI-PAPER

Mostteamsseemtouseaninternalguidanceof>=85%foranon-KappaadjustedStatisticalAgreementoftheAAA.Notethatthis is the opposite of variables gaging. In variables gaging, one is trying to minimize the amount of percentage of error in the system; theoretically to 0% and lower is better. However, in AAA, the team is striving for 100% agreement.

This article primarily discusses the non-adjusted “Statistical Agreement” format. In the sample on the previous page, 3 operators evaluated the pass/fail criteria of 14 units. 7 were fail standards and 7 were pass standards. I hope that some of you recognize this AAA format which was shared as a member value tool to Statistics Division members a long time ago.

This team wiped out on their first try. There are several things that can be done to help them better align themselves and improve upon their results:

1. They can be re-trained on the errors that were originally made. Give them additional instruction and ideas on how to evaluate the units.

2. More standards need to be added into the mix. Usually, 30–50 samples are preferred. A fair mix of good, borderline and bad units are needed.

3. Run the AAA again as timely as feasible.4. Share the new set of results. Retrain to the gaps. Repeat.5. Running the evaluation is usually very fast…All the person is doing is looking at the samples and saying, “Pass, Fail,

Pass…”6. The collecting of the standards is usually the tougher aspect of a successful AAA. Coming up with 30–50 units across the

gamut of the pass/fail and borderline range along with the differing types of failure modes can take some time.

Figure 2: AAA example two


MINI-PAPER

Leveraging the AAA to empower your team to make a consistent and accurate decision can have a profound effect on your processes. By re-affirming the standards and actually providing a broader palette of units, performance generally improves. Within 2–4 iterations of the evaluation process, inspection personnel usually become very accurate with the AAA regularly exceeding 85%. This skill needs to be practiced on a regular basis. The act of inspection does not constitute practice because the full gamut of good, borderline, and bad are not typically experienced.

In the effort to do a good job, people trend towards over-rejection over time (alpha). Recalibrating the operators to the standards on a regular basis is key to long term success. Manage the regular AAA re-training like a gage calibration program. Bring in the personnel, let them assess the standards, verify their effectiveness and then release them to the environment. Identify the optimum frequency to avoid slippage in skills. Generally:

• Developthestandardsfirst.Getasolidsetof30–50samples(ormore)• “Calibrate”everyoneacrosstheprocess:Engineering,Supervision,OperatorandtheCUSTOMERprior to launching the

process• Useasamethodtobothtrainexistingpersonnelandlatertoqualifynewpersonnel.ConductAAAsrepeatedly(every

“X” weeks) to keep people from “drifting” towards “over-rejection” over time. • UseasamethodtobettermitigateriskfromaRisk-Based-ThinkingapproachtosupportupcomingISO-9001:2015

changes • Anticipatethatin-houserework/scrap(duetoover-rejectedalphaerrors)willdecline• Anticipatelessarguing/settlingofdisputes• WARNING: If this process is launched without re-aligning the customer in advance, anticipate that customer complaints

may increase. They may have been accustomed to a different caliber of product coming through. Once managed, anticipate a reduction in external rejects

SummaryAAA is a powerful, risk-mitigation process that empowers personnel to make the right decision more consistently. Its team dynamics helps to align individuals to improve their overall results. It can largely reduce alpha-based internal scrap and rework and beta-based external returns. It should not be overly burdensome to pull together a large enough sample set needed for the personnel to train against. After that is established, a regular frequency helps to sustain the level of training needed to be effective. This tool is deceptively powerful! I wish you much luck with its implementation and execution.

While this is the Statistics Division, if you’re very interested in this topic, please consider attending the Inspection Division conference on 14–15 September 2017 in Grand Rapids, MI where this topic and other measurement system concepts will more thoroughly explored.

References1. Minitab 17. Minitab Assistant White Paper. Attribute Agreement Analysis 2. AutomotiveIndustryActionGroup(AIAG)MeasurementSystemAnalysis(MSA).Pg.215(defnKappa)3. AIAGMSAPg.137Kappa0.75

About the AuthorJd Marhevko is the VP of QLMS & EHS for Accuride. Four Accuride sites have been recipients of the AME Manufacturing Excellence Award. She has been involved in Operations and Lean/Six Sigma for almost 30 years across a variety of industries. Jd is an ASQ Fellow. In 2016, Jd was awarded both the Shainin Medal and honored as one of the top 100 Women in Manufacturing by Washington DC’s Manufacturing Institute. With ASQ she also holds the CMQ/OE, CQE, CSSBB and is a ASQ Certified Trainer. She is an MBB and Baldrige Assessor. Jd is a Past-Chair of the ASQ Quality Management Division, a 23,000 member organization. She holds a BSE from Oakland University in MI and an MSA from Central Michigan University.


COLUMNDesign of Experimentsby Bradley Jones, PhDJMP Division of SAS

Celebrating statisticians: George E.P. Box

DuringtheInternationalYearofStatistics,weatJMPcelebratedfamousstatisticiansonamonthlybasis.I chose Professor George E.P. Box as the subject of my celebration. I was looking forward to writing this piece because I knew George personally and have been an admirer of his since the beginning of my career.

Sadly, George passed away in late March 2013, and I wrote a remembrance of him for the JMP Blog at that time. That blog post expresses what I would have written in a post celebrating him. So, instead of speaking in general about his life and accomplishments, in this post I will focus on one of his many great papers. My plan is to write several such blog posts this month, each emphasizing a different one of his wonderful

publications. One of the benefits for me is that I get to reread these papers.

In this post, I want to focus on the first of his two-part paper with J. Stuart Hunter on the family of regular two-level fractional factorial designs that was published in Technometrics in 1961.

This seminal paper is 40 pages long, and one thing I found notable about it was that the mathematical content did not go past arithmetic and a little algebra! Despite this, there are many fundamental results in this paper, but all are stated in natural language without formal proofs. That was refreshing.

How does the paper begin?The paper starts with a brief exposition of two-level full factorial design in k factors. It shows how these designs can estimate interactions of all orders up to the k-factor interaction. This provides the motivation and background for introducing a half fraction of the full factorial design. They illustrate the construction method using the 2(4-1) design showing how one starts with the full factorial design in three factors and then adds a fourth factor by computing the elementwise product of the first three factors.

What happens next?They now have a design with 8 runs that is just half as many runs as are in the full factorial design with four factors. With the full factorial design, you can estimate 16 effects—the overall average, 4 main effects, 6 two-factor interactions, 4 three-factor interactions and 1 four-factor interaction (16 = 1 + 4 + 6 + 4 + 1). Now with 8 runs, you can only estimate 8 effects. It turns out that the construction the authors use confounds the 16 effects of the full factorial into 8 pairs of effects. The average is confounded with the four-factor interaction. The 4 main effects are each confounded with one of the 4 three-factor interactions. Finally, the 6 two-factor interactions are confounded in 3 pairs (8 = 1 + 4 + 3).

Figure 1 on the next page shows the analysis using Lenth’s t statistic to identify significant effects. The values reported are half of the quantities Box and Hunter report, because they define their effects as being the difference in the response when changing from one level of the factor to the other. An effect is the change in the response due to a one-unit change in the factor. Since one level of the factor is coded –1 and the other is coded +1, each factor changes by two units going from its low to its high level. Thus, the effect of a one-unit change is half the effect of going from low to high.

Bradley Jones

Dr. George E.P. Box

Republished from a 3 May 2013 JMP Blog Post.

http://blogs.sas.com/content/jmp/2013/03/29/george-box-a-remembrance/


How does the rest of the paper go?Of course, the paper is much too long for me to cover everything Box and Hunter introduce—especially not in this level of detail. Here are some of the big concepts:

• Generalizingtheir4-factorexample,theyshowthatthebestwaytocreateahalffractionofa k factor full factorial design is to start with a full factorial design in k – 1 factors and then calculate the last column by computing the elementwise product of the original k – 1 columns. They also show that one can reconstitute the full factorial by combining a half fraction with another half fraction that where every value in the second fraction is obtained by multiplying the corresponding value in the first fraction by –1. This leads to the concept of a foldover design—a term they also introduce here.

• TheyintroducetheideaofdesignresolutionanddefineresolutionIII,IVandVdesigns.Introducingtheideaofasaturated design, they describe resolution III designs of 7 factors in 8 runs, 15 factors in 16 runs and 31 factors in 32 runs. They also throw a bone to Plackett and Burman (1946) mentioning their constructions of 11 factors in 12 runs, 19 factors in 20 runs, 23 factors in 24 runs, etc.

• Theyintroducetheideaofdesigngeneratorsandusethisideatoshowhowtoblockthefractionalfactorialdesignsingroups of runs that each have 2, 4, 8 or some other power of 2 runs per block.

• TheyshowhowtoobtaindesignsofresolutionIVbyfoldingoveradesignofresolutionIIIandintroducetheideaofdesign projectivity. For example, they state that every resolution IV design projects to a full factorial (or replicated full factorial) in any three of the factors. The benefit of this is that if only three factors turn out to be important, it is possible to estimate all the interaction effects of those three factors. And, it does not matter which three are important.

Where has design for screening gone in the 50+ years since then?It is a tribute to the combined power and simplicity of this approach that the regular two-level fractional factorial designs are still in frequent use today. The construction and analysis of these designs does not require a computer, which made them popular when computers were rare. Of course, the calculations can be a bit tedious, so having a computer do them for you makes for fewer errors and more free time.

In the same year as the publication of this paper, Hall published 5 different orthogonal arrays for 15 factors in 16 runs. The saturated design in the Box and Hunter’s paper was one of the 5. This paper was also fundamental as it turns out that all the orthogonal arrays 16 runs for fewer factors are projections of the Hall arrays.

Forty years later, Sun, et al. (2002) catalogued all the orthogonal 16 run designs for 5 to 14 factors. For 9 to 14 factors, the 16 run designs of Box and Hunter are all of resolution III, which means that main effects are confounded with two-factor

Design of Experiments

Figure 1: Analysis using Lenth’s t statistic to identify significant effects


Design of Experiments

interactions. Sun, et al. found designs in these cases where none of the two-factor interactions confounds a main effect. Instead, some two-factor interactions may be correlated either plus or minus one-half with a main effect. The benefit of these designs is that main effects can be identified without the built-in ambiguity that resolution III designs entail.

ReferencesBox, G. E. P. and Hunter, J. S. (1961) “The 2k-p Fractional Factorial Designs Part I” Technometrics Vol 3, No. 3 311–351.Hall, M. Jr. (1961). Hadamard matrix of order 16. Jet Propulsion Laboratory Research Summary, 1, 21–26.Sun,D.X.,Li,W.,andYe,K.Q. (2002),“AnAlgorithmforSequentiallyConstructingNon-IsomorphicOrthogonalDesignsandIts

Applications,”TechnicalReportSUNYSB-AMS-02-13,StateUniversityofNewYorkatStonyBrook,Dept.ofAppliedMathematicsandStatistics.

SPC and Statistics

Many obstacles exist for those who would use statistics. In 46 years of teaching statistics to non-statisticians I have discovered some distinctions that help students overcome these obstacles. While the distinctions outlined here have not been widely taught, I have found that they are essential to the proper use of both SPC and the traditional statistical techniques.

Some BackgroundIn Economic Control of Quality of Manufactured Product, published in 1931, Walter Shewhart was very careful to make a distinction between the techniques of statistical inference and the approach he was taking for the analysis of observational data (pp. 275–277). However, in 1935 E. S. Pearson glossed

over these distinctions and sought to reformulate Shewhart’s ideas in terms of the elements of statistical inference. In Pearson’s own words: “While the broad lines of Shewhart’s attack could not be questioned, it seemed that there was scope for sharpening the statistical procedures which he was using . . . ” (Industrial Quality Control, August 1967). Among the things Pearson “sharpened” was the idea that the data have to be normally distributed prior to placing them on a process behavior chart.

In his rebuttal to Pearson’s book Shewhart wrote: “we are not concerned with the functional form of the universe [i.e. the probability model], but merely with the assumption that a universe exists.” [italics in the original] (Statistical Method from the Viewpoint of Quality Control, p. 54). While we will consider what these two different approaches mean in practice, we need to begin with some useful distinctions.

Two Types of DataThe first difference between SPC and the techniques of statistical inference has to do with the type of data used by each. The techniques of statistical inference were developed to analyze experimental data. Such data are generally collected under different conditions with the purpose of determining whether changes in the input variables have an effect upon a response variable.

SPC was created for the analysis of observational data. As the name suggests, observational data are a by-product of some routine operation. These data may be deliberately and intentionally collected, but they are collected while the process is operated in an ordinary manner. Observational data simply track the underlying process while the input variables are held constant.

COLUMNStatistical Process ControlDonald J. Wheeler

Donald J. Wheeler


Statistical Process Control

Thus, the first difference has to do with the type of data being considered. The difference in the conditions under which these two types of data are obtained immediately reveals two additional ways that experimental data differ from observational data. An experiment will always yield a fixed amount of data. In contrast, in an observational study we can usually obtain additional data by simply waiting. Moreover, because experimental data are collected under special conditions, they tend to be more expensive than observational data. Table 1 summarizes these differences between observational data and experimental data.

Table 2: Different Approaches to Analyzing Observational and Experimental Data

Observational Studies Experimental Studies

Should Be No Signals Should Be Some Signals

Sequential Analysis Procedure Required All Data Analyzed at One Time

Conservative Analysis Used Traditional or Exploratory Analysis Used

Table 1: Three Differences Between Observational Data and Experimental Data

Observational Data Experimental Data

One Condition Present Two or More Conditions Present

Additional Data Available Fixed Amount of Data

Less Expensive Data More Expensive Data

Different ExpectationsWe also have different expectations for these two types of data. We expect our experimental data to represent the differences between the special conditions being studied. Thus, in an experimental study we are looking for differences that we have paid good money to create and that we believe are contained within the data. Moreover, we only get one chance to detect these differences when we analyze the data. The expectation that there will be signals within our data, the finite amount of data available, and the fact that we only get one chance to analyze the data, will combine to make us choose a more exploratory approach to the analysis than we might otherwise use. Hence, we commonly use a traditional five percent alpha-level in a trade-off to gain increased sensitivity to those signals we have attempted to create with the experiment.

On the other hand, when conducting an observational study, our data will generally be collected under one condition. As a result we will not expect to find any signals within the data. Furthermore, since any signals will usually indicate unplanned changes in the process, we will want to be sure about any differences we find before we take action. Since additional data will usually be available, we can often afford to play a waiting game with observational studies. These two characteristics combine to make us want to use a conservative analysis. And indeed, the limits on a process behavior chart provide a conservative analysis for each new observation added to the chart. Thus we have a very small risk of a false alarm, so that before we take action we will have strong evidence of a process change. Table 2 summarizes these different approaches.

The Study of LionsAnother way in which experimental and observational studies differ is in the nature of the variables under consideration. In an experimental study the objective is to discover cause-and-effect relationships. This means that we will manipulate potential causes (the input variables) and observe what happens to certain response variables. (Do any of the causes have an effect upon the response?)


As soon as we start to do an experimental study we quickly discover that there are more input variables than we can reasonably include in the experiment. So we have to make a choice about what input variables to include in the experiment (call these the X variables), and what variables to leave out of the experiment. This latter set of variables can be further divided into those input variablesthatareheldconstantduringthecourseoftheexperiment(theYinputvariables),andthoseinputvariablesthatareignored during the experiment (the Z input variables). This latter group would include the environmental variables that are beyond the control of the experimenter as well as those input variables that are thought to have minimal impact upon the response variable. Thus, there are three categories of input variables for every experiment:

1. those that are studied by the experiment, the X input variables;2. thosethatareheldconstantduringtheexperiment,theYinputvariables;and3. those that are ignored by the experiment, the Z input variables.

To keep the Z variables from undermining the experiment we tend to randomize how they show up in the study. This will shuffle the effects of the Z variables. The idea behind randomization is to get the effects of the Z variables to (hopefully) average out within each treatment combination over the course of the experiment. (Be sure to keep your fingers crossed!)

As a result of these three classes of input variables all experimental results must be interpreted with two caveats:

1. The results have to be interpreted under the assumption that randomization worked to average out the extraneous effects of the Z variables. (This assumption becomes more reasonable as the number of observations per treatment combination increases.)

2. TheresultshavetobeinterpretedinthecontextofthosespecificlevelsthatwereusedfortheYinputvariables.(IfsomeoftheYinputvariablesinteractwithsomeoftheXvariablesthenyourresultsmightbedifferenthadyouchosendifferentlevelsfortheYvariables.)

This is why experienced statisticians are always careful to look for those things that might have gone wrong during an experiment. In the words of George Box, “If you are looking for gold and accidentally find some silver along the way, stop and mine the silver.” WhenthingsgowronginanexperimentitisoftenacluethateithertheYvariablesortheZvariablesarehavinganeffectuponthe response variable.

Observational studies collect data in a completely different way. To understand this difference we need to make a distinction between those input variables that are controlled during routine operations (call these the control factors) and all the remaining potential input variables that are not controlled during routine operation (call these the uncontrolled variables). (In an experimental context,theXandYvariablestendtocomefromthesetofcontrolfactorsandtheZvariableswilltendtocomefromthesetofuncontrolled variables, but the correspondence is not exact.) Observational studies tend to observe the response variables while all of the control factors are held constant. So what can we learn from an observational study?

With all of the control factors being held constant the process ought to behave very consistently and predictably. If it does not do so, then it has to be one of the uncontrolled variables that is making its effects known. Unplanned process changes that occur while the control factors are held constant are signs that the set of control factors is incomplete.

While we may not know which of the uncontrolled variables caused an unplanned change, the change itself will focus our attention on the time and place where at least one of the uncontrolled variables changed. Observational studies may not prove that a cause and effect relationship exists, but they can alert us to the possibility of certain relationships. They are a powerful tool for discovery. Once we have an idea about a possible cause-and-effect relationship we can use simple experiments to examine our idea.

So what about the study of lions? There are many things we can learn about lions at the zoo. They are large. They eat meat. They have an impressive roar. But if you want to become an expert on lions, you are going to have to study lions in the wild. (Hopefully you can avoid being on the menu.) As you watch lions interact with their environment you will discover things that you would



never have learned at the zoo. Experimental studies are like studying lions in a zoo. Here you are seeking the answers to specific questions. Observational studies are like studying lions in the wild. Here you are opening yourself up to discovering things never before dreamed of. There is a time and a place for each type of study.

Some statisticians belittle observational studies by claiming that they prove nothing. However, observational studies have been around for a long time. They have their origin in Aristotle’s teaching that the time to discover those causes that influence a system is to look at those points where the system changes. While experiments may allow us to obtain definitive answers for specific questions, they are of little use when we do not know what questions to ask. Discovery must plant the seeds for experimentation, and observation is the mother of discovery.

Applications of TechniquesTechniques for a one-time analysis such as t-tests, ANOVA, and ANOM were explicitly created for the analysis of finite sets of experimental data. Sequential analysis techniques such as process behavior charts were expressly created for the sequential analysis of continuing streams of observational data.

As shown in Figure 1, we may adapt the sequential analysis techniques for use with finite sets of experimental data, but we cannot easily go in the other direction. One-time analysis techniques cannot be meaningfully adapted for use in situations requiring a sequential analysis technique. So, the two types of data require different analysis techniques. They also result in different objectives for the analysis.

Inanexperimentalstudytheobjectiveistoestablishaparticularrelationshipbeyondareasonabledoubt.Youwanttobeableto say: “Do this and you will get that.” To this end you will need to estimate some parameters. These parameters may be the parameters for a probability model, or they may be parameters for a regression equation, but either way, the objective is to obtain


Figure 1: Analysis Techniques for Observational and Experimental Studies


some reliable estimates for a set of parameters. To express the uncertainty in these estimates we often use interval estimates. These interval estimates will be dependent upon our choice of an alpha-level and the amount of data available. Together these two quantities will usually determine the appropriate critical value. Thus, the analysis of experimental data is characterized by alpha-levels, critical values, and an emphasis upon having enough data to obtain reliable estimates.

With an observational study we are not evaluating known changes but are instead trolling for unknown and unintentional changes. Here there is no probability model or regression equation with parameters to be estimated. The problem is not one of estimation but rather one of characterization. Process behavior charts seek to characterize the past process behavior as being either predictable or unpredictable. Has the process been operated predictably in the past, or does the process show evidence of unplanned and unexplained changes?

To make this characterization SPC uses generic, fixed-width limits. These limits provide a reasonably conservative analysis with virtually every type of homogeneous data set. Thus, no probability model has to be specified. No alpha-level is required. No critical values are needed. With this conservative, one-size-fits-all approach any signals found are almost certain to be real, and this allows us to reliably characterize the process behavior without going through the multiple steps commonly associated with the customized, fixed-probability limits of statistical inference.

Thus, as shown in Table 3, SPC differs from statistical inference in many ways. It was created for a different type of data. It is intended for use with a sequential stream of data rather than performing a one-time analysis. It has the objective of characterizing process behavior rather than estimating the parameters for some model or equation. It uses generic, fixed-width limits that are conservative for all types of homogeneous data sets rather than fine-tuning the critical values to achieve some specific alpha-level. And SPC seeks to detect and discover unknown factors that affect an existing process rather than trying to establish that a specific relationship exists between known variables.


Table 3: SPC vs. Statistical Inference

SPC Statistical Inference

Observational Studies Experimental Studies

Sequential Analysis One Time Analysis

Characterize Process Behavior Estimate Parameters for Model

Use Conservative Analysis Use Traditional Alpha-Levels

Use Generic Fixed-Width Limits Use Customized Fixed-Probability Intervals

Discover Unknown Factors EstablishRelationshipsforKnownFactors

Consequences of the DifferencesIf the generic limits of SPC show the observational data to be reasonably homogeneous, then we judge the underlying process to be reasonably predictable. At this point it will make sense to talk about the data as if they all came from a single process. If the data all come from a single process then the notion of process parameters makes sense. When the notion of process parameters makes sense, we can use our descriptive statistics to estimate the process parameters and extrapolate beyond the data.

If the generic limits of SPC show the observational data to be non-homogeneous, then we can reasonably conclude that exceptional variation is present. Because of the conservative nature of the generic limits, the presence of exceptional variation will be strong evidence that the underlying process is changing. When the process is changing the process parameters will be changing. When



the process parameters are changing the question of estimating those parameters becomes moot. Rather than estimating process parameters we need to be asking: “What is causing the process to change?”

If we do not address this question regarding the source of the process changes we will have missed an opportunity to improve the process. When we do address this question we will hopefully be making some changes in the process, and our old data will no longer be of interest. Either way, the shape of the histogram and the various descriptive statistics computed from our non-homogeneous data become irrelevant. Such data will merely be a collection of values coming from different processes. When the data are non-homogeneous any discussion of the shape of the histogram, or any attempt to estimate the process characteristics, will be like talking about the constellations in the night sky—a triumph of imagination over substance.

As long as the analysis is conservative, when we find a signal we will be justified in taking action. The question is not “What is the alpha-level?” but rather “What caused the unplanned change in the process?” And this is the objective of SPC, to detect the unknown process changes so that actions can be taken to turn the assignable causes of exceptional variation into control factors that are held constant during production, and thereby to reduce both variation and costs.

Figure 2: When the Data Are Homogeneous


Confusion or Clarity?Thedifferencesoutlinedhereareprofound.Anyfailuretounderstandthesedifferenceswillinevitablyresultinconfusion.Yetthereis a built-in trap that almost guarantees that confusion will reign. It has to do with the nature of statistical training.

Statisticians are trained in the techniques of statistical inference. We see the world in terms of experiments, alpha-levels, critical values, and the estimation of parameters. Eventually this mind-set becomes so ingrained that it becomes difficult to escape. When those of us who operate with this mind-set are presented with Shewhart’s approach we will usually seek to remold Shewhart to fit in with the elements of statistical inference. Pearson did this in 1935. I did this in the 1970s. Fortunately, I had Professor David S. Chambers and Dr. W. E. Deming to help me out of my confusion.


Figure 3: When the Data Are Not Homogeneous


Scientific Experiments in Root Cause Analysis for Health Care

The Joint Commission accredits and certifies 21,000 health care organizations in the United States. Among Joint Commission accredited organizations, Root Cause Analysis is the most commonly used system analysis method for identifying causal factors in patient safety events1. That is events that are due to system or process flaws not primarily related to the patient’s illness. In health care, these safety events can have serious consequences. They could result in sentinel events that result in death or severe harm1.

The occurrence of these safety events are a continuing problem1. Health care organizations are experiencing dramatic change. Changes in reimbursement, new technology, regulations and staffing

COLUMNStatistics for Quality Improvementby Gordon Clark, PhDProfessor Emeritus of the Department of Integrated Systems at The Ohio State University and Principal Consultant at Clark-Solutions, Inc.

Gordon Clark


Since Pearson lead the way, many others have fallen into the same trap by trying to “fill in the gaps” between Shewhart’s approach and the techniques of statistical inference. Hallmarks of these attempts are statements like the following.

• “Thedatahavetobenormallydistributedforaprocessbehaviorchart.”• “Youneedtotransformyourdatabeforeyouputthemonachart.”• “Youneed todefine a referencedistributionbefore youcancompute appropriate limits thatwillhave the correct

alpha-level.” • “Yourprocesshastobein-controlbeforeyoucanplaceyourdataonaprocessbehaviorchart.”• “Youcan’tputautocorrelateddataonaprocessbehaviorchart.”• “Theprocessbehaviorchartworksbecauseofthecentrallimittheorem.”• “Youhavetohavesubgroupsofsizefiveinorderfortheaveragecharttowork.”• “Youhavetohaveatleast30subgroupsbeforeyoucancomputeyourlimits.”• “Youneedtoremovetheoutliersfromthedatabeforeyoucomputeyourlimits.”• “TherearetwostagestousingSPC:Shewharttalkedaboutstageone.Instagetwoyoucanuseothertechniquesto

fine-tune the limits.”

All of the statements above are wrong. Dr. Deming categorized statements like these as “such nonsense.” As I have tried to show here, any attempt to fill in the gaps is based upon a failure to understand how SPC is different from statistical inference. The two use different approaches to different types of data and have different objectives.

There is a time and a place for the techniques of statistical inference. There is a time and a place for the use of SPC. To use both effectivelyyouneedtounderstandhowtheydiffer.Youalsoneedtoavoidthosewhotrytomergethetwointoone“unified”approach. Alpha-levels, critical values, distributional assumptions, tests for lack-of-fit, and the like all belong to the world of experimental studies and statistical inference.

From the very beginning the elements of statistical inference had no place in SPC. Today nothing has changed—they still have no place in SPC. SPC and statistical inference offer solutions to different problems. Understanding these fundamental differences between statistics and SPC can bring clarity. Seeking to unify statistics and SPC will simply create confusion.

If you have questions or comments contact me at <[email protected]>.



Statistics for Quality Improvement

levels contribute to these changes and the organization become more complex. These changes increase variation and risk to patients. In September 2013, researchers estimated that US hospitals had more than 400,000 premature deaths associated with preventable harm to patients2. Instances of serious patient harm were estimated to be 10 to 20 times more frequent than lethal harm.

Historically, root cause analysis was used to analyze sentinel events, but it shows great promise as a proactive tool. Health care organizations are using root cause analysis to investigate other risky events. They develop an understanding of contributing factors and their relationships. They can implement an action plan to reduce the risk to patients.

Root Cause Analysis is commonly used after a safety event has occurred to identify contributing causes. Failure Mode and Effects Analysis (FMEA) is a proactive technique to prevent process risks before they occur. Health care organizations should be familiar with both techniques to reduce risk.

Literature ReviewPercarpio, Watts and Weeks examined articles written in English and published by 2007 to review the effectiveness of Root Cause Analysis. They found thirty case studies, but only three of them had measurements of clinical outcomes to estimate the effectiveness or Root Cause Analysis (RCA) to improve patient safety. The results are:

• Apatientdiedafterfailuretocommunicateanesthesiarisk.Theteamimplementedariskassessmentprocedure.Mortalityimproved from 5% before implementation to 1% three years after implementation. Described as a case study below.

• AnRCAteamanalyzed90deathsoftransplantpatients.Theyinitiateddocumentingpatientstatusbeforethetransplant.One year patient survival increased from 81% to 93%.

• AnorganizationusedRCAtoreduceadversedrugevents(ADEs)duringa29monthperiod.TheoccurrenceofADEswas reduced by 46%.

Eight of the case studies used process measures to indicate the effect of RCA on patient safety. That is, they published measures of compliance of the recommended actions based on the RCA.

If a Root Cause Analysis does not estimate changes in clinical outcomes, the analysis may end before they discover the true root cause. They may stop the analysis when the team agrees on a cause that meets their biases or when stopping is convenient.

Barsalou (2016) recommends using the scientific method for performing an analysis and it should be used when doing RCA4. The iterative process of forming hypotheses about the root cause, testing the hypotheses, analyzing the results, forming new hypotheses gives the RCA team a much better chance of discovering the true root cause. In addition, the empirical tests provides estimates of the improved performance based on corrections for the root cause.

Health Care Standard for Root Cause AnalysisThe importance of Root Cause Analysis in Health Care has resulted in The Joint Commission promoting and publishing a guide to Root Cause Analysis1. This guide emphasizes the use of scientific experiments to discover the underlying root causes and give empirical evidence to the benefits of avoiding the root causes. The elements of the scientific method are:

• Determinewhatisknownnow.• Decidewhatneedstobelearned,changedorimproved.• Developahypothesisabouttheeffectofachange.• Testthehypothesis.• Assesstheresultsofthetest.• Implementsuccessfulimprovementsorre-hypothesizeandconductanothertest.

The scientific method can be used in either the Plan-Do-Study-Act (PDSA) cycle as shown in Figure 1 or the Define, Measure, Analyze, Improve, Control (DMAIC) process.


Statistics for Quality Improvement

Case StudyMcGinn, Conte et al (2005) present the Root Cause Analysis anesthesia risk case study mentioned in the Literature Review section.ItoccurredattheStatenIslandUniversityHospitalinNewYork.

The Medical Staff Performance Improvement Committee (MSPIC) received a report about a 78 year old woman admitted in 2000 after a fall in a nursing home. She was diagnosed with a femur fracture. She had had pulmonary disease, hypothyroidism and laryngeal cancer. An anesthesiologist evaluated the patient as an ASA III/IV indicating elevated risk. The patient was taken to the operating room and died after anesthesia induction. The Root Cause Analysis identified an inadequate preoperative assessment, poor communication between providers, and lack of a framework for high-risk preoperative evaluations. A pre-operative assessment tool included evidence-based guidelines for management of hypertension, use of beta blockers, and deep vein thrombosis prophylaxis.

Prior to the Root Cause Analysis, the mortality rate was 4.9% during 2000. Following the corrective action, the mortality rate decreased to 2.7% in 2001 and 2002. In 2003, it was 1%. Using Fisher’s exact test, the mortality reduction in 2003 of 79% (p = .0245) was statistically significant.

References1. The Joint Commission Resources (2015). Root Cause Analysis in Healthcare: Tools and Techniques, Fifth Edition.2. James, JT (2013). “A New Evidence-based Estimate of Patient Harms Associated with Hospital Care”, Journal of Patient Safety, 9(3),

122–128. 3. Percarpio,K.,B.V.Watts,W.B.Weeks(2008),“EffectivenessofRootCauseAnalysis:WhatDoestheLiteratureTellUs?”,The Joint

Commission Journal on Quality and Patient Safety, 34(7), 391–398.4. Barsalou, M. (2016). “A Better Way”, Quality Progress, November 2016, 38–42.5. McGinn, T., J. Conte etal (2005), “Decreasing Mortality for Patients Undergoing Hip Fracture Repair Surgery”, The Joint Commission

Journal on Quality and Patient Safety, 31(6), 31(6), 304–307.

Figure 1: Plan-Do-Study-Act


Z-Score

If a normal distribution has a mean (µ) of zero and a standard deviation (σ) of one, it is referred to as the standard normal distribution. When this is the case, areas under any normal frequency distribution curve can be obtained by performing a change in scale. This change of scale converts the units of measurement from the original (or “x”) scale into standard units, standard scores, or z-scores, by means of the formula:

zx – μ

= σ

In this new z-scale, the value of z, i.e., the z-score, is the number of standard deviations the correspondingvalueofxliesaboveorbelowthemeanofitsdistribution.Knowingthevalueofzallowsyoutodeterminetheareaunder the normal curve from one point on the “x” scale to another. Because of the unique properties of the normal distribution, i.e., the mean, median and mode have equal values; the area under the curve between any two values of “x” is equal to the probability of this event taking place.

The concept of the z-score has been advanced to include special cases of “z,” e.g.: ZU is a dimensionless index used to measure the location of a process, i.e., its central tendency, relative to its standard deviation and the process upper specification limit (USL). If the process frequency distribution is normal, the value of ZU can be used to determine the percentage of the area located above the USL.

ZL is also a dimensionless index used to measure the location of a process, i.e., its central tendency, relative to its standard deviation and the process lower specification limit (LSL). If the process frequency distribution is normal, the value of ZL can be used to determine the percentage of the area below the LSL.

ZMIN results from a comparison of the ZU and ZL values and is the smaller of the two. It is used to calculate CPK, the mean-sensitive Process Capability Index.

COLUMNStats 101by Jack B. ReVelle, PhDConsulting Statistician at ReVelle Solutions, LLC

Jack ReVelle

Reprinted with permission from Quality Press © 2004 ASQ; www.asq.org. No further distribution allowed without permission.

Announcing new ASQ Fellows nominated by the Statistics Division

Michael Hamada, Los Alamos National Lab, Los Alamos, N.M. — For significant research contributions in the design and analysis of experiments, measurement system assessment and reliability; for leadership in interdisciplinary collaborations to improve the practice of science for national security; and for dedicated service to the practice of quality.

Abbas Saghaei, Azad University, Tehran, Iran — For effective training of students at universities and professionals across multiple industries; for implementing quality practices across many organizations that deliver quantifiable financial benefits; for outstanding contributions to the quality movement within Iran; and for significant research and publications at the national and international level.

http://www.asq.org


System Reliability as an Emergent Property

Reliability is the probability that an item will function as intended under stated conditions for a stated period of time. All products will eventually fail. This is the natural law. Product reliability should be viewed as an emergent property. Emergence is a strong concept in Systems Thinking. Simplistically put, Systems Thinking is the idea of holistic thinking—thinking in terms of the whole instead of looking at parts separately from the whole. A system consists of two or more interconnecting and interacting parts. From the system’s standpoint, emergent properties are properties of the system as a whole that emerge as a result of the interaction of the parts within a system or as a result of interaction of the system or its components with its environment. In this aspect, product reliability can be viewed as an emergent property.

The history of Reliability Engineering goes back to the twentieth century when vacuum tubes were used in electronic equipment like radios, televisions, radars and other technological equipment.[1] The vacuum tube was considered to be a great success for the Allies during the Second World War, also called the “wizard war.” However, the vacuum tubes suffered from reliability issues and were required to be replaced five times more than any other electronics component of that time. This prompted the US Department of Defense to launch initiatives to look into the vacuum tube failures. This and other military led initiatives by different countries launched the Reliability Engineering field. The Systems Thinking approach shifts the focus from “Component Reliability” to “System Reliability.” The famous American Systems Thinker Dr. Russell Ackoff has said—“It is not the part’s performance that is critical. It is the way that the part performs with other parts that is critical.” The structure of a system determines its behavior.

Consider the case of a smart phone as the product to look further into systems. A smart phone is made up of several components. A smart phone by itself does not have a function or a purpose. It is a mechanistic system. When the owner interacts with the smart phone, it has a purpose. The user and the smart phone are viewed as a purposeful system. The system’s reliability is dependent upon its design, its components, the way it was manufactured, the environmental conditions it went through, other accessories used with it and finally the way the user interacts with it. Another important concept in Systems Thinking is the presence of nonlinear cause and effect relationships. The cause and effect can also be far apart in time and most of the time there are multiple causes leading to the effect under multiple enabling conditions. All these factors make the emergent property of reliability hard to understand and predict. An example of the nonlinear effect is the 1986 Space Shuttle Challenger accident where the interaction of an O-ring with the cold environment led to the loss of seven crew members.

COLUMNTesting and Evaluationby Harish Jose

Harish Jose

Figure 1: System Reliablity- Smart Phone Example


Testing and Evaluation

Thinking in terms of the whole is quite useful since it puts the focus on the interactions between the parts within a system and also the interaction of the parts with its environment. A component does not generally fail in isolation. It fails when it is interacting with other components or with the environment. When viewed in the light of systems thinking, it becomes evident that we should focus on the interactions that the product and its components can face. Thinking in terms of the whole can also make us ask different questions. Instead of asking why something is failing, we might start to also ask why is something succeeding despite all of the identified risks. The two questions lead to different thinking processes.

One of the best examples of thinking in terms of System Reliability is Google. Google does not think of System Reliability in terms of component reliability. Google has not developed a strong and reliable data server to support its web- based products. Google’s goal is system optimization and not local optimization in terms of a reliable central drive. Google treats component failures as the norm rather than the exception.[2] Google has built a file system that consists of thousands of inexpensive commodity parts. It relies on the redundancy of the data by storing data across thousands of inexpensive drives, and for redundancy the data is replicated multiple times more. Thus even if a hard drive crashes, there is no disruption. The crashed hardware is immediately and automatically taken out of the system and another drive is used in its place. In a recent post, Eric Brewer, VP of Infrastructure at Google said; [3]

We need to optimize the collection of disks, rather than a single disk in a server. This shift has a range of interesting consequences including the counter- intuitive goal of having disks that are actually a little more likely to lose data, as we already have to have that data somewhere else anyway. It’s not that we want the disk to lose data, but rather that we can better focus the cost and effort spent trying to avoid data loss for other gains such as capacity or system performance.

Dr. Ackoff has also said that “Quality ought to be directed at effectiveness”. In a similar vein, I am paraphrasing him; “Reliabilityoughttobedirectedateffectiveness”—doingtherightthingsinsteadofjustdoingthingsright.Knowyoursystemand other systems interacting with it; focus on system’s reliability as the top priority and come up with holistic solutions that meet your needs.

Always keep on learning…

[1] Enrico Zio. Reliability engineering: Old problems and new challenges. Reliability Engineering and System Safety, Elsevier, 2009

[2] Sanjay Ghemawat, Howard Gobioff, and Shun- Tak Leung. The Google File System, 2003

[3] Eric Brewer. Google seeks new disks for data centers. Google Cloud Platform Blog Tuesday, February 23, 2016

About the AuthorHarish Jose has over 8 years experience as a Quality Engineer in the medical devices field. He is a graduate of the University of Missouri- Rolla (USA) where he obtained a master’s degree in Manufacturing Engineering and published two articles. He is an ASQ member with multiple ASQ certifications including Reliability Engineer, Six Sigma Black Belt and Quality Engineer. He has subject matter expertise in lean, data science, database programming and industrial experiments. He can be reached at [email protected]. His LinkedIn profile is available at https://www.linkedin.com/in/harishjose.


https://www.linkedin.com/in/harishjose


COLUMNStandards InSide-Outby Mark Johnson, PhDUniversity of Central FloridaStandards Representative for the Statistics Division

Some General Thoughts on Standard

In the course of these columns, I have focused primarily on statistical standards—after all, my involvement has been generously supported by the Statistics Division of ASQ. Since I spent the fall 2016 semester in Shanghai, China, I could not help but notice generally the contrast as to what constitutes “standards” generally in the two very different countries and cultures. By being immersed in the day to day activities in another country, one becomes very aware of how one’s own country operates and works in ways that we take for granted. It is analogous to understanding language—I personally did not appreciate English grammar until I took German in college. For this column, I have collected some rather random thoughts on our Shanghai “adventure,” as I am rushing to provide this overdue contribution to our Statistics Digest editor, Matt Barsalou.

For background on Shanghai, it is the world’s third largest metropolis in terms of population (24 million) located in the delta oftheYangtzeRiver,withtheHangpuRiverseparatingthehistoricalBundtfromthemagnificentandrelativelynewfinancialdistrict with its super tall skyscrapers. Shanghai Province is more populated than Florida while being about the size of Palm Beach County (which will be getting increased attention in the next four years). There are a lot of people, and attractions tend to be crowded. Below are two photos of Shanghai.

Shanghai is perhaps the most international of Chinese cities and is probably the easiest city for a western to reside in for an extended period. Shanghai has historically had a significant British (think Bundt for those who have visited the city) and French presence. The French concession area is a particularly attractive area of Shanghai. The Shanghai office of ASQ is located in the French Concession area and appears to be increasing its role in the Chinese community. A local I met remarked that in some respects China is where the US was with the onset of Six Sigma. I only saw a single instance of ISO 9000 certification displayed while in China, whereas this is quite common to see in Tokyo.

Living for four months in a foreign city cannot help but allow one to observe things that are likely overlooked compared to what is discovered while attending an international conference with limited time to explore the environs. Very short trips into a country tend to highlight what is different (and usually inconvenient) from one’s home country or state. Here I would like to mention some of the more interesting things that we encountered that could be beneficially adapted in the US or made us recognize how comfortable our own existence is back home. Ultimately, standards is all about making things the same in some respects.

Mark Johnson


Standards InSide-Out

Signal lights at intersections have countdowns for how much longer the light will be green as well as how much longer the light will be red. At large intersections, the countdown can commence at 95 (seconds). It can seem like an eternity and encourages hurrying to make a crossing. The traffic configurations for some intersections have the left turn lane in the rightmost lane. Since motorbikes stay on the right, this allows both types of vehicles to queue together on the right. It actually works quite well. Speaking of traffic flow, there are giant LCD displays posted strategically above major highways indicating the real time flow (green for full speed to red for stop- and-go). These graphical displays are easy to understand and compare favorably to US Interstate messages such as “State Rd 50 exit 15mi, 18mn.” Taxis in Shanghai are abundant and relatively cheap and the ultra- modern metro (subway) is about $0.50 end- to-end for most lines. At rush hour, pedestrian one- way traffic is in place, which facilitates the movement of massive numbers of commuters. The longer distance train service out of Shanghai was also impressive. I took a fast train (300+km/hr) from Shanghai to Jinan to attend a conference and a one way ticket was about $100. Door- to-door was faster and more comfortable by train than with flying with a domestic airline. The trains and metro were always on time and came with sufficient frequency that I was never squashed into a compartment. The most challenging part of traveling in China was procuring the tickets. On line with the help of a local was the easiest, while buying a ticket at a station was sometimes problematic—not every bus station even sells tickets with some requiring advance purchase online (with no English option on the website).

We take for granted having restrooms in restaurants, but that is not the case in Shanghai. A scarce few have their own, but most establishments in malls and shopping plazas rely on community restrooms operated by the plaza. Usually, an individual was assigned to the restroom facility to keep it cleaned. Even along shopping avenues, it could be that the available restroom might be across the street in another building, again with its own attendant. One quickly learns to carry along one’s own tissues, as these are not generally supplied.

Asatennisplayer,IwasdeterminedtoplayabitwhileinChina.Wefoundatennisclub(KomeTennis)nearourapartmentforwhich I arranged a weekly hitting session with a young tennis pro (called a “coach” but with the language barrier it was really just a rally session). All of the pros at the club seemed to have the same style of strokes (big topspin forehands, two handed backhands) while they park it on the baseline. It is as if they all learned from the same school of tennis instruction. Here this standardization may not be idle as my competition hated to return backspin. We were fortunate to be in Shanghai for a Rolex Master’s tournament in a new tennis park facility southwest of town with a retractable roof center court, two other lesser but excellent smaller stadia and loads of practice courts and training facilities. Buying the tickets was a bit of an ordeal, partly due to our non- Chinese language skills and clumsy ticketing service. We eventually got in and saw Tsonga, Monfils and Djokovic and best of all, the Bryan Brothers in action. The hour and a half taxi ride back to our apartment through Shanghai was only $30 while the round of 16 session was around $70 for center court tickets.

The most challenging aspects of the trip were the food and the internet—perhaps the two things we most take for granted at home. The internet is controlled and slow. To get anything done, one must purchase a VPN which permits the use of Google andreadingtheNYTimessite.Thebandwidthmadeitchallengingfor“telecommuting”sothatFacetimeorSkypewereerratic.The food may be challenging for Westerners. We cowardly tended to avoid the local mom and pop restaurants with a very few exceptions. We found an excellent Italian restaurant (Mylk) and there was an Outback Steakhouse not too far away. We ate a great deal of noodles and Western coffee shops were everywhere. Fortunately, there was also an international grocery nearby so we cooked a good deal in our apartment. Since we carried all our groceries from there (20 minute walk), we learned to appreciate the convenience of driving in the states.

A visa is required to visit China, and eventually we got 10 year tourist visas having a 60 day maximum stay (side trips to Taipei and Tokyo trips got us re- entry). There is plenty more to see and experience in China. We would very much like to return on a future date.


FEATUREPredictive Analytics: Is Your Problem Collinearity, Sparsity, or Both?by Marco S. ReisCIEPQPF, Department of Chemical Engineering, University of Coimbra, Portugal

Large datasets resulting from passive collection schemes (happenstance data) often present a number of features that may turn their analysis more complex, especially for practitioners not so well- acquainted with the nature of data and the methodologies available to cope with their characteristics. Apart from some formal training in the initial stages, successful analysts are able to learn and capitalize on their experience, extrapolating their findings to new situations in a process where a certain intuition regarding the data generating mechanisms and the inner mechanics of the analytical methods are successively assimilated and a conceptual map begins to be established between data (with their structure) and methods (with their algorithms). In this short article, a brief overview of these two elements is provided in the context of multivariate and high- dimensional processes, namely: i) features commonly found in data—the focus being in industrial data; ii) classes of methods available to handle them—centred around predictive analytics frameworks. The main goal is to catalyse the assimilation process of effective solutions to complex data analysis problems.

The Nature of Passively Collected (Happenstance) DataData collected in modern industrial facilities has a number of features that can raise serious difficulties for people trained in the analysis of clean and well- organized two- way data tables. There are essentially two levels of problems to cope with (Figure 1). One level regards the fundamental inner structure of data—the core data structure. This level is to a great extent independent of the way data is collected and is linked to the underlying phenomena taking place. Common features characterizing this level are: high- dimensionality (many variables characterizing the behaviour of the process), correlation (including the inner network of relationships; see Reis (2013b)), sparsity (not all variables are connected with the target of the investigation), autocorrelation (or dynamics—static, stationary, non- stationary, at a single scale or at multiple scales; Reis (2009)) and nonlinearity. These features can be present with different levels of intensity or even be absent (for instance, in well- controlled continuous processes nonlinearity is usually not present or perceived, given the tight operational windows). The most pervasive feature of industrial data is certainly the existence of many variables with mutual correlations. Correlation is just a natural consequence of the well- known fact that the number of sources of process variation are much smaller than the number of sensors collecting data, which implies that the information they convey is necessarily redundant. Data correlation is therefore a reflection of process redundancy, which is further promoted by the existence of underlying fundamental laws (e.g., mass, energy and momentum conservation) that increase the linking between variables, as well as by the existence of redundant sensors, motivated by accuracy and reliability considerations.

The second level of problems is closely connected to the data acquisition process—the tangible data structure. This second level can be an important source of complexity in data analysis, involving features such as the presence of noise, outliers, missing data, multirate data (data with different acquisition rates—for instance, quality variables are usually made available at much lower rates than process variables; see Reis, Rendall, Chin, and Chiang (2015)), multiresolution data (data with different levels of granularity—e.g., pointwise process data together with time averages computed with different aggregation periods or with samples that are composed along batches, shifts, etc.; see Reis and Saraiva (2006a, 2006b)). Hitherto, no method is able to cope with all the features appearing in the tangible level of the data structure, but approaches have been developed to handle subsets of them with increasing compatibility and resilience with respect to even the wildest scenarios of data collection.

This article is focused on how to handle some of the core data structure most pervasive features in the development of predictive analytics solutions, namely: collinearity (presence of correlation in the regressors or

Figure 1: The two levels of features that compose the data structure


Predictive Analytics: Is Your Problem Collinearity, Sparsity, or Both?

input variables) and sparsity (relevant predictive information concentrated only in a few critical regressors). From our experience, these features create a first line of challenges to practitioners interested in applying predictive methods to their problems, and therefore we find it opportune to review the classes of solutions available to handle them.

A Systematization of the Plethora of Predictive Analytics MethodsAs a part of formal training or after a few incursions to real problems, data analysts become well aware of the limitations of the application of ordinary least squares (OLS) methods for developing predictive solutions in high dimensional problems. Applying OLS in this context in order to predict a given response is hardly a solution by itself. Firstly, a large number of regressors are included in the model, most of them unnecessarily (due to variables sparsity), which must be discarded in a second stage of analysis. Secondly, it may not be even possible to estimate a first model, or such an estimate would be highly unreliable, due to the presence of collinearity in the regressors because the presence of collinearity leads to a problem in the OLS solution, namely in the inversion of the extended dispersion matrix of regressors, which becomes ill- conditioned if regressors are correlated or even singular if the matrix is rank deficient. Therefore, practitioners need to make use of their data analytics toolkits in order to proceed with the development of an adequate predictive solution and this is where some important decisions need to be made.

Even though data analysts tend, rather understandably, to rely on the tools they are more familiar with and/or that have brought them successful experiences in the past, a structured overview of the range of solutions available may open their perspectives to other more opportune approaches for certain types of problems. There is certainly no “one size fits all” analytical methodology and the most appropriate solution should be found on a case by case analysis, even though some methods do have a rather broader application scope. In this context, we propose an arrangement of the methods for handling collinearity and/or sparsity in four classes of techniques (Figure 2) (Rendall, Pereira, & Reis, 2016). The content of each class will now be briefly described together with a reference to some of its well- known representatives; the list of methods provided in each class is just for illustration purposes and is not intended to be exhaustive.

Variable Selection Methods: This class of methods address the collinearity problem using the following rational: if what is causing the problem is the presence of correlated inputs and if correlation means the existence of redundancy between them, then one way to solve it is to choose only those regressors that are more predictive of the response; all the others that do not bring any additional significant added- value in this regard are set aside—either because they are correlated with the ones selected to be included in the model (collinearity), or because they simply do not carry any prediction capability of the response and thus are useless to the model (sparsity), or both. A large number of variable selection methodologies are currently available, including the well- known forward stepwise regression method (FSR) based on the partial- F test, greedy and exhaustive enumeration algorithms such as the Best Subsets (BS) and methods inspired in nature, such as Genetic Algorithms (GA), that mimic evolution theory (in this case, the survival of the Darwinian fittest principle).

Figure 2: Classes of methods for handling collinearity and/or sparsity in predictive problems involving large datasets



This class of methods is adequate when variables are weakly correlated or when variables measurements are costly and therefore there is a pressure to make the model as parsimonious as possible. These models are quite easy to interpret, but the conclusions may be easily flawed by the presence of correlations among regressors. An interesting aspect of this class of methods is that they address simultaneously the presence of collinearity and sparsity, which gives them an interesting range of applications.

Latent Variable Methods: Complex systems call for truly multivariate approaches; an aspect that variable selection methods may not be able to accommodate. In this context, the latent variable class of methods has gained some popularity for dealing with high- dimensional correlated processes. The rational is that if correlation is a reflection of redundancy, then the true dimensionality of the system is much lower than the observed one, i.e., that given by the total number of variables under analysis. Therefore, these methods look for some lower dimensional subspace that concentrates most of X-data variability (as in Principal Components Regression; Jackson (1991)) or that presents high covariance with the response (as in Partial Least Squares; Wold, Sjöström, and Eriksson (2001)), and use the set of variables defined in such subspace, as the new regressors, also called latent variables. A latent variable is computed as a linear combination of the original ones, and its values correspond to projections of the original high- dimensional observations into its axis. No variables are discarded in this class of methods, but they receive different weights according to their predictive power. Therefore, this class of methods are most useful for handling collinearity, rather than sparsity.

Penalized Regression Methods: These methods circumvent the OLS collinearity problem by penalizing the size of the estimated regressor coefficients. In this way, these estimates will be slightly biased, but become more stable (with less variance) and thus on average, will fall closer to the “real” underlying parameters. This is the bias- variance trade- off interpretation of penalized regression approaches, but there are other equivalent rationals for these methods (e.g., the stabilization of the inversion operation in OLS).

One can found in this class of methods approaches such as ridge regression (RR), LASSO, elastic nets (EN) and support vector regression (SVR). RR imposes a square penalty in the coefficients magnitude, limiting large coefficients values: many variables tend to present small but nonzero coefficients. On the other hand, LASSO imposes a penalty on their absolute values, allowing some variables to have large coefficients while others are effectively shrink to zero. Thus, LASSO has also a built- in variable selection capability. EN combines both aspects of RR and LASSO. SVR is another regression method that attempts to minimize the values of the regression coefficients, while the estimation process does not tolerate (or penalize) errors above a given threshold.

Machine Learning Methods: We include in this class those methods proposed in the broad field of Machine Learning, that are mostly of an algorithmic nature (i.e., not necessarily built upon an a priori defined probabilistic model structure for the data generating mechanism). Examples include the so called ensemble methods, whose goal is to improve the prediction ability of individual models, by combining many estimates in a suitable fashion. Representatives of this category are: bagging of regression trees (RT), random regression forests (RF) and boosting of regression trees (BT).

Some Illustrative ExamplesIn this section we illustrate how the most suitable analytical methodologies for addressing a specific multivariate or high- dimensional problem can be interpreted on the basis of the underlying data generation mechanism. The examples shown are simple and should make us aware of the importance of studying the problem and analysing the process (the data generating mechanism) before applying our favourite predictive modelling tools for crushing the data. Let us consider the following three scenarios. The first two are simulated scenarios, whereas the third one corresponds to a real case study.

Scenario 1. Sparse linear model. The first scenario is based on simulated data from a standard linear model where some variables have influence on the response, whereas others have no effect at all. It consists of 100 observations from 20 predictor variables: for 10 variables the regression coefficients were generated from a uniform distribution, U(–4, 4), while the remaining 10 variables have 0 coefficients—so the problem is moderately sparse. Furthermore, a small amount of noise was added to the response (SNR = 20) to simulate realistic noisy conditions.

Scenario 2. Latent variable model. The second scenario consists of a latent variable model structure with 4 latent variables whichgoverntheobservedvariabilityinbothXandY.Sincethistypeofproblemsisusuallycharacterizedbyahighnumberof



variables,100inputvariableswereconsidered(asbefore,onlyoneoutputvariable,Y,wasgenerated).Thedatasetiscomposedby500 observations, also contaminated with a moderate amount of i.i.d. Gaussian noise.

Scenario 3. Wine age prediction. The third and last scenario concerns a real world data set, collected to develop predictive approaches for wine age prediction (Pereira, Reis, Saraiva, & Marques, 2011). Wines are characterized by many properties and among them age is one of the most relevant feature as it is directly related to the market value and quality. In this study, 52 samples of Madeira wine collected over an extended period of time (from the same grape variety, Malvasia) were analysed by 3 different analytical techniques with each one providing a set of predictor variables: high- performance liquid chromatography (HPLC), gas chromatography- mass spectroscopy (GC-MS) and UV- vis spectroscopy. In this article, we are going to focus on the information collected with HPLC, which consists of the compositions of 23 phenolic compounds, 2 furanic compounds and 7 organic acids. These are the regressors used for predicting the ageing time of Madeira wine.

A wide variety of methods for handling collinear and sparse data structures were applied to the three application scenarios. These methods belong to the four classes of methodologies illustrated in Figure 2 and were briefly referred above. The predictive performance of the methods was accessed using a double cross- validation scheme from which the cross- validation R 2

(R 2CV) was

computed (Rendall et al., 2016). Table 1, summarizes the results obtained in each scenario, namely the methods presenting the best predictive performances and their associated R 2

CV.

Scenario Best performing methods R2CV

1. Sparse linear model FSR / EN / BS 0.93 / 0.92 / 0.92

2. Latent variable model EN / PCR / PLS 0.95 / 0.95 / 0.95

3. Wine age prediction BT / RR / EN 0.99 / 0.96 / 0.96

Table 1: Summary of results obtained in the analysis of the three application scenarios

For the first scenario, FSR presented the best predictive performance and was also able to select the most important predictors (not shown). This method was followed by EN and BS. Thus, the class of variable selection methods are adequate for handling this data structure since they do have the appropriate mechanisms to extract the relevant model structure. The class of penalized regression methods also led to good results. A justification for this, is their embedded variable selection capability (EN and LASSO were in the top 4).

As to the second scenario, latent variable methods presented a very good performance with PCR and PLS having the second and third best overall performances, respectively. Rather curiously, EN presented the best performance even though the difference was not statistically significant when compared to PCR and PLS. Again, matching the models prior assumptions and the data generating mechanism leads to better predictive results as a consequence of achieving a more parsimonious description of the data that finally leads to more stable predictions.

Finally, the results for the third scenario indicate that BT is the best method, meaning that even though sparsity and collinearity are present (see the good performances of RR and EN), nonlinearity play also an important role, which it is not captured by the other approaches. In fact, the evolution of the wine chemical composition along time does not follow a linear trend, and this result is expected.

ConclusionsBesides generating large amounts of data, the evolution of industrial systems brings new types of processes and data generating mechanisms. In this article we illustrate, through three examples, that there is potential for improving data analysis by combining our knowledge of the analytical tools with process specific information. This can lead not only to better predictive performances as illustrated here, but also to improved diagnosis and interpretation of the results achieved , which is a very important aspect that is not covered in this article. In this context, four classes of predictive methodologies were proposed and cases where they are



expected to lead to good results presented. But the spectrum of analytical tools available for handling the increasing complexity of processes does not resume to this already rich and organized set of methods. In the quest for selecting the best match between analytics and the data generation mechanism, we often find out that data is organized in natural groups, whose composition may be known a priori (e.g., variables from different stages in a production line, etc.), or not (e.g., groups of genes that are active in a given physiological condition). In these cases, the above referred single- block methods should give place to multiblock methods, able to incorporate this knowledge about the underlying systems structure—either for the cases with known block compositions (MacGregor,Jaeckle,Kiparissides,&Koutoudi,1994;Tenenhaus&Tenenhaus,2014;Westerhuis,Kourti,&MacGregor,1998)or unknown block compositions (Reis, 2013a, 2013b).

In summary, the essential message is that we, as data scientists, statisticians or engineers, should actively look for the analytical tools providing the best fit to the underlying structure of the process and of available data, as well as the adequate complexity balance for achieving the desired goals. This certainly requires a complex mixture of good science and experience- driven common sense. I tried to share a bit of my experience—so please feel to share yours!

ReferencesJackson, J. E. (1991). A User’s Guide to Principal Components.NewYork:Wiley.MacGregor,J.F.,Jaeckle,C.,Kiparissides,C.,&Koutoudi,M.(1994).ProcessMonitoringandDiagnosisbyMultiblockPLSMethods.

AIChE Journal, 40(5). Pereira, A. C., Reis, M. S., Saraiva, P. M., & Marques, J. C. (2011). Development of a fast and reliable method for long- and short- term wine

age prediction. Talanta, 86, 293–304. doi: http://dx.doi.org/10.1016/j.talanta.2011.09.016Reis, M. S. (2009). A Multiscale Empirical Modeling Framework for System Identification Journal of Process Control, 19(9), 1546–1557. Reis, M. S. (2013a). Applications of a new empirical modelling framework for balancing model interpretation and prediction accuracy through

the incorporation of clusters of functionally related variables. Chemometrics and Intelligent Laboratory Systems, 127, 7–16. Reis, M. S. (2013b). Network- Induced Supervised Learning: Network- Induced Classification (NI-C) and Network- Induced Regression

(NI-R). AIChE Journal, 59(5), 1570–1587. Reis, M. S., Rendall, R., Chin, S.-T., & Chiang, L. H. (2015). Challenges in the specification and integration of measurement uncertainty in

the development of data- driven models for the chemical processing industry. Industrial & Engineering Chemistry Research, 54, 9159–9177. Reis, M. S., & Saraiva, P. M. (2006a). Generalized Multiresolution Decomposition Frameworks for the Analysis of Industrial Data with

Uncertainty and Missing Values. Industrial & Engineering Chemistry Research, 45, 6330–6338. Reis, M. S., & Saraiva, P. M. (2006b). Multiscale Statistical Process Control with Multiresolution Data. AIChE Journal, 52(6), 2107–2119. Rendall, R., Pereira, A. C., & Reis, M. S. (2016). Advanced predictive methods for wine age prediction: Part I—a comparison study of

single- block regression approaches based on variable selection, penalized regression, latent variables and tree- based ensemble methods. Talanta, Accepted (in press).

Tenenhaus, A., & Tenenhaus, M. (2014). Regularized generalized canonical correlation analysis for multiblock or multigroup data analysis. European Journal of Operational Reserach, 238, 391–403.

Westerhuis,J.A.,Kourti,T.,&MacGregor,J.F.(1998).AnalysisofMultiblockandHierarchicalPCAandPLSModels.Journal of Chemometrics, 12, 301–321.

Wold, S., Sjöström, M., & Eriksson, L. (2001). PLS- Regression: A Basic Tool of Chemometrics. Chemometrics and Intelligent Laboratory Systems, 58, 109–130.

About the AuthorMarco S. Reis, PhD in Chemical Engineering, Professor in the Department of Chemical Engineering of the University of Coimbra, Portugal. He is currently President of the European Network for Business and Industrial Statistics (ENBIS), President of PRODEQ—Association for the Development of Chemical Engineering and responsible for the Process Systems Engineering (PSE) research group at the department of Chemical Engineering. He lectures courses on process systems engineering, quality technology and management, management and entrepreneurship, and process improvement. His research interests are centred on the field of process systems engineering (system identification, fault detection and diagnosis, control and optimization), statistical process control of complex large- scale processes, data- driven multiscale modelling, Chemometrics, and industrial statistics. Other areas of interest include multivariate image analysis, systems biology and process improvement through initiatives such as six- sigma and lean manufacturing. (http://orcid.org/0000-0002-4997-8865).

http://dx.doi.org/10.1016/j.talanta.2011.09.016

http://orcid.org/0000-0002-4997-8865


Ellis R. Ott Scholarship For Applied Statistics and Quality Management

The Statistics Division of the American Society for Quality is pleased to announce the availability of $7500 scholarships to support students who are enrolled in, or are accepted into enrollment in, a master’s degree or higher program with a concentration in applied statistics and/or quality management. This includes the theory and application of statistical inference, statistical decision-making, experimental design, analysis and interpretation of data, statistical process control, quality control, quality assurance, quality improvement, quality management and related fields. The emphasis is on applications as opposed to theory. Studies must take place at North American institutions.

Year2016–17 scholarship winners are:

Mr. Andrew Walter,UniversityofKansasintheM.S.category,andMr. Matthew Keefe, Virginia Tech., in the Ph.D. category.

During the last 19 years, scholarships totaling over $280,000 have been awarded to 52 deserving students.

Qualified applicants must have graduated in good academic standing in any field of undergraduate study. Scholarship awards are based on demonstrated ability, academic achievement, industrial and teaching experience, involvement in student or professional organizations, faculty recommendations, and career objectives.

Application instructions and forms should be downloaded from:

http://asq.org/statistics/about/awards-statistics.html

Forms for the 2017–18 academic year will be accepted only between January 1 and April 1, 2017.

For more information, contact:

Dr. Lynne B. Hare55 Buckskin PathPlymouth, MA 02360

Email: [email protected]

Governing Board: Lynne Hare, J. Stuart Hunter, Tom Murphy, Dean V. Neubauer, Robert Perry, Susan Schall, Ronald Snee, J. Richard Trout and Neil Ullman.

http://asq.org/statistics/about/awards-statistics.html



Upcoming Conference CalendarConference on Statistical Practice23–25 February 2017Jacksonville, FLhttp://ww2.amstat.org/meetings/csp/2017/conferenceinfo.cfm

The Conference on Statistical Practice aims to bring together hundreds of statistical practitioners—including data analysts, researchers, and scientists—who engage in the application of statistics to solve real-world problems on a daily basis. The goal of the conference is to provide participants with opportunities to learn new statistical methodologies and best practices in statistical analysis, design, consulting, and statistical programming. The conference also will provide opportunities for attendees to further their career development and strengthen relationships in the statistical community.

Lean and Six Sigma Conference27–28 February 2017Phoenix, AZhttp://asq.org/conferences/six-sigma/about.html

Do you have technical proficiencies and leadership responsibilities within your organization? Are you actively involved in process improvement, organizational change, and development dynamics related to a successful lean and Six Sigma culture? This conference is for you!

2017 World Conference on Quality and Improvement1–3 May 2017Charlotte, NChttp://asq.org/wcqi

ASQ’s World Conference on Quality and Improvement has a 70-year tradition of educating, engaging, connecting, and inspiring leading professionals from around the globe. Each year thousands gather to share best practices, expand their network and further develop their professional growth. The theme was chosen as a way of centering on current and future business leaders and the growth they seek to better influence the work they do, organizations they work for, and lives they lead. The body of tools, techniques, and methods that aid in this is ever growing. The conference sessions will feature thought leaders and knowledge that best demonstrate the successes, tested solutions, and proven results these disciplines can bring.

Joint Statistical Meeting29 July–3 August, 2017Baltimore, Minn.http://www.amstat.org/meetings/jsm.cfm

JSM (the Joint Statistical Meetings) is the largest gathering of statisticians held in North America. The JSM program consists not only of invited, topic-contributed, and contributed technical sessions, but also poster presentations, roundtable discussions, professional development courses and workshops, award ceremonies, and countless other meetings and activities.

Fall Technical Conference5–6 October, 2017Philadelphia, PAhttp://asq.org/conferences/fall-technical/

The 2017 Fall Technical Conference theme is Statistics: Powering a Revolution in Quality Improvement.

http://ww2.amstat.org/meetings/csp/2017/conferenceinfo.cfm

http://asq.org/conferences/six-sigma/about.html

http://asq.org/wcqi/?utm_source=partner%20website&utm_medium=link&utm_campaign=marketing_wcqi%202017_wcqi2017eventpagetilelink

http://www.amstat.org/meetings/jsm.cfm

http://asq.org/conferences/fall-technical/


Statistics Division Committee Roster 2017

CHAIR Herb [email protected]

CHAIR-ELECTSteve [email protected]

TREASURER Mindy [email protected]

SECRETARYJennifer [email protected]

PAST CHAIR Theresa [email protected]

Operations

OPERATIONS CHAIRJoel [email protected]

MEMBERSHIP CHAIRGary Gehring [email protected]

VOICE OF THE CUSTOMER CHAIRJoel [email protected]

CERTIFICATION CHAIRBrian [email protected]

STANDARDS CHAIRMark [email protected]

Member Development

MEMBER DEVELOPMENT CHAIRGary Gehring [email protected]

OUTREACH/SPEAKER LIST CHAIR Steve Schuelka [email protected]

EXAMINING CHAIRDaksha [email protected]

Content

CONTENT CHAIRAmy Ste. Croix [email protected]

NEWSLETTER EDITORMatthew [email protected]+49-152-05421794

WEBINAR COORDINATORAdam [email protected]

SOCIAL MEDIA MANAGERBrian Sersion & Joshua [email protected]

WEBSITE AND INTERNET LIAISONLandon [email protected]

STATISTICS BLOG EDITORGordon [email protected]

STATISTICS DIGEST REVIEWER AND MEMBERSHIP COMMUNICATIONS COORDINATORAlex [email protected]

Awards

AWARDS CHAIR Peter Parker [email protected]

OTT SCHOLARSHIP CHAIRLynne [email protected]

FTC STUDENT/EARLY CAREER GRANTSJennifer [email protected]

HUNTER AWARD CHAIROpen

NELSON AWARD CHAIROpen

BISGAARD AWARD CHAIROpen

YOUDEN AWARD CHAIRTheresa [email protected]

Conferences

WCQIGary Gehring [email protected]

FTC STEERING COMMITTEEBill [email protected]

FTC PROGRAM REPRESENTATIVEMindy Hotchkiss [email protected]

FTC SHORT COURSE CHAIR Yongtao [email protected]

Auditing

AUDIT CHAIR Steve [email protected]

By-Laws

BY-LAWS CHAIRAdam [email protected]

Nominating

NOMINATING CHAIR Adam [email protected]

Planning

PLANNING CHAIR Theresa [email protected]

APPOINTED

OFFICERS





mailtp:[email protected]


























The ASQ Statistics Division Newsletter is published three times a year by the Statistics Division of the AmericanSociety for Quality.

All communications regarding thispublication, EXCLUDING CHANGE OF ADDRESS, should be addressed to:

Matthew Barsalou Editoremail: [email protected]

Other communications relating to the ASQ Statistics Division should be addressed to:

Richard Herb McGrath Division Chairemail: [email protected]

Communications regarding change of address should be sent to ASQ at:

ASQP.O. Box 3005Milwaukee, WI 53201-3005

This will change the address for all publications you receive from ASQ. Youcanalsohandlethisbyphone (414) 272-8575 or (800) 248-1946.

Upcoming NewsletterDeadlines for Submissions

Issue Vol. No. Due DateJune 36 2 April 16

ASQ Statistics Division

VISIT THE STATISTICS DIVISION WEBSITE:www.asq.org/statistics

ASQ Periodicals with Applied Statistics content

Journal of Quality Technologyhttp://www.asq.org/pub/jqt/

Quality Engineeringhttp://www.asq.org/pub/qe/

Six Sigma Forumhttp://www.asq.org/pub/sixsigma/

STATISTICS DIVISION RESOURCES:

LinkedIn Statistics Division Grouphttps://www.linkedin.com/groups/ASQ-Statistics-Division-2115190

Scan this to visit our LinkedIn group!

Connect now by scanning this QR code with a smartphone (requires free QR app)

Check out our YouTube channel at youtube.com/

asqstatsdivision

One accurate measurement is worth a thousand expert opinions.

Rear Admiral Grace H. Hopper


Documents

STATISTICS DIGEST - ASQasq.org/statistics/2016/10/statistics/statistics-digest-february-2017.pdf• Social Media (LinkedIn) • Conference presentations (Fall Technical Conference,