Design expert: An expert system application to clinical investigations

Expert Systems With Applications, Vol. 2, pp. 361-371, 1991 0957-4174/91 $3.00 + .00 Printed in the USA. © 1991 Pergamon Press plc

Design Expert: An Expert System Application to Clinical Investigations

CALVIN L. WILLIAMS

Clemson University, Clemson, SC USA

Abstract--In recent years several expert systems have been developed for practical applications in applied statistical methodologies. Existing expert systems in statistics have explored several areas, e.g., the determination of appropriate statistical tests, regression analysis, and determination of the "best" experimental design for industrial screening experiments. The DESIGN EXPER T, a prototype expert system for the design of complex statistical experiments is presented here. It is intended for scientific investigators and statisticians who must design and analyze complex experiments, e.g., multi-level medical experiments with nested factors, repeated measures, and both fixed and random effects. This system is "expert" in the sense that it is able to (i) recognize specific types of complex experimental designs, based on the application of inference rules to nontechnical information supplied by the user," (ii) encode the obtained and inferred information in a flexible general-purpose internal representation, for use by other program modules; (iii) generate analysis of variance tables for the recognized design and an appropriate Biomedical Computer Programs runfile for data analysis, using the encoded information. DESIGN EXPERT can recognize randomized block designs, including lattice designs within embedded Latin Squares, crossover designs, split plots, nesting, repeated measures, and covariates. It is written in an experimental programming language developed specifically for research in Artificial Intelligence.

1. INTRODUCTION

EXPERIMENTS WITH one or more experimental factors as well as several different types or sizes of experimental units are known as complex experiments. The most commonly used statistical technique to analyze complex experiments is analysis of variance (AOV). Anal- ysis of complex experiments, hence, forces a researcher to become familiar with terms such as between-subjects or case, within-subjects or case, repeated measure, split- plot, and, quite possibly, crossover designs. This is par- ticularly the case in most clinical studies. Clinical studies include not only the less complex between-subjects design, but also time-response studies, dose-response studies (both appropriately analyzed with a within- subjects analysis or repeated measures), or multicenter clinical trials. These types of analyses of designed experiments present a problem for many researchers with little knowledge about designing or analyzing complex experiments. The typical statistical consultation in a

This research was made possible through a fellowship from the Upjohn Company, Kalamazoo, MI.

Requests for reprints should be sent to Calvin L. Williams, De- partment of Mathematical Sciences, Clemson University, 0-323 Martin Hall, Clemson, SC 29634-1907.

medical environment may begin as follows: A young man or woman wearing a white lab coat and a desperate expression appears at the office door, clutching a sheaf of mixed data, notes, graphs, and grocery lists. Our client is the technician in a laboratory or, perhaps a clinician, and the latest experiment is producing results that are ambiguous at best. It is time to consult a statistician.

The ensuing interview is painful for all parties. It sometimes takes more than an hour just to discover what variables or factors were measured and when they were measured; the usual problems are a lack of common terminology and the peculiar experimental idio- syncrasies that each subdiscipline of science has evolved. Having solved the riddle of this particular experiment, we then scribble down the AOV tables which summarize the experimental design. To the statistically unsophisticated, these tables seem to be incantations of statistical wizardry from which flow spells of signif- icance and power. And it is true that they do indeed hold the keys to data analysis, interpretation, and eventual criticism. With the design issue clarified, the consultation enters a happier phase, with the technician and ourselves cooperating either to reanalyze the data or to redesign the experimental protocol.

First, a clinician could have designed a hypothetical experiment, one in which he/she plans to fully imple-

361

362 C. L. Williams

ment once he/she has (i) an understanding of the ap- propriateness of the experiment and (ii) the appropriate AOV for the hypothetical experiment as well as the terminology surrounding the characteristics of the experiment. Second, if the clinician has already conducted the experiment, similar considerations can be made. Ideally, the clinician could use such a system prior to conducting the experiment, thereby reducing the possibility of conducting an experiment that is in- adequate for answering his/her questions, or inappro- priate.

DESIGN EXPERT is designed to assist in the consultation between the statistician and the client. It does so by conceptualizing design attributes of the particular experiment. In this effort, DESIGN EXPERT acquires information about experimental procedure in a sys- tematic manner and encodes this information in frame data structures for easy retrieval and manipulation. This information is then refined through the application of analysis rules that contain recognizable design properties. These properties are properties of complex designs to which DESIGN EXPERT can be applied.

This research is not the first attempt at making experimental designs easier to recognize and apply. From the structural point of view, Taylor and Hilton (1981) use a structural diagram called a Hasse diagram to symbolize AOV. This symbolization relies on the work of Lee (1966, 1975) and presents a very viable way of representing experimental designs, as the research here attempts to do. Recent attempts have been made to enhance the capabilities of DESIGN EXPERT by developing a graphics interface that will draw a field lay- out or Hasse diagram from the information stored in the frames of the knowledge base. The C language was used in this implementation with hopes of translating the entire system over to this language. When complete, in this way, DESIGN EXPERT will provide the frames that describe the client's design and a structural picture for better understanding of the design.

Haaland, Liddle, and Yen (1985, 1986) have developed a system for determining the "best" experimental design for industrial screening experiments. DEXTER relies on a static database, whereas DESIGN EXPERT allows the experimentor to "experiment" with different designs. Hence the need for a statistician on hand is readily warranted.

1.1. Strategy Development for Effective Design of Clinical Experiments

The primary concern of statistical expert systems is developing the strategy that will be pursued to analyze the problem at hand. The analytical strategy must be expressed as criteria associated with specific statistical methods. Various statistical strategies have been discussed in a nonanalytical context by Cox and Snell (1981), Koch, Amara, Stokes, and Gillings (1980),

Koch, Amara, Carr, H irsch, and Marques (1985), and by Hopkins (1983). Artificial Intelligence aspects of statistical strategy have been described by Gale and Pregibon (1984a, 1984b, 1984c), Gale (1986), and by Oldford and Peters (1984). These discussions of statistical strategies emphasize the data analysis phase of experimentation. But, what about the strategy that goes into the initial design of the experiments yielding these data?

Experimental design is the practice of planning experiments with the desire to maximize the amount of information obtained while minimizing the use of the available resources. In most cases, the experimenter controls the factors or types of treatments that are to be studied. Before carrying out an experiment, a strategy to follow while executing the experiment must be developed. The strategy identifies (i) which factors (types of treatments) and levels of treatment are to be studies; (ii) how many times these factors must be ob- served; (iii) what the inference population for the sample is and, subsequently, what the experimental units are; (iv) how the factors to these experimental units should be assigned; and finally (v) an outline of the proposed analysis before the data are collected. This ensures the resulting data can be analyzed and that the desired comparisons can be made.

Jones (1980) described a strategy for generating the design of a simple comparative experiment. He main- tains that the limitations of the computer in the design of such a strategy is readily exhibited by the fact that the computer will be able to give little or no assistance in making important decisions regarding the following: the choice of appropriate treatments, choice of the proper experimental units, choice of what response to measure and where and when the experiment should take place (items i-iv above). In spite of this pessimistic report, this research demonstrates that a computer can be programmed to conceptually outline the analysis. That is, a system that can symbolically identify and represent the AOV table, key concepts, and interpret the design characteristics of the experiment provided that items i-iv have been appropriately determined and performed.

1.2. Ground Domain Knowledge

Knowledge domains in this presentation may be one of two types. The first type is ground domain knowledge, which can be classified as the domain or area from which the knowledge will be applied. The ground domain of an application can be any area in which the application can be applied successively. Although it is certainly possible to support ground domain knowledge from any area for which development of design is used, for this presentation we consider clinical knowledge. Hence, it should be noted that DESIGN EXPERT does not depend on a static database of specific designs. On

Design Expert 363

the contrary, it depends on a set of stored rules used to validate important attributes about a hypothesized design described by the user. It is for this reason that DESIGN EXPERT is as well suited for analyzing designs of medical and agricultural experiments as for multistaged sampling experiments.

1.3. Statistical Domain Knowledge

The statistical domain of DESIGN EXPERT consists of a set of stored rules that characterize attributes specific to certain experimental designs. Two items, concept key and type, are used to make distinctions among the classes of rules. There are five basic classes of rules: instantiation, appending, asserting, counting, and simple pattern matching. In other words, these rules are used to create statistical attributes of concepts and to append and assert these newly formed concepts. These rules define the key concepts that are pertinent to specific experimental designs.

1.4. Knowledge Representation

One of the major concerns in artificial intelligence is how to represent hu.man knowledge internally in a computer so as to allow to its efficient use. The representation of this human expertise in the knowledge base not only affects the search methods utilized by the system, but, in turn, is affected by these strategies. It is reasonable to expect that these strategies also affect the explanation facilities incorporated in the system.

Statistical as well as clinical concepts can be represented in frames. For this application there are four structural types of frames. These are Knowledge-Infor- mation (KI) frames which hold information and rules concerning a class of frames and also give the method by which the following frames are used; Typical frames, which are generic templates for the construction of specific instances of a class of frames; Specific-Instance frames, which are members of a class; and Rule frames which describe rules. Figure 1.1 shows the represen- tational form of these frames.

The choice in this research was to represent knowledge in terms of schemas, which are simply stereotyp- ical generic template frames. The schema, as described by Fox, Wright, and Adam (1986), contains slots and values. The slots represent the attributive, structural, and relational information about a concept. The Typ- ical frame is used to represent this type of construction. Typical frames organize the knowledge base as a col- lection of objects or concepts that share common properties through inheritance. For example, consider the frame in Figure 2 which represents a specific instance of a study. The study's name is Medical Center Trials; the session in which this study is considered is session& 1 (this is a unique symbol generated by the system); the objective of Medical Center Trials study

TypicalConcept attribute1 attribute2 attribute3

typical type

l [ Specific Instance attribute1

value1 ... specific instance attribute2

value2...

rds specific instance

Know/edge-Information instance typical

Rule frames class type pattern variables IF THEN

/

Used only du~ng inferencing

FIGURE 1. Types of frame structures.

is the determination of drug efficacy on hypoglycemia; the type of study is experimental; the sampling units of the study are medical centers, doctors, and patients; the dependent variable is blood sugar level," the response structure is univariate; and the design is multistaged. This frame was instantiated from the Typical frame of Figure 3, that is, it is a Specific-Instance of the class of frames Typical Study. When a specific object is instantiated from a generic template (a Typical frame) all of the slots are copied; then during subsequent execution these slots are filled. The derivation of these Typical frames are discussed later in this presentation.

The special Knowledge-Information frames play a significant role in the representation of domain knowledge. These frames contain description of statistical entities, dialog modules for controlled conversation with the user, help messages and necessary explanations about system execution. These frames also house the knowledge used by the specific instances once they have been instantiated from their Typical frames. Note that a record of each study is kept in the inst slot of the frame. An example of these types of frames is given in Figure 4. These frames give the method by which all concepts are created through the Typical frames. The key slot in this instance is the typical slot. It determines which Typical frame is referenced. Here, it is the Typ- ical Study frame. The query slot holds the query used in prompting the user for a study name. At execution time, the session number is inserted to create the prompts needed.

1.5. Representing Statistical Reasoning

Inference rules in DESIGN EXPERT are also encoded as frames. Frame patterns are found under the if and

364

Medical Cenler Trials class

study session

session& 1 objective

"The determination of drug efficacy on hypoglycemia" type

experimental sampling units

medical centers doctors patients

dependent variable blood sugar level

response structure univariate

design multistaged

FIGURE 2, A specific-instance of a study.

then slots. When the rule applier tries to apply a rule in the forward direction, the applier then looks for an assertion of a frame that matches the "if" pattern. If one is found, it uses the then pattern to modify the given frame (the goal). Backward chaining, where the goal is assumed true and an attempt is made to sub- stantiate the goal by finding assertions that match, is also possible, but it was not necessary in this research. Figure 5 shows a low-level rule used by DESIGN EX- PERT. This rule is statistically unobjectionable and noncontroversial. The benefit of rule-based programming, however, is that the rules can be altered without changing the program itself.

2. METHODOLOGY

The implementation of this consultatory system required a complete review of the methodologies and strategies used to create such a system. The aspects that warrant a careful review are (i) the knowledge rep- resentational structure used to represent the knowledge, (ii) the GIMLI (Cobb, 1986) language in which the system is written, (iii) the development of the knowledge frames that are used to house the special entities of the system, (iv) the implementation of the user interface for the acquisition of knowledge from the user, and (v) the refinement rules that make inferences from the information solicited from the user.

2.1. Frame Representation in GIMLI

As previously stated, the frame is the principal structure used in this consultatory system. This section describes

TypicalStudy class session objective type sampling units dependent variable response structure design

FIGURE 3. Typical Study frame for construction of study instances.

C. L. Williams

study inst

Medical Center Trials typical

TypicalS tudy query

single "What is the name of the study for"/*session number is inserted*/ "?(I0 characters or less, please)"

FIGURE 4, An abbreviated knowledge-information frame for the study concept.

its representation in the GIMLI language• This is given separate consideration from the other attributes of the language because of its complexity and because it is the fundamental data structure in GIMLI. The frame structure is prefixed in GIMLI with the key word frame, but is deleted from all frame structures shown here for readability• The frame is generally considered a tree with named nodes, the root node being the frame name. Figure 6 shows a frame as represented in the GIMLI language• Note that the braces "{ }", brackets"[ ]" and quotes are used by GIMLI to separate the syn- tactical entities of the frame structure, thus allowing the frame to be represented in a "free" format. The root node of this frame has the name root-name. The nodes of this frame, at the second level are slot 1, slot2, . . . . slotk. These nodes have sub-nodes A, B, C, D, • . . and so forth. Figure 7 shows how this frame is represented in a tree diagram.

The lines connecting the nodes are considered branches. The order in which these branches appear determines the order in which the pattern matching- facility performs its search. The type of search performed by the searching facility is called depth-first searching. Reconsider Figure 7. Depth first searches are performed by initiating the searching process at the root node, root-name, and proceeds by searching down

Rule2 class

Rule type

Append pattern variables

X Y Z

IF Y

class

THEN Z

unit experimental factor

X subunit

Z

class subunit

experimental factor X

Interpretation:

IF X is an experimental factor of the unit Y and Y has subunit Z,

THEN X is an experimental factor of Z.

FIGURE 5. A low-level inference rule.

Design Expert 365

root-name { slotl [ A B] slot2 [C D]

slotk [E{GH}F] }

FIGURE 6. A frame structure.

the node slotl, A and then B. If the searching on slotl is unsuccessful, slot2 and subsequently slotk are searched until a successful match is encountered. In the tree structure, generally all nodes must have a name. This requirement is slightly eased in frame representation in GIMLI since, a frame may use an anonymous name " _ " which matches any name in the frame.

2.2. Attributes of the GIMLI Language

The GIMLI language supports three basic variable types. These are strings, numeric values, and frame pointers. Strings, prefixed in GIMLI with a dollar sign ($), can be assigned any string values. Numeric value variable names are prefixed by a pound sign (#). These values can be used for algebraic manipulations. The frame pointers, prefixed by an asterisk (*), are used as pointers to frame structures. A variable is bound to a value in GIMLI, just as in LISP and PROLOG. Short of assigning a value to a variable using the GIMLI predicate assign, bindings are usually made during pattern matching.

The function routines of DESIGN EXPERT, implemented in GIMLI, are actually user-defined predicates that return a Boolean result, which implies the predicate either fails or succeeds. These routines are initiated with the key word function. The body of a function can be a logical conjunction of the form "predicate/and predicate2 a n d . . , predicatek", or a logical d is junct ion represented by " e i t h e r " conjunctionl or conjunction2 end., or a logical negation described by not (conjunction). These functions play a crucial role in the execution of the system. As previously stated, functions are predicates that either fail or succeed. When a function or predicate fails, as in Prolog, backtracking, the process of reversing program execution to perform the act of "exhausting all possible occurrences" of a specific outcome, takes place. During backtracking GIMLI retraces its steps, reexamining each predicate to determine if all possible occurrences have been encountered. Backtracking continues until a new occurrence is found, at which point forward execution is resumed. Backtracking can be intentionally forced with the GIMLI predicate fail. During the execution of a function one can force the process of backtracking to exhaust the set of all possibilities or out- comes.

Similar to forcing backtracking with the predicate fail, the predicate cut can be used to inhibit the back-

tracking process. Backtracking is a very important ac- tion that takes place during program execution, as will be shown. It also has some effects that may be disliked, if not properly understood. During backtracking all previous binding or assignment of function variables or frame manipulations are deleted. Ironically, this ac- tion is what provides the ability to detect all possible occurrences of a particular outcome. This is where the cut predicate plays a crucial role. If during backtracking a cut is encountered, all bindings or assignments performed before the cut in the forward execution of the function are preserved. That is, the cut causes the backtracking process to cease at the point at which the cut is encountered.

Another significant feature of the GIMLI language is its pattern-matching facility. This facility play a crucial role, not only in the execution of the knowledge base, but also in the program execution itself. Pattern matching is the act of comparing a frame structure, usually referred to as a pattern to a set or sequence of trial frames until a frame is found that matches the pattern. A pattern is generally a frame in its normal context with unbound string or numeric variables, initiated in GIMLI by the dollar sign ($) and the pound sign (#), respectively, used when node names are needed. The matching process exhibits the binding of the variables to the corresponding node names of the trial frame. A successful match is encountered when every feature or node is reflected in the trial frame. The trial frame may contain extraneous information, but this does not affect the successful completion of the match.

3. K N O W LED G E- IN F O RMA TIO N FRAMES FOR DESIGN ATTRIBUTES

There are two fundamental structural components of an experimental design, design structure and treatment structure. To determine the experimental design, these components must be determined for all types of experimental units in the study (Milliken & Johnson, 1984). For example, split-plot designs have two types of experimental units, whole-plot experimental units and sub-plot experimental units. The design structure is determined by the type of blocking or grouping im-

root -name

slot k

FIGURE 7. A tree diagram.

366 C. L. Williams

posed on the experimental units to form homogeneous groups. Ideally, the design structure of an experimental design involves the grouping of the experimental units so that the conditions under which the treatment combinations are to be applied are as uniform as nature permits. The treatment structure is determined by the various treatment types or treatment combinations that are to be studied, which are subsequently under the control of the experimenter.

Koch et al. (1980, 1985) describe strategies for the analysis of repeated measures and split-plot designs• These designs are considered quite complex in the domain of experimental designs. Koch's strategies are contingent upon the identification of the observational units of an experimental study• In other words, the identification of the observational units of an experimental study becomes a goal of the experimenter• Sim- ilarly, determining what are observational units is a major concern in the design of experiments. In determining the design structure, the first objective is to identify the primary sampling units, subsequent sam- piing units (i.e., possible multistaged sampling experiments), and the observational units for the experimental study•

To proceed with this objective, we first give some definitions. Sampling units are units sampled from the study population. Ideally these units are sampled from the target population• They are considered experimental units if some experimental treatment or treatment combination effects are to be applied to these units• Experimental units are those units of experimental material to which a treatment is applied in a single trial of the experiment (Cochran & Cox, 1957). Cox (1966) gives a more formal definition: Experimental units are the smallest division of experimental material such that any two of these units may receive different treatments in the actual experiment. Finally, the observational units are the units of observation. Note that observational units may be experimental units or some subsampling units thereof• These units are considered to be observational units if the response measurements are drawn from the units as a whole• Hence, the strategy for reaching the "identify the observational units" goal is determined.

DESIGN EXPERT's strategy is to determine the observational units. This is first achieved by a controlled sequence of queries presented to the user. The unit knowledge-information (KI) frame is used to solicit user information from the that will define the observational units. Subsequent inquiries are continued by references to the subunit KI frame, which is further used to determine subsampling and possible split-plot- ting structures. It should be noted that all queries stored in the KI frames are a part of the knowledge base. Hence, the queries can be altered to the user's vernac- ular without altering the consultation program.

The next major issue to be considered is that of

variable or factor classification. The user is queried for all experimental factors that are applied to each sampling unit (in this case experimental units) at each level of sampling. This is how the treatment structure is determined.

A major concern of all good statistical expert systems is the user's comprehension of statistical terms. The system attempts to ameliorate this problem by the use of a glossary of statistical terms to which the user may refer• During the execution of the application the user may at any prompt interrupt the design process by is- suing a term preceded with a question mark (?), (e.g., ?randomization). The system will refer to the glossary, which is also represented by a frame structure, and attempts to locate the term with which the user is having difficulty. If the term is located in the glossary, the definition appears on the terminal screen• If it is not located, the user is referred to a statistical text or a statistician to locate the definition of the term and is given the opportunity to update the glossary. The implementation of these type of queries in frames demonstrates that obtaining information from the user does not have to be "hard-wired" in the functions of the system• More importantly, since they are represented in this manner, changes can be made in the wording of each frame without modification of the system•

3.1. Analysis of Typical Concepts

The hierarchical structure used in DESIGN EXPERT comprises of seven concepts: a session; the study constructed during the session; the primary sampling units and subsampling units, both of which are referenced by the TypicalUnit frame; independent variables (e.g., experimental factors, blocking factors, etc.) referenced by the TypicalIndependent frame; dependent variables, referenced by the TypicalDependent frame; the analysis in which the resulting allowable effects are stored; and finally the effects generated by the TypicalEffect frame.

These concepts are represented in GIMLI in the following form:

Typical Concept subconcept 1 subconcept2

subconceptk,

where TypicalConcept is the name of the frame and the subconcepti 's are concept attributes. These attributes can assume the role of trivial attributes in the sense that they are arbitrary slot names, or, they can be nontrivial. In the latter instance, they are actually representatives of other typical concept frames in the knowledge base.

Each of these typical concept frames has a corresponding knowledge frame in the knowledge base.

Design Expert 36 7

3.2. Analysis of Inference Rule Types

The representation of design concepts by rules is a for- midable task. Fortunately, most designs have one or more properties that distinguish them from others. It is from these properties that the rules are produced. Unfortunately, some designs have one or more properties that are similar, if not identical, to others. The design structures that are considered in DESIGN EX- PERT are completely randomized, randomized complete and incomplete block, and Latin square designs. Design rules are embedded naturally in the knowledge base. They are represented in the following form:

sources of variation for that specific analysis are constructed by the use of the source demon located in the reactive processing module. The source demon con- structs all possible permutations of existing effects. The effects are the used to determine the degrees of freedom. Hilton and Taylor and Hilton (1981) give rules for generating the allowable effects of AOV and the degrees of freedom for each effect. These rules determine which effects are valid sources of variation and their degrees of freedom.

4. I M P L E M E N T A T I O N The system consists of the consultation program DE- SIGN EXPERT; the knowledge base which consists of

Rule{ number [000] type [rule-type] key [concept-class] varlist [varl, var2 . . . . . . . vark] IF [_{ situation-attribute 1 [value 1 ] situation-attribute2[value2] } ] THEN [_{ consequence-attribute 1 [value I ]consequence-attribute2[value2] } ],

}

where: • number is a slot whose value is the rule number, • type contains the rule type which is either instantia-

tion, appendage, assertion, count, or simple matching,

• key contains the concept class which the rule applier must match,

• varlist contains the pattern variables, if necessary, used in the unification of variables,

• IF holds the assertions pattern which is subsequently used to facilitate pattern matching,

and • T H E N holds the consequence pattern which is used

to alter the current state of the knowledge base if a match occurs. There are five basic types of rules: (l) instantiation

rules, in which objects are created; (2) appendage rules, in which values are appended to specific slots; (3) assertion rules, in which values are asserted (i.e. insertion by replacement); (4) simple matching rules, in which a simple pattern match occurs; (5) and count rules, in which the number of values in a slot are counted. Each rule type has it's own rule applier. Instantiation rules are applied with the procedure DoInstantiation. Ap- pendage rules are applied with the procedure Do- Appendage. Assertion rules are applied with the procedure DoAssertion. Simple pattern matching rules are applied with the procedure DoMatch, and Count rules are applied with the procedure DoCount.

3.3. Generation of Analysis of Variance The generation of effects for the analysis of variance table is rule determined. Upon the application of the rule governing the determination of an analysis, the

design production rules named ANOVARULES; a glossary of technical terms named GLOSSARY; typical concepts representative of the schematic frames used in the creation of design concepts referenced as TYP- ICALCONCEPTS; and a set of demon procedures (written in GIMLI code, to promote reactive processing during program execution) named DEMONS. The functions of DESIGN EXPERT are actually user-defined predicates that either succeed of fail, as with PROLOG predicates.

DESIGN EXPERT utilizes two inferencing mechanisms. First, it uses the predicate logic scheme as previously described. The second inferencing mechanism is a specialized rule matching as mentioned in Section 3.2. The ANOVARULES describe different attributes of experimental designs. In this regard it should be readily noted that DESIGN EXPERT uses a specialized scheme that relies on both rules (rule-base) and frames (knowledge-base). The knowledge used in developing these rules came from basically two sources: (i) "ex- perts" in the field of experimental design, and (ii) tex- tual material (Cochran & Cox, 1957; Cox, 1966; Dia- mond 1981; Milliken & Johnson, 1984). The schematic frames used in T Y P I C A L C O N C E P T S determine the objects or type of attributes used in ANOVARULES. There are "special" demons in the system. The demons, as previously mentioned, are used to facilitate reactive processing during the execution of the system.

4.1. Examples of Sample Sessions

Example sessions on DESIGN EXPERT, executed on an Apple Macintosh with l megabyte of memory, are presented below. The presentation of output to the user appears within windows. Two windows are used to

368 C. L. Williams

handle output. The transaction window displays all in- teractions between the user and DESIGN EXPERT. The output window is used to display only system output. These two types of output are presented as follows: all transactions are presented in italics, system queries italics and user responses bold italics; and all system output is presented as bold text. The first example shows the basic query process of the system. The second shows only the output from a session Carriage returns are denoted by the symbol (cr) . These have been altered slightly for readability. 4.1.1. Session 1. The following example is conducted from the following information. An experiment is conducted to determine the efficacy of three analgesics on pain. Twenty-five patients are grouped by age and are randomly administered one of the three pain formulas. The consultation begins:

Welcome to DESIGN EXPERT. DESIGN EXPERT is an expert system designed to assist researchers in the design and analysis o f experiments. You will be asked a series of questions concerning the experiment that you will be designing or one that you have already designed. Do you wish to get instructions on the operation of this system?

yes. (cr) There are two basic types of queries used in DESIGN EX-

PERT. The first are all queries ending in a QUESTION mark (?). All queries of this nature expect the user response to be in the form of a single term (i.e. all blanks are ignored). The second type of query ends in a COLON (.'). The response expected here is in the form of a list (i.e. all blanks are considered as separators. I f a query is not clear, there are (key words) embedded within each query that you may reference to clarify the query. To reference these key words enter the key word in the following form ?key word, the definition will then appear on the screen. I f there is still some confusion you should see a statistician. Beginning session&l on DESIGN EXPERT.

Do you wish to confirm your responses made throughout this session? no. (cr) What is the name of the study for session&l? Drug Efficacy. (er) What is the (objective) of this study? To evaluate the efficacy o f analgesics on pain relief. (cr) What are the (primary sampling units) of interest in Drug Efficacy? (subjects, patients, e t c . . . ) ?primary sampling units (cr) Primary sampling units are those units from which all sub. sequent sampling units are selected, (cr) What are the (primary sampling units) of interest in Drug Efficacy? (subjects, patients, e t c . . . ) subjects <cr) [low many subjects are there in this study? 18 (or) Are there (blocking factors) used to create more (homogeneous) groups o f subjects? yes (cr)

Please list the (blocking factors) by which subjects are grouped. age (cr) Is the factor age qualitative or quantitative? quan *At most queries with limited range responses short distinct entries can be made. Internal inferences suggest that in future sessions this variable shouM be considered a covariate since it is quantitative. Please list the (levels,) o f age: 10-20 30-40 50-60 (cr) Assumed 6 per block. (er) Is age a f ixed or a random factor? ran (cr) Are there (experimental factors) or treatment types being applied directly to subjects? yes (cr) Please list the experimental treatment types being studied on subjects: Analgesics (cr) Is the factor Analgesics qualitative or quantitative? quai (cr) Please list the (levels) o f Analgesics: aspirin tylenol ibruprofen (cr) Assumed 6 per treatment. (er) Is Analgesics a f ixed or a random factor? fiX (cr) Are the response measurements being made directly from subjects? yes (cr) Please list the (responses) or (dependent variables) being measured in the study Drug Efficacy." Paln-threshold (cr) Is the response Pain-threshold categorical or continuous? con (cr) What is the expected minimum observable value of Pain- threshold? 0 (cr) What is the expected minimum observable value of Pain- threshold? loo (cr) Is Pain-threshold measured more than once on each observational unit? no (cr) Current State constructed Applying Analysis Rules'. The system now attempts to apply rules to the current state.

APPLYING RULES TO CURRENT STATE

Rulel Rule3 appl i e s . . . ! Rule4 appl i e s . . . ! Rule5 Rule6 appl i e s . . . ! Rule9 appl i e s . . . ! Ralel0 appl i e s . . . ! Rulel3 Ralel6 appl i e s . . . ! Rulel9 Rule20

Design Expert 369

Is Analgesics (crossed) with (nested) by or (nested) within age? crossed (cr) Do you wish to see the current state frames? yes (cr>

CURRENT STATE FRAMES OF SESSION Session&l

Session&l class: session study: Drug Efficacy

Drug Efficacy class: study objective: To evaluate the efficacy of analgesics on pain relief. type: experimental sampling units: subjects dependent variable(s): Pain-threshold response structure: univariate method: parametric design: completely randomized suggested procedure: analysis of variance analysis of co- variance*

*Suggested due to quantitative blocking factor age.

Pain-threshold class: dependent variable(s) type: continuous min: 0 max: 100

Do you wish to get a summary o f the current state? yes (cr)

SUMMARY OF CURRENT STATE FRAMES OF SES- SION Session&l

This is a summary of the designed experiment developed during Session&l. The following describes the study Drug Efficacy.

The objective of the study Drug Efficacy is to evaluate drug efficacy on pain relief. The sampling units under consideration are subjects of which there are 25. The dependent variable(s) or response measurements made is Pain-threshold. The response structure of this study is univarlate. The method of analysis should be parametric. The suggested procedure is analysis of variance or analysis of covarlance.

Do you wish to see an ANO VA table? yes (cr) ANALYSIS OF VARIANCE

EFFECT DF

Analgesics 2 age 2 Analgesic×age 4 subjects 9

Are you through for this session? yes (cr)

End of Execution.

4.1.2. Session 2. Consider an experiment that is perhaps a little more complex. A clinical trial is conducted to determine the effectiveness ofhypoglycemics on the blood sugar levels in men and women across the United States. Three centers were chosen to conduct the experiment. Within each center, two doctors were randomly selected and from each of the doctors six patients suffering from hypoglycemia were treated with three different interventions. These 36 patients (l 8 male and 18 female) were randomly treated with one of the interventions. The system is then instructed of these characteristics and presents the following frames to the user:

Session&2 class: session study: Medical Center Trials

Medical Center Trials class: study session: session&l objective: "The determination of drug efficacy on hypoglycemia" type: experimental sampling units: centers doctors patients dependent variable(s): blood sugar level response structure: univarlate design: muitistaged method: parametric suggested procedure: analysis of variance

centers class: sampling units number: 3 analysis: centers-analysis

subjects class: sampling units number: 18 blocking factor: age experimental factor: Analgesics blocking restriction: randomized complete block treatment structure: one-way classification analysis: subjects-analysis

age

class: blocking factor type: quantitative levels: 10-20 30-40 50-60 universe: random units blocking on: subjects crossed with: Analgesics*

*Since each level of analgesic is present in each block.

Analgesics class: experimental factor type: qualitative levels: aspirin tylenol ibuprofen universe: fixed units studied on: subjects crossed with: age*

*Since each level of analgesic is present in each block.

370 C. L. Williams

subjects-analysis class: analysis source: Analgesics subjects age Analgesics×age analysis of: subjects

doctors class: sampling units number: in each

centers: 2 blocking factor: centers analysis: doctors-analysis

patients class: sampling units number: in each

centers: 12 doctors: 6

blocking factor: sex experimental factor: hypoglycemics blocking restriction: randomized complete block treatment structure: one-way classification analysis: subjects-analysis

s e x

class: blocking factor type: qualitative levels: male female universe: random units blocking on: patients crossed with: hypoglycemics*

*Since each level of hypoglycemic is present in each block. hypoglycemies

class: experimental factor type: qualitative levels: Hypl Hyp2 Hyp3 universe: fixed units studied on: patients crossed with: sex*

*Since each level of hypoglycemic is present in each block. centers-analysis

class: analysis analysis of: centers

doctors-analysis class: analysis source: centers analysis of: doctors

subjects-analysis class: analysis source: centers doctors sex centers×sex doctors×sex patients analysis of: subjects

TABLE 1 Analysis of Variance

Effect Degrees of Freedom

centers 2 doctors 3 sex 1 centers×sex 2 doctors×sex 3 patients 24

Table 1 shows the effects and their degrees of freedom.

blood sugar level class: dependent variable(s) type: continuous min: 100 max: 300

These frames show how the representation of the characteristics make the actual complex design easier to visualize. Table 1 shows AOV of the effects and their degrees of freedom.

5. CONCLUSIONS

One of the initial goals of expert systems developers was to develop systems that could simulate the cog- nitive activity of a human being. It has been shown that it is possible to create such systems. This is par- ticularly true in the area of decision-making consul- tative systems.

The limited number of expert statisticians suggests that the development of statistical expert systems to assist in the experimental design phase is a promising area of research. These systems are needed to work in conjunction with, not to replace, expert statisticians. The present expert systems in statistics have been developed mostly to assist in data analysis. With DESIGN EXPERT, expert systems move into the realm of designing experiments as well as proposing analytical models for the design before the data are collected.

The goal of this research was to create an efficient working example of an expert system that can be applied to a specific branch of statistics, experimental design and its attendant procedures. This required the identification of rules and key concepts and the development of their representations. Achievement of this goal required the following: 1. determination of appropriate representation(s); 2. choice of a computer language suitable for artificial

intelligence programming; 3. identification and comprehension of basic concepts

in designing experiments; 4. definition of structures, rules, and functions for de-

scribing design concepts; 5. compilation of these concepts for use in a knowledge

base for experimental designs; 6. development of strategies for appropriate interaction

mechanisms and data acquisitions from the user; 7. construction of a scheme for using rules and frames

for decision making; and 8. testing and validation of the system by use of ex-

amples. DESIGN EXPERT is expert in the sense that in-

formation about design concepts, and the rules that refine this information are stored as data structures, namely frames. The concepts themselves are structured hierarchically in schemas as previously described. DE- SIGN EXPERT relies on two types of inferencing. In-

Design Expert 3 71

ferencing at the language level (GIMLI) follows simple predicate logic, i.e., the predicate either fails or succeeds. The other inferencing mechanism is used in the application o f rules. Previous expert systems have shown that product ion rules are an efficient way of representing statistical knowledge (Aikins, 1984). In D E S I G N E X P E R T the application o f rules required the use o f a data structure that would enhance appli- cat ion o f the system's rules. The frame data structure accomplishes this task. Thus, the f rame-product rule paradigm is used. The frames lend themselves as a structural " language" for describing the concepts in the knowledge base o f D E S I G N E X P E R T and as a structure by which the rules can be represented in a uni form manner .

D E S I G N E X P E R T is written in a language developed specifically for use in artificial intelligence. The G I M L I language, a logic p rogramming language, provides the necessary manipula t ion o f tree structured components . These capabilities are shared with the traditional artificial intelligence p rogramming languages LISP and P R O L O G . GIMLI , however, is quite easy to use as an AI development language because o f its data structure, the f rame, and the built-in pattern matching facilities. The built-in pattern matching facilities c i rcumvent the problem encountered with the other traditional languages used in AI p rogramming LISP and P R O L O G . In these languages the pattern- matchers must be written by the user. This factor, cou- pled with the fact that the language is implemented on the Apple Macintosh (considered one o f the easiest, if not the easiest to use mic rocompute r because o f it user interface), made the language of choice GIMLI .

In conclusion, the evolut ionary process o f the de- ve lopment o f all knowledge systems will suggest short- comings o f the prototype, and hence suggest refine- ments o f the rules and concepts o f the system. Such is the case with D E S I G N E XPE RT . Although, D E S I G N E X P E R T represents a fully functional expert system, the evolut ionary process will cont inue to suggest changes that will be implemented in the future.

REFERENCES

Aikins, J.S. (1984). A representation scheme using both frames and rules. In B.G. Buchanan & E.H. Shortliffe (Eds.), Rule-based expert systems. The Mycin experiments of the Stanford Heuristic Pro- gramming Project (pp. 424 449). Reading, MA: Addison-Wesley.

Cobb, L. (1986). Programming in Gimli (AI Memo 3). Charleston, SC: Medical University of South Carolina.

Cochran, W.G., & Cox, G.M. (1957). Experimental designs. New York: John Wiley & Sons.

Cox, D.R. (1966). Planning of experiments. New York: John Wiley & Sons.

Cox, D.R., & Snell, E.J. (1981). Applied statistics: Principles and examples. New York: Chapman and Hall.

Diamond, W.J. ( 1981). Practical experimental designs. New York: Van Nostrand Reinhold.

Fox, M.S., Wright, J.M., & Adam, D. (1986). Experiences with SRL: An analysis of a frame-based knowledge representation. In L. Kerschberg (Ed.), Expert Database Systems: Proceeding for the First International Workshop. (pp. 161-172).

Gale, W.A., & Pregibon, D. (1984a). Constructing an expert system for data analysis by working examples. Proceedings of COMP- STA T (pp. 227-236). Prague: Physica-Verlag.

Gale, W.A., & Pregibon, D. (1984b). REX: an expert system for regression analysis. Proceedings of COMPSTAT (pp. 242-248). Prague: Physica-Verlag.

Gale, W.A. (1986). Artificial intelligence and statistics. Reading, MA: Addison Wesley.

Haaland, P.D., Liddle, R.F., & Yen, D. (1985). An expert system for the design of screening experiments. Proceedings of Statistical Computing Section of American Statistical Association (pp. 223- 228). Washington, DC: American Statistical Association.

Haaland, P.D., Liddle, R.F., & Yen, D. (1986, August). An expert system for experimental design. (Becton-Dickinson Memo).

Hopkins, K.D. (1983). A strategy for analyzing ANOVA designs having one or more random factors. Educational and Psycholog- ical Measurement, 43, 107-113.

Jones, B. (1980). The computer as a statistical consultant. BIAS, 7(2), 168-195.

Koch, G.G., Amara, I.A., Stokes, M.E., & Gillings, D.B. (1980). Some views on parametric and nonparametric analysis for repeated measurements and selected bibliography. International Statistical Review, 48, 249-265.

Koch, G.G., Amara, I.A., Carr, G., Hirsch, M.G., & Marques, E. (1985). The interface between analysis strategies and design structure in repeated measurements studies. Proceedings of the Statistical Computing Section of the American Statistical Asso- ciation (pp. 19-28). Washington, DC: American Statistical As- sociation.

Lee, W. (1975). Experimental Designs and Analysis. San Francisco: W. H. Freeman and Company.

Lee, W. (1966). Experimental design symbolization and model derivation. Psychornetrika, 31, 397-412.

Milliken, G.A., & Johnson, D.E. (1984). Analysis of messy data. New York: Van Nostrand Reinhold.

Oldford, R.W., & Peters, S.C. (1984). Building a statistical knowledge based system with mini-Mycin. Proceedings of Statistical Com- puting Section of American Statistical Association (pp. 85-90).

Taylor, W.H. Jr., & Hilton, H.G. (1981). A structure diagram sym- bolism for analysis of variance. The American Statistician, 35(2), 85-93.

Documents

Design expert: An expert system application to clinical investigations