Applying rule-base anomalies to KADS inference structures

Ž .Decision Support Systems 21 1997 271–280

Applying rule-base anomalies to KADS inference structures

Frank van Harmelen )

AI Group, Department of Mathematics and CS, Vrije UniÕersiteit Amsterdam, Amsterdam, The Netherlands

Abstract

The literature on validation and verification of knowledge-based systems contains a catalogue of anomalies forknowledge-based systems, such as redundant, contradictory or deficient knowledge. Detecting such anomalies is a methodfor verifying knowledge-based systems. Unfortunately, the traditional formulation of the anomalies in the literature is veryspecific to a rule-based knowledge representation, which greatly restricts their applicability. In this paper, we show how the

Ž .traditional anomalies can be reinterpreted in terms of conceptual models in particular KADS inference structures . For thispurpose, we present a formalisation of KADS inference structures which enables us to apply the traditional rule-baseanomalies to these inference structures. This greatly improves the usefulness of the anomalies, since they can now be appliedto a much wider class of knowledge-based systems. Besides this reformulation and wider applicability of the traditionalanomalies, further contributions of this paper are a novel formalisation of KADS inference structures and a number ofimprovements to the existing formalisation of the traditional anomalies. q 1997 Elsevier Science B.V.

Keywords: Validation; Verification; Knowledge-based systems; Anomalies; Inference structures

1. Introduction

Traditional anomalies for knowledge-based sys-Ž .tems KBSs concern properties such as redundant,

contradictory or deficient knowledge. Detectinganomalies is a method for verifying KBSs. Ideallyanomaly detection must be done using the concep-tual model of a KBS, and not its implementation.This is required to keep anomaly detection free fromdetails about the particular procedural properties ofthe inference engine for the knowledge-base. How-ever, traditionally, anomalies are formulated in termsof a rule-based knowledge-representation formalism.This has made the formulation of the anomaliesspecific to a particular representation language.

) Corresponding author. E-mail: [email protected]

In this paper, we show how the traditional rule-based anomalies can be reinterpreted in terms ofconceptual models.

In the literature on validation and verification ofKBSs, many researchers have devised, categorisedand extended definitions for structural anomalies ofrule-based systems. We have taken a specific formu-

w xlation of anomalies for rule-bases from Ref. 1Ž .henceforth, Pr&Sh for short as the basis for ourwork, but that paper in turn builds upon previouswork by a variety of researchers, including Refs.w x2–5 . These papers define a wide class of anoma-

w xlies, but we focus on those in Ref. 1 , which are animportant and representative set of anomalies.

We will show how to reinterpret these anomaliesin terms of the conceptual models as proposed by the

w xKADS method of KBS construction. KADS 6 is awell-known method of constructing conceptual mod-

0167-9236r97r$17.00 q 1997 Elsevier Science B.V. All rights reserved.Ž .PII S0167-9236 97 00045-6

( )F. Õan HarmelenrDecision Support Systems 21 1997 271–280272

els for KBSs, which has found widespread accep-tance both in industry and in academia.

The traditional anomalies are most relevant for aspecific part of KADS conceptual models, calledinference structures. These inference structures aredeclarative and implementation-independent descrip-tions of the problem-solving competence of a KBS.In order to apply Pr&Sh’s formalisation of the tradi-tional anomalies to these inference structures, werequire a formalisation of these inference structures.

Ž w xHowever, existing formalisations see Ref. 7 for an.overview are too far removed from Pr&Sh’s defini-

tions. We will use a recent formalisation from Ref.w x8 , which allows a direct interpretation of the tradi-tional anomalies in terms of KADS inference struc-tures. This greatly improves the usefulness of theanomalies, since they can now be applied to a wideclass of knowledge-based systems, instead of only torule-based systems.

Furthermore, anomalies were traditionally definedŽfor a rule-based formalism such as first-order logic

w x.in Ref. 1 , rather than for a particular rule-basedŽ .representation language e.g., OPS5 or Prolog . This

was done in order to re-use verification methods.Typically, the approach taken has been to translateimplemented systems written in concrete languagesto the formalism on which the anomalies were de-fined. In abstracting the implemented systems, infor-mation is lost, which is one of the reasons whyanomalies are not necessarily faults. This approachmay seem ‘backwards’, but it arose because untilrecently, the only concrete description of the KBSwas the implementation itself.

In the above light, the contributions of this paperŽ .are: i that the formalism upon which the anomalies

are defined is at a higher level of abstraction, namelythat of the conceptual model which exists before

Ždesign choices are made for example the choice to. Ž .use rules ; ii that verification no longer depends on

the ‘backwards’ creation of a verifiable abstractionof the implemented system, but instead proceeds inthe ‘forwards’ direction: the conceptual model iscreated, verified ‘accurately’, and only then, imple-mented.

The structure of this paper is as follows. In Sec-tion 2, we summarise the anomalies as formalised byPr&Sh, and introduce a number of improvements toPr&Sh’s definitions. Section 3 describes KADS in-

ference structures and introduces Aben’s formalisa-tion of them. Section 4 uses this formalisation tointerpret the anomalies for inference structures, andSection 5 concludes. Appendix A discusses and mo-tivates in more detail the alterations we have made tothe original definitions of Pr&Sh.

Notation: In what follows, we will use lower-caseletters from the middle of the alphabet for literals,lower-case letters from the end of the alphabet forfirst order variables, lower-case Greek letters forvariable substitutions, and calligraphic letters for setsof formula, literals or terms.

2. Summary of the work by Preece and Shinghal

In this section, we discuss the main definitions ofw xRef. 1 . We will not simply summarise these defini-

tions, but also introduce improvements. Commentson our changes to Pr&Sh’s original definitions havebeen relegated to notes at the end of this paper.Although our versions of Pr&Sh’s definitions oftendiffer significantly from the original ones, they are

w xall in the spirit of Ref. 1 .We first introduce the terminology and notation

w x Ž .used in Ref. 1 : 1 a rule R is formula of the formi

l n . . . n l ™m where each l and m are first1 n iŽ .order literals; 2 for each rule R s l n . . . n l ™i 1 n

Ž . � 4 Ž .m, we write antec R s l , . . . ,l and conseq R1 n

sm. We will sometimes abuse notation and inter-Ž . Ž .pret the set antec R as a conjunction; 3 a rule set

Ž .RR is a set of rules; 4 the goal-literals GG is the setof all ground literals that could possibly be output

Ž .from the rule set; 5 the input-literals II is the setof all ground literals that constitute all possibleinputs to the rule set. Since we are dealing withfirst-order formula, the sets GG and II can in general

Ž .be infinite; 6 a semantic constraint is a set of� 4literals l , . . . ,l such that their conjunction l1 n 1

n . . . n l is regarded as a semantic inconsistencynŽ � Ž . Ž .4.e.g., the set male x , pregnant x . Semantic con-straints form a way of introducing an extended no-tion of semantic inconsistency among rules evenwhen they are logically consistent. We write CC forthe set of all semantic constraints for a rule set; andŽ .7 an environment is a subset of II that does notimply any semantic constraint. We write ´ for theŽ .possibly infinite set of all such environments. For-mally, for all eg´ and all cgCC.

( )F. Õan HarmelenrDecision Support Systems 21 1997 271–280 273

w xThere are nine anomalies that Ref. 1 defines.w xA1 Unsatisfiable rule: a rule R is unsatisfiable

iff there is no way of deducing R’s antecedent fromany legal input:

! 'egEE , 's : Rje&s(antec RŽ .Ž .Žwe write s(f for the application of substitution s

to formula f and s s (f for the simultaneous1 2.application of multiple substitutions .

w xA2 Unusable rule: a rule R is unusable iff theconsequent of R subsumes neither a goal literal norany antecedent literal in the rule set:

;s : s(conseq R fGnŽ .ŽX � 4 X

!'R gRR_ R :s(conseq R gantec R .Ž . Ž . .w xA3 Subsumed rule: a rule R is subsumed iff

there exists a more general rule:X � 4 X

'R gSCR_ R , 's :R ™s(R .

w xA4 Redundant rule: a rule R is redundant inrule set RR iff R is not essential for the computationof any goal literal from any environment:

;egEE , ;ggGG :

� 4if SCRje&g then RR_ R je&g .

w x w x w xAnomalies A1 – A3 are special cases of A4 .w x XA5 Inconsistent rule pair: rules R and R are an

inconsistent pair iff R and RX are both applicable andderive a semantic constraint:

'egEE , 's , 'sX :

RRjeqs(conseq R nŽ .RRjeqs

X(conseq RX nŽ .

s(conseq R , sX(conseq RX gC.� 4Ž . Ž .

w xA6 Inconsistent rule set: a rule set RR is incon-sistent iff from some legal input it is possible toderive a semantic constraint from RR:

'egEE , 'cgC : RRje&c.

w x w xAnomaly A5 is a special case of A6 .w xA7 Circular rule set: a rule set RR is circular iffŽ .antec R cannot be derived from any environment,

except by adding R’s consequent:

'RgRR:;egEE : RRje & antec R nŽ .'egEE : RR j e j conseq R & antec R .Ž . Ž .

w xA8 Unused input: an input literal igII is un-used iff any result that can be computed from anenvironment can also be computed from that envi-ronment minus i:

� 4;egEE ;ggGG :if RRje & g then RRje_ i &g .

w xA9 Incomplete rule set: a rule set RR is incom-plete iff there exists some output that cannot becomputed from any environment:

'ggGG :;egEE RRje&g ,

3. KADS inference structures

In this section, we describe KADS inferencestructures, both informally and formally. The infor-mal description is by now well known in the litera-

Ž w x.ture see Ref. 9 . Our particular formalisation ofKADS inference structures is new, and is based on

w xthe notions from Ref. 8 , although we have simpli-fied the formal treatment.

3.1. Informal description

Ž .KADS inference structures are used to model iwhat the legal inferences are that can be made by a

Ž .system; ii the role that domain knowledge plays inŽ .these inference steps; and iii the dependencies be-

tween these inference steps. An inference structure isa collection of inference steps and knowledge roles.See Fig. 1 for a simple example. Each inference stepis a relation between its input knowledge roles andoutput knowledge roles. Each knowledge role is acollection of domain knowledge that, by being inputor output of a particular inference step, plays aparticular role in the inference process. Dependen-cies between inference steps are modelled by sharingknowledge roles among inference steps: the outputof one inference step can be the input for anotherinference step.

Notice that a KADS inference structure is indeedat the abstraction level of a conceptual model: itdescribes inference steps and knowledge roles with-out committing us to a particular data-structure to


Fig. 1. A KADS inference structure for computing the distance between two sets. Inference steps are ovals, knowledge roles are boxes,Ž .arrows indicate data dependency. In a , inference steps max and min construct from their input knowledge roles set and set the pairs of1 2

minima or maxima respectively. The inference step diff subsequently computes the difference between the elements of such a pair toproduce a measure of the distance between the two input sets.

represent the knowledge roles, or to a particularalgorithm to implement the inference steps. Further-more, an inference structure is indeed declarative andnot procedural, as required in Section 1: it expressesdata-dependencies among inference steps but doesnot specify control flow. For example, in Fig. 1,many different control regimes can still be imposed:first select all maximal elements or all minimalelements, or selecting one maximal and minimalelement alternatively, etc. Such control flows are notspecified in the inference structure, leaving themfully declarative.

It is exactly this set of properties which makesKADS inference structures so useful for validationand verification. They define the functionality of aKBS, but do so independently of particular datastruc-tures and control choices. This means that any prop-erties we will proof of the inference structures willalso be independent of such choices. This allowsvalidation and verification to be done in an earlierstage of KBS development, before choices are madeconcerning such implementation details. Any resultsobtained for the conceptual model will remain validfor different designs that we might choose in a laterstage. It will be convenient to use both a graph-theo-retical and a logical formalisation.

3.2. Graph-theoretical formalisation

An inference structure is a directed graph wherethe nodes are inference steps or knowledge roles, andwhere any edge connects either an inference step to aknowledge role or vice versa. All sources and sinksof such a graph must be knowledge roles.

3.3. Logical formalisation

The formalisation given here is compatible withthe formalisation of inference structures in our speci-

Ž .2 w xfication language ML 10 , but the precise detailsof the relation are beyond the scope of this paper.After this work was completed, we learned that itresembles the formalisation of data-flow diagrams,

w xas given in Ref. 11 . Although the spirit of thatformalisation is very similar to ours, many of thetechnical details are significantly different.

3.4. Knowledge role

Each knowledge role is a unique first-order vari-able of a unique type. It may seem strange that

Ž .knowledge roles which contain domain knowledgeŽare formalised by first order variables i.e., ranging


.over terms and not by sentences. By formalising aninference structure as a meta-theory of the domain-

Ž Ž .2 .knowledge as done in ML , we can still modeldomain knowledge as first-order sentences whileviewing knowledge roles as variables over terms.

w xAgain, we refer to Ref. 10 for the technical detailsof this solution.

3.5. Inference step

An inference step is a predicate with, as argu-ments, the variables corresponding to the connectingroles. The predicate characterises the inputroutputrelation of the inference step.

Example: the inference step diff from Fig. 1 isŽ .formalised as predicate diff y< pairs, z<measure ,

defined by:

; y :: pairs, z ::measure:diff y ; z lŽ .' y , y : ys y , y nzsy yy . 1Ž . Ž .1 2 1 2 1 2

The notation y< pairs indicates that variable y isof type pairs. We use the ; notation to separate inputarguments from output arguments, but this notationhas no formal relevance.

3.6. Confluent knowledge role

A knowledge role is confluent iff it is an outputrole of more than one inference step.

3.7. Confluence-free inference structure

An inference structure which is free from conflu-ent knowledge roles is formalised by the conjunctionof its inference steps. If a knowledge role r connects

Ž .two inference steps say p and p , then p and p1 2 1 2

use the same variable for the argument of type r.Example: the inference structure from figure Fig.

1b is formalised as:

² :x ::set , x ::set , y :: pairs, z ::measure1 1 2 2

min x , x ; y n diff y , z .Ž . Ž .1 2

Notice that all occurrences of the variables in this² :formula are free. The . . . notation is only an

informal annotation to indicate the typing of thesefree variables.

3.8. Terminal roles

A knowledge role is a terminal role iff it is notthe input role of any inference step.

3.9. Initial roles

A knowledge role is an initial role iff it is not theoutput role of any inference step.

3.10. Fully connected inference structure

An inference structure I is fully connected iffthere is a sequence of connected steps and rolesbetween any two roles if we ignore the direction ofthe edges.

3.11. Confluence-free sub-structure

J is a confluence-free substructure of inferencestructure I iff J is a fully connected subgraph of Ithat includes all terminal roles of I and where norole is the output of more than one inference step.Confluence-free substructures of I can be obtainedby ‘tracing back’ from the terminal roles of I, andchoosing one alternative when there is a choice ofpreceding inference steps.

Example: Fig. 1b,c are exactly the confluence-freesub-structures of Fig. 1a.

3.12. Formalisation of an inference structure

An inference structure I is formalised as thedisjunction of its confluence-free sub-structures.

Example: the inference structure I from Fig. 1b isformalised as: p

² :x <set , x <set , y< pairs, z<measure1 1 2 2

min x , x ; y n diff y ; z kŽ . Ž .Ž .1 2

max x , x ; y n diff y ; z .Ž . Ž .Ž .1 2

Prefix: the prefix of an inference p in an infer-ence structure I is the largest fully-connected sub-graph J of I of which the terminal roles are all theinput roles of p.


Example: the following formula is the prefix ofdiff in Fig. 1a:

prefix diff 'Ž .² :x <set , x <set , y< pairs, z<measure1 1 2 2

min x , x ; y k max x , x ; y .Ž . Ž .1 2 1 2

3.13. Assignment to a knowledge role

An assignment of a value Õal<type to a knowl-� 4edge role Õar<type is a substitution ss ÕarrÕal

Žwhere Õal is a ground term i.e., not containing.variables . We will also speak of an assignment for

m ultip le know ledge ro les w hen s s�Ž . Ž .4Õar rÕal , . . . , Õar rÕal .1 1 n n

3.14. Satisfying an inference structure

Let s be an assignment for all the knowledgeroles occurring in an inference structure I, and letDEF be the definition of all inference steps occurring

Ž Ž .in I in the same way that Eq. 1 is the definition.for diff , then s satisfies I iff DEF*s( I. This

definition allows I to consist of a single inferencestep, in which case we will speak of the satisfactionof the inference step. We will often implicitly as-sume the definition of all the inference steps in DEF,and will not keep including DEF in our formulae.Because of this, and to improve the readability,

Ž .instead of DEF*s( I we will write SAT s , I .Ž .When writing SAT s , I , s is assumed to be a

ground substitution, as mentioned before. Further-more, s must be an assignment for all knowledge

Ž .roles in I. The intuition behind SAT s , I is that s

is an assignment to knowledge roles in I correspond-ing to a legal execution of I.

Ž .Example: if DEF is definition Eq. 1 for diff plusthe obvious definitions for max and min, then thefollowing assignments s and s satisfy the infer-1 2

Ž .ence structure from Fig. 1 as formalised in 3 :

� 4 � 4s s x r 1,2,3 , x 4,6,8 , yr 1,4 , zr3� 4Ž . Ž .Ž .Ž . Ž .1 1 2

� 4 � 4s s x r 1,2,3 , x 4,6,8 , yr 3,8 , zr5 .� 4Ž . Ž .Ž .Ž . Ž .2 1 2

Notice that s and s assign the same values to the1 2

input roles of I, but contain different assignments forthe output roles. This reflects the non-determinacy of

the inference structure I, as revealed by the fact thatit has multiple confluence-free sub-structures.

4. Anomalies for inference structures

In this section, we will use the formalisation ofKADS inference structures from Section 3 to applyPr&Sh’s definition of rule-based anomalies to infer-ence structures. As explained in Section 1, this willprovide us with a formulation of anomalies in termsof a proper conceptual model, instead of the imple-mentation specific language of production rules.

The intuition behind our reformulation is to viewan inference step as the analogue of a rule, and a ruleset as an entire inference structure. Antecedent andconsequent of a rule then correspond to the input andoutput types of an inference step, and input- andgoal-literals become all possible values for the initialand terminal roles of an inference structure. Thisanalogy can be made precise by the following rules:Ž . Ž .1 for an inference structure I, inputs I is the set oftypes corresponding to all initial roles of I, and

Ž .outputs I is the set of types corresponding to allterminal roles of I. Additionally, we will write in-

Ž .tern I for the set of all types that are neither initialŽ .nor terminal roles. We will write sS inputs I if s

is a substitution for a set of variables of exactly theŽ . Ž .types in inputs I , and similarly for outputs I and

Ž . Ž .intern I ; 2 the goal set GG is the set of all possibleassignments to terminal roles. For example, if aninference structure has two terminal roles, r and r ,1 2

�then GG will have the form GG s r rÕal ,1 114 Ž .r rÕal , . . . , r rÕal , r rÕal , . . . ; 3 the input1 12 2 21 2 22

set II is the set of all possible assignments to initialroles. As before, GG and II will in general be

Ž .infinite; 4 the definitions of semantic constraint andŽ .the constraint set CC are as before and; 5 an envi-

ronment ´ is a subset of II that assigns a value toeach initial knowledge role in such a way that nosemantic constraint is implied. As before, we write

Ž .EE for the possibly infinite set of all such environ-Ž .ments. Formally: ´:II and ´S inputs I and ;´(c

for all ´gEE and all cgCC.Comparing this list with the one in Section 2

shows that a shift has taken place from predicates tow x Žterms in the formalisation: in Ref. 1 GG and II and


.consequently EE were sets of ground literals and arenow sets of ground term-substitutions. Similarly,

Ž . Ž .antec R and conseq R were sets of literals whileŽ . Ž .inputs I and outputs I are sets of term-types. This

Ž .is the result of no longer taking a sentence a rule asŽthe primitive element, but rather a predicate in-

.ference step . This shift will be noticeable in thereformulation of Pr&Sh’s anomalies to which weturn now. The important point to notice in the newformulation of the anomalies below is the closecorrespondence with the formulation of the anoma-lies in Section 2, both formally and informally.

w ) xA1 Unsatisfiable inference step: an inferencestep p is unsatisfiable iff there is no way of satisfy-ing p from any legal input:

! '´ g EE ,'s :SAT ´s , prefix p np ,Ž .Ž .Ž .

i.e., there exists no input environment e which canbe completed with another substitution s to form asatisfying assignment for p and its prefix.

w ) xA2 Unusuable inference step: an inference stepp is unusable iff no output value of p contributes toeither the goal-set or to satisfying another inferencepX:

;s ,;vSoutputs p :Ž .if SAT sv , pŽ .then v l SCGs0 and

X X X X� 4! 'p g I _ p ,'s : SAT s v , p nŽ Ž .X X

!SAT s , p ,Ž . .

i.e., for all possible satisfying assignments sv of p,the output roles v never contribute to the goal setGG, and they also never form an essential part of anysatisfying assignment s

Xv for another inference step

pX.An example of this is the following: if in the

Ž Ž . Ž .. Ž .inference structure p x ; y kp x ; y nq y; z ,1 1 2 2

the inference q never holds for any value of yderived by p , then p is unusable.2 2

w ) xA3 Subsumed inference step: an inference stepp is subsumed iff there exists another inferencewhich, for any inputroutput relation of p, specifies

at least the same output from at most the same input:

X � 4'p g I _ p :

;sS inputs p ,;vSoutputs pŽ . Ž .if SAT sv , pŽ .

X X X Xthen 's S inputs p ,'v Soutputs p :Ž . Ž .X X X X XSAT s v , p ns :snv:v .Ž .

An example is the following: in the inference struc-Ž . XŽ .ture p x , x ; y kp x ; y , if for any value of x ,1 2 1 1

pX computes the same value for y as p does, irre-Ž Žspective of the value of x in other words: p x ,2 1

. XŽ X. X.x ; y np x ; y ™ysy , then p is subsumed by2 1

pX.w ) xA4 Redundant inference step: an inference step

p is redundant in inference structure I iff p is notessential for the derivation of any output assignmentfrom any environment:

;´ g EE ,;vSoutput I :Ž .if 'sS intern I :SAT ´sv , IŽ . Ž .

X X � 4then 's intern I :SAT ´s v , I _ p ,Ž . Ž .

i.e., whenever some value v for I ’s output roles isŽcomputed from some input environment ´ via an.additional assignment s for I ’s internal roles , then

the same output values can be computed from thesame input values via an inference structure that

w ) xdoes not contain p. Notice that, as before, A1 –w ) x w ) xA3 are special cases of A4 .

w ) xA5 Inconsistent inference steps: we do nottreat this case separately, since it is a rather uninter-

w ) xesting special case of A6 .w ) xA6 Inconsistent inference structure: an infer-

ence structure I is inconsistent iff from some legalinput it is possible to derive a role assignment thatsatisfies I and which also satisfies a semantic con-straint:

'´ g EE ,'s , 'c g SCC :SAT ´s , I nSAT ´s ,c .Ž . Ž .

w ) xA7 Circular inference structure: we have notsucceeded in finding a suitable definition of circular-ity of an inference structure.

w ) x Ž .A8 Unused input value: an input value ÕarrigII is unused iff any result v that can be com-puted from an environment ´ can also be computed


Ž .from that environment minus Õarri :

;´ g EE ,;vSoutput I ,;sS intern I :Ž . Ž .� 4if SAT ´sv , I then SAT ´sv _ Õar r i , I ,Ž . Ž .

i.e., for any input environment ´ and for any outputassignment v that can be computed from ´ viasome internal assignment s , the same v can be

Ž .computed from the same ´ without using Õarri .As an example, in the inference structure men-

w ) xtioned under A3 all values for x would be2

unused.w ) xA9 Incomplete inference structure: an infer-

ence structure I is incomplete iff there is a terminalrole value which cannot be derived from any envi-ronment.

' Õar r g g SCG:;´ g EE ,;s :Ž .if SAT ´s , I then Õar r g f s ,Ž . Ž .i.e., no possible way ´s of satisfying I from some

Ž .input environment e contains Õarrg .

5. Conclusion

Ž .In this paper, we have presented i a number ofimprovements to the formalisation of rule-based

Ž .anomalies by Pr&Sh, ii a formalisation of KADSŽ .inference structures, and iii an interpretation of the

traditional anomalies in terms of these implementa-tion independent structures. This greatly improvesthe usefulness and applicability of the anomalies as amethod of KBS verification. It also opens up thepossibility to analyse other properties of inferencestructures as part of KBS verification, such as deter-minedness, consistency and coherence, as defined in

w xRef. 8 .Some technical problems remain to be solved for

our formalisation: the current formalisation can onlydeal with circular inference structures in a trivialway, which does not correspond to their actual use inpractice, namely to model non-monotonic reasoningprocesses. It remains to be seen if the current formal-isation can be extended to deal with these cases.

As Pr&Sh state, anomalies are not errors, theyare only indications of possible errors. As a result,anomaly detection is only half the work. After ananomaly has been detected in an application, we

need to know which errors are responsible for thatanomaly, and how these errors can be repaired.Further study of error-classes for inference structuresis required before this problem can solved satisfacto-rily.

As with any formal method, the proof of thepudding is in the eating. At first sight, it mightappear that our formulation of the anomalies is notdirectly applicable: many of the anomalies stateproperties about infinite sets such as input- or goal-literals. However, the same objection could be raisedagainst Pr&Sh’s original anomalies. Nevertheless,

w xRef. 1 reports a number of case studies that haveestablished both the feasibility and the usefulness ofthe anomalies on a number of realistic KBS applica-tions. This gives us sufficient confidence that thiswill also be the case for our reformulation of theanomalies for KADS models.

The next step in this work must no doubt be toapply these anomalies to realistic KADS models, andto subject our definitions to similar empirical tests asreported by Pr&Sh. To achieve this end, it willcertainly be necessary to build software that automat-ically checks inference structures for the anomaliesthat we have defined in this paper.

Acknowledgements

We are grateful to two anonymous referees, whosecomments have helped to improve this paper.

Appendix A. Comments on the work by Preeceand Shinghal

In this section we describe and motivate the alter-ations we have made in Section 2 to the definitions

w xfrom Ref. 1 .w xA1 Unsatisfiable rule: Pr&Sh’s original defini-

tion is:

'l g antec R :Ž .! 's : s( l g SCI kŽ

X X� 4'R g SCR_ R :'s :s( l g conseq R .Ž . .Because of the existential quantification over a sin-gle antecedent literal l, this definition disregards thecase when two antecedent literals are both satisfi-


able, but not under the same substitution. ConsiderŽ . Ž . Ž . �Ž .the rule l 1, y n l x,2 ™m x, y with EEs 1,1 ,1 2

Ž .42,2 . Pr&Sh’s definition does not capture this ruleas unsatisfiable, although it will never fire fromeither of the given environments. Besides being sim-pler, our definition does not suffer from this prob-lem. On the other hand, the original definition hasthe advantage of being more easily computable byautomatic checkers.

w xA2 Unusable rule: our formulation is equivalentto the one by Pr&Sh.

w xA3 Subsumed rule: Pr&Sh’s definition for asubsumed rule is:

'RX's : R™s(RX .

We believe this to be an error. Consider Rsa™band RX sanaX

™b. Then R™RX but it wouldseem to us that RX is subsumed, rather than R.

w xA4 Redundant rule: Pr&Sh’s definition for re-dundancy of R is:

< <� 4 � 4;e g EE : h SCR j e & h s h SCR_ R j e & h .� 4

Our definition differs from this in two aspects:first, if we regard this equality as a mutual inclusionof the left- and right-hand side, then the inclusion =

Ž .of this equality is trivial by monotonicity of & , sowe omit it from our definition. Secondly, we restrictthe inclusion to literals from GG instead of using anyliteral h. After all, rules are also redundant if theyonly contribute to irrelevant literals. Therefore, ourdefinition makes all rules redundant which do not

Ž .contribute directly or indirectly to the computationof goal literals, which is not the case in Pr&Sh’sdefinition.

w xA5 Inconsistent rule pair: besides our definition,Ž . Ž X.Pr&Sh also demand that antec R ™antec R .

Presumably, this is to ensure that R and RX areapplicable in the same cases. In our definition, weexplicitly demand that both R and RX are applicablefrom the same environment, rather than relying on

Ž . Ž X.the special case of antec R ™antec R . Pr&Shcall this anomaly ‘ambivalent rule pair’, but weprefer the term ‘inconsistent’, since the rules R andRX imply a semantic constraint which is interpreted

w xas a semantic inconsistency. Again, as with A3 , theoriginal version has the advantage of being moreeasily computable by automatic checkers.

w xA6 Inconsistent rule set: our definition is equiva-lent to the one by Pr&Sh.

w xA7 Circular rule set: Pr&Sh’s original definitionis:

'R g SCR ,'e g EE :SCR j e ; antec R nŽ .SCR j e j conseq R & antec RŽ . Ž .

This already makes R circular if its antecedentcannot be derived from some environment e exceptby adding R’s consequent to e. Our definition isstronger, and demands that R is only circular if itsantecedent cannot be derived from any environment,although it can be derived from some environmentby adding R’s consequent.

w xA8 Unused input: Pr&Sh’s original definitionfor an unused input i demanded that:

i f SCG n ! 'R g SCR ,'s :i g s`antec RŽ .Ž .which says that i is never used, either as a finalresult, or for triggering a rule. This definition suffersfrom the problem that i may be used by a rule that,when using i, does not contribute to computing anyelement of GG, even though the rule does contributeto elements of GG from other environments, andtherefore does not qualify as an unusable rule. Thiswould mean that i goes unnoticed as an unusedinput. The alternative definition that we proposeddoes not suffer from this problem.

w xA9 Incomplete rule set: Pr&Sh’s definition ofan incomplete rule set was:

'e g EE :;g g SCG: SCR j e ; g ,

Ži.e., there exists an environment from which RR will.not produce any output at all . This definition, which

has the reverse quantifier scheme from the one wepropose, says more about the environment e than itdoes about the rule set RR: the fact that RD e;g for

Žany g may be due to an erroneous environment e.g.,.esØ which was not prevented by sufficiently strong

semantic constraints. By universally quantifying overEE, our definition avoids this problem.

References

w x1 A. Preece, R. Shinghal, Foundation and application ofŽ .knowledge base verification, Int. J. Intelligent Syst. 9 1940

683–701.w x2 M. Suwa, A. Scott, E. Shortliffe, An approach to verifying

completeness and consistency in a rule-based expert system,Ž . Ž .AI Magazine 3 4 1982 16–21.


w x3 T.A. Nguyen, W.A. Perkins, T.J. Laffey, Checking an expertsystems knowledge base for consistency and completeness,IJCAI-85, 1985, pp. 375–378.

w x4 A. Ginsberg, Knowledge-base reduction: a new approach tochecking knowledge bases for inconsistency and redundancy,Proceedings of the 7th National Conference on ArtificialIntelligence AAAI’88, 1988, pp. 585–589.

w x5 C. Chang, J. Combs, R. Stachowitz, A report on the expertŽ . Ž .systems validation associate eva , Expert Syst. Appl. 1 3

Ž .1990 217–230.w x6 B.J. Wielinga, A.Th. Schreiber, J.A. Breuker, Modelling

expertise, In: A.Th. Schreiber, B.J. Wielinga, J.A. BreukerŽ .Eds. , KADS: A Principled Approach to Knowledge-basedSystem Development, Academic Press, London, 1993, pp.21–46.

w x7 D. Fensel, F. van Harmelen, A comparison of languageswhich operationalise and formalise KADS models of exper-

Ž .tise, Knowledge Eng. Rev. 9 1994 105–146.w x8 M. Aben, Formal methods in knowledge engineering, PhD

thesis, University of Amsterdam, Faculty of Psychology,ISBN 90-5470-028-9, February 1995.

w x9 A.Th. Schreiber, The KADS approach to knowledge engi-Ž . Ž .neering, Knowledge Acquisition 4 1 1992 1–4, Editorial

special issue.w x Ž .210 F. van Harmelen, J.R. Balder, ML : a formal language for

Ž .KADS models of expertise, Knowledge Acquisition, 4 11992. Special issue: the KADS approach to knowledge engi-neering, reprinted in KADS: a principled approach to knowl-edge-based system development, In: A.Th. Schreiber, et al.Ž .Eds. , 1993.

w x11 P.G. Larsen, N. Plat, H. Toetenel, A formal semantics ofŽ .data-flow diagrams, Formal Aspects of Computing 3 1993

1–21.

Ž .Frank van Harmelen 1960 studiedmathematics and computer science inAmsterdam. In 1989, he was awarded aPhD from the Department of AI in Edin-burgh for his research on meta-level rea-soning. While in Edinburgh, he workedwith Dr. Peter Jackson on Socrates, alogic-based toolkit for expert systems,and with Prof. Alan Bundy on proofplanning for inductive theorem proving.After his PhD research, he moved backto Amsterdam where he worked from

1990 to 1995 in the SWI Department under Prof. Wielinga. Hewas involved in the REFLECT project on the use of reflection inexpert systems, and in the KADS project, where he contributed to

Ž .2the development of the ML language for formally specifyingKnowledge-Based Systems. In 1995 he accepted a lectureship atthe Vrije Universiteit Amsterdam, where he is a member of the AIresearch group. His current interests include formal specificationlanguages for knowledge-based systems, meta-reasoning and re-flection, and the use of meta-reasoning in diagnosis. He is authorof a book on meta-level inference, editor of a book onknowledge-based systems, and has published over 30 researchpapers.

Documents

Applying rule-base anomalies to KADS inference structures