A flexible architecture for autonomous agents

This article was downloaded by: [Wageningen UR Library]On: 25 July 2014, At: 15:30Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Journal of Experimental& Theoretical ArtificialIntelligencePublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/teta20

A flexible architecture forautonomous agentsS. K. DAS , J. FOX , D. ELSDON & P. HAMMONDPublished online: 09 Nov 2010.

To cite this article: S. K. DAS , J. FOX , D. ELSDON & P. HAMMOND (1997)A flexible architecture for autonomous agents, Journal of Experimental &Theoretical Artificial Intelligence, 9:4, 407-440, DOI: 10.1080/095281397146979

To link to this article: http://dx.doi.org/10.1080/095281397146979

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

http://www.tandfonline.com/loi/teta20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/095281397146979

http://dx.doi.org/10.1080/095281397146979

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

J. E x p t . T h e o r . A r t i f . I n t e l l . 9(1997)407± 440

A ¯ exible architecture for autonomous agents*

S. K. DAS Œ , J. FOX � , D. ELSDON u and P. HAM M OND �

Œ W i l l i a m P e n n e y L a b o r a t o r y , I m p e r i a l C o l l e g e , L o n d o n SW 7 2AZ, UK

tel. : 44 171 594 8424

email : skd ! doc.ic.ac.uk

� Advanced Computation Laboratory, Imperial Cancer Research Fund,

London W C2A 3PX , UK

email : jf ! acl.icnet.uk

u Integral Solutions Limited, Basingstoke , Hampshire RG21 4RG , UK

email : davide ! isl.co.uk

� Department of Computer Science, Brunel University, Middlesex UB8

3PH , UK

email : p.hammond ! brunel.ac.uk

Abstract. A generic architecture for autonomous agents is presented. In common

with other current proposals the agent is capable of reacting to and reasoning about

events which occur in its environment, executing actions and plans in order to achieve

goals in its environment, and communicating with other agents. The work described

here proposes certain advances on other systems, notably the ability to reason about

and make decisions under uncertainty, including decisions about competing beliefs

and alternative actions. The framework is grounded in a non-classical decision model,

the `domino ’ model. This is formalized to ensure continuity with classical decision

theory and avoid ad hoc features. The domino model is embodied in a well-de® ned

knowledge representation language, R # L , which explicitly supports the central

concepts of decisions and plans, and associated constructs of goals, arguments,

commitments, obligations and constraints. The availability of such a language

provides a sound basis for building knowledge-based agents for practical applications.

A major issue for such applications, however, is how to ensure their safe operation.

This is a central issue whether the agents are used in an advisory role (e.g. decision

support systems) or an autonomous one (e.g. in a robot). Techniques for explicit

management of safety are described and some broader theoretical implications are

discussed.

* The RED project was initiated by the Imperial Cancer Research Fund and Queen Mary and

West® eld College and supported under the DTI } SERC project ITD 4 } 1 } 9053 : Safety-CriticalSystems Initiative. The authors would like to thank Integral Solutions Ltd for project

management, Masons Solicitors and Lloyds Register for their stimulating studies on sociolegal

issues and software safety, and Paul Krause of the Imperial Cancer Research Fund and SimonParsons of Queen Mary and West® eld College, for many helpful discussions. We would

particularly like to remember Mike Clarke who shared in the conception of the project but who

sadly died during its course. This paper was completed while SKD was working at theComputer Science Department of Queen Mary and West® eld College, London . PH was

previously at the Advanced Computation Laboratory, Imperial Cancer Research Fund,

London.

0952 ± 813X } 97 $12 ± 00 ’ 1997 Taylor & Francis Ltd

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014

408 S. K . Das et al.

1. Introduction

This paper reports results from a project which has carried out a series of experiments

on the design of intelligent agents. The project aimed to address theoretical and

practical issues, notably concerned with soundness and safety of such agents. It has

resulted in a consolidated architecture which has a number of features of theoretical

interest (Das et al. 1996) and has been demonstrated on a range of medical applications

(Fox and Das 1996). The paper is intended to provide a su� ciently comprehensive

presentation to permit applications designers and experimentalists to reconstruct it

and explore its operational properties, and theoreticians to analyse its properties with

respect to formal work in the literature.

The kind of agent that we are aiming at has the following capabilities : it should be

capable of operating in uncertain environments and detect and act in response to

events that occur in time. It should also be able to make decisions about the

interpretation of events whose nature is uncertain, choose between plans and actions

necessitated by those interpretations and manage the execution of plans in the face of

uncertain consequences. There are many potential applications for such an agent, in

equipment monitoring, industrial process management or vehicle control systems, for

example. A common class of application is in systems which need to detect faults that

arise in the system, diagnose their cause, and select and manage the execution of repair

procedures.

The design of agents for this class of applications overlaps strongly with a central

theoretical challenge in AI, namely the design of àutonomous ’ agents. The

prototypical AI agent is some form of automaton which may be mobile, monitor its

environment via sensors, and act upon its environment through eŒectors, without

external intervention or control. The concepts reported here have been developed

within the context of an engineering project concerned with the design of knowledge-

based software for safety critical applications. The speci® c requirements for the agent

model that is described did not include requirements for sensors, eŒectors etc., but in

many other respects the requirements, and the theory and architecture which have

emerged from the work, have much in common with agents being developed by other

groups in mainstream AI.

The principal applications that have informed the design of the agent described here

are in clinical medicine. M edicine has been a valuable forcing function on the

development of many generations of AI technology, because it is conceptually rich and

practically complex, and of course it is socially important. M edicine poses a wide

range of problems for theories of knowledge representation, problem solving, decision

making, planning, the management of uncertainty and so forth. It has played much the

same role in our work, namely to pose challenges which require not just pragmatic

engineering solutions but ones which are principled and provide general techniques

which are applicable to other domains.

The present paper reports certain novel theoretical ideas for agent theory, notably

in the area of decision making under uncertainty. A number of these results have been

reported in detail elsewhere, so the main contribution of the present paper is that it

presents an experimental synthesis of these results, and places them in a wider context

of research on intelligent agents. We believe that this experiment has shown that the

synthesis provides the basis of a sound and practical technology which has much to say

to the wider AI research community.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014

A ¯ exible architecture for autonomous agents 409

1.1 Overview of paper

The next section brie¯ y introduces a medical scenario which is intended to provide a

concrete illustration of the kind of application that has informed the work. W e then

position the functionality required by this scenario with respect to current views of

intelligent agents. In Section 3 we describe the design of the `domino ’ agent which has

been developed to address these requirements, emphasizing issues concerned with its

ability to make decisions and manage actions and plans. After providing an intuitive

presentation of the domino model we present it more formally in Section 4 by

describing a knowledge representation language R # L which has been designed to

embody the model’ s central concepts and represents the main synthesis of the work.

The discussion section returns to additional issues in agent theory (e.g. multi-agent

systems) and a ® nal appraisal.

Although the domino model and representation are presented with medical

examples we have tried to present it in a way that makes domain independent features

clear. We have also tried to ensure that the paper is technically rigorous, yet does not

sacri ® ce readability and accessibility to the need for formal clarity. To do this the body

of the paper concentrates on providing an intuitive presentation of the central ideas

and operation of the model, reserving formal detail for appendices dealing with the

management of uncertainty in decision making (Appendix A) ; the logic LR # L to

provide the semantics of agent knowledge representation language R # L (Appendix B)

and the method for translating R # L into LR # L (Appendix C). W e hope these

appendices provide su� cient technical information to permit others to implement and

experiment with the domino agent.

2. Theory of agents

2.1 Scenario

Suppose a patient complains of serious and unexplained loss of weight. As an

abnormal and potentially serious condition a decision has to be taken as to the most

likely cause of the complaint. Increasingly in medicine the decision making and

management required are represented by written guidelines and protocols. The

procedures involved can be represented and summarized with a simple graphical

notation, an example of which is given in Figure 1. In this notation circles represent

decisions and rectangles represent atomic actions and or compound action plans.

Figure 1. A medical protocol including decisions, plans and actions.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


The ® rst step in the protocol in Figure 1 is a diagnosis decision, shown on the left

of the ® gure. To make the decision we have to identify all potentially relevant causes

of weight loss (such as cancer and peptic ulcer) and then generate and evaluate

arguments for and against each of these candidates. Arguments will be based on

relevant information such as the patient’ s age and personal history, symptoms and so

on. The evaluation of these arguments will include not only an assessment of the

number and strength of each independent argument and the collective force of

argument for each hypothesis but also the plausibility of the evidence and justi ® cation

for each line of argument.

After evaluating the arguments let us suppose that we take the decision that cancer

is the most likely cause of the weight loss, i.e. we `commit ’ to acting on this conclusion.

Now we must take another decision, about the appropriate therapy. The possible

candidates for this may be chemotherapy, surgery and radiotherapy for example. As

before, in assessing these alternatives we need to consider arguments for and against

these options, taking into account the past e� cacy and known costs of each therapy

etc. Suppose that after weighing up the arguments we conclude that chemotherapy is

most appropriate for this patient and we are ready to take a decision. At this point,

however, we must be very careful since once an action is taken it cannot be reversed.

In taking the decision, therefore, we not only need to consider the general pros and

cons of each action, but also anticipate the speci® c consequences for this patient of

taking a wrong decision, whether the action is likely to achieve the speci® c clinical goal

and achieve it without unacceptable side-eŒects or other consequences.

In short, in order to safely carry out the decision making and patient management

required in such a medical scenario it is necessary to take into account the normal

forms of reasoning modelled in decision analysis, expert systems, etc., but also to be

able to re¯ ect upon the reasons for and consequences of our actions. The heart of this

paper is concerned with methods of knowledge representation and reasoning which

address these requirements.

2.2 The nature of an intelligent agent

A recent review of the theory and practice of intelligent agents (Wooldridge and

Jennings 1995) attempts to de ® ne the concept with respect to two general usages. The

® rst `weak ’ usage restricts the term to hardware or software systems which have

abilities to perceive and react to their environments ; take the initiative in order to

achieve goals in their environments ; operate autonomously without direct intervention

by humans or others, and interact with other agents, including humans, via some kind

of agent communication language.

Wooldridge and Jennings view these characteristics as uncontentious, but note that

many in AI have a stronger usage. In this, in addition to the above properties, an agent

is thought to resemble a human in certain interesting ways. For example, the language

used to describe agent behaviour is often mentalistic, depending on terms like

knowledge, belief, obligation and intention. This paper is about an AI system which

may be viewed as an agent in both these senses. However, to provide a basis for a

structured discussion we prefer to de ® ne the concept of an agent in a way that is more

mundane, but which leaves room to develop theories of increasingly complex agents

which do not necessitate, although do not exclude, anthropomorphic interpretations.

A mundane agent de® nition therefore runs as follows.

An agent is any natural or arti® cial system, whether logical or physical , which exists in an

environm ent and whose function is to preserve or achieve certain states in that environment. The

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


behaviour of an agent is to monitor the environment over time and act upon it to implement its

objectives.

In other words we may think of an agent as an abstract (if complex) function which

maps from states of an environment to actions on the environment :

agent : Environment U Action

Obviously a great deal of conventional hardware and software (including simple

devices like boiler thermostats or non-autonomous systems like word processors)

satisfy the above minimal de® nition. W e shall therefore refer to this class of agent as

a zero-order agent. Zero-order agents react to environmental conditions (mediated by

sensors, command processors, or whatever) and respond by evaluating some function

which yields a result or eŒect.

Countless pieces of conventional software (e.g. industrial manipulators, software

gophers) and AI systems (e.g. basic agents, Vere and Bickmore 1990, intelligent agents,

W ooldridge and Jennings 1995) satisfy the de® nition. It is consequently somewhat

unilluminating since these systems have various other capabilities which we wish to

position with respect to the theory of agents. We therefore propose to introduce an

additional descriptive dimension to discriminate among signi® cantly diŒerent classes

of agent. This dimension concerns the degree to which an agent makes its beliefs,

knowledge, etc. explicit, and hence the degree of re¯ ection of which the agent is

capable.

The ® rst level of re¯ ection concerns the explicit representation of the environment,

rather than leaving that representation implicit in the agent’ s procedures. A ® rst-order

agent, therefore, is a zero-order agent which maintains an explicit model of its

environment. This model represents the agent’ s current model of the state of the

environment (anthropomorphically its `beliefs ’ ) and may include a record of past

events and actions and future actions (its ` intentions’ ).

Event e M odel { Action

W hich is to say that the action behaviour of the agent may be derived from the event

and the current state of the agent’ s model. An agent can exploit such a model in many

ways, to detect change by comparing past and present states ; predict unknown or

future states by trend extrapolation ; schedule actions with respect to future events ;

answer queries from other agents about its beliefs and intentions, and so forth.

A second-order agent is a ® rst-order agent which maintains more than one world

model. Second-order agents have the potential to be more adaptive than ® rst-order

agents because they can compare the properties of alternative interpretations. This

permits not only an assessment of which is the most plausible (e.g. by comparing

probabilities), but also permits construction of action plans which cover for more than

one contingency (e.g. by selecting actions which optimize anticipated cost-bene® t over

the set of possible futures).

Event e ² Model ´ { possible(Action)

² possible(Action) ´ { commit(Action)

where ² X ´ is the set of all Xs, `possible ’ means that there is a reason to believe that the

action will achieve some goal of the agent, and `commit ’ means that the agent intends

to carry out a particular action selected from the set of possible actions.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


Third-order agents are second-order agents which maintain a higher-order (meta)

model of their beliefs and desires, including the justi® cations for their beliefs and

intentions and the expected consequences of their intended actions.

Event e ² Proposition V Justi ® cation ´ { possible(Action)

² possible(Action) ´ { commit(Action V Goal )

A meta-model introduces considerable additional representational power, such as the

ability to justify beliefs or actions to other agents. Third-order agents can also

construct counter-factual models, as in ` if such and such were diŒerent how would this

aŒect the interpretation? ’ , or `what would I need to do in order to achieve a certain

state in the currently preferred environment model which is not presently true in that

model ? ’ .

This paper is concerned with developing a theory of, and describing an experimental

architecture for, third-order agents. The agent we have built is capable of having

`goals ’ with respect to its environment ; monitoring events and states in the

environment ; detecting problem states (i.e. states which are inconsistent with its

goals) ; generating candidate solutions to such problems, deciding among the

alternative candidates, and executing actions and } or plans which are intended to

change the state of the environment to one that is acceptable.

We have several motivations for attempting to construct such a third-order agent :

(1) To contribute to the development of agent theory, within the tradition leading

from classical automata and planning systems to recent proposals for rational

or intelligent agents (Wooldridge and Jennings 1995).

(2) To experiment with a speci® c agent technology in order to evaluate its practical

advantages over current technologies. This work lies within the tradition of

knowledge-based systems for knowledge-rich domains, such as medicine.

(3) To explore ways of ensuring the safe operation of intelligent agents. This work

lies on the research line that starts with classical control systems, through

current robot technologies, to future advanced automata which, like the

® ctional robots of Isaac Asimov, are capable of autonomous ethical reasoning.

W e see no reason not to assume the existence of agents which are more complex than

third-order (e.g. in multi-agent systems agents may not only re¯ ect on their own beliefs

but also reason about the beliefs of others.) However, such agents are well beyond the

scope of the present paper.

3. Decisions and agents

W e believe that the diŒerent kinds of complexity outlined above are fundamentally

associated with an agent’ s abilities to address uncertainty about its environment and

to make adaptive decisions in the face of that uncertainty.

A decision is usually viewed as a choice between two or more competing hypotheses

about some world or possible courses of action in the world. For an autonomous agent

to make decisions, however, it must be able to do much more than make choices,

including (1) recognizing that a decision is required, (2) proposing alternative decision

options, (3) identifying relevant information sources (which will help choose among

the options), (4) formulating arguments for and against the options in the light of

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


available information, (5) choosing one of the options on the basis of these arguments,

and (6) committing to the choice (by updating the current belief set or implementing

the preferred action).

Two main traditions have in¯ uenced the design of practical decision systems in AI ;

statistical decision theory (e.g. expected utility theory) and knowledge-based (expert)

systems.

Expected-utility theory (EUT) is the longest established approach. EUT models a

decision process in terms of numerical probabilities (of hypothetical states of the

world, and of evidence conditioned on these states) and numerical utilities (which

measure the desirability of the states which may result from alternative actions). EUT

is well understood and, under certain assumptions, guarantees that a selected action

will maximize expected bene® t. Unfortunately EUT provides no guidance on when a

decision is needed, or how to identify candidate options (hypotheses or actions) or

relevant sources of evidence. Since there is no theory for these steps they are left for a

(human) decision analyst or software designer to de® ne explicitly or provide a method

for generating options by means of some other theory. In short EUT alone provides

a very impoverished knowledge representation, and does not currently provide a basis

for the design of autonomous agents other than those in which all possible events and

alternative responses are precoded into the agent implementation.

Knowledge-based systems (KBSs) introduced the use of symbolic inference (e.g.

predicate logic) which has the representational power to capture a wide variety

of specialized reasoning methods. Research on KBSs has shown how evidential

reasoning, including probabilistic reasoning, can be combined with methods for

reasoning about causality, time, space, physical structure (e.g. anatomical structure in

medicine), function (e.g. physiological functions) and many other `commonsense ’

ideas which cannot be explicitly captured with purely algebraic formalisms in general

and classical decision formalisms in particular. KBS techniques can also be used to

implement methods for reasoning about the relevance of knowledge and control of its

application. These oŒer more ¯ exibility (e.g. for hypothetical reasoning and action

management) than conventional quantitative algorithms.

Early knowledge-based systems, however, were rather ad hoc and formal theories of

knowledge representation remain somewhat immature. This is a matter of concern

both for developing AI theory and practice (e.g. if KBS are to be deployed in safety

critical applications). The situation is improving rapidly due to the increasing

availability of specialised reasoning formalisms (see Davis 1992 for a discussion of

formalization of common sense concepts and Krause and Clark 1993 for a review of

quantitative and logical approaches to hypothetical reasoning). Nevertheless, even

modern KBSs are not entirely satisfactory in that they lack a well-de ® ned framework

for dealing with all aspects of decision making, and hence a basis for establishing their

soundness.

A related area of weakness for the design of complex agents is in their re¯ ective

abilities. It is well known that logic-based languages and technologies have

considerable power for meta-level reasoning, but systematic development in this area

remains somewhat lacking.

In earlier work we have therefore developed an explicit framework to address these

problems. `Symbolic decision theory ’ incorporates a set of inference processes that,

together, implement a complete problem solving and decision cycle (Das et al. 1996,

Fox and Das 1996). Symbolic decision theory is formalized using classical predicate

logic and a number of non-classical logics for specialized forms of inference which are

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


required (e.g. for hypothesis generation, argumentation about decision options, and

commitment (Fox 1991, Fox and Krause 1992)). Other authors have proposed

classical and non-classical logics for building decision support systems (Bonczek et al.

1981, Chen and Henschen 1985, Bhargava and Kimbrough 1994), but we have placed

greater emphasis in our work on developing an explicit theory of decision-making

under uncertainty, as summarized in the above decision cycle.

In this paper we show how important commonsense concepts such as beliefs, goals,

intentions and constraints can be introduced in a theoretically sound way into a

decision procedure, and how this helps to address practical issues in AI systems, such

as those which must arise when deploying intelligent agents in unpredictable and

threatening environments. Our proposals are illustrated with medical examples,

notably for diagnosis decisions (hypothetical reasoning) and treatment decisions

(practical reasoning about actions and plans).

3.1 The domino agent

Our interest is in formalizing the reasoning and decision making required to support

decision making of the kind illustrated in the medical example above. Figure 2 presents

our framework informally, the `domino ’ agent model. Each node in the domino

represents part of a database state and each arrow is a class of inference mechanisms

which update database states. Inference procedures are formalized by an appropriate

axiomatization, or logic.

An implementation of the domino model operates as follows. First it maintains a

database of beliefs about a particular environment ; in the medical context this

includes data about a clinical situation including patient data. Certain beliefs (e.g.

unexplained weight loss) cause the system to raise goals (e.g. to explain the abnormal

weight loss). Such goals lead to problem solving to ® nd candidate solutions (e.g.

weight loss may be caused by cancer or peptic ulcer) and arguments are constructed for

and against the candidates, by instantiating general argument schemas on the

situation model (patient data) and general domain knowledge (medical knowledge).

As additional data are required a point may arise where an assessment of the

arguments for and against the various candidates permits the system to commit to a

most probable hypothesis (e.g. cancer). This is accepted as a new belief which, while

the belief is held, guides further problem solving and action.

Since the new belief concerns an undesirable ± indeed life threatening ± condition

Figure 2. The domino model for decision making.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


another goal is raised, to decide on the best therapy for the patient. This initiates a

further cycle of reasoning, summarized in the left half of the ® gure. As before,

candidate decision options are proposed (surgery, chemotherapy, etc.) and arguments

are generated for and against the alternatives. In due course a commitment may be

made to a single therapy (e.g. chemotherapy).

Many clinical therapies, such as chemotherapy, are in fact complex procedures

executed over time. Such therapies can usually be modelled as hierarchical plans that

decompose into atomic actions (e.g. administer a drug) and subplans (e.g. take

baseline measurements, administer several cycles of therapy, and then follow up the

patient for a period after the last cycle of treatment). Our framework acknowledges

this by providing ways of representing plans and specifying the control processes

required during plan execution. In particular the atomic actions of a plan must be

scheduled with respect to any other actions which have been previously scheduled as

a result of problem solving or decision processes required for achieving the goal, or

other goals raised by the agent (Fox et al. 1994). Note that the present work is not

concerned with the process of constructing plans, but only with making choices

between alternative prepared plans, and carrying them out.

It is particularly important in the present context that an agent knowledges that (1)

actions may have consequences beyond those implicit in its original goal (chemo-

therapeutic agents can have toxic side-eŒects, for example), (2) actions scheduled for

diŒerent purposes may compete or interact in undesirable ways (e.g. drugs for diŒerent

conditions may interact), (3) plans and actions are executed in a changing world, and

decisions may not remain valid, or something may go wrong during plan execution

(e.g. the patient’ s condition may deteriorate due to drug toxicity). Over and above the

basic requirements for decision making and action scheduling, therefore, is the

problem of continuously assessing the consequences of the agent’ s commitments. We

therefore require an explicit concept of a `hazard ’ and functions by which the agent

may forecast and avoid hazards that arise as a result of its commitments. The present

paper oŒers a formalisation of the above scheme, and some proposals for reasoning

about hazards.

4. The domino knowledge representation language : R2L

The R # L language is a knowledge representation language which explicitly incor-

porates the main concepts in the domino model. This is done by de® ning the language

around the two major constructs, namely the decision and the plan, and four

supporting constructs symbol, constraint, obligation and rule. Only the major

constructs are described in detail here ; a full technical de® nition of the language is

available on request.

4.1 Modelling the environment : properties and occurrences

R # L is built from two kinds of concept : static properties (e.g. the patient has a cold)

and dynamic occurrences (e.g. the patient is given chemotherapy) as described by Allen

(1984). An occurrence is either an event or a process. W e consider the set of all R # L

symbols sorted into properties (for example, weight j loss) and actions (for

example, injection ). The agent can execute atomic actions or composite plans

(themselves composed of actions and subplans). The underlying language of R # L is

temporal, in which the representation of time is interval-based. A closed interval is of

the form [t1, t2], where t1 and t2 are numerical values representing time points

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


such as hour(5), minute(40) , etc. The timestamp associated with a property or

action in a knowledge base may be represented as a temporal property.

holds([t1, t2], © property ª )

or a temporal action

execute([t1, t2], © action ª )

A belief concerns a temporal property or an action. The expression holds([t, t2],

© property ª ) means that the property starts to hold on or before t1 and ceases to

hold on or after t2. The expression execute([t1, t2], © action ª ) means that

the action is executed from t1 until t2 and therefore its duration is t2-t1.

An interval may have three relative positions with respect to the current time point ;

[t1, t2] may be completely in the past ; the current time point may lie in the interval

or the interval may be completely in the future. Consequently holds([t1, t2],

© property ª ) may be interpreted as © property ª which held in the past, holds now,

or is expected to hold in the future, or a combination of these. Similarly the formula

execute([t1, t2], © action ª ) can be interpreted as © action ª has been taken,

is in the process of being taken, or is intended to be taken.

If we say that someone had a cold last year then we do not mean that the patient had

a cold throughout last year but for only part of the year. Similarly, if we say that a

patient had an injection yesterday we do not mean that the injection took the whole of

yesterday to complete. For ¯ exibility, therefore, R # L represents open intervals in the

form (t1, t2). The expression holds((t1, t2), © property ª ) means that

© property ª started to hold on or after t1 and will cease to hold on or before t2.

Similarly, execute((t1, t2), © action ª ) means that the action is executed from

or after t1 until sometime before t2. In this case the duration of the action is less than

or equal to t2-t1.

As a concrete illustration of the use of temporal information in representing

properties and actions consider the important concept of a hazard , such as death

resulting from the inappropriate use of toxic chemotherapeutic agents. This is a state

which might result (in the future) from a planned action (also in the future). It is

important to be able to model such concepts explicitly since it may be desirable to take

future hazards into account in arguing for and against alternative therapies or, where

there are no alternatives, to raise an auxiliary goal to identify actions which may pre-

empt such hazards.

R # L provides the set of temporal meta-predicates past, next and previous to

specify properties and actions relative to the current time point :

past( © property ª ) : The speci® ed property held in the past, that is, holds([t1,

t2], © property ª ), where t2 is less than the current time point.

next([t1, t2], © property ª ) : The speci® ed property will hold for the interval

speci ® ed relative to the current time point t0, that is, for the absolute interval

[t0 t1, t0 t2].

previous([t1, t2], © property ª ) : The speci® ed property held for the interval

speci ® ed relative to the current time point t0, that is, for the absolute interval

[t0 ® t1, t0 ® t2].

W e also have similar operators for actions and open intervals.

first( © plan/action ª ) : Execute speci® ed plan } action ® rst.

start(T, © plan/action ª ) : Execute speci ® ed plan } action in T time units.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


after( © plan/action1 ª , © plan/action2 ª ) : As soon as the ® rst plan } action

is complete, start the second plan } action.

after( © plan/action ª , T, © plan/action ª ) : T units of time after the

completion of the ® rst plan } action, start the second plan } action.

repeat(every(T) , © plan/action ª , count(N)) : Schedule the speci® ed

plan } action now, then repeat it every T units of time for a total of N times.

repeat(every(T) , © plan/action ª , until( © condition ª )) : Schedule the

speci® ed plan } action now then repeat it every T units of time until the condition is

true.

on(N, © time j of j the j day ª , © plan/action ª ) : On the Nth mornings } even-

ings } nights } ¼ schedule the speci® ed plan } action.

4.2 Decisions

A decision schema has ® ve component parts :

E A situation : this describes, as a boolean expression on the database of beliefs, the

situation which will cause a goal to be raised (for example, a belief that an abnormality

is present).

E A goal, which is raised as soon as the evoking situation occurs (for example, a belief

that an abnormality is present may cause a goal to diagnose the abnormality to be

raised). A goal is removed if it becomes satis ® ed on the database of beliefs, i.e. the

decision has been made.

E One or more candidates ; a set of alternative decision options, such as cancer,

peptic j ulcer. In principle the set of candidates may be de® ned intentionally (by

® rst-order rules) or extensionally (as a set of facts), but we only consider the latter case

here.

E One or more argument schemas. These are rules which de ® ne the evidence or

preference-based arguments which are appropriate for choosing between candidates

for the class of decision.

E One or more commitment rules. These de® ne the conditions under which the

decision may be taken and may include quantitative and } or logical conditions on the

argument and belief databases.

Consider the following simpli ® ed example of a decision schema corresponding to

Figure 1 :

decision :: weight j loss j diagnosis

situation

weight j loss

goal

weight j loss j diagnosed

candidates

cancer ;

peptic j ulcer

arguments

elderly ¯ ª support(cancer, d1) ;

smoker ¯ ª support(cancer, d2) ;

positive j biopsy ¯ ª support(cancer, d3) ;

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


pain j in j upper j abdomen ¯ ª support(peptic j ulcer, d4) ;

young ¯ ª support( 4 cancer & 4peptic j ulcer, d2)

pain j after j meals ¯ ª support(cancer & peptic j ulcer, d5)

commits

netsupport(X, M) & netsupport (Y, N) & greater (M, N) ¯ ª add(X).

Where C is negation and d1 ¼ d5 are symbols representing the support given by an

argument to one of the candidates.

In classical logic an argument is a sequence of inferences leading to a conclusion.

The usual interest of the logician is in procedures by which arguments may be used to

establish the validity (truth or falsity) of a formula. In LA (a logic of argumentation,

Fox et al. 1992, Krause et al. 1995, a variant of intuitionistic logic which de® nes a set

of inference rules for constructing arguments) arguments do not necessarily prove

formulae but may merely indicate support for (or doubt about) them. Also in classical

logic, so long as we can construct one argument (proof) for F, any further arguments

for F are of no interest. In our system all distinct arguments of candidates are of

interest (intuitively, the more arguments we have for F the greater is our knowledge

about the validity of F). We therefore distinguish distinct arguments by identifying the

unique grounds of each (essentially a normalized proof term in LA, Krause et al. 1995)

and a sign drawn from some dictionary which indicates the support provided to F by

the argument.

An argument schema is like an ordinary inference rule with

support( © candidate ª , © sign ª )

as its consequent, where © sign ª is drawn from a dictionary of qualitative or

quantitative representations of support for the candidate (Fox et al. 1992, Krause et

al. 1995).

An example of an R # L argument is

elderly ¯ ª support(cancer, d1)

where F is cancer, the ground is elderly ¯ ª cancer and the support is d1.

The theory of argumentation and methods for aggregating arguments in diŒerent

representations of uncertainty is reviewed in Krause et al. (1995). An outline of the

concept of a dictionary and its use is given in Appendix A.

To simplify the presentation we shall restrict ourselves to the simple support

symbol in the examples below, meaning that the argument justi ® es a qualitative

increase in con ® dence in the decision candidate, without indicating the magnitude of

that increase.

A knowledge base for the domino agent consists of a set of de® nitions of this and

other kinds of task ; these are interpreted by a piece of software which will be referred

to from now on as the R # L engine (described later).

In this example the decision weight j loss j diagnosis will be considered for

activation by the R # L engine when a belief about weight j loss is added to the

database. W hen the engine detects this it checks whether any of the candidates has

already been committed. If not the decision will be activated and the goal

weight j loss j diagnosed is raised. Otherwise no action is taken. W hile the goal is

raised, further information about the situation (e.g. the patient’ s age, symptoms) can

be examined to determine whether the premises of any argument schemas are

instantiated.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


A commitment rule is like an ordinary rule with one of

add( © property ª } © temporal property ª )

schedule( © action ª } © temporal action ª )

as its consequent. The former adds a new belief to the knowledge base and the latter

causes an action to be scheduled as follows:

decision :: cancer j treatment

situation

cancer

goal

treatment j decided

candidates

chemotherapy ;

surgery

arguments

elderly ¯ ª support(chemoth erapy, ’ ’) ;

young ¯ ª support(surgery , ’ ’)

commits

netsupport(X, M) & netsupport(Y, N) & greater(M, N) ¯ ªschedule(X).

As arguments become valid the commitment rules are evaluated to determine

whether a decision is justi® ed. A commitment rule will often, though not necessarily,

make use of the netsupport meta-predicate. This aggregates collections of

arguments for and against any candidate to yield a measure of the overall strength of

the case for the candidate, permitting an ordering on the set of decision options. This

meta-predicate has the form :

netsupport( © candidate ª , © support ª )

It computes the support for the speci® ed candidate using an aggregation algorithm

(discussed in Appendix A) selected from a library of aggregation algorithms (Fox et al.

1992, Krause et al. 1995). The netsupport meta-predicate computes the support

for the speci® ed candidate by using the argument schemas speci® ed in arguments .

Not every decision (commitment) requires the netsupport meta-predicate.

Consider the situation in which we require an èager ’ commitment rule which will

immediately respond to any hazardous situation that occurs. A simple example of this

is in the management of acute asthma, as indicated in the following example :

Rescue courses of steroid tablets may be needed to control exacerbations of asthma ; indications

for such a course may include any of the following:

E symptoms and peak expiratory ¯ ow (PEF) are getting progressiv ely worse ;

E PEF falls below 60 ± sleep is disturbed by asthma ;

E symptoms persist ;

E there is a diminish ing response to inhaled broncho dilators;

E emergency use is made of nebulized or injected broncho dilators.

An R # L representation of this knowledge is as follows :

decision :: steroid j rescue j course

situation

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


high j asthma j exacerbation

goal

asthma j exacerbations j controlled

candidates

steroid j rescue j course

arguments

progressively j worse j symptoms & progressively j worse j pef ¯ ªsupport(steroid j rescue j course, ’ ’) ;

pef j below j 60 ¯ ª support(steroid j rescue j course, ’ ’) ;

disturbed j sleep ¯ ª support(steroid j rescue j course, ’ ’) ;

persistent j symptom ¯ ªsupport(steroid j rescue j course, ’ ’) ;

diminishing j bronch j response ¯ ªsupport(steroid j rescue j course, ’ ’) ;

emergency j nebulised j bronch ¯ ªsupport(steroid j rescue j course, ’ ’) ;

emergency j injected j bronch ¯ ªsupport(steroid j rescue j course, ’ ’)

commits

support(steroid j rescue j course, ’ ’) ¯ ªschedule(steroi d j rescue j course).

The decision rule in this example commits to scheduling a

steroid j rescue j course as soon as there is any argument for it.

4.3 Safe decision making

W hatever numerical calculus we adopt to compute relative support for decision

candidates the maximization of some quantitative measure has a weakness which is

particularly important when we consider safety critical applications. Suppose we rely

on a commitment rule that commits to an action if some threshold of con ® dence,

Theta, is exceeded. This quantitative approach to safety has a number of weaknesses.

For example a decision rule based solely on a numerically based preference ordering

is unstable, in that a few early items of information may lead to Theta being exceeded,

but if we wait and receive more information the con ® dence may be reduced below

Theta. Clearly we need a more stable decision rule which is less èager ’ and which takes

account of safety conditions, such as ònly commit to an action if there is no missing

information which, if it were available, would change the preference (possibly to a less

hazardous action) ’ . It would seem desirable in any domain, not just medicine, that an

agent should be able to re¯ ect upon the justi® cations for its preferences and

hypothetical states of argument and evidence in taking decisions which may be

hazardous.

An alternative approach might be to attempt to ensure that the agent is cautious

rather than eager by setting Theta, the threshold for action, to a high value.

Unfortunately this is a rather crude strategy because raising the uncertainty threshold

runs the risk of merely trading one kind of error (e.g. taking an unsafe action) for

another (e.g. delaying action too long or even not taking action). This is just an

instance of the problem of trading oŒfalse-positive and false-negative errors which is

well known throughout statistics.

In general we take the view that agent designers should adopt commitment rules

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


which use knowledge to explicitly model and reason about domain hazards as well as

(or possibly instead of) numerical preferences. Such safety rules may be domain

independent (e.g. even if a plan is the most preferred do not commit to it if it is

associated with a potential hazard when an alternative is available) or domain speci® c

(e.g. always refer a patient for specialist investigation if there is any candidate

diagnosis which is life-threatening and its NetSupport is " 0).

In fact it is possible to capture the necessary knowledge-based principles for

reasoning about potentially hazardous consequences of commitments in a very

general way. For example, the following rule re¯ ects upon a set of candidate actions

to reason explicitly about their possible consequences :

Action1 should not be performed during Action2 in Plan IF

Action2 is necessary part of Plan AND

Action2 produces Effect AND

Effect is potentially hazardous AND

Action1 aggravates or makes Effect more likely AND

Action1 has an alternative without Effect

This is a safety principle which was abstracted by a careful analysis of about 50 cancer

treatment protocols (Hammond et al. 1994). Nine safety principles of this kind which

can be evaluated during the commitment step were identi ® ed. It is interesting to note

that although the principles were abstracted from analysis of procedures in medicine

they appear to capture general commonsense safety principles which are applicable to

any domain.

4.4 Plans

A plan schema has four components :

E pre-conditions which must be true before the agent may commit to executing the

plan ;

E subtasks which specify the sub-plans and atomic actions from which this plan is

built ;

E scheduling j constraints which specify the ordering relation which hold between the

subtasks ;

E abort j conditions which specify any conditions which, if true, will cause the plan to

be aborted (and may schedule a new plan).

An example of a simple plan in R # L can be found in the basic structure of many

cancer chemotherapy plans :

plan : : chemotherapy

pre j conditions

cancer

subtasks

baseline ;

chemo

followup

scheduling j constraints

first(baseline) ;

after(baseline, chemo) ;

after(chemo), day(6), followup)

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


abort j conditions

patient j deteriorates ¯ ª abort(chemother apy) &

schedule(intens ive j care).

The plan chemotherapy consists of three subtasks : baseline, chemo and

followup . The scheduling constraint first(baseline) requires that baseline

is carried out ® rst, before any other tasks that are speci ® ed in the plan. The scheduling

constraint after(baseline, chemo) speci® es that chemo is to be performed

immediately after baseline while the ® nal scheduling constraint speci ® es that

followup is to be performed six days after chemo (a variety of scheduling

constraints in R # L such as repeat and on are listed in Section 4.1). The only abort

condition says that if the patient deteriorates then abort the whole of chemotherapy

and schedule another plan, intensive j care.

5. Safety reasoning

In software engineering, safety has been traditionally approached by means of

informal procedures for identifying and avoiding potential operational hazards at the

design stage. Formal methods for specifying and verifying safety critical components

of software are also increasingly advocated (Leveson 1986). To build knowledge-

based systems for safety critical decision making and planning, however, additional

techniques are needed because the formal integrity of software and knowledge bases

may not be su� cient to ensure that the advice given or actions recommended in

complex situations will always be appropriate. This section addresses this di� culty by

proposing that agents should be able to explicitly anticipate possible hazards and

manage them at runtime (Fox 1993, Hammond et al. 1994).

The safety of a decision support system (Das 1995a) is the property that any actions

recommended by the system will have minimal undesirable consequences. Such

consequences may result from recommendations which have arisen in any of the

following situations:

Group I (a) Hardware failure.

(b) Human error in the context of operating and maintenance, inst-

allation.

Group II (a) Incorrect design and speci® cation of the system.

(b) Incorrect implementation or one that diŒers from the actual

speci® cation.

Group III (a) Inconsistency, redundancy, inaccuracy or incompleteness of the

knowledge base.

(b) Incorrect update of the knowledge base.

(c) Lack of appropriate integrity and safety constraints imposed on the

system.

Ignoring hardware failure and human error issues (group I) for the moment we wish

to make a distinction between intrinsic safety and consequential or dynamic safety.

Intrinsic safety (group II) has traditionally been pursued by means of a rigorous and

sometimes formal approach to the design, testing and maintenance of system software.

The core functions such as the R # L agent software, are entirely algorithmic and

therefore demand similar good practice.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


Knowledge-based systems raise additional issues (group III). A medical decision

support system, for example, may use knowledge of diseases and treatments in order

to make diagnoses and select drugs, but knowledge is frequently heuristic : it may be

incomplete and uncertain, and the limits on its applicabilitymay be poorly understood.

M ethods for ensuring that knowledge bases are consistent, non-redundant, etc. are

still immature and while a knowledge base may contain no demonstrable errors or

inconsistencies, and the programs which apply it are intrinsically safe the advice given

may entail hazards. W e propose to address this problem by introducing dynamic safety

conditions, a form of integrity constraint, into the agent’ s knowledge base to permit it

to reason explicitly about hazards and, where possible, to modify recommendations in

the light of hazard forecasts.

Conventional integrity constraints may be de ® ned as properties which a knowledge

base is required to satisfy (Das 1992). For example, a person’ s age must be less than

or equal to 150. Such constraints help to avoid inconsistent, incomplete and inaccurate

knowledge bases and address problems in Group III, (a) and (b). Safety constraints on

the other hand are constraints on the behaviour of the system. W e model such

constraints here in terms of the deontic concepts of obligation and permission

(Wieringa 1989) on system actions. For example, in the context of chemotherapy, a

typical safety requirement is Ìt is obligatory that Cisplatin is stopped if anaphylaxis

(severe allergic reaction) occurs ’ .

5.1 Update constraints

W henever an attempt is made to add a property to the knowledge base, or to change

the status of a plan } action, the system must ensure that no constraint is violated. A

constraint declaration consists of a constraint speci ® cation and an action to be carried

out if the constraint is violated.

constraint : : after j baseline j chemo

specification

chemo ¯ ª past(baseline)

action

warning(’chemot herapy may not be done until after baseline ’).

This constraint speci ® es that baseline must have been completed before chemo

can be performed. The system will not permit the chemo task to be started unless

baseline has already been entered and the start time of chemo is after the

completion time of baseline . Any attempt to do this will cause the message to be

displayed. In principle the violation of a constraint could evoke any procedure ; we

restrict the discussion to the delivery of warnings to simplify the presentation.

5.2 Obligations

An obligation (Aqvist 1985, Chellas 1980, von Wright 1951) declaration consists of a

speci® cation in the form of a rule whose consequent is labelled with the meta-predicate

obligation and an action which is to be taken when an obligation arises :

obligation : : toxicity j monitoring j after j methotrexate

specification

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


methotrexate ¯ ªobligation(next ([hour(24),hour (48)], toxicity j monitoring))

action

warning(’toxici ty monitoring should follow methotrexate ’).

where methotrexate and toxicity j monitoring are events which involve

actions taken by clinicians and other medical staŒ. The above construct speci ® es that

if the drug methotrexate is given it is obligatory that toxicity monitoring is executed

during the interval between 24 and 48 hours following administration. If toxicity

monitoring is not started when it should be the system gives a warning as this is a

hazardous situation (though, as with constraints, other procedures could be executed).

6. Implementation of the domino agent

The current implementation of the domino agent uses a Prolog theorem prover for a

designated subset of LR # L which contains the translation (see Appendix B) of any R # L

speci® cation. W ith this prototype we have been able to demonstrate medical

applications dealing with the management of testicular cancer, diagnosis and

management of acute depression, and management of acute asthma. This suggests

that R # L has considerable expressiveness.

The domino agent acts as a decision maker, corresponding to the left side of the

`domino ’ , and as a plan manager or scheduler corresponding to the right side. The

agent monitors a database which can store information about properties and actions

representing any R # L construct (beliefs, goals, candidates, plans, etc.).

Initially the domino agent is quiescent. If new information is added, specifying a

temporal property or an action to be executed for a speci® ed time interval then

constraint checking is invoked. If an integrity constraint is violated, the new

information is rejected and appropriate warning messages generated. Otherwise, the

new information is added to the knowledge base and a cycle of the agent’ s engine is

executed. A cycle of the engine is also executed when time moves on by a unit (the size

of the unit is application-dependent).

When new information is supplied to the agent, there is no restriction on the times

speci® ed in the timestamp. This means that it is possible to say that a property held for

a particular period of time in the past, holds for some time interval around now, or will

hold during some speci® ed interval in the future. Similarly for actions, though in this

case there is additional meaning represented by control states which are associated

with actions (see below).

6.1 Decision making by the domino agent

W henever the knowledge base state changes, or time moves on, the agent will check

whether conditions which indicate an initiating situation for any decision schema in its

knowledge base is satis® ed. If so then the appropriate goal is raised (the decision

procedure becomes active). Next, all commitment rules of active decision processes are

evaluated. Rules are used by the engine in a backward-chaining mode only, i.e. when

the agent needs to evaluate the rule’ s consequent. If a commitment rule is satis ® ed then

the appropriate knowledge base update is attempted. This is handled in the same way

as when new information is supplied by an external agent, i.e. constraint checking is

invoked and, provided no constraint is violated, the commitment is made (the

knowledge base is updated) and the decision maker is called.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


The handling of situation and goal within a decision context is relatively

straightforward. As soon as the triggering situation for a decision is found to be true,

a check is made to see whether one of its candidate options already holds. If not, the

goal is raised, the decision becomes active, and the agent attempts to commit to one

decision candidate. It does this by attempting to ® nd a commitment rule whose

antecedent is satis ® ed so that the consequent can be performed, i.e. adding a new

property to the knowledge base or scheduling an action. Once a commitment to a

candidate is made, the goal is satis ® ed and this decision process ceases to be active.

6.2 Scheduling by the domino agent

W henever the state of the knowledge base changes, or time moves on, the status of its

intentions (plans } actions) may change. In particular once a plan or action has been

scheduled it can be started as soon as its pre-conditions are satis® ed. Then, over time,

it passes through a number of control states. The full set of control states is as follows:

E scheduled : ready to go but not time yet ;

E planned : the plan } action should start execution at a later time point;

E active : the plan } action is being executed ;

E pending : it is time to be active but not started yet ;

E hazard : it is obligatory that the plan } action is active but not started yet ;

E complete : normal completion ;

E aborted : the plan } action was active but aborted.

This particular control scheme has proved adequate for the scheduling required by

the medical applications we have looked at, though a more complex control scheme

could be introduced. M ore complex schemes have been suggested (e.g. for executing

general medical therapy plans (Herbert 1995) although it is not currently clear what

the general requirements of a more elaborate scheme are).

Currently the default behaviour of the agent’ s scheduling functions is only to

recommend that pre-conditions should be brought about or particular plan } actions be

performed at particular times. Such recommendations could be strengthened by

means of the obligation construct, or by the introduction of integrity or safety

constraints.

Constraints are checked by the engine whenever properties or actions are added to

or deleted from the knowledge base. This will typically occur when an external agent

(e.g. the user) requests the agent to update its knowledge base, or a commitment rule

attempts to add a property or schedule a plan } action. If a safety constraint would be

violated by the update, the knowledge base transaction is not carried out and the

message is displayed.

7. Summary and discussion

The domino agent combines a number of ideas in contemporary AI (reactive agents,

knowledge-based systems, non-classical logics, scheduling and planning) with a

number of less well known concepts (symbolic decision procedures, a logic of

argumentation, explicit safety management, etc.). Together these yield an apparently

powerful and ¯ exible architecture.

R # L is a specialized knowledge representation language in the AI tradition which

oŒers considerable representational power and versatility. Its well-de® ned semantics

provides con ® dence that applications making use of the unorthodox decision

procedure and other functions of the domino model are theoretically sound.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


The principal reasons for developing the symbolic approach to decision making are

that classical decision theory, while well understood, addresses a restricted range of the

functions required for intelligent systems, while knowledge based systems are versatile

but rather ad hoc and poorly understood.

However, soundness is a necessary but not su� cient condition for safe decision

making (Fox 1993). Even if an R # L decision engine is properly speci ® ed, formally

veri ® ed and correctly implemented it may still give advice which is wrong and even

unsafe. Consequently the basic domino structure and R # L formalism incorporate

constructs for explicitly representing safety knowledge, namely obligations, con-

straints and commitment rules.

7.1 Autonomous agents and decision support systems

M ost of our applications work has emphasized the use of the domino agent in a

decision support role, to assist doctors or other medical staŒin the management of

patients. Given the present state of the art in intelligent agent design we would not

expect doctors or patients to trust such an unproven technology, so none of our

applications actually take decisions (make commitments). On the contrary we require

that all commitments, whether to diagnostic hypotheses or therapeutic or other

clinical actions, are made by human users. The domino agent has therefore only been

used in a support mode rather than an autonomous mode to date. However, the

domino agent is designed for use in an autonomous mode, and we believe that this may

be simply achieved by introducing commitment rules which do not require con-

sultation with an external user or agent.

Throughout the development it has been our long-term aim to provide a framework

for decision making by autonomous systems, such as robotic systems which can

autonomously sense and act upon their environments as well as solve problems and

take decisions. There are many potential medical applications for such a technology.

If real-time perceptual-motor functions could be integrated with the functions

described here for recognizing and solving problems, and making decisions under

uncertainty without recourse to instructions or guidance from users or other external

agents, this would have considerable promise for constructing technologies for use in

high demand or otherwise challenging situations (e.g. intensive care systems,

autonomous vehicles for use in hostile environments).

We have not, of course, integrated the kinds of sensors or eŒectors, and we do not

assume that provision of such mechanisms would be easy. However, technological

systems for sensing, interpreting and acting upon their environments are being

developed in AI and other ® elds, and the framework presented here seems to oŒer

scope for interfacing such devices in a localized way at speci® c points in the domino

cycles. Recalling Figure 2 the outputs of perceptual systems could directly update the

belief database, thereby evoking problem solving, decision making and planning,

without needing to interact (or interfere) with other processes. This possibility has

been explored in work on the design of a workstation for radiologists. The workstation

provides support for clinical decision making and therapy plan management as

described here. In addition, however, certain data acquisition operations are

implemented with automated image analysis operators which extract information

from the image and feed this into the decision making process (Taylor 1995).

Autonomous robotic systems are a major goal of arti® cial intelligence research. We

believe that our main contributions to this goal are to (1) propose an intuitive and

formal framework within which a number of intelligent problem solving and plan

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


execution functions can be integrated ; (2) show how the framework can incorporate

functions to make decisions under uncertainty in a principled way, and (3) demonstrate

ways in which an agent-based on the framework can detect hazards and reason about

the safety of its actions.

7.2 Agent theory

As an agent theory the domino model is similar to the BDI agent concept (Rao and

GeorgeŒ1991) in that it shares mentalistic ideas (Shoham 1993) like beliefs, desires

(goals), and intentions (plans). The main theoretical extensions to agent theory are the

ability to reason about, and make decisions under, uncertainty. Generally, goals will

require hypothetical and } or practical reasoning but many problems may be solved in

more than one way. To deal with this we have introduced the idea that agents may

construct logical arguments for and against diŒerent candidate solutions. Decision

taking may involve the aggregation of arguments to induce a preference ordering over

the candidates, committing to one or other of the options based on these preferences,

or on other decision policies which oŒer diŒerent tradeoŒs for the agent.

The domino has features of ® rst, second and third-order agents, some of which are

the following:

First-order. The domino agent maintains an explicit set of beliefs about the environment, and an

explicit set of intentions to carry out actions on that environment. The agent can therefore

respond to queries and } or revise its plans by re¯ ecting on and modifying these explicit data

structures.

Second-o rder. The symbolic decision procedure provides a method for the creation of explicit

decision alternatives, or candidates. Candidates may be alternative beliefs about the world or

alternative actions on the world. The explicit representation of candidates perm its the argu-

mentation mechanism to search for reasons pro and con the alternative candidates under the

control of these knowledge structure s, and aggregate arguments to establish preference order-

ing on the alternatives.

Third-ord er. Beliefs are recorded and intentions (plans and actions) are scheduled along with

their justifying argum ents. Intentions are also recorded with the goals which lead to them, hence

permitting the agent to re¯ ect upon whether a completed action or plan has achieved the goal

and, if not, initiate some alternative action. The agent is also able to re¯ ect upon whether events

or actions represent threats to its goals and, if so, react appropri ately.

We believe that this sort of analysis provides a useful basis for comparing alternative

agent designs and assessing their likely properties. However, we also believe that such

an analysis needs to be formalized to achieve a full understanding of the concepts,

which we hope will be the focus of future work.

7.3 Multi-agent systems

A subject of widespread interest in AI is the use of agent systems as elements in

distributed (multi-agent) systems. Agents in distributed systems may include human

users (e.g. of a decision support workstation) or external programs, such as

communication links to other computer systems, other R # L agents etc. The use of

agents to implement distributed systems is particularly interesting, although chal-

lenging, as it may provide a valuable technology for important classes of application,

such as providing support for coordination and communication between diŒerent

functions and diŒerent sites.

We have explored ways in which agents based on our approach can be used to

support distributed patient management in multidisciplinary medical care. For

example the management of cancer patients may involve not only specialists in

radiotherapy, chemotherapy, etc., but also the patient’ s own doctor, nurses visiting the

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


patient’ s home and so on. Each member of such a team can be supported by an agent

with the capabilities described here, augmented with functions for communication of

beliefs and decisions and coordination of intentions, etc. In the same way that a user

can query, update or issue commands to an R # L agent, other agents can communicate

messages (e.g. by electronic mail) in order to request or provide information or issue

commands. Elsewhere we have proposed a set of primitive message schemas for

composing such inter-agent transactions, and for negotiating and revising com-

mitments between agents (e.g. where requests or commands cannot be satis ® ed due to

resource, timing or other limitations) (Huang et al. 1994). In this work a network of

autonomous agents simulating a doctor and a number of nurses sharing responsibility

for a patient and the tasks required in managing chemotherapy for breast cancer has

been successfully demonstrated.

7.4 Applications

It is one thing to propose a theory for an intelligent agent, and another to demonstrate

that it oŒers a viable architecture. For our purposes we believe that we have shown

viability in two ways. First, by providing a su� ciently formal description of the

underlying theory that others can analyse its properties and implement experimental

test versions of it. Second, by demonstrating practical applications which have been

implemented with it.

The present paper attempts to meet the ® rst objective by providing intuitive and

semiformal descriptions of the domino model and its main elements (in the body of the

paper above), and a formal syntax and semantics of the model (in the appendices).

In addition we have described a variety of practical medical applications elsewhere.

W e believe these oŒer prima facie evidence that the architecture can be used to address

non-trivial practical problems. For example we have described applications to drug

prescribing, and several systems for diagnosis and management of a wide range of

medical problems, including acute asthma, depression, otalgia, and various cancers.

The prescribing system was awarded the Perry Prize for technology innovation by the

British Computer Society’ s Primary Care Specialist Group.

However, we should acknowledge that these applications only represent a limited

evaluation of the architecture, because the full capabilities of the model are not

currently utilized. Speci® cally, none of the applications built so far take responsibility

for actually committing to clinical decisions. In principle they could take decisions

without human intervention but since this would not be acceptable to physicians the

responsibility for this step is reserved for the user. W ithin this limitation on the

autonomy of the applications built to date we believe that our examples provide good

evidence of the practical viability of the model. An overview of the medical applications

demonstrated so far can be found in Fox and Das (1996).

8. Conclusions

W e have presented a general framework for the design of knowledge-based agents

which is intended to incorporate a more comprehensive account of decision making

and action execution than other agent designs, and avoids the ad hoc features of many

knowledge-based systems. A formal foundation for such systems is particularly crucial

if they are to be used for safety critical applications. We believe that R # L (or more

strictly LR # L) satis ® es this requirement, but further support is also provided for

explicit reasoning about hazards by means of appropriately specialised constructs,

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


notably obligations, constraints and commitment rules. The domino engine imple-

ments an interpreter } theorem prover for the full R # L language. A number of medical

application knowledge bases have been speci® ed in the language and successfully

demonstrated on the engine.

Appendix A. Argumentation as a framework for uncertainty management

A dictionary is a set of symbols which can be used to label a proposition. In general,

a dictionary will be a semi-lattice with the partial-order relation % . For simplicity, we

shall consider a dictionary as a chain with one distinguished element ^ known

as the top element. Let d1 be an element from some dictionary. Then the argument

elderly ¯ ª support(cancer, d1) speci ® es that if a person is believed to be

elderly then this argument confers evidence level d1 on the candidate cancer . We

might consider d1 as a member of the quantitative dictionary of probabilities:

dict(Prob) ¯def

[0, 1]

However, there is no requirement that we should restrict dictionaries to dict(Prob).

Among the obvious dictionaries we may consider is the following:

dict(Qual ) ¯def

² , ´

The meaning of propositions associated with labels from the above dictionary are as

follows:

support(F, ) : an argument for F

support(F, ) : an argument that F is de® nitely true

where F is an arbitrary R # L sentence. As mentioned, a dictionary has always a top

element to represent the highest support for arguments. For example, elements and 1 are the top elements of the two dictionaries dict(Qual ) and dict(Prob),

respectively. A number of diŒerent dictionaries for reasoning under uncertainty have

been discussed in Fox et al. (1992) and Krause et al. (1995), together with their

mathematical foundations and their relation to classical probability and other

uncertainty formalisms.

Suppose a decision maker has a set of arguments for and against a set of mutually

exclusive decision options (`candidates ’ , that is, alternative beliefs or plans under

consideration) whose signs are drawn from a single dictionary. The decision maker can

aggregate these arguments to yield a sign representing the decision maker’ s overall

con® dence in each of the candidates. Every dictionary has a characteristic aggregation

function for aggregating arguments. Consider the argument presented above and the

following one :

positive j biopsy ¯ ª support(cancer, d3)

Considering the dictionary as dict(Prob), the two arguments can be aggregated by

using a special case of Dempster’ s epistemic probability (Krause and Clark 1993)

giving the value d1 d3 ® d1 ¬ d3. This formula can be generalized incrementally if

there are more than two arguments for the candidate cancer.

In general, suppose a decision maker has a set of arguments for and against a set of

mutually exclusive decision options, C (candidates, that is, alternative beliefs or plans

under consideration) whose signs are drawn from a single dictionary D . The decision

maker can aggregate these arguments to yield a sign drawn from D « which represents

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


the decision maker’ s overall con ® dence in each C . The general form of an aggregation

function is as follows :! : P (C ¬ G ¬ D ) U C ¬ D «

where P stands for `power set ’ and G is the set of all grounds. The simple

netsupport predicate in R # L implements the function ! . If D is dict(Qual ) then D «is the set of non-negative integers whereas D « is D itself when we consider dict(Prob)

as D . In the former case, ! assigns an aggregation number to each decision option,

giving a total preference ordering over the options. This suggests a simple rule for

taking a decision ; choose the alternative which maximises this value.

If we allow both F and 4F to occur in the support then by applying our usual

aggregation algorithm we compute total evidence for F (say, d1) and 4F (say, d2)

separately. If we have used the dictionary ² , ´ then we have the following four

cases :

d1 " d2 d1 ® d2 is the total evidence for F

d2 " d1 d2 ® d1 is the total evidence for 4 Fd1 ¯ d2 dilemma

d1 ¯ d2 ¯ ¢ inconsistency

If we have used the dictionary dict(Prob) then we have the following cases :

d1(1 ® d2)

1 ® d1 ¬ d2total evidence for F

d2(1 ® d1)

1 ® d1 ¬ d2total evidence for 4 F

d1 ¯ d2 dilemma

d1 ¬ d2 ¯ 1 inconsistency

Appendix B. Logic of the representational language LR 2L

The logic LR # L is essentially temporal propositional logic extended with a few modal

operators. The approach taken in LR # L towards reasoning about time (Das and

Hammond, 1996) is interval-based (Allen 1984, Humberstone 1979, Halpern and

Shoham 1991), ® rst-order (Haugh 1987, Shoham 1987, M cDermott 1982), discrete

and linear (Emerson and Halpern 1986, van Benthem 1991) in nature. Intervals are

represented by their boundary time points. All properties and actions in a knowledge

base have a time interval associated with them. Although the underlying language of

LR # L is temporal propositional we allow temporal quanti ® cations and arithmetic

expressions, for manipulation of temporal intervals.

Metalevel reasoning is involved in the R # L translation of LR # L for computing the

netsupport predicate through argumentation and aggregation. M eta-level reason-

ing is also required to ensure that the constraints satisfy the knowledge base, that is,

each constraint is a theorem of the knowledge base. W e have employed an extension

of the path ® nding method (Das and W illiams 1989) to achieve this.

Syntax of LR # L

Suppose 0 is the set of all propositions, divided into properties and actions and

includes the special property symbol w (true). The set of integers represents constants

or time points in the logic, and we have the usual arithmetic function symbols.

Ordering and equality relations are added to the logic as two special predicate

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


symbols. Suppose D is an arbitrary dictionary with the top element ^ . The modal

operators of LR # L corresponding to belief (Fagin and Halpern 1988, Hintikka 1962)

and goal (Cohen and Levesque 1990) are © bel ª and © goal ª respectively. In addition,

for each dictionary symbol d ` D , we have a modal operator © supdª for support. The

formulae (or assertions) of LR # L are as follows:

The domain of propositional formulae is extended to the domain of formulae as

follows:

E temporal propositions p(t", t

#) and p[t

", t

#] are formulae ;

E arithmetic expressions involving time points are formulae ;

E © bel ª F and © goal ª F are formulae, where F is a formula ;

E © supdª F is a formula, where F is a formula and d is in the dictionary D ;

E | F, F g G and c tF are formulae, where F and G are formulae.

W e take v (false) to be an abbreviation of | w . Other logical connectives and the

existential quanti ® er are de ® ned using ` | ’ , ` g ’ and ` c ’ in the usual manner. In the rest

of the paper we consider only closed LR # L formulae with respect to temporal

variables.

Axioms of LR # L

The axioms of LR # L are divided into classical, temporal (Das and Hammond, 1966)

and modal (Das et al. 1996).

Classical axioms

W e consider every instance of a propositional tautology to be an axiom. Those

instances of propositional tautologies can be constructed using temporal propositions

and may involve any number of modal operators, for example, © bel ª p(i", i

#) U

© bel ª p(i", i

#). W e also add the necessary axioms of the equality and ordering relations.

W e have the modus ponens inference rule and substitution rule, but do not require the

generalisation rule of inference since we are only considering closed formulae.

Temporal axioms of LR # L

If an action is taken in an interval then it is also taken in any interval containing the

interval. Similarly, if a property holds in an open interval then it also holds in any

interval containing the interval. Thus we have the following axiom (p is an action or

a property) :

c t"

c t#( p(t

", t

#) g t

$% t

"g t

"% t

#g t

#% t

%U p(t

$, t

%)) (1)

If a property u holds in a closed interval then it also holds in each of its closed

subintervals. In other words, every subinterval of a closed interval is closed. This is

axiomatized as follows:

c t"

c t#( u [t

", t

#] g t

"% t

$g t

$% t

%g t

%% t

#U u [t

$, t

%]) (2)

Conversely, if a property holds in two adjacent closed intervals then they can be

concatenated to obtain another closed interval in which the property holds. This is

expressed through the following axiom :

c ( u [t", t

$] g u [t

$, t

#] U u [t

", t

#]) (3)

A closed interval is also an open interval. This yields the following axiom :

c t"

c t#( p[t

", t

#] U p(t

", t

#)) (4)

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


M odal axioms of LR # L

W e adopt a set of standard axioms of beliefs which can be found in Cohen and

Levesque (1990), Fagin and Halpern (1988), Halpern and M oses (1985), and M eyer et

al. (1991).

| © bel ª v (5)

© bel ª F g © bel ª (F U G) U © bel ª G (6)

© bel ª F U © bel ª © bel ª F (7)

| © bel ª F U © bel ª | © bel ª F (8)

Axiom (5) expresses that an inconsistency is not believable by a decision maker. The

derivation of the symbol v from the database implies inconsistency. Axiom (6) states

that a decision maker believes all the logical consequences of its beliefs, that is, a

decision maker’ s beliefs are closed under logical deduction. The two facts that a

decision maker believes that s } he believes in something and a decision maker believes

that s } he does not believe in something are expressed by axioms (7) and (8),

respectively. We also have the rule of necessitation for beliefs :

if { F then { © bel ª F (9)

There is no support for an inconsistency and the following axiom re¯ ects this

property :

| © supd ª v , for every d ` D (10)

Support is closed under tautological implications by preserving degrees. In other

words, if F has a support d and F U G is an LR # L tautology then G too has a support

d :

if { F V U G then { © supdª F V U © sup

dª G , for every d ` D (11)

If an observation in the real world generates support d for F and if F U G is a decision

maker’ s belief then it is unreasonable to conclude that d is also a support for G . This

prevents us from considering supports closed under believed implications. The

following rule of inference states that an LR # L tautology has always the highest

support :

if { F then { © sup ª F (12)

A rational decision maker believes in something which has support with the top

element of the dictionary. Thus, the following axiom should be considered for a

rational decision maker :

© sup ^ ª F U © bel ª F

This axiom, of course, assumes that an assertion and its negation are not

simultaneously derivable with the top element as support, that is, an integrity

constraint (Das 1992) of the following form :

© sup ^ ª F g © sup ^ ª | F U v

It is di� cult to maintain consistency of a database in the presence of the above axiom,

particularly when the database is constructed from diŒerent sources ; mutual

inconsistency and mistakes sometimes need to be tolerated. In these circumstances, it

might be left to the decision maker to arbitrate over what to believe or not believe.

A decision maker might believe in something even if the database derives no support

for it. W e call a decision maker who does not believe in something unless there is

support with the top element a strict decision maker. If a decision maker is both

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


rational and strict then the concepts of believability and support with the top element

coincide. In other words,

© sup ^ ª F % © bel ª F

Note that we do not consider the following as an axiom which says that certain

evidence for an assertion also implies every evidence for the assertion lower than the

evidence :

© supd "

ª F U © supd #

ª F where d2 % d1

The reason for exclusion will be given in the context of model de® nition. The exclusion

also avoids the unnecessary contributions to the aggregation process for F.

We adopt the following two standard axioms of goals (Cohen and Levesque 1990,

W ainer 1994) :

| © goal ª v (13)

© goal ª F g © goal ª (F U G) U © goal ª G (14)

Axiom (13) says that something that is impossible to achieve cannot be a goal of a

decision maker. Axiom (14) states that all the logical consequences of a decision

maker’ s goal are goals themselves.

According to Cohen and Levesque (1990), worlds compatible with a decision

maker’ s goals must be included in those compatible with the decision maker’ s beliefs.

This is summarized in the following axiom :

© bel ª F U © goal ª F (15)

A database is full of a decision maker’ s beliefs. Consequently, many redundant goals

can be generated due to the presence of the above axiom. A goal will be considered

achieved (resp. active) in a state if it is derivable (resp. not derivable) in the state.

To keep our development practical and simple we have excluded a number of

axioms related to goals. Two such axioms concerned with goals (Wainer 1994) are (a)

if a decision maker has a goal of having a goal then s } he has this goal and the converse

(b) if a decision maker has a goal of not having a goal then s } he does not have this goal

and vice versa. If we had considered these axioms this would have introduced some

extra properties on the accessibility relations of our model de® nition in the following

section.

Semantics of temporal LR # L

A model of LR # L is a tuple

© W , V , Rb, R

s, R

gª

in which W is a set of possible worlds. A world consists of a set of temporally quali® ed

assertions outlining that which is known in diŒerent periods of time. V is a valuation

which associates each (world, interval) pair with a subset of the set of propositions. In

other words,

V : W ¬ I U P (P)

where I is the set of all intervals of the form [i", i

#], P is the set of propositions and P (P)

is the power set of P .

Consider the world w and interval [i", i

#] which is a member of I. Then the image of

© w, [i", i

#] ª under the mapping V , written as Vw[i

", i

#], is the set of all propositions

which are true in [i", i

#] in the world w . This means that p[i

", i

#] holds in w for each p in

Vw[i", i

#].

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


The relations Rb, R

sand R

gare the accessibility relations for beliefs, supports and

goals, respectively. Each of these relations is relativized in time, that is, each relation

de® nes an accessibility relation for every time point. For example, the relation R tb

relates a world w to a set of worlds considered possible by the decision maker from w

at time t. If there are n candidates in a decision construct which is active in a world w

at time t then the size of such a set of possible worlds will be 2n .

The relation R ts

is a hyperelation which is a subset of the set

W ¬ D ¬ P (W )

Semantically, if © w , d, W « ª ` R tsthen there is an amount of support d for moving to one

of the possible worlds in W « from the world w at time t, where W « is non-empty. In

other words, the support d is for the set of assertions uniquely characterized by the set

of worlds W « .Aggregation of arguments introduces a hierarchy of preferences (Das 1995b) among

the set of all possible worlds accessible from w by the relation R tb. The maximal

elements and possibly some elements from the top of the hierarchy of this preference

structure will be called goal worlds. The relation R tg, which is a subset of R t

b, relates the

current world to the set of goal worlds. Only one of the goal worlds is committed to

move from the current world and this world will be called the committed world.

For every time point t, the following additional restrictions are placed on the

relations of the model corresponding to some of the axioms of LR # L : (not including

axioms for rational and strict systems)

Rg

is serial

Rb

is serial, transitive and Euclidean

for every w in W and d in D , if © w, d, W « ª ` Rs

then W « 1 0

RgX R

b: for every w and w « in W , if wR

gw « then wR

bw «

W e also consider the following compactness and concatenation properties of the

valuation V of a model :

Vwu [i

", i

#] X Vw

u [i$, i

%], for every (i

$, i

%) contained in (i

", i

#).

Vwu [i

", i

$] f Vw

u [i$, i

#] X Vw

u [i", i

#], for every pair of intervals (i

", i

$) and (i

$, i

#).

where Vwu [i

", i

#] is the set of all property symbols of Vw[i

", i

#].

The semantics of supports, beliefs and goals are as follows. Given a model - ¯ © W ,

V , Rb, R

s, R

gª , the truth values of formulae with respect to pairs © w, i ª of worlds and

time are determined by the rules given below :

z © w,i ª- w .

z © w,i ª- p[i", i

#] iŒp ` Vw[i

", i

#].

z © w,i ª- p(i", i

#) iŒthere exist i

$, i

%such that i

"% i

$% i

%% i

#and p ` Vw[i

$, i

%].

z © w,i ª- © supdª F iŒthere exists © w , d, W « ª in R i

ssuch that z © w « ,i ª- F, for every w « ` W « .

z © w,i ª- © bel ª F iŒfor every w « in W such that wR ibw « , z © w « , i ª

- F.

z © w,i ª- © goal ª F iŒfor every w « in W such that wR igw « , z © w « ,i ª- F.

z © w,i ª- | F iŒz } © w,i ª- F.

z © w,i ª- F g G iŒz © w,i ª- F and z © w,i ª- G .

Note that the truth value of a temporal proposition p[i", i

#] (or p(i

", i

#)) with respect to

a world is the same at every time point, but the status of p changes when the current

time point lies within the interval [i", i

#].

A formula F is said to be true in model - if and only if z © w,i ª- F , for every world

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


© w , i ª in W ¬ I. A formula F is said to be valid if F is true in every model. The soundness

and completeness result can be stated as follows (see Das and Hammond, 1996, and

Das et al. 1996 for detail) : for every formula F ` LR # L , z F if and only if { F .

M edical example

This example illustrates the semantics presented in the previous section. First of all, we

consider the dictionary D as dict(Prob) and D « is D itself. Suppose the current world

w!

is described by a database consisting of the formulae in the above section (which are

translated from the decision constructs presented in Section 4) as hypotheses and the

following set as knowledge ( 3 F g © bel ª F ) :

² young, smoker, pain j in j upper j abdomen, weight j loss ´

The valuation V on w!

is de ® ned as follows:

V(w!) ¯ ² young, smoker, pain j in j upper j abdomen, weight j loss ´

Since there are two candidates in the weight j loss j diagnosis decision construct, there

Figure 3. Relation between the current world and possible worlds.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


will be 2 # , that is, four possible worlds w", w

#, w

$and w

%whose valuations are as follows

(see Figure 3) :

V(w") ¯ V(w

!) e ² cancer, weight j loss j diagnosed ´

V(w#) ¯ V(w

!) e ² peptic j ulcer, weight j loss j diagnosed ´

V(w$) ¯ V(w

!) e ² cancer, peptic j ulcer, weight j loss j diagnosed ´

V(w%) ¯ V(w

!) e ² weight j loss j diagnosed ´

The relations Rb

and Rs

in the model de ® nition are de® ned as follows :

Rb¯ ² © w

!, w

"), © w

!, w

#ª , © w

!, w

$ª , © w

!, w

%ª ´

Rs¯ ² © w

!, d2, ² w

", w

$´ ª , © w

!, d4, ² w

#, w

$´ ª , © w

!, d2, ² w

%´ ª ´

Note that weight j loss j diagnosed is true in each of the possible worlds and therefore

this is a goal as the set of goal worlds is a subset of the set of possible worlds. The goal

corresponds to the provability of © goal ª weight j loss j diagnosed in the current world

using © bel ª weight j loss in conjunction with the formula

© bel ª weight j loss U © goal ª weight j loss j diagnosed

The goal is active in w!. We are, of course, assuming that the theorem prover of LR # L

is able to derive the negation of © bel ª weight j loss j diagnosed from the current world by

a mechanism similar to negation by failure. The supports for the mutually exclusive

possibilities are computed by the aggregation process (using the domain knowledge

that cancer and peptic j ulcer are almost mutually exclusive candidates) as follows:

total support for C"(cancer g | peptic j ulcer) ¯ ! ( ² © C

", G

", d2 ª ´ ) ¯ d2

total support for C#( | cancer g peptic j ulcer) ¯ ! ( ² © C

#, G

#, d4 ª ´ ) ¯ d4

total support for C$(cancer g peptic j ulcer) ¯ ! ( ² © C

$, G

", d2 ª , © C

$, G

#, d4 ª ´ ) ¯ 0

total support for C%( | cancer g | peptic j ulcer) ¯ ! ( ² © C

%, G

%, d4 ª ´ ) ¯ d2

where each di is drawn from dict(Prob) and the grounds G", G

#and G

%are as follows:

G"

¯ weight j loss g smoker U cancer

G#

¯ weight j loss g pain j in j upper j abdomen U peptic j ulcer

G%

¯ weight j loss g young U | cancer g | peptic j ulcer

Assuming that d4 is less than d2, the preference relation ! among the set of possible

worlds is derived as w$

! w#, w

#! w

"and w

#! w

%. The maximally preferred possible

worlds are w"

and w#. The relation R

gin the model de® nition is now de® ned as follows:

Rg¯ ² © w

!, w

"ª , © w

!, w

%ª ´

This yields a dilemma. In case the decision maker cannot gather any more evidence,

s } he may commit to w%

by preferring w%

to w". This involves adding | cancer and

| peptic j ulcer to the current state of the database as either beliefs or knowledge

depending on the strength of support and the decision maker’ s con ® dence. The goal

weight j loss j diagnosis in the new situation will no longer be active due to the presence

of

© bel ª ( | cancer g | peptic j ulcer) U weight j loss j diagnosed

Note that we add only beliefs to keep the belief revision option open in case of a wrong

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


diagnosis. Alternatively, if we now add additional evidence © bel ª positive j biopsy into

the database that would increase the total support for C"

as follows :

total support for C"

¯ ! ( ² © C", G

", d2 ª , © C

", G «

", d3 ª ´ ) ¯ d2 d3 ® d2 ¬ d3

where the additional ground G !"

for C"

is the following:

G !"

¯ weight j loss g positive j biopsy U cancer

The revised valuation on each wi

will be as before except positive j biopsy changes its

truth value. The relations Rs

and Rg

will be rede® ned as follows :

Rs¯ ² © w

!, d2, ² w

", w

$´ ª , © w

!, d4, ² w

#, w

$´ ª , © w

!, d2, ² w

%´ ª , © w

!, d3, ² w

", w

$´ ª ´

Rg¯ ² © w

!, w

"ª ´

Since w"

is the only goal world, the decision maker considers w"

as the committed

world. Changing to the committed world from the current world involves adding

cancer and | peptic j ulcer to the database as decision maker’ s beliefs. Adding

© bel ª cancer to the database will trigger the decision for cancer treatment and the

decision making process continues as before.

Appendix C. Translation from R2L plans to LR 2L

First of all, if the situation in a decision construct is believed then the corresponding

goal is raised. Thus the situation and goal portion in decision weight j loss j diagnosis is

translated to the following temporalized rule :

© bel ª weight j loss(t) U © goal ª weight j loss j diagnosed(t)

For any particular situation a raised goal is considered as achieved if it is true. The

raised goal from a decision construct is true if any possible situation for the candidates

is believed. In the context of the decision weight j loss j diagnosis, this is re¯ ected in the

following temporalized formulae :

© bel ª (cancer(t) g | peptic j ulcer(t)) U weight j loss j diagnosed(t)

© bel ª (peptic j ulcer(t) g | cancer(t)) U weight j loss j diagnosed(t)

© bel ª ( | cancer(t) g | peptic j ulcer(t)) U weight j loss j diagnosed(t)

© bel ª (cancer(t) g peptic j ulcer(t)) U weight j loss j diagnosed(t)

The equivalent LR # L representations of the ® rst argument (rest are similar) in decision

weight j loss j diagnosis are given below :

© bel ª (weight j loss(t) g elderly(t)) U © supd "

ª cancer(t)

Each of rules, constraints and obligations are also translated to their equivalent

LR # L representation by appropriate temporalization. For example, the equivalent

LR # L representation of the constraint in Section 5 is given below (assuming the time

granularity to be an hour) :

c ( © bel ª chemo[t", t

#] U d t

$d t

%( © bel ª baseline[t

$, t

%] g t

%! t

"))

A plan in R # L is equivalently transformed to a principal rule. The body of a principal

rule is formed using the pre-condition and scheduling constraints. The temporal

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


variables of the principal rule are adjusted according to the scheduling constraints.

The principal rule corresponding to the plan named cancer j therapy is given below :

c (cancer(t") g baseline[t

", t

#] g chemo[t

#, t

$] g t

%

¯ t$ 6 g follow j up[t

%, t

&] U chemotherapy[t

", t

&])

By a normal completion of a plan we mean that the constituent actions and subplans

are executed in such a way that all scheduling constraints are satis ® ed. Under normal

completion of a plan an instance of the consequent of the principal rule will be

derivable. This instance conveys the information about the start and normal

completion time of the plan. In the above example if we have executed the actions as

baseline[1, 3], chemo[3, 4], and follow j up[10, 15] and cancer(1) is true then chemo-

therapy[1, 15] is derivable from the rule. This means that the plan started at 1 and

® nished at 15 and the completion is normal.

When a repeat-until type of scheduling constraint is present in a plan speci® cation

then additional recursive rules are generated in addition to the primary rule. Consider

the following plan related to chronic asthma treatment for children (BTA 1993) :

plan : : subsequent j treatment

pre j conditions

child & asthma

subtasks

oxygen j treatment ;

drug j salbutamol


first(oxygen j treatment) ;

after(oxygen j treatment,

repeat(every(mi nute(30)), drug j salbutamol, until

(patient j has j improved)))

abort j conditions

none.

The principal rule corresponding to the plan is presented below :

c (child(t") g asthma(t

") g oxygen j treatment[t

", t

$] g repeat j drug j salbutamol[t

$, t

#]

U subsequent j treatment[t", t

#])

The additional set of rules are provided below :

c ( patient j has j improved[t", t

#] U repeat j drug j salbutamol[t

", t

#])

c (drug j salbutamol[t", t

$] g repeat j drug j salbutamol[t

$, t

#]

U repeat j drug j salbutamol[t", t

#])

Note that a repeat-count type of scheduling constraint does not require any

additional recursive rules as the number of times the relevant action is to be executed

is ® nite. Consider a method for prescribing a steroid rescue course for asthmatic

children which requires that a dose for prednisolone of 1± 2 mg } kg body weight should

be used for ® ve days (BTA 1993). This can be represented in R # L as follows:

plan : : steroid j rescue j course

pre j conditions

child & high j asthma j exacerbation

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


subtasks

prednisolone j mg1 j 2/kg


repeat(every(ho ur(24)), prednisolone j mg1 j 2/kg, count (5))

abort j conditions

none.

The equivalent LR # L representation of this plan is given below :

c (child(t") g high j asthma j exacerbation(t

") g

prednisolone j mg1 j 2 } kg[t", t

#] g t

$¯ t

" 24 g

prednisolone j mg1 j 2 } kg[t$, t

%] g t

&¯ t

$ 24 g

prednisolone j mg1 j 2 } kg[t&, t

’] g t

(¯ t

& 24 g

prednisolone j mg1 j 2 } kg[t(, t

)] g t

*¯ t

( 24 g

prednisolone j mg1 j 2 } kg[t*, t

" !]

U steroid j rescue j course[t", t

" !])

ReferencesAllen, J. F. (1984) Towards a general theory of action and time. Arti® cial Intelligence, 23 : 123± 154.

Aqvist, I. (1985) Deontic logic. In D. Gabbay and R. Guenthner (eds) Extensions of Classical Logic, Vol.

2 of Handbook of Philosophical Logic, pp. 605± 714 (D. Reidel, Dordrech t).

Bonczek, R. H., Holsapp le, C. W. and W hinston, A. B. (1981) Foundation of Decision Support Systems

(Academic Press, New York).

Bhargava, H. K. and Kimbrough , S. O. (1994) Editor’ s introduc tion to the special issue on logic modelling.

Decision Support Systems, 11 : 101± 102.

BTA. (1993) Guidelines on the management of asthm a. THORAX : The Journal of the British Thoratic

Society, 48 : 1 ± 24.

Chen, M. C. and Henschen, L. J. (1985) On the use and internal structure of logic-based decision support

systems. Decision Support Systems, 1 : 205± 219.

Chellas, B. (1980) M odal Logic (Cambridge University Press, Cambridge).

Cohen, P. R. and Levesque, H. (1990) Intention is choice with commitment. Arti® cial Intelligence, 42 :

213± 261.

Das, S. K. (1992) Deductive Database and Logic Programming (Addison-W esley, Reading, MA).

Das, S. K. (1995) Form alising safety in decision support systems. In C. J. Mitchell and V. Stavridou (eds)

Proceedings of the IM A Conference on Mathematics of Dependable Systems, pp. 49± 61 (Oxford

University Press, Oxford).

Das, S. K. (1995) A logical reasoning with preference. Decision Support Systems, 15 : 19± 25.

Das, S. K., Fox, J. and Krause, P. (1996) A uni® ed framework for hypothe tical and practical reasoning (1) :

theoretical founda tions. Proceedings of the International Conference on Formal and Applied Practical

Reasoning (Springer-Verlag, Berlin), pp. 58± 72.

Das, S. K. and Hammond, P. (1996) M anaging tasks using an interval-based temporal logic. Journal of

Applied Intelligence, 6 : 311± 323.

Das, S. K. and Williams, M . H. (1989) A path ® nding method for checking integrity in deductive databases.

Data and Knowledge Engineering, 4 : 223± 244.

Davis, R. (1992) Foundation of Commonsense Reasoning (Addison-W esley, Reading, MA).

Emerson, E. A. and Halpern, J. Y. (1986) `Sometime ’ and `Not Never ’ revisited : on branching versus linear

time tempora l logic. Journal of the Association of Computing Machinery, 33 : 151± 178.

Fagin, R. and Halpern , J. Y. (1988) Belief, awareness and limited reasoning. Arti® cial Intelligence, 34 :

39± 76.

Fox, J. and Das, S. K. (1996) A uni® ed framework for hypothetical and practical reasoning (2) : lessons

from medical applications. In Proceedings of the International Conference on Formal and Applied

Practical Reasoning (Springer-Verlag, Berlin), pp. 73± 92.

Fox, J., Das, S. K. and Elsdon, D. (1994) Decision making and planning in autonom ous systems : theory,

technology and applications. In Proceedings of the ECAI Workshop on Decision Theory for DAI

Applications.

Fox, J. and Krause, P. (1992) Qualitative frameworks for decision support : lessons from medicine. The

Knowledge Engineering Review, 7 : 19± 33.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014


Fox, J., Krause, P. J. and Ambler, S. (1992) Arguments, contradictions and practical reasoning. In

Proceedings of the European Conference on Arti® cial Intelligence.

Fox, J. (1991) Decision support systems and qualitative reasoning. In Proceedings of the IMACS

International Workshop on Decision Support Systems and Qualitative Reasoning, pp. 43± 26 (North-

Holland, Amsterdam) .

Fox, J. (1993) On the soundne ss and safety of expert systems. Arti ® cial Intelligence in Medicine, 5 : 159± 179.

Haugh, B. A. (1987) Non-standard semantics for the method of tempora l arguments. In Proceedings of the

10th International Joint Conference on Arti® cial Intelligence, pp. 449± 455.

Herbert, S. I. (1995) Informatics for care protoco ls and guidelines : towards a European knowledge model.

In C. Gordon and J. P. Christensen (eds) Health Telematics for Clinical Guidelines and Protocols, pp.

27± 42 (IOS Press, Amsterdam).

Hammond, P., Harris, A. L., Das, S. K. and W yatt, J. C. (1994) Safety and decision support in oncology.

Methods of Information in Medicine, 33 : 371± 381.

Hintikka, J. (1962) Knowledge and Belief (Cornell University Press, Ithaca).

Huang, J., Jennings, N. R. and Fox, J. (1994) Cooperat ion in distribu ted medical care. In Second

International Conference on Cooperative Information Systems.

Halpern , J. Y. and Moses, Y. O. (1985) A guide to the modal logics of knowledge and belief. In Proceedings

of the 9th International Joint Conference on Arti ® cial Intelligence, pp. 480± 490.

Halpern , J. Y. and Shoham, Y. (1991) A proposit ional modal logic of time intervals. Journal of the

Associat ion for Computing M achinery, 38 : 935± 962.

Humberst one, I. L. (1979) Interval semantics for tense logics : some remarks. Journal of Philosophical Logic,

Kluwer Academic, Dordrecht.

Krause, P. J., Ambler, S. J., Elvang-Goransso n, M . and Fox, J. (1995) A logic of argumentation for

uncertain reasoning. Computational Intelligence.

Krause, P. and Clark, D. (1993) Representing Uncertain Knowledge : An arti® cial intelligence approach

(Intellect, Oxford).

Leveson, N. G. (1986) Software safety : why, what, and how. ACM Computing Surveys, 18,

New York.

McDermott, D. V. (1982) A temporal logic for reasoning about processes and plans. Cognitive Science, 6 :

101± 155.

Meyer, J.-J. Ch., van der Hoek, W. and Vreeswijk, G. A. W. (1991) Epistemic logic for computer science :

a tutorial (part one). EATCS , 44 : 242± 270.

Rao, A. S. and GeorgeŒ, M. P. (1991) Modelling rational agents within a BDI-arch itecture. In Proceedings

of the Knowledge Representation and Reasoning, pp. 473± 484.

Shoham, Y. (1987) Tempora l logics in AI : semantical and ontolog ical considerations. Arti ® cial Intelligence,

33 : 89± 104.

Shoham, Y. (1993) Agent-oriented programm ing. Arti® cial Intelligence, 60 : 51± 92.

Taylor, P. (1995) Decision support for image interpre tation: a mammography workstat ion. In Bizais,

Barillot and Di Paola (eds) Image Processing and M edical Imaging (Dordrecht, Kluwer) .

Vere, S. and Bickmore , T. (1990) A basic agent. Computational Intelligence, 6 : 41± 60.

van Benthem, J. (1991) The Logic of Time (Kluwer Academic, Dordrecht) .

von W right, G. H. (1951) Deontic logic. M ind, 60 : 1 ± 15.

Wainer, J. (1994) Yet another semantics of goals and goal priorities. In Proceedings of the 11th European

Conference on Arti ® cial Intelligence, pp. 269± 273.

Wooldridge, M. and Jennings, N. R. (1995) Intelligent agents : theory and practice. The Knowledge

Engineering Review, 10 : 1 ± 38.

Wieringa, R. J., M eyer, J. J. and Weigand, H. (1989) Specifying dynamic and deontic integrity constraints.

Data and Knowledge Engineering, 4 : 157± 189.

Dow

nloa

ded

by [

Wag

enin

gen

UR

Lib

rary

] at

15:

30 2

5 Ju

ly 2

014

Documents

A flexible architecture for autonomous agents