Artificial Intelligence Ubiquitous Computing … Computing ... Artificial Intelligence Technology with a Future. EUropean Certification of Informatics Professionals

Ubiquitous Computing

The European Online Magazine for the IT Professionalhttp://www.upgrade-cepis.org

Vol. III, No. 5, October 2002

Artificial IntelligenceTechnology with a Future

EUropean Certification of Informatics Professionals

An accepted European ICT certification standard

<http://www.eucip.com/>

1

UPGRADE

is the European Online Magazine for the Information Technology Professionals, published bimonthly at <http://www.upgrade-cepis.org/>.

Publisher

UPGRADE is published on behalf of CEPIS (Council of European Professional Informatics Societies,<http://www.cepis.org/>) by NOVÁTICA<http://www.ati.es/novatica/>, journal of the Spanish CEPIS society ATI (Asociación de Técnicos de Informática<http://www.ati.es/>).UPGRADE is also published in Spanish (full issue printed, some articles online) by NOVÁTICA, and in Italian (abstracts and some articles online) by the Italian CEPIS society ALSI<http://www.alsi.it> and the Italian IT portal Tecnoteca<http://www.tecnoteca.it/>.UPGRADE was created in October 2000 by CEPIS and was first published by NOVÁTICA and INFORMATIK/INFORMATIQUE, bimonthly journal of SVI/FSI (Swiss Federation of Professional Informatics Societies, <http://www.svifsi.ch/>).

Chief Editors

François Louis Nicolet, Zürich <[email protected]>Rafael Fernández Calvo, Madrid <[email protected]>

Editorial Board

Prof. Wolffried Stucky, CEPIS PresidentFernando Piera Gómez andRafael Fernández Calvo, ATI (Spain)François Louis Nicolet, SI (Switzerland)Roberto Carniel, ALSI – Tecnoteca (Italy)

English Editors:

Mike Andersson, Richard Butchart, David Cash, Arthur Cook, Tracey Darch, Laura Davies, Nick Dunn, Rodney Fennemore, Hilary Green, Roger Harris, Michael Hird, Jim Holder, Alasdair MacLeod, Pat Moody, Adam David Moss, Phil Parkin, Brian Robson.

Cover page

designed by Antonio Crespo Foix, © ATI 2002

Layout:

Pascale Schürmann

E-mail addresses for editorial correspondence:<[email protected]> and <[email protected]>

E-mail address for advertising correspondence:<[email protected]>

Copyright

© Novática. All rights reserved. Abstracting is permitted with credit to the source. For copying, reprint, or republication permission, write to the editors.

The opinions expressed by the authors are their exclusive responsibility.

ISSN 1684-5285

The European Online Magazine for the IT Professionalhttp://www.upgrade-cepis.org


Joint issue with N

OVÁTICA

2 AI: Past, Present and Future

– Federico Barber, Vicente J. Botti, and Jana Koehler

The guest editors present the issue and include a list of useful references for those interested in knowing more about Artificial Intelligence.

6 Spoken Communication with Computers

– Francisco Casacuberta-Nolla

This article deals with the development of systems which enable spoken interaction with computers, of widespread use in speech recognition systems, translation systems, etc.

10 Progress in AI Planning Research and Applications

– Derek Long and Maria Fox

In this paper the autors sketch the foundations of planning as a sub-field of Artificial Intelligence and the history of its development over the past three decades, and discuss some of the recent achievements within the field.

25 Trends in Automatic Learning

– Ramón López de Mántaras

This article looks at intelligent IT systems’ learning capacity, one of the fundamental characteristics of intelligence, and the techniques they employ to develop it presently.

32 Knowledge-Based Systems

– José Mira-Mira and Ana E. Delgado-García

In this article Knowledge Engineering is presented with special emphasis on methodological aspects (Knowledge Based Systems, Expert Systems), with the aim of approaching the rigour of other engineering disciplines.

39 Cooperating Physical Robots and Robotic Football

– Bernhard Nebel and Markus Jäger

In this article an analysis is made of the techniques and applications related to physical robots in tasks carried out in real environments, where the ability of the robots to cooperate correctly is especially important.

46 Autonomous Agents and Multi-Agent Systems

– Carles Sierra

This article presents the current state of multi-agent systems and their main applications.

53 Artificial Intelligence and Education: an Overview

– Maite Urretavizcaya-Loinaz and Isabel Fernández de Castro

This paper offers an overview of the different contributions AI is making to the world of educational IT, and a review of intelligent educational systems.

Artificial Intelligence: Technology with a Future

Guest Editors: Federico Barber, Vicente J. Botti, and Jana Koehler

Coming issue:“Security in E-Commerce/Business”


© Novática

UP

GRADE


2

Presentation

AI: Past, Present and Future

Federico Barber, Vicente J. Botti, and Jana Koehler

Artificial Intelligence (AI), defined as

“the science of makingmachines do things that would require intelligence if done bymen”

(Minsky), took on a viable scientific meaning as amodern Computer Science (CS) discipline during the secondhalf of the 20

th

century. It was the direct result of the conver-gence of various intellectual currents (Theory of Computation,Cybernetics, Information Theory, Symbolic Processing,Psychology, …) which had developed from the formal bases ofLogic and Discrete Mathematics, and had been given impetusby the development of digital computers. AI represents a seri-ous effort to understand the complexity of human experience ininformation processing terms. It deals not only with how torepresent and use complex and incomplete information logical-ly, but also with questions of how to see (vision), move (robot-ics), communicate (natural language, speech), learn, etc.

Human intelligent behaviour of the sort that AI tries to emu-late comprises several different aspects. One deals mainly withcognitive reasoning processes and is clearly related to logic.Another is more that of a

perceptive

nature (vision, speech,etc.) and, although it shares some problems and methods withthe previous aspect, it tends to be more rigorous in terms offormal expression and its specific problems, techniques andmethods constitute the discipline known as

Pattern Recogni-tion

. Finally we can talk about

symbolic

AI, concerned with theprocessing of symbols of knowledge, and

connectionist

AI, in

which the process of intelligence is simulated by means ofbasic, usually quantitive, elements of processing.

If we look at just the common core of AI, there is a widerange of trends which consider aspects of both human thoughtand human behaviour. Each of these trends can in turn receiveempirical approaches which use hypothesis and subsequentconfirmation by experiment, or rationalist approaches whichrequire a combination of logic-mathematical and engineeringprocesses (see Table 1).

The approaches contained in Table 1 define AI according toeach of these different aspects. The definitions in the top partare focused on processes connected with

reasoning

or

thought

and the ones in the lower part are focused on processes relatedto

behaviour

. The definitions in the left column measure thesuccess of AI from the human perspective (which requires anempirical approach) and those in the right column do the samefrom a

rational

perspective, a concept of intelligence whichcould be called rationality.

In recent years, research into AI has undergone a markedchange with regard to both content and the methodology beingused. It is becoming ever more common to build AI systemsbased on existing theories rather than putting forward newtheories; taking as a starting point rigorous theorems or solidexperimental evidence rather than intuition, and demonstratingthe use of AI applications in the real world rather than creating

Federico Barber, Telecommunications Engineer and Doctor ofComputer Science, is currently a Full Professor at the UniversidadPolitécnica de Valencia (Spain), where has been the Dean of theFaculty of Computer Science. He has been the editor of “Inteligen-cia Artificial, Revista Iberoamericana de IA”‚ (an Ibero-AmericanAI journal), and he is currently President of the Spanish Associationfor Artificial Intelligence(AEPIA). His areas of study are centredmainly on the problems of constraint satisfaction (scheduling,optimization, temporal planning with resources, temporal reason-ing, etc.) in which he has developed his own models and applica-tions, in addition to the field of knowledge engineering. He is jointleader of an extensive research group and has published a greatmany scientific articles. He has also participated in or led nationaland international research projects (CICYT, MC&T, ESPRIT, etc.),and technology transfer agreements, as well as sitting on variousscientific committees in his field. He is a senior member of ATI andco-editor of Novática’s AI section. <[email protected]>

Vicente J. Botti, Electrical Engineer and Doctor in ComputerScience, is currently a Full Professor at the Universidad Politécnicade Valencia (Spain), where he has also been the Head of the Dept.of Informatics Systems and Computation. His fields of study arefocused mainly on multi-agent systems, and more specifically, real

time multi-agent systems, real time systems, mobile robotics (inwhich he has developed his own models, architectures and applica-tions) in addition to the field of knowledge engineering. He is jointleader of an extensive research group whose general line ofresearch is Artificial Intelligence and has published about 100scientific articles. He has been and is a principal researcher onnationally and internationally funded projects (CICYT, MC&T,ESPRIT, etc.), and on technology transfer agreements, as well assitting on various scientific committees in his areas of interest. Heis a senior member of ATI and co-editor of Novática’s AI section.<[email protected]>

Jana Koehler is a research staff member and project leader at theIBM Research Lab in Zurich that she joined in Spring 2001. She gother Phd in 1994 from the University of Saarbruecken, where shehad worked at the German Research Centre for AI from 1990 to1995 in an AI planning project. From 1996 to 1999 she was anassistant professor at the University of Freiburg where she startedworking as a consultant for the technology management of Schin-dler Elevators in 1998. From 1999 to 2001 she worked as a projectleader for Schindler Elevators. At IBM, she works on new middle-ware technology for the integration and automation of businessprocesses based on webservices. <[email protected]>


3

UP

GRADE

Vol. III, No. 5, October 2002 © Novática

‘toy’ examples. In areas such as games, logical inference andtheorem proving, and medical diagnosis, systems based onrigorous theoretical principles are emerging which can performas well or better than human experts. In other areas – such aslearning – visual perception, robotics and natural languageunderstanding are making rapid steps forward thanks to theapplication of better analytical methods and a better under-standing of the problems involved.

A good example of the above is the field of natural languageunderstanding. In the 70s a great many architectures and ad hocapproaches were tested on some specially chosen examples.More recently these have given way to approaches based on

hidden Markov models

, founded on rigorous mathematicaltheory in which models are generated by means of a learningprocess based on a large body of real language data. The use ofthese models enables us to obtain ever better classifications,and language technology, together with its associated field ofhandwriting recognition, is currently moving towards industri-al and consumer applications.

During the 90s,

Fuzzy Logic

was consolidated in several AIcontexts, and the

connectionist paradigm

continued to gainfavour, as did

genetic algorithms

, leading to the development of

hybrid systems

in a quest for adaptability. New

knowledgeacquisition

methodologies were developed, such as KADS (byStewart Tansley). In

learning

significant advances were madeand new methods were put forward. With regard to cognitivearchitectures we saw the revolution that the introduction of

re-activity

sparked in the development of

autonomous agents

.Finally we witnessed a

change of paradigm

in artificial vision,from the classic passive approach to the

active approach

(AlanYuille) whereby the perceptual task is connected with theperformance of actions (perception-action). This had importantimplications for the development of robotic systems withenhanced performance.

The work done by Tate and Chapman has given rise to anelegant synthesis of planning programmes brought together ina unified framework. Planning systems are currently used for

programming the work in factories and for space shots. Mean-while,

intelligent scheduling

systems based either on theConstraint Satisfaction Problems (CSP) paradigm extended bythe inclusion of temporal reasoning techniques, or knowledgebased systems, provide an alternative answer to classic unsolv-able problems.

Pearl’s Probabilistic

Reasoning in Intelligent Systems

(1988)marked the arrival of the use of

probability

and

decision theory

in AI. The development of the

belief

network formalismresponded to the need to be able to reason efficiently whenfaced with a combination of uncertain knowledge. Thisapproach far outperforms the probabilistic reasoning systemsof the 60s and 70s, and is currently at the heart of research intoAI which is being currently being carried out on uncertainreasoning and expert systems (ES). The work of Pearl, Horvitzand Heckerman served to promote the idea of

ES rules

, that is,that they should act rationally in accordance with decisiontheory, without trying to imitate human experts. Following thisline of thought,

fuzzy logic

, based on possibility theory,emerged in response to the difficulty of providing problemswith precise data input. Possibility theory was introduced byZadeh in 1965 to handle uncertainty in fuzzy systems, and hasmuch in common with probability. Although mathematicians atfirst considered it to be a flawed theory, possibility theory actu-ally tackles a different problem. Fuzzy logic has been widelyused by the Japanese in the design and construction of house-hold appliances.

Similar trends have been seen in

robotics

,

computerizedvision

,

automatic learning

(including neural networks) and

knowledge representation

. A better understanding of the prob-lems and their complexity, together with greater computingcapacity, have enabled sound reasoning methods to be created.

Possibly encouraged by the progress made in the solution ofsubproblems in AI, researchers have gone back to work on theproblem of the “complete agent”, adopting this new, moreformalist, trend. Research by Newell, Laird and Rosenbloom(SOAR) is the best known example of a general architecture for

Empirical Approach Rational Approach

HumanThought

Systems which think like humans.

Cognitive Science.

"The exciting new effort to make computers think ... machines with minds, in the full and literal sense".

"The automation of activities that we associate with human thinking, activities such as decision-making, problem solving, learning...".

Systems which think rationally.

Logic-mathematical processes.

"The study of mental faculties through the use of computational models ".

"The study of the computations that make it possible to perceive, reason, and act".

HumanBehaviour

Systems which act like humans.

Cognitive task simulation.

"The art of creating machines that perform functions that require intelligence when performed by people".

"The study of how to make computers do things at which, at the moment, people are better".

Systems which act rationally.

Implementation of Inferential Processes.

"A field of study that seeks to explain and emulate intelligent behaviour in terms of computational processes ".

"The branch of computer science that is concerned with the automation of intelligent behaviour ".

Table 1: Various approaches to AI from different perspectivesSource: Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 1995


© Novática

UP

GRADE


4

an AI system. One of the fundamental aspects of a

generalarchitecture

is its capacity to incorporate many different kindsof

decision making

, from knowledge based deliberation toreflex action responses. The new agent architectures aim tostrike a balance between these two factors, reflex responses, forsituations in the which speed is of the essence, and knowledgebased deliberations, where the agent has time to take moreinformation into consideration, for forward planning, forhandling situations in which there is no immediate responseavailable and to propose better responses tailored specificallyto the situation in hand. Architectures such as SOAR haveprecisely this structure. By means of compilation processeslike explanation based learning, they convert declarative infor-mation at a deliberative decision making level into more effi-cient representations until the decision eventually becomes areflex action.

Research into

real time AI

looks into all the abovementionedaspects. Agents in real environments need to have the means ofcontrolling their own deliberations and also be capable of usingthe time allowed for reasoning to perform the calculationswhich will provide the best results. As AI systems are appliedto ever more complex domains, so all the problems willbecome real time problems, since the agent will never haveenough time to find an exact solution to a problem.

There is obviously a great need for methods which work wellin more general decision taking situations. In recent years twopromising techniques have appeared,

anytime algorithms

and

decision theory techniques

. The last element of an agent’sarchitecture is its

learning

mechanism. Inductive learning,reinforcement learning and compilation learning mechanismscan be used for all agent’s learning activities. These mecha-nisms will doubtless depend on the type of representationchosen. Representations based on logic, and neural and proba-bilistic networks, are well known and much studied formalismsfor which there are a great variety of learning methods. As newrepresentations are created, such as first order probabilisticlogics, it will be necessary to create new learning algorithmsfor them.

Agent/multi-agent system

(MAS) technology is makingimportant contributions to problem solving in various domains(e-commerce, e-auctions, medicine, stock market, manufactur-ing systems, telephony systems, etc.), where traditional ap-proaches do not provide satisfactory solutions. The study ofMulti-Agent Systems began nearly 20 years ago, within thearea of

Distributed Artificial Intelligence

(DAI) which is asubfield of artificial intelligence research. DAI is the study ofintelligent group behaviour stemming from the cooperation ofwhat are known as agents. It studies how a group of modulescooperate to divide up and share the knowledge of a problem,and how it reaches a solution. DAI focuses on global behaviour,with a predetermined agent behaviour. It studies the techniquesand knowledge required for the coordination and distributionof knowledge and actions in a multi-agent environment.

When we look at how AI has evolved in the last fifty years wecan see a transition from the initial embryonic theories andsystems to the adaptable, robust and user-friendly environ-ments of today; environments based on a wide range of logical

theories, cognitive models and engineering based approaches.

Technological development

and

progress in related fields

(Neu-rophysiology, Psychology, Biology) will have a great deal tosay in the future. An analysis of current AI systems and the waythey can be extended will enable us to pose a great many ques-tions, the answers to which will lead us towards generalpurpose intelligent systems.

In this monograph by

Upgrade

we have introduced a few ofthe areas and techniques involved in AI which, by their veryscope, are impossible to deal with comprehensively. We shall,pay special attention to the discipline’s applicability and use asan alternative solution where other techniques or methodolo-gies have failed or do not provide satisfactory solutions, orwhere these alternative techniques may provide bettersolutions.

The articles included are the work of very importantresearchers/developers and cover each of the areas dealt from amultiple viewpoint – generalist, scientific and applied – with aspecial emphasis on future development. These contributionsshould give the reader an idea of the historical perspective, thecurrent state and the future possibilities of AI. We hope thatthey will enable the reader to have a clearer understanding ofthese areas and a greater awareness of the current realities andthe challenges they pose. The articles included are:

“Spoken Communication with Computers”

, by FranciscoCasacuberta-Nolla, dealing with the development of systemswhich enable spoken interaction with computers, of wide-spread use in speech recognition systems, translation systems,etc.

“Progress in Planning Research and Applications”

, byDerek Long and Maria Fox, in which they take a look at the ap-plications and current challenges posed by intelligent planningtechniques, used in task planning, robots, resource scheduling,etc.

“Trends in Automatic Learning”

, by Ramon López deMántaras, in which he looks at intelligent IT systems’ learningcapacity, one of the fundamental characteristics of intelligence,and the techniques they employ to develop it.

“Knowledge-Based Systems”

, by José Mira-Mira and Ana E.Delgado-García. In this article Knowledge Engineering ispresented with special emphasis on methodological aspects(Knowledge Based Systems, Expert Systems), with the aim ofapproaching the rigour of other engineering disciplines.

“Cooperating Physical Robots and Robotic Football”

, byBernhard Nebel and Markus Jäger. In this article an analysis ismade of the techniques and applications related to physicalrobots in tasks carried out in real environments, where the abil-ity of the robots to cooperate correctly is especially important.

“Autonomous Agents and Multi-Agent Systems”

, by CarlesSierra. This article presents the current state of multi-agentsystems and their main applications.

“Artificial Intelligence and Education: an Overview”

, byMaite Urretavizcaya-Loinaz and Isabel Fernández de Castro.An overview of the different contributions AI is making to theworld of educational IT, and a review of intelligent educationalsystems.


5

UP

GRADE

Vol. III, No. 5, October 2002 © Novática

To close, we would like to thank all the participants in thismonograph for their interest and efforts, and to thank theeditors of

Upgrade

too for their support, suggestions and infi-nite patience in bringing this work to fruition.

Note from the Editors:

This monograph will be also published in Spanish (full issue print-ed, some articles online) by Novática, journal of the Spanish CEPIS

society ATI (Asociación de Técnicos de Informática) at <http://www.ati.es/novatica/>, and in Italian (online edition only, containingabstracts and some articles) by the Italian CEPIS society ALSI and theItalian IT portal Tecnoteca at <http://www.tecnoteca.it>.

Translated by Steve Turpin

Useful AI References

Compiled by Federico Barber and Vicente J. Botti

Artificial Intelligence is an exceptionally lively field. In addi-tion to its real practical applications, there is a great body ofresearch and development, a symptom of the great challengesstill to be met. Below we list just some of the references ofassociations, conferences, prestigious publications, interestinglinks, etc. which reflect the activity and development in thissubject.

Without claiming to offer an exhaustive list

we providea sample of some of the most important items, to which shouldbe added the references which appear in the articles comprisingthis issue.

Principal Associations

IJCAI (International Joint Committee on AI)ECCAI (European Coordination Committee for AI):

<http://www.eccai.org/>AAAI (American Association for AI) <http://www.aaai.org/>IBERAMIA (Association of Ibero-American Associations):

<http://www.iberamia.org/>.AEPIA (Spanish Artificial Intelligence Association):

<http://www.aepia.org>

Some e-publications

JAIR. Journal of A.I. Research: <http://www.jair.org>ETAIJ. Electronic Trans. on AI – ECAI: <http://www.etaij.org/>Cognitive Systems Research:

<http://www.elsevier.com/locate/cogsys>

Some specific publications

AI Magazine (AAAI)Artificial IntelligenceArtificial Intelligence ReviewCognitive Brain ResearchCognitive Science, etc.Computer Speech and LanguageData & Knowledge EngineeringData Mining and Knowledge DiscoveryElectronic Trans. on AI (AI Communications).Engineering Applications of Artificial IntelligenceFuzzy Sets and SystemsIEEE Trans. on Pattern Analysis and Machine IntelligenceIEEE Transactions on Fuzzy SystemsIEEE Transactions on knowledge and data engineering.IEEE Transactions on Man and CyberneticsIEEE Transactions on Neural NetworksInt. J. of Uncertainty Fuzziness and Knowledge-Based SystemsInt. journal of Approximate ReasoningInt. journal of Computer VisionIntelligent Systems and their applicationsInternational Journal of Pattern Recognition and A.I.

Knowledge Eng. Review.Machine learningNeural computation Neural NetworksPattern recognition.

Conferences and congresses

IJCAI: International Joint Conference on AI ECAI: European Conference on AIIEEE Conference on Artificial IntelligenceCAEPIA: Spanish Conference on Artificial Intelligence.

Distribution lists

INT-ARTIF: AI Distribution List (RedIRIS):<http://www.rediris.es/list/info/int-artif.html> (in Spanish).

Newsgroups

comp.aicomp.ai.edu comp.ai.digest comp.ai.doc-analysiscomp.ai.fuzzycomp.ai.games comp.ai.geneticcomp.ai.nat-lang comp.ai.nlang-know-repcomp.ai.neural-netscomp.ai.phiñlosophycomp.ai.shellscomp.ai.vision

Other interesting links

Spanish Artificial Intelligence Research Institute (IIIA), SpanishScientific Research Council (CSIC): <http://www.iiia.csic.es/>

News service of the AAAI:<http://www.aaai.org/Pathfinder/html/current.html>

AI Topics: <http://www.aaai.org/Pathfinder/pathfinder.html>TECNOCIENCIA. AI directory:

<http://www.portaltecnociencia.es/index/DirectorioSEC.jsp?ISI=0616>

Artificial Intelligence Laboratory (MIT):. <http://www.ai.mit.edu/>CMU Artificial Intelligence Repository:

<http://www.cs.cmu.edu/Web/Groups/AI/html/air.html>Artificial Intelligence Resources in the Institute For Information

Technology: <http://ai.iit.nrc.ca/ai_point.html>The Collection of Computer Science Bibliographies . Bibliographies

on Artificial Intelligence:<http://liinwww.ira.uka.de/bibliography/Ai/index.html>

Computer Science Bibliography. Artificial Intelligence:<http://www.informatik.uni-trier.de/~ley/db/ai.html>


© Novática

UP

GRADE

Vol. III, No. 5, October 2002 6

Spoken Communication with Computers

Francisco Casacuberta-Nolla

Present day technology allows us to build commercial IT systems able to transcribe speech into written text,interpret a spoken instruction in order to manage devices, or access information systems by means of adialogue between human being and machine. The same technology also allows us to build systems whichtranslate speech from one language into another for limited tasks. The great success of these systems is due,among other reasons, to the use of pattern recognition techniques and, in particular, to the fact that modelscan be built automatically from examples of the problem to be tackled. However, we are still a long way offachieving real spoken communication between human beings and computers. We may need a new frameworkin which to develop new models and techniques.

Keywords: Human-machine Speech Communication,Speech Recognition/translation, Statistical Pattern Recogni-tion.

Introduction Speech is the usual way for the transmission of information

from a human being to another. However, we need to use otherways for communication with a computer.

The current state-of-the-art of the so called speech technolo-gies allows us to build commercial IT systems capable of tran-scribing voice into written text controlling devices or accessinginformation systems via speech. However, we are still a longway off achieving real oral communication between humanbeings and computers.

Some types of systems that process speech are: • Linguistic decoders for written transcription of speech in

terms of word phrases. Examples of such systems are thoseused for dictation [Padmanabhan/Picheny 2002].

• Speech understanding systems to extract the meaning of aspoken message by searching for key words or convertingthe oral message into a semantic representation [Cole et al.1997]. Summarizers, which are typically used to obtainsimplified versions of news broadcast on the radio or televi-sion, could be included within this group [Kubala et al.2000].

• Speech dialogue systems for resolving ambiguities, answer-ing questions or checking for possible errors in spokenaccess to information systems [Gorin et al. 2002] [McTear2002].

• Speech-input translators for enabling communicationbetween human beings who speak different languages [Coleet al. 1997] [Amengual et al. 2000] [Paulson 2001].

• Machine aided translation systems for transcribing phrasesuttered by a human translator in the target language of thephrase he has read in the source language.

• Speech synthesis systems for converting written messagesinto spoken messages [Cole et al. 1997].

There are other types of systems involving speech whichshare many techniques with the abovementioned systems: • Language identification, for knowing what language the

speaker is using [Cole et al. 1997]. • Topic detection, for identifying the subject matter of spoken

text [Kubala et al. 2000]. • Speaker identification/verification, for identifying or check-

ing who the speaker is [Cole et al. 1997]. A wide range of disciplines play important roles in the devel-

opment of these systems, pattern recognition being one of themost important. However, algorithmics, phonetics, linguistics,signal processing, information systems, formal languagetheory and artificial intelligence are also important in thedevelopment of speech systems.

The techniques used up until the eighties were based on theexpert knowledge of phonetics and linguistics available at thetime. From a given utterance, the system produced a hypothesis(e.g. a sequence of words) which was compatible with the

1

Francisco Casacuberta-Nolla received his Doctorate in Phys-ics from the Universidad de Valencia (Spain) in 1981 and is ChairProfessor of the Department of Computer Systems and Computa-tion of the Universidad Politécnica de Valencia. He has been a re-searcher on 6 European Union scientific projects (head researcheron one of them) and on 16 scientific projects of the Spanish Min-istry of Education and Science (head researcher on six of them).He has been published in a number of international journals andis author/editor of two books. He is also published in internationalconference proceedings and has directed 9 doctoral theses. He isa reviewer for IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, IEEE Transactions on Speech and AcousticProcessing, Pattern Recognition Letters, Machine Learning Jour-nal and Journal of Computers. He is a member of the followingscientific societies: IEEE Computer Society, Spanish Associationof Pattern Recognition and Image Analysis (AERFAI), and theSpanish Association for Artificial Intelligence (AEPIA).<[email protected]>


7 UPGRADE Vol. III, No. 5, October 2002 © Novática

available knowledge via a heuristic search process. But theformalization of the phonetic and linguistic knowledge and thetransmission of this knowledge into a computer system consti-tuted open problems and the performance of the systems wasnot satisfactory.

Since the eighties, statistical pattern recognition techniqueshave made it possible to build better systems. Using thisapproach, (statistical) models are built automatically, in a proc-ess known as training or learning, from real examples: acousticspeech signal for the acoustic models, monolingual text for themodels of word sequences and parallel text for the translationmodels. The drawback of this approach is the requirement oflarge training corpora (plural of the Latin word corpus, “col-lection of examples of spoken and written usages”.) Thesesystems can interpret a given utterance using very efficientalgorithms as search engines that are based mainly on dynamicprogramming and A* schemes.

In the following section, the statistical framework for build-ing the above-mentioned systems is presented.

Statistical Framework for Speech Processing In speech processing, one of the basic problems is to search

for the word sequence that a speaker has uttered (speech decod-ing). Another related problem is to search for a sentence in atarget language other than the one which the speaker has used(speech translation).

2.1. Speech Decoding Let us suppose that x is an acoustic representation of a certain

utterance. This acoustic representation usually is a sequence ofvectors, in which the components of each vector contain spec-tral information about a fixed length segment of speech (nor-mally 10 msec.). These vectors are obtained by applying signalprocessing techniques and acoustic knowledge [Cole et al.1997]. The decoding of x can be formulated as the problem ofsearching for a sequence of words ê that maximizes the proba-bility that x is generated by ê:

After applying Bayes’ theorem:

where Pr(x | e) is usually a probability density function thatis approximated by means of acoustic models and the distribu-tion Pr(e) is approximated by means of a language model[Jelinek 1998]. The aim of acoustic models (typically hiddenMarkov models) is to represent the acoustic variability whichcan be found in the pronunciation of a word. A hidden Markovmodel is usually associated to each phonetic or phone unit[Young/Bloothooft 1997]. The parameters of these models areestimated from speech data by using very known algorithmssuch as Baum-Welch or Viterbi algorithms. These trainingalgorithms are available in software packages such as HTK

[Young/Bloothooft 1997]. The acoustic models of words areobtained by linking the hidden Markov models of the phonesthat define the word.

Language models aim to represent the laws (statistics) gov-erning the formation of phrases in a language in such a way thata correct sequence of words has a high probability, while anincorrect sequence of words has a low or zero probability. Themost commonly used language models are n-grams, which inpractice are a type of stochastic regular grammar. The parame-ters of these models can be estimated by computing thefrequencies of sequences of n words in a text corpus. Unfortu-nately, there are no corpora large enough to estimate all theparameters of the n-gram models adequately. In order to solvethis problem of the lack of enough training samples, there areseveral smoothing techniques which allow the assigning of anon-zero probability to unseen events [Young/Bloothooft1997]. As the models of n-grams and the hidden Markovmodels are finite state networks, the problem of computing theoptimum sequence of words in an utterance can be formulatedas a search process in an integrated finite state network. Thisnetwork is obtained from the n-gram model by substitutingeach transition of the model by the hidden Markov model of theword associated to the transition. This process is carried outvery efficiently by using the Viterbi algorithm with beamsearch. This last heuristic allows us to implement the searchengine in low cost computers [Padmanabhan/Picheny 2002].One suboptimal alternative with lower computational cost isthe use of a two phase process (serial architecture): the firstphase consists of using a speech decoder with a language mod-el which is fairly simple and, therefore, not very precise inorder to obtain a set of best hypotheses in an explicit or com-pact form by means of a word graph. In the second phase, adetailed language model is used to choose the best hypothesis.

2.2. Speech-input Translation Speech translation can be formulated as a process that

involves the speech decoding of an utterance into a sentencefrom a source language followed by a process of translation ofthat sentence into a sentence from a target language. Unfortu-nately, the systems which implement the first process are noterror free and most of the present day text translation systemsassume that the input is written correctly.

The statistical framework can also allow to tackle the prob-lem of speech translation. Let us suppose that x is an acousticrepresentation of a given utterance. The translation of x intoanother language can be formulated as the problem of search-ing for a sequence of words in the target language whichmaximizes the equation:

But this translation can be considered as a two phase process:

2

eê ≈ argmax Pr(e | x). (1)

sê ≈ argmax Pr(e) · Pr(x | e). (2)

s

ss ≈ argmax Pr(s | x). (3)^

x w t→ →


© Novática UPGRADE Vol. III, No. 5, October 2002 8

in which e is a possible decoding of x in the source languagewhich translation is s. Consequently,

where Pr(x | e) is usually approximated by means of hiddenMarkov models as in the case of speech decoding, and thedistribution Pr(e,s) is approximated by means of a translationmodel [Amengual et al. 2000]. Stochastic finite state transduc-ers permit a direct approximation to the probability distributionPr(e,s) [Casacuberta 2000], although other alternatives are alsopossible [Niessen et al. 2000] if we consider the followingdecomposition:

In this case Pr(s) is approximated by means of a targetlanguage model and Pr(e | s) by means of alignment models andstatistical dictionaries [Niessen et al. 2000]. Stochastic finitestate transducers are finite state models, and are similar toregular stochastic grammars in the sense that each transition islabelled by a word in the source language and a string (whichmay be empty) in the target language. The training of stochasticfinite state transducers is achieved by building these models insuch a way that they adequately represent (and generalize)finite samples of pairs of sentences in the source language andin the target language [Vilar 2000] [Casacuberta 2000].

As in speech decoding, stochastic finite state transducersallow hidden Markov models to be integrated into the transduc-er’s transitions. In this integrated architecture the Viterbi algo-rithm is used as the engine of the translation process [Amen-gual et al. 2000].

By using the equation (5) we can apply a serial architectureto speech translation, that consists of a speech decoding (usingPr(x | e) and Pr(e) as an approximation to Pr(e | s)) and thetranslating of the decoded sentence (using Pr(t) and Pr(w | t)).This approximation is used when the translation models do notallow the integration of acoustic models into the translationmodel.

Speech is also used as an aid to human translators. Let ussuppose that we have a source text e to be translated, the humantranslator can translate e by pronouncing x, an acousticsequence in the target language. In this case the goal is tosearch for a target sentence, such that

where Pr(x | s) is approximated by means of acoustic models(in the target language), Pr(s) with a target language model andPr(e | s) with a translation model. The latter two terms are thoseusually found in a conventional speech decoding system. In thiscase the translation model aids the search process for the wordsequence in the target language. This is performed by means ofthe Viterbi algorithm or a version of the algorithm A* [Brownet al. 1994].

2.3 Speech Understanding and Dialogue Many speech understanding systems are formulated as a

system whereby speech is translated into a semantic represen-tation. In this case, the semantics can be modelled by means ofa formal language either directly (query language, control lan-guage for a robot, etc.) or indirectly to build a final semanticrepresentation (frameworks). This point of view allows us totackle speech understanding in the statistical frameworkdescribed above.

Dialogue can be considered as a problem of management ofthe alternate user’s turns and computer’s turns. In the user’sturn, he or she tries to obtain information by asking questionsor providing the details that the computer requests. In the com-puter’s turn, the dialogue system should try to carry out someaction (access to a database, generate a sentence as a reply or toask for clarification). This action will depend on how well thecomputer understood the user’s last turn and also will dependon the history of semantic messages that the computer hasobtained up until that moment. This process can also be formu-lated within the statistical framework, bearing in mind that thesystem has to understand what the user is saying in his or herturn (understanding) and to choose an action (classification) tobe performed in the machine’s turn.

Current State of Speech Systems The current state of development of the so called speech

technologies has enabled us to build dictation (voice-to-text)systems with a vocabulary of about 20,000 words capable ofcorrectly recognizing nearly 90% of input words. Speech-inputtranslation systems have also been built which are capable oftranslating spontaneous speech for limited tasks with 80% ofwords correctly translated. Although a great deal of progresshas been made in the development of these technologies, thereare still a large number of problems to be solved.

The performance of speech recognition systems is reducedconsiderably when they work in adverse conditions such asnoisy environments. On the other hand, for a robust training ofacoustic or language models, it is necessary to acquire largetraining corpora. These corpora must be filtered and labelledmanually or semi-automatically, all of which are expensiveprocesses. Moreover, aligned parallel corpora are needed fortranslation. Without any doubt, current linguistic knowledgecan and must play an important role in the design of suchsystems. The problem is how to combine the two approaches:the linguistic approach and the example-based approach. Greatstrides forward have also been made in the algorithmic aspectsof the search process, but to tackle applications of greater scopewe need to look for some more efficient algorithmic solutions.Other important problems are: adaptability to new environ-ments or new applications; confidence measures of the hypoth-eses generated in the speech decoding process; inclusion ofprosody; handling of words not seen during the training phase,handling of spontaneous speech, and the lack of good modelsfor dialogue.

s s ( max(Pr(e,s) · Pr(x | e))e )argmax Pr(s | x) ≈ argmax (4)

s s ( max(Pr(s) · Pr(e | s) · Pr(x | e))e )argmax Pr(s | x) ≈ argmax (5)

sargmax Pr(s | e,x) ≈ argmax Pr(e | s) · Pr(x | s) · Pr(s), (6)

3



Conclusions Speech technologies that are based on pattern recognition

techniques allow us to build systems for spoken communica-tion between human beings and computers in not overlayadverse situations. The application of these systems may rangefrom dictation to device control, as well as speech translationin limited applications. However, there is still a long way to gobefore communication between human being and machinebecomes an everyday occurrence.


References [Amengual et al. 2000]

J. C. Amengual, J. M. Benedí, F. Casacuberta, A. Castaño, A.Castellanos, V. M. Jiménez, D. Llorens, A. Marzal, M. Pastor, F.Prat, E. Vidal, J. M. Vilar. “The EUTRANS-I spoken languagetranslation system”. Machine Translation Journal, vol. 15 (1–2),pp. 75–103, 2000.

[Brown et al. 1994]P. F. Brown, S. F. Chen, S. A. Della Pietra, V. J. Della Pietra, A.S. Kehle, R. L. Mercer. “Automatic speech recognition in ma-chine aided translation”. Computer, Speech and Language, vol. 8,pp. 177–187, 1994.

[Casacuberta 2000]F. Casacuberta. “Inference of finite-state transducers by usingregular grammars and morphisms”. Lecture Notes in Artificial Intelligence, vol. 1891, pp. 1–14.Springer-Verlag, 2000.

[Cole et al. 1997]R. Cole, J. Mariani, H. Uszkoreit, G. Batista-Varile, A. Zaenen,A. Zampolli, V. Zue. “Survey of the state of the art in humanlanguage technology”. Web edition at<http://citeseer.nj.nec.com/cole97survey.html>.Cambridge University Press and Giardini, 1997.

[Gorin et al. 2002]A. L. Gorin, A. Abella, G. Riccardi, J.H. Wright. “Automatednatural spoken dialogue”. Computer, vol. 35 (3), pp. 51–56, 2002.

[Jelinek 1998]F. Jelinek. Statistical Methods for Speech Recognition. MITPress, 1998.

[Kubala et al. 2000]F. Kubala, S. Colbath, D. Liu, A. Srivastava, J. Makhoul. “Inte-grated technologies for indexing spoken language”. Communica-tions of the ACM, vol. 43 (2), pp. 48–56, 2000.

[McTear 2002]M. F. McTear. “Spoken dialogue technology: enabling the con-versational user interface”. ACM Computing Surveys (CSUR),vol. 34 (1), pp. 90–169, 2002.

[Niessen et al. 2000]H. Ney, S. Niessen, F. Och, H. Sawaf, C. Tillmann, and S. Vogel.“Algorithms for statistical translation of spoken language.”. IEEETransactions on Speech and Audio Processing, vol. 8 (1), pp.24–36, 2000.

[Padmanabhan/Picheny 2002]M. Padmanabhan, M. Picheny. “Large-vocabulary speech recog-nition algorithms”. Computer, vol. 35 (3), pp. 42–50, 2002.

[Paulson 2001]L.D. Paulson. “Translation technology tries to hurdle thelanguage barrier”. Computer, vol. 34 (9), pp. 12–15, 2001.

[Vilar 2000]J. M. Vilar. “Improve the learning of subsequential transducers byusing alignments and dictionaries”. Lecture Notes in Artificial Intelligence, vol. 1891, pp. 298–312,Springer-Verlag, 2000.

[Young/Bloothooft 1997]S. Young, G. Bloothooft (eds.). Corpus-based statistical methodsin speech and language processing. Kluwer Academic Publishers,1997.

4



Progress in AI Planning Research and Applications

Derek Long and Maria Fox

Planning has made significant progress since its inception in the 1970s, in terms both of the efficiency andsophistication of its algorithms and representations and its potential for application to real problems. In thispaper we sketch the foundations of planning as a sub-field of Artificial Intelligence and the history of itsdevelopment over the past three decades. We will then discuss some of the recent achievements within thefield and provide some experimental data demonstrating the progress that has been made in the applicationof general planners to realistic and complex problems. We conclude by identifying some of the open issuesthat remain as important challenges for future research in planning.

Keywords: AI, Planning of Actions, Plans, Scheduling andPlanning

IntroductionPlanning is a sub-field of Artificial Intelligence (AI),

explored by researchers in the AI community for more thanthree decades. Newell and Simon’s work on GPS [Newell/Si-mon 1963], Green’s QA3 [Green 1969] and McCarthy’s situa-tion calculus [McCarthy/Hayes 1969] helped to define the clas-sical planning problem and many of the basic assumptionsmade then still influence planning research today. This papersurveys the objectives of the research field, the progress of re-searchers towards meeting them and some of the current activ-ities and themes in the area. It also considers the extent towhich modern planning techniques are ready for wider exploi-tation and what still remains to be achieved.

In the early days of AI research scientists pursued a broadand ambitious, if somewhat ill-defined, objective of creating anintelligent machine. Reasoning capabilities were seen ascentral to this objective, but were expected to be based on aninterlocking collection of generic mechanisms. This can beseen in the application of theorem proving as a general technol-ogy to all kinds of reasoning problems, including planning. Asthe subject has developed it has become clear that genericreasoning, if it can ever be achieved, can only be built on athorough understanding of more specific examples of humanproblem-solving enterprises. Consequently, researchers haveexplored different areas of problem-solving reasoning and AIhas splintered into a collection of different sub-fields. Planningemerged as a specific sub-field with the seminal work of Fikesand Nilsson [Fikes/Nilsson 1971] on the STanford ResearchInstitute Problem Solver (STRIPS).

This paper begins, in section 2, with a description of the plan-ning problem itself and the constraints that have been imposedto make tractable versions of it. In order to understand the foun-dations for the most recent developments in planning, it is help-ful to review the history of research in planning – this iscovered in section 4. In sections 5 and 6 the more recent devel-opments in the field are described. Finally, in section 7, some

of the many problems that remain to be solved, or solved moresuccessfully, are considered.

The Planning ProblemTo make the planning problem accessible, it is necessary to

have a precise definition of what the problem is and whatconstitutes a solution to an instance of the problem. In definingthe problem, several simplifications have been made that do notalways characterise planning problems in general application,but make the core of what remains a more manageable startingpoint for research. Different researchers have adopted slightly

1 2

Derek Long is a Lecturer in Computer Science at the Univer-sity of Durham (United Kingdom). His research interests lie inplanning, and applications of planning systems. Dr Long co-chaired the 3rd International Planning Competition, held at AIPS2002. He also co-developed the STAN planning system and theTIM planning domain analysis system, establishing the conceptof generic types and their role in planning domain constructionand decomposition. Within the Hybrid STAN extension of theoriginal STAN system, he and his colleague Maria Fox havedemonstrated the potential to automatically configure problem-solving technology to support a planning system by attackingsub-problems using specialised solvers. He is chairman of theUK Planning and Scheduling Special Interest Group, a group thatholds an annual meeting attracting international participation.<[email protected]>

Maria Fox is Reader in Computer Science at the University ofDurham (United Kingdom). Prior to joining Durham UniversityDr. Fox was a lecturer at University College London. Her earlywork in AI Planning considered the problems of generating andrefining abstract plans through soundness-preserving transforma-tions. More recently she has focused on representation and rea-soning issues in temporal and metric planning. With Derek Longshe co-chaired the third international planning competition whichstressed planning in temporal domains. She has developed plan-ning algorithms and domain analysis techniques for a range ofplanning domain description languages and is currently workingin the area of autonomous planning and execution.<[email protected]>



different formulations of the problem, but the following is themost widely adopted starting point:

A planning problem is described by a collection of actions,each characterised by their preconditions (what must be true inorder for the action to be executed) and their postconditions(which describe the effect of execution of the action), an initialstate of the world and a description of the goals to be achieved.The problem is solved by finding actions that will transform thegiven initial state into a state satisfying the given goals.

It can be observed that this action-centric view of the plan-ning problem is influenced by the notion of state, or situation,and of transition between states. This view has a very strongheritage based on McCarthy’s development of the situationcalculus. This calculus describes how situations, described in afirst order language, are affected by actions performed on themby an executive agent. Effect axioms describe how actionschange the situations in which they are applied into new situa-tions while frame axioms describe what aspects of a situationremain unaffected as actions are applied. These axioms arespecified in terms of the relations and predicates that describeconfigurations of objects in the world, together with situationvariables that enable the facts associated with one situation tobe distinguished from those associated with a successor situa-tion. Axioms are universally quantified over situations. Given acomplete set of such axioms it is possible to deduce the situa-tion that results from the application of a chain of actions andto determine which actions to apply to obtain a desired statechange.

An important shortcoming of the situation calculus is thedifficulty of defining a complete set of effect and frame axiomsfor a non-trivial world. Effect axioms are easier to definebecause the number of actions that need to be described iscontained and they are usually identified with only a smallnumber of positive state changing effects. Frame axioms, onthe other hand, are extremely difficult to define exhaustivelybecause the number of properties of a situation that do notchange when an action is applied is far greater than the numberthat do. Furthermore, it is not natural to think about the worldin negative terms so there is a great danger of providing only apartial collection of frame axioms which would lead tounsound reasoning within the calculus. This problem isreferred to as the frame problem.

The STRIPS system made a very important contribution toPlanning research by introducing the Strips Assumption as away to avoid the complexity of the frame problem for thepurposes of planning within the situation calculus. Theassumption is that the only changes that arise on application ofan action to a situation are those that are explicitly mentionedas positive effects of the action. All other relations and predi-cates, associated with the situation in which the action isapplied, are automatically deduced to hold in the successorsituation. The STRIPS project introduced a simple syntax fordefining action schemas, in terms of the preconditions, addeffects and delete effects of the action. An example can be seenin Figure 1. Despite the many advances that have been made inplanning research over the years the STRIPS assumptioncontinues to be a fundamental principle in the modelling of

planning problems and an important influence on the way plan-ning algorithms are designed.

Planning problems are fundamentally dynamic in structure –an initial situation is presented, a goal is defined and a plan isseen as a sequence of (sets of) state changes applied over time.It is therefore natural to interpret collections of action schemasas defining the transitions in a parameterised automaton (seeFigure 2) and a plan as the transitions traversed by an acceptingtrace through the instantiated automaton. This view is incontrast with the static view imposed by the situation calculus,

Action LOAD ?object ?container?location

Precondition: at(?object,?location)

at(?container,?location)

empty(?container)

Add: inside(?object,?location)

Delete: at(?object,?location)

empty(?container)

Action UNLOAD ?object ?container ?location

Precondition: at(?container,?location)

inside(?object,?location)

Add: at(?object,?location)

Delete: inside(?object,?container)

empty(?container)

Action MOVE ?container ?start ?destination

Precondition: at(?container,?start)

link(?start,?destination)

Add: at(?container,?destination)

Delete: at(?container,?start)

Initially: at(PickUp,Home)

at(Box,Office)

link(Home,Town)

link(Town,Home)

link(Town,Office)

link(Office,Town)

Goal: at(Box,Home)

Figure 1: Simple domain description and problem instance. Terms marked with "?’’ are variables.



in which theorem-proving determines whether a particularcollection of constraints is consistent and the passage of time ismodelled through the use of state variables at the object level.A goal state is achievable if the collection of effect and frameaxioms are satisfiable for some value of the state variable asso-ciated with the goal. Both the dynamic and static views haveinfluenced the design of algorithms for planning, although thedynamic view has dominated approaches taken to representa-tion of planning problems.

This dominance is apparent in the family of PlanningDomain Description Languages (PDDL1.2 [McDermott 1998]and PDDL2.1 [Fox/Long 2002]) that have been proposed asstandards for modelling planning problems. These languagesare based on the STRIPS assumption and support the model-ling of a planning problem in terms of a compact representationof the finite state automaton that describes its behaviour. As isdiscussed in the following sections, this style of modelling canbe extended to support reasoning about continuous as well aslogical change, and can provide sufficient expressive power forthe modelling of very complex realistic planning problems.The question of how algorithms might be developed to enableplans to be found, given such models, is a separate one and isdiscussed below.

Foundations of PlanningThe action-centred view of problem representation makes

a number of simplifying assumptions which define classicalplanning. First, it is assumed that the evolution of actionsequences, applied to a completely known initial situation, canbe completely and correctly predicted as though no externalinfluences were operating in the world. Second, planning is thetask of constructing a single completed plan that achieves thegoal, prior to the execution of any part of it. A third, related

point, is that the classical planning formulation assumes thatthe goals are known before planning starts – planners do not settheir own goals and goals do not change as execution progress-es. This makes classical planning a poor technology for realis-tic problems in which goals arise continually and importantthings can happen outside the control of the planner. Finally, inclassical planning, in which numeric reasoning is excluded,plan quality is determined solely by the number of actions inthe plan. This is, of course, a very simplistic measurement ofplan quality and one that is further discussed in section 6,where recent work going beyond this assumption is examined.

Even under these simplifying assumptions plan generation iscomputationally a very hard problem. Under the assumptionthat the space of reachable states is finite planning belongs tothe PSPACE-hard class of problems meaning that the numberof states that must be considered in attempting to find a pathfrom the initial state to the goal is likely to be exponential in thesize of the description of the planning problem. The space ofreachable states is certainly exponential in the size of theproblem description (the description is schematic whilst thestate space is fully instantiated) so the task of a planning algo-rithm is to find a path between the initial situation and onesatisfying the goal, whilst exploring as little as possible of thestate space in the process. This is what makes plan generationdifferent from the problem of finding a shortest path in a graph– the graph is too big to be built explicitly (see Figure 3) so aplan generation algorithm must intelligently build only thatpart of it in which the solution lies.

The intelligent exploration of the problem state spacedepends on the ability of the planning algorithm to exploitpowerful heuristics or control knowledge to guide its search.Many recent strides have been made in planning because of thediscovery of informative heuristics which can be very effective

in directing search towards a solution. Planning research has traditionally

relied on simple and relatively unstruc-tured models of the problem and placedthe research emphasis on the develop-ment of algorithms and powerful heuris-tic control methods. Although it has beenrecognized [Fox/Long 1998] [Gerevi-ni/Schubert 1998] that a model typicallycontains hidden structure that can beexploited by a planner the tendency hasbeen to persist with the construction andcommunication of traditional modelsand, either to supplement the traditionalmodel with problem-specific controlrules [Bacchus/Kabanza 2000] [Kvarn-strom/Doherty 2000] [Nau et al. 1999],or to use automated analysis to extractthe hidden structure and make it accessi-ble to the reasoning mechanisms of theplanner [Fox/Long 2000] [Porteous2001] [Gerevini/Schubert 1998]. Therecent successes of planners using prob-lem-specific control rules (see section

3

Figure 2: Illustration of a fragment of a state space for a planning problem. Note that the same action can cause transition between different pairs of states, with untouched parts of the start state preserved by the STRIPS assumption.

B: Pre: P, Q, RAdd: TDel: Q, R

C: Pre: P, QAdd: U, VDel: P, Q

D: Pre: P, TAdd: UDel: P

E: Pre: S, TAdd: V, WDel: S, T

A: Pre: PAdd: QDel: P

P, Q, R

Q, R, S

P, T

R, U, V

T, U

V, W

X

Initial state

A

B

C

S, T

A

E

D

A state satisfying goals



4.5) raises the question of how much modelling can impact onsearch efficiency. Other AI research communities have focusedon exploring the extent to which modelling choices can expe-dite the solution of a problem [Borrett/Tsang 2001] [West-fold/Smith 2001]. Certainly, the more human expertise isembedded in the model the less discovery has to be made by thesolver (whether planner, constraints problem solver, schedul-ing algorithm or other), but the burden on the human expert canbe prohibitive. The correctness of the reasoning systemdepends fundamentally on the correctness of the model, soerrors in the modelling can be catastrophic. The traditionalapproach taken in planning has been to limit this burden as faras possible, providing a standard means for modelling ofaction-centred behaviour and placing the problem-solvingemphasis on automated techniques.

The complexity of the search problem, and the difficulty ofidentifying powerful and general search control methods, hastended to limit the scope for fielded applications of planningtechnology. A central problem is that all decision making issuesare thrown into the same brute-force, often unexpected, searchstrategy and a planner quickly becomes unable to cope with thebranching factor of the problem. This approach, has been ques-tioned by researchers addressing real, and often highly struc-tured, problems. Indeed, some researchers [Ingham et al. 2001]have argued that, whilst a planner must be left free to discovernovel solutions to some parts of a problem there will always beother parts the solutions of which can be prescribed quite close-ly. Furthermore, a problem expert might be able to provide anoverall flexible plan structure which would guide and constrainthe search behaviour of a plan generation system whilst leavingmany choices to be resolved in closer contact with an execu-tive. The standard languages for representing planning prob-lems do not provide this facility, but researchers in the relatedfield of model-based reasoning have designed languages, suchas RMPL [Ingham et al. 2001], which support this purpose andmight provide a basis for combining the power of plan genera-tion with the ability to exploit the knowledge of a problemexpert.

The Development of Planning ResearchA wide variety of planning algorithms has been developed

since STRIPS, using different search strategies and exploringdifferent search spaces. In the 1970s and 1980s effort was fo-cused on puzzle-like problems, such as the blocks world, tile

puzzles and the towers of Hanoi, all of which involve highlyinter-dependent actions and constraints. The objective was tofind optimal plans (plans that would achieve the goal stateusing as few action instances as possible) and planningapproaches were considered flawed if they were fundamentallyunable to achieve optimality in particular puzzle domains. Forexample, the well-known Sussman anomaly led to the develop-ment of least-commitment planning as an alternative to thesequential planning style of STRIPS. The Sussman anomaly(depicted in Figure 4) is a feature of the blocks world demon-strating that goals cannot always be decomposed. Similarly, thetowers of Hanoi problem shows that progress cannot always bemade monotonically.

Although considerable progress was made in the develop-ment of planning algorithms, and in the understanding of theirformal properties, by means of consideration of these puzzles,planning did not make much progress towards practical appli-cation during these years. Indeed, the wider AI communityraised questions about the utility of classical planning given itsapparent inability to dispose even of puzzle problems of littleor no practical interest. During this period work began toemerge on the impressive results that could be obtained in thesolution of practical problems by planners equipped with prob-lem-specific knowledge [Currie/Tate 1991] [Wilkins/desJar-dins 2000] [Stefik 1981]. This gave rise to a tension in theplanning research community between classical, domain inde-pendent, and application-oriented planning. The advantages oftaking a classical approach is that the techniques developed arecompletely general and reusable, whilst the advantages of theapplication-oriented approach is that excellent (though notreusable) results can be obtained in specific problem domains.

This tension has always been constructive because it hashelped to drive classical planning research towards the solutionof ever more realistic problems. In the last ten years incrediblestrides forward have been made in terms of the complexity ofproblems that can be addressed, the efficiency with whichsolutions can be generated and the quality of those solutions. Inthe early 1990s no classical planner could produce plans ofmore than, say, thirty steps. Ten years later such planners canproduce plans consisting of hundreds of steps in a fraction ofthe time it used to take to produce plans an order of magnitudeshorter. Furthermore, modern planners are capable of handlingproblems that are much closer to practical utility than thepuzzle problems of old.

4

FreeCell 4 cards per suit13 cards per suit

1.1 x 1011 initial states1.75 x 1064 initial states

Logistics Largest problem in 1st IPCLargest problem in 2nd IPC

(fully automated)

3 x 1023 states (solved in more than 13 minutes)

2 x 1087 states (solved in 80 seconds)

Figure 3: The table gives a very brief indication of the sizes of planning problem state-spaces. FreeCell is a problem based on the well-known solitaire card game, introduced as a planning benchtest in the 2nd International Planning Competition (IPC) [Bacchus 2000]. The Logistics problem is a commonly used benchtest for planning systems. It is not easy to estimate the size of the state space in all problems. For FreeCell the number of essentially distinct initial states gives an impression of the size of the state space. In the Logistics domain it is easier to compute the number of different reachable states. It is interesting to see the contrast in the performances between the 1st IPC (1998) and 2nd IPC (2000).



One of the most important forces driving these developmentshas been the biennial planning competition which started in1998. The first competition was organized and run by DrewMcDermott in consultation with an international committee ofexperts whose task it was to standardize the representationlanguage to be used by planners and to define a range of benchmark problems. This competition led to the emergence of thePDDL family of domain description languages. The benchmark problems used in 1998 still owed much to the puzzleproblem heritage, although the Logistics domain, popularizedby Kautz and Selman [Kautz/Selman 1998], was used to beginto push planners towards the solution of problems with a morepractical emphasis. Only five competitors participated in thiscompetition, but it led to an enormous resurgence of interest inclassical planning and its potential for application. One of themost interesting consequences of the 1998 competition wasthat emphasis on plan optimality began to give way to a will-ingness to trade off optimality for speed of plan generation.Optimality is a property of plans that is too hard to check, andbecause of the difficulty of producing guaranteed optimalplans, plans produced by systems that do not guaranteeoptimality cannot be evaluated in terms of their distance fromoptimal. However, the heuristic forward search planner HSP[Bonet et al. 1997] emerged as a very exciting future planningtechnology because of its ability to find “acceptable” plans veryquickly (where a plan might be considered acceptable if it is nomore than, say, ten per cent longer than competitive plans).

This led to an intense period of research activity into heuristicforward search.

The two following competitions, in 2000 and 2002, intro-duced a range of new bench marks emphasising practical prob-lem features. In particular, in 2002 problems involving numericand temporal reasoning were introduced and planners wererequired to reason with intervals of time and the consumptionand production of numeric resources. Some of the bench marksintroduced in 2002 began to closely approximate problems ofreal practical interest. For example the Timed Rover domain,shown in Figure 4, closely models the planetary rover explora-tory problems being researched into by NASA and other spaceagencies (see also section 6). Planning has moved from being apuzzle-solving technology to being the foundation of autono-mous behaviour.

In the following sections we review some of the advances inplanning algorithms that have supported the developmentsoutlined above.

4.1. Graphplan Graphplan [Blum/Furs 1995] excited a great deal of interest

when it was introduced because it constitutes an approach toplanning that was radically new at the time (in the early 1990s).Graphplan constructs and then searches a compact reachabilityanalysis of the problem state space. The compactness of therepresentation, together with the informative nature of the datathat can be accumulated during both construction and search,

Figure 4: Examples of planning domains. (i) Simple blocks world (the problem shown is Sussman’s anomaly). (ii) The towers of Hanoi problem. Constraints require that the discs move between pegs without ever placing a larger disc on a smaller one. (iii) Logistics domain: packages must be transported between locations. The airplanes are constrained to move between airports, while the trucks are restricted to local regions. (iv) Depiction of rovers domain: differently equipped rovers rove around planet surface, examining sites of interest and recording data from experiments. Data is communicated back to a lander and from there to earth. Constraints limit data storage, fuel use and recharging, rover capabilities and so on.



leads to performance that far outstripped that of contemporaryplanning search strategies. The success of Graphplan led to thedevelopment of a number of Graphplan re-implementationsand extensions [Koehler et al. 1997] [Long/Fox 1999] andsome of its contributions remain very important today(although its planning performance has been surpassed).

Graphplan searches for a plan in two stages. The first stage isthe construction of a data structure, the plan graph, that effi-ciently represents information about what the executive couldpossibly achieve by executing actions from the initial state. Thesecond stage searches, backwards from the goals, for a sub-structure within the plan graph that represents a subset ofactions that will actually achieve the goals. An importantpreparatory step used by Graphplan, that has become acommon first step for many planning systems, is the groundingof all the actions. This is the process in which all the descrip-tions of parameterised actions are instantiated with all possiblevalues for the parameters selected from the objects in theproblem instance. Grounding for large problem instances canbe a memory intensive operation, but clever compact encodingscan allow tens of thousands and even hundreds of thousands ofaction instances to be constructed and stored efficiently.

The important information that is captured within the plangraph is the collection of propositions that could, individually,be made true after application of increasing numbers of actions.In addition, the graph shows which pairs of propositions aremutually incompatible. That is, pairs of facts that cannot bothbe made true in a reachable state of the world. This might bebecause the facts are simply inconsistent (such as that a door isboth open and closed) or it might be because achieving both ofthem requires execution of more actions than the graph current-ly allows. As the graph is extended by adding the effects ofexecution of more actions, these pairs of facts will becomecompatible. Once all the facts that form the goal set for theplanning problem appear in the graph and are pairwise compat-ible, the search phase commences.

Graphplan search uses several additional techniques toimprove efficiency, but is essentially an exhaustive backwardsearch from the goals, looking for achieving actions that havecompatible preconditions. The search is carried out depth-first,but is limited by the number of actions that the graph data struc-ture contains. If no plan is found, the graph is extended and anew search commences. This process makes Graphplanperform an iterated depth-first search for a plan. This searchstrategy combines the usual benefits of depth-first search (lowoverhead in memory consumption) with the important benefitof a breadth-first search, which is that it will find the shortestsolution. A failing of this search strategy is that it repeatedlyexplores the same set of choices as the depth bound increases.This failing impacts on Graphplan performance, so that if aplan cannot be found in the first few iterations of the searchthen it is often the case that Graphplan will exceed its resourcebounds before finding a plan.

4.2. Heuristics Search PlanningFollowing the significant success of Graphplan, interest in

the planning problem was revitalised and other new ideas were

explored. A very influential direction was initiated by work ofMcDermott in the planning system UNPOP [McDermott 1996]and Geffner and Bonet in HSP [Bonet et al. 1997]. The ideabehind this work is to use a classic heuristic guided search. Thisis the search strategy in which the choice between alternativesis made by evaluating each alternative using a heuristic evalua-tion function and then selecting the best valued option. Thereare several ways in which the heuristic guidance can beapplied, such as the well-known A* search, hill-climbingsearches, best-first searches and so on [Russell/Norvig 1995].Using heuristic search was not new in planning – many plan-ners, such as UCPOP [Penberthy/Weld 1992], allow a heuristicfunction to be used to guide the search. However, heuristicsearch had not previously seemed very promising. The difficul-ties that had been encountered by earlier planners attempting toexploit heuristic choice arose from the fact that the heuristicfunction was usually encoded by hand and it was often difficultto construct a function that could reliably guide the planner tomake all the right choices.

The novel contribution made by McDermott and by Geffnerand Bonet was to demonstrate a method by which a surprising-ly informative heuristic function could be constructed automat-ically, simply by analysing the domain. Although the details ofthe techniques differ, the underlying approach is the same: theheuristic value of a particular choice of action is based on anestimate of how much work remains to be accomplishedfollowing the addition of that action to the plan. To estimate theoutstanding work a very simple, yet very effective, measure-ment is made: the number of actions required to achieve all theoutstanding goals if the destructive effects of those actions areignored. Achieving goals using actions whose destructiveeffects are ignored is called relaxed planning. The measure ofoutstanding work is simply the size of a relaxed plan to achievethe goals. Unfortunately, finding an optimal relaxed plan is,perhaps surprisingly, technically as hard as finding a real plan[Erol et al. 1995]. Fortunately, however, it is relatively easy tofind arbitrary relaxed plans, and even to find “good” relaxedplans. The work inspired by McDermott and Geffner andBonet, then, uses efficient techniques to construct good relaxedplans which are treated as reasonably accurate measures of thework required to complete a plan if a given choice of action ispursued.

One of the most efficient planners based on this approach isthe FF system developed by Hoffmann [Hoffmann/Nebel2000]. This planner uses a relaxed plan graph (built usingactions with their destructive effects ignored) and an efficientplan graph search to find good relaxed plans. Using thisapproach yields a slightly different heuristic estimate from thatexploited in HSP, and the results seem slightly better. The heu-ristic exploited by HSP is admissible (it never over-estimatesthe distance to the goal) whilst the plan graph based heuristicof FF is inadmissible because it relies on extraction of a relaxedplan from the plan graph, and there is no guarantee that the planextracted will be the shortest one available. Geffner and Bonethave explored a wide variety of variants on the relaxed planheuristic [Bonet/Geffner 1997]. Hoffmann has carried out athorough exploration of the way in which the optimal relaxed



plan relates to the optimal real plan in a variety of bench markdomains, allowing the identification of several topologicalfeatures that govern the success or failure of the relaxed plan-ning heuristic as a guide towards real plans.

4.3. Transformation of Planning ProblemsA different direction has been explored by several research-

ers: the reformulation of a planning problem into a form that isamenable to solution using a technology developed in a differ-ent research field. There are at least three examples of thisapproach: the SAT-planners, in which a planning problem isconverted into an instance of a SATISFIABILITY problem andsolved using an efficient SAT-solver [Kautz/Selman 1995]; theCSP approach, in which a planning problem is reformulated asa constraint satisfaction problem and solved using a CSP-solver [vanBeek/Chen 1999] [Binh/Kambhampati 2000] andthe model-checking approach in which a planning problem isconverted into a large logical formula that can be model-checked [Edelkamp/Helmert 2000] [Cimatti/Roveri 1999] (thisis actually a variation on the SAT-planning approach, but usingsufficiently distinct technology for checking the formula that itis generally seen as a direction in its own right).

The SAT-solving approach has been explored most exten-sively in the Blackbox system developed by Kautz and Selman[Kautz/Selman 1995]. The approach exploits the expressivepower of the SATISFIABILITY problem. Briefly, this problemcan be summarised as that of attempting to determine whetherthere is an assignment of truth values to propositional variablesappearing in a given propositional formula that will make theformula true. The expressive power of propositional formulaeis such that it is possible to express the structure and constraintsof a planning problem. There is an interesting complication,which is that a SAT-instance can only express the possibility oftheir being a plan of a maximum given size. If the formulacannot be satisfied then a new formula must be constructed(usually by a straightforward extension of the existing formula)to express the possibility of there being a longer plan. Thus, theproblem of finding a plan becomes not one instance of SATIS-FIABILITY, but a sequence of instances as the search attemptsto find a plan with iteratively increasing numbers of actions.The Blackbox system encompasses several different SAT-solv-ing techniques, each configurable with different parameterscontrolling their behaviour, but the underlying translation of aplanning problem into a SATISFIABILITY instance is via aplangraph. Two extensions of SAT-planning have exploredextended expressive power that encompasses numeric valuedexpressions: LPSAT combines a SAT-solver with a linear-constraint solver to handle a restricted (but very powerful)collection of numeric-valued expressions [Wolfman/Weld1999] and constraints and a version in which an integerprogramming solver is combined with a SAT-solver has alsobeen tried [Vossen et al. 1999]. Integer programming is a simi-lar optimization of numeric valued expressions, subject toconstraints, to that tackled in linear programming (linear con-straint satisfaction). The difference lies in the added constraintthat variables take integer values instead of real values – anapparently minor addition, but one that has surprisingly far-

reaching consequences for the difficulty of the search for asolution.

The CSP approach has been explored by van Beek[vanBeek/Chen 1999], using hand-coded translations of plan-ning problems for his CSP solver. More recently, Binh Do andKambhampati have shown [Binh/Kambhampati 2000] thatautomatic translation to CSP is possible, using a Graphplan-style plan graph as an intermediate structure. CSPs consist ofcollections of variables, each with an associated domain ofpossible values, and a collection of constraints that specify howvalues of variables must interact. A solver searches for assign-ments of values to the variables that are consistent with theconstraints. Described at this abstract level, it is not hard to seehow CSP and SATISFIABILITY can be closely related to oneanother – the assignment of truth values to propositional varia-bles is a restricted form of the assignment of domain values tovariables in a CSP. The CSP community has explored a varietyof techniques for efficient CSP solving [Tsang 1993] and itmight be hoped that these techniques have something to offerto planning. There probably are many lessons that can belearned within the planning community from the CSP commu-nity, but it is now apparent that, as with other reformulationtechniques, the sacrifice of structure from the planning problemis very hard to repay with performance benefits in the technol-ogy that is made accessible for solving the newly expressedproblems.

The third approach, model-checking, was explored by sever-al of the competitors in the competition in 2000[Edelkamp/Helmert 2000] [Hölldobler/Störr 2000]. The idea inthis approach is to reformulate the planning problem as a logi-cal formula that can be expressed as a binary decision diagram– a particularly compact representation for manipulating suchformulae. Once again, the reformulation gives access to generictechnology for solving problems in the form of the reformulat-ed problem. In this case, an efficient model-checking approachcan be used to determine the status of the formula. Thereremains a very active application of the approach in the solu-tion of planning problems that involve actions with uncertainoutcomes (see, for example, [Cimatti/Roveri 1999]).

4.4. Hybrid PlanningExperimental results demonstrate that there is no single plan-

ning strategy suited to all planning problems. All of theapproaches discussed above display a wide range of perform-ance over different problems and the extent to which a particu-lar strategy is suited to a particular problem is determined bythe structure and organization of the problem. For this reasonhybrid planning approaches can be very successful. However,determining automatically which of the available strategies ina hybrid to invoke remains a difficult research problem.

Edelkamp and Helmert’s MIPS system [Edelkamp/Helmert2000] combined a Graphplan strategy with a model-checkingapproach, using time bounds to determine which of the strate-gies to apply to a problem. The Graphplan strategy was triedfirst, and if no solution was found within the time bound theproblem would be reformulated as a model-checking problem.Similarly, the BLACKBOX system [Kautz/Selman 1995] is



able to invoke a range of different SAT-solving techniques andthese are tried successively until a solution is found (orresource bounds are exceeded).

Sometimes no generic search strategy is suited to the solutionof a problem and, instead, some specialised solver should beinvoked. A direction that has been explored in some recentwork in planning [Fox/Long 2001] [Long/Fox 2000], is theidea of harnessing specialised solvers capable of efficientlytackling particular combinatorial problems, such as the well-known Travelling Salesman Problem (TSP), to support ageneral planning engine. The motivation for this is that manyhard specialised problems have themselves been the subject ofresearch and good solutions exist for them, while a genericproblem-solving technology is very unlikely to challenge thesespecialised solutions when applied to these problems. It is oftenthe case that these specialised problems appear as sub-prob-lems within a larger planning problem. For example, a problemthat involves transporting components between locations,constructing complex artifacts with the components and thendelivering them to customers can be seen to contain route-planning, resource allocation, job-shop scheduling andconstruction planning sub-problems. Each of these can, tosome extent (although not entirely) be decoupled from theothers and solved using specialised technology.

Several examples have now been demonstrated in which asuccessful automatic identification of sub-problems iscombined with automatic configuration of specialised prob-lem-solving technology and solution of planning problemsusing these specialised solvers [Long/Fox 2001] [Clark 2001][Long et al. 2000] [Long/Fox 2002]. Another planner exploit-ing a similar architecture, although without automatic problemdecomposition, is REALPLAN [Srivastava 2000], whichfactors out resource-scheduling sub-problems for specialisedtreatment. The decomposition approach is of interest because itoffers a way in which advances in particular problem-solvingtechnologies might be exploited without impacting on the taskof encoding the planning problem – automatic identification ofsub-problems relieves the domain-engineer of the requirementto understand the characteristics of the sub-problems and alsoof the obligation to understand the restrictions on the capabili-ties of the sub-solvers or of the ways in which collections ofsolvers might interact in deployment.

4.5. Planners Exploiting Hand-Coded Control Knowledge Although the development of fully-automated planning

systems, which do not rely on advice from human domain-experts to guide their search, has been the focus of most of thesurvey so far, progress in planners exploiting hand-codedcontrol knowledge has been equally dramatic in the past fewyears. The third IPC highlighted the performance of three suchsystems: TALplanner [Kvarnstrom/Doherty 2000], TLplan[Bacchus/Kabanza 2000] and SHOP2 [Nau et al. 1999]. Thefirst two of these share significant common ground in theirplanning strategy, which is essentially a forward search, explor-ing the states that can be reached by the execution of actionsfrom each state as it is visited, and selecting the next state tovisit using carefully crafted advice provided by a human

domain-engineer. The third system represents a class of plan-ners known as hierarchical task network (HTN) planners. Inthese systems the domain must be described in terms of hierar-chically structured actions, from an abstract level down to aconcrete level of detail. An abstract action connects to one ormore methods, which are possible “recipes” for accomplishingthe abstract task using less abstract tasks. The planner uses aprocess of refinement to construct a plan, replacing abstractactions that achieve its initial goals with their less abstractmethod expansions and then, iteratively, refining the compo-nents of these expansions. We will now consider each of thesetwo families of planners in a little more detail.

4.6. Hand-Coded Control Rules TLplan and TALplanner both rely on control rules which

guide the choices made by the planner as it considers whichaction to select at each step in the plan construction. Both ofthese systems use a temporal logic to express the rules, whichallows the control rules to explicitly refer to logical relation-ships between properties of the states before and after applica-tion of an action, to the goal state, the initial state and also tocollections of states such as all future states or all past states.Control rules can express advice such as “do not load an objectonto a transporter if it is already at the place it needs to reach”,or “if an unloaded object and a transporter are at the same placeand the object needs to go somewhere that the transporter isscheduled to visit, then in the next state the transporter muststill be at the same location and the object must either be loadedinto the transporter or still waiting to be loaded”. It can be seenfrom these examples that control rules can become quite cum-bersome to write, even if they express relatively intuitiveadvice. In particular, the second of these examples is only validif there is a single suitable transporter. If there is more than onetransporter that could be used to transport the object then therule becomes even more cumbersome to express. Someresearch has been conducted on automatically constructingrules in certain cases [Kvarnstrom et al. 2000] [Muñoz-Avila2002], but the encoding of a good set of advice for a domainremains a demanding task.

One of the most significant disadvantages of planners usinghand-coded control knowledge is that the task of writingrequires expertise in both the domain and knowledge of theway that the planner will respond to the advice. Constructingcollections of advice that are both consistent and allow a plan-ner to find all plans that are sought is a complex task, not unlikeprogramming. The benefits are considerable, however: TLplanand TALplanner produced plans of a consistently high quality,very fast, for a huge range of problems presented to it in thethird IPC. TLplan can plan with durative actions, exploitingconcurrency, and with numbers. TLplan is regularly used bygraduate students, who have constructed representations of awide variety of problem domains and achieved successful plan-ning performance with the system, demonstrating that domainconstruction is not a skill restricted to a small elite. Neverthe-less, the burden of modelling domains for TLplan has not beenproperly explored and it is interesting to observe that Bacchus



and Ady were unable to complete an encoding of the finaldomain for the third IPC due to time constraints.

4.7. HTN Planning HTN planning has been an active area of research for more

than a decade, with influential systems including Tate’s O-Plansystem [Currie/Tate 1991] [Drabble/Tate 1994] and Wilkin’sSIPE [Wilkins 1988]. SHOP2 [Nau et al. 1999] [Nau et al.2001] represents a particularly straightforward implementationof an HTN system. In contrast to the approach adopted in theplanners that use hand-coded control rules, HTN plannerscannot rely on the primitive actions available in the domain,alone, but must have these supplemented with constructions –abstract actions – that represent sequences of these primitives.The use of a hierarchy of abstractions allows alternative choic-es about the precise sequence of primitive actions that is usedto realise a given abstract action, by offering different expan-sions for abstract tasks. Thus, the search problem confronted byHTN planners is not the typical search between alternativeprimitive actions that is managed by other planners, but achoice between the particular refinements used to realiseabstract tasks.

HTN planning is a confounding mixture: on the one hand itis highly intuitive to exploit abstraction in planning, simplify-ing a planning problem by identifying strategic structures tosolutions before attempting to refine them into detailed activity.On the other hand, it is a common criticism of HTN systemsthat it seems that most of the planning problem is solved by thehuman domain engineer and that the “planner” is relegated to apurely administrative role of coordinating the recovery of plancomponents from a database. This is a simplistic view –domain constructions for SHOP are demanding to construct,requiring knowledge of both the planner and the domain, butnevertheless, the domain encodings represent specialisedprograms and the planner is certainly required to execute morecomplex activity than pure database look-up.

What is frustrating to many researchers is that the intuitiveappeal of abstraction as a route to efficient planning seems tobe submerged under a significant amount of problem-specificspecialised techniques, some of which are not at all intuitiveand do not fit easily into the HTN machinery. For example,managing the order in which goals are considered is an impor-tant element of the efficient heuristic control of all planners,including HTN planners. The mechanism by which this ishandled in SHOP is by defined artificial extensions to the pure-ly physical elements of the original problem that encode goalagenda management actions and the organisation of the agendais handled by use of these problem-specific actions.

The exploitation of abstraction remains a controversial andtantalising theme in planning research: it seems very reasona-ble that some form of abstraction should help to make planningmore efficient, but there remains work to be done to identifymechanisms for doing so that are flexible and convenient toexploit while realising their promise of efficiency gains.

Modern Planning Systems In this section we will review some of the modern planning

approaches that currently seem particularly promising and thatexhibit particularly interesting features. The enriched expres-sive power of PDDL2.1 [Fox/Long 2002], used in the 2002planning competition (the third IPC), makes the range of prob-lems that can be modelled far more interesting and far closer toreal applications than previously. Some of the recently emerg-ing planning approaches are capable of handling problemsrequiring at least the expressive power of PDDL2.1.

A search strategy that has recently entered the planningrepertoire is local search. The planner PbR [Ambite/Knoblock1997] was one of the first planners to demonstrate that planscould be obtained by means of iterative repair performed on aflawed initial plan, using rewriting rules selected by efficientlocal search techniques. ASPEN [Chien et al. 1999] [Rabideauet al. 1999] also constructs plans by means of local search-based iterative repair. This idea is explored in LPG [Gerevi-ni/Serina 2002] using locally extending plan graphs. LPG doesnot handle arbitrary use of numbers, but manages actions withduration and concurrency. LPG replaces the Graphplan searchwith a far more efficient and more powerful local searchtechnique. This involves identifying an initial candidate “plan”(which might, in fact, not even be executable) and refining it bygenerating alternative possible repairs or modifications to thecandidate. Evaluation of alternative refinements is carried outby a heuristic evaluation function that is partially based on therelaxed plan heuristics used in HSP and FF. LPG demonstratesthat there is significant potential in this technique coupled withtechniques capable of exploiting the reachability structure ofplanning problems.

Heuristic search planning remains a powerful strategy. Anextended version of FF, metric-FF [Hoffmann 2002], canhandle numeric expressions and typically generates good qual-ity solutions while maintaining its speed. The extension of theheuristic evaluation function to handle numbers essentiallyinvolves estimating how actions can contribute towards movinga number-valued expression towards its target threshold value.In the same way that the destructive propositional effects ofactions are ignored in generating the relaxed plan heuristicestimates of distances to goals, so in the metric version of FFthe “destructive” effects of actions that move numeric values inthe “wrong” direction are ignored in making the heuristic esti-mates of numbers of actions required to complete a plan. Thisapproach relies on numeric expressions being linear forms.Although it performs extremely well in many bench mark prob-lems it does not perform well in problems in which the numericexpressions represent complex interlocked resource manage-ment behaviour and it seems likely that the relaxation of prop-ositional parts of a plan is more widely applicable than therelaxation of the numeric parts. FF does not handle actions withduration and its current architecture does not obviously gener-alise to support construction of plans with concurrency.

An interesting system, which extends the notion of forwardsearch to planning in temporal and resource intensive domains,is SAPA [Binh/Kambhampati 2001]. SAPA uses a heuristicevaluation function based on a relaxed temporal plan, together

5



with a technique for estimating the implied cost associated withsupplying necessary resources for the relaxed plan. Thelanguage SAPA uses allows the representation of actions withduration, the effects of which can happen at arbitrary pointswithin the durative interval. The current implementation ofSAPA allows effects to be associated only with the end pointsof these intervals, because computing interactions betweenarbitrary time points is computationally expensive and compli-cates the reasoning mechanisms of the planner.

Most of the fully automatic planning systems that have beendeveloped in the past five years rely on grounding of actions.As previously mentioned, although this might seem excessive-ly memory-intensive, in practice, using compression tech-niques, it has proved possible to manage large numbers ofactions. However, there is clearly a question about the scalabil-ity of this technique in the face of problems with large numbersof objects. Although such problems might not seem likely toarise in practice, in fact, where a domain requires ranges ofnumbers as objects there can be very large collections ofobjects in a problem instance. One planner in the third IPC,VHPOP [Younes/Simmons 2002], demonstrated an interestingapproach to compromising between grounding, and the poten-tial for efficiency it offers, and leaving parameters unbound,which offers the benefits of far more compact representation.

Bridging the Gap between Research and Application Planning technology has been applied to a wide variety of

problems. For example manufacturing processes [Nau et al.1995] [Gupta 1998], satellite and spacecraft operations plan-ning [Muscettola 1994] [Muscettola et al. 1998] [Smith et al.1999], bridge play [Smith et al. 1998], chemical plant start-upplanning [Aylett et al. 1998], elevator scheduling [Koehler1998], evacuation planning [Muñoz-Avila et al. 2001] repre-sent only some of problems to which planning has beensuccessfully applied. However, appliedplanning systems are knowledge-intensivesystems, requiring significant technical in-put from both domain-experts and, perhapsprimarily, from planning experts. To makeplanning a more accessible and widely usedtechnology, domain-independent systemsoffer a route past the bottleneck of planningsystem expertise. In this section we consid-er the extent to which domain-independentplanning technology is reaching a maturitysufficient for broader and realistic applica-tion.

Simplifying assumptions of classicalplanning, described in section 3, have beenprogressively relaxed in modern systems.One of the stated goals of the third IPC wasthe encouragement of a broader commit-ment to more sophisticated planning capa-bilities. The development of a much richerexpressive power for logical preconditionsthan pure STRIPS was considered fromrelatively early days [Pednault 1989] and

achieved in least-commitment planners [Blum/Furs 1995], aswell as more recent systems such as the Graphplan-based IPP[Koehler et al. 1997] and heuristic-forward-search planner FF.Preprocessing of domain encodings has proved a powerful andsuccessful approach to managing this form of extended expres-siveness. This power was tested in a range of systems in the firstand, particularly, the second IPCs. In the second IPC, a domainbased on the scheduling of elevator use (Miconics-10 [Koehler1998]) exercised a wide range of the expressive power of thisfragment of PDDL.

In the most recent competition, the expressive power ofPDDL has been extended with the explicit intention of breach-ing some of the traditional restrictions of classical planning. Inparticular, domains used in the third IPC make use of numeri-cally measured resources and other numeric values. They alsomodel the temporal structure of actions in the domains,including the duration of actions that can be executed. Theinclusion of actions with duration implicitly introduces theneed for planners that can manage and exploit concurrency,including recognising harmful interactions between concurrentactivities even if they simply overlap, rather than synchronisetheir start or end points. Furthermore, plan-metrics have beenadded to the language, so that it is possible to identify howplans should be evaluated. The impact of this extension shouldnot be underestimated: it offers the power to harness planningfor practical use in a way that is simply impossible if the onlymeasure of plan quality is the number of steps in the plan. Inalmost every real planning application the cost of resourceconsumption, including the actual time over which the plan isexecuted, possibly offset by the profit generated by the plan, isan essential measure of plan quality, while number of steps isof limited interest.

These extensions in expressive power made it possible tointroduce several domains in the third IPC that make an inter-

6

Figure 5: Complex Satellite variant, plan quality: smaller values are better plans.



esting and convincing step towards real application domains.One domain extends the logistics domain by the addition oftime for travel between locations, time for loading and unload-ing and fuel consumption. With these additions, plan qualitycan be measured as a tradeoff between the total time requiredto complete all the deliveries and the fuel consumed in doingso. The extended domain also includes alternative means oftravel, offering a choice between slow, fuel-efficient travel andfaster, fuel-hungry travel. This choice is an important one whenthe plan is to be evaluated using combined costs of duration andfuel consumption.

A second realistic domain is that of planetaryexploratory-rovers, based on the Mars exploratoryrovers mission due for 2003 launch. In the compe-tition model of this domain rovers are equippedwith different equipment and have different capa-bilities for traversal of the terrain. The rovers mustcollect data and downlink it to a lander (for subse-quent retransmission to earth). Terrain blockscommunication between certain locations, makingit necessary to plan how to efficiently visit experi-ment sites, store data and communicate it aftermoving to a site from which transmission to thelander is possible. Rovers consume energy duringtheir activities and must recharge to remain active.Plan costs are measured in terms of the time andenergy spent acquiring the data, making efficientconcurrent use of rovers important for plan quality.

A third domain introduced is modelled after theproblem of satellite observation scheduling [Smithet al. 2000]. This problem involves using a collec-tion of differently equipped satellites to recordobservations, storing the data in a finite capacityon-board store for subsequent downlink. Carefulchoice of calibration targets to be used in setting

up instruments is an important aspect of this prob-lem, as well as efficient management of the datastore and of fuel in slewing the satellite betweenobservation and calibration targets. One version ofthis domain used in the competition required plan-ners to solve the problem of maximising acquireddata, without being presented with any logicalgoals at all. This is a dramatic contrast to the clas-sical planning problem formulation and demon-strates one of the ways in which plan metricsstretch the scope of the planning problem.

Although there remain aspects of the real prob-lems on which these domains are modelled that arenot yet captured, or are even impossible to capturewithin the PDDL language as it currently stands,the performance of planners on these problems andthe sophistication of the models demonstrates thedistance that planning has progressed in recentyears. Some of the outstanding issues are discussedin section 7. We end this section by consideringdata demonstrating the performance of planners inthe third IPC on some of the problems in various

versions of the satellite domain. Figure 5 shows plans produced for a variant of the satellite

domain using both numeric and temporal features. Here the ob-jective was to minimize the time for execution of the plan.Figure 6 shows how fast these plans were produced (on a1800MHz Athlon CPU PC, with 1Gb RAM), while Figure 7shows that these plans contain well over 100 steps in somecases. Note that the planners using hand-coded controlsproduced longer plans, reflecting the exploitation, by thedomain-engineers, of certain sequences of actions that do notobey a “triangle inequality”: for some actions A, B and C,

Figure 6: Complex Satellite variant, planning speed: note log-scale.

Figure 7: Complex Satellite variant, plan length. This plot shows the size of the plans produced, placing the planning times in context. Note that the fully-automated planners typically produce plans with fewer steps.



actions A and B achieve a goal that is otherwise achieved byaction C, but the sum of the lengths of actions A and B is small-er than the length of action C. The automatic planners tend tofavour selecting single action solutions to goals when they can,even though this is not always optimal.

Issues for Further Research As we have seen, planning has moved on a long way from

its STRIPS roots. Planners can now handle problems with timeand numbers, allowing the expression of complex mixedscheduling and planning problems, demanding concurrencyand resource management. In addition, the modelling languageallows expression of plan metrics, so that planners can seek tooptimise the plans they produce against a more useful measureof value than simple plan length. In the 2002 planning compe-tition the use of this metric confirmed that several planners cantailor their performance towards production of plans of higherquality. Using the plan metric it is even possible to constructplanning problems in which the objective is not to achieve anyparticular logical goal state, but to optimise the value of somemeasure of utility subject to the selection of actions forming theplan. For example, in the competition a problem was posed inwhich satellite observations should be collected into finitecapacity stores on board satellites. No specific observationtargets were required to be stored, but the value of the plan wasdetermined by the total amount of stored data achieved by theplan. This kind of problem represents a radical new challengefor the planning community. MIPS [Edelkamp 2002] was theonly fully automated planner that managed to produce plansthat acquired data. The hand-coded systems, TLplan andSHOP2, both produced very high-quality plans, apparentlyclose to optimal in the majority of problems; but this perform-ance relied on a human expert encoding the problem so thatnumeric rather than logical effects would be prioritized in thesearch process. The use of plan metrics is clearly an importantchallenge for the further development of fully automatedplanning systems, perhaps being used to supplement the guid-ance heuristic evaluation functions used by heuristic searchplanners.

The existing PDDL standards do not support the modellingof exogenous events. As discussed earlier, classical planningmakes the assumption that no change can occur that is notunder the direct control of the planner. However, in many real-istic situations it is necessary to plan around the fact that uncon-trollable changes will occur. For example, in the full satelliteobservation planning problem, it is vital to represent the factthat opportunities for making observations and for downlinkingdata both arise in time windows that are not under the controlof the executive (and, therefore, cannot be planned by theplanner), since they arise as a consequence of the process oforbiting the earth and the events of transition above or belowthe horizon for each opportunity location that are caused by thisprocess. Some planners in the application-oriented tradition[Muscettola 1994] [Laborie/Ghallab 1995] are capable of plan-ning with foreknowledge of such events. When such events canbe anticipated they can be encoded as constraints which can bethrown in with all the other constraints that describe a problem

and then any solution that emerges will respect the restrictionsthat they impose. However, planning strategies not based onreformulation of the problem into a CSP have not yet success-fully tackled planning with predictable exogenous events.

When exogenous events are not entirely predictable they giverise to uncertainty in the planning problem. Although manyresearchers have considered the problem of planning underuncertainty in various forms, there remain many questions toresolve. There is not yet a clear consensus even on the form ofa plan when managing uncertainty. It seems that uncertaintyarises in several different forms. Unpredictable exogenousevents give rise to a form which can be difficult to plan with –if unexpected changes are occurring frequently the resultinguncertainty can undermine any planning effort. More benignforms of uncertainty also exist. For example, there is uncertain-ty about the precise duration of actions, or consumption ofresources such as fuel by those actions. This uncertainty typi-cally can be described by continuous probability distributions,often normal or close to normal. This form of uncertainty mightbe considered benign in that plans can be made increasinglyrobust to the uncertainty by allowing more resources for theirexecution. Uncertainty about the successful outcome of theexecution of an action might be best described by a discreteprobability distribution. This form of uncertainty is more diffi-cult to manage because it leads to a significant branching in thepossible states of the world after execution, making it very hardto produce a robust plan without introducing contingentactions.

In addition to uncertainty, many real domains demand that aplanner should manage complete ignorance. In this case, theexecutive will typically have access to some form of informa-tion gathering actions and the planner can plan to acquire infor-mation and react to the results. In this situation, and also in thecase of handling uncertainty, it is often the case that to plan forlong sequences of activity is either impossible or else a poorinvestment of effort. A more useful approach to problem solv-ing in this case is to interleave planning and execution, usingexecution-monitoring, failure diagnosis and plan repair tech-niques to resolve the problems that arise during this process.Continuous planning [Knight et al. 2001], in which new goalscan arise as a consequence of discoveries made at executiontime, is an important development of planning taking it furtherin the direction of providing the basis for autonomous behav-iour. These are all areas of active research, but there is nocommonly accepted empirical framework for evaluating suchsystems, or even for describing problems that could be sharedacross the community. Putting problems such as these onto theagenda for the whole planning community is an important rolethat can be played by combinations of the competition series,the reporting of high-profile application areas demanding thesekinds of functionality and the continual striving of the commu-nity as a whole to extend and develop the technology at its core.

Conclusion Planning is a hard problem, even when reduced to the bare

form of the classical planning formulation. The hardness of thisproblem has confounded rapid development in the planning

7

8



research community for a long time, but over recent years thefield has seen significant advances. This progress has allowedthe restrictions of classical planning to be relaxed or overcome.In turn, this has brought extended expressive power anddomain-independent planning is now a technology capable ofapplication to realistic domains.

Over the past decade several new techniques have emergedas promising for future development of classical planningtowards application. In particular, these include heuristicforward search based on informative heuristics that can beautomatically generated and that exploit the structure of theproblem; local search techniques that can be applied to effi-ciently repair a rapidly generated, flawed plan; hybrid systemsthat exploit structure and systems that go beyond the simplenotion of batch planning and can anticipate execution-timediscoveries. In parallel there has been increasing interest inplanners that can reason about expert knowledge of a problem,expressed in terms of search control rules or in terms of skeletalplan structure or HTN-style action decompositions. The gapbetween classical planning and application-oriented planningis narrowing and the scope for exploitation of planning technol-ogy in industry, commerce and scientific application contexts iscontinuously increasing.

References [Ambite/Knoblock 1997]

J.L. Ambite and C.A. Knoblock. Planning by rewriting: efficient-ly generating high-quality plans. In Proceedings of 14th NationalConference on AI (AAAI), 1997.

[Aylett et al. 1998]R. Aylett, J. Soutter, G. Petley, P.W.H. Chung, and A. Rushton. AIplanning in a chemical plant domain. In Proceedings of EuropeanConference on AI ECAI 98, 1998.

[Bacchus 2000]F. Bacchus. The 2nd International Planning Competition homepage. <http://www.cs.toronto.edu/aips2000/>, 2000.

[Bacchus/Kabanza 2000]F. Bacchus and F. Kabanza. Using temporal logic to expresssearch control knowledge for planning. Artificial Intelligence,116(1–2):123–191, 2000.

[Binh/Kambhampati 2000]M. Binh Do and S. Kambhampati. Solving planning graph bycompiling it into a CSP. In Proc. of 5th Conference on AI Plan-ning Systems, pages 82–91. AAAI Press, 2000.

[Binh/Kambhampati 2001]M. Binh Do and S. Kambhampati. Sapa: a domain-independentheuristic metric temporal planner. In Proc. ECP-01, 2001.

[Blum/Furs 1995]A. Blum and M. Furst. Fast Planning through Plan-graph Analy-sis. In Proc. of 14th Internation Joint Conference on AI, pages1636–1642. Morgan Kaufmann, 1995.

[Bonet/Geffner 1997]B. Bonet and H. Geffner. Planning as heuristic search: newresults. In Proc. of 4th European Conference on Planning (ECP).Springer-Verlag, 1997.

[Bonet et al. 1997]B. Bonet, G. Loerincs, and H. Geffner. A robust and fast actionselection mechanism for planning. In Proc. of 14th NationalConference on AI, pages 714–719. AAAI/MIT Press, 1997.

[Borrett/Tsang 2001]J. Borrett and E.P.K. Tsang. A context for constraint satisfactionproblems formulation selection. Constraints, 6, No.4:299–327,2001.

[Chien et al. 1999]S. Chien, R. Knight, A. Stechert, R. Sherwood, and G. Rabideau.Integrated planning and execution for autonomous spacecraft. InProceedings of the IEEE Aerospace Conference (IAC), 1999.

[Cimatti/Roveri 1999]A. Cimatti and M. Roveri. Conformant planning via modelchecking. In Proceedings of the European Conference on Plan-ning (ECP99), 1999.

[Clark 2001]M. Clark. Construction domains: a generic type solved. In Proc.of 20th UK Planning and Scheduling Workshop, Edinburgh,2001.

[Currie/Tate 1991]K. Currie and A. Tate. O-plan: the open planning architecture.Artificial Intelligence, 52(1):49–86, 1991.

[Drabble/Tate 1994]B. Drabble and A. Tate. The use of optimistic and pessimisticresource profiles to inform search in an activity based planner. InProc. of 2nd Conference on AI Planning Systems (AIPS). AAAIPress, 1994.

[Edelkamp 2002]S. Edelkamp. Mixed propositional and numeric planning in themodel checking integrated planning system. In M. Fox and A.Coddington, editors, Planning for Temporal Domains: AIPS’02Workshop, 2002.

[Edelkamp/Helmert 2000]S. Edelkamp and M. Helmert. On the implementation of mips. InProceedings of Workshop on Decision-Theoretic Planning, Arti-ficial Intelligence Planning and Scheduling (AIPS), pages 18–25.AAAI-Press, 2000.

[Erol et al. 1995]K. Erol, D. Nau, and V.S. Subrahmanian. Complexity, decidabil-ity and undecidability results for domain-independent planning.Artificial Intelligence, 76(1–2):75–88, 1995.

[Fikes/Nilsson 1971]R.E. Fikes and N.J. Nilsson. STRIPS: A New Approach to theApplication of Theorem-Proving to Problem-Solving. ArtificialIntelligence, 2(3):189–208, 1971.

[Fox/Long 1998]M. Fox and D. Long. The automatic inference of state invariantsin TIM. Journal of AI Research, 9:367–421, 1998.

[Fox/Long 2000]M. Fox and D. Long. Utilizing automatically inferred invariantsin graph construction and search. In Proc. of 5th Conference onArtificial Intelligence Planning Systems (AIPS), Breckenridge,Colorado. AAAI Press, 2000.

[Fox/Long 2001]M. Fox and D. Long. Hybrid STAN: Identifying and ManagingCombinatorial Sub-problems in Planning. In Proc. of 17th Inter-national Joint Conference on AI, pages 445–452. MorganKaufmann, 2001.

[Fox/Long 2002]M. Fox and D. Long. PDDL2.1: An extension to PDDL for ex-pressing temporal planning domains. Technical Report Depart-ment of Computer Science, 20/02, Durham University, UK.Available at: <http://www.dur.ac.uk/d.p.long/competition.html>.Forthcoming in revised form as a JAIR article, 2002.

[Gerevini/Schubert 1998]A. Gerevini and L. Schubert. Inferring state constraints fordomain-independent planning. In Proc. of 16th National Confer-ence on AI, pages 905–912. AAAI/MIT Press, 1998.

[Gerevini/Serina 2002]A. Gerevini and I. Serina. LPG: A planner based on local searchfor planning graphs. In Proc. of 6th International Conference onAI Planning Systems (AIPS’02). AAAI Press, 2002.

[Green 1969]C. Green. Theorem proving by resolution as a basis for question-answering systems. In B. Meltezer, D. Michie, and M. Swann,



editors, Machine Intelligence, volume 4. Edinburgh UniversityPress, 1969.

[Gupta 1998]S.K. Gupta, D.S. Nau, and W.C. Regli. IMACS: A case study inreal-world planning. IEEE Expert and Intelligent Systems,13(3):49–60, 1998.

[Hoffmann 2002]J. Hoffmann. Extending FF to numerical state variables. InProceedings of European Conference on AI (ECAI’02), pages571–575, 2002.

[Hoffmann/Nebel 2000]J. Hoffmann and B. Nebel. The FF planning system: Fast plangeneration through heuristic search. Journal of AI Research,14:253–302, 2000.

[Hölldobler/Störr 2000]S. Hölldobler and H-P. Störr. Solving the entailment problem inthe fluent calculus using binary decision diagrams. In Workshopon Model-Theoretic Approaches to Planning at AIPS2000. Beck-enridge, 2000.

[Ingham et al. 2001]M. Ingham, R. Ragno, and B.C. Williams. A reactive model-based programming language for robotic space explorers. In Int.Symp. on Artificial Intelligence, Robotics and Automation inSpace, 2001.

[Kautz/Selman 1995]H. Kautz and B. Selman. Unifying SAT-based and graph-basedplanning. In Proc. of 14th Internation Joint Conference on AI,pages 318–325. Morgan Kaufmann, 1995.

[Kautz/Selman 1998]H. Kautz and B. Selman. The role of domain-specific axioms inthe planning as satisfiability framework. In Proc. of 4th Confer-ence on AI Planning Systems, Pittsburgh, PA, pages 181–189.AAAI Press, 1998.

[Knight et al. 2001]R. Knight, G. Rabideau, S. Chien, B. Engelhardt, and R. Sher-wood. Casper: Space exploration through continuous planning.IEEE: Intelligent Systems, 16(5):70–75, 2001.

[Koehler 1998]J. Koehler. Planning under resource constraints. In Proc. of 13thEuropean Conference on AI, pages 489–493, 1998.

[Koehler et al. 1997]J. Koehler, B. Nebel, J. Hoffmann, and Y. Dimopoulos. Extendingplanning graphs to an ADL subset. In Proc. of 4th EuropeanConference on Planning, Toulouse, pages 273–285, 1997.

[Kvarnstrom/Doherty 2000]J. Kvarnstrom and P. Doherty. TALplanner: A temporal logicbased forward chaining planner. Annals of Mathematics andArtificial Intelligence, 30(1–4):119–169, 2000.

[Kvarnstrom et al. 2000]J. Kvarnströn, P. Doherty, and P. Hasslum. Extending TAL-planner with concurrency and resources. In Proceedings ECAI-00, Berlin, Germany, August, 2000.

[Laborie/Ghallab 1995]P. Laborie and M. Ghallab. Planning with sharable resourceconstraints. In Proc. of 14th International Joint Conference on AI.Morgan Kaufmann, 1995.

[Long/Fox 1999]D. Long and M. Fox. The efficient implementation of the plan-graph in STAN. Journal of AI Research, 10, 1999.

[Long/Fox 2000]D. Long and M. Fox. Automatic synthesis and use of generictypes in planning. In Proc. of 5th Conference on Artificial Intelli-gence Planning Systems (AIPS), pages 196–205. AAAI Press,2000.

[Long/Fox 2001]D. Long and M. Fox. Multi-processor scheduling problems inplanning. In Proc. of International Conference on AI (IC-AI), LasVegas, 2001.

[Long/Fox 2002]D. Long and M. Fox. Reformulation in planning. In Proceedingsof Symposium on Abstraction, Reformulation and Approxima-tion. LNAI, Springer-Verlag, 2002.

[Long et al. 2000]D. Long, M. Fox, L. Sebastia, and A. Coddington. An examina-tion of resources in planning. In Proc. of 19th UK Planning andScheduling Workshop, Milton Keynes, 2000.

[McCarthy/Hayes 1969]J. McCarthy and P.J. Hayes. Some philosophical problems fromthe standpoint of artificial intelligence. In B. Meltzer and D.Michie, editors, Machine Intelligence 4, pages 463–502. Edin-burgh University Press, 1969. reprinted in McC90.

[McDermott 1998]D. McDermott. PDDL – the planning domain definition lan-guage. Technical report, Yale University,<http://www.cs.yale.edu/users/mcdermott.html>, 1998.

[McDermott 1996]Drew McDermott. A heuristic estimator for means ends analysisin planning. In B. Drabble, editor, Proceedings of the 3rd Interna-tional Conference on Artificial Intelligence Planning Systems(AIPS-96), pages 142–149. AAAI Press, 1996.

[Muñoz-Avila et al. 2001]H. Muñoz-Avila, D.W. Aha, D.S. Nau, R. Weber, L. Breslow, andF. Yaman. SiN: Integrating case-based reasoning with taskdecomposition. In Proceedings of International Joint Conferenceon AI (IJCAI-2001), 2001.

[Muñoz-Avila 2002]L. Murray. Reuse of control knowledge in planning domains. InL. McCluskey, editor, Knowledge Engineering Tools andTechniques for AI Planning: AIPS’02 Workshop, 2002.

[Muscettola 1994]N. Muscettola. HSTS: Integrating planning and scheduling. In M.Zweben and M.S. Fox, editors, Intelligent Scheduling, pages169–212. Morgan Kaufmann, San Mateo, CA, 1994.

[Muscettola et al. 1998]N. Muscettola, P. Pandurang Nayak, B. Pell, and B.C. Williams.Remote agent: To boldly go where no AI system has gone before.Artificial Intelligence, 103(1–2):5–47, 1998.

[Nau et al. 1999]D. Nau, Y. Cao, A. Lotem, and H. Muñoz-Avila. SHOP: Simplehierarchical ordered planner. In Proc. of 16th International JointConference on AI, pages 968–975. Morgan Kaufmann, 1999.

[Nau et al. 1995]D. Nau, S. Gupta, and W. Regli. Artificial intelligence planningversus manufacturing-operation planning: a case study. In Proc.of 14th International Joint Conference on AI, pages 1670–1676.Morgan Kaufmann, 1995.

[Nau et al. 2001]D. Nau, H. Muñoz-Avila, Y. Cao, A. Lotem, and S. Mitchell.Total-order planning with partially ordered subtasks. In Proceed-ings of International Joint Conference on AI (IJCAI-2001), 2001.

[Newell/Simon 1963]A. Newell and H. Simon. GPS, a program that simulates humanthought. In E.A. Feigenbaum and J. Feldman, editors, Computersand Thought. McGraw Hill, NY, 1963.

[Pednault 1989]E. Pednault. ADL: Exploring the middle ground betweenSTRIPS and the situation calculus. In Proc. of 1st InternationalConference on Principles of Knowledge Representation andReasoning, pages 324–332. San Francisco, CA, MorganKaufmann, 1989.

[Penberthy/Weld 1992]J. Penberthy and D.S. Weld. UCPOP: a sound, complete, partial-order planner for ADL. In Proc. Int. Conf. On Principles ofKnowledge Representation and Reasoning, pages 103–114, LosAltos, CA, 1992. Kaufmann.



[Porteous 2001]J. Porteous, L. Sebastia, and J. Hoffmann. On the extraction,ordering, and usage of landmarks in planning. In Proceedings ofEuropean Conference on Planning ECP’01, 2001.

[Rabideau et al. 1999]G. Rabideau, R. Knight, S. Chien, A. Fukunaga, and A. Govind-jee. Iterative repair planning for spacecraft operations in theASPEN system. In International Symposium on Artificial Intelli-gence Robotics and Automation in Space (ISAIRAS), 1999.

[Russell/Norvig 1995] S. Russell and P. Norvig. Artificial Intelligence: a ModernApproach. Prentice Hall, 1995.

[Smith et al. 1999]B. Smith, W. Millar, J. Dunphy, Y. wen, P. Nayak, E. Jr, and M.Clark. Validation and verification of the remote agent for space-craft autonomy. In Proceedings of the IEEE Aerospace Confer-ence, Snowmass, CO., 1999.

[Smith et al. 2000]D.E. Smith, J. Frank, and A.K. Jónsson. Bridging the gapbetween planning and scheduling. Knowledge EngineeringReview, 15(1), 2000.

[Smith et al. 1998]S.J.J. Smith, D.S. Nau, and T. Throop. Computer bridge: A bigwin for AI planning. AI Magazine, 19(2):93–105, 1998.

[Srivastava 2000]B. Srivastava. RealPlan: Decoupling causal and resource reason-ing in planning. In Proc. of 17th National Conference on AI,pages 812–818. AAAI/MIT Press, 2000.

[Stefik 1981]M. Stefik. Planning with constraints (MOLGEN: Parts 1 and 2).Artificial Intelligence, 16(2):111–170, 1981.

[Tsang 1993]E.P.K. Tsang. Foundations of Constraint Satisfaction. AcademicPress, London and San Diego, 1993.

[vanBeek/Chen 1999]P. van Beek and X. Chen. CPlan: A constraint programmingapproach to planning. In Proc. of 16th National Conference onArtificial Intelligence, pages 585–590. AAAI/MIT Press, 1999.

[Vossen et al. 1999]T. Vossen, M. Ball, A. Lotem, and D. Nau. On the use of integerprogramming models in AI planning. In Proceedings of Interna-tional Joint Conference on AI, (IJCAI-99), pages 304–309, 1999.

[Westfold/Smith 2001]S.J. Westfold and D.R. Smith. Synthesis of efficient constraintsatisfaction programs. Knowledge Engineering Reviews: SpecialIssue on AI and OR, 2001.

[Wilkins/desJardins 2000]D. Wilkins and M. desJardins. A call for knowledge-based plan-ning. In Proc. of AI Planning and Scheduling (AIPS) Workshopon Analyzing and Exploiting Domain Knowledge for EfficientPlanning, 2000.

[Wilkins 1988]D.E. Wilkins. Practical Planning: Extending the Classical AIPlanning Paradigm. Morgan Kaufmann Publishers Inc., SanFrancisco, CA, 1988.

[Wolfman/Weld 1999]S. Wolfman and D. Weld. The LPSAT engine and its applicationto resource planning. In Proc. of 16th International Joint Confer-ence on AI, pages 310–317. Morgan Kaufmann, 1999.

[Younes/Simmons 2002]H.L.S. Younes and R.G. Simmons. On the role of ground actionsin refinement planning. In Proceedings of the Sixth InternationalConference on Artificial Intelligence Planning and SchedulingSystems, pages 90–97, 2002.



Trends in Automatic Learning

Ramón López de Mántaras

The capacity for learning is one of the fundamental characteristics of intelligence. Learning forms part ofany activity requiring intelligence, such as diagnosis, planning, language, motor-sensory activities, etc. Forthis reason automatic learning plays a vital role in applications which have proven to be to difficult toprogramme manually. To describe each and every method of automatic learning would require a book of itsown and besides, the more classic methods are described in most introductory texts to Artificial Intelligence.For this reason, in this article I have decided to concentrate on the more modern techniques of automaticlearning.

Keywords: Learning, Reinforcement Learning, StatisticalLearning, Methods based on Ensembles of Classifiers, CaseBased Reasoning

IntroductionThe capacity for learning is one of the fundamental charac-

teristics of intelligence. In keeping with this irrefutable state-ment, Roger Schanck [Schank 1987], pioneer of ArtificialIntelligence, defines AI as “The science of endowingprogrammes with the ability to improve their performance byusing experience”. This definition practically identifies “Artifi-cial Intelligence” with “Automatic Learning”. It is true thatlearning is involved in any activity requiring intelligence, suchas diagnosis, planning, language, motor-sensory activities, etc.Automatic learning plays an essential role in applicationswhich have proven to be too difficult to programme manually.Examples of such applications are: automatic control of auton-omous vehicles, speech recognition, information retrieval,indexing and categorization of documents, scientific discovery,games, artificial vision, autonomous robots, music, etc.

Due to limitations of space the description of each and everyone of the methods of automatic learning there are would forceme to devote no more than a couple of line to each one. Besides,the more classic methods (induction of decision trees, explana-tion based learning, neural networks, rule learning, etc.) aredescribed in most introductory texts to Artificial Intelligence[Russell/Norvig 1995] [Ginsberg 1993]. For this reason, in thisarticle I have decided to concentrate on the more modern tech-niques of automatic learning. For further information I recom-mend Mitchell’s text book [Mitchell 1997], and the bookpublished by Paliouras, Karkaletsis and Spyropoulos [Paliour-as et al. 2001] based on an advanced course on automatic learn-ing. This book contains a representative collection of articlescovering both fundamental aspects and applications of auto-matic learning together with a series of tutorials on the subject.

Modern Methods in Automatic LearningIn the last 10 years we have seen some spectacular progress

in automatic learning and there are at least two reasons for this.

The first is the ever closer collaboration between scientificcommunities which until recently had ignored each other. I amreferring, of course, to the Artificial Intelligence communityand the statistical and pattern recognition communities. Thesecond reason is that learning techniques are being applied tonew and ever more difficult problems such as knowledgediscovery in databases, language processing, autonomousrobots, combinatorial optimization, games, music, and manyother applications which are extremely difficult to programme

1

2

Ramón López de Mántaras is a Research Professor at theSpanish Scientific Research Council (CSIC) and Deputy Directorof the Artificial Intelligence Research Institute of the CSIC. He isa Technical Engineer in Electronics from the Escuela Politécnicade Mondragón (Spain), a Computer Science graduate from theUniversidad del País Vasco (Spain), Doctor of Applied Physics(special subject: Automation) from the University Paul Sabatierof Toulouse (France), MSc in Computer Science from the Univer-sityof California-Berkeley, and Doctor of Computer Science fromthe Universidad Politécnica de Cataluña (Spain). He has workedin the Ikerlan Technological Research Centre in Mondragón, wasFull Professor of the Faculty of Computer Science, Barcelona,and Visiting Professor at the University Pierre et Marie Curie ofParis. He sits on the editorial committees of several internationaljournals and was Editor-in-Chief of the European journal on arti-ficial intelligence: “Artificial Intelligence Communications”. Hehas received the following awards: “City of Barcelona” for re-search in 1982, “European Artificial Intelligence Research PaperAward” in 1987, “International Computer Music AssociationSwets & Zeitlinger Award” in 1997 and he has been a Fellow ofECCAI since 2000. He was the coordinator of the European Net-work of Excellence in Machine Learning, and author or co-authorof some 170 scientific publications and the book ApproximateReasoning Models published by Ellis Horwood in their ArtificialIntelligence Series. He is currently researching into case basedreasoning and learning, multi-agent architectures for qualitativeautonomous robot navigation, and in applications of artificial in-telligence to music. His personal page is at <http://www.ii-ia.csic.es/~mantaras>.< [email protected]>



manually. This progress is reflected in a group of methodswhich could be termed as “modern” and which I will go on todescribe briefly.

2.1 Reinforcement LearningThis is a method of learning applicable in situations in which

there is interaction between an agent and its environment. Byinteracting with its environment, the agent has to learn the rightsequence of actions (control strategy) to achieve a pre-deter-mined objective. This problem is modelled by Markov process-es and, consequently, the agent must solve a Markov DecisionProblem (MDP). In an MDP, the agent can perceive a set S ofdifferent environment states and can carry out a set A ofactions. At each moment of time t, the agent perceives the statest, selects an action at and carries it out. The environmentresponds by giving a reward or penalty rt= r(st, at) and generat-ing the successor state st+1= _ (st, at). This response is whatenables it to learn the right sequence of actions. In an MDP, thefunctions r and _ depend only on the present state and actionand not on prior states and actions. Generally speaking, r and _may be non-deterministic. Therefore reinforcement learningmay be considered as the study of computationally tractablealgorithms for approximating the solution of Markov decisionproblems [Dietterich 1997]. For example, let us consider amobile robot equipped with a set of sensors (cameras, sonar,etc.) for observing the state of its environment, and capable ofcarrying out a set of actions (advance, turn, etc.) which modifythat state. Its task consists of learning a control strategy whichwill choose actions enabling it to achieve an objective, such asmoving from any initial position to a final position while avoid-ing obstacles.

There is a close relationship between reinforcement learningand the technique of dynamic programming [Bellman 1957]for solving MDPs. In fact, most fundamental algorithms arebased on considering reinforcement learning as a form ofdynamic programming [Barto et al. 1995]. The problem withthese algorithms is that the execution time grows exponentiallywith the dimension of the space of states (Bellman called thisthe “dimensionality problem”). This shortcoming is not impor-tant for simple problems but, for the problems involved in Ar-tificial Intelligence, it is enough to make their solution unfeasi-ble. Another problem is that these algorithms require acomplete model of the system; that is, knowledge of the rewardfunction r, and the transition between states function d (or ofthe corresponding transition and reward probabilities in thenon-deterministic case). In many realistic applications thismodel is unknown. This problem is known as the “modellingproblem”. Researchers in reinforcement learning have intro-duced important innovations which allow these problems to beovercome [Dietterich 1997]. These innovations are based onproposing ways to reach an approximate calculation of the val-ues of the state transition and reward functions in the event ofa modelling problem, and taking advantage of the often hierar-chical (or loosely hierarchical) structure of the space of statesin the event of a dimensionality problem. Another major diffi-culty is the hypothesis that the environment state is completelyobservable at all times. Evidently, in many cases this hypothe-

sis is false (for example, the sensors of a robot only providepartial information about the environment). We try to solve thisproblem by means of algorithms based on the concept of par-tially observable Markov processes [Littman et al. 1995]. Inspite of all these difficulties, there are interesting applicationsof reinforcement learning to problems which could not be tack-led by using any other methods of learning, such as navigationof autonomous robots [Kaebling et al. 1996] [Busquets et al.2002] or games [Tesauro 1995]. Interested readers will find arepresentative collection of papers on the subject in a recentspecial edition of the journal “Artificial Intelligence” [AI 2001]devoted entirely to reinforcement learning.

2.2 Statistical LearningOne important trend in current research is the growing

evidence of the close relationship between automatic learningand statistical techniques, especially with regard to problemmodelling and result interpretation. The clearest example of therelationship is the influence that statistical data analysis ishaving on data mining problems. Other examples are naïveBayesian classifiers, support vector machines (SVM), andBayesian networks. I will go on to give a brief description ofthese last two methods.

2.2.1 Support Vector MachinesSVMs [Vapnik 1998] are methods of inductive learning from

examples initially applied to binary classification problems.Basically, SVMs induce a hyperplane which separates thespace of characteristics describing the examples into tworegions, one for each of the two classes of the binary classifica-tion. When the set of training examples is linearly separable,the SVM finds a hyperplane which separates the two classes soas to have the maximum distance between the hyperplane andthe nearest example of each class, i.e. so that the hyperplane isoptimal. This distance is called the “margin” and could bedefined as the maximum width a band placed between the twoclasses could have without covering any of the examples. Theconcept of margin can be extended to non-linearly separabledata and even non-separable data [Cortes/Vapnik 1995], due toa remarkable property of SVMs which enables them to betransformed into non-linear learning systems. This is done byusing a non-linear application to transform the original vectorof attributes describing the examples into a characteristicvector of higher dimensionality. In this space of characteristics,the SVM finds the linear classification rule (hyperplane) whosemargin is maximal. This rule is converted into a non-linearclassification rule once it is projected once more to the originalattributes space. The drawback is that the calculation of thenon-linear transformation is computationally inefficient. Themain advantage is that neither the generalisation capacity northe training process of SVMs necessarily depends on thenumber of attributes, and consequently SVMs perform excel-lently well in high dimensionality problems [Joachims 1998].Among their many applications to tasks of classification,SVMs have been applied very successfully to problems ofartificial vision, for example in the detection of objects. Theyare also currently being applied to the problem of text catego-



rization, such as web pages dealing with a specific subject[Joachims 1998] [Joachims 2000]. Interested readers will finda representative collection of papers on the subject in [Cher-kassky/Mulier 1998].

2.2.2 Learning of Bayesian NetworksA Bayesian network is a graphical model in which probabil-

ity ratios between a given set of random variables are represent-ed [Heckerman et al. 1995].

These networks describe the probability distribution govern-ing a set of variables explicitly representing the conditionaldependencies and independencies between them. For example,the Bayesian network shown in Figure 1 (taken from [Diet-terich 1997]) corresponds to the following decomposition ofthe joint probability distribution of the 6 variables present:

P(E,N,M,I,G,D) = P(E) * P(N) * P(M/E,N) * P(D/M,E,N) *P(I/D) * P(G/I,D)

Where E= Age, N= Number of pregnancies, M= Body mass,D= Diabetes, I= Insulin (level in blood), G= Glucose (level inblood).

The structure of the network, and in particular the missingarcs, specifies conditional independencies in the followingway:

Two variables X e Y are conditionally independent given Z, if P(X,Y/Z) = P(X/Z) * P(Y/Z)For example, in Figure 1, E affects I only through the node

D; therefore E and I are conditionally independent given D.The aim is to be able to infer the probability that one of the

variables has a given value (for example that D is true) from theprobability values of the other variables (observations). To dothis, in addition to specifying the structure of the network, it isnecessary to specify all the probability distributions. For exam-ple, if E admits 4 possible values (4 age intervals), N admits 3values (0, 1 or more than 1 pregnancy) and M admits 3 (that is,3 weight intervals), the complete specification requires know-ing 31 parameters: 4 for the 4 a priori probabilities of the 4possible values of the variable E, 3 for the a priori probabilitiesof N, and 24 for all the combinations of values of the condi-tioned probability P(M/E,N) (there are 36 values but 12 of themare calculated by subtracting from 1 the 24 remaining ones,since the sum of all the probabilities is 1).

By repeating this process with the rest of the variables we cansee that even for such a simple network, it would be necessaryto know the value of an extremely high number of parametersto be able to infer the probability of any of the variables whichdepend on others (in the example, M, D, I, G are the dependentvariables) from the rest. This is where learning comes in since,from a set of sufficiently complete training examples, it ispossible to estimate the values of all these probabilities bymeans of classic statistical techniques. However, as exact tech-niques have an exponential complexity, approximate methodsare used such as the algorithm “expectation-maximization”(EM) [Dempster et al. 1976] or the “Gradient Descent” forBayesian networks [Russell et al. 1995].

In addition to learning the parameters, generally speaking itis also useful to be able to learn the structure of the network.Given that the space of possible structures grows at a rate great-er than exponential to the number of variables, we again findthat learning the exact structure is, generally speaking, anintractable problem and so it is necessary to find some approx-imate algorithms. One of the best known is the one known asK2 [Cooper/Herskovits 1992]. This algorithm is capable oflearning the approximate structure of the network when all thevariables are represented in the set of training examples. Aswith most structure learning algorithms, K2 carries out agreedy search in the space of possible structures, taking intoaccount the trade off between the complexity of the networkand the accuracy (that is, the quality of the approximation)obtained. This algorithm requires the user to decide on anordering of the variables. This ordering effectively restricts thesearch since the algorithm considers adding arcs only in onedirection (from variables earlier in the ordering to variableslater in the ordering). This restriction also guarantees that thenetwork being learnt will not contain cycles. K2 starts out witha network without arcs (all variables are initially assumed to beindependent from one another) and it evaluates the a posterioriprobability of adding each of the possible arcs between the var-iables (in accordance with the established ordering) and addsonly those that maximise that probability. K2 has been success-fully applied in medical diagnosis applications of some consid-erable size (in the order of fifty variables and more than ahundred arcs). [Heckerman et al. 1995] modified the algorithmK2 so as to be able to start out with an initial network (witharcs) corresponding to an a priori probability distribution. Thealgorithm carries out a local search considering modificationsto the network (adding an arc, deleting an arc, or changing thedirection of an arc) and retains the modification which maxim-ises the a posteriori probability of the network. [Spirtes et al.1993] developed an algorithm to find dependency and inde-pendency relations between variables by means of constraintsatisfaction techniques. Once those dependencies and inde-pendencies are determined, the algorithm constructs the corre-sponding network. Another important contribution is the TANalgorithm (tree-augmented naïve Bayes) from Friedman andGoldszmidt [Friedman/Goldszmidt 1996], which takes as itsstarting point a network in the form of a depth-one tree, that is,one root node (corresponding to a variable) with the rest of thevariables (x1, x2,…, xn) all connected with the root node (i.e.

E

M

N

D

I

G

Figure 1: Example of a Bayesian Network



all dependent on the variable associated with the root). Thealgorithm learns an arc structure which interconnects variablesxi. Interested readers will find a representative collection ofpapers on the subject in the book “Learning in GraphicalModels” [Jordan 1998].

Methods based on ensembles of classifiersUntil recently, the aim of learning methods was to find, by

using training examples, the best possible hypothesis among aset of hypotheses generated during the training phase. Forexample, let us consider a programme which must learn anunknown function y = f(x), based on a set of training examplesof the form {(x1, y1), … , (xn, yn)} where “xi” are generallyvectors of characteristics or attributes by which the examplesare described (colour, weight, size, etc.) and “y” are, in the caseof discrete classification problems, discrete values correspond-ing to the possible classes {1, …, k} or real values in the caseof regression problems. So, given the set of training examples,the algorithm generates a classifier. This classifier is a hypoth-esis about the true function f in such a way that given any newvalue (x), it predicts its class (y). In general, by suitably manip-ulating the training set or the learning algorithms themselves,different hypotheses (classifiers) are generated. Based on thisfact an alternative approach has been proposed [Schapire 1990]which consists of combining the different hypotheses generat-ed instead of selecting one of them. This alternative approachgoes by the name of Ensemble of Classifiers. In general, anensemble of classifiers is a set of classifiers whose individualdecisions about which class an example to be classified belongsto, is combined so as to propose a collective decision. The aimis for the collective decision to be better than all the individualdecisions. It is not hard to prove [Dietterich 1997] that if thevarious individual classifiers are not correlated (i.e. there is nounanimity in their decisions) and if the probability of error ofeach one is less than 0.5, then the collective probability of erroris less than the individual probabilities of error of the differentclassifiers.

The main problem is to construct good ensembles of classifi-ers. To this end a series of construction methods have beendeveloped. These methods can be divided into general ones,applicable to any learning algorithm, and specific ones forparticular algorithms.

A general method consists of generating multiple hypotheses(classifiers) by running the learning algorithm several timeswith different subsets, chosen at random, from the set of train-ing examples. Each of these runs generates a classifier, that is,a hypothesis about which class each example belongs to. Thefinal decision is based on the majority vote of the classifiers.This technique is known as “bagging” [Breiman 1996] andworks particularly well with “unstable” algorithms, that is,algorithms whose response (the classification they give to newexamples) varies substantially in response to small variations inthe set of training examples. Neural networks and decisiontrees, among others, belong in this category.

Another method which gives good results is “boosting”. Likebagging, boosting manipulates the training set with the aim ofgenerating multiple hypotheses. To explain how it works, let us

take an example in which we will construct three classifiers fora problem of classification into 2 classes. We begin by random-ly selecting a subset E1 of n1 examples of the training set E. Theboosting technique starts by training a first classifier C1 whoseaccuracy in classifying the examples of the training set E1 is alittle greater than what we would obtain by classifying atrandom (for example, if in the set E1 there are 60% of examplesof class 1 and 40% of class 2, classifying randomly meansalways predicting class 1, so, on average, we would get anaccuracy of 60%). Next we generate a second training set, E2,to be maximally informative given the first classifier C1 (thismeans that half of the examples in E2 should be correctlyclassified by C1 and the other half must be incorrectly classifiedby E1). This can be done as follows: we toss a coin and if itcomes up “heads” for example, we select the rest of the exam-ples in E and we present them one by one to C1 for it to classifyuntil C1 classifies an example incorrectly. We add this exampleto E2. Then we toss the coin again. If it comes up “heads” werepeat the operation, that is to say, we present the examples inE, one by one to C1, until it classifies another one incorrectlyand we add it E2. However, if it comes up “tails” we look for anexample that C1 classifies correctly and we add it to E2. Theprocess continues until there are no more examples left. In theend, approximately half of the examples in E2 are classifiedcorrectly by C1 and the rest are classified incorrectly, and thusE2 provides complementary information to that represented byC1, or to put it another way, E2 is maximally informative givenC1. Next we use E2 to train the second classifier C2. The nextstep is to construct a third training set, E3, which cannot beclassified unanimously by the set formed by C1 and C2. In otherwords, we randomly select an example in E and if C1 and C2classify it into different classes, we add it to E3; otherwise weignore this example and choose another one and so on and soforth. At the end of the process, E3 will contain those examplesnot classified by the set of classifiers formed by C1 and C2.Finally, we train the third and last classifier C3 with the set E3.

To classify each new example a voting technique is used insuch a way that if C1 and C2 coincide by classifying it in thesame class, this will be the definitive class; otherwise it isclassified in the class given by C3. It is interesting to note thatthis procedure of boosting can be applied recursively to theclassifiers making up the set (in the example, to E1, E2 and E3)which would give a set of 9 classifiers and so on (obtaining 27classifiers, then 81, etc.). In this way you can achieve an overallclassification error as small as you like, assuming that the prob-lem is separable. Of course, there is a computational price topay.

There are a series of variations on the basic boosting tech-nique I have described. The best known variation is the algo-rithm AdaBoost (adaptive boosting) [Freund/Schapire 1997].In AdaBoost a weight is assigned to each example of the train-ing set, E, which determines the probability of it being selectedto form part of a subtraining set. If a training example iscorrectly classified, the probability of its being reselected goesdown and if it is incorrectly classified the odds go up. In thisway AdaBoost focuses its attention on the most difficult exam-ples to classify. Specifically, the process starts by assigning the

3



same weight to all the examples (uniform probability distribu-tion). In each iteration, k, a subset of examples is selected atrandom, according to their different weights (probabilities),and classifier Ck is trained based on these examples. Next theweighting of the badly classified examples by Ck is increasedand the weighting of the correctly classified examples isdecreased. The examples selected according to the new proba-bility distribution, resulting from the adjustment of weightingcarried out previously, are used to train the next classifier, Ck+1and so on until the set of classifiers reaches a classification errorrate beneath a predetermined threshold. The final set of classi-fiers takes the classification decisions on new examples bymeans of a weighted vote. The weight of the vote from eachindividual classifier individual is decided by the accuracy ratewith which it classified the examples of the training set it wastrained with.

An alternative technique for constructing ensembles of clas-sifiers consists of manipulating the set of characteristics orattributes by which the examples are described. The basic ideais to train different classifiers by modifying those characteris-tics for each of them. For example, Cherkauer [Cherkauer1996] developed an algorithm for identifying volcanoes onVenus by training a set of 32 neural networks obtained bycombining 8 different subsets of characteristics (from a total of119 possible characteristics for describing volcanoes) with 4different sizes of networks. The resulting set of classifiersmatched the accuracy of human experts in the identification ofvolcanoes. Again, either a simple or a weighted vote decidesthe set.

Another alternative technique consists of manipulating thevalues of the classes in the training examples. The best knownalgorithm is ECOC (Error-Correcting Output Coding) [Diet-terich/Bakiri 1995]. ECOC is particularly interesting whenthere is a high number of classes. The technique consists ofmaking a random split of the initial K classes into two subsets,say A and B. Then the classes of the training examples are re-tagged in such a way that those whose class is in A become, forexample, class 1 and all those whose class is in B become class2. The re-tagged examples are processed by the learning algo-rithm giving rise to a classifier. By repeating this process Ntimes with different subsets A and B, we obtain a set of Nclassifiers C1, …, Cn. Each new example is classified bypresenting it to each classifier Ci. If Ci classifies it into class 1,each and every class in A receives a vote and if it classifies itinto class 2 all the classes in B receive a vote. After carrying outthis process with the N classifiers the votes each of the initial Kclasses split into A and B are added up and the one with thehighest number of votes is chosen.

Another technique used for constructing ensembles of classi-fiers consists of introducing randomness into the learning algo-rithm itself. For example, the values of the weights of a neuralnetwork can be initialised randomly. By applying the back-propagation algorithm with different initial weights we will getdifferent classifiers for the same training set. At first sight thistechnique does not appear to be a very general technique sinceit seems to be limited to parameter dependent algorithms, how-ever it can also be applied to algorithms which do not require

the initialisation of parameters, e.g. the learning of decisiontrees. In this case, the randomness is introduced in the attributeselection phase since, instead of selecting the one that maxim-ises or minimises the corresponding selection criterion [Lópezde Mántaras 1991], the N best attributes are selected and one ischosen at random. Evidently, the result of this process is a setof different decision trees (classifiers). As with the previoustechniques, the joint decision is based on a voting process.

In addition to the general techniques described, there are alsotechniques specific to particular learning algorithms. Most ofthem are for training sets for neural networks. For example,Optiz and Shavlik [Optiz/Shavlik 1996] use a genetic algo-rithm for finding a good population of neural networks. Geneticoperators are used to diversify the topology of the networks.After training each network, in each iteration the worst ones (interms of classification error and diversity) are eliminated fromthe population until a set is obtained whose accuracy exceeds apredetermined threshold.

In [Dietterich 1997] there is an interesting explanation ofwhy ensembles of classifiers work so well.

Which is the best method for constructing ensembles ofclassifiers? According to the “No-Free-Lunch theorem”[Wolpert/Macready 1995] “the best” method does not exist inthe same way that “the best” learning algorithm does not existeither. However some methods may be better than othersdepending on the particular problem to be solved. There havebeen few systematic studies made of these construction meth-ods for ensembles of classifiers so there is still a great deal ofwork to be done in this in this area. Another aspect open toimprovement is the lack of transparency with which ensemblesof classifiers take decisions. One single decision tree can beeasily interpreted by a user but it is not so simple to interpretthe result of the joint decision of hundreds of trees. Finally weshould mention the problem of the enormous quantity of mem-ory required and the high computational cost of these methods.A promising line of research would be to look for ways toreduce redundancy, for example by eliminating from theensemble those classifiers whose decisions are highly correlat-ed to others.

Case Based ReasoningCase based reasoning (CBR) is an important computation-

al model of problem solving by means of reasoning and learn-ing from experience. In CBR a problem is solved by adaptingthe solutions of similar problems which have been solvedpreviously. They are based on the hypothesis that similarproblems have similar solutions. The process consists of sever-al phases: Case retrieval (similar problems), Selection of themost similar one or ones, Reuse of the solution(s) to propose asolution to the new problem, Review of the solution(s)proposed and Memorization of that solution (in other words,learning from experience) for use in the future to solve a newproblem [Aamodt/Plaza 1994]. In simple problems (as, forexample, in classification), it is generally enough to retrievejust one case (the most similar) to obtain a solution to the newproblem (that is, to decide which class it belongs to). CBRclassification systems are based on the classification method

4



known as “nearest neighbours” [Duda et al. 2001]. In complexproblems, such as design and planning for example, it is gener-ally necessary to combine the partial solutions provided by thedifferent cases retrieved and then adapt the solution soobtained. A typical field of complex application is architecturaldesign. Architects do not start “from scratch” every time theydesign something; they reuse previous designs. There are alsoa great many applications to medical diagnosis and planning[Kolodner 1993] and even to music [Arcos/López de Mántaras2001]. There is extensive literature on CBR since as well asbeing a powerful method of problem solving, it has been usedin conjunction with such other methods as inductive learning(to generalize cases), genetic algorithms and neural networks.There are still many aspects open to improvement. One of themis the problem of validation of the case base (do we have theright cases? are they all necessary? etc.) and its maintenance (astime goes by it is necessary to adapt the content of the casebase) since the quality of the case base, and consequently thequality of the CBR system as a problem solver, depend on thesetwo aspects. Another complex aspect is the reuse (also referredto as adaptation) of solutions of previous cases to adapt them tothe new problem. Currently a set of adaptation rules are used,however the acquisition of this adaptation knowledge is, gener-ally speaking, a complex problem for which techniques ofautomatic learning are being studied. Interested readers willfind a representative collection of papers on the subject in[Aha/Watson 2001]

ConclusionIn this article I have made a brief description of some of the

learning methods which I believe reflect the most moderntrends in the subject. For reasons of space I have not been ableto go into as much depth as I would have liked in the descrip-tion of the methods I have been able to include and, obviously,I have not been able to include many others of equal impor-tance. However I did not want to close without mentioning afew of the many aims which will doubtless set the agenda forfuture research into automatic learning:

Active learning/Experience design: the aim is for the learn-ing system itself to decide which experiments to carry out andselect the data for those experiments.

Cumulative and continuous learning: the aim is to designagents which learn by the continuous accumulation of knowl-edge which is used to guide their own learning process.

Learning incorporated into programming languages: theidea is to develop new programming languages and environ-ments incorporating some kind of automatic learning just asnow they incorporate abstract data types, for example.

Learning from texts and natural language discourse: theobjective is to develop systems capable of accumulating knowl-edge by reading texts and understanding spoken language.

Learning from multimodal data: instead of learning from justone type of data, the aim is to develop systems capable of learn-ing from multimodal data (numerical, acoustic, images, etc.)

As we can see there is still a great deal left to do in this fas-cinating subject. Without a doubt the future of Artificial Intelli-

gence is completely dependent on the progress made in auto-matic learning.


References [Aamodt/Plaza 1994]

A. Aamodt, E. Plaza, 1994. Case-Based Reasoning: FoundationalIssues, Methodological Variations, and System Approaches, AICommunications 7(1): 39–59.

[Aha/Watson 2001]D.W. Aha, I. Watson (editors), 2001. Case-Based ReasoningResearch and Development (ICCBR 2001), Lecture Notes inArtificial Intelligence 2080.

[AI 2001]A.I. 2001. Artificial Intelligence Journal, 49 (2/3).

[Arcos/López de Mántaras 2001]J.L. Arcos, R. López de Mántaras, 2001. An Interactive Case-Based Reasoning Approach for Generating Expressive Music,Applied Intelligence 14(1): 115–129.

[Barto et al. 1995]A.G. Barto, S.J. Bradtke, S.P. Sing, 1995. Learning to Act UsingReal-Time Dynamic Programming, Artificial Intelligence 72:81–138.

[Bellman 1957]R.E. Bellman, 1957. Dynamic Programming, Princeton Universi-ty Press.

[Breiman 1996]L. Breiman, 1996. Bagging Predictors, Machine Learning 24(2):123–140.

[Busquets et al. 2002]D. Busquets, R. López de Mántaras, C. Sierra, T.G. Dietterich,2002. Reinforcement Learning for Landmark-based Robot Navi-gation, 1st Autonomous Agents and Multiagent Systems Confer-ence, Bologna, Italy.

[Cherkassky/Mulier 1998]V. Cherkassky, F. Mulier, 1998. Learning from Data: Concepts,Theory, and Methods. Wiley, New York.

[Cherkauer 1996]K.J. Cherkauer, 1996. Human Expert-Level Performance on aScientific Analysis Task by a System Using Combined ArtificialNeural Networks, AAAI Workshop on Integrated MultipleLearned Models, 15–21. Available at www.cs.fit.edu/imlm/.

[Cooper/Herskovits 1992]G.F. Cooper, E. Herskovits, 1992. A Bayesian Method for theInduction of Probabilistic Networks from Data, Machine Learn-ing 9: 309–347.

[Cortes/Vapnik 1995]C. Cortes, V. Vapnik, 1995. Support-Vector Networks, MachineLearning 20: 273–297.

[Dempster et al. 1976]A.P. Dempster, N.M. Laird, D.B. Rubin, 1976. Maximum Likeli-hood from Incomplete Data via the EM Algorithm, Journal of theRoyal Statistical Society B 39: 1–38.

[Dietterich 1997]T. G. Dietterich, 1997. Machine-Learning Research, AI Maga-zine 18(4): 97–136.

[Dietterich/Bakiri 1995]T.G. Dietterich, G. Bakiri, 1995. Solving Multi-Class LearningProblems via Error-Correcting Output Codes. Journal of Artifi-cial Intelligence Research 2: 263–286.

[Duda et al. 2001]R.O. Duda, P.E. Hart, D.G. Stork, 2001. Pattern Classification,Wiley-Interscience.

[Freund/Schapire 1997]Y. Freund, R.E. Schapire, 1997. A Decision-Theoretic Generali-zation of On-Line Learning and an Application to Boosting. 2nd

5



European Conference on Computation Learning Theory, 23–37,Springer-Verlag.

[Friedman/Goldszmidt 1996]N. Friedman, M. Goldszmidt, 1996. Building Classifiers usingBayesian Networks. 13th National Conference on Artificial Intel-ligence, 1277–1284, AAAI Press.

[Ginsberg 1993]M. Ginsberg, 1993. Essentials of Artificial Intelligence. MorganKaufmann.

[Heckerman et al. 1995]Heckerman, D. Geiger, D.M. Chickering, 1995. Learning Baye-sian Networks: The combination of knowledge and statisticaldata, Machine Learning 20: 197–243.

[Joachims 1998]T. Joachims, 1998. Text Categorization with Support VectorMachines: Learning with Many Relevant Features, Lecture Notesin Artificial Intelligence 1998 (ECML’98), 137–142.

[Joachims 2000]T. Joachims, 2000. Estimating the Generalization Performance ofa SVM Efficiently, International Conference on Machine Learn-ing, 431–438, Morgan Kaufmann.

[Jordan 1998]M.I. Jordan (ed.) 1998. Learning in Graphical Models, MITPress, Cambridge, Massachusetts.

[Kaebling et al. 1996]L.P. Kaebling, M.L. Littman, A.W. Moore, 1996. ReinforcementLearning: A Survey, Journal of Artificial Intelligence Research 4:237–285.

[Kolodner 1993]J. Kolodner, 1993. Case-Based Reasoning, Morgan Kaufmann.

[Littman et al. 1995]M.L. Littman, A. Cassandra, L.P. Kaebling, 1995. LearningPolicies for Partially Observable Environments: Scaling Up. 12thInternational Conference on Machine Learning, Morgan-Kaufmann, 362–370.

[López de Mántaras 1991]R. López de Mántaras, 1991. A Distance-Based Attribute Selec-tion Measure for Decision Tree Induction, Machine Learning 6:81–92.

[Mitchell 1997]T.M. Mitchell, 1997. Machine Learning, McGraw-Hill.

[Optiz/Shavlik 1996]D.W. Optiz, J.W. Shavlik, 1996. Generating Accurate andDiverse Members of a Neural-Network Ensemble. Advances inNeural Information Processing Systems 8, editors: D.S. Touretz-ky, M.C. Mozer and M. Hesselmo, 533–541, MIT Press.

[Paliouras et al. 2001]G. Paliouras, V. Karkaletsis, C.D. Spyropoulos (editors), 2001.Machine Learning and Its Applications: Advanced Lectures,Lecture Notes in Artificial Intelligence 2049, Springer-Verlag.

[Russell et al. 1995]S. Russell, J. Binder, D. Koller, K. Kanazawa, 1995. Local Learn-ing in Probabilistic Networks with Hidden Variables, 14th Inter-national Joint Conference on Artificial Intelligence, 1146–1152.

[Russell/Norvig 1995]S. Russell, P. Norvig, 1995. Artificial Intelligence: A modernapproach. Prentice-Hall.

[Schank 1987]R. C. Schank, 1987. What is AI, Anyway?, AI Magazine 18(4):59–65.

[Schapire 1990]R.E. Schapire, 1990. The Strength of Weak Learnability, MachineLearning 5(2): 197–227.

[Spirtes et al. 1993]P. Spirtes, C. Glymour, R. Scheines, 1993. Causation, Prediction,and Search, Springer-Verlag.

[Tesauro 1995]G. Tesauro, 1995. Temporal Difference Learning and TD-GAM-MON. Communications of the ACM 28(3): 58–68.

[Vapnik 1998]V. Vapnik, 1998. Statistical Learning Theory, Wiley.

[Wolpert/Macready 1995]D.H. Wolpert, W.G. Macready, 1995. No Free-Lunch Theoremsfor Search, Technical Report SFI-TR-95-02-010, Santa FeInstitute.



Knowledge-Based Systems

José Mira-Mira and Ana E. Delgado-García

In this article we introduce the concept of a Knowledge-Based System (KBS) as a computer program, whichcodifies the formal model underlying the knowledge which a human expert uses to solve a task within alimited field. We also examine some of its basic characteristics, such as the separation between domainknowledge, and inference knowledge, which specifies the reasoning steps followed to solve the task. Then,some methodological issues concerning the taxonomy of levels and domains of description of our models ofhuman knowledge are introduced. We also present the different phases in the development of a KBS(modelling at the knowledge level, “operationalization” of the inferences, implementation, validation, andevaluation) and end drawing links with the tendencies of component reuse and integration of the symbolicand connectionist perspectives of KBS. An essential characteristic in the perspective of this article is theemphasis on the methodological aspects, whose aim is to reduce the gap between knowledge engineeringand electronic engineering.

Keywords: Knowledge Modelling, Methodology, Compo-nents Reuse, Ontologies, Library of Methods.

Concept of Knowledge - Based System (KBS)When there is a system in which intensive use is made of

the knowledge of the domain and it is separated from the mech-anism controlling its use in inference, we say we have a KBS(Knowledge-Based System). Within the KBS there is a groupof systems in which knowledge comes from a human expertwho specializes in a specific task (diagnosis, supervision,therapy, etc...) in a technical domain. We then say that we havean Expert System (ES). Currently, both concepts are used inter-changeably although, the ES explosion finished, there is atendency to return to the more comprehensive KBS, regardlessof what the source of knowledge may be [Mira et al. 1995].

The development of the basic methods of representation ofknowledge has taken place alongside KBS. Each form ofrepresentation is historically associated to the first system,which used it, thus demonstrating its practical possibilities. Forexample, the possibility of programming the solution to a prob-lem using rules is known since Post, in 1943, and Newell andSimon, in 1972, introduced it into Artificial Intelligence (AI).It was, however, necessary for the MYCIN system to developfor programming by rules to be consolidated into a generalmethod for developing expert systems [Buchanan/Shorliffe1984].

The fundamental characteristics of a KBS are: limiteddomains, competence in its field, separation of domain knowl-edge and inference and task knowledge, capacity to explain thereasoning steps, flexibility in dialogue and treatment of uncer-tainty.

The basic modules of a KBS of the first period are the baseof true facts, the knowledge base, and the inference mechanismused by the elements in the knowledge base, guided by true

facts, to produce new facts constituting the KBS responses andtwo additional modules. The first one is a conventional data-base, and the second one is an explanation and dialoguemodule. This description of a KBS represents a Rule-BasedSystem. When knowledge is represented by means of frames orsemantic nets, the structure does not correspond to thesemodules in such a clear fashion. An important conclusionwhich can be drawn from this stage of AI is that Rule-BasedSystems constitute an effective procedure for representing andusing knowledge that is essentially different from conventionalprogramming. Here, computation is carried out by means of aniterative cycle of first identifying the adequate rule for a truefact, and then applying it. Afterwards one checks if the problemhas been solved as a consequence of the application of this rule

1

José Mira-Mira is full professor of Computer Science andArtificial Intelligence and Head of the Department of ArtificialIntelligence at the National University for Distance Education(UNED) in Madrid (Spain). His research interests include (1)Knowledge Modelling at the knowledge level (2) Neural model-ling of biological structures and application of these models to thedesign of more realistic artificial neural nets (3) Integration ofsymbolic and connectionist methods in the design of knowledgeedition environments for real applications in Medicine and Indus-try. He is the general Chairman of the biennial interdisciplinarymeetings on the interplay between Neuroscience and Computa-tion, IWANN (International Work Conference on Natural andArtificial Neural Networks). <[email protected]>

Ana E. Delgado-García is an associate professor of ComputerScience and Artificial Intelligence of the Department of ArtificialIntelligence, National University for Distance Education (UNED)in Madrid (Spain). Her current research interests include AIfundamentals and methodology, neural modelling of biologicalstructures, and co-operative processes. <[email protected]>



and, if not, a new rule must be sought, whose condition fieldcan be adapted to that of the previous one.

Characteristics of a KBSAlthough it is obvious that a KBS is more than merely a

final program, which includes the modules described in theprevious section, when the development phase is finished wedo have a program. The program as such possesses, at least, thefollowing distinct characteristics:• Competency in its field: For a program to be really consid-

ered a KBS, it must be able to resolve problems with an effi-ciency and quality of results comparable to those of a humanexpert.

• Limited domain: This is a distinguishing feature of the KBSwhen compared with other artificial intelligence programs.Domain limitation is a prerequisite to reaching competency.

• Explanation capacity: One of the characteristic features of ahuman expert is the capacity to explain how he has resolvedthe problem, i.e. the method applied and the reasons why.Likewise, a KBS should be able to explain to the user thereasoning process it employs.

• Treatment of uncertainty: This is a requirement derived fromthe complexity of the problems broached by KBS. In con-trast with formal worlds, in most real applications of KBSthere are numerous sources of uncertainty, the treatment ofwhich is complex.

• Flexibility in the dialogue: The human expert – e.g. a doctor– has tremendously flexible dialogues, accepting any infor-mation provided at any moment and generating questionsflexibly and adapting to each particular situation. It would bean asset if the KBS also possessed this capacity, being ableto communicate (understand and express itself) in naturallanguage, all within reasonable limits. We are still far fromthis situation which depends on progress being made in theunderstanding of natural language. Developments in multi-media resources are however encouraging.

• Explicit representation of knowledge: Every computerprogram is based on the knowledge possessed by its design-er about the problem which needs to be resolved. However,when we say that a system is based on knowledge, we areusing the expression in a restricted manner to indicate thatthe system has an explicit representation of knowledge.

Advantages and Limitations of KBSWe have indicated that a KBS should be able to reproduce

a human experts capacity to carry out a specific scientific-tech-nical task within a limited domain. However, KBS have certainadvantages over the real specialist, although in other aspectthey are still clearly inferior. According to Waterman [Water-man 1986] the advantages of KBS can be stated as: permanen-cy, duplication (the human expert can only be at a single phys-ical location and, obviously, is non-reproducible), reliability (aKBS will always respond in the same manner), speed, and lowcost.

In spite of these advantages, KBS also manifest great short-comings when compared with human beings.

• Common sense: There is nothing obvious to a computer. AKBS in a medical domain will easily accept that there is amale patient weighing 80 grams and measuring 175 meterswho has been pregnant for three years unless the programdesigner has explicitly indicated that this data is absurd. Nomatter how great the attempts to purify the models of knowl-edge, we can never be entirely certain that a KBS will notproduce a result which goes against the most elementarycommon sense.

• Flexibility: The flexibility of a human being is unlimited.They immediately recognize when there is an exception orwhen the case exceeds the limits of their knowledge, adapt-ing themselves to unexpected situations, seeking new solu-tions. Despite the progress being made KBS at present, andin the near future, will continue to be terribly rigid.

• Natural language: We still have a long way to go before asystem capable of formulating flexible questions and main-taining an informal conversation with a user or patient isavailable. The same can be said with regards to the capacityto generate explanations and reports.

• Sensory experience: Almost all the current expert systemsare limited to receiving information through a keyboard andmouse. Only some are equipped with image treatment andeven these are incomparable to human visual capacity. Thesame can be said about the human auditory, tactile and olfac-tory capability.

• Global perspective: A human expert is able to detect, imme-diately and reliably, the core questions and the secondaryones, making a prompt separation of relevant data and insig-nificant detail.

Along with the above, initially proposed by Waterman, wecan add the following limitations:• Lack of capacity for learning: All human experts are able to

learn from experience, constantly acquiring knowledge byinductive and abductive processes, by analogy, and by othertechniques difficult to formulate. This is so even within thefield of cognitive psychology itself, such as the formation ofconcepts or insight, and characteristics of scientific andartistic creativity.

• Capacity to handle non-structured knowledge: The humanexpert organizes and employs the available information andknowledge in a rather unorganized manner structured byinternal mechanisms which are not very well known, but areevidently capable of self-organizing this data and thatknowledge in a highly effective manner. The robustness ofthe human expert against syntactic error in natural languageis representative of that self-organizing capacity and estab-lishes a clear difference to any computational alternative, beit conventional or Knowledge-Based.

• Genuinely human functions: In the previous sections, wehave assumed that the tasks carried out by the KBS and thehuman expert were the same (both making reference toscientific-technical knowledge and having a limited domainwas limited). There is in addition a clear limitation where thetype of task and domain in which KBS are not adequatebecause the nature of the knowledge involved makes itgenuinely human and difficult to recreate in a program.

2

3



Everything related to natural language, concept forma-tion, common sense knowledge and creativity are themost obvious examples of genuinely human functionsthat still remain outside the field of KBS, at least withinthe current state of knowledge.

Therefore, to substitute a human expert with a KBS plusa computer – notwithstanding the judicial, psychological,and ethical considerations – is not possible today, and willnot be so in the short and middle term. Conversely, theimportant issue is not whether machines will duplicate orsurpass human capacities, but that the two should cooper-ate, taking advantages of each part's individual qualities. Inthis sense, the following statement which appeared as“information for the authors” in the publication Knowl-edge Acquisition is quite significant:

“What is important is not Artificial Intelligence, but theexpansion of Natural Intelligence through Knowledge-Based Systems.”

Levels and Domains of Description of a KBSIt is usual to describe KBS without including any

distinction between levels and domains of description ofthe calculus carried out by this program. As a result, it isgenerally difficult to determine what actually the KBSdoes, which knowledge has been represented, and which isartificially injected during the human interpretation of theKBS behaviour.

In order to clarify this methodological issue we considerthe taxonomy of levels introduced by Marr and Newell (knowl-edge, symbols, and hardware) superimposed on the twodomains of description of a calculus (the domain proper of eachlevel and the observer domain – that is external to the compu-tation of the level).

According to Marr [Marr 1982] it is not enough to know thecomputation at the hardware and software levels, there is also aneed for an additional level of understanding in which thenature of the information processing tasks carried out duringthe computations of the KBS is analysed and understood. Thisadditional level of description (the “theory of the calculus”) isindependent of the particular data structures and algorithms,which implement this model. To analyse or synthesize a KBS,it is necessary to use three levels (Figure 1): 1. Theory of calculus.2. Formal representation of data and algorithms.3. Implementation.

At the first level we have the theoretical grounds of the com-putation, the presentation of the problem in natural language,and a possible solution scheme in terms of the knowledge of thedomain. This Marr’s “theory of calculus” is nearly equivalentto Newell’s knowledge level [Newell 1981] put forward as ameans of modelling the knowledge we need to inject into aKBS. Nevertheless, Newell introduced the knowledge level asa new level of description beyond the symbol level whichcharacterizes the behaviour of a system in terms of its goals, itsknowledge, its beliefs, its inference procedures and a general“rationality” principle which guaranteed that the agents usedthat knowledge and those reasoning mechanisms to reach their

goals (Figure 1). The key to the level is its abstract, generic, andindependent nature, both in the application domain and in thelanguages used to represent the knowledge (frames, rules, logicor nets), and to use it (induction, deduction or abduction). Itthus looks for general and reusable architectures to specifytasks, and Problem- Solving Methods (PSMs) in line with theconcept of competence introduced by Chomsky [Chomsky1968], and KADS-type methodologies [Tansely/Hayball1993].

In order to complete our understanding of the significance ofthe calculus carried out by a KBS at the three levels previouslymentioned (knowledge, symbols and hardware) and to fairlyassess many AI results, it is necessary to make use of a distinc-tion between two domains of description. These are the level’sown domain (OD) and the domain of the external observer(EOD) which interprets computation at that level and, in mostcases also acts first as analyser then as programmer and finallyas interpreter [Mira et al. 1995], [Maturana 1975], [Varela1979] [Herrero/Mira 2000], [Mira/Delgado 2002]. The “pro-gram-interpreter” (instrument-user) dyad is necessary for deal-ing with all causality levels underlying a computation. Whenwe acknowledge the existence of an observer external to thecomputation in the prescription and descriptions of the func-tions of an AI program, we are introducing the idea of twodifferent systems of reference in which formal entities andrelations and their significances are represented (Figure 2).

Each level is characterized by a phenomenology, and byproperties and causality laws, which are intrinsic to the entitiesand relations, which constitute them. Hence, everything that

4

Marr Newell

I: Theory of Calculus

II. Algorithm

III. Implementation

Knowledge Level: (natural language, goals,purpose, beliefs, inferenceprocedures, rationalityprinciple)

Symbol Level

Physical Level

Human“compiler”

compiler

Models

Program

Hardware

Figure 1: Newell’s proposal of the three levels of description of a calculus (knowledge, program and hardware). Note the correspondence with Marr’s proposal (theory of calculus, algorithm, implementation)



happens in the descriptions of a KBS at the level’s own domainis causal, they are relations of necessity. That which “has tohappen”, happens. Thus, the relationships between the primi-tives of a programming language are specified by its compilerand follow the laws of its grammar. And the same happenswithin the formal side of the knowledge level, where logic,mathematical relations and finite state automata are used.

On the other hand, the observer can always act at the threelevels. His language includes not only natural language but alsoformal and programming languages. His function is first tomodel the knowledge to include in the KBS and then re-writethese models taking account of the limitations of the symbollevel and the knowledge loosed in the “operationalization” ofthe inferences and in the programming processes. This knowl-edge lost in the reduction (but not more) has to be injected inthe interpretation of the functions of the KBS. In the knowledgelevel of the EOD there is always more knowledge than thatwhich can be reduced to the symbol level. The rest of theknowledge (the meaning of the entities and relations of theconceptual model) is always outside the program, at the knowl-edge level and in the domain of the external observer. Only theformal model underlying these models of knowledge is passedto the computer [Mira et al. 1995], [Herrero/Mira 2000],[Mira/Delgado 2002].

A frequent source of confusion and misunderstanding in AIis the mixing of OD entities and relations with other entries in

the EOD with which they have no causal relation. The basicpoint in this section is to understand that in a KBS the part ofthe knowledge that is not formal always remains at the knowl-edge level and in the external observer’s domain.

Different Phases in the Development of a KBSWe have seen the methodological frame of levels and

domains of description of a KBS. In this frame there are fourphases, which can be distinguished in the development of aKBS, starting from the identification and preliminary analysisof the problem. If the construction of the KBS is feasible, justi-fied and appropriate, then the subsequent phases are:1. Knowledge modelling2. Operationalization of the inferences3. Implementation4. Validation and evaluation

5.1 Phase 1: Knowledge modelling at the knowledge level and in the domain of the external observer

This first task of knowledge modelling is doubtless the mostimportant in the development of KBS. It seems obvious to statethat to make a human expert's knowledge computational, wemust obtain this knowledge from the dialogue with the expertor other documentary sources (books, reports) such that weobtain a version in some intermediate representationallanguage which can facilitate its subsequent encoding using aprogramming language.

Now the question arises: what is knowledge? It is neithermatter nor energy. It is pure form. Actually, we only know of itsexistence through the human expert's effectiveness in the reso-lution of a scientific-technical problem and it is the humanexpert who possesses the knowledge. We are mere observers ofthe phenomenon of knowing and the acquisition of that knowl-edge consists, in the construction of a conceptual model of theentities and relations which the human expert supposedly uses,and of his way of reasoning, combining qualitative and numer-ical evidence about those relations with certain memorycontent which includes “what the expert knows”.

Times have changed since the acquisition of knowledge wasconceived as the task of transferring “something” from thehuman expert's head to the computer. Now it is generallyaccepted [Puerta et al. 1993] [Ford et al. 1993] that the acqui-sition of knowledge is a constructive modelling task in variouslevels, in which the aim is to analyse and understand the natureof the task, the possibility of breaking it down by one or morePSM in terms of subtasks, until we arrive at the sub-level ofprimitive inferences where we only need to specify the inputand output roles played by the domain factual knowledge foreach of the inferences.

To carry out this modelling process, the current methodolog-ical approaches like commonKADS [Schreiber et al. 1999],distribute these components of knowledge into two layers(Figure 3):

1. Layer of task, methods and inferencesThe layer of task, PSMs and inference play the role equiva-

lent to that of integrated circuits in the hardware level. Task

5ConceptualModel Formal Model

non-computableknowledge

Symbol Level (SL)



Physical Level (PL)

Program

Hardware

Knowledge Level (KL)

Ext. Observer Domain(EOD)

Own Domain(OD)

Figure 2: Introduction of the external observer agent and the subsequent distinction between two domains of description for each level of a KBS.



knowledge provides a description of which are the goals of theKBS (diagnosis, system configuration, learning, ...). PSMsknowledge specifies ways of decomposing this task down tothe level of primitive inferences, and inference knowledgedescribe the most basic reasoning steps (cover, select, estab-lish, refine, ...) in which the selected PSM decompose the task.The description of the inferences is completed with the specifi-cation of the input and output roles played by the domainknowledge entities. Currently, there are libraries of PSMs[Benjamins 1995], [Benjamins/Fensel 1998] [Benjamins/Aben1997] for different tasks, such as diagnosis [Benjamins 1995]or planning [Chandrasekaran 1990].

2. Layer of the domain knowledgeThe domain knowledge layer models the set of entities and

relations specific to a domain of application (medicine, econo-my, ...) which are independent of its use in inference. Thesestructures are usually instantiated from ontology [Schreiber etal. 1999], [Tu et al. 1995] and include static and factual knowl-edge, both of quantitative and qualitative nature. For examplehierarchies of concepts (frames) and causality relationsbetween these concepts. These data structures will play the roleequivalent to electrical signals (physical magnitudes) in anelectronic circuit.

At the end of this analysis phase we have:

1. A set of entities and relations of the domain model selectedor adapted from a domain ontology.

2. A set of inferences with the corresponding input and outputroles.

3. An inferential circuit connecting these inferences throughthe dynamic roles. This inferential circuit, again, can beselected or adapted from a library of PSM.

4. A control structure.That is to say, we have a model at the knowledge level (KL)

and in the external observer domain (EOD) (conceptual algo-rithm) to solve the task. The next step in the way to build thecode (SL into the OD) is to make operational each one of theseinferences. That is, to rewrite them in formal terms (from theEOD to the OD, without leaving the KL).

5.2 Phase 2: Operationalization of the inferencesHaving modelled the task in Phase 1, the process of reducing

the content of the two layers of the model to symbol level (theprogram) can begin.

The activities of phase 2 are:• Specify the data-structures for all the input and output roles

played by the entities of the domain layer.• Specify the formal operators used for each one of the infer-

ences. These operators can be symbolic (rules) or connec-tionist (neural nets, fuzzy, probabilistic).

The selection of formal operators is made according to thesuitability to accommodate the available balance of data andknowledge for the task, PSM, and specific domain underconsideration. If we have more knowledge than data, then“hard” operators are proper. Alternatively, if we have more datathan knowledge, then fuzzy or neural operators are moreadequate. Finally, if we have labelled data then supervisedlearning algorithms are preferred. If the data available areunlabelled, unsupervised or self-organizing operators are theonly alternative. The most usual situation in real world prob-lems is a hybrid one. We usually have insufficient data andknowledge and have to use data to improve knowledge-drivenoperators and knowledge to improve data-driven inferences.This means that the final operational architecture of all KBS ishybrid.

5.3 Phase 3: ImplementationHaving reduced the content of the two layers of the KBS

model to symbol level in Phase 2, the implementation canbegin. The activities of phase 3 are:• Selection and use of knowledge edition environments.• Use of complementary tools for the selected operators

(rules, bayes, fuzzy, neurons).• Design of user interface.• Prototype integration and preliminary validation.

Initially, the environments were oriented to representationlanguages. Nowadays, the knowledge edition environments aremore general support systems, integrating the functionalities ofthe previous ones with others closer to the dominant methodol-ogies, with explicit ontologies and libraries of reusable compo-nents of modelling such as CommonKADS [Schreiber et al.

Task

Input role(dynamic)

Output role(dynamic)

Static role

Domain Ontology(Hierarchies of Enitities & Relations)

Inference

PSM

Subtask Subtask Subtask... ...

PSM

Inference Inference Inference... ...

I

II

Figure 3: Knowledge modelling at the knowledge level: (I) Task, PSMs and inferences. (II) Domain layer.



1999] and UPML [Ford et al. 1993] or the general-purposeframework PROTÉGÉ-II [Eriksson et al. 1995].

5.4 Phase 4: Validation and evaluationHaving completed implementation in Phase 3 validation and

evaluation can begin. This is the final stage in the developmentof a KBS, before the learning process can begin. A non-validKBS is useless, as is a valid one which is not used because itdoes not fulfil the functional specifications at its origins. Inother words, to what extent does it behave like the humanexpert it seeks to substitute or advise? If we eliminate from theevaluation process those components of the KBS, which arecommon to any other type of software, what remains to be eval-uated is:• The knowledge model it incorporates in the two layers

(domain and task-PSM-inference) and in the “operationali-zation” of the entities of these layers.

• Its acceptance in real environments (validity, efficiency,completeness, integration, reliability and user friendliness).

To carry out these evaluations, there are no established anduniversally accepted methods or techniques. Evaluationsinclude preliminary trials on initial prototypes, surveys andinterviews with the final user, a series of experimentsperformed on the behaviour of the KBS when facing prototypi-cal situations, and statistical sampling representative of variouscases which the system will encounter.

Actual Tendencies in KBS DevelopmentAt present, the same features are required in the final prod-

uct both for software engineering and for knowledge engineer-ing. A KBS must exist side by side with other software productsthat involve the same pre-requisites concerning planned devel-opment, operational robustness, validity, evaluation, and bothupdating and tailoring possibilities [Mira et al. 2001].

If we were to choose three dimensions where KBS develop-ment is advancing, we would select the following:1. Emphasis in methodology. The methodological perspective

that has most contributed to bringing the development ofKBS closer to the other material and energy-engineeringdisciplines is based on:

• The introduction of knowledge-level conceptual model-ling, in the sense of Newell [Newell 1981], and the accept-ance of natural language as the primary source of informa-tion from where the entities of any knowledge model areextracted[Mira et al. 2000].

• The search for recurrent abstractions that can be detachedfrom tasks and application domain and be reused eitherdirectly or with small modifications [Clancey 1985][Chandrasekaran 1986].

• The taxonomy of knowledge in at least two layers: a moregeneric layer of tasks, methods, primitive inferences andcontrol; and a more specific, application-domain depend-ent layer, with reusable components in both cases[Schreiber et al. 1999]

2. Reuse of components from ontologies and libraries ofPSMs. The reuse of components in the buildup of KBS hasbeen manifested in the following points:

• Attempts to agree on libraries of PSM and inferences[Benjamins 1995], [Benjamins/Aben 1997], [Fensel et al.1999], [Eriksson et al. 1995] with different degrees ofproximity to the final implementation.

• The implementation of development environments, as wellas of complementary libraries of elicitation schemes andreduction methods, which guarantee that the models areoperational [Herrero/Mira 1998] [Herrero/Mira 1999].

• The introduction of intermediate entities (adapters, filters,etc.) [Fensel 1997] [Taboada et al. 2001], [Wielinga et al.1998] that cope with the difficult problem of reusability:what initially seems generic ends up being dependent, to alarge extent, on the method used and on the applicationdomain.

• The development of ontologies [Chandrasekaran et al.1999], [Guarino 1997] of reusable components in thedomain layer (entities, and relations) and the subsequentlibraries of these entities and relations for each sort of tasksand PSMs.

• Acceptance of the restrictions and limits that are commonto any computable model. Computable knowledge is a verysmall part of the knowledge that the human expert uses forthe same task. Its discourse only includes the logic-rela-tional components, the facts and concepts that can bedefined by means of operators with numerical values orlabels (also numbers at the end) that have a pre-definedmeaning. The rest is interpretation and this makes theinstrumental nature of the whole of knowledge engineeringobvious [Herrero/Mira 2000] [Mira et al. 1995].

3. Hybrid architectures integrating hard and soft computingPSMs, inferences and formal operators.

Finally, the time when connectionism and symbolic AIcompeted against each other is now over. The rivalry was dueto ignorance [Mira et al. 1995]. Now is generally accepted thatthey are different PSMs and formal tools that belong to a broadlibrary of reusable components for knowledge modelling and“operationalization”, according to the nature of the applicationand the specific balance between data and knowledge available[Mira et al. 1995], [Mira/Delgado 2002], [Mira et al. 2001].

References[Benjamins 1995]

Benjamins, R. Problem-Solving Methods for Diagnosis and theirRole in Knowledge Acquisition. Int J of Expert Systems:Research and Applications, vol 2 no 8: 93–120. 1995.

[Benjamins/Aben 1997]Benjamins, R., Aben, M. Structure-Preserving KBS Develop-ment Through Reusable Libraries: A Case-study in Diagnosis.Int. J. Human-Comput. Studies, 47, 259–288. 1997.

[Benjamins/Fensel 1998]Benjamins, R., Fensel, D. Editorial: Problem-Solving Methods.Int J Human-Computer Studies vol 49 no 4: 305–313. 1998.

[Buchanan/Shorliffe 1984]Buchanan, B.G., Shorliffe, E.H. Rule-Based Expert Systems: TheMYCIN Experiments of the Stanford Heuristic ProgrammingProject. Addison-Wesley, Reading, MA. 1984

[Clancey 1985]Clancey, W.J. Heuristic Classification. Artificial IntelligenceNeural Netw, Complex Probl.-Solving Technol 27: 289–350.1985.

6



[Chandrasekaran 1986]Chandrasekaran, B. Generic Tasks in Knowledge -Based Reason-ing: High-Level Building Blocks for Expert System Design.IEEE Expert; 1: 23–9. 1986.

[Chandrasekaran 1990]Chandrasekaran, B. Desigm Problem Solving: A Task Analysis.AI Magazine, 11. 59–71. 1990.

[Chandrasekaran et al. 1999]Chandrasekaran, B., Josephson, J.R., Benjamins, R. What areOntologies, and Why do we Need Them?. IEEE IntelligentSystems vol 14 No 1: 20–26. 1999.

[Chomsky 1968]Chomsky, N. Language and Mind. Harcourt Brace & World Inc.,N. York. 1968.

[Eriksson et al. 1995]Eriksson, H., Shahar, Y., Tu, S.W., Puerta, A.R., Musen, M.A.Task MODELLING with Reusable Problem-Solving Methods.Artificial Intelligence 79 2: 293–326. 1995.

[Fensel 1997]Fensel, D. The Tower-of-Adapter Method for Developing andReusing Problem-Solving Methods. In: Benjamins, R., Plaza, E.et al. (eds) Knowledge Acquisition, Modelling and Management.Proceeding of the 10th European Workshop EKAW’97. LNAI1319. Springer-Verlag, Berlin, 97–112. 1997.

[Fensel et al. 1999]Fensel, D. et al. UPML: A Framework for Knowledge SystemReuse. Proc 17th Int Joint Conf Artificial Intelligence (IJCAI99),Morgan Kaufmann, San Francisco. 1999.

[Ford et al. 1993]Ford, K., Bradshaw, J.M., Adams-Webber, J.R., Agnew, N.M.Knowledge Acquisition as a Constructive Modelling Activity.International Journals of Intelligent Systems; 8: 9–32. 1993.

[Guarino 1997]Guarino, N. Understanding Building and Using Ontologies. Int JHuman-Comput Studies 46, (2–3): 293–310. 1997.

[Herrero/Mira 1998]Herrero, J.C., Mira, J. In Search of a Common Structure Under-lying a Representative Set of Generic Tasks and Methods: TheHierarchical Classification and Therapy Planning Cases Study.In: Mira J, del Pobil AP, Moonis A. (eds) Methodology and Toolsin Knowledge-Based Systems. LNAI 1415. Springer-Verlag,Berlin, 21–36. 1986.

[Herrero/Mira 1999]Herrero, J.C., Mira, J. SCHEMA: A Knowledge Edition Interfacefor Obtaining Program Code from Structured Descriptions ofPSM’s. Two Cases Study. Applied Intelligence 10: 2/3, 139–153.1999.

[Herrero/Mira 2000]Herrero, J.C., Mira, J. Causality levels in SCHEMA: A Knowl-edge Edition Interface. IEE Proceeding-Software Engineering,vol 147 No 5 pp 191–200. 2000.

[Marr 1982]Marr, D. Vision. Freeman, New York. 1982.

[Maturana 1975]Maturana, H. The Organization of the Living: a Theory of theLiving Organization. Int. J. Man-Machine Studies; 7: 313. 1975.

[Mira et al. 1995]Mira, J., Delgado, A.E., Boticario, J.G., Díez, F.J. Aspectos Bási-cos de la Inteligencia Artificial. Editorial Sanz y Torres, Madrid.1995.

[Mira et al. 2000]Mira, J., Alvarez, J.R., Martínez, R. Knowledge Edition andReuse in DIAGEN: A Relational Approach. IEE Proceeding-Software Engineering, Vol. 147, No. 5: 151–162. 2000.

[Mira et al. 2001]Mira, J., Martínez, R., Álvarez, J.R., Delgado, A.E. DIAGEN-WebDB: A connectionist approach to medical knowledge repre-sentation and inference. In: J. Mira, A. Prieto (eds.) Connectionistmodels of Neurons, Learning Processes, and Artificial Intelli-gence. LNCS 2084, Springer Verlag, Berlin, 772–782. 2001.

[Mira/del Pobil 2000]Mira, J., del Pobil, A.P. Knowledge Modelling for Software Com-ponent Reuse. IEE Proc–Softw., vol 147 no 4: 149–150. 2000.

[Mira/Delgado 2002]Mira, J., Delgado, A.E. Where is Knowledge in Robotics? SomeMethodological Issues on Symbolic and Connectionist Perspec-tives of AI. In Ch. Zhou, D. Maravall and Da Ruan (eds.) Auton-omous Robotic Systems. Ch. I, 3–34. Physica- Verlag. Springer-Verlag. 2000.

[Newell 1981]Newell, A. The Knowledge Level. AI Magazine, summer 1–20.1981.

[Puerta et al. 1993]Puerta, A.R., Tu, S.W. Musen, M.A. Modelling Tasks with Mech-anisms. International Journals of Intelligent Systems; 8: 129–52.1993.

[Schreiber et al. 1999]Schreiber et al. Engineering and Managing Knowledge: TheCommonKADS Methodology. The MIT Press. 1999.

[Taboada et al. 2001]Taboada, M., Des, J., Mira, J., Marín, R. Diagnosis Systems inMedicine with Reusable Knowledge Components. IEEE Intelli-gent Systems vol 16 no 6: 68–73. 2001.

[Tansely/Hayball 1993]Tansely, D.S.W., Hayball, C.C. Knowledge-Based Systems Anal-ysis and Design. A KADS Developer´s Handbook. Prentice Hall,.1993.

[Tu et al. 1995]Tu, S.W., Eriksson, H., Gennari, J.H., Shahar, Y., Musen, M.A.Ontology-based Configuration of Problem-solving Methods andGeneration of Knowledge-acquisition Tools: Application ofPROTÉGÉ-II to Protocol-based Decision Support. ArtificialIntelligence in Medicine; 7: 257–89. 1995.

[Varela 1979]Varela, F.J. Principles of Biological Autonomy. The NorthHolland Series in General Systems Research. North Holland.1979.

[Waterman 1986]Waterman, D.A.: A Guide to Expert Systems. Addison-WesleyPub. Comp. Reading. Mass. 1986.

[Wielinga et al. 1998]Wielinga, B.J., Akkermans, J.M., Schreiber, A. A CompetenceTheory Approach to Problem-Solving Method Construction. IntJ Human-Computer Studies vol 49 no 4: 315–338. 1998.



Cooperating Physical Robots and Robotic Football

Bernhard Nebel and Markus Jäger

Having a robot that carries out a task for you is unquestionably of some help. Having a group of robots seemsto be even better because in this case the task may be finished that much quicker and more reliably. In spiteof this dealing with a group of robots can make some problems more difficult. In this paper we sketch someof the advantages and some of the disadvantages that come up when dealing with groups of robots.

Keywords: AI, Robots, Multi-Robot Systems, CooperativeSensor Interpretation, Cooperative Cleaning, Team Coordina-tion Robots, Robotic Football, Multirobot Systems, RoboCUP.

Introduction Having a robot that carries out a task for you, for example,

cleaning the floor or fetching the mail, is unquestionably ofsome help. Having a group of robots seems to be even betterbecause in this case the task may be finished that much quickerand more reliably. Sometimes a group may even be required toget the task done, as in the case of robotic football, which re-quires a team of robots.

In general, some problems can be more easily, more reliably,or more quickly solved by a group of robots. For example,cleaning a large room [Jäger 2002] or distributing mail ormessages to many recipients can be completed faster with agroup of robots. This has been demonstrated by the team of SRIrobots winning one of the robot competitions at AAAI’96[Guzzoni et al. 1997]. Basic tasks such as self-localization canalso be more reliably solved by a group of robots. In contrast,dealing with a group of robots can make some problems thatmuch more difficult. A good example of this is path planning,which is easier for one robot than for a group of robots.

So what is it that makes such multi-robot systems interest-ing? A cooperating group of robots appears to simply be aspecial case of a multi-agent system. While this is certainly atrue statement, it is true in the same sense as the statement thatrobots are “simply” special cases of agents. Mobile robots arespecial in a variety of ways. For this reason, one has to dealwith problems that do not arise with other agents, such as soft-ware agents. Firstly, there is the problem that a robot has toperceive and act in a physical world. This leads to the secondproblem, sensing and acting. This is inherently uncertain.Finally, when considered in conjunction with the two formerpoints, communication between the robots might not be possi-ble, it may be restricted to low bandwidth, or it may be possibleonly over restricted distances. Apart from these issues a groupof robots can be viewed as a multi-agent system.

One should note, though, that a group of robots might also beviewed as one centrally controlled multi-bodied robot. Whilesuch a viewpoint is indeed possible there are a number of argu-ments against such a perspective. Firstly, this multi-bodied

robot has a very large number of degrees of freedom, thus mak-ing it computationally infeasible to control. Secondly, wemight be unable to communicate between the different parts ofthe robot or the communication bandwidth may be very low.Thirdly, we might be unable to estimate the global system statebecause for some parts of the robot we do not know that state.Fourthly, failures of parts of a multi-bodied robot are muchmore naturally dealt with when one takes a multi-agentperspective.

In the rest of the paper, we will present some case studies oftechniques developed in the context of multi-robot systems. Inthe next Section, we have a look at cooperative sensing. InSection 3 and 4, we then turn to particular forms of coordinatedbehaviour, namely, cooperative motion planning and areapartitioning. In order to support cooperative behaviour in agroup, often roles are assigned to the group members. How thiscan be done for a group of robots in a highly dynamic environ-

1

Bernhard Nebel is a professor in the Department of ComputerScience at Albert-Ludwigs-University Freiburg (Germany), anddirector of the research lab on foundations of Artificial Intelli-gence. He received a Ph.D. degree from the University of Saar-land (Germany) in 1989 for his work on knowledge representa-tion and belief revision. He is a member of the IJCAI Inc board oftrustees and a member of the graduate school on mathematicallogic and its applications at Albert-Ludwigs-University. He is afellow of the ECCAI. Among other professional services, heserved as the program chair for the 17th International Joint Con-ference on Artificial Intelligence (IJCAI’01). In addition, he is theresearch note editor of Artificial Intelligence, and used to be anassociate editor of the Journal of Artificial Intelligence Research.His research interests include temporal and spatial reasoning,constraint-based reasoning, action planning, and robotics.<[email protected]>

Markus Jäger is a graduate student in Computer Science andis currently finishing his doctoral thesis at Albert-Ludwigs-Uni-versität Freiburg (Germany) about cooperating cleaning robots.Since June 2000 he has had a Ph.D. grant from Siemens AG(Germany). He has published a number of papers on the subjectof cooperating robots, especially on fully decentralized collisionavoidance and area partitioning.<[email protected]>



ment will be studied in Section 5, where the role assignmentproblem in robotic football is investigated. Finally, in Section6, we will summarize and discuss how robotic football helps toadvance the state of the art in the area of multi-robot-systems.

Cooperative SensingIf there is a group of robots that can communicate with

each other, it follows that these robots should share their obser-vations with each other. In this manner they can compensate forsensor limitations that, for example, restrict the range in whichan object can be sensed. Furthermore, by combining estimates,robots may be able to narrow down their hypotheses or tocorrect their estimates.

As mentioned in the Introduction, all sensor measurementsare uncertain. There is always some (normally distributed)noise and in addition there may be some systematic error thatwe cannot anticipate. For instance, when using the odometry –measuring how often a wheel has turned – there is a normallydistributed measurement error and, depending upon the floor,there may also be an additional systematic error. Carpets inparticular can lead to systematic diversions that cannot beanticipated. There may also be a large displacement from timeto time when the robot collides with an obstacle or with anotherrobot. All of these errors accumulate, leading to high degree ofuncertainty about the robot’s position after a very short amountof time.

For these reasons other means are used to solve the so-calledself-localization problem. Measurements from other sensorsare used to correct the estimates derived from the odometry.The mathematical tool that is used to deal with this problem isoften the Kalman filter [Maybeck 1990]. This is a method ofcombining all measurements in order to arrive at an optimalestimate. Intuitively it involves computing a weighted averageover sensor measurements, where sensors that are more accu-rate have a higher weighting than those which are known to beless accurate.

Known positions of recognized landmarks are also oftenused in the self-localization process for correcting the position-al estimates. When one wants to explore an unknown territory,however, there are no known landmarks. With a group of robotsthat have initially known positions, though, it is possible to dosomething similar to landmark-based navigation. This isbecause some of the robots themselves can be used as land-marks.

2.1 Cooperative Self-LocalizationRekleitis et al. [Rekleitis et al. 1997] proposed a scheme for

multi-robot exploration with a group of robots. In this approachit is assumed that the robots can track each other with reasona-ble reliability and accuracy as long the line of sight betweenthem is free of obstacles. Under this assumption, one or morerobots can move using one or more temporarily immobilerobots as landmarks. After a while the roles of the moving andunmoving robots can be exchanged. By using such a method,the odometry error can be reduced dramatically [Rekleitis et al.1997].

In most applications, however, we already know the environ-ment and “only” have to solve the self-localization problem. Inthis case it often happens that one robot can come up with mul-tiple position hypotheses. If we have a group of robots that areable to recognize other group members when they are closeenough, it is possible for the robots to narrow down the set ofposition hypotheses when they meet [Fox et al. 1999].

2.2 Cooperative Object LocalizationYet another scenario for multi-robot cooperative sensing is

when we can assume that the position and orientation of allrobots are almost always accurate and reliable, but there issignificant uncertainty and unreliability in the sensing of otherobjects. This situation occurs, for instance, in the robotic foot-ball context as proposed by the RoboCup initiative [Kitano etal. 1997].

The players of the CS Freiburg team [Gutmann et al. 2000][Nebel et al. 2000] [Weigel et al. 2001] use laser range findersin order to solve the self-localization problem [Gutmann et al.2001], and for this reason can be assumed to know their ownposition very reliably. They are not, however, very good atrecognizing the ball and estimating its position on the field –this is done using a monocular vision camera.

There is a significant measurement error when estimating thedistance to the ball, an error that increases with the distancebetween camera and ball. The angular error, on the other hand,is smaller and does not depend on the distance. Additionally,there is a restriction as to the maximum distance over which theball can be recognized, which is approximately 4–5 meters. Asa final point, the robots frequently recognize false positives, i.e.phantom balls. Ignoring this latter problem, one can again usea Kalman filter to combine observations, with time stamps,from different robots in order to get estimates that are more ac-curate than any single measurement. This gives us, in fact, asort of stereo vision with a group of robots. Assuming that theangular error is much smaller than the distance error gives a tri-angulation effect as shown in Figure 1.

As already pointed out, sometimes the robots will sometimesobserve phantom balls. An example of such a situation is illus-

2

Figure 1: The Kalman filter for integrating ball observations leads to triangulation. Grey discs denote position estimates for single robots, the ellipses around the grey discs denote measurement errors, and the white disc denotes the fused combined estimate.



trated in Figure 2. Two players see a ball close to the goal andanother player sees a ball on the centre line.

If we were now to take the weighted average of the sensedball positions, we would get an extremely inaccurate estimate.For this reason, it seems preferable to exclude obviously wrongmeasurements. One way to do so would be to ignore measure-ments that are implausible given the current estimate. This,however, could lead to a situation where a robot tracks a phan-tom ball and the other robots are all diverted from sensing theright ball because they believe the hallucinating robot.

The best (and most democratic way) to deal with such a situ-ation is to believe in what the majority of robots sense. In theexample depicted in Figure 2 we would rather believe the play-ers 1 and 3 than player 2. One way to put such a voting schemeinto effect is to use the so-called Markov localization approach[Fox et al. 1999] for the ball. In this approach one basicallymaintains a discrete probability distribution for the positionprobability of the object of interest. Usually, this is the robotitself. But in our case it is the ball. Each observation updates theposition probability by increasing the probability at the loca-tion where the ball has been observed and lowering the proba-bility where no observation has been made, using conditionalobservation probabilities and Bayes’ rule. In addition, the posi-tion probability is flattened out for each time step to model theloss of certainty over time. Using such an approach it appearsmore probable that the ball is around the locations where it hasbeen observed by two robots, rather than at the that which it hasbeen observed by only one. Combining Markov localization asa plausibility filter with a Kalman filter, one gets a quite reliableand accurate global ball estimation mechanism [Dietl et al.2001].

While all these methods might not appear to be overlysophisticated, the real value of these approaches is that they arebased on a solid theoretical basis and work in practice. Almostall of these approaches, however, are still passive in the sensethat they do not involve the interplay between sensing andacting, i.e., active sensing [Burgard et al. 1997].

Cooperative Path and Motion PlanningLatombe [Latombe 1991] starts his book with the follow-

ing remark:

This capability [motion planning] is eminently necessarysince, by definition, a robot accomplishes tasks by moving inthe real world.

And what is true for single robots is, of course, also true forteams of robots.

The basic motion planning problem is usually stated[Latombe 1991] as the problem of moving a single rigid object– the robot – in an Euclidian (2- or 3-dimensional) space, theso-called workspace, from an initial position and orientation toa target position and orientation. There can, of course, beobstacles in the workspace that must be avoided. Mapping theproblem to the (so-called) configuration space usually solves it.This space is generated by the degrees of freedom the robot has.In the 2-dimensional case, these degrees of freedom are (x, y,θ), i.e., the x and y coordinates of the robot position as well asits heading θ. In the 3-dimensional configuration space the ro-bot is just a point and we have to find a path from the start tothe target avoiding obstacles. In the special case that we havedisk-shaped robots, the configuration space can be describedby the x and y coordinates alone and so the configuration spaceis only 2-dimensional.

In the previous Section, we have seen how sensing can leadto more accurate and reliable estimates if we have a group ofrobots. Furthermore, the additional computational costs arereasonable. In contrast to that, path and motion planning iscomputationally much more difficult if a group of robots isinvolved. This becomes obvious when one generalizes theconfiguration space planning method described above to a mul-ti-robot system. In this case 3 dimensions have to be added tothe configuration space for each robot. Of course, this might bean indication that the configuration space approach is notappropriate, however, the multi-robot path planning problem isindeed inherently difficult. Indeed it is PSPACE-hard in thenumber of robots, as follows from results by Hopcroft[Hopcroft et al. 1997].

3.1 Cooperative Path Planning with Global Communica-tion

If we assume that all robots can communicate with each otherthen the multi-robot path planning problem can be solvedcentrally, for example, by using the configuration spaceapproach sketched above. While this guarantees optimality and

3

Figure 2: Player 2 observes a phantom ball, which may lead to an incorrect estimate of the ball position.

Figure 3: Paths planned in a decoupled approach



completeness, it is usually not efficient enough for even a mod-erate number of robots.

Instead of a centralized approach one can use decoupledplanning [Latombe 1991]. In this approach, we first plan inde-pendent trajectories for all of the robots and then combinethem, resolving conflicts wherever they arise. This reduces thecomplexity, but it also sacrifices optimality and completeness.

There are two decoupled planning methods that have beenconsidered in the literature. Firstly, there is the prioritizedplanning approach [Erdmann/Lozano-Pérez 1987], which con-siders the multi-robot path planning problem as a sequence ofpath planning problems. It starts with the first robot and all theimmobile obstacles. Then it adds another robot and plans a pathavoiding all immobile obstacles and the moving robot from thefirst phase, and so on. The critical decision is the order which itshould use. Depending on this order, it is quite possible that nosolution is found although one does exist.

The other decoupled approach is the so-called path coordina-tion method [O’Donnell/Lozano-Pérez 1989], in which therobots plan their paths independently (see Figure 3), and after-wards coordinate their movements without leaving theirplanned paths. As is obvious from this example, there is achance that the two robots collide if they follow their plannedpaths without coordination. Thus what we need here is a colli-sion-free schedule that the robots can follow. For two robots,this problem can be solved using so-called coordinationdiagrams, like the one shown in Figure 4. Whilst this coordina-tion diagram shows that with two robots the problem can beeasily solved, it also gives a hint that the problem mightbecome computationally difficult when the number of robotsincreases – the generalized coordination diagram wouldcontain as many dimensions as there are robots. For this reasonprioritization schemes are often used [Azarm/Schmidt 1996].

Path coordination is, of course, more restrictive than theprioritized planning approach and for this reason may find

fewer solutions, which may be less cost-effective. This isindeed the case even in natural environments, as has beenshown by Bennewitz and Burgard [Bennewitz/Burgard 2000].

3.2 Cooperative Path Planning with only Local Communi-cation

Although the decoupled path planning methods do notattempt to control the group of robots as one entity – and reducethe algorithmic complexity by that – they do still presupposethat there is a central coordinator and a global communicationsnetwork. If one assumes that only local communicationbetween pairs of physically close robots is possible, then thedecoupled approaches do not work.

Similar to the decoupled approaches, we will assume that thepaths are planned independently by each robot. However,instead of relying on a central component that deals withconflicts, we will now assume that only local coordination ispossible [Jäger/Nebel 2001].

If two robots are close to each other, they establish a coordi-nation link, which means that they create a coordination dia-gram, which determines their schedule. It now can happen thatone robot (A) has to wait for the other one (B). Unfortunately itmight happen that robot B may also have to wait for anotherrobot C, which in turn waits for A, i.e., we have a deadlock.These deadlocks have, of course, to be detected and resolved.Resolution of a deadlock can mean that one tries to find analternative path in the coordination diagram or that a new path(segment) is planned [Jäger/Nebel 2001]. The right tools foruse here are distributed algorithms for deadlock detection andresolution [Chandy et al. 1983].

Area PartitioningIf multiple robots are used to cooperatively clean a large

room, for example in an airport, then the area that has to becleaned must be partitioned amongst the robots. This partition-ing can be done either statically [Bast/Hert 2000] or dynami-cally. A static approach assigns each robot a certain subarea atthe beginning. Each robot is then only responsible for itsassigned subarea. The main disadvantage of a static assignmentis that the whole system can not be adapted dynamically to anew situation. If, for example, a robot breaks down, the other

4

Figure 4: Coordination diagram. The s1 and s2 axes represent the length robot 1 or robot 2, respectively, has already traveled on the respective planed path. Black cells represent collisions. The bold line shows a collision-free schedule.

Figure 5: Division of an area into polygons



robots can not take over its work, or if a robot is slower thanassumed, the other robots are not able to help him.

A dynamic partition assigns the subareas during runtime. Itis therefore possible to react to unpredictable events. If, forexample, a robot breaks down, the other robots can dynamical-ly take over his work. Here, a completely dynamic approach isintroduced. The main idea is to divide the room into polygons,as shown in Figure 5. The robots then allocate and clean thearea covered by these polygons. To allocate a polygon meansthat a robot intends to clean the area covered by the polygonand announces this to the other robots. In Figure 6, three robotsare processing a large room. The dark polygons are cleanedregions, the light polygons are allocated regions, and the whiteregion is not yet allocated to any robot.

The developed allocation strategy allows the robots to parti-tion the area in a reasonable way, although they are not able tocommunicate with each other all the time. The radius of thecircles around the robots (Figure 6) is half of the communica-tion radius of the robots, i. e. the robots are able to communi-cate with each other only if their circles intersect.

Jäger and Nebel [Jäger/Nebel 2002] give more informationon this area partitioning strategy and some simulation results.

Role Assignment in Dynamic EnvironmentsThe previous two Sections have focused on problems such

as sensing and path planning. In these cases, the solutionsappeared to be very robotics specific and the overlap withmulti-agent systems seems to be minimal. However, there are,of course, other multi-robot problems that have a true multi-agent flavour.

One such problem is the assignment of roles to members ofa group [Stone/Veloso 1999]. Such an assignment serves thepurpose of associating a set of behavioural patterns with theagents in order to support the coordination between all of theagents. In football, for example, we distinguish at least betweenthe roles goalie, defender, and forward player. There may be

additional roles such as midfielder and supporter. In general wewant to determine a one-to-one function from the set of agentsA to the set of roles R. Sometimes, we may also want toconsider different sets of roles, i.e., different formations. Thuswe may want to deal with a 4-3-3 and a 3-3-4 formation and toswitch between these formations.

A very simple way of dealing with this problem is to use afixed assignment, as for example the CS Freiburg team did in1998 [Gutmann et al. 2000]. Each robotic football player has afixed role, which has an associated home position and an areaof competence, as shown in Figure 7.

When the ball moves into such an area of competence, therespective robot becomes active and tries to move the ball intothe direction of the opponent goal – without leaving its ownarea of competence. Whilst this strategy does not appear to beoptimal, it avoids the problem that a swarm of robots approach-es the ball. In fact, only one robot of the team can be at the ballat any one point in time.

It is, however, also clear that this delegation of duties has anumber of severe problems. First of all, defending robots cannever run with the ball over the entire field and score a goal –they always have to pass the ball to a forward player. It was forthis reason that very early on a “shouting” protocol was imple-mented that permitted a robot with the ball to make a runtoward the opponent goal without being stopped by its ownteam members (see Figure 8).

A second disadvantage of the scheme described above is thedisjointed decomposition of the field. It happened that the ballwas in one of the competence areas, but the respective playerwas unable to go for the ball for some reason. No other player

5

Figure 6: Three robots processing a large supermarket in Bussum, Netherlands.

Figure 7: Role assignment and areas of competence

Figure 8: Shouting in order to get a free run to the opponent goal



would come to provide assistance. This problem can be (andhas been) solved by allowing overlaps of the competence areasand using the “shouting protocol” to avoid the situation thattwo players block each other at the ball. Finally, there is theproblem that once a robot breaks down, its role will not be filledby another robot on the field, even if that role is very important.

Although the CS Freiburg team became RoboCup worldchampion of the F2000 league in 1998, this was certainly notbecause its role assignment and coordination method weresuperior to that of the other teams. Indeed, there are a numberof issues one has to address in order to build a flexible androbust team:• role assignments should be changed dynamically to account

for the current positioning and to support team reconfigura-tions after the break down or removal of individual teammembers; and

• flexible positioning that takes into account the entire situa-tion on the field.

The latter point has been addressed by CMUnited’s SPARmethod [Stone et al. 1999], a method that tries to find the opti-mal position by using a linear programming method given thevalues for a number of important parameters such as ball posi-tion, position of team members, etc. Stone and Veloso[Stone/Veloso 1999] also addressed the issue of dynamic rolere-assignments. However, this approach was built on so-calledlocker room agreements, i.e., pre-built plans, and on fillingmore preferable roles if they became vacant.

A more flexible scheme for the dynamic assignment of roleshas been proposed and used by the ART Italy team in 1999[Castelpietra et al. 2001]. They consider roles such as• active player, the player which possesses the ball or goes to

the ball;• supporter, the player that moves parallel with the active

player;• defender, the player staying behind defending the goal.

Each agent can contribute some utility when filling a role.For instance, if a robot is already close to the defending posi-tion, it can contribute a high utility value when it fills thedefender role. If it is close to the ball, it can contribute a highutility value if it fills the active player role. Each robot deter-mines these utility values for each role and transmits the com-puted values to all other robots.

The roles for the field players are ordered by importance andassigned dynamically (in the importance order) to the playerthat can contribute most by filling the role according to thecomputed values. In fact, this assignment is done in a distribut-ed manner, i.e. each robot decides on the basis of the receivedutility values which role to take. This can lead to situations,where two players temporarily fill a role. However, this doesnot happen very often and is resolved after a fraction of asecond [Castelpietra et al. 2001].

While this scheme appears to work very well, its efficiencyseems to rely on ordering the roles by importance. A moregeneral scheme would view the role assignment problem as anoptimization problem, where we want to maximize the socialwelfare of the entire group. While this sounds like a combina-torial, i.e., a computationally difficult problem, it is simply the

problem of finding a maximal weighted match [Galil, 1986],which can be solved in polynomial time. In the CS Freiburgteam, this more general scheme together with a communicationprotocol for changing roles in a consistent manner was used[Weigel et al. 2001]. This is complemented by a variant of theSPAR method [Stone et al. 1999] to find the right position foreach role.

Conclusions and DiscussionAs should have become obvious from what has been said

so far, research in multi-robot systems is an interesting researcharea with challenging problems. In particular, service robotswill become a reality only if we are able to develop flexible androbust methods for the coordination of robot groups – be it forsensing or acting.

Robotic football as proposed by the RoboCup initiative is anattractive area for developing and comparing different methodsunder “hostile” conditions. The most interesting aspect ofrobotic football, however, is the need to design and implementsystems that “close the loop” from observation over actionplanning and action selection to action execution and monitor-ing. It is not enough to come up with methods that work inisolation under ideal conditions, but one has to use methodsthat can be integrated into a large system and that show gracefuldegradation when something goes wrong.

Of course, the RoboCup environment has a number of prop-erties which that may limit transferability to other multi-robotapplications. For example, on a robotic football field one canassume that communication radius is not a limiting factor inestablishing successful communication links between theagents, while in larger scale applications one might be verywell forced to consider local, temporary, ad-hoc communica-tion links, as described in Section 3.2. Also, the robots canalmost always observe the rest of the group, which may not betrue when operating in larger scale environments. The necessityto be responsive to dynamic changes, to substitute roles of ro-bots that have failed and to be robust in general, however, areproperties that one will find in almost all multi-robot applica-tions.

References [Bast/Hert 2000]

H. Bast and S. Hert, The Area Partitioning Problem, In Proceed-ings of the 12th Canadian Conf. on Computational Geometry,pages 163–171, 2000

[Azarm/Schmidt 1996]K. Azarm and G. Schmidt. A decentralized approach for theconflict-free motion of multiple mobile robots. In Proceedings ofthe International Conference on Intelligent Robots and Systems(IROS’96), pages 1667–1675. IEEE/RSJ, 1996.

[Bennewitz/Burgard 2000]M. Bennewitz and W. Burgard. An experimental comparison ofpath planning techniques for teams of mobile robots. In Auto-nome Mobile Systeme. Springer-Verlag, 2000.

[Burgard et al. 1997]W. Burgard, D. Fox, and S. Thrun. Active mobile robot localiza-tion. In Proceedings of the 15th International Joint Conference onArtificial Intelligence (IJCAI-97), Nagoya, Japan, Aug. 1997.Morgan Kaufmann

6



[Castelpietra et al. 2001]C. Castelpietra, L. Iocchi, M. Piaggio, A. Scalza, and A. Sgorbis-sa. Communication and coordination among heterogeneous mid-size players: ART99. In P. Stone, G. Kraetzschmar, and T. Balch,editors, RoboCup-2000: Robot Football World Cup IV, LectureNotes in Artificial Intelligence, Berlin, Heidelberg, New York,2001. Springer-Verlag.

[Chandy et al. 1983]K. M. Chandy, J. Misra, and L. M. Haas. Distributed deadlockdetection. ACM Transactions on Computer Systems,1(2):144–156, May 1983.

[Dietl et al. 2001]M. Dietl, S. Gutmann, and B. Nebel. Cooperative sensing indynamic environments. In Proceedings of the InternationalConference on Intelligent Robots and Systems (IROS’01).IEEE/RSJ, 2001.

[Erdmann/Lozano-Pérez 1987]M. A. Erdmann and T. Lozano-Pérez. On multiple movingobjects. Algorithmica, 2(4):477–521, 1987.

[Fox et al. 1999]D. Fox, W. Burgard, and S. Thrun. Markov localization formobile robots in dynamic environments. Journal of ArtificialIntelligence Research, 11:391–427, 1999.

[Fox et al. 2000]D. Fox, W. Burgard, H. Kruppa, and S. Thrun. Collaborativemulti-robot localization. Autonomous Robots, 8(3), 2000.

[Galil, 1986]Z. Galil. Efficient algorithms for finding maximum matchings ingraphs. ACM Computing Surveys, 18:23–38, 1986.

[Gutmann et al. 2000]J.-S. Gutmann, W. Hatzack, I. Herrmann, B. Nebel, F. Rittinger,A. Topor, and T. Weigel. The CS Freiburg team: Playing roboticfootball based on an explicit world model. The AI Magazine,21(1):37–46, 2000.

[Gutmann et al. 2001]J.-S. Gutmann, T. Weigel, and B. Nebel. Fast, accurate, and robustself-localization in polygonal environments. Advanced RoboticsJournal, 2001.

[Guzzoni et al. 1997]D. Guzzoni, A. Cheyer, L. Julia, and K. Konolige. Many robotsmake short work: Report of the SRI International mobile robotteam. The AI Magazine, 18(1):55–64, 1997.

[Hopcroft et al. 1997]J. E. Hopcroft, J. T. Schwartz, and M. Sharir. On the complexityof motion planning for multiple independent objects: PSPACE-hardness for the ‘warehousman’s problem’. International Journalof Robotics Research, 3(4):76–88, 1984.

[Jäger/Nebel 2001]M. Jäger and B. Nebel. Decentralized collision avoidance, dead-lock detection, and deadlock resolution for multiple mobilerobots. In Proceedings of the International Conference on Intelli-gent Robots and Systems (IROS’01), pages 1213–1219.IEEE/RSJ, 2001.

[Jäger/Nebel 2002]M. Jäger and B. Nebel, Dynamic Decentralized Area Partitioningfor Cooperating Cleaning Robots, In Proceedings of the Interna-tional Conference on Robotics and Automation (ICRA’02), pages3577–3582. IEEE, 2002

[Jäger 2002]M. Jäger, Cooperating Cleaning Robots, In Proceedings of Dis-tributed Autonomous Robotic Systems (DARS 2002), pages237–246. 2002

[Kitano et al. 1997]H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, E. Osawa, and H.Matsubara: RoboCup: A challenge problem for AI. AI Magazine18(1):73–85, 1997.

[Latombe 1991]J.-C. Latombe. Robot Motion Planning. Kluwer, Dordrecht, Hol-land, 1991.

[Maybeck 1990]P. S. Maybeck. The Kalman filter: An introduction to concepts. InI. J. Cox and G. T. Wilfong, editors, Autonomous Robot Vehicles.Springer-Verlag, Berlin, Heidelberg, New York, 1990.

[Nebel et al. 2000]B. Nebel, J.-S. Gutmann, and W. Hatzack. The CS Freiburg ’99team. In M. Veloso, E. Pagello, and H. Kitano, editors, RoboCup-99: Robot Football World Cup III. Springer-Verlag, Berlin,Heidelberg, New York, 2000.

[O’Donnell/Lozano-Pérez 1989]P. A. O’Donnell and T. Lozano-Pérez. Deadlock-free and colli-sion-free coordination of two robot manipulators. In Proceedingsof the IEEE International Conference on Robotics and Automa-tion (ICRA’89), pages 484–489, 1989.

[Rekleitis et al. 1997]I. M. Rekleitis, G. Dudek, and E. E. Milios. Multi-robot explora-tion of an unknown environment, efficiently reducing the odom-etry error. In Proceedings of the 15th International Joint Confer-ence on Artificial Intelligence (IJCAI-97), pages 1340–1345,Nagoya, Japan, Aug. 1997. Morgan Kaufmann.

[Stone/Veloso 1999]P. Stone and M. Veloso. Task decomposition, dynamic roleassignment, and low-bandwidth communication for real-timestrategic teamwork. Artificial Intelligence, 110(2):241–273,1999.

[Stone et al. 1999]P. Stone, M. Veloso, and P. Riley. The CMUnited-98 championsimulator team. In M. Asada and H. Kitano, editors, RoboCup-98: Robot Football World Cup II, pages 61–76. Springer-Verlag,Berlin, Heidelberg, New York, 1999.

[Weigel et al. 2001]T. Weigel, W. Auerbach, M. Dietl, B. Dümler, J.-S. Gutmann, K.Marko, K. Müller, B. Nebel, B. Szerbakowski, and M. Thiel. CSFreiburg: Doing the right thing in a group. In P. Stone, G. Krae-tzschmar, and T. Balch, editors, RoboCup-2000: Robot FootballWorld Cup IV, Lecture Notes in Artificial Intelligence. Springer-Verlag, Berlin, Heidelberg, New York, 2001.



Autonomous Agents and Multi-Agent Systems

Carles Sierra

In this article we make a brief overview of the current situation of the field of autonomous agents,commenting on the lines of research which are attracting most attention, and the outlook for the future.

Keywords: Intelligent Agents, Multi-agent Systems, Artifi-cial Intelligence.

Agent Based ComputingThroughout the last decade there has been a marked

increase in the interest shown by the academic and industrialworld in the agent-based programming paradigm [Bradshaw1997], though many IT engineers are still either unaware of itor confuse it with the object oriented paradigm. In this article Iwill try to introduce the agent-based programming paradigm insome of its many facets and give an overview of the future linesthat research may follow, although my main aim is to encour-age readers to go on to read more papers on the subject.

In my opinion, the agent concept arose out of the concur-rence of two ideas: firstly the understanding that complexsystems are more easily designed and explained as intentionalsystems, which were studied at length in the mid 20th century[Dennet 1987], and secondly, that the design of large IT appli-cations by means of top-down methodologies is expensive anddifficult to maintain and upgrade.

The key nexus point between these ideas and their materiali-zation from an IT point of view is the notion of autonomy. ITcomponents are conceived and designed as autonomous enti-ties, independent from one another. It is the dialogue betweenthem which causes a system to run, not the procedure or func-tion calls, or the running of a method as a result of receiving amessage (or an illocution). Agents decide autonomously how toact and when to intervene in the collective solution (made by agroup of agents) to a problem.

Trends in computing in recent years reinforce the idea thatdistributing the solution via autonomous components is theway to go[Berners-Lee 1999]. In particular the trend towardsubiquitous computing, using devices with computational capa-bility in PDAs, mobile phones and even in household applianc-es, and the interconnection of all of them in networks meansthat problem solving becomes distributed naturally. It alsoseems evident that in order for us to survive in the complexnetwork of components which is going to surround us in thenear future, these components should be endowed with intelli-gence so as to interact with one another rationally and thus freeus from the burden of taking difficult – or impossible – deci-sions. We will want to delegate tasks to these devices in such away that they perform them autonomously and without bother-ing us (if the washing machine needs to negotiate with the drierabout who should have first option on the power supply and to

which particular supplier they should buy it from, we will notwant to know anything about it). The concept of an intelligentautonomous agent which fulfils all the tasks it is given withoutour direct intervention may be just the technology we need forthese activities.

When it comes to identifying the defining characteristics ofthe agent concept, developers tend to agree on three require-ments, in addition to the abovementioned one of autonomy[Wooldridge/Jennings 1995] These are: reactivity, agents haveto be systems capable of perceiving their environment (physi-cal or otherwise) and of reacting to it quickly enough for theirresponse to be effective; proactivity, agents must have objec-tives and act in such a way that those objectives are fulfilled;and, finally, social behaviour, agents must know that othersystems like them also exist and they must be able to interactwith them in order to cooperate, coordinate and/or negotiate. Inaddition to these requirements, some authors add mobility[Rothermel/Popescu-Zeletin 1997], learning capacity [Weiss1997] or benevolence before they consider a programme canrightly be called an agent.

The societies of agents which act in solving a problem areknown as multi-agent systems.

In section 2 we will look at some essential aspects to bear inmind when building agents. In section 3 we will take a look ata set of technologies associated with agent design and whichhave recently become increasingly important. Finally, we will

1

Carles Sierra received his doctorate from the UniversidadPolitécnica de Cataluña (Spain) in 1989. he is currently a Scien-tific Researcher for the Spanish Council for Scientific Research inthe Institute of Research on Artificial Intelligence. He lectures atthe School of Engineering of the Universidad Autónoma deBarcelona, and he also runs doctorate courses there and at theUniversidad Politécnica de Cataluña. He is currently president ofthe Catalan Association for Artificial Intelligence (ACIA). He hasworked in different fields since 1984: knowledge based systems,reflexive architectures, applications to medicine and his mostrecent work has been on multi-agent systems. During his careerhe has written in the region of one hundred and fifty scientificpapers and he has received various awards among which is thePrize awarded in the XI edition of the Awards of Mundo Elec-trónico (Electronic World) 1986 for the best Professional Elec-tronics entry for his book “LISP”, and the European DigitalAward for the best article on Artificial Intelligence.<[email protected]>



take a look at two of the most promising areas of applicationand see what their future prospects are.

Agent Design From the outset, agents have been used as an abstraction

tool permitting the analysis of complex systems made up ofindividualized and autonomous components, enabling an inte-grating vision of different components or modules, for exam-ple, learning, reasoning, etc., as an integrated whole. The as-pects we will be looking at here are those that refer to agentarchitecture, the infrastructures which support its execution,and methodological proposals, which are beginning to appearin spite of the comparative infancy of these systems.

2.1. Agent Oriented Software Engineering Up until now work on agent oriented software engineering

has focused on establishing a close relationship between soft-ware engineering communities and knowledge engineeringcommunities and has attempted, with a highly pragmaticvision, to provide relatively ad hoc solutions for their use inindustry [Ciancarini/Wooldridge 2001]. The key question thatresearch in this field has to find an answer to has been how thefunctionalities of an autonomous agent relate to and interactwith those of the execution infrastructure and those of the otheragents in a multi-agent system. Thus issues such as agentrequirements analysis, verification techniques, ontologydesign, or the construction of tools to facilitate the constructionand validation process, are coming under the scrutiny ofresearchers in this field. To date the results are at a relativelyearly stage although already some methodological proposalshave appeared such as AUML [Bauer et al. 2001] (based onUML [Booch et al. 1998]) and GAIA [Wooldridge et al. 1999].

2.2. Agent Architectures The design of the structure of an agent, that is, its architec-

ture, was the object of argument and debate from this area’sinception. There were initially two opposing standpoints. Thefirst standpoint was that held by those in favour of reactivearchitectures, based on the seminal ideas of Brooks’ subsump-tion architecture[Brooks 1990], whose vision of agents focusedon a direct relationship between stimulus and response (theirproposal was based on a behavioural hierarchy in which eachone was implemented as a set of situation → action type rules).The other standpoint, held by those in favour of deliberative ar-chitectures, proposed that the agents required a process ofreasoning which would mediate between perception andaction. The most paradigmatic case of this kind of deliberativearchitectures was that of BDI architectures proposed by Raoand Georgeff, in which the reasoning process was founded ona logical representation based on three modal logic operators torepresent Beliefs, Desires and Intentions (hence BDI)[Rao/Georgeff 1995] [Kinny et al. 1996] [Fischer et al. 1996].Recently hybrid architectures have been proposed combiningthe two approaches which see agent architecture in the form ofa series of interrelated layers of software. Examples of sucharchitecture are InterRap [Muller/Pischel 1994] or Touring-

Machines [Ferguson 1992]. Nowadays hybrid architectures arethe most commonly adopted.

2.3. Agent Infrastructures As we commented in the introduction, agents make most

sense when they use their social dimension, that is, they inter-act with one another to solve problems jointly. With the aim offacilitating this interaction, in recent years a great many mid-dleware technologies have been developed to enable intercon-nection between agents over heterogeneous environments. Thelayer of middleware separates the operating system and net-work levels from the application level which contains theagents (Jini [Oaks/Wong 2000] is an example of this type oftechnology). Also work has been done to extend this middle-ware and increase its functionality with a view to achievinggreater simplicity in the design of applications. Among theinitiatives in this direction we might mention: • UDDI: is an open framework for describing services,

discovering them and integrating them over the Internet. It isan initiative which is being supported by many softwarecompanies, market operators and e-business leaders.

• RosettaNet: is a consortium of tech companies formed topropose standards for eBusiness. ROSETTANET standardsinclude a non-proprietary solution which contains data dic-tionaries, an implementation framework and eBusiness andXML standards.

• ebXML: is an initiative which aims to standardize businessspecifications.

• Soap: is a lightweight mechanism for interchanging infor-mation between peers in a distributed environment usingXML.

Finally we should mention the efforts made by the founda-tion FIPA (Foundation for Intelligent Physical Agents <http://www.fipa.org>) to define agent interaction standards. The dif-ferent versions of the interaction standard have given rise to aseries of implementations (mostly open source) of agent infra-structures, JADE and FIPA OS being examples of these.

2.4. Electronic Institutions As the complexity of multi-agent systems increases so it

becomes more apparent that the level of functionality that mid-dleware can provide to facilitate agent interaction is low (basi-cally name management and message routing). Many multi-agent system applications, such as e-marketplaces for example,not only need to enable communication between agents butthey also require this communication to be governed by con-ventions, that is, by strict rules which regulate the interactions,just as happens in human institutions. Thus there is clearly aneed to incorporate organisational concepts in multi-agentsystems [Castelfranchi 1990]. The metaphor would be that ofplacing a layer between the middleware and the applications,which we could call the “social layer” which would facilitateenriched, social, and regulated interaction between the agentsof a multi-agent system. The concept of electronic institutionaims to fulfil this role in the design of complex multi-agentsystems [Noriega/Sierra 1999a]. To design these electronicinstitutions it becomes necessary to use concepts of organiza-

2



tion theory and of how knowledge of such organizations chang-es and evolves in the course of time. The fields of sociology andsocial psychology thus become allies of IT engineering in thedesign of electronic institutions.

The concept of the “electronic institution” plays a methodo-logical role (with a different approach from that proposed byGAIA) in the development of multi-agent systems, obliging thedesigner to carry out a three level specification: 1. Dialogic framework: description of the dialogic component

of the institution, including which illocutions to use, thespecific ontology which structures the content of the illocu-tions and the set of roles that the agents taking part in aninstitution may play.

2. Performative structure: set of activities (dialogic) that theagents may perform within an institution and their interrela-tions. Each activity is specified as a scene in the theatricalsense, in which the agents establish a dialogue according toa pre-established and specified protocol, in the manner of afinite state machine in which the transitions between statescorrespond to messages sent between the agents.

3. Normative rules: formal expressions which determine theconsequences of the agents’ actions (dialogic), in the formof commitments or constraints for agents’ future actions.

A tool, ISLANDER, has been developed to support thismethodology [Esteva et al. 2002].

Agent Technologies The design of agent based systems has prompted the devel-

opment of a series of technologies and has initiated a series oflines of research. In this section we will take a look at some ofthe most interesting ones to have emerged in recent years.

3.1. Communication Languages As has already been mentioned in previous sections, one of

the fundamental characteristics of multi-agent systems is theircapacity for interaction to solve complex problems [Cian-carini/Hankin 2001]. Agents need to communicate with oneanother which is why they need to be endowed with languagesthat allow them to cooperate, collaborate, negotiate, etc. Com-mon communication languages enable communicationbetween various agents, programmed over several differentlanguages, platforms and methodologies [Labrou et al. 1999].In the last decade, two languages have emerged which have metwith a great degree of acceptance among the agent designercommunity. One is KQML, developed in the early 90s thanksto an initiative by the US Defence Department, which consistsof a language and a protocol for the exchange of informationand knowledge [Finin et al. 1993]. KQML’s semantics is simi-lar to that of planning languages like STRIPS, using intentionalpredicates (beliefs and intentions) in the description of precon-ditions and postconditions [Cohen/Perrault 1979]. Morerecently the previously mentioned not-for-profit organisationFIPA (Foundation for Intelligent Physical Agents ), whose aimis to produce standards for interoperability among heterogene-ous software agents has proposed an alternative called FIPAACL based on similar principles. Currently it seems that forceshave been joined to concentrate on this second proposal. The

philosophical and linguistic roots of these languages are to befound in the work on pragmatics carried out by Austin andSearle in the 60s [Austin 1962] [Searle 1969].

3.2. Coordination MechanismsIn the design of multi-agent systems one of the key points to

consider is the design of agent interaction protocols [Axelrod1984]. These protocols determine the order in which illocu-tions are exchanged, restrict their content and establish theobligations which the dialogue participants acquire as certainillocutions are uttered. The idea of a protocol to coordinatedifferent agents appeared back in the 60s with the Contract NetProtocol [Smith/Davis 1981] and is still present in currentmarketplace coordination mechanisms [Decker 1995]. Thetheoretical properties of these protocols has been the subject ofintense study in the last ten years. In recent years the theory hasevolved and now understands these interaction protocols asdialogue games in which the participants “play” by generatingsentences according to some specific rules. Thus games havebeen proposed for forming groups, persuasion or negotiation.

3.3. Negotiation Taking as their starting point the corpus of work on game

theory [Binmore 1992] which began to be developed in the 40sby von Neuman [Neumann/Morgenstern 1944], researchersinto multi-agent systems have developed adaptations of thosetheoretical results to construct coordination protocols in com-mercial interchanges [Kraus et al. 1995] [Rosenschein/Zlotkin1994]. In particular protocols have been implemented for vari-ous types of auctions, as well as for multi-attribute bilateralnegotiation strategies. Some papers have researched into nego-tiation protocols to allow the use of persuasion techniques ena-bling agents to influence their opponents set of beliefs[Amgoud 1999] [Dung 1995] [Parsons et al. 1998]. The workin these last areas has been mainly of an experimental natureand has given rise to several simulation environments.

3.4. MarketplacesAuction and negotiation protocols are usually structured in

what is known as a “marketplace”, a place where an arbitrarilylarge number of participants interact. The marketplaces wherethe use of agents is being introduced, or where they are plannedto be introduced in the near future, tend to have some of the fol-lowing characteristics: • The interactions are very fast. There is no time to re-consult

the user between auction rounds. • The interactions are repetitive with either (A) high commu-

nication overloads or (B) limited domain, which makes theagent’s learning of the user’s behaviour effective. Many B2Bareas are of this type.

• Each transaction is of a relatively small amount. If eachtransaction is of a relatively small amount it is possible tomonitor the process and halt the automatic commerce aftera while without significant losses. It is necessary to stress theimportance of the word ‘relative’ in this context: a smallamount for a company may be a large sum for an individual.

3



It is very likely we will see agents in B2B environmentsbefore they reach any other environments.

• The process is repeated time and time again. There are tworeasons for this: the first is that there has to be an expectationof profit which will justify investment in software, hardwareand training. The second is that the repetition of the processfacilitates the learning techniques in order to know the user’spreferences, thus obviating the need for explicit elicitationwhich is tedious and time consuming.

3.5. Preferences Preference modelling is essential so as to endow an agent

with the capability to act according to a user model (to be ableto negotiate, look for the right information, etc.) The challengesfacing this area nowadays are not only how to find a way torepresent the preferences of an agent, but also to develop mod-els which will indicate what the opponent’s preferences arefrom observing their behaviour, since it can give us a greatmany clues about interesting aspects of their interaction strate-gy. Different techniques are being used with that aim in mind:data mining, fuzzy logic and learning. Among the most imme-diate challenges with regard to preference modelling are: • Dynamic preference. Interests and preferences change over

the course of time. Artificial intelligence approaches such asinductive algorithms, belief revision techniques and learningin general are very useful for this purpose.

• Different ontologies. Agents will probably use differentontologies to characterize domains and interests. It is there-fore necessary to design reasoning techniques which willenable interaction in this context, without the need to resortto common or personal ontologies.

• Fuzziness. Preferences are fuzzy by their very nature and forthat reason we will need to use fuzzy logic, as well as statis-tics, to model user preferences.

• Learning. To build a user preference model without usingtedious questionnaires it will be necessary to develop tech-niques enabling us to observe the behaviour of the user, forexample, by presenting alternatives or making suggestions.

3.6. Mobility and Security A substantial percentage of researchers into agents consider

mobility as one of their most interesting characteristics[Rothermel/Popescu-Zeletin 1997]. By mobility is understoodthe capability of an agent to move its code and state betweendifferent platforms by using communication networks. Themost important advantage of mobility is the possibility ofdisconnecting the device from which the agent was activatedonce the latter has move to another device, and increasingapplications’ performance by moving the code of the agentmaking the computation to the platform which contains thedata, thus avoiding the limitations which bandwidth might oth-erwise impose. Currently the most important challenge for thedeployment of this technology is how to guarantee the securityof agents with regard to malicious platforms and how to protectthe platforms from malicious agents. Since the emergence ofthe concept of agent mobility, research has mainly beenfocused on such basic aspects as the design of mobility proto-

cols and the design of the minimum elements of security.Among the important milestones in this line of work have beenthe design of telescript [White 1994], General Magic, Aglets[Lange/Oshima 1999], IBM, and at a European level, thedesign of the platform MOLE by the University of Stuttgart<http://mole.informatik.uni-stuttgart.de>.

Areas of Application There are a good many areas of application where multi-

agent systems can be deployed, although perhaps the twowhich attracted the earliest attention were those of informationretrieval and e-commerce. Doubtless because both are tedioustasks which to a large extent appear ripe for automating.

4.1. Information AgentsThe emergence of large data communication networks like

the Internet has had a profound effect on the way we under-stand computing. For a start, distributed systems and concur-rent computation, which seemed like the pipedreams of aminority of IT people barely ten years ago, have become thenorm, commonplace. Nobody would dream now of not consid-ering distributed designs for the vast majority of their applica-tions.

One of the consequences of all the above is that the net hasbecome the depositary of a vast number of semi-structured doc-uments which are very hard to analyse and search for. We areall conscious of suffering from an information overload. Infor-mation agents were designed to help in the task of retrievingand handling distributed information on the net [Klusch 1999].One of the pioneers in this field was Pattie Maes, who designedseveral systems with these aims in mind [Maes 1994]. One ofthem was MAXIMS, an e-mail wizard which learned how toprioritise, delete, order and file messages. The system learnedfrom the actions that the user performed and constructed situa-tion → action type patterns. In these patterns the situation wasrepresented via message descriptors such as the message’ssource, receiver, target, or key words in the body of themessage, and the action consisted of showing the message,saving it or deleting it depending on the comparison betweenthe degrees of certainty obtained from the previous rules andsome pre-established thresholds.

Other Web-based agents will emerge in the near future: guideagents to help with navigation by suggesting “where to gonext” and which learn user preferences; indexing agents whichgo beyond the limitations of current browsers such as Googleand which take into account the aims and wishes of the users;common question browsers or expert browsers which “under-stand” what they are being asked for, not as a simple sequenceof words which are used to access an index.

Finally, a model has been proposed which would bringtogether existent information systems by means of a wrapperagent which “understands” the structure of the system and actsas a broker for other agents and information systems. Thus wecan imagine a multi-agent system distributed over the net inwhich the agents interact with brokers which are responsiblefor getting into contact with several agents or of redirecting

4



questions to those agents best equipped to answer on anyspecific topic.

4.2. E-commerce It should come as no surprise that e-commerce has been one

of the first areas to use autonomous agents [Noriega/Sierra1999b]. This is because commerce is basically interaction.Interaction between buyers and sellers at different levels: prod-uct search, purchase, delivery. The agents which have beendeveloped for e-commerce tend to have a high degree of auton-omy – they make their own decisions as to what, when andunder what conditions they are going to perform their actions[Chavez/Maes 1996]. Very often these agents need to interactwith other agents to achieve their objectives (either becausethey do not have enough resources to do it by themselves orbecause there are interdependencies between agents). The aimof these interactions is to get other agents to behave in a certainmanner, to modify a plan or agree on a joint action. The twomost commonly used mechanisms for this are auctions andnegotiation. In recent years several platforms have been devel-oped which enable auctions to take place in which agentsparticipate [Rodríguez-Aguilar et al. 1997]. Similarly, duringthe European Union’s Fifth Framework Programme severalprojects were undertaken with a view to setting up auctionbased electronic marketplaces. Given that agents have no directcontrol over each other, they must persuade one another to actin a certain way. It is not surprising, therefore, that negotiationprotocol modelling is seen as a requirement for e-commerceapplications. By negotiation is understood a process in whichtwo parties take a joint decision based on initially conflictingproposals which are successively refined via a process of con-cession making or a search for alternatives. The actual applica-tions are very varied and cover needs ranging from marketdesign, electronic management, banking, telecommunications,finance, or the shopping wizards. Most of the applications areprototypes which still require consensual and, if possible,standardized solutions to such issues as security and the guar-antee of privacy.

4.3. Future ProspectsIn the course of this present decade we are going to witness

a series of technological developments and economic changeswhich are very likely to have an effect on the use of autono-mous agents: • Mobile devices. The computer industry is gradually making

computational devices more accessible for the private indi-vidual. In the coming years we will witness, as we arealready are today, the appearance of many lightweight port-able personal devices. Agent applications will adapt to thisnew situation, for example with agents which know our pref-erences and will tell us when we are near a certain shop in acity, or will ask us for urgent instructions to close a dealwhile we are having dinner. Agent technology will have acentral role to play in all these services.

• Perception of the context. One of the great technologicalchallenges is how to detect what context a human is in. Thisinformation will be an indispensable complement to the user

profile. Knowing the user context will enable a more intelli-gent interaction between agents and humans, for example,we will want our mobile devices to be able to detect that weare in a restaurant and switch to an interactive mode inwhich no messages will be passed unless they are veryurgent (in that context we will not want to be interrupted byour PDA or any other device we may be using at thatmoment to be offered a new tomato sauce from our new e-supermarket but we might be interested in being asked if wewant to close a deal worth millions). Once outside therestaurant we will be able to receive the less importantmessages. The learning of behaviours in different contextswill be one of the more essential techniques.

• Deregulation. The process of deregulation which begantowards the end of the 20th century in many areas of com-merce (telecommunications, energy, transport, etc.) opensup enormous possibilities of interaction with new players inthe market. Many areas which up until recently were in thehands of monopolies are now sprouting dozens of new com-panies. We will need to develop new mechanisms of interac-tion among them. In some particular cases, techniques likeautomatic negotiation, mentioned in this article, will beapplied as a matter of course. We will soon be seeing theemergence of many agent-based marketplaces which willallow interaction among their participants.

• The vanishing computer. Voice recognition, emotion detec-tion and mood and movement detection mechanisms will bevital in the bid to improve the perceptive capabilities ofcomputational devices. These sensorial capabilities will bedeployed over an ever more distributed computational struc-ture. The use of networks in house construction and the pos-sibility of having low price computational devices linked tohousehold appliances open up a new world of potential ap-plications for autonomous agents. For example, devices ca-pable of connecting a fridge which is aware of what is insideit with a local trader to meet the needs of some tenants, ornegotiations between the different household appliancesabout when to consume energy, after negotiating with thedifferent electricity suppliers in order to bring the electricitybill down as low as possible.

Where to go for more information It is currently possible to find some text books about the

field of autonomous agents and multi-agent systems. Thebooks by Mike Wooldridge [Wooldridge 2001] and JacquesFerber [Ferber 1999] are good introductions to the subject anduseful for preparing graduate or postgraduate courses. Withregard to scientific publications, these tend to concentrate on aseries of conferences held since the early 90s: AutonomousAgents, International Conference on Multi Agent Systems,Workshop on Agent Theories, Architectures and Languages,which have been brought together recently under the banner ofthe International Conference on Autonomous Agents and MultiAgent Systems <http://autonomousagents.org/2002/>. There iscurrently a journal published by Kluwer called AutonomousAgents and Multi Agent Systems which is fast becoming theprestige journal of the field.

5



In the field of information agents, the series of CIA work-shops contain a large amount of useful and up to date material.Similarly the workshops Agent-Mediated Electronic Com-merce contain both theoretical and applied papers in the area ofe-commerce.

Finally we should mention that the publishing house Spring-er has a fairly good collection of publications in the area ofagents; in particular the proceedings of the CIA, ATAL, AMECand some other conferences have been systematically pub-lished as Lecture Notes on Artificial Intelligence and are animportant source of information.


References [Amgoud 1999]

Leila Amgoud. Contribution à l'intégration des préférences dansle raisonnement argumentatif. PhD thesis, Université Paul Saba-tier, Toulouse, 1999.

[Austin 1962]J. L. Austin. How to Do Things With Words. Oxford UniversityPress, 1962.

[Axelrod 1984]R. Axelrod. The Evolution of Cooperation. Basic Books, Inc.,Publishers, New York, USA., 1984.

[Bauer et al. 2001]Bernhard Bauer, Jean Pierre Muller, and J. Odell. Agent UML: aformalism for specifying multiagent software systems. In Agent-Oriented Software Engineering (LNCS-1957), pages 91–104.Springer-Verlag, 2001. Paolo Ciancarini and Michael WooldridgeEds.

[Berners-Lee 1999]Tim Berners-Lee. Weaving the web. Orion Business. London,1999.

[Binmore 1992]K. Binmore. Fun and Games. D. C. Heath and Company, Lexing-ton, MA, USA, 1992.

[Booch et al. 1998]G. Booch, J. Rumbaugh, and I. Jacobson. The Unified ModellingLanguage User Guide. Addison-Wesley, Reading, MA, 1998.

[Bradshaw 1997]Jeff Bradshaw (ed.). Software Agents. MIT Press, Cambridge,MA, 1997.

[Brooks 1990]Rodney Brooks. Elephants don't play chess, pages 3–15. MITPress, Cambridge, MA, 1990.

[Castelfranchi 1990]C. Castelfranchi. Social Power: A Point missed in Multi-Agent,DAI and HCI. In Y. Demazeau and J. P. Müller, editors, Decen-tralised AI, pages 49–62. Elsevier, 1990.

[Chavez/Maes 1996]A. Chavez and Pattie Maes. Kasbah: An agent marketplace forbuying and selling goods. In First International Conference on thePractical Application of Intelligent Agents and Multi-AgentTechnology (PAAM'96), pages 75–90, 1996.

[Ciancarini/Hankin 2001]Paolo Ciancarini and C. Hankin (Eds). Coordination Languagesand Models. Springer-Verlag, 2001. LNCS 1061.

[Ciancarini/Wooldridge 2001]Paolo Ciancarini and Michael Wooldridge (Eds). Agent-OrientedSoftware Engineering. Springer-Verlag, 2001. LNCS 1957.

[Cohen/Perrault 1979]P. R. Cohen and C. R. Perrault. Elements of a plan based theoryof speech acts. Cognitive Science, (3):177–212, 1979.

[Decker 1995]K. S. Decker. Environment Centred Analysis and Design ofCoordination Mechanisms. Ph.D. thesis, University of Massa-chusetts, 1995.

[Dennet 1987]D. C. Dennet. The Intentional Stance. MIT Press, Cambridge,MA, 1987.

[Dung 1995]P. M. Dung. On the acceptability of arguments and its fundamen-tal role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence, 77:321–357, 1995.

[Esteva et al. 2002]Marc Esteva, David de la Cruz, and Carles Sierra. ISLANDER:an electronic institutions editor. In AAMAS'02, page in press.ACM Press, 2002.

[Ferber 1999]Jacques Ferber. Multi-Agent Systems. Addison-Wesley, Reading,MA, 1999.

[Ferguson 1992]I. A. Ferguson. TouringMachines: An Architecture for Dynamic,Rational, Mobile Agents. PhD thesis, Clare Hall, University ofCambridge, Cambridge, UK, 1992.

[Finin et al. 1993]Tim Finin, Jay Weber, James McGuire, Stuart Shapiro, and ChrisBeck. Specification of the kqml agent-communication language,plus example agent policies and architectures. Technical report,The DARPA Knowledge Sharing Initiative External InterfacesWorking Group, June 1993. Draft.

[Fischer et al. 1996]K. Fischer, J. P. Mueller, and M. Pischel. A Pragmatic BDI archi-tecture, pages 203–218. Intelligent Agents II (LNAI-1037). M. Wooldridge and J. P.Mueller and M. Tambe (eds.). Springer-Verlag, 1996.

[Kinny et al. 1996]David Kinny, Michael Georgeff, and A. Rao. A methodology andmodelling technique for systems of bdi agents. In W. Van de Vel-de and J. W. Perram, editors, Agents Breaking Away: Proceedingsof the Seventh European Workshop on Modelling AutonomousAgents in a Multi-agent World, volume 1038 of Lecture Notes inArtificial Intelligence, pages 56–71. Springer-Verlag, 1996.

[Klusch 1999]Matthias Klusch (ed.). Intelligent Information agents. Springer,1999.

[Kraus et al. 1995]S. Kraus, J. Wilkenfeld, and G. Zlotkin. Multiagent negotiationunder time constraints. Artificial Intelligence, 75:297–345, 1995.

[Labrou et al. 1999]Y. Labrou, T. Finin, and Y. Peng. Agent communication languag-es: the current landscape. IEEE Intelligent systems, 14(2):45–52,1999.

[Lange/Oshima 1999]D. B. Lange and M. Oshima. Programming and deploying JavaMobile Agents with Aglets. Addison-Wesley, Reading, MA,1999.

[Maes 1994]Pattie Maes. Agents that reduce work and information overload.Communications of the ACM, 37(7):31–40, 1994.

[Muller/Pischel 1994]J. P. Muller and M. Pischel. An architecture for dynamicallyinteracting agents. International Journal of Cooperative Informa-tion Systems, 3(1):25–45, 1994.

[Neumann/Morgenstern 1944]J. V. Neumann and O. Morgenstern. Theory of Games and Eco-nomic Behaviour. Princeton University Press, Princeton, NJ,1944.



[Noriega/Sierra 1999a]P. Noriega and C. Sierra. Auctions and multi-agent systems. InMatthias Klusch, editor, Intelligent Information Agents, pages153–175. Springer, 1999.

[Noriega/Sierra 1999b]Pablo Noriega and Carles Sierra (eds.). Agent Mediated Electron-ic Commerce. LNAI 1571, Springer, Berlin, 1999.

[Oaks/Wong 2000]S. Oaks and H. Wong. Jini in a Nutshell. O'Reilly and Associates,Inc, 2000.

[Parsons et al. 1998]S. Parsons, C. Sierra, and N. Jennings. Agents that reason andnegotiate by arguing. Journal of Logic and Computation, 1998.To appear.

[Rao/Georgeff 1995]A. S. Rao and M. P. Georgeff. BDI agents: From Theory toPractice. In Proc 1st Int Conf on Multi-Agent Systems, pages312–319, San Francisco, USA, 1995.

[Rodríguez-Aguilar et al. 1997]Juan A. Rodríguez-Aguilar, Pablo Noriega, Carles Sierra, andJulian Padget. Fm96.5 a java-based electronic auction house. InSecond International Conference on The Practical Application ofIntelligent Agents and Multi-Agent Technology(PAAM'97),pages 207–224, 1997.

[Rosenschein/Zlotkin 1994]J. S.Rosenschein and G. Zlotkin. Rules of Encounter. The MITPress, Cambridge, USA, 1994.

[Rothermel/Popescu-Zeletin 1997]Kurt Rothermel and Radu Popescu-Zeletin, editors. MobileAgents. Number 1219 in Lecture Notes in Computer Science.Springer, 1997.

[Searle 1969]J. R. Searle. Speech Acts. Cambridge U. P., Cambridge, UK.,1969.

[Smith/Davis 1981]R. G. Smith and R. Davis. Frameworks for cooperation in distrib-uted problem solving. IEEE Trans on Systems, Man and Cyber-netics, 11(1):61–70, 1981.

[Weiss 1997]Gerhard Weiss, editor. Distributed Artificial Intelligence MeetsMachine Learning. Number 1221 in Lecture Notes in ArtificialIntelligence. Springer Verlag, 1997.

[White 1994]J. E. White. Telescript technology: the foundation for the elec-tronic marketplace (white paper). General Magic inc., 1994.

[Wooldridge 2001]Michael Wooldridge. An introduction to Multiagent Systems.John Wiley & Sons Ltd, 2001.

[Wooldridge/Jennings 1995]Michael Wooldridge and Nicholas R. Jennings. Intelligentagents: Theory and practice. The Knowledge Engineering Review, 10(2):115–152, 1995.

[Wooldridge et al. 1999]Michael Wooldridge, Nicholas R. Jennings, and David Kinny. Amethodology for agent-oriented analysis and design. In Proceed-ings of the Third International Conference on AutonomousAgents (AGENTS'99), May 1999.



Artificial Intelligence and Education:an Overview

Maite Urretavizcaya-Loinaz and Isabel Fernández de Castro

In this paper we present an overview of various contributions which Artificial Intelligence has made to theworld of computer-aided learning. After a short introduction to the field, including a brief history ofintelligent educational systems, we present some of the pedagogical trends which have influenced thedevelopment of these systems, a non-exhaustive review of intelligent educational systems, the ArtificialIntelligence techniques used in them and some current lines of research.

Keywords: Computer-aided Learning, Artificial Intelli-gence, Intelligent Educational Systems.

IntroductionThe terms “Artificial Intelligence” and “Education” define

the distinctive nature of research activity focused on the devel-opment of educational systems based on advanced technolo-gies [IA 2001] which take various aspects of knowledge intoconsideration. Nowadays the combination of new informationand telecommunication technologies, and new pedagogicaltrends, provide the driving force behind a constant evolution;from the first computer-assisted learning instruction programsin the 50s (CAI), through intelligent tutoring systems (ITS)[Ohlsson 86] [Self 1999] to the present day when the possibil-ities of current hypermedia and multimedia systems [Lustosa1993], the World Wide Web and trends in group work (collab-orative systems) are fuelling the ceaseless development ofdistance and virtual instruction [Brusilovsky 2000] [Dillen-bourg 2000] (Figure 1). We should also mention the construc-tion environments of: ITS [Arruarte et al. 1997] [Arruarte et al.2002] [Murray 1999], hypermedia systems [Stayanov et al.1999] and collaborative systems [Bourdeau et al. 2002].

CAI systems are characterised, among other things, bycovering complete courses, being static, being built in an adhoc way and lacking good interfaces. ICAI, Intelligent CAI,systems, however, are characterised by articulating the domain,inferring the student’s knowledge so as to direct and adapt theprocesses of instruction and learning and thereby improvingTutor-Student communication.

The Intelligent Educational Systems (IES) are tools support-ing learning at all levels whose capacity for adaptation, both tolearning needs and the characteristics of the user/student, qual-ify them as intelligent. The aim of IES’s is to aid, collaboratewith and enhance learning processes as an integrated part of themost state-of-the-art models of instruction. In other words, theyshould be considered as a complementary tool enabling us toincrease the quality of learning, rather than as a tool to replacean entire classic training system.

In the following sections we present, firstly, different peda-gogical trends and their relevance to IES environments. Then

we will go on to give a vision of the Artificial Intelligence tech-niques used in the development of instruction and learningenvironments. We will later show several lines of work, includ-ing some that were presented in one of the most prestigiousinternational conferences on this subject, Intelligent TutoringSystems – June 2002, San Sebastián (Spain)-Biarritz (France) <http://www.itsconference.com>. We will end with our conclu-sions.

Bases of Educational Technology and IES’sIn the last 10 years intelligent systems developed in the

field of education have undergone a great revolution, proposingsystems from various pedagogical and didactic points of view.

1

2

Maite Urretavizcaya-Loinaz received her bachelor and Ph.D.degrees in Computer Science in 1986 and 1992 respectively, fromthe Universidad del País Vasco - Euskal Herriko Unibertsitatea(UPV-EHU), Spain. Currently she is Associate Professor andmember of the Grupo Galan, group on ITS at the Computer Sci-ence Department of UPV-EHU in San Sebastián. Her researchinterests include diagnosis systems for different domains andtools to generate them. Now she leads the Erakusle project – aWeb-oriented Tutor System, whose goal is to build intelligenttutoring systems endowed with flexible learning strategies and aWeb-oriented architecture by reusing courses written with con-ventional editors. <[email protected]>

Isabel Fernández de Castro received her bachelor degree inPhysics and Ph.D. in Computer Science in 1981 and 1989 respec-tively, from the Universidad del País Vasco - Euskal HerrikoUnibertsitatea (UPV-EHU), Spain. Now she is a Professor at theComputer Science Department of UPV-EHU leading the GrupoGalan, group on Intelligent Tutoring Systems (ITS). Her researchinterests include the design and implementation of ITS, authoringtools, self-improved ITS’s, knowledge acquisition facilities, andso on. During the last ten years she has participated in and led anumber of European, national and local projects. Some of themhave major industrial implications, such as FROGALAN – avirtual reality system to help in the assessment of technical abili-ties and INGALAN – intelligent tools based on virtual realitytechnology to support the learning and evaluation of technicalabilities. <[email protected]>



The instructive approach is the traditional educationalapproach in which instruction is understood to be “the trans-mission of knowledge requiring the teacher to monitor thestudent constantly, especially in the problem solving process-es.” ITS’s have this approach [Trojahn et al. 2002]. They are aparticular type of ICAI system or IES which teach knowledge(as transmission) in an individualised way, taking into accountthe capacity for learning and the knowledge of the student inthat subject. The answer to the questions What is taught?, Whois taught? and How are things taught? gave rise to the classicmodule-based architecture (Figure 2) [Wenger 1987].

ITS’s are adapted to each student by means of their diagnos-tic skills which examine the student’s knowledge and the struc-turing and presentation of knowledge. They also make use of avariety of techniques to hold the user’s attention and facilitatethe transmission of the desired knowledge. Intelligent trainingsystems also share this approach, although in these cases theprocesses are aimed more towards specific problem solvingactivities. The tutor guides the instruction process according totraditional practices.

Meanwhile, the constructivist approach [Jonassen et al.1992] [Gil et al. 1999] establishes that “the student should takecontrol of his or her own learning process.” In other words,learning is seen as an active process of knowledge construction.This approach defines the bases of discovery and experimenta-

tion environments – a paradigm that can be immediately seenin network-based learning. Hypermedia systems [Heller 1990][Meyer 2002] are perfectly adapted to the requirements of thisapproach. However, students’ great mobility through hyper-space may disorient them which is why it is necessary to offerthem complementary tools to give them guidance and adapt totheir learning.

Both ITS’s and hypermedia systems share the principle ofindividualized learning. However, these days current trends arepointing towards group or collaborative learning [Vigotsky1978] [White 1999] as an alternative in order to improve qual-ity. A network of computers becomes a basic tool for the groupof students to communicate, collaborate and share the activitiesneeded to complete a task. Systems with this collaborativeapproach allow the student to carry out activities such as jointproblem solving, criticism of own and others’ proposals, justi-fication and explication of the solutions given and received, oraccess to information. A whole set of methods of pedagogicalintervention have been developed to organise collaborativelearning [Barros et al. 2000], competition and cooperation[Rasseneur et al. 2002], whether it be in small or large groups,face-to-face or virtual, in synchronous or asynchronous mode.

Computer Technology of IES’s and Artificial Intelli-gence.Each of the various types of learning environments

described displays a series of requirements and qualitieswhich characterise the kind of conceptualisation and formal-isation suitable in each case, while requiring the use of thesame number of techniques or technologies. In this sectionwe will focus on the relation of these aspects and the mostpromising or most useful techniques of Artificial Intelligencein the present day, providing specific references which maycomplement the qualities mentioned.

Techniques coming from Artificial Intelligence concernedwith symbolic knowledge modelling, such as frame or object

3

ITSconstructionenvironments

Computer Assisted Instruction

CAI

Flexible ITSsITS

Adaptative hypermedia

Distance

Collaborative

ICAI- Intelligent CAI

systems

learning

systems

Figure 1: Evolution of Computer Assisted Instruction Systems

Evolution of IES:

web, on-line interactivity,

IES construction

Agents.

Virtual reality.

Dialogues:

Intelligent educational systems IES

1950 1980 1990 2000

chats, etc.

environments.

written and spoken.

InterfacePedagogical component

Domainmodule

Studentmodel

Figure 2: Classic architecture of an Intelligent Tutoring System



taxonomies, have been extensively used to define knowledge ofthe domain or subject of the instruction of ITS’s [Fernández-Castro 1989] (Figure 3). In this case, the attributes of thedifferent nodes and their relations can represent propertiesdescribed by educational theories.

Other techniques such as qualitative reasoning [Bredeweg2002], Bayesian networks [Vanlehn et al. 2001] [Millán et al.2000], or procedural networks have enabled the conceptualmodelling of both the learning domain (especially in ITS) andthe student model. Bayesian networks can also be used tomodel the relations between the observed actions of thestudent, his or her internal state and the outputs produced.

Planning techniques [Martens et al. 2002] and rule systems[Prentzas et al. 2002] (Figure 4) have allowed us to model thepedagogical behaviour of the systems in a satisfactory way. Thedynamic generation of instruction plans enables both thedomain and its grade of difficulty to be adapted to the user orusers who interact with the instruction system.

While qualitative reasoning1 is a technique which has beenused for some years now, [Sime et al. 1992] [Vadillo et al.1998] the use of qualitative models [Bredeweg 2002] – causalbehaviour models, which together with simulation technology(outside the scope of AI) satisfactorily meet the needs of expla-nation, visualisation and representation of different levels ofknowledge – is currently gaining a special importance.

Meanwhile, agent technology, which has revolutionised soft-ware technology in general, is having a considerable effect onhow we conceptualise a classic software architecture.

Web technology’s arrival on the scene and its growingacceptance by society has prompted IES to be reorientedtowards the net. With this new perspective we can come acrossdefinitions such as “ITS’s are a kind of system based on Artifi-cial Intelligence which act as wizards (which automate compli-

cated and lengthy manual functions) in instruction-learningprocesses” [Trojahn et al. 2002]. This definition can be extend-ed to IES’s.

Obviously we cannot fail to mention all those computerscience techniques which, to a greater or lesser degree, formpart of IES’s: databases, distributed computation, a wide rangeof multimedia technologies, Internet, etc.

Some Lines of Research into IES’sIES’s basically involve four different areas of knowledge

related to: 1) the development of the task of instruction per se,2) the corpus of knowledge or the subject you want to learn, 3)the evolution of the student’s state of knowledge and 4) thecomponent responsible for the communication between theuser and the system. Also, if the system is to permit groupwork, it will be necessary to add a specific area for the repre-sentation of the task and group and individual behaviour.

The research carried out in this regard has been approachedin various ways, from research associated with each of theareas– knowledge representation, planning of the instruction[Fernández-Castro et al. 1993] [Bretch 1990] [Elorriaga et al.2000], natural language [Diaz de Ilarraza et al.1992], [Woolf1992] to the development of generic architectures [McCalla etal. 1990], the study of automatic learning techniques [Dillen-bourg 2000] and finally the development of design aid environ-ments and the building of instruction systems [Murray 1999].In the latter two cases it is even possible to model the instructor,

1. MONET <http://monet.aber.ac.uk/> is an European excellencenetwork on Model-Based Educational Systems and QualitativeReasoning.

4

outputfunction

expression

functiondeclaration

functioncall

logicalexpression

arithmeticexpression

p_d

es_a

es_a

p_d

pr

pr

co

“prerequisite” relation

“part of” relation

“is_a” relation

“co-requisite” relation

Domain concept

pr

p_d

es_a

co

Figure 3: Domain fragment in Tutor [Fernández-Castro 1989].

(-2.4) if teach-meth-bias is examples –exercises (1.5),concentration-level is low (1.2),knowledge-level is low (1.0),percent-accessed –cunits < 0.30 (0.9),teach-meth-bias is theory-examples-exercises (0.9)

then teaching-method is theory-examples-exercises

Figure 4: Part of a Neurorule



with the aim of increasing the quality of the resulting ITS andeven increasing the knowledge of the instructors themselves.Information could be acquired about the instructor’s preferenc-es, interests, or regular activities related to the ITS underconstruction; this could be used to increase the level of adapta-tion and aid to the instructor’s needs. We should also highlightother lines of interest evidenced in [ITS 2002]: the Web andagents as a means of developing IES’s, evaluation of IESsystems and, finally, dialogues as a means of Tutor-Studentcommunication, including studies on human motivation andemotions.

In the last five years, research on dialogue [Peinstein et al.2002] has been taken up again with renewed vigour as a meansof enabling tutor-student interaction incorporating more natu-ral means of communication, including emotional aspects inthe interaction; for example, the Reading Tutor of the LISTENproject. The maturity of the research carried out is amplydemonstrated by its presence in specialised conferences oninstruction, both in developed and evaluated systems.Researchers are currently studying the emotional factors aris-ing during the learning discourse, with the aim of making edu-cational systems more effective by stimulating the student andkeeping him or her motivated. Several groups of researchershave incorporated dialogue into their instruction systems: forexample CIRCSIM-tutor, PSA, AutoTutor, WHY2 system,Atlas-Andes and Ms.Lindquist Tutor. Other systems such asBEETLE and PACO are procedural task-based tutors, in whichthe student needs to carry out certain steps in a loose hierarchi-cal order that he or she may or may not be aware of.

The handling and recovery of students’ errors also deservesa special mention. There are two aspects involved: the recogni-tion or identification of the error and diagnosis of its underlyingcause, and the planning of suitable pedagogical strategies inorder to recover them [Urretavizcaya 1991] [Ferrero et al.2000] [Mitrovic 1998].

ITS’s have also been applied in industrial environments[Frasson 1996] [Vadillo et al. 1998] [Ferrero et al. 2000] tocomplement their conventional training programmes. In thiscase multiple knowledge representation models (mental mod-els, levels of abstraction) [Sime et al. 1992] and graphical inter-faces [Rickel et al. 1999], which are often the main obstacle tothe success of such systems, take on a special importance.

Group learning techniques and the integration of telecom-munication technologies, chats, e-mail, debate forums, etc.,form part of the line of research known as Computer SupportedCollaborative Learning (CSCL), which allows resources,spaces and activities shared by groups of individuals to beorganised and defined [Verdejo et al. 2002].

Finally, we should not fail to mention a new line of interestwhich aims to integrate the pedagogical capabilities of intelli-gent learning environments by the use of virtual reality (VR) inorder to improve the instruction process [Eisenberg et al. 1997][Rickel et al. 1999]. These new graphical interfaces are usedwith the aim of obtaining immersive systems which canemulate the real productive environment more faithfully toachieve more effective instruction. As occurs with simulationtechniques, these environments are especially useful in

domains in which real training is dangerous or expensive;however VR can provide a more realistic perception (visual,auditory, tactile) suitable for a wider range of situations. Theusers, immersed in 3D simulations of their working environ-ments, improve their skills by carrying out real tasks. The tutorscan also inhabit the virtual world with the students, collaborat-ing with them “physically” and interacting and communicatingby non-verbal means, thus widening the bandwidth of man-machine communication. It is also possible to access newinformation characterizing the user, such as visual attentionand physical movements (e.g. position and orientation ofhands). Thus VR opens up fresh possibilities for the instructionof physical tasks in new educational environments.

ConclusionsIn this article we have presented an overview of intelligent

educational systems. It was not our intention to provide eitheran exhaustive, extensive or in-depth study, though we havetried to supply enough bibliographical references to enablereaders to go more deeply into the subjects that interest themmost. As we have seen there are many lines of research open,but we would like to highlight three in particular. Firstly, theevaluation of existing educational systems which will enableus to improve and ensure the quality of developed systems, andtherefore integrate them more effectively into centres of learn-ing. Secondly, the new graphical possibilities afforded bypersonal computers, as a result of more sophisticated interfac-es- multimedia (video, sound, animation, etc.), simulation andvirtual reality techniques– together with written and spokendialogue systems, will enhance man-machine communication.All of this will have a positive influence on the acceptance ofthese tools in different environments, from training centres inindustrial environments to centres for special educational needs(for the visually or aurally handicapped, etc.). And finally, theWorld Wide Web and the new possibilities afforded by the Inter-net, which have already left their mark on the field of educa-tional IT, will be a compulsory reference for both the evaluationand popularisation of IES’s and the effective use of educationaltools.

Educational IT in general, and IES’s in particular are under-going an important and exciting revolution. On the one hand,the new telecommunication technologies, the great connectivi-ty of computers, digital lines, satellite communications, etc.and on the other, the most recent trends in cognitive psychologyand the new pedagogical approaches, all have a marked effectand influence on the development and orientation of IES’s.

AcknowledgementsThis work has been partially funded by the Spanish Commission of

Science and Technology (CICYT) through the project TIC1999-0252,and the Universidad del País Vasco - Euskal Herriko Unibertsitatea(1/UPV00141.226-T-13995/2001). We would like to give our sincerethanks to all the members of the Grupo Galan, the intelligent tutoringgroup of the Computer Science Faculty of San Sebastián, for theirwork and dedication.


5



References[Arruarte et al. 1997]

Arruarte A., Fernández I., Ferrero, B. J. Greer. “The IRIS Shell:how to build ITS’s from Pedagogical and Design Requisites.”Journal of Artificial Intelligence in Education, Vol. 8, N. 3/4, pp341 – 381. 1997.

[Arruarte et al. 2002]Arruarte A., B. Ferrero, Fernández-Castro I., Urretavizcaya M.,Greer J. The Iris Authoring Tool. 2002.

[Barros et al. 2000]Barros, B, Verdejo, F. (2000). “Analysing student interactionprocesses in order to improve collaboration. The DEGREEapproach” In Int. Journal of Artificial Intelligence in Education,Vol 11, pp 221–241. 2000.

[Bourdeau et al. 2002]Bourdeau J. & Mizoguchi R. Collaborative Ontological Engi-neering of Instructional Design Knowledge for an ITS AuthoringEnvironment.

[Bredeweg 2002]Model-Based Educational Systems and Qualitative Reasoning:The next generation Workshop Proceedings. In 6th. InternationalConference on Intelligent Tutoring Systems. Biarritz (France)-San Sebastián (Spain) 2002.

[Bretch 1990]Bretch, B. “Determining the Focus of instruction: Content,Planning on ITS’s.” Phd Dissertation: University of Saskatch-ewan. 1990.

[Brusilovsky 2000]Brusilovsky P. “Course sequencing for Static Courses? ApplyingITS Techniques in Large-Scale Web_Based Education.” InProceedings of Intelligent Tutoring Systems, Eds. Gilles G.,Frasson C and VanLehn K. Springer Verlag, pp.625–634. 2000.

[Diaz de Ilarraza et al.1992]A. Díaz de Ilarraza, J.A. Elorriaga, I. Fernández de Castro, J.Gutiérrez, “User Modelling and Architecture in industrial ITS’s.”Intelligent Tutoring Systems ITS’92, Lecture Notes in ComputerScience 608, Springer-Verlag, pp 661 – 668. 1992.

[Dillenbourg 2000]Dillenbourg, P. “Virtual Learning Environments.” EUN Confer-ence 2000, “Learning in the new millennium. Building neweducation strategies for schools”, 2000.

[Eisenberg et al. 1997]Eisenberg M., Nishioka A. and Schreiner M. E. “Helping UsersThink in Three Dimensions: Steps Toward Incorporating SpatialCognition in User Modelling”. Proceedings of the InternationalConference on Intelligent User Interfaces. Johanna Moore, Er-nest Edmonds and Angel Puerta (Eds.) ACM-ISBN: 0-89791-839-8, pages 113–120. 1997.

[Elorriaga et al. 2000]Elorriaga J.A., y Fernández-Castro I., (2000c). “Evaluation of aHybrid Self-improving Instructional Planner.” Proceedings of theInternational Workshop on Advanced Learning Technologies.Palmerston North, pp. 133–136. 2000.

[Fernández-Castro 1989]Fernández-Castro I. Estrategias de Enseñanza en un SistemaInteligente de Enseñanza Asistida por Ordenador. Tesis doctoral,Universidad del País Vasco. 1989.

[Fernández-Castro et al. 1993]Fernández-Castro I., Díaz-Ilarraza A. & Verdejo F. Architecturaland Planning Issues in Intelligent Tutoring Systems. Journal ofArtificial Intelligence in Education, Vaol. 4(4), pp. 357–395.1993.

[Ferrero et al. 2000]Ferrero, B., Fernández-Castro, I., Urretavizcaya, M. “Diagnosticet évaluation dans les systèmes de ‘training’ industriel.” InSciences et techniques éducatives, Vol. 6, No. 1, pp.189–217.2000.

[Frasson 1996]Frasson C., “Some Characteristics of Instructional Design forIndustrial Training.” Computer Aided Learning and Instruction inScience and Engineering CALISCE’96, LNCS 1108, pp. 1 – 8.1996.

[Gil et al. 1999]Gil D., Carrasco J., Dumas-Carré A., Furió C., Gallego R., GenéA., González E., Guisasola J., Martínez-Torregrosa J., Pessoa deCarvalho A., Salinas J., Tricárico H. Y., Valdés P. “¿Puedehablarse de consenso constructivista en la educación científica?”Debates. Enseñanza de las ciencias. Vol. 17, num. 3, pp. 503–512.1999

[Heller 1990]Heller R. (1990) The Role of Hypermedia in Education: A look atthe Research Issues. Journal of Research on Computing in Edu-cation, 22, pp. 431–441. 1990.

[IA 2001]“Inteligencia Artificial” 12. Número monográfico: SistemasInteligentes en el ámbito de la educación. Primavera 2001.<http://sensei.ieec.uned.es/cgi-bin/aepia/contenidoNum.pl?nu-mero=12>.

[ITS 2002]Intelligent Tutoring Systems. 6th. International Conference, ITS2002. Cerri S. A., Gouardères G. & Paraguaçu F. (Eds.) Springer.2002.

[Jonassen et al. 1992]Jonassen D., Mayes T. and McAleese R. “A Manifesto for aconstructivist Approach to Uses of Technology in Higher Educa-tion.” Eds. Duffy, Lowyck and Jonassen, pp. 231–147. 1992.

[Koschmann 1996]Koschmann, T. “Paradigm shifts and Instructional technology:An Introduction.” En CSCL: Theory and Practice of an emergingparadigm, pp 1–23, Lawrence Erlbaum Associates.1996.

[Lustosa 1993]Lustosa Vieira, L. “Vers l´integration des Systemes TutoriauxIntelligents dans les Hypermedias: une etude bibliografique.” InLe journal de la formation continue. 1993.

[Martens et al. 2002]Martens A. & Uhrmacher A.M. Adaptative Tutoring Processesand Mental Plans. (2002) In Intelligent Tutoring Systems - ITS2002. Cerri S.A., Gouardères G. & Paraguaçu F. (Eds). Springerpp. 71–80.

[McCalla et al. 1990]McCalla, G.I., Greer, J.E. & the SCENT Research Team.“SCENT-3: An Architecture for Intelligent Advising in ProblemSolving Domains.” En C. Frasson & G. Gauthier (Eds), Intelli-gent Tutoring Systems: At the Crossroads of Artificial Intelli-gence and Education, pp 140–161. Norwood, NJ: Ablex. 1990.

[MacLaren et al. 2002]MacLaren b. & Koedingir K. When and Why Does MasteryLearning Work. In 6th. International Conference, ITS 2002proceedings. Biarritz (Francia) and San Sebastián (Spain), pp.356–366. 2002.

[Meyer 2002]Meyer C. Hypermedia Environment for Learning ConceptsBased on Inter-domain Analogies as an Educational Strategy. InIntelligent Tutoring Systems - ITS 2002. Cerri S.A., GouardèresG. & Paraguaçu F. (Eds). Springer pp. 281–290. 2002.

[Millán et al. 2000]Millán E., Pérez-de-la-Cruz J., Suárez E. “Adaptive BayesianNetworks for multilevel Student Modelling”, Proc 5th ITS,Springer -Verlag, pp 534 – 543. 2000.

[Mitrovic 1998]Mitrovic A. “Experiences in Implementing Constraint-BasedModeling in SQL-Tutor.” Intelligent Tutoring Systems ITS’98,LNCS 1452, pp 414 – 423. 1998.



[Murray et al. 2002]Murray T., Blessing S., & Ainsworth S. (Eds.) Authoring Toolsfor Advanced Technology Learning Environments: Toward cost-effective adaptive, interactive, and intelligent educational soft-ware, 32 pgs. In press.

[Murray 1999]Murray, T. “Authoring Intelligent Tutoring Systems: An analysisof the State of the Art”. Int. Journal of Artificial Intelligence inEducation, 10(1), 98–129. 1999.

[Ohlsson 86]Ohlsson, S. “Some principles of intelligent tutoring.” In Instruc-tional Science 14, pp 293–326. 1986.

[Peinstein et al. 2002]Peinstein R.C. & Aleven V. Empirical Methods for TutorialDialogue Systems Workshop Proceedings. En ITS 2002.

[Prentzas et al. 2002]Prentzas J., Hatzilygeroudis I. & Garofalakis J. (2002) A web-based Intelligent Tutoring System Using Hybrid Rules. In Intelli-gent Tutoring Systems - ITS 2002. Cerri S.A., Gouardères G. &Paraguaçu F. (Eds). Springer pp. 119–128. 2002.

[Rasseneur et al. 2002]D. Rasseneur, E. Delozanne, P. Jacoboni, B. Grugeon. Learningwith Virtual Agents: Competition and Cooperation in AMICO.Intelligent Tutoring Systems 2002: 61–70

[Rickel et al. 1999]Rickel J. & Johnson L.W. Virtual humans for Team Training inVirtual Reality. In 9th Artificial Intelligence in Education, OpenLearning Environments, Lajoie S.P. & Vivet M. (Eds.), pp578–585. 1999.

[Self 1999]Self, J.A. “The defining characteristics of intelligent tutoringsystems research: ITS care, precisely.” En Int. Journal of Artifi-cial Intelligence in Education, Vol 10, N. 3–4, pp 350 – 364. 1999.

[Sime et al. 1992]Sime, J.A., Leitch, R. “A Learning Environment Based onMultiple Qualitative Models.” En Intelligent Tutoring Systems(ITS-92). Frasson, C., Gauthier, G., McCalla, G.I. (Eds). pp116–124. Springer-Verlag. 1992.

[Stayanov et al. 1999]Stayanov S., Aroyo L. & Kommers P. Intelligent Agent Instruc-tional Design Tool for a Hypermedia Design Course. In ArtificialIntelligence in Education (AI-ED’99). Lajoie S.P. & Vivet M.(Eds.), pp. 101–108. 1999.

[Trojahn et al. 2002]Trojahn dos Santos C., Frozza R., Dharmer A. & Paschoal G.DORIS - Pedagogical Agent in Intelligent Tutoring Systems. InIntelligent Tutoring Systems - ITS 2002. Cerri S.A., GouardèresG. & Paraguaçu F. (Eds). Springer pp. 91–104. 2002.

[Urretavizcaya 1991]M. Urretavizcaya, “Sistema para la detección, diagnóstico ycorrección de errores en un sistema tutor inteligente.” Tesis doc-toral, UPV/EHU, Facultad de Informática. 1991.

[Vadillo et al. 1998]Vadillo J. A., A. Díaz de Ilarraza,I. Fernández, J. Gutiérrez, J.E.Elorriaga. “Behavioural Explanations in ITSs for training usingCausal Models.” Interactive Learning Environments Journal.Special Issue: The Use of Qualitative Reasoning in ILE, Vol. 5(1),pp. 125–134. 1998.

[Vanlehn et al. 2001]Vanlehn K, Zhendong N. (2001) “Bayesian student modelling,user interfaces and feedback: a sensitivity analysis.” InternationalJournal of Artificial Intelligence in Education, Vol. 12, (2) pp.155–184.

[Verdejo et al. 2002]Verdejo M.F., Barros B. Read T. Y Rodríguez-Artacho M. Asystem for the Specification and Development of an Environmentfor Distributed CSCL Scenario. In Intelligent Tutoring Systems-ITS 2002. Cerri S.A., Gouardères G. & Paraguaçu F. (Eds).Springer. pp. 139.148. 2002.

[Vigotsky 1978]Vigotsky L.S. “Mind in society: The development of higherpsychological processes.” Cambridge MA: Harvard UniversityPress, 1978.

[Wenger 1987]Wenger, E. “Artificial Intelligence and Tutoring Systems: compu-tational approaches to the communication of knowledge.” LosAltos, CA: Morgan Kaufmann Publishers, Inc. 1987.

[White 1999]White R.T. “Condiciones para un aprendizaje de calidad en laenseñanza de las ciencias. Reflexiones a partir del proyectoPEEL” Traducción de la ponencia del V congreso Internacionalsobre Investigación en la Didáctica de las Ciencias (Martínez C.).Enseñanza de las ciencias, 1999. Vol. 17, num.1, pp. 3–12.

[Woolf 1992]Woolf, B.P. “Towards a Computational Model of Tutoring.” InAdaptative Learning Environments. Foundations and Frontiers.Eds M. Jones & P. Wihne, Nato ASI Series. 1992.

Documents

Artificial Intelligence Ubiquitous Computing … Computing ... Artificial Intelligence Technology with a Future. EUropean Certification of Informatics Professionals