Tasks as Context for Intelligent Agents

Proceedings of the IEEE/WIC International Conference on Intelligent Agent Technology (IAT’03) 0-7695-1931-8/03 $ 17.00 © 2003 IEEE

Tasks as Context for Intelligent Agents

Kendall Lister and Leon Sterling Intelligent Agent Lab

Department of Computer Science and Software Engineering The University of Melbourne, Parkville 3010, Melbourne, Australia

{krl,leon}@cs.mu.oz.au www.cs.mu.oz.aulagentlab

Abstract

The context of a statement or assertion can be critical to a useful understanding of its meaning. In huge knowledge bases such as the CYC project, capturing context explicitly encounters significant problems. Research at the Intelligent Agent Lab at The University of Melbourne has been investigating lightweight approaches to incorporating context in the construction of knowledge-based information agents. The prototypical example of a successful agent built in the lab is SportsFinder. SportsFinder extracts sporting match results from a large variety of web sites without any formal knowledge representation. This is a task for which understanding the context of knowledge on the web pages is critical, as the individual elements that make up a page of sports results are very ambiguous without consideration of the surrounding information. This paper advocates incorporating context via a task specification, without resort to general purpose knowledge techniques.

1. Introduction

As general ontologies and knowledge bases grow in number and size, the issue of when certain facts are true and relevant and when they are not becomes more important. It is quickly apparent, even after making only a small number of assertions, that the relevance and even truth of many statements only holds under a set of assumptions. Most evaluations of information inevitably depend on subjective criteria such as importance, relevance, urgency or reliability. Different people typically give different answers to the same questions, and make different comments about the validity of a statement. If knowledge models and ontology-based applications are to be widely usable by large numbers of people and software agents then it is necessary to accommodate this subjectivity.

154

Given the difficulty of modelling and representing knowledge and reasoning in the first place, it is not clear how to extend such models and representations to deal with SUbjectivity. The idea usually employed as an aid in approaches to extending models is that of context, that the meaning of a word or text is determined by the texts and statements that surround it. Additionally, context incorporates the circumstances or setting in which an event occurs, such as a task being performed. For example, if a news report includes a local temperature forecast of 25 degrees, it is reasonable to act on that knowledge and choose clothing or make plans accordingly; but if the report was generated in a foreign country, it would no longer be sensible to base your choices on the information that it contains. Although the statement "It will be 25 degrees tomorrow" is true for some people in a certain place, it is most unlikely to be true for us (coincidence aside). The question of whether the forecast temperature has been given in degrees Celsius or Fahrenheit is another good example of the importance of context for correctly interpreting information. Without some extra information, such as the country of origin of the forecast, it is very difficult to decide whether tomorrow will be quite warm or very cold.

This obvious example of a temperature forecast needing a context to be meaningful bears examination with a view to what exactly it is that renders a single piece of information relevant or irrelevant, useful or useless. In the case of the weather report, it is the presence in the surrounding information, or context, of a fact that indicates that the report originated in a place sufficiently removed from your current location that the contents of the report are not applicable to you. A key observation from this example is that rather than asking about the meaning of an assertion, we could ask about its relevance to us, in our current situation, given our current activities and goals. The relationship between meaning and relevance is not completely clear; it can be difficult to say whether the meaning or the relevance of an assertion is more valuable. Arguably, the meaning is useless without

IE E E~ COMPUTER

SOCIETY


some consideration of the relevance, while relevance would seem to be difficult to determine without an idea of the meaning. In the above example of a weather report, the interests of the viewer are served equally well by judging the forecast irrelevant based on its context as they are by considering its meaning in light of the origin of the report - if the former can be done quickly, it seems unnecessary to perform the latter.

In many situations, context extends beyond the contents of a particular information source or report. Motivation, immediate and future goals, attitude and available resources all tend to influence the interpretation of both the truth and the relevance of information. In general, the meaning and the relevance of a piece of information are bound to be dependant on not just the obvious context, that is the extra information surrounding a particular piece of data, but also whatever facts are already known or believed - even if a weather forecast is judged to be applicable to your neighbourhood, if you have no intention of going outside at all then the report has little relevance.

In group environments, individuals will interpret information differently according to the roles they are playing and the tasks they are performing. For example, to the commander of an entire theatre of war, the particular formation of a group of enemy aircraft is not at all relevant - what matters is their apparent destination. However, to the pilots tasked to engage the enemy aircraft, the formation may be a crucial factor in choosing which tactics to employ. Similarly, to the average person it is true that it is impossible for something to jump instantly from one place to another; simple Newtonian physics and their everyday experiences rule out anything else. But to a physicist, quantum theory reverses that truth. Likewise, software agents in multi-agent systems increasingly have to deal with different interpretations and viewpoints, as theories based on ideas such as joint intentions, dynamic groups and hierarchies, collaborative planning and heterogeneous communities are becoming increasingly popular.

In the next section, the CYC project and its explicit handling of context is discussed. In Section 3 we consider task specification as inherently incorporating contextual information, without explicit representation. The SportsFinder information agent is analysed in Section 4, and in Section 5 we discuss the close relationship between meaning and relevance, and the way that context influences both. The final section considers future directions for the ideas in this paper, particularly with regard to the development of multi-agent systems. Note that for the purposes of this paper, there is no need to distinguish between ontology-based systems and knowledge-based systems. We choose the former term to refer to them.

155

2. Context in CYC

A useful study in the practical implementation of context in a functioning ontology-based system is found in the long-standing, highly-visible CYC project [I, 2]. The CYC project is interesting because its knowledge base/ontology is huge and general - it attempts to be completely objective in the hope of being understandable and reusable by anyone. During the project several approaches have been taken to context, as is described in this section. The initial direction of CYC was to amass the vast collection of facts that form general knowledge and common sense by explicit specification, in essence compiling the contents of an encyclopaedia. When that endeavour appeared unlikely to be sufficient for intelligence to emerge, the CYC direction expanded to specifying the presumably even larger body of knowledge that surrounds and informs our understanding of the facts in the encyclopaedia, without which we are left struggling to communicate in a sea of knowledge; as Brian Smith put it, "everything we know, but have never needed to write down" [3]. The purpose of this section is not to critique the CYC project or its goals, but instead to observe its experiences with the implementation of context-sensitive knowledge manipulation in the light of the content of rest of this paper.

Currently the CYC project handles context explicitly by using microtheories [4]. In contrast, for the first half dozen years of its life CYC contained no explicit consideration of context [3, 4]. From 1989 to 1991, contextualisation was added to the various attributes or characteristics that concepts and assertions possess in Cye. Contexts were defined and said to have assumptions and content, assertions could be imported from one context by another, and contexts were actual first-class terms in the CYC representation language that were partially ordered by specialisation. This implementation was unsatisfactory. Doug Lenat, the CYC guru, lists the primary drawbacks as:

• the expense of importing assertions from one context to another,

• the burden on the ontology builders of explicating the assumptions of each context, and

• the cost of placing every assertion into the proper context [4].

What can be very usefully learnt from this episode in the life of CYC is two-fold: firstly, that some direct consideration of context is absolutely necessary for a large ontology-based system, and secondly, that representing context as just another piece of knowledge is not an effective way to capture the particular effects that context has on information. Rather than establishing context by

rFFE~ COMPUTER

SOCIETY


adding more knowledge to determine meaning, context should be set by an alternative mechanism that can clarifY and bound meaning.

CYC now has a more sophisticated mechanism for explicating and manipulating context using microtheories (in the CYC glossary, the entry for the term 'context' says "often used interchangeably with 'microtheory'" [1]). Microtheories are based on a tenet that context is a multidimensional space [4]. The dimensions reflect the attributes of an assertion that define under what conditions it is valid. The twelve dimensions of context-space chosen for CYC are: absolute time, type of time, absolute place, type of place, culture, sophistication/security, topic/usage, granularity, modality/disposition/epistemology, argument preferences, justification and 'lets'. By choosing bounded segments along the axes of some or all of these dimensions, regions of the context-space are delineated and a 'context' is defined. Assertions in the CYC knowledge base can have the appropriate region of context-space specified for them, and queries can be restricted to certain regions of the context-space, thus limiting the set of assertions that can be referenced when CYC tries to supply an answer. Specified regions of context-space can be reified within the Cye knowledge base and then re-used for a set of assertions - these are the actual microtheories.

In practice, a microtheory seems to be used as a grouping device that ties together some number of assertions under a label [2] . Those assertions must be true in the context referenced by the label of the microtheory. Thus, each assertion (truth) in the cye ontology is tagged with the microtheory in which it is true. This opens the way not only for the inclusion of assertions that are apparently contradictory when taken out of context (e.g. using microtheories it should be possible to add to CYC the two assertions "Jesus Christ is the son of God" and "Jesus Christ was just a wise teacher", which cannot both be true in the same context), but also for a calculus of microtheories, allowing various manipulations of not just assertions but whole contexts at a time. Importantly, queries to CYC are also tagged with a microtheory, effectively adding to the questions an attempt to specifY the context in which the question is being asked. For example, when asking the question "Was Jesus Christ the son of God?", one could specifY the microtheory 'AtheistDoctrineMt' and be told by eye "No, Jesus Christ was not the son of God". If, however, the microtheory label provided with the query was 'ChristianDoctrineMt', the answer would be "Yes, of course". The microtheories implementation in eye is based on a set of principles detailed in [4] that include the notion that breaking a huge knowledge base into contexts will speed both knowledge entry and inference. The need is removed for the knowledge engineer to specifY myriad

156

assumptions on which each truth that they enter hinges. Additionally, opportunities are provided for an inferencing engine to efficiently remove large chunks of the knowledge base from consideration based on what is known about, for example, the context of a query. To paraphrase Lenat, when asking if given that it is raining one should take an umbrella, it is not worth considering the number of legs on a spider or the birth date of Julius Caesar.

The current CYC approach to handling and exploiting context may look promlsmg. The microtheory implementation produces some seemingly clever and effective demonstrations, but leaves more questions than it answers. The consideration of context to be a multidimensional space is well and good, except that it is not clear exactly how multi-dimensional it is or should be. Lenat has explained the choice of twelve dimensions. However, his choices may not be comfortable or even comprehensible for others who intend to use CYC. He admits that there is no real limit to the number of dimensions of context-space that could be identified and that each dimension is almost certainly continuous rather than discrete.

More problematic is the question of how many actual contexts are enough. Already CYC has thousands, and there seems to be a clear preference on the part of people adding to the CYC knowledge base to name their contexts and work with them as labels, rather than specifYing in detail the location of each assertion in context-space. It seems likely that the number will continue to grow, particularly as the CYC knowledge base specialises. For example, the ACM Computing Classification System [5] classifies technical papers in the area of computing into nearly one hundred topics, each with two sub-levels of up to ten or more further topics. Perhaps CYC's general knowledge will make it unnecessary to specifY all of these hundreds or thousands of contexts, but even starting from such a specialised area as 'Computing', it takes a further four levels and possibly thousands of new contexts to specifY 'Object-oriented programming languages' and that doesn't seem like an over-specialisation for the context of a query. And when other fields are considered, such as medicine, engineering or folklore, the number of contexts appears to grow without limit. Possibly this will not be a problem at all once CYC understands natural languages and is configured to take advantage of large gridcomputing resources. In the meantime, which could be a long time given CYC's rate of progress so far, practical implementations of ontology-based systems require a much lighter approach to context.

3. Exploiting the task at hand

rF.F.F.~ COMPUTER

SOCIETY


In his exhaustive exploration of the world of 12-dimensional context space, Lenat makes a significant remark that context enables people to, among other things, "ignore 99.999% of our knowledge so that we can focus on the task at hand" [4]. Lenat's statement can appear to be a broad and sweeping consideration of the issue of context. If taken at face value it can be very revealing about what might a practical way to enable a machine to include context in its ontological reasoning mechanisms.

The problems associated with capturing complete knowledge have lead many to limit their efforts to domain-specific approaches to ontology-based systems. Limited-focus knowledge bases or ontologies have been referred to as purposive, a term that we believe neatly expresses the goal which is to specify all and only the knowledge required for a particular purpose or task. So when Lenat writes that context is some intangible thing that permits us to identify the 99.999% of our knowledge that is not relevant to our current purposeful activity, the conclusion could be drawn that context, practically considered, is defined as much by the purposive knowledge that has been included as relevant to the task at hand as by the knowledge that has been excluded. In other words, when trying to understand what makes some knowledge relevant to a task and what gives it specific meaning for that context, don't set aside the knowledge identified as necessary to the current purpose and looking for the context in the vast remainder of our knowledge. Instead, look to the defining characteristics of the knowledge already deemed relevant and seek the context in place.

In many ontology-based systems that perform information-oriented tasks, purposive knowledge bases or ontologies provide an efficient way to specify and represent the knowledge required to successfully operate in the required environment. Lightweight techniques such as knowledge unit analysis [6, 7] attempt to explicitly model common sense knowledge for a strictly limited domain, without trying to use a general purpose formal knowledge representation technique such as description logic. The lightweight approach runs counter to the view that the context that lends meaning to a statement must be something unspecified, that "knowledge we have never needed to write down". It avoids the mysticism with which context sometimes seems to be viewed by considering the context to be inherently captured as the purposively relevant domain knowledge units are specified. Put flippantly, if you have defined the task adequately that it can be successfully performed in the desired circumstances, you have surely defined the context sufficiently, or else how is the task being performed?

Even if it is not explicitly discemable, the incorporation of sufficient context can be determined to be present by considering whether the task is being

157

performed correctly. Knowledge unit analysis works, to a degree, at the sub-concept level, by assuming that where there are knowledge units in a text, the concepts that the knowledge units represent are present, rather than trying to decide if the concepts themselves are present or not [7]. In this way, because concepts are not formally defined and generalised with theoretically complete assertion and axiom systems, the problem of formally locating those which are not concepts or assertions but elements of context is avoided. The context is explicated, but it is not discerned from the concepts and other elements that comprise domain knowledge.

The close relationship between information and context is due primarily to the importance of the environment in defining and performing a task. The environment in which a task is to be performed plays a crucial part in the definition of the task, often implicitly. The crucial nature of the environment is true for tasks performed by people and by information agents.

4. The SportsFinder information agent

The SportsFinder information agent, developed in the Intelligent Agent Lab at the University of Melbourne, was a successful implementation of a knowledge unit based on-line system to report to its user the results of recent matches played by a chosen sports team [8, 9]. Developed in a predominantly evolutionary way, SportsFinder showed not only significant success in assessing a given web page to interpret its contents and highlight the information about a certain team, but also remarkable adaptability. Developed to report the outcomes of soccer matches, it was adapted to a range of team sports including basketball, rugby and Australian Rules football. The SportsFinder agent was able to cope well with new information sources and requests concerning unknown teams, and even more impressively, was able to incorporate new sports on the fly by the user filling out a simple template. Despite - or perhaps due to - the fact that it understood nothing about natural language, Sports Finder coped equally well with the Italian Serie A results as it did with the FA Premier League. A successful demonstration was the reporting of results from a Dutch draughts league with no customisation.

rF.EE~ COMPUTER

SOCIETY


The knowledge units that SportsFinder dealt with included the following:

Score format: Team name:

Winning team:

<number> '-' <number> (a sequence of) <capitalised string> the team with the numerically greater score

Score correspondence: the left-most team is given the left-most score, regardless of the ordering of actual score numbers and team names

Conceivable scores: 0 ~ potential score ~ 13 (for example)

Irrelevance: anything between parentheses, e.g. given the information Inter Milan 2 (Baggio 12. Baresi 67) de! Lazio 1 (Simone 34), SportsFinder does not report that Inter Milan lost to Baggio 12 goals to 2.

So where is the context? The point of describing the SportsFinder information agent is to show the success that a lightweight, purposive knowledge base or ontology gave, despite the clearly high dependency on context of the task of being able to identify which words and number on a sport results page are actually teams and scores, and which teams and scores together represent the outcome of an actual match. This example has clear relevance to the issue of having to rely on context to interpret a text - does the text '3' represent the number of goals scored by a team, the minute in which one goal was scored, the day or month of the fixture, the number of a player dismissed from the field for foul play, the minute in which she was sent off or the position on the league ladder one of the teams now occupies as a result of losing the match? Obviously the immediately surrounding texts define the meaning of the number. How much of the surrounding text is required? SportsFinder deliberately started with the minimum consideration necessary to solve the problem of reporting match results in one instance, and slowly grew in knowledge until it could handle a very wide range of soccer results information sources. (Interestingly, SportsFinder did not actually require a list of team names in order to function, being able, given one team name as its user input initial conditions, to deduce the opposing team's name via its knowledge units.)

It is not, however, necessary to build up the required knowledge units slowly on an informed trial and error basis. The conversion of SportsFinder from soccer to Dutch draughts demonstrated that it is possible to efficiently identify the knowledge units for a new domain that will permit the agent to migrate with success. This is

158

especially encouraging, as it implies that newly created agents could be informed from their beginning with the results of knowledge unit analysis of their intended domain and environment.

We want to emphasise that people can and do perform tasks in a corresponding manner to the behaviour exhibited by SportsFinder. The second author enjoys the challenge when travelling internationally of looking up sports scores in local newspapers. He is usually successful despite the wide variety of formats of the newspapers and the fact that he rarely understands the local languages. The task of locating the scores is made achievable by its context. For example, the assumption that the scores will be present explicitly in a certain part of the newspaper makes it possible to ignore large parts of knowledge, such as the syntax of foreign languages.

5. Meaning and relevance

The knowledge units used by SportsFinder, and the heuristics by which they were generated, raise interesting issues concerning meaning and relevance. At first glance, it is satisfying to imagine that SportsFinder is at some level determining the meaning of individual elements of information, or knowledge units. Certainly, it seems appropriate to consider the information agent analysing a line that contains several numbers and deciding that this number is probably not a score and that that number most likely is. From this perspective, the meaning of the numbers has been determined and they have been ruled out as candidates for inclusion in the fmal answer to the user's question about how their team fared. But an alternative consideration could view the actions of SportsFinder as assessing the relevance of each knowledge unit to the original task. As elements of the web site being processed are deemed unlikely to warrant inclusion in the fmal response, they are removed. This is where domain-specific knowledge such as "content within parentheses is not a score" comes into play. Only then is what is left interpreted and an answer formulated for the user, via other domain-specific knowledge such as "the left-most team is given the left-most score".

Literally, context is concerned with meaning. It describes the influence that related facts and assertions have on the meaning of a particular statement. But very often it is not clarification of meaning that we require, but an evaluation of relevance. In these cases it is not so much that context is used to fully determine the meaning of a text, but merely to determine that the meaning is not something that we are interested in - that whatever it means, it's not relevant. This goes to the core of Lenat's hope for CYC that incorporating context into the huge knowledge base will greatly reduce the processing required to answer queries and infer new knowledge. It

rFFE~ COMPUTER

SOCIETY


also provides a compelling reason to consider the specifics of a task or purpose as defining the context. As SportsFinder shows, the approach of defining the knowledge units required for a task can leverage context by excluding all other knowledge. Whether, theoretically, this is by determining the meaning via context or evaluating the relevance via context seems to be a point of semantics rather than practical concern.

The techniques demonstrated by SportsFinder have been employed in a number of other successful information agents within the Intelligent Agent Lab [10]. These agents operate in a wide variety of domains, including CASA, for searching semi-structured real-estate classified advertisements [6], Justice, which uses the context of a legal case to report summary information about judgements and rulings [11], and CiFi, an agent that retrieves citations from the WWW [12]. Each agent uses lightweight purposive knowledge that both defines the target information and bounds the context of the agent's task. Additionally, a wide range of crude information agents developed by students in a graduate software agents class attest to the robustness of a task being made possible by restricted context. Rather than dismissing these agents as ad-hoc, we view them as exhibiting purposive knowledge as a feature.

6. Future work

The danger with a series of ad-hoc successes is that they remain nothing more than success in isolation. The challenge we see is to generalise the process by which agents with purposive knowledge can be designed, as participants in multi-agent environments as well as individual information agents. The issue of formalising the development of purposive knowledge-based agents is one we are addressing in the Intelligent Agent Lab.

As mentioned in Section 1, multi-agent systems are an area in which multiple view-points and independent interpretations of data and events are increasingly becoming a topic of interest. Organising the activities of individual agents within such systems into roles and responsibilities is a promising approach that is being integrated into agent-oriented software engineering methods.

In the ROADMAP method being developed by our group within the Intelligent Agent Lab [13], for example, roles represent a set of activities and constraints that together define a comprehensible unit of tasking. In GAIA [14], the agent-oriented software engineering method on which ROADMAP is based, the domain-specific knowledge required for each role is implicitly encoded in the attributes of the individual roles [13]. This corresponds quite closely to the idea presented in this paper that the context is in the task. For the future,

159

however, it is hoped that it will be possible to further examine the nature of the knowledge used to defme a task or role with a view to modularising it to permit reuse -ROADMAP aims to do just this [15].

It would be useful to investigate the possibility of determining what, if anything, makes a piece of knowledge contextual rather than simply informative or instructive. Our feeling is that this will not be possible completely, but will require packaging knowledge into task definitions and then replacing individual units of knowledge as necessary for reuse. The basis for this suggestion comes from observing that although people very readily categorise things such as tasks loosely, and can happily say that, for example, searching for a person is a task and that task can be performed in a variety of contexts, it seems obvious that searching for a person in an office building is a fundamentally different task from searching for a person at an airport or lost at sea. When it becomes necessary to define a task so completely that a computer can perform it, many assumptions and generalisations fall away and it becomes clear that context is not a mystical cloud of extra knowledge but something intrinsic to the task itself.

7. Acknowledgements

This research is supported by Discovery Grant DP0209297 from the Australian Research Council. Thanks to the members of the Intelligent Agent Laboratory at the University of Melbourne for many useful discussions, especially Maia Hristozova and Thomas Juan.

8. References

[I] Cycorp Inc., "CYC," 2002. [2] C. Legg, "Implementation of a Large-scale

General Ontology at Cycorp," presented at Departmental Seminar, Department of Computer Science and Software Engineering, The University of Melboume, 2002.

[3] B. Smith, "The owl and the electric encyclopaedia," Artificial Intelligence, vol. 47, 1991.

[4] D. Lenat, "The Dimensions of Context-Space," 2002. Accessible via Cycorp Inc. web site http://www.cyc.com/contextspace.doc.

[5] Association for Computing Machinery Inc., "ACM Computing Classification System," 2002.

[6] X. Gao and L. Sterling, "A Methodology for Building Information Agents," in Web Technologies and Applications, Y. Yang, M. Li,

IEEE~ COMPUTER

SOCIETY


[7]

[8]

[9]

[10]

[11 ]

and A. Ellis, Eds. Beijing: International Academic Publishers, 1998. L. Sterling, "Surveying and Reviewing Methods of Representing Scenarios (Technical Report)," Department of Computer Science and Software Engineering, The University of Melbourne, Melbourne, Australia 1998. H. Lu, L. Sterling, and A. Wyatt, "Knowledge Discovery in SportsFinder: An Agent to Extract Sports Results from the Web," presented at 2nd Pacific-Asia Conference Knowledge Discovery and Data Mining (PAKDD 1998). Melbourne, Australia, 1998. A. Wyatt, "SportsFinder: An informationgathering agent to return sports results," in Department of Computer Science and Software Engineering. Melbourne: The University of Melbourne, 1998. L. Sterling, "On Finding Needles in WWW Haystacks," presented at 10th Australian Joint Conference on Artificial Intelligence (AI 1997). Perth, Australia, 1997. J. Osborn and L. Sterling, "Automated Concept Identification within Legal Cases," The Journal

160

[12]

[13]

[14]

[15]

of Information, Law and Technology (JILT), vol. 1, 1999. S. Loke, A. Davison, and L. Sterling, "CiFi: An Intelligent Agent for Citation Finding on the World-Wide Web," presented at 4th Pacific Rim International Conference on Artificial Intelligence (PRICAI 1996), Cairns, Australia, 1996. T. Juan, A. Pearce, and L. Sterling, "ROADMAP: Extending the Gaia methodology for Complex Open Systems," presented at First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2002), Bologna, Italy, 2002. M. Wooldridge, N. Jennings, and D. Kinny, "The Gaia Methodology for Agent-Oriented Analysis and Design," Journal of Autonomous Agents and Multi-Agent Systems, vol. 3, 2000. T. Juan and L. Sterling, "A Meta-model for Open, Intelligent and Adaptive Multi-Agent Systems," presented at Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2003), Melbourne, Australia, 2003.

I EEE~ C OMPUTER

SOCIETY

Documents

Tasks as Context for Intelligent Agents