Representing temporal knowledge in conceptual graphs

Representing temporal knowledge in conceptual graphs

Bernard Moulin and Daniel C6t6

This study was motivated by some difficulties encountered by the authors when trying to express temporal knowledge using Sowa's conceptual graph (CG) approach. An overview of Sowa' s approach is given and the difficulties encountered when trying to model temporal knowledge are outlined: the disparity of notations allowed by CG theory for expressing temporal information; the ambiguity and incompleteness of tense specification; the difficulty of harmonizing tenses and intergraph temporal relations. Various approaches sug- gested for representing time both in artifcial intelligence and linguistics are presented, and an extension to Sowa's approach is proposed in which temporal and non- temporal knowledge are differentiated. In this model points in time are represented as well as time intervals. A semantic interpretation of verbs is provided based on an extension of Reichenbach's model of temporal markers. The authors show how their approach enables the representation of tenses as well as the aspectual properties of natural language sentences.

Keywords: conceptual graph theory, temporal knowledge representation

This study took place during a long-term project called GENTEXT (Generation of Text) in which we aim to develop a system that will be able to generate texts in the French language, starting from knowledge structures expressed in a form equivalent to Sowa's conceptual graphs 1. These knowledge structures can be proposed by a user or generated by a planning system 2,3.

We chose to use the conceptual graph theory ~ because it provides a knowledge representation approach which is compatible with most conceptual modelling techniques used in artificial intelligence, database design, cognitive psychology, linguistics etc. Natural language generation (NLG) is a research domain in which are studied the various approaches and techniques which enable a system to respond to a

Drpartement d'informatique, Universit6 Laval Ste Foy, Qurbec G1K7P4, Canada

Paper received 18 October 1990. Revised 1 July 1991

user using natural language utterances 4-6. The conceptual graph (CG) approach provides a rich conceptual framework for representing semantic knowledge especially for natural language (NL) processing. Sowa has proposed various guidelines for NL generation 1, but few papers 7 have been reporting developments in this area.

The present study was motivated by some difficulties encountered when we tried to apply the conceptual graph approach for NL generation, especially when we had to express temporal knowledge. In order to overcome these difficulties we developed a conceptual framework for specifying temporal knowledge, using Sowa's CG approach.

In the next section we give an overview of Sowa's approach and we indicate the difficulties we encountered when we tried to model temporal knowledge: the disparity of notations allowed by CG theory for expressing temporal information; the ambiguity and incompleteness of tense specification; the difficulty in harmonizing tenses and intergraph temporal relations. Later we present various approaches which have been proposed for representing time both in artificial intelligence and linguistics, then we propose an extension to Sowa's approach in which we differentiate temporal and non-temporal knowledge. In our model we represent points in time as well as time intervals. We provide a semantic interpretation of verbs based on an extension of Reichenbach's model of temporal markers. We show how our approach enables us to represent tenses as well as the aspectual properties of natural language sentences.

CONCEPTUAL GRAPH APPROACH

Sowa indicates that 'conceptual graphs may be viewed in any of the following ways: a form of logic with a graph notation instead of a linear notation; a set of data structures that are generated in computer storage as a result of processing natural language; a psycho- logical hypothesis about mental representations that arise in the human brain while people are thinking and talking; a notation for stating linguistic hypotheses about semantic patterns in natural language without making any assumption about how these patterns are

Vol 4 No 4 December 1991 0950-7051/91/040197-12 (~) Butterworth-Heinemann Ltd 197

related to brain activity; a knowledge representation language for implementing expert systems and natural language processors TM.

We consider that conceptual graphs and the related operations present all the characteristics of a 'pivot conceptual language' which can be used to support the representation of networks of concepts (or semantic nets) in artificial intelligence and cognitive sciences, especially in the domains of knowledge representation and natural language processing. Below we present briefly the main characteristics of Sowa's approach which are relevant for expressing semantic knowledge. We then identify some problems which arise when we try to express temporal knowledge using conceptual graphs.

Conceptual graphs Concepts are the representations of objects of the application domain. A concept is characterized by two elements:

• a type which represents the set of all the occurrences of a given class: human, animal, man, woman, student are examples of types;

• a referent which represents a given occurrence of the class which is associated with the concept; for example man and woman may have respectively John and Mary as referents.

Using a graphical notation, a concept is represented by a rectangular box. Using a linear notation, we specify it between square brackets: [TYPE-NAME : referent].

Concept types are organized in a hierarchy which supports operations on concepts: generalization and specialization. An ordering relation between two types ci and cj is specified by C i ~ Cj if ci is a generalization of cj. For instance HUMAN I> WOMAN, ANIMAL I> MONKEY, ACT I> EAT . . . Sowa has transformed the type hierarchy into a lattice structure, by adding a 'universal type' UNIV which is the most generalized concept (for every type t, UNIV I> t), and a type ABSURD which is the most specialized concept (for every type t, t I> ABSURD).

A link between two concepts is called a conceptual relation. A conceptual relation is always associated with a 'sink concept' (by means of an output arrow) and zero, one or several 'source concepts' (by means of input arrows). Using a graphical notation the conceptual relation is represented by an oval box from which exits only one arrow and to which may enter several arrows. Using a linear notation the conceptual relation is represented between brackets and associated with concepts by means of arrows:

[TYPE. : *x.] ~ (RELATION)- ~--[TYPE1 : *Xl]

~--[TYPE,_1 : *x,_l].

Conceptual relations are usually binary. They correspond often to 'semantic cases '9 such as 'agent' (AGNT), 'object' (OBJ), 'instrument' (INST), 'patient' (PTNT), 'characteristic' (CHRC), 'point in time' (PTIM), 'duration' (DUR), 'successor' (SUCC), 'beneficiary' (BENF), 'experiencer' ( E X P R ) . . . Sowa

pE R SO N : ~ 1 ~ ( ~ ~ F - - - - ~ ( ~ ~ CITY: Boston

EARRIVE,- " ' " ~ ~ (AGNT) ~ ]PERSON: Mary] (LOC) ~ ]CITY: Boston]

(PTIM) ~ ]DATE: 90-7-3]

Figure 1. Graphical and linear presentations of a conceptual graph representing 'Mary arrives in Boston July 3rd 1990'

introduced unary relations for expressing modalities (i.e. 'obligation' (OBL) or 'possibility' (PSBL)) and tense information: 'past' (PAST) or 'future' (FUTR).

Conceptual graphs (CG) are finite, connected and bipartite graphs, whose nodes are either concepts or conceptual relations: in the graph an edge can only relate two nodes whose categories are different. In Figure 1 we find an example of CG in both linear and graphical notations. CG are also organized with respect to a generalization hierarchy.

Conceptual graphs are used to represent knowledge structures at a semantic level. Hence, they support the abstraction operation which is most important when it comes to processing and organizing knowledge.

A lambda abstraction A al,... ,an U, consists in a canonical graph U, called the body, together with a list of generic concepts al,...,a, contained in U, called 'formal parameters'. The parameter list following )~ dis- tinguishes the formal parameters from the other concepts in U. The abstraction operation allows us to represent various types of conceptual structures, namely type definition, prototype specification, schema definition, individual definition and conceptual relation specification.

For instance a type label is a kind of a ;~ abstraction. In any concept, the type label can be replaced by the A expression that defines it. For example the definition of PET-CAT can be represented using a type A abstraction:

type PET-CAT(x)/s [CAT : *x] ~-- (OBJ) ~-- [OWN] -o (AGNT) --~ [PERSON].

New types of conceptual relations can also be defined by a A abstraction, for example Sowa u defined the relations BFOR ('before') in terms of the PTIM and SUCC relations and interpreted them as 'x corresponds to a point in time which is a successor of a point in time associated with y':

relation BFOR (x, y)/s [*x] ~ (PTIM) -~ [TIME] ~ (SUCC) -~ [TIME] ~-- (PTIM) ~-- [*y].

In natural language it is common to use sentences which are composed of several propositions (principal and relative propositions) and to use anaphorical terms. For instance in the sentence 'Mary tells John that she will not arrive in Boston on 90-7-3, 'she' is an anaphora representing 'Mary'. Sowa 1 introduced the notion of 'embedded conceptual graphs' (or 'nested conceptual graphs') to represent the nesting of contexts within conceptual graphs. In order to specify embedded graphs, Sowa uses either the concept of 'proposition'

198 Knowledge-Based Systems

or of 'situation' whose referent may be one or more CG*.

Figure 2 presents an interpretat ion of the preceding sentence in the form of conceptual graphst . When concepts of equal nature appear in different contexts such as 'Mary ' and 'person ' , we represent the corre- spondence between them by an identity line. As a mat ter of fact this enables us to express anaphoric references.

Although Sowa's theory is a fairly advanced one for knowledge representat ion, we have encountered various problems when trying to model temporal knowledge: the disparity of notations allowed by C G theory for expressing tempora l information; the ambiguity and incompleteness of tense specification; the difficulty in harmonizing tenses and intergraph temporal relations.

Disparity of CG notations for expressing temporal knowledge In Sowa's work 1,1°,11 different indications for expressing temporal knowledge can be classified in three categories.

Monadic conceptual relations such as PAST, FUTR are used to indicate that a C G describes a situation which takes place respectively in the past and future. The relation PRST which indicates that a situation takes place in the present t ime is considered to be implicit and not represented in CG. As we shall see below the semantics of these relations are not precise enough to express the richness o f tenses in NL.

Dyadic conceptual relations such as PTIM (point in t ime), FREQ (frequence) or DUR (duration) are used to relate concepts to t ime concepts such as [TIME] and [ T I M E - P E R I O D ] within a conceptual graph. The relation SUCC is used to indicate that a t ime point is a successor of another. These specialized concepts and conceptual relations are useful for expressing temporal knowledge associated with points in time, but cannot be used to describe the propert ies of t ime intervals 13 which is necessary for NL generation.

Dyadic conceptual relations such as BFOR (before), AFTR (after) are used to express temporal information relating conceptual graphs and are called ' intersentential relat ions ' lh Indeed these relations are necessary

* Sowa has refined the semantics and notations for nested contexts in several papers~m2: 'A situation is defined as a state of affairs that occurs at a single place and time' . . . 'A context is a set of propositions that describe a situation'. In the graphical notation, a context is represented by a rectangle including one or more CG. Sowa provides an example ~2 (Fig. 5, p E06.7) where a situation is nested in a proposition which is nested in another situation. In some cases, it is not easy to decide whether we should use a 'situation node' or a 'proposition node' in a graph. This decision can be quite subjective and dependent on epistemologic arguments. For instance, somebody may choose that the object of the ACT of 'thinking' is a proposition. But we can argue that the object of an ACT of 'thinking' corresponds to ideas that can be viewed as 'abstract situations'. In this paper we will use only the node 'SITUATION' for introducing an embedded CG with respect to Sowa's notation. In the model that we propose we will use a more generic node called 'NTCG' (non- temporal conceptual graph'). t This is a technical note on Sowa's notation. Tense relations like FUTR ('future') in our example or PAST, should be attached to 'SITUATION' nodes according to Sowa TM. However, if we want to associate a tense indication with an embedded proposition, we have to attach the tense relation to a PROPOSITION NODE, as we do in this example.

SITUATION 1

SITUATION2

Figure 2. Embedded conceptual graphs representing the sentence 'Mary tells John that she will arrive in Boston July 3rd 1990'

but, as we shall see later, some problems arise when we have to combine intersentential relations and monadic tense relations for expressing the propert ies of temporal adverbs or temporal conjunctions.

Difficulties in specifying verb tenses within CG theory

The monadic relations PAST and F U T R (and the implicit PRST) are not precise enough and are even ambiguous when we need to express verb tenses encountered in NL. The reason is that the choice of verb tenses, temporal adverbs ( tomorrow, today etc.) or temporal expressions depends on the temporal properties of the described situations, which are conveyed not only by tense indications but also by aspectual characteristics of verbs 14,15.

For instance using the PAST relation, we cannot differentiate the tenses of the C G associated with the following sentences: ' John opened the door and called Mary ' (sent. 1) and 'John was opening the door when he called Mary ' (sent. 2).

Fur thermore , temporal adverbs or expressions may change the semantic interpretation of verb tenses. Consider the following sentences which express two situations taking place in the future: ' John will arrive by plane tomorrow' and ' John arrives in a minute ' . In the second sentence using the present in conjunction with the expression 'in a minute ' corresponds to a semantic interpretation of a situation that takes place in the future.

If we want to describe aspectual propert ies of situations, we need to consider the time intervals during which situations take place. For instance in sentence 2, the event ' John called Mary ' (sent. 2.1) took place during another activity, when ' John was opening the door ' (sent. 2.2). Hence the t ime interval associated with the situation described by sent. 2.1 is included in the time interval associated with the situation described by sent. 2.2.

The present is a notion which is not easily understood and modelled. In Sowa 1 the present is considered as a default option: when we do not attach a specific monadic tense relation to a CG, it is supposed to correspond to a situation taking place in the 'present t ime' (usually indicated as 'Now' when required). For

Vol 4 No 4 December 1991 199

example, the PAST relation is defined by the following CG (ibid p113).

Relation PAST (x) is" [SITUATION: *x]---> (PTIM)--> [TIME]--~ (SUCC) --> [TIME: Now].

'Now' is replaced by 'the speech time', the time at which the sentence is uttered 1°. This example illustrates also that in CG theory temporal knowledge is mainly expressed by points in time (see above).

Physical time, which is usually measured with respect to periodic physical phenomena, is considered to be objective: the measurement does not depend on the measuring subject. But most NL temporal expressions cannot be expressed according to physical time. Basi- cally 'the time we speak about in our sentences' is often subjective and relative to the speaker (or locutor). Hence 'Now' is not an absolute value and must be defined relatively to the 'speech time' (the time interval when the locutor utters the sentence).

It should be possible to specify the time associated with the situation relatively to the speaker's time, as well as in an absolute way relatively to the 'official physical time'.

Difficulty in harmonizing tenses and intergraph temporal relations

Using the conceptual graph theory, it is difficult to associate different tenses in principal and subordinate sentences, as well as the required verb mode (infinitive, indicative, subjunctive, conditional). For instance, consider the sentences: 'the dog that is barking, belongs to John' (sent. 3.1) and 'the dog that was barking, belongs to John' (sent. 3.2). Usually a relative clause is expressed by a 3. abstraction in the type field of the concept such as:

[(3. x) [DOG : *x] *-- (AGNT) *-- [BARK]: #] *-- (PAT) <--- [BELONG] ---> (RCPT) ---> [John].

To differentiate sentences 3.1 and 3.2, we should introduce a PAST relation in the conceptual graph which defines the 3. abstraction. But that is not really consistent with the fact that 3. abstractions are mainly used for defining concepts and that definitions should be time independent.

In order to relate two propositions, we use temporal connectives such as 'when', 'while', 'before' etc. Sowa u proposed to express in CGs such connectives by intersentential relations (BFOR, AFTR). Sometimes, the use of a temporal connective induces a change in the tense and/or the mode of the verb of the second sentence. Such a subtlety is difficult to express using conceptual graphs. For instance, consider the following sentences and the associated CGs: 'John bought an apple and came to school' (sent. 4.1)*:

CG 4.1: (PAST) --~ [[PERSON : John] *-- (AGNT) ~-- [BUY] ~ (OBJ) ---> [APPLE]]

(AND)

* Sowa indicates 11 that 'to simplify the graph, the labels SITU- ATION and PROPOSITION may be omitted, implying the default type UNIV'. That is what we did in this example where the contexts a r e represented by the first level of square brackets.

[[PERSON : JOHN l <--- (AGNT) <-- [COMEI --~ ( D E S T ) ~ [SCHOOL]I ~ (PAST)

and 'John bought an apple before coming to school' (sent. 4.2):

CG 4.2 (PAST) ---> [[PERSON : John] ,-- (AGNT) ~-- [BUY]---> ( O B J ) ~ [APPLE]]

(BFOR) [[PERSON : John] ~-- (AGNT) <--- [COME] --> (DEST) ~ [SCHOOL]] *- (PROG).

In CG 4.2 we have chosen to express the temporal relations (PAST) and (PROGressive) to reflect the tenses required by the use of the conjunction 'before'. But semantically both the actions of buying an apple and coming to school are occurring in the past. Hence, for expressing temporal knowledge, we need a formalism which is independent of the surface form of sentences which expresses this knowledge.

Sowa never mentioned any process that would enable us to check the coherence of temporal knowledge in conceptual graphs. Such a process is an important one for an NL generation system since it is necessary to verify that the input knowledge structures are coherent before trying to generate sentences.

REPRESENTING TIME

Researchers have studied time from different perspec- tives in physics, philosophy, linguistics, cognitive psychology as well as in artificial intelligence. Some fundamental questions arise: Is time discrete or contin- uous, absolute or relative, linear, parallel or branching, bounded or unbounded? Various approaches have been proposed to answer these questions, but studying them goes beyond the scope of this paper (for an overview see Reference 16). First, we will present informally general reflections about the perception and the representation of time, then we will discuss some fundamental questions related to the study of time in linguistics. Finally, we will summarize the main issues involved in time modelling from the perspective of knowledge representation.

How can we represent time?

Intuitively, we consider situations (or states of affairs) or phenomena that occur in the real world or in some hypothetical world. Temporal information is used to relate situations together or to position them according to an official time scale (universal time, calendar etc.)

Vendler 17 proposed a classification of situations by distinguishing between non-terminative situations ("states" and "activities") and terminative situations ("accomplishments" and "achievements"). "States" are durative situations which do not entail changes such as 'to possess a diploma'. "Activities" have a duration and are homogeneous such as 'he runs'. "Achievements" are considered to have no extension in time (such as 'to open the door') in contrast to "accomplishments" which are furthermore directed towards a result (such as 'to paint a picture').

Dorfmuller-Karpusa ~5 summarizes some important points related to human perception and intuitive use of time: 'The time concept which originates in a natural


feeling of the succession of events is based not only upon the intuitive relations "before and after", but also upon the intuition of simultaneousness. The latter forces us to replace time points on the time axis by time intervals and to consider the relations of t h e s e . . . Physical time, i.e. the time which can be measured with respect to a periodic physical phenomenon, is considered to be objective in the sense that the result of the measurement does not depend upon the measuring subject. On the other hand, the time we currently speak about is basically subjective, but in our culture we are accustomed to connect it directly to the physical time as measured by our watches and determined by our daily rhythms. Both are considered to be located in a one-dimensional continuum which enables us to arrange our perceptions and generally states of consciousness'.

In natural language we are able to speak (or reason) about the relationships existing between situations either relatively or with respect to some time reference. Hence we must be able to represent basic notions such a s :

• situations considered as time points, and situations having a duration;

• the position of a situation with respect to some time scale using various units (second, day, year, millen- ium etc.);

• the relations between two situations must bear notions of anteriority, posteriority and simultaneousness;

• time is often considered as a relative notion, with respect to some situation of reference (i.e. the time when a sentence describing a situation is uttered);

• some situations are depicted with a 'temporal structure' (for instance: duration, progressivity, iterativ- ity etc.);

• time may be described in a fuzzy way (using expressions such as 'few days ago', 'in the near future') or even implicitly (i.e. 'I will come and see you').

Time and natural language

Natural language offers a variety of ways for expressing temporal information, at a lexical level as well as at sentence and discourse levels. Figure 3 presents a classification of temporal markers with respect to the time localization and the structure of events (the French version is in Reference 18). Ehrich 19 proposes an equivalent classification for verbs and adverbs for German.

Vendler's categorization of situations 17 provided an initial frame for developing lexical semantics of verbs and for classifying them in categories such as state verbs, activity verbs, accomplishment verbs, achieve- ment v e r b s 18,19.

Studying time in NL raises various questions related to tenses and the aspectual properties of verbs. Reichenbach 2° proposed a model for interpreting semantically verb tenses. He localized on a time axis the point of speech (S), the point of reference (R) and the point of event (E) and gave a semantic interpretation of English tenses:

R

The o n a

I saw John

I have seen John

I will see John

S is the time at which the statement is produced (time of locutor);

E is the time at which occurred the event represented by the statement; is the time of the temporal reference according to which are situated the locutor and the event.

markers E, R, S have different relative positions temporal axis according to the verb tense:

E,R S ~t

E R,S ~t

S E,R ~t

Using the concomitance relation (noted ',') and the precedence relation (noted ' _ ' ) between these markers, we have the following representations for verb tenses:

Perfect (E,R _ S) I saw Present perfect (E __ R, S) I have seen Pluperfect (E R S) I had seen Present (E, R, S) I see Future (S _ R, E) I will see Future perfect (S E R) I will have seen

This model bears some limitations. With regards to the relations introduced by Reichenbach, the markers correspond to points in time. They cannot be used to describe aspectual properties of sentences such as the distinction between the perfective aspect (for instance 'I saw') and the imperfective aspect (for instance 'I was seeing').

Dorfmuller-Karpusa 15 extended this approach by considering the three temporal markers as time intervals. She indicates: 'The economy of language entails a large temporal indeterminacy, especially if we take in account the fact that states of affairs occupy intervals on the time axis and not points' . . . 'The difference between the perfective and the imperfective aspects is often considered as follows: in the perfective aspect the state of affairs is felt as an inseparable whole without considering its different phases. On the other hand, a state of affairs is described in an imperfective aspect if the producer takes into account its internal structure. Temporal as well as aspectual relations are producer- dependent and consequently subjective, the degree of subjectivity being, however, different. Whereas the temporal relations are defined by the point of speech, the aspectual relations are defined by the point of reference, the latter being dependent upon the producer's attitude towards the state of affairs he describes.'

For instance consider the following sentences and the corresponding positions of temporal markers S, E = [el, es] and R = [ri, rs].

'John opened the door' (sent. 6.1)

ei=ri es=rs S - - [ ] ~ [ - - ] - - > t

'John was opening the door' (sent. 6.2)

ei ri rs es S - - [ ~ [ ] ~ ] [--]--~ t

Both sentences correspond to past events (E is anterior to S). In the perfective case (sent. 6.1), the event E is


Verbs Adverbs Temporal conjunctions

Present Time localization of situations

Temporal structure of situations

Past (i.e. pluperfect)

Future (i.e. future perfect)

1. durative / punctual 2. perfective /

imperfective 3. progressive 4. iterative etc.

1. to spend, to take 3 days for

2. to finish to 3. to be working 4. to rewrite etc.

Temporal reference

deictic

yesterday tomorrow nowadays

anaphoric

later before after

Temporal structure

suddenly long periodically progressively often etc.

Simultaneousness anteriority posteriority

when, as soon as, before tha t . . .

Aspectual properties

each time that during etc.

Figure 3. Temporal markers in natural language

considered as a whole, and we can consider that the reference interval R coincides with it (according to Reichenbach's model). To represent the impeffective case, the reference interval R must be included in the event interval E.

Reichenbach's model was initially applied at the sentence level. It can also be extended to describe the temporal and aspectual structure of a text as a whole. Dorfmuller-Karpusa ts indicates: 'In the case of a text, we have to abandon the principle of permanence of the reference point. It appears that in a text we are faced with a complex nexus of described states of affairs in such a way that the same state of affairs may represent a point of event and a point of reference for other states of affairs'.

Borillo et alJ 8 extended Reichenbach's approach in a similar way: they represented the temporal markers as time intervals and used Allen's temporal log ic ~3 to reason on their relationships. These authors demon- strated that their approach can be used to represent verb tenses in the indicative mode, as well as temporal adverbs ('yesterday', 'today', 'tomorrow') and temporal conjunctions ('when', 'while' etc.). Algorithms can be implemented in order to check automatically the proper use of tenses, temporal adverbs and conjunctions in sentences.

Time considered from a knowledge representation perspective Kwong 2~ identifies some issues which must be addressed when modelling time from a knowledge representation perspective:

• temporal determinism (we can attach to the non- temporal part of a statement either an explicit date or time, or a relative temporal reference);

• granularity (we must provide representations for various time spans such as nanosecond, day, year, century etc.);

• points or intervals (which approach is more appropriate to represent temporal phenomena?);

• fuzziness (fuzzy time information is given because it is unnecessary to precis it in phrases containing expressions like 'yesterday', 'one year ago', 'in a month');

• persistence (any fact asserted to the knowledge system should remain in the fact base until it is explicitly contradicted or removed);

• specification of the boundary between past and future (the present can take 'different durations' depending on the context);

• coexistence of time and other knowledge (in some cases temporal knowledge is insignificant or irrele- vant compared to other aspects of the representation).

Philosophers and logicians have proposed to model time using first order predicate logic (Russel and Quine) or modal logic (Taggart, Von Wright) (see Reference 16 for an overview). McDermott 22 proposed a temporal logic based on first order logic and allowing the modelling of possible futures along different time lines. In this model situations are described as sets of partially ordered states (equivalent to "an instanta- neous snapshot of the universe") related to the linear time line; the indeterminacy of future is represented by different successions of possible states in a tree-like structure. Although McDermott's model provides an interesting formal background, it is difficult to base on it the representation of temporal knowledge as it is expressed in natural language.

Allen's temporal m o d e P attracted much attention in the AI community. This model is concerned with temporal relationships relating time intervals and rejects the notion of point in time. Everything is considered to be taking place along some time interval. Allen identified 13 binary relations that can hold between two different intervals I and J: I Before J, I


Meets J, I Overlaps J, I During J, I Starts J, I Finishes J, and the symmetrical relations in addition to I Equal J.

When a new fact is added to a network of interval relations, the propagation of relations between pairs of intervals must be checked. Allen's temporal theory is based on three concepts: the "property" which holds over every subinterval of any interval over which it holds; the "event" whose occurrence defines a unique interval over which it occurs (and it does not occur over any subinterval of that interval); the "process" refers to some activity not involving a 'culmination or anticipated result' such as 'I am running'. In accordance with Galton 16 we find this last notion difficult to under- stand and in some ways ambiguous. The rejection of time points is also questionable because it is contradict- ing everyday practice, when we use a universal time scale. Nevertheless, Allen's approach emphasized appropriately the need for defining precisely the relationships that relate time intervals, and that is what we will hold from it.

AN E X T E N D E D C O N C E P T U A L G R A P H MODEL

We propose to extend the conceptual graph theory with a temporal model in order to overcome the problems we have encountered when we tried to represent temporal knowledge with conceptual graphs. This model takes its roots in different approaches from artificial intelligence and linguistics. Here are some guidelines that have oriented our research.

• We need a formalism which allows us to express concepts and conceptual relations related to points in time as well as time intervals.

• This formalism must provide means to indicate time references with respect to 'an absolute physical time', as well as time references relatively to the speaker's time.

• Within this formalism some constructs must enable us to express from a semantic point of view, tenses as well as aspectual properties of verbs.

• In NL some temporal structures provide intersentential relationships which support the discourse coherence. The required formalism must provide constructs which enable us to interpret combina- tions of verb tenses, temporal adverbs and connectives.

Representation of points in time and time intervals

The initial version of this model was proposed with the goal of representing temporal knowledge structures in a planning system 2,3,23. We decided to differentiate temporal knowledge from semantic knowledge in our knowledge structures.

In this model the world is described by a collection of facts. A fact is a couple <NTCG, TS> where NTCG is called a 'non-temporal conceptual graph' and TS is the time span over which the NTCG is known to be true in the world. An 'NTCG' is a non-temporal knowledge structure described by a conceptual graph. For instance:

[NTCG : [John] ~-- (AGNT) ~-- [BRING] --~ (OBJ) --~ [GIFT]].

The time span is an interval [LB, UB] defined on a discrete time scale. The time unit can be chosen according to the characteristics of the application and is provided by a clock which is used to determine the time points associated with the facts perceived by the system. The lower bound LB of the time span TS represents the time point when the fact starts its existence in the world, and the upper bound UB represents the time point when it disappears.

Temporal constraints are expressed using compara- tive operators < , ~< and = which relate known time points (instantiated with a constant) or unknown time points (represented by variables). We defined the equivalent of Allen's 13 relations between time intervals using the three basic relations defined between time points. Within the system, a function equivalent to a 'time specialist '24 manages the network of time constraints. This model can account for a point in time perspective, as well as a time interval perspective. In addition, we can specify temporal information either precisely (by instantiating time bounds with known values) or vaguely (by associating variables to time bounds).

We extended this first version of our temporal model in order to express time granularity and to represent intervals with unknown bounds and a fixed duration as we can find in expressions like 'in two years he learned Spanish', 'in five days he constructed a prototype' etc. In this new model 25 temporal information is specified by a 'temporal label' which is related to an NTCG by a conceptual relation that we will call ' temporal period' (PER).

A temporal label is specified by a triplet [unit, lwr- bnd, upp-bnd], where:

• unit is the precision of the observations (second, minute, hour, day etc.),

• lwr-bnd is the point in time when the situation was first observed,

• upp-bnd corresponds to the point in time when the situation was last observed*.

The lower and upper bounds can be either instantiated if the speaker knows their values, or set as variables if the values are unknown. In order to specify intervals comprising a fixed duration, we can also assign to these bounds expressions in the form (x + k), where x is a variable and k a constant instantiated according to the interval unit.

Again, using temporal labels we can specify information about points in time as well as time intervals and define relations between temporal labels similar to the relations that Allen introduced between intervals. Note

*We represent temporal labels in a way which is slightly different from the standard concept specification. Our notation can be considered as a shorthand for a concept; [TEMPORAL-LABEL : (unit, lwr-bnd, upp-bnd)] where (unit, Iwr-bnd, upp-bnd), appears in the referent part of the concept and specifies the temporal label parameters. Our notation was chosen in order to emphasize the fact that temporal labels are not processed in the same way as non-temporal concepts. In our model, temporal information may be viewed as located in a 'temporal plan' which is different from the 'conceptual plan' where non-temporal CGs are represented. In the temporal plan we can reason on temporal intervals associated with temporal labels and can apply the inference rules of an appropriate temporal logic.


that to distinguish relations between time points introduced by Sowa (BFOR, AFTR) and relations between time intervals, we will specify the latter in full words: BEFORE, EQUAL, MEETS, OVERLAPS, DURING, STARTS, FINISHES (see Appendix 1 for the definition of these relations). Here follow some examples. 'John buys a gift before visiting the town' is expressed in our model by:

[NTCG: [John] ~-- (AGNT) ~-- [BUY] ~ (OBJ) --, [GIFT]]--~ (PER)---> [ul, bl , b2]

[NTCG : [John] ~-- (AGNT) ~ [VISIT] ---> (DEST) --~ [TOWN]] --, (PER) --~ [ul, b3, b4]

[ul, bl, b2] - , (BEFORE) ~ [ul, b3, b4].

'In five days John can repair his house':

( P S B L ) - - * [NTCG: [John] ~ (AGNT) ~-- [REPAIR] ~ (OBJ)---, [HOUSE]]

(PER) ~ [day, hl, bl + 5].

Representing non-temporal semantic knowledge Sowa's approach LH provides a fairly complete model for representing semantic knowledge. We are restat- ing here some notions introduced by Sowa in order to fit them in the description of non-temporal knowledge structures.

An NTCG ('non-temporal conceptual graph') is a conceptual graph within which all conceptual relations are non-temporal. All the notions that were defined on Sowa's conceptual graphs can be rede- fined in a similar way on our NTCGs (canonical states, operations for manipulating states, ~ abstractions etc.).

Sowa uses the concept of PROPOSITION in order to describe embedded CGs (see Footnote on page 199 of this paper). In our model NTCGs can be embedded in other NTCGs. For instance the graph representing the sentence 'Mary thinks that John takes the Concorde plane' is represented by (italic represents comments):

[NTCG: ........ NTCG1 [WOMAN : Mary] ~-- (EXPR) ~ [THINK]

(PTNT)---> [NTCG: ........ NTCG2

[MAN : John] ~-- (AGNT) ~-- [TAKE] (OBJ) --* [PLANE : Concorde]

] 1. Earlier in this paper we indicated that in Sowa's approach we need to use a ~. abstraction to represent the CG corresponding to a subordinate proposition in the type field of a concept. This way of using ~. abstractions breaks the symmetry of the network that represents the conceptual graph. Let us examine the following two examples.

'Mary thinks of the Concorde plane that John takes' is represented in Sowa's model by:

[PROPOSITION: [WOMAN : Mary] ~ (EXPR) ~ [THINK] --->

(PTNT) --> [(). x) [ P L A N E : *x] ~ - ( O B J ) ~ - [TAKE]- -~

(AGNT)--~ [ M A N : J o h n ] :Concorde]].

and 'Mary thinks of John who takes the Concorde plane' is represented by:

[PROPOSITION: [WOMAN : Mary] ~ (EXPR) ~ [THINK]---,

(PTNT) [()~ x) [MAN : *x] <-- (AGNT) <-- [TAKE l

(OBJ)--> [PLANE: Concorde]: John]].

In both cases we have the same situation 'John takes the Concorde plane', but in order to specify precisely the object of 'Mary's thinking' we have to use two different ), abstractions.

In our model, instead of using ~, abstractions for representing these subordinate propositions, we allow the designer to mark a concept in the embedded NTCG, that we call an 'entry point'. The entry point indicates that the marked concept is the sink concept (resp. source concept) of the conceptual relation that points to (resp. from) the NTCG. The notion of entry point was introduced in Janas and Schwind z6 to represent 'extensional semantic networks'. In our approach of NL generation, the notion of entry point is very useful because it indicates the node from which starts the exploration of an embedded CG when mapping the conceptual graph to a sentence (see Sowa's proposal for NL generation 1 (p 230). In our model, when we describe a CG with the linear notation, a black circle is put after the concept marked by the entry point. From the technical perspective of manipulating graphs, entry points are processed more easily than ), abstractions.

In the preceding example 'Mary thinks that John takes the Concorde plane', there is no entry point because the object of 'Mary's thinking' is the whole situation 'John takes the Concorde plane'. But consider for instance the following example: 'Mary thinks of the Concorde plane that John takes' corresponds to:

[NTCG: [WOMAN : Mary] <-- (EXPR) <-- [THINK] --~

(PTNT) --> [NTCG:

[MAN : John] <-- (AGNT) <-- [TAKE] (OBJ) ---> [PLANE : Concorde]e

] 1. On the other hand, 'Mary thinks of John who takes the Concorde plane' corresponds to:

[NTCG: [WOMAN : Mary] <-- (EXPR) <-- [THINK] --~

(PTNT) ---> [NTCG:

[MAN : John]* <-- (AGNT) ~-- [TAKE] (OBJ)---> [PLANE : Concorde]

] ]. Note that in the preceding examples we specify situations as non-temporal CGs, even if the sentences that motivate our representations are expressed in the present tense. Our purpose here is to show that the operations that Sowa defined on CGs and embedded CGs can be applied on our NTCGs.


Situations

We consider in this approach that the world is described in terms of situations which are perceived or imagined by agents (this definition is equivalent to Sowa's definition).

An elementary situation is specified by an NTCG associated with a temporal label by a conceptual relation named temporarily 'period' (PER).

[NTCG: non-temporal CG] ~ (PER) ~ [unit, lwr- bnd, upp-bnd].

The lower-bound lwr-bnd of the temporal label corresponds to the point in time (with respect to a given reference time scale) when the agent first perceived (or imagined) the situation: The upper-bound upp-bnd of the temporal label corresponds to the last point in time when the agent perceived or imagined the situation. In the world which is associated with the agent perception (or imagination), the NTCG is associated with the truth value 'true' for any time point or time interval included in the interval [lwr-bnd, upp-bnd], and to the truth value 'false' otherwise.

Situations may be more complex. A compound situation can be the result of either the aggregation of situations or of the inclusion of situations in other situations; the embedded situations may be either elementary or compound ones.

When a compound situation is the result of the aggregation of two other situations it is specified by:

[SITUATION: [SITUATION: id-NTCG1] (log.rel)--~ [SITUATION: id-NTCG2]]

[SITUATION: id-NTCG1] ~ (PER) ~ [unitl, lwr-bndl, upp-bndl]

[SITUATION: id-NTCG2] ~ ( P E R ) ~ [unit2, lwr-bnd2, upp-bnd2]

where

• [SITUATION: id-NTCGI] represents an embedded s i t u a t i o n , i d - N T C G i represents the non-temporal CG associated with the situation;

• log-rel is a conceptual relation relating the embedded situations and can be instantiated by 'AND', 'OR' or 'COND' (expressing a condition equivalent to IF statel THEN state2);

• each embedded situation is associated with a temporal label [uniti, lwr-bndi, upp-bndi].

When a compound situation is the result of the inclusion of situations in other situations, it is specified by embedded NTCGs as described earlier, each NTCG being associated with a temporal label.

Since situations may be possible, necessary, allowed etc., we can attach a modality to an NTCG. We introduce a conceptual relation called 'MODAL- ITY', which relates the NTCG and a concept which represents the required modality type (POSSI- BILITY, NECESSITY, ALLOWANCE etc.). For instance 'John must take the plane Concorde' is represented by:

[NECESSITY] --~ (MODALITY) - . [NTCG : [MAN : John] *- (AGNT) ~-- [TAKE] ( O B J ) - . [PLANE : Concorde]].

This notation extends Sowa's specification of modalities using unary relations (see earlier).

Semantic interpretation of verb tenses

Let us remark that in the preceding representation we do not have any indication related to verb tenses. In order to specify verb tenses, we use temporal markers 2° considered as time intervals. In our model the temporal markers S, E and R correspond to the temporal labels [us, si, ss], [ue, ei, es] and [Ur, ri, rs].

The markers S, E and R are related to the non- temporal part of the conceptual graph (denoted 'NTCG') respectively by the conceptual relations MRK-S, MRK-E and MRK-R. MRK-E replaces the PER relation we introduced above. We can relate the temporal markers using our relations on temporal labels, which are compatible with Borillo's approach. It is worth remarking that these markers enable us to introduce pragmatic knowledge in the model since we take into account the context in which sentences are formulated.

For instance, Figure 4 represents the situation associated with the sentence 'Mary called John before the storm arrived'. Note that the units and bounds of the temporal labels are undefined and represented in the graph by variables.

The global situation (SITUATION G) corresponds to the locutor's perspective (associated with marker S1 = (ux, sa, s2) which is the same marker for both situations included in the global situation. The situation 'Mary called John' is represented by NTCG1 associated with markers E1 = (ul,e~,e2), R1 = (ul,r~,r2). The situation 'the storm arrived' is represented by NTCG2 associated with markers E3 = ( U l , e 3 , e 4 ) , R3 = ( u l , r 3 , r 4 ) .

The relation [El] ~ (BEFORE) ~ [E3] expresses the precedence order existing between the two situations.

Since both situations represented by NTCG1 and NTCG2 are considered as 'inseparable wholes '~5, we have the relations [El] -~ (EQUAL) --~ [R1] and [E3] --~ (EQUAL) ~ [R3].

Since both embedded situations take place in the past we have the relations [El] - . (BEFORE) - . [S1] and [E3] --~ (BEFORE) -~ [$1].

Consider now the global situation associated with the sentence 'Two days ago, Mary called John while the storm was blowing' (Figure 5). The temporal relation existing between the embedded situations 'Mary called John' (represented by NTCGI') and 'the storm was blowing' (represented by NTCG2') is different from the temporal relation relating NTCG1 and NTCG2 in the preceding example: the situation 'Mary called John' took place during the situation 'the storm was blowing'. Hence we can position the association temporal markers on the time axis in the following way:

e3, r3 , e l , = - r 1, e 2 , ~ - r 2, r 4, e 4, s 1, 8 2,

-[ [ [ ] ] ] [ - - ] , t

NTCGI' is now considered as an 'inseparable whole' and we have the relation [El'] ~ (EQUAL) [RI']. We position SITUATIONI' within SIT-


1 I o1,,1,,2 I

SITUATION G:

NTCG 1: L, PERSON: Mary I . G T r > - I cAL,

ul, el,e2 I ~l-~..: i i iEOUAL!i i~ ul, rl, r2

I NTCG 2: I

~ii~i~i~i~i!~ I ul, r3, r4 I

~iiiili M R K- R ii!i~ 7 i

STORM: # I ARRIVE I I

PERSON: John

Figure 4.

LEGEND

) conceptual relation

marker

temporal relation

state or validity interval

Representation of 'Mary called John before the storm arrived'

I I concept

UATION2', hence we have the relations [R3'] --+ (DURING)--+ [E3'], and [RI'] ~ (DURING)--* [R3']. Here we have indicated the relation [RI'] --+ (DURING) --+ [R3'] because we consider that the locutor's focus of attention which is first positioned on the SITUATIONI' by the marker RI ' , shifts to SITUATION2' with the second marker R3'. A more restricted representation would have introduced the relation [RI'] ~ (EQUAL) --+ [R3'], considering that the focus of attention is positioned on the situation associated with NTCG1.

In our model we consider that each situation is associated with a marker E and a marker R, and that in compound situations the relations existing between the markers R of the embedded situations indicate the shifts of the locutor's focus of attention. Our approach differs in that way from Borillo et al.'s modeP 8 (p28). These authors consider that in a sentence which is composed of a principal proposition and a subordinate proposition, the reference is settled by the subordinate proposition.

It is worth noting that temporal connectors and adverbs influence the semantic interpretation of the temporal information associated with the situations. In our example the adverb 'two days ago' indicates that there is a temporal distance of 'two days' between the locutor's time (marker S') and the event time of situation 'Mary called John' (marker El ' ) .

Hence the marker S' is represented by [day, el,, + 2, s21.

The study of the influence of adverbs and temporal connectors on the semantic representation of temporal knowledge in conceptual graph structures deserves further investigation. See, for instance, a study about the connector 'quand' ('when') by Borillo =v.

Temporal markers manipulation Temporal markers may be considered as 'special concepts' related to non-temporal CG by 'special conceptual relations' (MRK-i). Temporal markers are related together by temporal relations. From a graph manipulation perspective, the graphs that we obtain possess properties which are equivalent to standard CG ones. Hence the standard CG operations (join, pro- jection etc.) can be applied on our extended graphs.

In addition, special operations can be defined on the temporal portion of our graphs. For instance, we have developed an algorithm which is used for checking the consistency of temporal relations relating temporal markers. Inference rules of an appropriate temporal logic could be applied on temporal relations in order to reason on the temporal information embedded in conceptual graphs.


SITUATION G':

I NTCG 1': I' ' PE RSON: Mary I " I I ' G " ~ ' N " ~

day, e1'+2, s2' • u 1' , e 3 ' , e4 '

CALL PERSON: John

u l ' , r l ' , r2'

u 1', r3', r4'

NTCG2' ISTO.M I -CZ -- BLOWl

Figure 5. 'Two days ago, Mary called John when the storm was blowing'

CONCLUSION

We have evoked some difficulties that we encountered when we tried to represent temporal knowledge using the conceptual graph theory. We had to cope with these difficulties because in natural language it is necessary to consider time intervals as well as points in time, and because Sowa's monadic relations such as PAST and FUTURE do not convey enough information for representing properly verb tenses in sentences.

We have proposed to distinguish non-temporal knowledge from temporal knowledge in conceptual graphs. We specified temporal knowledge by means of time intervals and used temporal markers in order to indicate the relative positions of the locutor's time, reference time and the event time. These markers are useful for specifying semantically tenses and the aspectual properties of verbs.

The proposed model is far richer that we could present in this paper. Further research is needed, especially in the cases where we want to use conceptual graphs to model semantically the effect of temporal adverbs and conjunctions on the propositions they modify.

ACKNOWLEDGEMENTS

This research is partially supported by the Natural Sciences and Engineering Research Council of Canada (Grant A2218).

REFERENCES

1 Sowa, J F Conceptual Structures: Information Pro- cessing in Mind and Machine Addison-Wesley (1984)

2 Lizotte, M and Moulin, B 'A temporal planner for modelling autonomous agents' in Proc. European

Workshop on Modelling Autonomous Agents in a Multi-Agent World London, UK (August 1989)

3 Lizotte, M and Moulin, B 'SAIRVO: a planning system which implements the actem concept' Knowl.-Based Syst. Vol 2 No 4 (1989) pp 210-218

4 Danlos, L 'Un survol des recherches en g6n6ration automatique' in Revue qu~b~coise de linguistique Universit6 du Qu6bec ~ Montr6al, Vol 14 No 2 (1985)

5 MeKeown, K R Text generation: Using discourse strategies and focus constraints to generate natural language text Cambridge University Press (1985)

6 McDonald, D D 'Natural-language generation' in Shapiro, S C (ed) Encyclopedia of Artificial Intelli- gence (vol 1) John Wiley (1987) pp 642-655

7 Velardi, P, Pazienza, M T and De' Giovanetti, M 'Conceptual graphs for the analysis and generation of sentences' IBM J. Res. Devel. Vol 32 No 2 (1988) pp 251-267

8 Sowa, J F 'Notes on conceptual graphs' IBM Tech- nical Paper (November 1986)

9 Fillmore, C H 'The case for case' in Bach, Harms (eds) Universals in Linguistic Theory Holt, Rinehart and Winston, New York, USA (1968)

10 Sowa, J F 'There's more to logic than the predicate calculus' in Carbonell, J and Fuechi, K (eds) Proc. US-Japan AI Symposium Tokyo, Japan (December 1987)

11 Sowa, J F 'Conceptual graph notation' in Esch, J W (ed) Proc. Fourth Annual Workshop on Conceptual Graphs St Paul, MN, USA (August 1988)

12 Sowa, J F 'Towards the expressive power of natural language' in Gerholz, L and Eidund, P (eds) Proc. Sixth Annual Workshop on Conceptual Graphs Detroit, Michigan, USA (July 1990)

13 Allen, J F 'Maintaining knowledge about temporal intervals' Commun. ACM Vol 26 No 11 (1983)


14 Comrie, B Tense and Aspects Cambridge University Press (1985)

15 Dorfmuller-Karpusa, K 'Temporal and aspectual relations as text-constitutive elements' in Pet0fi, J S (ed) Text and Discourse Constitution Walter de Gruyter (1988)

16 Gaiton, A (ed) Temporal Logics and their Applica- tions Academic Press (1987)

17 Vendler, Z 'Verbs and times' in Vendler, Z (ed) Linguistics and Philosophy Ithaca, Cornell Univer- sity Press (1967)

18 Borillo, A, BoriHo, M and Bras, M 'Une approche cognitive du raisonnement temporel' in Teknea (ed) Actes des journ~es nationales en intelligence artifi- cielle Toulouse (March 1988)

19 Ehrich, V 'The generation of tense' in Kefrnpfen, G (ed) Natural Language Generation NATO Advanced Institute Series, Martinus Nijhoff (1987)

20 Reichenbach, H Elements of Symbolic Logic McMillan, New York, USA (1947)

21 Kwong, C 'Representing time' in Ringland and Duce (eds) Approaches to Knowledge Representa- tion: An Introduction John Wiley (1988)

22 McDermott, D 'A temporal logic for reasoning about processes and plans' Cognitive Sci. No 6 (1982) pp 101-155

23 Moulin, B 'Le concept d'actem pour la gestion de sc~nes' in Proc. GIRICO'87, L'intelligence arttfi- cielle au service des organisations Montr6al, Canada (June 1987)

24 Kahn, K and Gorry, G A 'Mechanizing temporal knowledge' Artif. Intell. No 9 (1977) pp 87-108

25 Moulin, B and C6t6, D 'Extending the conceptual graph model for differentiating temporal and non- temporal knowledge' in Proc. Conceptual Graph

Workshop, AAAI Conf. Boston, MA, USA (July 1990)

26 Janas, J M and Schwind, C B 'Extensional semantic networks: their representation, application and generation' in Findler (ed) Associative Networks Academic Press (1979)

27 Borillo, A 'Quelques remarques sur "quand" con- necteur temporel' in Larousse (ed) Langue Fran- faise No 77 Paris (1986)

BIBLIOGRAPHY

Moulin, B and Kabbaj, A 'Architecture de SMGC: un syst~me de manipulation de graphes conceptuels' in Proc. Int. Informatique et Langue Naturelle Nantes, France (October 1988)

Moulin, B and Kabbaj, A 'SMGC: a tool for conceptual graphs processing' Commun. Cognition-Artif. Intell. (June 1990)

APPENDIX 1 Allen's temporal relations

Illustration X: _ _ . Y :

Relations Symmetric relations , t

X BEFORE Y Y AFTER X X EQUAL Y Y EQUAL X X MEETS Y Y MET-BY X X OVERLAPS Y Y OVERLAPPED-BY X X DURING Y Y CONTAINS X X STARTS Y Y STARTED-BY X X FINISHES Y Y FINISHED-BY X


Documents

Representing temporal knowledge in conceptual graphs