Chapter 8

CASE STUDY AND EVALUATION

Case studies give the concrete proof of realizing the hypothesis which had initiated

the research work. They help to illustrate the application of the solutions that have

been proposed for realizing the various aspects of the hypothesis.

This chapter describes two case studies illustrating the application of the models that

have been progressively developed for reallzing agents with language ability. The first

case study exactly maps the proposed models onto a MULtilingual natural Language

Agent Interface for mail service (MULLAI), where the language ability is attributed

for a single agent. The second case study describes how the proposed architecture has

been extended to adapt the same to realize the language ability of a multi-agent

system (Multil~ngual Multi-agent System (MMAS). Though, the language ability of

multi-agent system IS not within the scope o f t h ~ s work, yet it has been described here

in order to illustrate that the model applies for an MAS also with small extensions. In

addition, evaluation of an agent with two-dimensional autonomy and the existing

multilingual d~alogue agents has also been carried out.

The objectives of this chapter are to

Illustrate the application of the proposed solutions in realizing the language

faculty for a

- Single Agent - MULLAI

- Multi-Agent System - MMAS.

s Evaluate the agent with two-dimensional language autonomy and the existing

multilingual dialoguing agents.

8.1 MULLAI -Multilingual Natural Language &gent Lnterface for Mail Service

Providing a natural language interface to the mail service operations would help to

bridge the language barrier that arises when mail services are to be used by people

who are familiar in their natural language only. A conventional form-based GUI

means of interaction may not be convenient for these users as they would prefer

interacting with the system in the same way as they interact with people. Hence, a

collaborative form of natural language interface that interacts with the user in a

comprehensive manner so as to infer his intentions from his abstract task specification

and carry out the task is required. This interface may have to support many languages,

especially when it has to be used in language diversified country like India. The

prospect of having to augment the language ability to new languages is also high.

These requirements instigate the need for a multilingual natural language agent

interface for mail service (MULLAI). These requirements have also been the subject

ofthe thesis for which suitable solutions have been provided in the earlier chapters.

Hence, these solutions have been used to realize MULLAI. Thereby, it contributes as

a test bed for analyzing the appropriateness of the proposed solutions towards

realizing MULLAI. The natural language interface has been provided for mail service

operations like

List

Read

Delete

Compose

Reply

Save.

Multilingualism has been realized using two languages - English and Tamil. Of these

languages, the system was initially built with support for English language only. The

Tamil language has been acquired and implemented in the system. The Surgemail has

been used as the back end mail server. In the subsequent discussion, how the various

solutions described in the previous chapters have been used in realizing the agent is

described. Especially, the case-study would explicit how the management functions

are accomplished with respect to their manager and behavior roles. In MULLAI, the

manager role gains predominance in certain management functions and in others the

behavior roles gain predominance. For example, the manager role is of significant

importance in the case of the knowledge management and the behavior role is of

importance in new behavior acquisition management. Dynamic role binding to the

required language behavior is of importance to the Interface and Function

Management. Hence, the description of the case-study is done by considering the

management functions with respect to their predominant role.

8.1.1 Manager Role - Knowledge Behavior Management

Here, the various responsibilities of the Manager role are considered and how these

are fulfilled in the above management functions are described. The sample screen

shots help to illustrate the accomplishment of the same in MULLAI.

The varlous responsibilities of the manager role are.

Communicating with other management functlons

Maintaming Task Context

Assume the requ~red behav~or role

Ma~ntaining Behavior References

Perform Task Level Inference

Implement newly acqu~red behaviors.

Communicat~ng wlth other management functions happens using send() and receive()

functlons as illustrated in the design and implementation model in the previous

chapter.

The task context of Knowledge Behav~or Management is constituted of the following

beliefs as described in the BTB paradigm:

(Bel TCk, language knowledge (Ik, DO)) +(Be1 TCk, language resources(lk ))

A (Bel TCA, , vocabulary(l~. DO))

"(Bel TCi, grammar rules(ii, DO))

A (Bel TCk,, knowledgeserv~ces(Ik))

The Language resources correspond to those required to facilitate input and output.

For example, in order to facilitate input output in Tamil, resource files corresponding

to font file (Bamini Font), coding scheme mapping file between Ascii to Tascii for

perception and vice-versa for response are required. Tascii is a character-based coding

scheme which is used to represent the characters of the languages supported,

internally. The Tascii codes used for Tamil language are given in Appendix A. The

keyboard layout is air0 given in the appendix A. The following example illustrates the

need for the mapping.

Typed keys

Ascii value in English Fonts

Corresponding character in Tamil : 61csn

Tascii Code for character Q 6 n , 2 9 5

Ascii value for the glyphs '61' 's' 'n' in Bamini Font : 78 70 72

That is, when the user types the characters 'k' .o' in the keyboard, it corresponds to

the ascii values 75 and 79. This should be interpreted internally as character '61en'

with Tascii code as '295'. In Bamini font which is used as the font file for Tamil, the

glyphs '61' '16' .r' are mapped to characters 'n' 'f 'h'. Hence. in order to display the

glyphs corresponding to '61csn' using the Bam~ni font, the ascii values 78 70 and 72

are required. Therefore, during input 75 and 79 are mapped to 295 (Ascii to Tascii).

To display the same character, 295 IS mapped to glyph codes 78, 70 and 72 (Tascii to

ASCII)

In MULLAI, a relational knowledge representation is used. The vocabulary is given

in terms of tables for each of the word types - noun, verbs, adjectives, pronouns etc.

These tables contain the word and the suffix code. The suffix code ~ndicates what

~nflect~on has been carried out on the word and what that ~nflectlon is. For example,

for nouns, the inflection may be with respect to gender, number, etc. In order to keep

track of the tables that contaln the various word types, another metadata table as

shown below in Table 8. i is maintained.

Table 8.1: Metadata of language details

tam11 ,-- I%- - sentence structure~

The grammar indicates the rules to be followed in the formation of words, inflection

of words, formation of phrases, formation of sentences etc. For example in Tamil,

noun phrases can be formed as follows:

np: noun ( pronoun

np: adj ; np

The following patterns are used in the formation of a sentence consisting of subject S.

verb V, object 0 1 and object 0 2 , adjective A

S v S O V

O S V

S 0 1 0 2 V

0 1 0 2 S V

0 2 0 1 S V

S O A V

O S A V

The metadata about the files containing the lexicon and grammar rules are maintained

separately so that they can be referenced and updated whenever required. The

follow~ng table in Table 8.2 glves the path details about where the knowledge files

corresponding to the varlous languages are stored.

Table 8.2: The path details of languages supported

The knowledge services that are to be rendered by the knowledge manager role are.

Servicing knowledge requests

Update of knowledge

Organlzlng and storing knowledge

The tirct servlce 15 e x p l ~ c ~ t l y not visible as 11 1s belng used b? the naturdl 1angudi.r

pr(1ces\lng hehav1or5 lor proceas~ng a naturdl language request The screen shot In

I;~gure X I shows k ~ i o ~ l e d g e update The update IS belng iarrled out lor noun

c'ircgor! uf'\\ord In the table with name Noun I The other command buttons pro\ idc

opl~onb l i)r modif)lng character?. phrases and sentences

I - L - u s . -.. .. . .. . . dlb B

L - W I I I I M ICI

i - ~~-

I I

WIlCtl U , . L I Y !

. I- .. .LJ 1 1 " ' - 1 a.rlj;. ---- ...-- ... Y P l l , , -I--

I !

m-rra . ; , irra.npa

-,--

Figure: 8.1: Interface for update of language konalcdge

I LII orgdnl/lng and \tclrlng k~io\\ledge. thc knowlcdpc rilaliagrr hclps to crcnte

\cp.ir,itc J ~ r e ~ l c ~ r ~ c a lor \toilng all files pertdlnlng to thdt laiigu.ige I: uI\L) help, to

ire;ltc thc rcqu~rcd tnbic structures fc>r stor~ng thc hriin\lcdge In relationdl

replc~cnt,ltluli du r l~ ig Idnpii.igr azqulsltlon This 15 deplited 111 I'lgure X ?

I-he w e e n sliot dep~cts I i < ~ u the hnouledgr englnccl createa .I inhie Sor htorlng \crb

dctail5 I'hc table IS irearcd ~1111 fields word, sul'li\ code dnd root I hc t?pe o f tliche

lields can also be specllied I he iirlnmand buttons. ddd dr rcino\e ficldr hclp ro

pc l fo r~ i i the iorrc>pund~ng operauonr In the table U hen the t'lhlc ~ ~ c a r l o n I\ r , \ c ~ tlir

b u ~ l d idblc hclps l u create a tablc In the bdch-end database \\ 1111 the \pec~tied liclds

and t lpea Thus. the knowledge manager helps to orgdnlze '111d \tore hrio\\ledge

Ftgurc: 8.2: Interface for creating tables for storlng language knowledge

I lie t.i\h LL>I~IL\I 0 1 111c \en Behd\i<)r . ~ C ~ U I \ I ~ I L > I ~ ~ I d ~ i ~ ~ g e i ~ i c ~ ~ ~ lid, i ~ i t e i n ~ i l bcl~ci'\

pzrl.ilriilig r i , t l ~ c td\h orltvlu_c and lrlngudgc ontolog! 'I he IL~ i lg i i r l g~ ~oiiti~log! I 0

< I \ c \ deta~l \ , rh i ) i~~ the rc\oiircc,. hnmrledye m d io i~ lpe tc~ lc ia hi acqiiiic 111 ordc~ to

t c . ~ l i / ~ J i ~ c \ \ I.~nyiiayc iii tlic dycnt ;ind thc i i inct~vnal dt,iiiaiii i>ill\>log! DO L O I ~ ~ I , I I ~ %

t l i ~ di,in.~iti o l Ilir 1.1iiguagc i1ch11l\ LO he r l ~ q i ~ i r c d Thia 15 ~cprc\e i~ ied JS.

( 1 3 ~ 1 IC , , , L O D O )

ll tlic lrilig~l.iye JL~~II~I~IIIII tdbh perII>rmb ~ C L I U I \ I ~ I O I ~ ~ > f rl inti\ IJI~~LI.I~C -I,,,,\' rhcil 11

ai~gnicnla the hellel set o f rlic language tocult! \ \ ~ t l i bs l ic t i ill the nc\\ Ia~igudpu I Ilia

I\ rcp~caentcd ah lo l lo \ rJ

o(Uel I lu r igu~gr (I,,,,, 110))

1 hcrc I,,,,, 15 thc nr\\ language dcyu~red

In order to ach~eve this. thc path deta~la of the nc\\ I! ncqulred Idngiinge I S appended in

the 1,111guogc dccrlils t.~bic. and tlic table details rablc IS upda~cd n ~ t h tiic n.irnr o l the

I'ingudgc and the ct~ric\pundlng tables name:. thdt colitdln I!> I,inguogr. hno\\ledge

resuulcca 'l'h~a la dwne \r it11 thc help ol'ttlc. lrliigi~rlgc h i l ~ l ~ ledye 11li1ilrlg~1

I he acrcen \hot In Flgure 8 3 lndlcates the Inltlatlon of the acqulrltlon hehavlur ~ o l e

~1 iTd1 i i l l language

i

I

I I 1 I I I

F l g ~ l r c 8.3: Acquiring new language ('Tamil)

I lic ,ILL]LII~I~RII~ p1~11ci~ h ~ g l n b \%ith the acqulsitron i i t ih,ii,iitcri h! ~iht.11n111; tile

IIIII~I~ICI (>I L l i G ~ ~ c ~ L t c i t!pe> 1Iie11 lia1nca slid I~IC ~ i u ~ i i b e r o I ' ~ l i ' ~ ~ ~ i ~ t e r \ 111 edcll 1)pe

l l icn. 11 \ I d r r ohtnlnlng the Ion1 f i le, thc '1aac11 code dnd tlic ASCII code till the

.I~~r, lcter\ I hen. 11 ohtdln, the I y p h code\ ~orrespondlng ta thc fbnt file that ale ti)

hc used tor d ~ ~ p l a ! 11ig 111c clidractrr glbpha Attcr th~s. the lo< 5 l i i c that I\ requircd to

I ~ ~ i i l ~ l a t e 11ipu1 and i ~ u t p u l b! convening hr thcen dscl! ILI t ax i1 and 1d\c11 tu a a ~ l i 15

~ ~ q u l r c d Th15 IOCS la a i laas l i le that la d!narnlcall! ~ n c l ~ l d c d b! nslnp d!llamli

i l a s loading Onl! 11 this IOCS class t i lc ia Included d > n a i n ~ ~ a l l ? . thc suhsrqucnt

\rurd acqulaltlun and aenIenct. acqulsltlon IS pi~sslhlc hrcdux! nnl! 1h1$ ICKY cdn

1 ~ ~ 1 l l t s t c Input and uutput 111 the correspondlny Idnguape tu cnter tl>c \rord and

icntcricc detalla l'heae are dep~cted In F ~ g u r c 8 4

' i l l l , " .n l i , , t . a,,",",

r d , 1

Ulsr l l l l l " . l~ ,nu 2

UI "l '$6 1

M1" n,, vwx, 1

.I..l.lu rr.I,,,IIIIIIII, 258

'"I illbIIa11 DwIDI.tN.cmIIo..ol...

MI1, I I I I ' I l l , . lSY<I< . WIO/.tN..IY.mlll.~,,~m.'!nfnfr,...

F ~ g u r e 8.4: Acquiring basrc language details

I c\cr! t!pr <)I clidrd~ler and for all the characlera In ever! hpe . tlic aacli code u icd

lur rcprc\enllllg l!ic ii1.1rdcter on the Lrybodrd. the t d \ i i i cridr v,!i~cli 15 u,cd lo

rcprc\c!ll the ihnr,tiler ~ntrrnall) a n d thr glypll code\ wed tor d~ydaying itir

L ~ ~ . I I J L I C I I !pI>\ Jrc : l i i l ~ ~ ~ r e d l h ~ \ 15 ileplcted in rigilrc X 5

Figure 8.5: Character Coding Details Acquiclhoo

1 his IS followed by word detail\ acquisltlon Here. the number of typcs o f words dnd

t h e ~ r name\ are ohtamed ~ n l t ~ a l l ? Then for each word type. a table u ~ t h the r rq i i~red

Wucture 15 crcated 'Then the table d c t a ~ l s are entered This 15 c d r r ~ e d out for all the

k o r d typcs r i g u r e 8 6 shows an cvample of 'word detdll acqulaltlon

F,gurc 8.0: Word Dctails acquisit~on

l l ic \dr luus plird>,il struziurer and scntence structure\ a l c also ohtdincd Tlicsc

~ i r r r c s p ~ r n d to tile \ d i ~ o i i a pdrreriis In \ \hlili tlirhe htructurci i o ~ i ?\)at iii tile

i o r r c \ p o n d ~ n g Iangudge 1 hi> 15 depicied in t lgures 8 7 and X t;

I , I j 1 I I

I I F ~ g u r e 8.7: Phrase details acqulsltloll

Figure 8.8: Sentence details acquisition

Subsequent to thls. the statlc sentences that correspond to the standard rrsponrc

Inessages lo he ionbeyed to the user are acqu~red This 1s because, for standard

rerpon\es. thc language generation process need not be carrled out unnecessarll) and

the acquired nle\sdge\ could he readll? con\,eyed

8.1.3 Dynamic Role Binding - Function Behavior Management a n d Interface Behavior %lanagenlent

In thc case o l funitlon hrhe\lar and imerhce heha\lor nianageinent. the manager role

d)nd~nliall! bind\ ~ t \ c l f to thc requlred language beha\lor role b! \%ea\'lng the

requ~red language aspect T h ~ s I S ~llustratcd In the ld lou iny screenshot5 In Figures

8 Y and 8 10 \%here the ~ntert'ace and competence funcllons \ \ caw the Tamll language

n\pect to pro\. ~ d e Interaction In Tam11 language

The s~recnahotb In the tigures ~llustrate how the reaearch contrlbutlons have been

practically real17ed In MlJLLAl

8.2. Multilingual Multi-Agent System

For applylng the language faculty models for a multi-agent system, the architecture

needs to be extended The need for thls extension, the extended 117odel and thc

appllcatlon of the model for a mult~l~ngual multi-agent system are explained in the

following d~scusslons

Figurc 8.9: Interface Behawor Management weaves w ~ t h 'I an111 language aspect

F ~ g u r e 8.10: F u r ~ c t ~ o r ~ Behavior Managenlent weakes Tan111 language processing Aspect

8.2.1 Extending the Behav~or hlanagcment Architecture for hlulh-agent Systenl

A Multi-agcnt a!stclll (MAS) 15 one in ~ h ~ c h tuurc than LIIIC dycnt e\ists dnd e\L.t!

agem possesses d 5pecilic tash e\penl&e The) arc l~aed ti>r \ul\ 1111. i.r>mplc\ problcin~

which rcqulrc the conlr~hut~on of mult~ple tash c\prrtlrr I'hc dyentr 111 M/\S

cooperate or negotlatc ~ ~ t h each other to solve the problem

The language ability of MAS should help to cater to the interaction requirements of

all the agents present in the MAS. That is, the language ability should suppon the

domain ontology of every agent. Mapping this requirement to the defined architecture

components, the function of every management function is as given in Table 8.3.

The table shows that the responsibility of each of the above management functions

become more as they have to manage additional behaviors and their complexity also

increases. Hence, each of the language behavior management components could be

realized as a separate agent with the corresponding behaviors under each of them.

Thus. language handling in MAS becomes a dtstributed problem solv~ng activity,

which is achieved by the cooperation ofthe behavior management agents.

Table 8.3: Functions of the Management components in the context of MAS

/ Behavior Management Components

Interface Behavior Management

The architecture which is depicted in Figure 8.1 1 consists of two layers - the

functional agent layer and language faculty agent layer. The language faculty agent

layer consists of the agents corresponding to the various management functions of the

language faculty. Except being divided into individual agents, the functions of each of

these management agents are similar to the functions defined for these components in

the architecture described for the language faculty of a single agent.

Functions

This component is not affected by the presence of multiple agents in the MAS and

Function Behavior Management

Knowledge Behavior Management

New Behavior Acquisition Management

The functional agent layer consists of the various functional agents of the MAS. There

is a facilitator agent which takes care of registering and deregistering of the funct~onal

remains the same. Help to comprehend a given request which is expressed in any of the supported languages and map to the appropriate domain ontology. . Manage the language knowledge and resources corresponding to all the task agents in all the supported languages. 8 Acquire the language resources in all the

supported languages corresponding to the task agents which are registered. Acquire new language knowledge.

agents in the MAS. Whenever a new agent registers with the facilitator, it registers its

domain ontology also with the facilitator. The facilitator immediately initiates new

behavior acquisition management agent to acquire the language resources pertaining

to this domain ontology in all the languages supported. When a functional agent

deregisters from the functional agent layer, the facilitator informs the behavior

knowledge and the behavior competence management agents to dismount the

language knowledge pertaining to this agent in all the languages.

1

Knowledge Competence New Behavlor Behav~or Behav~or

Management Management Acquls~t~on Agent Agent Management

Agent

Intellace Language Behavlor Faculty

Management Agent Agent Layer

Percept I Response

Figure. 8.11: Extended Behavior Management Architecture Model for the language Ability of Multi-Agent System

The request for the task to be performed by the MAS is received through the Interface

Behavior Management Agent. It transfers it to the Function Behavior Management

Agent which processes it and converts it to an intermediate structure. This

intermediate structure is compared with the domain ontologies of the various

functional agents that are registered with the facilitator to determine to which of the

agent the task is delegated. This conversion to intermediate structure and determining

the appropriate functional agent are the additional functions required to be managed

by the behavior competence management. After this determination, the request is sent

to the corresponding functional agent through the facilitator. This request may then be

executed by the functional agent either independently or as a coordinated activity

among the various agents of the MAS.

When a new language has to be acquired, the New Behavior Acquisition Management

agent configures to the language acquisit~on role and uses the registered domain

ontologies of the functional agents and the generic language ontology to acquire the

language knowledge and resources corresponding to all the domains.

The main advantage of this model is that it is able to provide the language ability of a

language faculty to a multi-agent system also. But, the limitation is that this model in

its current state is most suitable for closed multi-agent systems or open multi-agent

systems with minimum number of agents.

8.2.2 MMAS - Multillnpd Multi-agent System

This case study describes how the above model is applied for developing a

Multilingual MAS. The multi-agent system was developed with two functional agents

where one of the agents is an e-mail service agent and the other agent is a file service

agent. The e-mail service agent makes use of the file service agent to store or retrieve

mails. If any file is to be e-mailed, the file service agent contacts the e-mail service

agent. The MAS supports two languages Tamil and English.

The screen shots below indicate the operations of the MMAS. The screen shots in

Figures 8.12 and 8.13 indicate the registration of the functional agents (Email service

agent and File service agent) along with their functional domain ontology with the

facilitator.

I'rgurc. 8 I?. Rep~stratton of >]ail Agent wt t l~ tlie fi~cilrtator

Figure 8.13: Regrstrstrori of File Ser\rcc Agent arth tlir facrl~tt~tor

Figure. 8.1 4: Ernail Service Operation ru Tamil

,,,= ",,. *,,, ,N*, . >.n,-. **

","I*.>< .A",

Figure 8.15: File Ser~rce Operatiou ~n English

8.3 Eva1u;ltion

I h e case studies dsscr~bcd a b o ~ c help to proke the vai~d~t! ol the miidels dc.\elc~ped

In order 10 ' A ~ I I C I I the advdlltagcs 01 the agent k\~th t\\o-d~mens~un;rl languagc

autonoln?, an evaluatli~n o f t h c dgent \ \ ~ t h the ehlstlng ~nult i l~ngual dlnlogulng dgent,

(MDAa) 1s carried out Among the varlour tbpea ofdgcni, ldent~tied In thc ld\oiiom!

prusrnled In the Ilterature revlelr, onl) the MDAs are churrn becaurc 0111) t11e.c

agents fulfill the b a r ~ c language d b ~ l ~ t ! rcqulrcllienl~ of dl1 dgent \+h~cli linvu ~ C C I I

ldentrfied aa CNLl and DM II I t h~h thearr ,411 tlir MDAs are ionaldrrrd .I\ a s ~ n g l r

category irrespective of the technique used for achieving multilingualism. This is to

provide a clear representation of the advantages of the existing MDAs with respect to

an agent with twodimensional language autonomy.

In order to perform the evaluation, the criteria for evaluation have to be identified

first. Also, the mode of assessment has to be identified. The following are the

evaluation criteria considered:

Language functionality properties for CNLI

Language functionality properties for DM

Internal State Language Properties

Language Management Properties

Language Management Architectural Properties

Multiple Language Handling Complexity Properties

The above set of criteria is a consolidation of the requirements or properties against

which the conceptual, internal state, architecture, design and implementation models

have been developed in this thesis. The assessment is represented quantitatively on a

scale of 0 to 1. The values are awarded against the properties based on the extent to

which the corresponding property is fulfilled.

Table 8.4 shows the evaluation of the agents. The awarded values indicate whether

the corresponding property is fulfilled completely (I), not considered at all in the

agent (0) or is partially fulfilled (0.1 to 0.9). The values of the existing MDAs are

found to be less except for the properties pertaining to CNLI. The reasons for the

reduction of values are given in the table as comments. The values for the agent with

twodimensional autonomy are given based on the results of analysis discussed at the

end of each of the chapters. In these analysis it has already been proved that the

properties arc fulfilled and hence the value of 1 is given. Finally, using these values, a

graph as shown in figure 8.16 is drawn to show the overall evaluation results. From

the evaluation it is clear that an agent with two-dimensional language autonomy not

only fulfills the properties / requirements, but also excels the existing MDAs in many

ways.

- The dynamic configuration to the required language should have minimum latency.

-- -

languages and the task performed by the agent.

Dy..mieily Should help to support multiple languages and dynamically configure to the required language. Perlorrmna - Should pmvide for a performance

which is comparable to that of an agent supporting a single language.

I

I

1

(There is no Language Management Architecture

available)

Scalability Should enable to acquire more languages. M.inWnabUity Should enable to update and maintain the language knowledge and competence. Reuubility The architecture should be suitable not only for language faculty, hut also for any other task faculty of the agent.

I

1

I

8.4 Summary

The descrited case studies and the screen shots provide illustrations of the application

of the solutions proposed for realizing the language faculty of an agent as well as a

multi-agent s9stem. The extended architecture model shows how the same behav~or

management architecture could be adapted for MAS to &Ifill its language

requirements.

The contributions of t h~s chapter are:

Proof of hypothesis proposed in problem statement.

Proof that the proposed solutions are suitable even when hypothesis is

extended to accommodate the language requirements of MAS.

Evaluation of an agent with two-dimensional language autonomy with existing

multilingual dialoguing agents.