3D Electronic Institutions eMarkets Research Group Helmut Berger

3D Electronic Institutions3D Electronic Institutions

eMarkets Research GroupeMarkets Research GroupHelmut BergerHelmut Berger

22

OutlineOutline

• The VisionThe Vision• ObjectivesObjectives• Electronic InstitutionsElectronic Institutions• 3D Virtual Worlds 3D Virtual Worlds • A Framework for 3D Electronic A Framework for 3D Electronic

InstitutionsInstitutions• ImplementationImplementation• Conclusion & Future WorkConclusion & Future Work

33

The VisionThe Vision

44

ObjectivesObjectives

• New Metaphor: New Metaphor: 3D Electronic 3D Electronic InstitutionsInstitutions

• Establish TrustEstablish Trust• Interact within a Social EnvironmentInteract within a Social Environment• Participants are Participants are

– Human UsersHuman Users– Autonomous Software AgentsAutonomous Software Agents

55

ObjectivesObjectives

• Reliability and SecurityReliability and Security• Immersive 3D ExperienceImmersive 3D Experience• Identify an Application DomainIdentify an Application Domain• Information-rich EnvironmentInformation-rich Environment• LearningLearning

66

TrustTrust

1.1. Trust is the assured reliance on theTrust is the assured reliance on the character, ability, strength, or truth ofcharacter, ability, strength, or truth of someone or something.someone or something.

2.2. One in which confidenceOne in which confidenceis placed.is placed.

(Merriam Webster‘s Collegiate Dictionary)(Merriam Webster‘s Collegiate Dictionary)

77

TrustTrust

• One of the most important social concepts One of the most important social concepts • Helps humans to cope with their social Helps humans to cope with their social

environmentenvironment• Is present in all human interaction Is present in all human interaction • Implicit Implicit trust assessmenttrust assessment underlies every underlies every

traditional bargaintraditional bargain• Drawback in electronic markets: face-to-Drawback in electronic markets: face-to-

face interaction is missingface interaction is missing• Increased risk level related to online Increased risk level related to online

marketsmarkets

88

Social EnvironmentsSocial Environments

• Social Interaction is the key featureSocial Interaction is the key feature• ConversationsConversations• Virtual Worlds are spaces wereVirtual Worlds are spaces were

people people meetmeet• Strong relation to TrustStrong relation to Trust

““Whom do you know?“Whom do you know?“

99

Electronic InstitutionsElectronic Institutions

• A A FrameworkFramework and and MethodologyMethodology developed by a Research Group led by developed by a Research Group led by Prof. Carles Sierra at the Artificial Prof. Carles Sierra at the Artificial Intelligence Research Institute (IIIA), Intelligence Research Institute (IIIA), Barcelona, SpainBarcelona, Spain

• Electronic Institutions are „regulated“ Electronic Institutions are „regulated“ Virtual Environments which allow Virtual Environments which allow Agents to Agents to interactinteract autonomously autonomously

1010


• Natural extension of the Natural extension of the social conceptsocial concept of institutions as regulatory systems of institutions as regulatory systems shaping human interactions shaping human interactions

• Software systems inhabited by Software systems inhabited by autonomous entities = software agentsautonomous entities = software agents

• Agents interact according to predefined Agents interact according to predefined conventions on conventions on languagelanguage and and protocolprotocol

• Conventions guarantee that certain Conventions guarantee that certain normsnorms of behaviour are enforced of behaviour are enforced

1111


• Conventions Conventions – on Languageon Language = = Dialogical Dialogical

FrameworkFramework– on Activitieson Activities = = Performative Performative

StructureStructure– oon Behaviourn Behaviour = = NormsNorms

1212

Dialogical FrameworkDialogical Framework

• Language (Ontology)Language (Ontology)• Illocutionary ParticlesIllocutionary Particles

– Theory of Speech ActsTheory of Speech Acts– Particles are Particles are inform, failureinform, failure or or requestrequest

• Organizational Structure of the Agent Organizational Structure of the Agent SocietySociety

• RolesRoles– External Vs. InternalExternal Vs. Internal– IncompatibilitiesIncompatibilities– RelationshipsRelationships

1313

Dialogical FrameworkDialogical Framework

OntologyOntology

(start_double_auction: -> boolean)(start_double_auction: -> boolean)

(offer:int*float -> boolean)(offer:int*float -> boolean)

(demand:int*float -> boolean)(demand:int*float -> boolean)

(want_to_buy:String -> boolean)(want_to_buy:String -> boolean)

(want_to_sell:String -> boolean)(want_to_sell:String -> boolean)

(close_meeting: -> boolean)(close_meeting: -> boolean)

(close_wait: -> boolean)(close_wait: -> boolean)

Role Role HierarchyHierarchy

1414

Performative StructurePerformative Structure

• Defines the Defines the DialoguesDialogues in which Agents in which Agents might participatemight participate

• Each Each ActivityActivity an Agent performs is an Agent performs is associated with a Dialogue among the associated with a Dialogue among the group of agents involved in this group of agents involved in this ActivityActivity

• Dialogues are referred to as Dialogues are referred to as ScenesScenes

1515


• Fixes the Fixes the ProtocolProtocol (possible dialogues) (possible dialogues) which might be enacted in each Scenewhich might be enacted in each Scene

• Defines the applicable Defines the applicable Sub-LanguageSub-Language of a particular Sceneof a particular Scene

• Minimum and Maximum number of Minimum and Maximum number of participantsparticipants

1616


• Agents transit is regulated by special Agents transit is regulated by special scenes referred to as scenes referred to as TransitionsTransitions

• Transitions are used toTransitions are used to rre-routee-route agents agents

• Agents areAgents are synchronized synchronized via different via differenttypestypes of transitions of transitions

1717

Performative Structure: An ExamplePerformative Structure: An Example

1818


• Behind the ScenesBehind the Scenes– A A Finite-State MachineFinite-State Machine (FSM) regulates the (FSM) regulates the

Flow of AgentsFlow of Agents within particular scenes within particular scenes– The FSM contains special states defining The FSM contains special states defining

possible possible EnterEnter and and ExitExit points for agents points for agents– ArcsArcs between states are labelled with between states are labelled with

illocutionary patternsillocutionary patterns– Uttered illocutions change the state of the Uttered illocutions change the state of the

FSMFSM

1919

FSMFSM:: Behind the ScenesBehind the Scenes

Example Illocution:

(W0 W1(inform (?s seller) (?t tradeMgr) offer(?qty ?price)) (?qty > 0 ))

2020

NormsNorms

• NormsNorms impose restrictions on actions impose restrictions on actions agents are allowed to perform in scenesagents are allowed to perform in scenes

• Agents acquire Agents acquire CommitmentsCommitments / / ObligationsObligations within an Electronic within an Electronic Institution while talkingInstitution while talking

• CommitmentsCommitments / / ObligationsObligations restrict restrict future actions of the agentfuture actions of the agent

2121

3D Virtual Worlds3D Virtual Worlds

2222


• Space is designed and arranged Space is designed and arranged according to according to human everyday human everyday experiencesexperiences

• Space and objects in space produce anSpace and objects in space produce animmersive environmentimmersive environment

• Construct a Construct a virtual representationvirtual representation of a of a particular domainparticular domain

2323


• A convenient and alternative A convenient and alternative interfaceinterface for user interaction for user interaction

• Social experience: already Social experience: already through the simple presence of through the simple presence of othersothers

• Versatile ways of InteractionVersatile ways of Interaction

2424


• Humans live in well-structured Humans live in well-structured environments designed according to environments designed according to well-known well-known MetaphorsMetaphors– Rooms, Buildings, Streets, Districts, ...Rooms, Buildings, Streets, Districts, ...

• Virtual Worlds might be designed Virtual Worlds might be designed according to similar metaphors according to similar metaphors

• AvatarsAvatars are the embodied are the embodied representation of human users representation of human users within the within the Virtual WorldVirtual World

2525

Design ConsiderationsDesign Considerations

• SpaceSpace is used to model impressions is used to model impressions – objects in space convey implicit meaning objects in space convey implicit meaning

(buildings)(buildings)– social power is expressed by “height”social power is expressed by “height”

• ProximityProximity might indicate things of a similar might indicate things of a similar type or grouptype or group– eg. eg. avatars located close to an object talking avatars located close to an object talking

about that object (social context)about that object (social context)

• SpaceSpace and and ProximityProximity relate strongly to a relate strongly to a person’s cultural backgroundperson’s cultural background

2626

Design ConsiderationsDesign Considerations

• Navigation: „Don‘t get lost!“Navigation: „Don‘t get lost!“• Visual Visual DistanceDistance

– the distance to an object influences the the distance to an object influences the

level of detaillevel of detail• Audibility DistanceAudibility Distance• ViewpointViewpoint

– „„What is an Avatar looking at?“What is an Avatar looking at?“• Appearance of AvatarsAppearance of Avatars

2727

2D vs. 3D?2D vs. 3D?

Consider: „Besides the benefits obtained Consider: „Besides the benefits obtained by adding an additional dimension for by adding an additional dimension for visualization purposes, this new visualization purposes, this new degree degree of freedomof freedom might introduce might introduce new new difficultiesdifficulties. . Not every application domain has a Not every application domain has a suitable and usable representation in a suitable and usable representation in a 3D.“3D.“

2828

No, 2D and 3D!No, 2D and 3D!

•2D elements support the 2D elements support the usability of 3D representationsusability of 3D representations

•Combine the AdvantagesCombine the Advantages

•Selection techniquesSelection techniques

2929


• retain the features and advantages of the retain the features and advantages of the original metaphorsoriginal metaphors

• the the essenceessence is „opening“ Electronic is „opening“ Electronic Institutions to human usersInstitutions to human users

• explore the relationship between humans explore the relationship between humans and software agents in a 3D Virtual Spaceand software agents in a 3D Virtual Space

3D Electronic Institutions3D Electronic Institutions combine the two combine the two metaphors of metaphors of Electronic InstitutionsElectronic Institutions and and 3D Virtual Worlds3D Virtual Worlds into one single into one single metaphormetaphor

3030


A contradiction?

3131


• Framework consists of Framework consists of tthreehree layers layers

• Social User Interface Layer (top)Social User Interface Layer (top)– 3D Virtual World3D Virtual World

• Communication Layer (middle)Communication Layer (middle)– Connection and CommunicationConnection and Communication

• Electronic Institution Layer (bottom)Electronic Institution Layer (bottom)– The actual Institution executionThe actual Institution execution

3232


3333

Electronic Institutions Electronic Institutions LayerLayer

• Electronic InstitutionElectronic Institutionss are designed and are designed and specified with ISLANDERspecified with ISLANDER

• AMELI AMELI – Runtime Environment for the Multi-Agent SystemRuntime Environment for the Multi-Agent System– Electronic Institution Specification acts as inputElectronic Institution Specification acts as input– Security and reliabilitySecurity and reliability– Opens a „communication“ channel for each agentOpens a „communication“ channel for each agent

3434

Electronic Institutions Electronic Institutions LayerLayer

• Agent-mediated Environment: Agent-mediated Environment: The The GovernorGovernor

• Agents are connected to the infrastructure Agents are connected to the infrastructure via Governorsvia Governors

• ““safe-guarding” the institutions and check safe-guarding” the institutions and check whether a particular message is allowed whether a particular message is allowed to be said at the current stage or notto be said at the current stage or not

3535

3D Electronic Institution3D Electronic Institution

3636

Communication LayerCommunication Layer

• Consists of two partsConsists of two parts– Adobe Atmosphere Community ServerAdobe Atmosphere Community Server– Causal Connection ServerCausal Connection Server

• Adobe Atmosphere Community Adobe Atmosphere Community Server handles multiple users; Server handles multiple users; i.e. i.e. the the Virtual World is shared among usersVirtual World is shared among users

• Causal Connection Server builds on Causal Connection Server builds on top top

3737


• Causal Connection: „whenever a change Causal Connection: „whenever a change is made in the representation of the is made in the representation of the system, the system itself changes to system, the system itself changes to maintain a consistent state and vice maintain a consistent state and vice versa“versa“

• Reflective systems: „the representation Reflective systems: „the representation of the system is part of the system of the system is part of the system itself“itself“

3838


• Each instantiated Virtual World (i.e. Each instantiated Virtual World (i.e. the human user) communicates via the human user) communicates via the Communication Layerthe Communication Layer with its with its associated Governorassociated Governor

• Actions performed by users are Actions performed by users are passed in terms of passed in terms of messagesmessages from from the user interface to the the user interface to the communication layercommunication layer

3939


• The Causal Connection Server captures The Causal Connection Server captures these messages and postpones the these messages and postpones the actual executionactual execution

• Messages are sent to AMELI in order to Messages are sent to AMELI in order to determine their „validity“ and checks determine their „validity“ and checks whether a particular message goes in line whether a particular message goes in line with the Electronic Institution rules or notwith the Electronic Institution rules or not

• A positive validation results in executing A positive validation results in executing the requested actionthe requested action

4040

3D Electronic Institution3D Electronic Institution

4141

Social User Interface LayerSocial User Interface Layer

• 3D User Interface3D User Interface• Adobe Atmosphere PlayerAdobe Atmosphere Player

– Free download from Adobe WebsiteFree download from Adobe Website– Player is embedded in HTML PagePlayer is embedded in HTML Page– Multimedia (Sound, Movies, Live Multimedia (Sound, Movies, Live

Streams ...)Streams ...)– A combination of JavaScript and Java is A combination of JavaScript and Java is

used to communicate with „outer world“used to communicate with „outer world“• One Player per UserOne Player per User

4242

Social User Interface Social User Interface ObjectivesObjectives

• Immersive and fancy ;-)Immersive and fancy ;-)• UsableUsable

– consistent, convenient, efficient, clearconsistent, convenient, efficient, clearlyly designdesigneded

– not overcrowded, not distracting not overcrowded, not distracting

• Social interactionSocial interaction– visual (e.g. via gestures) visual (e.g. via gestures) – facial expressionsfacial expressions– sound (even speech)sound (even speech)– textualtextual

4343


• AwarenessAwareness– knowledge about my own knowledge about my own locationlocation

and the location of other participantsand the location of other participants– being aware of different being aware of different typestypes of of

participants (human users or software participants (human users or software agents)agents)

– distinguish between internal and distinguish between internal and external participants (avatar external participants (avatar visualization code)visualization code)

4444


• BelievabilityBelievability– Real vs. Surreal visualization; for Real vs. Surreal visualization; for

instance, do we need instance, do we need gravitygravity or or collisionscollisions

– „„Can a visualization be too real?“Can a visualization be too real?“– Spatial layout of the Virtual World: Spatial layout of the Virtual World:

Euclidian approach vs. „teleportation“Euclidian approach vs. „teleportation“

4545

Social Virtual WorldSocial Virtual World

4646

ImplementationImplementation

• 3D Electronic Institution Specification3D Electronic Institution Specification– DevelopedDeveloped an XML schema which an XML schema which

enhances the Electronic Institution enhances the Electronic Institution SpecificationSpecification

• Specification TransformerSpecification Transformer– AAutomatically converts the specification utomatically converts the specification

of an Electronic Institutions into a 3D of an Electronic Institutions into a 3D Electronic Institution SpecificationElectronic Institution Specification

4747

Annotation EditorAnnotation Editor

• Design Tool for System EngineersDesign Tool for System Engineers• Input: Electronic Institution SpecificationInput: Electronic Institution Specification• Output: 3D Output: 3D Virtual WorldVirtual World• A basic conversion is done automaticallyA basic conversion is done automatically• Additional features are incorporated by Additional features are incorporated by

the System Engineer (the System Engineer (eg.eg., , furniturefurniture) )

4848

System ArchitectureSystem Architecture

4949

TechnologiesTechnologies

• Electronic Institutions LayerElectronic Institutions Layer– IDE-eli (Integrated Development IDE-eli (Integrated Development

Environment for Electronic Institutions)Environment for Electronic Institutions)– Multi-Agent System is implemented in Multi-Agent System is implemented in

JavaJava– It utilizes JADE and materialized in AMELIIt utilizes JADE and materialized in AMELI– Institution Specification is an XML file Institution Specification is an XML file

specified with ISLANDER specified with ISLANDER

5050


• Communication LayerCommunication Layer– LinuxLinux– Apache Web ServerApache Web Server– Specifications are stored in XMLSpecifications are stored in XML– Adobe Atmosphere Community Adobe Atmosphere Community

ServerServer– Open Source DatabaseOpen Source Database (mySQL) (mySQL)– JavaJava

5151


• Social User Interface LayerSocial User Interface Layer– Adobe Atmosphere Editor Adobe Atmosphere Editor

(commercial)(commercial)– Adobe Atmosphere Player (free)Adobe Atmosphere Player (free)– Player is embedded in HTML Player is embedded in HTML – JavaScriptJavaScript– JavaJava

5252

ConclusionConclusion

• New Metaphor: New Metaphor: 3D Electronic 3D Electronic InstitutionsInstitutions3D 3D Virtual Worlds & Electronic InstitutionsVirtual Worlds & Electronic Institutions

• Human-centred View Human-centred View • Trust Trust • Social InteractionSocial Interaction• Application Domains: e-Commerce, Application Domains: e-Commerce,

E-Government, E-Learning ...E-Government, E-Learning ...

5353

Next StepsNext Steps

• Fully implement the Causal Connection Fully implement the Causal Connection Server at the Communication LayerServer at the Communication Layer

• Conduct a Field TrialConduct a Field Trial– test acceptance, usability, performancetest acceptance, usability, performance

• LearningLearning– Relationship between Agent and Principal Relationship between Agent and Principal

• Make the Environment Information-RichMake the Environment Information-Rich– e.g., incorporate information botse.g., incorporate information bots

5454

DiscussionDiscussion

Documents

3D Electronic Institutions eMarkets Research Group Helmut Berger