1DTI/EPSRC 7th June 2005

Reacting to HCI Devices:Initial Work Using Resource

Ontologies with RAVE

Dr. Ian GrimsteadRichard Potter BSc(Hons)

2DTI/EPSRC 7th June 2005

Presentation Structure

Aims and Objectives Background to Ontologies Background to RAVE Initial extension of RAVE Demonstration Questions and Answers.

3DTI/EPSRC 7th June 2005

Aims

To enhance RAVE Extend resource-awareness Include user input devices

To prototype ontological approach In an existing system.

4DTI/EPSRC 7th June 2005

Objectives

Implement “Widgets” Enable Widgets to request interaction

support Enable RAVE to access Ontology

RAVE widgets announce interaction style required

Ontology finds best fit Resource interfaced with RAVE for

information exchange.

5DTI/EPSRC 7th June 2005

Background to Ontologies

Ontology: Formal collection of inter-referenced taxonomies

Taxonomy – a hierarchical classification Resources described using 6 taxonomies:

Hardware User actions Sensory experiences

Software interfaces Software interface items Variables.

6DTI/EPSRC 7th June 2005

Resource description

Hardware User Actions

Sensory Experience

Software Interface

Item

Software Interface

Variable

provides*

can detect*

has* consists of*

provides access to

represents

represents

7DTI/EPSRC 7th June 2005

Categorisation

• Each taxonomy categorises information, for example the user interaction taxonomy:

User Interaction

Vocal Command

Gesture

Vocal Interaction

Extremity Interaction

Movement Interaction

8DTI/EPSRC 7th June 2005

Libraries and APIs

Sesame (RDF/RDFS repository) OwlIm (Sesame ontology plugin) Sesame-to-Jena Jena (Ontology API and inference engine) OWL/RDF/RDFS (Ontology implementation

languages).

9DTI/EPSRC 7th June 2005

Backgroundto RAVE

RAVE – Resource-Aware Visualization Environment

Supports collaborative visualization Static datasets, or Real-time feed from remote process

Remote process can be instrumented RAVE can then steer a simulation.

10DTI/EPSRC 7th June 2005

Data Distribution First component: Data Server Acts as a distribution point & interpreter

Understands many types of data Uses Java3D+Xj3D as importer

DataServer

Internetor remote machine

VisualizationData

RAVEClient

RAVEClient

RAVEClient

RawData

11DTI/EPSRC 7th June 2005

Displaying Data Second component: Active RAVE Client

“Active” – facilities to draw on its own Accepts feed from Data Server Presents images of data to user

VisualizationData

DataServer

Active RAVE Client

Visual drawn on local machine

12DTI/EPSRC 7th June 2005

Computational Steering

Independent simulation can supply Data Server

Simulation code instrumented Transmits scene creation and

updates to Data Server Data Server reflects updates Multiple clients can view live

simulation Client interact with “widgets”

Steer simulation.

13DTI/EPSRC 7th June 2005

Summary

Data Server reads data and distributes Active Client renders locally Data Server can link to live simulation All resources shared where possible Uses Java to support (most) platforms Also:

Thin Client (PDA) renders via Render Server But that’s another story.

14DTI/EPSRC 7th June 2005

Resource support

Semantic reasoning over the ontology: Appropriate resources can be chosen

Ontology also describes: Each resource’s available software interfaces

Joins visualization to selected hardware Supports intuitive interaction Selects most appropriate resource.

15DTI/EPSRC 7th June 2005

Resource support

RAVE

Hardware Resource

Ontology

Support System

Widget

Visualization

implements

notifies requirements

detects availability

getsdescription

from

describes

16DTI/EPSRC 7th June 2005

Initial Extensionto RAVE

Database to store ontology RAVE extended with new Widget type Issues:

Lack of driver support! Only available: mouse / mouse with wheel Shows proof of concept.

17DTI/EPSRC 7th June 2005

Demonstration

We will show: RAVE interacting with a molecular dynamics

simulation RAVE discovering available input devices Ontology support automatically selecting

available devices RAVE being unaware of available input

devices.

18DTI/EPSRC 7th June 2005

Questions and Answers

Questions…

…and possibly answers?