Activity presentation

Activity presentation

BP 53 - 38041 Grenoble Cedex 9 - FranceFédération IMAG

Communication Langagière etInteraction Personne-Système

Jean.Caelen@imag.frwww-clips.imag.fr

Director: Jean Caelen (DR CNRS)Vice-Director : Catherine Berrut (Prof. UJF)

Institutions

UMR 5524

Research TopicsMan-Machine Interaction(s)Collaborative interactions with machinesSpeech, Image, Language Processing

E-LearningPerceptive Rooms and Augmented RealityMultimedia Document IndexingMan Machine Interaction EngineeringSpeech Recognition and Natural Language DialogueCollaborative and Multimodal InterfacesInformation RetrievalMachine TranslationLanguage Processing

Structure

Direction

Research• ARCADE• GEOD• GETA• IIHM• MRIMLa

bora

toty

cou

nci l

Dire

ctio

n co

unci

l Technical staff• Maintenance• Projects

Administration• Secretariat• People Management• Financial• Subsistance• Communication

Valorisation• MultiCom

People : 72 (staff + PhD students)UJF, CNRS, INPG, UPMF, US

Research Teams

ARCADE (J.P. Peyrin, prof. UJF)e-Learning (Ateliers de Réalisation et de Conception d’Applications Destinées à l’Education)

GEOD (J.F. Serignat, MCF INPG)Speech Group (Groupe d’Etude sur l’Oral et le Dialogue)

GETA (C. Boitet, prof. UJF)Machine Translation Group (Groupe d’Etude sur la Traduction Automatique)

IIHM (J. Coutaz, prof. UJF)Man Machine Interaction Engineering (Ingénierie de l’Interaction Homme-Machine)

MRIM (M.F. Bruandet, prof. UJF)Multimedia Information Retrieval (Modélisation et Recherche d’Informations Multimédia)

People 1/1/2002

Regular Staff 73(a) Profs + Researchers 29(b) Engineers + Admin. 11(c) PhD + Post-docs 26(d) Under contracts 7

(e) Masters and others 24(g) Invited 10Total 107

Technological transfert

MultiCom : Usability platform for the design and the evaluation of interactive systems

Start-up creations

1998 : NOVADIS, ergonomics, cognitive engineering1999 : ARLYS (MENRT, CARMI, Microsoft, Axe et Cible), e-

learning, education software development2000 : DMAILER, e-mail indexing and processing

MultiCom

Technological Platform :• To study human factors in situated tasks• To design and evaluate innovative and interactive services (or

systems)by,• Real usage experimentation• Simulation (Woz for instance)

Partnership Linguists, Sociologists, Ergonomists

Relationship and contractsMultiCom• 6 industrial : (Schneider, France-Télécom, Leti-CEA, STM), • 2 european : SIRLAN IST 5th PCRD, ADAMOS, Finlandia • 2 regional : SHIVA RRA 99 et Thématique 2000Research• 6 industrial : (Thalès, France-Télécom, EDF, Alcatel) in

RNRT, RNTL, CTI CNET, DGA• 2 international : C-STAR, UNL (17 countries)• 5 european : NESPOLE, TACIT, TIPS, ARIADNE, GLOSS,

CAMELEON, FAME• 3 research networks : CNRS (french networks)• 2 regional• 5 local (C-STAR, RICOM, RESIDE-HIS, EPIC, e-Xpérience)

Multi-team projects

C-STAR/NESPOLE! : Face to face speech translation (GEOD, GETA)

RICOM : Multimodal interaction for Multimedia information retrieval (GEOD, IIHM, MRIM)

Smart Rooms : disable people monitoring, home services… (GEOD, IIHM)

FORMID : e-learning, virtual teaching (ARCADE, IIHM)

C-STAR : Consortium for Speech Translation Advanced Research (Japan, Corea, USA, Germany, Italy, France)NESPOLE! (IST project) : NEgotiating through SPOken Language in E-commerce

Projet C-STARSpeech translationApplications to : tourism, hot line, e-commerce

2 ways possible

source

Speech recognition

Translation

Speech synthesis

target

speakingspeaking

listeninglistening

texte IFtexte IF

IF coding

Generation

Interchange Format

• Meaning representation• Based on

– Speech acts• What to want or to do

– Concepts• What to say

– Arguments• Discourse predicates

give-information

+availability+room

room-type=(single ; double)

The 22th july 1999at Grenoble

RICOM : Information Retrieval: Interactive, Collaborative andMultimodale HALPIN : Man-Machine DialogueCoVitesse : Information Representation

I found 1206 documents on the topic  artificial intelligence. Do you want refine your research ?

What have you on ... e, artificial intelligence ?

HALPIN

Man-Machine Dialogue : speech recognition + dialogue management + information retrieval + speech synthesis

History of dialogue

Input

Outputs

Documents

Speech ButtonsSpeech Recognition

Collaborative surfingTrace of navigation

Collected documents

Session controlDocument annotation

RESIDE-HIS : Smart Home for disable peopleCOMEDI : Informal Communication

RESIDE-HIS

TelemonitoringRéseau Téléphonique Public

Modem

HModem

Home bus

Physiological and biological sensorsMicrophones

Monitoring on :EventsAlarmsStatistics

Plan HIS

CoMedi : Informal communication

Feed-back, privacy control

Awareness, Focus

Acces control

Feed-back control

Example of filters

Poster

Curtain

EigenSpace

EigenSpace filter: Main Component Analysis

Image base Eigen Space

Image captured Image to send

Eigen vector ArgMin(distance)

FORMID : FORmation Interactive à DistanceMagic Board : Collaborative Space and augmented reality

FORMID : Toolkit for teaching by example I learn what I do.

FORMID

GeneSimuOasis

Mag

ic B

oard

Platform for design and evaluationof interactive systems

Analyse des besoins

Conception

Conception logicielle

Codage

Tests Unitaires

Tests d’intégration

Tests UtilisateursEvaluation

ergonomique

Development

Observations

Task analysis

Design

Software design

Lifecyle

Objectives

To anticipate the usage and to predict the usabilityTo design and to evaluate interactive systems along the lifecycle

Participatory design methodAnd simulation techniques (as Moz) for the experimentation and the test

using

A good starting point : enrich the first idea

Taking into account all the criteria about (a) sociological, (b) ergonomical, (c) economical,(d) technological points of view

Idea

Functions

Specifications

Scenario

ObservationDiagnostic

Validation

Mockup

Evaluation

Sociology…Practice analysisUsage meaning• By questionnaire• By interviews• By recording practicesApplying sociological criteria

Recommendations for the design

Ergonomics...Simulation, Woz, Scenarios…

• Test human perceptive• Test human behavior (after capture) under definite and limited

scenarios• Task and activity analysis

Recommendations for the design

The first cycle ends…

Functionalities definitionDesign constraintsCriteria Evaluation definitionSpecifications

Then development can start along a second iterative cycle

The hardware platform

Client Magicien - Agent

Echange de- images- plans- textes

DATVidéo-disque

1 Magicien "Gestion du dialogue" 1 Magicien "Annotation"

UserDevices for observation

Wizard 1 Wizard 2

Control video

Multimedia recording

Interactive devices

Conclusion… CLIPS-IMAG is…

• A research lab on interactive systems• Open on the technological transfert• Focused on user centered design methods

BP 53 - 38041 Grenoble Cedex 9 - FranceFédération IMAG

Communication Langagière etInteraction Personne-Système