S S1

Intelligent MultiModal Interfaces

Manuel J. FonsecaJoaquim A. Jorge

http://sketch.inesc-id.pt

S S2

Presentation Outline

• Problem

• Tools/Solution

• Objectives

• Project Overview

• Task Analysis

• 2D Sketch Editor

• Sketch-Based Retrieval

• Current Status

S S3

Problem

• Longer product cycles• Inflexible separation - design and engineering• Raise manufacturing costs and Time-to-market

• CAD systems allow very complicated models• “Bad” Usability at early stages of product design

(pencil/paper reign)• Early design phase dissociated from product

definition• Large conceptual gaps

Mental model vs Geometric models

S S4

S S5

 

S S6

S S7

• 9 Weeks to get the Virtual model

S S8

Tools/Solution

• Alternative input modalities and new interface technologies

• New sketching applications supporting what we call Calligraphic Interfaces. – Remove artificial dialogue constraints (WIMP Uis)

– Allow designers to work as they would with more traditional media (pencil/paper)

S S9

Objectives

• Rethinking Usability of design

• Research into better input modalities

• RISC Interfaces, Drawing as metaphor

• User-centered approach

• Computer-based tools in initial stages of

product design

S S10

Constraint BasedConstraint BasedModel AssemblyModel Assembly

Personal Design AssistantsPersonal Design Assistants

Immersive Surface ModelingImmersive Surface ModelingRetrieval by ExampleRetrieval by Example

S S11

Task Analysis

• We visited Industrial partners– CENTIMFE– ItalDesign– ELASIS

• Observed/Videotaped users in action• Interviewed them and presented

questionnaires• They answered to 11 questions• Reports for each site

How ?

S S12

Task Analysis

1. Who is going to use the system?2. What tasks do they perform?3. What tasks are desired?4. How are the tasks learned?5. Where are the tasks performed?6. How often are the tasks performed?7. What are the time constraints?8. etc.

What did we ask?

S S13

Task Analysis

• Identify critical tasks

• Define user requirements

• Recommendations and guidelines for the project

• Assure that our work is based on Real Data not on our imagination!

Why ?

S S14

Task Analysis• User Requirements

– Larger drawing area– Bigger and easier to identify icons– New tools for the left hand – No mouse and no keyboard– Access to past projects– Possibility to use the touch sense for modelling– Software integration (2D+3D+presentation+...)– ...

• Final TA meeting (Sept 9-10, Lisbon)– Final report almost done

Results/Conclusions

S S15

On Going WPs

• API Specification

• 2D Sketch Editor for Modelling

• 3D Surface sketching in Immersive

Environments

• Constraint-Based Scene Modelling

• Sketch-Based Retrieval

S S16

2D Sketch Editor

• Recognition-based Interfaces

• Reduced Instruction Set Interfaces

• Handle Imprecision and Ambiguity

in uniform manner

• Less (More expressive) Commands

S S17

Current Status

• Simple prototype using IRIT (geometric kernel)

• Make it more robust using better

geometric kernel such as ACIS or ICEM

• Add more functionality

2D Editor

S S18

Sketch-Based Retrieval

• Designer re-use drawings

• CAD Systems do not support retrieval

• Text indexing not sufficient for CAD

• Need Innovative methods for CAD databases

• Simple geometric features + Spatial relations

• Indexing + Matching algorithms

S S19

Query by Sketch

• How do designers retrieve?(Library Component, Past project) – Videos from TA

• Our proposalSketched Query Suggested Results

Retrieval

S S20

Matching

System ArchitectureClassificationQuery

Retrieval

S S21

Current Status

• Description using Topology Graphs

• Graph Matching using Eigenvalues

• New Indexing Structure, NB-Tree– High-Dimensional Data points

– Outperforms recent structures• SR-Tree [Katayama 97]

• Pyramid Technique [Berchtold 98]

• A-Tree [Sakurai 00]

Retrieval

S S22

Performance Evaluation

• 100,000 points

• PII@233MHz

• 384MB RAM

• Knn = 10

• Uniform Data

Retrieval

S S23

Project Outline and Status

Integration and Final Prototype

Integration and Final Prototype

Final User Evaluation

Final User Evaluation

User Requeriments

& Task Analysis

User Requeriments

& Task AnalysisAPI

Specification

APISpecification

Field and User Tests

Constraint-Based Scene Modelling

Constraint-Based Scene Modelling

Sketch-Based Retrieval

Sketch-Based Retrieval

2D Sketch Editor for Modelling

2D Sketch Editor for Modelling

3D Surface Sketching

3D Surface Sketching

Prototypes

Done Almost Done

Working On It