Ravin Balakrishnanravin@dgp.toronto.edu

www.dgp.toronto.edu/~ravin

Physics, projection, piles, and pens: user interfaces in the post GUI era.

2

DGP lab

• Since 1968• 6 faculty: 3 HCI, 3 Graphics• ~50 graduate students, ~5 postdocs• Several tech/admin staff

My group:• 15 graduate students, 2 postdocs

3

Projects• BumpTop: Desktop with Physics and Piles (CHI 2006)

• Hybrid pointing (UIST 2006)

• Handheld projector interaction (UIST 2006)

• Volumetric Display Interaction (UIST 2004, UIST 2006, AVI 2006)

• Mnemonic rendering (UIST 2006)

• Under the table interaction (UIST 2006)

4

Physical vs. GUI Desktops

• GUIs lack rich expressiveness• Casual but meaningful placement• Loose vs. rigid organization

5

Physically Inspired Desktop

• Enrich desktop metaphor with real world characteristics • Support casual organization, Piling• Pen centric

6

Support for Piling

• Ethnographic studies of office organization(Malone 83, Whittaker et al. 01)

• Filing• Cognitively difficult, multiple categories• Premature, duplicate filing

• Piling• Lightweight, easy to maintain• Deferred classification• Reminds, accessibility of recent information

7

Related Work• Ethnographic Studies of Office Organization

• Malone 1983• Whittaker et al. 2001

• Piling in GUIs• Mander et al. 1992• Presto 1999• DynaPad 2004

• Physically Inspired Interfaces• WebBook, Web Forager 1996 • Beaudouin-Lafon 2001• Fold’n’Drop 2004• 3D Paper-flyer metaphor 2003

• 2½D, 3D Spatial Organization• Data Mountain 1998• Task Gallery 2000

8

Design Goals

• Realistic Feel• Disable Physics as Necessary• Tangible, Paper-like Icons• Optimize for Pen Interaction• Discoverability

9

Video Demonstration

• Object Movement• Pile Interaction• Paper-like Icons• Arbitrarily sized objects• Beyond Physics

10

LassoMenu

• Fluid selection and parameter adjustment in one stroke

• Advantages:• Avoids tricky gestures, memorization• Doesn’t interrupt flow with pen lift, etc.

11

Real and Virtual Piles

• Items pulled out for emphasis

12

Pressure

• Trigger interaction with PressureLock• Pressure Cursor States:

0% Pressure 75% Pressure 100% PressurePressureLock

Hollow cursor,indicates functionality

13

Piling in GUIs

• We significantly expand on previous designs• Pile-pile, intra-pile interaction• Emphasize discoverability• Pile widgets• Integrate browsing advances • Transition to pile• Physical simulation

(Mander et al. 92) DynaPad 2004Presto 1999

14

Addional Interaction Techniques

• Manipulation• Sort, Delete, Move• Emphasize contents• Insertion: Tossing, Drag’n’Cross

• Paper-like Icons• Crease, Crumple, Pin up• Enlarge, Reduce size• Shelving

• Arbitrarily sized objects• Beyond Physics

• Enforcing Axis Alignment• Hierarchical Piles

15

Initial User Evaluation• 12 participants (5 female, 7 male)• 1 hr videotaped think-aloud sessions

• Simulated tasks

• Realistic Feel• Novices found realistic techniques empowering• Playfully rearranging, balancing items

• Optimize for Pen Interaction• Crossing “smoother”, more realistic• Crossing deteriorated at screen edges• Drag’n’Cross troublesome

• Discoverability• All users discovered interaction techniques• Improved repetition of tasks

16

Some Next Steps

• Convey meta-data as physical attributes• Other physics: fields, magnets, cloth• Beyond reality?• Deployable applications

17

Hybrid Pointing

18

Hybrid Pointing

19

Hybrid Pointing Evaluation

20

Handheld Projectors

21

Prototype

22

Video

23

Volumetric Displays

• Points illuminated in true 3D

• 360° viewing angle

• Available commercially

24

Implementation - Display

• Perspecta Spatial 3D System • 10" spherical image

• 198 2D slices, 768x768 each

• 24Hz refresh rate

25

Implementation – Finger Tracking

• Vicon motion tracking system• 5 Cameras

• Tracked 5mm markers• Real Time, 120Hz, sub mm precision

26

Issues & Assumptions

• Objects within arm’s reach

• Objects inside enclosure

• Direct link between input and display

• Volumetric display as exclusive platform

27

Video

28

Acknowledgements

• DGP Lab members• John Hancock

29

Prior Physically-Inspired Interfaces

• Windows as paper

Beaudouin-Lafon 2001

WebBook, Web Forager 1996

3D Paper flyer metaphor 2003 Fold’n’Drop 2004

30

Prior Spatial Organization

• Benefit of 2½ D, 3D spatial memory

Data Mountain (1998) Task Gallery (2000)