Human-Centered ComputingTelepresence, AR, tangible interfaces

John Canny UC Berkeley

HCC overview

Telepresence, AR, tangible interfaces

John Canny

• PRoPs (Personal Roving Presences) and gesturing avatars.

• UPM - a universal planar manipulator.

• Bearable computers - turning laptops into augmented reality systems and wearable computers.

HCC overview

PRoPs: Personal Roving PRoPs: Personal Roving PresencesPresences

Space Browser + Eric PaulosSpace Browser + Eric Paulos Surface cruiser

HCC overview

PRoPs: Toward being there

• A PRoP is a Personal Roving Presence.

• PRoPs seek to bridge the gap between telepresence and being there.

• There are two essential aspects of remote presence that only telerobotics can provide:

Non-verbal communication, i.e. gaze, proxemics, gesture etc. which are highly spatialized.

A gamut of human behaviors, such as chance encounters, informal meetings, showing or being shown, browsing, exploring etc, which are affordances of a physical presence.

HCC overview

Terrestrial PRoPs: Carts

• Carts are a good match to human capabilities.

• Our strategy was to list important physical communication skills, and to implement as many as possible.

HCC overview

The rest of the story:A taxonomy of interaction behaviors• Verbal: symbolic and prosodic

• Nonverbal communication theory (psychology, linguistics, anthropology). Main cues are:

Gaze Proxemics Facial expression Body and hand gesture Posture Touch

• Example: A Handshake depends on all the cues.

HCC overview

Differences between verbal and nonverbal communication

• Adam Kendon (83) summarizes the difference very well:

...gesture has properties different from speech. In particular, it employs space as well as time in the creation of expressive forms, whereas speech can use only time. We find, therefore, that the way information may be preserved in speech, as compared to gesture, tends to be very different.

HCC overview

Why nonverbal skills are spatial

• Gaze - very sensitive to mutual direction. Center-right-left.

• Proxemics - Depends on distance to others and the surroundings.

• Facial expression - the least spatial, directed by gaze.

• Gesture - deictic gesture is completely spatial, so are other pictographic forms.

• Posture - Directional.

• Touch - Cant fake it without space.

HCC overview

You need social context as well

• Communication.

• Persuasion.

• Trust-building.

• These are radically different contexts that rely on different non-verbal cues. We need to study telepresence in representative scenarios.

HCC overview

The two-way street:

• PRoPs provide selective control over particular cues. They can be turned on and off at will.

• So they provide a potentially interesting testbed for fine-grained study of non-verbal cues.

• We can create a rich variety of media and study their influence on the overall perception of the other person.

HCC overview

The input side

• How do you achieve rich, expressive input for PRoPs/avatars with today’s technology?

• Cross-modal mapping from pen gesture to body gesture. Both have symbolic and expressive character:

HCC overview

Future research?

• Gaze cue repair: Face to face is not perfect. There are common breakdowns in face-to-face communication, such as gaze aversion.

• Beyond Being There: It may be possible to rectify gaze aversion using image processing and/or a synthetic gaze cue.

HCC overview

Project 2:UPM: A universal planar manipulator

• A smart desk that can move a large number of objects placed on it independently.

• Provides a tangible interface in both directions for 2D layouts: floor plans, landscapes, stage directions etc.

• Uses a small numbers of motors (4) and the non-linearity of friction.

HCC overview

UPM: A universal planar manipulator• First prototype used linear voice-coils from ancient

disk drives, capable of about 2g acceleration. New prototype should manage about 50gs.

HCC overview

Bearable Computing (new project)

• An exploration of issues in personal, persistent computing (augmented reality, worn interfaces) using ordinary laptop computers.

• Avoid head-mounted displays (expensive and low-res) head-tracking, and cables.

• The approach: use optics to overlay computer images on reality, but use laptop or pocket-mounted displays.

• Testbed: Grad course in HCC next semester.

HCC overview

An augmented reality classroom

Virtual image with notes, questions, private chat

space

Physical space

Glasses to rotatethe laptop image

HCC overview

An AR classroom

• Students work in groups of 5-7, communicating silently via pen or keyboard chat.

• Each group has one main note-taker, the others add their own comments or questions to the transcript.

• Students can mark up the group transcript, or the lecturer’s notes. There is non-archived chat also.

• One student per group works as facilitator or TA, posing questions to the others, and testing understanding.

HCC overview

Bearable computers on the go

• The laptop can serve as a “bearable” computer, attached wirelessly to its owner.

Laptop in briefcase with wireless TV

and keyboard TXs.

Pocket-worn portable LCD TV

Double-mirror element in glassesHeads-up virtual image

Wireless chordal keyboard or Palmpilot in pocket for input.

HCC overview

Goals of this research

• Can we enhance attention in class with live group chat/note-taking?

• Do we get a richer transcript with collaborative note-taking?

• Does the head-up display make note-taking less distracting?

• Will people do more tasks with a mobile bearable computer because the threshold for computer use is lower?