1

Interaction Devices

Interaction Devices The Standard: QWERTY (or Sholes) Keyboard

Mobile devices are driving the need for future input devices

Pointing devices (mouse, touchscreen)

Gestural input

Two-handed input

3-d pointing

Voice input/output

Wearable devices

Whole body involvement

Niche applications

• Eye-trackers

• DataGloves

• Haptic/force-feedback

• Brain-controlled mouse movement

2

Interaction Devices

Interaction Devices

Multimodal interfaces

• Combine several modes of input/output

• E.g., voice commands with pointing devices Likely direction

• Giving users the ability to switch between modes depending on their needs

• E.g., Driving a car

– Operate navigation systems with touch or voice input

– Invoke visual or voice output based on location (e.g., moving in traffic vs. at

a stop sign)

– DB adjustment for ambient noise Context-aware computing

• Sensors:

– Global Positioning System (GPS)

3

Interaction Devices

4

Interaction Devices

Interaction Devices

Context-aware computing

• Sensors:

– Global Positioning System (GPS)

– Cell-phone sources

– Wireless connections

• Make detailed information available about the users surroundings

– Restaurants, gas stations

– Museum visitors or tourists

– Auto-connect to a printer based on room location

– Yelp

• Privacy issues

5

Interaction Devices

Keyboards and Keypads

QWERTY

• Beginners approximate 1 keystroke per second

• Average office worker is 5 keystrokes per second (50 words per minute)

6

Interaction Devices

Keyboards and Keypads

Potential for higher rates of information input

• Courtroom recorders (300 words per minute)

• Piano Keyboard

– Allows several finger presses at once

– Responsive to different pressures and durations

• Chord keyboards

• One-handed keyboards

– Useful for tasks requiring one hand manipulation of an object

7

Interaction Devices

Keyboards and Keypads

Reduction of ulnar abduction and pronation

Ulnar tunnel syndrome

• Ulnar tunnel syndrome is caused by pressure on the ulnar nerve at the wrist

• This nerve is found on the pinkie-finger side of the wrist

8

Interaction Devices

Keyboards Layouts

QWERTY

• Keep frequently used letters apart,

• Slow down users to avoid key jamming Dvorak

• Reduces finger travel time

• Increases speed, reduces errors

9

Interaction Devices

Keyboards Layouts

Number Pads – phone pads versus calculator pads

• Most computer keyboards use the calculator layout

• Performance is slightly better with the phone layout

10

Interaction Devices

Keyboards Layouts

Keyboard for those with disabilities

• Combination of small hand movements and small finger

presses selects the letters and controls the cursor

• No finger or wrist movement is needed

• Helpful to users with carpal tunnel syndrome or arthritis

• Each dome slides into one of eight zones to type a

character

• Either dome can slide first or move both at the same

time.

• Domes slide toward the center of their color or

character zones. (not directly at the characters)

• Slide the right dome to the zone of the character you

wish to type; slide the left dome to the color of that

character.

11

Interaction Devices

Keyboards Layouts

Dasher

• Predicts probable characters and words as users make their selections

• Continuous two dimensional pointing with a mouse, touchpad or eye-tracker

• http://www.youtube.com/watch?v=K6jnY2bNIzw

12

Interaction Devices

Keys

Modern keyboards

• ½ inch square

• Slightly concave surfaces

• Matte finish

• 40 to 125 gram force

• 3 to 5 millimeters of displacement (when the key is pressed far enough to send a

signal, both tactile and audible

• Larger keys (ENTER, SHIFT, CTRL)

• Keys with indicators (CAPS LOCK, NUM LOCK)

• Combination Keys (Ctrl-V)

• Cursor movement keys

• Other special keys (HOME, PAGE UP, END, PAGE DOWN)

• Number Pad

13

Interaction Devices

Keyboards and Keypads for Small Devices

Foldable Keyboards

Virtual Keyboards

14

Interaction Devices

Keyboards and Keypads for Small Devices

Mobile Phones and Devices

• Multi-tap to select a letter

• Word prediction

• Palm Graffiti

15

Interaction Devices

Pointing Devices

Avoids learning commands

Reduces the chances of typographical errors

Keeps the user’s attention on the display

Operations

• Select

• Position (e.g., move)

• Orient (e.g., rotate)

• Path (e.g., curving line in a drawing program)

• Quantity (e.g., numeric selection)

• Text (e.g., enter, move, edit) Direct Control Devices

• Examples: Light Pen, Touch Screen Indirect Control Devices

• Examples: Mouse, Trackball, Joystick

16

Interaction Devices

Pointing Devices

Criteria for success

• Speed and accuracy

• Learning time

• Cost and reliability

• Size and weight Example: Touch screen

• Gestures

• Touch screen is the only input that has survived the

Disney theme parks Indirect Control Pointing Devices

• Eliminate hand fatigue and hand obscuring the

screen of direct input

• Required more cognitive processing and eye/hand

coordination

• Examples: Mouse, trackball, joystick, graphics tablet

17

Interaction Devices

Comparison of Pointing Devices

Light pen and touch screen (fast but inaccurate)

Mouse (fast and accurate)

Keyboard entry in sometimes faster than the mouse (e.g., data entry)

Joysticks and trackballs are often preferred over mice by users with motor disabilities

Alternative keyboard entry should be provided

Touch screen and trackball are durable for public access

Mouse, trackball, graphic tablet and touch pad are good for pixel level pointing

18

Interaction Devices

Fitts’s Law

Predicts the amount of time to point at an object

Helps to decide the location and size of buttons

Helps to decide on the types of pointing devices as a function of task

The time for hand movements was dependent on the distance users had to move (D) and the target size (W)

Doubling the distance took longer, but not twice as long

Increasing the target size enabled the user to point at the object more rapidly

MT = a + b log2(D/W + 1)

a = approximates the start/stop time in seconds for a given device

b = inherent speed of the device

if a = 300 msec, b = 200 msec/bit, D = 14 cm, W = 2 cm

then

MT (movement time) = 900 msec

19

Interaction Devices

Fitts’s Law

The farther away a target is, the longer it takes to acquire it with the mouse

The smaller a target is, the longer it takes to acquire it with the mouse

The inverse of both statements is true as well (closer and bigger targets can be more quickly acquired)

There are two main ways to improve mouse efficiency: put the controls closer, or make them bigger

As monitors have gotten bigger and screen resolutions have increased, Fitts' law dictates that actual mouse efficiency has gone down

In other words, the same button takes much longer to click than it did fifteen years ago

Office 2007 and Fitts' Law

• Most controls in the Ribbon are labeled. This helps discoverability and usability

considerably, but it also makes the buttons bigger and easier to target.

20

Interaction Devices

Fitts’s Law

The Mini Toolbar was designed with Fitts' Law in mind as well. Whenever you select text or right-click selected text, a small toolbar appears directly next to the mouse cursor. As you move closer to it, it fades in; as you move away, it fades out.

The controls on the Mini Toolbar are small, but because they're located directly next to your cursor, they're easy to target

21

Interaction Devices

Fitts’s Law

Mile-High Menus and Magic Corners

• One of the most useful aspects of applying Fitts' Law to computers is that screen

size is bounded. No matter how far you move your mouse to the left, the cursor will

never go farther than the left side of the screen

• Because the Quick Access Toolbar is in the title bar, the buttons are effectively

infinitely tall. You can target and click each of the buttons very quickly; they're a

"mile high."

Office Button

22

Interaction Devices

Novel Devices

Foot Controls – Foot Mouse (takes twice as long as a hand operated mouse)

Eye-tracking – still mostly a research tool

Data Glove

Haptic Feedback – moving a control (e.g., a mouse) and feeling resistance

Bimanual Input – non-dominant hand makes a coarse selection, the dominant hand performs a precise movement

Ubiquitous Computing

• Embeds sensing technologies in a room

• The user wears badges so the sensors detect the presence of a user

• Application: tracking patients in a hospital

23

Interaction Devices

Novel Devices

Digital Pen (by Logitech)

• Records the strokes written on digital paper

• When the pen is placed back in its holder, the data is transferred to the computer

Digital Microscopes

• Stores magnified pictures and annotations

24

Interaction Devices

Speech and Auditory Interfaces

The vision of computers chatting with users may be more of an uninformed fantasy than a desirable reality

Voice commanding is more demanding of user’s working memory than is hand/eye coordination

Speech requires use of limited resources, while hand/eye coordination is processed elsewhere in the brain, enabling a higher level of parallel processing

Planning and problem solving can proceed in parallel with hand/eye coordination, but they are more difficult to accomplish while speaking

Variations related to speech and audio technologies

• Discrete-word recognition

• Continuous-speech recognition

• Voice information systems

• Speech generation

• Non-speech auditory interfaces

25

Interaction Devices

Speech Systems

Potential uses

• Users with vision impairments

• When the user’s hands are busy

• When mobility required

• When the speaker’s eyes are occupied

• When harsh or cramped conditions preclude use of a keyboard Technologies

• Speech store and forward

• Discrete-word recognition

• Voice information systems

• Speech generation Issues with speech recognition

• Increases cognitive load when compared to pointing

• Speaking commands or listening disrupts planning and problem solving

• Interference from noisy environments

• Unstable recognition across changing users, environments and time

http://www.youtube.com/watch?v=EySAW1C60is&feature=related

Siri

http://www.youtube.com/watch?v=0ZeJwWVbx5U&feature=related

26

Interaction Devices

Speech Systems

Issues with speech output

• Slow pace of speech output when compared to visual displays

• Ephemeral nature of speech

• Difficulty in scanning/searching

27

Interaction Devices

Discrete-word recognition

Recognition of individual words

• Accurate 90% to 98% for 100 to 10,000 word vocabularies

• Speaker-dependent training (the user trains the system)

• Speak-independent (not as accurate)

• Keyboards and pointing devices are more rapid, and actions and command are

more visible for easy editing

• Error handling is difficult and slow

Continuous-speech recognition

• Difficult to recognize the boundaries between spoken words

• Problems with accents, variable speaking rates, disruptive background noise, and

changing emotional conditions

• Fields with distinct terminology may be able to use this technology due to the

limited and precise vocabulary

28

Interaction Devices

Voice information systems

Interactive Voice Response (IVR)

• Examples: Help support, banking, ordering, voice mail Challenges

• Complex, deep structures

• Slow pace of voice output

• Difficulty scanning Apple’s iPod

• Management of large audio databases

• Retrieval of specific segments

29

Interaction Devices

Speech Generation

Automobiles systems (e.g., “Turn left onto Route 896)

Control rooms (e.g., “Danger, temperature rising”)

Privacy related to audio messages

Is the human voice a better means of communication (e.g., Atlanta Airport)?

Are tones sometimes better than voice (e.g., “Headlights on”)

Microsoft’s Windows Narrator (reads text display on screens)

Voice tagging of web pages (VoiceXML)

Googles TalkBack

Non-Speech Auditory Interfaces

MIDI (Musical-instrument digital interfaces) for music composition

Auditory icons – distinctive familiar sounds

Earcons – abstracts sounds whose meanings must be learned

Example Earcons

http://upload.wikimedia.org/wikipedia/commons/0/07/Emergency_broadcast_system.ogg

http://www.youtube.com/watch?v=IvEwOfL21Uo

30

Interaction Devices

Displays – Small and Large

The primary source of feedback to the user

Physical dimensions (diagonal)

Resolution (number of pixels)

Number of available colors

Luminance, contrast and glare

Power consumption

Refresh rates

Cost

Reliability

31

Interaction Devices

Displays Technologies

Raster-scan cathode-ray tubes (CRTs)

• Similar to televisions

• Electron beams sweeping out lines of dots to form letters and graphics

Liquid-crystal displays (LCDs)

• Voltage changes influence the polarization of tiny capsules of liquid crystals

32

Interaction Devices

Displays Technologies

Plasma display panels (PDPs)

• Rows of horizontal wires are slightly separated from vertical wires by small glass-

enclosed capsules of neon-based gases

• When the horizontal and vertical wires receive a high voltage, the gas glows

Light-emitting diodes (LEDs)

• Certain diodes emit light when a voltage is applied

• The curved display in New York’s Time Square

33

Interaction Devices

Displays Technologies

Electronic ink

• Paper-like resolution (80 to 200 dpi)

• Uses tiny capsules containing negatively charged black particles and positively

charged white particles

34

Interaction Devices

Displays Technologies

Smart Phone penetration

35

Interaction Devices

Displays Technologies

Large Displays

• Informational wall displays (control room usage)

36

Interaction Devices

Display Technologies

Large Displays

• Interactive Wall Displays

• SMART board

37

Interaction Devices

Display Technologies

3D Smart Phones

Visual Strain?

38

Interaction Devices

Display Technologies

Perceptive pixel on CNN

Check this out: CNN Perceptive Pixel

Notice

• Gestures

• Resizing capabilities

• Selection capabilities

A world of Glass

http://www.youtube.com/watch?v=bBjvqnKQsTI&feature=related

39

Interaction Devices

Display Technologies

Multiple desktop displays

Six 19” Monitors

Facilitate side-by-side comparisons of documents, software debugging, information visualization and analysis

40

Interaction Devices

Display Technologies

Heads-up and helmet-mounted displays

• Project information on a partially silvered windscreen of an airplane cockpit or car

• Operators can remain focused on the surroundings while receiving computer

generated information

Mobile device displays

• Making phone calls

• Texting

• Checking appointments

• Reading email

• Finding a locations

• Obtaining directions

• Listening to music

• Internet browsing http://www.youtube.com/watch?v=T5YOffkw0_I

8 minutes into video

41

Interaction Devices

Display Technologies

Printers

• Speed

• Quality

• Cost

• Compactness

• Quietness

• Type and size of paper

• Support for special formats

• Reliabiltyhttp://www.youtube.com/watch?v=KGKJQ2WMTdY