Perceptual Issues in Haptic Interface Design

Lynette Jones,Department of Mechanical Engineering, Massachusetts Institute of Technology.

Overview• Tactile, thermal and kinesthetic sensors• What can we sense?• Field of view – is more better?• Haptic interface configuration

- performance and perception• Tactile illusions and inter-sensory

interactions – perceptual enhancement?• Conclusions

7 classes of Mechanoreceptor17,000 – glabrous skin of hand

2 classes of Thermoreceptorwarm and cold receptors

4 classes of Nociceptorsmechanical, thermal, chemical, polymodal

3 classes of Proprioceptormuscle spindle receptors (Group Ia and Group II)Golgi tendon organs

Information Channels in Human Hand

(Visual Encyclopedia, 1999)

Tactile ReceptorsMechanoreceptors ModalityMeissner corpuscle Stroking, fluttering

Merkel disk receptor Pressure, texture

Pacinian corpuscle Vibration

Ruffini ending Skin stretch

Hair tylotrich, hair-guard Stroking, fluttering

Hair-down Light stroking

Field Skin stretch(Johansson & Vallbo, 1983)

Thermal Sensors

ObjectThermal

interaction

Thermal perceptionThermal

sensation

Change in skin temperature

• Thermoreceptor range: 13-45 °C• Cold receptors respond to decreases in temperature, warm

receptors to increases in temperature• Resolution: 0.02-0.05 °C (transient)• Thermal discriminability: 2%• Neutral thermal sensation: 31-36 °C• Thermoreceptors are NOT thermometers

Proprioceptors

• Muscle spindles– Afferent neurons

• Ia (primary)• II (secondary)

– Efferent neurons• γ-static motor

neurons• γ-dynamic motor

neurons– An actively

controlled sensor• Golgi tendon organs

– Afferent neurons• Ib

(www.blackwellpublishing.com)

Why an active sensor?1. Intrafusal muscle fibers keep the spindle in its sensitive range2. Muscle spindle efferents also change the sensor gain

(http://fig.cox.miami.edu/~lfarmer/BIL265/locomotion.html)

What do we sense? HapticTactile

• Surface texture- Roughness- Hard/soft

• Orientation• Shape• Size• Pressure• Curvature• Motion

• Mass/weight• Force• Stiffness/compliance• Viscosity

Thermal

• Cooling• Warming

Tactile Spatial Acuity

(Johnson & Phillips, 1981)

Grating-orientation discrimination thresholds

(Revised from Weinstein, 1968) (Vega-Bermudez & Johnson, 2001)

Tactile Spatial Acuity

On smooth glass surface, dot of height 1-3 µm and diameter of 550 µm can be detected by the fingertip

Spatial frequency of grating: fingertip can distinguish 40-50 µm in spatial period of 0.7-1.0 mm

35 mm

(Eye – 1 min arc, ear – 1°)(Morley et al., 1983) (Johansson & LaMotte, 1983)

Tactile Temporal Acuity

Successiveness Two stimuli (1 ms duration) must be separated by 5.5 ms to be perceived as two at a single locus

Temporal orderTwo successive stimuli at separate sites must be separated by 20 ms in order to determine which site first

20 ms

5 ms

(Ear - 0.01 ms, eye -25 ms)

Vibrotactile stimuli• Common in tactile

and haptic displays• Convey tactile

sensations in VE and teleoperation

• Medium of communication

BrailleNote BT

Vibrotactile stimuli• Frequency • Amplitude• Waveform• Locus - Localization (12 sites on torso),

Optacon (24 by 6 pin array)

• Duration- Burst duration, pulse repetition

rate, number of pulses

Frequency discrimination

Amplitude discrimination

Vibration• Frequency

Range: 0.4-1000 HzDifference Threshold: 4-50%

• AmplitudeThreshold: 0.01µm at 200 HzDifference Threshold:11%BUT

Frequency and amplitudeinteract (Brisben et al., 1999)

0.01

0.1

1

1 10 100 1000

Frequency (Hz)

Diff

eren

ce th

resh

old

(∆F/

F)

finger

forearm

hand

finger

-5

-4

-3

-2

-1

0

1

20 30 40 50 60 70

PSE

(db

re: s

tand

ard)

Comparison frequency (Hz)

Standard 25 Hz

Vibration ThresholdsGlabrous skin

0.01

0.1

1

10

100

1000

1 10 100 1000

Disp

lace

men

t (µ

m)

Frequency (Hz)

Hairy skin

(Bolanowski et al., 1994)(Brisben et al., 1999)

DorsalPalmar

Field of view: Spatial and temporal summation

• Vibrotactile sensitivity- Increases with contact area on hand (above

40 Hz)- Increases with stimulus duration or number of

stimuli in a sequence• Perceived roughness- Increases when surface explored with two

fingers as compared to one• Perceived heaviness- Increases when adjacent fingers lift weights

simultaneously• Haptic object recognition- Whole hand is faster and more accurate than

manual exploration with a single finger

BUTTactile processing of spatial charactersNo improvement when characters are displayed across two adjacent fingers

(Brisben et al., 1999)

1

10

100

1000

1 10 100 1000

Diff

eren

tial t

hres

hold

(N.s/

m)

Reference Viscosity (N.s/m)

19%

0.01

0.1

1

10

0.1 10 1000

Force (N)

Diff

eren

tial t

hres

hold

(N)

6%

Force

ViscosityHaptic Thresholds

Stiffness

0

200

400

600

800

1000

0 900 1800 2700 3600

Stiffness (N/m)

Diff

eren

tial t

hres

hold

(N/m

)

17%

Mechanics of Contact and PerceptionMechanical properties of interface:

mass, stiffness, viscosity.Workspace

Joystick/Mouse/KnobStylus/ProbeThimble

PHANTOM™ PHANTOM™

Interface Design: Control and SensingWhat is optimal?

Keyboard - activation force - 1 N - tactile feedback - 0.04-0.25 N Joystick- thumb control- grasp control

StylusThimbleTrackball - force discrimination

Glove- position resolution

Exoskeleton

Pantograph

Cyberforce™Immersion Corp.

Performance Issues in Haptic Displays

150

200

250

300

0 500 1000 1500 2000 2500

Interface Stiffness (N/m)

Del

ay (m

s)

450

500

550

600

650

700

750

10 1000

Interface Viscosity (N.s/m)

Tim

e to

0.5

pt o

n St

ep

Res

pons

e Fu

nctio

n

Mechanical properties of the display (mass, stiffness, viscosity)Limb mechanical properties and magnitude of changes during tasksDelays associated with processing – models of neuromuscular, sensory, and cognitive systems

Thermal sensing•Thermal cues are useful for identifying objects haptically

– Thermal characteristics of the object– Thermal sensation associated with contact

Bar

rier

Haptic display

Thermal Feedback

Virtual objectObject at remote place

Thermal Model

ObjectThermal

interaction

Thermal perceptionThermal

sensation

Change in skin temperature

T

t

t

Material Skin

Tskin,i

material, i T

2/12/1

2/1,

2/1,

)()()()(

skinmaterial

skiniskinmaterialimaterials ckck

ckTckTT

ρρρρ

++

=

2/1,"

)()(

tTTk

qskin

iskinsskinskin πα

−=

q”

Ts

Heat transfer process

Thermal Display

Thermistor 1

Thermistor 2

A/D

D/ABipolar Power Supply

&Voltage AmplifierPI Control

Temperature Curve

20

25

30

35

40

0 2 4 6 8 10 12Time (s)

Tem

pera

ture

(C) Peltier device

Index finger

Thermistor 1

Thermistor 2

A/D

D/ABipolar Power Supply

&Voltage AmplifierPI Control

20

25

30

35

40

0 2 4 6 8 10 12Time (s)

Peltier device

Index finger

Real Simulated

0

0.2

0.4

0.6

0.8

1

Copper SS Granite ABS Foam

Material

Prop

ortio

n of

cor

rect

resp

onse

s

RealSimulated

Threshold for reliable identification

0

0.2

0.4

0.6

0.8

1

Copper-SS Copper-ABS SS-ABS

Material Combination

Prop

ortio

n of

Cor

rect

Res

pons

e

Threshold for discrimination

R-R

R-S

Thermal sensing: Real vs simulated

Model Evaluation: Physiological Results

Changes in skin temperature during contact

0

1

2

3

4

5

6

7

0.00 4.00 8.00 12.00

Contact pressure (kPa)

Dec

reas

e in

ski

nte

mpe

ratu

re d

urin

gco

ntac

t (o C

)

Measured dataModel prediction

5.9 kPa 11 kPa

Thermal Field of View

0

0.2

0.4

0.6

0.8

1

Urethan

e

Woo

d

Glass SS

Copper

Prop

ortio

n of

cor

rect

res

pons

es

Improvement in identifying materials when cues are presented to three fingers but no further improvement with five fingers.

13

5

Tactile Illusions:Space-time interactions

Saltation

(Geldard, 1975)

Illusory displacement of tactile stimuli

(Cholewiak & Collins, 2000)

(Geldard & Sherrick, 1972)

Haptic Illusions

0

5

10

15

20

0 200 400 600 800

Weight (g)

Perc

eive

d W

eigh

t

LARGE

SMALL

1.4

1.8

2.2

2.6

3

100 10000Volume (cc)

350 gm

904 gm

Size-weight illusion - visual(Masin & Crestoni, 1988)

Texture-force interaction(Flangan et al., 1995)

Size-weight illusion - haptic(Ellis & Lederman, 1993)

Thermal Illusions

Thermal referral Temperature-force illusions(Stevens & Hopper, 1982)

Conclusions• Properties of the somatosensory

system provide a framework for the design of tactile and haptic interfaces

• Psychophysical studies have revealed how humans respond to changes in mechanical and thermal stimuli, and the interactions that occur between these stimuli

• Design of interfaces –prehension, gesture, single digit? What is optimal?