Haptic Interfaces - Courses

02/14/2010 EECE596: Copyright since 1999, Sidney Fels5/01/2018 EECE596: Copyright since 1999, Sidney Fels

Haptic Interface Technologies

05/01/2018 EECE596: Copyright since 1999, Sidney Fels

Haptics: Overview

• Haptics: Introduction – tactile – proprioception

• Tactile Interface Technologies • Force Feedback Technologies


Haptics: Introduction

• Sense of touch and kinesthesia – tactile sensing – proprioception – Somatosensory system

• Bidirectional – sense environment

• temperature, vibration, weight, etc. – manipulate environment

• push, pull, pinch, hit, rotate, etc.


Haptics: Tactile Sensing• Can sense:

– texture/vibration – temperature of object or environment – slip detection – surface compliance – elasticity – viscosity – electrical/thermal conductivity – vibration (other than for texture) – initial contact detection – gauging force required for manipulation


Haptics: Tactile I/F

• Critical for performance in many tasks – Massimino and Sheridan, 1993 – tracking (Patrick, Sheridan and Massimino, 1990) – target pointing (Akamatsu, 1994) – degraded visual condition (Massimino and Sheridan,

1993) • See Lederman’s work for many experiments

looking at attributes of tactile sensing • Considered to be critical for virtual environments

– needed for immersion and sense of reality


Haptics: Tactile Sensing• Four main sense organs

– Meissner’s Corpuscles • surface curvature, velocity, local shape, flutter, slip • poor spatial resoln’ • 43%

– Pacinian Corpuscles • vibration, slip, acceleration • 70-1000Hz response frequency range • 13%

– Markel’s Disks • skin curvature, local shape, pressure • 25%

– Ruffini Endings • skin stretch, local force • 19%


Haptics: Tactile Sensing• Skin properties:

– finger pad (Reynier and Hayward, 1993) • spatial resolution, 0.15mm; two-point discrimination 1-3mm • smooth glass feature, 2µm for single dot, 0.06µm for grating,

0.85µm for straight lines • line orientation detection, 8.7mm separation (sequential);

13.1mm separation (simultaneous) • 5.5msec separation of two 1msec stimulti

– increase stimuli duration -> reduce threshold

– Other results available for different properties • strain, texture properties, temperature, etc. • check Burdea and Coiffet (1994)


Haptics: Tactile Actuators


Haptics: Tactile Displays (commercial)• CyberTouch (Virtual Technolgies)

– little vibrators on fingers • 0-200Hz

– $14,000 US • TouchMaster (Exos)

– voice coils on fingers – 210-240Hz

• Tactool System (Xtensory) - sold as kits – pins driven by shape memory alloys

• impulse sensation (30g) • vibration (20Hz) • $1,500

• Displace Temp. Sensing System (CM Research) – thimble – thermoelectric heat pump (Peltier effect) – $10,000


Haptics: Peltier Effect

P - Bismuth N - Telluride


Haptics: CyberTouch


Haptics: Tactile Displays (commercial)

• Intelligent Systems Solutions – air bladders + pnematics – not available anymore

• Braille Displays – Freedom of Speech (85-8 dot cells)

• Games – Playstation controller – fishing, driving, baseball


Haptics: Tactile Displays (research)

• Reading aids – Sherrick, 1984; Shimizu, 1986; Barfield and Furness,

1995 • tools for studying haptics

– Cholewiak and Sherrick, 1981; Schneider, 1988, Lederman, 1999

• tools for rehabilitation – Wise et al., 1990

• tools for teleoperation


Haptics: Tactile Displays (research)• Disney - TeslaTouch

– http://www.ivanpoupyrev.com/projects/teslatouch.php

– https://www.youtube.com/watch?v=zo1n5CyCKr0



• Armstrong Labortatory – 5x6 array tactile stimulator, SMA wires – solenoid attached to Phantom

• Harvard (Howe) – 4x6 array of pins driven by SMA actuators

• 2.1mm spacing, 3mm height, 62msec rise/fall time, BW 6-7Hz • studied palpation,

– shape display (single row) + force feedback display – 3 subjects, 60 trials, tumor finding task, 4mm cylinder inside

foam rubber • error <1mm in 50%; <3mm in 95%; no shape info > 13mm



– voice coils + force feedback device (teleoperation) • investigated 3 types of tasks

– detection of vibration is fundamental goal • worn ball bearning • vibration feedback helps discrimination

– vibrations indicate state of task • piercing plastic membrane • vibration feedback reduces force exerted and increases response

time – vibration not important to task

• peg-in-hole task • no effect, but subjective improvement



• Hokkaido (Ino) – displays for shearing, pressure and temperature – Shear display

• table mounted pneumatic driver – Temp. display



• MIT - Touch Lab (Srinivasan) – linear and planar graspers

• used for psychophysical experiments • use with Phantom

– libraries available for • compliance, viscosity, mass, shape, texture, friction, walls and

corners – created “force shading” for creating feel of smoothly

curved surfaces – what about contact sounds?



• Karlsruhe Research Centre (Germany) – three 24 needle print heads – needles vibrated at 600Hz to simulate contact pressure – actuated by sensors attached to forceps

• Many others: – Queens, (Lederman) - spinning disks – Sandia Labs, 2x3 array of electromagnetic actuators – TiNi Alloy Comp., 5x6 array of tactor pins – U of Salford, UK, vibration with piezo-electric actuator, Peltier

effect heat-pump for temp., pneumatic bladders for contact force, glove mounted


Haptics: Kinesthetic Interfaces

• Awareness of position and movement and forces on body parts

• Force feedback has been shown to be important – Teleoperation

• Hill and Salisbury, 1976; Hanaford, 1989 and Howe, 1992 – Molecular docking

• Ouh-Young, Beard and Brooks, 1989 – Grasping tasks

• Gomez, Burdea and Langrana, 1995 (reduced error and learning time by 50%)


Haptics: Kinesthetic Interfaces• Where is it useful?

– Virtual reality/Augmented reality – medicine

• surgery • diagnosis

– scientific visualization – data manipulation – interactive art – situations where auditory and visual feedback are

limited • aids to disabled • peripheral tasks • 3D manipulation



• Basic idea: 1. Measure movement and forces exerted by user (fingers,

hand, arm, body) 2. Calculate effect of forces on manipulated objects and

resulting forces on user • virtual or real objects

3. Present forces to the user’s fingers, wrist, arms etc. as appropriate



• Devices are either – earth grounded – off-the-body – exoskeletons


Haptics: Kinesthetic Technologies


Haptics: Kinesthetic Technologies

• Other technologies – magnetic levitation – tendons – ???


Haptics: Human Kinesthetic Properties• In addition to cells mentioned we have:

– Golgi endings in joint ligaments • joint torque

– Ruffini type endings in joint capsules • capsule stretch

– Golgi tendon organs • muscle tension

– muscle spindle organs • muscle stretch and rate of change • probably most important for kinesthetic sense

– nothing known for weight or effort • inside central nervous system


Haptics: Human Kinesthetic Properties• Somatosensory system is not symmetric

– force control and perceptual bandwidths differ (Brooks, 1990)

• deliver forces at 5-10Hz • receiving position and force signals >20-30Hz

• JND for force sensing is about 7% • rigid body - 153 to 415 N/cm • force production: 16.5N distal finger to 192.3N

shoulder joint • output resoln’: high at finger tip, low at shoulder


Haptics: Human Grasp Properties• Some force outputs for different types of grasps

– Jacobus et al, 1992


Haptics: Human Grasp Properties

• Current technologies generally change way they work based on grasp types

• several ways to categorize grasps – Schlesinger (1919)

• cylindrical, finger-tip, hook, palmar, spherical, lateral – Napier (1956)

• distinguish between power and precision grasp – MacKenzie (1990s)

• prehensile vs non-prehensile


Haptics: Human Grasp Properties

Cutkosky and Howe, 1990


Haptics: Force-feedback Issues

• Should keep grasping/reflected forces to less than 15% of max (Wiker, 1989) – comfort and fatigue issues – I.e., index finger 7N, middle 6N, ring 4.5N


Haptics: Issues for Creating FF/B Devices

1. Sampling data from user – lag, update rates etc.

2. Computing forces – S/W libraries needed like graphics libraries

3. Presenting forces – calculating control parameters in real-time

• 1000Hz update • Impedance control (force) vs. Compliance control (position)

– overcome inertia


Haptics: Force-F/B Devices (comm)

• Phantom (SensAble Devices, Massey) – thimble on finger

• single point force feedback only – motors and cables and good engineering – can use more than one at a time (if you have $$$)


Haptics: Force-F/B Devices (comm)• Impulse Engine (Immersion Corp.)

• 5 dof – 2 dof pivot – insertion translation – rotation of tool – open-close motion of instrument

• servo-motor actuators • laproscopic and endoscopic simulation



• Laproscopic Impulse Engine



• CyberGrasp (Virtual Technologies) – motor actuated tendons attached to fingers

• 1 dof/finger • used to be flappers on end for contact force

– mounted on CyberGlove


Cyberforce• Exoskelton

– arm forces • Tendons

– grasp forces



• Exos – 4 dof Force Feedback Master

• surgical simulator • hand and arm

– Force Exoskeleton



• SaFiRE (Exos) – exoskeleton for hand

• applies forces to thumb, index finger and wrist • 8 dof: 3 dof to thumb, 3 dof to index, 2 dof to wrist • links grounded to forearm supply 3D Cartesian forces to

fingertips and palm • DC motors used with wires and gears


Haptics: Force-F/B Devices (comm)• Hand Exoskeleton Haptic Display (Exos)

– based on SaFiRE hand exoskeleton • 1 dof to thumb, 2 dof to index • slip display on thumb and index • boom mounted; 2 dof position sensing, 1 dof vertical FF/B


Haptics: Force-F/B Devices (comm)• Per-Force Handcontroller and Finger Forcer

Option (Cybernet Sys. Corp) – 6 dof force feedback joystick – finger force feedback added to top of joystick

• thimbles attached to finger tips



• Haptic Master (Iwata in Tsukuba) – 3D force and torque – three sets of pantograph links


• Interactor (Aura)



Haptics: Force-F/B Devices (research)• Many issues remain to be solved for force

feedback devices – commercial FF/B displays only scratch the surface

• Extensive research on how to use these displays – not a lot of practical uses yet

• Most research effort to develop FF/B displays has come from teleoperation – applies to virtual environments – other effort from psychophysical testing labs – effort to explore application of FF/B displays done in

well funded labs


Haptics: Force-F/B Devices (research)• Ino, Hokkaido

– elbow joint force feedback – metal hydride actuator

• temperature changes in alloy controls pressure of hydrogen gas • pressure converted into propulsion • 300g, cylinder is 20.62 mm, 6g of hydride • lift 10kg to 50mm with vel. of 9mm/sec • noise free, no sudden impact force • similar compliance to elbow joint

– report good results (obviously) • probably not good for hand • probably work for knee and other large joints


Haptics: Force-F/B Devices (research)• MIT (Salisbury)

– continuing with Phantom • changing tip

– creating software • haptic rendering - calculating and generating forces • still point interaction

– contact forces – contact persistence – impedance – some curvature, texture and friction

• want general purpose s/w libraries to allow for haptic rendering

– Interval was working on some of this as well


Haptics: Force-F/B Devices (research)

• Institut des Systemes Intelligents et de Robotique, Université Pierre et Marie Curie (Hayward previously McGill) – 9 dof Stylus - tendon driven single point – Pantograph

• 2 dof translation - 10cm X 10cm • FF/B mouse • found acceleration best for shock and hard contacts • for effective performance need

– wide frequency response – high accuracy in presenting forces – mechanical impedance controllable over wide range – dynamic response up to 50-100Hz



Pantograph

Stylus


Haptics: Force-F/B Devices (research)• Northwestern University (Colgate)

– 4 dof Force Reflecting Manipulandum • translation + rotation in horizontal plane • high impedance control

– stiff springs and hard walls • no singularities in the work space

– study physics based simulation for haptic interfaces


Haptics: Force-F/B Devices (research)• Rutgers (Burdea)

– Rutgers Master • uses pneumatic microcylinders positioned in palm • ends attached to tips of 3 fingers and thumb

– Experiment 1: • manipulate ball without deforming it • compared

– visual f/b • (ball, bar graph and mono/stereo)

– audio f/b – force f/b

• Results – FF/B most significant non-redundant modality – FF/B + audio best redundant combo.



• U. of North Carolina (Brooks) – Argonne Remote Manipulator (ARM) – 6 dof large workspace device

• hand grip display, joint action at shoulder and outward – used mostly for molecular docking



• Others: – Suzuki Motor Corp: SPICE

• 6 dof, mechanical + motors

– Tokyo Inst. Of Tech • single point FF/B • strings and motors



• CCRMA (Gillespie) – piano/keyboard key feedback

• simulate touch of different keyboards • 7 keys, DC motor driven, dynamic simulation of key action

• UBC (Salcudean) – maglev joystick force feedback – motor controlled pantograph with additional dof

• UBC (Maclean) – 1 dof rotary devices

• Cobots (Peshkin and Colgate) – COBOTS

• use mechanical constraints to set up forces


Haptics: Cobots


Haptics: Summary• Need good haptic interfaces for:

– immersive VR – 3D direct manipulation i/f – effective teleoperation – mediated communication

• Very active area of research


Haptics: Summary• Current status of Tactile displays

– limited area (finger tip) – don’t scale up – large physical dimensions - not practical – no software models + psychophysical info available + devices capable of meeting resolution and B/W reqd. in

specific domains + >20-30Hz BW + pin displays are close to human resolution + vibration available at wide range of frequencies

– not high enough for skillful manipulation + temperature displays have sufficient resolution


Haptics: Summary• Force feedback displays

– generally application specific – generally small workspace – encumbering – most use servo motor actuation

• stability problems • backdrivability • friction

– Specs for good force feedback • force resolution: 12 bits, • position resolution: 0.001 in • passive friction less than 1% of max. force • minimum sampling rate: 2000Hz • latency <1msec

– Phantom is closest to meeting all of these for single point


Haptics: Summary– Still can’t represent solid, immovable wall or rigid

objects • instability at solid edge

– add viscosity to whole environment – reduce stiffness - everything is a sponge

• More problems – sensor accuracy – latency of computer – actuator performance – difference between actuator and sensor placement – mechanical transmission difficulties – non-linear system due to dynamics of user characteristics – safety – software models lacking – more psychophysical testing needed


Haptics: Summary

• Currently – no general purpose haptic interfaces – no practical applications in common use

• Future: – robotic graphics? – New actuators? – Non-linear, deformable manipulators

Documents

Haptic Interfaces - Courses