Implementation of Arbitrary Path Constraints using Dissipative Passive Haptic Displays

Implementation of Arbitrary Path Constraints using Dissipative Passive Haptic Displays

Davin K. Swanson

PhD Defense

George W. Woodruff School of Mechanical Engineering

April 2, 2003

Committee: Wayne Book, ME, Chair

Tom Kurfess, ME

Kok-Meng Lee, ME

Julie Jacko, ISyE

Chris Shaw, CoC

Swanson PhD Defense – April 2, 2003

Haptic Displays

• Definition: a physical man-machine interface which interacts with a user’s sense of touch

• Types of haptic effects– Kinesthetic: movement of hands,

limbs; point forces and torques

– Tactile: fine touch; texture, temperature

Introduction


Energetically Active Haptic Displays

• Most haptic displays are active– Electric motors

– Pneumatics

– Hydraulics

– Voice coils

• Advantages of active devices– May generate wide array of control efforts, haptic effects

– Amplification of human effort

– Rich control literature

• Disadvantages of active devices– Machine failure or instability can lead to uncommanded motion

– High forces may cause injury

– Delicate environments may be damaged

Introduction


Energetically Passive Haptic Displays

• Passive displays may only dissipate, redirect, store energy– Brakes, clutches, dampers (dissipative)

– Continuously variable transmissions / CVTs (steerable)

• All motive energy comes from user

• Advantages of passive devices– Safety

– Better acceptance by some operators (surgeons, astronauts)

• Disadvantages of passive devices– Limited by passive constraint

– May not generate arbitrary control efforts

– Difficult to control; conventional controls not always suitable

Introduction


Applications of Haptic Displays

• Teleoperation – force-reflective masters

• Virtual reality

• Synergistic devices– Direct contact between payload/tool, user, interface

– Example: cooperative manipulation

Payload

Human Userguides payload

Synergistic Manipulatorsupports payload,constrains motion

Introduction

indirect coupling between user and

environment


Passive Haptics as Synergistic Devices

• Passive devices are attractive for synergistic applications due to safety advantages

• Tasks required of synergistic devices:

Suitability to task

Task Dissipative Steerable

Free motion Excellent Average

Gravity compensation Below Average Below Average

Path following Average Excellent

Obstacle avoidance Good Excellent

Haptic effects Good Poor

Investigated previously by Swanson, Book

Focus of this work

Introduction


Goals of this Research

• Implementing path constraints is a weakness of dissipative devices (compared to steerable)

• How well can dissipative devices perform this task?

• How to fully evaluate performance?

• Goals:– Develop control methodologies to implement path following on dissipative

passive devices

– Generate performance measurements to evaluate these controllers

– Use human subject testing to evaluate these controllers

– Correlate physical measurements with qualitative user opinion

Introduction


Overview of Presentation

• Background

• Controller Development

• Experimental Testbed

• Human Subject Testing – Design of Experiments

• Human Subject Testing – Data Analysis

• Conclusions

Overview


Existing Passive Haptic Devices

• PTER – Passive Trajectory Enhancing Robot– Charles, Book

– 2 DOF

– 2 dissipative, 2 coupling actuators

– Used in this work

Background

• Cobots– Colgate, Peshkin, et.al.

– Steerable devices

– Use CVTs or steerable casters

PTER

“Scooter”



• PADyC – Passive Arm with Dynamic Constraints– Troccaz, et.al.

– Overrunning clutches limit velocities

• Large workspace brake-actuated device– Matsuoka, Miller

– 3 DOF (2 rotational, 1 prismatic)

– particle brakes

PADyC

Background



• Florida 6 DOF hand manipulator– Will, Crane, Adsit

– Particle brakes

• PALM-V2

– Tajima, Fujie, Kanade

– Variable dampers

• That’s about it…

Background


Control of Dissipative Devices

• PTER path following control (Davis, Gomes, Book)– Modified impedance controller

– Velocity controller; computed desired forces

• PTER obstacle avoidance (Swanson, Book)– Gomes velocity controller

– Single degree-of-freedom (SDOF) control; selective actuator locking

• PALM-V2

– Change damping to control velocity

– Does not deal with sign differences between actual, desired velocity

• Brake-actuated lower body orthosis (Goldfarb, Durfee)– Power comes from stimulated muscle contraction

– PD / adaptive control of position and velocity

– Applied force will always be in direction of desired velocity

Background



• PADyC– Free motion, position constraint, region constraint

– Trajectory constraint

• Only velocity limits may be controlled

• Define “box” of possible future endpoint positions

• Velocity limits alter shape, size of box

• Large-scale 3 DOF display (Matsuoka, Miller)– Viscous fields

– Stiffness modeling

– Virtual walls (similar to SDOF control)

Background



• Very limited previous work in path-following control of dissipative interfaces– PALM-V2 does not address situations where force and velocity signs differ

– Controlled brake orthosis always has force and desired velocity of same sign

– PADyC has unique actuators (velocity magnitude constraints)

• No directed work at providing path-following control for:– Arbitrary path shapes

– Unknown external motive forces

– Dissipative passive haptic displays

• The door is wide open!

Background



• Background





• Conclusions

Overview


Path Following Control

• Goal: Allow user free motion along an arbitrary path while preventing motion orthogonal to that path

• Conventional control methods– Assume active device

– Typically calculate forces / torques to be applied

– Example: impedance control

Controller Development


Velocity Field Control

• Choice of high level controller

• Control velocities rather than forces / torques

• “Passive VFC” used by Li, Horowitz to control active manipulators

• Define velocity field based on desired path

• Low-level controller deals with achieving desired velocity

• Velocity direction controlled, magnitude left to the user



Low Level Controllers

• Form bulk of control work

• Must drive link velocities towards desired velocity specified by velocity field

• Three control concepts:

– Velocity ratio control

– Velocity ratio control with coupling elements

– Optimal controller


0,limt

txxxxd


Velocity Ratio Controller

• Desired velocity may be transformed into link-space

• Magnitude is unimportant… direction should be controlled

• Control velocity ratios– Reduces controlled DOF by one

– Makes sense! User has control of DOF along desired path

dd x 1 J


11

3

1

2 , ,

n



• Compute ratio vector

• Members represent amount each link must slow down– Lower number means more deceleration required

– Negative number means direction change is necessary

niccci

dii

Tn

1 , where1




• Normalize the ratio vector by largest positive member

• Goal of controller: guide system towards populated with all ones

• Special case: no positive elements in– All axes must change direction

– Solution: immobilize device

• Use to generate control law

)max(c

ccn

nc

c

nc

ncu 1



Velocity Ratio with Coupling Elements

• Some interfaces may contain both dissipative and steerable elements

• 2 DOF testbed used in this work– Two purely dissipative actuators

– Two dissipative/coupling actuators

– Allows for greater control flexibility

• If coupling actuators are feasible, they are preferred

• Strategy– Use a coupling actuator if feasible

– Otherwise, fall back to standard velocity ratio controller



Velocity Ratio with Coupling Elements

• Scale desired velocity for kinetic energy equivalence

• Generate vector of signs of required accelerations

• Compute matrix which represents effect of each actuator on each link velocity (-1, 0, or 1)

• If any row of equals , the actuator represented by that row will be used

• Otherwise, fall back on velocity ratio controller

ddeds TTs ,, such that

niidii 1 ,sgn

nm

T



Optimal Controller

• In previous controller, dissipative and coupling elements separated

• Use optimal control– Single control law dealing with both types of actuators

– Often used to control “overactuated” systems

• Minimize a cost function

• Normally done offline to compute gains or control law– Dissipative haptic interfaces have serious nonlinearities

– Signs of control efforts dependent on signs of link velocities

• Perform minimization at every time step– States considered constant

– Nonlinearities fall out



Optimal Controller

• Optimization at each timestep– System is linear– If linear cost function is chosen, linear programming can be used– Fast, accurate, achievable

• Goals of cost function– Drive system towards desired velocity

• Primary goal of controller– Minimize energy loss

• Secondary goal to favor coupling elements

• Constraints– EOM of system– Actuator limits



Optimal Controller – CF Elements

• Velocity control element– Controller must be free to deviate from desired velocity direction

– Set of optimal inputs are control efforts and “optimal” desired velocities

– Minimize angle between desired velocity and “optimal” desired velocity

– To make it linear, maximize the numerator

onomnnno uuuuu 111

odd

odd

acos

odd



Optimal Controller – CF Elements

• Energy element– Minimize the reduction in kinetic energy

– Use negative time derivative as member in the cost function

– Simple, effective way to favor the coupling actuators

– Use “optimal” desired velocity and actual velocity to estimate link accelerations

• Final cost function

Tccf T




• Background





• Conclusions

Overview


PTER – Experimental Testbed

• PTER – Passive Trajectory Enhancing Robot

Experimental Testbed

A

B

C

D

1

2

3

4


PTER – Experimental Testbed

• Five-bar linkage; two DOF

• Actuators: electromagnetic friction brakes– Two dissipative (1, 2)

– Two dissipative/coupling (3, 4)

• PWM power supplies

• 6-axis force/torque sensor on handle

• Digital encoders (50,000 count/rev)


A

B

C

D

1

2

3

4


PTER – Dynamics and Clutch Effects


A

B

C

D

1

2

3

4

BAA

BAB

B

A

BA

BA

B

A

cos370227.0

cos370227.0

85984.1sin370227.0

sin370227.002283.4

2

2

BIN,432

AIN,431

B

A


PTER – Control Software

• Pentium II/450 with Servo-to-Go 8-axis interface card

• QNX RTOS v6.1

• Serial port for force sensor

• 500 Hz update rate

• Link velocities computed from encoder measurements– Backwards difference + 25 Hz 4th order digital Butterworth filter


Position Unfiltered Velocity EstimateFiltered Velocity Estimate


PTER – Controller Verification

• Proof-of-concept tests of the three control concepts

• Desired path: line at y=0.6 m

• Starting point: (-0.1, 0.8)

• Force applied by hand, roughly in (3, -1) direction

• 5cm “buffer distance”

-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.60.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

X position (m)

Y p

ositi

on (

m)

Desired PathStarting PointApplied Force



PTER – Controller Verification

• Two actuation smoothing routines; used to improve feel

• Low velocity smoothing– Reduces chattering due to velocity sign changes

– Velocity limit = 0.11 rad/s

• Velocity direction error smoothing– Reduces chattering due to switching sides of the desired velocity vector

– Angle limit = 0.10 rad

LiL

i

ii uu

when

bb

d

d

uu

when

acos



PTER – Velocity Field Controller



PTER – VF Controller w/Coupling Elements



PTER – Optimal Controller




• Background





• Conclusions

Overview


Motivation for Human Subject Testing

• Controller evaluation– Any haptic device has a human in the control loop

– Human is very difficult to model

– Comprehensive evaluation of controllers requires human subjects

• Quantitative measurement of user opinion– User opinion important part of device operation

– Typically requires multiple subjects, survey questions

– Physical measurements are more accessible, predictable

– Correlate survey responses with measured physical data

Human Subject Testing – Design of Experiments


Experimental Design

• Task: point-to-point motion while following path

– User instructed to move from start box to end box:

• As quickly as possible

• While following path


Focus more on speed


Template Design

• Four templates representing different paths, areas of workspace



Experimental Setup

• Templates plotted full-scale

• Locating board positioned on floor

• Laser pointer provides visual feedback to user

• Three locating pins to position templates

• For each condition, user performs task six times– First 2 trials of each condition are practice

• Data file recorded for each trial



Experimental Conditions

• Four templates

• Nine control configurations– No control

– Velocity ratio controller – low and high gains

– Velocity ratio controller w/coupling elements – low and high gains

– Optimal controller with no force input – low and high gains

– Optimal controller with force input – low and high gains

• Each subject uses all 36 combinations of conditions– Four templates presented in random order

– For each template, nine control setups presented in random order



Recorded Data

• Physical data recorded for each trial– Positions

– Endpoint forces

– Actuator commands

• Survey questions after each condition– NASA Task Load Index (TLX)

– User ranks components of workload on 0-20 scale

• Physical Demand (PD)

• Mental Demand (MD)

• Temporal Demand (TD)

– Weighted combination of these used to calculate total workload

• Weights based on subjects’ opinions of importance of each component

– “Smoothness” component added (not used in workload computation)


• Effort (E)

• Performance (P)

• Frustration (F)



• Background





• Conclusions

Overview


Collected Data

• Nine total subjects– Three female, six male

– Eight right-handed, one left-handed

– Age: 19 – early 30s

• 1292 total analyzed trials– Nine subjects

– Four templates

– Nine conditions

– Four trials per condition

• One set of four trials corrupted – not used

Human Subject Testing – Data Analysis

-0.1 0 0.1 0.2 0.3 0.4 0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

PTER - Endpoint Trace - S11 X16 C07 T23 num004

X (m)

Y (

m)

Desired PathActual Path


Physical Measurements

• Path-average path error– Accuracy

• Average desired-path velocity– Velocity estimated with six-step

balanced difference + smoothing filter– Speed

• Time-average endpoint force– Effort / fatigue

• Endpoint acceleration FFT sum– Smoothness

1

2

1 N

iii

pa L

Le

1tt

L

t

Lv

N

p

c

pa

N

iyixia ff

Nf

1

221

1

0

1

0

22

whereN

iik

k

kf

Njik

N

eaAN

Ah



Statistical Methods

• Compute sample means of data by group

• Compute confidence intervals based on standard error– 95% C.I.

• Compare confidence intervals to determine whether population means of different groups are different

kn

iki

kk Mkx

nx

1

1for 1

k

nk

k

nkk

k

n

i kki

n

stx

n

stx

n

xxs

kk

k

1,1,

1

2

22 ,

1



Controllers – Path Error

• All controlled cases better than uncontrolled– VCLo better with a

90% C.I.

• High gains better than low gains, except for optimal controllers

• All optimal similar to VLo and VCLo NoCon VLo VHi VCLo VCHi OnFLo OnFHi OFLo OFHi

6

6.5

7

7.5

8

8.5

9

9.5

10x 10

-3

Controller

Me

an

Av

era

ge

Pa

th E

rro

r (m

)

Mean Average Path Error vs. Controller with 95% CI



Controllers – Path Error

0.1 0.2 0.3 0.4 0.5 0.6

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Start

End

Endpoint Trace - All trials - Template 22 - Uncontrolled

X (m)

Y (

m)

0.1 0.2 0.3 0.4 0.5 0.6

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Start

End

Endpoint Trace - All trials - Template 22 - OnFHi Controller

X (m)

Y (

m)



Controllers – Path Speed

• VHi and VCHi slower than all other conditions

• Other controllers’ speeds similar to uncontrolled case

NoCon VLo VHi VCLo VCHi OnFLo OnFHi OFLo OFHi0.15

0.2

0.25

0.3

0.35

0.4

Controller

Me

an

Av

era

ge

Pa

th S

pe

ed

(m

/s)

Mean Average Path Speed vs. Controller with 95% CI



Controllers – Tip Force

• Non controlled case lowest

• VHi and VCHi significantly higher

• All others slightly higher than uncontrolled

NoCon VLo VHi VCLo VCHi OnFLo OnFHi OFLo OFHi10

15

20

25

30

35

Controller

Me

an

Av

era

ge

Tip

Fo

rce

(N

)

Mean Average Tip Force vs. Controller with 95% CI



Controllers – High and Low Gain Cases

• Gain makes a big difference in velocity ratio controllers

• Gain does NOT make a big difference in optimal controllers

• Why?

• Gains tuned by hand to have similar “feel” across same-gain controllers

• One subject used for this tuning

• Not an ideal way to adjust gains for accuracy / feel trade-off

• If optimal controller high gains were set even higher, difference between high and low gain conditions would be seen



Survey Data – Total Workload

• Average tip force shows best correlation

• Strong linear trend

1 2 3 4 5 6 7 8 9 10 11 12 13 14 1610

15

20

25

30

35

Total W orkload (0-20)

Me

an

Av

era

ge

Tip

Fo

rce

(N

)

Mean Average Tip Force vs. W orkload with 95% CI



Survey Data – Total Workload

• Secondary influences?

• No trend with path error

• Downward trend with path speed– Likely a secondary effect

– Higher endpoint forces = lower path speed

1 2 3 4 5 6 7 8 9 10 11 12 13 14 165

5.5

6

6.5

7

7.5

8

8.5

9

9.5

10x 10

-3


Me

an

Av

era

ge

Pa

th E

rro

r (m

)

Mean Average Path Error vs. W orkload with 95% CI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 160

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5


Me

an

Av

era

ge

Pa

th S

pe

ed

(m

/s)

Mean Average Path Speed vs. W orkload with 95% CI



Survey Data - Smoothness

• TipaAcceleration FFT sum showed strongest correlation

• Very strong, linear downward trend

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

2000

4000

6000

8000

10000

12000

14000

16000

Smoothness (0-20)

Me

an

Av

era

ge

Tip

Ac

ce

l F

FT

In

teg

ral

Mean Tip Accel FFT Integral vs. Smoothness with 95% CI



Survey Data by Controller

• Workload vs. Controller– VHi and VCHi significantly

higher

– No difference between other controllers and non-controlled case

• Smoothness vs. Controller– VHi and VCHi significantly

lower

– Uncontrolled case very high

– Other cases similar


5

6

7

8

9

10

11

Controller

To

tal

Wo

rklo

ad

(0

-20

)

W orkload Rating vs. Controller with 95% CI


6

8

10

12

14

16

18

Controller

Sm

oo

thn

es

s (

0-2

0)

Smoothness Rating vs. Controller with 95% CI




• Background





• Conclusions

Overview


Conclusions – Controller Development

• Three path following controllers proposed…

• All shown to work on experimental testbed

• May be applied to any dissipative display with or without coupling elements

• Velocity ratio controllers do not require a dynamic model

• Optimal controllers require dynamic model

Conclusions


Conclusions – Controller Performance

• Controlled cases result in better path-following performance– Higher path following accuracy– Same average speed– No change in workload– Higher endpoint forces

• Differences between controllers are slight– Use of coupling actuators not significant– Likely due to nature of task

• User aware of desired path• User attempting to follow path

– Proof-of-concept test shows use of coupling actuators better when input force and desired velocity are dissimilar

• Gain tradeoffs– High = better accuracy, slower, higher forces– Low = reduced accuracy, faster, lower forces

Conclusions


Conclusions – Survey Metrics

• Two physical measurements with very strong correlation to survey data– Total workload: average tip force

– Smoothness: tip acceleration FFT sum

• Controller effects– Workload not effected with low gain controllers compared to

uncontrolled case

– Low gain controllers resulted in lower smoothness compared to uncontrolled case

– High gain velocity ratio controllers had high workload, low smoothness

Conclusions


Contributions of this Work

• Three arbitrary path-following controllers which may be applied to any dissipative passive haptic display with or without coupling elements

• Set of performance metrics to evaluate such controllers

• Set of physical metrics which may be used to measure or predict user opinion about perceived workload and smoothness

• Human subject testing framework for evaluation of path-following haptic displays

Conclusions


Future Directions

• Application of controllers to different dissipative devices– Higher numbers of degrees of freedom

– Active device could be used to simulate passive actuators, virtual coupling elements

• Application of controllers to different tasks– Surface simulation / virtual walls

– Obstacle avoidance (expand on previous work – SDOF controller)

• Evaluate workload and smoothness measurements with other tasks– Surface exploration

– Impedance simulation

– Teleoperation

• Improvement of optimal controller– Nonlinear optimization

– Other terms in cost function (perhaps based on workload/smoothness?)

• Determine if physical demand still primary source of workload on smaller interfaces

• Investigate use of coupling actuators in other tasks

Conclusions


Questions?


Extra slides


Classes of Passive Displays

• Dissipative– Remove energy from system

– Resist motion of the device

– Focus of this work

• Steerable– Constrain one or more DOF

– Kinematic DOF < Workspace DOF

• Hybrid– Contains both types of elements

– Typically one type is dominant

– Addressed in this work

Introduction


Applications of Synergistic Devices

• 6 DOF version of PADyC for surgical tool

positioning

• Automobile assembly– Many active applications

– Scooter cobot

Background


Applications of Synergistic Devices

• Active surgical robots

• Kazerooni material handling systems

• USAF active munitions handler

• Human-robot load sharing

Background

Documents

Implementation of Arbitrary Path Constraints using Dissipative Passive Haptic Displays