Cursus Doelgericht Handelen(BPSN33)

R.H. Cuijpers, J.B.J. Smeets and E. Brenner (2004). On the relation between object shape and grasping kinematics. J Neurophysiol, 91: 2598-

2606. R.H. Cuijpers, E. Brenner and J.B.J. Smeets (2006). Grasping reveals visual misjudgements of shape. Exp Brain Res 175:32-44

Topics• 1st hour: Control Variables in Grasping

– Opposing views on visuomotor control– Research question

• 2nd hour: Grasping elliptical cylinders– Real cylinders

• Which positions?• How to get there?

– Virtual cylinders• Constant haptic feedback• Veridical haptic feedback

– If time permits: Modeling grip planning– Conclusions

Control variables in grasping

Many levels of description:• Activity motor neurons• Muscle activity (EMG)• Posture (Joint angles)• Kinetics (Forces, torques)• Kinematics (Position, speed

etc.)• Task level

Degrees of Freedom

(DoF)

low

high

Control variables in grasping

How does the brain ‘plan’/compute the desired motor neuron output?

• If movements are planned in task space:– little computational power needed for planning stage

• But …– Need to solve DoF-problem (Motor primitives)– Cannot control everything (Stereotypic movements)– Need low-level on-line control (e.g. stiffness control)

Control variables in grasping

• What is/are the correct level(s) of description for movement planning and visuomotor control?

Method of research in visuomotor control:• Manipulate visual information / haptic feedback /

proprioceptive feedback• Measure effect on motor output

• Variables that have an effect are ‘controlled’• Variables that have no effect are redundant

Haptic = by touch Proprioceptor = sensory receptor in muscles, tendons or joints

Opposing views on visuomotor control

Fingertip positions and object size• Milner & Goodale: perception vs. action• Franz et al: common source model• Smeets & Brenner: position vs. size

Fingertip positions and object orientation• Glover & Dixon: planning vs. on-line control• Smeets & Brenner: position vs. orientation

perception vs. action

Goodale (1993); Milner, Goodale (1993)

• RV: lesions in occipito-parietal cortex (dorsal).

• DF: damage in ventrolateral occipital areas due to CO poisoning.

Grasping DiscriminationRV DF

Occipito-Parietal

Ventrolateral oocipital

perception vs. action

• Dorsal pathway for guiding movements (should be veridical)

• Ventral pathway for perception (perception of shape, colour etc.)

Dorsal = Action

Ventral = Perception

perception vs. action

Agliotti, De Souza, Goodale (1995):

• Grip aperture NOT influenced by size-illusion.

• Due to separate processing of information for perception and action.

Common source model

• Franz et al (2000): equal effects of illusion

Position vs. size

Brenner, Smeets (1996):

• Size-illusion does not affect grip aperture, but does affect the initial lifting force.

• Explanation: not size information is used but position information. They are inconsistent.

Planning vs. on-line control

Glover & Dixon (2001)

• Relative effect of illusion decreases with time

Illusion mainly affects planning

Position vs. orientation

Smeets et al. (2002)• Assumption:

illusion affects orientation, not position

• Also explains data of Glover and Dixon

Research Question: How is shape information used for

grasping?

• The visually perceived shape is deformed• Shape (ventral) determines where it is best

to grasp an object (dorsal)– Grip locations not veridical

• Shape information could be used during planning (ventral) or on-line control (dorsal)– Grip errors arise early or late in the movement

Grasping elliptical cylinders:real cylinders

Experimental design

• seven 10cm tall cylinders

• elliptical circumference with fixed 5cm axis

• variable axis: 2, 3, 4, 5, 6, 7 and 8 cm

Experimental Design

Experimental design• Optotrak recorded traces of fingertips• 2 distances x 7 shapes x 6 orientations = 84 trials• 3 repetitions• 10 subjects

Experimental Design

Example

Which positions?

• Geometry: grasping is stable at principle axes

Which positions?• Principle axes preferred. But systematic errors…

Which positions?• Systematic "errors" depending on orientation.

• Scaling grip orientation 0.7 except for aspect ratios close to 1, 0.5

Which positions?

Scaling grip orientation = slope + 1

Comfortable grip

Prediction:

Slope a = w-1

Offset b = -(w-1)0

0)1()1( ww 0)1( ww

0ab

Suppose: grip orientation = mixture between cylinder orientation + comfortable grip

Thus …

• Subjects grasp principle axes, but make systematic errors

• Cannot be explained by comfort of posture

• Additional effect of deformation of perceived shape

How to get there?

How to get there?

How to get there?

•High correlation despite errors!

•Sudden drop at end: Grip aperture automatically corrected

•Correlation much higher for max. grip aperture than final grip aperture

•Gradual increase: grip errors were planned that way

Thus …

• Systematic errors already present in the planning of the movement

• Maximum Grip Aperture reflects planned size rather than true size

Grasping virtual cylinders

Experimental design

Experimental Design

Experimental design

Experimental design

• Constant haptic feedback:– Real cylinder is always circular– Virtual cylinders: 15 aspect ratios, 3 orientations

• Veridical haptic feedback:– Virtual and real cylinders are the same,

7 aspect ratios and 2 orientations

Constant haptic feedback

• Only half of the subjects scale their grip orientation

• If they do, the scaling of grip orientation is similar to real objects (0.42)

Constant haptic feedback

• Subjects hardly scale their max. grip aperture

• Scaling of max. grip aperture is much smaller than for real objects (0.14 instead of 0.57)

Thus

• Inconsistent haptic feedback reduces scaling gains

Possible cause:• All subjects scale their grip aperture based

on the felt size • Scaling of grip orientation based on seen

orientation for only half of the subjects, and the felt orientation for the other half

Veridical haptic feedback

• Similar pattern of grip orientations for all subjects

• Scaling of grip orientation (0.58) close to those for real objects (0.60)

Veridical haptic feedback

• All subjects adjust their maximum grip aperture

• Scaling of max. grip aperture (0.39) much higher and closer to real objects (0.57)

Thus

With consistent haptic feedback

• Scalings of grip orientation and grip aperture close to those for real cylinders

• Less variability between subjects

Comparison of experiments

Real Cylinders

Consistent Feedback

Inconsistent Feedback

Thus

• Natural grasping of virtual cylinders requires veridical haptic feedback

• Grip orientation and grip aperture can be scaled independently

Modeling grip planning

Modeling grip planning

• Physical constraints– Grip force through centre of mass– Grip force perpendicular to surface– Optimal grip along major or minor axis

• Biomechanical constraints– For a given cylinder location there is a most

comfortable grip– Evident when grasping circular cylinder

Modeling grip planning

• Assumptions:– The planned grip orientation is a weighted

average of the optimal and the comfortable grip orientation

– The weights follow from the expected cost functions for comfort and mechanical stability

Modeling grip planning

maximal is 0)(

)|()(

*)(

))(1()(

)(22

EGEG

dpG

ondistributiGEG

ww

GGG

planned

planned

planned

planned

comfort

comfortoptimal

comfortplanned ww )1(

If

Then (required)

Modeling grip planning

• Perceptual errors change the perceived cylinder orientation

• The comfortable posture may also be uncertain

)|ˆ(~ˆ p

)|ˆ(~ˆ0000 p

Modeling grip planningIf distributions are Gaussian with zero mean, we get:

For the circular cylinder w=0, so that:

22

22222

))(1(

)1(var

)1(

comfortmotor

motor

comfortplanned

wwMEG

ww

wwE

comfort

22varcomfortmotorcircular

comfortcircularE

Modeling grip planning

• Each grip axis may be grasped in different modes:

• Model predicts probability of each mode

kk 21

axismajor

Modeling grip planning

• The model describes the relative costs for grip comfort and mechanical stability

• It predicts the relative probability of choosing the major or minor axis

• We can incorporate biases in the perceived cylinder orientation

• We can extend to more general shapes

Conclusions

• Subjects plan their grasps to suboptimal locations based on the perceived shape and the anticipated (dis)comfort

• Upon touching the surface the errors are corrected• Haptic feedback is necessary for natural grasping• With our model we can identify relative

contributions of comfort, stability and perceptual errors

Conclusions

• Visual shape information (slant, curvature) is used for planning suitable grip locations (position information)– Perceptual bias– Bias due to comfort of posture

• No substantial on-line corrections On-line control uses position information

• When inconsistent, haptic and visual shape information is combined differently for the planning of grip aperture and grip orientation

The end

Stable grip of an ellipse