Computational Motor Control: Optimal Control for Deterministic Systems (JAIST summer course)

Computational Motor Control Summer School02: Optimal control for deterministic systems

Hirokazu Tanaka

School of Information Science

Japan Institute of Science and Technology

Optimal control for deterministic (noiseless) systems.

In this lecture, we will learn:

• Invariant laws of body movements

• Calculus of variation

• Lagrange multiplier method

• Karush-Kuhn-Tucker condition

• Pontryagin’s minimum principle

• Boundary problem

• Minimum-Jerk model

• Minimum-torque-change model

• Kinematic vs dynamic planning in reaching movements

Straight paths and bell-shaped velocity in reaching.

Morasso (1981) Exp Brain Res

12

3 4 5

6

joint angles

angular velocity

angular accel.

hand speed

T1->T4 T3->T5 T2->T5 T1->T5

Power laws for curved movements.

Lacquaniti et al. (1983) Acta Psychologica

2

3 1

3v

: angular velocity

: h

: curvature

and speedv

or

curvature

angu

lar

velo

city

Fitts’ law for movement duration in rapid pointing movements.

Fitts (1954) J Exp Psychol

2

2log

f

Dt a b

W

“Main sequence” for saccadic movements

Bahill et al. (1975) Math Biosci

Optimality principle: how a unique trajectory is chosen.

http://en.wikipedia.org/wiki/Snell%27s_law

Snell’s law

Fermat’s principle of least time

1 1 1

2 2 2

sin 1/

sin 1/

v n

v n

Q Q

0 P P

1T dsT s dt n s ds

v s c

Q

P

10T s n s ds

c

Optimality principle: how a unique trajectory is chosen.

Newton’s equation of motion

Hamilton’s principle of least action

Vm

qq

q

2

1

,t

tS dtL q q q

21,

2L V q q q q

0S q

http://en.wikipedia.org/wiki/Principle_of_least_action

Mathematics: Calculus of variation.

0

, ,ft

xS x x L x dt

0

0 0

0

0

, ,

, , ,

f

f f

f

f

t t

t

t t

t

t

L x dt L x dt

L

S x x S x x x x S x x

x

Ldt

x x

L d L Ldt

x dt

x x x

x

x

x

x xx

0d L L

dt x x

0

0

ft t t

L L

xx

xx

S: Action, L: Lagrangian

Euler-Lagrange equation

x t x t x t variation


0

, ,, ,, ,ft

S x dtLx xx xxx x

0 0

0

2 3

2 3

2

0

2

, , , , ,, ,, ,f f

f

f

t t

t

t L d L d L d L

x dt x dt x dt

L d L d Lx

x

S x x x x x xx dtL x dtL x

dt

dt x dt x

x x x x x x

x

x

x

00

f ft t

L d L Lx x

x dt x x

2 3

2 40 0 f

L d L d L d Lt t

x dt x dt x dt x

2

2

000

0

f f ft t t

L d L d L L d L Lx x x

x dt x dt x x dt x x

Lagrangian with higher derivatives

Euler-Poisson equation


2 3

2 30 0 f

L d L d L d Lt t

x dt x dt x dt x

2

2

000

0

f f ft t t

L d L d L L d L Lx x x

x dt x dt x x dt x x

2 3

2 30 0 f

L d L d L d Lt t

x dt x dt x dt x

0 0 00f f ft t t

x x x

Euler-Poisson equation with general boundary conditions

Euler-Poisson equation with fixed boundary conditions

Smoothness criterion: Minimum-jerk model.

2 2

3

30

3

3MJ0

, , , , , , ,ftft d x d y

x x x x y y y y dt dtdt dt

C L

Flash & Hogan (1985) J Neurosci

2 3 3

2 3 6

6

30 2 2

L d L d L d L d d xx

x dt x dt x dt x dt dt

6

6 6

60

d x d y

dt dt

Intuition: The observed movement trajectories are smooth, so “smoothness” of trajectory may be selected in the brain.

: position, : velocity, : acceleration, : jerkx x x x

Smoothness criterion: Minimum-jerk model.


6

6 6

60

d x d y

dt dt

00

f f

x x

x t x

0 0 0

0f f

x x

x t x t

Boundary conditions for a point-to-point movement:

Euler-Lagrange equation:

45

0 0

3

6 15 10f

f f f

t t tx xt x x

t t t

Solution of minimum-jerk trajectory (5th order polynomial)

Smooth trajectory and bell-shaped velocity explained by the model.


speed y accel x accel speed y accel x accel

Via-point movements also explained by the model.


3 4 5

0 1 2

2

3 4 5 10x t a a t a t a t a t a t t t

3 4 5

0 1 2 3 4

2

5 1 fx t a a t a t a t a t a t t tt

00 , 0 0 0x x x x

, 0f f ffx t x x t x t

1 1 1 1 1 1

1 1 1 1 1 1

, , ,

, , .

x t x x t x x t x t

x t x t x t x t x t x t

00x x

1 1x t x

f fx t x

x t

x t

Twelve unknown coefficients can be determined by twelve boundary conditions.

ia

Power law predicted by the minimum-jerk model.

Viviani & Flash (1995) J Exp Psychol

Smoothness criterion: path-constrained Minimum-jerk model.

Huh & Sejnowski (2015) PNAS

Cartesian coordinates Frenet-Serret coordinates

ˆv v t

4 26 ˆ ˆ32 1v z z h z t z n

v v

h

0 0

log , logv

z hv

Smoothness criterion: path-constrained Minimum-jerk model.


4 26 ˆ ˆ32 1v z z h z t z n

v v

h

2

20

d

dzdz z

d

4 6 2 3 4

3

2 2 2

2 2

5 2 30 10 25 82 40

2 14 90 12

20 55 75 15

82 8 22 20 0 129

v z z h h h z z

z h h h h h

z h h z

z z h h zh z z h

Minimum-jerk Lagrangian in Frenet-Serret coordinates:

Derive Euler-Lagrange equation:

Or explicitly:

Derivation of two-thirds power law.


4 6 2 3 4

3

2 2 2

2 2

5 2 30 10 25 82 40

2 14 90 12

20 55 75 15

82 8 22 20 0 129

v z z h h h z z

z h h h h h

z h h z

z z h h zh z z h

0

ae

0

logh a

0

bv ev

0

logv

v bv

Euler-Lagrange equation:

Spiral path:

or

Assume a solution in an exponential form:

or

4 6 2 3 4 3 2 2

0

4 6

4 6

0 5 25 82 40 90 12 55

(constant)

75a b

a b

v e a ab b b a a b a

e

Substituting the exponential forms into the Euler-Lagrange equations:

2

3b a

Model predicts the power law in spiral movements.


0

ae

2

30ev v

therefore, 2/3v

That’s what was found in experiment!

Scaling law with figure-dependent exponents: model prediction and experimental confirmation.


sinh

Optimization with equality constraints: Lagrange multiplier method.

Minimize a function f(x) under a constraint g(x)=0.

0J f g

x x x

0J

g

x

Image source: Wikipedia

,J f g x x x

λ: Lagrange multiplier

Necessary condition:

Karush-Kuhn-Tucker (KKT) condition for constrained optimization.

Minimize a function f(x) under an inequality condition .

,J f g x x x

0

0

0

0

J f g

g

g

x

x

x x

x

Image source: Wikipedia

0g x

λ: Lagrange multiplier

KKT condition:

Optimization under dynamic constraint: Pontryagin’s minimum principle.

,x f x u

0

,ft

fJ g dt u x x u

0 0

0

0

T

T T

T

, , , ( , )

, ,

, ( , )

f f

f

f

t t

t

t

f

f

f

J g dt dt

g dt

dt

x u p x x u p x f x u

x x u p f x u p

x xu p

x

x p

Minimize:

under constraint of EOMs:

T, , , ,g x u p x p ufu xHamiltonian

Optimization under dynamic constraint: Pontryagin’s minimum principle.

0

0

T T

TT T

, , , , , ,

, , , ,f

f

f

f

t

t

f f

T

t

t

x

J J J

dt

dt

x u p

x u p x u p p x

x p ux x p

x u p x u p x u p

x x x

x u p p x

u

p x

p p x

T

,

0

g

p

f

x x x

x f x u

p

u

p

f

f

t t

t

xp

EOMs

Terminal condition

Smoothness criterion: Minimum-torque change model.

T

1 2 1 2 1 2 x

22

2

2 2 2 2

11

1 2 1 11

2 1

11

22

,

,

, ,,

, ,

f

f

u

u

x f x u

0 0

T T1 1

2 2

f ft t

J dt dt τu u uτ

T T1, , ,

2 x u p u u p f x u

State vector

EOMs

Torque-change cost

Uno, Kawato & Suzuki (1989) Biol Cybern



T

T

,

0

x f xp

f

x x

fu

u u

u

p p

p

00 x x

00 p p

Initial condition for x:

Initial condition for p:

p0 must be chosen so that .fft x x



Experiment Model



Experiment Model



Experiment Model

Is movement planning in extrinsic or intrinsic space?

Wolpert et al. (1993) Exp Brain Res

Experiment

Visual perturbation experiment: MJ prediction: adapted path under visual perturbationMTJ prediction: non-adapted path under visual perturbation.

Summary

• Human movements exhibit a variety of invariant features (straight paths, power law, Fitts’ law, main sequence, …).

• Those invariant features are explained in terms of optimality principles.

• There are mathematical methods for solving optimization problems (calculus of variation, Lagrange multiplier methods, Pontryagin’s minimum principle, etc.).

• Smoothness in trajectory or joint torques is one of the most successful criterion for reaching movements.

References

• Morasso, P. (1981). Spatial control of arm movements. Experimental Brain Research, 42(2), 223-227.

• Lacquaniti, F., Terzuolo, C., & Viviani, P. (1983). The law relating the kinematic and figural aspects of drawing movements. Acta Psychologica, 54(1), 115-130.

• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.

• Bahill, A. T., Clark, M. R., & Stark, L. (1975). The main sequence, a tool for studying human eye movements. Mathematical Biosciences, 24(3), 191-204.

• Flash, T., & Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical model. The journal of Neuroscience, 5(7), 1688-1703.

• Viviani, P., & Flash, T. (1995). Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 32.

• Huh, D., & Sejnowski, T. J. (2015). Spectrum of power laws for curved hand movements. Proceedings of the National Academy of Sciences, 112(29), E3950-E3958.

• Uno, Y., Kawato, M., & Suzuki, R. (1989). Formation and control of optimal trajectory in human multijoint arm movement. Biological Cybernetics, 61(2), 89-101.

• Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study. Experimental Brain Research, 103(3), 460-470.

• Flanagan, J. R., & Rao, A. K. (1995). Trajectory adaptation to a nonlinear visuomotor transformation: evidence of motion planning in visually perceived space. Journal of neurophysiology, 74(5), 2174-2178.

Exercise

• Point-to-point minimum-jerk solution: For given initial and final positions, draw a minimum-jerk trajectory (path and velocity).

• Via-point minimum-jerk solution: Find a via-point trajectory by determining the twelve coefficients with given boundary conditions.

• Write a MATLAB code to solve the two-boundary problem of the minimum-torque change model.