Inverse Dynamics Daniel Hennes Maastricht University December 21, 2010

Outline •  Learning Dynamic Tasks •  Gaussian Process Regression •  Learning Inverse Dynamics •  Packages • What’s next?

Learning Dynamic Tasks •  Policy Gradient RL •  Pole balancing and

pendulum swing-up •  Works in simulation •  Problems during

transition to PR2 –  Image pipeline delays – Arm dynamics –  ...

Learning Inverse Dynamics

Applications of Inverse Dynamics •  Feed forward control • Optimal control (ILQR)

D. Mitrovic, S. Klanke, and S. Vijayakumar. Adaptive Optimal Feedback Control with Learned Internal Dynamics Model. 2010.

•  Collision detection A. De Luca et al. Collision Detection and Safe Reaction with the DLR-III

Lightweight Manipulator Arm. 2006 (- 2009).

•  Haptic feedback

Gaussian process regression •  How does it work?

1.  Provide training data 2.  Select kernal function (easy: Gaussian kernel)

3.  Tune hyperparameters (easy: Quasi-Newton minimization)

4.  Inference (complexity O(n3))

5.  Prediciton (complexity O(n))

Motion data

Avg. nMSE performance (GPR)

0

0.05

0.1

0.15

0.2

0.25

circle figure8 reaching sinoid

shoulder_pan

shoulder_li8

upper_arm_roll

elbow_flex

forearm_roll

wrist_flex

wrist_roll

nMSE performance (sinoid motion)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

GPR IDM

shoulder_pan

shoulder_li8

upper_arm_roll

elbow_flex

forearm_roll

wrist_flex

wrist_roll

GPR interpolation

GPR extrapolation

Packages •  : : inverse_dynamics – GPR model – Newton-Euler model (KDL or WBC) – Motion generation + other tools

•  : : dynamics_markers – Joint torque visualization in rviz

: : dynamics_markers

Conclusion •  GPR model – extrapolation is limited – suffers from curse of dimensionality – can still be used for repetitive tasks

•  IDM – performance strongly depends on clean signals

What’s next? •  Future work

– NN or SVR extrapolation performance – Predicting residuals – Dimensionality reduction

•  NLPCA, elastic maps – Online learning

•  Local GPR •  Projection based sampling

–  IDM Identification •  a lot to improve, e.g. modeling friction, estimating inertia

parameters