Cyber-Physical Systems with Humans in

the Loop

Ruzena BajcsyElectrical Engineering & Computer Sciences

University of California, Berkeley

April 2014

Motivation

• Humans interact with cyber-physical systems on a daily basis

• Scenarios span many levels of complexity, from simple, task

specific communication to complex dynamic interactions

• We wish to understand the mechanics of these interactions

and to design algorithms for control-sharing between humans

and autonomous systems

• We will illustrate with example applications in a number of

domains

Why is this problem important?

• A person is a complex kinematic/dynamic system with many

degrees of freedom and parameters which vary from person to

person

• Not all degrees of freedom are used in all activities

Central question: what is the appropriate representation of human

physical actions for a specific application?

• Is there a systematic way to detect and classify what the

appropriate sparse representation is for a given class of activities?

• What features should be used?

• What should be the generalized coordinates of this system?

Human Model Classes

Musculoskeletal

Kinematic/Dynamic

Kinematic

InteractionAgent

Interaction

Individualized Driver Modeling

Human Model Classes: Agent Interaction

Musculoskeletal

Kinematic/Dynamic

Kinematic

Agent Interaction

Goal

Model human interaction with a system

using a simple, application-specific set of

observed variables

Application

Application: Individualized Driver Modeling

Big Questions:

• If we have an idea of the state of the driver, can we develop more intelligent active safety systems?

• How do we guarantee safe interaction between a driver and an autonomous car that can be trusted?

• How will the autonomous car handle a heterogeneous environment?

• What exactly needs to be known by the autonomous vehicle to make correct, informed decisions?

Human-in-the-Loop Experiments

• Monitor the driver for improved

active safety systems

• Develop smarter active safety systems

that rely on driver monitoring to

predict how humans will behave

• Collect data on driver control inputs,

vehicle trajectories

• Goal: Set up realistic experimental

platform to collect realistic driver

data in dynamic environments

Katherine Driggs-Campbell, Guillaume Bellegarda, Victor Shia, S. Shankar Sastry, Ruzena Bajcsy. Experimental Design for Human-in-the-Loop

Driving Simulations. arXiv:1401.5039

Driver Modeling Motivation

Idea: Reachable set prediction based on driver state

• Current active safety systems consider the entire reachable set of the

vehicle

– If an obstacle intersects, the system will intervene by braking

• Issues:

– Only works at low-speeds and for a short time horizon

– Doesn’t utilize the human intent (or driver state)

• By utilizing likely human behavior, we provide a useful and

informative reachable set of the human-in-the-loop system

K. Campbell, V. Shia, R. Vasudevan and R. Bajcsy, “Probabilistic Driver Models for Semiautonomous Vehicles,” in Digital Signal Processing for In-

Vehicle Systems, 2013.

V. Shia, Y. Gao, R. Vasudevan, K. Campbell, T. Lin, F. Borrelli and R. Bajcsy, “Driver Modeling for Real-Time Semi-Autonomous Vehicular Control,”

IEEE Transactions on Intelligent Transportation Systems, to appear.

Driver Modeling Method

Predict Future Behaviors

• Driver inputs:

• Steering, Throttle,

Braking

• Long time horizon

trajectory sets

Outside Environment

• Future road

• Obstacle data

• Obtained from CarSim

Driver State

• Position/orientation of

hands and head

• Obtained from MS Kinect

Driver Modeling Results

No Driver Model With Driver Model

Att

en

tiv

eD

istr

act

ed

Intelligent Transportation Systems

Idea: Human-inspired autonomous driving

• High-level decision making for an autonomous system

integrates:

– Sensor Fusion — V2V Communication

– Artificial Intelligence —Hybrid Control Theory

Human Model Classes: Kinematic

Musculoskeletal

Kinematic/Dynamic

Kinematic

InteractionAgent

Interaction

Bio-Inspired Action

Recognition

Goal

Model human motion using a rigid-body

kinematic model

Applications

Automated Coaching/

Quantitative Outcome

Measures

Application: Action Recognition

Feature Selection

• Different actions require humans to engage different joints of

the skeleton at different intensity (energy) levels at different

times

• Ordering of joints based on their level of engagement across

time should reveal significant information about the

underlying dynamics - the invariant temporal structure of the

action itself

Jumping Punching

Ferda Ofli, Rizwan Chaudhry, Gregorij Kurillo, René Vidal, and Ruzena Bajcsy. "Sequence of the Most Informative Joints (SMIJ): A New

Representation for Human Skeletal Action Recognition“ Journal of Visual Communication and Image Representation (JVCI). 2013.

People performing the same activity use different styles. This

represents a challenge for action recognition

Variation Across People

Ordering of Most Significant Joints

Feature Selection

1. Convert the time series: 3D position into joint angles

2. Partition the action sequence into a number of congruent

segments

3. Compute the variance of the joint angle time series of each

joint over each temporal segment

4. Rank order the joints within each segment based on the

variance in descending order

Most Important Features

EverybodyJump.mp4

Ordering of most active/engaged joints:

Most Informative Joints

Idea: Represent actions using the Sequence of Most Informative

Joints (SMIJ)

Evaluation of Kinematic Representation

• Results suggest there is a sparse basis of features given

different activities

• SMIJ representation does not suffer from database bias as

much as other reference representations

Ferda Ofli, Rizwan Chaudhry, Gregorij Kurillo, René Vidal, and Ruzena Bajcsy. "Sequence of the Most Informative Joints (SMIJ): A New

Representation for Human Skeletal Action Recognition“ Journal of Visual Communication and Image Representation (JVCI). 2013.

Application: Quantitative Outcome Measures

• Applicable to Muscular Dystrophy studies, assessment of

physical abilities of elderly

• Development of non-invasive, scalable system for motion and

function assessment locally and remotely

• Takes advantage of off-the-shelf technologies, with potential to

bring them to patients’ homes

III (I)

IV (II) II (IV)

I (III)

R

x

zx

z

L

Video

3D Data

Kinematics

Microsoft Kinect

3D Camera

Standardized Video Instructions

Standardized Movement Protocol

Data Processing

Database Storage

3D Visualization

Muscular Dystrophy: collaboration with UC Davis Medical Center

Quantitative assessment of elderly: Oregon Health and Science University

Reachable Workspace (RW)

• Reachable workspace is well-

established in robotics to

graphically represent the

boundaries of the working volume

• Method of obtaining RW is data-

driven in our case

• RW is quantified by relative

surface area - other features are

possible

• The analysis of RW is based on 3D

kinematic trajectory input

• The input can be generated by

Kinect, motion capture, or other

motion sensing devices

Determination of the reachable workspace

G. Kurillo, J.J. Han, R.T. Abresch, A. Nicorici, P. Yan and R. Bajcsy, "Development and Application of Stereo Camera-Based Upper Extremity

Workspace Evaluation in Patients with Neuromuscular Diseases", PLOS ONE journal, September 2012.

Standardized Movement Protocol

• The movement protocol includes

cardinal movements of the shoulder in

vertical and horizontal directions

• Video-feedback is applied for

standardized performance

• 3D feedback obtained from Kinect data

assists user as a virtual mirrorHealthy Subject

Subject w/Advanced Muscular Dystrophy

Accuracy Evaluation

• We compared Kinect-based assessment of reachable workspace

alongside a commercial motion capture system (Impulse,

PhaseSpace inc.)

• 10 healthy subjects, 3 repetitions per trial

• The mean difference between the two systems was -0.03 in RSA

• Results show high agreement between consecutive trials with CC

of 0.86 and 0.93

The total reachable workspace area as captured by the motion capture and the

Kinect and the corresponding Bland-Altman analysis.

Elevation angle in the shoulder as average over 10

subjects and three repetitions each (using time

warping)

0 20 40 60 80 1000

50

100

150

200

Ele

vatio

n A

ngle

( °)

0 20 40 60 80 1000

50

100

150

200

Ele

vatio

n A

ngle

( °)

Motion Capture

Kinect

Human Model Classes: Kinematic/Dynamic

Musculoskeletal

Kinematic/Dynamic

Kinematic

InteractionAgent

Interaction

Human Dynamic

Stability Analysis

Goal

Model human motion using a rigid-body

kinematic/dynamic model

Applications

Human-Robot

Collaborative

Manipulation

Dynamics of Human Motion

• Given the reduced kinematics degrees of freedom modulo

task , the question is how precise the model of dynamics

must/or should be

• The answer to this question determines the complexity of the

system and in turn our ability to evaluate the stability of the

human during the task

• Dynamics of motion vary with each individual

• Considering the multi-degree freedom system, there will be

multiple points of stable configurations - can we enumerate

them?

Interaction Dynamics

• Let us assume that we can develop a dynamical model of

human physical activity. Similarly we can assume that we have

a dynamical model of an mechanistic ,yet multi-degree

freedom system.

• If these two systems are mechanically coupled, then the

dynamics changes of the overall system. Here is the

bottleneck of compositionality.

Application: Collaborative Manipulation

Goal: Enable intelligent

control of robots

providing direct physical

assistance to humans

• Create unified model

of the human-robot

coupled mechanical

system

• Predict intent of

human operator

based on physical cues

Human/Autonomous System Interaction

• Interaction begs for solving the problem of

communication between the two systems

• In which coordinate system should they

communicate?

– Most natural space for physical interaction is the task

space

– For remote, non-physical communication, the question is

open

Human-Robot Collaboration: Methods

• Individualized, data-driven

modeling of human

kinematics and dynamics

• Operational

space/impedance control to

generate control commands

and provide compliance

• Future: environment

contacts change dynamics

and kinematics of the

system

ξ1

ξ2

ξ3…

�� �, = � exp ������∈� �

exp ��� �� ∈ ��(3)

Human-Robot Collaboration: Applications

• Define safe subsets

of the person’s

configuration space

for lifting and

moving tasks

• Optimize motion to

minimize

interaction forces,

velocity, or other

variables

• Use task-specific

learned models to

predict human

actions

Application: Human Dynamic Stability Analysis

Given a hybrid dynamical model of a system, can we:

• Fit the model to real motions/trajectories and determine the

discrete and continuous inputs

• Analyze movement in terms of:

– Safety

– Reachability

Goal: Use these ideas to develop a controller to increase the

safety/reachability for assistive systems

Switched System Optimal Control

R. Vasudevan, “Modeling Biolocomotion - Computationally tractable tools for the identification of hybrid dynamical models of human

movement,” in SIAM Conference on Dynamical Systems, 2013.

R. Vasudevan, H. Gonzalez, R. Bajcsy and S. S. Sastry, “Consistent Approximations for the Optimal Control of Constrained Switched Systems -- Part

1: A Conceptual Algorithm,” SIAM Journal on Control and Optimization, vol. 51, no. 6, pp. 4463-4483, 2013.

Hybrid Reachability

Using occupation measures and Liouville’s equation, calculate an

outer-approximation of the reachable set for hybrid systems

with bounded inputs.

• Polynomial autonomous systems with affine dynamics

• Convex program with guarantees of convergence.

• Up to 6 states

V. Shia, R. Matthew, R. Vasudevan and R. Bajcsy, “Relaxing Global Decrescence Conditions for Hybrid Systems Using SOS,” in International

Federation on Automatic Control (IFAC), 2014, 2014.

V. Shia, R. Vasudevan, R. Bajcsy, and R. Tedrake. “Convex Computation of the Reachable Set for Controlled Polynomial Hybrid Systems.”

Submitted

Hybrid Reachability Examples

Compass Gait Walking Model (2 states)

Backwards Reachable Set for CG

(black dotted line is the limit cycle)

Bicycle Model for Vehicles (6 states)

Forward Reachable Set with bounded inputs

(green dots are in the FRS, red are not)

Gait Phases

Human Model Classes: Musculoskeletal

Musculoskeletal

Kinematic/Dynamic

Kinematic

Discrete Discrete States

Goal

Combine a kinematic/dynamic model with a

nonlinear model of muscle characteristics to

predict biomechanical properties throughout

the human’s workspace

Application

Medical Diagnostics/

Assistive Robotics

Neuromuscular Disease

Prevalence:

• Inherited: 1 in 3500 [Emery 1991]

• Aquired: Stroke, Heavy metal

poisoning , Auto-immune disease

• Total: 1 in 850 [Pohlschmidt 2010]

Symptoms:

• Weakness,

• Joint rigidity,

• Loss of muscular control

• Muscle pain,

• Twitching/Spasms

A. Emery, “Population Frequencies Of Inherited Neuromuscular Diseases - A World Survey,” Neuromuscular Disorders, 1991

M. Pohlschmidt, R. Meadowcroft, “Muscular Disease: The Impact,” Muscular Dystrophy Campaign, 2010

Change in musculature due to

Duchennes Muscular Dystrophy

[Benayoun 2008]

Human Musculoskeletal Model

Consists of:

• 206 bones

• Between 640 and 850 muscles

• Muscles are nonlinear, variable

dynamic actuators

• Highly deformable structure

• Variation in body morphology

between individuals

Hill Muscle Model

��� = � � � , � ! , � !, !" , �#� , #�, $�, %� = �& + �( �#�

Passive Component

�( ≈ *+, �-+

*.

Active Component

�& ≈ � � � � � " �

Muscle Activation

� � = *&/ � − +*& − +

Processed

EMG

Non-linear

Shape

Constant

Force Length Curve Force Velocity Curve Tendon Force Strain Curve

A. Hill. "The heat of shortening and the dynamic constants of muscle,“ Proceedings of the Royal Society of London. Series B, Biological Sciences, 1938.

F. Zajac. "Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control." Critical Reviews in Biomedical Engineering,

1988.

T. Buchanan, et al. "Neuromusculoskeletal modeling: estimation of muscle forces and joint moments and movements from measurements of neural

command." Journal of Applied Biomechanics, 2004.

Hill Muscle Model

��� = � � � , � ! , � !, !" , �#� , #�, $�, %� Muscle Systems are non-trivial

• Highly non-linear dynamics

• Muscle lengths 1, vary non-trivially with each joint

angle �• Biarticular muscles span several joints, further

complicating the length terms (Gray 1918)

• Muscles can co-contract (Veeger 1991)

• Significant errors during dynamic movements

(Perrault 2003)

• Model is highly sensitive to model parameters (Scovil

2006)

H. Veeger, et al. "Inertia and muscle contraction parameters for musculoskeletal modelling of the shoulder mechanism." Journal of Biomechanics, 1991.

E. Perreault, C. Heckman, T. Sandercock. "Hill muscle model errors during movement are greatest within the physiologically relevant range of motor unit firing

rates." Journal of Biomechanics, 2003.

C. Scovil, J. Ronsky, "Sensitivity of a Hill-based muscle model to perturbations in model parameters." Journal of Biomechanics, 2006.

Our Focus

• Capturing the musculoskeletal model of an

individual

• Analyzing and quantifying the range of

motion, joint forces and torques for

diagnosis

• Developing control strategies to assist

deficient joints/ to aid rehabilitation

• Intelligently design assistive devices for

restore functionality while minimizing the

number of actuators and sensors

Methodology

Data Acquisition Musculoskeletal Modelling

Musculoskeletal AnalysisAssistive Device Design and Control

System ID, State Estimation

Constrained, task specific optimization

Hill muscle

Models

Human/Robot

Controller

Conclusions

• Robotic technology has great utility for modeling and

predicting human physical capabilities and limitations

• Hybrid systems results can help to assess

stability/safety of human physical performance and

tolerance to disturbances (in walking, other activities)

Open Problems

Big Question: How to communicate intent to cooperative

human-autonomous systems (industrial, transportation,

assistive, etc.)

• Analytical and computational tools cannot handle the

complexity of detailed human kinematic/dynamic

models

• What subset of a person’s musculoskeletal parameters

need to be known for a given task? How do you

estimate them?

Acknowledgments

UC Berkeley:

S. Shankar Sastry, Claire Tomlin,

Francesco Borelli, Gregorij Kurillo,

Ferda Ofli, Stepan Obdrzalek, Ram

Vasudevan, Victor Shia, Katie Driggs-

Campbell, Robert Matthew, Aaron

Bestick

OHSU:

Misha Pavel, Holly Jamison

UC Davis Medical Center:

Jay Han

Johns Hopkins University:

René Vidal

Support from National Science

Foundation (NSF) and Center for

Information Technology Research in

the Interest of Society (CITRIS) at UCB.