Visual Comput (2005)DOI 10.1007/s00371-005-0284-4 O R I G I N A L A R T I C L E

Vincent De SapioJames WarrenOussama KhatibScott Delp

Simulating the task-level control of humanmotion: a methodology and framework forimplementation

Published online: 24 May 2005 Springer-Verlag 2005

Vincent De Sapio · JamesWarren · Oussama KhatibArtificial Intelligence Laboratory,Computer Science DepartmentStanford University, Stanford, CA 94305,USAe-mail: {vdesap, warren,khatib}@robotics.stanford.eduSend offprint requests to: V. De Sapio,Clark Center Room E100, 318 CampusDr., Stanford, CA 94305-5450

Scott DelpNeuromuscular Biomechanics Laboratory,Mechanical Engineering andBioengineering DepartmentsStanford University, Stanford, CA 94305,USAe-mail: delp.stanford.edu

Abstract A task-level control frame-work is proposed for providingfeedback control in the simulationof goal-directed human motion. Anoperational space approach, adaptedfrom the field of robotics, is usedfor this purpose. This approach isaugmented by a significant newextension directed at addressingthe control of muscle-driven sys-tems. Task/posture decompositionis intrinsically exploited, allowinghuman musculoskeletal properties todirect postural behavior during theperformance of a task. This paperalso describes a simulation architec-ture for generating musculoskeletalsimulations of human characters. Theevolving capabilities of the collectiveenvironment are directed toward

autonomously generating realisticmotion control for virtual actorsin interactive computer graphicsapplications, as well as synthesizingthe control of human-like motion inrobotic systems.

Keywords Human animation · Mus-culoskeletal dynamics · Robotics ·Task-level control

1 Introduction

In recent years, there has been an emerging desire to syn-thetically generate human-like motion in both simulatedand physical settings. In the computer graphics commu-nity, this desire is directed toward autonomously gener-ating realistic motion for virtual actors [5, 6, 29]. The in-tent is to direct these virtual actors using high-level goal-directed commands for which low-level motion control isautomatically generated.

Motivated by similar desires, the robotics communityseeks a high-level control framework for robotic systems.With the recent advent of complex humanoid robots [19,25] this challenge grows more demanding. Consistentwith their anthropomorphic design, humanoid robots are

intended to operate in a human-like manner within man-made environments and to promote interaction with theirbiological counterparts. Common control strategies in-volve generating joint space trajectories [19] or learningspecific motions [10], but these approaches require exten-sive motion planning computations and do not generalizewell to related tasks.

The challenge of synthesizing low-level human mo-tion control from high-level commands can be addressedby integrating approaches from the biomechanics androbotics communities. The biomechanics community hasinvestigated the phenomenon of neuromuscular dynam-ics and control through the use of computational musclemodels [3, 4, 31]. This characterization allows us to de-scribe muscle strength limitations, activation delays, andoverall muscle contraction dynamics. Properly accounting

V. De Sapio et al.

for these characteristics is critical to authentically simulat-ing human motion since all human motion is rooted in, andbounded by, physiological capabilities. In a complemen-tary manner, the robotics community has investigated thetask-level feedback control of robots using the operationalspace approach [12, 13, 15]. This approach recasts the dy-namics of the robotic system into a relevant task spacedescription. This provides a natural mechanism for speci-fying high-level motion commands that can be executed inreal-time using feedback control.

While the use of computational musculoskeletal modelsand the task-level control of robots have been researchedin the past, the integration of these approaches into a uni-fied framework offers a promising methodology for syn-thesizing goal-directed human-like motion control. Recentwork [15] addresses multi-objective task-level control ofhumanoid systems. However, the authors do no not ad-dress the biomechanical factors that influence humanmotion control. Such factors, like musculoskeletal dynam-ics, form an important part of emulating human motion ina synthetic environment.

This paper focuses on the biomechanical aspects ofhuman motion in order to provide a physiological frame-work for simulating human motion. We present a controlarchitecture that integrates elements of biomechanical androbotic approaches. This control architecture extends theoperational space approach by addressing the control ofmuscle-driven systems. This extension complements otherwork done in operational space control of humanoid sys-tems [15] that addresses multiple task objectives but notthe physiological basis for motion.

In addition to the new control framework, this paperpresents a simulation architecture for the task-level controlof human-like motion. This consists of a simulation im-plementation that generates feed-forward musculoskeletalsimulations of human-like characters. This is built upona core environment that has already been developed for thephysics-based simulation of robotic systems. The evolvingcapabilities of the overall environment are facilitated by anobject-oriented design, which provides a natural mechan-ism for connecting the musculoskeletal plant to task-levelcontrollers. The environment offers real-time simulationfunctionality, allowing the user to control and interact withmany degree-of-freedom muscle-driven models in a task-level setting.

2 Previous work

The operational space framework [12, 13] provides thepoint of departure for the task-level control approach thatwill be addressed in this paper. This framework providesa natural vocabulary for studying human motion control.It does so by providing dynamic models at the task leveland structures for decoupled task and posture control. This

allows for posture objectives to be controlled without dy-namically interfering with the primary task(s). More re-cently a multi-objective formulation has been proposed forthe control of humanoid systems [15]. The efficacy of theoperational space framework has been demonstrated forrobotic systems; however, to date, it has only had limitedapplication for biomechanical systems [27]. Specifically,past work in formulating operational space control, in-cluding the multi-objective formulation [15], makes noaccommodation for muscle-driven control. This paper ex-tends the application of the operational space approach tobiomechanical simulations.

Several tools have emerged in the biomechanics com-munity for studying the dynamics and neuromuscular con-trol of movement. Many biomechanics labs have made useof SIMM (Software for Integrated Musculoskeletal Mod-eling) [4], a modeling environment for defining skeletaltopology, muscle geometry, and joint kinematics. Whenused in conjunction with SD-FAST (Symbolic Dynam-ics) [9] to generate the multi-body equations of motion forthe skeletal system, the user can simulate the feed-forwarddynamic response of the musculoskeletal system to neuralinputs. There are no native control capabilities in SIMM,so users must specify the control via open-loop using theirown feed-forward optimization routine [22] or via closed-loop using their own feedback control routine [28]. Inaddition to SIMM, there have been several other mus-culoskeletal simulation systems described in the litera-ture [8, 17, 18, 23].

3 Musculoskeletal model

The musculoskeletal model presented here involves a de-scription of the skeletal system as a rigid multi-body sys-tem spanned by a set of musculotendon actuators. Themusculotendon actuator model consists of a standard two-state Hill-type model [26, 30]. A stiff tendon simplifica-tion of the two-state model will also be presented.

While this musculoskeletal model has been broadlyused, Sect. 5 describes a unique integration of this modelinto a task-level control framework. This involves encod-ing musculoskeletal parameters into a postural field to fa-cilitate control.

3.1 Skeletal dynamics

The skeletal system is modeled as a system of constrainedrigid bodies whose configuration is described by a set ofgeneralized coordinates, q. These generalized coordinatesare usually taken to be the joint angles between limb seg-ments but may also involve other displacement parame-ters. The n equations of motion can thus be represented instandard form as

Γ =A(q) q+b(q, q)+g(q), (1)

Simulating the task-level control of human motion: a methodology and framework for implementation

where Γ is the set of joint torques, A(q) is the systemmass matrix, b(q, q) is the vector of centrifugal and Cori-olis terms, and g(q) is the vector of gravity terms. Forconciseness, we will often refrain from explicitly denotingthe functional dependence of these quantities on q and q.This practice will also be employed with other quantitiesthroughout the paper.

3.2 Muscle kinematics

Having defined a dynamical model of the skeletal system,we may now assign a system of musculotendon actuatorsto it. For the purposes of this model, we will assume thatall musculotendon lengths, l, can be uniquely determinedfrom the system configuration, q. That is, l = l(q). Asa consequence of this assumption, differential variations inl are given by

δl=L(q) δq, (2)

where L(q) is the muscle Jacobian. From the principle ofvirtual work, we conclude that

Γ = −LTf T , (3)

where f T is the vector of net muscle forces (active andpassive components). The negative sign is due to the con-vention of taking contractile muscle forces as positive.

3.3 Neuromuscular dynamics

Neuromuscular dynamics refers to the time-evolution ofmuscle states (length and force) in response to neural ex-citation (control input) [11, 21]. This net dynamic effectcan be divided into activation dynamics and musculoten-don contraction dynamics. A Hill-type active state model[26, 30] has been used to model these effects.

3.3.1 Activation dynamics

Activation dynamics refers to the process of muscle acti-vation in response to neural excitation. This process canbe modeled by the following equation of state written interms of the activation, a:

a = u −a

τ(u, a), (4)

where u ∈ [0, 1] is the neural input. The term τ(u, a) isa time constant given by

τ(u, a) ={(τa − τd)u + τd for u � aτd for u < a

, (5)

where τa and τd are the activation and deactivation timeconstants, respectively.

3.3.2 Contraction dynamics

Musculotendon contraction dynamics refers to the processof force generation in the muscle based on muscle con-traction, rate of contraction, and activation. A number ofdifferent active state models exist. One such model is for-mulated with muscle length, lM , as the state [26, 30]. Fig-ure 1 depicts the configuration of forces in this model. Inaddition to an active element, there is a passive viscoelas-tic element (in parallel) and an elastic tendon element(in series). The relative angle associated with the musclefibers, α, is referred to as the pennation angle.

Based on the geometry of Fig. 1, we have the followingrelationships:

l(q) = lM cos α+ lT

l(q, q) = lM cos α+ lT (6)

and the following force equilibrium equation:

fT = ( f A + f P) cosα. (7)

Using this force equilibrium equation in conjunction withconstitutive relationships (Fig. 2) we can express an equa-tion of state in the following functional form:

lM = lM(l(q), l(q, q), lM, a). (8)

For a system of r musculotendon actuators, we canexpress the following system of 2r first-order state equa-

Fig. 1. Active state musculotendon model. The active contractileelement and passive viscoelastic element are in parallel. The pas-sive elastic tendon element is in series

V. De Sapio et al.

Fig. 2. Force-length-velocity relationship showing muscle force,fM , as a function of muscle length, lM , and rate of contraction,lM(all quantities are normalized)

tions:

a=

1/τ1 0. . .

0 1/τr

u1 −a1...

ur −ar

, (9)

and

lM = lM(l(q), l(q, q), lM,a) (10)

3.4 Musculoskeletal dynamics

Combining neuromuscular dynamics with multi-bodyskeletal dynamics allows us to model the overall muscu-loskeletal system. Given a set of neural inputs, u, a set

Fig. 3. Musculoskeletal system (feed-forward path). Neural excitations provide input to the activation dynamics. Output of the activationdynamics provides input to the contraction dynamics. Output of the contraction dynamics provides input to the skeletal dynamics throughthe joint torques

of muscle activations, a, arise as dictated by the activa-tion dynamics. Based on these activations, as well as theskeletal configuration, q and q, a set of muscle forces,f T , arises. These muscle forces are related to the jointtorques, Γ , through the muscle Jacobian, L. The skele-tal dynamics are driven by these joint torques, resulting inmotion of the system. This overall process is described bythe feed-forward path of Fig. 3. This path, however, doesnot address the feedback mechanisms used by the centralnervous system to arrive at appropriate compensation forcontrolling the musculoskeletal plant.

3.4.1 Stiff tendon model

If we make the assumption that the tendon is infinitelystiff, lM is no longer an independent state, but rather,it is algebraically related to the overall musculotendonlength, l(q). This provides a useful simplification to thefull musculoskeletal dynamics described in the previ-ous sections. If we define the muscle saturation force,fS(q, q) � f A(q, q, 1), we can then define the muscleforce-activation gain, K f , as the magnitude of force gen-eration in the muscle per change in unit activation. Thus,

K f �∂ fT

∂a= fS cos α. (11)

For a system of muscles, the muscle force-activation gainmatrix is then

K f =

fS1 cos α1 0. . .

0 fSr cos αr

, (12)

and the muscle forces can now be expressed in terms ofthis gain:

f T (q, q,a) = f P(q, q)+K f (q, q)a, (13)

Simulating the task-level control of human motion: a methodology and framework for implementation

where

f P = (f P1 cos α1 · · · f Pr cos αr

)T. (14)

As with the musculotendon forces, it is useful to ex-press our generalized forces using a gain relationship. Tothis end, we will define the muscle torque-activation gainmatrix,

KΓ �−LTK f . (15)

The generalized forces can now be expressed in terms ofthis gain,

Γ (q, q,a) = Γ P(q, q)+KΓ (q, q)a, (16)

where

Γ P = −LTf P. (17)

The overall musculoskeletal dynamics can then be ex-pressed as

Γ (q, q,a) =A(q) q+b(q, q)+g(q). (18)

4 Task-level control framework

Within the context of our control framework, a task is anyformal description of desired activity that can explicitlybe represented as a function of the generalized coordi-nates. This could be as simple as specifying the positionof a limb to be in a certain location. Multiple tasks can becombined into a single task definition, as long as they arekinematically consistent with each other. The task coordi-nates will be denoted by x = x(q). The Jacobian,J(q), ofthe task is defined as

Jij �∂xi

∂qj(19)

Defining the dynamically consistent inverse of the Ja-cobian as [12] [13]

J �A−1JT (JA−1JT )−1, (20)

we can map the configuration space description of theskeletal dynamics (Eq. 1) into an operational space de-scription:

JT (Aq+b+g = Γ )

⇓Λ(q) x+µ(q, q)+p(q) = f

. (21)

This process provides us a description of the dynamicsin task coordinates rather than generalized coordinates

(while generalized coordinates are still present in Eq. 21,the inertial term involves task space accelerations ratherthan generalized accelerations). This is useful for controlapplications, since we can frame our control problem interms of the relevant task coordinates, x, that we wish tocontrol using a relevant operational space force, f . Theterms in Eq. 21 are defined as the operational space massmatrix,

Λ(q)�(JA−1J T )−1

, (22)

the operational space centrifugal-Coriolis vector,

µ(q, q)� J Tb−ΛJq, (23)

and the operational space gravity vector,

p(q)� JTg. (24)

We will always take the dimension of the task to be lessthan or equal to the dimension of the configuration space.In the specific case that the dimension of the task is strictlyless than the dimension of the configuration space, we re-fer to the system as redundant with respect to the task (seeFig. 4). In this case, the operational space dynamics do notfully describe the system behavior, and f does not fullydescribe the system control. However, the null space of thetask is associated with a complementary posture space ofmotion that does not interfere with the task [13]. This pos-ture space may be utilized to satisfy other criteria throughthe following decomposition of the control torques:

Γ = Γ task +Γ posture = JT f + (I−JT J

T )Γ o. (25)

Fig. 4. A task description illustrating two corresponding task-consistent postures. Redundancy with respect to task introducestask dynamics as well as posture dynamics (simulation sequencecreated by L. Sentis)

V. De Sapio et al.

A more concise expression of this is

Γ task +Γ posture = JT f +N TΓ o, (26)

where

N T � I−J T JT. (27)

Khatib et al. [15] considered a multi-objective formu-lation of operational space control, allowing for a hierar-chy of separate tasks that may conflict with each other.This multi-objective approach will not be used for the ex-position presented throughout the rest of this paper. Thebasic formulation summarized above will suffice, given anassumption of non-conflicting tasks.

5 Task-level musculoskeletal control

We now wish to apply a task/posture control decompos-ition to our musculoskeletal system. This entails choos-ing physiologically characteristic task and posture controlstructures that model certain aspects of the motion con-trol behavior of the central nervous system. Because weare choosing feedback control structures, the system canbe designed to behave in a robust manner in the presenceof disturbances. This is important for real-time interactionwith a physics-based simulation or a physical system. Inthe absence of such feedback mechanisms, feed-forwardoptimization approaches like dynamic optimization wouldbe required. Such approaches have the drawback of beingcomputationally costly, thereby precluding real-time exe-cution. Additionally, due to the open-loop nature of thecontrols they generate, the behavior of the system is notrobust in the presence of disturbances and external userinteractions.

5.1 Task control

We will begin our task/posture control synthesis by choos-ing a task control structure. This entails designing a con-trol response, f . An artificial potential field, Ut(x), de-scribed in task coordinates, serves as a straightforwardcontrol primitive. For example

Ut(x)� 1

2(x− xo)

TKx(x− xo) (28)

can be used as a simple quadratic potential field, relative toa goal location, xo. Seeking to minimize this potential, wegenerate the following control force:

f � = −∇Ut = −Kx(x− xo). (29)

Adding a damping term, we have

f � = −Kx(x− xo)−Kv x (30)

as the control input of the decoupled system. The gainmatrices,Kx andKv, can be chosen to reflect physiolog-ical properties, such as natural limb endpoint impedances.Using estimates (denoted by ·) of the operational spacedynamic properties, we have the following dynamic com-pensation expression:

f = Λ f � + µ+ p. (31)

This expression will be complemented by a posture ex-pression, to be developed in the following section.

5.2 Posture control

We will complete our task/posture control synthesis bychoosing a posture control structure. This entails design-ing a control input, Γ p, to serve as the null space term,Γ o, in Eq. 26. A potential field, Up(q), can be chosen torepresent any of a number of different physiological mea-sures. Specifically, where these measures relate to muscu-loskeletal properties, we will refer them as muscular effortmeasures. Our control then seeks to minimize Up whilesatisfying the task.

The expression for Up(q) proposed the gravityterms [16] involves muscle strength capacities projected atthe joint level and is given by

Up(q) =r∑

i=1

wi

(gi(q)

ΓBi (q)

)2

, (32)

where, depending on the sign of the gravity terms gi , theterm ΓBi is either the upper or lower muscle inducedtorque boundary about a joint i . These terms are computedusing the stiff tendon musculoskeletal model described inSect. 3.4.1. The wi terms are weighting terms.

The physiological rationale for this measure was dis-cussed in [16] and will only be briefly described here. Inthis study, a number of motion-capture experiments wereperformed on human subjects. The experimental posturesassociated with a given reaching task were compared withother postures in the kinematic null space of the task. Theset of these postures comprise the self-motion manifoldassociated with a fixed task point, x(q) = xo. This set isgiven by M(xo)� {q|x(q) = xo}. In [16], good correlationwas observed between the experimental postures and thepostures that minimized the effort measure of Eq. 32.

It is noted that by projecting muscle strength capaci-ties to the joint level in a decoupled manner, as is donein Eq. 32, the cross-joint coupling associated with multi-articular muscles (muscles that span more than one joint)is ignored. Consequently, in this paper we propose an en-hanced and more general muscle effort measure to thatwhich was proposed in [16]. This measure is given by

Up(q) = g(q)T (KΓK

TΓ

)−1g(q), (33)

Simulating the task-level control of human motion: a methodology and framework for implementation

whereKΓ is the muscle torque-activation gain matrix de-fined in Eq. 15.

Using the measure of Eq. 33, experimental studieswere performed and analyzed. Motion capture experi-ments were performed on five subjects who were in-structed to move different weights to various target loca-tions. Figure 5 depicts the results of a particular subjecttrial. The effort associated with the subject’s nominal con-figuration, qo, at the target can be compared with the effortassociated with other configurations in a subset of the self-motion manifold, q = [q1, q2], of the target.

Not all trials show as precise a correlation between thesubject’s nominal configuration and the minimum of themuscle effort curve, as can be seen in the trial of Fig. 5.However, for most trials, the minimum of the effort curve

Fig. 5. Variation of muscle effort (Eq. 33) over the self-motionmanifold for a given hand target location. The horizontal axis rep-resents variation from the subject’s nominal configuration, qo, indegrees along the self-motion manifold. For most trials, the mini-mum of the effort curve is typically within several degrees of thesubject’s nominal configuration

is typically within several degrees of the subject’s nominalconfiguration. More experiments are planned; neverthe-less, the results are promising in that they suggest goodcorrelation between muscle effort minimization and sub-ject motion. The reader may refer to the work of Khatibet al. [16] for a more extensive discussion of experimentalvalidation with human subjects.

In the work presented by Khatib et al. [16], the ap-proach employed with the muscle effort measure (Eq. 32)was effectively a kinematic one. While a prioritized task-level control framework is summarized, the authors do notexplicitly integrate the musculoskeletal model into the dy-namics and control setting. Further, the analysis done byKhatib et al. [16] is purely kinematic. By contrast, we wishto apply an effort-minimization approach to the control,where we seek to minimize the muscle effort potential ina task-consistent manner. To do this, we generate the fol-lowing posture control torque:

Γ p = −∇Up. (34)

Adding a damping term and gains, we have

Γ p = −Kp ∇Up − Kd q. (35)

5.3 Unified musculoskeletal task/posture control

With the individual task and posture control structuresdefined, we can implement the overall feedback controlstructure. In summary, our control torque is given by

Γ = JT f +N TΓ p, (36)

where the operational space compensation is

f = Λ f � + µ+ p, (37)

and the control input of the decoupled system is

f � = −Kx(x− xo)−Kv x. (38)

Finally, the posture term is

Γ p = −Kp ∇Up − Kd q, (39)

where Up is the muscle effort measure given by Eq. 32,Eq. 33, or some other muscle-based potential. Figure 6depicts a block diagram of the task-level neuromuscularcontroller described by Eqs. 36 through 39.

Figure 7 depicts responses associated with the task-level neuromuscular controller of Fig. 6. In this illustrativeexample, a muscle-driven redundant chain is controlledto move to a target. In one case, the distal muscles arestronger than the proximal muscles. In the other case, theproximal muscles are stronger than the distal muscles.

V. De Sapio et al.

Fig. 6. Task-level feedback control employing a neuromuscular criterion for posture control. The task servo block represents a controllaw in task coordinates. This can be defined as the gradient of a task potential field plus a damping term (Eq. 38). Estimates of the op-erational space dynamic parameters, Λ, µ, and p are used to provide the appropriate dynamic compensation, f (Eq. 37). The posturecontrol emerges in a task-consistent manner, where the posture servo block represents a neuromuscular-based control law. This can bedefined as the gradient of a muscle effort potential plus a damping term (Eq. 39). The sum of the task and posture terms is applied to theskeletal plant as a control input, Γ (Eq. 36)

Fig. 7. A muscle-driven redundant chain controlled by the system of Fig. 6. The case where the distal muscles are stronger than the prox-imal ones is illustrated in (a). The controller drives the musculoskeletal plant to the target (task) from to to t f in a manner that minimizesthe effort (Kx = 100, Kv = 20, Kp = .3, Kd = 3). The case where the proximal muscles are stronger than the distal ones is illustrated in(b). Again, the controller drives the musculoskeletal plant to the target in a manner that minimizes the effort (Kx = 100, Kv = 20, Kp = .4,Kd = 4)

Simulating the task-level control of human motion: a methodology and framework for implementation

Different behaviors autonomously emerge from the con-troller based on these different physiological conditions.

The tight integration of our musculoskeletal model andtask-level control framework is unique in that it poses thecontrol objective for the musculoskeletal plant in termsof a concise set of task coordinates rather than the muchlarger set of generalized coordinates. Additionally, whilethe musculoskeletal plant can be driven by individual neu-ral excitations, as is conventionally done, its integrationwith the task-level control framework allows it to be con-trolled through a unique task/posture decomposition. Inthis way, the posture controller encodes the properties ofthe musculoskeletal plant in an efficient manner, usinga potential, without the need for specifying a large set ofcontrol inputs (neural excitations).

This approach also offers advantages over other ap-proaches. For example, a standard approach like dynamicoptimization exhibits a significant computational burden,since a large input space of neural excitations must besearched to drive the musculoskeletal plant over a fixedtime interval. Additionally, the control is inherently open-loop as the feed-forward path of the musculoskeletal plantis repeatedly cycled in order to generate the control inputs.This prohibits the simulation from being interactive andresponding in a robust manner to disturbances.

6 Simulation architecture

A simulation environment has been developed for simu-lating musculoskeletal dynamics and task-level control.This has been built upon a simulation and active interfacesframework (SAI), which is comprised of a set of librariesdeveloped to perform interactive simulation of complexrobotic systems [14].

6.1 Core simulation environment

At the core of SAI is a multi-body dynamics engine forsimulating kinematic chains. Additionally, collisions andcontact between objects are simulated. A haptics inter-face library allows for interaction with the system usinga force-feedback device. Outside of the core dynamics andinteraction capabilities, most of an SAI’s infrastructure isdevoted to algorithms for task-level planning and control.

The multi-body dynamics component of SAI focuseson recursive algorithms [1, 7, 20] for computing the dy-namics and control of n-joint branching, redundant roboticmechanisms with m operational degrees of freedom.The complexity of these algorithms scales according toO(nm +m3) as compared to O(n3 +m3) for many exist-ing symbolic algorithms. Typically, m is small comparedto n, which implies near-linear scaling, O(n). A muchmore detailed description of the multi-body dynamics al-gorithms used in SAI is given [1].

The collision-handling capabilities of SAI implementan impulse-based approach for modeling rigid contact andcollision. Objects in the simulated world are modeled asthe union of convex polyhedra. A contact space is definedto describe the contact state between any two polyhedrawith a finite number of contact points. When a contact orcollision occurs, a Jacobian and operational space massmatrix, defined with respect to the contact space, are com-puted. In this approach, only the true degrees of freedomof the multi-body system are modeled, providing addedefficiency by eliminating the internal constraints of thesystem. The collision model employs a standard coeffi-cient of restitution relation as the constraint on velocitiesbefore and after collision. A much more detailed descrip-tion of the collision approach used in SAI, as well as itsbenefits over standard approaches, is given in [24].

While collision capability is useful in the general ap-plication of physics-based simulation, we have not specif-ically exploited it in our task-level musculoskeletal con-trol approach. As a result, we have only provided a briefsummary of this core simulation capability and have notdiscussed it within the specific context of task-level mus-culoskeletal control. Future applications of our muscu-loskeletal control approach may exploit the collision andcontact capabilities of SAI. These capabilities would beparticularly relevant for gait and contact-force controllers,both of which would require force-sensing information inthe feedback loop. Figure 8 depicts simulations in an SAIillustrating its handling of multi-body dynamics as well ascollision.

6.2 Musculoskeletal simulation environment

The core SAI environment has been extended to incorpo-rate the simulation of muscle-driven systems. Characterscan be specified with a set of musculotendon actuators,modeled as described in Sect. 3. The environment allowsfeed-forward operation, in which control inputs are sentto each of the muscles, as well as an implementation ofthe task-level control methodology of Sect. 5. The object-oriented design employed provides a natural mechanismfor connecting the musculoskeletal plant to task-level con-trollers. Appendix A provides a detailed description of theclass design.

The addition of muscle simulation capabilities to SAIis illustrated in Fig. 9, which depicts a sequence froma task-level control simulation. The model shown in thesequence consists of 30 joints and 144 muscles. Muscu-loskeletal data used in this model has been derived fromSIMM models [4]. The hands are defined as task-controlpoints that can be moved interactively by the user. A bal-ancing task constraint is imposed as well.

Because the control of the human characters in thesesimulations is robust and executed in real-time (or nearreal-time), the user can interact with the simulation andapply external disturbances to the characters. Figure 10

V. De Sapio et al.

Fig. 8. Sequences illustrating core simulation capabilities of SAIs. Virtual acrobats interacting in a dynamic environment subject to real-istic physical constraints (a). This SAI simulation sequence illustrates real-time multi-body dynamics and collision response (simulationsequence created by F. Conti). Interaction with a virtual skeleton in an SAI (b). The skeleton is commanded to sit on the block. The userthen pulls on the block with a virtual force, causing the skeleton to summersault (simulation sequence created by V. De Sapio)

shows an example of disturbance forces interactively ap-plied by the user. The character’s controller applies com-pensation in response to the disturbance, resulting in sta-ble behavior of the character’s dynamics.

7 Conclusion and future work

We have presented a framework for the task-level con-trol of muscle-driven human motion. This framework in-tegrates a musculoskeletal model and a task-level controlapproach, allowing the control objective for the muscu-loskeletal plant to be stated in terms of a natural set oftask coordinates. The unique task/posture decompositionimplements a posture controller, which efficiently encodesmusculoskeletal properties into a posture potential. The

feedback nature of this framework inherently provides sta-ble response to disturbances and external interactions, asopposed to feed-forward approaches like dynamic opti-mization.

A simulation environment that implements our task-level musculoskeletal control framework has also beenpresented. This environment is built upon an efficientmulti-body dynamics engine. The overall environmentprovides a user-friendly interface for producing physicallyrealistic task-driven human-like motions. We believe thatsuch a system is useful in the study and synthesis ofhuman-like motion for computer graphics applications aswell as the synthesis of human-like motion control for an-thropomorphic robotic systems.

The current focus of our work has been on whole-bodypostural control rather than walking. There are gait con-trollers currently being tested for future use. Regarding

Simulating the task-level control of human motion: a methodology and framework for implementation

Fig. 9. Task-level control ofa simulated human in a dynamicsetting. While the environmentallows feed-forward operationin which control inputs are sentto each of the 144 muscles,a more efficient task-level con-trol is used in this interactivesequence. Users can interac-tively move the control points,defined at the wrists, while thesimulated human responds

Fig. 10. Response of a human character to disturbanceforces (vectors) interactively applied by the user. Thefeedback controller applies compensation resulting instable behavior of the system and a return to equilib-rium. This is not possible with feed-forward simulationapproaches that generate open-loop controls

the muscle models, while the current system simulates fullmuscle dynamics in the feed-forward path, a stiff tendonmuscle model is employed for control purposes. This sim-plifies the control problem but neglects some importantdynamical properties of muscle. Plans for improvementof the system include the implementation of other mus-cle control routines. This will include an operational spaceversion of the computed muscle control method [28]. Suchan approach involves performing a task-constrained min-imization of muscular effort defined in terms of mus-cle activations. The conceptual framework for this ap-

proach has been developed and an implementation isplanned.

The implementation of constraint enforcement capa-bilities is planned as well. This step will be useful formodeling closed kinematic chains in addition to chainsthat involve constraints between the generalized coordi-nates. Such constraints are particularly relevant in thecase of the human shoulder complex and the knee joint.A control methodology has been developed to addresssuch cases [2]. This methodology makes use of a uniqueoperational space decomposition, which generates sepa-

V. De Sapio et al.

rate structures for task-motion control and constraint-forcecontrol.

The muscle path generation facilities of the SAI mus-culoskeletal environment do not include wrapping sur-faces, which are useful for modeling complex muscle rout-ing. SIMM includes these facilities and has been used toaugment the capabilities of SAI in this area. Consequently,an implementation of wrapping surfaces would be a par-ticularly useful addition to a future version of the SAIenvironment.

Acknowledgement The contributions of Luis Sentis, Irena Pa-schenko, Francois Conti, Diego Ruspini, and Kyong-Sok Chang tothe SAI simulation environment are gratefully acknowledged. Theauthors would also like to thank Katherine Holzbaur for provid-ing an enhanced model of the human shoulder complex, which wasimplemented in some of our work.

A musculoskeletal class design

Skeletal dynamics has been modeled in the SAI environ-ment using a set of link, joint, and robot classes. At the topof the class hierarchy is a world class that contains a listof pointers to any number of robot objects. Each robot canbe loaded individually into the world by the user. An XMLfile structure is used for storing geometric, kinematic, andinertial properties for each robot.

Muscle mechanics have been modeled using a class hi-erarchy consisting of an activator class, a contractor class,a muscle class, and a muscle system class. Collectively, theclasses are used to define a system of controllable muscu-lotendon actuators. Each robot object points to a musclesystem object. An XML file structure is used for storingproperties for the muscles that are contained in each mus-cle system. A set of structures are used for storing muscle

Fig. 11. Muscle paths are generatedas Catmull-Rom splines throughmuscle origin, insertion, and viapoints

Fig. 12. The activator class state model. Activation, a, is the systemstate (and output) responding to neural input

Fig. 13. The contractor class state model. Muscle length, lM , isthe system state responding to activation input. The output is thetendon force, fT

and tendon lengths, velocities, and forces, as well as ac-tivation properties, contraction properties, and path data.The path structure contains a list of attachment points anda list of pointers to corresponding attachment links. Thereare also linear and cubic spline interpolation functions.The linear interpolation function is used for interpolatingover the muscle force-length data. The spline interpola-tion function employs a Catmull-Rom spline algorithm togenerate muscle paths (Fig. 11).

Simulating the task-level control of human motion: a methodology and framework for implementation

Fig. 14. The muscle classmodel consisting of point-ers to a contractor and anactivator object. A muscleobject coordinates activa-tion and muscle dynamicsfor a given muscle-tendonunit.

The activator class encapsulates muscle activation dy-namics. Each object contains activation property data spe-cific to an individual muscle. Based on a neural input, u,muscle activation, a, is computed at each time-step. Fig-ure 12 depicts the block diagram for this class.

The contractor class encapsulates musculotendon con-traction dynamics. Each object contains musculotendonpath data and contraction properties data specific to an in-dividual muscle-tendon pair. The overall musculotendonpath length, l, is computed given the path data and thecurrent configuration of the skeleton, q. Based on an ac-tivation input, a, the force generated in the tendon, fT ,as well as active and passive forces generated in the mus-cle, f A and f P, are computed at each time-step. Figure 13depicts the block diagram for this class.

The muscle class integrates the activator and contrac-tor classes. It contains a pointer to an activator object andcontractor object and thus represents a single musculoten-don actuator (motor nucleus and associated muscle-tendonbody). Methods are provided to interface to the activatorand contractor objects. Figure 14 depicts the block dia-gram for this class.

The muscle system class is composed of a vector ofpointers to individual muscle objects. There are methods

to coordinate the initialization and update of individualmuscle objects. Additionally, at each update, system-widequantities are computed. These include the muscle Jaco-bian, L, and the vector of joint torques, Γ . The muscleJacobian is computed given the path data and the currentconfiguration of the skeleton.

The set of classes previously described simulate feed-forward musculoskeletal dynamics. A set of controls(neural excitations) can be specified as input to drive thefeed-forward dynamics. In practice, these controls wouldbe produced by a dynamic optimization routine [22] or inclosed-loop by a feedback control routine [28]. To imple-ment the feedback control structure described in Sect. 5,a set of classes have been implemented that employ thestiff tendon assumption described earlier. These are sim-plifications of the muscle classes already described andare used specifically for task-level musculoskeletal con-trol. At each update, additional system-wide quantities arecomputed. These include the muscle force-activation gainmatrix,K f and the muscle torque-activation gain matrix,KΓ , as well as the muscle-induced torque boundaries ΓBi .Various muscle effort measures, like the ones described inSect. 5.2, can be computed from these system-wide quan-tities.

References1. Chang K-S, Khatib O (2000) Operational

space dynamics: efficient algorithms formodeling and control of branchingmechanisms. In: Proceedings of the 2000IEEE International Conference on Roboticsand Automation 1:850–856

2. De Sapio V, Khatib O (2005) Operationalspace control of multibody systems withexplicit holonomic constraints. In:Proceedings of the 2005 IEEE InternationalConference on Robotics and Automation(in press)

3. Delp SL, Loan JP (1995) A softwaresystem to develop and analyze models ofmusculoskeletal structures. Comput BiolMed 25:21–34

4. Delp SL, Loan JP (2000) A computationalframework for simulating and analyzinghuman and animal movement. IEEEComput Sci Eng 2(5):46–55

5. Faloutsos P, van de Panne M,Terzopoulos D (2003) Autonomousreactive control for simulatedhumanoids. In: Proceedings of the 2003IEEE International Conferenceon Robotics and Automation1:917–924

6. Faloutsos P, van de Panne M,Terzopoulos D (2001) Composablecontrollers for physics-based characteranimation. In: Proceedings of the 2001SIGGRAPH Conference, pp 251–260

7. Featherstone R (1987) Robot dynamicsalgorithms. Kluwer, Boston

8. Hase K, Miyashita K, Ok S, Arakawa Y(2003) Human gait simulation witha neuromusculoskeletal model andevolutionary computation. J Visual ComputAnim 14(2):73–92

9. Hollars M, Rosenthal D, Sherman M(1991) SD/Fast users manual. Technicalreport, Symbolic Dynamics Incorporated

10. Ijspeert A, Nakanishi J, Schaal S (2002)Movement imitation with nonlineardynamical systems in humanoid robots.In: Proceedings of the 2002 IEEEInternational Conference on Robotics andAutomation 2:1398–1403

V. De Sapio et al.

11. Kandel ER, Schwartz JH, Jessell TM(2000) Principles of neural science, 4thedn. McGraw-Hill, New York

12. Khatib O (1987) A unified approach tomotion and force control of robotmanipulators: the operational spaceformulation. Int J Robot Res 3(1):43–53

13. Khatib (1995) Inertial properties in roboticmanipulation: An object level framework.Int J Robot Res 14(1):19–36

14. Khatib O, Brock O, Chang K-S, Conti F,Ruspini D, Sentis L (2002) Robotics andinteractive simulation. Commun ACM45(3):46–51

15. Khatib O, Sentis L, Park J, Warren J(2004) Whole body dynamic behavior andcontrol of human-like robots. Int JHumanoid Robot 1(1):29–43

16. Khatib O, Warren J, De Sapio V, Sentis L(2004) Human-like motion fromphysiologically-based potential energies. In:Lenarcic J, Galletti C (eds) On advances inrobot kinematics. Kluwer, Boston, pp149–163

17. Komura T, Shinagawa Y, Kunii TL (1999)Calculation and visualization of thedynamic ability of the human body. JVisual Comput Anim 10(2):57–78

18. Komura T, Shinagawa Y, Kunii TL (2000)Creating and retargetting motion by themusculoskeletal human body model. VisComput 16(5):254–270

19. Kuroki Y, Blank B, Mikami T, Mayeux P,Miyamoto A, Playter R, Nagasaka K,Raibert M, Nagano M, Yamaguchi J (2003)Motion creating system for a small bipedentertainment robot. In: Proceedings of the2003 International Conference onIntelligent Robots and Systems2:1394–1399

20. Lilly KW (1992) Efficient dynamicssimulation of robotic mechanisms. Kluwer,Boston

21. McMahon TA (1984) Muscles, reflexes,and locomotion. Princeton University Press,Princeton

22. Pandy MG, Anderson FC, Hull DG (1992)A parameter optimization approach for theoptimal control of large-scalemusculoskeletal systems. J Biomech Eng114:450–460

23. Rasmussen J, Damsgaard M, Surma E,Christensen S, de Zee M (2003) Designinga general software system formusculoskeletal analysis. In: Proceedingsof the International Symposium onComputer Simulation in Biomechanics

24. Ruspini D, Khatib O (2000) A frameworkfor multi-contact multi-body dynamicsimulation and haptic display. In:Proceedings of the 2000 InternationalConference on Intelligent Robots andSystems 2:1322–1327

25. Sakagami Y, Watanabe R, Aoyama C,Matsunaga S, Higaki N, Fujimura K (2002)The intelligent asimo: system overview andintegration. In: Proceedings of the 2002International Conference on IntelligentRobots and Systems 3:2478–2483

26. Schutte LM (1992) Using musculsokeletalmodels to explore strategies for improvingperformance in electricalstimulation-induced leg cycle ergometry.Dissertation, Stanford University

27. Thelen DG, Anderson FC (2001) Anoperational space tracking algorithm forgenerating dynamic simulations ofmovement. In: Proceedings of the 5thInternational Symposium on ComputerMethods in Biomechanics and BiomedicalEngineering

28. Thelen DG, Anderson FC, Delp SL (2003)Generating dynamic simulations ofmovement using computed muscle control.J Biomech 36:321–328

29. Yamane K (2004) Simulating andgenerating motions of human figures.Springer, Berlin Heidelberg New York

30. Zajac FE (1989) Critical reviews inbiomedical engineering. In: Bourne JR (ed)Muscle and tendon: properties, models,scaling, and application to biomechanicsand motor control. CRC Press, Boca Raton17(4)359–411

31. Zajac FE (1993) Muscle coordination ofmovement: a perspective. J Biomech26:109–124

VINCENT DE SAPIO is a Ph.D. student in the Ar-tificial Intelligence Laboratory at Stanford Uni-versity and a senior member of the technicalstaff at Sandia National Laboratories. He re-ceived a B.S. degree from Rensselaer Polytech-nic Institute in 1990 and an M.S. degree fromStanford University in 1994, both in Mechan-ical Engineering. He is currently investigatingtask-level control within the context of biome-chanical systems. This involves the implemen-tation of neuromuscular models in conjunctionwith a task/posture control decomposition. Thegoal is to emulate aspects of human motor con-trol in robotic systems.

JAMES WARREN is a Ph.D. student in the Artifi-cial Intelligence Laboratory at Stanford Univer-sity. He received his B.S. degree in Mathematicsfrom Texas A&M University in 1996 and hisM.S. degree in Scientific Computing and Com-putational Mathematics from Stanford University

in 2000. His research is in identifying humanmotion behaviors from motion capture data andapplying these strategies for humanoid roboticcontrol.

OUSSAMA KHATIB is Professor of ComputerScience at Stanford University. He received hisPh.D. in 1980 from Sup’Aero, Toulouse, France.His current research is in human-centeredrobotics, human motion synthesis, humanoidrobotics, dynamic simulation, haptic interfaces,and human-friendly robot design. This buildsupon a large body of work pursued over the past25 years and published in over 200 contributionsin the field. Professor Khatib was Program Chairof ICRA2000 and editor of The Robotics Review.He served as director of the Stanford ComputerForum and is currently president of the Inter-national Foundation of Robotics Research andeditor of Springer Tracts in Advanced Robotics.Professor Khatib is an IEEE Fellow, a Distin-

guished Lecturer of IEEE, and a recipient of theJARA Award.

SCOTT DELP is Professor and Chair of the Bio-engineering Department at Stanford University.After receiving a Ph.D. degree from StanfordUniversity in 1990, he joined the faculty ofNorthwestern University, where he was jointlyappointed in the Schools of Engineering andMedicine. He returned to Stanford in 1999 inthe departments of Mechanical Engineering andBioengineering. His work draws on computa-tional mechanics, biomedical imaging, and neu-romuscular biology to study human movement.He has received numerous awards, includinga National Young Investigator Award from theNational Science Foundation and Faculty Fel-lowships from the Baxter Foundation and PowellFoundation. He is a Fellow of the American In-stitute of Medical and Biological Engineering.