1

Development of Infant Reaching

Neil E. BerthierDepartment of Psychology

University of Massachusetts, Amherst

berthier@psych.umass.edu

http://people.umass.edu/neb

The Development of ReachingSlow improvement over a long period of time with a great deal of

practice.– Arm movements in late-term fetuses (Myowa-Yamakoshi &

Takeshita, 2006; Sparling et al., 1999).– Directed, but non-successful reaching movements in

neonates (Ennouri & Bloch, 1996; Ruff & Haltom, 1978:Rader & Stern, 1982; von Hofsten, 1982; von Hofsten &Rönnqvist, 1993).

von Hofsten (1982)

Developmental Overview– Changes in movement kinematics throughout early childhood

(Berthier & Keen, 2005; Konczak & Dichgans, 1997; Thelen,Corbetta, & Spencer, 1996; von Hofsten, 1991).

– Not reaching, but:• Keen et al. (in prep). Longitudinal study of spoon use,

protracted learning even with teaching.• Adolph (2002): 9,000 steps and over 2.5 km/day over varied

terrain.

Konczak and Dichgans (1997)

Arm DynamicsDynamics of arm play a central role:•Well modeled by a single shift in equilibrium point.•Both an opportunity and an obstacle.

JerkMaximum Speed

Early Reaching

Hand Paths

Berthier et al., 1999

2

Overview of the Development ofReaching

• Within a context of a changing plant– Changes in the lengths and masses of the arm segments.– Changes in the strength and sizes of the arm muscles– Changes in the brain and the ascending and descending

motor systems.

Wells, Hyler-Both, Danley, Wallace (2002)

Wells, Hyler-Both, Danley, Wallace (2002)

Theories of Development of Reaching• Theories of the development of reaching from CS/AI background,

those that exist are generally inconsistent with the data:Kuperstein (1991) or Sanger (1994).

• Bernstein (1967).• Sporns and Edelman (1993), Sporns (1997).• Theories from developmentalists:

– Thelen and coworkers.– Von Hofsten (1993), search of a task space.– Berthier and coworkers. Approximate optimal control modeled

by reinforcement learning (RL; Q-learning with tables,kinematic arm, Berthier, 1996,1997, and Actor-Critic RL withCMAC, dynamic arm, Berthier, Rosenstein, & Barto, 2005).

WorldActionSelection

Arm

Proprioception

•Vision of hand, target,and surround•Touch

Efference copy (forward models?)

Visuomotor planning, gripselection, constraints, etc.

Working Model Is early reaching feedforward or does itinvolve feedback control?

Bushnell (1985) concluded that early reaching was visually elicited andballistic. She reached this conclusion because of the belief that infantscould not relate visual errors to motor corrections.In the middle of thefirst year of life infants could make corrections, but as practiceproceeding visual correction becomes unnecessary.

… The important point is that erroneous arm movements resultingfrom whatever source of imprecision cannot be compensated forwith visually elicited reaching. An inherent feature of such reachingis that errors are not informative. Upon extending the arm in thedirection presumed to be appropriate and then missing the target,there is no way to perceive to what extent and in what direction thearm should be moved to obtain the object after all. The situation isanalogous to when one fails to pinpoint an object on top of a tablewith a the hand pointing underneath the table.

3

Van der Meer, van der Weel, & Lee (1995)

18-day-old infants can use vision tomove the hand into sight

Reaching and Vision

Berthier & Carrico, in prep

•Infants could use sight of hand early, but don’t.•Infants begin to use vision as adults do starting at 9 months-of-age.•They become sensitive to object size in planning and grasp at aboutthe same age.

Developmental Learning• While the ultimate goal is rapid and efficient grasping, the child reaches this state

by solving a set of increasingly difficult tasks:– Gain control of arm dynamics– Accomplish transport of hand to target reliably– Place hand on target in a way that simplifies grasp– Pre-shape hand for target– All the while improving speed and accuracy of hand transport

• Initial movements seem to be made with high stiffness (high spring constant) andwith elbow locked. Both important to reduce motor error. Van Roon et al. (inpress) suggest that proximally controlled movements would have lower motorerror than distally controlled movements. Examples of neural and peripheralproximodistal development?

• Improves early learning, but obviously not adaptive in the long run. Interestingfrom a learning perspective: learn a solution that you later avoid. Gradualrelaxation of these early modes of operation. (However, Sanger (2000) notes thatlow dimensionality is common in adult movement.)

• If understood from an equilibrium point control view, these early patterns define aset of synergies that are appropriate for early learning. However, with increasingcontrol the members of that set change.

Developmental Learning• Early reaching relies on proprioception and vision of the hand

becomes more important with age and experience.• Perhaps, proprioception is the more informative sense when one

is making high speed, poorly controlled reaches during earlylearning. Perhaps, it provides information about the internalforces experience by the arm in a way vision can’t.

• Visual acuity is poor in young infants (average 6-month-oldsvision is approximately 20/100), maybe quality of information isnot good early on.

• Vision of hand is then used later when the hand is more undercontrol and the precise spatial information conveyed by vision isof great use.

• There may be a general principle here. While vision is availableat young ages, attention to a restricted set of sensory informationallows for timely updated of movement. Sufficiency.

Exploratory learning has serious computational problems. The biggest isprobably the prohibitive size of the search space and the problem ofstarting with random movements. In infant reaching there are some

factors that allow for exploratory learning to be successful:• Early freezing of degrees of freedom: proximodistal development.

Reduces degrees of freedom, simplifies mapping from control signals toeffects.

• Short arm length and small workspace allows for success even whencontrol is poor early.

• Co-contraction puts a bound of the movements, increases safety duringearly exploration.

• Learning begins with a crude controller, not random motion. Increasessafety, allows the hand to reach the target eventually (Kawato, Sanger).

• Infants appear to be intrinsically motivated to repeatedly reach. Whilethey appear to want the target object, the act of obtaining the targetseems motivating in itself.

Exploratory Learning RL as a Model of Learning inDevelopment

• No teacher necessary, no requirement to calculate trajectory norto regulate movement around that trajectory.

• Defines task appropriately. Getting to the target efficiently allowsfor the infant to discover useful arm dynamics.

• Allows for “shaping.” First goal may be to simply obtain thetarget, next get it rapidly, next to preshape fingers.

• Search of action space is around an already appropriate solution.• As the plant changes with age, action selection adjusts.

4

Complex Representation Not Necessary

• Stores best actions for different states. Requires arepresentation that generalizes appropriately.

• No explicit modeling. No forward model necessary.• Unlikely, there is a qualitative shift at any age that results in

reaching success.• But, could provide a basis for deeper understanding of space.

Berthier, Rosenstein, and Barto, 2005

Simulations

Before Training After Training

Action Representation

Catching a Moving Object100 mm/s 200 mm/s

73

33

6392

50 mm

100100

50 mm

50 mm

50 mm

Berthier, 1997Solved with tables and Q-Learning

Manual Search

Failure due to sensorimotor mode of responding?

Do very young infants have knowledge of hidden object movements?

Where’s the ball? Two- and three-year-oldsreason about unseen events.Berthier,DeBlois,Poirier, Novak, & Clifton(2000)

Haddad and Keen

5

Shutts, Keen, & Spelke (2006) Questions

• What perceptual, motor, cognitive and social capacities dohumans/animals start off with?

• What are the constraints imposed by physical bodies, thephysical environment, the social environment and how do theydetermine change during development in humans/animals?

• What is an object? An affordance? An event? How could youmake a machine recognize one?

• What do humans/animals understand about space, time andcausation, how is that understanding represented, what is innateand what learnt, and in the latter case how is it learnt?

• What other kinds of affective states and processes are likely toarise, or be useful in robots, and why?

• What are the sources of motivation in humans and other animals,and how do they develop over time? What kinds of motivation, ormeans of acquiring new sorts of motivation, might a robot need?