Abstract—In this paper, an optimized central pattern generator (CPG) network is proposed for humanoid walking control. The CPG controller targets three joints (hip, knee and ankle) of each leg including 4 degrees of freedom (DOFs). The connections for CPG units of related joints are simplified and optimized hierarchically. The total number of CPG parameters is greatly decreased in this way. Moreover, the genetic algorithm (GA) is adopted to acquire the optimal parameters in batch and the complexity of the algorithm is decreased greatly. Finally, the proposed CPG controller is applied to the straight and circular walking on a commercial humanoid robot NAO, both in simulations and practices.

Keywords-Central pattern generator; genetic algorithm; humanoid robot; NAO; bipedal walking

I. INTRODUCTION Central pattern generator (CPG) is a concept derived from

the neurobiology and it has been proved to exist in the spinal cord of animals, which can generate rhythmic movements such as locomotion, heart beating, respiration and flying [1]. CPG is widely used as a bio-inspired control strategy in the robotic community, which does not need an accurate model of robot dynamics. CPG can generate complex locomotion behaviors and even switch between different gaits while receiving only simple input signals. Compered to finite-state machines [2], sine-generators and zero movement points (ZMP)-based methods [3], Ijspeert identified some advantages of CPG-based locomotion control method [4]. One advantage is the stable limit cycle behavior of CPG, which can recover rhythmic movement quickly from external perturbations or transient perturbations of the state variables. Usually, the frequency, wave shape and even the duration of CPG outputs can be adjusted by modifying a few control parameters. Moreover, CPG is ideally suited to integrate sensory feedbacks and they usually offer a good substrate for learning and optimization algorithms.

Some models developed to simulate CPG, such as Matsuoka model, lamprey model and Van der Pol model and so on. CPG models have been used to control various types of

This work is supported by the National Natural Science Foundation of China (51075265) and the State Key Laboratory of Robotics and System of HIT (SKLRS-2012-ZD-04).

Qing Zhang and Dingguo Zhang are with Institute of Robotics, Shanghai Jiao Tong University, SH 200240 CN. They are members of IEEE. * Dingguo Zhang is the corresponding author (e-mail: dgzhang@sjtu.edu.cn).

Te Tang, Shichao Yang and Yunli Shao were undergraduate students from Shanghai Jiao Tong University.

robots and modes of locomotion, such as hexapod and octopod robots, swimming robots, quadruped robots and bipedal robots [4]. However, there are some challenges to apply CPG-based approaches. A sound design methodology and a solid theoretical foundation for describing CPGs are still missing. It is not easy to apply CPG because of its nonlinear and coupling property. Especially, it is difficult to tune the parameters of a complex CPG network with multiple neural oscillators. At present, there are three methods to solve this problem: trial-and-error method, analytic methods such as describing function, piecewise linear analysis and contraction theory, as well as optimization methods such as genetic algorithm (GA) [5-8] and quantum evolutionary algorithm [9-11]. Trial-and-error method may cost a lot of time and it is hard to perfectly match the desired outputs. The analytic methods require complicated and strict mathematical analysis. Here, we utilize the third method because of its high feasibility and manageability.

The goal of this work is to construct a relatively simple CPG network to generate specific gaits and use the generated pattern to control a biped robot. Taga initially built a simple network structure but it seems unable to generate specific gaits due to the limitation of parameters [12]. Inada et al. used a highly coupled CPG network to control 8 DOFs of bipedal locomotion and the amount of parameters is 271 [5]. Saif controlled a humanoid NAO of 6 DOFs and the number of parameters is 144 [13]. While their network structures are complex and not easy to be implemented.

In this work, a simplified CPG network is proposed. Hereby, the number of parameters is greatly reduced. Based on the proposed structure, we hierarchically optimize these parameters via GA and obtain well-fitted outputs. By simplifying the network and optimizing the parameters in batch, the calculation complexity is tremendously decreased and the efficiency is improved. Furthermore, we use the proposed CPG controller to generate straight and circular walking patterns for humanoid robot NAO. Both simulations and experiments are successfully performed, which demonstrates the feasibility of the proposed method.

The rest of the paper is organized as follows. In Sec. II, we construct a simplified CPG network. Then we use GA to obtain all the parameters in Sec. III. Following Sec. III, we experimentally verify the proposed CPG network on humanoid robot NAO and show the straight and circular walking results. At last, we make the conclusions.

Optimized Central Pattern Generator Network for NAO Humanoid Walking Control

Qing Zhang, Te Tang, Dingguo Zhang*, Shichao Yang, and Yunli Shao

978-1-4799-2744-9/13/$31.00 ©2013 IEEE

Proceeding of the IEEEInternational Conference on Robotics and Biomimetics (ROBIO)

Shenzhen, China, December 2013

1486

II. METHODOLOGY

A. Central Pattern Generator The mathematical model of the CPG utilized in this work is

based on Matsuoka oscillator model. As shown in Fig. 1, a CPG unit is constituted by two neurons, namely extensor neuron and flexor neuron [14]. The oscillator net output outy , is determined by the two neuron outputs 1y and 2y . Three parts contribute to the neuron output: self-inhibition 1ν , mutual inhibition of related neurons (i.e. coupling weights) ω and external inputs 1 2,r r . The model could be expressed as:

1 1 1 1 2 1x x v y rτ β ω= − − + +

2 1 1 1v v yτ = − +

1 2 2 2 1 2x x v y rτ β ω= − − + +

2 2 2 2v v yτ = − + m ax ( , 0), 1, 2i iy x i= =

1 1 2 2o u ty c y c y= − where 1 2 1 2, , , ,c cβ τ τ are internal parameters. β represents the effect of adaptation, 1 2,τ τ are time constants, 1 2,c c are coefficients of the two neurons’ outputs and 1 2 1 2, , ,x x v v are the state variables.

Fig. 1. Structure of a Matsuoka CPG unit. The circle with an “e” represents extensor neuron and the circle with “f” represents flexor neuron.

B. Network Structure In CPG network, one oscillator's output controls one DOF

of humanoid robot. The arrangement of joints for the humanoid robot NAO is illustrated in Fig. 2. As shown in Fig.2, there are 3 joints, namely hip, knee and ankle joints with 4 DOFs on each leg.

For a completely coupled Matsuoka network, there are 40 internal parameters, 16 external inputs and 232 inhibition parameters to be assigned. That is a huge and difficult task. To reduce the complexity and improve the efficiency, we make some simplifications to the CPG network. We assume the left and right parts of the body are connected via hip pitches only. And hip pitch has unidirectional connections with other oscillators. 'Unidirectional connection' means hip pitch oscillator has inhibitions on other oscillators, whereas there is no inhibition in turn. Based on the assumptions, the proposed CPG network structure is shown in Fig.3.

Left Right

Hip Pitch

Knee Pitch

Ankle Pitch

Hip Roll

Fig. 2. Joint arrangement to represent the humanoid robot. It has 8 DOFs for two legs, i.e. left and right hip rolls, hip pitches, knee pitches and ankle pitches.

Fig. 3. CPG network structure for humanoid robot. One pair of neurons forms an oscillator in charge of a DOF. The arrows show the inhibitory directions among different neurons. The characters ‘e’ and ‘f’ indicate extensor neuron and flexor neuron respectively.

In straight walking, a further simplification is that we equate the external inputs 1 2,r r within an oscillator. Taking into account the fact that the desired outputs of two sides are out of phase during straight walking, and the assumption that the left and right leg are joined by hip pitches only, we presume the left hip pitch and the right hip pitch have the same internal parameters and external inputs, as well as symmetrical inhibition parameters. Through these simplifications, the totaled parameters are now reduced to 99, including 35 internal parameters, 7 external inputs and 57 inhibition parameters. In circular walking, we assume the inhibition parameters are the same as straight walking and there are 40 internal parameters and 8 external inputs. The optimization process and results are presented in Sec. III.

III. RESULTS

A. Optimization Method The optimization method we adopt is GA. Improved Euler

method is used when solving the differential equations in (1). Sheffield genetic arithmetic toolbox is chosen to implement the GA [15]. The number of the individual is set to 200 and maximum number of generations to 500. Each parameter of

(1)

1487

the CPG network is expressed as 10 digits in binary coding. The generation gap is set as 0.9. The mutation rate is 0.5 and crossover rate is 0.7. All the calculations are performed in Matlab.

The reference data are obtained from the angle sensors installed in humanoid NAO. The data includes the angles of hip, knee, ankle pitches and hip rolls of both legs. In view of the assumed relationship between hip, knee and ankle pitches described above, the optimization process is divided into five steps to simplify the calculation.

Step.1: Optimize two hip pitches Step.2: Optimize left knee pitch and left ankle pitch Step.3: Optimize right knee pitch and right ankle pitch Step.4: Optimize left hip roll Step.5: Optimize right hip roll

The execution of latter steps will not affect the results of former steps due to the unidirectional inhibition relationship described above. The process is described in Fig. 4.

Fig. 4. Five steps for optimization of CPG network in sequence: (a) Step.1: optimize the connections between two contralateral hip pitches. (b) Step.2: optimize the connections among unilateral hip, knee and ankle joints for one leg. (c) Step.3: optimize the connections among unilateral hip, knee and ankle joints for the other leg. (d) Step.4: optimize the connections between hip pitch and hip roll for one leg. (e) Step.5: optimize the connections between hip pitch and hip roll for the other leg. Note that (b) and (d) have symmetry structures to (c) and (e) respectively.

B. Optimization Results The optimization results for straight and circular walking

are demonstrated in Fig. 5(a)-(h) and Fig. 6(a)-(h). The relative error of each step for straight walking is 7.5%, 4.4%, 6.7%, 4.5% and 5.2% in sequence. The results manifest that all the outputs of CPG are well fitted to the desired outputs, which validates our simplification to the network. Fig. 5 (i) illustrates that there is only a phase difference between the angles of each hip joint approximately during straight walking.

In contrast, Fig. 6 (i) indicates that both the amplitude and the phase are different. That’s the key to circular walking. The optimized internal CPG parameters for straight and circular walking are shown in Table 1 and Table 2, and the inhibition parameters are shown in Appendix.

10 15-0.6

-0.4

-0.2

10 15-0.6

-0.4

-0.2

10 150.5

1

1.5

10 15-0.8

-0.6

-0.4

10 150.5

1

1.5

10 15-0.8

-0.6

-0.4

10 15-0.2

0

0.2

10 15

-0.2

0

0.2

Angle(Rad)

Angle(Rad)

Angle(Rad)

10 15-0.6

-0.4

-0.2

Fig. 5. Optimized results of all joints in straight walking: (a) Left hip pitch angle. (b) Right hip pitch angle. (c) Left knee pitch angle. (d) Left ankle pitch angle. (e) Right knee pitch angle. (f) Right ankle pitch angle. (g) Left hip roll angle. (h) Right hip roll angle. Dashed lines represent the desired outputs and solid lines represent the outputs of optimized CPG network. (i) Reference data of two Hip pitches.

10 15-0.6

-0.4

-0.2

10 15

-0.4

-0.2

10 150.5

1

1.5

10 15-0.8

-0.6

-0.4

10 150.5

1

1.5

10 15-0.8

-0.6

-0.4

10 15

-0.4

-0.2

10 15-0.5

0

0.5

10 15-0.5

0

0.5

Angle(Rad)

Angle(Rad)

Angle(Rad)

Fig. 6. Optimized results of all joints in circular walking: (a) Left hip pitch angle. (b) Right hip pitch angle. (c) Left knee pitch angle. (d) Left ankle pitch angle. (e) Right knee pitch angle. (f) Right ankle pitch angle. (g) Left hip roll angle. (h) Right hip roll angle. Dashed lines represent the desired outputs and solid lines represent the outputs of optimized CPG network. (i) Reference data of two Hip pitches.

Table 1: Optimized internal CPG parameters of straight walking

lhp/rhp lkp lap rkp rap lhr rhr 1τ 0.399 1.358 2.906 2.370 2.722 1.390 1.728 2τ 1.511 0.050 1.238 0.912 0.722 1.531 2.018

β 5.191 12.00 5.467 9.279 8.540 6.370 8.059 r 2.452 0.645 0.646 2.235 2.510 4.581 2.862 1c 1.261 2.070 1.138 6.000 0.868 1.860 3.971 2c 0.610 1.883 9.655 2.188 2.252 1.155 1.003

Table 2 Internal CPG parameters of circular walking

1488

lhp rhp lkp lap rkp rap lhr rhr 1τ 0.350 0.350 0.874 1.590 0.771 1.815 1.865 0.824 2τ 1.343 1.343 0.155 0.680 0.982 0.651 0.510 1.636

β 9.854 9.854 11.95 5.056 8.985 8.751 7.261 11.99 r 0.880 0.880 1.408 0.575 0.704 1.132 0.880 1.050 1c 2.311 5.278 4.123 1.965 20.00 2.159 4.534 5.226 2c 2.827 2.985 0.839 11.70 1.255 6.393 3.062 3.267

IV. EXPERIMENTAL IMPLEMENTATION After the optimization, the proposed CPG controller is

applied to simulation environment (NAOsim) and real bipedal robot NAO. NAO is a programmable humanoid robot developed by Aldebaran Robotics with 25 degrees of freedom. NAO’s joint angles and corresponding motor speeds can be accurately controlled using programming language (C++/Python). The software NAOsim offers a real-time robot simulation in a customizable virtual world so that the proposed controller can be tested before examining it on the real robot.

The CPG based controller for NAO's walking is realized by C++ programs. After online computation, all joint-angle data are sent to virtual robot in NAOsim and real robot through Wi-Fi. The structure diagram of experimental work is shown in Fig.7.

Fig. 7. Structure diagram of experimental work. Robot walking is realized both in simulations and experiments. DCM stands for Device Communication Management, which is used for communication with all electronic devices in NAO.

A. Straight Walking Each gait cycle of straight walking contains two steps and

the period is set to 3 seconds. The robot moves forward 0.04 meter per gait cycle. Fig. 8 shows the successful application to virtual robot in NAOsim.

Fig. 8. One cycle of straight walking of virtual robot in NAOsim environment. Photos were captured every 0.6 sec. The cycle starts from double stance posture and ends with swinging.

Fig. 9. One cycle of successful straight walking of real robot NAO. Photos were captured every 0.6 sec. in patch.

Fig. 9 demonstrates the performance in real robot NAO. In the snapshots, the robot starts from double stance posture, then it lifts the right foot and at the same time the torso tilts towards the left. After the right foot touches ground, the robot recovers the center of mass by swinging. The process of left leg's lifting and dropping is similar.

B. Circular Walking In circular walking, NAO's left leg's stride is larger than the

right leg's, which leads to NAO turning right. The gait cycle is also set to 3 seconds. Under this condition, NAO rotates 0.349 radians per step. The demonstration in simulation and experiment is shown in Fig.10.

(a) (b)

Fig. 10. Circular walking of the robot: (a) simulation in NAOsim (b) demonstration in reality.

V. DISCUSSION AND CONCLUSION Compared to the previous work [5, 13], whose joint

arrangements are similar to ours, the joint number is equal to that in [5] and more than that in [13], but the CPG network is much simpler than theirs. The network structure proposed in this article has 3 advantages. Firstly, the number of parameters is decreased to 99, compared with 271 in [5]. and 144 in [13], while the relative error does not rise (about 5%). The simplification of the network greatly decreases the complexity for optimization and computing. Secondly, by changing mutual inhibitions into unidirectional connections between hip pitch oscillator and other oscillators, all the 99 parameters can be separately optimized in 5 steps, instead of being calculated all together. This also reduces the optimized parameters in one step. We adopt the strategy to compute in block and update parameters locally, which further simplify the optimization and improve the efficiency. Finally, fewer inhibited connections between two legs facilitate their separate controls. Changing the CPG parameters in one leg's oscillators has little influence on the other’s outputs.

We simplify the CPG network and realize two different

1489

biped gaits. An innovative and simplified CPG network structure is proposed. Based on our assumption, we omit some unnecessary connections between left and right leg and build unidirectional connections from hip to other joints.

GA is used to optimize CPG parameters. On the basis of the unidirectional connections, we divide the optimization process into five steps, instead of optimizing all the parameters in one time and this reduces the complexity and improves the efficiency of calculation.

Furthermore, straight and circular walking based on humanoid robot NAO is successfully realized in simulations and experiments, which proves the feasibility of the proposed CPG controller.

However, this work needs some further improvements. Firstly, we haven’t added sensory feedback to this network, which limits the performance of the controller. This drawback causes problems in circular walking control. In future, the adaptivity of CPG network should be improved in response to the environmental changes. Feedback learning algorithms [16, 17] may be good alternatives for enabling CPG to adjust the parameters on different terrains. Secondly, the theoretical analysis may be used to assist the optimization process. Finally, we will apply the proposed network into more cases to see if it can fulfill more tasks in bipedal walking.

Appendix Matrix of inhibitory connections among the units in CPG network:

,

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

1

1 2

2 3

4

A 0 0 0B A 0 0

YB 0 A 0B 0 0 A

0 4.092 1.543 0.944 0 1.179 -5.977 0.8044.921 0 0.944 5.941 5.660 0 2.317 2.4461.542 0.944 0 0.942 2.258 0.135 0 0.4400.944 5.941 4.921 0 3.959 2.235 -5.267 0

0 4.710 3.924 2.8455.860 0 1

− − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥− −⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥− −⎢ ⎥ ⎢ ⎥− − −⎣ ⎦ ⎣ ⎦

−

=

1 2

3

A A

A

， ，

0 6.000 0 0.953 1.331 1.672 0 0 0

1.097 0.006 0 1.191 0 0 0 0.5460.628 1.367 -11.202 0 0 0 0 0

2.938 5.003 0 0 0 0 5.414 -1.1202.821 -2.991 0 0 0 0 1.601 -0.422-1.672 5.026 0 0 0 02.199 2.024 0 0

−⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

⎡ ⎤⎢ ⎥⎢ ⎥= =⎢ ⎥⎢ ⎥⎣ ⎦

4

1 2

A

B B

， ，

，-1.414 4.804

0 0 10.874 -2.258

4.815 2.282 0 0-0.370 0.346 0 0

0 0 -3.584 -1.4020 0 6.000 -0.522

=[ _ _ _ _ _ _ _ __ _ _ _ _ _ _

y lhpe y lhpf y rhpe y rhpf y lkpe y lkpf y lape y lapfy rkpe y rkpf y rape y rapf y lpre y lhrf y rhre y

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦

3B

Y

，

，

_ ]Trhrf

where lhpe represents left hip pith extensor neuron, rhrf represents right hip roll flexor neuron, and so on.

References [1] F. Delcomyn, "Neural basis for rhythmic behaviour in animals,"

Science, vol. 210, pp. 492-498, Oct. 1980. [2] R. A. Brooks, "A robot that walks; emergent behaviors from a

carefully evolved network," Neural computation, vol. 1, pp. 253-262, 1989.

[3] M. Vukobratović and B. Borovac, "Zero-moment point—thirty five years of its life," International Journal of Humanoid Robotics, vol. 1, pp. 157-173, Mar. 2004.

[4] A. J. Ijspeert, "Central pattern generators for locomotion control in animals and robots: a review," Neural Networks, vol. 21, pp. 642-653, May 2008.

[5] H. Inada and K. Ishii, "Behavior generation of bipedal robot using central pattern generator (cpg)(1st report: CPG parameters searching method by genetic algorithm)," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas, NV, 2003, pp. 2179-2184.

[6] K. Hase, K. Miyashita, S. Ok, and Y. Arakawa, "Human gait simulation with a neuromusculoskeletal model and evolutionary computation," The Journal of Visualization and Computer Animation, vol. 14, pp. 73-92, Apr. 2003.

[7] N. Ogihara and N. Yamazaki, "Generation of human bipedal locomotion by a bio-mimetic neuro-musculo-skeletal model," Biological Cybernetics, vol. 84, pp. 1-11, Jan. 2001.

[8] C. Paul, "Bilateral decoupling in the neural control of biped locomotion," The 2nd International Symposium on Adaptive Motion of Animals and Machines, Kyoto, 2003.

[9] C.-S. Park, Y.-D. Hong, and J.-H. Kim, "Full-body joint trajectory generation using an evolutionary central pattern generator for stable bipedal walking," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, 2010, pp. 160-165.

[10] K. I. Na, C. S. Park, I. B. Jeong, S. Han, and J. H. Kim, "Locomotion Generator for Robotic Fish Using an Evolutionary Optimized Central Pattern Generator," IEEE International Conference on Robotics and Biomimetics, Tianjin, China, 2010.

[11] C.-S. Park, J.-K. Yoo, Y.-D. Hong, K.-B. Lee, S.-J. Ryu, and J.-H. Kim, "Walking pattern generator using an evolutionary central pattern generator," in Trends in Intelligent Robotics, ed: Springer, 2010, pp. 65-72.

[12] G. Taga, Y. Yamaguchi, and H. Shimizu, "Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment," Biological cybernetics, vol. 65, pp. 147-159, Jul. 1991.

[13] S. M. Saif, "Central Pattern Generator Parameter Search for a Biped Walking Robot," European Journal of Scientific Research, vol. 57, pp. 466-477, 2011.

[14] K. Matsuoka, "Mechanisms of frequency and pattern control in the neural rhythm generators," Biological Cybernetics, vol. 56, pp. 345-353, Jul. 1987.

[15] J. H. Holland, Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence, 2nd ed.: U Michigan Press, 1975.

[16] G. Endo, J. Nakanishi, J. Morimoto, and G. Cheng, "Experimental studies of a neural oscillator for biped locomotion with QRIO," IEEE International Conference on Robotics and Automation (ICRA), Barcelona, 2005, pp. 596-602.

[17] L. Righetti and A. J. Ijspeert, "Programmable central pattern generators: an application to biped locomotion control," IEEE International Conference on Robotics and Automation (ICRA), Orlando, Florida, 2006, pp. 1585-1590.

1490