View
221
Download
2
Category
Preview:
Citation preview
Traffic Police Gesture Recognition using Accelerometers
Ben Wang Department of Automation
Tsinghua University Beijing 100084, China
Email: wben06@mails.tsinghua.edu.cn
Tao Yuan Department of Automation
Tsinghua University Beijing 100084, China
Email: yt@mail.tsinghua.edu.cn
Abstract—When an automatic traffic light system is not used due to too heavy traffic, the traffic would be controlled by traffic police gesture. This paper is about the design of a system so that the traffic lights can follow the traffic police gestures. To simplify the system, a unique mapping between the traffic police gestures and the orientation and movement of hands is defined. The hand motion characters are extracted by fixing a 3-axis accelerometer on the back of each hand. A 2-level hierarchical classifier is used to recognize the gestures. First the gestures are categorized into three groups, according to the movement of each hand. Then a gesture is recognized by comparing it with the predefined templates. This real-time recognition algorithm is implemented by a micro-controller. It is envisaged that this will help drivers.
I. INTRODUCTION When traffic is very heavy, an automatic traffic light
system is not efficient to control traffic, causing traffic jam. In this case, it is necessary to switch off the traffic light and let traffic police guide traffic by gestures. In the case of bad weather or obstruction by other vehicles, however, sometimes it is difficult for all drivers to recognize the gestures. It would be useful if the traffic light can follow the traffic police gestures.
Two methods are considered suitable for gesture recognition. The first one is to use vision sensors like cameras to acquire images, which are analyzed to recognize the gestures [1] [2]. The second one is to place inertial sensor on the traffic police hand and extract the motion characters.
The most advantage of the vision method is that it can recognize gestures without adding any extra hindrance to the police. However, it suffers from poor illumination, e.g. at night or in fog weather.
Due to the advantages of low cost and small size, MEMS accelerometers have been used widely in gesture recognition. As different signals correspond to different gestures, hand gestures can be used to control home appliances, like TV and DVD [3]. An accelerator-glove has been designed by fixing
a 2-axis accelerometer on the back of each finger. By comparing the output signals of the accelerometers and the magnitude of gravity, the bending angles of the fingers can be recognized. It is used further to recognize 26 letters of American Sign Language (ASL) [4].
This paper presents a Chinese traffic police gesture recognition system. By fixing two 3-axis accelerometers on the back of both hands, so the arm movement and hand position, when the arm is steady, can be extracted. By recognizing traffic police gestures and synchronizing the traffic lights with them, it is envisage that this application will give help to vehicle drivers.
II. CHINESE TRAFFIC POLICE GESTURE SYSTEM The Chinese traffic police gesture system is defined and
regulated by Chinese Ministry of Public Security [5]. Essentially, it is an arm gesture system, which means the gestures are mainly expressed by two arms, which are either steady or oscillation. Currently, it consists of 8 gestures:
(1) stop,
(2) move straight,
(3) left turn,
(4) left turn waiting,
(5) right turn,
(6) lane changing,
(7) slow down,
(8) pull over.
In the recognition system described in this paper, a “stand in attention” gesture is added. It is police stand in attention. The reason for this is that traffic police will keep stand in attention when they do not change the current traffic. If a “stand in attention” gesture is recognized, it is unnecessary to change the traffic lights. Fig.1. shows the nine gestures.
1-4244-2581-5/08/$20.00 ©2008 IEEE 1080 IEEE SENSORS 2008 Conference
Fig. 1 Chinese traffic police gestures
In these gestures, “stop”, “move straight”, “left turn”, “left turn waiting” and “right turn” can be followed by standard traffic lights. If these gestures are detected, the traffic lights can be synchronized with them. Other gestures, like “lane changing”, “slow down” and “pull over”, cannot be shown in standard traffic lights. However, it is necessary to recognize all the gestures. First, the gestures, like “lane changing” and “slow down”, can be shown in some emergency traffic lights. Second, in long term, with the coming of intelligent automobile, unmanned or auto-driving vehicles, it is necessary to have them under some special conditions to follow the commands of traffic police. There are two possible solutions to this: the active way or the passive way. The first way is to use cameras on unmanned vehicles to recognize the traffic gestures. The latter way is to use on body sensors to recognize the traffic police gestures. Due to the illumination problems mentioned before, the latter way is a better choice than the first one.
III. THE SYSTEM
A. System description The interconnection of all the components is shown in
Fig. 2. The key components of this system are two 3-axis accelerometers. They can provide signals of arm movement and hand position with respect to the gravitational vector. The signals from the accelerometers are collected by a micro-controller and then analyzed. An NEC78K0547 micro-controller is used in this application [6]. This micro-controller has 128 Kbyte flash, 7 Kbyte RAM, 8 channels of 10-bits ADC, 2 UART (Universal Asynchronous serial receiver and transmitter) ports and 3 SPI (Serial Peripheral
Interface) ports. The micro-controller is connected to a PC via a USB port. In our laboratory, a PC is used to simulate traffic lights.
PC
Microcontroller
Left hand accelerometer
Right hand accelerometer
SPI
USB
Fig. 2 System block diagram
B. Selection and placement of sensors The LIS3LV02DQ is a successful product form ST, Inc.
[7]. It is a low cost, low power, digital output 3-axis accelerometer on a single IC chip, which can provide both dynamic and static acceleration signals. The measurement range is selectable to be ±2g or ±6g. Each axis output is represented by two bytes, in which 12 bits are significant. When the measurement range is set to ±2g, the sensitivity can be 1 mg. The accelerometer communicates with the micro-controller via a SPI port, and no additional A/D converter is needed. In the system, 12 bytes data are read sequentially by the micro-controller, beginning with the right hand accelerometer, X axis followed by the Y axis and then the Z axis, then the left hand accelerometer with the same sequence.
It is crucial to place the sensors in appropriate positions for obtaining unique gesture measurements with the least sensors. In this paper, after a close scrutiny of Chinese traffic police gestures, it is decided to fix two accelerometers on the back of hand, which is shown in Fig. 3. The X and Y axis are parallel to the hand, and the Z axis is perpendicular to the hand and points to the palm.
Fig. 3 Sensors on back of both hands
1081
C. Data collection and visualization In the system, the data processing and gesture recognition
are performed in the micro-controller. During the initial stage, to reach a better effect, it is necessary to modify the parameters for gesture recognition. It would be useful if the signals can be displayed directly on the PC screen. The micro-controller sends the data to the PC by sets, and the signals are displayed on the PC screen using the Matlab software.
IV. SIGNAL PROCESSING
A. Signal preprocessing Because the output of accelerometer may suffer form the
interference of high frequency white noise, it is necessary to process the signal by a low-pass filter [8]. Considering the capacity of the micro-controller, a 3rd order Butterworth filter is used.
B. Oscillation and steady detection If an arm is steady, the hand acceleration is almost
constant. If the arm is oscillating, the hand acceleration verifies much along with the time. In this paper, the resultant
acceleration 2 2 2r x y za a a a= + + is used to represent
the arm movement, where xa , ya and za represent the acceleration outputs of the X axis, Y axis and Z axis. As the standard deviation can be used to describe the variety of a data stream, the following criteria are used for steady and oscillation classification.
Criterion 1: For a sampled time instance M, if ( )S
N thMσ σ> for M = 1A , 1 1A + ,… 1A N+ , then
1A is the beginning of an oscillation.
Criterion 2: For a sampled time instance M, if ( )S
N thMσ σ< for M = 2A , 2 1A + ,… 2A N+ ,
then 2A is the end of an oscillation.
Where N is the number of data samples, ( )SN Mσ is a
standard deviation of ra over N samples, and thσ is the threshold of the standard deviation.
In Fig. 4, a typical profile of the magnitude of acceleration ra and its standard deviation are given. Fig. 4 also shows how to find the beginning and end of an oscillation.
0 50 100 150 200 250 300-1000
-500
0
500
1000
Samples
acce
lera
tion(
mg)
0 50 100 150 200 250 3000
200
400
600
Samples
Sta
ndar
d de
viat
ion(
mg
)
Threshold
2
M
A1 A2
Figure 4. A typical process containing oscillation and steady state is
separated
V. GESTURE RECOGNITION A two stage classification is used for gesture recognition.
First, according to whether the arm is oscillating or steady, the dispatcher divides the gestures into three categories. After that, the recognizer I and II are used to recognize the gesture. Fig. 5 shows how the gestures are classified and recognized.
Fig. 5 Classifier’s Hierarchy, each number representing a numbered
gesture in Fig 1 The recognizer I and II are introduced to demonstrate
how they work in the left hand steady and right hand oscillating category.
First, in recognizer I, the output of left hand accelerometer is used to make judgment. Several “if-then” statements are implemented.
1082
If((X axis is vertical-up)&&(Y axis is horizontal)&& (Z axis is horizontal))
Then gesture is “move straight”
If((X axis is horizontal)&&(Y axis is vertical-up)&& (Z axis is horizontal))
Then gesture is “right turn”
If((X axis is horizontal)&&(Y axis is vertical-down)&& (Z axis is horizontal))
Then gesture is “lane changing” or “slow down”
Else gesture is “unknown”.
Because the human gesture is not very precise, a fuzzy definition is used. In our experiment, an angle of -20° to 20° is defined as the horizontal region, 60° to 90° as vertical up and -115° to -75° as vertical down.
Because the gesture “lane changing” and “slow down” have the same steady left hand position, they have to be recognized in recognizer II. As can be seen in Fig. 1, in both gestures the right hand is in oscillation movement. However, the hand oscillates horizontally in the gesture “lane changing” and vertically in “slow down”. The mid value of Z axis is used to distinguish the two gestures. Fig. 6 shows how the two gestures are separated.
0 1 2 3 4 5-2000
-1500
-1000
-500
0
500
1000
1500
2000
time(s)
acce
lera
tion(
mg)
red, Z axis of "lane changing"blue, Z axis of "slow down"
0 1 2 3 4 5-2000
-1500
-1000
-500
0
500
1000
1500
2000
time(s)
mid
val
ue o
f ac
cele
ratio
n(m
g)
red, mid value of "lane changing"blue, mid value of "slow down"
VI. CONCLUSION A novel traffic police gesture recognition system is
presented in this paper. Two 3-axis accelerometers fixed on the back of hand are used as the sensing devices to detect the arm movement and hand position. An algorithm run in an MCU is used to realize the gestures in real time. The software in the PC is used to simulate the traffic lights.
The system has achieved a high recognition rate even when the gestures are not very standard. The main advantage of the on-body system compared to a vision based system is that it will not suffer form occlusion or illumination problems. Future work is to use a wireless module to substitute the USB port. Two nRF2401 wireless modules from NORDIA Company are under test. In addition, energy consumption is always an important concern in a battery-operated system. A “movement wakeup” strategy is being considered in the system.
The gesture recognition approach introduced in this paper can be easily transferred to traffic police gestures in other countries, such as India traffic police gestures.
ACKNOLEDGEMENT The authors would like to thank Engineer Shengli Yang
at NEC-Tsinghua Embedded Research and Education Center for his help in hardware establishment, also, Professor Wuqiang Yang at The University of Manchester is thanked for his help in English.
VII. REFERENCES [1] Yin X M and Xie, M, “Finger identification and hand posture
recognition for human-robot interaction,” Image and Vision Computing, 25 (8), pp. 1291-1300, 2007.
[2] Prieto Abraham, Bellas F, Duro R.J, Lopez-pena F, “An adaptive visual gesture based interface for human machine interaction in intelligent workspaces,” Proc. of IEEE International Conference on Virtual Environments, 2006, pp 43-48
[3] Kallio S, Kela J, Korpipaa P and Mantyjarvi J, “User independend gesture interaction for small handheld devices,” International Journal of Pattern Recognition and Artifical Intelligence, 20 (4), pp. 505-524, 2006.
[4] Hernandez-Rebollar J L, Lindeman R W, and Kyriakopoulos N, “A multi-class pattern recognition system for practical finger spelling translation,” Proc. of 4th IEEE International Conference on Multimodal Interfaces, Oct. 2002, pp 185-190.
[5] Chinese Ministry of Public Security website: www.maps.gov.cn [6] NEC Company website: www.necel.com [7] ST Company website: www.st.com [8] I.J. Jang, W.B. Park, “ Signal processing of the accelerometer for
gesture awareness on handheld devices,” Proc. of IEEE International Workshop on Robot and Human Interactive Communication. Oct. 2003, pp 139-144.
Fig. 6 “lane changing” and “slow down” separated by mid value of Z axis
1083
Recommended