An Untagged Human Hand Motion Classification

by Pyroelectricity

Chun-Ching Hsiao Department of Mechanical Design Engineering, National Formosa University, No. 64, Wunhua Rd., Huwei Township,

Yunlin County 632, Taiwan

Email: cchsiao@nfu.edu.tw

Abstract—Pyroelectric sensors are the most suitable for

detecting human motions because they are sensitive to

human motions induced variances in the infrared radiation.

A pyroelectric sensor array is designed and used to build an

event-time sequence in an infrared radiation domain. Then,

the sequence is used to contrast with a password sequence

for further recognizing human hand motions. This is

applied on the intelligent life for controlling doors to open or

close as a pyroelectric lock. For estimating the functions of

the pyroelectric lock, six password sequences are used to

control the proposed device via discreet manipulation of six

persons. The experimental results verify the feasibility and

efficacy of the proposed design. The false recognition is

lower than 2%.

Index Terms—pyroelectricity, human hand motion,

pyroelectric infrared sensors, thermal vision

I. INTRODUCTION

To provide a supplement and alternative to traditional

vision-based approached, pyroelectric infrared

technology is capable of detecting human motion by

sensing their thermal radiation variations. Therefore,

infrared sensors are more suitable for tracking an

untagged human limb motion induced variances in the

infrared radiation. A human limb tracking system based

on infrared technology can be low-cost, low-data-

throughput, fast response speed, small volume, low

power consumption, nonintrusive sensory approaches

without interference, and unnecessary complex

computations compared with the expensive and

complicated computer vision-based system [1], [2].

The pyroelectric effect is the change in spontaneous

polarization that appears in ferroelectric materials with a

time dependent temperature variation. When a

pyroelectric material is applied to a temporal temperature

change, then the net dipole moment of the polar material

is modified and hence the value of the spontaneous

polarization. The pyroelectric material is associated with

the top and the bottom electrode and an electrical charge,

a current will appear as long as the temperature changes.

Considering a pyroelectric material with charge

collection electrodes, the pyroelectric current flows from

these electrodes in an electric load can be written as [3]:

Manuscript received November 19, 2015; revised February 16, 2016.

IP = η × p × A × dT / dt

where, IP is the pyroelectric current, η is the absorption

coefficient of radiation, p is the pyroelectric coefficient, A

is the active surface area, dT/dt is the temperature

variation rate. Pyroelectric materials have been widely

used for dynamic temperature measurements, infrared or

terahertz detection but also waste heat energy harvesting.

Moreover, pyroelectric infrared detectors exhibit broad

and uniform spectral response which makes them

desirable for many applications such as fire monitoring,

gas detection and security system. The operation of

pyroelectric detectors is to convert infrared radiation

energy into heat and further into electrical signal. When

the dimensions and the materials of pyroelectric elements

are determined, various moving velocities of thermal

objects resulting various temperature variation rates will

directly affect the pyroelectric current and voltage

responsivities. Moreover, the temperature variation rate

has a huge maneuverability according to the design of the

patterns, trenches, cavities and structures in the

pyroelectric materials. However, the temperature

variation rate is difficult to extract from pyroelectric

layers by experimental measurements. Some finite

element models were built using the commercial software

package COMSOL to explore the temperature variation

rate in pyroelectric cells, with some designs of cavities by

wet etching, trenches by a precision dicing saw, and

grooves by a sandblast etching technique, in order to

improve the energy conversion efficiency of PZT cells by

pyroelectricity [4]-[6]. Moreover, Hsiao et al. [7] used an

aerosol deposition to grow the ZnO films by laser

annealing for rapidly fabrication on pyroelectric sensors.

The pyroelectric effect has been proven to be

advantageous in smart and intelligent system and focused

on the applications of sensors and power generators.

Recently, the pyroelectric power generations have been

widely used and studied from time-dependent

temperature fluctuations. The efficiency of the generators

was improved when Olsen et al. [8] proposed using the

Ericsson cycle in the pyroelectric energy harvesting. It

enables the generators to approach Carnot efficiency.

However, using the pyroelectric effect for energy

harvesting lies in finding promising temperature

variations or a circulating thermal loading in real

circumstances. Hsiao et al. [9] proposed a novel

pyroelectric harvester integrating solar radiation with

International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016

©2016 Int. J. Electron. Electr. Eng. 426doi: 10.18178/ijeee.4.5.426-429

wind power. A thermal source is caught from solar

radiation and wind is a dynamic power. This design used

a mechanism to convert the rotary energy of the disk

generator to drive a shutter for generating temperature

variations in pyroelectric cells using a planetary gear

system. The pyroelectric harvester could play a

complementary role when the disk generator was inactive

in situations of low wind speed.

The pyroelectric infrared sensors with the visibility

modulations have been capable of identifying and

classifying the human motions with high sensing

efficiency. When the pyroelectric materials are heated,

the net polarization decreases and the entropy increases

due to a reorientation of the dipole moment. The decrease

of polarization results a current for applying to an electric

circuit. Hence, monitoring the current or voltage

responsivities of the pyroelectric cells can probe human

limb motions. The pyroelectric sensor array can further

detect the orientations and velocities of human limb

motions. Human limb tracking could be extensively

applied in surveillance, robotics, intelligent home and gait

biometrics. This study proposes a low-cost human limb

motion tracking system based on a pyroelectric infrared

sensor array, and this system is further applied on the

intelligent life for controlling doors to open or close as a

pyroelectric lock. The pyroelectric lock is based on the

pyroelectric infrared technology with a sensor array for

sensing human limb motion, then it further confirms the

orders for controlling an electromagnetic valve as an

entrance guard.

II. MATERIALS AND METHODS

The pyroelectric lock consists of pyroelectric infrared

sensors, a shadow mask, an electromagnetic valve and a

NI LabVIEW system, which is depicted in Fig. 1. Nine

pyroelectric infrared sensors with D205B type are used to

construct a 3×3 square array. Fig. 2 shows the assembly

of the sensor array for tracking the human hand motions.

The output signals of the pyroelectric sensors are further

treated via an operational amplifier (LM741), as shown in

Fig. 3. The amplifier is a non-inverting amplifier and has

a close-loop gain with a ratio of output voltage to input

voltage as 1 + RO/RI. Moreover, the shadow mask is used

to insulate the interference between the infrared sensors

and restrict the sensing region of infrared sensors. The NI

LabVIEW system consists of a case of NI PXIe-1082, a

controller of NI PXIe-8135, a data acquisition card of NI

PXIe-6366 and NI LabVIEW 2014 software, which

mainly acquires the pyroelectric signals from nine sensors

to further filter, amplify, modulate and digitize the

analogy signals as the digital signals. The digital signals

are transferred into event-time sequences. The sequences

are used to compare with set sequences as password

sequences for further confirming the orders and

controlling the electromagnetic valves as the entrance

guards. For digitizing pyroelectric signals, a simple and

effective signal processing method, as depicted in Fig. 4,

is adopted to integrate pyroelectric signals and time

domain signals as follows:

(1) Calculate the incremental data from the raw data,

and further take them as absolute data fluctuation. The

incremental data is the data difference between current

moment and last moment.

(2) Integrate the incremental data with the raw data as

the characteristic signal.

(3) Set a threshold and further digitize the

characteristic signal.

(4) Smooth the digital signal and integrate the digital

signal and time as the event-time sequence.

Figure 1. Schematic diagram for the setup of the pyroelectric lock.

Figure 2. Schematic diagram for the sensor array.

Figure 3. The electrical circuit of the pyroelectric sensor.

International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016

©2016 Int. J. Electron. Electr. Eng. 427

Figure 4. The treatment flow of the pyroelectric signals.

The Event-Time Sequence (ETS) can be expressed as

a matrix with the number of Pyroelectric Sensors (PS) as

column elements and the duration (DT) as row elements.

EST=ETSi,t=[

𝐸𝑆𝑇1,1 ⋯ 𝐸𝑆𝑇1,𝐷𝑇

⋮ ⋱ ⋮𝐸𝑆𝑇𝑃𝑆,1 ⋯ 𝐸𝑆𝑇𝑃𝑆,𝐷𝑇

]

iϵ{1, 2, …., PS}, tϵ{1, 2, …., DT}

The PS indicates the sensing spatial resolution for a

given sensing area, which is a trade-off between the

required spatial resolution and the limited physical

response of the pyroelectric sensors. The issue of human

limb motion recognition for further confirming the orders

can be solved through measuring motion similarities

between a test and a password sequence in the event-time

domain. A training set is selected from the ETS as a

password sequence (P), and then a test sequence (Q) is

measured from the ETS associated with a query motion.

For evaluating the similarity between the test and the

password sequence, a criterion of the Median Hausdorff

Distance (MHD) is used and expressed as [10]:

MHD(Q, P)=𝑚𝑒𝑑𝑖𝑎𝑛

𝑸𝒕 ∈ 𝐜𝐨𝐥(𝑸)(

𝑚𝑖𝑛𝑷𝒕 ∈ 𝐜𝐨𝐥(𝑷)

(‖𝑸𝒕 − 𝑷𝒕‖))

where col(∙) represents the set of the column vectors of

the embraced matrix, Qt is the column vector extracted

from the test sequence, Pt is also the column vector

extracted from the password sequence, ║∙║ denotes the

Euclidean distance from Qt to Pt, and the operator

‘median’ selects the median ranked value from min ║∙║

over Q. Moreover, the median Hausdorff distance is a

rather robust criterion with less sensitive to outlier

observations. A smaller median Hausdorff distance

represents the more similar between the password and the

test sequence. The exactitude for confirming the orders

can set the median Hausdorff distance. An excessively

small median Hausdorff distance will increase the

difficulty to confirm the orders. However, an excessively

large distance will induce an erroneous judgement. The

set value of the median Hausdorff distance will decide the

sensitivity of the pyroelectric lock to recognize human

limb motion.

III. RESULTS AND DISCUSSION

For validating the usability of the pyroelectric lock,

human right hand motions are adopted to operate the

pyroelectric lock. All exercises are performed in a

30cm×30cm×10cm cube. The pyroelectric sensor array is

placed in front of the human right hand. The distance

between the array and the hand is nearly 10cm. Fig. 1

shows the sketch for using the human right hand with a

fist gesture to operate the pyroelectric lock. The fist

gesture can minimize the size of heat radiation for

confirming the path of hand motions detected by the lock

consisted of the pyroelectric sensor array. Moreover, six

right hand motions as the exercises are transferred to the

password sequences for verifying the capability of the

pyroelectric lock. The images of these motions are

depicted as in Fig. 5. These right hand motions are Z

type(RM1), × type(RM2), ㄇ type(RM3), ㄩ type(RM4),

┴ type(RM5) and ⌂ type(RM6). The database of these

motions is constructed from six persons to perform the

exercises. These persons have various physical

characteristics of 160-180cm in height and 60-80kg in

weight and perform these motions with both the motion

styles and velocities respectively, which provides more

realistic data for testifying the versatility of the

pyroelectric lock.

Figure 5. Schematic diagram for the hand motions as password sequences.

The database from the sensor array capturing six

persons, each person performing six hand motions and

each motion repeated six times, which is built as 216

(6×6×6) sequences. Thus, each motion generates 36 (6×6)

sequences, each sequence is associated with a 9×DT

matrix. Here DT represents the duration of each motion

and the array consists of nine pyroelectric sensors.

Furthermore, 36 sequences built by six persons repeated

six times to present a hand motion, these sequences are

merged as a password sequence for charactering the

motion. Fig. 6 shows the median Hausdorff distances

between the password sequence and the test sequence,

where rm1 to rm6 represent the test motions and RM1 to

RM6 represent the password sequence. It is obvious that

the each test motion is nearest to the corresponding

International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016

©2016 Int. J. Electron. Electr. Eng. 428

password motion due to the smallest median Hausdorff

distance. Fig. 7 shows a confusion matrix between the

password motion and the test motion for presenting a few

false recognition lower than 2%. Hence, the spatio-

temporal sequence can achieve the classification of hand

motions with the low-dimension.

Figure 6. Median Hausdorff distance between the password sequence and the test sequence.

Figure 7. Confusion matrix for the classification of hand motions.

IV. CONCLUSIONS

The spatiotemporal sequence is used to characterize

the human hand motion features by the pyroelectric

sensor array. The test sequence is compared with the

password sequence for recognizing the human right hand

motions with six patterns. The proposed device can

capture the low-dimensional feature sequences in real

time without extra computation. The six right hand

motions can be classified from 216 series data of the

pyroelectric sensor array. This is used on the intelligent

life for controlling doors as a pyroelectric lock. The false

recognition is lower than 2%. The pyroelecticity is

successfully applied on tracking the untagged human

hand motion in the infrared radiation domain without the

need of heavy computations.

ACKNOWLEDGMENT

The authors are thankful for the financial support from

the Ministry of science and Technology (Taiwan) through

Grant No. MOST 104-2221-E-150-013, and the

experimental support from the Common Laboratory for

Micro and Nano Science and Technology at National

Formosa University, and Nano-Electro-Mechanical-

Systems (NEMS) Research Center at National Taiwan

University.

REFERENCES

[1] Q. Guan, C. Li, X. Guo, and G. Wang, “Compressive

classification of human motion using pyroelectric infrared sensors,”

Pattern Recognit. Lett., vol. 49, pp. 231-237, 2014.

[2] B. Yang, J. Luo, and Q. Liu, “A novel low-cost and small-size

human tracking system with pyroelectric infrared sensor mesh

network,” Infrared Phys. Techn., vol. 63, pp. 147-156, 2014.

[3] R. W. Whatmore, “Pyroelectric devices and materials,” Rep.

Progr. Phys., vol. 49, pp. 1335-1386, 1986.

[4] C. C. Hsiao, J. C. Ciou, A. S. Siao, and C. Y. Lee, “Temperature

field analysis for PZT pyroelectric cells for thermal energy

harvesting,” Sensors, vol. 11, pp. 10458-10473, 2011.

[5] C. C. Hsiao, A. S. Siao, and J. C. Ciou, “Improvement of

pyroelectric cells for thermal energy harvesting,” Sensors, vol. 12,

pp. 534-548, 2012.

[6] C. C. Hsiao and A. S. Siao, “Improving pyroelectric energy

harvesting using a sandblast etching technique,” Sensors, vol. 13,

pp. 12113-12131, 2013.

[7] C. C. Hsiao and S. Y. Yu, “Rapid deposition process for zinc

oxide film applications in pyroelectric devices,” Smart Mater.

Struct., vol. 21, 2012.

[8] R. B. Olsen, “Ferroelectric conversion of heat to electrical energy

- A demonstration,” J. Energy, vol. 6, pp. 91-95, 1982.

[9] C. C. Hsiao, J. W. Jhang, and A. S. Siao, “Study on pyroelectric

harvesters integrating solar radiation with wind power,” Energies,

vol. 8, pp. 7465-7477, 2015.

[10] D. M. Mount, N. S. Netanyahu, and J. L. Moigne, “Efficient

algorithms for robust feature matching,” Pattern Recogn., vol. 32,

pp. 17-38, 1999.

Chun-Ching Hsiao is an associate professor of

the Department of Mechanical Design Engineering at the National Formosa

University, Taiwan. His researches focus on

pyroelectric sensors, flexible devices, thermoelectric generators, pyroelectric

harvesters, pyroelectric human-machine

interface and residual stress of thin film.

International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016

©2016 Int. J. Electron. Electr. Eng. 429