Proceedings of the 2006 IEEEInternational Conference on Robotics and Biomimetics

December 17 - 20, 2006, Kunming, China

A Novel Real-Time Error Compensation Methodology fortIMU-based Digital Writing Instrument

Chi Chiu Tsang, Gary Chun Tak Chow andPhilip H. W. Leong

Department ofComputer Science and EngineeringThe Chinese University ofHong Kong

Hong Kong SAR, China

Guanglie Zhang, Yilun Luo, Zhuxin Dong,Guangyi Shi, Sze Yin Kwok,

Heidi Y. Y. Wong and Wen J. Li*Centrefor Micro andNano SystemsThe Chinese University ofHong Kong

Hong Kong SAR, China

Abstract - A Micro Inertial Measurement unit (jtIMU) whichis based on Micro-Electro-Mechanical Systems (MEMS)accelerometers and gyroscope sensors is developed for real-timerecognition of human hand motion. By using appropriatefiltering, transformation and sensor fusion algorithms, a

ubiquitous digital writing instrument is produced for recordinghandwriting on any surface.

In this paper, we propose a method for deriving an error

feedback to a Kalman filter based on the assumption that writingoccurs only in two dimensions i.e. the writing surface is flat. Byimposing this constraint, error feedback to the Kalman filter canbe derived. Details of the feedback algorithm will be discussedand experimental results of its implementation are comparedwith the simple Kalman filter without feedback information.

Index Terms - Error Compensation; jt-IMU; Human MotionSensing; Kalman Filtering; Digital Writing Instrument.

I. INTRODUCTION

The "Electronic Whiteboard" and "Digital Pen" are new

paradigms in the office automation industry that may somedaycompletely replace the computer keyboard, which is still thepreferred alphanumeric human-to-computer input device. AUbiquitous Digital Writing Instrument has been developed byour group to capture and record human handwriting or

drawing motions in real-time based on a MEMS Micro InertialMeasurement Unit (pIMU) [1].

A Micro Inertial Measurement unit ([tIMU) which isbased on Micro-Electro-Mechanical Systems (MEMS)accelerometers and gyroscope sensors is developed for real-time recognition of human hand motions. By using withappropriate filtering, transformation and sensor fusionalgorithms, a 3D ubiquitous digital writing instrument isinvented for recording handwriting on any surface [1].

In inertial kinematics theory [2], the position is computedby the double integration of acceleration with respect to time.However, due to electronic and mechanical noises in theMEMS inertial sensors, random noise exists in the systemmeasurement output, thus the measured acceleration errors

will be rapidly propagated to the position estimate. Kalmanfiltering can be used to reduce this effect but depends on themeasurement inputs.*Contact author: wen@acae.cuhk.edu.hk.

This project is sponsored by the Hong Kong Innovation and TechnologyCommission (ITF-UIM-151) and by DAKA Development Ltd., Kowloon,Hong Kong.

This application is similar to inertial navigation systemsbut global positioning system (GPS) accuracy and the fact thatit is likely to be used indoors precludes directly using GPS as

the reference. Given that most of the handwriting is on a

surface, there is zero displacement in z-axis in earth frame.Since the pen is oblique to the earth frame during writing, theerror in three axis accelerometers will contribute in the z-axisin the earth frame. We suggest making use of the informationassociated with the z-axis in earth frame to act as the feedbackmeasurement input for Kalman filter to perform a real-timeerror compensation to the acceleration measurement forinertial navigation.

The paper is structured as follows: Section 2 describes thearchitectural design of the ubiquitous digital writinginstrument, including the hardware and software structure. Wewill describe the error compensation algorithm in Sections 3.Simulation and experiment results will be discussed in Section4. Finally, we present conclusions and proposed futureimprovements in the last section.

II. ARCHITECTURE OF WRITING SYSTEM

A. Hardware Architecture ofthe Digital Writing SystemFig. 1 illustrates the block diagram of tIMU with the real-

time position tracking system. The system can be divided intotwo parts. The first part is the hardware for the wirelesssensing unit and the second is the software for data access and3D navigation tracking.

The pIMU is developed for a wireless digital writinginstrument and used to record human handwriting. The iJMUintegrates the 3D accelerometers and 3D gyroscopes withstrapdown installation [1]. The sensor unit is affixed on a

commercially available marker to measure the inertialinformation in the pen's body frame.

The output signals of the accelerometers [Ax, Ay, A] andthe gyroscopes [Wroii, Wpitch, Wyaw], are the body frameaccelerations in three-axes and the angular rates, roll, pitch,yaw respectively. These are measured directly with an

Atmega32L A/D converter microcontroller. The serialBluetooth transceiver is implemented via a USARTconnection with the MCU for wireless communications. Thedigital sample rate of the sensor unit is 200 Hz and thetransmit baud rate is 57.6 Kbps, which ensures rapid reactionto human handwriting [3].

1-4244-0571-8/06/$20.00 C)2006 IEEE

Ming Yiu Wong

DAKA Development LtdHong Kong SAR, China

678

y -axi

Magn eticxUSB Sender Sensors

Bluetooth

Wire WirelessTransmission Transmission ---------------------------

Bluetooth Raw Data Obberer Data DispRec ver Collector

MM _ ~~Algorithmrs Dlata StoredReIF ce ver Al I

User Computer ( Software)Fig. 1 The System Architecture of the 3D Digital Writing System

Fig. 2 shows the prototype of piMU with the Bluetoothmodule for wireless connection. The sensor system utilizesfour-layer printed circuit board techniques for noise reduction.The dimensions are within 56x23x 15mm.

USB

VJ is the velocities in the earth frame: [Vs, Vy, VJ],

Se is the positions in the earth frame: [St, Sy, S],

q is the pen attitude in quaternion format,

G is the gravity vector: [0,0,-g] andL is the distance of the gyroscopes from the pen tip.

.!....

RYUsl_ns

M__ krs

Magnetic Sensor

Ze

> vOe

Xe

I I01JIYb

Fig. 3 The coordinate system of the 3D Digital Writing System

yroscopes

Fig. 2 The Prototype of pIMU with Bluetooth module

III. ERROR COMPENSATION ALGORITHMS

A. Kalman Filtering AlgorithmOwing to the random nature of the noise in the

accelerometers and gyroscopes, a Kalman filter, which is a

stochastic based real-time recursive filter using measurementand noise information, is used to reduce the error of theposition estimate.

B. Algorithmfor Handwriting in the Digital Writing SystemFig. 3 illustrates the pIMU sensor structure of the digital

writing system for position tracking. According to strapdownkinematics theory [2], the body frame accelerations are

transformed to the earth frame by a direct cosine matrix(DCM). After compensating for the gravitational androtational accelerations, the translation accelerations are

integrated into 3D trajectories in space. Thus any 2D humanhandwriting is recorded in real time if the pen touches thewhite board plane.

Ab -AIMU -ARotation (wroll pitchl S yaw L) (1)

Ve =DCM( Ab-G (2)

Se dtdt (3)

where A1Mu is the body frame accelerations: [Ax, Ay, A].

Ab is the translational acceleration in the body frame,

Initialppioestlimate xoand its error covarianice po- Measuereent ilput:- Y

= t] ]2- -, Estim

Fig. 4 The Kalman Filter Algorithm

Fig. 4 demonstrates the real-time recursive process of theKalman filtering algorithm. Pang et al. [4] suggested to use theaccelerometer output and the following process noise

679

3D Digital Pen ( Hardwware)ix - axis x - axis

I

c

covariance, Q, and transition matrix, A, to compensate thenoise in the accelerometer in each axis i, where i= x, y, z.

Xk, i [Sk,i earth Vk, i earth ak,i earth ]

Yk,i =[ak,i earth ]

1 AtI At22

A= 0 1 At0 0 1

W At' WAt4 WAt320 8 6W WAt4w At3 WAt28 3 2WAt3 WAt2 WAt6 2

(4)(5)

(6)

0 0 0

DCM2= 0 0 0

Cos 0sin fCOSbCOS +- sin 0 cos Yf +

sin 0 sin sin f) Ycostl5sin0sinVw)

DCM3= O O O

--sinO sinocosO cosocosO-Asub

0

Afeedback o0

0(7)

B. Feedbackfrom the Pen-tipSince Kalman filtering algorithm is one of the solutions of

the least square problem, the increase of the measurementinput information will improve noise removal in theaccelerometer output and also the position estimation of thepen.

Traditionally, aircraft navigation systems make use ofGlobal Positioning Systems (GPS) to act as the second inputof the measurement input of the Kalman filter as it provides anabsolute position estimation. However, GPS is not suitable forwriting instruments.

Fortunately, most handwriting is done on a twodimensional surface; hence we can make use of the constraintthat the z-axis in the earth frame is always zero when the pen-tip touches the writing surface. This is used as the secondmeasurement input of the Kalman filter to allow an error to bederived and feedback to be used. We also note that the pen isoblique to the earth frame during writing so a 3-axisaccelerometer is not orthogonal to the earth frame z-axis andhence will project the errors onto the earth frame z-axis.

With this idea, we propose to modify the Kalman filteringalgorithm as shown:

Xk = [Xk,X Xk,y Xk,z ak body

Yk =rak -body Sk,z _ earth Vk,z _ earth]

1 AtIAt2

2

Asub= 1 At00 0

(8)

(9)

0 0 0

DCM1= 0 0 0

coscos - cosobsinVur+ sinosin+sin 0 sin cos t Ycos 0 sin cos

0

Asub00

00

Asub0

DCM1

DCM2

DCM3

I3

Q 0 0

Qfeedback Q0

0 0 Q 0

0 0 0 Wdt1-3_

(10)

(1 1)

IV. EXPERIMENTAL RESULTS

In order to test the accuracy of the algorithm, wesimulated stationary accelerometers and rotated the pen tocause the accelerometers to fluctuate. We assume Gaussiandistributed noise in the accelerometers with covariance (u2) of10 cm2s-4 as shown in Fig. 5. As a comparison, the actualnoise level of acceleration sensors is around 7 cm2s-4.

By comparison, the overall noise level using our feedbackerror compensation technique shown in Fig. 6 is much lowerthan that using Pang's algorithm shown in Fig. 7, especially inthe steady state.

Also by comparing Fig. 8 and Fig. 9, which shown ouralgorithm with larger noise covariance and smaller noisecovariance respectively, the measurement noise covariance(R) controls the error compensation performance. If R issmaller, a longer time is required to reach a steady state. Incontrast, ifR is larger, a shorter transient is required.

V. DISCUSSION

Through simulation with realistic sensor noise levels, theproposed Kalman filtering technique can be demonstrated tosignificantly reduce the noise in a pen-based inertialmeasurement unit. Further investigation will be made onfinding the suitable parameter and the acceptable noise levelfor the digital writing system in the near future from buildingan error model of the accelerometer.

VI. CONCLUSIONS

In this paper, we have demonstrated a novel Kalman filterbased algorithm for real time error reduction in a pIMU-baseddigital writing instrument. This approach can be used to moreaccurately determine position in motion sensing based digitalwriting instruments.

680

Noise vs Time ( Larger Variance )

CN

E 1,

Time (sec)

Fig. 5 Random Noise in the three-axis Accelerometers

Noise2 vs Time

E,CN.2_

E

t

E

N

Time (sec)

Fig. 6 Noise level after using our feedback error compensation algorithm

Noise vs Time

E

20 40 60, 80 oo 120 140 160 180 2

Time (sec)

Fig. 8 Noise level after using our feedback error compensation algorithmwith larger measurement noise covariance

Noise2 vs Time ( Smaller Variance)0.01

tE 0008

tE 0 008 -

.0

>o20 40 60 80 100 120 140 160 1802(

x0

N 20 40 6 80 oo 120 10 160 8

Time (sec)Fig. 9 Noise level after using our feedback error compensation algorithm

with smaller measurement noise covariance

ACKNOWLEDGMENT

This project is sponsored by the Hong Kong Innovationand Technology Commission (ITF-UIM-151) and by DAKADevelopment Ltd, Kowloon, Hong Kong.

E

.2

.cnE

N

Time (sec)Fig. 7 Noise level after using Pang's algorithm

REFERENCES

[1] Guanglie Zhang, Guangyi Shi, Yilun Luo, Heidi Wong, Wen J. Li, PhilipH. W. Leong and Ming Yiu Wong, "Towards an Ubiquitous WirelessDigital Writing Instrument Using MEMS Motion Sensing Technology",Proceedings of IEEEIASME International Conference on AdvancedIntelligent Mechatronics, 2005 pp.795-800.

[2] D. H. Titterton, J. L. Weston, Strapdown Inertial Navigation Technology,2nd Edition, AIAA, Reston, USA, 2005.

[3] Yilun Luo, Chi Chiu Tsang, Guanglie Zhang, Zhuxin Dong, Guangyi Shi,Sze Yin Kwok, Wen J. Li, Philip H.W. Leong, Ming Yiu Wong, "AnAttitude Compensation Technique for a MEMS Motion Sensor BasedDigital Writing Instrument", Proceedings of 2006 IEEE InternationalConference on Nano/Micro Engineered and Molecular Systems, 2006.

[4] Grantham Pang, Hugh Liu, "Evaluation of a Low-cost MEMSAccelerometer for Distance Measurement", Journal of Intelligent andRobotic Systems, Volume 30, Issue 3, Mar 2001, pp. 249-265.

681

Noise2 vs Time

E

1(

a)a

E 0

E 0

cs

Et

'O 1(-rm!

N