Inertial Odometry on Handheld Smartphones · Inertial odometry is concerned with estimation of the change of position over time. The extensive survey of Harle [17] covers many approaches

Inertial Odometryon Handheld Smartphones

Arno Solin1 Santiago Cortes1 Esa Rahtu2 Juho Kannala1

1Aalto University 2Tampere University of Technology

21st International Conference on Information Fusion

July 12, 2018

Inertial odometry on handheld smartphonesSolin, Cortes, Rahtu, Kannala

2/20

Introduction

I Phones have accelerometers andgyroscopes

I Should in theory enable inertialnavigation

I Cheap and small sensorsI Low quality data

ú inertial navigation not possible

U We demonstrate that it can be done


3/20

Inertial navigation: How it should work

I Velocity is the integral of acceleration.I Position is the integral of velocity.I We can observe acceleration and

angular velocity in the mobile phone.

Inertial navigation Velocity is the integral of acceleration.

Position is the integral of velocity.

We can observe acceleration and angular velocity in the mobile phone.

Position Velocity Acceleration


4/20

Inertial navigation: Why it does not work

I All inertial navigation systems suffer from integration drift.I Small errors in the measurement of acceleration and

angular velocity...I Progressively larger errors in velocity...I Even greater errors in position.

I The dominating component in acceleration is gravity.I Even slight error in orientation makes the gravity ‘leak’.

I The sequential nature of the problem makes the errorsaccumulate.


5/20

Inertial navigation: How to make it work

I Input: accelerometer data ak and gyroscope data ωk .I Accelerometer and gyroscope biases part of the state:

ak = Tak ak − ba

k ωk = ωk − bωk

I Dynamical model:pkvkqk

=

pk−1 + vk−1∆tkvk−1 + [qk (ak + εa

k )q?k − g]∆tk

Ω[(ωk + εωk )∆tk ]qk−1

for position pk , velocity vk , and orientations qk at time tk .

I Inference by an Extended Kalman filter / smoother


6/20

Inertial navigation: How to make it work

I Additional constraints (observations) are requiredI This framework can use

I Zero-velocity updates (ZUPTs)I Position fixesI Loop-closuresI Barometric air pressure for relative height

I A pseudo-measurement keeping the velocitycomponent from exploding

I Sensor timing infoI A matter of learning the biases


7/20

How does this compare to previous approaches?

On mobile devices PDR typically based on:I Movement detectionI Step and heading systemsI Visual featuresI All of these are limited in some way


8/20

Example studies

I Equipment used:• Off-the-shelf iPhone 6

I Sensors:• Built-in gyroscope and accelerometer• Sampling rate: 100 Hz

I Computations:• Off-line

(runnable on device hardware)


9/20

Example: Conventional PDR


10/20


Position fix

Bag PocketPosition fix0

24

6

z-d

ispla

cem

ent

(m)

2

02

Vel

oci

ty(m

/s)

x y z

0 10 20 30 40 50 60 70 80 90 100 110 120

010

20

Time (seconds)

Ori

enta

tion

(rad)

Figure 2: The altitude (vertical) profile, the velocities, and the orientations of the phone along thepath in Figure 1. The shading shows the stationarity detection outcome, where zero-velocity updateswere triggered. The subtle periodicity in the path is due to walking, and the drop in altitude on thefirst floor is because of the phone being in a bag for parts of the path.

tic filtering theory can be applied to estimation of thedouble-integrated accelerometer readings for inferringpositions. The exact model is presented in detail, andmeasurement updates for fusing measurements with dy-namics are described. Section 4 presents empirical stud-ies where the inertial navigation algorithm is employedin pedestrian dead-reckoning examples, a generalizeddead-reckoning example, and as a measurement tool.Finally, the results are discussed in Section 5.

2. RELATED WORKInertial navigation systems have been studied for

decades. The classical literature cover primarily naviga-tion applications for aircraft and large vehicles [1, 2, 15,16]. The development of handheld consumer-grade de-vices has awakened an interest in pedestrian navigationapplications, where the challenges are slightly di↵erentfrom those in the classical approaches. That is, the lim-ited quality of smartphone MEMS sensors and abruptmotions of hand-held devices pose additional challengeswhich have so far prevented generic inertial navigationsolutions for smartphone applications.

In order to focus on the relevant previous literature,we restrict our scope to tracking algorithms that usethe sensors available in a smartphone. Standard sensorsin navigation applications are accelerometers (used formovement estimation), gyroscopes (turn-rate), magne-tometers (compass heading), and barometers (altitude).

Inertial odometry is concerned with estimation of the

change of position over time. The extensive survey ofHarle [17] covers many approaches with di↵erent con-straints for the use of inertial sensors for pedestriandead-reckoning (PDR). Typically INS systems eitherconstrain the motion model or rely on external sensors.In fact, we are not aware of any previous system whichwould have all the capabilities that we demonstrate inthis paper.

One prominent INS solution relying on external hard-ware is the OpenShoe project [8, 18]. It uses foot-mounted inertial sensors with several pairs of accelerom-eters and gyroscopes to estimate the step-by-step PDR(in an INS-SHS framework, see below). The model isconstrained by zero-velocity updates (ZUPTs) on eachstep once the foot touches the ground. Co-operativevariants of the system use inter-agent ranging to main-tain a global reference frame.

Step and heading systems (SHS, see e.g. [5, 19–21])use the inertial sensor to estimate the heading and thestep length of the user. These are introduced intoa constrained model that estimates the walking pathby accumulating the step vectors in order to do PDR.These systems have been proven to work well for PDRin short and medium range but they typically imposeconstraints for the device orientation. For example, thedevice orientation is often known or fixed with respectto the walking direction. Further, they are very sensi-tive to changing gaits and are prone to false positives(see discussion in [17]). A recent approach [22] usesbipedal locomotion models to model the periodical be-


11/20


8 4 0 48

6

4

2

02

Floor level 3

Floor level 2

Floor level 1

Ele

vato

rri

de

Stairs

Horizontal (y) displacement (meters)

Ver

tica

l(z

)dis

pla

cem

ent

(met

ers)

Figure 3: A PDR example with first descend-ing two levels down and then taking the eleva-tor back up. The path was started at origin andthe path ends with a loop-closure in the sameplace. The points where the phone touches thefloor level (the sharp drops in vertical position)are zero-velocity updates. No absolute positioninfo was given to the model.

3.6 Pseudo-Measurement UpdatesWithout position fixes, loop-closures, or ZUPTs the

inertial navigation system quickly becomes unstable.Once the estimates start diverging, they easily loosetheir numerical precision. The main source of theseproblems is gravity ‘leaking’ into the acceleration inputand corrupting the velocity vector. Once the velocitystarts to drift, the position diverges almost instantly.

However, even without other auxiliary information, itis possible to keep the system informed about a reason-able scale of velocity. In our model, we present a simpleyet powerful pseudo-update formulation that keeps thespeed in the range of some meters per second and dis-courages the system from accelerating into higher ve-locities.

The pseudo-update model is defined in terms of thespeed of the object, when it is not stationary. The speed(the Euclidean norm of the velocity) is

h(x) = kvk. (11)

In our experiments the pseudo-updates areparametrized as follows. The speed observationy = 0.75 m/s with a measurement noise = N(0, 22).The large measurement noise variance keeps the updatenon-informative in comparison to other informationsources.

4. EXPERIMENTSThis section is dedicated to showing the capability,

versatility, and performance of the inertial navigationsystem presented in the previous sections.

In the examples, the interest was put on Apple phonesand tablets—mostly because of their uniform hardwareand good software compatibility between devices. Thedevice models used in the examples are the iPhone 6and the iPad Pro (12.9-inch model). Both these modelsare equipped with built-in IMUs (InvenSense MP67B)and a barometric sensor (Bosch Sensortec BMP280). Inall experiments, the IMU sensor data and the associatedtimestamps were collected at 100 Hz, and the barom-eter data (when used) at approximately 0.75 Hz. Thedata was collected using an in-house developed data col-lection application, and the paths were reconstructedo↵-line in Mathworks Matlab.

4.1 Pedestrian Dead-ReckoningThe most apparent use case for the presented model

is to apply it to pedestrian dead-reckoning, where themobile phone (iPhone 6) is carried by the user indoors.There exist a multitude of methods for dead-reckoningusing data provided by mobile phones. Therefore theaim of this experiment is to show how this method dif-fers from others by its generality.

Figures 1 and 3 summarize features of the proposedINS system; the example includes the use of zero-velocity updates, position fixes, pseudo-measurementsconstraining the speed, and barometer observations.This experiment covers traditional navigation-like PDRuse cases (walking with the phone in a fixed orienta-tion), where SHS systems are often used, cross-floortracking, where visual tracking methods are usually themethod of choice, and bag/pocket use cases, which cur-rently often require resorting to radio based position-ing. The generality of our INS system can cover themall with only one method and no external hardware.

In the first example, the path was started onthe ground floor with zero-velocity updates (no pre-calibrations done). First the user walked up a flight ofstairs to the first floor holding the phone in the hand.On the first floor a position fix was given, after whichthe phone was put in a bag. Next, the phone was takenout of the bag and put in the trouser pocket. Beforedescending to the ground floor, the phone was takenout of the pocket and a second position fix was given,which aligned the path to the map and was able to pro-vide absolute information of the scale. On the groundfloor the a manual loop-closure was given to indicatethat the phone had returned to the starting point, andthe phone was placed on the floor for some final zero-velocity updates. The tracking path is accurate andfollows the true path up to decimetres.

Figure 2 shows the altitude profile of the path. TheZUPTs where the phone is placed on the floor are clearlyshowing, as well as the stair climbing. The drop in


12/20

Example: General dead-reckoning

iPhone 6

Baby

Freely turningfront wheels

Figure 4: The experiment setup with a wheeledbaby stroller and an iPhone placed on thestroller for tracking.

altitude on the first floor is due to the phone being inthe bag for a part of the path. The figure also shows theestimated velocity. The periodicity is due to walking.This e↵ect is less evident when the phone is in the bag,and at clearest when the phone is in the trouser pocketfirmly attached to the body. The third subfigure inFigure 2 shows the orientation of the phone over theentire path.

We briefly present a second PDR example which isshown in Figure 3. In this example the path was startedat origin with some zero-velocity updates in di↵erentphone orientations. After this the user walked two floorsdown. When waiting for the elevator on floor level 1,further zero-velocity updates were done. The path iscompleted with taking the elevator back to floor level 3and closing the loop at the starting point. In this ex-ample no absolute position information was given. Thescale comes entirely from the accelerometer data. Inboth examples, a backward smoother pass is run afterevery update, thus also correcting the past estimates.

4.2 Generalized Dead-ReckoningWheel based motion and general non-legged motion

are use cases which are not covered by conventionalstep counting PDR methods. This demonstration setsthe method in a more general scope for general dead-reckoning that can be applied to any wheeled, sliding, orflying objects indoors or outdoors. Applications includepush-carts, trolleys, robots, hover boards, quadcopters,etc.

We chose to include an example with a human

05

10

15

t = 18.7 s

05

10

15

t = 37.0 s

05

10

15

t = 55.8 s

05

10

15

t = 74.2 s0

510

15

t = 91.5 s

05

10

15

t = 108.3 s

Figure 5: The evolution of the position esti-mate and path are visualized along the way atthose points where the stroller was momentarilystopped. The dashed line shows the final pathfor reference. The phone remained leaning onthe stroller for the entire experiment. No posi-tion fixes nor loop-closures were used.


13/20

Example: General dead-reckoning

The video is available on YouTube:https://youtu.be/L-E9fNsrvII

https://youtu.be/L-E9fNsrvII


14/20

Example: Inertial measurements

The video is available on YouTube:https://youtu.be/L-E9fNsrvII

https://youtu.be/L-E9fNsrvII


15/20

Example: Inertial measurements

7.36m

8.35

m

7.42m

8.33m

(a) Measurement #1

7.36m

8.41

m

7.39m

8.47m

(b) Measurement #2

Figure 6: Two examples of measuring the wall placements of an indoor space with an iPad Pro.The walls (the equations of the planes) are a part of the state variable. Points where the phone isstationary (against the wall) are visualized in red. The true size of the room is 7.30 m 8.45 m.

manoeuvred wheeled object with an intrinsic noisesource—that is a baby stroller with a baby on board.Figure 4 shows the test setup, where the phone is placedon the stroller leaning on the top. The phone (iPhone 6)remains fixed to the stroller body for the entire experi-ment.

The experiment setup is as follows. The walking wasstarted from a stationary state, where the phone auto-matically performed a number of zero-velocity updates.Along the route the stroller was stopped irregularly andzero-velocity updates triggered if the stroller becamestationary enough. In this experiment no position fixesnor loop-closures were used. The path only measure-ment data is the automatic loop-closures, the barome-ter observations, and pseudo-updates constraining themomentary speed. The total path length was approxi-mately 93 m.

Figure 5 shows the path estimate at the times whenthe stroller was stopped. The dashed line in the fig-ures is the path estimate at t = 108.3 s which is shownfor reference. The zero-velocity updates in the variousheading angles of the stroller are clearly enough to cap-ture the bias estimates and stabilize the system. Thefinal estimate is o↵ from the starting point by approxi-mately 0.73 m.

4.3 Sensing of SurroundingsThe model proposed in this paper has many potential

applications beyond simple odometry and tracking. Forexample, the orientation and ZUPT information can beused as a measurement tool per se. By consecutivelyplacing the phone flat on the walls of a room, a modelof the geometry of the room can be built. From eachZUPT on the trajectory, a wall can be projected parallelto the phone screen, thus capturing the geometry of the

room. Associating several ZUPTs to the same wall withthe additional knowledge that the points span a planethrough the space, can also make it possible to betterestimate the wall placement and orientation.

The model in this paper is flexible enough thatnew constraints—in form of estimated quantities andprior information about them—can be introduced. Forthis particular application there are several useful con-straints, such as coplanarity between some ZUPT po-sitions. For this purpose, a smartphone applicationthat delivers these functionalities is publicly available.Therefore we seek to deliver comparable results to theRoomScan (Locometric Ltd, http://locometric.com)application.

In this case the conventional loop-closures are notsuited for this particular purpose. The loop-closurepoints are not indicating that the phone is in the sameposition, but touching the same plane. This plane is aline in the xy-plane, and the coecients of the equationof the line for each wall can be augmented in the statevector. Each ZUPT is thus an observation of a pointand orientation on a line representing a wall. In oursetup, we do not enforce any prior information aboutwalls being orthogonal to each other, whereas we specu-late that the RoomScan application enforces some shapeconstraint for the room.

The experiment is as follows. Markers were placedupon the walls of a room of known geometry (rectangu-lar measuring 7.30 m 8.45 m) the distance betweeneach marker is kept as close to 1 m as possible. Due toobstacles, some markers had to be skipped. The deviceis then pressed flat against the wall at one of the cor-ner markers, after which the user waits three secondsand places the device flat against the wall at the nextmarker. This is repeated until the user arrives at the


16/20

Recap

I Inertial navigation on asmartphone is possible

I The key is learning the sensortransformations (biases)

I Not possible withoutmeasurements (consraints)• ZUPTs• pseudo-speed

I Future work towards relaxingthese further


17/20

I Homepage:http://arno.solin.fi

I Twitter:@arnosolin

http://arno.solin.fi

Documents

Inertial Odometry on Handheld Smartphones · Inertial odometry is concerned with estimation of the change of position over time. The extensive survey of Harle [17] covers many approaches