14
I am a Smartphone and I can Tell my User’s Walking Direction Nirupam Roy University of Illinois (UIUC) [email protected] He Wang University of Illinois (UIUC) [email protected] Romit Roy Choudhury University of Illinois (UIUC) [email protected] ABSTRACT This paper describes WalkCompass, a system that exploits smartphone sensors to estimate the direction in which a user is walking. We find that several smartphone localization sys- tems in the recent past, including our own, make a simplify- ing assumption that the user’s walking direction is known. In trying to relax this assumption, we were not able to find a generic solution from past work. While intuition suggests that the walking direction should be detectable through the accelerometer, in reality this direction gets blended into var- ious other motion patterns during the act of walking, includ- ing up and down bounce, side-to-side sway, swing of arms or legs, etc. Moreover, the walking direction is in the phone’s local coordinate system (e.g., along Y axis), and transla- tion to global directions, such as 45 North, can be chal- lenging when the compass is itself erroneous. WalkCompass copes with these challenges and develops a stable technique to estimate the user’s walking direction within a few steps. Results drawn from 15 dierent environments demonstrate median error of less than 8 degrees, across 6 dierent users, 3 surfaces, and 3 holding positions. While there is room for improvement, we believe our current system can be imme- diately useful to various applications centered around local- ization and human activity recognition. Categories and Subject Descriptors C.4 [Performance of Systems]: Measurement techniques; C.5.3 [Computer System Implementation]: Portable devices General Terms Design, Experimentation, Performance Keywords Heading direction; Force analysis; Mobile phones; Sensing; Compass correction; Localization; Activity recognition; Ori- entation; Magnetic field Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. MobiSys’14, June 16–19, 2014, Bretton Woods, New Hampshire, USA. Copyright 2014 ACM 978-1-4503-2793-0/14/06 ...$15.00. http://dx.doi.org/10.1145/2594368.2594392 . 1. INTRODUCTION Despite tremendous amount of academic research, indoor localization is still struggling to come to mainstream. Based on numerous discussions with industries and start-ups, we are beginning to realize that much of the academic research has optimized for “location accuracy”, while making sim- plifying assumptions about other aspects of the end-to-end system. Example assumptions include the availability of indoor maps, availability of dense WiFi infrastructure, ori- entation of the phone, known walking directions, etc. The consistent feedback we have received is that relaxing these assumptions is crucial at this point, perhaps even more than pushing the boundaries of location accuracy. Driven by this feedback, this paper concentrates on automatically sensing a user’s global walking direction regardless of the orientation of the phone. If solved well, a user’s walking direction can oer benefits beyond inertial localization. Stitching crowd-sourced sensor data to infer detailed indoor maps is of increasing interest – the user’s walking direction is a critical component in this. Elevator companies are envisioning that future elevators will be proactively fetched when the resident of a building begins to walk towards the elevator [2] – the walking direction is valuable here. Walking directions may also translate to the user’s facing direction, enabling additional applications in augmented reality, social activities, and context-awareness. Of course, these benefits would be available only if walking direction is estimated through a generic stand-alone mod- ule, without relying on external information (such as maps, locations, WiFi access points). Moreover, to be able to tell the global walking direction (e.g., 45 North), the compass errors need to be mitigated. WalkCompass is focused on addressing these challenges. It is natural to question the diculty of solving this prob- lem – one may ask why not estimate the walking direction by projecting accelerometer signals to the horizontal plane? While this is true in abstraction, the real situation is far more complicated when the phone is held in dierent ori- entations at various body positions. In pant pockets, for example, the phone swings along with the leg, continuously altering the phone’s local coordinate system; free-flowing hands are similar. Users sway sideways in varying degrees while taking each step, which becomes pronounced when the phone is in the palm (e.g., when the user is checking emails). Inherent sensor noise further pollutes the signal. The precise walking direction is actually a micro-motion

Walk Compass

Embed Size (px)

DESCRIPTION

walk compass

Citation preview

  • I am a Smartphone and I can Tell my UsersWalking Direction

    Nirupam RoyUniversity of Illinois (UIUC)[email protected]

    He WangUniversity of Illinois (UIUC)[email protected]

    Romit Roy ChoudhuryUniversity of Illinois (UIUC)

    [email protected]

    ABSTRACTThis paper describes WalkCompass, a system that exploitssmartphone sensors to estimate the direction in which a useris walking. We find that several smartphone localization sys-tems in the recent past, including our own, make a simplify-ing assumption that the users walking direction is known.In trying to relax this assumption, we were not able to finda generic solution from past work. While intuition suggeststhat the walking direction should be detectable through theaccelerometer, in reality this direction gets blended into var-ious other motion patterns during the act of walking, includ-ing up and down bounce, side-to-side sway, swing of arms orlegs, etc. Moreover, the walking direction is in the phoneslocal coordinate system (e.g., along Y axis), and transla-tion to global directions, such as 45 North, can be chal-lenging when the compass is itself erroneous. WalkCompasscopes with these challenges and develops a stable techniqueto estimate the users walking direction within a few steps.Results drawn from 15 dierent environments demonstratemedian error of less than 8 degrees, across 6 dierent users,3 surfaces, and 3 holding positions. While there is room forimprovement, we believe our current system can be imme-diately useful to various applications centered around local-ization and human activity recognition.

    Categories and Subject DescriptorsC.4 [Performance of Systems]: Measurement techniques;C.5.3 [Computer System Implementation]: Portabledevices

    General TermsDesign, Experimentation, Performance

    KeywordsHeading direction; Force analysis; Mobile phones; Sensing;Compass correction; Localization; Activity recognition; Ori-entation; Magnetic field

    Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected], June 1619, 2014, Bretton Woods, New Hampshire, USA.Copyright 2014 ACM 978-1-4503-2793-0/14/06 ...$15.00.http://dx.doi.org/10.1145/2594368.2594392 .

    1. INTRODUCTIONDespite tremendous amount of academic research, indoorlocalization is still struggling to come to mainstream. Basedon numerous discussions with industries and start-ups, weare beginning to realize that much of the academic researchhas optimized for location accuracy, while making sim-plifying assumptions about other aspects of the end-to-endsystem. Example assumptions include the availability ofindoor maps, availability of dense WiFi infrastructure, ori-entation of the phone, known walking directions, etc. Theconsistent feedback we have received is that relaxing theseassumptions is crucial at this point, perhaps even more thanpushing the boundaries of location accuracy. Driven by thisfeedback, this paper concentrates on automatically sensing ausers global walking direction regardless of the orientationof the phone.

    If solved well, a users walking direction can oer benefitsbeyond inertial localization. Stitching crowd-sourced sensordata to infer detailed indoor maps is of increasing interest the users walking direction is a critical component in this.Elevator companies are envisioning that future elevators willbe proactively fetched when the resident of a building beginsto walk towards the elevator [2] the walking direction isvaluable here. Walking directions may also translate to theusers facing direction, enabling additional applications inaugmented reality, social activities, and context-awareness.Of course, these benefits would be available only if walkingdirection is estimated through a generic stand-alone mod-ule, without relying on external information (such as maps,locations, WiFi access points). Moreover, to be able to tellthe global walking direction (e.g., 45 North), the compasserrors need to be mitigated. WalkCompass is focused onaddressing these challenges.

    It is natural to question the diculty of solving this prob-lem one may ask why not estimate the walking directionby projecting accelerometer signals to the horizontal plane?While this is true in abstraction, the real situation is farmore complicated when the phone is held in dierent ori-entations at various body positions. In pant pockets, forexample, the phone swings along with the leg, continuouslyaltering the phones local coordinate system; free-flowinghands are similar. Users sway sideways in varying degreeswhile taking each step, which becomes pronounced whenthe phone is in the palm (e.g., when the user is checkingemails). Inherent sensor noise further pollutes the signal.The precise walking direction is actually a micro-motion

  • that needs to be carefully identified, weighted, and averagedover at least two steps.

    Even if the users walking direction is successfully extractedin the phones local coordinate system, translating it toglobal directions, say 45 North East, is an additional chal-lenge. This is because the global (magnetic) coordinatesystem relies on the compass, which itself is susceptible toferromagnetic interference in many indoor environments.All in all, the accelerometer signal that corresponds to thedirection of walking gets blended into various other motionpatterns that vary across users, environments, and phonemodels. Solving it generically, we believe, is non-trivial.

    The core intuitions in WalkCompass are simple and canbe explained in two parts. First, when a user walks, herheel strikes the ground creating a distinct vibration thatresonates through the entire body. This vibration reflectson the accelerometer data across all holding positions, evenwhen the user is holding the phone against her ears. Walk-Compass uses this vibration as a reference, scans the signalbackwards, and extract specific samples from a time windowwhen the bodys movement is dominantly in the headingdirection. The signal is then processed with the gyroscopedata to compensate for instability of the phones coordinatesystem, such as when the entire phone is swinging in thepant pocket. This motion vector is then projected to theplane orthogonal to gravity, and averaged over few steps toconverge upon the local walking direction.

    The second problem pertains to translating the walkingdirection to the global magnetic coordinate system, but forthis, the compass needs to be improved. WalkCompasssintuition is to treat the compass measurements as a signal,shaped by the earths magnetic field, and by (the resultantof) other interferers in the ambiance. While separating thisresultant interference is a dicult problem, the opportunityarises from walking. Since the phone moves in small steps,it observes staggered snapshots of the same interference as if watching an object from dierent viewing angles thereby enabling the possibility of triangulation. Of course,triangulation is possible only when the signal exhibits cer-tain properties. WalkCompass exploits these properties tocorrect the compass direction in certain locations, and usesgyroscope based dead reckoning to track the walking direc-tion between these locations.

    Of course, the above is an over-simplification of our de-signed algorithm several constraints need to be accountedfor that are detailed in the paper. WalkCompass incorpo-rates these algorithms into a functional system, with certainoptimization for human variations and holding positions.Experiment results from 15 dierent settings show that themain limitation of WalkCompass is in its time to conver-gence. While an ideal system should be able to oer thedirection in 2 walking steps, WalkCompass sometimes re-quires up to 5 steps. However, the direction accuracy ispromising, with the 75th percentile error being less than12 degrees, in comparison to the compass which can be26 even when held in the direction of walking. The accu-racies scale across users, varying placements on the body,and while walking over dierent surfaces such as concrete,carpets, etc. Micro-benchmarks on magnetic interference

    calculation also exhibit promising results, demonstratingthat the native compass errors can be appreciably reduced(when the user is walking). While there is still room for im-provement, we believe WalkCompass can already be usefulto other applications. We have demonstrated the system toa few companies an YouTube video of the demonstrationis posted here [3].

    The main contributions of WalkCompass may be brieflysummarized as follows:

    1. Analyzing the anatomy of walking patterns from theperspective of smartphone sensors. This is performedthrough a synchronized analysis of a walking video andsensor readings, revealing meanings of each short seg-ment of accelerometer readings. We believe our analy-sis tool could benefit other smartphone-based activityrecognition systems.

    2. Coping with magnetic interference in indoor environ-ments. Designing various techniques to localize, quan-tify, and isolate the interference, with certain inspira-tions borrowed from noise cancellation techniques inwireless communication.

    3. Implementation of a functional prototype on multiplemodels of Android phones. Experiment results from 15dierent buildings indicating median error of 8 witharound 5 steps to achieve convergence.

    2. FEW NATURAL QUESTIONS(1) Despite substantial research on activity recogni-tion and phone orientations, why is walking direc-tion still an unsolved problem?While we have been somewhat surprised as well, our litera-ture survey revealed some insights into this question. Mostpapers that have attempted this problem have done so inthe context of a broader application, and have leveragedapplication-specific opportunities to resolve the challenges.For instance, [9] analyzes human motion in indoor settingsand inherently requires a map of the place; authors leveragethe same map to estimate the users walking direction. Au-thors in [30] perform localization using information aboutthe WiFi APs location, and use the same AP locationsto infer crude walking direction. [22, 19] assumes that thephones initial orientation is known, and tracks the usersheading direction for a short time window after that. Zee[28] is one of the localization papers that addresses the prob-lem without other assumptions, but admit that the solutionis not stable and oers ambiguity they mention that usingmaps can solve the problem at the expense of some latency.

    A recent paper [5] attempts to solve the problem as astand-alone module but uses highly controlled conditionsfor testing, namely, phone held in hand. MATLAB resultsare presented for two path traces from outdoor environ-ments. Moreover, the solution is only in the local coordinateframe. Finally, several papers estimate the orientation of asmartphone on the human body. Phone orientation, whilenecessary, is insucient for translating a users walking di-rection to a global coordinate system; moreover, detectingthe (local) walking direction is a separate problem. Know-ing the orientation does not make this problem any easier.

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

  • (2) Why not use the sequence of users location tocompute walking direction?In outdoor environments, this is true and trivial (assum-ing GPS energy consumption is not an issue). For indoors,however, walking direction is needed to compute locationin the first place. This reliance is growing stronger due totwo reasons. First, with more systems moving away frommanual war-driving to crowd-sourced approaches, it is im-portant to understand the motion patterns of the crowd,so that sensed data can be appropriately oriented. Second,pedestrian dead reckoning is becoming increasingly popularfor localization due to its ability to track fine scale humanmovements (otherwise dicult with say WiFi or cell towersignals). As a result, WalkCompass cannot assume the avail-ability of the users location the estimated walking direc-tion will be needed to facilitate localization.

    3. SYSTEM OVERVIEWFigure 1 shows the key building blocks underlying Walk-Compass, and their connections. We describe the flow ofoperation here, and expand on the technical details in thenext sections.

    Human Walk Analysis (one time)

    Video-sensor co-analysis

    Semantically labeled data

    Time Domain Filtering

    Gyro. based Dead Reckoning

    Projection to Horizontal Plane

    Sway Detection & Averaging

    Freq. Domain Low Pass Filter

    Iterative Triangulation On Low Freq.

    Magnetic Interference Cancellation

    Global Walk

    Direction

    Conversion to Global Direction (GDE)

    Estimating Walking Direction (LDE)

    Local Walk

    Direction

    Phone Position Classifier

    Wal

    kC

    om

    pas

    s

    Figure 1: Flow of operations in WalkCompass. Theinput is the phones sensor signals and the output isthe local and global walking directions.

    The overall WalkCompass system has 3 main modules,namely: (1) Human Walk Analysis (HWA), (2) Local WalkDirection Estimator (LDE), and (3) Global Walk DirectionEstimator (GDE). HWA is a one-time analysis of humanwalking patterns that helps us understand how dierentforces act in dierent directions to keep the human bodystable during a walk. We observe how these forces aremanifested on the smartphones accelerometer in dierentpositions on the body, and use the insights to make keydesign decisions in LDE. Specifically, we identify the exacttime-segment of the accelerometer signal that contains theforward motion of the body-part (e.g., leg, hand, chest) nearwhich the phone is located.

    The Local Walk Direction Estimator (LDE) receives theraw accelerometer reading from the smartphone, and first

    classifies the data to infer the phones location on the body.The (time domain) accelerometer signal is then analyzedand the appropriate segment extracted based on HWAsrecommendations. This segment contains the forces in thedirection of the walk, but they are in 3D; moreover it ispolluted by a constantly-rotating coordinate system (saywhen the upper leg swings while pivoted to the hips). Thegyroscope is engaged to compensate for this rotation, andafter some processing, the motion vector is projected to thehorizontal plane, orthogonal to gravity.

    Now, this vector is only for one step of one leg, and notnecessarily the direction of the walk humans extend theirfeet in side-ward directions, called sway, and the walkingdirection is an average of sway. The output from this aver-aging operation yields the walking direction in the phoneslocal coordinate system. Observe that this implies thatwith respect to the humans walking direction, the phonesorientation is now known.

    The local walking direction may suce for some appli-cations; others need the global direction. In a perfectworld, the global walking direction would be the angle sub-tended by the walking vector and the compass direction(i.e., North). For instance, if the walking vector points inthe opposite direction of the phones compass, then theglobal walking direction would be South. However, thecompass can be erroneous indoors due to ambient ferromag-netic interferences the GDE module is tasked to computethe global walking direction despite these errors. To thisend, the GDE module operates on the frequency domain,treating the sequence of compass readings as a digital signal.

    As a person walks through indoor spaces, WalkCompass hy-pothesizes that the eect of nearby interferers would changequicker over time, and would be captured by the high fre-quency components in the compass signal. Filtering out thehigh frequency components, the low frequency componentsare expected to be from further away sources of interference,adding an oset to the compass reading. GDE employs aniterative algorithm that essentially attempts to localize thisfar-away interferer. The key intuition is to decompose themeasured magnetic vector into the earths magnetic vector(G) and the interference vector (I), and adjust the direc-tion of G until all the I vectors intersect at the same point1.This value of G is inferred to be the actual earths mag-netic field. Of course, this triangulation may not be feasiblein all locations (e.g., where many strong and opposing in-terfering sources are located far away). However, if somelocations oer feasible results, the global walking directioncan be computed there, and gyroscope-based tracking canbe used at other (in-between) locations. The subsequent sec-tions expand on each of the techniques, followed by micro-benchmark and full-scale system evaluation.

    4. SYSTEM DESIGNWe describe the three modules, HWA, LDE, and GDE, forthe case where the phone is carried in the pant pocket. Thisis actually the harder case, compared to the palm or in the

    1Note that the magnitude of the earths magnetic vector ata given location can be looked up in a database, so knowingthe users crude location, say the zip code, is adequate.

  • Primary'Step'Primary'Step'

    Ampl.&

    stance&phase& swing&phase&

    20%& 40%& 60%& 80%& 100%&&0%&

    Leg&pivoted&Body&advances&

    heel>strike& heel>strike&Leg&swings&toe>o&

    Figure 3: Accelerometer data from human walking patterns with the smartphone placed in the pocket: (a)Correlated jerks on all 3 axes on the accelerometer indicating a heel-strike on the ground. (b) The peakdetection algorithm detects the timing of the heel-strikes. (c) The gait cycle of human walk composed of 2phases: the first 60% between two heel strikes is called the stance, and the next 40% is called the swing.

    Figure 2: Matching video and accelerometer for la-beling measured readings the green circles aretracked across video frames to compute their respec-tive motion vectors, and later matched against thevectors derived from the accelerometer.

    swinging hand. At the end of this section, we will discusshow the techniques can be generalized to other placementswith small modifications.

    4.1 Human Walk Analysis (HWA)Our goal in HWA is to be able to label the dierent partsof an accelerometer reading with the corresponding micro-actions that the user is performing at each time point.The hope is that this analysis would reveal at least onesegment of the signal that is precisely along the walkingdirection, and more importantly, is least polluted by othermicro-actions of the limbs. Of course, numerous studies,especially in the domains of physical therapy and computeranimation, have analyzed the acceleration of various bodyparts during the walk cycle [13, 14, 12]. Our study, how-ever, is from the view of the smartphone sensor, placed atdierent parts of the body, inside dierent outfits.

    We set up a system in which 2 video cameras were placedfocusing various body parts of a walking person, say John.John carried phones at multiple positions in the body ateach of these locations, we attached a green marker to helptrack the motion of the phone through the video frames. All

    phones and the video cameras were time synchronized (tosub-milliseconds) by shining a bright light on all their cam-eras simultaneously. From the gathered videos, we trackedeach green marker over time and translated them into mo-tion in the X, Y, and Z axes of the phone. We matched thevisually-computed motion with the accelerometer signal tosynchronize them as well as possible. This process helps inmatching the dierent ranges of motion (one in pixel spaceand the other in sensor signal space), as well as for syn-chronizing timing shifts due to fluctuations in accelerometersampling (details omitted in the interest of space). Figure2 shows the outcome we are now able to click on anytime window on the accelerometer signal and observe thecorresponding video clip.

    We distill 2 useful findings from this experiment that we uselater to design the system. Here the limb that carries thephone is referred to as the primary limb, and the other iscalled secondary.

    (1) Heel Strike is an Anchor PointThe maximum amplitude and fluctuation in the accelera-tion, along all the 3 axes, occurs when the heel of the hu-man strikes the ground. This holds true across all positionson the body, including hand, backpack, pant pocket, shirtpocket, etc. only the magnitude of the jerk is dierentin each of them. This general property serves as a valuablesignal marker an anchor point from which other stagesof the walk can be analyzed. To isolate this heel-strike withhigh reliability, we add all the 3 dimensions of the accelerom-eter signal, and take the signal envelope of this summation,and pass through a simple peak detection algorithm (Figure3a and 3b). The outputs of the algorithm are the precisetimings of the peaks.

    (2) Swing Before Heel StrikeBetween consecutive heel strikes, the legs go through twomain phases a stance and a swing as shown in Figure3(c). Stance is the phase when the upper body and leg moveforward while pivoted to the feet on the ground; swing is thephase when the leg swings forward, pivoted to the hip, toovertake the upper body. During the swing, the accelerationis positive in the first half of the swing, and negative in thesecond half (to bring the leg to a stop for a heel-strike).

  • Further, say the swing lasts from time t1 to t9 (Figure 4) the acceleration is maximum at time t3, then its zero attime t5, and then the deceleration is maximum at time t7.

    Stance&(60%)&

    t1&&&&&t2&&&&&t3&&&&&t4&&&&&t5&&&&&t6&&&&&t7&&&&&t8&&&&&t9&

    Accelera;on&

    heel>strike&toe>o&

    Swing&(40%)&

    Figure 4: The acceleration at various points of theswing phase during the human walk cycle.

    Our first approach was to extract the accelerometer dataaround t3 and t7, reverse the sign for the data at t7, andaverage them to derive the users walking direction. Thisshould cope well with accelerometer noise since hardwarenoise is Gaussian with zero mean, hence should cancel wellupon averaging. However, we observed that the data aroundt3 is polluted by the motions of the secondary leg and theupper body. On the other hand, these secondary and upperbody motions are much less prevalent around time t7 per-haps because the body is trying to stabilize making thesignal pristine. Hence, the Human Walk Analysis (HWA)module prescribes the region around this t7 time point asthe signal containing the walking direction. Of course, thisis an approximation, since the above analysis assumes thatthe swing is occurring uniformly in time. To be accurate,we cannot assume that the mid point of the swing is at t5,and the deceleration is maximum mid-way between t5 andt9. The subsequent discussions in LDE will account for thesefactors.

    4.2 LocalWalking Direction Estimator (LDE)The LDE module receives a users accelerometer readings asan input and runs the raw data through a position classifierto detect where the phone is located on the body. Past workhas reliably solved this problem [29, 27] we have borrowedthese solutions, and depending on the placement, applyminor variants of LDE. For ease of explanation, we describeLDE entirely for the case of pant pockets, and discuss thevariants at the end of the section.

    Filtering: Assuming that the phone is in the pocket, thedata is then received by the Time Domain Filtering mod-ule. This module computes the heal-strike peaks in thesignal, looks back from the peak to extract the segmentwith maximum deceleration in the second half of the swing(as described earlier in HWA). More precisely, the start ofthe second half needs to be estimated first, for which Walk-Compass looks for the change in the sign of accelerationfrom positive to negative. Denote this time ti and the timeof next peak, tj . WalkCompass extracts the time point ofmaximum deceleration within [ti, tj ] denote this as time

    tx. To cope with accelerometer noise, typically Gaussianwith zero mean, WalkCompass extracts a short segmentaround tx

    2. WalkCompass passes this segment through alow pass filter to remove high frequency noise and poten-tially other signal pollutants, and forwards the samples tothe gyroscope based dead reckoning module.

    Dead Reckoning: The dead reckoning module fetchesthe gyroscope readings at these exact time points (corre-sponding to these signal samples) and observes the angularrotation of the phone. By rotating back the phones coor-dinate system to a reference time point (i.e., rotating thephone in the reverse direction of the gyroscope readings),the samples are now brought to a stable coordinate system.The samples are now averaged, and the sign reversed. Thisis the walking vector in 3D space.

    Projection to Horizontal: The walking direction in 3Dspace is then projected to the plane orthogonal to gravity,also called the walk plane. The gravity vector is of coursenot obvious from the raw accelerometer reading we es-timate the gravity vector by fusing the accelerometer andgyroscope data, and applying the complementary filter [1].WalkCompass projects the 3D walking direction into theplane perpendicular to this computed gravity vector.

    Sway Detection: In bipedal movement, each leg moves thebody in a slightly lateral direction the walking direction isactually a resultant of these two lateral motions. Thus, thewalking direction computed so far is slightly oset from theactual walking direction. Further, the body also sways dur-ing the walk, introducing additional forces unaligned to thewalking direction. Fortunately, all these movements are rea-sonably symmetric with respect to the heading direction inthe horizontal plane. Therefore, the resultant of the primaryand secondary movements mitigates these lateral movementsand sway-induced errors. WalkCompass estimates the walk-ing direction on the secondary leg using the same techniqueas the primary, except that it uses a second smaller peak inthe accelerometer signal. These smaller peaks are indicativeof when the secondary heel strikes the ground. The finalresultant vector is passed through a median filter to removerandom jerks on the phone during the walk. The output ofthe median filter is forwarded as the final output of LDE this is WalkCompasss estimate of the walking direction inthe phones local coordinate system.

    4.2.1 Coping with Phone PlacementPast work has developed classifiers that reliably discrimi-nate the phones location on the body (palm, pant pocket,swinging hands, etc.). The key technique is to observe theenergy density from acceleration, as well as the magnitude ofpeaks during the walk. WalkCompass borrows this classifierwith a few minor adjustments this is not our contribution.

    Once the phones location is known, we apply a few mi-nor modifications for the case when the phone is held inthe hand (both palm and swing). Specifically, instead ofsumming the (x, y, z) accelerometer signals, we first com-pute the variation along the gravity dimension (which is

    2Taking a sample only at tx is susceptible to noise, insteadaveraging over a few samples is expected to dampen thenoise.

    ahrarnaqviHighlight

  • Time (sec.)

    Comp

    ass h

    eadin

    g (de

    g.)

    N"

    Figure 5: (a) Magnetic fluctuations in 3 indoor environments far higher than outdoors. (b) Heat map ofmagnetic intensity across the entire corridor of a science laboratory. WalkCompass performs evaluation inthese settings.

    not necessarily the z axis). The intuition is that the phonebounces maximally along the gravity, and the HWA mod-ule shows that the bounce-peaks are well correlated to theheel strikes. Thus, once we anchor the accelerometer sig-nal to these bounces (which are symmetric for both legs),we apply the same segment extraction scheme. Fortunately,the time point when the legs deceleration is maximum, isalso the time when the upper body is getting stabilized fromfull-fledged forward motion. Selecting this time window iseective for hand-held phones. The case for shirt pockets isslightly more dicult, since the bounce is softer. Nonethe-less, the bounce is still visible among the experiments weperformed (but may be less reliable for a soft walker, or achild). We leave jacket pockets and bags to future work.

    4.3 Global Direction Estimator (GDE)Translating LDEs output to the global coordinate systementails 2 steps: (1) finding the true magnetic north in thephones coordinate system, and (2) measuring the angle be-tween the true magnetic north and LDEs output. Step 2is trivial, and in an ideal world, even step 1 is simply thedirection given by the smartphones compass reading. In re-ality, however, the smartphone compass is heavily influencedby magnetic interferers in the surroundings, especially in in-door environments. Figure 5(a) shows the compass readingsfrom Android and iPhones in 4 dierent environments anoutdoor sidewalk, a bio-engineering building, a computer sci-ence building and a residential hall. In 3 of these 4 cases, thecompass deviates heavily from the actual walking direction.Figure 5(b) shows a more detailed view a heat map of themagnetic intensity when the user walks with the phone ina rectangular corridor. The intensity varies frequently andranges from 10 to 80 micro-tesla, indicating high magneticinterference. As a result more than 95% of the corridor in-curs errors greater than 23 degree oset. However, on theoutdoor path around the building, the magnetic intensity re-mains close to its expected value, around 53 micro-tesla atthat location. In light of this, GDE concentrates on improv-ing the compass, which is likely to benefit other applicationsthat utilize the native compass app.

    Background on the Magnetic CompassThe Earth behaves like a giant magnet surrounded by linesof magnetic flux [24]. A freely suspended magnetic needlealigns itself with these lines and directs itself towards the

    magnetic North of the Earth. Although these magnetic fluxlines converge at magnetic poles, they appear parallel in agiven (small) region, due to Earths large diameter. Thus,when we walk with a smartphone in outdoor settings withzero magnetic influence, we observe that the compass read-ings a 3D vector are parallel to each other. However,as we introduced an artificial interference near the phone3,we observed that the measured compass vectors were dis-torted. The distortions were in the direction of the resul-tant of the Earths magnetic field (G) and the field causedby the interferer (I) shown in Figure 6(a). Now, with mul-tiple interferers close to the phone, the measured compassreadings changed quickly since small displacements broughtthe phone relatively closer to some interferers and furtherfrom others. When the interferers were moved far away, themagnitude of the interference reduced, and the fluctuationsin the readings naturally subsided. The measured compassvector exhibited an oset from the actual magnetic north,and this oset changed slowly over time (Figure 6(b)).

    Figure 6: (a) Measured compass vector (R) is theresultant of earths magnetic field (G) and the inter-ference vector (I). (b) Compass vector R changingslowly as user moves.

    Intuition for Compass CorrectionThe above observations, although unsurprising, oered usan intuition. We recognized that when the compass is mov-ing, the data it records is essentially multiple snapshotsof the magnetic interference from slightly changing angles.Assuming the interferer is far away and stationary, the con-secutive compass vectors should make small angles betweeneach other. Now, if the earths magnetic north vector was

    3Interferers can be permanent magnets, called Hard Ironinterference, or any ferromagnetic material, called Soft Ironinterference.

    ahrarnaqviHighlight

  • Figure 7: (a) With incorrect G vectors, the I vectorsdo not coincide. (b) Upon iterating over , for somevalue of G, the vectors coincide.

    indeed known, then subtracting this vector from the com-pass vectors, should have oered the interference vectors,and more importantly, these interference vectors shouldhave intersected at the location of the interferer. In reality,since the earths magnetic north is not known, we asked:what if we iterate over all possible vectors of magnetic northuntil we find the interference vectors intersecting at onelocation.

    Figure 7 illustrates the idea graphically. Compass readings[R1, R2, R3...] correspond to a moving user recording thereadings at times [t1, t2, t3...]. Now, since the direction ofG is not know, it is possible to assume an arbitrary direc-tion, say G, and subtract from the R vectors, resulting ininterference vectors, I1, I2, and I3. If G is incorrect, theinterference vectors are not expected to intersect at a com-mon point. However, when iterated for all values of , thecorrect value of G may be expected to oer a common pointof intersection, shown in Figure 7(b). If this intersectionindeed occurs, then we can select the corresponding G, anduse that as an estimate of the true magnetic north.

    A natural question is: does the IMT algorithm assume onlyone interferer in the ambiance? While it may appear to beso, observe that the interference vector we estimate couldactually be the (vectorial) sum of all interferences in theambiance. This is modeled under the principle that dier-ent force vectors can be linearly added and represented by asingle resultant vector. Of course, there is no reason to be-lieve that these resultant vectors would originate or culmi-nate at a single source (the point of intersection). However,given that the spatial gaps between R1, R2, R3 is small (say,three consecutive samples at 25 Hz), the resultant vectorsare likely to be oset by small amounts, and hence, couldbe expected to meet at nearby points (a heuristic). The rest

    of this section details this algorithm, called Iterative Mag-netic Triangulation (IMT). Several pre-processing steps andfine-tuning are necessary to make the algorithm generic androbust to widely varying real-world conditions.

    Iterative Magnetic Triangulation (IMT)We describe 5 steps of the IMT algorithm next.

    Step 1: Vector SelectionThe compass continuously provides a series of Ri vectors,however, not all these vectors are suitable for the IMTalgorithm. For example, when the user is not moving,the compass reports unchanging magnetic vectors that areparallel to each other. The interference vectors for thosesamples will also be parallel, rendering triangulation infea-sible. On the other hand, if the phone sways significantly(on the axis perpendicular to the of walking direction), thenit breaks IMTs model that the phone is dominantly mov-ing along the walking direction. Finally, the triangulationheuristic requires the interferers to be suciently far, sothat within a small time window, the magnitudes of the in-terference vectors are almost equal. Recall that with nearbyinterferers, the magnitudes change much quicker.

    In view of these constraints, IMT treats the compass dataas a signal and first passes it through a low pass filter.This eliminates the high frequency components correspond-ing to nearby interferers, and leaves the influences of faraway and strong interferers. On this residue signal, IMTopportunistically selects 3 consecutive vectors that are notparallel, and whose projections on the sway-axis have negli-gible variation. We find reasonable number of vector triplets< Ri, Ri+1, Ri+2 > that satisfy this criteria. Recall thateven if WalkCompass obtains the global walking directionin a few spots, it can use the gyroscope to track the user inbetween those spots (since the gyroscope is not aected bymagnetic interference).

    Step 2: Iteration and TriangulationIMT has a reasonable (though not precise) estimate of themagnitude of G, based on the phones crude location (atthe granularity of, say, zip codes). This can be found fromthe International Geomagnetic Reference Field database [6].However, the direction of G is unknown and IMT iteratesover all possible values of . Thus, for a given , IMTsubtracts G from each of < Ri, Ri+1, Ri+2 > to compute< Ii, Ii+1, Ii+2 >, and then observes how the interferencevectors intersect. When the intersection points are tightlyclustered (defined in more detail later), the correspondingG is selected as the earths magnetic field. If multiplevalues of present tight clusters, IMT chooses the values of that is closest to the compass reading. Finally, if no tightclusters are found, IMT attempts the same operation on thenext valid vector triplet, or waits a few seconds to get freshdata from the walking user.

    Step 3: Refining Magnitude of GUnfortunately, we observed that in some cases, the roughmagnitude ofG (looked up from the Reference Field database)provides unreliable results. This is due to inaccurate lo-cations as well as unknown heights (i.e., a smartphoneon the 5th floor of a building may observe dierent ge-omagnetic forces compared to sea level values from the

    ahrarnaqviHighlight

  • 1000 500 0 500 1000 15003000

    2000

    1000

    0

    1000

    2000

    3000

    X Axis

    Y Ax

    is

    100 200 300 400 500300

    250

    200

    150

    100

    50

    0

    50

    X Axis

    Y Ax

    is

    0 100 200 300 400100

    200

    300

    400

    500

    600

    Angle (degree)

    Tigh

    tnes

    s Met

    ric

    Figure 8: The locus of two interference vectors when iterating over geomagnetic vectors. If the magnitude ofthe geomagnetic vector is incorrect then the loci creates (a) an irregular shape, otherwise, (b) the loci createsa regular ellipse/circle shape. (c) Varying values of tightness with increasing IMT picks G correspondingto minimum tightness at around 260.

    database). Importantly, we find that when the magnitudeof G is erroneous, the locus of the intersection points of< Ii, Ii+1, Ii+2 >, computed from all values of , followerratic shapes (Figure 8(a)). On the other hand, when themagnitude of G is close to the correct value, this locus formsa reasonably closed-formed shape, resembling a circle or anellipse (Figure 8(b)). In view of this, IMT checks the shapeof the locus and iteratively reduces the magnitude of G untilthe locus forms the expected shape.

    Step 4: Picking Tight Intersection ClustersIdeally, each pair in the inference vector triplet should in-tersect at the same point. In practice, however, the pointsof intersection form a cluster. IMT picks the value of forwhich it finds the tightest cluster of intersections. We definethe tightness of a cluster as the sum of all pairs of pointsin that cluster.

    Step 5: Final Global Walking DirectionFigure 8(c) shows the variation of tightness values for a fulliteration of G for a given vector triplet. IMT finds thelowest point of this graph (i.e., the minimum value of tight-ness), and the corresponding G (263

    in this example) isannounced as the estimated geomagnetic north. Note thatthis can also be used as the new compass output, improvingthe inherent quality of the compass. Finally, WalkCompasscompares the local walking direction against this estimatednorth, and outputs the global walking direction of the user.Algorithm 1 presents the pseudo code for the IMT algorithm.

    5. EVALUATION

    5.1 Implementation and MethodologyWalkCompass has been implemented on Android, using theJellybeans version, and tested using a variety of Samsungphones. The codebase has also been replicated on MAT-LAB to test optimization modules oine the appropriateoptimization have are ported back into the phone. Figure9 shows a screenshot with the gray cone(thicker) showingthe compass direction; the green cone(thinner) denotes theusers walking direction. The width of the cone is pro-portional to the variance of walking direction a usefulvisualization for debugging in real conditions.

    Algorithm 1: Pseudocode for the IMT algorithm

    Data: continuous magnetometer dataResult: est< Ri, Ri+1, Ri+2 > = Vector Selection(filtered input);rG = geomagnetic intensity;while rG > threshold do

    for G 0 to 360 doG = vector with magnitude rG and angle G ;Ii = Ri G ;Ii+1 = Ri+1 G ;Ii+2 = Ri+2 G ;< Pi, Pi+1, Pi+2 > = pairwise intersections of< Ii, Ii+1, Ii+2 > ;t = sum of all pairwise distances between Pi,Pi+1 and Pi+2. ;Store < (t, G) > in T ;if Locus of < Pi, Pi+1, Pi+2 > goes irregularthen

    reduce rG exponentially ;break ;

    endendest = G for which t is minimum in T ;return est ;

    end

    WalkCompass experiments were performed with 6 users whovolunteered to walk with the phone in dierent environ-ments, using varying holding positions. The environmentswere mostly UIUCs science and engineering buildings withheavy magnetic influence this subjects WalkCompass to astronger test. For each test, we asked the users to walk alongestablished corridors this is because we used the Googlesatellite view to compute the global truth in walking direc-tions. Some experiments were also performed in Wal-mart,houses, and apartments, and the global truths were com-puted similarly. We also walked along with the users notingdown when they changed the orientation of the phone, orhow they placed the phones in the pocket. We report re-sults against the baseline of the phones compass.

    5.2 Performance ResultsWe intend to concentrate on the following questions:

  • 150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    compassw/ turnsw/o turns

    150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    w/ turnsw/o turns

    150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    w/ turnsw/o turns

    Figure 10: CDF of WalkCompasss LDE error with and without turns. (a) Phone held in the palm. (b)Phone carried in the pant pocket. (c) Phone held in swinging arms.

    Figure 9: (a) WalkCompass screenshot. (b) QRcode for WalkCompass demo.

    The accuracy of recognizing the walking direction in thephones local coordinate system (i.e., results from out-doors and some interference-free indoor environments) Figures 10, and 11. Speed of change detection, especiallyafter sharp turns (Figure 12).

    Coping with dierent holding positions (Figure 13), anddierent orientations such as portraits to landscapes (Fig-ure 14). Robustness when user is bear foot or wearingshoes, and walking on various surfaces, such as carpet,concrete, etc. (Figure 15).

    Accuracy of compass error correction with IMT Figures17, and 18. Its performance under various regimes ofmagnetic interference (in labs,oce,house, and outdoorsettings) Figures 19, and 20.

    Accuracy of Local Walk Direction (LDE)Figure 10 plots the CDF of local walking direction error,estimated from all traces, across all users, and across allenvironments. Figure 10(a) plots the case when the phoneis held in the palm (in the browsing position) some are inportrait, some landscape, and some users held the phone atan angle to the walking direction. Figure 10(b) and (c) plotthe case of pant pocket and swinging hands, respectively.WalkCompasss error is computed as the dierence with thetrue walking direction (computed manually for every tracefrom Googles satellite view). Since we are unsure aboutthe global truth near the turning positions (recall that usersmay be making soft turns), we show the error distributionsincluding and excluding the turns. While excluding turns,we have removed 6 steps at the each turn.

    We could not develop a meaningful baseline scheme forcomparison, hence, we use the following. For the palm posi-tion in Figure 10(a), we record the orientation of the phoneonly when it is pointed in the direction of walking. Wesmoothen this data through a low pass filter (as a way ofcanceling out periodic perturbations) and plot its error CDF denoted compass. Evidently, even when the phone ispointing in the forward direction, and the data smoothened,the fluctuation is appreciable. Since we could not learn thephones orientation in Figure 10(b) and (c), we do not plotthe compass data in these two graphs.

    Evident from the graph, WalkCompass exhibits low medianerror for a sizable fraction of the scenarios, and the errorsare well distributed on both sides of the walking direction.This suggests that the heel-strike jerk manifests itself promi-nently across all these phone postures. The performancedegrades when the hand swings in fact, performance forone of the 5 users is poor, significantly skewing the distri-bution from the center. On close observation, we noticedthat this users hand-swings are embedded with several ro-tational motions of the wrist, and even though they arerepetitive, WalkCompass is unable to cancel them out. Forall other cases, the performance is reasonably consistent.For the palm, the median error is at 5, compared to23 for the phones native compass app (with low passfiltering). For the pant pocket, the median is at 3, andfor the swinging-hand, 8.

    Figure 11 breaks down the results for each individual user,demonstrating that the LDE module is fairly robust to dif-ferent walking patterns. In all these traces, the user walkedfor around 20 steps on average before taking a turn, and hastaken around 25 turns in total. We show both the medianand the 75th percentile to reflect the robustness of the sys-tem. When carried in the palm or pant pocket, the errors arearound 6, except for one user who experiences around 20

    error. While this is not ideal, and leaves room for improve-ment, we believe it is still useful for various applications.An elevator company keen on automatically dispatching el-evators based on approaching users finds such error marginstolerable.

    Direction Change DetectionFigure 12 zooms into the turning behavior of users this is arepresentative graph that plots the error when a user makes

  • All U1 U2 U3 U4 U5 U60

    5

    10

    15

    20

    25Er

    ror (

    degr

    ee)

    Median75th Percentile

    All U1 U2 U3 U4 U5 U60

    5

    10

    15

    20

    25

    Erro

    r (de

    gree

    )

    Median75th Percentile

    All U1 U2 U3 U4 U5 U60

    5

    10

    15

    20

    25

    Erro

    r (de

    gree

    )

    Median75th Percentile

    Figure 11: The median and 75th percentile of error per user without turns. Phone carried in (a) palm, (b)pocket, (c) swinging-hand.

    a 180 turn. This is the worst case behavior, and the graphshows the time it takes to converge to the actual walkingdirection. Evidently, the error decreases at a steady pacewith more number of steps after the turn, and converges ataround 5 or 6 steps. We believe these results are slightlyconservative since the ground truth is assumed to be theintended direction in which the user is walking, and doesnot account for how the user actually walked. WalkCompassmeasures each of the micro-deviations the user makes at eachstep, and is hence slightly penalized here.

    0 10 20 300

    50

    100

    150

    Number of Steps

    Abso

    lute

    Erro

    r (de

    gree

    )

    Median75th Percentile

    Figure 12: Error convergence after sharp turns.Both median and 75th percentile error converge afteraround 4 or 5 steps.

    Toying with the PhoneUsers orient their phones in various ways, perhaps changingfrom a portrait to landscape for a video, raising the phonefor a phone call, or just tilting to check an email. Figure13(a) shows the changes in the phones compass directions,while WalkCompasss estimated direction continues to pointin the users heading direction. Since LDE computes theforce in 3D, and takes the projection on to the horizontalplane, the walking direction is estimated for all orientations(even when the user is holding the phone against the ears fora call). Figure 13(b) plots the distribution of errors acrossall users and traces.

    Figure 14(a) zooms in to the same results and shows a scatterplot from many users toying with the phone. For easiervisualization, all the traces have been oset to a commonwalking direction of 250. Evidently, LDE copes consistently

    20 40 600

    100

    200

    300

    Time (sec.)He

    ading

    (deg

    .)

    EstimatedCompass

    150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    EstimatedCompass

    Figure 13: (a) Examples of traces in which usersfrequently changes the orientation of the phone. (b)CDF of error across all traces.

    well with toying of phones and tracks the walking directioncorrectly. Some points fall far from the 250 line, however,observe that not too many points are consecutive, indicatingthat the large errors do not persist for long. To provide asense of how much the phone was toyed with, Figure 14(b)shows the angles to which the phone was pointing across allthe experiments.

    Impact of Walking on Different SurfacesGiven that the success of LDE relies on correctly detect-ing the accelerometer signatures during a walk, we evaluatewalking on various surfaces, with and without shoes. Usersin our experiments walked barefoot and with shoes, on dif-ferent surfaces, namely, carpet, and tiled floors. Figure 15

  • 0 20 40 600

    100

    200

    300

    400

    Time (sec.)

    Head

    ing (d

    eg.)

    EstimatedGround truth

    0 20 40 600

    100

    200

    300

    Time (sec.)

    Com

    pass

    Ang

    le (d

    eg.)

    Figure 14: (a) Estimated heading for multiple userswhen they randomly change the phones orientation.(b) Tracking the angle in which the phone is pointingto understand the extent of toying.

    shows the median and 75th percentile errors for each of these[footwear, surface] tuples. Observe that the errors are notaected by any specific surface the heel strike is reasonablyrobust and lends itself across all these scenarios.

    Staircase and Backward WalkWe evaluate dierent modalities of walking, such as walk-ing up and down staircases, and walking backwards. Figure16(a) plots the CDF for staircases while Figure 16(b) showsthe performance when the user walks backwards. For bothscenarios, the phone was carried in dierent orientations andpositions in the body. As with regular walking, the per-formance of a specific individual was relatively worst thanothers, especially for the hand-swing. Otherwise, the per-formance was stable. Finally, we note that in all cases, theerror of walking did not accumulate over time; even whenusers walked for long duration and experienced some errordue to some jerks on the phones or other actions, a new heel-strike would reset the errors. We believe this is a desirableresilience property in any sensing system, and WalkCompasspossesses it. We have omitted these results in the interestof space.

    Global Walking Direction Estimation (GDE)This section focuses on evaluating the accuracy of the globalwalking direction estimator (GDE) module, and specificallythe Iterative Magnetic Triangulation (IMT) algorithm. Totest this algorithm in diverse conditions, we select 15 dif-ferent locations from various regimes, including science and

    carpet w/o shoe

    carpet w/ shoe

    tiles w/ shoe

    tiles w/o shoe

    Figure 15: Walking on dierent surfaces (carpet andtiled floors) with and without shoes.

    engineering laboratories, social places, houses, libraries, andoutdoors. We apply the IMT algorithm on the magnetome-ter data and compare the results with the smartphones na-tive compass app. Since IMTs task is to only mitigate themagnetic interference (and not estimate walking directions),we hold the phone in the palm with the front of the phonefacing the walking direction. We carefully record the groundtruth from Google Satellite view.

    5.2.1 Error distributionFigure 17 plots the angle histogram across all traces, com-paring IMT and the phones compass against ground truth.Ideally this plot should have a spike only at 0, which is theground truth. Evidently, the compass measurements scat-ters around 60 on both sides of 0, occasionally reachingup to 90. The long histogram bar around 25 suggest thatthats the common case error. IMT, in contrast, produceserrors of 10 from the ground truth. Figure 18 shows thecumulative distribution of absolute error for both IMT andcompass. The curve rises steeply for IMT and its value re-mains less than 15 for 75% of the traces, whereas the rawcompass is at 37 for the same 75th percentile. The me-dian error of IMT is around 7, less than a third of compasserror. However, for very high magnetic interference, wherethe compass error is more than 50, the performance of IMTdegrades, although still remaining better than the compass.Figure 19 compares the median error of IMT and nativecompass for each of the 15 places. IMT is consistently halfor one-third of the compass. This is encouraging since evenoutside WalkCompass, the techniques from this paper canbe useful in improving the compass app in phones, especiallyin magnetic rich places where they are needed most.

    5.2.2 Distribution of interval between effective tripletsRecall that IMT estimates the true compass direction on aset of three magnetic vectors, called a triplet. Ideally, manytriplets should be available, so that WalkCompass can fre-quently estimate the true North. It is also advantageous tohave the triplets evenly spread over the path of a walk thisallows better interpolation (using the gyroscope) betweenthese correct points. Figure 20(a) shows the distribution ofthe intervals, in seconds, between two consecutive tripletsfound during the experiments. Although triplets tend to becollocated, we have found triplets separated by around 7 sec-

  • 150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    All UsersUser 16

    150 75 0 75 1500

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    All UsersUser 16

    Figure 16: (a) The cumulative error distribution forwalking up and down staircases. (b) CDF of errorfor walking backwards.

    onds (i.e. every 7 steps) even at the 90th percentile. Figure20(b) shows the number of triplets found in each individualexperiment. The density is sucient to continuously andreliably track the true geomagnetic North during the walk.

    6. LIMITATIONS AND DISCUSSIONWe discuss a few limitations and opportunities.

    Tail of Error Distribution: While median and 75thpercentile of accuracy is quite reliable with WalkCompass,in certain settings the performance drops sharply. This isparticularly evident in areas where the compass error is ex-tremely high greater than 45 and in other cases wherecertain users vary the phones orientation while swingingtheir arms. We do not have a strong handle on why these

    30

    210

    60

    240

    90

    270

    120

    300

    150

    330

    180 0

    30

    210

    60

    240

    90

    270

    120

    300

    150

    330

    180 0

    Figure 17: The angle histogram of (a) phones nativecompass app, (b) IMTs estimates.

    20 40 60 800

    0.25

    0.5

    0.75

    1

    Error (degree)

    CDF

    CompassIMT

    Figure 18: Comparing CDF of absolute error be-tween the compass and IMT.

    a b c d e f g h i j k l m n o0

    10

    20

    30

    40

    LocationsM

    edian

    Erro

    r (de

    g.)

    CompassIMT

    Figure 19: The median error comparison betweenIMT and compass across all 15 places.

    cases occur deeper engineering and fine tuning is necessaryto cut-back on the tail of the error distribution.

    Need to Walk a Few Steps: The proposed IMT al-gorithm cannot use any magnetometer data to infer themagnetic north only certain magnetometer snippets pro-duce the correct answer. This indicates that the user wouldhave to walk a distance before WalkCompass can estimatethe magnetic North. While this may be tolerable for humanwalking applications, additional research is needed if thephones native compass has to be improved. Our ongoingwork is investigating methods to infer the true North fromany snippet of magnetometer data.

    Beyond Walking: While WalkCompass has not beentested for other forms of human locomotion, such as wheelchairs, skate-boards, biking, etc., we believe the core tech-niques may still apply so long as there is a repetitive force inthe heading direction. Pushing the wheel for wheel chairs,swinging the leg for skateboards, and rotation of the legsduring cycling, all seem to oer this opportunity. We planto test this in future.

    3DWalkCompass: This paper investigates human walk-ing direction on 2D, but we believe that the core techniquescan be scaled to 3D, albeit some additional complexity inthe IMT algorithm. We leave this to future work.

    7. RELATEDWORKWe discuss past work on detecting walking directions.

  • 10 20 30 400

    0.25

    0.5

    0.75

    1

    Interval (sec.)

    CDF

    a b c d e f g h i j k l m n o0

    10

    20

    30

    40

    Locations

    Tripl

    et C

    ount

    Figure 20: The distribution of the gap between twoconsecutive triplets(a) and number of triplets foundper location(b).

    Finding Heading DirectionBody-pasted sensors: Early work from anatomy analy-sis, skeletal parsing, and computer animation have studiedthe problem of walking direction using body-pasted sensors[16, 32, 17]. Smartphone sensors are characterized with farmore degrees of freedom, given that it can be carried indierent clothing, dierent parts of the body, or handled inunknown ways by the user. The challenges in coping withthe variations are fundamentally dierent.

    Sensor dead reckoning: Attempts have been made tosolve the problem with smartphones, however, solutionsmake certain assumptions. Specifically, [22, 19] assume thatthe initial orientation of the phone is known, and the phoneis held stable in the hands. Authors in [20] infer orientationby identifying specific gestures like texting, at which pointthe phone is assumed to be in a known stable orientation.In the outdoor setting, GPS can oer heading direction,although with limited responsiveness; GPS may also falterin Manhattan-like settings. WalkCompass relies only on thedirection of the forces on the smartphone, thereby eliminat-ing reliance on other technologies.

    Map based: The application of particle filters on floor-plans is a popular technique [7, 25, 28, 18, 11]. However,the approach fails in open spaces such as halls, airports,libraries, atriums of hotels, etc. Moreover, the reliance offloorplans restrict the applicability of these solutions. Zee[28] proposes an interesting stand-alone approach to inferthe heading direction, essentially deriving hints from the

    frequency response of the accelerometer. However, the tech-nique alone is insucient and leaves an ambiguity betweentwo opposite directions. It relies on the map for disam-biguation.

    Vehicle dynamics: In essence, the analysis of vehicle dy-namics using inertial sensors is also related to WalkCompass,despite dierences in approach and methodology. Some re-cent papers in this domain have leveraged sensor data fromsmartphones to track the movement of vehicles for applica-tions like driver detection [8], driving behavior analysis [15,31, 10], and estimation of road conditions [23].

    Correcting Compass ErrorGeomagnetic fields and compass errors are well investigatedareas of study many surveying organizations [6, 26] pre-cisely record and tracks the behavior of global magneticfields and anomalies. For local magnetic distortions, somemathematical models are available [4] to quantify the errorand apply corrective algorithms like Ellipsoid Fitting [21].However, these approaches rely on calibrating the magneticfield at various locations using specific devices and method-ology. These techniques are therefore suited for one-time ap-plications, say robotic motion planning or war-driving, butdoes not scale to anywhere, anytime computing. WalkCom-pass attempts to correct the compass in a calibration-free,stand-alone manner.

    8. CONCLUSIONThis paper shows that the humans walking direction can beestimated from the smartphones inertial sensors, regardlessof the orientation of the phone on the body. The core tech-niques are rooted in analyzing the relationship between hu-man walking and its eect on the phone, as well as methodsto estimate and cancel magnetic interference from the com-pass data. We believe that WalkCompass can immediatelyhelp a variety of apps that make assumptions on the userswalking direction. More importantly, we believe that withsome more eort, the phones native compass can be dra-matically improved, ultimately helping all applications thatrely on the compass.

    9. ACKNOWLEDGMENTWe sincerely thank our shepherd, Dr. Xiaofan Jiang as wellas the anonymous reviewers for their invaluable feedback.We are also grateful to Intel and NSF for partially fundingthis research through the following grants: NSF 0910846 andNSF 1040043.

    10. REFERENCES[1] Complementary filter.

    http://web.mit.edu/scolton/www/filter.pdf.[2] Unibros. http://hfid.olin.edu/sa2013/s_

    engr3220-unibros/inspirational.php.[3] Walkcompass demo. http://synrg.csl.illinois.

    edu/projects/localization/walkcompass.[4] Afzal, M. H., Renaudin, V., and Lachapelle, G.

    Assessment of indoor magnetic field anomalies usingmultiple magnetometers. Proceedings of ION GNSS10(2010), 19.

    [5] Ayub, S., Heravi, B. M., Bahraminasab, A., andHonary, B. Pedestrian direction of movement

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

  • determination using smartphone. In Next GenerationMobile Applications, Services and Technologies(NGMAST), 2012 6th International Conference on(2012), IEEE, pp. 6469.

    [6] Barton, C. International geomagnetic reference field:the seventh generation. Journal of Geomagnetism andGeoelectricity 49 (1997), 123148.

    [7] Chintalapudi, K., Padmanabha Iyer, A., andPadmanabhan, V. N. Indoor localization without thepain. In Proceedings of the sixteenth annualinternational conference on Mobile computing andnetworking (2010), ACM, pp. 173184.

    [8] Chu, H., Raman, V., Shen, J., Kansal, A., Bahl,V., and Choudhury, R. R. I am a smartphone and iknow my user is driving.

    [9] Constandache, I., Choudhury, R. R., and Rhee,I. Towards mobile phone localization withoutwar-driving. In INFOCOM, 2010 Proceedings IEEE(2010), IEEE, pp. 19.

    [10] Dai, J., Teng, J., Bai, X., Shen, Z., and Xuan, D.Mobile phone based drunk driving detection. InPervasive Computing Technologies for Healthcare(PervasiveHealth), 2010 4th International Conferenceon-NO PERMISSIONS (2010), IEEE, pp. 18.

    [11] Evennou, F., Marx, F., and Novakov, E.Map-aided indoor mobile positioning system usingparticle filter. In Wireless Communications andNetworking Conference, 2005 IEEE (2005), vol. 4,IEEE, pp. 24902494.

    [12] Faure, F., Debunne, G., Cani-Gascuel, M.-P.,and Multon, F. Dynamic analysis of human walking.In Computer Animation and SimulationaAZ97.Springer, 1997, pp. 5365.

    [13] Gafurov, D., Helkala, K., and Sndrol, T. Gaitrecognition using acceleration from mems. InAvailability, Reliability and Security, 2006. ARES2006. The First International Conference on (2006),IEEE, pp. 6pp.

    [14] Inman, V. T. Human locomotion. Canadian MedicalAssociation Journal 94, 20 (1966), 1047.

    [15] Johnson, D. A., and Trivedi, M. M. Driving stylerecognition using a smartphone as a sensor platform.In Intelligent Transportation Systems (ITSC), 201114th International IEEE Conference on (2011), IEEE,pp. 16091615.

    [16] Kim, J. W., Jang, H. J., Hwang, D.-H., and Park,C. A step, stride and heading determination for thepedestrian navigation system. Journal of GlobalPositioning Systems 3, 1-2 (2004), 273279.

    [17] Krach, B., and Robertson, P. Integration offoot-mounted inertial sensors into a bayesian locationestimation framework. In Positioning, Navigation andCommunication, 2008. WPNC 2008. 5th Workshop on(2008), IEEE, pp. 5561.

    [18] Kwon, W., Roh, K.-S., and Sung, H.-K. Particlefilter-based heading estimation using magneticcompasses for mobile robot navigation. In Roboticsand Automation, 2006. ICRA 2006. Proceedings 2006IEEE International Conference on (2006), IEEE,pp. 27052712.

    [19] Lee, S.-W., Jung, P., and Song, S.-H. Hybridindoor location tracking for pedestrian using a

    smartphone. In Robot Intelligence Technology andApplications 2012. Springer, 2013, pp. 431440.

    [20] Li, F., Zhao, C., Ding, G., Gong, J., Liu, C., andZhao, F. A reliable and accurate indoor localizationmethod using phone inertial sensors. In Proceedings ofthe 2012 ACM Conference on Ubiquitous Computing(2012), ACM, pp. 421430.

    [21] Li, Q., and Griffiths, J. G. Least squares ellipsoidspecific fitting. In Geometric Modeling and Processing,2004. Proceedings (2004), IEEE, pp. 335340.

    [22] Link, J. A. B., Smith, P., Viol, N., and Wehrle,K. Footpath: Accurate map-based indoor navigationusing smartphones. In Indoor Positioning and IndoorNavigation (IPIN), 2011 International Conference on(2011), IEEE, pp. 18.

    [23] Mednis, A., Strazdins, G., Zviedris, R.,Kanonirs, G., and Selavo, L. Real time potholedetection using android smartphones withaccelerometers. In Distributed Computing in SensorSystems and Workshops (DCOSS), 2011 InternationalConference on (2011), IEEE, pp. 16.

    [24] Merrill, R. T., McElhinny, M. W., andMcFadden, P. L. Magnetic Field of the Earth,vol. 63. Academic Press, 1996.

    [25] Nandakumar, R., Chintalapudi, K. K., andPadmanabhan, V. N. Centaur: locating devices in anoce environment. In Proceedings of the 18th annualinternational conference on Mobile computing andnetworking (2012), ACM, pp. 281292.

    [26] Olsen, L., Major, G., Shein, K., Scialdone, J.,Vogel, R., Leicester, S., Weir, H., Ritz, S.,Stevens, T., Meaux, M., et al. Nasa/global changemaster directory (gcmd) earth science keywords.Version 6, 0.0 (2007), 0.

    [27] Pei, L., Liu, J., Guinness, R., Chen, Y.,Kuusniemi, H., and Chen, R. Using ls-svm basedmotion recognition for smartphone indoor wirelesspositioning. Sensors 12, 5 (2012), 61556175.

    [28] Rai, A., Chintalapudi, K. K., Padmanabhan,V. N., and Sen, R. Zee: Zero-eort crowdsourcingfor indoor localization. In Proceedings of the 18thannual international conference on Mobile computingand networking (2012), ACM, pp. 293304.

    [29] Susi, M., Renaudin, V., and Lachapelle, G.Motion mode recognition and step detectionalgorithms for mobile phone users. Sensors 13, 2(2013), 15391562.

    [30] Wang, Y., Jia, X., Lee, H., and Li, G. An indoorswireless positioning system based on wireless localarea network infrastructure. In 6th Int. Symp. onSatellite Navigation Technology Including MobilePositioning & Location Services (2003), no. 54.

    [31] Wang, Y., Yang, J., Liu, H., Chen, Y., Gruteser,M., and Martin, R. P. Sensing vehicle dynamics fordetermining driver phone use. In Proceeding of the11th annual international conference on Mobilesystems, applications, and services (2013), ACM,pp. 4154.

    [32] Woodman, O., and Harle, R. Pedestrianlocalisation for indoor environments. In Proceedings ofthe 10th international conference on Ubiquitouscomputing (2008), ACM, pp. 114123.

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight

    ahrarnaqviHighlight