Detecting human activities using smartphones and maps

University of Illinois, Chicago 1

Detecting human activities using smartphones and maps

Leon StennethAdviser: Professor Ouri WolfsonCo-Adviser: Professor Philip Yu


Road map

• Outdoor transportation mode detection• Indoor and outdoor transportation mode

detection• Parking status detection• Parking availability estimation


Sensors

Image source:www.i-micronews.com

http://www.i-micronews.com/lectureArticle.asp?id=5147


Maps

• Bus stop locations, real time bus locations, road network, rail line trajectory, location of parking pay boxes, etc.


Transportation mode detection using mobile phones and GIS information

• Patent filed• Paper published at ACM SIGSPATIAL GIS 2011• 20 external citations


Problem

• Detecting a mobile user’s current mode of transportation based on GPS and GIS.

• Possible transportation modes considered are:


Motivations

• Value added services (e.g. in Google Maps)

• More customized advertisements can be sent

• Providing more accurate travel demand surveys instead of people manually recording trips and transfers

• Determining a traveler’s carbon footprint.


Contributions

• Improve accuracy of detection by 17% for GPS only models

• Improve accuracy of detection for 9% compared to GPS/GIS models

• Introduce new classification features that can distinguish between motorized and non-motorized modes.


Technique

• A supervised machine learning model

• New classification features derived by combining GPS with GIS

• Trained multiple models with these extracted features and labeled data.


Data model

• GPS sensor report: pi = <lat, lon, t, v, h, acc>

• GPS trace: T = p0 → p1 → · · · → pk


Approach

• In addition to traditional features on speed, acceleration, and heading change. We build classification features using GPS and GIS data

Mobile Phone’s GPS sensor report

Bus stop spatial data

Rail line spatial data

Real time bus locations

Training example


Features

• Traditional – Speed, acceleration, and heading change

• Combining GPS and GIS– Rail line closeness– Average bus closeness– Candidate bus closeness– Bus stop closeness rate


Rail line closeness

• ARLC - average rail line closeness• Let {p1, p2, p3, p4…pn} be a finite the set of GPS

reports submitted within a time window. ARLC = ∑i=1 to n di

rail / n


Average bus closeness (ABC)

• Let {p1, p2, p3, p4…pn} be a finite the set of GPS reports submitted within a time window.

ABC = (∑i=1 to n dibus) / n


Candidate Bus closeness (CBC)

• dj.tbus 1≤j≤m - Euclidian distance to each bus busj

• Dj - total Euclidian distance to bus j over all reports submitted in the time window

Dj = ∑t=1 to n dj.tbus 1≤j≤m

• Given Dj for all the m buses, we compute CBC as follows.

CBC = min (Dj) 1≤j≤m


Bus stop closeness rate (BSCR)• | PS | is the number of GPS reports who's

Euclidian distance to the closest bus stop is less than the threshold

BSCR = | PS | / window size

0 50 100 150 200 250 300 350 400 450 5000

20

40

60

80

100

120

140

GPS sensor report number

Eucl

ilidi

an d

ista

nce

from

clo

sest

bus

stop

(m

)


Machine learning models

• We compared five different models then choose the most effective– Random Forest (RF)– Decision trees (DT)– Neural networks (MLP)– Naïve Bayes (NB)– Bayesian Network (BN)

• WEKA machine learning toolkit


Evaluation matrices

• Precision(M)=(number of correctly classified instances of mode M) / (number of instances classified as mode M)

• Recall (M) = (number of correctly classified instances of mode M) / (number of instances of mode M)


Data set

• 6 individuals • 3 weeks


Results

• Random Forest was the most effective model

train bus

stationary

walk car bike

average

0

10

20

30

40

50

60

70

80

90

100

Traditional features onlyTraditional and GIS features

mode

prec

isio

n

train bus

stationary

walk car bike

average

0

10

20

30

40

50

60

70

80

90

100

Traditional fea-tures onlyTraditional and GIS features

mode

reca

ll


Feature Ranking

• Below we rank the features to determine the most effective.


Results

• Using the top ranked features only• Precision and recall is shown below

train bus

stationary

walk car

bike

average0

10

20

30

40

50

60

70

80

90

100

Top ranked features only

mode

prec

isio

n

train bus

stationary

walk car

bike

average0

10

20

30

40

50

60

70

80

90

100

Top ranked features only

mode

reca

ll


Deployed System

• We can provide further information (i.e. route, bus id) on the particular bus one is riding.


Related work with GPS

• Liao et. al (2004) – consider the user’s history such as where one parked or bus stop boarded.

• Zheng et. al (2008) – Robust set of GPS only features and a change point segmentation method.

• Reddy et. al (2010) – Combined accelerometer and GPS to achieve high accuracy.


Conclusion

• Using GIS data improves transportation mode detection accuracy.

• This improvement is more noticeable for motorized transportation modes.

• Only a subset of our initial set of features are needed.

• Random forest is the most effective model• We can provide further information about the

bus that a user is riding


Limitations and solutions

• Using GPS consumes battery power aggressively [explore low power sensors such as BT or accelerometer]

• Misclassification of car as rail [map matching using both road and rail artifacts]

• The effects of window size on classification feature effectiveness [more experiments]


Adding Accelerometer sensor to the model

• Acceleration in all three axes• Consumes less energy than GPS• Common on today’s mobile phone (e.g.

iPhone)

University of Illinois, Chicago

Adding accelerometer to the model

Mobile Phone’s GPS sensor report

Bus stop spatial data

Rail line spatial data

Real time bus locations

Training example

Noise fileter

Mobile Phone’s accelerometer sensor report


Contribution of accelerometer

• 4 % increase in outdoor detection accuracy

• Effective for indoor transportation mode detection (stairs, elevator, escalator)

• Finer granularity on mode detection (e.g. calorie trackers)


Accelerometer readingsSimple effective features

• DC component • Rxy ,Rxz ,Ryz(i.e. correlation

coefficient)• σx , σy , σz

• yx ratio• High and low peaks in time

window


Accelerometer and body position


Results

• Random Forest is most effective • Increase in 5.5% for outdoor transportation

mode• Detects each indoor (i.e. stairs, elevator,

escalator) mode by over 80% accuracy• GPS and GIS model by itself is not effective for

indoor transportation mode detection


Limitations of accelerometer study

• Small data set• Constrained mobile phone position


Real time street parking availability estimation

• Motivation– Vehicles searching for parking in LA business

district• CO2 emission (730 tons in 1yr)• Waste gasoline (burnt 47K gals 1yr)• Waste time (38 trips around the world)


Real-time street parking availability estimation

• The traffic product – sparse probes, map matching, map, travel speed, tta, color maps indicating current travel speed.

Parking status detection (PSD)

• Determines spatial-temporal property of parking event (maybe parking probes)

Image sources: http://videos.nj.com/, http://pocketnow.com/smartphone-news/http://sf.streetsblog.org

36

http://videos.nj.com/

http://videos.nj.com/

http://pocketnow.com/smartphone-news/

http://pocketnow.com/smartphone-news/

http://sf.streetsblog.org/


Parking status detectors (PSD)

Contribution to PSD: Three less expensive techniques to detect spatial and temporal property parking events using mobile phones [patent pending]

38

Our schemes for PSD

driving stop off-car park

passenger

unpark

stationary

car

stationary

walk

driving accompanied

driving unaccompanied

walk,stationary,bus,train

carcar

car

stationary

walk,stationary,bus,train

car


Our schemes for PSD


Street parking estimation model

location errorsfalse +false -false –

false +

• Estimate the number of available parking spaces on a street block.

• PSD – Parking status detector• HAP – Historical availability profile• PAE – Parking availability estimator

41

HAP construction scheme

• estimates the historic mean (i.e. ) and variance (i.e. ) of parking

• relevant terms– prohibited period, permitted period (PPi), fp, fn, b, N

Historical availability profile (HAP) Algorithm

• Start with a time at which the street block is fully available, e.g., end of a prohibited time interval (start permitted period)

• When a parking report is received, availability is reduced by:

• Deparking causes increase of availability by same factor

)1(1

fnbfp

b: penetration ratio(uniform distribution)

fn: false negative probability

fp: false positive probability

Justification:1. Each report (statistically) corresponds to 1/b actual parking2. 1/(1fn) reports should have been received if there were no false negatives3. The report is correct with 1fp probability

HAP algorithmPermitted period 1

m

tatq

m

ii

1

)(ˆ)(ˆ

m

tqtatQ

m

ii

1

2))(ˆ)(ˆ()(ˆ

43

Permitted period 2

Permitted period 3

Permitted period m

HAP algorithm termination condition

• HAP terminates when the difference between q(t) and is less than x parking spaces with k% confidence.

• Automatically determines m.

44


Computing confidence

• Assumptions– PSD vehicles are uniformly distributed among all vehicles

– Parking activities are detected independently of each other.

– are identically and independently distributed

• See upcoming lemmas:

)(),...,(),( 21 tatata m



• Lemma 1:• Proof

– pi(t)|Pi(t)Binomial(Pi(t), b(1fn)) 1.– di(t)|Di(t)Binomial(Di(t), b(1fn)) 2.

–

– From 1. – Thus,

)())(|)(ˆ( tatataE iii

)1(1))(|)((

)1(1))(|)(())(|)(ˆ(

fnbtDtdE

fnbtPtpENtataE

ii

iiii

)1()())(|)(( fnbtPtPtpE iii

)()()())(|)(ˆ( tatDtPNtataE iiiii



• Also showed that for i=1,2,…, m.

)())(ˆ( tqtaE i .

)())((

)})({(

))(|)(ˆ(})({())(ˆ(

0

0

tqtaE

kktaprob

ktataEktaprobtaE

i

i

N

k

iii

N

ki

More specifically:

• Example:– If we want error < 2 with 90% confidence,

• standard deviation of the estimation is 10 (i.e., the average fluctuation of estimated availability at the 8:00am is 10).

– then we need 68 permitted periods. • i.e. about two months of data.

1))(ˆ(2}|)()(ˆ{|Prob tQmtqtq

Estimation average Estimation varianceTrue average

Number of samples , or permitted periods

Cumulative distribution function of normal distr.


Evaluation of HAP

• Real parking signals from SF Park• Simulated errors (i.e. fp and fn)

HAP Results

• RMSE between q , b = 1%

Polk St. block12 spaces available

50

HAP Results

• RMSE between q , b = 1%

Chestnut St. block4 spaces available

51

Parking availability estimation (PAE) algorithms

• Proposed four algorithms – Solely real time observations

• scaled PhonePark (SPP) – capped

– Solely historical parking data (HAP)• historical statistics (i.e. HAP)

52

Parking Availability Estimation (PAE)

• Combining history (i.e. HAP) with real time– Weighted average with pre-fitted weights

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

RMSE

of e

stim

ated

mea

n

wHS

b=1%, fn=fp=0,Chestnut

b=1%, fn=fp=0.1,Chestnut

b=50%, fn=fp=0, Polk

b=50%, fn=fp=0.1, Polk

b=50%, fn=fp=0.25,Polk

53

Parking Availability Estimation (PAE)

• combining history (i.e. HAP) with real time– Kalman Filter estimation (KF)

.

54

PAE results

• RMSE between x for street block• b =1 % , see for b = 50% in paper

0

0.5

1

1.5

2

2.5

fn=fp=0.05 fn=fp=0.15 fn=fp=0.25

RMSE

of e

stim

ated

ava

ilabi

lity

WA

KF

SPP

HS

0.44

0.45

0.46

0.47

0.48

0.49

0.5

0.51

0.52

0.53

0.54

fn=fp=0.05 fn=fp=0.15 fn=fp=0.25

RMSE

of e

stim

ated

ava

ilabi

lity WA

KF

SPP

HS

55

PAE results

• Boolean availability i.e. at least one slot available • b =1 %

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

fn=fp=0.05 fn=fp=0.15 fn=fp=0.25

bool

ean

avai

labi

lity

accu

racy

WA

KF

SPP

HS

0.5

0.55

0.6

0.65

0.7

0.75

0.8

fn=fp=0.05 fn=fp=0.15 fn=fp=0.25

bool

ean

avai

labi

lity

accu

racy

WA

KF

SPP

HS

56


Conclusion

• We can provide reasonable parking availability estimation that does not deviate from the true availability by too much.

• Works under low penetration ratio (e.g. b=1%)

• Robust to false+ and false- errors


Limitations and solutions

• PSD penetration ratio can be low. [Can we use signals from neighboring blocks?]

• PAE algorithms did not consider the previous known parking availability at time t-1 for a street block [try to combine history & previous & current parking observations]


Current work• Increasing parking signals by using signals

from neighboring blocksSpear St (200-298) , The Embarcadero (61-69)

aver

age

park

ing

avai

labi

lity

0

5

10

15

20

0

5

10

15

20

time (1 hour epoch and starts at 9am each day of the weekstarting Sun. UTC)

0 12 24 36 48 60 72 84 96 108 120 132 144 156 168

0 12 24 36 48 60 72 84 96 108 120 132 144 156 168

block1block2

scatter plot (Spear St (200-298) , The Embarcadero (61-69))

bloc

k 2

avai

labi

lity

0

10

20

30

40

50

60

Y A

xis

Titl

e

0

10

20

30

40

50

60

block 1 availability0 10 20 30 40 50 60

0 10 20 30 40 50 60

R = 0.92distance between blocks = 0.13km


Current workspatial correlation

R

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

distance (km)0 1 2 3 4 5 6

0 1 2 3 4 5 6


Future work

• Temporal correlations• Incorporating neighboring signals in Kalman

Filter• Incorporating parking availability at previous

epoch in the model• New parking status detectors(e.g. acoustic

sensors)


The end

• Thanks you for your time

• Questions……….

Documents

Detecting human activities using smartphones and maps