Pervasive Indoor Localization and

Tracking Based on Fingerprinting

Gary Chan

Professor, CSE

HKUST

2

Catchphrase: Location, Location,

Location!

3

Outdoor Location-Based

Services (LBS)

• Based on GPS (Global Positioning System) with satellites– Widely and successfully commercialized

• Navigation, places of interest, hotels, parks, gas stations, etc.

4

Indoor Location-Based

Services (LBS)

Even larger and more lucrative market: more than 70%

of our time is spent indoors Don Dodge, developer advocate at Google

5

Market Opportunities

• A multi-billions market

• Malls, super-marts or exhibition centers– Help customers to find items, shops or booths

– Location-based recommendations, mobile advertisement and marketing

– Enable O2O (online-to-offline) business

– Track people and crowd/flow analysis

– Customer behavior

• Airport– Help tourists to find their gates for flights

– Shop while waiting for flights

• Social applications: finding nearby people sharing the same interest

• Location-based games (e.g., treasure hunt)

• Track assets

• Health– Track people, patients, etc.

Indoor Positioning:

Localization and Tracking

• Localization: Where am I?

• Tracking: Where are you?

User experience depends largely on the

accuracy of the technique!7

Challenges of Indoor GPS

• Global Positioning System (GPS) simply does not work indoor!– Line-of-sight not easily achievable indoor

• Complicated indoor environment leads to complex signal fading, shadowing and interference

• High requirements– Cost-effective to deploy

– High accuracy (~5m)

– Computationally light-weighted

– Power and memory efficient

– Privacy, etc.

8

Application Venues

• Malls are the most

mapped venues around

the world

• Airports come second

• Growing works on

advertising and analytics

to target customers in

these venues

Commerce79%

Airports13%

Other8%

4th Qtr0%

9

Wherami: The HKUST

Indoor Positioning Technology

10

Turning Research to

Commercialization

• Government– ITC LSCM

– Innovation and Technology

Funds (ITF)

• Technology Transfer

Center, Department and

School of Engineering at

HKUST– Advices and support on

contractual agreements

– R&D facilities and human

resources

• Industry sponsors and

collaborators– Funding, technical

support, trial sites,

equipment, etc

Tech.

Transfer

Wherami

11

Use of Wi-Fi for Positioning

• Pervasive deployment– In HKUST atrium, we can detect more than 400 AP hot

spots (HKUST 2013-08-08)

• The dominant localization technology by

2015

• Easy and cost-effective to deploy

(source: ABI research indoor location report)

12

Wi-Fi (Possily combined

with sensor fusion, Cell or

other)75%

Bluetooth 4.015%

Other (LED...)

10%

Wherami: Pervasive Indoor

Positioning

• Supported by ITC LSCM and industries

• Smart guide on smart-phones– Beyond just “digital map”

• Indoor navigation: How can I get there?

• Are any good promotions being offered around me?

• Are my friends around me? Where are my kids?

• Where are my customers? What are their behaviors?

• …

13

Fingerprint Concept

• Each person has a unique fingerprint

– Stored in a database

• Given a fingerprint

– Match it to the database

– Find the person

• Wi-Fi fingerprinting

– Fingerprint = Wi-Fi signals

– Person = Physical location

14

Wi-Fi Signals Collected at a

Location

15

The Wi-Fi Signals Form a Unique

Fingerprint for the Location

Site Survey and Fingerprint

Collection

User Location Calculation

Research Challenge: Signal

Measurement Noise

• Minimize the overall expected signal difference in signal map

• Subject to constraints on possible regions the target may be in

• 𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒

– 𝑖𝑁𝑚 Δ 𝑝, 𝑞𝑖 ⋅ 𝜔𝑖

• 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜– 𝑥 = 𝑖 𝜔𝑖 ⋅ 𝑟𝑖 , 𝑟𝑖 ∈ 𝐶𝑚 ,

– 𝑥 − 𝑥𝑙 ≤ 𝑑𝑙 , 𝑙 ∈ Λ

– 𝑖𝑁𝑚 𝜔𝑖 = 1, 0 ≤ 𝜔𝑖 ≤ 1

Stochastic Optimization

under Noisy Signals

Client-based Localization

• App-based

– Android + iOS

• Non-intrusive and transparent to existing

WiFi infrastructure

• “No string attached” besides your phones

Integration with Other Signals

• Bluetooth (BLE)/iBeacon

• Dead reckoning/INS (inertial

navigation system)

– Step counter, gyroscope,

accelerometer, etc.

• Magnetic field

• GPS/AGPS/NLP (Network

location provider)

• Barometer

• Ultrasound

• RFID

• Etc.22

Beacon 1 Beacon 2

Beacon 4

Beacon 3

• Good improvements on the area where WiFisignals are weak

BLE (Bluetooth Low Energy)

integration

23Case I – do nothing Case II – Restrict the estimation

Automatic Fingerprint Updating

for Survey Reduction

• Using Implicit crowdsourcing and Gaussian process regression (GPR)

• The samples has a joint Gaussian distribution

• Regression function

• Specified by mean and covariance function

• Joint Gaussian

24

𝑣 = 𝑓 𝑙 + 𝜀

𝑓 𝑙 ~ 𝐺𝑃(𝑚 𝑙 , 𝑘(𝑙, 𝑙′))

𝑐𝑜𝑣 𝒗 = 𝑲 + 𝜎𝑛2𝑰𝒗 ∼ 𝑵(𝒎 𝑳 , 𝑐𝑜𝑣 𝒗 )

𝒇∗|𝑳, 𝒗, 𝒍∗ ∼ 𝑵(𝝁∗, 𝜎∗

2)𝒇∗

𝒗∼ 𝑵(

𝒎(𝒍∗)

𝒎 𝑳,𝒌(𝒍∗, 𝒍∗) 𝒌 𝒍∗, 𝑳 𝑻

𝒌(𝒍∗, 𝑳) 𝑲 + 𝜎𝑛2𝑰

)

𝜀 ∼ 𝑵(𝟎, 𝜎𝑛2)

AP-Based Solution for

User Tracking

25

Design Dimensions

Without infrastructure

installation

Infrastructure outlay

(APs and iBeacons)

No client app installation

Client-based apps

26

User tracking and

analytics

User localization

and navigation

Better user positioning

accuracy and analytics

AP-based Tracking vs Client-

based Localization

AP-based Client-based

Device

Capability

Any Wi-Fi devices (including iOS,

Andriod, Wi-Fi tag, Windows, etc)

Require Wi-Fi API

Need ibeacon to localize iphones

Tracking

Transparency

Transparency to client side Client app interacts with the users

Accuracy

Improvement

Installation of more APs

Antenna improvement

Fusion of different signals

Computation Server Client, hence the device can be offline

Positioning

interval

Seconds Real time (less than a second)

Wherami Technology Features

• Driven by system research and advanced development

• Full solution suite for user positioning

• AP-based tracking

• Client-based localization– Joint AP-client localization to further improve accuracy

• Indoor-outdoor and multi-storey roaming (3D routing)

• Low cost, easy and fast to deploy

• Accurate and low computation – Precision 2m ~ 10m

• Low site survey cost– Adaptive to signal changes

• Ready to be integrated into other applications– Location API is provided

28

29

Main Functions

• Real-time positioning

• Real-time navigation– Destination search

– Best path selection

– Moving trajectory

– Panoramic visual clue

• Targeted services– Nearby facility search

– Program search

– Event/Ad prompt

Shopping Mart or Superstore

• Aim for better shopping

experience in large area

with many aisles

– Location-based promotions

– For bargain hunters

– For shoppers with a

shopping list

• Targeted advertisement

(behavior analysis)

30

Shopping Mall: Pushed Ads

31

Hospital, Out-Patients, Kids

and Elderly home

• Smart Wrist Band– Bluetooth or Wi-Fi enabled

– Detect the heart rate, step count, sleep status,

etc

• Tracking by SmartAP– Track and trace the location of patient/elderly

– Real time monitor and report the health

condition

33

Context-aware Computing or Smart

Space: Museum and Theme Parks

Location-based Games and

Treasure Hunting

34

Intelligent Transportation

Deployment Trials: HKIA,

Cyberport, Olympian City, Union

Hospital, HKCEC, HKUST, etc.

36

• Where is the check-in counter for Airline X?

• How to get to Gate 24?

• The closest restaurant?

• Push ads: there is a discount in the coffee shop nearby!

Hong Kong International Airport

37

HKIA trials

38

Checkin asile

K, J, H

Gate 20~26

Meeting Hall

Trials in HKIA

39

How we conduct trials

40

1. Identify reference points

2. Collect WiFi fingerprinting

3. WiFi data analysis and pre-process

Demonstration (youtube

video)

41

Cyberport Trial

42

Trial area:

1. The Arcade – whole

building

2. Cyberport 1 – floor L

3. Cyberport 2 – floor L

4. Cyperport 3 – floor L3

and L4

43

LBS in Cyberport

Location advertisementLocation game: treasure hunt Panoramic Visual Clue

Trial at Olympian City

44

• Where is the Pathology Centre?

• Where is the nearest washroom or food court?

Hospital

45

• Where is the xyz company booth?

• Are there any forum discussions this afternoon and where are they held?

• Where is the “3D Printing Solutions” zone?

• Where is my friend?

Hong Kong Convention and

Exhibition Centre

46

HKUST Campus

47

Trial area:

1. Academic Building

2. Lee Shau Kee

Business Building

3. Institute of Advanced

Study Building

• Where is classroom for Course X or Seminar Y?

• How to get to Room Z?

• My friend shares his position to me – I am going for a discussion with him!

HKUST Deployment

48

49

Wherami main functions

Navigation Content SearchLocalization

50

Wherami Trial Venues

Core Features Value Added Features

Wherami Trial Venue

Offline M

ap

Navig

ation

Targ

et

Searc

h

Augm

ente

d

Panora

mic

Vie

w

Location-b

ased A

ds

Cours

e/F

light

Searc

h

Sem

inar/

Pro

motion

Events

Pro

motion g

am

es

(e.g

. tr

easure

hunt)

Outp

atient

Pro

cedure

Flo

w

Real Tim

e Q

ueue

Sta

tus

Patient

Identity

Verification

Real Tim

e F

low

Contr

ol

University Campus• HKUST

Shopping Mall• Cyberport and OlympianCity

Airport• HKIA

Hospital• Union Hospital

Exhibition• HKCEC

Visit us at:

http://mwnet.cse.ust.hk/wherami

http://www.cse.ust.hk/~gchan

We are Happy to

Demonstrate Our

Technology at Your Sites

51