Edge Computing with Jetson TX2 for Monitoring Flows of...

Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrians and Vehicles

Dr J. Barthélemy, Dr N. Verstaevel, Dr H. Forehead, Senior Prof. P. Perez

Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrian and Vehicles

At SMART, we believe that People with good

information and good tools will make good

Decisions and change our world

2

Technology and community: DLL

3

Solving problems: The case of Liverpool

CBD is growing fast: new UoW campus, airport development,…

What does it mean for the city and its community? What are the problems?

Smart Cities and Suburbs Program: How can we use IoT to solve the problems?

4

Which problems?

Let’s ask the people! Pedestrians- Where are they going?- What are the most popular routes?- What are the most congested locations?- Impact of city activity?

Cyclists- Which route are they taking?- How can we improve bike usage?

Cars- Live traffic?

5

Sensors locations

20 sensors

Image credit: OpenStreetMap

6

How about using CCTV?

7

Privacy!

Sensor requirements

• Mobile units + leveraging CCTV infrastructure

• Privacy is important!

– On board video analytics

– Only indicators transmitted (no raw data!)

• Real-time image processing

• LoRaWAN network

– Long range, low bandwidth (200 bytes/message)

– Free to use by the community8

LoRaWAN: an IoT network

9

Image credit: https://ttnmapper.org

History of the prototypes

10

The sensorAn edge computing device

Components

— NVIDIA Jetson TX2 for onboard processing

— Pycom LoPy 4 for data transmission on The Things Network

— Camera (USB webcam / existing CCTV)

11

Architecture of the solution

Fixed counters

Air quality x20

Noise level x20

x5

x15

Mobile counters

Sensors IoT Core ApplicationsTransport

+Private and Public APIs Traffic

modelling

Analytics

Dashboard

Citizens app

12

From the input to the dashboard

Image Acquisition

• CCTV/Webcam (OpenCV)

Detection

• Deep Convolutional Neural Network

Tracking

• Kalman Filtering

Data transmission

• LoRaWAN/OneM2M

Dashboard/Database

Image credit: NVIDIA Corporation

Image credit: Pycom

13

Detection: YOLO v3

Inference: Jetson TX2 @ 10 fpsTraining: Titan Xp

• Fully convolutional DNN• 106 hidden layers• Detections at 3 scales• 3 classes: person, bicycle, vehicles• Pascal VOC and COCO datasets

cuDNN

FP16+ +

14

NVIDIA GPU Grant

Detection: YOLO v3• Detecting locations of pedestrians and vehicles

• Number of objects of each type

15

VIRAT Video Dataset

Tracking: Kalman Filtering• Associating IDs with the detections

• Trajectories

16

VIRAT Video Dataset

Final output

No image Privacy OK!

17

Dashboard

18

DashboardHeatmap of the maximum number of detections

19

DashboardTrajectories of the detections

20

DashboardTrajectories of the detections (inside a building)

21

DashboardTrajectories of the detections (inside a building)

22

Dashboard

23

Next step: inferring network dynamics

Image credit: OpenStreetMap Image credit: Google Maps

24

ConclusionsIt’s only the beginning!

• Jetson TX2 for real time object detection and tracking

• Privacy compliant… but meaningful information

• Open data for people centric approach

– citizen applications

– city and traffic planners

• Scalability and interoperability

• Framework can integrates other sensors

– air quality, noise

Traffic modelling

AnalyticsDashboardCitizens app

25

An ecosystem around the Jetson TX2

26

Keep In Touch

johan@uow.edu.au

linkedin.com/company/smart-

infrastructure-facility-university-

of-wollongong

@SMART_Facility

smart.uow.edu.au

uowblogs.com/smartinfrastructureSMART Infrastructure Facility

27