GeoAI: Deep Learning meets ArcGIS - Harris Geospatial€¦ · AI with ArcGIS: End to End Cycle 4....

GeoAI: Deep Learning meets ArcGIS

Omar Maher

Practice Lead, Advanced Analytics. ESRI

omaher@esri.com

1. AI > ML > DL

Reasoning Knowledge Representation

Perception

RoboticsNLP Machine Learning

features Labels1. Training

2. Predicting

Supervised Learning

Unsupervised LearningReinforcement Learning

Deep Supervised Learning

Artificial Intelligence Machine Learning Deep Learning

Dog

Machine Learning

Deep Learning

Artificial Intelligence

CNTK TensorFlowTheano

Natural Language Processing

Video game behavioral AI

Robotics

Keras

IBM Watson

scikit-learn

Computer Vision

ArcGIS has Machine Learning Tools

ArcGIS

Classification

Clustering

Prediction

Machine Learning Tools in ArcGIS

•Maximum Likelihood Classification•Random Trees• Support Vector Machine

Clustering

• Empirical Bayesian Kriging•Areal Interpolation• EBK Regression Prediction•Ordinary Least Squares

Regression and Exploratory Regression•Geographically Weighted

Regression• Spatially Constrained Multivariate Clustering•Multivariate Clustering•Density-based Clustering• Image Segmentation•Hot Spot Analysis•Cluster and Outlier Analysis• Space Time Pattern Mining

Classification Prediction

Integration with External Frameworks

ArcGIS

GeoAI Sample Use-Cases (Videos)

Accidents Probability Prediction

High Resolution Land CoverObject Detection from Imagery Object Detection from Videos

Object Detection from Imagery Building Detection

Advanced Object Detection from Imagery Discover Deep Hidden Insights from Imagery Data

CAFO Site Detection using Deep Learning + ArcGIS Pro

1. Sample Training Data (Feature Layer of 700 CAFO Site)

3. Export Training Data 4. Train CNN

5. Detect Objects6. Call the model directly from Pro.. ..Via Python Raster Function

2. Add Imagery Source (NAIP)

Imagery Tools for Deep Learning

Bounding Boxes

Labelled Pixels+

Export Training Data For Deep LearningAvailable with Spatial Analyst / Image Analyst license

GIS Feature Class Imagery Data

Parking Utilization per Timeusing Fast RCNNs and Operations Dashboard

Object Detection from Satellite Imageryusing Deep Learning with ArcGIS Pro

Electric Substation Detection

AI with ArcGIS: End to End Cycle

4. ArcGIS Field Apps like Workforce + Collector and Survery123 could be used to plan inspections for detected sites. Operations Dashboard could be used to monitor execution of assigned tasks in Real-TimeInspections Results could then be analyzed in ArcGIS

1. ArcGIS “Export Training Data For Deep Learning” GP tool used to Prepare the a labelled training data set from feature class

2. CNTK or TensorFlow used to train a CNN to detect objects of interest using the labelled training data set

3. ArcGIS Imagery tools used for imagery management and analytics. A Raster Function is used to call the trained CNN and generate the results directly at Pro, allowing for further vector and raster analytics

Unassessed Pools DetectionIntegrating Deep Learning with ArcGIS

Road Detection from Satellite Imageryusing UNet with ArcGIS Pro

High Resolution Land CoverUsing Deep Learning to achieve 1-meter resolution land cover at scale

manually created a high-resolution land cover map for precision conservation of the Chesapeake watershed

100kmi2Area of watershed to map

2TBFile size of imagery to classify

18monthsTime to create map

By the time the land cover map was completed in December 2016, it was already out of date, and an update would be time-intensive and costly.

Land Classification Model

Convolutional

Network

Architecture

23 layer U-Net

Test ImagesLabeled

Training Images

Chesapeake

Conservancy

Dataset

Land

Classification

Model

Land Cover Map

Working Platform: GeoAI Virtual Machine

Dataset: 120k mi2 of imagery at 1-meter resolution,

split in half geographically into train and test sets

Algorithm Results

91%Average land

classification accuracy

16xFaster than Chesapeake

Conservancy’s previous

methods

High Resolution Land Classificationusing CNTK Convolutional Neural Networks

Real-Time Activity Detectionusing Deep Learning with ArcGIS API for Python

Blight/Graffiti/Cracks Detection from Street view Imagesusing Azure Custom Vision Service with PhotoSurvey

Buildings DetectionDetect Buildings, Roof Type, Height, and More!

Workflow

Labelled Lidar or Aerial Imagery

Export Training Data for Deep Learning

tool

Train Model using Mask RCNN

Output: Detected Buildings

+ Roof Type, Footage, Height and # Floors (if we have

Lidar)

Raster Function Runs Model w/ New

Imagery

Buildings Converted to 3D

Regularize Building Footprint (GP)

1. Labelled Lidar or Aerial Imagery 2. Detected Buildings

3. 3D Buildings w/ Roof Types

Automated Buildings Detection using Deep Learning

Combining the AI power of Microsoft with the geospatial analytics of Esri.

Pre-Configured environments in the cloud for GeoSpatialData Science & AI Modelling, Development & Deployment.

https://azure.microsoft.com/en-us/blog/microsoft-and-esri-launch-geospatial-ai-on-azure/

GeoAI Virtual Machine

ArcGIS Pro

Accidents PredictionPredict Accident Probability per Segment per Hour

What would Cause an Accident?

Temperature

Sun, Mon, Fri..

Visibility

High/Low

Wind Speed

Fast, Slow..

Snow Depth

High/Low

Time of the Day

12:45, 23:00

Day of the Week

Sun, Mon, Fri..Month

Feb, Dec..

Road Alignment

Straight / Curved

Proximity to

Intersections

Speed Limit

120 km/h

Sun Direction

East, West

Daily Traffic

AADT

Proximity to

Billboards

…

7 Years of Data400,000 Accidents500,000 Segments

10s of Variables

Road Width

20-30 M

Impossible to Manually Analyze

Train a Machine to do?

Accidents Probability Prediction using Scikit Learn XGBoost with ArcGIS Pro