SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING · Tornado Damage. L3HARRIS SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING 15 Hands-on Tutorial: Tornado Damage Detection Label “Here

SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING

And an introduction to the recently updated ENVI Deep Learning Module

May 12, 2020 Zach Norman and Bill Okubo – Product Management

L3HARRIS 2SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING

Contact Information and Introductions

Zachary Norman

Product Manager

[email protected]

Bill Okubo

Product Manager

[email protected]


Agenda

Introduction

Scenario One: Environmental Monitoring

Scenario Two: Disaster Response

• Hands-on Tutorial Overview

• Change Detection with Deep Learning

The ENVI Deep Learning Module

Additional Examples

Questions and Discussion


EMPLOYEES

50K~LOCATIONS IN

COUNTRIES

30~LOCATIONS

400~CUSTOMERS IN

COUNTRIES

130~

L3Harris Technologies is an agile,

global aerospace and defense

technology innovator, delivering

end-to-end solutions that meet

customers’ mission-critical needs.


Automatic Language Translation

Driverless CarsCaption Generation

Social Media & Online Shopping

What is Deep Learning: Applications as a Glance

Object Detection


Environmental Applications of Deep Learning

Deep learning is poised to work well for large-scale

monitoring applications, such as the impact of the oil

and gas industry

There can be regulations for companies to return the

environment to its original state

Information derived from deep learning can be used to

keep industry accountable or can help companies verify

they are on track with government regulations

1 2

3 1. A well pad preceding reclamation.

BLM photo.

2. Native plants in the first stage of

vegetative growth since reclamation.

BLM photo.

3. Yucca, Shinnery Oak, and various

native shrub and grass species serve as

evidence of successful reclamation

efforts. BLM photo.

Source: https://www.blm.gov/blog/2018-05-15/reclamation-success-stories-new-mexico-state-office

Success story from the U.S. Bureau of Land Management (BLM) showing reclamation progress

over time

https://www.blm.gov/blog/2018-05-15/reclamation-success-stories-new-mexico-state-office


Well Pads in Texas

31.527075°N, 102.062456°WN

This shows just how

prevalent the oil and gas

industry is in some parts

of the US and why it is

important to monitor the

environment


Well Pad Extraction with ENVI Deep Learning

How did we solve this problem?

• Used PlanetScope 4 band data

• 15 scenes from the Texas and Oklahoma area of the U.S.

• 11,202 labels as point features for rapid labeling

• Custom training parameters:

– Loss weight = 0.0

– Class weight = [2,3]

Training performance: 99.5% accuracy, 89.5% precision, 80% recall after 2.5 hours of training

IDL workflow using the

ENVI API that

completely automates

the data preparation

and training process

Yes, it is only 12 lines

of code!


Well Pads in Texas

31.527075°N, 102.062456°WN

This shows just how

prevalent the oil and gas

industry is in some parts

of the US and why it is

important to monitor the

environment


Detected Well Pads

31.527075°N, 102.062456°WN

ENVI Deep Learning

detected nearly 2900

well pads in this area


Lessons Learned and Takeaways

Easier Detects Harder Detects


What Comes Next: Assessing Environmental Recovery

Once you know where well pads were, monitoring them

is easy with the out-of-the-box tools in ENVI

Change detection, spectral indices, or other spectral

analysis in ENVI to monitor vegetation returning to its

original state

Looking at something as simple as the NDVI (Normalized Difference Vegetation Index) over well pads which, over

time, could be used to verify vegetation has returned after the site is cleared

1. Hot-spot analysis

2. Vegetation health

classification

1 2


Disaster Response

As we understand, some of the top priorities after a

disaster are to:

1. Rapidly assess the extent of the damage

2. Task resources to help in the recovery effort based on

where, and how severe, the damage is

With deep learning, once you have the data, you can

reduce the timeline down to hours rather than days

For reference, it can take up to 6 days to get accurate

maps on where damage has occurred after disasters like

tornadoes

Examples of different natural disasters and how you can see them with remotely sensed data. Examples: Fire extent (top-left and lower-left), landslides (top-right),

flooding (lower-right).


Tornado Damage


Hands-on Tutorial: Tornado Damage Detection

Label

“Here are some examples of the feature I’m interested in”

Train“Learn what the feature looks like”

Classify“Find more features like this elsewhere”

Use the Deep Learning Labeling Tool to create your classes, manage your training data, and automate

the training process

Use the ENVI ROI tool

to create accurate

training data using

points, lines, or

polygons



Label




Get real-time feedback on the performance of your models using TensorBoard which is included in the

ENVI Deep Learning Module



Label




Seamless integration with ENVI to easily classify your imagery, visualize results, cleanup results, and

generate vectors of your detections


Other Applications of Damage Detection with Deep Learning

We can also use change detection with deep learning,

including two separate images over the same area and

just focus on extracting the changes in the scene

Source data from the “xView 2: Assess Building

Damage” challenge (https://xview2.org/)

Data:

• Sensor: Maxar Worldview

• Images cover the same spatial extent and are

registered

• Used IDL to parse training data and create the labels

for training

Time 1 Time 2

https://xview2.org/


A Look at the Details

Use the ENVI Modeler to automate your deep learning workflow, train with the click of a button,

and not write a single line of code!


Results


The ENVI Deep Learning Module – Making Deep Learning Easy

Applied deep learning for geospatial imagery in ENVI, the

leading remote sensing and image analysis software

Without needing to program, the capabilities include:

• Segmentation (i.e. cloud masking, road surface)

• Object detection (i.e. cars, ships)

• Linear feature extraction (i.e. roads, rivers)

• Support for nearly any image format and data modality

• Can use points, lines, and polygons for trainingCreate Training

Data

Create Models

Evaluate Performance

Deep learning workflow in ENVI, built on TensorFlow and

Keras

Assess building damage after hurricanes and

tornadoes

Automated flood detection using SAR


Deep Learning Guide Map

Entry point to access all of the deep learning tools in ENVI such as:

• Training and retraining

• Training metrics

• Validating system setup and configuration

• Help content

• Pre-built ENVI Modeler workflows


Advanced ENVI Modeler Usage

Example workflow that fully automates the training

process from data preparation to classifying validation

images for performance review


Easy-to-use IDL API

Classifier Generation – 17 Lines of CodeCustom Data Preprocessing – 21 Lines of Code


TensorBoard Integration

TensorBoard is a tool for visualizing

the performance of your neural

networks during and after training

You can also view a diagram of your

network architecture

Makes it easy to compare training

sessions to one another

Ships with a built-in tool to manage

your metrics


TensorBoard: Metrics

Four metrics are reported during training:

• Accuracy

• Loss

• Precision

• Recall

During training these should increase:

• Accuracy

• Precision

• Recall

While the loss should decrease


Data Management and Labeling Tool

An interface which will:

• Manage training data

• Track classes for deep learning

• Save training parameters

• Train with a single click


Examples


Road Networks and Solar Panels

Automatically extract road networks from high-resolution

UAV or aerial imagery

Use UAV imagery to determine the locations of solar panels in

a neighborhood


SAR - Severe Burn Detection

Sentinel 1

Intensity series


Landcover Classification

36.318410°N, 119.316403°WN

Landsat 8 landcover classification

generated using the Cropland Data Layer

produced by the USDA


Field Boundary - Example 1

39.556898°N, 97.491028°WN


Sugar Beet Classification Image Created with ENVI Deep LearningGEO: 50.614780°N/6.989323°E

N

Source: Remote Sensing 2018 Weed Map Dataset, available from https://projects.asl.ethz.ch/datasets/doku.php?id=weedmap:remotesensing2018weedmap.


L3Harris Geospatial

www.L3HarrisGeospatial.com

[email protected]

303-786-9900

Questions and Discussion

Release details can be found here: https://www.l3harrisgeospatial.com/Support/Maintenance

Look for the “What’s New in ENVI Deep Learning 1.1” item for more information

Zachary Norman

Product Manager

[email protected]

Bill Okubo

Product Manager

[email protected]

https://www.l3harrisgeospatial.com/Support/Maintenance

Documents

SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING · Tornado Damage. L3HARRIS SOLVE GEOSPATIAL PROBLEMS WITH DEEP LEARNING 15 Hands-on Tutorial: Tornado Damage Detection Label “Here