From remote sensing to Unmanned Aircraft Systems (UAS ... · Reference on Drones • When we talk about drones, we are not only referring to big Predator and the likes, but we are

© 2019 BAE Systems. All Rights Reserved. ClearFlite, GXP, GXP OpsView, GXP WebView, GXP Xplorer, SOCET GXP, and SOCET SET are registered trademarks of BAE Systems. This document gives only a general description of the product(s) or service(s) offered by BAE Systems. From time to time, changes may be made in the products or conditions of supply. EXPORT-CONTROLLED DATA. This document contains technical information whose export is governed by the U.S. Export Administration Regulations (EAR). This informationmust not be transferred to a foreign person without proper authorization of the U.S. Department of Commerce. Violations may result in administrative, civil or criminal penalties. E-20190317-62. NOT INTENDED FOR PUBLIC RELEASE.

Presented by Rocco Corsetti, Phill Williams, Stephen Foster, and Joey Medina

From remote sensing to Unmanned Aircraft Systems (UAS), how drones are changing photogrammetric workflows

GXP360º Professional Exchange and WorkshopMarch 18-22, 2019 | San Diego, California

© 2019 BAE Systems. All Rights Reserved. EAR CONTROLLED. E-20190317-62. NOT INTENDED FOR PUBLIC RELEASE. Photogrammetric Workflows with UAS Imagery | Medina, Corsetti

Agenda

• Introduction

• Unmanned Aerial Systems (UAS) imagery processing capabilities:• Unmanned Aerial Vehicle (UAV) import • Triangulation

• Terrain generation-editing / Helicopter Landing Zone (HLZ) generation / orthomosaics

• Stereo, mono, and Automatic Feature Extraction (AFE)

• Data discovery, 3-D visualization, flythrough, measurements on 3D Multiport™ (simulated ops)



Introduction

Reference on Drones

• When we talk about drones, we are not only referring to big Predator and the likes, but we are also referring to small commercially built UAVs, built for commercial and private companies

Statistics

• Reuters: “The FAA on Tuesday also predicted the number of pilots of drones is expected to increase from 20,000 in 2016 to a range of 10 to 20 times as many by 2021

• Since August, the FAA has approved more than 300 waivers for drone use without some restrictions, including Union Pacific Railroad, BNSF Railway Co owned by Berkshire Hathaway, Intel Corp, Walt Disney Parks and Resorts, Time Warner’s HBO and CNN units”

https://www.reuters.com/article/us-usa-drones/u-s-commercial-drone-use-to-expand-tenfold-by-2021-government-agency-idUSKBN16S2NM

https://www.reuters.com/article/us-usa-drones/u-s-commercial-drone-use-to-expand-tenfold-by-2021-government-agency-idUSKBN16S2NM



Introduction

• DroneDeploy: “The Construction Industry is the Fastest Growing Adopter of Commercial Drones. Construction is currently the fastest growing sector— surging 239% in the last year — and is now the top industry using DroneDeploy. Builders deploy drones on job sites around the world to increase safety, document and understand site progress, and identify potential issues before they become costly.”

https://blog.dronedeploy.com/2018-commercial-drone-industry-trends-70b83e0a2e6f

• Allianz: “By 2020, there will be $2.7 million in commercial use US alone, with $127 billion value of global market for commercial applications, and 1 billion or more value of global commercial UAS insurance market”

https://www.agcs.allianz.com/solutions/aviation-insurance/drone-insurance.html

https://blog.dronedeploy.com/2018-commercial-drone-industry-trends-70b83e0a2e6f

https://www.agcs.allianz.com/solutions/aviation-insurance/drone-insurance.html



Introduction

Why UAVs?• Always costs less than flying small airplanes with photogrammetric cameras, and of satellites, "not to mention the cost

of a metric mapping camera and survey-grade GPS/IMU system, which could run $500K-1M “

• Operating costs in relation to area imaged is considerably lower

• More transportable and often safer to operate

• Easily used by military and civilian sectors

• DroneApps: “A Cessna 172 airplane, a model regularly used for aerial imaging, will set you back roughly $300,000. A professional automated mapping drone like senseFly's eBee RTK will cost you about $25,000, while DJI's Phantom 3 drone, which hovers on the border of the consumer and professional markets, is around $1,000.”

https://droneapps.co/price-wars-the-cost-of-drones-planes-and-satellites/

https://droneapps.co/price-wars-the-cost-of-drones-planes-and-satellites/



UAV processing workflow in SOCET GXP® v4.4

GXP Ecosystem

UAVWizard

•Triangulate Imagery•Generate 3-D Point Cloud•Generate DSM/DEM•Generate Orthomosiac

ProductGeneration

•High Resolution Terrain•Helicopter Landing Zones•Automatic Feature Extraction

3D Visualization

•Flythroughs•3-D Mensuration•Simulated Operations

Catalog•Discover UAV Imagery

Alert•Notify the Analyst

Open•Open Imagery in SOCET GXP

GXP Xplorer® SOCET GXP

Unmanned Aerial Vehicle (UAV)

File System

UAV Imagery



Data discovery, 3-D visualization, flythrough, measurements on 3D Multiport

7

• In our scenario, we have collected an area where we could possibly land one or more helicopters in support of a mission in the area

• The main objective is to extract some hostages from a building reported adjacent to a red vehicle• We need to determine the area suitable for landing one or more helos• Determine any obstacles (height / width / coordinates)• Route to use by other supporting vehicles in the area (Humvees, APC’s, etc.)

Assignment: • Generate a terrain model of the area• Build an orthomosaic to cover the entire area• Create products identifying obstacles• Routes• Possible observation points for enemy or friendly snipers• Flythrough of the area


Rocco Corsetti

UAV import / triangulation



Frame collection geometry: sensor behind or in front of the lens

9

• In a frame camera, a lens mirrors imagery onto a focal plane

• A rectangular sensor on the focal plane measures pixel intensity/color

• The focal length is the distance between the lens and the sensor

• The relative sizes of the sensor and focal length determine the field of view angles

• The same geometry results from a virtual sensor considered in front of the lens

lens

field ofview angles

focallength

sensor (true)

sensor (virtual)focal

length



35mm Equivalent Focal Length

10

• In this example, the actual physical dimensions are:• Focal length, f = 9mm• pixel size, ps = 2.41mm• An image size of 4864 x 3648 pixels equals a sensor

size of 11.72 x 8.79mm

• EXIF metadata does not include pixel/sensor size• Focal length alone is ambiguous• Ground footprint changes depending on pixel size

• EXIF metadata often provides a “35mm Equivalent” focal length

• Uses a film size of 36 x 24mm• Same aspect ratio• “Equivalent” yields the same viewing angles

f = 9mm

11.72mm8.79mm2.41µm

f = 9mmps=2µm?

f = 9mmps=4µm?

f35mm = 24mm

f = 9mm

8.79mm11.72mm

2.41µm

24mm

36mm

f = 9mm

6.58µm

8.79mm11.72mm

2.41µm

66deg52deg

f35mm = 24mm



Why do Triangulation?

11

• At the time an image is captured, certain information about the sensor may be recorded:

• Position & external orientation (POSEO), focal length, pixel size and other interior orientation (IO)

• In our case, we only have approximations for focal length and pixel size, along with rough position

• No exterior orientation, lens distortion, principal point offset

• Need to calculate/improve image metadata• Images don’t align with each other (relative accuracy)• Images don’t align with ground (absolute accuracy)• Provide/improve predicted errors (error propagation)

Images

Ground

A

C

B

X1,Y1,Z1ω1, ϕ1, κ1

X2,Y2,Z2ω2, ϕ2, κ2


Stephen Foster

Terrain generation-editing / HLZ generation / orthomosaics



Terrain generation – Automatic Terrain Generation (ATG)

13

• Powerful algorithms for Surface and Bare Earth Modelling

• Automatic Terrain Extraction (ATE) and Next Generation Automatic Terrain Extraction (NGATE)

• Automatic Spatial Modeler (ASM)

• 3-D Point Cloud generation

• Takes advantage of latest optimized graphics processing units (GPU) functionality



Terrain editing

14

• Flexible tools to give the analyst total control

• Full suite of terrain visualization options

• Post, Area, and Geomorphic editing capability

• Cue Cards to provide visual assistance



Helicopter Landing Zone generation

15

• Helicopter Landing Zone tool

• Specify safe values for any Rotary Winged aircraft

• Clear and unambiguous results show Go / No-Go areas for HLZs



Orthomosaics

16

• Orthomanager

• Orthorectify• Mosaic

• Seam Line Creation and Editing

• Radiometric Balancing

• Distributed processing


Phill Williams

Stereo, 3-D Feature Extraction, and Automatic Feature Extraction



Stereo building extraction

18

• Stereo image visualisation methods supported by SOCET GXP

• Stereo Multiport – for use with 3-D monitor with NVIDIA 3-D Stereo enabled graphic’s cards or similar

• Anaglyph Stereo – for use with red and blue glasses enabling the user to work in true stereo without needing to use expensive stereo enabled hardware

• Split Screen Stereo – for use when equipment needed by the first two selections is not available



Stereo building extraction …2

19

• 3D Shape (Building tools)• Modifier: Extrude Up and Down, Arbitrary, Wall Fence, and Hole• Flat, Gable, Pyramid, Parapet, Minaret, etc.

• Select Apply (Editing)• Selection tools (Green)• Rubberstamp (Copy)• Rotate• Extrude Up and Down

• Draw (modes)• Static• Arc• Curve

• Active Tool• Generate textures• Union• Subtraction



Automatic Feature Extraction

20

• AFE can extract buildings and trees from high quality terrain data

• Not as accurate as manual digitizing, but saves significant amount of time

• Quality of results highly dependent on quality of terrain data• Designed to work with LiDAR (raw LAS)• Can extract buildings and trees when LiDAR point density

is > 4 points per square meter• Much more difficult to distinguish between trees and

buildings using NGATE


Joey Medina

Data discovery, 3-D visualization, flythrough, measurements on 3D Multiport



Data discovery, 3-D visualization, flythrough, measurements on 3D Multiport …2

22

• Data discovery using GXP Xplorer• Search for existing products • Saved Searches for Area of Interest (AOI)

• Stream to GXP WebView® or direct file open to SOCET GXP v4.4



Data discovery, 3-D visualization, flythrough, measurements on 3D Multiport …3

23

• 3-D visualization• Terrain analysis • Measurements• Flythrough



UAV imagery providers

24

• Ing Civil Geomatico Jaime Orlando Guzman, Chile BAE Systems would like to thank the following organizations for providing data for use throughout the conference:



We value your feedback

25

We improve because of your feedback!

Please take a moment to complete the brief paper survey that was provided to you at the beginning of this session.

If you did not get a survey, see your session monitor in the back of the room or your session instructor.

We appreciate your time and suggestions!


Thank you!

26

Documents

From remote sensing to Unmanned Aircraft Systems (UAS ... · Reference on Drones • When we talk about drones, we are not only referring to big Predator and the likes, but we are