Common Approach for UAS Data Geoprocessing

®

Common Approach for

UAS Data Geoprocessing

UAV-g 2015 August 2015

George Percivall

Chief Engineer and CTO [email protected]

Copyright © 2015 Open Geospatial Consortium

OGC ®

Common Approach for UAS Data Geoprocessing

• UAVs bring new geographic information to many application domains

• UASs similar to other geographic imagery systems so existing frameworks are applicable

• But challenges unique to UASs exist in processing and creation of geospatial products

• Challenges motivate use of existing standards and extending standards


OGC ®

UAS frameworks similar to previous geographic observing system frameworks


!

UAS Production Process (USGS)

Figure 1. The frameworks of application from UAVs

II. PROJECT WORKFLOW OF IMAGE DATA ACQUISITION AND PROCESSING

A. Data Acquisition

a Scheme of the main navigation sensors and camera

b Flight planning example

Figure 2. Data acquisition equipment and planning

In this thesis, UAVs were described as a photogrammetric data acquisition platform for further applications. The structure of UAV is showed in fig 2 a, including UAV, camera, GPS/INS. Because of the limit of playload, the quality of imageries is impacted seriously by the posture of the UAV without a self-stabilization platform. And the imageries always have a huge distortion compared to manned aircraft, due to the roll, pitch, and heading of UAV. Here, an overview of the

overall workflow for the acquisition of UAV images will be given. The main data files are defined as follows:

• Camera calibration files: measured automatically, focal length c, the coordinates of the principal point xP, yP and the radial lens distortion parameters were estimated within a bundle adjustment. Further parameters of lens distortion were not significant, therefore they were neglected [2].

• Imageries acquired with UAVs. The imageries with consumer camera always had large image scale, irregular overlap, strong geometric, and radiometric changes.

• Position Orientation System (POS) data: with the Position Orientation System, a GPS coordinate and elevation are recorded for each image, yet the roll, pitch, and heading of the platform. However, even with the autopilot system, the plane was affected by wind and thermals, which are common in the low-altitude environment [3]. The roll, pitch, and heading of the platform were useless for imageries processing, and only the coordinate was used to depict the aerial strips.

• Ground Control points: acquired by Real-time kinematic (RTK), including the coordinates, recording of the point and the image with position of point.

B. Image Processing for Photogrammetry Here we focus on the UAV image orientation and

photogrammetric products like DSM, orthoimages. Subsequent to a brief description of the “UAV Block Triangulation”, we will focus on the DSM generation and on an alternative or supplementary to the generated DSM using only image information.

For the orientation of the images software packages, like LPS (Leica Photogrammetry Suite 8.9-9.2), the embedded ORIMA (Orientation Management Software, ORIMA-LPS-M Rel. 9.10), ISDM (Z/I Imaging, Image Station Digital Mensuration Version 04.04.17.00) originally developed for the processing of aerial images, and Photomodeler 6. In addition, the INPHO model is generated in Germany, and has a particular advantage for low-altitude imageries in automatically triangulation and terrain extraction. DPGrid model developed by Wuhan University is also a professional software package

Image matching and mosaicking

Flight planning Autonomous flight Flight trajectory

Data processing

Environment and agriculture

Terrain extraction

3D visualization

Monitoring of hazards

Data acquisition

Photography Classification

Applications

UAVs

Framework for research on UAVs (Ma)

OGC ®

Challenges with UAS technology

•  Image distortion with inexpensive digital cameras •  Sensors have low or no metadata which hinders use of

sensor data •  Limited accuracy of the exterior information: position,

orientation •  Need for smooth, fast workflow: raw to classified imagery

•  Requirements for accountability increase requirements on provenance in data processing


OGC ®

Challenges motivate the use of standards

•  Diversity of alternatives in UAVs show a lack of standardization at all levels: sensors, platforms, processing

•  To advance, UASs need to increase use of existing standards and in some cases new standards will need to be developed.

•  Standards for geographic observations are quite mature and UASs benefit from using them


OGC ®

What is a Standard?

•  “An agreed way of doing something”

© 2015 Open Geospatial Consortium

EC: Practical standards guide for researchers - en

OGC ®

What is a Standard?

•  “An agreed way of doing something”

• Standards are distilled wisdom of people with expertise in their subject matter and who know the needs of the organizations they represent – people such as manufacturers, sellers, buyers, customers, trade associations, users or regulators.

• Standards are knowledge. They are powerful tools that can help drive innovation and increase productivity. They can make organizations more successful and people’s everyday lives easier, safer and healthier.

© 2015 Open Geospatial Consortium

EC: Practical standards guide for researchers - en

OGC ®

What is an OGC Standard?

•  Document; Established by consensus; Approved by OGC membership (balance of interest, all members have vote) – Provides, rules, guidelines or characteristics –  Implementable in software

•  Open Standards are not the same as Open Source software – OGC/OSGeo Paper on Open Source and Open Standards:

http://wiki.osgeo.org/wiki/Open_Source_and_Open_Standards

•  OGC standards are Open Standards – Freely and publicly available – No license fees – Vendor neutral

Copyright © 2015 Open Geospatial Consortium 8

OGC ®

OGC Sensor Web Enablement


Webcam

Environmental Monitor

Airborne Imaging Device

Health Monitor

Vehicles As Sensor Probe

Satellite-borne Imaging Device

• Sensors connected to and discoverable on the Web • Sensors have posi9on & generate observa9ons • Sensor descrip9ons available • Services to task and access sensors •  Local, regional, na9onal scalability • Enabling the Enterprise

OGC ®

OGC Sensor Web Enablement Standards Discover and Task Sensors

Access and process Observations

•  Sensor Model Language (SensorML)

•  Observations & Measurements (O&M)

•  Sensor Planning Service (SPS)

•  Sensor Observation Service (SOS)

•  Catalogue Service •  Sensor Alert Service (SAS) •  PUCK


OGC ®

Visualization / Decision Tools and Applications GeoAPI

OpenLS

SLD

SE

Data Models and Encodings WMC

FE

GML

GeoXACML

KML

CityGML OpenGeoSMS

IndoorGML GeoSparql

WaterML GeoPackage

NetCDF GMLJP2

OGC Services Architecture

Other Data

Processing Services OpenMI WPS TJS WCPS

Geospatially Enabled Metadata

Discovery Services

CSW OpenSearch Geo ebRIM

WMS

WMTS WFS

Simple Features Access

Access Services

Geospatial Feature Data

Geospatial Browse/Maps

Geospatial Coverage Data

WCS

Other Services Workflow, Alerts,

Security

Sensors

Puck

SOS SPS O&M SensorML

Sensor Web Enablement

Discover Task Access


OGC ®

Copyright © 2014 Open Geospatial Consortium 12

Empire Challenge OV-1

OGC ®

Tigershark UAV in Empire Challenge 2008

•  On-demand geolocation and display of HD video from Tigershark UAV

•  Client: – UAH Space Time Toolkit

•  Services: – SOS – Tigershark video and

navigation (ERDAS) – SOS – Troop Movement

(Northrop Grumman) – SensorML – On-demand

processing (Botts, Inc.) – Virtual Earth – base maps

OGC ®

Tigershark UAV-HD with SWE processing

OpenGL

SensorML-enabled Client

SLD

Tigershark SOS

JP2

NAV

Tigershark SOS offerings served in O&M:

(1) time-tagged video frames (in JP2)

(2) aircraft navigation (lat, lon, alt, pitch, roll, true heading)

SensorML process chain (using CSM frame sensor model) geolocates

streaming imagery on-the-fly Source: Mike Botts

OGC ®

NASA and US Forest Service UAS missions

•  Ikhana UAV with multispectral sensor •  Fire intelligence to management teams •  Web access to geospatial processing services


Source: Ambrosia, G., Sullivan, D., Buechel, S., GSA Special Paper 482

OGC ®

Framework for UAS using OGC SWE

•  UAV challenges –  sensors publish data in

unpredictable manner. –  proprietary access to data

•  Need to integrated data stream web publishing

•  Framework to simplify integration in an interoperable way using OGC SWE standards


Source: Rieke, M., Foerster, T., Broering, A. 14th AGILE International Conference

OGC ®

Framework to combine UAS with other sensors

•  Precision farming: variety of vendor-specific sensor systems, control units and processing software

•  SWE-based infrastructure: control, access, transmission and storage of of sensor data for web services

•  Field trial proved applicability of the infrastructure.


SWE infrastructure for precision farming (Source: Geipel)

OGC ®

Using SensorML to manage UAS complexity

•  Manage proliferation of sensors on UAV platforms – Mission planning: after the most appropriate UAV is determined, it is

time to choose which kind of sensor will be accompany to the UAV.

•  Using SensorML to manage specifications – Platforms: helicopter, quadcopter, blimp and airplane – Sensors: micro analog, HD camera, lowlight and thermal camera –  In a database to support processing, e.g., MATLAB, BPEL


28

Figure 23: System 1 and 2- Cameras with Quad copter and Cameras with Blimp

For instance, in a mission that happens at night and if it takes a lot of time and it must be

fast, the UAV will probably be an aero plane with these four UAV and the camera must be

thermal or low light camera from five camera sensors.

Figure 24: System 3 and System 4-Cameras with aero plane and cameras with helicopter

Datasheets Including UAV and the general characteristics of camera sensors are used to

determine this situation. The Information Inside datasheets transform Into databases In XML

form by using SensorML. Basically, every UAV has five camera sensors and this makes these

databases hierarchic databases stored In XML form by using SensorML.

In the second part of the thesis, an interrogation system has been designed with the aim of

using the databases with the description of the sensors and the UAVs. The main concept of

Source: C. Avci,, Halmstad University

OGC ®

OGC Point Cloud Working Group

•  Established in July 2015

•  Focus on all types of point clouds: LiDAR/laser, bathymetric, meteorologic, photogrammetric…

OGC ®

Common Approach for UAV Data Geoprocessing

•  Open standards provide alternatives to “stove-pipe” vertical integration of data collection, database management, analysis, portrayal and user interface.

•  Pick and choose components that work well together because of open standards – “plug and play”

•  Efficient processing and dissemination of the data achieved using software and systems that implement open standards

•  Gain full benefit of the explosion of UAV platforms and sensors that will be interchangeable based on open standards


OGC ®

For Details on OGC Standards…

OGC Standards – Freely available –  www.opengeospatial.org/standards

George Percivall gpercivall at opengeospatial.org @percivall

21

Engineering

Common Approach for UAS Data Geoprocessing