Upload
emory-eaton
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
www.mtri.org
J. Garbarino, C. Roussi, B. Whitewww.mtri.org/unpaved
ALGORITHM/SYSTEM OVERVIEWRITARS-11-H-MTU1
Technical Advisory CommitteeProject Update – March 24, 2015
Overview
Review– Data Collection System
• Airborne platform(s)• Camera, lens, GPS• Intervalometer
– Data Processing System
• Algorithm Suite• Results
Future steps
2
Unmanned Platform
Bergen Folding Hexacopter– 7kg flight-ready– Gyro-stabilized platform
Nikon D800 w/GPS
Custom Intervalometer
3
Manned Platform
Cesna 152 or equivalent ($170/hr)
Nikon D800 w/ 70-200mm f/2.8 lens ($5100)
Intervalometer ($200)
Tyler mini-gyro stabilized mount ($500)
4
System Specifications
5
Hexacopter
Batteries 26Ah, 14.8V
Weight 4kg empty, 5.5kg w/ batteries
Flight Time 20min hover, 10min full power
Range 5km
Ceiling ~3000m
Misc accessories
Charger, flight controller, tools
Cost $5200
D800
Resolution
36Mp (7360x4912)
Weight 1kg (1.3kg w/ lens)
Frame advance
5 fps max
Speed 1/8000s – 30s
ISO 100 - 6400
Cost $2800
Nikkor 50mm Lens
F-stop f/1.4 – f/16
Weight 0.3kg
Field of view 31 deg
Cost $480
Intervalometer (custom)
Interval Range 5s – 1/4s
Weight 200g
Battery 9V alkaline
Cost $200Costs Unmanned Manned
Equipment $8700 $5800
Operating 1 FTE 1 FTE + $170/hr
Nikkor 70mm-200mm
Lens
F-stop f/2.8 – f/22
Weight 1.5kg
Field of view 12-34 deg
Cost $2400
Tyler Mount gyro
Size 25” x 20” x 13”
Weight 27kg w/ batteries
Cost $500
Unmanned Concept of Operations
Typical Collection involves:– Assemble hexacopter and pre-
flight check – 7min– Determine camera settings and
controller setup – 2min– Flight collection – 2min for
100m– Stow equipment – 5min– Charge batteries – 20min
Typical selection and processing – 4 hrs
6
Collect Data
Select Site
Select Data
Process Data
Evaluate in Roadsoft
Manned Concept of Operations
Typical Collection involves:– Emplacing tyler mount
(may involve a modified aircraft w/ port in hull) – 10min
– Determine camera settings and controller setup – 2min
– Flight collection • 4s for 100m• 2-passes needed for
coverage– Stow equipment – 10min– Charge batteries – 20min
Typical selection and processing – 4 hrs
7
Collect Data
Select Site
Select Data
Process Data
Evaluate in Roadsoft
Platform Comparison
8
Hexacopter Pros Cons
Easy to transport and deploy Can only collect 1km of road on a set of batteries
Data can be collected by a single person May require closing road for safety
Operating costs only involve 1 person’s hourly rate
Some ground-truth can be collected manually, if needed
No recurring costs
Manned Aircraft Pros Cons
Can collect more road segments during a flight
Requires a modified aircraft for gyro-stabilized mount
Can operate over a wider range of weather conditions
Involves a pilot and an operator
Requires multiple passes over the same road to get sufficient coverage for 3D reconstruction
Requires more careful selection of images (manually) for processing
Software Architecture
Because we are incorporating legacy code, third-party tools, and custom code, we need a flexible architecture
– Developed in C, C++, Python, bash– Flexible control, with tools calling each other as needed
9
Algorithm
Use Structure from Motion (SIFT+ Bundler + PVMS) to turn 2D images into 3D point-cloud reconstruction
– SIFT = scale-invariant feature transform– PVMS = patch-based multi-view stereo
Form a surface from the 3D point-cloud– Form grid, Fourier Filter, Marching Cubes to triangulate
Find the depth/height map of the surface– Singular Value Decomposition (SVD)– Rotate so z-axis is “up” (depth)
10
Algorithm
Find and select the road in the scene– Image entropy measure (road is “smoother”)
Rotate extracted road into new coordinate system– Makes it easier to take cuts along and across road
Analyze for features of interest– Gabor Filtering, Circular Hough Transform, Cuts for profiles of
road and drainage
Convert to PASER-like metrics
Generate XML output suitable for RoadSoft processing
11
Algorithm
12
Data Collection
Image Q/C
Preprocessing
SIFT
Bundler
Surface from Point Cloud
SVD to Find Depth Map
Distress Extraction
Characterization
Translation to RoadSoft
PMVS Analysis
RoadSoft Processing
Algorithm Details
13
Data Collection
Road Finding (image entropy metric)
Orient road in image
Ruts, Corrugations (Gabor Filtering)
Potholes (Circular Hough Transform)
Edge Cuts for Berm, Loose Agregate
Apply Detection Maps to Depth Map
Compute Statistics on Distresses
Convert to PASER-like ratings
Output to RoadSoft
Transverse Cuts for Crown Loss
Generate XML
Example Image
Taken from 25m, 2m/s
14
Example Reconstruction
15 images used for reconstruction
15
Bundler output Densified point cloud
3D surface from point cloud Height-field from surface
Road Segmentation from Depth Map
16
Rotated Depth Map Mask of Road Surface from image Entropy
Extracted Road
Depth Map Detail
17
Input to Crown Measurement
18Across Road
Alo
ng
R
oad
Example crossection plot
meters
Pothole Detection
19
XML Report
Ruts and Corrugations
20
Performnace Summary
Crown estimates vary from manual ratings slightly– We measure the crown everywhere; manual inspections
sample the surface poorly
Ruts: Pd = 67%, Pfa = 19%
Corrugations: Pd = 100%, Pfa = 38%
Potholes: Pd = 95%, Pfa = 4%
21
Observations
Deep ruts are sometimes labeled as potholes
Strings of potholes along the driving direction are sometimes labeled as corrugations
Strips of grass on the road surface cause false alarms
Manual scoring is a trade-off between accuracy and time– Spot checks– Spend long enough to get a “good enough” estimate
Automated scoring finds everything– Can be both good and bad
22
Room for Improvement - General
Each step in the process can be addressed– Collection parameters– Image quality– Image pre-processing to enhance– Processor operating points– Algorithm choices for current distresses
• Old can be refined• New can be tried
Expanded Applications
23
Room For Improvement - Specific
Automatic rating/rejection of unsuitable images– Blurred images limit reconstruction accuracy
Tuning of algorithms– Each process has “knobs” to adjust performance– Internal operations can be refined
• E.g. changing entropy estimation routine
– Throughput optimization
Script additions to expose more controls– Adding switches makes it more flexible– Set/reset detection points
Adding data exploitation routines (not all information that can be gleaned from the data has been)– Intersection geometry– Texture changes
24
Example
One way of characterizing an intersection is by abstracting its geometry– Leveraging computer vision morphological tools
Sample intersection w/ non-perfect segmentation, followed by medial-axis transformation– Finding the “skeleton” of the intersection– Imaging a brush-fire starting at the boundary; the place where
the fire meets is the media axis
25