Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Structural Inspection Coverage Path Planner
CS686, Paper Presentation
유병호(Yu Byeong-ho)
Andreads, B. et al., “Structural Inspection Path Planning via Iterative Viewpoint Resampling with Application to Aerial Robotics,” ICRA 2015.
Sungwook, J. et al., “Multi-layer Coverage Path Planner for Autonomous Structural Inspection of High-rise Structures,” IROS 2018.
Recap.
Autonomous Robotic Exploration Based on Multiple RRT
- Overall schematic diagram of the exploration algorithm -
RRT-basedFrontier detector
moduleFilter module
Task allocator module
“Detecting frontier
points”
“Clustering the frontier
points”
“Assigning a target
point”
Reference: Hassan Umari, Autonomous robotic exploration based on multiple rapidly-exploring randomized trees (IROS 2017)
Reference: Lukas Janson, Fast Marching Tree: A fast marching sampling-based method for optimal motion planning in many dimensions (IJRR 2015)
Fast Marching Tree
1.Structural Inspection Path Planning via Iterative Viewpoint Resampling with Application to Aerial Robotics
Introduction
In the fields of inspection operations, autonomous complete coverage 3D structural path planning is required.
A robot needs fast algorithms that result in full coverage of the structure while respecting any sensor limitations and motion constraints.
▪ Especially, drones are limited due to its payload.
A novel fast algorithm that provides efficient solutions to the problem of inspection path planning for complex 3D structures is proposed.
6
Problem description
Conventional 3D methods:
▪ Two-step optimization
• Compute the minimal set of viewpoints that cover whole structure (solving Art Gallery Problem (AGP[1])).
• Compute the shortest connecting tour over these viewpoints (Travelling Salesman Problem (TSP[2])).
▪ Large cost of computing efficiency (expensive)
▪ They are prone to be suboptimal due to the two-step separation of the problem.
▪ In specific cases they can lead to unfeasible solutions/paths (e.g. in the case of non-holonomic vehicles)
7
[2] G. Dantzig, R. Fulkerson, and S. Johnson, “Solution of a large- scale traveling-salesman problem,” Journal of
the operations research society of America, vol. 2, no. 4, pp. 393–410, 1954.
[1] J. O’rourke, Art gallery theorems and algorithms. Oxford University Press Oxford, 1987, vol. 57.
Contributions
Not minimizing the number of viewpoints, it samples them such that connecting path is short while ensuring full coverage
A two step optimization paradigm to find good viewpoints that together provide full coverage and a connecting path that has low cost
▪ First: In every iteration, each viewpoints is chosen to reduce the cost-to-travel between itself and its neighbors
▪ Second: the optimally connecting tour is recomputed
8
Methodology: Algorithm with pseudo-code
1. Load the mesh model2. k = 03. Sample Initial Viewpoint Configurations (Viewpoint Sampler)4. Find a Collision-free path for all possible viewpoint
combinations (boundary value solver (BVS), RRT*)5. Compute the Cost Matrix and Solve the Traveling Salesman
Problem (Lin-Kernighan-Helsgaun Heuristic (LKH))6. While running
1. Re-sample Viewpoint Configurations (Viewpoint Sampler)
2. Re-compute the Collision-free paths (BVS, RRT*)
3. Re-populate the Cost Matrix and solve the new Traveling Salesman Problem to update the current best inspection tour (LKH)
4. k = k + 17. end while8. Return BestTour, CostBestTour
9
Methodology: Path computation and Cost estimation
To find the best tour among viewpoints, TSP solver requires a cost matrix of all pairs of viewpointsPath generation and cost estimation is done by either
▪ BVS - directly connect the two viewpoints
▪ BVS+RRT* - due to obstacles, connection is not feasible
The cost of a path segment corresponds to the execution time 𝒕𝒆𝒙» 𝒕𝒆𝒙 = 𝐦𝐚𝐱 Τ𝒅 𝒗𝒎𝒂𝒙 , Τ𝝍𝟏 − 𝝍𝟎
ሶ𝝍𝒎𝒂𝒙
10
Where 𝒅 is the Euclidean distance, translation speed limit is 𝒗𝒎𝒂𝒙,
rotational speed limit is ሶ𝝍𝒎𝒂𝒙, and 𝝍 is yaw angle, respectively
Methodology: Viewpoint sampling(1)
For every triangle in the mesh, one viewpoint has to be sampled, the position and heading is determined while retaining visibility of the corresponding triangle.
First, the position is optimized for distance to the neighboring viewpoints using a convex problem formulation and then heading is optimized.
To guarantee a good result, the position solution must be constrainedsuch as to allow finding an orientation for which the triangle is visible.
11
Methodology: Viewpoint sampling(2)
The constraints on the position 𝒈 =[𝒙, 𝒚, 𝒛] consist of the inspection sensor limitation of minimum incidence angle, minimum and maximum range (𝒅𝒎𝒊𝒏, 𝒅𝒎𝒂𝒙) constraints.
▪
▪ Where 𝒙𝒊 are the corner of the mesh triangle, 𝒂𝑵 is the normalized triangle normal and 𝒏𝒊are the normal of the separating hyperplanes for incidence angle constraints, respectively.
12
Three main planar angle of incidence constraints on all threesides of the triangle. For a finitenumber of such constraints theincidence angle is only enforcedapproximately. The red line (and 𝒏+) demarks asample orientation for a possibleadditional planar constraint at acorner. Minimum (green plane) and maximum (red plane) distanceconstraints are similar planarconstraints on the sampling area. These constraints bound thesampling space, where g can bechosen, on all sides (gray area).
Incidence angle constraints on a triangular facet
Methodology: Viewpoint sampling(3)
To account for the limited FoV with fixed pitch angle of camera, it imposes a revoluted 2D-cone constraint which is nonconvex problem and then convexified by dividing the space into 𝑵𝑪 equal convex pieces.
The optimum is computed for every slice.
13
• where 𝒙𝒍𝒐𝒘𝒆𝒓𝒓𝒆𝒍 , 𝒙𝒖𝒑𝒑𝒆𝒓
𝒓𝒆𝒍 are the respective relevant corners of the
mesh triangle, m the middle of the triangle and 𝒏𝒍𝒐𝒘𝒆𝒓𝒄𝒂𝒎 , 𝒏𝒖𝒑𝒑𝒆𝒓
𝒄𝒂𝒎 ,
𝒏𝒓𝒊𝒈𝒉𝒕 and 𝒏𝒍𝒆𝒇𝒕 denote the normal of the respective separating
hyperplanes.
Methodology: Viewpoint sampling(4)
Optimization objective is to minimize the sum of squared distances to the
preceding viewpoint 𝒈𝒑𝒌−𝟏, the
subsequent viewpoint 𝒈𝒔𝒌−𝟏 and the
current viewpoint in the old tour 𝒈𝒌−𝟏.
The heading is determined according to
14
Computational Analysis
To evaluate the capabilities, a simple scenario is used.
15
The time complexity:LKH: 𝑶(𝑵𝟐.𝟐)VP Sampling: 𝑶(𝑵)Distance compute.: 𝑶(𝑵𝟐)
Evaluation Test - Simulation
405m Tower
Large scale structure to be inspected: The 405m high Central Radio & TV Tower in Beijing. The mesh used to compute the path contains 1701 triangular facets. After a computation time of 92s the cost for the inspection is 2997.44s
The red point denotes, start– and end–point of the inspection.
Evaluation Test – VTOL UAV
Online: image processingOffline: 3D reconstruction (Image->Pix4D)
Path plannerCost for the inspection: 151.44s
Visual-Inertial Sensor, ATOM CPU (Linux)
Summary & Conclusions
A practically–oriented fast inspection path planning algorithm capable of computing efficient solutions for complex 3Dstructures represented by triangular meshes was presented.
With the help of 3D reconstruction software, the recorded inspection data were post-processed to support the claim of finding full coverage paths and the point cloud datasets are released to enable evaluation of the inspection quality.
https://github.com/ethz-asl/StructuralInspectionPlanner
2.Multi-layer Coverage Path Planner for Autonomous Structural Inspection of High-rise Structures
Intro.
Structural inspection and maintenance of large structure is becoming significantly important.
Using UAV, it is faster, safer, cheaper !
20
Lotte World Tower, Seoul Central TV Tower, Beijing Oriental Pearl Tower, Shanghai Eiffel Tower, Paris
Intro.
21
Sensor limitations
Operational restrictions
Completeness of coverage
Efficient and tidy path
e.g. payload e.g. f light time, weather condition
Contribution
22
i layer
i+1 layer
n layer
n-1 layer
Spiral path
Computational complexity
Efficient and tidy
𝒪(𝑁2.2) 𝒪 𝑛12.2 +⋯+ 𝒪 𝑛𝐾
2.2
𝑁 = 𝑛1 + 𝑛2 +⋯+ 𝑛𝐾
K: # of layern: # of viewpoint
in each layer
Methodology
23
Coverage Path
PlanningAutonomous flying Localization On-line inspection
Prior map generation• Manual process
• 3D volumetric map
• Ground elimination
Viewpoint generation• Layer separation
• Down-sampling
Path Planning• TSP Solver
• Layer connecting
Viewpoint resampling• Check duplication
• Viewpoint update
Coverage
Completeness
Evaluation
Methodology
Multi-Layer Coverage Path Planner
Input: DistToStruct, FOV, VoxelSize, StartPoint, NumOfLayers
1: Generating voxelized 3D map
2: Calculate a surface normal vector (𝑛1, 𝑛2⋯ , 𝑛𝑁) of every center point (𝐶1~𝑘)
3: Divide the structure with K layers by height
4: while i < K do
5: Sample initial viewpoint (𝑣1, 𝑣2, … , 𝑣𝑁) at i-th layer6: Down-sample essential viewpoints ( ො𝑣1, ො𝑣2, … , ො𝑣𝑁)7: Solve TSP problem with LKH solver at i-th layer8: Update VPs in (i+1)-th layer by checking duplication 9: Connect i layer and i+1 layer10: i ← i+1
Output: TourLength, CalcTime
24
Methodology
Multi-Layer Coverage Path Planner
Input: DistToStruct, FOV, VoxelSize, StartPoint, NumOfLayers
1: Generating voxelized 3D map
2: Calculate a surface normal vector (𝑛1, 𝑛2⋯ , 𝑛𝑁) of every center point (𝐶1~𝑘)
3: Divide the structure with K layers by height
4: while i < K do
5: Sample initial viewpoint (𝑣1, 𝑣2, … , 𝑣𝑁) at i-th layer6: Down-sample essential viewpoints ( ො𝑣1, ො𝑣2, … , ො𝑣𝑁)7: Solve TSP problem with LKH solver at i-th layer8: Update VPs in (i+1)-th layer by checking duplication 9: Connect i layer and i+1 layer10: i ← i+1
Output: TourLength, CalcTime
25
Methodology
Multi-Layer Coverage Path Planner
Input: DistToStruct, FOV, VoxelSize, StartPoint, NumOfLayers
1: Generating voxelized 3D map
2: Calculate a surface normal vector (𝑛1, 𝑛2⋯ , 𝑛𝑁) of every center point (𝐶1~𝑘)
3: Divide the structure with K layers by height
4: while i < K do
5: Sample initial viewpoint (𝑣1, 𝑣2, … , 𝑣𝑁) at i-th layer6: Down-sample essential viewpoints ( ො𝑣1, ො𝑣2, … , ො𝑣𝑁)7: Solve TSP problem with LKH solver at i-th layer8: Update VPs in (i+1)-th layer by checking duplication 9: Connect i layer and i+1 layer10: i ← i+1
Output: TourLength, CalcTime
26
Methodology
Multi-Layer Coverage Path Planner
Input: DistToStruct, FOV, VoxelSize, StartPoint, NumOfLayers
1: Generating voxelized 3D map
2: Calculate a surface normal vector (𝑛1, 𝑛2⋯ , 𝑛𝑁) of every center point (𝐶1~𝑘)
3: Divide the structure with K layers by height
4: while i < K do
5: Sample initial viewpoint (𝑣1, 𝑣2, … , 𝑣𝑁) at i-th layer6: Down-sample essential viewpoints ( ො𝑣1, ො𝑣2, … , ො𝑣𝑁)7: Solve TSP problem with LKH solver at i-th layer8: Update VPs in (i+1)-th layer by checking duplication 9: Connect i-th layer and (i+1)-th layer10: i ← i+1
Output: TourLength, CalcTime
27
Experiment
Experiment
No layer▪ Initial viewpoints : 3441
▪ Selected viewpoints : 83
▪ Computation time
• Down sampling: 435.206 seconds
• TSP: 0.840438 seconds
• VP update: -
• Total: 436.0464 seconds
▪ Total distance: 3505.15m
▪ Missed voxel: 311/19935
Experiment
5-layers (every 20m)▪ Initial viewpoints: 2885
▪ Selected viewpoints: 95
▪ Computation time
• Down sampling: 79.65832 seconds
• TSP: 1.069598 seconds
• VP update: 4.358884 seconds
• Total: 85.0868 seconds
▪ Total distance: 1943.623m
▪ Missed voxel: 156/19935
Experiment
12-layers (every 8m)▪ Initial viewpoints : 1642
▪ Selected viewpoints : 102
▪ Computation time
• Down sampling: 15.20832 seconds
• TSP: 0.07194 seconds
• VP update: 5.385357 seconds
• Total: 20.66561 seconds
▪ Total distance: 2165.702m
▪ Missed voxel: 36/19935
Results
A. Bircher et al. [1] No-Layer 5-Layers 12-Layers
Results
Total Comparison
[1] Bircher, Andreas, et al. "Structural inspection path planning via iterative viewpoint resampling with application to aerial robotics," IEEE International Conference on Robotics and Automation (ICRA), pp. 6423-6430, Seattle, USA, 2015.
Appendix.1
Art Gallery Problem (AGP)
Travelling Salesman Problem (TSP)
Suppose you have an art gallery containing priceless paintings and sculptures. You would like it to be supervised by security guards, and you want to employ enough of them so that at any one time the guards can between them oversee the whole gallery. How many guards will you need?
Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?
Appendix.2
Boundary Value Problem (or Solution)
▪ A boundary value problem has conditions specified at the extremes ("boundaries") of the independent variable in the equation
▪ whereas an initial value problem has all of the conditions specified at the same value of the independent variable (and that value is at the lower boundary of the domain, thus the term "initial" value).
Appendix.3
Lin-Kernighan-Helsgaun heuristic (LKH, K. Helsgaun, 1998)
▪ LKH is an effective implementation of the Lin-Kernighan heuristic for solving the traveling salesman problem.
▪ Even though the algorithm is approximate, optimal solutions are produced with an impressively high frequency.
▪ Employ the concept of k-opt moves
Visualization of the k-moves process