INPE Course 2013 Mitishitar Integration

8/12/2019 INPE Course 2013 Mitishitar Integration

1/103

PHOTOGRAMMETRIC AND LIDAR DATA

GRADUATE PROGRAM IN GEODETICSCIENCES

Professor: Edson A. Mitishtia

13/04/2013 INPE_Course_2013 1


2/103


Combining multiple datasets acquired by different

sensors in order to get better accuracy and enhancedinference about the environment than could bea a ne roug e use o a s ng e sensor

13/04/2013 INPE_Course_2013 2


3/103


Why to integrate Photogrammetric Image

13/04/2013 INPE_Course_2013 3


4/103


LIDAR Pros Photogrammetric Cons

Dense information fromhomogeneous surfaces

Almost no positional informationalong homogeneous surfaces

Direct acquisition of 3D coordinates Complicated and sometimesunreliable matching procedures

Vertical accuracy is better than its

planimetric accuracy

Vertical accuracy is worse than the

planimetric accuracy

13/04/2013 INPE_Course_2013 4


5/103


High redundancy No inherent redundancy

Rich in semantic information Positional; difficult to derivesemantic information

object space breaklines

breaklines

Planimetric accuracy is better than Planimetric accuracy is worse thanthe vertical accuracy the vertical accuracy

13/04/2013 INPE_Course_2013 5


6/103


Photogrammetry is the art and science of derivingaccurate 3D metric and descriptive object information

from digital images

n er or r en a on arame ers

Exterior Orientation Parameters EOP

13/04/2013 INPE_Course_2013 6


7/103


Coordinates of the principal pointRadial and descentering parameters

Calibration Proceduren epen en a ra on

In Situ Self-CalibrationMount arameters calibration

13/04/2013 INPE_Course_2013 7


8/103


Position and Orientation parameters of the imagerelated with Geodetic Frame

Direct and Indirect Procedure

13/04/2013 INPE_Course_2013 8


9/103


Direct Georeferencing

A block of imagery georeferenced using GPS/INSs stems

13/04/2013 INPE_Course_2013 9


10/103


Direct Georeferencing (Kersting, 2011)

13/04/2013 INPE_Course_2013 10


11/103


Direct Georeferencing (Hutton et al, 2005)

In order to accurately compute the ground coordinates of a point using

Direct Georeferencing, a number of requirements need to be met :1. The physical misalignments of IMU with respect to the Camera need to becalibrated;

. e ever-arm o se s rom e camera perspec ve cen er o e anto the GPS antenna need to be calibrated;3. The exact time of image exposure needs to be recorded by the GPS-AidedINS;4. The position and orientation from the GPS-Aided INS navigation data

5. The camera interior geometry (principal point, lens distortion, focallength) must be well calibrated and stable.

13/04/2013 INPE_Course_2013 11


12/103


Indirect Georeferencing

A block of imagery georeferenced using ground controloints Bundle Ad ustment

13/04/2013 INPE_Course_2013 12


13/103


LIDAR Point Computaion (Kersting, 2011)

13/04/2013 INPE_Course_2013 13


14/103


LIDAR and Photogrammetry Registration

Using LIDAR data as the source of control for the

allows for establishing a common reference frame for multi-temporal and multi-source photogrammetric datasets

13/04/2013 INPE_Course_2013 14


15/103


eg s ra on e o o ogy

rs , a ec s on as o e ma e regar ng e c o ce oprimitives for the registration procedure

The second issue is concerned with a registrationrans orma on unc on a ma ema ca y re a es e

datasets under consideration

13/04/2013 INPE_Course_2013 15


16/103


REGISTRATION PRIMITIVES

Registration problems involving spatial data, the threefundamental, and commonly, used registration primitivesare po n s, nes an area reg ons

Potential features include road intersections, corners ofbuilding, rivers, coastlines, roads, lakes, or similar

dominant man-made or natural structures

13/04/2013 INPE_Course_2013 16


17/103


Registration Primitives

Distinct Points Linear FeaturesAreal Feature

13/04/2013 INPE_Course_2013 17


18/103


Mathematical Model Collinearity Equation

Collinearity equations is based on the fact that image point, object point,

and the perspective center are collinear

The image coordinates of a point areex ressed as a function of the Interior

Orientation Parameters (IOP), theExterior Orientation Parameters,

the corresponding object point

Collinearity condition of the rays

13/04/2013 INPE_Course_2013 18


19/103


Mathematical Model Collinearity Equation

x, y : Image point coordinates corresponding to object point (X, Y, Z)X, Y, Z : Corresponding ground point coordinates

Xo, Yo, Zo, , , : Exterior orientation parameters: Xo, Yo, and Zo represent theposition of perspective center with respect to ground coordinate system, where ,

systemsr11 ... r33 : The rotation matrix between the image and ground coordinates systems

13/04/2013 INPE_Course_2013 19


20/103


Distinct Points

Aerial Image Intensity Image Range Image

13/04/2013 INPE_Course_2013 20


21/103


Distinct Points

13/04/2013 INPE_Course_2013 21


22/103


Distinct Points

Point get by intersection of three planes

13/04/2013 INPE_Course_2013 22


23/103


on os ar nos p anos o e a o

13/04/2013 INPE_Course_2013 23


24/103


Distinct Points Delaunay Triangulation and Normal Vector

Planes Detection

13/04/2013 INPE_Course_2013 24


25/103


Distinct Points Delaunay Triangulation and Normal Vector

13/04/2013 INPE_Course_2013 25


26/103


13/04/2013 INPE_Course_2013 26


27/103


13/04/2013 INPE_Course_2013 27


28/103


0=+++ dcZbYaX

13/04/2013 INPE_Course_2013 28


29/103


Interseo dos trs planos Concorrentes

=ca

01111 dXcba

0. 2222 =+ dYcba p

03333 dZcba p

13/04/2013 INPE_Course_2013 29


30/103


The centroid of regular building roof is equivalent a single control point

with 3D coordinates allowin its use in traditional hoto rammetricsystems

Re ular buildin roof

13/04/2013 INPE_Course_2013 30


31/103


LIDAR Centroid

roof in the ground space has five steps

First: Mathematic procedure selects the raw laser scanning pointsaround the building

Raw LIDAR points close to the building

13/04/2013 INPE_Course_2013 31


32/103


33/103


LIDAR Centroid

Third: Mathematic procedure selects selects interpolated LIDARpoints that lie only on the roof

Interpolated LIDAR points on the roof

13/04/2013 INPE_Course_2013 33


34/103


LIDAR Centroid

Fourth: Planimetric centroid coordinates of regular building roof inthe LIDAR coordinates are determined

n : Number of interpolated LIDAR points that lie on the roof

13/04/2013 INPE_Course_2013 34


35/103


LIDAR Centroid

Fifth: Z centroid of regular building roof in the LIDAR coordinatesis determined

e centro coor nate w e eterm ne us ng t e two pro es,with almost equal Z coordinates, closer to the borders of the roof

m : Number of interpolated

profileProfiles close to the building border

13/04/2013 INPE_Course_2013 35


36/103


Image Space centroid

The 2D centroid coordinates (xc,yc), in the space image will be

determined by calculating the 2D mean coordinates of these fourpoints, considering omega and phi close to zero.

21 axay cc +=

21 bxby

cc +=

Image points used to determine 2D centroidcoordinates in the image system

13/04/2013 INPE_Course_2013 36


37/103


LIDAR Linear Feature

Intensity Image Range ImageAerial Image

13/04/2013 INPE_Course_2013 37


38/103



Line get by intersection of two planes

13/04/2013 INPE_Course_2013 38


39/103



Plane fitting and blunder detection from LIDAR patches(a) and plane intersection for extracting LIDAR lines (b)

13/04/2013 INPE_Course_2013 39

O OG A C A A A A


40/103


Intermediate Points

13/04/2013 INPE_Course_2013 40


41/103



42/103


Mathematical Model Coplanarity Constraint

Perspective transformation between image and LIDAR control straightlines and the coplanarity constraint for intermediate points along the line

13/04/2013 INPE_Course_2013 42



43/103


Mathematical Model Co lanarit Constraint

13/04/2013 INPE_Course_2013 43



44/103


Photogrammetric Patches

by three 2D points in the image space while in the object space three 3D

points will be usedLIDAR patches are represented by the set of 3D points that comprise thepatch under consideration in its raw format as collected by the scanner

Photogrammetric planar patch

, , .LIDAR patch points are also shownon the roof

13/04/2013 INPE_Course_2013 44



45/103


Mathematical Model

,namely the photogrammetric SPH= {A, B, C} set and the LIDAR SL=

{(XP, YP, ZP), P=1 to n}Since the LIDAR points are randomly distributed, no point-to-pointcorrespondences can be assumed between the datasets; nevertheless, all

13/04/2013 INPE_Course_2013 45


46/103


47/103

TRUE ORTHOPHOTO GENERATION


48/103


13/04/2013 INPE_Course_2013 48



49/103


Problema Principal:

(Edificaes, rvores, Pontes, Postes, etc);

Existncia de sobras e ocluses Estado da Arte da Gerao de Ortoimagens:

Automa o da reconstru o dos modelosrepresentativos do terreno e das edificaes

Levantamentos LIDAR: Maior exatido e maior densidade dos pontos (DSM)

Indefinio das bordas das edificaes

Obteno da correta geometria das edificaes(DBM)

13/04/2013 INPE_Course_2013 49

MAPEAMENTO INVERSO


50/103

MAPEAMENTO INVERSO

13/04/2013 INPE_Course_2013 50

MAPEAMENTO INVERSO


51/103

MAPEAMENTO INVERSO

13/04/2013 INPE_Course_2013 51

MAPEAMENTO DIRETO


52/103

MAPEAMENTO DIRETO

13/04/2013 INPE_Course_2013 52

MAPEAMENTO DIRETO


53/103

13/04/2013 INPE_Course_2013 53

IMAGEM AREA PROJEO CENTRAL


54/103

13/04/2013 INPE_Course_2013 54

ORTOIMAGEM DTM


55/103

13/04/2013 INPE_Course_2013 55

ORTOIMAGEM DTM Exatido Planimtrica


56/103

Pto DX DY DP21 -7,44 1,63 7,6221 -7 44 1 63 7 62

Pto DX DY DP34 -9,84 1,85 10,01

22 -7,19 0,91 7,2523 -9,17 1,81 9,35

- , , ,36 -7,54 1,75 7,7437 -8,82 1,74 8,99

- , , ,25 -6,52 1,77 6,7526 -9,07 2,37 9,38

38 -8,76 1,55 8,9039 -7,97 1,54 8,1240 -6,70 1,79 6,93

27 -8,87 2,24 9,15

28 -7,36 2,32 7,7229 -6,76 1,96 7,04

41 -9,36 2,91 9,80

42 -9,20 2,63 9,57-

30 -6,68 1,80 6,91

31 -8,34 2,66 8,75

-

44 -7,44 1,63 7,62

, , ,33 -9,30 2,02 9,51 Mdia -8,14 1,93 8,38

Desvio-Padro 1,04 0,45 1,05

13/04/2013 INPE_Course_2013 56

ORTOIMAGEM DSM Duplo Mapeamento


57/103

13/04/2013 INPE_Course_2013 57

TRUE ORTHOPHOTO DSM


58/103

13/04/2013 INPE_Course_2013 58

TRUE ORTHOPHOTO DSM


59/103

Pto DX DY DP21 -0,02 0,42 0,42

Pto DX DY DP34 -0,63 0,20 0,66

- , , ,

23 -0,39 0,11 0,41

24 0,05 0,97 0,97

35 -0,07 0,63 0,6336 0,07 0,02 0,07

37 0 16 -0 07 0 1825 -0,42 0,03 0,4226 -0,31 0,30 0,4327 0 08 0 43 0 43

38 0,35 -0,47 0,59

39 0,15 -0,32 0,35

28 -0,48 0,32 0,5829 -0,04 -0,02 0,04

- , - , ,

41 0,21 -0,15 0,2642 -0,56 0,23 0,60, , ,

31 -0,07 -0,20 0,2132 -1,41 0,37 1,45

43 -0,05 -0,53 0,5344 0,35 -0,14 0,38

33 -0,03 -0,41 0,42 Mdia -0,13 0,12 0,48

Desvio-Padro 0,38 0,40 0,32

13/04/2013 INPE_Course_2013 59

ORTHOPHOTO Urban Area Using DTM


60/103

13/04/2013 INPE_Course_2013 60

ORTHOPHOTO Urban Area Using DSM


61/103

13/04/2013 INPE_Course_2013 61

Urban Area Occluded areas


62/103

13/04/2013 INPE_Course_2013 62



63/103


13/04/2013 INPE_Course_2013 63



64/103


13/04/2013 INPE_Course_2013 64

Occluded areas Z BUFFER METHOD


65/103

13/04/2013 INPE_Course_2013 65

Occluded areas ANGLE-BASED METHOD


66/103

13/04/2013 INPE_Course_2013 66


67/103

cc u e areas -

Varredura radial adaptativa Varredura em espiral

13/04/2013 INPE_Course_2013 67


68/103

-

13/04/2013 INPE_Course_2013 68


69/103

13/04/2013 INPE_Course_2013 69


70/103

13/04/2013 INPE_Course_2013 70


71/103

13/04/2013 INPE_Course_2013 71


72/103


73/103

13/04/2013 INPE_Course_2013 73


74/103

E UA O DE COLINEARIDADE


75/103

INVERSA

X X Z Z m x m y m c

o o= ++ +

( ) 11 21 31

13 23 33

m x m y m c+ +12 22 32

m x m y m c

=

+ +13 23 33

X f c X Y Z x y Zo o o= ( , , , , , , ), , ,

Y c X Y Z x y Z o o o= , , , , , ,, , ,

13/04/2013 INPE_Course_2013 75

MONOPLOTTING


76/103

13/04/2013 INPE_Course_2013 76

MONORRESTITUIO DE EDIFICAES


77/103

Monorrestituio sem Iteraes Integrao com dados Laser

m X X m Y Y m Z Zo o o=

+ + 11 12 13( ) ( ) ( )

m X X m Y Y m Z Zo o o + + 31 32 33( ) ( ) ( )

y cm X X m Y Y m Z Z

X X Y Y Z Z

o o o

o o o=

+ +

+ +

21 22 23( ) ( ) ( )

Ponto Laser (X,Y,Z) Transformao - (x,y,Z)

13/04/2013 INPE_Course_2013 77

MONORRESTITUIO DE EDIFICAES


78/103

Monorrestituio sem Iteraes Integrao com dados Laser

13/04/2013 INPE_Course_2013 78

MONORRESTITUI O DE EDIFICA ES


79/103

INTERPOLAO

Equao de colinearidade: coordenadas laser

, ,fotogramtricas (xp,yp)

]),,[( ,1 niiii Zyx pp =

n = Nmero de pontos laser

13/04/2013 INPE_Course_2013 79



80/103

Digitalizao Vetorial Dados 2D

13/04/2013 INPE_Course_2013 80



81/103

Representao Vetorial contornos das edificaes

13/04/2013 INPE_Course_2013 81



82/103

Representao Vetorial - Exatido

Area 1 Monorrestituio versus bundle adjustment

Number of corners used 52

RMSE discrepancy (E,N,h) (m) 0.310 0.375 0.561

Maximum discrepancy (E,N,h) 1.347 1.202 -1.155

13/04/2013 INPE_Course_2013 82



83/103


13/04/2013 INPE_Course_2013 83



84/103


13/04/2013 INPE_Course_2013 84

MONORRESTITUI O


85/103

Digitalizao rea Urbana

13/04/2013 INPE_Course_2013 85

MONORRESTITUI O


86/103

Representao - Planimtrica

13/04/2013 INPE_Course_2013 86



87/103

Bundle Adjustment

control points in the photogrammetric bundle adjustment

Thirty-six pre-signalized points and twenty-eight centroid points were used

These points were used to perform the experiments proposed andverify the accuracy of the results

Type of pre-signalized control points and a typical centroid point determined

13/04/2013 INPE_Course_2013 87



88/103

Used datasetsThe LIDAR dataset was captured using an OPTECH ALTM 2050 laser scannerwith an average flying height of 975m and mean point density of 2.24 points/m2

(~0.7m point spacing). The range and intensity data were recorded. According tothe sensor and flight specifications, 0.5m horizontal and 0.15m vertical accuracies

are expected

13/04/2013 INPE_Course_2013 88

pec ca ons o e p o ogramme r c a ase



89/103

Block configuration- -

relative survey and the twenty-eight centroid points have 3D LIDARcoordinates determined by methodology proposed

13/04/2013 INPE_Course_2013 89


90/103


B dl Adj t t i i t t id t l


91/103

Bundle Adjustment using points centroid as control

Bundle ad ustment results usin oints centroid as control oints

13/04/2013 INPE_Course_2013 91

and check points analysis

Bundle Adjustment Low cost digital camera


92/103

13/04/2013 INPE_Course_2013 92

C C lib ti


93/103

Camera Calibration

13/04/2013 INPE_Course_2013 93

Signalized Control Point


94/103

13/04/2013 INPE_Course_2013 94

Horizontal Discrepancies


95/103

13/04/2013 INPE_Course_2013 95

Vertical Discrepancies


96/103

13/04/2013 INPE_Course_2013 96

Bundle Adjustment Results GPS Coordinates


97/103

13/04/2013 INPE_Course_2013 97



98/103

13/04/2013 INPE_Course_2013 98



99/103

13/04/2013 INPE_Course_2013 99

Bundle Adjustment Results LIDAR Coordinates


100/103

13/04/2013 INPE_Course_2013 100



101/103

13/04/2013 INPE_Course_2013 101



102/103

13/04/2013 INPE_Course_2013 102

EOPs differences from GPS and


103/103

EOPs differences from GPS andLIDAR Bundle Adjustments

13/04/2013 INPE_Course_2013 103

Documents

INPE Course 2013 Mitishitar Integration