G E O M A T I C A

ERROR ANALYSIS OF A MOBILE

TERRESTRIAL LIDAR SYSTEMM. Leslara, B. Hua, J.G. Wanga

aDepartment of Earth and Space Science and Engineering, Lassonde School of Engineering, York University, Toronto, Canada

The understanding of the effects of error on Mobile Terrestrial LiDAR (MTL) point clouds has notincreased with their popularity. In this study, comprehensive error analyses based on error propagationtheory and global sensitivity study were carried out to quantitatively describe the effects of various errorsources in a MTL system on the point cloud. Two scenarios were envisioned; the first using theuncertainties for measurement and calibration variables that are normally expected for MTL systems asthey exist today, and the second using an ideal situation where measurement and calibration values havebeen well adjusted. It was found that the highest proportion of error in the point cloud can be attributedto the boresight and lever arm parameters for MTL systems calibrated using non-rigours methods. Inparticular, under a loosely controlled error condition, the LiDAR to INS Z lever arm and the LiDAR toINS roll angle contributed more error in the output point cloud than any other parameter, including theINS position. Under tightly controlled error conditions, the INS position became the dominant source oferror in the point cloud. In addition, conditional variance analysis has shown that the majority of theerror in a point cloud can be attributed to the individual variables. Errors caused by the interactionsbetween the diverse variables are minimal and can be regarded as insignificant.

Introduction

Achieving high accuracy surveys using MobileTerrestrial LiDAR (MTL) systems is a challengefaced by many surveyors. Many factors contribute tothe outcome of the final product, from base stationlocations and local GPS conditions to MTL systemcalibration and LiDAR measurement capabilities.

In fact, the accuracy of an MTL survey can greatlydepend on factors as variable as the weather.Controlling as many of these factors as possiblewould seem to be called for; however, too muchattention and care in collecting the data can becomecostly in both time and money. A surveyor therefore

GEOMATICA Vol. 68, No. 3, 2014, pp. 183 to 194 dx.doi.org/10.5623/cig2014-303

M. Leslarmikel@optech.ca

B. Hu

J.G. Wang

La compréhension des effets des erreurs sur les nuages de points du LiDAR terrestre mobile (MTL)n’a pas augmenté avec sa popularité. Dans le présent article, nous avons effectué des analyses globalesdes erreurs fondées sur la théorie de propagation des erreurs et une étude globale de la sensibilité dansle but de décrire quantitativement les effets, sur le nuage de points, de diverses sources d’erreurs dansun système MTL. Nous avons envisagé deux scénarios, le premier utilisant les incertitudes dans lesvariables de mesure et de calibration qui sont normalement attendues pour les systèmes MTL quiexistent actuellement et le second utilisant une situation idéale où les valeurs de mesure et de calibrationont été bien ajustées. Il a été établi que la proportion la plus élevée d’erreurs dans le nuage de pointspeut être attribuée aux paramètres de visée et de levier pour les systèmes MTL calibrés en utilisant desméthodes non rigoureuses. Plus particulièrement, dans une condition d’erreurs mal contrôlées, le levierentre le LiDAR et l’axe Z du INS (système de navigation par inertie) et l’angle de roulis entre le LiDARet l’INS ont contribué à des erreurs plus grandes dans le nuage de points résultant que tout autreparamètre, y compris la position du INS. Dans des conditions d’erreurs fortement contrôlées, laposition du INS est devenue la principale source d’erreurs dans le nuage de points. De plus, l’analysede la variance conditionnelle a démontré que la majorité des erreurs dans un nuage de points peut êtreattribuée aux variables individuelles. Les erreurs causées par les interactions entre les diverses variablessont minimes et peuvent être considérées comme négligeables.

G E O M A T I C A

needs to know what factors to spend time andeffort controlling and by how much they should becontrolled. A balance between controlling the errorsources and producing results, as always, must bestruck.

An MTL system usually consists of a LiDARsensor integrated with an Inertial NavigationSystem (INS). The parameters used to convert rawMTL data into a georeferenced point cloud includeboth fixed parameters and measurements. The fixedparameters include the LiDAR calibration, theboresight and lever arms between the LiDAR andthe INS. The measurements made by the MTL sys-tem include the position and orientation determinedby the INS and the angles and ranges measured bythe LiDAR. Some measurements are external to theMTL system, such as those made at the GPS basestation, while others are confined to the movingplatform on which the MTL system operates. Thispaper will focus on the error sources containedwithin the MTL system, specifically the measure-ment sources used to produce a point cloud and theerrors that occur because of the integration of thesemeasurements.

The calibration and measurement sources usedin MTL are similar to those used by MobileAirborne LiDAR (MAL). Similar to MTL, MALcollects INS positions and orientations and com-bines them with raw angles and ranges measuredby the LiDAR. MAL also requires boresight andlever parameters between the LiDAR and INS to bedetermined. Unlike MAL, MTL is far moreimmersed in the target scene. This means that, gen-erally, MTL systems have a larger range of possibleboresight parameters between systems, the field ofview of the LiDAR is much larger, and the rangesare generally much shorter.

Measurements for the position and orientationof the MTL system are generated by the INS for anytime t along the vehicle’s path. The INS usuallyprovides estimates of the errors for these positionand orientation measurements. These error estimatesare calculated in real time by the INS and are usual-ly given in the form of root mean square (RMS)errors or one sigma standard deviations ( ), depend-ing on the INS system in question. On the otherhand, the measurements made by the LiDAR pro-vide information on the location of any point p inrange of the system. It is usual for the LiDAR tohave fixed error quantities determined by the manu-facturer or the surveyor in a controlled environment.These error estimates are almost always given asone sigma standard deviations. Many of these tech-niques have been well documented in static LiDARsystems [Reshetyuk 2009].

The errors associated with the integration of theLiDAR with the INS are usually the least under-stood and generally the hardest to quantify. Whilemultiple methods have been proposed for LiDAR toIMU boresighting, few offer a reliable or rigorousapproach for determining error. The documentedmethods for determining sensor to INS calibrationparameters include manual visual methods [OptechInc. 2008], software assisted visual methods[Hypack Inc. 2007; Reson B.V. 2011; VirtualGeomatics Inc. 2010; Jeeninga 2012; Dix et al.2011] and algorithm based methods [Rieger et al.2010; Picard et al. 2012; Chan et al. 2013]. Onlythe algorithm-based methods really give a meansfor determining error estimates for the LiDAR toINS calibration parameters.

Some work has already been done to analyzethe errors inherent in MTL systems. Sources, suchas Alshawa et al. [2007], Glennie [2007a],Glennie [2009], Glennie [2007b], discuss currentworks on the development of an error model forMTL systems that incorporate modified staticterrestrial LiDAR systems in their design. Inaddition, overall performance data with respectto a control field can be found in Leslar [2009].

Despite the work that has already beenpublished, the error sources for MTL remainunder-analyzed. A comprehensive study of errorsources involved with MTL systems is needed tounderstand and identify which factors can bebetter controlled for the purpose of producinghigh accuracy MTL point clouds. This paper pres-ents the results of a comprehensive error analysisconducted on data collected with the Lynx MobileMapper [Optech Inc. 2008]. The Lynx system lends

184

Figure 1: A typical configuration of a mobile terrestrial system plotted inthe Earth Centered Earth Fixed (ECEF) coordinate system.

G E O M A T I C A

itself to this type of testing since the calibrationnumbers are provided by the manufacturer in ASCIItext file format. Unlike some other systems, wherethe calibration is stored in proprietary binaryformats, the Lynx calibration values are open tothe end user.

The Mathematical Models

The LiDAR Equation

Mobile LiDAR systems consist of two distinctcomponents, the INS and the laser ranging system,as shown by a typical system configuration inFigure 1. As shown in Figure 1, the coordinates ofpoint p in the Earth Centered and Earth Fixed(ECEF) coordinate system are a function ofthe laser range vector, the relative location and ori-entation of the LiDAR sensor to the INS systemand the position and orientation of the INS. Thebase equation of this relationship can be expressedby Equation 1.

(1)

In Equation 1, is the position of the

INS, is the vector from the INS to the

LiDAR sensor, and is the vector from the

LiDAR to the target point p. and

can be further deconstructed, as expressed inEquations 2 and 3.

(2)

(3)

where is the lever arm vector from the INS

body frame to the LiDAR body frame, is thelaser range vector between the LiDAR and thetarget point p, R1 is the rotation matrix betweenthe local geodetic coordinates and the ECEF

PECEF

= + +− −P P L L pECEF

INSECEF

INS LiDARECEF

LiDARECEF

PINSECEF

−LINS LiDARECEF

−LLiDAR pECEF

−LLiDAR pECEF

−LINS LiDARECEF

( ) ( )= ⋅−L R B, L R r, p,hINS LiDARECEF

1 2

( )⋅ l l ,l ,lLiDARINS

X Y Z

( ) ( )= ⋅−L R B, L R r, p,hpLiDARECEF

1 2

θ θ θ α β( ) (⋅ ⋅R , , Z l , ,d,KpX YLiDAR

3

lLiDARINS

lpLiDAR

coordinate frame, R2 is the rotation matrix betweenthe INS body frame and the local geodetic frame,and R3 is the rotation matrix between LiDAR body

frame and the INS body frame. The vectors ,

and the matrices R1, R2, R3 are themselvesproducts of various measurements. We can groupthese measurements based on their dependencies.Group 1 consists of the INS position in Cartesian

ECEF coordinates . TheGroup 1 parameters are converted from thegeodetic latitude (B), longitude (L) and ellipsoidalheight (h) measured by the INS. Group 2 consistsof r, p, h, which represent the roll, pitch and head-ing of the INS with respect to the local geodeticcoordinate frame. Group 3 consists of lX, lY, lZ,which represent the components of the lever armvector between the INS and the LiDAR. Group 4consists of , which represent the x, yand z rotations of the LiDAR coordinate framein the INS frame. Group 5 consists of the LiDARmeasurements, where α represents the horizontalangle measured by the LiDAR in the LiDAR bodyframe, β represents the vertical angle measured bythe LiDAR in the LiDAR body frame, d representsthe distance measured by the LiDAR to the targetpoint p and K represents the zero error of theLiDAR instrument.

Each of the 18 aforementioned variableshave an associated error estimate, usually in theform of an RMS or standard deviation. It is usualto assume that each of the measurements isnormally distributed and that the RMS valuesreported by the INS can be considered equal toone sigma standard deviations (σ). These 18standard deviations can be assigned to the samegroups previously discussed.

Error Propagation Analysis

The error propagation analysis is a standardtechnique used to estimate errors in calculatedvalues propagated from the errors of the meas-ured parameters based on a known relationship.The equations for linear error propagation arefairly well understood and used across manydisciplines [Glennie 2007b; Goulden 2009;Tellinghuisen 2001]. Performing linear errorpropagation first requires the non-linear equationpresented in Equation 1 to be linearized asshown in Equation 4.

lLiDARINS

lpLiDAR

( )X , Y , ZINSECEF

INSECEF

INSECEF

θ θ θ, ,X Y Z

185

G E O M A T I C A

(4)

In Equation 4, represents Equation 1evaluated at the initial approximate variables

for the 18 measurements previously identified.Using the first derivatives from Equation 4, evalu-ated at each component (X, Y, Z) of the ECEFposition, the design matrix (U) for a least squareadjustment may be formed as shown in Equation 5.

(5)

+ ⋅ Δ + ⋅ Δ + ⋅ ΔdP

drr

dP

dpp

dP

dhh

ECEF ECEF ECEF

+ ⋅ Δ + ⋅ Δ + ⋅ ΔdP

dlxlx

dP

dlyly

dP

dlzlz

ECEF ECEF ECEF

θθ

θθ

θθ+ ⋅ Δ + ⋅ Δ + ⋅ ΔdP

d

dP

d

dP

d

ECEF

XX

ECEF

YY

ECEF

ZZ

αα

ββ+ ⋅ Δ + ⋅ Δ + ⋅ ΔdP

d

dP

d

dP

ddd

ECEF ECEF ECEF

+ ⋅ ΔdP

dKK

ECEF

PECEF0

( ( ) ( ) ( )X , Y , Z , B , L , r , p , INSECEF

INSECEF

INSECEF0 0 0 0 0 0 0

θ θ θ α β )h , l , l , l , , , , , , d , KX Y Z X Y Z0 0 0 0 0 0 0 0 0 0 0

+ ⋅ Δ + ⋅ ΔdP

dBB

dP

dLL

ECEF ECEF

+ ⋅ Δ + ⋅ ΔdP

dYY

dP

dZZ

ECEF

INSECEF INS

ECEFECEF

INSECEF INS

ECEF

≈ + ⋅ ΔP PdP

dXXECEF ECEF

ECEF

INSECEF INS

ECEF0

=

⎡

⎣

⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥

U

dP

dX

dP

dX

dP

dX

dP

dK

dP

dK

dP

dK

ECEF

INSECEF

X

ECEF

INSECEF

Y

ECEF

INSECEF

Z

ECEF

X

ECEF

Y

ECEF

Z

Using the additional information providedfrom the individual error estimates of the 18 inputparameters and assuming no correlation betweenthese variables, the variance-covariance matrix (Q)may be formed as shown in Equation 6.

(6)

This allows us to form the error propagation modelshown in Equation 7.

(7)

Conditional Variance Analysis

Conditional variance analysis is a method ofglobal sensitivity analysis. Global sensitivityanalysis is primarily used to study how the uncer-tainty in the output of a model can be apportioned todifferent sources of uncertainty in the model inputs[Saltelli et al. 2008]. First order sensitivity indicesare a quantitative measure of the importance of theinput variables on the calculated results. They applyto additive models. The more sensitive the calculat-ed results are to small variances in any particularinput variable, the greater the first order sensitivityindex. The total effect indices are also a quantitativemeasure of the importance of the input variables onthe calculated results. Unlike first order indices,however, they are applicable to all types of modelsindependent of their model characteristics[Schwieger 2004].

Performing conditional variance analysisinvolves rewriting the terms of Equation 1 in termsof generic variables; Equation 1 can be expressedas shown in Equation 8.

(8)

Variance-based sensitivity analysis is carriedout using simulated values for each variable (Xi). Toacquire these simulated values one must generate N

σ

σ

=

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥

Q

0 0

0

0

0 0

X

K

2

2

INSECEF

σ = ⋅ ⋅−U Q UP

T2 1ECEF

( )= ≤ ≤P f X for i1 18ECEFi

186

G E O M A T I C A

random samples according to a specific probabilitydensity function for each of the variables [Glen andIsaacs 2012; Bowyer and Danson 2004]. Startingwith the estimated values and their estimatedstandard deviations that we have for the 18 variables,these random samples can be generated based onthe normal distribution.

Holding variable (Xi) fixed at the mean valueof its randomly generated samples, while allowingthe other variables to vary, N random values canbe generated from Equation 8. The variance of the

outputs of Equation 8 can be cal-culated. If we then average this variance over allpossible values of Xi we get the expectation of the

variance, . Based on linearalgebra we can compute the main effect of Xi on

from Equation 9[Saltelli et al. 2008].

(9)

The larger is, the moreinfluential Xi is on the output of Equation 8.Therefore, from the relationship shown in Equation9, the first order Sobol index for the variable Xi canbe computed according to Equation 10 [Saltelli etal. 2008].

(10)

Higher order indices are computed by holdingmore than one variable fixed as shown in Equation11 [Saltelli et al. 2008].

(11)

Computing all the higher order terms becomesinefficient due to the large number of combinationsfor models with a large number of variables. Forexample, the 18 variables in Equation 1 will gen-erate 48,620 9th order Sobol indices alone. It istherefore desirable to calculate the total effect for a

variable . This can be done by calculating the

( )( )V P XECEFi

( )( )E V P XECEFi

( )( )( )P V E P XECEF ECEFi

( ) ( )( ) ( ) ( )= +V P E V P X V E P XECEF ECEFi

ECEFi

( )( )V E P XECEFi

( )( )( )=S

V E P X

V Pi

ECEFi

ECEF

( )( )( )= − − −S

V E P X , X , X ,

V PS S Si, j ,k ,

ECEFi j k

ECEF i j k

( )Sti

variance of the expected values when all thevariables in the model are held fixed while

allowing only Xi to vary .The total effect can be computed as shown inEquation 12 [Saltelli et al. 2008].

(12)

For additive models

For non-additive models

Implementation and Testing

The error propagation described above wasimplemented in Microsoft Visual C++ 6.0 and theconditional variance analysis was implemented inMatlab R14. For testing purposes, MTL data werecollected using the Lynx Mobile Mapper. Thesedata were collected in a mostly empty parking lotto allow the LiDAR to reach its maximum range.The data used is shown in Figure 2.

The northern section of the data contains a treeline that is located approximately 50 m from the

∼( )( )( )V E P XECEFi

∼( )( )( )= −S

V E P X

V P1t

ECEFi

ECEFi

∑ = ==

S , S S1i t ii

n

1 i

>S St ii

187

Figure 2: Lynx data collected in a parking lot inVaughan, Ontario, Canada, on May 19, 2012. The colourscale indicates the height above the WGS84 ellipsoid.

G E O M A T I C A

LiDAR sensor; the southern section of the datacontains an open parking lot with some hedges anda lamp post. A building and two cars exist in theparking lot at long range for the LiDAR sensor.They are not seen in Figure 2 as they are at extremelong range for the LIDAR system, but randomshots were collected off of them.

To analyze the effect of errors in the 18parameters on the output point cloud, it is firstnecessary to collect the error estimates for eachparameter. The error estimates concerning theposition and orientation of the Lynx system in theECEF mapping frame are generated in real timeby the INS. In the case of the Lynx system, theINS used is an Applanix POS.

Out of the five variables related to the positionof the INS, it was found that the latitude and longi-tude contribute less than a millimetre of error to thefinal solution. For the purposes of testing, theywere therefore excluded from further considerationsince their contribution to the overall error isinsignificant. The LiDAR errors are published byOptech, the manufacturer of the Lynx sensor head.They state that the one sigma range accuracy of theLiDAR is 0.008 m. Examining planar surfaces inmultiple points taken by the Lynx system reveals that

the range accuracy of the system is generally betterthan this, usually around 0.003 m. Therefore thesenumbers will be compared to see the effect of each.

That leaves the six errors associated with theintegration of the LiDAR and INS system, the so-called boresight and lever arm parameters.Typically, boresight operation for many MTL andmultibeam sonar systems is done using non-rigours methods such as the so-called “Patch Test”[Hypack Inc. 2007]. Multiple manufacturers ofthis equipment and the software used to processthe data use these methods for determining bore-sight parameters for MTL [Optech Inc. 2008;Hypack Inc. 2007; Reson B.V. 2011; VirtualGeomatics Inc. 2010; Jeeninga 2012; Dix et al.2011]. Table 1 gives a breakdown of theseparameters, their groups, expected error valuesand the ideal error values for these parametersobtained by accurate calibrations.

Some literature sources agree that the expectedlever arm uncertainties listed in Table 1 are typicalfor most systems [Glennie 2007b]. However, thesame source indicates that the typical manual bore-sight method should facilitate an uncertainty of0.005° for roll and pitch and 0.008° for heading[Glennie 2007b]. The same source further states

188

Table 1: Expected and ideal uncertainties in MTL parameters.

Parameter Group Expected Ideal

, Position X [m] 1 Estimated from INS Estimated from INS

, Position Y [m] 1 Estimated from INS Estimated from INS

, Position Z [m] 1 Estimated from INS Estimated from INS

, Platform Roll [degrees] 2 Estimated from INS Estimated from INS

, Platform Pitch [degrees] 2 Estimated from INS Estimated from INS

, Platform Heading [degrees] 2 Estimated from INS Estimated from INS

, LiDAR X Lever Arm [m] 3 0.02 0.004

, LiDAR Y Lever Arm [m] 3 0.02 0.004

, LiDAR Z Lever Arm [m] 3 0.02 0.004

, LiDAR Roll [degrees] 4 0.02 0.001

, LiDAR Pitch [degrees] 4 0.02 0.001

, LiDAR Heading [degrees] 4 0.02 0.001

, LiDAR Horizontal Angle [degrees] 5 0.0055 0.0055

, LiDAR Vertical Angle [degrees] 5 0.0055 0.0055

, LiDAR Distance [m] 5 0.008 0.003

, LiDAR Zero Error [m] 5 0.01 0.005

σXINS

ECEF

σYINS

ECEF

σZINS

ECEF

σ r

σ p

σ h

σ lx

σ ly

σ lz

σθX

σθY

σθZ

σ a

σ β

σ d

σ K

G E O M A T I C A

that the expected results from a least squaresadjustment should have a typical uncertainty of0.001° for roll and pitch and 0.004° for heading.

The values from the manual boresight seemoptimistic. When performing a manual boresight, amore likely value for the boresight roll, pitch andheading uncertainty is 0.02°, as reflected in Table 1.

Results and Discussion

The Lynx data was run through the errorpropagation model for both the expected errorestimates and the ideal error estimates. Theresults were converted to a colour scale and

applied to the point cloud. The results from theerror propagation analysis for the expected errorestimates are shown in Figure 3.

The colour scales applied to the point clouds inFigure 3 show that the largest source of error in theresultant point cloud is consistently caused by theINS positional errors. The poor quality of the leverarm estimates in this case makes the lever armsbetween the LiDAR and the INS the second mostconsistent source of error in the system. The angularerrors from the INS orientation variables, as well asthe LiDAR to INS boresight variables, take onincreasing significance as the range from theLiDAR sensor increases. The LiDAR measure-ments themselves comprise the least source of errorin the point cloud.

The colour-mapped data from the first orderconditional variance analysis for the expected errorestimates are shown in Figure 4. The colour mapsin Figure 4 show that when the expected errorestimates exist in the point cloud, the largest pro-portion of the error in the MTL point cloud is givenby the LiDAR to INS lever arm values in Group 3.The rotations between LiDAR coordinate frameand the INS coordinate frame (Group 4) in turntake on increasing significance as the range fromthe LiDAR sensor increases.

189

Figure 3: Results of linear error propagation using theexpected error estimates. Each of the five identifiedgroups of variables was isolated and the errorpropagation performed.

Figure 4: Results of first order conditional varianceanalysis using the expected error estimates. Each ofthe five identified groups of variables was isolatedand the Sobol indices computed.

Figure 5: Scatter plots of the first order Sobol indices for Zlever arm and boresight roll in Lynx Mobile Mapper data.

G E O M A T I C A

Since the color maps in Figure 4 refer togroups of variables instead of individual variables,the Sobol indices for each variable were calculated.It was found that the lever arm almost exclusivelyresponsible for this result is the Z lever arm. Thescatter plots in Figure 5 show that at close rangesthe Z lever arm from the LiDAR to INS contributes

up to 90% of the error in the point cloud. Figure 5also shows that as the range from the sensorincreases, the significance of the Z lever arm tapersoff and is replaced by the LiDAR to INS roll angle.At the maximum range of the LYNX sensor, theeffect of roll under these conditions becomes thesingle most significant source of error in the pointcloud. Figure 5 shows that at the maximum range ofthe Lynx sensor, the error generated by the LiDARto INS roll angle is upwards of 70%.

In addition to the first order indices of theconditional variance analysis, the total effectindices for the five groups were also computed.These total effect indices were converted to acolour scale and applied to the point cloud. Figure6 shows the total effect indices for the expectederror estimates. Figure 7 shows the two mostimportant scatter plots of the total effect indices forthe expected error estimates. The colour-mappeddata in Figure 6 show that when the expected errorestimates exist in the point cloud, the total effectindices from the conditional variance analysis arepractically identical to the first order effect.

Breaking down the groups into individualvariables as we did with the first order indices,again we see that the LiDAR to INS Z lever armand the LiDAR to INS roll angle are the mostsignificant parameters in terms of their proportionaleffect on the point cloud. The scatter plots shown inFigure 7 indicate that the LiDAR to INS Z leverarm contributes upwards of 90% of the error atranges close to the LiDAR sensor and decreases asthe range increases. Figure 7 also shows that theLiDAR to INS roll angle has little significanceclose to the LiDAR sensor, but makes up almost70% of the error at maximum range.

For this MTL point cloud, error propagationwith the expected error estimates (Figure 3) indicatesthat the largest source of error in the point cloudcomes from the INS position. The first order condi-tional variance analysis tells a different story. Figure4 indicates that the INS position only accounts forabout 12% of the total error in the point cloud. Thelargest source of error is actually the LiDAR to INSlever arm, not the INS position. The fact that bothFigures 5 and 7 show the Z-lever arm and theLiDAR to INS roll angle contributing upwards of90% and 70% of the total error at various times high-lights the significance of these two parameters. Thiswas apparent from the error propagation model.

The fact that the first order sensitivity indices inFigure 4 add up to one, and that the first order indicesclosely mirror the total effect indices shown inFigure 6, indicates that Equation 1 is an additivemodel. This implies that the majority of the total

190

Figure 7: Scatter plots of the total effect Sobol indices for Zlever arm and boresight roll in Lynx Mobile Mapper data.

Figure 6: Results of the total effect conditional variance analysis using the expected error estimates.Each of the five identified groups of variables wasisolated and the Sobol indices computed.

G E O M A T I C A

effect can be attributed to the individual variables,i.e., the first order effect. Cross-effects caused by thevarious variable combinations are minimal and canbe regarded as insignificant. The higher orderindices from Equation 11 can therefore bedisregarded from further error analysis.

The results from the error propagation for theideal error estimates are shown in Figure 8. Figure8 shows that using the ideal error estimates, theerror in the resulting point cloud will be heavilyinfluenced by the errors inherent in the INS system.The most significant source of error in Figure 8 isin the INS position (Group 1) at a constant 3–4 cm.The second largest source of error according toFigure 8 is given by the INS orientation data

(Group 2). This error is insignificant at the LiDARand increases as the range from the LiDARincreases. The LiDAR to INS lever arms, therotations of the LiDAR coordinate frame in the INSframe and the LiDAR measurements themselves areshown to contribute an insignificant amount of errorin Figure 8.

The colour maps in Figure 9 show that whenthe ideal error estimates exist in the point cloud, thelargest proportion of error in the MTL point cloud isgiven by the INS position parameters of Group 1.The LiDAR measurement parameters of Group 5contribute the next highest percentage of error inthe MTL point cloud, while the other groupsdisappear into insignificance.

191

Figure 9: Results of first order conditional varianceanalysis using the ideal error estimates. Each of thefive identified groups of variables was isolated andthe Sobol indices computed.

Figure 8: Results of linear error propagation usingthe ideal error estimates. Each of the five identifiedgroups of variables was isolated and the errorpropagation performed.

Figure 10: Scatter plots of the first order Sobol indices for Zcoordinate of the INS position, the LiDAR range and theLiDAR measured vertical angle.

G E O M A T I C A

The scatter plots for the ideal error estimateswere calculated and it was found that the Z coordi-nate of the INS position, the LiDAR range and theLiDAR vertical angle were the most significant.Figure 10 shows that the Z coordinate from the INScontributes up to 45% of the total error at closeranges under ideal error conditions. The LiDARrange contributes another 40% of the error at thesesame ranges. In some spots where laser saturationsoccur, spikes can be seen in the LiDAR range scat-ter plot in Figure 10; however, the general trend ofthe data is for the laser range to contribute less errorto the overall solution as range increases. As thesignificance of the LiDAR range decreases, thevertical angle of the LiDAR takes over as the majorsource of error from the LiDAR, peaking at around40% at long ranges.

The colour-mapped data in Figure 11 showthat when the ideal error estimates exist in the pointcloud, the total effect indices from the conditionalvariance analysis are also very similar to the firstorder effect. The INS position (Group 1) consis-tently contributes the largest proportion of error tothe MTL point cloud, with the LiDAR measure-ments (Group 5) contributing the next largestproportion of error to the MTL point cloud. Theother groups vanish into insignificance.

Looking at the total effect Sobol indices for theideal case, we see similar results as those found withthe first order Sobol indices. Figure 12 shows the totaleffect Sobol indices for the same parameters that wereshown in Figure 10, namely, the Z coordinate of theINS position, the LiDAR range and the LiDARvertical angle. As expected, the results for the totaleffect Sobol indices show results identical to the firstorder Sobol indices of Figure 10. Just as in Figure 10,Figure 12 shows that the INS Z coordinate contributesup to 45% of the overall error and that there is areciprocal relationship between the amount of errorcontributed by the laser range finder and theLiDAR encoders.

Figures 9, 10, 11 and 12 reinforce the implicationthat the majority of the total effect can be attributed tothe first order effect. The higher order effects fromEquation 11 can therefore be treated as insignificant.

Contrasting the result obtained using theexpected error estimates in the error propagationmodel, we see that the ideal error estimates in theerror propagation model clearly give us the INSposition as the main source of error in the pointcloud (Figure 8). In Figure 8, these three groups ofvariables show error estimates less than 1 cm for allthe points in the point cloud. Only the INS positionand orientation errors play a significant role in thefinal point cloud error. The conditional variance

192

Figure 12: Scatter plots of the total effect Sobol indices forZ coordinate of the INS position, the LiDAR range and theLiDAR measured vertical angle.

Figure 11: Results of total effect conditional varianceanalysis using the ideal error estimates. Each of thefive identified groups of variables was isolated andthe Sobol indices computed.

G E O M A T I C A

analysis bears this out, showing that the INSposition errors make up more than 50% of the errorsthroughout the point cloud (Figure 9). Figure 9shows that the INS orientation, LiDAR lever armsand the LiDAR boresight collectively make up asmall percentage of the total error. We also see inFigure 9 that the second most significant source oferror is from the LiDAR and that it occurs directlyalong the MTL system’s path. This is due to effectsin the LiDAR receiver caused by the extreme shortranges measured by the system. The rest of the errorfrom the LiDAR increases as the range increases.This indicates that the angular uncertainties in theLIDAR’s encoder combined with range and zeroerror uncertainties become more pronounced in theresulting point cloud at range.

Conclusions

The nature of error in MTL is still beingdiscovered. While research into the nature of errorgenerated by Airborne LiDAR systems has beenconducted for several years, research into errors inMTL has just begun.

Conditional variance analysis has shown thatthe contribution by the interactions among themeasured parameters to the variances of the pointsin an MTL point cloud is insignificant. The sizes ofthe variances for the measurements used to producea point are the primary source of error in the outputpoint cloud. In particular, under a loosely controllederror condition, the LiDAR to INS Z lever arm andthe LiDAR to INS roll angle contributed more errorin the output point cloud than any other parameter,including the INS position. Under tightly con-trolled error conditions, the INS position becomesthe dominant source of error in the point cloud.

Therefore, MTL systems that have beenmounted and boresighted using non-rigorousmethodologies provide the greatest source of errorin the resulting point cloud. It is therefore essentialthat high quality rigours methods be used to derivethe integration parameters so that high qualityLiDAR data may be collected with an MTL system.

References

Alshawa, M., E. Smigiel, P. Grussenmeyer, and T.Landes. 2007. Integration of a terrestrial LIDAR ona mobile mapping platform: first experiences. 5thInternational Symposium on Mobile MappingTechnology. Padua, Italy. 2007.

Bowyer, P. and F.M. Danson. 2004. Sensitivity of spectralreflectance to variation in live fuel moisture contentat leaf and canopy level. Remote Sensing ofEnvironment. 92: 297-308.

Chan, T., D. Lichti, and C. Glennie. 2013. Multi-featurebased boresight self-calibration of a terrestrialmobile mapping system. ISPRS Journal ofPhotogrammetry and Remote Sensing. 82: 112-124.

Dix, M., A. Abd-Elrahman, B. Dewitt and L. Nash.2011. Accuracy evaluation of terrestrial LiDARand multibeam sonar systems mounted on a surveyvessel. J. Surv. Eng. doi: 10.1061/(ASCE)SU.1943-5428.0000075

Glen, G. and K. Isaacs. 2012. Estimating Sobol sensitivityindices using correlations. Environmental Modelling& Software. 37: 157-166.

Glennie, C. 2007a. Reign of point clouds: a kinematicterrestrial LIDAR scanning system. Inside GNSSMagazine. 2(7), Fall 2007.

Glennie, C. 2007b. Rigorous 3D error analysis ofkinematic scanning LIDAR systems. Journal ofApplied Geodesy. 1(3): 147–157, November 2007.doi: 10.1515/jag.2007.017

Glennie, C. 2009. A kinematic terrestrial LIDAR scanningsystem. Proceedings of the Transportation ResearchBoard 88th Annual Meeting, Washington D.C.January 11–15, 2009. Paper No. 09-0122.

Goulden, T. 2009. Prediction of error due to terrain slopein LiDAR observations. Thesis, MS in Geodesy andGeomatics. University of New Brunswick, 2009.Published as technical report No. 265, University ofNew Brunswick.

Hypack Incorporated. 2007. Hysweep User’s Manual.Middletown, Connecticut, USA: 2007.

Jeeninga, B. 2012. Mobile laser calibration. Presentationgiven by QPS. Zeist: The Netherlands, 2012.

Leslar, M. 2009. Extraction of geo-spatial information fromLiDAR-based mobile mapping system for crowdcontrol planning. IEEE Xplore Special Issue, Scienceand Technology for Humanity (TIC-STH). 2009 IEEEToronto International Conference. 468-472.

Optech Incorporated. 2010. Lynx Software Manual–Revision B. Vaughan, Ontario, Canada: 2008.

Picard, A., T. Touze, N. Seube, J.G. Nistad, and M.Rondeau. 2012. Calibration of vessel mountedLiDAR. Proceedings of the Canadian HydrographicConference, May 15–17, 2012. Niagra Falls,Ontario, Canada.

Reshetyuk, Y. 2009. Self-calibration and direct georefer-encing in terrestrial laser scanning. Thesis, PhD inInfrastructure Geodesy. Royal Institute ofTechnology, Department of Transport andEconomics, Stockholm, Sweden.

Reson B.V. PDS2000 Multibeam Office Training.PDS2000 v3.7. Rotterdam: 2011.

Rieger, P., N. Studnicka, M. Pfennigbauer, and G. Zach.2010. Boresight alignment method for mobile laserscanning systems. Journal of Applied Geodesy. 4(1):13–21, June 2010. doi: 10.1515/jag.2010.002

Saltelli, A., M. Ratto, T. Andres, F. Campolongo, J.Cariboni, D. Gatelli, M. Saisana, and S. Tarantola.

193

G E O M A T I C A

2008. Global sensitivity analysis. The Primer. JohnWiley & Sons, Ltd. 20-25.

Schwieger, V. 2004. Variance-based sensitivity analysisfor model evaluation in engineering surveys.INGEO 2004 and FIG Regional Central andEastern European Conference on EngineeringSurveying. Bratislava, Slovakia, November11–13, 2004.

Tellinghuisen, J. 2001. Statistical error propagation.Journal of Physical Chemistry. 105: 3917-3921.

Virtual Geomatics Incorporated. LiDAR Calibrator.Austin Texas USA: 2010.

Authors

Michael Leslar is a PhD candidate in theGeomatics Engineering Program, Department ofEarth and Space Science and Engineering, LassondeSchool of Engineering at York University, Toronto,Canada. He has a Bachelor’s degree in CivilEngineering (Geomatics) and a Master’s degree inApplied Science (Geomatics) from the Departmentof Civil Engineering at Ryerson University,Toronto, Canada. In addition to being a part timestudent, he has been a full time employee of OptechIncorporated for the last eight years. His currentwork and research interests focus on terrestrialLiDAR data processing, feature extraction, LiDAR

sensor calibration, sources of error in terrestrialLiDAR and their correction.

Baoxin Hu received her Bachelor’s andMaster’s degrees in Electrical Engineering fromTianjin University, Tianjin, China and PhD degreein Remote Sensing and GIS from BostonUniversity, Boston, USA. She is currently anassociate professor in Geomatics Engineering, YorkUniversity, Toronto, Canada. Her research interestsinclude 3D scene reconstruction from LiDAR andoptical imagery and vegetation characterizationfrom remotely sensed data.

Jian-Guo Wang is an Associate Lecturer inGeomatics Engineering, faculty member of LassondeSchool of Engineering and Faculty of GraduateStudies of York University in Toronto, Canada. Heobtained his Bachelor’s and Master’s Degrees inSurveying Engineering from Wuhan TechnicalUniversity of Surveying and Mapping, Wuhan,China, and his Dr-Ing in Surveying Engineering fromUniversity of The Federal Armed Forces, Munich,Germany. His current research interests focus onmultisensor integrated positioning and navigationtechnology, GNSS, mobile mapping, engineeringsurveying, and advanced data processing algorithmsin geomatics. q

194