Certifiable Relative Pose EstimationMercedes Garcia-Salguero∗, Jesus Briales† and Javier Gonzalez-Jimenez‡

Machine Perception and Intelligent Robotics (MAPIR) Group, System Engineering and Automation Department,University of Malaga, Campus de Teatinos, 29071 Malaga, Spain

Email: ∗mercedesgarsal@uma.es, †jesusbriales@uma.es, ‡javiergonzalez@uma.es

Abstract—In this paper we present the first fast optimality certifier for the

non-minimal version of the Relative Pose problem for calibratedcameras from epipolar constraints. The proposed certifier isbased on Lagrangian duality and relies on a novel closed-form expression for dual points. We also leverage an efficientsolver that performs local optimization on the manifold of theoriginal problem’s non-convex domain. The optimality of thesolution is then checked via our novel fast certifier. The extensiveconducted experiments demonstrate that, despite its simplicity,this certifiable solver performs excellently on synthetic data,repeatedly attaining the (certified a posteriori) optimal solutionand shows a satisfactory performance on real data.

Index Terms—Relative Pose; Essential Matrix; Epipolar con-straint; Convex programming; Certifiable algorithm; LinearIndependence Constraint Qualification.

I. INTRODUCTION

In this work we consider the central calibrated relativepose problem in which, given a set of N pair-wise featurecorrespondences between two images coming from two cali-brated cameras, one seeks the relative pose (rotation R andtranslation t, up-to-scale) between both cameras (see Figure(1)) that minimizes the epipolar error.

Estimating the relative pose between two calibrated views ofa scene is specially relevant for visual odometry and also as abuilding block for more complex problems like Structure fromMotion (SfM) [1] or Simultaneous Localization and Mapping(SLAM) [2], [3].

1

2Fig. 1: In the relative pose problem, we aim to estimate therelative relative rotation R and the relative translation t up-to-scale between two calibrated cameras 1− 2 given a set ofN correspondence pairs of unit bearing vectors {f ′i ,fi}Ni=1.

[Ropt, topt][Rloc, tloc]

K-TEf

Kf'i

iK-TE

Tf'

Kfi

i

12

2'

Fig. 2: Suboptimal local minima (green) may lie far fromthe (globally) optimal solution (blue) and hinders subsequentalgorithms, e.g., Bundle Adjustment, even if it agrees withthe data {fi,f ′i}Ni=1. For the sake of exactness, we depictthe image counterparts of the data in pixels by applyingthe intrinsic camera parameters through K, assuming bothcameras have the same K and no lens distortion [6].

Whereas the gold standard for relative pose estimation isposing this as a 2-view Bundle Adjustment problem, this isa hard problem and it is common practice (see e.g. [4]) tobootstrap its initialization with a simpler formulation basedon the epipolar error.

Despite this simplification, the problem to optimize is stillnon-convex and presents local minima [5], which hinders theapplication of iterative approaches. These suboptimal minimamay lie arbitrarily far from the optimal solution, yet stillexplain the input data. Figure (2) illustrates this situation,where the local minimal solution (green) leads to a relativepose [Rloc, tloc] far from the optimal solution (blue) [Ropt, topt].Note that, in the presence of noise, the optimal solution maynot longer be the ground truth pose.

A suboptimal minima thus represents a wrong solution that,when passed to incremental methods can quickly cascadeleading to failure. When passed to global methods, where thesolution is averaged with many other estimations, it provokestwo negative effects. First, we are missing the opportunityto provide the method with more valid data, which, whenhealthy, improves the quality of the estimate. Second and

arX

iv:2

003.

1373

2v2

[cs

.CV

] 1

9 Fe

b 20

21

most important, this generally far from correct solution willturn into anything from mild to gross outliers, introducingbiases and hindering the performance of global estimators ingeneral. Thus, suboptimal local minima should be detectedand avoided, yet the Relative Pose problem is still non convexand hard to solve globally.

In this context, a recent line of research has evincedthat for some so-far-considered hard problems (in GeometricComputer Vision but also in other fields [7]), though worst-case instances can remain intractable in terms of resolutionor for the certification of optimality, real-world instances donot usually tend to these worst-cases. Interestingly, for manyproblem instances found in practice it is possible to attain andeven certify optimality.

Certifiable algorithms can be attained in multiple ways.Perhaps one of the most straightforward realizations consistsof characterizing a tight convex relaxation for the givenproblem instance and jointly solving for its primal and dualproblems [8], recovering at the same time a solution to theoriginal problem and a (dual) certificate of optimality. Thisis the recent proposal of Briales et al. [9] or Zhao [10] forthe Relative Pose problem, where they propose a (probably)tight Semidefinite Problem (SDP) relaxation that can be solvedefficiently (in polynomial time).

While the approach above is simple and often provides acertifiable solution to the Relative Pose problem, it is notthe only nor the most efficient way to devise a certifiablesolver [7, Sec. 1]. One can devise a much faster certifiableapproach by combining a fast solver for the original problem(one that returns the optimal solution with high probabilitybut no guarantees) with a fast standalone certification methodthat produces an optimality (dual) certificate leveraging thissolution (see e.g. [11], [12]), which finally brings us to thecore contribution of the present work.

Contributions In this line, we conceive a novel closed-form (linear) approach which allows to certify a posterioriif the potentially optimal solution to a Relative Pose probleminstance is indeed the optimal one. With this certifier available,we unblock the ability to build faster certifiable solvers inthe fashion proposed by Bandeira [7]. i.e. by combining fastheuristic solvers with a fast optimality certifier. To prove thevalue of this approach in the context of the Relative Poseproblem, we propose a novel, simple and efficient iterativeRiemannian Trust-Region solver that operates directly on theessential matrix manifold and tends to return the optimalsolution when initialized, for example, with the classical8-points (8pt) algorithm. The conducted experiments (inSection (VII)) show that, in practice, one can bootstrap theiterative method with the trivial identity matrix or a randomessential matrix and still retrieve the optimal solution, mostlywhen considering more than 40 correspondences. Combiningboth, we get a novel certifiable approach for solving theRelative Pose problem. This pipeline represents our mainpractical contribution and we refer the reader who is onlyinterested in its application to Section (VI) for a conciseexplanation. Moreover, given the simplicity of its component

blocks, we see great potential on this kind of pipeline to bestreamlined, in order to achieve excelling computational times,so that it becomes the new go-to state-of-the-art solver for thecommunity.

The main technical contributions contained in the paper thatwere required to achieve the above are:• Characterize a family of relaxed quadratic

formulations for the Relative Pose problem, whoseKarush–Kuhn–Tucker (KKT) conditions fulfill theLinear Independence Constraint Qualification (LICQ)(in Section (IV)).

• Based on the above, design a fast approach to computethe potential dual candidate solution in closed-form, giventhe (potentially) optimal solution to the original problem(in Section (V)).

• Define how to perform optimality certification, giventhe candidate dual solution from the approach above (inAlgorithm (1)).

• Develop the required calculus to implement the newiterative solver taking advantage of the optimization pro-totyping framework MANOPT [13] (in Section (VI)).

Extensive experiments with both synthetic and real data,covering a broad set of problem regimes, support the claimsof this paper and show that our proposed pipeline performsexcellently on synthetic data, consistently reaching the optimalsolution with few iterations of the iterative solver when ini-tialized with the 8pt algorithm and certifying this optimality,for all but a few exceptional (0.53%) cases among all testedproblem instances. The preliminary results on real data show asatisfactory performance, while still leaving margin for futureimprovement. Note that, although this empirically supportsthat the strong duality condition usually holds and that theproposed relaxed formulation for the Relative Pose problemis indeed tight, a formal demonstration is not available (yet).

Finally, please notice that, although the proposed pipelineestimates the essential matrix, the relative pose (rotation andtranslation) can be recovered from it by classic ComputerVision algorithms [6].

II. RELATED WORK

A. Minimal Solvers

The essential matrix has five degrees of freedom (threefrom 3D rotation, three from 3D translation and one less fromthe scale ambiguity) and therefore, only five correspondences(except for degenerate cases [6]) are required for its estimation.This is the so-called minimal problem and since it providesus with an efficient hypothesis generator, it can be embeddedinto RANSAC paradigms to gain robustness against wrongcorrespondences, i.e. outliers [5], [14]. In this context, differentworks [15], [16] have reported efficient algorithms to solvethis minimal problem, although they involve nontrivial (tenthdegree) polynomial systems which are commonly solved bymethods based on polynomial ideal theory and Gröbner basis,which are not always numerically robust [17]. Alternativeapproaches have tried to overcome this instability, such as

[18], where it was proposed an eigenvalue-based method, morestable than state-of-art approaches [15], [16] for the 5 and 6points algorithms.

B. 8-point Algorithm

The seminal work of Longuet-Higgins in [19] showed forthe first time that the relative pose between two calibratedviews is encoded by the essential matrix and proposed the(linear) 8-point (8pt) algorithm, which led to many linear andnonlinear algorithms, among others [20], [21], [22]. Despitebeing designed for the fundamental matrix estimation, the8pt algorithm can be adapted to the essential matrix i.e., forcalibrated cameras. Special attention must be given to the cel-ebrated normalized 8-point algorithm proposed by Hartley in[20]. However, in both cases the solution is not guaranteed tobe an essential matrix [6], but an approximation. Nevertheless,due to its simplicity, the 8pt algorithm can be considered asthe state-of-the-art initialization for further refinements.

C. Iterative Optimization on the Essential Matrix Manifold

Minimal solvers or the 8-point algorithm typically providesuboptimal solutions for the non-minimal N-point problemand therefore it is a common practice to refine these initialestimates by local, iterative methods [14]. Contrary to op-timization problems on flat (Euclidean) spaces, these localoptimization methods must respect the intrinsic constraints ofthe search space. In this context, the essential matrix manifoldhas been characterized via different, yet (almost) equivalentformulations. As it was shown in [21], these parameterizationsmay lead to different performances and convergence rates fornon-linear optimization methods. In [23], an iterative methodbuilt upon the 5 points estimation, which directly solvesfor the relative rotation, was proposed, achieving frame-ratespeed. In [22] it was proposed the refinement of the initialestimation from the 8pt algorithm on the manifold of theessential matrices, although the approach only converges ina small neighborhood of the true solution for their chosenmanifold parameterization. Helmke et al. [21] improved theconvergence properties of the iterative solver by proposinga different parameterization of said manifold. Recently, Tronand Daniilidis [24] present a characterization of the essentialmatrix manifold as a quotient Riemannian manifold whichtakes into account the symmetry between the two views andthe peculiarities of the epipolar constraints. Interestingly, thereported optimization instances were able to converge in fiveiterations.

D. Globally Optimal Solvers

Despite its attractive as fast solvers, the above-mentionedproposals do not guarantee nor certify if the retrieved solutionis optimal. In fact, finding said guaranteed optimal solutionsfor non-convex problems, such as the Relative Pose problem,is in general a hard task. In [25], the authors extended theapproach in [24] to incorporate the presence of outliers asan inlier-set maximization problem, which was solved inpractice via Branch-and-Bound global optimization. In [26],

it was first proposed the estimation of the essential matrixunder a L∞ cost function by Branch-and-Bound. In [5], aneigenvalue formulation equivalent to the algebraic error wasproposed and solved in practice by an efficient Levenberg-Marquardt scheme and a globally optimal Branch-and-Bound.Nonetheless, Branch-and-Bound presents slow performanceand exponential time in worst-case scenarios.

A different approach which also certifies the optimality ofthe solution a posteriori, relies on the re-formulation of theoriginal problem as a Quadratically Constrained QuadraticProgram (QCQP). QCQP problems are still in general NP-hardto solve. However, one can relax this QCQP into a Semidefi-nite Relaxation Program (SDP) by Shor’s relaxation [8], [27]and solve this SDP by off-the-shelf tools in polynomial time. Ifthe convex relaxation happens to be tight, one can recover thesolution to the original problem with an optimality certificate.This was the approach followed in [9], where it was shownfor the first time that the non-minimal (epipolar) RelativePose problem for calibrated cameras could be formulated as aquadratic (QCQP) problem whose Shor’s relaxation resulted inan empirically tight (SDP) convex relaxation. Beyond its valueas a proof-of-concept convex approach, the QCQP formulationchosen by the authors there resulted in a quite large SDPproblem, which led to computation times of 1 second witha MATLAB implementation using SDPT3 [28] as solver. Asubsequent contribution by Zhao [10] follows a similar ap-proach and proposes an alternative (still equivalent) QCQPformulation, featuring a much smaller number of variablesand constraints (with the essential matrix at its core). ApplyingShor’s relaxation to this formulation results in a much smaller,although not always tight, SDP relaxation with 78 variablesand 7 constraints. Thanks to the significantly smaller size ofthe SDP problem, a C++ based implementation and by fine-tuning an off-the-shelf solver like SDPA [29] they attain anefficient solver with times around 6ms. This is fast by SDPsolver standards, but not as fast as desirable by real-timeComputer Vision standards [2], [30].

Although tractable, solving these convex problems fromscratch may not be the most efficient way to obtain a solution.As an alternative approach one may found the so-called FastCertifiable Algorithms recently characterized and motivatedin [7], [31]. These algorithms typically leverage the existenceof an optimality certifier which, given the optimal solutionobtained by any means, may be able to compute a (dual)certificate of optimality from it. A straightforward approachto get such dual certificate relies on the resolution of thedual problem [8] from scratch, whose optimal cost valuealways provides a lower bound on the optimal objective forthe original problem. In many real-world problem instancesthis bound is tight, meaning both cost values are the sameup to some accuracy and one can certify optimality from it.However, this naive approach would still be as slow as directlysolving the problem via its convex relaxation.

On the other hand, in the context of Pose Graph Op-timization it has been shown that the (potentially optimal)candidate solution to the original problem can be leveraged to

obtain a candidate dual certificate in closed-form, providinga much faster way to solve the dual problem [12], [11],[32]. This enables a fast optimality verification approach withwhich we can augment fast iterative solvers with no guaranteesinto Fast Certifiable Algorithms [33], [34] while maintainingtheir efficiency. For the Relative Pose problem, though, suchstandalone fast optimality certifier has not been proposed yet,as none of the SDP relaxations previously proposed [9], [10]allow for computing a dual candidate in closed-form as thereexists for the Pose Graph Optimization case.

III. NOTATION

In order to make clearer the mathematical formulation inthe paper, we first introduce the notation used hereafter. Bold,upper-case letters denote matrices, e.g. E,C; bold, lower-casedenotes (column) vector, e.g. t,x; and normal font letters de-note scalar, e.g. a, b. We reserve λ for the Lagrange multipliers(Section (V)) and µ for eigenvalues. Additionally, we willdenote with Rn×m the set of n × m real-valued matrices,Sn ⊂ Rn×n the set of symmetric matrices of dimension n×nand Sn+ the cone of positive semidefinite (PSD) matrices ofdimension n × n. A PSD matrix will be also denoted by �, i.e., A � 0 ⇔ A ∈ Sn+. We denote by ⊗ the Kroneckerproduct and by In the (square) identity matrix of dimensionn. The operator vec(B) vectorizes the given matrixB column-wise. We denote by [t]× the matrix form for the cross-productwith a 3D vector t = [t1, t2, t3]T , i.e., t× (•) = [t]×(•) with

[t]× =

0 −t3 t2t3 0 −t1−t2 t1 0

. (1)Last, we employ the subindex R across the board to indicate arelaxation of the element w.r.t. the element without subindex.For example, we denote by ER the set that is a relaxation ofE and therefore, a superset of the latter, i.e. ER ⊃ E.

IV. RELATIVE POSE PROBLEM FORMULATION

We consider the central calibrated Relative Pose problemin which one seeks the relative rotation R and translationt between two cameras, given a set of N pair-wise featurecorrespondences between the two images coming from thesedistinctive viewpoints. In this work, the pair-wise correspon-dences are defined as pairs of (noisy) unit bearing vectors(fi,f

′i) which should point from the corresponding camera

center towards the same 3D world point. A traditional wayto face this problem is by introducing the essential matrixE [19], [6], a 3× 3 matrix which encapsulates the geometricinformation about the relative pose between two calibratedviews. The essential matrix relates each pair of correspondingpoints through the epipolar constraint fTi Ef

′i = 0, provided

observations are noiseless. With noisy data, however, theequality does not hold and fTi Ef

′i = �i defines what is called

the algebraic error.In the literature one can find previous works which exploit

this relation and seek the essential matrix E that minimizesthe squared algebraic error �2 and its variants [22], [10], [21].

We will follow this approach and address the Relative Poseproblem as an optimization problem. The cost function can bewritten as a quadratic form of the elements in E by definingthe positive semi-definite matrix S9+ 3 C =

∑Ni=1Ci, with

Ci = (f′i ⊗ fi)(f ′i ⊗ fi)T ∈ S9+. Formally, the Relative Pose

problem reads:

f? = minE∈E

N∑i=1

(fTi Ef′i)

2

︸ ︷︷ ︸f(E)

= minE∈E

vec(E)TCvec(E). (O)

See the Supplementary material for a formal proof of thisequivalence.

A. The Set of Essential Matrices

E above stands for the set of (normalized) essential matrices,typically defined as

E .= {E ∈ R3×3 | E = [t]×R,R ∈ SO(3), t ∈ S2}. (2)

Note that in (2) the translation is identified with points inthe 2-sphere S2 .= {t ∈ R3|tT t = 1} since the scale cannot berecovered for central cameras [6]. The rotation is representedby a 3 × 3 orthogonal matrix with positive determinant R ∈SO(3) and SO(3) .= {R ∈ R3×3|RTR = I3, det(R) = +1}.

Other equivalent parameterizations are possible for thisset [24], [21]. E.g. Faugeras and Maybank [35] proposed:

E .= {E ∈ R3×3 | EET = [t]×[t]T×, tT t = 1}. (3)

This parameterization, recently leveraged by Zhao in [10],features a lower number of variables (12) and constraints (7),yet it provides excellent results in the context of buildingSDP relaxations for the relative pose problem [10], resultingin a smaller problem than relaxations based on previousformulations (2) [9].

B. Relaxed Formulation of the Relative Pose Problem

Despite its advantages, the parameterization by Faugerasand Maybank (3) above still does not allow for the devel-opment of a fast optimality certifier for the Relative Poseproblem in the fashion of that proposed e.g. for Pose GraphOptimization in [11], [32]. To attain this kind of certifiers,it will be necessary to leverage a relaxed version ER of theessential matrix set E:

E ⊂ ER.= {E ∈ R3×3 | hi(E, t) = 0,∀hi ∈ CR; t ∈ R3},

(4)with CR the relaxed constraint set defined as

CR ≡

h1 ≡ tT t− 1 = 0h2 ≡ eT1 e1 − (t22 + t23) = 0h3 ≡ eT2 e2 − (t21 + t23) = 0h4 ≡ eT3 e3 − (t21 + t22) = 0h5 ≡ eT1 e3 + t1t3 = 0h6 ≡ eT2 e3 + t2t3 = 0

, (5)

where we have denoted the rows of E by ei ∈ R3,∀i ∈{1, 2, 3}.

These are almost the same constraints used by Zhao in [10],but we dropped the constraint eT1 e2 + t1t2 = 0. Even thoughthis constraint set differs from that by Faugeras and Maybankby just one constraint (6 versus 7), it turns out this difference isinstrumental to eventually enable our fast optimality certifier,as motivated later in Section V-A. A formal proof of howER in (4) defines a strict superset of E is provided in theSupplementary material.

With this relaxed set at hand, we define a relaxed version(R) of the original Relative Pose problem (O):

f?R = minE∈ER

vec(E)TCvec(E). (R)

Problem (R) is a relaxation of the original Problem (O) andtherefore the inequality f?R ≤ f? holds, with equality onlyif the solution to (R) is also an essential matrix, and hencefeasible for (O).

Interestingly enough though, we observed that equalityholds (f?R = f

?) in many problem instances in practice,meaning that the relaxed problem (R) is very often a tightrelaxation of the original problem (O). We have no theoreticalproof as to why the behavior above holds so often, and oursupport to this claim is fundamentally empirical (given byextensive experiments in Section (VII)).

There exists other examples in the literature of problemrelaxations that remain tight for common instances, such as therelaxation of SO(3) onto O(3) in the context of Pose GraphOptimization [11], [32]. Yet, it is remarkable that whereasO(3) ⊃ SO(3) has two disjoint components, ER ⊃ Ehere features a single connected component, which makes theobserved behavior less expectable.

C. Relaxed QCQP Formulation

We can now re-formulate our relaxed optimization prob-lem (R) as a canonical instance of QCQP by writing explicitlythe constraints in CR. Let us define for convenience the 9Dvector e = vec(E) ∈ R9 and the 12-D vector x = [eT , tT ]T .The relaxed canonical QCQP formulation employed in thiswork for the Relative Pose problem (also referred to as theprimal problem) is

f?R = minx∈R12

xTQx

subject toxTA1x = 1

xTAix = 0, i = 2, ..., 6(P-R)

where {Ai}6i=1 are the 12 × 12-symmetric correspondingmatrix forms of the quadratic constraints, so that hi(E, t) ≡xTAix − ci = 0, ci ∈ R, and Q is the 12 × 12-symmetricdata matrix of compatible dimension with x, i.e.

Q =

[C 09×3

03×9 03×3

]∈ S12+ . (6)

Problem (P-R) is exactly equivalent to the relaxed Problem(R). Nonetheless, Problem (P-R) is still a Quadratically Con-strained Quadratic Program (QCQP), in general NP-hard to

solve. However, it allows us to derive an optimality certifierby exploiting the so-called Lagrangian dual problem, whichwe present next.

V. FAST OPTIMALITY CERTIFIER

Our interest in the dual problem is twofold. First, the dualproblem presents a relaxation of the primal program (P-R)and hence provides a lower bound for the optimal objectiveof the latter, principle known as weak duality [8]. In manysituations, as it is shown in Section VII-A, this relaxation isexact, meaning that the optimal objective of the dual (d?R) andprimal (f?R ) problems are the same up to some accuracy. Whenthis occurs, we say there is strong duality and that the dualitygap f?R − d?R is zero. Second, when strong duality holds, wecan recover the primal solution from the dual and vice versa(assuming some conditions hold), without actually solving theprimal (or dual) problem.

Definition V.1 (Dual problem of the Primal program (P-R)).The Dual problem of the program in (P-R) is the followingconstrained SDP:

d?R = maxλ

λ1 (D-R)

subject to M(λ) � 0

whereM(λ) .= Q−∑6i=1 λiAi is the so-called Hessian of the

Lagrangian and λ = {λi}6i=1 are the Lagrange multipliers.The derivation of this problem is given in the Supplementarymaterial.

A classic duality principle relates the objectives attained by(P-R) and (D-R) as the chain of inequalities [8]

dR(λ̂) ≤ d?R ≤ f?R ≤ fR(x̂), (7)

where we employ λ̂, (resp. x̂) to denote dual (resp. primal)feasible points, i.e., points which fulfill the constraints requiredby the dual (D-R) (resp. primal (P-R)) problem. The first andthird inequalities hold by definition of optimality, while thesecond stands for the weak duality principle. Further, for anyessential matrix Ê ∈ E the following chain of inequalitiesalways holds:

dR(λ̂) ≤ d?R ≤ f?R ≤ f? ≤ f(Ê), (8)

where the first two inequalities come from (7), the relationf?R ≤ f? stands due to the fact that (P-R) is a relaxationof (O) and the inequality f? ≤ f(Ê) holds by definition ofoptimality.

Therefore, the dual problem allows us to verify if a givenprimal feasible point x̂ is indeed optimal. Although the dualproblem (D-R) is a SDP and can be solved by off-the-shelfsolvers (e.g., SeDuMi [36] or SDPT3 [28]) in polynomial time,inspired by [11], [32] we propose here a faster optimality veri-fication based on a closed-form expression for dual candidates,thus avoiding the resolution of the SDP from scratch.

(Original problem) Eq.(17)Refinement

&

Verification of optimality

Alg.(I)

Proposed certifiable pipeline

Closed-form for dual point

Eq.(13)

Sec. (5)

Sec. (6)

Sec. (5)Initialization

e.g.,

(RANSAC +) 8pt

Sec. (6)

Primal QCQP problem

(P-R)

Lagrangian duality

Sec. (4)

Dual SDP problem

(D-R)

Sec. (5)

2,...,6

Relaxed constraint set

KKT &

LICQ

Dual feasible point

Direct application

Underlying technical contributions

Correspondence pairs {fi', fi}

Solution: essential matrix Optimality certificate

i=1

N

R

R

Fig. 3: Proposed certifiable pipeline: its user-supplied input Correspondence pairs {f ′i ,fi}Ni=1; and two outputs (essentialmatrix and optimality certificate). We show the underlying formulations (primal and dual problems), along with the novelclosed-form expression for potential dual points λ̂ given a potential optimal primal point x̂. This dual point is not providedby the user.

A. Closed-form Expression for Dual Feasible Points

Following, we show how to compute dual feasible points(candidates) in closed-form. Assuming strong duality holds,we know from duality theory [8] that a primal optimal point x?

is a minimizer of the dual problem (D-R), that is, a minimizerof the Lagrangian evaluated at the dual optimal point λ?.SinceM(λ) is positive semidefinite for any feasible dual pointλ, by definition xTM(λ)x ≥ 0 for any 12D vector x and itsminimum value is achieved at 0. Next, we can re-formulatethis requirement as:

x?TM(λ?)x? = 0⇔M(λ?)x? = 012×1, (9)

that is, x? lies in the nullspace of M(λ?). This is known asthe complementary slackness condition [8], which provides uswith a set of linear constraints relating the optimal values forthe primal and dual variables (always under the assumption ofstrong duality).

With this in mind and recalling the structure of M(λ) in(V.1), we re-write (9) as

012×1 = M(λ?)x? = (Q−

6∑i=1

λ?iAi)x? ⇔ (10)

⇔ Qx? =6∑i=1

λ?iAix?. (11)

We stack the 12D vectors {Aix?}6i=1 column-wise in the12× 6 matrix J(x?) and the Lagrange multipliers as the 6Dvector λ?, obtaining the following linear system w.r.t. λ?:

J(x?)

12×6

λ?

6×1= Qx?

12×1

. (12)

(12) enables us to compute a candidate dual solution λ̂given a potential primal solution x̂. With this, we can certifyif the given (feasible) solution x̂ is indeed optimal through thefollowing Theorem:

Theorem V.1 (Verification of Optimality). Given a putativeprimal solution x̂ for Problem (P-R), if there exists a uniquesolution λ̂ to the linear system

J(x̂)λ̂ = Qx̂, (13)

and M(λ̂) � 0, then we have strong duality and the putativesolution x̂ is indeed optimal.

Proof. Assume λ̂ is a solution of (13) and dual feasible. Then,by (9): M(λ̂)x̂ = 012×1 ⇔ x̂TM(λ̂)x̂ = 0. Given thedefinition of M(λ̂) in (V.1) and the quadratic constraints in(P-R),

x̂TQx̂ = x̂T6∑i=1

(λ̂iAi

)x̂ =

= λ̂1

(x̂TA1x̂

)= λ̂1 ⇔ fR(x̂) = dR(λ̂), (14)

which implies that the chain on inequalities given in (7) istight: d(λ̂) = d?R = f

?R = fR(x̂), that is, the primal candidate

x̂ achieves the optimal objective fR(x̂) = f?R , therefore it isthe optimal solution and we have strong duality.

From Theorem (V.1), the next statement follows:

Corollary V.1.1. Given a potentially optimal solution Ê forproblem (O) and its equivalent form x̂ = [vec(Ê)T , t̂T ]T

where t̂ is the associated translation vector, if there exists aunique solution λ̂ to the linear system in (13) and it is dualfeasible (i.e. M(λ̂) � 0), then we can state that: (1) strongduality holds between problems (P-R) and (D-R); (2) therelaxation carried out in (P-R) is tight; and (3) the potentiallyoptimal solution Ê is optimal for both (P-R) and (O).

Proof. Since Ê is feasible for (O), it is also a primal feasiblepoint for (P-R) since the feasible set of (P-R) is a relaxationof the set in (O). Therefore, we can apply Theorem (V.1) tothe feasible point x̂ considering it as a potentially optimalsolution for (P-R). If there exists a unique dual feasible pointλ̂, by Theorem (V.1) the chain of inequalities in (7) is tight,which implies that strong duality holds (statement (1) of thecorollary). Further, since the objective functions of (P-R) and(O) are equivalent ∀E ∈ E, the attained cost values agreef(Ê) = fR(x̂) and the chain of inequalities in (8) becomesalso tight: dR(λ̂) = d?R = f

?R = f

? = fR(x̂) = f(Ê),which implies that the relaxation is tight f?R = f

? (provingthe statement (2)) and that the same solution is optimal forboth problems f?R = f

? = f(Ê) = fR(x̂) (statement (3) inthe corollary).

Before we continue, we want to point out that Theorem(V.1) and Corollary (V.1.1) can only either certify the givenprimal solution is indeed optimal, or it is inconclusive aboutits optimality. In the latter case it might be that the solutionis suboptimal or that the chosen dual problem is not tight.

That being said, notice that Theorem (V.1) requires theexistence of a unique solution to the system (13), that is, itrequires the existence and uniqueness of the Lagrange multi-pliers. However, this is not necessary to obtain strong duality,and there exist other conditions under which the problemhas it, while still being the first-order optimality conditions(KKT) satisfied by the pair primal/dual optimal points [8,Sec. 5.5]. Nevertheless, this uniqueness is the cornerstone ofour optimality certifier. Luckily, both the existence and theuniqueness of the Lagrange multipliers are assured by theregularity condition or constraint qualification (CQ) knownas Linear Independence Constraint Qualification (LICQ) [37,Sec. 12.2], [38]. The following theorem adapts the LICQ toour primal non-convex problem (P-R).

Theorem V.2 (LICQ for the primal problem (P-R)). We saythat the Linear Independence Constraint Qualification (LICQ)holds at a primal feasible point x̂ ∈ R12 for (P-R) with the setof 6 (differentiable) equality constraints {ci−x̂TAix̂ = 0}6i=1if

rank(∇(1− x̂TA1x̂), . . . ,∇(−x̂TA6x̂)

)= 6, (15)

where ∇(f(x)) denotes the gradient of the function f(x) w.r.t.x.

Therefore, LICQ assures that the Lagrange multipliers areunique if and only if the gradients of the active set of con-straints (all the equality constraints in (P-R) for our problem)are linearly independent or equivalently, the Jacobian −2J(x̂)of these constraints evaluated at the feasible point x̂,

R12×6 3 [∇(1− x̂TA1x̂), . . . ,∇(−x̂TA6x̂)] == −2[A1x̂, . . . ,A6x̂] = −2J(x̂), (16)

is full (column) rank.

As introduced before, the analysis of the dual problem(D-R) and concretely, the linear system in (13), allowed usto detect the constraint in the original set employed in [10]that blocked the development of our fast optimality certifier.While we include the full analysis in the Supplementarymaterial, we briefly sketch the main conclusions here. Theset in [10] generates a 12× 7 Jacobian matrix with rank 6 forall feasible primal points, yielding to a pencil of solutions tothe linear system in (13); that is, the solution is not unique andneither LICQ nor Theorem (V.1) hold. This rank deficiency iscorrected by eliminating one of the constraints associated withthe expression EET in (3), leading to the Jacobian matrixin (16) which is a scaled version of the coefficient matrixJ(x) ∈ R12×6 in (13). The system becomes fully determinedand over-constrained in all scenarios (proof is given in theSupplementary material) and either 1 or 0 solutions exist. Inpractice, and mainly due to numerical errors, the exact solutionmay not exist, i.e. the vector Qx̂ does not lie on the rangespace of J(x̂). In these cases, one can always compute the“closest” solution in the least-squares sense.

Note that if we drop more constraints, we can still de-rive a closed-form expression for dual candidates and a fastcertification algorithm. However, for each constraint that wediscard we are creating an even looser relaxation of the originalproblem (O). At some relaxation, the dual problem may notprovide with a tight lower bound on the original problem,and hence, the certifier associated to that relaxation will notdetect the optimal solution. Formally, let us consider a relaxedset of the essential matrix Ei, which differs from the originalone E in i constraints and, with the same notation, an evenlooser relaxed set Ej , with 5 > j > i ≥ 1. Let us denoteby f?i , f

?j the optimal costs of the associated minimization

problems (akin to (R)). Since Ej is a superset of Ei, bythe same reasons our relaxed set ER was a superset of E,the problem with Ej as feasible set is a relaxation of theproblem with Ei, and we have that d?j ≤ f?j ≤ f?i ≤ f?always. We can then apply Corollary (V.1.1) to the relaxationwith Ej . See that if the relaxed problem has strong dualityfor an essential matrix E, then all the tighter relaxations(more constraints, i.e. sets Ei) will also have strong duality for

this same problem 1; the contrary is not true. The conditionis then sufficient but not necessary. This can be extendedto the performance of the associated certifier. In practice,when we discard some constraints, we are only changingthe form of the closed-form expression for the candidatesto dual points (equation (13)), and the associated Hessian ofthe Lagrangian. While the computation cost for this certifiercould be potentially lower, we do not know a priori whichrelaxation performs better in terms of certification of essentialmatrices. A study of the performance of each relaxation is outof the scope of this paper. In summary, while looser relaxationscan still be leveraged to develop similar certifiers, in termsof number of detected optimal solutions they will generallyperform worse than tighter relaxations; the tightest relaxationfor the parameterization of the essential matrix set employedin this work that still assures that a closed-form expressionexists with unique solution is obtained by dropping only oneconstraint.

In practice, one can certify the optimality of a given primalfeasible point x̂ for (P-R) and, by Corollary (V.1.1), of agiven feasible solution Ê for (O) by following in both casesAlgorithm (1). To universalize the Algorithm, let us denoteby x̂ = [vec(Ê)T , t̂T ]T the feasible solution for (P-R) or for(O). Further, for any Ê ∈ E, the attained objective value in (O)(f(Ê)

)agrees with the objective value in (P-R)

(fR(x̂)

)since

E ⊂ ER; hence we employ fR(x̂) to denote the correspondingobjective value in both cases without confusion. Recall thatour certification has two possible outcomes, either POSITIVE(the solution is optimal) or UNKNOWN (the certification isinconclusive). From a practical point of view, we write thecondition M(λ̂) � 0 as its smallest eigenvalue µM beinggreater than a negative threshold τµ and assure strong dualityby applying a (positive) threshold τgap to the absolute valueof the dual gap |fR(x̂) − dR(λ̂)|, which allow us to accom-modate numerical errors. In practice, we fix the tolerancesto τµ = −0.02 and τgap = 10−14. If either the minimumeigenvalue is negative and/or the dual gap is greater thanzero (considering the tolerance), the verification procedure isinconclusive. This could occur if the solution is not optimalor if strong duality does not hold for this particular probleminstance and/or the chosen relaxation.

VI. PROPOSED FAST CERTIFIABLE PIPELINE

Rather than solving the original problem via its convexSDP relaxation, in this work we propose to solve the relativepose problem through an iterative method that respects theintrinsic nature of the essential matrix set, but comes with nooptimality guarantees, and to certify a-posteriori the optimalityof the solution leveraging our fast optimality certifier. Currentiterative methods work well and usually converge to the global

1See that the constraints that were not present in Ej can be seen asredundant constraints in Ei for this problem since the certified optimal solutionE (an essential matrix) for the problem with Ej trivially satisfied the rest ofthe constraints. It has been shown in the literature, see e.g. [9], [31] thatredundant constraints tighten the dual problem. Since the relaxation (Ej ) isalready tight, it follows that the “redundant“ relaxation with Ei is also tight.

Algorithm 1: Verification of Optimality1 Input: Compact data matrix C; putative primal

solution x̂2 Output: Optimality certificate ISOPT∈ {True, unknown}

3 Compute fR(x̂) from (P-R);4 Compute λ̂ by solving the linear system in (13) and

set dR(λ̂) = λ̂1;5 Compute min. eigenvalue µM of M(λ̂);6 if µM ≥ τµ and |fR(x̂)− dR(λ̂)| ≤ τgap then7 ISOPT = True;8 else

// Dual candidate is not feasible9 ISOPT = unknown;

optima despite initialization, in addition to be faster thanthe methods employed in convex programming, e.g., InteriorPoint Methods (IPM). Following, we enumerate and brieflyexplain the three major stages in which the proposed pipelineis separated (see also Figure (3) for a graphic representation):

1) Initialization: One starts by generating an initial guesswith any standard algorithms, e.g., (RANSAC + ) 8ptalgorithm [6], or simply providing the trivial identitymatrix or a random essential matrix.

2) Refinement (Optimization on Manifold): We seekthe solution to the original primal problem (O) byrefining the initial guess with a local iterative methodthat operates within the essential matrix manifold ME(always fulfilling constraints):

Ê = arg minE∈ME

vec(E)TCvec(E). (17)

3) Verification of optimality: The candidate solution Êreturned by the iterative method can be verified asglobally optimal with the proposed Algorithm (1) if theunderlying dual problem (D-R) is tight.

Note that the above-mentioned pipeline estimates the essentialmatrix, which encodes the relative pose. Both the rotation andtranslation up-to-scale can be extracted from it by any classiccomputer vision algorithm [6].

A. Implementation Details about the Optimization on Mani-fold Stage

Riemannian optimization toolboxes, such as MANOPT, de-couple the optimization problem into manifold (domain),solvers and problem description, making it quite straightfor-ward to implement an iterative solver for (17) as proposedabove:

Solver We choose here an iterative truncated-Newton Rie-mannian trust-region (RTR) solver [39]. RTR has shownbefore [33], [34] a very good trade-off between a large basinof convergence and superlinear convergence speed.

Manifold As mentioned earlier in Section (II), many char-acterizations have been proposed in the literature for theessential matrix manifold. Here we pick the state-of-the-art proposal from Tron [24]. This characterization and itsassociated operators are already provided by MANOPT asESSENTIALFACTORY.

Problem Hence we only need to specify the cost functionand its Euclidean derivatives (gradient ∇f(E) and Hessian-vector product ∇2f(E)[U ]) which only depend on E:

f(E) = vec(E)TCvec(E) (18)∇f(E) = 2Cvec(E) (19)

∇2f(E)[U ] = 2Cvec(U). (20)

We provide these developments in the Supplementary materialfor completeness.

VII. EXPERIMENTAL VALIDATION

In this Section we validate through extensive experimenta-tion with synthetic and real data the utility of the presentedverification technique (Algorithm (1)) and the performance ofthe proposed certifiable pipeline (Section (VI)).

A. Experimental Validation with Synthetic Data

Similarly to [9], we generate random problems as follows:We place the first camera frame at the origin (identity orien-tation and zero translation) and generate a set of random 3Dpoints within a truncated pyramid (frustum) zone with depthranging from one to eight meters (approx. from 75 to 615focal units) measured from the first camera frame and insideits Field of View (FoV). Then, we generate a random pose forthe second camera whose translation parallax is constrainedwithin a spherical shell with minimum radio ||t||min, maximumradio ||t||max and centered at the origin. We also enforcethat all the 3D points lie within the second camera’s FoV.This configuration is closer to those found in real scenarios.Next, we create the correspondences as unit bearing vectorsand add noise by assuming a spherical camera, computingthe tangential plane at each bearing vector and introducing arandom error sampled from the standard uniform distribution,considering a focal length of 800 pixels for both cameras andimages of approx. 1900 pix . The principal point is placedat the center of the image plane for both cameras.

In the following, we will fix the FoV to 100 degrees and thetranslation parallax ||t||2 ∈ [0.5, 2.0] (meters). We generatefour sets of experiments, each of them with a different levelof noise σnoise ∈ {0.1, 0.5, 1.0, 2.5} pix. Further, for eachlevel of noise, we consider instances of the Relative Poseproblem with different number of correspondence pairs inN ∈ {8, 9, 10, 11, 12, 13, 14, 15, 20, 40, 100, 200}. We gener-ate 500 random problem instances for each combination ofnoise/number of correspondences.

Effectiveness of the Verification Algorithm. First, weshow the effectiveness of the verification technique in Theorem(V.1) under the setup provided by the above-mentioned foursets of experiments, which consider different combinations

of noise/number of correspondences for the Relative Poseproblem. For each instance, we compute the optimal solutionx? by solving2 the SDP relaxation of (O) as it was executed in[10] and consider the returned solution as optimal if the rankguarantees the tightness of the relaxation. If the relaxation isnot tight, we treat the solution as suboptimal x̂ and project itonto the essential matrix set to obtain a primal feasible point(we refer the reader to the original work for further details).We detected in 343 out of 24000 occasions that the relaxationwas not tight. Further, for each instance of the problem we alsocompute the essential matrix by the 8pt algorithm and treatit as suboptimal. Next, we apply the verification technique inTheorem (V.1) for each instance and compute the well-knownclassification metrics precision and recall. Let us denote theglobally optimal x? classified as such by TP (True Positive),the suboptimal point x̂ classified as optimal by FP (FalsePositive) and the true optimal that couldn’t be verified (theverification was inconclusive) by FNP (False Non-Positive)3.Hence, the classification metrics take the form:

precision =TP

TP + FP, recall =

TP

TP + FNP. (21)

Figure (4a) depicts the metrics for the four sets of experimentsas a function of the number of points. As expected, the preci-sion is always 1 while the recall decreases with the number ofcorrespondences when the noise level is high [1.0, 2.5] pix,but remains stable (above 95 %) with the typical level of noise[0.1, 0.5] pix. Note that the set of experiments with noise0.1 pix attains also a recall metric of one for all the testsregardless the number of correspondences considered.

Further Experiments on Effectiveness: Focal LengthWith the default set of parameters, we let the image sizeconstant (approx. 1900 pix ) and modify the focal length ofboth cameras, selecting values from f ∈ {300, 400, 500, 700}pix ; the FOV of the camera is changed accordingly to(approx.) FOV ∈ {145, 134, 124, 107} degrees, respectively.For each focal length, we generate random problem instanceswith the same range of correspondences employed before.We then compute the precision and recall metrics for theseexperiments w.r.t. the solution obtained from the SDP solver.Figure (5) depicts the results, where we can observe thatvarying the focal length in this way leads to similar resultsthan changing the noise level. For the smallest focal length(f = 300 pix) we obtain the worst results, but we are still ableto certify 90% of the optimal solutions. For all the cases, weobtain a precision of 100%, which is not shown in the graphicfor clarity. We perform similar experiments with constant FOVand varying focal length, hence changing the image size. Forthe same set of values for the focal length, we obtain similar

2Since the original code is not publicly available, we execute our ownimplementation: The SDP was modelled in MATLAB with CVX [40] andsolved with SDPT3 [28].

3One may note that the classic classification metrics employ False Negativeinstead of False Non-Positive. However, our algorithm does not outputpositive/negative but positive/inconclusive, and hence False Non Positiveseems a better choice for the cases in which our verification technique outputsinconclusive for a known true optimal solution.

Nr. correspondences

Precisi

on/Rec

all

(a)

Nr. correspondences

(outer)

iteratio

ns

(b)

Nr. iterations

Cost va

lue (log

)

(c)

Fig. 4: (a) Precision (solid line) and recall (dashed line) metrics for the four levels of noise considered as a function of thenumber of points. (b) Averaged number of (outer) iterations for different initialization (identity matrix IDEN, random matrixRAND and 8pt algorithm 8PT) and levels of noise (0.1, 0.5, 1.0, 2.5 pix ). (c) Evolution of the cost value as a function ofthe number of iterations; an example of each instance is shown. Note the logarithmic scale in the last figure and the non-linearscale for the X axis.

8 9 10 11 12 13 14 15 20 40 100 200Nr. correspondences

0.9

0.92

0.94

0.96

0.98

1

Per

cent

age

optim

al s

olut

ions

cer

tifie

d

f:300f:400f:500f:700

Fig. 5: Recall metrics for the set of experiments with vary-ing focal length from f ∈ {300, 400, 500, 700} and defaultparameters. The precision metrics are 100% for all cases andare not shown for clarity.

results that are not included here due to space limits. Thissimilarity between focal length and noise level agrees on howwe are introducing the noise in our correspondences: we scalethe (unit norm) vectors by the focal length, and then add noisesampled from a uniform distribution, which is independent ofthe focal length. Hence, the same amount of noise has a greatereffect when the focal length is smaller.

Performance of the Proposed Certifiable PipelineWe consider the same four sets of experiments with the

different combinations of noise/number of correspondences.We feed the proposed pipeline with the initial guess estimatedby the 8pt algorithm [6] and apply the verification techniqueto the solution returned by the iterative method to certifyits optimality a posteriori. Due to space limits, we onlyshow the results for the cases with noise 0.1 and 1.0 but

include the remainders in the Supplementary material. Figure(6a) depicts the percentage of cases in which the verificationalgorithm could certify optimality as a function of the numberof points. One may note how the number of cases certified asoptimal decreases with high levels of noise and large numberof correspondences, following the tendency of the RECALLmetric in Figure (4a). As a more intuitive measurement, Figure(6b) and (6c) plot the error in rotation (in degrees) for the setof experiments. The error is measured in terms of the geodesicdistance between the estimated rotation R̂ and ground truthRgt:

�rot = arccos( tr(R̂TRgt)− 1

2

)180π

[degrees]. (22)

For completeness, under the same setup, we generate ad-ditional instances of the Relative Pose problem by fixing thelevel of noise to 0.5 pix and varying the FoV, parallax andnumber of correspondences, one at a time. While the statisticsare given in the Supplementary material due to space limits,we can sketch the following conclusions: (1) the number of(outer) iterations remains stable under five steps for all theconducted experiments when the iterative method is initializedwith the 8pt; and (2) while the number of certified optimalsolutions does not seem affected by the varying FoV, thevariation of the parallax produces a similar behavior to thechange of noise. In [22] it was remarked that, for a fixed levelof noise, a variation on the parallax is equivalent to varyingthe signal-to-noise ratio, which agrees with our results.

Further, we observe in all these cases that instances of therelative pose problem whose numbers of correspondences arecloser to the minimum (8-9) are more sensible to the noiselevel, FoV and parallax (see e.g. Figure (6c)).

Further Experiments:A good initial guess is crucial in many problems which rely

on iterative solvers in order to avoid local minima (for exam-ple, Pose Graph Optimization [12]). To analyze the sensibilityof the proposed pipeline to initialization quality, we generate

Nr. correspondences

Certifie

d optim

al solu

tions

(a)

Nr. correspondences

Error ro

t (log de

gree)

(b)

Nr. correspondences

Error ro

t (log de

gree)

(c)

Fig. 6: (a) Percentage of cases in which the algorithm could certify optimality as a function of the number of correspondencesand noise level. Error in rotation for the instances of the problem with noise 0.1 pix (b) and 1.0 pix (c). Please, note thelogarithmic scale in the Y axis in figures (b) and (c).

Nr. correspondences

Certifie

d optim

al solu

tions

(a)

Nr. correspondences

Error ro

t (log de

gree)

(b)

Nr. correspondencesErro

r rot (lo

g degree

)(c)

Fig. 7: (a) We plot the percentage of cases in which the algorithm certifies optimality for instances of the Relative Pose problemwith fixed noise 0.5 pix and different initialization. Error in rotation [degrees] for the final estimations whose refinementstages were initialized with a random guess (b) and identity matrix (c). Please, note the logarithmic scale in the Y axis infigures (b) and (c).

500 random instances of the Relative Pose problem follow-ing the above-mentioned procedure, with fixed parametersFoV = 100 (degrees), ||t||2 ∈ [0.5, 2.0](m), σ = 0.5pix andvarying the number of correspondences, as it was previouslydone. In this case, however, we feed the iterative method withthree different initial guesses: the trivial identity matrix, arandom essential matrix and the resulting estimate from the8pt algorithm. Then, we apply our verification technique toeach instance.

We plot the percentage of cases in which the verificationtechnique certified the solution as optimal as a function of thenumber for each initialization considered in Figure (7a) andthe error in rotation (degrees) in (7c), (7b) for the instancesinitialized with the identity matrix and a random guess,respectively. Figure (4b) shows the number of (outer) iterationsrequired by the RTR solver to converge. For the cases withthe estimate from 8pt algorithm as initial guess, the iterativemethod required less than five iterations to converge, whileboth the random and identity cases increase the iterations upto sixteen. We want to point out the (almost linear) decreasingtendency in the cases detected as suboptimal and the error in

rotation with the number of correspondences for the set ofexperiments with identity and random initial guesses. Further,with a large number of correspondences, the iterative methodtends to return the optimal solution and the initial guess onlyaffects the convergence rate.

Analysis of Computational Cost: The greatest advantageof our proposal against the state-of-the-art solvers [9], [10]is the low computational cost, both for the actual solver(initialization and optimization on manifold) and for the cer-tificate of optimality (Algorithm (1)). Note however that weimplement our proposal in matlab; a faster, more efficientimplementation is considered as future work. Nevertheless wecan still draw some intuition for the computational cost of ourproposal and its potential. The following times are the averagevalues for all our experiments, which were performance with astandard PC: CPU i7-4702MQ, 2.2GHz and 8 GB RAM. Thecandidate computation (solving Equation (13)) takes 0.23723milliseconds, while defining and computing the minimumeigenvalue of the Hessian goes to 0.13332 msecs. In total,considering initialization (here we employ the 8pt as standardinitialization), refinement on manifold (3 − 4 iterations till

Certifie

d optim

al solu

tions

boulde

rsexh

ibition

_hall

relief

terracestatue

playgr

ound

observ

atory

living_r

oomlounge

lecture

_room

terrain

soffi

cekick

erfaca

decou

rtyard

botani

cal_gar

denreli

ef_2

terrace

_2(a)

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Certifie

d optim

al solu

tions

boulde

rsexh

ibition

_hall

relief

terracestatue

playgr

ound

observ

atory

living_r

oomlounge

lecture

_room

terrain

soffi

cekick

erfaca

decou

rtyard

botani

cal_gar

denreli

ef_2

terrace

_2

(b)

Fig. 8: (a) Averaged percentage of cases in which the algorithm certified optimality for the eighteen different sequences withpre-filtered outlier-free correspondences and (b) embedded in a RANSAC scheme.

boulde

rsexh

ibition

_hall

relief

terracestatue

playgr

ound

observ

atory

living_r

oomlounge

lecture

_room

terrain

soffi

cekick

erfaca

decou

rtyard

botani

cal_gar

denreli

ef_2

terrace

_2

Error ro

t (degree)

(a)

Error ro

t (degree)

exhibit

ion_ha

ll

botani

cal_gar

den

boulde

rsreli

efterr

acestatue

playgr

ound

observ

atory

living_r

oomlounge

lecture

_room

terrain

soffi

cekick

erfaca

decou

rtyard

relief_

2

terrace

_2(b)

Fig. 9: (a) Error in rotation (degrees) for the eighteen different sequences with pre-filtered outlier-free correspondences anddirectly (b) embedded in a RANSAC scheme. We plot all the results from the optimization (ALL) and the errors for thosecases detected as optimal (OPT).

convergence, see Figure (4c)) and certification, together withsome other operations, e.g. projection on manifold or definitionof problem, our certifiable pipeline takes only 41 milliseconds.Since the faster certifiable solver 4 is the one proposed byZhao in [10], we only measure the computational cost ofthis proposal. The interested reader is referred to that workfor a comparison against other state-of-the-art solvers. Toperform a fair comparison, this SDP solver was implementedin MATLAB, modeled with CVX and the IPM was solved with

4Date of this document

SDPT3. As expected, the computational cost is larger andtakes around 150 milliseconds, without including modelingor other operations, such as data matrix creation, low rankdecomposition or relative pose recovery.

B. Experimental Validation with Real Data

We conclude this Section with the evaluation of the pro-posed certifiable pipeline on real data. We sample pairs of im-ages from 18 different (multi-view) sequences in the ETH3Ddataset [41], which covers both indoor and outdoor scenes and

provides with ground-truth camera poses and intrinsic cameracalibration.

To generate the correspondences, we proceed as follows.First, we extract and match 100 SURF [42] features per pairof images. Next, we obtain the corresponding bearing vectorsby using the pin-hole camera model [6] with the intrinsicparameters provided for each frame. From here, we conducttwo types of experiments with all the image pairs. Thesesets of experiments are aimed to mainly reflect: (first type)the performance of the proposed pipeline under real noise;and (second type) the importance of our novel certificationtechnique when dealing explicitly with outliers. Further, foreach image pair we execute 100 times each type of experimentto provide statistically meaningful results.

Experiments on real data with pre-filtered outliers: Wefilter outliers by removing all the correspondences whoseassociated algebraic error w.r.t. the ground truth essentialmatrix is greater than a given fixed threshold �error, i.e. weconsider as inliers all the correspondences (f ′i ,fi) such that(fTi Egtf

′i)

2 < �error. We run the proposed certifiable algorithmwith this set of outlier-free correspondences. Figure (8a) showsthe averaged percentage of cases in which we could certify anoptimal solution and, as in the previous section, we also plotthe error in rotation measured in degrees w.r.t. the providedground-truth in Figure (9a). We depict the rotation error for allthe returned solutions regardless their optimality (ALL) andonly the values for those cases detected as optimal (OPT).We certify more than 70% of image pairs as optimal in allthe sequences, which is reflected as low rotation errors inFigure (9a). We want to point out that those cases with optimalsolutions tend to attain lower errors w.r.t. the ground truthrotation.

Experiments on real data embedded in a RANSACscheme:

In this second type of experiments, we filter outliers by em-bedding the initialization stage into a RANSAC [43] schemein order to both detect the set of inliers and obtain the initialguess. Note that in this case, both the initialization and theset of employed points during the subsequent Riemannianoptimization depend on the RANSAC solution, which maynot be accurate enough. Bad RANSAC solutions may lead toan optimization problem with corrupted data (outliers remainin the inlier set) and/or missing inliers [14]. However, evenin these cases, one can still find the optimal solution to theproblem given the provided (corrupted) data. Nevertheless,in these cases it is expected that the optimal solutions mayattain large rotation errors w.r.t. the ground truth. We wantto remark that one can employ other paradigms to select theset of inliers; here we choose RANSAC as example for beingwidely employed in the literature [43].

Figure (8b) shows the averaged number of cases in whichour algorithm could certify optimality given the same set ofimage pairs and points that in the previous type of experiments,while Figure (9b) shows the error in rotation. We again depictthe error for all the cases after the optimization ALL and the

error for only those cases with certified optimal solution OPT.As expected, the rate of certified optimal solutions decreases,which is reflected as larger rotation errors (a similar behavioris depicted in [14]). Note, however, how we obtain lower errorswhen only considering the optimal solutions; we also note thatsome instances with optimal solutions still attain large errors,behavior which is expected if the estimate set of inliers byRANSAC contain outliers and/or present missing inliers, as itwas outlined above.

VIII. CONCLUSIONS AND FUTURE WORK

In this work we have proposed a formulation of the non-minimal Relative Pose problem whose dual problem admits aclosed-form solution given the primal solution. This allowsus to build a very fast a posteriori certifier for candidateprimal solutions. We have provided the first certifiable pipelinefor the Essential matrix estimation which, given a set of Ncorrespondence pairs: (1) generates the initial guess (e.g. withthe 8pt algorithm or simply with the identity matrix); (2)refines the initialization with a local, iterative method oper-ating directly on the Essential Matrix manifold; and last, (3)certifies the optimality of the returned solution with our novelcertification procedure, which employs the proposed closed-form expression for dual candidates. Extensive experiments onboth synthetic and real data under a wide variety of conditionssupport our claims. The preliminary results obtained with ourMatlab implementation show highly promising for hardeninginto a much faster C++ implementation, that we intend toexplore and benchmark in future work. Further, we contem-plate the exploration of fast dual solvers leveraging tighterrelaxations in order to tackle the failure cases the currentcertifier may present.

DECLARATION OF COMPETING INTEREST

The authors declare no conflict of interest. The fundingentities had no role in the design of the study; in the collection,analyses, or interpretation of data; in the writing of themanuscript, or in the decision to publish the results.

ACKNOWLEDGEMENTS

This work was supported by the research project WISER(DPI2017-84827-R), as well as by the Spanish grant programFPU18/01526. The publication of this paper has been fundedby the University of Malaga.

REFERENCES

[1] Bill Triggs, Philip F McLauchlan, Richard I Hartley, and Andrew WFitzgibbon. Bundle adjustment—a modern synthesis. In Internationalworkshop on vision algorithms, pages 298–372. Springer, 1999.

[2] Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: a versatile and accurate monocular slam system. IEEE transactionson robotics, 31(5):1147–1163, 2015.

[3] Ruben Gomez-Ojeda, Francisco-Angel Moreno, David Zuñiga-Noël,Davide Scaramuzza, and Javier Gonzalez-Jimenez. Pl-slam: A stereoslam system through the combination of points and line segments. IEEETransactions on Robotics, 35(3):734–746, 2019.

[4] Onur Özyeşil, Vladislav Voroninski, Ronen Basri, and Amit Singer. Asurvey of structure from motion*. Acta Numerica, 26:305–364, 2017.

[5] Laurent Kneip and Simon Lynen. Direct optimization of frame-to-frame rotation. In Proceedings of the IEEE International Conferenceon Computer Vision, pages 2352–2359, 2013.

[6] Richard Hartley and Andrew Zisserman. Multiple view geometry incomputer vision. Cambridge university press, 2003.

[7] Afonso S Bandeira. A note on probably certifiably correct algorithms.Comptes Rendus Mathematique, 354(3):329–333, 2016.

[8] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cam-bridge university press, 2004.

[9] Jesus Briales, Laurent Kneip, and Javier Gonzalez-Jimenez. A certifiablyglobally optimal solution to the non-minimal relative pose problem. InProceedings of the IEEE Conference on Computer Vision and PatternRecognition, pages 145–154, 2018.

[10] Ji Zhao. An efficient solution to non-minimal case essential matrixestimation. IEEE Transactions on Pattern Analysis and MachineIntelligence, 2020.

[11] Jesus Briales and Javier Gonzalez-Jimenez. Fast global optimalityverification in 3d slam. In 2016 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), pages 4630–4636. IEEE, 2016.

[12] Luca Carlone and Frank Dellaert. Duality-based verification techniquesfor 2d slam. In 2015 IEEE international conference on robotics andautomation (ICRA), pages 4589–4596. IEEE, 2015.

[13] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. Manopt, a Matlabtoolbox for optimization on manifolds. Journal of Machine LearningResearch, 15:1455–1459, 2014.

[14] Tom Botterill, Steven Mills, and Richard Green. Refining essential ma-trix estimates from ransac. In Proceedings Image and Vision ComputingNew Zealand, pages 1–6, 2011.

[15] Henrik Stewenius, Christopher Engels, and David Nistér. Recentdevelopments on direct relative orientation. ISPRS Journal of Pho-togrammetry and Remote Sensing, 60(4):284–294, 2006.

[16] David Nistér. An efficient solution to the five-point relative pose prob-lem. IEEE transactions on pattern analysis and machine intelligence,26(6):756–770, 2004.

[17] Zuzana Kukelova and Tomas Pajdla. Two minimal problems for cameraswith radial distortion. In 2007 IEEE 11th International Conference onComputer Vision, pages 1–8. IEEE, 2007.

[18] Zuzana Kukelova, Martin Bujnak, and Tomas Pajdla. Polynomialeigenvalue solutions to the 5-pt and 6-pt relative pose problems. InBMVC, volume 2, page 2008, 2008.

[19] H Christopher Longuet-Higgins. A computer algorithm for reconstruct-ing a scene from two projections. Nature, 293(5828):133–135, 1981.

[20] Richard I Hartley. In defense of the eight-point algorithm. IEEETransactions on pattern analysis and machine intelligence, 19(6):580–593, 1997.

[21] Uwe Helmke, Knut Hüper, Pei Yean Lee, and John Moore. Essentialmatrix estimation using gauss-newton iterations on a manifold. Inter-national Journal of Computer Vision, 74(2):117–136, 2007.

[22] Yi Ma, Jana Košecká, and Shankar Sastry. Optimization criteria andgeometric algorithms for motion and structure estimation. InternationalJournal of Computer Vision, 44(3):219–249, 2001.

[23] Vincent Lui and Tom Drummond. An iterative 5-pt algorithm for fastand robust essential matrix estimation. In BMVC, 2013.

[24] Roberto Tron and Kostas Daniilidis. The space of essential matrices asa riemannian quotient manifold. SIAM Journal on Imaging Sciences,10(3):1416–1445, 2017.

[25] Jiaolong Yang, Hongdong Li, and Yunde Jia. Optimal essential matrixestimation via inlier-set maximization. In European Conference onComputer Vision, pages 111–126. Springer, 2014.

[26] Richard I Hartley and Fredrik Kahl. Global optimization throughsearching rotation space and optimal estimation of the essential matrix.In 2007 IEEE 11th International Conference on Computer Vision, pages1–8. IEEE, 2007.

[27] Yichuan Ding. On efficient semidefinite relaxations for quadraticallyconstrained quadratic programming. Master’s thesis, University ofWaterloo, 2007.

[28] Kim-Chuan Toh, Michael J Todd, and Reha H Tütüncü. Sdpt3—amatlab software package for semidefinite programming, version 1.3.Optimization methods and software, 11(1-4):545–581, 1999.

[29] Makoto Yamashita, Katsuki Fujisawa, and Masakazu Kojima. Im-plementation and evaluation of sdpa 6.0 (semidefinite programmingalgorithm 6.0). Optimization Methods and Software, 18(4):491–505,2003.

[30] Shiyu Song, Manmohan Chandraker, and Clark C Guest. Parallel, real-time monocular visual odometry. In 2013 ieee international conferenceon robotics and automation, pages 4698–4705. IEEE, 2013.

[31] Heng Yang, Jingnan Shi, and Luca Carlone. Teaser: Fast and certifiablepoint cloud registration. arXiv preprint arXiv:2001.07715, 2020.

[32] Luca Carlone, David M Rosen, Giuseppe Calafiore, John J Leonard, andFrank Dellaert. Lagrangian duality in 3d slam: Verification techniquesand optimal solutions. In 2015 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), pages 125–132. IEEE, 2015.

[33] Jesus Briales and Javier Gonzalez-Jimenez. Cartan-sync: Fast andglobal se (d)-synchronization. IEEE Robotics and Automation Letters,2(4):2127–2134, 2017.

[34] David M Rosen, Luca Carlone, Afonso S Bandeira, and John J Leonard.Se-sync: A certifiably correct algorithm for synchronization over thespecial euclidean group. The International Journal of Robotics Research,38(2-3):95–125, 2019.

[35] Olivier D Faugeras and Steve Maybank. Motion from point matches:multiplicity of solutions. International Journal of Computer Vision,4(3):225–246, 1990.

[36] Jos F Sturm. Using sedumi 1.02, a matlab toolbox for optimization oversymmetric cones. Optimization methods and software, 11(1-4):625–653,1999.

[37] Jorge Nocedal and Stephen Wright. Numerical optimization. SpringerScience & Business Media, 2006.

[38] Gerd Wachsmuth. On licq and the uniqueness of lagrange multipliers.Operations Research Letters, 41(1):78–80, 2013.

[39] P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimizationalgorithms on matrix manifolds. Princeton University Press, 2009.

[40] Michael Grant and Stephen Boyd. Graph implementations for nons-mooth convex programs. In V. Blondel, S. Boyd, and H. Kimura, editors,Recent Advances in Learning and Control, Lecture Notes in Control andInformation Sciences, pages 95–110. Springer-Verlag Limited, 2008.

[41] Thomas Schops, Johannes L Schonberger, Silvano Galliani, TorstenSattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-cameravideos. In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, pages 3260–3269, 2017.

[42] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded uprobust features. In European conference on computer vision, pages 404–417. Springer, 2006.

[43] Martin A Fischler and Robert C Bolles. Random sample consensus:a paradigm for model fitting with applications to image analysis andautomated cartography. Communications of the ACM, 24(6):381–395,1981.

APPENDIX AEQUIVALENCE BETWEEN THE OBJECTIVES IN PROBLEM (O)

Following we prove the equality in the objective function for Problem (O).Let us express the original cost function as,

f(E) =

N∑i=1

(fi(E)

)2, (23)

where each element in the objective function is defined as,

fi(E) = fTi Ef

′i . (24)

The identities

vec(AXB) = (BT ⊗A)vec(X) (25)(aT ⊗ bT ) = (a⊗ b)T , (26)

allow us to reformulate each term as

fi(E) = (f′Ti ⊗ fTi )vec(E) = (f ′i ⊗ fi)T vec(E). (27)

The contribution of each element(fi(E)

)2is then written as(

fi(E))2

=((f ′i ⊗ fi)T vec(E)

)2= (28)

=((f ′i ⊗ fi)T vec(E)

)T(f ′i ⊗ fi)T vec(E) = (29)

= vec(E)T (f ′i ⊗ fi)(f ′i ⊗ fi)T vec(E) = (30)= vec(E)TCivec(E), (31)

where we have defined the PSD matrix Ci as

Ci = (f′i ⊗ fi)(f ′i ⊗ fi)T ∈ S9+, (32)

with ⊗ being the Kronecker product.The cost function can be therefore written as

f(E) =

N∑i=1

fi(E) =

N∑i=1

vec(E)TCivec(E) = (33)

= vec(E)T( N∑i=1

Ci)vec(E) = vec(E)TCvec(E), (34)

with C =∑Ni=1Ci.

The expression (34) can be re-formulated in terms of x = [vec(E)T , t]T ∈ R12 by defining the extended matrix Q andpadding with zeros:

Q =

[∑Ni=1Ci 09×303×9 03×3

]. (35)

The objective function is therefore expressed asf(E) = xTQx, (36)

which is the expression that appears in the primal problem (P-R).

APPENDIX BTHE RELAXATION SET ER IN (4) IS A STRICT SUPERSET OF E

We prove here that the constraint set in (4) is indeed a relaxation of the space defined in (3) and therefore E ⊂ ER. Forthat, let us define the matrix EET whose entries are given by the constraints in (4):

EET =

t22 + t23 k −t1t3k t21 + t23 −t2t3−t1t3 −t2t3 t21 + t22

, ∀E ∈ ER, t ∈ S2, k ∈ R. (37)

For E as defined above to belong to E, its determinant must be zero and so will be the determinant of EET . ApplyingLaplace’s formula and reordering:

det(EET ) = −k2(t21 + t22) + 2kt1t2t23 + 2t21t22t23 + t21t22(t21 + t22), (38)

whose roots are given by

k1 = −t1t2 (39)

k2 = t1t21 + t231− t23

. (40)

Since these conditions do not hold in general ∀k ∈ R, the matrices defined by (4) do not need to be rank deficient. Therefore,the set defined by ER is larger than E, and thus, it represents a relaxation of the latter, proving the claim of the main document.

APPENDIX CLAGRANGIAN DUAL PROBLEM OF THE PRIMAL PROBLEM (P-R)

In order to define the dual problem [8], we follow the usual procedure. This development was originally given in [10]with seven constraints instead of the six employed in this work. The adaptation is therefore trivial, but we include it here forcompleteness. We start by writing the Lagrangian function as

L(x,λ) = xTQx+6∑i=2

λi(−xTAix) + λ1(1− xTA1x) =

= xTQx+

6∑i=1

λi(−xTAix) + λ1 = xTM(λ)x+ λ1, (41)

where λ = {λi}6i=1 are the Lagrange multipliers and M(λ) is the Hessian of the Lagrangian defined as

M(λ).= Q−

6∑i=1

λiAi (42)

By definition, the Lagrangian dual problem for (P-R) is

d?R.= max

λinfxL(x,λ) = max

λinfx

(xTM(λ)x+ λ1

)︸ ︷︷ ︸

d(λ)

, (43)

where d(λ) is known as the dual function with d(λ) ≤ f for any value of λ by definition.We note that xTM(λ)x is a quadratic form whose only finite minimum value is achieved at 0 when M(λ) � 0 i.e.,

d(λ) = infxxTM(λ)x+ λ1 =

{λ1, M(λ) � 0−∞, otherwise

(44)

Since we are trying to find the maximum, we can restrict the problem to the finite values, i.e., the Hessian of the Lagrangianis positive semidefinite. Hence, we can write the (unconstrained) dual problem in (43) as the constrained SDP problem,

d?R = maxλ

λ1 (45)

subject to M(λ) � 0,

which is the problem that appears in Theorem (D-R), proving the claim.

APPENDIX DEXISTENCE AND UNIQUENESS OF THE LAGRANGE MULTIPLIERS FOR THE DUAL PROBLEM (D-R)

We want to point out that the following statements have been adapted to our concrete problem, where only equality quadraticconstraints are present. We refer the reader to [8], [37] for a full characterization of the general case.

That being said, we proceed as follows. Any pair of primal and dual optimal points must satisfy the KKT conditions [8] tohave strong duality (assuming both the objective and the set of constraints are differentiable). For the problem in (P-R) and apair of primal-dual optimal points (x?,λ?), the KKT conditions (first-order necessary conditions) read:

(1) Primal feasibility Aix? = 0,∀i = 1, ...,m (46)

(2) Stationary ∇(x?TQx?) +m∑i=1

λ?i∇(−x?TAix?) = 0, (47)

where m is the number of equality constraints.For the relation in (47) to be a necessary optimality condition, some restrictions must be applied to the equality constraints

of the primal problem. These conditions are known as constraint qualifications (CQ) [37, Sec. 12.2][38]. Many CQ’s havebeen proposed in the literature; in this work, however, we are interested in those that characterize the set of dual solutions (i.e.the Lagrange multipliers) of the dual problem. The strongest CQ is the so-called Linear Independence Constraint Qualification(LICQ), which assures both the existence and the uniqueness of the Lagrange multipliers [38], i.e. it implies that the set ofdual solutions is a singleton.

LICQ holds if the gradients of the active constraint {∇(−x?TAix?)}mi=1 are linearly independents. If LICQ holds, the dualpoint λ? that satisfies the KKT conditions is unique and exists, which is the cornerstone of our optimality certifier. We notethat weaker constraint qualifications exist which still guarantee strong duality; however they do not in general guarantee theuniqueness of Lagrange multipliers [38]. Therefore, for our optimality certifier to be purposive, we must then assure that LICQholds and list the degenerated cases. These two aspects are tackled next.

A. Assuring Linear Independence Constraint Qualification

The third KKT condition for optimality in (47) is explicitly written as the linear system in λ? as:

Qx? =

m∑i=1

λ?iAix? = JC(x

?)λ?, (48)

where we have definedJC(x

?).= [A1x

?,A2x?, . . . ,Amx

?] ∈ R12×m

the Jacobian (up-to-scale) of the constraint matrices evaluated at x?. Hence, the LICQ condition assures that if the matrixJC(x

?) is full rank, then the Lagrange multipliers λ? are unique, i.e., the linear system has either one or zero solutions. Notethat if the system has no solution, one can always find the ”closest” point in the least-squares sense.

We prove now that the original set of constraints given in [10] is not suitable for our optimality certifier as is. Considerthis original set {Ai}7i=1 and its associated Jacobian JC(x̂) evaluated at a feasible primal point x̂ = [vec(Ê)T , t̂T ]T , wherevec(Ê) = [ê1, ê4, ê7, ê2, ê5, ê8, ê3, ê6, ê9]T . JC(x̂) has the following structure:

JC(x̂) =

0 ê4/2 ê1 ê7/2 0 0 00 ê1/2 0 0 ê4 ê7/2 00 0 0 ê1/2 0 ê4/2 ê70 ê5/2 ê2 ê8/2 0 0 00 ê2/2 0 0 ê5 ê8/2 00 0 0 ê2/2 0 ê5/2 ê80 ê6/2 ê3 ê9/2 0 0 00 ê3/2 0 0 ê6 ê9/2 00 0 0 ê3/2 0 ê6/2 ê9t̂1 t̂2/2 0 t̂3/2 −t̂1 0 −t̂1t̂2 t̂1/2 −t̂2 0 0 t̂3/2 −t̂2t̂3 0 −t̂3 t̂1/2 −t̂3 t̂2/2 0

(49)

Next, we show how JC(x̂) is indeed (column) rank deficient for all primal feasible x̂. For that, we need to find a non-nullsubspace Φ(x̂) ∈ R7×r, where 7− r is the rank of JC(x̂), such that JC(x̂)Φ(x̂) = 012×r for all primal feasible point x̂. Afterthat, we will show that this nullspace is indeed one-dimensional (r = 1).

Proof that the matrix JC(x̂) is rank deficient: Let us assume that the matrix JC(x̂) is rank deficient with nullspace givenby the 7D vector Φ = [Φ1, . . . ,Φ7]T (possible dependent on the feasible point x̂, although in what follows we will avoid this

dependence on the formulation for the sake of clarity), such that JC(x̂)Φ = 012. First, recall that any essential matrix has asleft nullspace the translation vector t [6]:

E = [t]×R =⇒ tTE = tT [t]×R = −([t]T×t

)TR = 03. (50)

From (50) one can obtain the following three equalities:

t̂1ê1 + t̂2ê4 + t̂3ê7 = 0 (51)

t̂1ê2 + t̂2ê5 + t̂3ê8 = 0 (52)

t̂1ê3 + t̂2ê6 + t̂3ê9 = 0 (53)

Let us denote the rows of the matrix JC(x̂) by ji, i = 1, ..., 12. Then, the explicit expressions for JC(x̂)φ = 012 are:

j1φ = Φ3ê1 + Φ2ê4/2 + Φ4ê7/2 = 0 (54)j4φ = Φ3ê2 + Φ2ê5/2 + Φ4ê8/2 = 0 (55)j7φ = Φ3ê3 + Φ2ê6/2 + Φ4ê9/2 = 0 (56)j2φ = Φ4ê1/2 + Φ6ê4/2 + Φ7ê7 = 0 (57)j5φ = Φ4ê2/2 + Φ6ê5/2 + Φ7ê8 = 0 (58)j8φ = Φ4ê3/2 + Φ6ê6/2 + Φ7ê9 = 0 (59)j3φ = Φ2ê1/2 + Φ5ê4 + Φ6ê7/2 = 0 (60)j6φ = Φ2ê2/2 + Φ5ê5 + Φ6ê8/2 = 0 (61)j9φ = Φ2ê3/2 + Φ5ê6 + Φ6ê9/2 = 0 (62)

j10φ = φ1t̂1 + φ2t̂2/2 + φ4t̂3/2− φ5t̂1 − φ7t̂1 = 0 (63)j11φ = φ1t̂2 + φ2t̂1/2− φ3t̂2 + φ6t̂3/2− φ7t̂2 = 0 (64)j12φ = φ1t̂3 − φ3t̂3 + φ4t̂1/2− φ5t̂3 + φ6t̂2/2 = 0 (65)

From Equations (54), (55), (56), one can see that a feasible parameterization for φ2, φ3, φ4 is given by:

φ2 = 2t̂2α2 φ3 = t̂1α2 φ4 = 2t̂3α2, (66)

by the relation given in (50), being α2 ∈ R an unknown scalar. Following the same argument with (57), (58), (59), one obtainthat:

φ4 = 2t̂1α3 φ6 = 2t̂2α3 φ7 = t̂3α3, α3 ∈ R. (67)

Further, Equations (59), (60), (61) yield a similar relation

φ2 = 2t̂1α4 φ5 = t̂2α4 φ6 = 2t̂3α4, α4 ∈ R. (68)

Since all the elements of the vector φ must be compatible, all the parameterizations must attain the same value. Concretely,we have two different expressions for φ2, φ4, φ6 and hence,

From φ2 : 2t̂2α2 = 2t̂1α4 =⇒ α2 = t̂1γ, α4 = t̂2γ, γ ∈ R (69)From φ4 : 2t̂3α2 = 2t̂1α3 =⇒ α2 = t̂1β, α3 = t̂3β, β ∈ R (70)From φ6 : 2t̂2α3 = 2t̂3α4 =⇒ α3 = t̂3ψ, α4 = t̂2ψ, ψ ∈ R. (71)

Further, γ = β = ψ since all the relations must hold at the same time. Let us fix this common scale to one without loss ofgenerality. Therefore, we obtain the explicit expressions for φ2, . . . , φ7 as:

φ2 = 2t̂1t̂2, φ3 = t̂21, φ4 = 2t̂1t̂3 (72)

φ5 = t̂22, φ6 = 2t̂2t̂3, φ7 = t̂

23. (73)

Introducing these explicit forms into (63), (64), (65):

φ1t̂1 + t̂1t̂22 + t̂1t̂

23 − t̂1t̂22 − t̂1t̂23 = 0 =⇒ φ1t̂1 = 0 (74)

φ1t̂2 + t̂1t̂22 − t̂1t̂22 + t̂2t̂23 − t̂2t̂23 = 0 =⇒ φ1t̂2 = 0 (75)

φ1t̂3 − t̂1t̂23 + t̂1t̂23 − t̂3t̂22 + t̂3t̂22 = 0 =⇒ φ1t̂3 = 0 (76)

Since tT t 6= 0, in order to fulfill all the relations it is necessary that φ1 = 0. The 7D vector which lies in the nullspace ofJC(x̂) takes the final form:

φ = [0, 2t̂1t̂2, t̂21, 2t̂1t̂3, t̂

22, 2t̂2t̂3, t̂

23]T . (77)

Since φ depends on t 6= 03, it is not null by construction. We note that the same vector was given in [10] for a differentpurpose, although the explicit construction was not provided there. Moreover, we can explicitly give the dependence as:

0 = JC(x̂)φ(x̂) =

7∑k=2

φkAkx̂⇔ (78)

⇔ φiAix̂ = −7∑

j=2,j 6=i

φjAjx̂. (79)

Equation (79) shows that any term φiAix̂, for i = 2, ..., 7 can be written as a function of the other terms φjAjx̂, for j 6=i, j = 2, ..., 7 for any feasible point. Recall that the constraint matrix A1 corresponds with the unitary constraint in t, while{Ai}7i=2 relate the essential matrix to its left nullspace EET = [t]×[t]T×. That is, the gradients of the constraints associatedwith the expression EET = [t]×[t]T× are linearly dependents. By dropping one of the columns associated with these constraints(here we choose to drop the second column, while we postpone a full analysis about their influence for future projects), weobtain the following (reduced) Jacobian JC−2(x̂) ∈ R12×6.

JC−2(x̂).=

0 ê1 ê7/2 0 0 00 0 0 ê4 ê7/2 00 0 ê1/2 0 ê4/2 ê70 ê2 ê8/2 0 0 00 0 0 ê5 ê8/2 00 0 ê2/2 0 ê5/2 ê80 ê3 ê9/2 0 0 00 0 0 ê6 ê9/2 00 0 ê3/2 0 ê6/2 ê9t̂1 0 t̂3/2 −t̂1 0 −t̂1t̂2 −t̂2 0 0 t̂3/2 −t̂2t̂3 −t̂3 t̂1/2 −t̂3 t̂2/2 0

(80)

We have shown that there exists a 7D vector which lies in the nullspace of JC . Next, we will prove that in fact the nullspaceof JC is one-dimensional..

Proof that the nullspace of JC is one-dimensional:To proof that JC−2(x̂) is full rank and thus the nullspace of the original Jacobian matrix JC(x̂) is one-dimensional, let us

assume that there exists a vector R6 3 ξ(x̂) .= [ξ1, ξ2, ξ3, ξ4, ξ5, ξ6]T such that JC−2(x̂)ξ(x̂) = 06. In what follows, we willomit the dependence of ξ(x̂) on the feasible point x̂ and simply write ξ for clarity. Hence:

ξ2ê1 + ξ3ê7/2 = 0 (81)ξ2ê2 + ξ3ê8/2 = 0 (82)ξ2ê3 + ξ3ê9/2 = 0 (83)ξ4ê4 + ξ5ê7/2 = 0 (84)ξ4ê5 + ξ5ê8/2 = 0 (85)ξ4ê6 + ξ5ê9/2 = 0 (86)ξ3ê1/2 + ξ5ê4/2 + ξ6ê7 = 0 (87)ξ3ê2/2 + ξ5ê5/2 + ξ6ê8 = 0 (88)ξ3ê3/2 + ξ5ê6/2 + ξ6ê9 = 0 (89)

ξ1t̂1 + ξ3t̂3/2− ξ4t̂1 − ξ6t̂1 = 0 (90)ξ1t̂2 − ξ2t̂3/2− ξ5t̂3/2− ξ6t̂2 = 0 (91)ξ1t̂3 − ξ2t̂3/2 + ξ3t̂1/2− ξ4t̂3 + ξ5t̂2/2 = 0 (92)

Four different set of equations can be discerned depending on the elements from ξ involved: Equations (81),(82), (83)involving ξ2, ξ3 (the set C23); Equations (84),(85), (86) involving ξ4, ξ5 (the set C45); Equations (87),(88), (89) involving

ξ3, ξ5, ξ6 (the set C356); and Equations (90),(91), (92) involving ξ1, ξ2, ξ3, ξ4, ξ5, ξ6 (the set Call). Further, given this block-alikestructure, we can treat four different cases separately.

Case I: Consider the set C23 and form the linear system in ξ2, ξ3 as:ê1 ê7/2ê2 ê8/2ê3 ê9/2

G13

(ξ2ξ3

)= 03. (93)

If the 2D vector [ξ2, ξ3]T is non-null, then the matrix G13 has one singular value equal to zero or equivantely, the matrixGT13G13 has one eigenvalue equal to zero. Note that if it has 2 eigenvalues equal to zero, then the matrix is null, i.e.ê1 = ê2 = ê3 = ê7 = ê8 = ê9 = 0 and thus, the first and third of E are null. This is not an essential matrix, as it will beexplained in the next subsection. For now, just consider the case when GT13G13 has one zero eigenvalue, i.e. the first and thirdrows of the essential matrix E are linear dependent.

For simplicity, let us denote the rows of E by {ei ∈ R3}3i=1. If e1 or e3 are null, then the essential matrix with theserows is not an essential matrix (we refer again the reader to the next subsection for these cases). This implies that e1 = αe3with α ∈ R/{0} and the essential matrix will have two identical rows up to (signed) scale. Note that α = 0 is excluded sinceotherwise e1 = 0e3 = 03. We will show in the next subsection that a 3 × 3 matrix with one (or two) zero rows cannot bean essential matrix. Considering the definition of essential matrix in (2) and the above-mentioned condition about the first andthird rows, we obtain the following relation between the elements in R and t:r1 r4 r7r2 r5 r8

r3 r6 r9

RT

αt2−αt1 − t3t2

= 03. (94)Since R is a rotation matrix, it is invertible5, which implies that the 3D vector [−t2,−t1 ± t3,±t2]T must be zero for thesystem to hold, i.e. t2 = 0, t3 = −αt1. The translation vector t takes the form t = [t1, 0,−αt1]T /||t||2.

We are left to show how this result yields to the trivial solution for the nullspace of JC−2.Recall the original set of Equations. First we can compute from the set C23: ξ2 = − 12ξ3e

†1e3 = − 12αξ3. Let us assume

that the second and third rows of E are not dependent (otherwise the rank of the essential matrix will be one and hence,not a essential matrix by definition). Therefore, ξ4 = ξ5 = 0 (set C45), which leads thought the set