Center for Machine PerceptionDepartment of Cybernetics, Faculty of Electrical Engineering

Czech Technical University in Prague

MAKING MINIMAL SOLVERS FAST

Zuzana Kukelova, Martin Bujnak, Tomas Pajdla

2/24Motivation

Recognition & Tracking

Placeholder text

Placeholder text

3D Reconstruction

Augmented reality

Robotics

Require fast solvers

3/24Motivation

Systems of polynomial equations

Camera geometry problems - calibration, relative/absolute pose estimation

4/24

PropertiesProperties RequirementsRequirements

all “non-degenerate” instances of the problem result in systems of equations of the same form - consisting of the same polynomials and differing only in coefficients

all “non-degenerate” instances of the problem result in systems of equations of the same form - consisting of the same polynomials and differing only in coefficients

Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)

High or even real-time performance

Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)

High or even real-time performance

Solve many instances of the same problem, of the same system of polynomial equations only with different coefficients, very fast

Co

nc

lus

ion

Co

nc

lus

ion

Camera geometry problems

5/24Complicated systems5-point relative pose problem

f 2 = 4:68x +6:62y+2:25z + 1:81+ 6:68y2 ¡ 0:806z2 +12:6xy+ 4:03xz +6:99yz +1:01x3+2:04y3 ¡ 0:873z3+6:13xyz +5:53x2y+1:94x2z +6:95xy2 ¡1:13xz2+4:27y2z ¡ 0:130yz2+4:53x2

f 3 = 0:866x+2:28y+3:10z+3:50y2+3:04z2+4:15xy+6:07xz+8:09yz ¡0:076x3+1:34y3+0:497z3+8:03xyz+3:05x2y+1:88x2z+4:69xy2+3:40xz2+5:24y2z +4:21yz2+0:402+0:199x2

f 10 = 5:35x +4:46y+ 3:06z + 3:85y2 +1:48+ 2:57z2 +9:21xy+ 8:05xz +7:16yz +1:41x3+0:991y3+1:09z3+8:06xyz +4:06x2y+4:06x2z +3:83xy2+3:50xz2+3:68y2z+2:60yz2+5:72x2

f 1 = 0:058x +0:331y ¡ 0:671z + 0:698y2 ¡ 0:813z2 ¡ 0:057xy ¡ 0:036xz +0:653yz + 0:534x3 ¡ 0:167y3 ¡ 0:323z3 + 0:633xyz ¡ 0:282x2y + 0:770x2z ¡0:974xy2 ¡ 0:146xz2+0:584y2z +0:239yz2 ¡ 0:170+0:593x2

: : :

6/24General methodsExisting software – well known general Grőbner basis or resultant algorithms

Reliablebut

SLOW!

7/24Specific methods• Design special algorithms to achieve numerical robustness and

computational efficiency

• Not general

• Fast !!!

• Intersection ellipse-line• No need to use general algorithms• Closed form solution

8/24

Online solver• Efficient solver for

solving systems of polynomial equations of one form

• Not general but fast

• This is what the user call

Offline phase• Study the problem • Solve it in some finite

prime field• Design a specific

efficient solver• Needs to be

performed only once

Specific methods

9/24Grőbner basis specific methods

Parameters identification• Number of solutions, basis• Computations in finite prime field

- fast, stable

Elimination template design• Which polynomials

should be multiplied with which monomials and eliminated to obtain the solution

G-J elimintation of the found template• With concrete

coefficients

Eigenvalue computations• Computation of

eigenvalues and eigenvactors of the action matrix created from the eliminated template

10/24Offline phase

25min15min

Offline phase & Elimination template – hard to understand and implement

Requires knowledge from algebraic geometry

1+1=?

ideal

mul

tiplic

atio

n m

atrix

Quotient ring

xy2+7y+..

field

11/24Automatic generator

Final “online” solver

List of known coeffs

List of unknowns

Equations

Offline phase

12/24How to make the final solver fast

Reduce the size of the elimination template• Kukelova Z., Bujnak M.,

Pajdla T., Automatic Generator of Minimal Problem Solvers, ECCV 2008, Marseille, France, October 12-18, 2008.

Sparse G-J elimintation of the found template

The slowest part • Idea: Exchange it for

computation of roots of a single-variable polynomial using Sturm-sequences

• Bujnak M., Kukelova Z., Pajdla T., Making Minimal Solvers Fast, CVPR 2012.

13/24Grőbner bases of the ideal• System of input polynomial equations

• The ideal generated by is a set of all polynomials that can be generated as polynomial combinations of

f 1; : : : ; fm

where are arbitrary polynomials from

I = fP m

i=1 f i hi : hi 2 C [x1; :::;xn]g

f 1; : : : ; fm

hi

I

𝒇 𝟏

𝒇 𝟐

𝒇 𝒎

+ …

I

14/24Grőbner bases of the ideal

𝒇 𝟏

𝒇 𝟐

𝒇 𝒎

+ …

I𝒈𝟏

𝒈𝟐

𝒈𝒎

+ + …

I

• An ideal can be generated by many different sets of generators which all share the same solutions

¿

• Grőbner bases - special bases• Useful in solving system of polynomial equations

15/24

Lexicographic GBLexicographic GB Graded Reverse Lex GBGraded Reverse Lex GB

Contains polynomial in one

variable Very expensive to compute Not feasible for computer

vision problems

Contains polynomial in one

variable Very expensive to compute Not feasible for computer

vision problems

Doesn’t contain a single variable polynomial

Special multiplication matrix – eigenvalues & eigenvectors give solutions

Easier to compute Used to solve CV problems

Doesn’t contain a single variable polynomial

Special multiplication matrix – eigenvalues & eigenvectors give solutions

Easier to compute Used to solve CV problems

Grőbner bases of the ideal

g1 (x1)

g2 (x1;x2)

: : :

gn (x1; : : : ;xn)

q1 (x1; : : : ;xn)

q2 (x1; : : : ;xn)

: : :

ql (x1; : : :;xn)

16/24

• We have the Graded Reverse Lexicographic GB and multiplication (action) matrix

1. FGLM conversion algorithmW

e h

av

eW

e h

av

e

• We want the variable polynomial from the Lexicographic GB

We

wa

nt

We

wa

nt

• Well known FGLM algorithm for converting grevlex GB to Lexicographic GB

• Needs polynomial division• Doesn’t bring a speed up over eigenvalue computations

So

luti

on

So

luti

on

q1 (x1; : : : ;xn)

q2 (x1; : : : ;xn)

: : :

ql (x1; : : :;xn)

g1 (x1)

17/24

• Replace time consuming polynomial division performed in standard FGLM algorithm with efficient matrix-vector multiplication using the action matrix

New Matrix FGLM algorithmT

ric

kT

ric

k

• New Matrix FGLM algorithm• To obtain a single variable polynomial for a system with solutions

• matrix-vector multiplications• Solve system of linear equations in unknowns

• Solve single variable polynomial using efficient Sturm-sequences

Alg

ori

thm

Alg

ori

thm

• Significant speedup over the eigenvalue computation• Only feasible solutions – positive feasible focal lengths, depths or radial

distortion coefficientsPo

sit

ive

sP

os

itiv

es

18/24

• We have a multiplication (action) matrix for multiplying with some variable • Eigenvalues of this action matrix give solutions to

2. Characteristic polynomial methodW

e h

av

eW

e h

av

e

• We want a single variable polynomial

We

wa

nt

We

wa

nt

• Characteristic polynomial of the action matrix • Krylov’s method• Faddeev-Leverrier method• Danilevsky method

So

luti

on

So

luti

on

g(x)

19/242. Characteristic polynomial method

• Faddeev-Leverrier method• Well-known method, compute traces of matrices• Suffers from a large numerical instabilityF

ad

de

ev

Fa

dd

ee

v

• Danilevsky method• Transforms input matrix to its companion matrix by s − 1 similarity

transformations• very efficient and numerically stable

So

luti

on

So

luti

on

• Krylov’s method• Based on Cayley-Hamilton theorem• Equivalent to the proposed matrix FGLM algorithm• Computes - for some matrices may be unstable

Kry

lov

Kry

lov

20/24

• 5-point relative pose problem

• Nister – SOTA “closed form solution”• GB+eig – standard Grőbner basis solution with eigenvalue

computation

• New “single variable polynomial”solutions – • ~5x speedup over GB+eig• Comparable to the closed form solution

Speedup over the eigenvalue method

Danilevsky

• 14.2μs

mFGML

• 13.7μs

Faddeev

• 17.2μs

GB+eig

• 61.2μs

Nister

• 10.6μs

21/24

• 6-point focal length relative pose problem

• New “single variable polynomial”solutions • ~8x speedup over GB+eig

Speedup over the eigenvalue method

Danilevsky

• 22.6μs

mFGML

• 21.3μs

GB+eig

• 176.3μs

22/24

• P4P+f absolute pose problem

• New “single variable polynomial”solutions – • ~3x speedup over GB+eig

Speedup over the eigenvalue method

Danilevsky

• 47.4μs

mFGML

• 46.2μs

Faddeev

• 51.2μs

Sparse GB

• 82.9μs

GB+eig

• 127.4μs

23/24

• Many problems in computer vision and other fields require fast solvers of systems of polynomial equations

• General methods – not feasible• Specific solvers – not general but fast

Conclusion

• Automatic generator of Grőbner basis solvers

• New methods for speeding up Grőbner basis solvers• Matrix FGLM method + Sturm sequences• Characteristic polynomial method + Sturm sequences

• Significant speed up over existing GB solvers

24/24http://cmp.felk.cvut.cz/minimal

THANK YOU FOR YOUR ATTENTION