Nister iccv2005tutorial

Using Algebraic Geometry for Solving Polynomial Problems

in Computer Vision David Nistér and Henrik Stewénius

Very Rough Outline• 14:00-15:30 “1st Round", administered by David.• - What is this Tutorial About?• - Motivation, RANSAC, Example Problems• - Gröbner Bases• - Action Matrix

• -15:30-16:00 Coffee break

• 16:00-18:00-“2nd Round“, administered by Henrik.• - Background Material• - Gröbner Bases• - Exercises in Macaulay2• - Q & A

What is this Tutorial About?

• If you are interested in solving polynomial equations, this tutorial is for you

• This tutorial is a ‘real’ tutorial

• The main focus will be on Gröbner bases and how the theory of Gröbner bases can be used to solve polynomial equations

History• Euclid (325BC-265 BC), • Apollonius (262 BC-190BC), • Descartes (1596-1650), • Fermat (1601-1655)

• Bézout (1730-1783)• Hilbert’s (1862-1943) Nullstellensatz JJ ))I(V(

RISCResearch Institute for Symbolic ComputationLinz, Austria

Wolfgang Gröbner (1899-1980) Bruno Buchberger

1966: Ph.D. in mathematics. University of Innsbruck, Dept. of Mathematics, Austria. Thesis: On Finding a Vector Space Basis of the Residue Class Ring Modulo a Zero Dimensional Polynomial Ideal (German). Thesis Advisor: Wolfgang Gröbner

Bezout’s Theorem

2x2=4

Mixed Volume, see for example [CLO 1998] provides a generalization for non-general polynomials.

With two variables,a solution according to the Bezout boundcan typically berealized with resultants.With three or more variables, thingsare less simple.

Resultants

• Provides Elimination of variables by taking a determinant

]2[]1[]0[

]2[]1[]0[

]2[]1[]0[

]2[]1[]0[1xx2x3

= det = [4]

a1x2+a2y2+a3xy+a4x+a5y+a6

b1x2+b2y2+b3xy+b4x+b5y+b6

652

2431

652

2431

652

2431

652

2431

bybybbybb

bybybbybb

ayayaayaa

ayayaayaa

det

1xx2x3

Various States of Mind You May Have, I:

• Skip the theory, what is this all about? –Answers from David

1st Quadrant2st Quadrant

3rd Quadrant 4th Quadrant

Difficult TheoryEasy Theory

Useful Theory

Useless Theory

Useful but HardUseful and EasySuper!

Useless and EasyUseless and impossibleto penetrate – Unfortunately survives

Algebraic Geometry and Gröbner Bases

Proofs of the Power of Gröbner Bases

• So far, approx. 600 publications and 10 textbooks have been devoted to Gröbner Bases.

• Gröbner Bases are routinely available in current mathematical software systems like Mathematica, Maple, Derive, Magma, Axiom, etc.

• Special software systems, like CoCoa, Macaulay, Singular, Plural, Risa-Asir etc. are, to a large extent, devoted to the applications of Gröbner Bases.

• Gröbner Bases theory is an important section in all international conferences on computer algebra and symbolic computation.

• Gröbner Bases allow, for the first time, algorithmic solutions to some of the most fundamental problems in algebraic geometry but are applied also in such diverse areas as functional analysis, statistics, optimization, coding theory, cryptography, automated theorem proving in geometry, graph theory, combinatorial identities, symbolic summation, special functions, etc.

• Stop the handwaving, what is the rigorous theory?

Answers from David and Henrik + books

Various States of Mind You May Have, II:

Suggested Literature• D. Cox, J. Little, D. O’Shea, Ideals, Varieties,

and Algorithms, Second Edition, 1996.

• D. Cox, J. Little, D. O’Shea, Using Algebraic Geometry, Springer 1998.

• T. Becker and Weispfennig, Gröbner Bases, A Computational Approach to commutative Algebra, Springer 1993.

Suggested Literature• B. Buchberger, F. Winkler (eds.) Gröbner

Bases and ApplicationsCambridge University Press, 1998.

• Henrik’s Thesis: H. Stewénius, Gröbner Basis Methods for Minimal Problems in Computer Vision, PhD Thesis, 2005

• Planned scaffolding paper, keep a lookout on the tutorial web page

• Skip the theory, how do I use it? – Answers from Henrik, Exercises in Macaulay 2.

Various States of Mind You May Have, III:

RANSAC- Random Sample Consensus

Least Squares

RANSAC

Robust

RANSAC- Random Sample Consensus

Line Hypotheses

Points

Robust

RANSAC

HypothesisGenerator

ObservationLikelihood

Hypotheses

Observations

500

1000

500 x 1000 = 500.000

?

Observed Tracks

Hypothesis Generation

Preemptive RANSAC

Data Input

Estimate orposterior likelihoodoutput

ProbabilisticFormulation

HypothesisGenerator

PreciseFormulation

2D-2DRelative Orientation

2D-3DPose

3D-3D 2D-2DAbsolute Orientation

Bundle Adjustment Robust StatisticsTriangulation

Geometry Tools

For Which Problems Did We UseGröbner Bases?

The Generalized 3-Point Problem

YesNo Yes, you bet

The 3 View 4-Point Problem

8(4)

10

2048

The 5-Point Relative Pose Problem

0 (or thousands)

473 View

Triangulation

Unknown FocalRelative Pose

15

64Generalized Relative

Pose

Microphone-SpeakerRelative Orientation

8-38-150-344-??

The Generalized 3-Point Problem

The Generalized 3-Point Problem

The Five Point Problem

What is R,t ?

Given five point correspondences,

EEEtraceEEE )(2

det

Sturm Sequencesfor Bracketing

Root Polishingby Bisection

ER,t

The 5-point algorithm (Nistér PAMI 04)

det

Sturm Sequencesfor Bracketing

Root Polishingby Bisection

R,t E

The 5-point algorithm (Nistér PAMI 04)

EEEtraceEEE )(2

EEEtraceEEE )(2

ER,t

10 x 10Action Matrix

Eigen-Decomposition

The 5-point algorithm (Stewénius et al)

Easy Conditions Realistic Conditions

Correct Calibration

5-Point Matlab Executable

Recent Developments on Direct Relative Orientation, Henrik Stewenius, Christopher Engels, David Nister, To appear in ISPRS Journal of Photogrammetry and Remote Sensing

www.vis.uky.edu/~dnister

6-point generalized relative orientation (64 solutions) (Stewenius, Nistér, Oskarsson and Åström, Omnivis ICCV 2005)

6-point relative orientation with common but unknown focal length (15 solutions) (Stewenius, Nistér, Schaffalitzky and Kahl,to appear at CVPR 2005)

Further Examples of Solved Problems

Triangulation

• Max-Norm -> Quartic (Closed form, Nistér)

• Directional Error -> Quadratic (Oliensis)

Triangulation, 2 Views

• L2-Norm -> Sextic (Hartley & Sturm)

• One parameter family – Balance the error

47 Stationary Points

Optimal 3 View Triangulationwork with Henrik Stewenius and Fred Schaffalitzky

ICCV 2005

Microphone-Speaker Locationwork with Henrik Stewenius

The 3 View 4 Point ProblemWork with Frederik Schaffalitzky

Geometry-Algebra ‘Dualism’

JJ ))I(V(• Hilbert’s Nullstellensatz

How Hard is this Problem?

Approximately This Hard

For Which Problems Did We UseGröbner Bases?

The Generalized 3-Point Problem

YesNo Yes, you bet

The 3 View 4-Point Problem

8(4)

10

2048

The 5-Point Relative Pose Problem

0 (or thousands)

473 View

Triangulation

Unknown FocalRelative Pose

15

64Generalized Relative

Pose

Microphone-SpeakerRelative Orientation

8-38-150-344-??

Camera Geometry

• Often leads to polynomial formulations,

or can at least very often be formulated in

terms of polynomial equations

Polynomial Formulation

• p1(x) , … , pn(x)= A set of input polynomials (n polynomials in m variables)

x=[y1 … ym]

Main Point

• Gröbner Basis Gives Action Matrix

(because it provides the ability to compute unique ‘smallest’ possible

unique remainders)

• Action Matrix Gives Solutions

PolynomialEquations

GröbnerBasis

ActionMatrix

Solutions

Intensively Studied

Algebraic Ideal

• I(p1 , … , pn)= The set of polynomials

generated by the input polynomials

(through additions and multiplications by a polynomial)

p and q in I => p+q in I

p in I => pq in IThe ideal I consists of ‘Almost’ all the polynomials impliedby the input polynomials(More precisely, the radical of the ideal consists of all)I

Basis (for Ideal)

• A basis for the ideal J is a set of polynomials

{p1 , … , pn} such that J=I(p1 , … , pn)

Algebraic Variety

• The solution set

(the vanishing set of the input polynomials)

V(I)={x:I(x)=0}

More precisely

p(x)=0 for all p in I

Quotient Ring J/I

• The set of equivalence classes of polynomials when only the values on V are considered (i.e. polynomials are equivalent iff p(V)=q(V))

V(I) p in J/I

Action Matrix

• For multiplication by polynomial on finite dimensional solution space

V(I)

Action Matrix

TransposedCompanion Matrix

Action Matrix

An ‘Equivalence’

Finding the Roots of a Polynomial

Finding the Eigenvalues of a Matrix

Compute CompanionMatrix

Compute Characteristic Polynomial

Finding the Roots of a Multiple

Polynomial Equations

Finding the Eigenvalues of a Matrix

Compute Action Matrix in Quotient Ring(Polynomials modulo Input Equations)

Compute Characteristic Polynomial

RequiresGröbnerBasis forInput Equations

Companion Matrix

x2 1x3x4x5x6

x7+ a6x6 +a5x5+ a4x4+ a3x3+ a2x2+ a1x+a0

-a6

11

11

1-a0-a4 -a3 -a2 -a1-a5

x

1

Multiplication by x modulo the seventh degree polynomial

can be expressed as left-multiplication by the matrix

Action Matrix

I

Action Matrix

V(I)

I

Action Matrix

V(I)

I

p in J

Action Matrix

V(I)

I

p in J

p in J/I

Action Matrix

Ip in J/I

Action Matrix

Ip in J/I

q in J/I

Action Matrix

Ip in J/I

q in J/I

pq in J/I

Action MatrixMultiplication by a polynomial q is a linear operator Aq

(αp+βr)q=α(pq)+β(rq)

The matrix Aq is called the action matrix for multiplication by q

Action Matrix

Ib0 b1 b2

x0 x1 x2

Action Matrix

Ib0 b1 b2

x0 x1 x2

q in J

Action Matrix

I

q(x1)b1

q(x2)b2q(x0)b0

q in J

Action Matrix

I

The values q(xi) of q at the solutions xi are the

eigenvalues of the action matrix

q(x1)b1

q(x2)b2q(x0)b0

q in J

Action MatrixThe values q(xi) of q at the solutions xi are the eigenvalues of the action matrix

If we choose q=y1 , the eigenvalues are the solutions for y1

Action Matrix

b’(x)Aq p=q(x)b’(x)p for all p in J/I and x in V(I)

b’=[r1 … ro]

b’(x)Aq =b’(x)q(x) b(x) is a left nullvector of Aq corresponding to eigenvalue q(x)

Monomial Order

• Needed to define leading term of a polynomial

• Grevlex (Graded reverse lexicographical) order usually most efficient

y_1

y_2

Gröbner Basis• A basis for ideal I that exposes the leading

terms of I (hence unique well defined remainders)

• Easily gives the action matrix for multiplication with any polynomial in the quotient ring

y_1

y_2

A Reduced Gröbner Basis is a Basis in the normal sense

• A polynomial in the ideal I can be written as a unique combination of the polynomials in a reduced Gröbner basis for I

• The monic Gröbner basis for I is unique

Buchberger’s Algorithm

Gaussian Elimination

Buchberger’s Algorithm

Euclid’sAlgorithm for the

GCD

Gaussian EliminationExposes all the leading terms, which are simplyall the variables in the case of general linear equations

xn x1

Remember Row Operations:

• Multiplying a row by a scalar

• Subtracting a row from another

• Swap rows

Add:• Multiplying a row by any polynomial

More General EliminationWith non-linear equations, there are relations betweenthe monomials that matter when multiplying

x2 xyy2 xyMultiply by x

Multiply by y

x2 xyy2 xy

x2 xyy2 xy

Multiplying by a Scalar

p(x)

3.8p(x)

Transitions through zero remain

Adding

p1(x)

p2(x)

Common transitions through zero remain

p1(x) + p2(x)

Multiplying

p(x)

f(x)

Transitions through zero remain

p(x)f(x)

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Buchberger’s AlgorithmCompute remainders of S-polynomials until

all remainders are zero

Elimination Schedule

Approach

Pose Problem over R

Compute Gröbner basis

End

Compute Action Matrix

Solve Eigenproblem

Backsubstitute

Begin (online)

Pose Problem. Port to Zp

Compute number of solutions

End

Build matrix based Gröbner basis code

Port to R

Begin (offline)

Prime Field Formulation

• Reals => Cancellation unclear

• Rationals => Grows unwieldy

• Prime Field => Cancellation clear, size is limited, only small risk of incorrect cancellation if prime is large

Gaussian Elimination

• Expanding all polynomials up to a certain degree followed by Gaussian elimination allows pivoting

Unwanted Solutions

Can be removed by ideal quotients, or more generally saturation

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Elimination Example

Action Matrix