View
221
Download
0
Category
Tags:
Preview:
Citation preview
3D Computer VisionGeneralized Hough
BoundariesRepresentationsSnakes (Trucco: 109-113)
Computational Vision / Ioannis Stamos
Generalized Hough TransformDetecting shapes
not described by an analytic equation
CSc 83020 3-D Computer Vision – Ioannis Stamos
CSc 83020 3-D Computer Vision – Ioannis Stamos
FromShree Nayar’sNotes
CSc 83020 3-D Computer Vision – Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
Computational Vision / Ioannis Stamos
FromShree Nayar’sNotes
FromShree Nayar’sNotes
Find template t in image f
t and f are both gray-scale images
FromShree Nayar’sNotes
FromShree Nayar’sNotes
FromShree Nayar’sNotes
FromShree Nayar’sNotes
FromShree Nayar’sNotes
FromShree Nayar’sNotes
FromShree Nayar’sNotes
DiscussionCan you search for matches in
Different scale (size) than template?Different orientation than template?
Computational Vision / Ioannis Stamos
ComparisonTemplate matching
vs.Hough
Computational Vision / Ioannis Stamos
To MATCH image boundary/region with MODEL
boundary/region, they must represented in the same
manner.•Boundary Representation•Snakes – Extraction of arbitrary contours from image.•Region Representation
Representation IssuesCompact: Easy to Store & Match.Easy to manipulate & compute properties.Captures Object/Model shape.Computationally efficient.
Polylines: concatenation of line segments.
Breakpoint
Matching on the basis of:# of line segmentslengths of line segmentsangle between consecutive segments
ei
Move along the boundaryAt each point find line that fits previous points
(Least Squares)Compute the fit error E=Sum(ei) using previous pointsIf E exceeds threshold, declare breakpoint and start
a new running line fit.
•Draw Straight line between end-points of curve•For every curve point find distance to line.•If distance is less than tolerance level for all points, Exit•Else, pick point that is farthest away and use as breakpoint.
Introduce new segments.•Recursively apply algorithm to new segments.
s ψ
x
y
s
Ψ
1 2
3 4
5 6
x
y
12
3
4
5
6
π
2π
s ψ
x
y
s
Ψ
1 2
3 4
5 6
x
y
12
3
4
5
6
Ψ-s is periodic (2π)Horizontal section in Ψ-s curve
=> straight line in the ImageNon horizontal section in Ψ-s
=> arc in Image
π
2π
s ψ
x
y H(Ψ)
Ψ
π 2π
HISTOGRAMLines
Arcs
s ψ
x
y H(Ψ)
Ψ
π 2π
HISTOGRAM
H(Ψ) shifts as objects rotates.H(Ψ) wraps around
Lines
Arcs
s ψ
x
y
Find: Ψ(s)Define:Φ(s)= Ψ(s)-(2πs)/P
P: Perimeter.2π: Period of Φ(s)
Φ(s) is a Continuous, Periodic function.Fourier Series for Periodic Functions:
Fourier Coefficients:
Pes
k
sikk
2,)( 0
1
0
P
sikk dsesP 0
0)(1
Φk’s capture shape informationMatch shapes by matching Φk’sUse finite number of Φk’s
Fourier DescriptorsInput Shape Power Spectrum Reconstruction
# of coefficients
Φk
Note: Reconstructed shapes are often not closed sinceonly a finite # of Φk’s are used.
1
0
)()(N
iii sBvsX
•Piecewise continuous polynomials used to INTERPOLATE between Data Points.•Smooth, Flexible, Accurate.
s=0
s=1s=2
x0
x1x2
xi
x
s
Spline X(s)
Data Point
We want to find X(s) from points xiCubic Polynomials are popular:
BASIS FUNCTIONS
COEFFICIENTS
N
Bi(s) has limited support (4 spans)
i-2 i-1 i i+1 i+2
Each span (i->i+1) has only 4 non-zero Basis Functions:Bi-1(s), Bi(s), Bi+1(s), Bi+2(s)
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
s:
s:
3-D Computer Vision CSc 83020 – Ioannis Stamos
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
CubicPolynomials
1/6
4/6
3-D Computer Vision CSc 83020 – Ioannis Stamos
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
6/)()(
6/)1333()()(
6/)463()()(
6/)133()()(
332
2321
231
2301
ttCsB
ttttCsB
tttCsB
ttttCsB
i
i
i
i
CubicPolynomials
1/6
4/6
3-D Computer Vision CSc 83020 – Ioannis Stamos
0141
0303
0363
1331
6
1
1)(
)()()()()(
21123
2312110
C
vvvvCttttx
vtCvtCvtCvtCtxT
iiiii
iiiii
If we compute v0, … vN => continuous representation for the curve.
)0(
...
)0()0()0()0(...............................)0(
)0()0()0()0(................)0(
)0()0()0()0()0(
4332211022
3322110011
2312011000
NN xx
vCvCvCvCxx
vCvCvCvCxx
vCvCvCvCxx
NN v
v
v
v
x
x
x
x
...
410......0
0............0
0...1410
0...0141
0...0014
6
1
...2
1
0
2
1
0
We have our N+1 data points:
And in matrix form:
We can solve for the vi’s from the xi’sBoundary condition for closed curves: v0=vN, v1=vN+1.
Tiiiii vvvvCttttx 21123 1)(
So, for any i we can find:
xi xi+1
xi(t)
t
t=0 t=1
Note: Local support.Spline passes through all data points xi.
B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html
Fitting a curve to an arbitrary shape.Active/Deformable contour or Snake.
Elastic band of arbitrary shape.Located near the image contour.
Attracted towards the target contour.
Stops when Energy Functional is minimized.
[Davatzikos and Prince][Davatzikos and Prince]
[Davatzikos and Prince][Davatzikos and Prince]
Demo using gimp“Intelligent scissors”
User-Visible OptionsInitialization: user-specified, automaticCurve properties: continuity, smoothnessImage features: intensity, edges, corners, …Other forces: hard constraints, springs,
attractors, repulsors, …Scale: local, multiresolution, global
Behind-the-Scenes OptionsFramework: energy minimization, forces
acting on curveCurve representation: ideal curve, sampled,
spline, implicit functionEvolution method: calculus of variations,
numerical differential equations, local search
Snakes: Active Contour ModelsIntroduced by Kass, Witkin, and TerzopoulosFramework: energy minimization
Bending and stretching curve = more energyGood features = less energyCurve evolves to minimize energy
Also “Deformable Contours”
Snakes Energy EquationParametric representation of curve
Energy functional consists of three terms
)(),()( sysxs v )(),()( sysxs v
)()()(int sss conimg vvv )()()(int sss conimg vvv
Internal Energy
First term is “membrane” term – minimum energy when curve minimizes length(“soap bubble”)
Second term is “thin plate” term – minimum energy when curve is smooth
2)()()()()(22
int sssss sss vvv 2)()()()()(22
int sssss sss vvv
Internal Energy
Control and to vary between extremesSet to 0 at a point to allow cornerSet to 0 everywhere to let curve follow
sharp creases – “strings”
2)()()()()(22
int sssss sss vvv 2)()()()()(22
int sssss sss vvv
Edge AttractionGradient-based:
Laplacian-based:
In both cases, can smooth with GaussianSnakes paper, figures 3 and 4
2),( yxIwimg 2),( yxIwimg
22 ),( yxIwimg 22 ),( yxIwimg
Corner AttractionCan use corner detector we saw last timeAlternatively, let = tan-1 Iy / Ix
and let nbe a unit vector perpendicular to the gradient. Then
Snakes paper, figures 5 and 6
n
wimg
n
wimg
Variants on SnakesBalloons [Cohen 91]
Add inflation force
Helps avoid getting stuck on small features
)(skFinfl n )(skFinfl n
[Cohen 91][Cohen 91]
dsEsEsEs imagecurvcont ))()()((
)(sc
)(),()( sysxs c
Parameterized by arc-length
[Kass, Witkan and Terzopoulos ’87]
Minimize
dsEsEsEs imagecurvcont ))()()((
)(scContinuity – compactness
2
1
2
iicont ds
dE pp
c
)(),()( sysxs c
Parameterized by arc-length
[Kass, Witkan and Terzopoulos ’87]
Minimize
dsEsEsEs imagecurvcont ))()()((
)(scContinuity – compactness
21
2
)( iicont dds
dE pp
c
)(),()( sysxs c
Parameterized by arc-length
[Kass, Witkan and Terzopoulos ’87]
Minimize
dsEsEsEs imagecurvcont ))()()((
)(scContinuity – compactness
)(),()( sysxs c
Parameterized by arc-length
[Kass, Witkan and Terzopoulos ’87]
Minimize
Smoothness
2
11 2 iiicurvE ppp
21
2
)( iicont dds
dE pp
c
dsEsEsEs imagecurvcont ))()()((
)(scContinuity – compactness
)(),()( sysxs c
Parameterized by arc-length
[Kass, Witkan and Terzopoulos ’87]
Minimize
Smoothness
2
11 2 iiicurvE ppp
Edge Attraction Term
2IEimage
21
2
)( iicont dds
dE pp
c
N
iimageicurviconti EEE
1
Problem Statement:
Start with chain of image locations representing initial position of contour.
Find the deformable contour
by minimizing the energy functional:
Nppp ,...,, 21
Nppp ,...,, 21
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
Set βj=0 for points of high curvature (corners)
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
Set βj=0 for points of high curvature (corners)
Stop when a user-specified fraction of points does not move.
Synthetic Results
Real Experiment
Snakes demo: http://www.markschulze.net/snakes/
ScaleIn the simplest snakes algorithm, image
features only attract locallyGreater region of attraction: smooth image
Curve might not follow high-frequency detailMultiresolution processing
Start with smoothed image to attract curveFinish with unsmoothed image to get details
Looking for global minimum vs. local minima
Diffusion-Based MethodsAnother way to attract curve to localized
features: vector flow or diffusion methodsExample:
Find edges using CannyFor each point, compute distance to
nearest edgePush curve along gradient of distance field
Xu and PrinceXu and Prince
Xu and PrinceXu and Prince
Simple SnakeSimple Snake With Gradient Vector FieldWith Gradient Vector Field
Xu and PrinceXu and Prince
3-D Computer Vision CSc 83020 – Ioannis Stamos
1 11 1 1 1 1
1 1 1 1 11 1 1 1
1
Spatial Occupancy Array
•Easy to implement.•Large Storage area.•Can apply set operations (unite/intersect).•Expensive for matching!
3-D Computer Vision CSc 83020 – Ioannis Stamos
Quad TreesEfficient encoding ofSpatial Occupancy Arrayusing Resolution Pyramids.
Black: Fully Occupied.White: Empty.Gray: Partially Occupied.
3-D Computer Vision CSc 83020 – Ioannis Stamos
Quad TreeLevel 0Level 1
Level 2
Level 3
Quad Tree Generation:Start with level 0. If Black or White, Terminate. Else
declare Gray node & expand with four sons.
For each son repeat above step.NW NE
SW SE
Recommended