Upload
latrell-pont
View
229
Download
0
Tags:
Embed Size (px)
Citation preview
OpenCV IntroductionHang Xiao
Oct 26, 2012
History 1999 Jan : lanched by Intel, real time machine vision library for UI, optimized code
for intel
2000 Jun : OpenCV alpha 3。
2000 Dec : OpenCV beta 1 for linux
2006 : the first 1.0 version supports Mac OS
2008 mid : obtain corporate support from Willow Garage
2009 Sep : OpenCV 1.2( beta2.0
2009 Oct : Version 2.0 released。
2010 Dec : OpenCV 2.2。
2011 Aug : OpenCV 2.3。
2012 Apr : OpenCV 2.4.
Overview
Goals Develop a universal toolbox for research and development in the
field of Computer Vision
Algorithms More than 350 algorithms, 500 API
Programming language C/C++, C#, Ch , Python, Ruby, Matlab, and Java (using JavaCV)
OS support Windows, Android, Maemo, FreeBSD, OpenBSD, iOS, Linux and Mac
OS.
Licence BSDlisence, free for commercial and non-commmercial
Overview - Applications
2D and 3D feature toolkits
Egomotion estimation
Facial recognition system
Gesture recognition
Human–computer interaction (HCI)
Mobile robotics
Motion understanding
Object identification
Segmentation and Recognition
Stereopsis Stereo vision: depth perception from 2 cameras
Structure from motion (SFM)Motion tracking
Overview - A statistical machine learning library
Boosting (meta-algorithm)
Decision tree learning
Gradient boosting trees
Expectation-maximization algorithm
k-nearest neighbor algorithm
Naive Bayes classifier
Artificial neural networks
Random forest
Support vector machine (SVM)
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Image Analysis
Thresholds
Statistics
Pyramids
Morphology
Distance transform
Flood fill
Feature detection
Contours retrieving
Image Thresholding
Fixed threshold;
Adaptive threshold;
Image Thresholding Examples
Source picture Fixed threshold Adaptive threshold
Statistics
min, max, mean value, standard deviation over the image
Norms C, L1, L2
Multidimensional histograms
Spatial moments up to order 3 (central, normalized, Hu)
Multidimensional Histograms
Histogram operations calculation, normalization, comparison, back project
Histograms types: Dense histograms
Signatures (balanced tree)
EMD algorithm The EMD computes the distance between two distributions, which are
represented by signatures.
The signatures are sets of weighted features that capture the distributions. The features can be of any type and in any number of dimensions, and are defined by the user.
The EMD is defined as the minimum amount of work needed to change one signature into the other
EMD – a method for the histograms comparison
.),(
,
,
),(
),(
,1,,1,
,
,
jiji
ij
jiij
jijiij
ji
qandpelementsthebetweendistancetheqpd
tscoefficienweightf
f
qpdf
QPEMD
historamstwoQjQqPiPp
Image Pyramids
Gaussian and Laplacian pyramids
Image segmentation by pyramids
Image Pyramids
Gaussian and Laplacian
Pyramid-based color segmentation
On still pictures And on movies
Morphological Operations
Two basic morphology operations using structuring element: erosion
dilation
More complex morphology operations: opening : erosion + dilation
closing : dilation + erosion
morphological gradient : the difference between the dilation and the erosion of
an image
top hat : the difference between an input image and its opening
black hat : the difference between the closing and its input image
Morphological Operations Examples
Morphology - applying Min-Max. Filters and its combinations
Opening IoB= (IB)BDilatation IBErosion IBImage I
Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)
Distance Transform
Calculate the distance for all non-feature points to the closest feature point
Two-pass algorithm, 3x3 and 5x5 masks, various metrics predefined
Flood Filling
Simple
Gradient
Feature Detection
Fixed filters (Sobel operator, Laplacian);
Optimal filter kernels with floating point coefficients (first, second derivatives, Laplacian)
Special feature detection (corners)
Canny operator
Hough transform (find lines and line segments)
Gradient runs
Canny Edge Detector
Hough TransformDetects lines in a binary image
• Probabilistic Hough Transform• Standard Hough
Transform
Another Sample of the Hough Transform Using
Source picture Result
Contour Retrieving
The contour representation: Chain code (Freeman code) Polygonal representation
Initial Point
Chain code for the curve: 34445670007654443
Contour representation
Hierarchical representation of contours
Image Boundary
(W1) (W2) (W3)
(B2) (B3) (B4)
(W5) (W6)
Contours Examples
Source Picture(300x600 = 180000 pts total)
Retrieved Contours (<1800 pts total)
After Approximation(<180 pts total)
And it is rather fast: ~70 FPS for 640x480 on complex scenes
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Structural Analysis
Contours processing Approximation Hierarchical representation Shape characteristics Matching
Geometry Contour properties Fitting with primitives PGH: pair-wise geometrical histogram for the
contour.
Contour Processing
Approximation: RLE algorithm (chain code) Teh-Chin approximation (polygonal) Douglas-Peucker approximation (polygonal);
Contour moments (central and normalized up to order 3)
Hierarchical representation of contours
Matching of contours
Hierarchical Representation of Contours
A contour is represented with a binary tree
Given the binary tree, the contour can be retrieved with arbitrary precision
The binary tree is quasi invariant to translations, rotations and scaling
Contours matching
Matching based on hierarchical representation of contours
Geometry
Properties of contours: (perimeter, area, convex hull, convexity defects, rectangle of minimum area)
Fitting: (2D line, 3D line, circle, ellipse)
Pair-wise geometrical histogram
Pair-wise geometrical histogram (PGH)
),( jip
.),(/),()(
,),(/),()(
,)](),2(),1(),(),2(),1([
iic
jjr
TcccrrrPGH
jipjipijE
jipjipjiE
MEEENEEEf
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Object Recognition
Eigen objects
Hidden Markov Models
Eigen Objects
.
.},,,{
,,1},,,,{
),(/1
,/1),()(
,cov}{
,,1,},,{
21
111
1
21
vectorseigenandvalueseigenand
basiseigenmmieeee
uuve
umuuuuuc
matrixariancecC
nmmiRuuuu
im
iiii
in
iii
l
k
kl
iki
il
m
k
klll
jl
l
lilij
ij
nin
iii
Eigen objects (continued)
Hidden Markov Model
Definitions
},...,,{ 21 NsssS },...,,{ 21 MoooO
Oqt )}|(\{ 1 itjtijij sqsqPaaA
)}|(\{ jiijij soPbbB
)}(\{ 0 iii sqP
- The set of states
- The set of measurements
- The state at time t
- The transition probability matrix
- The conditional probability matrix
- The starting states distribution
Embedded HMM for Face Recognition
Model-
- Face ROI partition
Face recognition using Hidden Markov Models
One person – one HMMStage 1 – Train every HMM
Stage 2 – Recognition
Pi - probability
Choose max(Pi)
…1
n
i
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Motion Analysis and Object Tracking
Background subtraction
Motion templates
Optical flow
Active contours
Estimators
Background Subtraction
Background model (normal distribution)
Background statistics functions: Average Standard deviation Running average
Motion Templates
Object silhouette
Motion history images
Motion history gradients
Motion segmentation algorithmsilhouette MHI
MHG
Motion Segmentation Algorithm
Two-pass algorithm labeling all motion segments
Motion Templates Example
Motion templates allow to retrieve the dynamic characteristics of the moving object
Optical Flow
Block matching technique
Horn & Schunck technique
Lucas & Kanade technique
Pyramidal LK algorithm
6DOF (6 degree of freedom) algorithm
y
xt
yyx
yxx
I
IIb
III
IIIGyxX
bXG
dtdyyIdtdxxItI
tyxIdttdyydxxI
,,
,),,(
,
);/(/)/(//
);,,(),,(
2
2
Optical flow equations:
Pyramidal Implementation of the optical flow algorithm
J image I image
Image Pyramid Representation
Iterative Lucas – Kanade Scheme
Generic Image
(L-1)-th Level
L-th Level
Location of point u on image uL=u/2L
Spatial gradient matrix
Standard Lucas – Kanade scheme for optical flow computation at level L dL
Guess for next pyramid level L – 1
Finally,
Image pyramid building
Optical flow computation
2
2
,
,
yyx
yxx
III
IIIG
)(21 LLL dgg
00 gdd
dUV
6DOF Algorithm
N
i ROI
Tit
N
i ROIi
Ti IIdsII
sXIsII
i
11
///
).(sX
Parametrical optical flow equations:
Active Contours
Snake energy:
Internal energy:
External energy:
Two external energy types:
extEEE int
curvcont EEE int
conimgext EEE
min
,)(
,
imgcurvcont
img
img
EEEE
IgradE
IE
Estimators
Kalman filter
ConDensation filter
Kalman object tracker
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
3D reconstruction
Camera Calibration
View Morphing
POSIT
Camera Calibration
Define intrinsic and extrinsic camera parameters.
Define Distortion parameters
],[],,,[,,,
100
0
0
,][
3
2
1
333231
232221
131211
vupZYXP
t
t
t
T
rrr
rrr
rrr
Rcf
cf
A
PRTAp
yy
xx
.
)],2/(2[)(~)],2/(2[)(~
222
212
42
21
221
42
21
yxr
yyrpxprkrkcvvv
xxrpyprkrkcuuu
y
x
Camera Calibration
Now, camera calibration can be done by holding checkerboard in front of the camera for a few seconds.
And after that you’ll get:3D view of etalon Un-distorted image
View Morphing
POSIT Algorithm
Perspective projection:
Weak-perspective projection:
iiiiii YZfyXZfx )/(,)/(
./,, ZfsYsyXsx iiii
OpenCV Websites
http://opencv.org OpenCV official webpage.
http://opencvlibrary.sourceforge.net/ OpenCV documentation and FAQs.
OpenCV Examples
adaptiveskindetector : detect skin area
fback_c : dense Franeback optical flow
contours : calculate contours on different levels
delaunay : delaunay triangle
find_obj : SURF Detector and Descriptor using either FLANN or brute force matching on planar objects
morphology : open/close, erode/dilate
motempl :motion templates
mser_sample : Maximal Extremal Region interest point detector
polar_transforms : illustrates Linear-Polar and Log-Polar image transforms
pyramid_segmentation : color pyramid segmentation
Thanks !!!