INFORMATIK

Prof. Didier Stricker

Kaiserlautern University

http://ags.cs.uni-kl.de/

DFKI – Deutsches Forschungszentrum für Künstliche Intelligenz

http://av.dfki.de

2D Image Processing Introduction to OpenCV

1

INFORMATIK

§ Presentation of the OpenCV library § History of the development § Basic concepts and data types

§ Practical considerations § Examples of use

Goals

2

INFORMATIK

§ General overview

§ Structure and datatypes

§  Installation instructions

§ Examples

Outline

3

INFORMATIK

OpenCV  –  Introduc0on  }  OpenCV  stands  for  the  Open  Source  Computer  Vision  Library.    

Founded  at  Intel  in  1999  OpenCV  is  free  for  commercial  and  research  use.    It  has  a  BSD  license.    The  library  runs  across  many  plaAorms  and  acBvely  supports  Linux,  Windows  and  Mac  OS.    

}  OpenCV  was  founded  to  advance  the  field  of  computer  vision.    It  gives  everyone  a  reliable,  real  Bme  infrastructure  to  build  on.    It  collects  the  most  useful  algorithms.  

4

INFORMATIK License

•  Based on BSD license •  Free for commercial or research use

–  In whole or in part –  Does not force your code to be open –  You do not need to contribute back

5

INFORMATIK

OpenCV at glance •  BSD license, 10M downloads, 500K+ lines of code •  Huge community involvement, automated patch testing and

integration process •  Runs everywhere

SSE, NEON, IPP, OpenCL, CUDA, OpenCV4Tegra, …

core, imgproc, objdetect …

OpenCV HAL

OpenCV

face, text, rgbd, … OpenCV Contrib

Bindings: Python, Java

Samples, Apps, Solutions

6 Slide  from  G.  Bradsky  

INFORMATIK Recent Stats

~200K downloads/month

> 10M downloads

World wide Rated highly

NOTE: This is only for source forge. Many more downloads come from Git and many more come on Unix distros.

7 Slide  from  G.  Bradsky  

INFORMATIK

8

OpenCV history

Slide  from  G.  Bradsky  

INFORMATIK

OpenCV Algorithm Modules Overview

9

Image Processing

Object recognition Machine learning

Transforms

Calibration Features VSLAM

Fitting Optical Flow Tracking

Depth, Pose Normals, Planes,

3D Features

Computational Photography

CORE: Data structures, Matrix math, Exceptions etc

Segmentation

HighGUI: I/O, Interface

Slide  from  G.  Bradsky  

INFORMATIKOpenCV  Overview:    General Image Processing Functions

Machine Learning: • Detection, • Recognition

Segmentation

Tracking

Matrix Math

Utilities and Data Structures

Fitting

Image Pyramids

Camera Calibration, Stereo, 3D

Transforms Features

Geometric Descriptors

Robot  support  

opencv.willowgarage.com  

> 500 functions

Slide Courtesy OpenCV Tutorial Gary Bradski

10

INFORMATIK

Other  Languages  

OpenCV  Conceptual  Structure  

Python    

Java  (TBD)  

Machine  learning  

HighGUI  

       SSE  TBB  GPU  MPU          

Modules  

CORE  

imgproc  

Object  DetecBon   Features2d   Calib3d  

Stereo  

VO  SLAM  (TBD)  

SBtching  (TBD)  

User  Contrib  

Opera0ng  system  

C  

C++  

INFORMATIK

CLASSIFICATION / REGRESSION (new) Fast Approximate NN (FLANN) (new) Extremely Random Trees CART Naïve Bayes MLP (Back propagation)‏ Statistical Boosting, 4 flavors Random Forests SVM Face Detector (Histogram matching)‏ (Correlation)‏ CLUSTERING K-Means EM (Mahalanobis distance)‏ TUNING/VALIDATION Cross validation Bootstrapping Variable importance Sampling methods

Machine Learning Library (ML)‏

AACBAABBCBCC  

AAA  

AACACB   CBABBC  

CCB  

B  CC  

ABBC  CB  

B  C   A   BBC  

C  BB  

12  12

INFORMATIK

Where  is  OpenCV  Used?  § Google  Maps,  Google  street  view,  Google  Earth,  Books  

§ Academic  and  Industry  Research  

§  Safety  monitoring  (Dam  sites,  mines,  swimming  pools)  

§  Security  systems  

§  Image  retrieval  

§ Video  search  §  Structure  from  mo0on  in  movies  

§ Machine  vision  factory  produc0on  inspec0on  systems  

§  Robo0cs  

INFORMATIK

OpenCV  –  GePng  Started  

}  Download  OpenCV  hVp://opencv.org  

}  SeYng  up  Comprehensive  guide  on  seYng  up  OpenCV  in  various  environments  at  the  official  wiki.  

}  Online  Reference:  hVp://docs.opencv.org    

}  Two  books  

14

INFORMATIK

OpenCV Book(s)

15 15 Gary Bradski, 2009

Book on OpenCV (2008): http://oreilly.com/catalog/9780596516130/ Or, direct from Amazon:http://www.amazon.com/Learning-OpenCV-Computer-Vision-Library/dp/0596516134 Code examples from the book: http://examples.oreilly.com/9780596516130/

INFORMATIK

16

INFORMATIK

Base  structure  of  OpenCV  >2.0  l  OpenCV  2.x  is  a  C++  library  as  opposed  to  OpenCV  1.x  l  Significant  changes  in  module  structure  since  version  2.2  l  Modules:  core,  imgproc,  video,  calib3d,  features2d,  

objdetect,  highgui,  gpu...  

INFORMATIK

New  C++  API:  Usage  Example  

double calcGradients(const IplImage *src, int aperture_size = 7) { CvSize sz = cvGetSize(src); IplImage* img16_x = cvCreateImage( sz, IPL_DEPTH_16S, 1); IplImage* img16_y = cvCreateImage( sz, IPL_DEPTH_16S, 1); cvSobel( src, img16_x, 1, 0, aperture_size); cvSobel( src, img16_y, 0, 1, aperture_size); IplImage* imgF_x = cvCreateImage( sz, IPL_DEPTH_32F, 1); IplImage* imgF_y = cvCreateImage( sz, IPL_DEPTH_32F, 1); cvScale(img16_x, imgF_x); cvScale(img16_y, imgF_y); IplImage* magnitude = cvCreateImage( sz, IPL_DEPTH_32F, 1); cvCartToPolar(imgF_x, imgF_y, magnitude); double res = cvSum(magnitude).val[0]; cvReleaseImage( &magnitude ); cvReleaseImage(&imgF_x); cvReleaseImage(&imgF_y); cvReleaseImage(&img16_x); cvReleaseImage(&img16_y); return res; }

double contrast_measure(const Mat& img) { Mat dx, dy; Sobel(img, dx, 1, 0, 3, CV_32F); Sobel(img, dy, 0, 1, 3, CV_32F); magnitude(dx, dy, dx); return sum(dx)[0]; }

C: C++:

18

INFORMATIK

OpenCV  core  func0onality  

l  All  the  OpenCV  classes  and  func0ons  are  placed  into  the  cv  namespace  

l  core  -­‐  compact  module  defining  basic  data  structures  and  basic  func0ons  used  by  all  other  modules  

l  Basic  image  class  cv::Mat

26/10/11

INFORMATIK

cv::Mat memory management  

l  OpenCV  handles  all  the  memory  automa0cally  

l  Memory  is  freed  automa0cally  when  needed  

l  When  a  Mat  instance  is  copied,  no  actual  data  is  really  copied  

l  To  make  a  real  copy,  use  Mat::clone  

26/10/11

INFORMATIK

The  Mat  class  I  • cv::Mat covers  the  old  CvMat  and  IplImage

– Data  representa0on  • Data  is  row  ordered  • Colour  pixels  are  interleaved  (e.  g.  RGBRGBRGB...)  

• Let's  see  some  important  members  of  the  class:  – Create  and  ini0alize  

26/10/11

//  Mat(int  _rows,  int  _cols,  int  _type);  //  Mat(Size  _size,  int  _type);  type  =  CV_8UC3,  CV_32FC1,  ...  //  Mat(Size  _size,  int  _type,  const  Scalar&  _s);  fill  with  values  in  _s  Mat  M(7,7,CV_32FC2,Scalar(1,3));//7x7,  float,  2  channels,  fill  with  (1,3)  M.create(Size(15,15),  CV_8U);//reallocate  (if  needed)  //Matlab-­‐like  initializers  Mat  ident  =  Mat::eye(3,3,  CV_32F);//also  Mat::ones(..)  and  Mat::zeros(..)  int*  data  =  {1,2,3,9,0,-­‐3};  Mat  C  (2,3,CV_32S,  data);  //no  data  copied.  C  =  C.clone();  //clone  the  matrix  -­‐>  now  the  data  is  created.  

   

INFORMATIK

The  Mat  class  II  l  Important  things  to  know:  

l  Shallow  copy:  Mat A = B; does  not  copy  data.  l  Deep  copy:  clone() and/or  B.copyTo(A);  (for  ROIs,  etc).  l  Most  OpenCV  func0ons  can  resize  matrices  if  needed  

l  Lots  of  convenient  func0onality  (Matrix  expressions):  l  s is  a  cv::Scalar,  α  scalar  (double)  l  Addi0on,  scaling,  ...:  A±B, A±s, s±A, αA l  Per-­‐element  mul0plica0on,  division...:  A.mul(B), A/B, α/A

l  Matrix  mul0plica0on,  dot,  cross  product:  A*B, A.dot(B),  l  A.cross(B)

l  Transposi0on,  inversion:  A.t(), A.inv([method]) l  And  a  few  more.  

  26/10/11

INFORMATIK

Mat  class:  element  access  I  

l  Rows,  columns,  ROIs,...  Mat A = B.row(int row); //same for B.col() A = B.rowRange(Range rg);//same for B.colRange() A = B(Rect r);//use a rectangle to set ROI l  Ranges,  ROIs,  etc...  only  create  new  headers.  l  Where  is  a  ROI  in  the  bigger  matrix?  

Mat A = B(Rect r); Size s; Point offset; A.locateROI(s, offset); //'s' and 'offset'

will define the rectangle 'rect' l  Element  access:  3  op0ons  

l  Using  at<>()  

double val = M.at<double>(i, j);//You have to know the type

26/10/11

INFORMATIK

Mat  class:  element  access  II  l  Old  C  style.  // compute sum of positive matrix elements double sum=0; for(int i = 0; i < M.rows; i++) { const double* Mi = M.ptr<double>(i); //we know it's double data for(int j = 0; j < M.cols; j++) sum += std::max(Mi[j], 0.); } l  STL-­‐like  iterators  

// compute sum of positive matrix elements, iterator-based variant

double sum=0; MatConstIterator_<double> it = M.begin<double>(),

it_end = M.end<double>(); for(; it != it_end; ++it) sum += std::max(*it, 0.);

-  This  iterators  can  be  used  with  STL  func0ons,  like  std::sort()

26/10/11

INFORMATIK

Manipula0on  with  images  using  Mat  class  l  Reading  and  wri0ng  images  is  easy  Mat imread(const string& filename, int flags=1);

//flags =0 -> always grayscale

//flags >0 -> always color

//flags <0 -> read image as-is bool imwrite(const string& filename, const Mat& img,

const vector<int>& params=vector<int>());

//params set compressions values. defaults are fine. l  example: Mat img = imread("filename.jpg", 1); imwrite("file.png", myImage);

26/10/11

INFORMATIK

Examples:  thresholding  #include  <cv.h>  

#include  <highgui.h>  

using  namespace  std;  

using  namespace  cv;  

int  main(  int  argc,  char**  argv  )  

{  Mat  src,  gray,  grayThresh;  src  =  imread(argc  >=  2  ?  argv[1]  :  "fruits.jpg",  1);  namedWindow("src",  CV_WINDOW_AUTOSIZE);  namedWindow("gray",  CV_WINDOW_AUTOSIZE);  namedWindow("grayThreshold",  CV_WINDOW_AUTOSIZE);  cvtColor(src,  gray,  CV_BGR2GRAY);  //color  images  are  BGR!  threshold(gray,  grayThresh,  100,  250,  CV_THRESH_BINARY);  imshow("src",  src);  imshow("gray",  gray);  imshow("grayThreshold",  grayThresh);  waitKey(0);  //waits  for  a  key:  it  also  handles  the  GUI  events.  

}  

return  0;  //no  need  to  free  the  matrices,  they  are  deleted  automatically  

26/10/11

INFORMATIK

Examples:  Canny  edge  detector  #include  <cv.h>  #include  <highgui.h>  using  namespace  std;  using  namespace  cv;  int  main(  int  argc,  char**  argv  )  {  

 Mat  src,  dst;    src  =  imread(argc  >=  2  ?  argv[1]  :  "fruits.jpg",  0);    //  dst  =  Mat(src.size(),  src.type());    Canny(src,  dst,  100,  150,  3);    namedWindow("src");  imshow("src",  src);    namedWindow("canny");  imshow("canny",  dst);    WaitKey(0);    return  0;  

}  

26/10/11

INFORMATIK

HighGUI:  Crea0ng  Interfaces  I  l  Start  off  by  crea0ng  a  program  that  will  constantly  input  images  from  a  camera  

         #include  <cv.h>            #include  <highgui.h>            int  main()  

 {    CvCapture*  capture  =  0;    capture  =  cvCaptureFromCAM(0);    if(!capture)    {      printf("Could  not  initialize  capturing...\n");      return  -­‐1;    }    cvNamedWindow("video");  

l  This  code  creates  a  capture  structure  poin0ng  to  camera  #0  and  creates  a  window  named  “video”  

26/10/11

INFORMATIK

HighGUI:  Crea0ng  Interfaces  II  l  Create  two  variables  holding  the  values  of  the  trackbars  we’ll  create  

         int  bright=128,  contrast=26;  

l  And  now  we  actually  create  the  trackbars:  

 cvCreateTrackbar("brightness",  //name  of  the  trackbar      "video",    //name  of  the  window      &bright,    //pointer  to  a  variable  that  will  hold  the  value  of        the  

trackbar)      255,    //maximum  value  of  the  trackbar  (minimum  is  always  0)      NULL);    //A  callback  function  (which  is  called  whenever  the        position  

of  the  trackbar  is  changed)      cvCreateTrackbar("contrast",  "video",  &contrast,  50,  NULL);  

•  Start  the  infinite  loop  reques0ng  for  frames:  

 while(true)    {      IplImage*  frame  =  0;      frame  =  cvQueryFrame(capture);      if  (!frame)        break;  

26/10/11

INFORMATIK

HighGUI:  Crea0ng  Interfaces  III  

l  bright  is  in  range  [0,255],  thus  subtract  128  to  have  a  convenient  range  -­‐127...128  to  reduce  to  increase  brightness.  

l  Modify  image  contrast  and  brightness  by  adding  to  every  pixel  bright  value  and  scaling  by  contrast  cvAddS(frame,  cvScalar(bright-­‐128,bright-­‐128,bright-­‐    128),  

frame);  l  Display  the  image  in  the  window  “video”  un0l  the  Esc  key  (ASCII  =  27)  is  pressed  

   cvShowImage("video",  frame);      int  c  =  cvWaitKey(20);      if  ((char)c==27)        break;    }    cvReleaseCapture(&capture);    return  0;  

     }  

26/10/11

INFORMATIK

HighGUI:  trackbar  example  

26/10/11

INFORMATIK

OpenCV:  image  filtering  

• It  is  possible  to  smooth  images  using  OpenCV  func0ons  such  as:  

• Blur  • Gaussian  blur  • Median  blur  • Bilateral  filter  

26/10/11

INFORMATIK

Theory  • Smoothing  (blurring)  is  a  simple  and  frequently  used  opera0on  • There  are  many  reasons  for  smoothing,  e.g.  noise  suppression  • To  perform  a  smoothing  opera0on  we  will  apply  a  filter  to  our  image.  The  most  common  type  of  filters  are  linear,  in  which  an  output  pixel’s  value  (i.e.  𝑔(𝑖,𝑗))  is  determined  as  a  weighted  sum  of  input  pixel  values  (i.e.    𝑓(𝑖+𝑘,𝑗+𝑙))  :  

𝑔(𝑖,𝑗)=∑𝑘,𝑙↑▒𝑓(𝑖+𝑘,𝑗+𝑙)ℎ(𝑘,𝑙)   • ℎ(𝑘,𝑙)  is  called  the  kernel,  which  is  nothing  more  than  the  coefficients  of  the  filter.  

• It  helps  to  visualize  a  filter  as  a  window  of  coefficients  sliding  across  the  image.  

26/10/11

INFORMATIK

Normalized  Box  Filter  

• This  filter  is  the  simplest  of  all!  Each  output  pixel  is  the  mean  of  its  kernel  neighbors  (all  of  them  contribute  with  equal  weights)  

• The  kernel  is  below:  𝐾= 1/𝐾↓𝑤𝑖𝑑𝑡ℎ ∗𝐾↓ℎ𝑒𝑖𝑔ℎ𝑡  [█■1&…&1@…&…&…@1&…&1 ]  

26/10/11

INFORMATIK

Gaussian  Filter  I  

• Probably  the  most  useful  filter  (although  not  the  fastest).  Gaussian  filtering  is  done  by  convolving  each  point  in  the  input  array  with  a    Gaussian  kernel.  

• 1D  Gaussian  kernel  

26/10/11

INFORMATIK

Gaussian  Filter  II  • Pixel  located  in  the  middle  has  the  biggest  weight.  • The  weight  of  its  neighbors  decreases  as  the  spa0al  distance  between  them  and  the  center  pixel  increases.  

• 2D  Gaussian  kernel  

• where  𝜇  is  the  mean  (the  peak)  and  𝜎  represents  the  variance  (per  each  of  the  variables  𝑥  and    𝑦)  

26/10/11

INFORMATIK

Median  filter  

• The  median  filter  run  through  each  element  of  the  signal  (in  this  case  the  image)  and  replace  each  pixel  with  the  median  of  its  neighboring  pixels  (located  in  a  square  neighborhood  around  the  evaluated  pixel).  

• The  median  of  a  finite  list  of  numbers  can  be  found  by  arranging  all  the  observa0ons  from  lowest  value  to  highest  value  and  picking  the  middle  one.  

26/10/11

INFORMATIK

Bilateral  Filter  

• Considered  filters  main  goal  were  to  smooth  an  input  image.  However,  some0mes  the  filters  do  not  only  dissolve  the  noise,  but  also  smooth  away  the  edges.  To  avoid  this  (at  certain  extent  at  least),  we  can  use  a  bilateral  filter.  

• In  an  analogous  way  as  the  Gaussian  filter,  the  bilateral  filter  also  considers  the  neighboring  pixels  with  weights  assigned  to  each  of  them.  

• These  weights  have  two  components  – The  first  component  is  the  same  weigh0ng  used  by  the  Gaussian  filter  

– The  second  component  takes  into  account  the  difference  in  intensity  between  the  neighboring  pixels  and  the  evaluated  one.  

26/10/11

INFORMATIK

Usage  examples  • Box  filter    

blur(src, dst, Size( filt_size_x, filt_size_y), Point(-1,-1)); – src:  Source  image  – dst:  Des0na0on  image  – Size(  w,h  ):  Defines  the  size  of  the  kernel  to  be  used  (  of  width  w  pixels  and  height  h  pixels)  – Point(-­‐1,  -­‐1):  Indicates  where  the  anchor  point  (the  pixel  evaluated)  is  located  with  respect  to  the  neighborhood.  If  there  is  a  nega0ve  value,  then  the  center  of  the  kernel  is  considered  the  anchor  point.

• Gaussian  blur  GaussianBlur( src, dst, Size(filt_size_x, filt_size_y ), 0, 0 );

– Size(w,  h):  The  size  of  the  kernel  to  be  used  (the  neighbors  to  be  considered).    and    have  to  be  odd  and  posi0ve  numbers  otherwise  the  size  will  be  calculated  using  the  and    arguments.  

– sigma_x:  The  standard  devia0on  in  x.  Wri0ng  0  implies  that    is  calculated  using  kernel  size.  – sigma_y:  The  standard  devia0on  in  y.  Wri0ng  0  implies  that    is  calculated  using  kernel  size.  

26/10/11

INFORMATIK

TD  

• Détec0on  de  visages  • Masquage  

26/10/11

INFORMATIK

Usage  examples  II  • Median  blur  medianBlur(src, dst, filt_size

– src:  Source  image  – dst:  Des0na0on  image,  must  be  the  same  type  as  src  – i:  Size  of  the  kernel  (only  one  because  we  use  a  square  window).  Must  be  odd.  

• Bilateral  filter  bilateralFilter ( src, dst, filt_size , filt_size *2, filt_size /2 )

– d:  The  diameter  of  each  pixel  neighborhood.  – sigma_col:  Standard  devia0on  in  the  color  space  (pixel  values).  – sigma:  Standard  devia0on  in  the  coordinate  space  (in  pixels)  

26/10/11

INFORMATIK

Median  blur  example  

26/10/11

INFORMATIK

43

OpenCV  cheatsheet  (2.4)  

INFORMATIK

44

INFORMATIK

Main  Structures  

INFORMATIK New  “Image”:  cv::Mat  

Mat does reference counting, so it does the right thing when it goes out of scope you can also easily make stl vectors or maps out of Mat.

INFORMATIK Mat  are  simple  

Mat  M(480,640,CV_8UC3);  //  Make  a  640x480  img  Rect  roi(100,200,  20,40);          //  Make  a  region  of  int  Mat  subM  =  M(roi);                            //  Take  a  sub  region,                                                                                                    //    no  copy  is  done      

Mat_<Vec3b>::iterator it= subM.begin<Vec3b>(), itEnd = subM.end<Vec3b>(); //0 out places in subM where blue > red for(; it != itEnd; ++it) if( (*it)[0] > (*it)[2]) (*it)[0] = 0;

INFORMATIK

Matrix    Manipula0on  

INFORMATIK

Simple  Matrix  Opera0ons  

INFORMATIK

Simple  Image  Processing  

INFORMATIK

Image  Conversions  

INFORMATIK

Histogram  

INFORMATIK

Input/Output  

INFORMATIK

Serializa0on  I/O  

INFORMATIK

Serializa0on  I/O  

INFORMATIK

GUI  (“HighGUI”)  

INFORMATIK

Camera  Calibra0on,  Pose,  Stereo  

INFORMATIK

Object  Recogni0on  

INFORMATIK

59

Examples  of  funcBonaliBes  

INFORMATIK

Canny Edge Detector

60

INFORMATIKDistance  Transform  

• Distance  field  from  edges  of  objects  

Flood    Filling  

61

INFORMATIK

Hough Transform

62 Gary Bradski, Adrian Kahler 2008

INFORMATIKScale Space

void cvPyrDown( IplImage* src, IplImage* dst, IplFilter filter = IPL_GAUSSIAN_5x5);

void cvPyrUp( IplImage* src, IplImage* dst, IplFilter filter = IPL_GAUSSIAN_5x5);

63

Chart by Gary Bradski, 2005

INFORMATIKThresholds

64

Screen shots by Gary Bradski, 2005

INFORMATIK

Histogram Equalization

65 Screen shots by Gary Bradski, 2005

INFORMATIKContours

66

INFORMATIK

Morphological  Opera0ons  Examples  §  Morphology  -­‐  applying  Min-­‐Max.  Filters  and  its  combina0ons  

Opening IoB= (IΘB)⊕B Dilatation I⊕B Erosion IΘB Image I

Closing I•B= (I⊕B)ΘB TopHat(I)= I - (IΘB) BlackHat(I)= (I⊕B) - I Grad(I)= (I⊕B)-(IΘB)

INFORMATIK

Image  textures  •   Inpain0ng:  •  Removes  damage  to  images,  in  this  case,  it  removes  the  text.  

INFORMATIK Segmentation •  Pyramid, mean-shift, graph-cut •  Here: Watershed

69 69

Screen shots by Gary Bradski, 2005

INFORMATIK

Op0cal  Flow  Algorithms  // opencv/samples/c/lkdemo.c int main(…){ … CvCapture* capture = <…> ? cvCaptureFromCAM(camera_id) :

cvCaptureFromFile(path); if( !capture ) return -1; for(;;) { IplImage* frame=cvQueryFrame(capture); if(!frame) break; // … copy and process image cvCalcOpticalFlowPyrLK( …) cvShowImage( “LkDemo”, result ); c=cvWaitKey(30); // run at ~20-30fps speed if(c >= 0) { // process key }} cvReleaseCapture(&capture);}

lkdemo.c, 190 lines (needs camera to run)

∑∑ ⎥⎦

⎤⎢⎣

⎡=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=∂∂=∂

=∂⋅

⋅∂∂+⋅∂∂=∂∂−

=+++

y

xt

yyx

yxx

II

IbIII

IIIGyxX

bXG

dtdyyIdtdxxItItyxIdttdyydxxI

,,

,),,(

,

);/(/)/(//);,,(),,(

2

2

INFORMATIK Projections

Screen shots by Gary Bradski, 2005

INFORMATIK

Stereo  Rec0fica0on  • Algorithm  steps  are  shown  at  right:  • Goal:  

– Each  row  of  the  image  contains  the  same  world  points  – “Epipolar  constraint”  

72

Result:  Epipolar  alignment  of  features:  

All:  Gary  Bradski  and  Adrian  Kaehler:  Learning  OpenCV  

INFORMATIK

Features2d  contents  Detec0on  

Detectors  available  •  SIFT •  SURF  •  FAST •  STAR  •  MSER •  HARRIS •  GFTT  (Good  Features  

To  Track)  

DescripBon  

Descriptors  available  •  SIFT •  SURF  •  Calonder •  Ferns  •  One way  

Matching  

Matchers  available  •  BruteForce •  FlannBased •  BOW

Matches filters (under construction) •  Cross check •  Ratio check

 

INFORMATIK

74

OPENCV  3.0  

INFORMATIK

OpenCV 3.0 at glance

•  Mostly compatible with OpenCV 2.x; OpenCV 1.x C API is deprecated and partially removed

•  Highlights: – even more modular and extendible – very stable API tailored for a long-term support – decent out-of-box performance: IPP, OpenCL(T-API) and NEON – lot’s of new functionality!

Aug’14 Nov’14 Apr’15 Jun’15

3.0 alpha 3.0 beta 3.0rc 3.0 3.1

Q4’15

Goal of 3.0: make a better OpenCV 2.0, cleanup API, get better performance (with T-API, IPP, NEON), shift to modular structure and enable user contributions

INFORMATIK

opencv ! opencv + opencv_contrib

OpenCV 2.x

OpenCV 3.x contributions

OpenCV 3.x

•  The “main” opencv repository includes mature algorithms and is fully supported

•  A separate contribution repository is for new CV algorithms that people want to share: http://github.com/itseez/opencv_contrib

•  Patches to the contrib repository are tested as well by our buildbot to ensure integrity!

INFORMATIK

New-style C++ API •  Unified API of all the high-level vision algorithms (face detection, optical flow estimation, stereo

matching etc.): •  Interfaces are pure abstract classes •  Implementations are completely hidden •  Properties are set/retrieved using special dedicated methods (intellisense-friendly)

// stereo StereoSGBM stereo; stereo.P1 = 5; stereo.P2 = 50; Mat left, right, disp; stereo(left, right, disp); // features 2D ORB orb; orb.set(“nfeatures”, 1000); Mat image, descriptors; vector<KeyPoint> keypoints; orb(image, keypoints, descriptors); // machine learning CvSVM svm; CvSVMParams params; params.kernel_type=CvSVM::LINEAR; param.svm_type=CvSVM::C_SVC; param.C=1; svm.train(data, resp, 0, 0, params);

// stereo Ptr<StereoSGBM> stereo = StereoSGBM::create(); stereo->setP1(5); stereo->setP2(50); Mat left, right, disp; stereo->compute(left, right, disp); // features 2D Ptr<ORB> orb = ORB::create(); orb->setMaxFeatures(1000); Mat image, descriptors; vector<KeyPoint> keypoints; orb->detectAndCompute(image, noArray(), keypoints, descriptors); // machine learning Ptr<ml::SVM> svm=ml::SVM::create(); svm->setType(SVM::C_SVC); svm->setKernel(SVM::LINEAR); svm->setC(1); svm->train(data, ROW_SAMPLE, resp);

OpenCV 2.4.x OpenCV 3.0

INFORMATIK

New Functionality and other improvements

•  Results from 20+ successful projects from GSoC 2013, 2014: –  Computational photography, Text detection, Object Tracking,

Matlab bindings etc. •  1500+ Pull Requests @ github (~800 PR’s between alpha &

the gold) •  18 new OpenCV modules! (mostly in opencv_contrib)

INFORMATIK

The easiest way to start with OpenCV

•  clone  hwp://github.com/itseez/opencv  &  opencv_contrib  •  build  it;  do  not  install  it!  •  use  the  following  CMake  file  for  you  sample  

cmake_minimum_required(VERSION 2.8)project(myopencv_sample)find_package(OpenCV REQUIRED)include_directories(${OpenCV_INCLUDE_DIRS})set(the_target "myopencv_sample")add_executable(${the_target} main.cpp) # add other .cpp # and .h files heretarget_link_libraries(${the_target} ${OpenCV_LIBS})

•  locate  OpenCVConfig.cmake  when  cmake  complains  •  you  are  good  to  go!  

INFORMATIK

Thank you!

80