Upload
others
View
19
Download
0
Embed Size (px)
Citation preview
Introduction to Computer Vision Harsh Sinha
Introduction to Computer Vision for Robotics
AE640A Autonomous Navigation5th March, 2019
1
Introduction to Computer Vision Harsh Sinha
Lecture Outline● Introduction
○ What is CV?○ Overview of the field○ A look at history○ Hard Problem?
● Human Vision System & the Machine○ The human vision system○ Fooling humans○ The computer vision system
● Images as matrices.○ How cameras work to produce these matrices○ Meaning of Intensity, Color etc○ Shoutout to Image Processing
2
Introduction to Computer Vision Harsh Sinha
Lecture Outline● Camera Model
○ Pinhole Camera Model○ Intrinsic Camera Matrix○ Camera Calibration
3
Introduction to Computer Vision Harsh Sinha
Introduction
4
Introduction to Computer Vision Harsh Sinha
What is Computer Vision?
5
Computer Vision
SystemImage Information
Image Processing Image
Uni
vers
e
Introduction to Computer Vision Harsh Sinha
What is Computer Vision?
6
Image Credits: CS131, Fall ‘18, Stanford
Introduction to Computer Vision Harsh Sinha
What is Computer Vision?● Computer Vision is deals with extracting information regarding the 3D world
we live in using a single or a bunch of images.
● Computer Vision like most other fields today, is at the junction of numerous disciplines from Biology to Computer Science and has applications only limited by our imagination.
7
Introduction to Computer Vision Harsh Sinha
Overview of the field
8
Image Credits: XKCD, 1425, 2014
Introduction to Computer Vision Harsh Sinha
Overview of the field
9
Image Credits: XKCD, 1425, 2014Image Credits: https://tinyurl.com/y53by9prImage Credits: https://tinyurl.com/y53by9pr
Introduction to Computer Vision Harsh Sinha
Overview of the field
10
Image
Uni
vers
e
(Image Processing + )
Computer Vision System
What kind of Information?
Introduction to Computer Vision Harsh Sinha
Overview of the field
11
What kind of Information?
Image Credits: https://tinyurl.com/lxuex6o
Image Credits: Karpathy, CVPR’15
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection
12
Recognition: Cat?Image: https://tinyurl.com/yanp2o5e
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection
13
Recognition: Cat?Localization: Where is the cat?
Image: https://tinyurl.com/yanp2o5e
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection
14
Object Detection: Which Objects are here and where?
Image: https://tinyurl.com/y4ly96rd
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation
15
Segmentation: Which pixels belong to which object?
Credits: Own Work
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements
16
Image Colorization: From Grayscale to Colored Images
Credits: Richard Zhang, CVPR 2016
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements
17
Image Enhancement: Real Time Image Enhancement
Credits: Michael Gharbi, ACM Graphics 2017
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements
18
Super Resolution: Upsampling Images while preserving quality
Credits: https://github.com/tensorlayer/srgan
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text
19
Image Description: Automatic semantic description for images
Credits: Karpathy, CVPR 2015
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text5. Image Generation
20
Image Generation: A style based generator architecture for GANs
Credits: Tero Karras, arXiv 2018
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text5. Image Generation6. Motion Estimation
21
Optical Flow: Lucas Kanade method for motion estimation
Credits: https://tinyurl.com/y5rloh3g
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text5. Image Generation6. Motion Estimation7. 3D reconstruction from Images
22
3D Reconstruction: REMODE, Real Time Reconstruction
Credits: Matia Pizzoli, ICRA 2014
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text5. Image Generation6. Motion Estimation7. 3D reconstruction from Images8. Visual SLAM
23
3D Reconstruction: REMODE, Real Time Reconstruction
Credits: Matia Pizzoli, ICRA 2014
Introduction to Computer Vision Harsh Sinha
Overview of the fieldPrimary themes in Computer Vision are:
1. Object Detection2. Segmentation3. Image Modifications/Enhancements4. Image to Text5. Image Generation6. Motion Estimation7. 3D reconstruction from Images8. Visual SLAM9. Biometrics and more ...
24
Biometrics : Fingerprint Detection, Apple Face ID
Credits:https://tinyurl.com/y2a7wybz,TheVerge Youtube
Introduction to Computer Vision Harsh Sinha
A look at history● Robert Nathan started writing
computer programs for enhancing images from NASA’s spacecraft’s at Jet Propulsion Lab, NASA.
● The Summer Vision Project: Project at MIT to solve a significant part of visual system. Primary Objective was to divide the image into object, background and chaos regions, over the course of a summer.
25
Credits: EE604, nasa.gov
Credits: https://tinyurl.com/y6bpo4nk
Introduction to Computer Vision Harsh Sinha
A look at history
26
Credits: Prof. Tanaya Guha, EE698K
Introduction to Computer Vision Harsh Sinha
A look at history
27
Credits: Prof. Tanaya Guha, EE698K
Introduction to Computer Vision Harsh Sinha
Hard Problem?● Why are we still working on roughly the same problem as the “summer vision
project”?● Why is it that creating 3D models of chairs is easier than identifying them?
28
Introduction to Computer Vision Harsh Sinha
Hard Problem?● Why are we still working on roughly the same problem as the “summer vision
project”?● Why is it that creating 3D models of chairs is easier than identifying them?
➔ There is a large between some ~1920x1080x3 numbers and the high-level abstract meaning we associate with them.
➔ Images are 2D representation of information from 3D world.
29
Introduction to Computer Vision Harsh Sinha
Human Vision System & Computer Vision System
30
Introduction to Computer Vision Harsh Sinha
The human vision system
31
Credits: https://tinyurl.com/y6bkhnqa
Introduction to Computer Vision Harsh Sinha
The human vision system
32
Introduction to Computer Vision Harsh Sinha
The human vision system
33
Credits: Ulas Bagci, UCF
Introduction to Computer Vision Harsh Sinha
Fooling humans
34
Credits: Oleg Shuplyak, Pinterest
Credits: https://tinyurl.com/y49rp7sd
Credits: Wikipedia, Spinning Dancer
Introduction to Computer Vision Harsh Sinha
The (human) (computer) vision system
35
Credits: CS131, Stanford
Introduction to Computer Vision Harsh Sinha
Fooling Computers
36
Credits: Wikipedia, Barber Pole IllusionCredits: https://tinyurl.com/l5pwp6t
Introduction to Computer Vision Harsh Sinha
Images as Matrices
37
Introduction to Computer Vision Harsh Sinha
Camera Models
38
Introduction to Computer Vision Harsh Sinha
Camera Models
39
Credits: https://tinyurl.com/y6qen2vb
Introduction to Computer Vision Harsh Sinha
Camera Models
40
Credits: https://tinyurl.com/y6qen2vb
Not this one but models as in modelling a phenomena
Introduction to Computer Vision Harsh Sinha
Camera Models● Like so many things in engineering, we create a simple “model” of a camera
to which is easy to understand and can approximate the actual functioning of a camera to a good degree.
● There are different models:■ Pinhole camera model■ Lens model■ ...
41
Introduction to Computer Vision Harsh Sinha
Pinhole camera model
42
aperture
Credits: Wikipedia, Pinhole Camera Model
Introduction to Computer Vision Harsh Sinha
Pinhole camera model
43
aperture
Credits: Wikipedia, Pinhole Camera Model
Introduction to Computer Vision Harsh Sinha
Pinhole camera model
44
aperture
Credits: Wikipedia, Pinhole Camera Model
Introduction to Computer Vision Harsh Sinha
Pinhole camera model
45
where x’i = yi and z = x3
where c is an offset in pixels
Can we make this into a matrix multiplication of the form p’ = Mp?
Introduction to Computer Vision Harsh Sinha
Intrinsic camera matrix
46
Credits: Edwin Olson, University of Michigan
Introduction to Computer Vision Harsh Sinha
Intrinsic camera matrix
47
Credits: Edwin Olson, University of Michigan
Introduction to Computer Vision Harsh Sinha
Intrinsic camera matrix
48
Credits: Edwin Olson, University of Michigan
Introduction to Computer Vision Harsh Sinha
Intrinsic camera matrix
49
Credits: Edwin Olson, University of Michigan
Introduction to Computer Vision Harsh Sinha
Intrinsic camera matrix
50
Credits: Edwin Olson, University of Michigan
Introduction to Computer Vision Harsh Sinha
Camera calibration
51
Credits: Gaurav Pandey, Ford
Calibration Rig ImagePi
Pci