Jack Pinches

INFO410 & INFO350 S2 2015

INFORMATIONSCIENCE

Computer Vision I

INFO410 S2 2015 COMPUTER VISION I SLIDE 2INFORMATION SCIENCE

Topics Definition Related Fields Application Examples Application Overview

Typical Tasks Methods

Feature Detection Feature Description Feature Matching Exam Questions Resources

Defining Computer Vision

INFO410 S2 2015 COMPUTER VISION I SLIDE 4INFORMATION SCIENCE

Formal Definition Making useful decisions about real physical objects and

scenes based on images (Shapiro & Stockman, 2001). Extracting descriptions of the world from pictures of

sequences of pictures (Forsyth & Ponce, 2003). Analysing images and producing descriptions that can be

used to interact with the environment (Horn, 1986). Designing representations and algorithms for relating images

to models of the world (Ballard & Brown, 1982).

INFO410 S2 2015 COMPUTER VISION I SLIDE 5INFORMATION SCIENCE

Informal Definition

Consists of acquiring, processing, analysing images/video of the real world

To produce information or descriptions Often used as part of decision making

INFO410 S2 2015 COMPUTER VISION I SLIDE 6INFORMATION SCIENCE

Definition Extract “meaning” from

pixels

What we see What a computer sees

http://www.cs.unc.edu/~lazebnik/spring11/lec01_intro.pdf

INFO410 S2 2015 COMPUTER VISION I SLIDE 7INFORMATION SCIENCE

Why is it important? Images and videos are everywhere, and growing in

numbers. Trying to mimic human vision and understanding with

computers. Humans have good visual understanding – large

amount of the brain devoted to visual processing. Difficult to mimic.

Human level visual perception may be strong AI or “AI complete”.

INFO410 S2 2015 COMPUTER VISION I SLIDE 8INFORMATION SCIENCE

Related Fields

https://en.wikipedia.org/wiki/Computer_vision#/media/File:CVoverview2.svg

Application Examples

INFO410 S2 2015 COMPUTER VISION I SLIDE 10INFORMATION SCIENCE

ApplicationsAgriculture

Augmented Reality

Autonomous Vehicles

Biometrics

Character Recognition

Forensics

Industrial Quality Inspection

Face Recognition

Medical Image Analysis

Pollution Monitoring

Process Control

Remote Sensing

Robotics

Security and Surveillance

Transport

http://www.bmva.org/visionoverview

INFO410 S2 2015 COMPUTER VISION I SLIDE 11INFORMATION SCIENCE

Application Examples

Face Detection: “Feature” included

on many digital cameras

https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect1.pdf\

INFO410 S2 2015 COMPUTER VISION I SLIDE 12INFORMATION SCIENCE

Application Examples Automatic number plate recognition:

Used by law enforcement Automatic tolling stations Road surveillance

https://en.wikipedia.org/wiki/File:California_license_plate_ANPR.png

INFO410 S2 2015 COMPUTER VISION I SLIDE 13INFORMATION SCIENCE

Application Examples

Facial Recognition: Airport Security Check the person

matches the passport photo

Used in NZ

http://www.secureidnews.com/news-item/uks-stansted-airport-deploys-biometric-e-passport-gates/

Application Overview

INFO410 S2 2015 COMPUTER VISION I SLIDE 15INFORMATION SCIENCE

Typical Tasks Detection:

Image scanned for a specific condition. E.g. detection of humans as part of a security system, or vehicles at automatic toll system.

Recognition: Pre-learned objects can be recognized. E.g. recognition of

a specific album cover.

Description: Extracting data about specific object of interest. E.g.

colour information, 2D position or 3D position.

INFO410 S2 2015 COMPUTER VISION I SLIDE 16INFORMATION SCIENCE

Application Methods Image Acquisition:

Image is produced by one or several image sensors. The image may have some associated meta data.

Pre-processing: Process image to ensure it meets requirements. Examples:

Re-sample the image to check it’s what’s expected to be outputted.

Noise reduction so sensor noise doesn’t add false information.

Contrast alteration so important features can be detected.

INFO410 S2 2015 COMPUTER VISION I SLIDE 17INFORMATION SCIENCE

Application Methods Feature extraction:

Image features are extracted from the image data. Examples:

Lines, edges, intersections Complex examples could include texture, shape, or

motion

Detection: Decide which image regions are required for further

processing.

INFO410 S2 2015 COMPUTER VISION I SLIDE 18INFORMATION SCIENCE

Application Methods High level processing:

Usually small set of the original data. Image region which assume contains a specific object.

Examples: Estimation of object pose or size Object recognition

Final decision: Make final decision for the application. Examples:

Match/no match in recognition application Flag if human review needed (medical, military

applications)

Feature Detection

INFO410 S2 2015 COMPUTER VISION I SLIDE 20INFORMATION SCIENCE

Feature Detection Methods that try to extract

image information. Decisions made if there is an

image feature or not. Features will be a subset of the image. Different approaches available.

INFO410 S2 2015 COMPUTER VISION I SLIDE 21INFORMATION SCIENCE

Feature Detection – Image Features Edges:

Points where there is a boundary between two image regions. Can be almost any shape.

http://stackoverflow.com/questions/11319937/c-sharp-paint-image-within-edges

INFO410 S2 2015 COMPUTER VISION I SLIDE 22INFORMATION SCIENCE

Feature Detection – Image Features Corners (interest points):

Originally called corner as early algorithms performed edge detection. Then looked for edges which have rapid direction change.

Can look for high level of curvature.

http://fahmifahim.com/2010/10/22/opencv-corner-detection/

INFO410 S2 2015 COMPUTER VISION I SLIDE 23INFORMATION SCIENCE

Feature Detection – Image Features Blobs:

Aimed at detecting regions of an image. The region may differ in properties

(such as brightness or colour)compared to surrounding images.

https://wwwx.cs.unc.edu/~sjguy/CompVis/Features/

Feature Description

INFO410 S2 2015 COMPUTER VISION I SLIDE 25INFORMATION SCIENCE

Feature Description

Feature Description Methods: SIFT: Scale Invariant Feature Transform SURF: Speeded up Robust Features GLOH: Gradient Location and Orientation Histogram HOG: Histogram of Orientated Gradients

INFO410 S2 2015 COMPUTER VISION I SLIDE 26INFORMATION SCIENCE

Feature Description - SIFT Detect and describe features in images. Developed by David Lowe in 1999. Algorithm described in:

Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision 2. pp. 1150–1157. doi:10.1109/ICCV.1999.790410

C++ Implementation: http://www.robots.ox.ac.uk/~vedaldi/code/siftpp.html

https://copilosk.fbk.eu/images/f/f6/Sift.pdf

INFO410 S2 2015 COMPUTER VISION I SLIDE 27INFORMATION SCIENCE

Input an image. Output is a list of points on the image each associated to a vector

of low level descriptors. These points are called the keypoints.

Provide a local image description. The points and their descriptions are invariant by rescaling,

orientation, and some changes of illumination. Used to find visual correspondences between images for different

applications, such as image alignment or object recognition.

Feature Description - SIFT

INFO410 S2 2015 COMPUTER VISION I SLIDE 28INFORMATION SCIENCE

Feature Description – SIFT Image Description

813 Keypoints

https://copilosk.fbk.eu/images/f/f6/Sift.pdf (slide 4)

INFO410 S2 2015 COMPUTER VISION I SLIDE 29INFORMATION SCIENCE

Feature Description – SIFT Application

Align two imagesto produce a newimage.

Feature Matching

INFO410 S2 2015 COMPUTER VISION I SLIDE 31INFORMATION SCIENCE

Feature Matching – Why do it? Fusing image from different sensors or times Detect changes between images Produce a new large image by overlapping/stitching smaller images

E.g. Panorama Object recognition by comparing an image to a reference image

or model

INFO410 S2 2015 COMPUTER VISION I SLIDE 32INFORMATION SCIENCE

Feature Matching

INFO410 S2 2015 COMPUTER VISION I SLIDE 33INFORMATION SCIENCE

Feature Matching Different Techniques:

SIFT RANSAC (Random Sample Consensus) Cross Correlation Nearest Neighbour Techniques Exhaustive Comparison

INFO410 S2 2015 COMPUTER VISION I SLIDE 34INFORMATION SCIENCE

Feature Matching – What makes a good feature? Uniqueness:

No ambiguous matches between images Look for “interest points”: image regions that unique are

(or at least unusual) Defining unusual?

INFO410 S2 2015 COMPUTER VISION I SLIDE 35INFORMATION SCIENCE

Feature Matching – With SIFT Can detect interest points, create feature descriptors. Compare feature descriptors between images.

INFO410 S2 2015 COMPUTER VISION I SLIDE 36INFORMATION SCIENCE

Feature Matching – With Cross Correlation Determine the important features:

Unique colour/structure of the neighbourhood Geometry or topology of neighbourhood

Vectors show the movement of the neighbourhood.

www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt

INFO410 S2 2015 COMPUTER VISION I SLIDE 37INFORMATION SCIENCE

Feature Matching

www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt

1) Determine points2) Match Neighbourhoods

Potential Exam Questions

INFO410 S2 2015 COMPUTER VISION I SLIDE 39INFORMATION SCIENCE

Exam Questions

Define computer vision, and include the typical tasks that could be expected in a computer vision application.

Explain why it is important to use good feature descriptors for feature matching, use applications examples to support your reasoning.

Explain how you would design and implement a license plate detection and recognition application.

INFO410 S2 2015 COMPUTER VISION I SLIDE 40INFORMATION SCIENCE

Resources Computer Vision:

http://www.cs.unc.edu/~lazebnik/spring11/ http://users.eecs.northwestern.edu/~yingwu/teaching/EECS432/Notes/intro.pdf

SIFT: https://copilosk.fbk.eu/images/f/f6/Sift.pdf http://

www.quora.com/What-is-the-best-explanation-of-SIFT-that-you-have-seen-or-heard Features, Matching, and Detection:

http://www.cs.toronto.edu/~kyros/courses/2503/Handouts/features.pdf https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect6.pdf www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt www.cse.msu.edu/~stockman/RegTutorial/matchingPart2.ppt http://people.cs.umass.edu/~elm/Teaching/ppt/370/370_10_RANSAC.pptx.pdf