Image Processing and Computer Vision with MATLAB and … · Artificial Intelligence Signal Processing Computer Vision System Toolbox™ Neural Network Toolbox™ Simulink™ Signal

1© 2014 The MathWorks, Inc.

Image Processing and Computer Vision

with MATLAB and SIMULINK

By Joss Knight

Senior Developer, GPU and Parallel Algorithms

2

Computer

Intelligence

Robotic Vision

Non-linear SP

Multi-variable SP

Cognitive

Vision

Statistics

Geometry

Optimization

Biological Vision

Optics

Smart

Cameras

Computer

Vision

Machine

Vision

Image

Processing

Physics

Imaging

Neurobiology

Mathematics

Machine

Learning

Control

RoboticsArtificial

IntelligenceSignal Processing

Physics

Imaging

Neurobiology

Mathematics

Machine

Learning

Control

RoboticsArtificial

IntelligenceSignal Processing

Computer Vision

System Toolbox™

Neural Network Toolbox™

Simulink™

Signal Processing

Toolbox™

SimScape™

Image Processing

Toolbox™SimBiology™

Embedded

Coder™

Optimization

Toolbox™

MATLAB™

Statistics Toolbox™

SimMechanics™

Control System

Toolbox™DSP System

Toolbox™

Image Acquisition

Toolbox™

The Computer Vision Ecosystem

3

Applications: Image and Video Processing

Medical imaging

Surveillance

Robotics

Automotive safety

Consumer electronics

Geospatial computing

Machine vision

and more…

4

Agenda

• Introduction • Feature Detect, Extract & Match

• Image Acquisition • Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration • Embedded Applications

5

http://www.mathworks.co.uk/products/image-video-processing


6

Agenda





• Feature Detect, Extract & Match




• Introduction





• Introduction





• Introduction


• Object Detection and Tracking



• Introduction


• Image Acquisition Demo: Live video in MATLAB

Demo: Image Acquisition App

Demo: Live video in SIMULINK




7

Image Acquisition

Acquire images and video directly into MATLAB and Simulink

Configure device properties

8

Agenda

• Introduction


• Image Acquisition Demo: Live video in MATLAB

Demo: Image Acquisition App

Demo: Live video in SIMULINK




• Introduction





• Introduction


• Image Acquisition




• Introduction






• Introduction




• 3D Vision


• Introduction




• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation

Demo: Image analysis Apps

Demo: Digital display reader

• 3D Vision


9

Standard Segmentation Workflow

10


Explore image

11


Explore image

Enhance image

12


Explore image

Enhance image

Segment image

13


Explore image

Enhance image

Segment image

Analyze image

14

Demos: Explore-Enhance-Segment-Analyze

15

Image Processing Apps

NEW!NEW!

Image Viewer

Color Thresholder

Region Analyzer

16

Optical Character Recognition

Support for English, Japanese + others

Simple interface:

>> results = ocr(I)

17

Segmentation – other techniques

Watershed and Active Contours (NEW)

18

Agenda

• Introduction




• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation

Demo: Image analysis Apps

Demo: Digital display reader

• 3D Vision


• Introduction




• 3D Vision


• Introduction




• Explore-Enhance-Segment-Analyze

• 3D Vision


• Introduction





• 3D Vision


• Introduction





• 3D Vision

• Embedded Applications

• Introduction





• 3D Vision

• Image Registration

Demo: Video Mosaicking (Simulink) • Embedded Applications

19

Image Registration

2D and 3D

Align multi-model images

20

Image Registration

NEW: Non-rigid image

registration and warping

21

Demo: Video Mosaicking

22

Agenda

• Introduction





• 3D Vision


Demo: Video Mosaicking (Simulink) • Embedded Applications

• Introduction





• 3D Vision


• Introduction





• 3D Vision



• Introduction





• 3D Vision



• Introduction




• 3D Vision




Geometry Estimation




• 3D Vision



23

Feature Detection, Extraction and Matching

Corner Template SURF MSEREdge

24

Point Tracker

Simple interface:

points = detectXFeatures(I);

tracker = vision.PointTracker;

initialize(tracker, points.Location, frame);

while ~finished

frame = step(videoFileReader);

[points, validity] = step(tracker, frame);

end

25

Geometry Estimation

RANSAC – Random Sampling and Consensus

Gets geometry from a minimal sample set and checks it against all matches

26

Agenda


Geometry Estimation




• 3D Vision



• Introduction




• 3D Vision







• 3D Vision







• 3D Vision






• 3D Vision





Demo: Multi-object Tracking


• 3D Vision



27

Object Detection and Tracking

28

Face Detection, People Detection and Foreground Detection

29

Training Image Labeler App

30

Demo: Tracking multiple objects with a Kalman Filter

31

Demo: Tracking multiple objects with a Kalman Filter

Kalman Filter interface:

– Configure:

– Predict:

– Update:

predictedCentroid = predict(kalmanFilter);

kalmanFilter = configureKalmanFilter( ...

'ConstantVelocity', centroid, ...

[200, 50], [100, 25], 100);

c2 = correct(kalmanFilter, centroid);

32

Agenda



Demo: Multi-object Tracking


• 3D Vision






• 3D Vision






• 3D Vision






• 3D Vision










• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App

Demo: 3D Reconstruction

Demo: Depth Mapping



33

3D Vision

-4-3-2-10123

-8

-6

-4

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

x (meters)z (meters)

y (

mete

rs)

Pyramid with dynamic programming and sub-pixel accuracy

0

5

10

15

34

NEW Apps: Camera Calibrator and Stereo Camera Calibrator

Simplified workflow estimating

camera intrinsic and extrinsic

parameters

Removes the effects of lens

distortion from an image

Automatically detects

checkerboard patterns

35

Demo: 3D Reconstruction of Point Cloud

Detect calibration grid to define

world coordinates

Detect and match features on

object

Project into scene using triangulate

36

Demo: Disparity Maps and Distance detection

37

Agenda



• Explore-Enhance-Segment-Analyze • 3D Vision Demo: Camera Calibrator App

Demo: 3D Reconstruction

Demo: Depth Mapping

























• Image Registration • Embedded Applications Demo: Ball Tracker

Demo: Quad-copter Model

38

Embedded Applications

56 Image Processing functions

support code generation

Most Computer Vision functionality

supports code generation

Many features generate platform-

independent code

bwdist

bwlookup

bwmorph

bwpack

bwselect

bwtraceboundary

bwunpack

conndef

edge

fitgeotrans

fspecial

getrangefromclass

histeq

im2double

im2int16

im2single

im2uint16

im2uint8

imadjust

imbothat

imclearborder

imclose

imcomplement

imdilate

imerode

imextendedmax

imextendedmin

imfill

imfilter

imhist

imhmax

imhmin

imlincomb

imopen

imquantize

imreconstruct

imregionalmax

imregionalmin

imtophat

imwarp

iptcheckmap

iptcheckconn

label2rgb

mean2

medfilt2

multithresh

ordfilt2

padarray

rgb2ycbcr

strel

stretchlim

ycbcr2rgb

imref2d

imref3d

affine2d

projective2d

39

Demos: Ball Tracker and Quad-copter model

40

Agenda









41

Explore-Enhance-Segment-Analyze

3D Vision

Apps

Feature Detection

Object Detection and Tracking

Image Registration

Image

AcquisitionImage

Viewer

Color

Thresholder

Region

Analyzer

Camera

Calibrator



43© 2014 The MathWorks, Inc.

Documents

Image Processing and Computer Vision with MATLAB and … · Artificial Intelligence Signal Processing Computer Vision System Toolbox™ Neural Network Toolbox™ Simulink™ Signal