1Copyright © 2015 Intel Corporation

Mostafa Hagog

12 May 2015

Developing High-Performance, Energy-

Efficient Vision Solutions Using Open-Standard

Libraries and APIs

2Copyright © 2015 Intel Corporation

Computer Vision Ecosystem From Cloud to

Wearables

Smaller

Low Power

Low Cost

High Performance

Efficiency &

High-end

Cloud &

Server

Automotive

Surveillance

& Retail

PCs &

Mobile

Devices

Wearables

Embedded

& Robotics

3Copyright © 2015 Intel Corporation

Computer Vision in the Cloud

Social Media Tagging

Web Visual SearchImages and

Videos

Across the

Web

Listing

Cloud ApplicationsPersonal

Files Stored

in the cloud

Video &

Image

Indexing

Image

Indexing

* Other names and brands may be claimed as the property of others

Shared

Images &

Video

4Copyright © 2015 Intel Corporation

Computer Vision in the Cloud

Social Media Tagging

Web Visual SearchImages and

Videos

Across the

Web

Listing

Cloud ApplicationsPersonal

Files Stored

in the cloud

Video &

Image

Indexing

Image

Indexing

Large scale high performance systems

High accuracy and high throughput computer vision solutions

Deployment in distributed systems

Generally fits highly programmable processors

* Other names and brands may be claimed as the property of others

Shared

Images &

Video

5Copyright © 2015 Intel Corporation

Computer Vision in PCs and Mobile Device

Gesture

Recognition

Intel RealSense

Microsoft Kinect

Samsung

Gesture core UIQUALCOMM

Embedded Gesture

Vision

Indoor

navigation

Intel RealSense

Google Tango

Augmented RealityVuforia

Metaio

RealSense

Low Latency

UI Integration

Low Latency

UI Integration

Sensor integration

Low latency

Scene Capturing

Scene Rendering

* Other names and brands may be claimed as the property of others

6Copyright © 2015 Intel Corporation

Computer Vision in PCs and Mobile Device

Gesture

Recognition

Intel RealSense

Microsoft Kinect

Samsung

Gesture core UIQUALCOMM

Embedded Gesture

Vision

Indoor

navigation

Intel RealSense

Google Tango

Augmented RealityVuforia

Metaio

RealSense

Low Latency

UI Integration

Low Latency

UI Integration

Sensor integration

Low latency

Scene Capturing

Scene Rendering

The Key:

Heterogeneous compute efficiency and scenario based utilization of

compute resources

* Other names and brands may be claimed as the property of others

7Copyright © 2015 Intel Corporation

Wearables and Embedded

Embedded

Non Traditional UI Low Power

Size & Cost

Constraints

Autonomous and

Interactive

8Copyright © 2015 Intel Corporation

Intel® RealSense™ Technology—Example for

PC & Mobile Devices

Real time (30 fps) augmented reality

experiences in 2m x 2m x 2m region

of interest

OpenCL™ as a heterogeneous

computer vision acceleration API (CPU

and GPU)

Highly accurate real time back-

ground subtraction using Intel®

RealSense™ camera

Using OpenCV 3.0 to transparently

select CPU & GPU

9Copyright © 2015 Intel Corporation

City Wide

Cameras

Computer Vision in the Cloud—Example

License plate recognition

Vehicle model detection

Match plate to model

Real-Time Fake Plate Tracking

Using OpenCV, OpenCL™ and Intel® Media SDK. Real-

time fake plate tracking on 64 camera feeds on a

single server chip

10Copyright © 2015 Intel Corporation

GRAY GRAY GRAY

YUVRGBRGB

Tile

RGB GRAY

Resize

Remap

Conv

TileAverag

eTile

Threshold

Color Conv

Tile

Hough

ChnlExtract

Kalman

ValidLines

Ransac Lines

FillLine

Lines

Tile

REMAP

Mean Std Dev

Tile

Tile

HoughValidLines

Lines

LinesLines

RGBFill

Line

Lines

ControlTile

Example Embedded Domain—

(A Working Example of Lane Departure Warning)

11Copyright © 2015 Intel Corporation

GRAY GRAY GRAY

YUVRGBRGB

Tile

RGB GRAY

Resize

Remap

Conv

TileAverag

eTile

Threshold

Color Conv

Tile

Hough

ChnlExtract

Kalman

ValidLines

Ransac Lines

FillLine

Lines

Tile

REMAP

Mean Std Dev

Tile

Tile

HoughValidLines

Lines

LinesLines

RGBFill

Line

Lines

ControlTile

Example Embedded Domain—

(A Working Example of Lane Departure Warning)

OpenVX™ abstraction

Transparent tiling , data/task Parallelism

HW agnostic: Targets CPU, GPU and dedicated HW

12Copyright © 2015 Intel Corporation

Similar Aspects Across Domains

Common CV problems

Detection and recognition

Motion analysis and tracking

Scene reconstruction

Classification

CV building blocks

Image processing

Feature detection,

extraction and matching

Machine learning and

classification

CV application

Solve one/more CV

problems

Uses several CV

building blocks

Tune

algorithms

differently

Common

optimization

practices*CV (Computer Vision)

13Copyright © 2015 Intel Corporation

Many Libraries and APIs

Cuda* BLAS

Other SDKs/Libraries:

Matlab*, Caffe*,

Point-Cloud …

Intel®

MKLCNN OpenCL

Computer Vision Applications

APIs for Parallel

Programming

OpenCV

FastCV* Intel®

IPP

OpenCV like DSP

Libraries

OpenCL™, CUDA, OpenMP*, Render-Script

NPP* OpenCV OpenCL™

Optimized

Libraries

High-level Libraries

Today across domains:

HW/vendor proprietary libraries/APIs

Developers have to change libraries/APIs

And, change optimization practices

* Other names and brands may be claimed as the property of others

14Copyright © 2015 Intel Corporation

Standard APIs and Libraries to the Rescue

OpenCV

portable cross

platforms

OpenCL

supported in many GPUs

OpenVX™

graph / vision functions

abstraction

Vendor/HW specific

optimization

Computer Vision Applications

Domain/Vendor Specific Libraries and SDKs

15Copyright © 2015 Intel Corporation

Standard APIs and Libraries to the Rescue

OpenCV

portable cross

platforms

OpenCL

supported in many GPUs

OpenVX™

graph / vision functions

abstraction

Vendor/HW specific

optimization

Computer Vision Applications

Domain/Vendor Specific Libraries and SDKs

OpenCV/OpenCL™ for cross domain portability

OpenVX™ to achieve power, performance, cost and size optimization

across the computer vision domains

16Copyright © 2015 Intel Corporation

Questions?

17Copyright © 2015 Intel Corporation

• "Harnessing Heterogeneous Computing for Cloud- and Mobile-Based

Visual Search," A Presentation From Baidu

• October 2013 Embedded Vision Summit Technical Presentation: "Vision-

Based Gesture User Interfaces," Francis MacDougall, Qualcomm

• Building an Integrated Vision and Robotics Packaging Line for Cosmetics

Using NI LabVIEW and Vision Hardware

• October 2013 Embedded Vision Summit Keynote: "Embedding Computer

Vision in Everyday Life," Mario E. Munich, iRobot

References