"Lessons Learned from Bringing Mobile and Embedded Vision Products to Market," a Presentation from ARM

Copyright © 2016 ARM Ltd 1

Lessons Learned from Bringing Mobile and

Embedded Vision Products to Market Tim Hartley, Product Manager, ARM

May 3, 2016


ARM Ltd


Increased processing

power

Improving computer

vision techniques

Opportunities for new and

existing markets

The Vision Opportunity


Growth Maturity Innovation

Embedded Vision Industry Growth

Embedded

Vision Markets

Platforms &

Tools Mark

et

penetr

ation


Challenges for Companies Developing

Vision Products

Sensors

Processors & Platforms

Power & Thermal

Software Enablement


Sensors


Image Sensors: a Critical Choice

© VanGogh Imaging


Sensor Summary

Speed

• 90 /120fps necessary for VR & AR

• Above 30 or 60 fps often not supported

3D scanning

• Quality of calibration

• Depth map technology implementation

Accuracy & Range

• Consistent color characteristics

• Accurate focus

2D Scanning

• Quality of textures

• Low light performance


Processors & Platforms


Choosing the Right Platform: Processors

CPU

• Core count?

• Core groups?

• SIMD?

GPU

• GPGPU?

• OpenCL?

DSP

• Features?

• Flexibility?

• SW library?

FPGA

• SW library?

• OpenCL?

ASIC

• Dev costs


Processor Choices: Costs & Benefits

Ease/speed of

development

Performance/

watt

Flexibility

Economy


Choosing the Right Platform: a Case Study

Size and weight

Price

Performance

(Power)

© Raspberry Pi Foundation

© Hardkernel

Odroid XU4

• 8-core CPU

• 6-core GPU

• $74

Raspberry Pi 2

• 4-core CPU

• $35


Power & Thermal


Low power Sustained

performance

Burst for

responsiveness

Power & Thermal P

ow

er/

Perf

orm

ance

Time

Maximum Sustainable Power

Trip points


Avoiding Thermal Throttling

© Aerobot S.A.

© Dyson Ltd.


Software Enablement


Over 2,500 algorithms & growing!

But not optimal on low power platforms when targeting…

• Most specialized vision processors

• Heterogeneous platforms

• GPUs

• CPU vector extensions like NEON

Looking forward…

• New developments coming — e.g., better heterogeneous low power support

• Unlikely to provide long term cross-platform optimal performance

Doesn’t Everyone Just use OpenCV?


• Platform lock-in

• Proprietary vision hardware often means proprietary middleware

• Refactoring software for new platforms is time-consuming

• Feature lock-out

• Does the middleware let you exploit latest developments in vision algorithms?

• Sub-optimal use of processors

• Can the middleware target all available cores?

• How do you target multiple core types?

Vision Software: Key Risks


GPU DSP CPU GPU DSP CPU

CPU GPU DSP

Application

Middleware

Drivers

Hardware

Traditional middleware Middleware based on nodes and graphs

Middleware Architecture Today vs. Tomorrow

Graph

APIs

Vision

Nodes

Compute

Cores


OpenVX to the Rescue?

• A new computer vision API from Khronos

• Based on vision nodes and graphs

• Aimed at low power platforms from the outset

• Has potential for:

• Performance portability

• Heterogeneous deployment of vision pipelines

• Initial version defines 40 vision building blocks

• Some omissions makes extensions important

• OpenVX-SC (safety critical) version in development


Optimise

Build nodes into

dependency

graphs

Load-balance

Merge /

recompile

Merge /

recompile

Merge /

recompile

Runtime

Framework

Application

Vision API

Interface

Graph

Builder

GPU Vision

Nodes

CPU Vision

Nodes

DSP Vision

Nodes

Create

GPU

binaries

Create

DSP

binaries

GPU

DSP

CPU

HW

Repository of CV

algorithms Vision Middleware

Heterogeneous Vision Framework

Compile

Power Performance


In Summary

Expect and plan for power & thermal problems

Be aware of the risks of different middleware choices

Expect change


• The trademarks featured in this presentation are registered and/or

unregistered trademarks of ARM Limited (or its subsidiaries) in the EU

and/or elsewhere. All rights reserved. All other marks featured may be

trademarks of their respective owners

• Copyright © 2016 ARM Limited

Thank you!

Technology

"Lessons Learned from Bringing Mobile and Embedded Vision Products to Market," a Presentation from ARM