Create Natural User Interfaces with the Intel® RealSense™ SDK Beta 2014

Hon Pong (Gary) HoIntel Perceptual Computing

2

Contents:• The Intel® RealSense™ SDK for Windows beta

• What is Intel® RealSense™ Technology?

• Why should developers care about Intel® RealSense™ Technology?

• Disclosures of the technical details of Intel® RealSense™ Technology

• Intel® RealSense™ SDK Developer Programs

• How We Help You to Help Us

• Next Steps: Getting Started with Intel® RealSense™ SDK for Windows beta

• Q&A

3

Interfaces for computing devices have evolved very little in 30 years

What comes next?

4

Click icon to add picture

The Intel® RealSense™ SDKFree Tools and APIs for building natural user interfaces. Public Beta for Windows available Q3 2014

Accessible for beginners. Open and extensible for experts.

Support for Windows 8.1 today. Android support coming.

Reach a broad-base of customers by writing apps for the Intel RealSense 3D Camera.

Stand out in a crowded market. Delight your customers.

Works with languages, frameworks, and game engines you already use

5

HandsFaceSpeechEnvironment

The Intel® RealSense™ SDK understands 4 basic types of inputUse standalone or in combinationCategories of

InputCapabilities Features

Hands • Hand and Finger Tracking

• Gesture Recognition

• 22-point Hand and Finger Tracking• 9 static and dynamic mid-air gestures

Face • Face Detection and Tracking

• Multiple Face Detection and tracking• 78-point Landmark Detection (facial features)• Emotion Recognition (7 emotions, coming post-

Beta)• Pulse Estimation• Face Recognition (Coming post-beta)

Speech • Speech Recognition • Command and Control• Dictation• Text to Speech

Environment • Segmentation• 3D Scanning• Augmented Reality

• Background Removal• 3D Object / Face / Room Scanning (Coming post-

beta)• 2D/3D Object Tracking• Scene Perception (coming post-beta)

6

Intel® RealSense™ 3D Camera for Windows: Designed for close-range interactions

Competitive technologies focus on a living-room experience or a sub-set of Intel RealSense technology features

20 cm

120 cm

Intel®RealSense™ 3D camera

7

The Intel® RealSense™ 3D Camera

* Other names and brands may be claimed as the property of others

8

The Developer KitWhen we roll out the public beta SDK in Q3 of 2014, we’ll release a developer kit to help you get started

Kit includes The Intel® RealSense™ SDK for Windows Beta

Peripheral depth camera

Full VGA depth resolution

1080p RGB camera

0.2 – 1.2 meter range

USB 3.0 interface

Requires 4th generation Intel® Core™ processor (or later)

Available for purchase at intel.com/software/realsense

9

Windows today….Android on the wayLook for more details on Intel® RealSense™ SDK for Android later this year

* Other names and brands may be claimed as the property of others

10

What Kinds of Applications can Intel® RealSense™ Technology Enable?

Capture and Share

Immersive Collaboration/Creation

Interact Naturally

Gamingand Play

Learning andEdutainment

Innovate on top of our application categories. YOU

will come up with the next big thing

11

Immersive Collaboration/

Creation• Via the 3D camera, software

can “segment” an environment, allowing an app to remove the background

• With the background removed, developers can create applications ideal for a combination of video chat and collaboration

• Make apps that allow presenters to be overlaid on top of applications, allow two people to share an experience on the web, or for a single person to use this “digital green screen” to create compelling original content

12

Interact Naturally• Use our extensive library

of gestures, facial analysis algorithms, and speech recognition algorithms to build interfaces that are truly natural

• Leverage our carefully developed user-experience guidelines document to ensure the interfaces you build will delight your customers

13

Gaming and Play• Differentiate your game

with gesture, speech, augmented reality enhanced with depth, facial analysis, and more

• Use the user’s pulse to determine the stability of a scope. Allow a user to physically move their head to peak around a corner. Use gesture for a lock-picking challenge. The possibilities are endless

14

Capture and Share• Partnership with

scanning/printing experts, 3D Systems*

• Rotate a small object in front of the 3D camera to create a 3D mesh + color

• Developers can import scans from a proprietary 3DS* app or eventually build scanning into their own application

• The model is scan, edit, share. Upload scans to social media, or sent to a 3D printer.

* Other names and brands may be claimed as the property of others

15

Learning and Edutainment

• Use augmented reality, powered by Metaio to track 2D or 3D objects and incorporate them into a learning experience

• Tracking is significantly improved by the 3D camera. Enjoy both enhanced precision and accuracy

• Interact with stories using both facial/emotion recognition and gesture. Disguise learning as play.

16

SDK Supported FeaturesRequired Hardware A system with a minimum of a 4th generation Intel® Core™ processor

(codenamed Haswell), either IA-32 or Intel® 64, with integrated depth camera

Required OS Microsoft* Windows 8.1 OS (32- and 64-bit)Microsoft* Windows 8.1 Metro (coming soon)

Supported Programming Languages

C++, C#, JavaScript, Java (JDK 1.7.0_11 or later for face, voice, and gesture)

Supported IDE Microsoft* Visual Studio C++ 2010-2013 with service pack 1 or newerEclipse.org IDE

Supported Development Tools

-Microsoft* .NET 4.0 Framework for C# development-Unity* PRO 4.0.0 or later for Unity game development-Processing* 2.0.7 or later for Processing framework development-openFrameworks* v0071 or later for openFrameworks development-Havok* Vision SDK 2012.2.1 or later for Havok Vision SDK development-Intel® Media SDK 2014-Intel® SDK for OpenCL Applications 2013

* Other names and brands may be claimed as the property of others

17

Additional SDK Features

Power Management State

Apps can manage battery life

Privacy Notification Tool Notifies user when camera is turned on by an app

Input Device Manager Multiple applications can access data from the Camera simultaneously

Multi-Mode Support Support multiple usage modes within a single app (e.g. finger tracking + speech + face tracking) or between apps

Extensible Framework Developers can plug in their own algorithms. New usage modes can be added. New devices can be supported.

18

18

Web / HTML5

Gestures and Facial Analysis Unity* Web Player games HTML5 / JavaScript (interactive websites)

Unity Web Player App

HTML5 / JavaScript

SDK Local Web-Socket Server

Intel® RealSense™ SDK runtime

Operating System

Web Socket Communication

127.0.0.1:4181 Internet Browser

Trigger face and gesture events, just like in C# applicationsSwitch to the web player platform for deployment

* Other names and brands may be claimed as the property of others

1919

Unity* Productivity ExtensionA set of scripts that provides configurable actions/rules based on the capabilities provided in the SDK.

• Programming: Associate your game objects with the action scripts.

• Programming: Create rules. Associate your game objects with rules.

Drag and drop programmingWrite less code!

* Other names and brands may be claimed as the property of others

20

Enabling the broad ecosystem through contests and investments

How We Help You to Help Us

• RealSense Resource Center (software.intel.com/RealSense)

• RealSense Community (Blogs & Forums)

Online

• Hackathons (12+ worldwide planned for 2014)• Tradeshow classes/labs

Physical Events

• Intel® RealSense™ App Challenge 2014• Share Your App Showcase

Promotions

21

Enabling the broad ecosystem through contests and investmentsOnline Resources

22

Enabling the broad ecosystem through contests and investmentsPhysical Events

23

Enabling the broad ecosystem through contests and investmentsPromotions

24

“perceptual computing [now RealSense]…a phrase that you’re about to hear a lot of…it’s the future of computing and it’s freaking awesome.”− Luke Hopewell, Gizmodo

http://www.gizmodo.com.au/2013/06/the-future-of-computing-is-perceptual/

25

Getting Started

• Sign up now to learn when the Intel® RealSense™ SDK and development kit are available

• Provided by Intel Free Intel® RealSense™ SDK for Windows Beta in Q3 2014 Intel® RealSense™ Developer Kit available for purchase Huge opportunity to reach customers with integrated 3D camera Works with languages/frameworks/engines you already use High-level APIs for NUI beginners. Low-level APIs for NUI experts

All products, software, computer systems, dates and figures specified are preliminary based on current expectations, and are subject to change without notice.

software.intel.com/RealSense

Seize The Opportunity

26

Q&A