Embed Size (px)
Citation preview
Introduction to Google Project Tango and Intel® RealSense™Technologies and Usages
Francesca TosiIntel Software Innovator
1
#droidconIT
About me
Francesca Tosi
• Intel Software Innovator
• Freelance Developer
• GDG-Firenze/Ancona Manager
@francescatosi
2
#droidconIT
Short Intro
3
#droidconIT
Computer Vision
Computer Vision on Android
- OpenCV C/C++ lib http://opencv.org
- BoofCV pure Java lib http://boofcv.org
4
#droidconIT
Computer VisionComputer Vision
- Face recognition - Object tracking - Space & lines detection
5
#droidconIT
2D World
6
#droidconIT
2D World
7
#droidconIT
3D World
8
#droidconIT
Project Tango & Intel RealSense
3D World
9
#droidconIT
Intel RealSense ZR300 Details
10
Video here: https://youtu.be/8ceDOk0qJbM
#droidconIT
Intel RealSense ZR300
11
RealSense R200
Fisheye camera
#droidconIT
Google Project Tango
12
#droidconIT
Google Project Tango
• Motion Tracking
• Area Learning
• Depth PerceptionRealSense solution
Fisheye camera
13
#droidconIT
Google Project Tango
Motion Tracking
Gives the relative position and orientation of the device
14
#droidconIT
Google Project Tango
Motion Tracking
how it works in practice
—> EXPLORER
15
#droidconIT
Google Project Tango
Area Learning
Gives the absolute position and orientation in pre-registered environments
16
#droidconIT
Google Project Tango
Area Learning
Gives the absolute position and orientation in pre-registered environments
17
#droidconIT
Google Project Tango
Depth Perception
Gives the distance of all the elementsin front of the camera
18
#droidconIT
Google Project Tango
Depth Perception
how it works in practice
—> Constructor
19
#droidconIT
Google Project Tango - SDK
Tango SDK
developers.google.com
20
#droidconIT
Tango Cube Stacker & Tango BlasterDemos – Tango
21
#droidconIT
Intel RealSense Technologies
22
#droidconIT
23
RealSense Cameras - 2015
devkitsavailable today
products
User-facing (F200) Rear-facing (R200)
World Facing
Intel® RealSense™ SDK Modules
Scene Perception
Enhanced Photography
Augmented Reality
Person & Face Tracking
Gesture Recognition
Background Removal
Object Recognition and
Tracking
Speech Recognition
3D Object Scanning
Hand & Blob Tracking
User Facing
24
#droidconIT
Intel® RealSense™ SDK Architecture (Windows-only atm)
*
Your Application HERE
- C++
25
#droidconIT
Intel® RealSense™ SDK
- C++
26
Technical Specification
• Required OS •Microsoft Windows* 8.1 or 10 OS 64-bit •Support for Android* Coming Soon**
• Supported Intel® RealSense™ Cameras
•SR300 – Front-facing •R200 – Rear-facing
• Supported Languages •C++, JavaScript* •C# (Microsoft .NET* 4.0 Framework is required) •Java* (JDK 1.7.0_11 or later) •Processing* 2.1.2 or later
• IDE used for Samples and Tutorials •Background Segmentation •3D Object / Face / Room Scanning •2D/3D Object Tracking
• Supported browsers for JavaScript* development
•Microsoft Internet Explorer* 10.0.13 •Google Chrome* 33.0.1750.146 •Mozilla Firefox* 27.0.1
#droidconIT
Intel® RealSense™ Driver for Linux & OS X
- C++
27
Librealsense
• Features •Multi-camera capture: librealsense can be used with all 3 camera models (F200, SR300, and R200) and can provide multi-camera capture with either single or multiple models simultaneously. •Native streams: depth, color, infrared •Synthetic streams: rectified images, depth aligned to color and vice versa, etc. • Intrinsic/extrinsic calibration information •Most of the hardware-specific functionality (UVC XU controls)
• Supported OS •Ubuntu 14.04.03 x64 with an updated 4.4 kernel with GCC 4.9 (Other distros may require a patched uvcvideo driver) •OS X 10.8. with Clang toolchain •Windows 8.1 with Visual Studio 2013 Update 5
• https://github.com/IntelRealSense/
#DroidconIT
User Facing TechnologiesCapabilities Features
• Finger Tracking
•Gesture Recognition•22-point Hand and Finger Tracking •Poses and Dynamic Gestures
• Face Tracking
• Face Recognition•Multiple Face Detection and tracking •78-point Face Landmarks •Facial Expressions •Face Recognition •Pulse Estimator
•Speech Recognition
•Voice Synthesis•Command and Control •Dictation •Text to Speech
•Segmentation
• 3D Scanning
•Augmented Reality
•Background Segmentation •3D Object / Face / Room Scanning •2D/3D Object Tracking
#DroidconIT
User Facing Technologies
Capabilities Features• Finger Tracking
•Gesture Recognition•22-point Hand and Finger Tracking •Poses and Dynamic Gestures
Video here: https://youtu.be/r42z259-HHE?t=177
#DroidconIT
User Facing Technologies
Capabilities Features• Facial Analysis Capabilities Features
• Identify the presence of faces in the camera’s range
• 78 landmarks points
• 3D face detection
#DroidconIT
Facial Analysis
Video here: https://youtu.be/r42z259-HHE?t=89
#droidconIT
User Segmentation / Background Subtraction
32
#DroidconIT
Background Removal
Video here: https://youtu.be/HSr_Cw1dlO8
#droidconIT
Opentrack and IRFaceRigDemos – head and F200
34
#droidconIT
Demos – hands and F200Hands Viewer, Longbow, Hoplites
37
#droidconIT
Demos – R200
39
Placeholder Footer Copy / BU Logo or Name Goes Here 41
Video here: https://youtu.be/btwU_pT-tao
#droidconIT
Demos – ZR300
42
Hands Viewer, Longbow, Hoplites
#droidconIT
Additional Usages
43
#DroidconIT
• Artistic Filters• Depth layers • Segmentation (Object/Human)• Refocus
• Measurements• Euclidian Distance• Surface / Area measurements
• Motion Effects• Point of View Shift• Parallax• Dolly Zoom
• Editing Effects• 2D/3D object insertion• Stickers
Enhanced Photography
#DroidconIT
Person Tracking
- Person Detection- Detect all people within the scene- Track the same person throughout the session
- Person segmentation- Rough segmentation- Remove, fully/partially replace person in the frame
- Body parts detection- Detect body parts and add virtual accessories in the
frame- Four points tracking
- Track head, hands and center of mass- Person Recognition
- Identify specific people
![ScanComplete: Large-Scale Scene Completion and ......sors such as Microsoft Kinect, Intel RealSense, and Google Tango, 3D reconstruction of indoor spaces has gained mo-mentum[21,10,23,41,5]](https://img.pdfslide.net/doc/110x75/5f8f408ccf231250377cb107/scancomplete-large-scale-scene-completion-and-sors-such-as-microsoft-kinect.jpg)
