9/9/2015 EEC492/693/793 - iPhone Application Development 1 EEC-492/592 Kinect Application Development Lecture 1 Wenbing Zhao [email protected]

04/19/23EEC492/693/793 - iPhone Application

Development 1

EEC-492/592EEC-492/592Kinect Application Kinect Application

DevelopmentDevelopmentLecture 1Lecture 1

Wenbing ZhaoWenbing Zhao

[email protected]@ieee.org


Development 2

IntroductionIntroduction Instructors

Wenbing Zhao, [email protected] Office Hours

Mondays 6-8pm Wednesdays noon-2pm

Course Web site http://academic.csuohio.edu/zhao_w/teaching/EEC492-S14/e

ec492.htm


Development 3

Quick Survey

Have you programmed in C#? Have you programmed in C++? Have you programmed in Java? Have you ever played Kinect games? Have you installed Visual Studio and done

Windows application development?


Development 4

Requirements

Prerequisite: Moderate programming background Basic knowledge of data structure and algorithms Basic knowledge of operating systems


Development 5

Textbooks and References

Main textbook Kinect for Windows SDK Programming Guide, by

Abhijit Jana, PACKT publishing, 2012. References:

Kinect Hacks, by Jared St. Jean, O’Reilly, 2013 Hacking the Kinect, by Jeff Kramer et al., Apress,

2012.Reference materials C# in a Nutshell, By Peter Drayton, Ben Albahari,

Ted Neward, O'Reilly, 1st Edition March 2002


Development 6

Required Hardware & Software Hardware

Windows PC or laptop Kinect sensor

Software Visual Studio 2010 Kinect SDK Other related libraries


Development 7

Kinect’s Capability Capturing real-time video using the color sensor Tracking a human body and then responding to its

movements and gestures as a natural user interface Generating depth map of the objects tracked Recognizing a human voice and developing hands-

free applications that can be controlled by voice


Development 8

Real-World Applications of Kinect Healthcare: exercise measurement, monitoring patients, their

body movements, and so on Robotics: as a navigation system for robots either by tracking

human gestures, voice commands, or by human body movements

Education: help them to learn subjects either by their gesture and voice commands

Security system: track human body movement or face and send the notifications

Virtual Reality Trainer: measuring the movements of human body joints,

providing live feedback to users if the joints are moving in an appropriate manner by comparing the movements with previously stored data

Military: build intelligent drones to spy on enemy lines


Development 9

Healthcare with Kinect Reflexion Health

Advancing physical therapy http://blogs.msdn.com/b/kinectforwindows/archive/2013/05/

09/reflexion-health-advancing-physical-therapy-with-kinect-for-windows.aspx


Development 10

Healthcare with Kinect ReMotion360: uses Kinect to measure and provide

on screen feedback for range of motion exercises either in a clinic or at home http://www.infostrat.com/solutions/Remotion360 https://www.youtube.com/watch?feature=

player_embedded&v=-jhpYslVygs


Development 11

Improving Safety at Workplace Siemens uses Kinect for Windows to improve

nuclear plant employee safety virtual 3-D human avatars integrated to model motion-

controlled actions input with a Kinect http://blogs.msdn.com/b/kinectforwindows/archive/2013/04/

19/siemens-uses-kinect-for-windows-to-improve-nuclear-plant-employee-safety.aspx


Development 12

Enabling Retailers to Drive Business Benefits:

Capture a customer's attention Educate customers about products Move a customer through the decision-making cycle to

close a sale Ski Shop scenario:

https://www.youtube.com/watch?feature=player_embedded&v=ORWYr1gx5eU

Toy store scenario: https://www.youtube.com/watch?

feature=player_embedded&v=wZfcDOsBs2E Retail clothing scenario

https://www.youtube.com/watch?feature=player_embedded&v=Mr71jrkzWq8


Development 13

Styku: Apparel Shopping Help find clothes that fit Virtually try on clothes before they buy To be offered by major retailers, such as Brooks

Brothers, IM-Label, and others


Development 14

Shoppers who approach the Kinect-based showroom are guided through an array of natural movements to activate hotspots on the Pathfinder model, allowing them to inspect the car inside and out https://www.youtube.com/watch?

feature=player_embedded&v=rQVycUmeqB4

Nissan Pathfinder Virtual Showroom


Development 15

Education Tool nsquared sky spelling

makes it fun for children to learn to spell in an interactive and collaborative environment

capture letters that complete the spelling of various words


Development 16

Hospitality Tool Nsquared Hotel Kiosk

augments the concierge function in a hotel by providing guidance to hotel guests through an intuitive, interactive experience

use a “grip” gesture to drag content across the screen and a “push” gesture to select content


Development 17

Ubi Interactive: Turn any Surface into a Touch Screen Instruct classes in an interactive lecture hall Access product information on a store’s window front Recipes can be projected onto kitchen countertops without

having to worry about getting anything dirty Use wall to play interactive games like Angry Birds Video: https://www.youtube.com/watch?

feature=player_embedded&v=Iu2XH5p_hMM

3D Scanning Kinect fusion: perform 3D scanning with a single

Kinect sensor! Nice match with 3D printers


Development 18


Development 19

New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch A combined 2-D and 3-D camera from Intel will

be built into laptops and tablets from a range of manufacturers, Intel announced at the 2014 International Consumer Electronics Show (CES)


Development 20

New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch Toshiba also demonstrated a dual-camera module for

smartphones, tablets, and laptops at 2014 CES Gesture recognition, refocus images, etc.

New Development on Depth Cameras SoftKinetic demonstrate a depth camera for 3D scanning

Need to move the camera around the subject

Apple will also come to the depth camera field Acquired Primesense, the company invented the technology

behind Kinect Filed a patent in 3D camera for iOS devices


Development 21


Development 22

Main Topics Understand the Kinect sensor and how they work Learn how to capture data streams from the Kinect

sensor Learn the uses of color, depth and IR data and how to

leverage them in your applications Use skeleton-tracking for interactive applications Learn basic gesture recognition methods and develop

gesture-enabled applications Build speech-enabled applications with Speech API, and

text to speech as well Learn 3D game development with Unity3D for Kinect

(virtual reality!) Applied computer vision with OpenCV and Kinect


Development 23

Assignments and Grading Assignments

In-class exercises Term project

Reading several research papers on the application of Kinect Develop a working prototype program using Kinect Project presentations and demonstration

Grading policy Class participation: 60% (any missed session will lead to

credit reduction) Project: 40% No exams


Development 24

Demos, Software Installation and

Q/A

Documents

9/9/2015 EEC492/693/793 - iPhone Application Development 1 EEC-492/592 Kinect Application Development Lecture 1 Wenbing Zhao [email protected]