Upload
roberto-reto
View
1.810
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Source / Fuente: http://channel9.msdn.com/Series/KinectQuickstart
Citation preview
KINECT FOR WINDOWS
QUICKSTART SERIES
KINECT FOR WINDOWS
INSTALLING &USING KINECT
KINECT FOR WINDOWS
INTRODUCTION
• Designed to get you started• Assume you have some programming
experience• Managed API - Concepts work in both
VB/C#
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
KINECT SENSORSIR Emitter Color Sensor
IR Depth Sensor
Tilt Motor
Microphone Array
KINECT FOR WINDOWS
GET STARTED
• http://kinectforwindows.org • Order Kinect Hardware• Download Kinect SDK
KINECT FOR WINDOWS
INSTALLING AND USING KINECT
DEMO
KINECT FOR WINDOWS
SETTING UP YOUR DEVELOPMENTENVIRONMENT
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development
Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
KINECT API BASICS
• Manage Kinect state • Connected• Enable Color, Depth, Skeleton• Start Kinect
• Get Data• Events - AllFramesReady• Polling – OpenNextFrame
KINECT FOR WINDOWS
RESOURCES
• Install Kinect Explorer• KinectWpfViewers
• Coding4Fun Toolkit• Skeletal scaling
KINECT FOR WINDOWS
SETTING UP YOUR DEV ENVIRONMENT
DEMO
KINECT FOR WINDOWS
CAMERA FUNDAMENTALS
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
CAMERA DATA
• Events return ImageFrame• PixelDataLength• FrameNumber• Timestamp• Dimensions: Height, Width
• Use AllFramesReady event to synchronize
KINECT FOR WINDOWS
RESOLUTIONS
• Color• 12 FPS: 1280X960 RGB• 15 FPS: Raw YUV 640x480• 30 FPS: 640x480
• Depth • 30 FPS: 80x60, 320x240, 640x480
KINECT FOR WINDOWS
CAMERA FUNDAMENTALS
DEMO
KINECT FOR WINDOWS
WORKING WITHDEPTH DATA
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
DEPTH DATA• Returns the distance and player for every
pixel• Ex: 320x240 = 76,800 pixels
• Distance• Distance in mm from Kinect ex: 2,000mm (6.56
feet)
• Player• 1-6 players
KINECT FOR WINDOWS
MODE
Mode Depth & Player Center Hip Joint Other 19 Joints
Default Yes Yes Yes
Near Yes Yes No, for v1.0
KINECT FOR WINDOWS
DISTANCES1.3’ 2.6’ 9.8’ 13.1’ 26.2’
.4 .8 3 4 8
DefaultMode
Near Mode
Feet
Meters
KINECT FOR WINDOWS
FORMULAS
• Distance Formulaint depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth;
• Player Formulaint player = depthPoint & DepthImageFrame.PlayerIndexBitmask;
KINECT FOR WINDOWS
WORKING WITH DEPTH DATA
DEMO
KINECT FOR WINDOWS
SKELETAL TRACKINGFUNDAMENTALS
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
SKELETON DATA
KINECT FOR WINDOWS
SKELETAL JOINTS• Each player with set of <x, y, z> joints in meters• Each joint has associated state
• Tracked, Not tracked, or Inferred
• Inferred - Occluded, clipped, or low confidence joints• Use TransformSmoothParameters to smooth joint data to
reduce jitter
KINECT FOR WINDOWS
SKELETAL TRACKING• Maximum two players tracked at once
• Six player proposals per Kinect
• Each skeleton has a unique identifier - TrackingID • SkeletonStream
• Set AppChoosesSkeletons to false• Use ChooseSkeletons method to choose which skeletons to
track
KINECT FOR WINDOWS
SKELETAL TRACKING
DEMO
KINECT FOR WINDOWS
AUDIOFUNDAMENTALS
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
KINECT FOR WINDOWS
SOUND POSITION• Sound Source Angle – the angle and confidence level of
where audio is coming from• Beam Angle – The angle used to record audio that you can
set as a “directional microphone”
-50 0 +50
KINECT FOR WINDOWS
RECORDING
• KinectAudioSource audio buffer• Use Echo Cancellation to not record PC
sound
KINECT FOR WINDOWS
SPEECH RECOGNITION
• Grammar – What we are listening for• Code – GrammarBuilder, Choices
• Note: Ensure AutomaticGainControl = false
KINECT FOR WINDOWS
GRAMMAR<!-- Confirmation_Yes._value: string ["Yes"] --><rule id="Confirmation_Yes" scope="public"> <example> yes </example> <example> yes please </example> <one-of> <item> yes </item> <item> yeah </item> <item> yep </item> <item> ok </item> </one-of> <item repeat="0-1"> please </item> <tag> out._value = "Yes";</tag>
var grammar = new Choices();grammar.Add(“yes please");grammar.Add(“yes");grammar.Add(“yeah");grammar.Add(“ok");
KINECT FOR WINDOWS
MULTI-MODAL FEEDBACK• Engagement model–Mic indicator for speech-enabled menus– Keyword to engage
• Feedback and confirmation
KINECT FOR WINDOWS
AUDIO
DEMO