Upload
poo-kuan-hoong
View
630
Download
2
Embed Size (px)
Citation preview
VIRTUAL INTERACTION USING MYO AND
GOOGLE CARDBOARD Name : Chong Wee Nee
ID : 1122702155
Supervisor : Dr Poo Kuan Hoong
Co-Supervisor : Ian Tan
Moderator : Dr Wong Ya Ping
Project Objectives
• To develop a Google Cardboard application and integrate the Myo Armband into the 3D environment.
• To enable user to control the 3D environment using the Myo Armband.
• To provide an immersive experience of learning Japanese characters
Project Scope
• Use only 5 gestures
• Positional tracking not included
• Capture hand movement and transform the data to display air drawn characters in the virtual world.
• The Japanese character will be limited to a few (Hiragana).
• The application will be able to determine if the Japanese character drawn is correct or wrong.
INTRODUCTION TO MYO ARMBAND AND
GOOGLE CARDBOARD
Myo Armband
• Reads muscle activity in forearm
• Electromyography technique
• EMG sensor & Motion sensor (9-axis IMU)
• Detect 5 gestures
• Connect to devices via Bluetooth
Google Cardboard
BACKGROUND STUDY
Virtual Reality Devices Oculus Rift HTC Vive Sony’s
Project Morpheus
Samsung Gear VR
Host Device Windows PC Windows PC Play Station 4 Galaxy Note 4, S6, S6 Edge
Field Of View 110°
Approx. 100° Approx. 100°
96°
Latency Less than 20ms
22ms Less than 18ms
Less than 20ms
Refresh rate 90Hz 90Hz 120Hz Mere 60Hz
Positional Tracking
Yes Yes (Lighthouse)
Yes No
Gesture Recognition Devices Device Method of Gesture Recognition
Kinect Depth Sensor
Play Station Move Camera, Orb on the controller
Wii Wii Remote, Sensor bar
Leap Motion 2 cameras, 3 infrared LED
Requirements
Hardware
• Android phone (Android 4.3 and up & Bluetooth LE 4.0)
• Google Cardboard
• Myo Armband
Software
• Android Studio 1.0 or higher
• Google Cardboard sdk
• MYO Android sdk
• Unity 5.0
• Blender
Flow Diagram
User
Myo Armband
Phone
Identify the Pose
hand pose
head movement
raw EMG data
gestural data
gestural data
gestural data
manipulated classroom
phone movement
updated orientation of VR world
Plugin
Identify the control assigned
to the gesture
Track Rotation & Track Position
FYP 1 Accomplishments
• 3D classroom environment
• Integrate Myo Armband into 3D environment
• Created plugin to connect Android-Myo, Unity
FYP 2 Accomplishments
• Capture hand movement and convert into drawing (gyroscope)
• Included two Japanese characters for users to trace.
• Added sound effect and background music.
• Identify whether the strokes are drawn correctly (compare points).
Future works
• Improve plugin (slow response)
• Add gestures
Screenshot
Screenshot
Screenshot
Screenshot
DEMONSTRATION
Q&A