Introduction to Virtual RealityThe Road to Virtual Immersion

• Jared Sheehan• Twitter: @jayroo5245• http://meetup.com/DCAndroid• http://slideshare.net/jayroo5245

Android Summit – August 24th and 25th

https://www.meetup.com/DCAndroid

Android Summit – August 24th and 25th

https://tinyurl.com/AndroidSummit

Agenda

• Augmented Reality vs Virtual Reality

• VR Use cases

• What is Virtual Reality

• Daydream hardware

• How it works

• Developing

• Questions?

Augmented Reality vs Virtual Reality

• Augmented Reality

• computer-generated visual layers atop an existing reality

• Improve an existing visual scene

• Interact with visual layers and touchpoints

• Virtual Reality

• Completely new computer-generated environment

• Full immersion in a scene

• Interact with the full environment

Augmented Reality vs Virtual Reality

• Augmented reality and Virtual reality are inverse reflections of one in another with what each technology seeks to accomplish and deliver for the user.

• Virtual reality offers a digital recreation of a real life setting, while augmented reality delivers virtual elements as an overlay to the real world.• Reality–virtuality_continuum

VR Use Cases – Google’s Version of Milgram’s Continuum

VR Use Cases – Interactive environment

VR Use Cases – Interactive environment

What is Virtual Reality – Hardware – Display

What is Virtual Reality – Hardware – Display – Stereoscopic 3D View

What’s next? Anything else?

VR Use Cases – 360 Video

What is Virtual Reality – Hardware

VR Use Cases – 360 video vs VR explained

VR Use Cases – 360 video

VR Use Cases – 360 video

VR Use Cases – 360 video

VR Use Cases – 360 video

What is Virtual Reality – Hardware

What is Virtual Reality – Hardware – Optics

What is Virtual Reality – Hardware – Optics

• Optical lenses make it possible to focus on a screen that is very close to your face.

• Try holding your phone really close to your face. Can you see anything?

• It focuses the light from a near eye display onto your eye in a way that makes it clear.

• It greatly magnifies the image from your phone.

• What are the side effects of magnifying images that close to your eye?

• Reduces the total resolution of the user’s display.

• Depending on the display the experience may seem pixelated.

What is Virtual Reality – Hardware – Display

• Right behind the optical lenses sit one or more VR Displays:• Usually: High resolution, OlED displays that support low persistence features

• Sometimes the Displays are built into the Headset:• HTC Vive and Oculus Rift

• Displays can be external devices:• Daydream View, Cardboard, Samsung Gear

• Low Persistence and motion blur:• Instead of showing a full image, VR will show you parts of an image.• Reduces motion blur and keeps the image clear as you navigate about.• Basically, hides the pixels as they change so the user doesn’t see/perceive blur.

What is Virtual Reality – Hardware – Tracking

• Vital to VR technology:• The computer must know where the user is and is looking in space.

• Rotational Tracking• All VR systems rely on an Inertial Measurement Unit (IMU)• Enables high speed rotational tracking• IMU’s cannot tell where an object is in space, only how it is rotated

• Positional Tracking techniques:• Cameras• Lasers• Magnetic Fields• This is an active field of discussion and research. IE no accepted best practice yet

What is Virtual Reality – Degrees of Freedom (DOF)

• 3 – DOF Tracking:• Head Rotation Tracking are

accurate• Pitch / Yaw / Roll

• X, Y and Z axis• No depth in the room

• 6 – DOF Tracking:• Head Rotation Tracking are

accurate• X, Y and Z axis• Pitch / Yaw / Roll

• And Positions or body movement around a room• X, Y and Z axis• Surge / Heave / Moving Left

or right

What is Virtual Reality – 3 DOF – Inertial Measurement Unit (IMU)

• Inertial Measurement Units• Really good at quickly and accurately measuring movement• IMU Hardware -

• Accelerometer• Gyroscope• Magnetometer

• Constantly infer where the headset is pointed • All major 3 - DOF VR systems use some form of IMU

For more information on usingAndroid Device sensors:• Contextual Awareness - Android

Summit 2016• Slides• Github Project - makingsense

The Daydream platform makes use of internal Android Device sensors to determine movement

What is Virtual Reality – 6 DOF – Major Players

• Oculus Rift:• Tracking System: Constellation• Infrared LED system in the Headset• Camera captures the LED light patterns• Transmits to a computer and transposes the

• images to the headset• This Model along with an internal IMU

• Calculates the users head position• HTC Vive:

• Tracking System: Lighthouse• Infrared Laser system in base stations

• around the room• IMU Hardware:

• Accelerometer• Gyroscope• Magnetometer

• Constantly infer where the headset is pointed

Daydream!

Daydream – Google I/O 2017

• 8 Daydream ready phones• Standalone Daydream Headsets

• no cables, phone or PC• 150+ Daydream apps

• 3x improvement from Early 2017• Daydream 2.0 Euphrates

• Cast support – Share with friends on your TV

• Coming later 2017

Daydream – What’s New in Daydream – 2017 - WorldSense

• “WorldSense"• “Inside-out” Tracking• Does not need external sensors • Version of Tango for VR/AR

• 6 Degrees of Freedom (6DOF)• Rotational and Positional Tracking• Dramatically improves the

experience• VPS – navigate indoors using visual

reference points as a sort of indoor GPS• Previous Daydream devices only had

3DOF• Only Rotational Tracking

Daydream – What’s New in Daydream – 2017 - WorldSense

Daydream – Google I/O 2017

• What are the new devices going to look like?

Cardboard versus Daydream

• Cardboard – Inexpensive VR that works

• No device hardware requirements

• phone + headset-only

• Input:• Uses the device’s magnetometer to

register the movement of the included magnet as a click

• 20% Google Project - $15

• Daydream – Virtual Reality for Grownups

• Strict device hardware requirements

• phone (optional) + headset + controller

• Input:• Controller IMU sensor data• Click events

• Relatively inexpensive - $79• Standalone Headset - $TBD

Daydream Hardware

Daydream Hardware – Device Hardware Requirements

• Device Hardware Requirements:• Android 7.0 Compatibility Definitions Document

• Hardware Requirements Highlights:• Bluetooth 4.2 LE• Display between 4.7 and 6 inches• Resolution at least 1080p @ 60Hz display with

• 3ms or less latency and 5ms or less persistence. • (Quad HD or higher recommended.)

• OpenGL ES 3.2 and Vulkan• Able to decode 2 instances of 60fps video simultaneously• Consistent 60fps rendering• Temperature sensors capable of reading device surface temperature

Daydream Hardware – Headset Requirements

• Headset Requirements:• Lenses• Location to strap in a device that is 4.7 to 6.0 inches• Strap of some sort to hold the device place• Near Field Communication Chip

• For automatically launching the VR experience• Daydream View:

• Has no electronic components, so feel free to wash it

Daydream Hardware – Controller Requirements

• Controller Requirements:• Sensor Hardware:

• 3-DOF - Motion-sensing capabilities• Must include its own IMU

• Accelerometer• Gyroscope• Orientation

• Touchpad (Clickable)• Will send click events to device• Single click and Long press

• Buttons:• Volume Buttons• App Button – Open to developer• Home Button

• System functions• long-press to re-center the headset

• 3D Pointer:

Touchpad

App Button

Home Button

Volume Buttons

Daydream Hardware – Controller Emulator

• Controller Emulator App:• Most Mobile Phones have the

correct hardware to act as a Daydream Controller

• Download the Daydream Controller App

• Pair your headset phone to your controller phone via Bluetooth

• Configure the Controller Emulator device through Developer Options

• Documentation• Off you go!

Touchpad

App Button

Home Button

Volume Buttons

Double tap to emulate a click

Daydream Hardware – Sticker overlay - Controller Emulator

Daydream: How it works

Developing – VR development types and docs

• Daydream Android VR SDK and VR NDK:• Google - Android VR support docs

• Daydream iOS VR SDK:• Google - iOS VR support docs

• Unity:• Unity - VR Support docs• Google - Unity support docs

• Unreal Engine:• Unreal - VR Support docs• Google - Unreal support docs

What is Virtual Reality – Google VR for Android

Developing – GoogleVR Repo

• https://github.com/googlevr

Developing – Platforms!

• Scene Creation / Manipulation

• Unity

• UnReal

• Asset / Model creation

• Blender.org• Open source

• Autodesk Maya• 3D Modeling and rendering tool

Virtual Reality – Unity Engine

Developing – Unity Platform

Hierarchy

Assets

Scene

Inspector

Daydream – Application Discovery – Daydream Home in VR Mode

Discovery Windows

Google Play

Developing – Publishing Daydream apps

• Daydream App Quality Requirements:• “To ensure a great user experience, apps for Daydream must follow specific

requirements for performance and usability”.• https://developers.google.com/vr/distribute/daydream/app-quality

1. Design Requirements:• UX-D1 – Users can focus on objects and read necessary text.• UX-C1 – Controller must be used as a laser pointer when clicking on UI targets.

2. Functionality requirements:• FN-M1 – App manifest does not request the NFC permission.• FN-S2 – App uses Daydream API calls to transition between activities.

3. Performance and stability requirements:• PS-P3 – App does not display a thermal warning during 30 minutes of usage.

4. Publishing requirements:• PB-P2 – App has a VR Icon.

Developing – Challenges

• Simulator Sickness (Also called Cybersickness):• Can make people uncomfortable• Sick, headaches or nauseous• Anything that causes a mismatch sense of motion can do this

• "Screen door" effect: • When users look at the display in their VR headset, they will often see a regular

grid of lines. These are the spaces between the pixels, which you can only see because the screen is magnified and several inches away from your face.

• Requires a Headset, Display (and perhaps a controller) of some sort:• Barrier to adoption

Developing – Daydream’s Future

• Google Assistant integration:• Very natural Fit• Use Voice integration when users cannot physically touch their device

• Multi Controller:• Why not?• A users offhand is just sitting there doing nothing

• iPhone support:• Would require an SDK of some sort• Not sure if possible in Daydream’s current form• iOS provides much less Bluetooth control/customization• Hack it together:

• https://www.techworm.net/2016/12/somebody-just-hacked-google-daydream-vr-make-work-iphone.html

Questions?

Virtual Reality – References

• https://developers.google.com/vr/daydream/overview• http://www.theverge.com/2016/11/10/13578012/google-daydream-view-vr-

review-mobile-headset-pixel• http://features.shopomo.com/electronics/google-daydream-new-mobile-vr-

standard-compares-google-cardboard/• http://www.popsci.com/google-daydream-vr-review• http://www.phonearena.com/news/Google-Daydream-VR-vs.-old-mobile-VR-

Whats-the-difference_id86140• http://www.androidcentral.com/google-reveals-hardware-requirements-daydream-

vr• Unite 2016 - Making Daydream Real: Building for Google's New VR Platform • https://www.udacity.com/