Embed Size (px)
Citation preview
Getting Started in VR with JS
Getting Started in VR with JSThe Dream of the 90s is Alive!
#empirejs2015
MARRY ME, ALICIA!
GO SARNIA BEES!
20 years later …
Why would this time be any different?
VR SHE WROTE
JavaScript FTWWebVR allows us to work in JS “native” browser environment.
What’s a headset?
WOAH! your app renders
1. delivers position data to
2. surface to displaystereoscopic image
.js
A device that gives positional data & a surface to draw on?Sounds awful like a smart phone!
I’ll be giving these outCome see me in the Q&A lounge after this and throughout the conference.
Forgiveness please …
I AM A (DEMO) GOD HERE
Download ALL THE THINGS!Drivers and SDK from developer.oculus.com Firefox WebVR Browser from mozvr.com/downloads Chromium WebVR Browser from bit.ly/1DPjgDQ
What is WebVR?
What it's not!
Not a Virtual Reality DOM
Not a WebGL replacementWebGL is a framework to build your own 3D graphics rendering pipeline.
Math is HARDComputing per pixel transform and coloring takes a lot of work.
Use three.jsProvides a higher level abstraction that is easier to work with.
no_vr.html<!DOCTYPE html> <html lang="en"> <head> <title>No VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/no_vr.js"></script> </body> </html>
no_vr.html</style> </head> <body> <script src="js/three.js"></script> <script src="js/no_vr.js"></script> </body> </html>
no_vr.js - Create a Renderervar renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 );
var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light);
var bitGeometry = new THREE.DodecahedronGeometry(0.5); var bitMaterial = new THREE.MeshLambertMaterial({
no_vr.js - Let Your Scene be Seenvar renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 );
var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light);
var bitGeometry = new THREE.DodecahedronGeometry(0.5); var bitMaterial = new THREE.MeshLambertMaterial({
no_vr.js - Let there be light!var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 );
var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light);
var bitGeometry = new THREE.DodecahedronGeometry(0.5); var bitMaterial = new THREE.MeshLambertMaterial({
no_vr.js - Have Something to Seevar light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light);
var bitGeometry = new THREE.DodecahedronGeometry(0.5); var bitMaterial = new THREE.MeshLambertMaterial({ color: 0x00ffff, shading: THREE.FlatShading }); var bit = new THREE.Mesh(bitGeometry, bitMaterial); bit.position.z = -2; scene.add(bit);
var planeGeometry = new THREE.PlaneBufferGeometry(1000, 1000, 1000); var planeMaterial = new THREE.MeshPhongMaterial({ color: 0x0000ff, shading: THREE.DoubleSide
no_vr.js - Get Animatedscene.add(plane);
function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
no_vr.js - Get Animatedscene.add(plane);
function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
no_vr.js - Get Animatedscene.add(plane);
function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
no_vr.js - Get Animatedscene.add(plane);
function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
no_vr.js - Get Animatedscene.add(plane);
function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
Apply some WebVR and …
What is WebVR?
WebVR is a device interfaceDetect and poll Head Mounted Displays (HMDs) and other position reporting devices.
navigator.getVRDevices()Through callback or promise returns a list of VR devices of two types …
PositionSensorVRDeviceInformation about position and orientation
HMDVRDeviceInforms you have a surface to render on and information about the eyes.
Math is HARDTranslating all this is difficult, but luckily three.js comes to the rescue.
Nearly all WebVR demos use VRControls and VREffectFound in three.js examples alongside other great utilities.
github.com/mrdoob/three.js/tree/master/examples/js
basic_vr.html<!DOCTYPE html> <html lang="en"> <head> <title>Basic VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head>
<body>
<script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
basic_vr.html</style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
basic_vr.js - Introduce VRControlsvar renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 );
var controls = new THREE.VRControls(camera); var effect = new THREE.VREffect(renderer); effect.setSize(window.innerWidth, window.innerHeight);
var light = new THREE.PointLight(0xffffff, 1.0, 15); light.position.set(0,0,0); scene.add(light);
VRControlsHandles PositionSensorDevice and manipulates the camera orientation.
VRControl.js - Initializationfunction gotVRDevices( devices ) devices = filterInvalidDevices( devices );
for ( var i = 0; i < devices.length; i ++ ) { if ( devices[ i ] instanceof PositionSensorVRDevice ) { vrInputs.push( devices[ i ] ); } }
if ( onError ) onError( 'HMD not available' ); }
if ( navigator.getVRDevices ) { navigator.getVRDevices().then( gotVRDevices ); }
basic_vr.js - Introduce VREffectvar renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 );
var controls = new THREE.VRControls(camera); var effect = new THREE.VREffect(renderer); effect.setSize(window.innerWidth, window.innerHeight);
var light = new THREE.PointLight(0xffffff, 1.0, 15); light.position.set(0,0,0); scene.add(light);
VREffectHandles HMDVRDevice and uses it to render a stereoscopic view
VREffect.js - Initialization
function gotVRDevices( devices ) { for ( var i = 0; i < devices.length; i ++ ) { if ( devices[ i ] instanceof HMDVRDevice ) { ///… }
if ( vrHMD === undefined ) { if ( onError ) onError( 'HMD not available' ); } }
VREffect.js - Remember the "..."
var eyeParamsL = vrHMD.getEyeParameters( 'left' ); var eyeParamsR = vrHMD.getEyeParameters( 'right' );
eyeTranslationL = eyeParamsL.eyeTranslation; eyeTranslationR = eyeParamsR.eyeTranslation; eyeFOVL = eyeParamsL.recommendedFieldOfView; eyeFOVR = eyeParamsR.recommendedFieldOfView;
basic_vr.js - Go Full Screenvar vrMode = false;
function enterVR() { effect.setFullScreen(true); vrMode = true; }
function exitVR() { effect.setFullScreen(false); vrMode = false; };
function toggleVR() { if (!vrMode) { enterVR(); } else {
VREffect - Full Screen VR display
this.setFullScreen = function ( boolean ) { if ( vrHMD === undefined ) return; if ( isFullscreen === boolean ) return;
if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else if ( canvas.webkitRequestFullscreen ) { canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); } };
basic_vr.js - Let Reality Take Controlfunction animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
controls.update(); effect.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
VRControls.js - Get State from VR Devicethis.update = function () { for ( var i = 0; i < vrInputs.length; i ++ ) { var vrInput = vrInputs[ i ]; var state = vrInput.getState();
if ( state.orientation !== null ) { object.quaternion.copy( state.orientation ); }
if ( state.position !== null ) { object.position.copy( state.position ) .multiplyScalar( scope.scale ); } } };
basic_vr.js - The Full VR Effectfunction animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01;
controls.update(); effect.render(scene, camera);
requestAnimationFrame( animate ); }
animate();
Two Cameras for the price of ONE!It creates a camera for each eye, shifts them from the main camera location based on HMDVRDevice eye information, then renders the view for each!
Confession
This doesn’t work on mobile browsers!WebVR is still in development.
webvr-polyfillSupplies Mobile device gyroscope as PositionSensorVRDevice, screen as HMDDevice.
github.com/borismus/webvr-polyfill
webvr-boilerplateA very basic skeleton much like you’ve seen here today.
github.com/borismus/webvr-boilerplate
basic_vr_mobile.html - Drop It In<!DOCTYPE html> <html lang="en"> <head> <title>Basic VR Mobile</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head>
<body>
<script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
basic_vr_mobile.html - Drop It In
</style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
I know what you’re thinking …
Simulator Sickness is Real …Need to hit 60-90 fps. Motion data already laggy. Additional disadvantage of being in browser. More graphics you push the harder this gets.
Consider 360 Video
Spherically map videos around each camera/eyeAgain, math is HARD! See three.js vr_video or eleVR-Web-Player for example.
Interaction
All the HTML5 ThingsKeyboard, mouse, gamepad …
Leap Motion is Strongly Embracing WebVRWorks directly with three.js
leapmotion.com/product/vr
leap_motion.html<!DOCTYPE html> <html lang="en"> <head> <title>Leap Motion VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="//js.leapmotion.com/leap-0.6.3.min.js"></script> <script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></script> <script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js"></script> <script src="js/leap_motion.js"></script> </body> </html>
leap_motion.html</style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="//js.leapmotion.com/leap-0.6.3.min.js"></script> <script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></script> <script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js"></script> <script src="js/leap_motion.js"></script> </body> </html>
leap_motion.jsLeap.loop(); Leap.loopController.use('transform', { vr: true, effectiveParent: camera });
Leap.loopController.use('riggedHand', { parent: scene, renderer: renderer, materialOptions: { emissive: new THREE.Color(0x00aaaa) } });
animate();
Confession 2.0
This doesn’t work on mobile*
* without hacks
One interaction to rule them all
GazeWhere you are looking can trigger changes, from environment to storyline.
raycast.js - Gazevar raycaster = new THREE.Raycaster(); var center = new THREE.Vector2(); function gaze() { if(!audioPlaying) { raycaster.setFromCamera(center, camera); var intersects = raycaster.intersectObjects([bit]);
if (intersects.length > 0) { audioPlaying = true; if (Math.random() < 0.5) { switchBitTo(1); yes.play(); } else { switchBitTo(0); no.play(); }
Responsive VRFor “simple” devices, it’s a roller coaster ride. More complex, more interactions.
Math is HARDCollision detection, physics, etc.
Unity publishes to multiple platformsPlugins for VR available. Support directly baked in 5.1
#pragma strict
var center : GameObject; var axis : Vector3;
function Start () { }
function Update () { transform.RotateAround( center.transform.position, axis,30 * Time.deltaTime ); }
Orbit.js
Cores.jsfunction beingLookedAt() { var hit : RaycastHit; var ray = new Ray(Camera.main.transform.position, Camera.main.transform.forward); return GetComponent .<Collider>() .Raycast(ray, hit, Mathf.Infinity); }
Unity now exports to WebGLAn idea …
Build a WebVR Unity pluginBridges when publishing to WebGL. Design once, distribute everywhere, run anywhere. NOW YOU’RE PLAYING WITH POWER
Gotchas
Again, Simulator Sickness is Real …It’ll get better, but start with easy motions
3D Modelling is HardSpent most of my time on it. Develop interactions first! Then find a friend.
The ground is constantly changingBeta Software running on Beta Hardware. Things break constantly.
So that’s it …
Except for one thing …?
Why would this time be any different?
Answer: Because of you …
![[WORKSHOP] Getting Started with 360-Video, Augmented Reality, and VR](https://img.pdfslide.net/doc/110x75/5a647b177f8b9a70568b46db/workshop-getting-started-with-360-video-augmented-reality-and-vr.jpg)
