Embed Size (px)
DESCRIPTION
Slides from fall 2014 HTML5 Dev Conf talk on Browser-Based VR
Citation preview
HACKING REALITY:
HTML5 AND JAVASCRIPT
FOR CROSS-PLATFORM VR
TONY PARISIOCTOBER, 2014
ABOUT ME
http://www.tonyparisi.com 04/09/2023
[email protected]: auradeluxehttp://twitter.com/auradeluxehttp://www.tonyparisi.com/http://www.learningwebgl.com/
GET THE BOOKS!
WebGL: Up and Runninghttp://www.amazon.com/dp/144932357XProgramming 3D Applications with HTML and WebGLhttp://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966
GET VIZIhttps://github.com/tparisi/Vizi
MEETUPShttp://www.meetup.com/WebGL-Developers-Meetup/
http://www.meetup.com/Web-VR/
BOOK CODEhttps://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAMhttp://www.glamjs.org/https://github.com/tparisi/glam/
WORKhttp://www.vizi.gl/
CREDSCo-creator, VRML and X3D
04/09/2023
http://www.tonyparisi.com
THE PROMISED LAND!CONSUMER VR
04/09/2023
http://www.tonyparisi.com
GIANT DOWNLOADS
SILO EXPERIENCES
CUMBERSOME NATIVE APPS AND INSTALLS
PROPRIETARY PLATFORMS AND DEVELOPMENT STACKS
VR DOTH BE
HARD!
VR TODAY
04/09/2023
http://www.tonyparisi.com
WHY I LOVE THE WEB
INSTANT PUBLISHING
INSTANT ACCESS TO INFORMATION
NO TOLLS
NOBODY CONTROLS IT
CULTURE OF COLLABORATION
VIEW SOURCE
…THE WEB WILL NEVER CLOSE UP SHOP.
image: Mark Surmanhttp://commonspace.wordpress.com/2014/03/12/happybirthday/
04/09/2023
http://www.tonyparisi.com
THE THREE D’S OF THE WEB
DEVELOPMENT CROSS-PLATFORM VENDOR-NEUTRAL OPEN SOURCE
DEPLOYMENT CLOUD PUSH-BUTTON UPDATE AND PUBLISH
DISTRIBUTION AND DISCOVERY EMBED IN OTHER PAGES SHARE WITH A HYPERLINK NO APP TO DOWNLOAD
04/09/2023
http://www.tonyparisi.com
VR AND THE WEB:TWO GREAT TASTES… ?
04/09/2023
http://www.tonyparisi.com
WEB VRFAST, CHEAP, AND TOTALLY
DEMOCRATIZED.
BROWSER-BASED VIRTUAL REALITY WEBGL CSS3 VR SUPPORT NOW IN
BROWSER DEV BUILDS!!!
NO BIG APP DOWNLOADS AND INSTALLS!!! JUST ADD SMART
PHONE “SMARTVR” USING
GOOGLE CARDBOARD
$25 CHEAP!
04/09/2023
http://www.tonyparisi.com
INFORMATION DIVING SHOWCASEhttp://mozvr.github.io/infodive/
POWERED BY FIREFOX BUILT WITH VIZIhttps://github.com/tparisi/Vizi
AN EXAMPLE
04/09/2023
http://www.tonyparisi.com
THE WEBVR API
1. QUERY FOR VR DEVICE(S) TO RENDER
// polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */
var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrHMD; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof HMDVRDevice ) { vrHMD = devices[i]; self._vrHMD = vrHMD; self.leftEyeTranslation = vrHMD.getEyeTranslation( "left" ); self.rightEyeTranslation = vrHMD.getEyeTranslation( "right" ); self.leftEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "left" ); self.rightEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "right" ); break; // We keep the first we encounter } } }
get left/right eye(camera) positions
get per-eye camera field of view; use WebGL to render scene from two cameras
04/09/2023
http://www.tonyparisi.com
THE WEBVR API
2. GO FULLSCREEN (VR MODE) var self = this; var renderer = this._renderer; var vrHMD = this._vrHMD; var canvas = renderer.domElement; var fullScreenChange = canvas.mozRequestFullScreen? 'mozfullscreenchange' : 'webkitfullscreenchange';
document.addEventListener( fullScreenChange, onFullScreenChanged, false ); function onFullScreenChanged() { if ( !document.mozFullScreenElement && !document.webkitFullScreenElement ) { self.setFullScreen( false ); } } if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else {
canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); }
handle exiting fullscreen mode
request fullscreen modefor this VR device
fullscreen mode requires a DOM element
04/09/2023
http://www.tonyparisi.com
THE WEBVR API3. HEAD TRACKING
// polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */
var self = this; getVRDevices().then( gotVRDevices );
function gotVRDevices( devices ) { var vrInput; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof PositionSensorVRDevice ) { vrInput = devices[i] self._vrInput = vrInput; break; // We keep the first we encounter } } } …
// called once per tick from requestAnimationFrame() var update = function() { var vrState = this.getVRState(); if ( !vrState ) { return; }
// vrState.hmd.rotation contains four floats, quaternion x,y,z,w setCameraRotation(vrState.hmd.rotation); };
get head-tracking VR device
update camera to match HMD device orientation
query HMD device state
04/09/2023
http://www.tonyparisi.com
WEBVR AND CARDBOARD
GOOGLE CARDBOARD SHOWCASE Mobile Chrome http://g.co/chromevr Desktop Chrome http://vr.chromeexperiments.com/
EVEN EASIER RENDER WEBGL SIDE-BY-SIDE STEREO (NO NEED TO
QUERY DEVICES) USE BROWSER DEVICE ORIENTATION API FOR HEAD
TRACKING
04/09/2023
http://www.tonyparisi.com
WEBVR AND THREE.JS
THE MOST POPULAR WEBGL LIBRARY
http://threejs.org/
LATEST STABLE VERSION (r68) INCLUDES STEREO RENDERING AND HEAD TRACKING RENDERING
examples/js/effects/StereoEffect.js (Cardboard)
examples/js/effects/VREffect.js (Rift)
HEAD TRACKINGexamples/js/controls/DeviceOrientationControls.js (Cardboard)
examples/js/controls/VRControls.js (Rift)
04/09/2023
http://www.tonyparisi.com
VIZI: A FRAMEWORK FOR WEBVR APPLICATIONS
GOAL: MAKE IT EASY TO QUICKLY BUILD INTERESTING 3D APPLICATIONS
ARCHITECTURE INSPIRED INSPIRED BY MODERN GAME ENGINE DESIGN COMPONENT-BASED DESIGN – EASILY PLUG NEW FEATURES INTO OBJECTS
APPLICATION OBJECT – HANDLES EVENT LOOP, THREE.JS CREATION, PAGE RESIZE, ROUTING EVENTS TO OBJECTS
INTERACTIONS – MAKE IT EASY TO PUT MOUSE AND TOUCH INTERACTION ON OBJECTS
BEHAVIORS – PREBUILT BEHAVIORS AUTOMATICALLY ROTATE, MOVE ETC.
PRE-BUILT OBJECTS – FOR COMMONLY USED DESIGN PATTERNS
SIMILAR IN DESIGN TO UNITY3D
USES THREE.JS FOR ALL GRAPHICS
ENHANCES THREE.JS VR RENDERING AND CONTROLLERS
04/09/2023
http://www.tonyparisi.com
OPEN TOOLSFOR CROSS-PLATFORM VR
game engines/IDEs
Goo Enginehttp://www.gootechnologies.com/
Verold http://verold.com/
Turbulenz https://turbulenz.com/
PlayCanvas http://www.playcanvas.com/
Artillery Engine
https://artillery.com/
Sketchfab https://sketchfab.com/
Unreal https://www.unrealengine.com/
Unity http://unity3d.com/#unity-5
scene graph libraries/page frameworks
Three.js
http://threejs.org/
SceneJS
http://scenejs.org/
BabylonJS
http://www.babylonjs.com/
Vizi
https://github.com/tparisi/Vizi
Voodoo.js
http://www.voodoojs.com/
PhiloGL
http://www.senchalabs.org/philogl/
tQuery
http://jeromeetienne.github.io/tquery/
04/09/2023
http://www.tonyparisi.com
PRO TOOLS FOR WEB VR
ported in 5 days
Unreal native C++ engine -> JavaScript
Emscripten + asm.js
60FPS
Unreal 4 in WebGLhttps://www.youtube.com/watch?v=c2uNDlP4RiE#t=42
EMSCRIPTEN - THE COOLEST HACK EVER!
https://github.com/kripken/emscripten
CROSS-COMPILE C++ NATIVE CODE TO JAVASCRIPT
asm.js- LOW-LEVEL OPTIMIZED JAVASCRIPT
UNITY, UNREAL ENGINES USE THIS TO DEPLOY ON THE WEB
WATCH OUT: HUGE DOWNLOADS AND HARD TO DEBUG…. !
04/09/2023
http://www.tonyparisi.com
WEBVR AND CSS
MOZILLA VR CSS SHOWCASEhttp://mozvr.github.io/vr-web-examples/domvr-birds/
04/09/2023
http://www.tonyparisi.com
WEBVR AND CSS<script type="text/javascript" src="js/vrutils.js"></script>
<script>
/* VR Headset and its associated orientation/position sensor */var vrHMD, vrSensor;
/* Element that will serve as our camera, moving the rest of the scene around */var cssCamera = document.getElementById("camera");
/* the camera's position, as a css transform string. For right now, we want it just in the middle. */var cssCameraPositionTransform = "translate3d(0, 0, 0)";
/* Request animation frame loop. */function animate() { /* Acquire positional data from VR headset and convert into a transformation that can be applied to CSS. */ if (vrSensor !== undefined) { var state = vrSensor.getState(); var cssOrientationMatrix = cssMatrixFromOrientation(state.orientation, true); }
/* Apply positional data to camera element */ cssCamera.style.transform = cssOrientationMatrix + " " + cssCameraPositionTransform;
window.requestAnimationFrame(animate);}
query HMD device state
calculate “camera” orientation
update “camera’s” CSS 3D transform
04/09/2023
http://www.tonyparisi.com
VR + MLA MARKUP LANGUAGE FOR THE METAVERSE?
GLAM (GL AND MARKUP) - A DECLARATIVE LANGUAGE FOR 3D WEB CONTENThttps://github.com/tparisi/glam/
DEFINE 3D SCENE CONTENT IN MARKUP; STYLE IT IN CSS
<glam> <renderer type="rift"></renderer> <scene> <controller type="rift"></controller> <background id="sb1" class="skybox"> </background> <group y ='1' z='-3'> <sphere class="bubble skybox”> </sphere> <sphere class="bubble skybox”> </sphere> </group> …
THE MARKUP<style> .skybox { envmap-right:url(../images/Park2/posx.jpg); … } .bubble { radius:.5; shader-vertex:url(../shaders/fresnel.vs); shader-fragment:url(../shaders/fresnel.fs); shader-uniforms:mRefractionRatio f 1.02 mFresnelBias f 0.1 mFresnelPower f 2.0 mFresnelScale f 1.0 tCube t null; } #sb1 { background-type:box; }</style>
THE CSS
04/09/2023
http://www.tonyparisi.com
CHALLENGES
WEBVR ON OCULUS IS NOT READY FOR PRIME TIME (THAT’S OK NEITHER IS OCULUS!) LATENCY IS THE BIGGEST ISSUE – BROWSER NEEDS
TO UN-THROTTLE AT 60FPS
BUT WE’RE GOOD TO GO ON CARDBOARD! 60FPS WORKS GREAT NEARLY 2 BILLION VR DEVICES ALREADY DEPLOYED!
FRAMEWORKS AND TOOLS ARE TYPICALLY WEB-ISH: UNDER DEVELOPMENT, ALWAYS IN FLUX, PRETTY MUCH OUT OF CONTROL BUT OPEN SOURCE SO WE CAN LIVE WITH IT
04/09/2023
http://www.tonyparisi.com
LINKS
BROWSER DEV BUILDSFirefox http://blog.bitops.com/blog/2014/08/20/updated-firefox-vr-builds/
Chrome https://drive.google.com/folderview?id=0BzudLt22BqGRbW9WTHMtOWMzNjQ
MOZILLA NEXT STEPS FOR VR (THURSDAY 23 OCT 2014)https://air.mozilla.org/virtual-reality-the-web-next-steps/
SAN FRANCISCO WEBVR MEETUPhttp://www.meetup.com/Web-VR/
WEBVR MAILING [email protected]
CARDBOARD VR SHOWCASEhttp://vr.chromeexperiments.com/
KEEP IN TOUCH
http://www.tonyparisi.com
04/09/2023
[email protected]: auradeluxehttp://twitter.com/auradeluxehttp://www.tonyparisi.com/http://www.learningwebgl.com/
GET THE BOOKS!
WebGL: Up and Runninghttp://www.amazon.com/dp/144932357XProgramming 3D Applications with HTML and WebGLhttp://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966
GET VIZIhttps://github.com/tparisi/Vizi
MEETUPShttp://www.meetup.com/WebGL-Developers-Meetup/
http://www.meetup.com/Web-VR/
BOOK CODEhttps://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAMhttp://www.glamjs.org/https://github.com/tparisi/glam/
WORKhttp://www.vizi.gl/
CREDSCo-creator, VRML and X3D
HACKING REALITY:
HTML5 AND JAVASCRIPT
FOR CROSS-PLATFORM VR
TONY PARISIOCTOBER, 2014
