AR/VR in perspective
Tech , Effect and ValueCarl Henrikson & Ahmed Assal
Building the right things
Building thingsright
Running thingsefficiently
Carl HenriksonSenior Consultant Business Advisory
Virtual reality - an old dream which is finally becoming reality
Stanford University Libraries
1966
AUGMENTED REALITY
The Ultimate Display – a.k.a. Damocles Sword
Head mounted augmented reality display
Ivan Sutherland 1968
80-ies and 90-ies...
But finally...
2014: Rirtualenaissance
V
VRAR
Learning and training
Decision making
Communication
Sales
Marketing
Immersive design
Location based
Entertainmentand information
Emerging applications
Learning and training
Virtual manuals
SimulationAR and Industrial
maintenance
Assistive AR/VR tools
Decision making
Real time data and analytics visualization
What if scenarios
Remote systems control and analysis
Communication
Augmented vision
Remote experts Remote collaboration
Telepresence
Remote Collaboration
Remote Expert
Telepresence
25
Sales
Showcases and demos
Configurators & customizers
Trials
Marketing
Experential campaigns
Brand recognition Virtual characters
Store attraction
Immersive design
Conceptual design Design review
Location based
Virtual tours On location gaming
Entertainment& information
Games
Movies
Concerts
Emerging applications
Crime scene reconstruction
Therapy, Pain relief Virtual workspaces
Immersive journalism
MemoAR
Technology Effect Value
The TECHNOLOGY The EFFECT that can be / need to be achived The business/customer MOTIVE or DRIVER
Interactive VR
Consuming VR
VR 360
Cameras
Empathy
engine
360 experience
Virtual
presence
Real time
virtual
presence
Near real
experience /
simulation
Capture your
experience
Anyone can
create a VR
environment
Be able to build/interact with a world around you
Cheap, Easy
access
Time & place
travel
Manipulate the
brain
Experience something
which is not built
Different for each customer and use case
The business value and customer
motive
Begin with...
The effect
which is
needed
then...
The technology needed to
achieve the effect
and finally...
But, how? ...
Outside-in metodology to understand users
gain
gain
gain
gain
Charity organization working with homeless people
Main objective: communicate what it’s like to be homeless so that people get emphatic with their cause and donate money and time
”Empathy engine”
Outside-in metodology to understand users
To get the most from this technology we must...
Become imagineers!
* Stolen with pride from Walt Disney Inc.
*
KEEP CALMAR + VR
IS CURRENTLY AT 1%
AR / VR / MR / 3D RealtimeGraphicsA Technical Perspective
VR / AR / MR / Realtime 3D
Current - Future Future+
Software
Hardware
Content
Procedural
Content
Generation
Automatic and Semi –automatic
Content Generation
Automatic
Animation
Creation Automatic
Mesh Creation
Automated, Mixed-authoring and Procedural Content Generation
44
Automatic Animation Generation
45
Natural Environements
47
Spectacular Hyper-realism
48
Mapbox & Unity 3D
49
Mapbox Styling
50
L-systems and Plants – SpeedTree
51
L-systems and Plants – Xfrog
52
Cities, Streets, Buildings & CityEngine
53
Cities, Streets, Buildings & CityEngine
54
Crowd Creation & Animation – Anima 3
55
Today’s journey
56
AR Works - Occlusion
57
“AR systems never “work” or “don’t work”. It’s always a question of do things
work well enough in a wide enough range of situations.”
58
Google search volumes (AR) over time (trends)
PokemonGo Distribution: July 2016
22-sept-2017: initial release Iphone 819-sept-2017: Initial release of iOS 11 iOS
NEW API: ARKit
April 2012: Google unveiled Project Glass
“Similarly, in your business, be sure that before you focus on theresults or outcome, that you make it crystal clear what problemsyou solve, or why people might need what you offer” - Ian Altmanat Forbes
Jan 2015
AR technologies compared
61
Microsoft Hololens Meta 2 ODG R-7
Magic Leap, Google Glass 2.0, Vuzix, ODG R-9, Daqri Smart glasses, ...
Daqri Moverio BT-300
+ universal use+ poweful, open platform+ standalone device+ strong go-to-market push
- limited battery life
+ universal use+ poweful platform
- connected device
+ universal use+ lightweight
- limited FoV- limited battery life
+ industry use tailored+ powerful platform
- industry use only- limited battery life
+ industry use tailored+ powerful platform
- industry use only- limited FoV
AR SDKsRegular Handheld Devices
62
Vuforia
ARCore
ARKit
Dedicated HWHMD Tango Tablet
Vuforia
ARCore
ARKit
Hololens
Tango
AR SDKs on Handheld and HMD Devices63
Augmented Reality
SLAM
Sensor Fusion
Computer VisionPose Deblurring
Visual-inertial Odometry
Optimal CorrectionSurface Estimation
Camera Calibration
Nonlinear OptimizationScene Understanding
Inertial Measurement Unit
Triangulation
AR SDKs on Handheld and HMD Devices64
It enables developers to position and orient virtual objects, such as
3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device.
Motion TrackingVisual Inertial Odometry (VIO) – to track your position in space (your 6dof pose) in real-timeYour pose is recalculated in-between every frame refresh on your display by matching a point in the real world to a pixel on the camera sensor each frame
AR SDKs on Handheld and HMD Devices65
It enables developers to position and orient virtual objects, such as
3D models and other media, in relation to real world images when these are viewed through the camera of a mobile or HMD device.
3D Reconstruction (point cloud)
66
ARKit vs Tango vs Hololens
HW Design
SW Stack
"RGB-IR" camera for color image and infrared depth detectionFisheye motion cameraAccelerometer, gyroscope & magnetometerAmbient light sensorsMicrosoft Holographic Processing Unit
VIO (motion tracking) Sparse Mapping (area learning)Dense 3D reconstruction (depth perception).
Regular phone cameraAccelerometer & gyroscope
Highly accurate Dead Reckoning Guess to
compensate lack of stereoscopic images
ARKit – Augmented Reality for iOS
67
• API that lets third-party developers to build Augmented Reality apps for iOS• Available only for processors Apple A9 and later (6S, 6S Plus and later) due to performance• Available only operatig system iOS 11 and later (officially released on 19-sept-2017)
Optimal performance from model iPhone 8
• Depth sensing camera system calibrated for AR• Lo-light and 60 fps• New gyro and accelerometer• Accurate motion tracking
Microsoft Hololens
68
• Accelerometer, gyroscope, magnetometer
• Spatial Mapping
• Gesture recognition
• 3D display (FOV: 30°×17.5°)
• Video (2.4 megapixel)
• Sound
• Bluetooth
• Wi-Fi
VR everywhere at MWC!
VR was the big attraction! ..but the real use cases were few!
VR devices in all ranges!
March $599April $799
2nd gen $99
May $200
Oct $499
Multiple version $5 Co
nsu
min
gIn
teractin
g
But Google DayDream and ARM next Gen processors will drive mobile VR from consuming to interactive
71
THANKS FOR LISTENING!