Upload
others
View
33
Download
2
Embed Size (px)
Citation preview
1
Enable Your Virtual Reality
& Augmented Reality
Technology on ARM
Nizar Romdan
Director of Ecosystem
Media Processing Group
17th June 2015
2
AR is open and partly immersive – you can see through and around it
AR puts virtual things into users’ real worlds, augmenting them
AR is great fun for games, but maybe not as much fun as VR
AR could be used outdoors for various use cases
VR is closed and fully immersive – you are inside it
VR puts users inside virtual worlds, immersing them
VR is great for games and 3D films, that’s what it was designed for
VR is primarily a living room, office or seated experience
Augmented Reality vs Virtual Reality – Similar or Different?
Augmented Reality (AR) and Virtual Reality (VR) are inherently different!
3
Augmented Reality
4
AR is overlaying location based/user
specific/geographic information on a
real-time camera feed
Several other definitions exist
Current AR techniques tend to be
more CPU focused than GPU
Process the camera feed Feature
Recognition Overlay information
Gravimetric, Fiduciary, Markerless, QR,
Natural Feature
Utilise the power of the Mali GPU to overlay
3D effects on to the view of the real world
Introducing Augmented Reality
5
Augmented Reality Use Cases
GAMING INTUITIVE UI
NAVIGATION, SECURITY SHOPPING, SOCIAL MEDIA, LBS
Applications Registration & Tracking
Featu
re R
eco
gn
itio
n
3D
Gra
ph
ics
6
Augmented Reality Challenges
“True to Life”
3D Graphics
Immersive
User Experience
Battery Life
Responsiveness
Robustness
Relevant Content
Reliable
Network
Ergonomically
Attractive
Fast
Sensors, Tracking
Key
Challenges
Influenci
ng
fac
tors
7
Virtual Reality
8
VR is not new – People were discussing VR the 90’s
Expensive and bulky hardware
Inadequate technology offered limited graphics capabilities
Virtuality – Sega VR – Nintendo Virtual Boy
VR is now accessible
Emergence of low-cost hardware based on existing platforms
Mobile platforms offering console-like performance offering more
realism
VR is about immersing users into Virtual worlds
All the virtual world is graphically rendered
Introducing VR Virtuality
Sega VR
9
VR Use Cases
Games Experiences Training
Education TV/Film Advertising
10
Virtual Reality Segmentation
Samsung Gear VR Zeiss VR One Google Cardboard
Oculus Rift Sony Morpheus HTC Vive (Steam VR)
Mobile
Desktop/Console
11
Headset based on Smartphone
Partnership between Samsung Mobile & Oculus
First version for Galaxy Note 4 (Mali-T760)
Second version for Galaxy S6 (Mali-T760)
Resolution 1080p – 60 Hz
First VR appstore
Introducing GearVR
12
Stereoscopic Display (via headset) – Providing wearer with a sense of depth
Render left eye and right eye independently
Barrel distortion effect applied to rendered output via post processing
Sensor input for head-tracking – To add realism to the output image
Sensor input for interaction (hands-tracking) – To provide an immersive experience
How Virtual Reality Works
Left eye
Right eye
Eye gap (IPD)
Left eye Right eye
Lens
Lens
2x CPU
Processing
2x Vertex
processing
Increased Fragment
Processing
13
Virtual Reality Technical Challenges
14
Mali Well Positioned for Mobile VR Samsung Gear VR is first real VR device (based on Mali-T760)
Mali Bandwidth Reduction Features ASTC & AFBC provide significant advantage
Mali Driver Extensions Minimize latency for mobile VR under development
Mali Ecosystem Engagements with middleware and game studios started
Mali VR SDK Under planning – H2’15 release
Geomerics Enlighten
Best lighting solution for VR
ARM Enabling Mobile VR Experiences
Enlighten Enables Immersive and Realistic VR Experiences
VR requirements:
→90+ fps for best user experience
→High resolution
→One view per eye
→Low latency
Limited GPU resources
for immersive 3D effects
Enlighten computes indirect lighting on the
CPU, independent of GPU
Engine
CPU Light and
colour data
Precompute
data
Enlighten output
Geometry Final scene
GPU
16
ARM VR Ecosystem
VR HARDWARE VR MIDDLEWARE & TOOLS VR CONTENT
Resolution
Games
17
Thank You!
malideveloper.arm.com/cn
Special Thanks to
nDreams & Starship, UK