Upload
nguyenmien
View
237
Download
3
Embed Size (px)
Citation preview
Tim Leland
VP, Product Management
Qualcomm Technologies, Inc.
August 2015
1
2
Immersive experiencesThe experiences worth having, remembering, and reliving
Draw you in…Take you to
another place…
Keep you present
in the moment…
Intuitive
interactions
Immersive is the intersection of three areas
3
Visual
quality
Sound
quality
Immersion
Intuitive
Interaction
Virtual reality is the ultimate level of immersion
Visuals so vibrant that they are indistinguishable from
the real world
Sound so accurate that it is true to life
Interactions so intuitive that you forget there is even an
interface
Stimulating our senses with realistic feedback
Still a few questions to be answered….
5
Sources: CNET: Business Insider, Oct. 30 2014
“There seems to be a lot of excitement around something that, to me, is a
yawn, frankly,” James Cameron said when asked about the rapidly growing
VR space.
“The question that always occurred to me is, when is it going to be mature,
when is it going to be accepted by the public at large, when are people
going to start authoring in VR and what will that be?”
2/26/2016 Qualcomm Confidential and Proprietary
When will the devices come? Global VR Headset Forecast
>100M Units in 2020
$62B Market Size in 2025
Wireless/Mobile = Most of
the VR Headset volume
Too early to know if this is
an accurate ramp, but there
will be new VR uses for
premium tier Snapdragon
Source - Piper Jaffray, May 2015
VR Headset Sales (In Millions)
Far more Immersive Gaming
360 Degree Video
− Sports
− Concerts
− Cinema
Social Experiences
− Virtual Tourism, Virtual Shopping
− New Education & Working Environments
So what will the new VR experiences be?
Visual
qualitySound
quality
Intuitive
interactions
Immersive virtual reality has extreme requirements
8
High resolution audioUp to human hearing capabilities
Extreme pixel quantity and qualityScreen is very close to the eyes
3D surround soundRealistic 3D positional audio that is
accurate to the real world
Precise motion trackingAccurate on-device motion
tracking
Broad field of viewHumans have a 180 degree field
of view
Minimal latencyMinimized system latency to
remove perceptible lag
Visual
quality
Sound
quality
Intuitive
interactions
Immersion
Stereoscopic displayHumans see in 3D
Natural user interfacesSeamlessly interact with VR using
natural movements, free from wires
An end-to-end approach is required for virtual realityOptimizing for motion to display latency
Gyro, accelerometer, camera, or other
sensors detect movement
Motion detectionCalculate movement and create
appropriate image through multi-stage
visual processing
Visual processingImmediately update display with image to minimize latency
Screen update
9
End-to-end approach
A heterogeneous computing approach is needed for VRSpecialized engines to meet processing requirements at low power and thermals
10
Location
GPU
DPU (Display)
Camera ISP ModemDSP
(audio, sensor, &camera co-processor)
CPUs
Memorysubsystem
VPU (Video)
Virtual realityComputer vision, image processing, sensor processing, graphics, video processing, location, and cloud interaction
* Not to scale
Entire SoC is used!
High-utilization
Still photos & captured videos that are:
− Never blurry, always focused on what the user wants to see
− Clear and sharp regardless of motion & lighting conditions, with leading performance
Color as nature & artists intended it to be
− Accuracy & color gamut matching all the way through the pipeline, from camera, video,
graphics out to display
Highest quality mobile movies and games
− Mobile performance that rivals what users currently get in the living room
Responsive, smooth interaction with home screen UIs, App UIs & Browsers:
− Free from “jank” and low frame rate that can detract from user experience
Other user experience goals for Snapdragon Visual Processing
Taking an end-to-end approach to improve visual qualityThe journey of light from the mobile camera to a correct pixel on the display is challenging
End-to-end solutionsColor management, artifact removal, optimized click-to-shoot time
Component tuning | Image processing | System optimization
Mobile size constraints create challenges:
• Poor light gathering
• Increased noise
Lens and sensorAddresses camera and display
challenges while enhancing images
SoCPhysical limitations of LCDs and LEDs:
• Partial color gamut
• Limited contrast and brightness
Display
Qualcomm SpectraTM Camera ISP
• Improved image quality, with more natural skin tones
• Pre & Post processing on HVX for new UX
• Improved power efficiency and higher throughput
Benefits
What’s New in Snapdagon 820
• Better Image Quality
• Enhanced de-mosaicing for new ~1.0u image sensors
• New hardware for radial noise reduction & color artifact
correction
• Camera sensor pre-processing streaming to DSP/HVX
• Improved hybrid autofocus framework
(PDAF/Contrast/Laser)
• GPGPU Optical Zoom
• Universal Bandwidth Compression (UBWC)
• Serial C-PHY interface
Without
SpectraTM ISP
Using
SpectraTM ISP
The new Adreno 530 GPU for Snapdragon 820
Adreno 530 is the top tier in a new family GPUs
− Designed for mobile UIs, Games, Web and VR
− Up to 40+% better performance than Adreno 430
Will include next generation capabilities
− Supports latest in Graphics and Compute APIs
− Support for 64-bit Virtual Addresses
Will deliver superior performance per mW
− New sorting and compression techniques
− A new GPU Power Manager that provides finer
granularity of power control
Improved compute and quality of service
“Paris Apartment”
Snapdragon 820 +
Epic Unreal Engine4Photorealistic 3D Graphics
A heterogeneous computing approach is neededSpecialized engines to meet visual processing requirements at low power and thermals
15
Location
GPU
Display Processor
Camera ISP ModemDSP
(audio, sensor, &camera co-processor)
CPUs
Memorysubsystem
Video Processor
Camera image captureLens correction, noise removal, and 3A
Image processingColor correction, tone mapping, and computational photography
High-end gamingRendering, tessellation, and geometry shading
Video compressionVideo encode and decode
Display imageCompositing, gamut mapping, and calibration to displays * Not to scale
High-utilization
Low-utilization
Kryo™
CPUs
Location
Adreno™ GPU
Adreno™ DPU
Snapdragon
“X” LTE
Modem
Memory
Subsystem
Spectra™
ISP
Adreno™ VPU
Using an efficient heterogeneous computing solutionSpecialized engines to meet processing requirements at low power and thermals
• TruPalette™ color enhancement
• ecoPix™ power savings
• Composition
• Superior performance per mW
• GPU compute for image processing
• Graphics rendering with HW tessellation
• Universal bandwidth compression
• Memory management for heterogeneous
computing
• Custom instructions for image, audio, and
sensor processing
• High-quality voice with Fluence™ Pro
• 3D immersive audio
• Ultra-low power sensor processing and fusion
• 14-bit twin ISP
• Ultra-fast laser & phase detect autofocus
• Triple noise reduction: Bayer, wavelet, and
temporal
• Dual camera support for depth, sensor fusion,
and ultra low light
• Efficient, high quality 4K encode/decode
• Industry leading Wi-Fi Display
• Efficient custom CPU architecture
• Image and audio processing
* Not to scale
16
• High bandwidth at low power and latency
• Multiple radio technologies and bands
Snapdragon processors
Hexagon™ DSP (audio, sensor, &
camera co-
processor)
Some features are only available on specific Snapdragon processors. Consult processor specifications for feature availability. TruPalette, ecoPix, Adreno, Spectra, Kryo, Krait, Fluence, and Hexagon are products of QTI.
Offering superior development and optimization toolsEnabling content creation and tuned devices
17
Content creation tools
Specialized solutions for development
• Adreno SDK
• Vuforia™ SDK
• Hexagon SDK
Debugger for solving
• Snapdragon Debugger
Profiler for optimization
• Snapdragon Profiler
Third-party middleware support
• Unity & Epic Unreal Engine
Device optimization tools
Calibration and tuning
• Chromatix™ color optimization tool
• Qualcomm Display Color
Management
• Qualcomm Audio Calibration Tool
Analysis and debugging
• Qualcomm Commercial Analysis
Toolkit
• Qualcomm eXtensible Diagnostic
Monitor
Other ecosystem enablement
Development devices
• Mobile Development Platform
• DragonBoard™
Customer support
• Customer engineers
Vuforia and Chromatix are products of QTI. DraonBoard is distributed by Intrinsyc.
Efficient heterogeneous
computing architecture
Within device constraints
Development time / Sleek form factor /
Power and thermal efficiency / Cost
Delivering superior immersive experiencesProviding efficient, comprehensive solutions
18
Visual quality
Immersive experiences
Sound quality
Intuitive interactions
Commercialization
Consistent, accurate color
In-focus images
Low-light video & photography
3D surround sound
Noise removal
Dynamic sweet spot
Seamless, responsive,
smooth user interfaces
Intelligent, contextual
interactions
Via Snapdragon™
solutions
Via ecosystem
enablement
Comprehensive solutions
across tiers
Custom designed
processing engines
Snapdragon development
platforms
Ecosystem collaboration
App developer tools
Snapdragon is a product of Qualcomm Technologies, Inc.
For more information on Qualcomm, visit us at:
www.qualcomm.com & www.qualcomm.com/blog
Qualcomm is a trademark of Qualcomm Incorporated, registered in the United States and other countries. All trademarks of
Qualcomm Incorporated are used with permission. Other products and brand names may be trademarks or registered
trademarks of their respective owners
Thank youFollow us on:
© 2015 Qualcomm Technologies, Inc. 19