Photonics in Sony’s novel display technologies · 4 LETI INNOVATIONDAYS VR&AR WORKSHOP June 24 2019 AI. Toward immersive and augmented visual experience. Panorama VR VR Goggles

LETI INNOVATION DAYS VR&AR WORKSHOP June 24 2019

Photonics in Sony’s novel display technologies

Hiroki KikuchiR&D Center, Sony Corporation

Copyright 2019 Sony Corporation

2 LETI INNOVATION DAYS VR&AR WORKSHOP June 24 2019

Outline

• VR/AR Display Technology in Sony

• Large Screen Display• Laser Projector• Crystal LED Display System (Micro-LED Video Wall)

• Augmented Reality Glass

https://youtu.be/6K_6b1wMmiM

https://youtu.be/6K_6b1wMmiM


AI

Toward immersive and augmented visual experience

Panorama VR

VR Goggles

Reality

Immersiveness

Flat Panel Display

AR GlassAugmented Reality

Laser Projector Crystal LED Display System

Immersiveness

VR/AR Display Technology


Laser Projector Crystal LED Display System

Large Screen Displays


A brief history of laser projector

50 m

10 m

• Grating Light Valve (GLV) Laser Projector• RGB laser-based light source• Projection size: 50 m x 10 m (2005 inches)

5000Lm x 12 stacks

World’s first large-sized laser projector by Sony (2005)

High Power RGB Laser

Grating Light Valve

Laser Projector


LCOS(Liquid-Crystal on Silicon)

• Up to 4K resolution• Fine pixel pitch• High contrast• Fast drive speed

Laser Display Key DevicesHigh Power Visible Lasers Projection Optics

I-LV 525nm

CWTC = 25°C

I-LV 455nm

CWTC = 25°C

Visible Lasers for Display and Lighting ApplicationsMurayama, et al Laser Display Conference2019

I-LV 640nm

CWTC = 25°C

Ultra short throw lens

Phosphor Color conversion

4K 0.74inch2k 0.37inch 480Hz (Single panel color sequential)


Projection for Virtual Reality ~ Warp Square

Education

EntertainmentIn-car experience


Projection for Virtual Reality ~ Warp Square

https://www.sony.co.jp/SonyInfo/design/stories/warpsquare/

https://www.sony.co.jp/SonyInfo/design/stories/warpsquare/


Projection for Augmented Reality

Projection area

Projector

Infrared light emission unit

Camera

Portable projector module with hand touch sensor

GM advanced tech windowhttps://youtu.be/0dIia553wVU

AR Smart Window for automotive


A brief history of Crystal LED Display System(Micro LED Video Wall)

100 μm 10 μm LED Size1 mm

Crystal LED DisplaySystem(2016)

Crystal LED Display (CES 2012)

Activ

eM

atrix

Pass

ive

Mat

rix

SMD Mini LED Micro LED

ConventionalLED displays LED

micro-sizing

Activ

e &

Tilin

g

55-inch 1,920×1,080×RGB Micro LEDs

Crystal LED Display System

Technologies for the Crystal LED Display SystemBiwa, et al The Society for Information Display (SID) Display Week 2019

Early stage experiment


Crystal LED Display SystemWorld First Scalable display system using micro LEDs

Display Unit

The Society for Information Display 2019 Display Industry Award

Display of the Year Winner

16K×4K, 792-inch(15,360×4,320 pixels,

19.2 m×5.4 m)

Shiseido Company, LimitedGlobal Innovation Center

Yokohama, JapanOpened on April 13, 2019


Augmented Reality Glass

http://www.google.co.jp/url?url=http://www.smartwatchpro.ru/sony-%D0%B2%D1%8B%D0%BF%D1%83%D1%81%D1%82%D0%B8%D1%82-%D0%BE%D1%87%D0%BA%D0%B8-smarteyeglass-%D0%B2-%D0%BC%D0%B0%D1%80%D1%82%D0%B5/&rct=j&frm=1&q=&esrc=s&sa=U&ei=x6bhVNCEFOGxmwWvyICwCw&ved=0CC4Q9QEwDA&usg=AFQjCNELrUzYUJ3QTfLMQVk2ugYBwht_3Q


A brief history of AR Glass in Sony

First prototype SID 2008 and at CES 2009

Prototype

2008 Sony Entertainment Access Glasses in movie theater

2014

Closed-Caption system for theater


Smart Glass AR/MR HeadsetsValue Propositions

Hands-free Information display

Mixed-reality experience of virtual image

Form Factor Light weightSocially acceptable design

Nicely fashioned

Display Narrower FOVHigh MTFVisibility under sun light

Wider FOV

Video processing

Stress-free superimposing of virtual image to real world

Availability Already there Still challenges for consumer product

Terminology: Smart Glass & AR/MR Headsets






Nicely fashioned


Wider FOV

Video processing



Smart Glass


SED-100A Holographic Waveguide Display

https://developer.sony.com/develop/lmx-001-holographic-waveguide-display/

Resolution: 419 (H) × 138 (V) pixel

Field of view: Diagonal 20°Brightness (max): 2,000 nit

See-through transmittance: More than 85%

Eye-box size: 9 mm (H) x 6 mm (V)

Dimension: H: 27.8 x W: 93.5 x D: 27.2 mm

Waveguide thickness: 1.0 mm (*3,14 mm incl protective plating)

Weight: Approximately 13 g

Smart Glass


In-couplingHOE

Out-couplingHOE

Blue

Red

GreenCollimator

Micro Display

LEDEye

1 mm

Plastic waveguide

Full-color plastic waveguide

Next Generation ~ Full-Color Plastic-waveguide

A Plastic Holographic Waveguide Combiner for Light-weight and Highly-transparent Augmented Reality Glasses Yoshida, et al The Society for Information Display (SID) Display Week 2018

Smart Glass


Hologram recording by roll-to-roll film process

Laser exposure

Optical block

Roll-to-roll alignment and transfer process.Machine Vision

Roller

HOE roll-film

Feed directionPlastic substrate

Plastic holographic waveguide

HOE

R2R HOE alignment and transfer process HOE: Holographic Optical Element

3 degrees

8 degrees

Sharp image quality maintained after transfer process

Smart Glass

Transfer to multiple designs of glass


Electrochromic Dimming Device for AR Glass

A Plastic Electrochromic Dimming Device for Augmented Reality GlassesMachida, et al The Society for Information Display (SID) Display Week 2018

Bleached state (T>70%)

Colored state(T<10%)

Viewing imageSunglasses lenstransmittance 26%

PlasticDimming Device

V

Plastic

ITOHxWO3

Ta2O5

ITOIrSn(OH)x

Plastic

Polymer Passivation layer

AR

AR

Al2O3

Al2O3

H+

+ -

+-

【WO3 layer】 WO3 + xH+ + xe- HxWo3

【IrSnO layer】 IrSnO + xOH+ - xe- IrSn(OH)x

Smart Glass






Nicely fashioned


Wider FOV

Video processing



AR/MR Headsets


Latency Compensation of Super-imposed AR Image

Correction disabled Correction enabled

AR/MR Headset

Latency Compensation for Optical See-Through Head-Mounted with Scanned DisplayAga, et al The Society for Information Display (SID) Display Week 2019

Not "floating" but "pinned to”the real world


Smart Glass AR/MR Headsets Human Eyesight Enhancement

Value Propositions



Augmentation of human’s vision


Nicely fashioned


Wider FOV Ultimate visual experience

Brightness, FOV, 3D Cues…Video processing



Moon-shot challenge?

How can we present virtual images as the same perception of real objects?

?

https://www.google.co.jp/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwigkYqk_MbUAhWEWbwKHUlHBswQjRwIBw&url=https://diamondvision.com/lasik-and-human-enhancement/&psig=AFQjCNEFzp3Dqg1RBBuYwCCG2dvgXRKVmg&ust=1497860724973994


Ultimate visual experience ～ Retinal Scan

A Compact Retinal Scan Near-Eye DisplayAkutsu, et al SIGGRAPH ’19 Emerging Technologies, July26-August01, 2019, Los Angeles, CA, USA

Hologr

EyeMEMS

θλ2

θλ1

θλ3λ1,2,3

Eye

n2

Λ

Holographic combiner

θλ2θλ1

θλ3

λ1,2,3

Holographic compensator

https://s2019.siggraph.org/conference/programs-events/emerging-technologies/


Retinal Scan display


Summary


Photonics Technology for AR/VR~ Building BlocksGenerate light Modulate light Deliver to eye

LaserProjector

AR Glass

Laser MicrodisplayLCD Optics

Micro LED Driving & Tiling

MicrodisplayLCD, OLED

HolographicWaveguide

Crystal LED Display System

http://live.wired.jp/wp-content/uploads/2015/02/sony-smarteyeglass-displayjpg.jpg


In summary…

• VR & AR is not only for Glass/HMD but also for large screen display

• AR Glass still has room for significant progress

• Photonics are key technology driver of VR & AR• Micro OLED, Micro LED, Laser

• HOE, DOE

• Metasurface optics, Nano-photonics

• Eventually, AR display technology may enhance human’s ability of eyesight • Ultimate visual experience of FOV, brightness, contrast, 3D cues,…

SONY is a registered trademark of Sony Corporation.

Names of Sony products and services are the registered trademarks and/or trademarks of Sony Corporation or its Group companies.

Other company names and product names are registered trademarks and/or trademarks of the respective companies.

Thank you

Documents

Photonics in Sony’s novel display technologies · 4 LETI INNOVATIONDAYS VR&AR WORKSHOP June 24 2019 AI. Toward immersive and augmented visual experience. Panorama VR VR Goggles