View
1
Download
0
Category
Preview:
Citation preview
1
Virtual RealityOverview & Update
Philip LelyveldUSC Entertainment Technology Center
PLelyveld@ETCenter.org
Founded 1993 by George Lucas tobringtogethertechnologyandentertainmentvisionariestocollaborateonthefutureofentertainmenttechnology
Adamantlyneutral• Discussions• Collaborations
• Research• Coverage
PLelyveld@ETCenter.org 2
CURRENT MEMBER COMPANIES
3
CURRENT MEMBER COMPANIES
4
In Augmented Reality, the world that you would normally see if you walked down the aisle without the screen is being augmented – enhanced, supplemented, added to - with additional data and experiences.
In Virtual Reality you are virtually – nearly, almost – but not really there in the grocery store aisle.
PLelyveld@ETCenter.org 5
RealEnvironment
AugmentedReality
VirtualReality
6
TransitionalMixed Reality
Blended Reality
TheAR– Reality– VRContinuum
PLelyveld@ETCenter.org
DifferentwaystohaveaVRexperience
MagicWindow
Cellphone-based viewer
HeadMountedDisplay(HMD)
GroupVR
7PLelyveld@ETCenter.org
8
VirtualRealityonCellphoneorTablet“MagicWindow”
Near$0costofentry
Swipe
Tilt and wave
PLelyveld@ETCenter.org
9
Cellphone-BasedViewerwithHeadphones
Near$0costofentryPLelyveld@ETCenter.org
Oculus Rift with Oculus Touch10
HeadMountedDisplayswithPeripherals
PLelyveld@ETCenter.org
TetheredversusUntethered
• BandwidthIssue• Compression• Latency
Sitting,Standing,WalkingHeadMountedDisplayswithPeripherals
PLelyveld@ETCenter.org 11
12
GroupVRExperiencesLocationBasedEntertainment
PLelyveld@ETCenter.org
TheVoidopensTimesSquareVRCenterFirstattraction:“Ghostbusters:Dimensions”OpensJuly1,2016,developedincooperationwiththemovie
TheLanguageofVR:Part1
Existingtoolsandskillsarebeingreworkedtocreatesomethingverydifferent
13PLelyveld@ETCenter.org
PLelyveld@ETCenter.org 14
“Virtual Reality is not a media experience. When it is done well, it is an actual experience.” Professor Jeremy BailensonStanford Virtual Human Interaction Lab
VideoPLelyveld@ETCenter.org 15
16
Senseof‘Presence’
1.Moresensesaccurately engaged=increasedpresence
2.FidelityContract
PLelyveld@ETCenter.org
17
HeightenedSenseofEmpathy
HenryOculusStoryStudio
7/28/15PLelyveld@ETCenter.org
“Ifeelhorribleaboutmakingthis,”…“Youactuallyfeelguilty.MymouthdroppedthefirsttimeIshotsomeonein
my GTA:V VRsetup.”
GrandTheftAutovideo-gamedesignerJosephDelgado,uponexperiencingtheVRversionofGTA:V
5/13/16PLelyveld@ETCenter.org 18
19
HeightenedSenseofEmpathy
3D facial expressions evoke stronger emotionsthan their 2D counterparts
due to the illusion of non-mediation
PLelyveld@ETCenter.orgSource: 11/24/15 research from Aalto Univ. & Univ. of Helsinki
“Hunger”(VirtualFoodBank)byNonnydelaPena20
Agency
PLelyveld@ETCenter.org
“My Mother’s Wing”By VRSE for UN
AgencyImmersiveJournalism
PLelyveld@ETCenter.org 21
TheultimateVRexperiencewillbeastorythatknowsyouareinthestory.
MarkBolas
PLelyveld@ETCenter.org 22
You are in a snow stormYou must get warm in 5 minutes or you will die
Audio is a key ‘presence’ componentPlayers ‘feel’ colder as they playYour body moves slower as time goes on
Languagestory telling + engagement design elements
From “Storm” by Andrick BregmanPLelyveld@ETCenter.org 23
Tools
PLelyveld@ETCenter.org 24
NextVR3603DRig6RedPro6Kcameras60fpsSpatialaudiocapture
JauntJ1-24REarly201632K(16Kpereye)120fps 25
VRCameras
Captured 3D
PLelyveld@ETCenter.orgAlgorithmic 3D
Nokia OZO
PLelyveld@ETCenter.org 26
Captures 360 stereoscopic 3D through:- 1950 capture with 50% overlap for 3D- 8 synchronized global shutter sensors- 8 integrated mikes for spatial audioJaunt committed to support Ozo for camera & production workflow
Surround 360 by Facebook
PLelyveld@ETCenter.org 27
Not a product; an open source design for others to build on2048x2048 60 fps genlocked cameras – 14 around, 1 up, 2 down
Orah 4i live-streaming Prosumer VR camera from VideoStitch
PLelyveld@ETCenter.org 284fisheyelenses,4mikes,4Koutput,30secdelay,~$3,600
VRCameras(consumermkt.)
Bublcam from Bubl ($799)
Kodak SP360 (2X$290)
Ricoh Theta ($350)
Giroptic ($499)PLelyveld@ETCenter.org 29
Samsung 360Two 15MP cameras360-degree video in 3,840-by-1,920 resolutionGalaxy S7 and S7 Edge, Gear VR
Nikon KeyMission 360 ($?)Waterproof to 100’
30
Wild– TheExperience,VRshortfromFox(ReeseWitherspoon, LauraDern)
Sound
PLelyveld@ETCenter.org
BinauralAudio
3Dio
Ambisonic Audio
TetraMic
PLelyveld@ETCenter.org 31
32
Wild– TheExperience,VRshortfromFox(ReeseWitherspoon, LauraDern)
SoundAudiobreakdown• Livespatialaudiocapture• Miked audiowithposition-tracking• Audiowithnoposition(musicscore)
PLelyveld@ETCenter.org
33
Stitching
PLelyveld@ETCenter.org
Stitching
FromtalkbyAnrick BregmanPLelyveld@ETCenter.org 34
Stitching
StandardVRcamerarigscanhavebuilt-in stitchingsoftwareFromtalkbyAnrick Bregman
PLelyveld@ETCenter.org 35
Editing tools
Skyboxiscreatinga‘directtoRift’ playerfortheAdobesuitethathangsthemenuswithintheHMDspace
PLelyveld@ETCenter.org 36
37
UserInterface
Tomaintainpresence,UIfornavigationdesign andinformationcommunicationmustbeconsistentwiththerestoftheexperience
PLelyveld@ETCenter.org
GameEngines
DigitalDistributionofVR
• YouTube360• AppleTV360• Hulu360• Amazon360• Netflix360
• Littlstar• Vrideo• WearVR• ….
PLelyveld@ETCenter.org 38
PLelyveld@ETCenter.org 39WearVR’s new home page 4/16/16
40
Technical ConsiderationsBandwidth and QoSCompressionLatencyProcessing powerLocal buffer/storage …
Option: Distribute a 4K frame, then client warps it into a sphere.
Capture at high-resolution, distribute at what client-processing, bandwidth, and display tech allow.
Google Fiber expressed interest in streaming VR
PLelyveld@ETCenter.org
HeadMountedDisplayTechnicalTargets
4K-8Kresolutionpereye
>90framespersecondrendering (vscapture)
<20msec latency
2200 fieldofview(FOV)
PLelyveld@ETCenter.org 41
Github VR standards discussions
Google put out RFComments for embedded spatial audio standards
FB camera released as open source to drive innovation and open standards
Google DayDream VR for Android N, including reference specs for phones and Gear-like HMDs
Standards?Standardization efforts.
PLelyveld@ETCenter.org 42
TheLanguageofVR:Part2
LiveActionorCGI?
PLelyveld@ETCenter.org 43
44
Camera-capturedVRLockedtocamerapositionTheDirector’sJourney
ComputerRenderedVRFreetowander(Storyworld)
Subtlecoercion
Decision
PLelyveld@ETCenter.org
MixedRealitySphericalcaptureliveandinsertintoCGI
“Volumetriccapturewillchangeeverything.”ChrisEdwards,TheVRCompanyPLelyveld@ETCenter.org 45
Hybrid- OutsidetoInside‘VolumetricVR’
Replay-technologies.com8i.comMS Hololens Holoport PLelyveld@ETCenter.org 46
Capture from many anglesCreate a model that can be explored
LytroCinemalight-fieldcameraCinemaframetoday,volumetricVRinthefuture
Everypixelhascolor,direction,x/y/zpositioninfoEveryobjectcanbeseparatedfromitssurroundings– greenscreen755RAWmegapixel40Kresolution,300FPS, 400Gbpsofdata
PLelyveld@ETCenter.org 47
Deluxe Launches VR Unit to Facilitate Industrial-Scale Virtual Reality Productions
Deluxeassembledateam ofspecialistsfromboththeworldofvisualeffectsandvideogameproduction.
Variety, 4/11/16PLelyveld@ETCenter.org 48
TheLanguageofVR:Part3
Thingstothinkabout
PLelyveld@ETCenter.org 49
TheCameraisYourAudience’sHead!!!
We don’t see shaky-cam when we run
The world doesn’t tilt when we tilt our heads
We see the world differently sitting and standing
PLelyveld@ETCenter.org 50
VRputsyouinthesceneThereisnothirdwalltobreak
Maintainingpresence dependsonmaintainingexpectationsintheexperience
HIDE(orbeabletoremove)EVERYTHINGTHATDOESN’TBELONGTHERE
CrewLightsEquipmentShadowsofequipment…NewLEDlightformfactorscanbehiddeninthesceneOrShootinmultipledirections(ex.180then180)andblendinpostIndoorscenes;sampletheceiling,putlightsthere,paintthemoutlater
Real-timeremotemonitoringoftheshootisbeingtested(ex.Teradek Sphere,Ozo system)ViewdailiesinHMDforsenseoftheexperienceANDlookatindividualsensorfeedsforproblems
PLelyveld@ETCenter.org 51
Solvingproblems
Greenscreenhelmetontoheadless torso.
Headbouncesdifferently frombody. FromtalkbyAnrick Bregman
PLelyveld@ETCenter.org 52
Thefinalimagemustbeerror-freeImperfectionsstandoutglaringly,andbreakpresence
Stitchingisa3-passprocess;- algorithmicreal-timefirstpass- quickpassforeditingwork- compositingheavyworkflowwithclean-upstitching
PLelyveld@ETCenter.org 53
54
Wild– TheExperience,VRshortfromFox(ReeseWitherspoon, LauraDern)
SoundAudioEngineering• Difficulttoproperlypositionfromaflatscreen• Toolsandroomsforspatialaudioarebeingdeveloped• Gameenginebasedaudioknowledge(Unity,Unreal)
PLelyveld@ETCenter.org
What’syourmessagetoSMPTEmembers?
Buyacheapcameraandexperiment!
PLelyveld@ETCenter.org 55
Resources
PLelyveld@ETCenter.org 56
LosAngelesVRandImmersiveTechnologiesMeetup
PLelyveld@ETCenter.org 57
August5th &6th 2016LosAngelesConvention
Center
SoCal VROctober 30, 2016Long Beach CC
New York
San Francisco
PLelyveld@ETCenter.org 59
For-Profitclasses
PLelyveld@ETCenter.org 60
MonthlySaturday,May28thVRLASchoolfocusesonthetoolsandprocessesforeditingstories
Websites
Road to VR – www.roadtovr.com - latest newshttps://www.facebook.com/groups/Virtual.Realities/ - latest newshttps://www.facebook.com/RoadtoVR/ - latest newsWearVR, Littlstar, Vrideo – latest contentReddit VR – community discussion groups
UploadVR - http://uploadvr.comVR Reporter - https://virtualrealityreporter.comVR Scout - http://vrscout.comWomen in VR – www.wivr.netWomen in VR Los Angeles - http://www.meetup.com/Women-in-VR/All Things VR – allthingsvr.coKzero - http://www.kzero.co.uk/blog/
PLelyveld@ETCenter.org 61
Podcasts
Voices of VR - dailyReal Virtual Show - weeklyhttp://a16z.com/?s=virtual+reality
http://www.apoip.org/home/events-calendar/http://allvirtualreality.com/events/list-virtual-reality-events-vr-expo-exhibitions-
conferences-2016.htmlhttps://www.jvrb.org/events
Lists of VR and AR conferences
PLelyveld@ETCenter.org 62
Modern era consumer VR is <3 years old
New art form and communication tool
WeneedtobringcinematicvisualandaudiocraftspeopleintothisfieldParaphrasedfromconversationwithJacquelineBosnjak,CEO,QDepartment
PLelyveld@ETCenter.org 63
Think about…
How to direct attention in an open space- Theme park and theme park ride design- Theatre in the round- Architecture and design
Existing skills and tools that port over to VR
Experimenting with a cheap VR camera
Learning Unreal, Unity, and other game engine UI tools
PLelyveld@ETCenter.org 64
ImagefromNotesonBlindnessVRexperience
Consumersbuyexperiences,nottechnologies
PLelyveld@ETCenter.org 65
66
Thank YouPhilip Lelyveld
USC Entertainment Technology Center
PLelyveld@ETCenter.org
www.etcentric.orgwww.PhilipLelyveld.com
Recommended