OZO Post Production Workflow - docs.ozo.nokia.com · PDF fileaudio, and as Nokia’s OZO Audio Interchange format. The camera records both picture and sound either to its own internal

O Z O + P O S T P R O D U C T I O N W O R K F L O W

Updated 29th August 2017

References Creator v2.1

2

Camera Overview ..................................................................................................................................... 4

Post Workflow Overview .......................................................................................................................... 4

Importing Assets ...................................................................................................................................... 5

Importing Raw Captures ........................................................................................................................... 6

Frames and Timecode ............................................................................................................................... 8

Viewing Metadata and Mastering Parameters ........................................................................................... 9

Raw Capture Metadata Tab .......................................................................................................................................... 9

Raw Capture Mastering Parameters ............................................................................................................................ 9

DPX Metadata/Mastering Tab .................................................................................................................................... 12

OpenEXR Metadata/Mastering Tab ............................................................................................................................ 12

Stitched Metadata/Mastering Tab ............................................................................................................................. 13

EXB Metadata/Mastering Tab .................................................................................................................................... 13

WAV Metadata/Mastering Tab .................................................................................................................................. 13

Overview of Media Exports ..................................................................................................................... 13

OZO Creator Fast-Turnaround Workflow Example Outline ....................................................................... 15

Export Equirectangular DPX/EXR Files ..................................................................................................... 17

Export Editorial Previews (360 Editorial) ................................................................................................. 18

Batch Queuing the Export of Multiple Assets ............................................................................................................. 20

The Job Queue ....................................................................................................................................... 20

Export Audio Files .................................................................................................................................. 22

Exporting Processed Audio ......................................................................................................................................... 22

Exporting Ambisonic Audio ........................................................................................................................................ 23

Exporting OZO Audio .................................................................................................................................................. 23

Export Per-sensor Files from Editorial Selects ............................................................................................................ 23

Marking In/Out Points in Captures ............................................................................................................................. 24

Frame Number Offset ................................................................................................................................................. 24

Exported Metadata Information ................................................................................................................................ 25

Editorial Decision-making Using OZO Creator .......................................................................................... 25

Selecting “Hero” Takes ............................................................................................................................................... 25

Basic Editing using Compositions ............................................................................................................................... 25

Exporting Compositions .......................................................................................................................... 27

Frame Rate Conversion using OZO Creator .............................................................................................. 27

Color Match and Denoise Imagery .......................................................................................................... 29

Using the Tones Controls ............................................................................................................................................ 29

Using the Color Wheel parameters ............................................................................................................................ 31

Using Denoise Parameters to Improve Imagery ......................................................................................................... 32

Using the Clarity parameters ...................................................................................................................................... 34

Using the Sharpening Parameters .............................................................................................................................. 35

Comparing the Results of Different Filter Settings on Stitched Footage .................................................................... 36

Stitching Overview ................................................................................................................................. 36

Stitching imagery using OZO Creator ....................................................................................................... 37

Importing Fisheye Imagery for Stitching..................................................................................................................... 37

Stitching Directly from Raw Captures ......................................................................................................................... 37

Setting Seam Positions ............................................................................................................................................... 37

Manually Setting Backseam Convergence .................................................................................................................. 39

Exporting Stitched Files .............................................................................................................................................. 41

3

Automated Backseam Convergence ........................................................................................................................... 43

Best Practices and Additional Notes for Stitching using OZO Creator ........................................................................ 43

Stitching Files Using 3rd-Party Services and Tools ..................................................................................... 44

Stitching DPX Files using Deluxe ................................................................................................................................. 44

Stitching DPX or EXR Files using The Foundry’s Cara VR tools for Nuke ..................................................................... 45

Stitching High Bitrate MP4s using Jaunt Cloud ........................................................................................................... 45

Stitching DPX Files with other 3rd-Party Software ..................................................................................................... 45

VFX ........................................................................................................................................................ 46

Metadata Parameter Definitions ................................................................................................................................ 46

Using the OpenEXR Depth Map Export ...................................................................................................................... 47

Using OZOEXB Captures to Create an HDR plate ........................................................................................................ 47

Final Color .............................................................................................................................................. 48

Titles and Credits .................................................................................................................................... 49

Exporting Encoded files .......................................................................................................................... 49

Encoding MP4VR Files with Depth Maps .................................................................................................................... 53

Viewing MP4 Files using OZO Preview ..................................................................................................... 53

Sound Mixing ......................................................................................................................................... 54

Reviewing the Mix using an HMD ............................................................................................................................... 55

Scripting with OZO Creator ..................................................................................................................... 57

Scripted Command Line Usage Examples ................................................................................................................... 57

4

Camera Overview The OZO+ camera is a 3D, 360-degree virtual reality camera that consists of eight 2048 x 2048 (2K x 2K)

sensors, with approximate sensitivity of ISO 400. Each sensor is paired with a wide-angle 195-degree FOV lens,

and captures images that overlap adjacent sensors to provide natural stereoscopic views. Cam1 (Side L), Cam2

(Front L), Cam3 (Front R), and Cam4 (Side R) represent the four sensor views around the “equator” of the

camera. Cam 5 (Top L), Cam 6 (Top R), Cam 7 (Bottom L) and Cam 8 (Bottom R) complete the 8 camera sensor

viewpoints.

By design, the camera captures stereoscopic (3D) imagery through approximately 260 degrees (+/- 130

degrees from center) oriented to the camera front and sides, with the balance of the 360-degree field of view

captured with monoscopic (2D) imagery. Please note however, that when the images are processed by OZO

Creator, the final stitched output only has 230 degrees of stereoscopic imagery.

There are additionally eight microphones spaced around the camera body for recording spatial audio, which can

be exported as a 4.0, 5.0, 7.0, or 8.0 channel 48kHz 24-bit PCM WAV file, first-order AmbiX Ambisonic spatial

audio, and as Nokia’s OZO Audio Interchange format.

The camera records both picture and sound either to its own internal 500GB Digital Cartridge, or to an external

Blackmagic HyperDeck Studio Pro 2 or Blackmagic HyperDeck Shuttle recorder via a 1.5Gbps HD-SDI cable.

The camera compresses the eight 2K x 2K pixel images captured by the sensors into a single compressed data

stream that emulates a 10-bit 1920 x 1080 full HD resolution video frame. Sound captured by the

microphones is recorded as 8 discrete PCM channels, synced with the video. To post produce the sound and

images from these raw camera recordings, first export to appropriate media and industry file formats using

OZO Creator software.

OZO Creator and the associated system requirements can be downloaded from Nokia’s website:

https://ozo.nokia.com/vr/ozo-creator-virtual-reality-software/

Post Workflow Overview Post production with OZO+ is conceptually similar to a standard post production workflow. Just like with a

typical production, editing, color correction, VFX, titles, sound mixing, transcoding to deliverable formats, etc.

are required

The main difference is that instead of starting with a single stream of images from the capture, or two streams

of stereoscopic footage (on a 3D project) a single “raw” capture is exported, and becomes either four or eight

individual streams of images (depending on camera settings). Then a stitching process must be applied to

create either a Left/Right eye stereoscopic output, or optionally a monoscopic output.

This document will help clarify where the stitching process best fits into the process, and step through a typical

professional post workflow with OZO+ footage, concentrating on the usage of OZO Creator.

https://ozo.nokia.com/ozo_en/ozo-software/

5

A high-level overview of production with the OZO camera

Importing Assets OZO Creator allows for the import of several kinds of raw and processed assets.

Captures are either .ozoraw files recorded onboard the camera, or .mov files recorded using a

Blackmagic Hyperdeck Studio Pro 2 or Blackmagic Shuttle

DPX are the per-sensor DPX files exported from OZO Creator. These can additionally be de-noised or

color-matched using 3rd party tools, prior to importing back into OZO Creator

EXR are the per-sensor OpenEXR files exported from OZO Creator. These can additionally be de-

noised or color-matched using 3rd party tools, prior to importing back into OZO Creator

Stitched files are Left and/or Right eye DPX or EXR files that have been stitched into

spherical/latlong/equirectangular format panoramas from the raw captures or per-sensor DPX or EXR

files.

Please note that to be imported into Creator, these stitched files must conform to the filename

structure “*[left/right]*[%07d].[dpx/exr]“. So frame 1 of the stitched stereoscopic sequence might

be “myProject_left_0000001.dpx” and “myProject_right_0000001.dpx”, for example. The left and right

eye files may reside in separate folders.

6

When importing stitched files, OpenEXR format Depth Maps generated by OZO Creator may also be

imported. These depth maps will be “linked” to the stitched files that are imported concurrently.

These can be used together with the stitched files to export MP4VR files with both stitched and depth

map video tracks.

EXB files are .ozoexb files recorded onboard the OZO+ camera. These 31-frame-long files are

generated when using the Exposure Bracketing feature in OZO Remote, and are meant for the

purpose of creating of high-dynamic-range (HDR) plates.

Audio files are multi-channel interleaved files either exported directly from OZO Creator, or multi-

channel mixes exported from sound mixing programs such as Pro Tools or Steinberg Nuendo. Please

note that these WAV files must conform to a particular channel order specification for proper spatial

audio reproduction (discussed in the Sound Mixing portion of this document) and must be 48KHz 24-

bit PCM files. The supported number of channels for imported audio include 4.0, 5.0, 7.0 and 8.0.

Creator also supports the export/import of first-order AmbiX Ambisonic audio with a W,Y,Z,X channel

order.

When more than one type of asset is imported in a project, a list of abbreviations is displayed at the top of the

Stage that correspond to those assets. Just click on an abbreviation to access the list of corresponding assets

currently imported into the project.

The Stage with imported Captures (CAP), DPX files (DPX), EXR files (EXR), Stitched files (STI), and WAV files (WAV).

Importing Raw Captures The first step after capturing imagery is to follow best practices and make backup copies of the captures.

After backups have been made, start Nokia’s OZO Creator software, and import captures into the program by

either dragging and dropping them onto the Stage pane of Creator, or by clicking the “+” button in the Stage

pane or navigating to Stage in the main menu, and then selecting Import Capture. Then select the capture(s)

that need(s) to be imported.

7

Close up of the “Stage” pane located at the upper-left of the Creator UI

At this point, the full raw captures can be played back using the transport controls. An individual capture is

selected in the Stage when it is highlighted and UI elements in the interface are blue, and the “[N] Asset(s)

Selected” message is shown next to the Export button.

Stage focus Creator UI

If the UI elements in the interface are green, then the Composition is active, and the transport controls play

back any content that is loaded on the timeline. A green “Composition Selected” message will also be

displayed next to the Export button.

8

Composition focus Creator UI

To edit and play back edited footage when in the Composition focus mode, drag-and-drop material from the

Stage to the Composition timeline (alternatively, control + click (Mac) or right click (Windows) on an asset and

select Add to Composition).

Frames and Timecode OZO Creator can display either timecode or frames in the UI. The UI can be switched between the two using the

following methods.

Click on the Duration heading above the timecode/frame duration readout to switch between

displaying frames and timecode.

Ctrl + click (Mac) or right-click (Windows) on either the Current position or Duration readout to bring

up a contextual menu that allows selection between Timecode and Frames.

Navigate to OZO Creator>Preferences and click the appropriate radio button to select either

Timecode or Frames.

The Current Position, Duration, Mark In, and Mark Out fields allow for direct number entry as well. This allows

for typing or copy-and-pasting values into the fields.

With Frames selected, click to place a cursor at the mouse pointer location, or double-click to select the

entire field.

With Timecode selected, click to place a cursor at the mouse pointer location, double-click to select the

two-digit field at the mouse location, or triple-click to select the entire field.

9

Viewing Metadata and Mastering Parameters

Raw Capture Metadata Tab Double-clicking an asset or single-clicking on the “tag” icon of a raw capture in the Stage will open up the

Mastering/Metadata tab.

If Metadata is selected, information about the selected capture can be viewed:

Camera ID

Date of capture

Time of capture (UTC)

Duration of capture

File size

Exposure setting

Illuminant setting

The metadata tab of a raw capture.

Raw Capture Mastering Parameters In the Mastering tab, a number of controls are available to adjust the results of exported imagery and audio.

Please note that adjustments to these parameters affects nearly all exported imagery and audio.

When an asset’s “tag” icon is highlighted, it indicates that some of the Mastering parameters have been

adjusted from their default values. To reset all the Mastering adjustments on an asset, the “Reset Mastering”

menu option can be selected from the asset’s context menu. Additionally, individual Mastering groups (Colors,

Tones, Denoise, Sharpening, etc.) can be toggled on and off to compare before/after changes to default

values. Please note that these toggles also affect exported media. Meaning that if a group is turned off, the

group’s parameters will have no effect on the output.

From a workflow perspective, it is highly recommended to set the Audio, Rotation, and confirm the Illuminant

selection in the Colors mastering section on all newly-imported per-captures as the very first step in the

process.

10

The Audio section allows for adjustment of a capture’s audio gain.

The audio volume can be changed by -/+20dB by adjusting the Gain. The waveform in the Creator UI

will animate in conjunction with adjustments to this control, and peaks in the waveform will turn red to

indicate audio clipping if the Gain is raised too much. Pressing the Auto (“A”) button will analyze the

audio of the capture and automatically maximize the Gain value without clipping.

The Rotation section contains only a single parameter for rotating footage.

The Rotate Asset 180° selection will rotate the imagery and audio of a capture by 180 degrees to

accommodate an underslung (upside-down) mounting of the OZO+ camera.

The Colors section contains options for adjusting the color conversion applied to the raw captures:

If needed, the Illuminant setting of the capture can be overridden by de-selecting the As Captured

checkbox and then adjusting the Illuminant value using the slider. While different Illuminant settings

visually change the color temperature, it is not a standard color temperature control. The Illuminant

setting will provide for the most accurate color reproduction when correctly matched to the specific

spectrum of lighting used in the scene. If fine color adjustments are still needed when the correct

Illuminant is selected, those adjustments can be applied using the Temperature, Tint and Tones

controls, discussed later in this documentation.

Import CSV file is used to import a file containing spectral data that is captured by taking a

measurement using a high-end color meter (e.g. Sekonic C-7000). Once imported into Creator, this

data will be constructed into a “custom” Illuminant which will be applied in place of the standard

Illuminant settings. This can potentially provide more accurate color reproduction when multiple lights

with different spectra (e.g. fluorescent, LED, sunlight) are simultaneously used to light a scene. Example

CSV files with this information in the proper format can be found in:

o For Windows systems: C:\Program Files\Nokia OZO\OZO Creator\Resources\Illuminants.

o For Mac systems:

Using Finder, navigate to:

~ /Applications/OZOCreator.app/Contents/Resources/Illuminants. Please note that

to access the Contents folder, either right click or Ctrl + click on the OZO Creator app

and select Show Package Contents.

Copy the.csv files within the folder.

Create an Illuminants directory on the Desktop and copy the files to this new directory.

Navigate to this folder in Creator and select the .csv file.

Temperature sets the white balance to a custom color temperature. Decrease Temperature to correct

a video taken with a lower color temperature of light; this setting makes the image colors bluer to

compensate for the lower color temperature (yellowish) of the ambient light. Conversely, increase

Temperature to correct a video taken with a higher color temperature of light; the image colors become

warmer (yellowish) to compensate for the higher color temperature (bluish) of the ambient light.

11

Tint sets the white balance to compensate for a green or magenta tint. Decrease Tint to add green to

the image; increase Tint to add magenta.

The Saturation control allows for the saturation of the image to be adjusted from -100 (fully

desaturated) to +50 (increasingly saturated). This control can also be used to help counteract the

desaturating effect of the Color denoise filter. This parameter defaults to a value of 0 (no adjustment).

In a typical workflow, the following Tones, Denoise, and Sharpening parameters can be adjusted later, after the

editing process has narrowed the field down to only the captures needed for the final piece.

The Color Wheels section contains controls to adjust the saturation and luminance of the Shadows, Midtones

and Highlights. These controls are discussed in more detail in the Using the Color Wheel controls section of

this documentation.

Each section contains the following controls:

A Color Wheel

Saturation

Luminance

Luminance Range

The Tones section contains controls for adjusting the levels of each of the red, green, and blue channels of the

imagery on both a Global and Per-sensor basis. These controls are discussed in more detail in the Using the

Tones controls section of this documentation.

Controls include:

White level

Gamma

Black level

Highlight recovery

The Denoise section contains controls for three filters that can improve image quality by removing different

kinds of noise from the imagery. These filters are discussed in more detail in the Using Denoise Parameters to

improve imagery section of this documentation.

Available filters include:

Temporal

Spatial

Color

12

The Clarity filter allows the user to to increase mid-frequency contrast within a specified luminance range of

the image such that images become clearer, potentially reducing any unwanted “haze.”

Available filters include:

Amount

Radius

Luminance change limit

Range

The Sharpening section contains controls for a sharpening filter that can be applied to imagery. These controls

are discussed in more detail in the Using the Sharpening controls section of this documentation.

Controls include:

Amount

Radius

Masking

DPX Metadata/Mastering Tab The Metadata tab for Fisheye DPX and Stitched DPX assets provides metadata about the camera, exposure

and Illuminant values, etc. of the imported footage.

In the Mastering tab, a Rotate Asset 180° control is provided to accommodate an underslung (upside-down)

mounting of the camera.

OpenEXR Metadata/Mastering Tab The metadata tab for Fisheye OpenEXR and Stitched OpenEXR assets provides all the information that the

DPX Metadata tab provides.



Additionally, two controls in the OpenEXR section are provided for the conversion of these files. Please note

that adjustments to these values will affect the appearance of exported imagery.

The Scene referenced check-box enables a conversion from scene referenced gamma (gamma 1.0) to

monitor gamma (approx. 2.2 gamma). Unchecking this box makes the EXR files become quite dense

and contrasty. It is recommended that this is left checked.

The Max value slider adjusts the clip point of the input floating point values when reading the EXR files.

By default, this is set to 1.0. This setting may be useful to adjust if color correction using a floating

point value other than 1.0 as “white” is applied to EXR files prior to re-importing them. However, under

typical conditions, leave this set to 1.0.

13

Stitched Metadata/Mastering Tab The Metadata tab for Stitched DPX and EXR assets provides more limited metadata than other assets,

primarily including the date and time of the stitch process, the duration, and the file size of the imported

footage.



EXB Metadata/Mastering Tab The Metadata tab for EXB captures provides the same metadata as with raw captures.

In the Mastering tab, the same set of controls are provided as with raw captures, with the exception of Audio,

as EXB files are recorded without sound.

WAV Metadata/Mastering Tab The Metadata tab for WAV files contains information about the imported audio file, such as duration, number of

channels, etc.

In the Mastering tab, a Gain control is provided for adjusting the gain of the selected WAV file.

Overview of Media Exports A number of different media formats can be exported from OZO Creator. Simply select an asset or multiple

assets in the Stage and press the Export button to open up the Export Asset window.

The Format menu will be populated with different options, depending on the format of the selected Asset in

the Stage.

Fisheye DPX files are 10-bit rec709 color space files. These are 16.8 MB per frame, or about

234GB/minute.

Fisheye OpenEXR files are 16-bit floating point linear gamma (scene referenced) files. These will match

the DPX files in color and contrast when a rec709 color space is applied to them. There are two

advantages of OpenEXR over DPX. The first is that EXR is a 16 bit floating point format versus DPX’s 10

bit integer format. And the second is that EXR files have lossless compression, whereas DPX files have

no compression and thus take up more drive space. However, there is no significant dynamic range,

color resolution, or image quality advantage over DPX. EXR files do take longer to export due to the

processing time associated with compressing them. The size varies due to the content, but they tend

to be around 9 – 14 MB per frame, or 126 – 197GB/minute.

14

Fisheye CinemaDNG files are raw sensor data files. These will need to be opened and converted in a

program such as Adobe Premiere Pro or DaVinci Resolve to de-mosaic/de-Bayer the raw sensor data

and apply a color space conversion. So DNG files add some complication to the workflow as compared

to DPX or OpenEXR. However, the main advantages of CinemaDNG over those two formats are that

these files tend to be smaller, and they provide considerably more control over dynamic range and

color to the user. The size is approximately 8.4MB/frame, or 118 GB/minute.

Fisheye High Bitrate MP4s are YUV 4:2:0 rec709 color space h.264-encoded MP4 files. This is the

format of choice if stitching with the Jaunt cloud. As these are compressed files, the bitrate and size

can vary considerably, depending on the detail captured by the sensor. Typical values range from

70Mbps – 180Mbps per MP4 file, or approximately 3.6 – 10.3GB/minute.

Per-Sensor Equirectangular DPX are spherical/latlong/equirectangular versions of each fisheye image

(per-sensor) in DPX Format. Files are 10-bit rec709 color space files.

Per-Sensor Equirectangular EXR are spherical/latlong/equirectangular versions of each fisheye image

(per-sensor) in EXR Format. Files are 16-bit floating point linear gamma.

Stitched DPX will stitch a 360-degree panorama and output a latlong/spherical/equirectangular DPX

image per frame. These have the same color space/gamma/bit-depth as the Fisheye DPX.

Stitched OpenEXR will stitch a 360-degree panorama and output a latlong/spherical/equirectangular

EXR image per frame. These have the same color space/gamma/bit-depth as the Fisheye OpenEXR.

Stitched MP4 creates an h.264-encoded file for play back using many applications or platforms, such as

YouTube, Facebook, or Samsung Gear VR. A common example would be to encode a MP4 at UHD

resolution with top-bottom packing for use on a Samsung Gear VR with the Oculus Video app. Note

that this selection has different audio options than MP4VR.

Stitched MP4VR creates an h.264-encoded file that is primarily intended for play back using the

applications that incorporate the Nokia Player SDK as it adds support for 2 channel spatial OZO audio

and encoded depth map video tracks. This format can be played back with any applications that play

back standard MP4 files. However please note that the audio will not be properly spatialized if played

back in an application that does not utilize the Nokia Player SDK.

Depth Maps OpenEXR will export Left and Right eye OpenEXR format “stitched”

latlong/equirectangular files that contain scene depth information in the depth.Z layer. These images

utilize a scale of 1 cm to 1.0 floating point value and correlate to stitched imagery that is exported from

OZO Creator using the same settings.

360 Editorial MOV is an h.264 encoded MOV file created by taking the side left (cam1) and side right

(cam4) images, applying a fisheye-to-spherical/latlong transformation on them, and then simply placing

them side-by-side in a single frame. It also contains a SMPTE timecode track and a stereo downmix of

the audio. The data footprint typically lands in the 15 to 40 MB per minute range.

15

Processed Audio WAV converts the native 360° OZO audio format to a “ring” of loudspeaker channels.

These are exported as interleaved 48KHz 24-bit PCM WAV files with either 4.0, 5.0, 7.0, or 8.0 channels.

Note that the “elevation” component of the captured audio is lost with this output.

Ambisonic Audio WAV converts the native OZO audio format to the broadly-compatible first-order

AmbiX ambisonic format with a W,Y,Z,X channel order. This provides for a full 360° of spatial audio, but

with lower spatial precision as compared to the native OZO audio format.

OZO Audio Interchange outputs the native OZO audio format as an 8.0 channel WAV file with additional

metadata. This can be used to import lossless OZO audio into Steinberg Nuendo and achieve full 360°

audio playback with maximum spatial precision.

OZO Creator Fast-Turnaround Workflow

Example Outline There are a number of different possible workflows for the post-production of OZO+ footage, each with

different advantages and disadvantages, and utilizing a wide array of 3rd party tools. It is unfortunately not

possible to cover all permutations in this document.

However, the following step-by-step outline that concentrates primarily on the use of OZO Creator for a “fast-

turnaround” use case may help in understanding the optimal workflow for a given project. This is just one

possible suggestion for a workflow. So deviate as needed to accommodate different production decisions,

schedules, availability of different 3rd party tools, etc.

1. Ingest raw captures into OZO Creator

2. Under Audio in the Mastering tab, press the Auto button to set the maximum Gain level on the

selected captures

3. If the camera was underslung on any shots, enable Rotate Asset 180° under Rotation for those assets

4. Identify the desired takes, and delete the unwanted takes from the Stage

5. Select all remaining takes, export 360 Editorial files

6. Import 360 Editorial files into the 3rd party non-linear editor (NLE) of choice and edit the footage

7. Re-open the project in OZO Creator. Use those editorial decisions to manually trim each of the desired

takes in OZO Creator to the needed length.

a. Only one range can be selected at a time on a given asset in the Stage. If multiple selects from

a single take are needed, the same take will need to be imported into OZO Creator multiple

times to allow for a given range to be individually selected on each imported version of the

asset.

8. Set the Mastering parameters for each of the trimmed takes, periodically saving the project in OZO

Creator

a. First, adjust the Illuminant to the setting that best represents the actual lighting in the scene

(or import an associated CSV file).

b. Adjust the Temperature and Tint to arrive at the desired overall color.Adjust Highlight

recovery and Shadows recovery.

16

c. Adjust Saturation.

d. Use the Color Wheels to adjust the color, saturation, and luminance in the Highlights,

Midtones, and/or Shadows if required.

e. Ensure Spherical View is selected, drag view to the right or left to expose the Back seam, and

then match the Per-sensor color if needed.

f. Ideally using a head-mounted display (or “HMD,” e.g. Oculus Rift), set the Global tones to

achieve the desired look.

g. Iterate between steps b. through g. as needed until the desired color is achieved.

h. Move the playhead several frames into the timeline, adjust the Temporal filter if needed.

i. Adjust the Spatial filter if needed.

j. Adjust the Color filter if needed.

k. Ideally using an HMD, set the Clarity parameters to taste.

l. Ideally using an HMD, set the Sharpening parameters to taste.

9. Adjust the seam locations taking into account the width and position of the seams. Note: If "3D

Panorama (monoscopic poles)" is selected in the export options. the position and widths of the

horizontal seams are used to control the falloff from stereo to mono at the zenith and nadir.

10. Select all raw captures, press Export

a. Select Stitched MP4

b. Choose the appropriate options to generate the desired format (often based on the specs for

the intended distribution platform)

c. Queue exports and start Job Queue

11. Import the stitched footage into the NLE and conform to the offline edit

12. Render from the NLE to generate mezzanine/transcode masters or direct-deliverable formats.

17

Export Equirectangular DPX/EXR Files The Per-sensor Equirectangular Export options will create per-sensor latlong/spherical/equirectangular/layers

from the fisheye DPX/EXR files. Since these files are output prior to the stitching process, these files can be

useful when manually fixing any artifacts created during automated stitching.

Equirectangular Side Left/Cam 1 image Equirectangular Front Left/Cam 2 image

Equirectangular Front Right/Cam 3 image Equirectangular Side Right/Cam4 image

Equirectangular Top Left/Cam 5 image Equirectangular Top Right/Cam 6 image

Equirectangular Bottom Left/Cam 7 image Equirectangular Bottom Right/Cam 8 image

18

Export Editorial Previews (360 Editorial) If an external Non Linear Editing (NLE) system is being used for editing, such as Final Cut Pro, Avid Media

Composer, or Adobe Premiere Pro, the first step is to export 360 Editorial files from OZO Creator.

The 360 Editorial is an h.264 encoded MOV file created by taking the side left (cam1) and side right (cam4)

images, applying a fisheye-to-spherical/latlong transformation on them, and then simply placing them side-by-

side in a single frame.

Due to the physical separation of these two sensors on the camera, a visible seam in the very center of the

frame will be present. This is normal, as this image is not truly “stitched.”

As these are compressed files, the data size can vary considerably, depending on the content. The data

footprint typically lands in the 15 to 65 MB per minute range.

An example of a 360 Editorial with burned-in 4 channel loudspeaker angles and timecode

A SMPTE timecode track is encoded into the 360 Editorial. The timecode can optionally be burned into the

picture at the upper-center of the image by enabling the Show timestamp option.

Additionally, an overlay for the intended audio mix (4.0, 5.0, 7.0, 8.0) can be optionally burned in using the Show

speaker layout option. This way, the sound mixer has a visual reference for producing a 360-degree positional

audio mix from watching 2D playback.

The 360 Editorial is exported with a 2-track stereo mixdown of the recorded audio to use as a sync reference.

19

To export a 360 Editorial from a full capture, either control + click (Mac) or right click (Windows) on a capture

in the stage or navigate to Create in the main menu, and then choose Export from the menu. Or simply click

the Export button in the UI. This will open the Export Asset window.

Exporting a 360 Editorial in the Export Asset window

By default, the In and Out point selection is set to the length of the full asset. But if these selections were

adjusted, the Range value can be changed to All to easily export the full length of the asset. The Single frame

(Playhead) selection will export only the single frame located at the current playback cursor/playhead location.

It is highly recommended that the full length of the assets is exported to 360 Editorials first to do the offline

edit. Then use those offline edit decisions to determine the takes and selects that are to be used for the online

edit. If the 360 Editorials are trimmed before exporting, it will be difficult to accurately reference the edits back

to the original, un-trimmed raw captures.

There are a few options for naming the exported asset under the File Naming menu.

Use asset name will use the exact name of the raw capture + “editorial.mov” (e.g.

“Capture0001_editorial.mov”)

Use camera ID and time of capture (UTC) will use the name of the camera that captured the footage,

along with the Coordinated Universal Time (UTC) date and time that the capture was initiated (e.g.

“20160221_151007_ PC01001071”). In most circumstances, this will match the output of the Use

asset name option if the asset was recorded on the media module in the camera.

20

Custom allows the user to specify the exact name that they want for the file. If batch exporting from

multiple captures, the software will auto-increment a number onto the end of the filename prefix to

avoid overwriting files. (e.g. “360Editorial-1.mov,” “360Editorial-2.mov,” etc.)

Then choose 360 Editorial under the Format drop-down, Browse to a location to choose the Destination

folder to which files will be exported. Select the Resolution, Show speaker layout and Show timestamp

options. Finally, click Queue. The export task is now added to the Job Queue.

Batch Queuing the Export of Multiple Assets Batch queuing the export of multiple jobs to the Job Queue can be a time saver for any part of the workflow

where a number of assets all need to be exported using the same settings.

For example, the raw capture assets in a project typically all need to be exported to 360 Editorial files using

exactly the same export settings. If this is the case, simply shift + click or ctrl + click (Windows) or cmd + click

(Mac) to select the needed assets in the Stage and then click the Export button. Choose the desired settings in

the Export Asset window and click Queue. Export jobs for all of the selected assets will be added to the Job

Queue.

However, please note that stitched exports must be individually selected and exported, otherwise the locations

of seams will not be accurately rendered into the exported file.

The Job Queue The Job Queue provides a way to queue up a list of tasks, and then execute those tasks in one single rendering

pass. This is optimally used to set up a queue of exports during the day, and then execute the processing

overnight, which can be valuable from a workflow efficiency perspective.

Any exports that are set up will only be queued. So these jobs will not begin to process until the Job Queue is

started. To initiate the execution of the queue, click on the Job Queue button at the lower left of the

application.

The Job Queue button located in the lower left of the UI

Pressing this button will bring up the Job Queue window. Clicking Start will start the queue, clicking Clear All

will remove all the jobs, and clicking the “X” button to the right of individual jobs removes those jobs from the

queue.

21

If the queue extends beyond the bottom of the window, scrolling up and down the queue can be done by

clicking and dragging anywhere within the center of the window.

Prior to starting the queue, the Close button will simply close the window. After the queue has been started,

the Close button will turn into a Cancel button and, if pressed, will provide a prompt to cancel the entire queue.

Note that all queued jobs will be lost if cancelling after starting the execution of the queue.

The Import stitched image assets automatically once exporting finishes option will be highlighted if Stitched

DPX or Stitched OpenEXR jobs have been queued. Clicking the check-box will trigger OZO Creator to import the

stitched imagery into the existing project as soon as the Job Queue has been completed. This can be a time

saver if stitching a number of assets, or stitching single frames using different settings from the same asset, in

order to quickly compare and contrast the results in OZO Creator.

Three different jobs queued in the Job Queue window

The Job Queue is saved when the project is saved. So it is good practice to save often when generating a large

queue of jobs.

22

Export Audio Files

Exporting Processed Audio Multi-channel audio can be exported concurrently when exporting the 360 Editorials. These files can be synced

to the 360 Editorial MOV files on a timeline in an NLE such as Premiere Pro or Final Cut Pro, synced in a DAW

such as Pro Tools, or muxed to the MOV files using a program such as ffmpeg.

Select Processed Audio in the Format window of the Export Asset window, and then choose the Range, the

File Naming option, the Destination, and the Selected Speaker Layout. Although there is a selection of

outputs, the typical speaker layout selections are 5.0 or 7.0.

Exporting a 5 channel 48KHz 24 bit PCM WAV File from a raw capture in the Stage

As long as the audio is exported with the same In and Out point selection Range as the 360 Editorial, the

exported audio will be in perfect sync with the 360 Editorial file.

The file size for audio will depend on the number of output channels selected:

4 channels - 23 MB/minute

5 channels - 28.8 MB/minute



23

Despite the coincidence that the camera has eight microphones, and that Creator provides an option to export

an eight channel audio file, please note that there is actually no direct 1:1 mapping of microphone signals to

exported audio tracks with this selection.

OZO Creator applies a conversion from mic signals to standard WAV audio when exporting Processed

Audio. And as part of this process there is a loss of elevation (i.e. the height, or “Z” component) in the exported

audio. So exporting Processed Audio is essentially a conversion from a "sphere" of captured mic signals to a

"ring" of loudspeaker format audio.

Exporting Ambisonic Audio Ambisonic audio can also be exported from raw captures using OZO Creator.

The ambisonic audio format is exported as a first-order AmbiX WAV file with channel order W,Y,Z,X. This

format provides for less spatial precision than a typical 5.0 channel Processed Audio export. But the ambisonic

format has the advantage of providing a full 360° of spatial audio, including the elevation component that the

Processed Audio exports lack.

Exporting OZO Audio The OZO Audio Interchange format is only supported by the Steinberg Nuendo DAW software. But if mixing

using Nuendo, this export is the best option as it will provide for a full 360° of spatial audio with full spatial

precision.

Export Per-sensor Files from Editorial Selects Once the offline editorial process using the 360 Editorial proxy files is completed, the next step is to trim and

export the full resolution imagery. If a 3rd party stitching tool will be used, and/or 3rd party tools will be used to

color-match, denoise, or apply any other processes prior to stitching, then export per-sensor files to begin the

online conform and finishing process.

An example of an exported DPX frame. The angled horizon is normal.

24

Referencing the burned-in timecode in the 360 Editorial or the information in an Edit Decision List (EDL)

generated by a Non Linear Editor (NLE), manually trim the asset of the corresponding raw capture to match.

Note that OZO Creator does not support the input of EDL files for automated batch trimming and exporting.

To trim the capture, the timecodes listed in the EDL/XML/AAF file have to be manually entered.

Marking In/Out Points in Captures There are a few methods for exporting a section of a capture. First, start by selecting an asset in the Stage.

Set the selection in and out points by using the mouse to position the playback cursor and click Mark In,

and then reposition the cursor and click Mark Out.

Click on the timecode/frames value for each of the Mark In and Mark Out and type in or paste the

desired values.

Adjust the triangular selection handles that hang just below the timeline to the desired in and out

points.

Once the asset has been trimmed on the timeline, and all Mastering controls are appropriately set, open the

Export Asset window and select either Fisheye DPX, Fisheye OpenEXR, Fisheye Cinema DNG, or Fisheye High

Bitrate MP4s from the Format drop-down menu.

These export selections will output an image sequence or compressed file for each of the eight sensors on the

camera.

Once the output image format is chosen, select a Destination, and then choose the File Naming for the folder

into which the exported files will be written. Each format exports to a slightly different folder structure, but the

frames ultimately end up written to their respective camera’s directories, e.g.:

/cam1/20161017_105500_PC01001093_cam1_0000000.dpx

/cam2/20161017_105500_PC01001093_cam2_0000000.dpx

etc.

Frame Number Offset For all DPX, DNG, and EXR (i.e. image sequence) exports, a Frame number offset option is provided. This

provides the ability to set the start frame of the output to any integer value from 0 through 999,999.

This option is not available for audio or MOV/MP4/MP4VR (i.e. encoded) exports.

25

Exported Metadata Information Additionally, OZO Creator will make another folder, labeled with the term “metadata” in some fashion, which is

located inside the named export folder. There will be two files inside of this folder.

The first file is a “metadata” file that Creator and other Nokia software will use to properly de-warp the fisheye

lens distortion of the exported image files. This is a binary file and can only be read by Nokia software.

The second file is a human-readable “.txt“ file that contains a number of parameters that can be used in

compositing software, such as Nuke, to de-warp the individual fisheyes. These metadata parameters are

explained in the VFX section of this documentation.

A third, optional file is a text file labeled [name of export]_motionmetadata.txt which contains the gyroscopic

metadata from the camera recorded during the capture. This metadata may be helpful in stabilizing the

footage in a post process. This file is only generated when exporting either Fisheye DPX, EXR, or DNG files in

conjunction with selecting the option Include motion metadata with export. This file provides comma-

separated values for the frame number, roll, pitch, and heading (yaw), as well as a timestamp.

Editorial Decision-making Using OZO Creator OZO Creator provides basic review and editing functionality for a simple edit of footage from the camera (i.e.

no fades/dissolves or import of external media such as titles), or for the selection of “hero” takes prior to

exporting footage.

Selecting “Hero” Takes Once a number of captures are imported into the Stage pane of OZO Creator, simply click on one capture after

another and play them back using the transport controls. This allows a quick selection of the captures that

contain the take(s) that are to be used in the edit, and an elimination of unwanted captures.

Basic Editing using Compositions Please note that assets cannot be mixed-and-matched on the timeline (e.g. a raw capture cannot be put on the

same Composition timeline as fisheye DPX files, or stitched DPX files).

Use the steps below to assemble a rough edit:

First, use the Mark In and Mark Out functionality to isolate (“trim”) the desired portion of all needed

captures in the Stage.

Drag the first selected asset from the Stage to the timeline. This will put the UI into Composition

focus mode (green UI elements).

Continue to drag “trimmed” captures from the Stage to the timeline, and drag them around on the

timeline to assemble them.

26

If an asset requires further trimming, use the mouse to position the playback cursor on the timeline to

the start point of the selection, and click Trim In. Then set the timeline cursor to the end point of the

selection, and click Trim Out. This can also be done by clicking and dragging on the head or tail of the

green “block” that represents the asset. Click and drag on the middle of the green asset “block” to

reposition the clip on the timeline.

If multiple ranges from a single capture have to be used, pull the same capture back onto the timeline

after the first trim action, and trim the new “pull” of the capture to the additional selection. It is not

possible to make multiple selections from a single instance of a capture on the timeline.

The original “source” timecodes of these selections will not be readily seen after trimming, as the timecode

display in OZO Creator only indicates the “timeline” or “output” timecode and not the “source” timecode of

each segment on the timeline. However, any exported files do retain the “source” timecode in the metadata of

the file header, regardless of their position on the timeline.

The full length of the edited Composition can be exported by pressing the Export button and choosing All. To

export just a selection, set the In and Out points, press the Export button, and choose Selection (In-Out).

To set the In and Out points, simply click and drag on the two triangular “flags” just below the timeline. These

will “snap-to” the edges of individual cuts on the timeline. This can also be done by positioning the playback

cursor and pressing the Mark In and Mark Out buttons (or typing in a timecode/frame value above the

respective button) to respectively set the In and Out points.

If more precision is needed when editing, click and drag the Zoom control to the right to zoom into the timeline.

The green scroll bar below the timeline can then be used to scroll the view. To see the entirety of a long

timeline, click the Auto zoom (“A”) button to the right of the Zoom slider, or drag the Zoom control to the left

to zoom out.

If assembling a Composition from raw captures, the audio of the raw captures will be trimmed and assembled

along with the video and exported separately if needed. But please note that imported WAV files cannot be

edited on the timeline. If Processed Audio or Ambisonic WAV import is needed, it is recommended to utilize a

single interleaved multi-channel WAV file, edited in a 3rd party program, which syncs to the edited Composition

timeline.

Once the timeline is edited, it can be exported to many supported export formats, depending on the format of

the assets used in the Composition.

Press the Export button (ensure the UI is displaying the green Composition focus mode), and make the

appropriate selections for the playback platform of choice. This is discussed in more detail in the Exporting

Encoded Files section later in this documentation.

27

Exporting Compositions Please note that the naming conventions are a bit different for the Export Composition window as compared

to the Export Asset window, as the timeline of a Composition can be made up of multiple captures.

Use project name will use the name of the current OZO Creator project.

Use project name and current time will combine the name of the OZO Creator project and the time of

the initiated export to create the name of the exported file.

Custom allows the user to specify the exact name that they want for the file.

The Export Composition window set up to export Stitched DPX files

Frame Rate Conversion using OZO Creator The Frame Rate Conversion feature in OZO Creator applies an optical-flow-based motion interpolation process

to generate additional frames in between those frames natively captured by the camera. This allows for the

possibility of producing smoother motion when up-converting to 60fps and outputting a 60fps encoded file.

Alternatively, a slow-motion effect can be created when up-converting to a higher frame rate (e.g. 120fps) but

outputting a file encoded with a lower frame rate (e.g. 30fps). So if the edit calls for a slow motion version of a

shot, this feature can be used to provide a smooth effect, rather than using simple (and juddery) frame-

doubling to slow down the motion.

Frame rate conversions can be applied to raw captures, fisheyes and stitched image sequence sources.

28

There are two parameters available to control this effect. The selection of the Format will determine if one or

both parameters will appear in the Export Asset window:

Interpolated Frame Rate appears when any RGB or YUV image export is selected in the Format menu

(i.e. DNG files cannot be frame rate converted). This setting determines the amount of frames

generated by the interpolation process. When raw captures are chosen as the source for interpolation,

options include 30, 60, 90, 120, 180, and 240. When image sequences such as Stitched DPX or

Stitched OpenEXR are chosen as the source for interpolation, options include None, 2x, 3x, 4x, 6x, and

8x.

Playback Frame Rate only appears when exporting to a stitched and encoded file format (e.g. MP4,

MOV). This parameter defines the playback frame rate of the encoded file. Options include 30 and 60

frames per second. The final effect on the footage is determined by the combination of this parameter

and the Interpolated Frame Rate parameter.

Below are examples of the effect when combining different Interpolated Frame Rate and Playback Frame Rate

parameter values to raw captures and exporting encoded files (e.g. MP4s).

Effect: None, native frame rate and duration:

Interpolated Frame Rate: 30

Playback Frame Rate: 30

Effect: 2x speed, “fast-motion”:



Effect: Smoothed motion, no change to duration:



Effect: Half-speed, “slow-motion”:



If a maximum-quality frame rate conversion is the focus, it may be advantageous to first apply the frame rate

conversion and export image sequences (DPX, EXR), manually address any possible artifacts introduced by the

frame rate conversion process, and then reimport the “touched up” image sequences into OZO Creator before

finally encoding to MP4.

It should be noted that using extremes in the interpolated frame rate and the playback frame rate could lead to

undesired effects. The user is encouraged to become familiar with these features and compare output effects

using different combinations of the interpolated and playback frame rates to verify if the desired output is

achieved.

29

Color Match and Denoise Imagery Once the per-sensor files of the select takes are exported, color match the sensor outputs to one another to

ensure the most seamless stitched presentation.

It has so far proven best to select the sensor output with the best color and contrast as the “hero” camera, and

do a “best light” or “pre-grade” color correction on that output using mostly basic color correction tools. Then

proceed to match the other sensor outputs to the color corrected “hero” camera.

The idea is not to do a “creative” color correction at this point because there are a number of time-consuming

steps that follow, including stitching. And any significant change to the color downstream may require backing

up and re-doing several steps. So it is recommended that a very basic color correction be applied at this point

to give the most flexibility for setting the final “look” after stitching and VFX have been completed.

It is also recommended to apply a denoising process on the individual DPX files, as long as the denoising

process does not significantly soften the image. This will help reduce the visibility of noise and compression

artifacts in the footage, and likely improve the results of any optical-flow-based stitching process. The

Temporal denoise filter provided in OZO Creator will do a good job removing random noise. If a 3rd party

motion-compensated temporal denoising pass is needed, many of our customers and partners have suggested

that they get good de-noising results using the Neat Video noise-reducing plugin.

Another reason that it is best to apply de-noising prior to stitching is because the stitching process will convert

to spherical (also called “latlong” or “equirectangular”) imagery. This results in the visible stretching and

warping of the image towards the “poles” of the sphere (i.e. the top and bottom of the imagery), which has the

effect of stretching out and warping the noise as well. So it will likely be more difficult to remove the noise after

stitching, rather than beforehand.

Using the Tones Controls The tone of exported footage can also be adjusted within OZO Creator using the Tones parameters. This

adjustment is applied after the color conversion using the selected Illuminant parameters in the Colors

mastering section. So first ensure that all Illuminant parameters are set appropriately, before using the RGB

color controls.

It is also recommended to set the Saturation value, located in the Colors mastering section, prior to adjusting

the per-sensor controls.

Please note that it will take a moment after releasing the mouse button for OZO Creator to process the

adjustments made to these parameters. So there will be a short delay between setting a value, and the image

updating with the results of that change.

30

The parameters available to adjust tones include:

White level adjusts the level of the white point in the image. Note that this parameter will not reveal

any additional highlight detail. Use the Highlight Recovery parameter for that function.

Gamma adjusts the level of the midtones in the image.

Black level adjusts the level of the black point of the image.

The Highlight recovery slider controls how much highlight detail is recovered in the brightest (clipped)

portions of the image. A setting of 0 results in no recovery, and a setting of 140 applies maximum

recovery. Values between 140 – 150 additionally darken the highlights of the image to allow for extra

range if post compositing/painting is desired in the highlight areas. Please be aware that any highlight

detail added into the image will not not have full color accuracy. This value can be adjusted if undesired

desaturation effects are seen in the highlights. If this is not a significant concern, then a setting of 140

is recommended to generate maximum dynamic range imagery.

The Shadows recovery slider controls how much shadow detail is recovered in the darkest (crushed)

portions of the image. A setting of 0 results in no recovery, and a setting of 100 applies maximum

recovery.

Shadow/Highlight Recovery Options

These parameters can be adjusted on both a Global (all-sensor) basis and on a Per-sensor basis, with the

exception of Highlight/Shadow recovery, which is always applied globally to all sensors. Simply click the

appropriately-labeled button in the RGB Levels section to select the mode.

These parameters can also be adjusted on each of the red (R), green (G), and blue (B) channels independently,

as well as on an “overall” basis (again, with the exception of Highlight/Shadow recovery). Simply select either

RGB to apply an equal correction to all three channels simultaneously, or click one of the R, G, or B buttons to

select the individual channel to be adjusted.

31

The curve in the displayed graph will animate along with these parameters to help indicate how the adjustments

are affecting the output imagery. Please note that adjustments to these parameters may result in additional

clipping of highlights or crushing of black detail.

Adjusting the tone on a per-sensor basis can be especially helpful to correct for less-than-optimal lighting

conditions (e.g. the sun located near the horizon and glaring directly into one sensor, resulting in “warm” tones

and/or “fogged” black levels).

Per-sensor tone adjustments may best be achieved in Spherical view (View>Spherical View), as this view will

most visibly show tone and color differences between sensors. Additionally, click-and-drag the panorama

slightly to the left or right in Spherical view so that the back seam is visible for matching the color between the

Side L and Side R sensors.

Global tone adjustment, however, is best achieved when the adjustments are reviewed using an HMD. This will

provide a viewing environment closer to the final consumption display, as many HMDs have significantly

different dynamic range capability and color space than traditional monitors. Thus, while it can be slightly

awkward to switch between adjusting the UI on a monitor and reviewing the footage in an HMD, making global

color decisions using an HMD will provide the most accuracy in achieving the desired “look.”

Using the Color Wheel parameters The color wheel parameters in the Mastering tab can be used to independently adjust the color, brightness, and

saturation in the Shadows, Midtones and Highlights.

The Color Wheel controls

32

The Color Wheels controls are as follows:

Color Wheel – The control point in the center of the Color Wheel is used to select the hue and the

intensity of that hue for each of the Shadows, Midtones, and Highlights.

o Positioning the control point at different angles will change the hue to that indicated on the

Color Wheel.

o Placing the control point further away from the center of the wheel increases the intensity of

the hue.

o The angular position of the control point is displayed above the color wheel. This value ranges

from 0° to 360°.

Luminance – allows adjustment to the brightness of the defined luminance range.

Saturation – allows adjustment to the color saturation of the defined luminance range.

Luminance Range – this control is used to set the luminance ranges that define the Shadows,

Midtones, and Highlights.

The user can return to the default settings by clicking the return button .

Using Denoise Parameters to Improve Imagery When exporting imagery from a raw capture, it is encouraged to use OZO Creator’s image processing filters to

improve the quality of the imagery.

The controls for these parameters are located in the Mastering tab of each individual raw capture under the

Denoise heading, and the values are saved on a per-asset basis. This makes it possible to tailor these settings

for all of the different raw captures in a project. These per-asset settings will be saved when the project is

saved.

Please note that it will take a moment after releasing the mouse button for OZO Creator to process the

adjustments made to these parameters. So there will be a short delay between setting a value, and the image

updating with the results of that change.

It is important to note that only one filter, the Temporal denoise filter, is applied to the raw sensor data. As the

DNG files are un-color-converted raw sensor data, the Temporal filter is the only filter whose effects can be

seen in all image exports, including exported DNG files. This is unlike any other available filters, as the remaining

filters are applied after the sensor data has been de-mosaiced/de-Bayered and color converted to RGB space.

So their effects can only be seen in the DPX, EXR, MP4, and MOV files.

33

The Temporal parameters control a filter that removes random noise using information from multiple adjacent

frames:

The Strength parameter adjusts the aggressiveness of the filter. This parameter primarily determines

if noise is removed. Higher values are more aggressive, and lower values are less aggressive. The higher

the value, the more frame-to-frame variation in pixel values the algorithm accepts when removing

noise. Please note if the value is set too high, too much variation may be allowed, and artifacts may

appear in some content. Also note that if the value is too low, the algorithm may interpret the frame-

to-frame random noise as movement, and the filter becomes less effective. This parameter defaults to

a value of 40.

The Framecount parameter adjusts the number of frames used to reduce the noise. This parameter

primarily determines how much noise is removed. Higher values equate to potentially lower noise

levels. So, under typical circumstances, it is recommended to set this value as high as possible. This

parameter defaults to a value of 10.

Please note that the Temporal denoise algorithm uses previous frames to do the filtering. Therefore, it is not

as effective in the first 10 frames of the clip. Move the playhead at least 10 frames into the timeline before

adjusting the filter to see the full effect.

The Spatial parameter controls a spatial noise filter which helps to further reduce the intensity of random

noise. It is strongly recommended to first set the Temporal filter before adjusting the value for this filter, as

the temporal filtering is applied to the imagery first:

The Strength parameter adjusts the aggressiveness of this spatial filter. Higher values will more

aggressively reduce noise, but this comes at the expense of potentially softening some image detail.

This parameter defaults to a value of 20.

The Color parameters control a filter that reduces color/chroma artifacts. These saturated “false-color”

chroma artifacts can often be seen on the edges of high-contrast, high-frequency detail:

Strength adjusts the aggressiveness of the filter. Higher values will more aggressively remove these

artifacts, but may also increasingly desaturate real image content. This parameter defaults to a value of

40.

Radius controls the size of the filter. Larger values will more thoroughly remove “blotchy” lower-

frequency (larger-sized) chroma artifacts. These are often seen in darker areas of the image. Please

note that larger values will also desaturate more and larger regions of real image content. This

parameter defaults to a value of 1.

To more closely inspect the results of these filters when adjusting the values, it is recommended to zoom in on

the fisheye imagery using the Image Zoom control. This is achieved either by scrolling with the mouse wheel

when the cursor is placed over the image, or by clicking on the zoom percentage value to open an Image Zoom

pop-up window, and then adjusting the slider. The maximum zoom value is 300%, which approximates the size

34

of the imagery when stitched and viewed in an HMD. To close the pop-up window, click on the zoom

percentage value again.

The Image Zoom pop-up window

When zoomed in, the image can also be repositioned by clicking and dragging on it.

It is also important to note that, in addition to improving image quality, the proper use of these image

processing filters can also improve the results of the optical flow used for the Fine Stitch (highest quality)

stitching and the Depth Map outputs in OZO Creator. So it is especially recommended to spend the time

optimizing these parameters on each of the raw capture assets in a project, prior to stitching the assets with

OZO Creator.

More in-depth information on the Denoise parameters can also be found in the Tips for Improving Image

Quality on OZO Footage document available in the Learn page on ozo.nokia.com.

Using the Clarity parameters The Clarity filter allows the user to increase mid-frequency contrast within a specified luminance range of the

image such that images become clearer, potentially reducing any unwanted “haze.”

This filter is applied to imagery at the fisheye stage when exporting fisheye DPX, EXR, or MP4 frames.

The Amount parameter adjusts the overall amount of clarity applied to the footage. Higher values add

more contrast to areas in the selected Range. Lower values decrease the contrast to areas in the

selected Range making the image appear softer.

The Radius parameter adjusts the kernel size of the Clarity filter. The scale ranges from 50 to 300, with

a default set to 175. The smaller Radius values primarily affect higher frequencies in the image and so

the effect more closely approximates to a sharpening filter. Larger Radius values additionally affect

lower frequencies, and thus can change the apparent contrast of the image.

The Luminance change limit defines the maximum amount of change possible in the affected portions

of the image. In practical terms, this parameter is used to prevent “crushing” of black areas and

“clipping” of white areas, and crucially helps to reduce visible “overshoots” created by this filter which

appear around the edges of high-contrast detail.

35

The Range parameter defines the luminance range in the image to which the filter will be applied. The

scale ranges from 0 to 100. Shadows fall in the 0 – 25 range, Midtones fall in the 25 – 75 range,

Highlights fall into the 75 – 100 range. The default value is 25 – 75 and thus only affect the Midtones.

Leaving the range set to the Midtones also helps to prevent clipping/crushing, and focuses the effect of

the filter where it has the most impact on the image.

It is strongly recommended to review the results in an HMD to best gauge how these parameters will affect the

experience on the final consumption display. For example, the image displayed in native fisheye geometry on a

2D computer monitor is much different in appearance than when viewing it mapped to a sphere in 3D using an

HMD.

Using the Sharpening Parameters The Sharpening filter is used to provide detail enhancement on OZO+ footage, optimally resulting in a “crisper”

output image with improved apparent resolution.

This filter is applied to imagery at the fisheye stage when exporting fisheye DPX, EXR, or MP4 frames.

Conversely, the sharpening is applied to imagery after stitching when exporting stitched footage.

The Amount parameter adjusts the overall amount of sharpening applied to the footage. Higher values

apply more sharpness, and lower values apply less.

The Radius parameter adjusts the kernel size of the sharpening filter. Larger values will more intensely

sharpen larger details in the image, but can also result in more visible overshoots. Lower values

sharpen only the finest detail in the image, but the generated overshoots tend to be less visible than

with larger Radius values. The optimum Radius selection will depend very heavily on the kind of detail

being sharpened.

The Masking parameter is used to control the effect of an edge mask used to help reduce visible

overshoots and the increased visibility of noise when sharpening details in the image. Lower values will

reduce the effect of the edge mask and allow for more high-contrast edges and low-contrast noise to

be sharpened, which may result in visible overshoots on edges of sharpened content and visible noise

patterns. Higher values will increase the effect of the edge mask and reduce the visibility of overshoots

and sharpened noise. But values significantly higher than the default may reduce the visible effects of

the sharpening filter.

It may be best to first turn up the Amount parameter to 100% and the Radius parameter to 3.0 to very clearly

see the effects of the filter. Then set the Masking parameter to ensure the sharpening on the most crucial

detail in the footage is not being masked out. Once the Masking parameter is set appropriately, adjust the

Amount and Radius parameters to taste.

It is strongly recommended to review the sharpened result in an HMD to best gauge how these parameters will

affect the experience on the final consumption display. For example, the sharpness of an image displayed in

36

native fisheye geometry on a 2D computer monitor is much different in appearance than when viewed mapped

to a sphere in 3D using an HMD.

Comparing the Results of Different Filter Settings on Stitched

Footage One way to quickly compare and contrast multiple different sets of parameters on stitched footage is to select

the Single frame (Playhead) option in the Range menu of the Export Asset window. This selection works best

when also enabling the Import stitched image assets automatically once exporting finishes option available in

the Job Queue window. Please note that this option is only present when selecting Stitched DPX or Stitched

OpenEXR exports.

Adjust the desired Mastering parameters

Export Stitched DPX or EXR using the Single frame (Playhead) option

For easy reference, name the file using a Custom name that reflects the chosen settings

Queue the job

Repeat until all desired variations of Denoise parameters have been queued for rendering.

After opening the Job Queue, but before executing the queue, ensure that the Import stitched image

assets automatically once exporting finishes option is selected

Using this procedure, a number of these jobs can be queued, and then automatically imported into the list of

stitched assets in OZO Creator upon completion of the queue. The resulting single frame versions with

different mastering parameters can then be compared within the Creator UI, or optimally using an HMD.

Stitching Overview Stitching is the process by which a single panoramic image is created by combining imagery from multiple

sensor positions. The desired result is to create a visually seamless transition from one sensor’s perspective to

another, so that the appearance is that of one contiguous image. Stitching OZO+ footage can be done with

OZO Creator, or with a number of 3rd party applications.

After any optional color matching and de-noising has been applied to the per-sensor files, the files proceed to

the stitching process in order to create left and right eye spherical/latlong/equirectangular images for use in

viewing stereoscopic 3D imagery using a head mounted display (HMD), such as a Samsung Gear VR, Oculus Rift,

or HTC Vive.

Expect that any stitching process will create some visible artifacts in the interpolated (“seam”) areas that will

require manual intervention to fix. These artifacts tend to be the most visible on objects that are closest to the

camera, as this is the point at which the parallax difference between sensor angles is the greatest. Large-

parallax imagery provides the greatest challenge for any stitching process.

For further information on optimizing footage for the stitching process, please refer to the Shooting for

Stitching guide in the Learn page on ozo.nokia.com.

37

Stitching imagery using OZO Creator OZO Creator uses computational stitching to create 360° panoramic frames from raw captures and per-sensor

fisheye images.

Importing Fisheye Imagery for Stitching To stitch panorama frames from the per-sensor fisheye images, the per-sensor images must first be imported

into OZO Creator. Either navigate to Stage in the main menu, or click the “+” in the Stage pane. Then select

either Import DPX or Import EXR (depending on the file format of the per-sensor fisheyes).

The DPX folder (or EXR folder) field should be set to point into a folder that contains per-sensor DPX

or OpenEXR frames.

The Metadata file field should be set to point into a folder that contains the metadata for the specific

DPX or EXR frames that are being used as the source (the metadata path is auto-populated when

selecting DPX / EXR files and metadata that conform to the default path and filename output from

Creator).

Then click Import to add these assets to the Stage.

To select these imported DPX or EXR frames for stitching, click on the asset in the Stage.

Note that if there is more than one kind of asset imported into the project, select the appropriate file format at

the top of the Stage pane first (i.e. “DPX” or “EXR”) to access the proper list of assets, and then select the

specific asset to stitch.

Stitching Directly from Raw Captures Stitching can also be done directly from the raw captures. Simply ensure a raw capture is selected in the Stage,

optimize the Mastering parameters, and then proceed with the stitching process.

Setting Seam Positions Once the correct asset is selected, enable Spherical view by clicking the “square” icon in the image viewing

pane, pressing “V,” or by navigating to View>Spherical View from the main menu. This view will provide a

latlong/spherical/equirectangular panorama view using a fast stitch process.

Once in Spherical view, enable the Seam Mode by pressing the Seams icon in the image viewing pane or via the

Seams parameter in the Mastering tab. This will highlight the seam areas with a magenta color. A spot check

can be done on the footage (the speed of the drives will likely not allow for full speed playback, especially if

stitching from DPX or EXR files) and note if there is close proximity detail or movement within seam areas.

Vertical stitching seam area locations and sizes are to some extent adjustable. As the content in seam areas is

interpolated, its quality may not be as good as the image quality in non-interpolated areas. Thus it may be

38

useful to move or resize seams to place them in areas where less important content is located. The seams

must be contained in an area of the 360-degree panorama where two adjacent cameras’ field of view overlap,

and thus cannot be positioned outside of this overlap area.

Horizontal seams are also present, and highlighted in green. These seams are not used for stitching. They are,

however, used to control the falloff from stereo to mono at the zenith and nadir poles of the camera. Like the

vertical seams, the position and width of the horizontal seams can be adjusted to some degree. The changes

will be applied if 3D Panorama (monoscopic poles) is selected in the export options.

To widen or narrow a seam, click and drag on the edges of the seam. To move the seam, click and drag in the

center of the seam. Please be aware that there are limitations on the extent the seams can be repositioned.

The seams can be reset to the original values by clicking the return button in the Seams parameters.

To get out of the Seams mode, simply press the Escape button on the computer’s keyboard.

The default placement of the third seam may lead to stitching artifacts on this particular footage. Please note that the visualization here is not

necessarily an accurate rendition of any artifacts that may be generated.

39

Note how the second seam has been narrowed so that the character’s hand is no longer within the seam area. The third seam has been moved to the

left and narrowed so that the character is no longer in the seam area. This will likely result in a stitch with fewer visible artifacts.

Adjustments made to the seam positions will be saved on a per-asset basis when the project is saved. So any

adjustments made to the locations of the seams here will affect how each of the shots are stitched.

Manually Setting Backseam Convergence It is recommended to adjust the Backseam Convergence parameter in the Mastering tab whilst viewing the

image in Spherical view to find the best setting for the visual convergence of detail in the captured scene.

Increasing this value will reveal closer proximity details (thus reducing the magnitude of the vertical “slice”

through them). But it will potentially cause distant objects to be viewed “doubled.” In other words, the same

distant detail may be seen to the left and right of the seam. So setting the Backseam Convergence parameter

will likely require a compromise between near and far detail reproduction.

Click and drag the mouse cursor in a non-seam area of the image, and pan the image left or right to better

expose the back seam. The image will “wrap around” and show how the left and right edge of the

latlong/spherical/equirectangular image will connect.

40

With the image panned to the right, and Backseam Convergence set to 1.090, notice how the buildings within the blue circled area don’t align

perfectly and appear doubled.

With the Backseam Convergence now set to 1.027, the buildings line up much more accurately.

This setting is saved on a per-asset basis, and will be applied to the back seam if Manual is selected in the

Backseam convg. menu of the Export Asset window when exporting stitched footage.

41

Exporting Stitched Files Once the correct asset is selected, and the optimal seam positions are determined, click the Export button at

the lower left of the UI (Stage focus, blue UI elements). Then select the necessary stitched format.

The Export Asset window

The Range, File naming, File name prefix, Destination, and Frame number offset fields should already be

familiar.

Format selects the output file format:

Stitched DPX will stitch a 360-degree panorama and output a latlong/spherical/equirectangular DPX

image per frame.

Stitched OpenEXR will stitch a 360-degree panorama and output a latlong/spherical/equirectangular

EXR image per frame. These are compressed files, so expect that EXR will take longer to process than

the DPX files, as computation is required to apply the compression.

Stitching Quality has two options:

Fine Stitch (highest quality): Frames are stitched with computational stitching at high quality. This

option should be used to get the highest possible output quality. But this comes with a cost of

rendering speed. The processing time is highly variable and will depend heavily on the content in the

42

seam areas and the speed of the input and output drives. In general, a faster stitching rate can be

achieved with:

o higher speed drives used for input and output

o content with less complexity of motion, especially in the seam areas

o the selection of Stitched DPX instead of Stitched OpenEXR for the export format

Fast stitch (normal quality): Frames are stitched with a computationally simpler process as compared

to the Fine Stitch quality. But the result is that the stitching runs at a much faster rate. Similar to the

Fine Stitch (highest quality) option, a faster stitching rate can be achieved with:

o higher speed drives used for input and output

o the selection of Stitched DPX instead of Stitched OpenEXR for the export format

Horizon stabilization provides the option to automatically stabilize the footage based on the OZO’s orientation

sensors.

Frame type specifies the panorama type:

3D panorama (monoscopic poles) outputs separate left and right eye panoramas with a blend to mono

at the zenith and nadir of the spherical panorama. This is the recommended stereoscopic setting for

most content.

3D panorama outputs separate left and right eye panoramas with stereoscopic poles. Note that this

option typically provides a less comfortable viewing experience than the 3D panorama (monoscopic

poles) option.

2D panorama (Left eye) outputs frames for the left eye only.

2D panorama (Right eye) outputs frames for the right eye only.

Exporting 2D panoramas is a bit faster than 3D panoramas because the other eye output is omitted.

Resolution provides several options for specifying the output resolution of the panoramic images. Please note

that 4K is the highest “native-resolution” output:

8K (8192x4096)*

8K UHD (7680x4320)*

6K (6144x3072)*

4K (4096x2048)

4K UHD (3840x2160)

3K (3072x1536)

HD (1920x1080)

*denotes a selection that is upscaled using an advanced algorithm during the stitching process

43

Automated Backseam Convergence The Export Asset window also includes an option to calculate and apply an automated Backseam convergence

when exporting Stitched DPX/EXR/MP4/MP4VR files.

The Backseam convg. parameter includes the following options:

The Manual option uses the user-specified Backseam convergence value set in the UI.

The Automatic (fixed value) option uses the first frame of the sequence to calculate the optimal

convergence values for a number of points along the vertical back seam. This can better account for

the different convergence distances typically needed to properly converge all the detail at differing

depths along the entirety of the seam. This option typically creates a more “invisible” back seam than

the Manual option does and thus is the recommended option for most static camera shots.

The Automatic (15 frame re-calculation) option works similarly to the Automatic (fixed value) option,

except that it re-calculates and applies the optimal convergence every 15 frames. The re-calculation

is not smoothly “animated” between keyframes. Thus, if there is significant change in the content in

the back seam area, the detail in the seam may visibly “shift” on the frames when the re-calculation is

applied.

Best Practices and Additional Notes for Stitching using OZO Creator Stitching seam areas are interpolated from two or more fisheye images, and at best the stitching seam areas

are not noticeable. But in certain situations various kinds of interpolation artifacts such as ghosting, blurring,

twitching, or garbage pixels may show up.

To ensure the best possible panorama stitching quality, here are some best practices and notes that may be

helpful:

During filming it is best to keep most of the action in non-interpolated areas, if possible. The area right

in front of the camera has a quite large FOV of seamless area. Thus it provides the best opportunity to

stage action close to the camera and still keep subjects out of interpolated seams.

The closer the filmed content is to the camera, the more disturbing the interpolation artifacts are. Slow

moving content at close distances in interpolated areas is perhaps the most "dangerous" combination.

Foreground content in which color is similar to the background may more readily result in interpolation

artifacts than content with high contrast.

Thin items such as electric wires, ropes, poles, etc. in the interpolated areas close to the camera are

very likely to cause ghosting artifacts.

Recommended minimum distance for content in interpolated areas is about 1.5 - 2.0 meters, or 5 - 6

feet.

44

The 3D effect is gradually faded out to 2D at the sides of the camera, around -90 to -120 and 90 to

120 degrees from the camera’s "zero" angle (directly forward).

The location of the seams, other than the back seam, can be manipulated in OZO Creator. Proper

adjustment of the Backseam convergence slider and/or choosing the most appropriate selection of the

Backseam convg. parameter in the Export Asset window will help provide the best visual convergence

of detail to the rear of the camera.

Bright light sources, especially above or below the camera, may cause lens flares or light streaks which

have different shapes and/or locations in different sensors due to the fisheye lens optics. This may

potentially confuse the interpolation algorithm.

In general, image quality & the stereo effect is not optimal directly above and below the camera.

Proper use of the 3D panorama (monoscopic poles) selection and adjustment of the stereoscopic-to-

monoscopic horizontal seams can help to account for this.

The five first and five last frames of the stitching process may typically contain more unwanted artifacts

than subsequent frames. Thus it is suggested to have a few extra frames of handle at the start and end

of the stitching range.

Panorama stitching takes a lot of time. Depending on the computer’s capabilities, hard drive read/write

speed, and the chosen settings, stitching one frame at high quality and high resolution settings may

take a number of seconds.

Other best practices and suggestions can be found in the Shooting for Stitching document available in the

Learn page on ozo.nokia.com.

Stitching Files Using 3rd-Party Services and

Tools

Stitching DPX Files using Deluxe The same proprietary technology developed by Nokia for the automated stereo stitching of footage has been

licensed for use by Deluxe, a full-service, internationally present post-production company. Their services can

be used for stitching OZO+ footage to receive stereoscopic output from color-matched and de-noised per-

sensor files.

Deluxe can additionally handle end-to-end workflow for the post production of footage captured using the

OZO+, including editing, audio mixing, color, de-noising, stitching, and the encoding and packaging of

deliverables.

45

Stitching DPX or EXR Files using The Foundry’s Cara VR tools for

Nuke Nokia has worked with The Foundry to provide native support for stitching OZO+ footage using the Cara VR

toolset within Nuke. This allows for image workflows that fully remain inside Nuke nearly from start to finish.

OZO+ specific procedures for stereoscopically stitching footage using Cara VR include:

In the c_CameraSolver node, be sure to select Nokia OZO in the Preset menu.

Click Setup Rig and then point the browser to the “.txt” file containing the metadata that was exported

concurrently with the DPX/EXR/DNG files from OZO Creator. This will properly set up all of the intrinsic

and extrinsic parameters of the particular camera used to shoot the footage within the

c_CameraSolver node.

If required, use Match and Refine in C_CameraSolver to fine tune the camera parameters.

In the c_Stitcher node, ensure that Enable Stereo Stitch is selected. Under the Stereo tab, the Eye

Separation parameter can be adjusted to control the apparent depth in the scene, and the Falloff Type

can be adjusted to control depth merging at the poles.

Under the Stereo tab of the c_Stitcher node, it may be desired to set the Eye Separation parameter to

a value of 0.086, as that is closer to the true inter-axial distance of each of the lenses on the camera.

3D elements can be integrated in a stereoscopic stitch using the c_RayRenderer node. Set the

Projection Mode to Spherical and select Stereo Scan Enable on the Camera tab. The Eye Separation

and Falloff Type can then be set to match the stitch.

Stitching High Bitrate MP4s using Jaunt Cloud Jaunt’s cloud stitching platform also supports stereoscopic stitching using footage captured using the camera.

This platform accepts the Fisheye High Bitrate MP4 export (not DPX/EXR/DNG files) as the source to stitch, and

ambisonic audio as the compatible spatial audio format.

The Fisheye High Bitrate MP4s are h.264 compressed files. So their size will vary depending on content. But

expect a data footprint in the neighborhood of 2.5GB – 5.5GB per minute of exported High Bitrate MP4

footage.

For further information on stitching using the Jaunt platform, visit https://www.jauntvr.com/camera/ozo/

Stitching DPX Files with other 3rd-Party Software There are also multiple software stitching solutions available for stitching OZO+ footage monoscopically,

including VideoStitch and Kolor Autopano. These are compatible with content captured using the camera.

Please note that these tools cannot create stereoscopically stitched OZO+ footage.

46

VFX After stitching, the images will likely need to go through visual effects and paint to manually clean up and repair

any artifacts from the stitching process.

The artists will likely require the per-sensor color corrected/de-noised/etc. images to use in repairing artifacts

or reconstructing imagery. Providing the artists with the “human readable” metadata information created

during the export of the DPX/EXR/DNG files, as well as the definitions below, should give them the information

they need to take a fisheye image, and create a spherical/latlong/equirectangular image than can be used to

paint/composite into the stitched image. OZO Creator exports the metadata for DPX/EXR/as a text file with the

filename ending in _metadata.txt. This will be found in the metadata folder associated with the export.

Metadata Parameter Definitions Each lens will have the following parameters:

The Center parameter refers to the X and Y coordinates of the center of the lens, measured with

respect to the center of the sensor. Since the sensor is 2048X2048, the two values should both be

close to 1024 (1024 being half of 2048, and thus the value of a perfectly centered lens)

The Focal parameter describes the image circle radius in pixels at 1.0 radians (about 57 degrees) FOV.

Typically Nuke or other compositing programs will use "Degrees" for this value. So it is likely that a

conversion has to be applied to get to an accepted value.

The Distortion parameters (which are named k1 - k4) are always bundled with Focal value. They

describe a polynomial function that approximates lens mapping curve. Four distortion parameters

should in theory give a bit more accuracy than three parameters.

o The distortion parameters use the Open CV 3.0 fish eye model described here:

http://docs.opencv.org/3.0.0/db/d58/group__calib3d__fisheye.html

o Specifically regarding the distortion function, the deviation from f.theta is described by a

polynomial, theta_D = theta ( 1 + k_1 . theta^2 + k_2 . theta^4 + k_3.theta^6 +

k_4.theta^8).

The Position parameter is used to describe the location in 3D space of each of the eight sensors using

the mid-center-right sensor and optical axis as the origin frame of reference.

Alternatively, OZO Creator, Nuke’s Cara VR toolset, and Deluxe can optionally output per-sensor

spherical/latlong/equirectangular images as part of the stitching process (e.g. the Convert input images into

spherical images option in OZO Creator), bypassing the need for VFX artists to create these images

themselves. This is likely the best option from a workflow efficiency perspective.

http://docs.opencv.org/3.0.0/db/d58/group__calib3d__fisheye.html

47

Use the post-stitched footage to composite special effects into the imagery. CGI needs to be composited into

stitched spherical imagery.

Using the OpenEXR Depth Map Export To assist with applying depth-based effects and compositing of CG elements, OZO Creator provides an option

to export depth maps by selecting the Depth Maps OpenEXR export option.

This export option uses multiple vantage points (camera viewpoints) to provide the calculated depth in the

scene for the stereo portion of the image. The depth maps are exported as Left and Right eye OpenEXR

format latlong/equirectangular files, with the depth information contained in the depth.Z layer. These

OpenEXR files containing the depth information are exported separately from the stitched Left and Right eye

panoramas. Please note that this export will take at minimum several seconds per frame, the speed of which will

vary depending primarily on the speed of the computer’s GPUs.

This depth map correlates to Fine Stitch (highest quality) stitched Left and Right eye files that are output with

exactly the same resolution. The Resolution options provided for the Depth Maps OpenEXR export option

correlate to those for the various stitched export formats.

Please note, however, that the back of the camera (i.e. the far left and far right portions of the latlong depth

map images) will not have all of the depth provided. As the back of the camera has only monoscopic viewpoints,

depth cannot be calculated in these areas.

In this depth map, 1 centimeter of depth is equal to a floating point value of 1.0. So the Gain for the viewing

window of a compositing application will need to be reduced quite considerably to see something other than a

pure white image (a gain of approximately 0.0005 should be sufficient). Please note that the depth in the

depth map is truncated beyond a distance of about 22.4 meters, as depth values beyond this distance currently

cannot be accurately calculated.

Using OZOEXB Captures to Create an HDR plate OZO Remote allows for the easy capture of bracketed exposures in a scene using the Exposure Bracketing

functionality. The format of a recording taken with Exposure Bracketing is an OZOEXB file, which is 31 frames

in length, with each frame captured at a different exposure setting.

When exported to standard image formats, this capture enables the creation of a single high-dynamic-range

(HDR) plate using 3rd party tools. This HDR plate can then be composited into a shot which is captured at the

same camera location to provide for imagery with more apparent dynamic range than the camera can capture

natively.

Note that Temporal denoise filtering is disabled on these captures. Since the difference from frame to frame is

so great due to the difference in exposure, this filter would have little practical effect anyway.

48

It is suggested that Highlight Recovery also be left at a value of 0, since the dynamic range of the HDR plate will

be generated from the bracketed exposures. Additionally, adding in significant amounts of Highlight Recovery

may introduce undesired desaturated highlight details into the generated HDR plate.

There are a number of workflow possibilities to extend the dynamic range of a shot using these EXB captures.

If OZO Creator will be used to stitch the shot into which HDR elements will be composited, it may be best to

export the EXB file as Fisheye DPX or EXR frames. Then use a 3rd-party tool to generate a single HDR fisheye

for each viewpoint, as long as the HDR result can be made consistent between viewpoints. Finally, import and

stitch those HDR fisheyes into Creator to generate the equirectangular HDR plate.

Alternatively, it is also possible to export the EXB file as Fisheye DPX or EXR frames, re-import those fisheye

frames, and then export them as Stitched DPX or EXR frames. Then, use those Left and/or Right eye stitched

frames as the source to generate the HDR plate(s). Be aware that the Fine Stitch (highest quality) stitching

may not provide optimal, temporally-consistent results on some content in the extreme exposure ranges that

contain significant white clipping or black crushing. So with this workflow, manual cleanup may be needed on

either the stitched exposure brackets prior to HDR generation, or on the final HDR 360° panorama itself.

If a 3rd party stitching solution will be used, then it is recommended to start by exporting Fisheye DPX or EXR

frames from the EXB file and then run those frames through the 3rd party stitching and HDR-generating tool

pipeline. This way, the alignment of the images will be optimal as the HDR plate will be stitched using the same

tool used to stitch the scene into which it will be composited. Depending on the stitching and HDR-generating

tools available, it may be simplest to stitch first, and then generate the HDR plate from the stitched files.

Otherwise, generate HDR versions of each of the fisheye viewpoints, and then stitch the result to generate the

equirectangular HDR plate. Experiment to see what works best for the combination of tools available.

It is also possible that, rather than combining all of the exposures into a single HDR plate to use as a composite

source, each alternate-exposure can be stitched and then used individually as a composite source for different

elements of the image. This is another perfectly valid way to extend the dynamic range of the final imagery.

Final Color Final color on the stitched images can typically be started in parallel with the VFX work. This is an industry-

standard full color correction pass to set the “look” of the show.

The left and right eyes are both made up of image information from different portions of the four equatorial

sensors on the camera. Thus there shouldn’t be a need to do too much color balancing between the eyes, as

long as the color matching pass on the per-sensor fisheye imagery was precisely done.

One significant thing to take into consideration is the multitude of HMDs that will be used to view this material.

It is recommended to output color bars and profile the intended display (Samsung phones, Oculus Rift, HTC

Vive, etc.) as much as possible prior to starting the color correction pass, so that colorists have a better sense

of the target they need to reach.

49

While most phones and other displays profess to use a standard Rec709 or sRGB gamut, a custom viewing look

up table (LUT) may be needed to best match the color correction environment to the consumption display.

Alternatively, a color space that best represents the color and contrast of the HMD may be needed. Phones in

particular (the Samsung Galaxy S6 and S7 being popular models for viewing VR content using the Gear VR

system) may have a much bluer white point than is standard for television monitors.

Titles and Credits Titles and credits should, under normal circumstances, be made small enough so that the viewer does not have

to look around in the HMD to view them. The FOV in most HMDs is quite narrow (usually between 96 and 110

degrees). So appropriately-sized credits will look quite small on the stitched latlong left and right eyes when

viewing on a monitor.

It is good practice to present multiple copies of the credits which are spaced out from one another at 90 or 120

degrees so that the viewer will see one of the copies, regardless of the direction they are looking when the

credits begin.

The credits can also be scrolled vertically or horizontally. It is generally not recommended to do this over

complete black, as it might induce slight motion sickness in the viewer. In the case of scrolling credits, it might

be better to “anchor” the viewer with some kind of static background over which the credits move.

Credits can also be placed in 3D space. But if the credits are over black, then a monoscopic presentation is

most often used.

Exporting Encoded files It is possible to export 360 Editorial or MP4 “review” versions of the footage as work progresses, or even export

directly to a stitched MP4 output from the raw captures as part of a “fast-turnaround” workflow. Depending on

the input, certain options may or may not be selectable.

There are two options in the Format menu of the Export Asset window for exporting stitched and encoded

files:

Stitched MP4 – this selection creates an h.264-encoded file for playback using the platform or

application of choice, for example, YouTube, Facebook, or Samsung Gear VR. A common example would

be to encode a MP4 at UHD resolution with top-bottom packing for use on a Samsung Gear VR with the

Oculus Video app. Note that this selection has different audio options than Stitched MP4VR.

Stitched MP4VR – this selection creates an h.264-encoded file that is primarily intended to play back

using applications that incorporate the Nokia Player SDK. This file can additionally be encoded with

depth map video tracks if a stitched image sequence and an associated depth map image sequences

are imported into OZO Creator and used as the source for encoding. However, this format is

50

compatible with any applications that play back standard MP4 files. Note that this selection has

different audio options than Stitched MP4.

Stitching Quality is available only if exporting from non-stitched imagery (captures, fisheye DPX/EXR files), as

OZO Creator will first stitch and then encode/export the footage. These options are the same as with the

Stitched DPX/Stitched OpenEXR functionality:

Fine Stitch (highest quality): Frames are stitched with computational stitching at high quality.

Fast stitch (normal quality): Frames are stitched with a computationally simpler process as compared

to Fine Stitch quality.

Output Quality determines the quality/bitrate of the encoded MP4 file:

Very High provides the best quality and a fairly high bitrate (typically in the range of 96 – 184 Mbps for

UHD, depending on all the export settings). This option takes a bit longer to process than either

Normal or High, and will also require approximately twice as much space as files encoded with the High

setting.

High provides good quality and reasonable bitrate (typically in the range of 48 – 92 Mbps for UHD,

depending on all the export settings). This option provides probably the best balance of quality, speed,

and space, as it produces noticeably better image quality than Normal, but doesn’t take as long to

encode or use as much disk space as Very High.

Normal will take less time to encode and create a smaller file as compared to High (typically in the

range of 6 – 21 Mbps for UHD, depending on all the export settings). Compression artifacts will likely be

visible in Normal encodes.

Backseam convg. Is available only if exporting from non-stitched imagery (captures, fisheye DPX/EXR files).

These options are the same as with the Stitched DPX/Stitched OpenEXR functionality:

Manual utilizes the manually set Backseam convergence parameter.

Automated (fixed value) calculates optimal back seam convergence using the first frame of the

sequence.

Automated (15 frame re-calculation) calculates and applies optimal back seam convergence every 15

frames, starting with the first frame of the sequence.

Profile provides a selection of presets for generating encodes compatible with Nokia SDK-enabled applications

on phones and on computers, as well as allowing for user-defined custom encoding options:

OZO Player Mobile changes the encoding settings to a preset configuration that is readable by mobile

applications, especially those that support the OZO Player SDK. This outputs a stereoscopic MP4 file

which is output with a top (Left eye) – bottom (Right eye) configuration at UHD resolution.

51

OZO Player Desktop changes the encoding settings to a preset configuration that is readable by

desktop computer applications using the OZO Player SDK, such as OZO Preview. This outputs a

stereoscopic MP4 file that contains two full UHD streams, one for each of the Left and Right eyes.

Online 360° 3D changes the encoding settings to a preset configuration that is broadly accepted by

most online distribution platforms. This selection will also reveal the Inject spatial media metadata

checkbox, which if enabled will inject metadata into a copy of the output file. This metadata is needed

for online video platforms to properly identify and play back the video in a 360° format

stereoscopically, and optionally with ambisonic audio. Please ensure that the target online video

platform is compatible with the preset configuration. Any one of these settings can be adjusted to suit

the target online video platform. If changes are made, the profile will change to Custom.

Online 360° 2D changes the encoding settings to a preset configuration that is broadly accepted by

most online distribution platforms. This selection will also reveal the Inject spatial media metadata

checkbox, which if enabled will inject metadata into a copy of the output file. This metadata is needed

for online video platforms to properly identify and play back the video in a 360° format

monooscopically, and optionally with ambisonic audio. Please ensure that the target online video

platform is compatible with the preset configuration. Any one of these settings can be adjusted to suit

the target online video platform. If changes are made, the profile will change to Custom.

Custom allows for a user-defined configuration to encode for the playback platform of choice.

Frame type works essentially the same as it does in the Stitched DPX/Stitched OpenEXR options. This

specifies the panorama type:

3D panorama (monoscopic poles) encodes a file with both left and right eyes, and if encoding from

non-stitched imagery, will stitch with a blend to mono at the zenith and nadir.

3D panorama encodes a file with both left and right eyes, and if encoding from non-stitched imagery,

will retain stereoscopic imagery at the zenith and nadir.

2D panorama (Left eye) encodes a file with the left eye only.

2D panorama (Right eye) encodes a file with the right eye only.

Frame packing determines how the left and right eyes are encoded into the file. This option is only available

when 3D panorama is selected under Frame Type:

Separate encodes the left and right eye into two full-resolution individual streams within the file. This is

the highest-quality option, as it retains full resolution for each of the individual eyes. However, it is less

compatible than Top-bottom. Please ensure that the playback device and playback application can

accommodate this encode.

Top-bottom (sometimes called “over-under”) encodes both the left eye and right eye into a single

frame. The eyes are stacked on top of one another in this frame. The left eye is on top, and the right

eye is on the bottom. Please note that this option reduces the vertical resolution of each of the eyes in

order to pack both into a single frame.

52

Top-bottom (double height) also encodes the left and right eye with a similar arrangement as the Top-

bottom selection. However, please note that this option uses the full vertical resolution of each eye,

and so this selection will double the height of the Resolution options. Additionally, many players are

unable to decode the large resolution frames this option outputs. So this selection is primarily used for

outputting a “master” encoded file that can be used to transcode to multiple playback formats, or for

uploading to platforms such as YouTube that will transcode for playback.

Resolution provides several options for specifying the output resolution of the panoramic images:

When Separate or Top-bottom is selected:

8K (8192x4096)*

8K UHD (7680x4320)*

6K (6144x3072)*

4K (4096x2048)

4K UHD (3840x2160)

3K (3072x1536)

HD (1920x1080)

When Top-bottom (double height) is selected:

8K (8192x8192)*

8K UHD (7680x8640)*

6K (6144x6144)*

4K (4096x4096)

4K UHD (3840x4320)

3K (3072x3072)

HD (1920x2160)

* denotes a selection that is upscaled using an advanced algorithm during the stitching process

Audio provides multiple options for encoding the audio of the MP4 file (please note that these options are

different if Stitched MP4VR is selected):

4.0 will encode four channels of loudspeaker-configuration audio into the MP4 file

5.0 will encode five channels of loudspeaker-configuration audio into the MP4 file

7.0 will encode seven channels of loudspeaker-configuration audio into the MP4 file

8.0 will encode eight channels of loudspeaker-configuration audio into the MP4 file

Mono will encode a single monophonic track of audio into the MP4 file

Ambisonics will encode first-order, four-channel AmbiX ambisonic audio into the MP4 file.

Disabled will disable the encoding of audio altogether and output an MOS MP4 file.

53

If MP4VR is selected, there will only be two Audio options when stitching directly from a raw capture:

OZO Audio 2 channels + meta will encode spatial audio that is compatible with the Nokia Player SDK.

Applications that do not use this SDK will not be able to properly play back this audio. This option is

only available when stitching to MP4VR directly from a raw capture.

Disabled will disable the encoding of audio altogether and output an MOS MP4VR file.

Further information on how to generate MP4 files for different platforms can be found on the Learn page on

ozo.nokia.com:

YouTube-compatible MP4 files, please refer to Exporting stitched MP4s to YouTube

Samsung GearVR-compatible MP4 files, please refer to Sideloading content to Samsung Gear VR.

Facebook-compatible MP4 files, please refer to Exporting stitched MP4s to Facebook

Vimeo-compatible MP4 files, please refer to Exporting MP4s to Vimeo

Encoding MP4VR Files with Depth Maps MP4VR files can be used in conjunction with Nokia’s Player SDK to provide playback of a single file with stitched

video and synced depth information to an application such as Unity. This assists with the real-time depth-

based compositing of CG assets and effects.

To generate an MP4VR file that includes depth map information:

Export Stitched DPX or EXR frames using OZO Creator

Export Depth maps using OZO Creator. Other than the Format selection, be sure to utilize the same

settings used to generate the stitched DPX or EXR frames

Import the stitched footage into OZO Creator, browsing to the appropriate locations of the stitched

footage and the associated depth maps when prompted

Select the stitched footage under the STI tab of the Stage, and export the footage to the MP4VR

format, ensuring that the Include depth map checkbox is selected in the Export Asset window

Viewing MP4 Files using OZO Preview OZO Preview can be freely downloaded from Nokia’s website:

https://ozo.nokia.com/vr/get-ozo-virtual-reality-products/

OZO Preview can be used to view MP4 files generated by OZO Creator. Exported footage can be reviewed using

either the computer monitor and mouse for 2D playback, or a supported HMD for 3D head-tracked playback,

along with a pair of regular headphones for audio reproduction (including binaural audio, if the export was

encoded with it).

Preview will be able to identify the stereoscopic/monoscopic format of the file through the file suffix.

https://ozo.nokia.com/vr/get-ozo-virtual-reality-products/

54

_TB.mp4 designates a top-bottom (over/under) stereoscopic file

_2LR.mp4 designates an MP4 containing two, full resolution separated left and right eye streams

_M.mp4 designates a monoscopic (2D) panorama of either a left eye or a right eye

When viewing the exported footage in OZO Preview, select Show Info Overlay to display an angle overlay, as

well as the current time and total run time of the footage.

The Info Overlay displays the current angle, the current time (upper left), and the total run time (upper right)

Additionally, OZO Preview can sync to Pro Tools using Midi time code (MTC) generated by the Pro Tools

application. This allows synced playback of audio in Pro Tools while viewing the footage in an HMD environment

in order to create the most accurate spatial audio mix for presentation. This is discussed in more detail in the

Sound Mixing section of this documentation.

Sound Mixing OZO+ captures spatial audio with eight microphones positioned over the surface of the device. The microphone

signals are stored with the raw captures. These microphone signals are later processed with software to enable

accurate headphone and loudspeaker playback.

Audio capturing itself does not require anything specific for VR playback. Typical means can be used to capture

the audio, such as close miking, coincident microphones, microphone arrays, etc. In addition, ADR dialogue or

Foley effects can be added in the mixing phase as is normally done.

Audio mixing for the Nokia OZO+ can be performed in the same way as normally done for films with 5.1 or 7.1

mixes. Essentially, if the mix works with a properly configured loudspeaker playback system, it should work with

Nokia’s binaural headphone playback. No special compensation is needed during the mixing process to account

for the conversion from multi-channel playback to spatial binaural, as the binaural rendering algorithms will

handle that.

However, one large difference between mixing for traditional theatrical or home video displays and mixing for

VR is the precision required in panning and defining the perceived distance to the source.

55

VR mixes must be panned extremely precisely, otherwise the auditory direction of the source and the visual

direction of the same source in the image will not match, and this will negatively impact the immersion for the

viewer, and potentially result in confusion.

Similarly, the perceived auditory distance to the source should match as closely as possible the perceived visual

distance to the same source. This can be achieved through the application of varying levels of location-

appropriate reverb and adjustments to the volume of the audio source.

That being said, realism may have to be compromised a bit to preserve clarity, and thus the mix typically should

not re-create a perfect auditory reproduction of the environment. This is because the environment, if

reproduced perfectly, will likely be too “ambient” to provide the viewer with enough clarity to easily discern

dialogue, especially from distant sources.

The currently supported loudspeaker layouts are (positive degrees to the right of center):

4.0 channels: -45, 45, -135, and 135 degrees

5.0 channels: -30, 30, 0, -110, and 110 degrees

7.0 channels: -30, 0, 30, -90, 90, -150, and 150 degrees

8.0 channels: 0, 45, 90, 135, 180, -135, -90, and -45 degrees

Please note that when muxing audio to picture using OZO Creator to encode MP4 files for the Nokia Player SDK,

the channel order in the final mixed multichannel 48kHz 24-bit WAV file has to match the aforementioned

supported loudspeaker layouts for proper spatial positioning. So, for a 5 channel mix, track 1 would correspond

to the Left channel (-30), track 2 to the Right (30), track 3 to the Center (0), track 4 to the Left Rear (-110), and

track 5 to the Right Rear (110).

It is recommended to mix using loudspeakers equidistantly located from the listener at the specified angles for

the chosen number of channels. Loudspeakers still currently provide the best “master” audio reproduction for

mixing. The mix can be reviewed with a binaural rendering plugin and a pair of high-quality headphones.

However, it is advised not to mix entirely using headphones, as the mix may not translate well to other sets of

headphones.

This mix should be synced to monitor playback of a 360 Editorial with the same loudspeaker angles burned in,

which provides a 2D visual representation of the position of the source within the 360-degree sound field. This

is probably the best way to “rough-in” the mix.

Reviewing the Mix using an HMD The mix can be reviewed using an Oculus DK2 HMD by using Midi time code (MTC) to sync OZO Preview to the

playback of Pro Tools when on a Macintosh. Loudspeaker audio playback using a video-synced HMD is currently

the most accurate way to judge if the mix of the audio matches the visual content.

56

Midi control can be used to start OZO Preview playback from digital audio workstations (DAWs) when they are

running in different computers in the same network. This enables audio playback from DAW and video playback

from OZO Preview to run in sync.

On a Mac, to use this feature with Pro Tools, first make sure "Network midi" is set up in "Audio Midi Setup" in OS

X preferences and that in OZO Preview "MTC Sync" command in "View" menu is enabled.

Then do the following in Pro Tools:

Set up a session ("Setup" > "Session") to start at 00:00:00:00.

In "Setup" > "Peripherals...", set the MTC Generator to be the network session.

Go to the next tab "Machine Control" and enable the master.

Select the network midi session.

In "Setup" > "Preferences", go to the "Synchronization" tab and activate both "Machine Chases Memory

Location" and "Machine Follows Edit Insertion/Scrub".

In the transport controls activate "GEN MTC".

The project might need some adaptation to account for approximately 600ms of startup buffering latency of

the player, after which the content should be in sync.

Another, perhaps less complex way to review the mix in an HMD is simply to have the locked picture edit

sideloaded to a Gear VR system. Then manually sync the start of the playback in the HMD and the playback in

the DAW of choice.

Simply ensure that you are in the optimal location within a properly arranged loudspeaker array while reviewing

playback with an HMD, and you should get a fairly accurate representation of how the mix will work with the

sequence after being encoded for binaural playback.

Reviewing footage in any HMD will block the view of the controls, so it is unlikely that immediate adjustments

can be made while reviewing the mix in this fashion. You will likely have to make mental notes, take off the HMD

to make adjustments, and then review again. Despite the inconvenience, it is highly recommended that the mix

is reviewed using this method before finalizing, because it best represents the end user experience.

57

Scripting with OZO Creator OZO Creator also includes scripting functionality for the distribution of batch exports. The scripting

functionality is run by executing OZO Creator from a command line prompt and specifying a JSON script that

contains all of the import/export/parameter information necessary.

JSON scripts can easily be generated through the OZO Creator application. The procedure is exactly the same

as queueing an export, except instead of pressing the Export button in the Export Asset window to add the

process to the Job Queue, press the Save As JSON button instead. OZO Creator will then write out a JSON file

to the specified Destination folder.

If on a Mac, run the OZO Creator scripting functionality by using Terminal to navigate to

/Applications/OZO/OZO\ Creator.app/Contents/MacOS/ (assuming that the application is installed in its

default location) and then type “./OZO\ Creator [path to and name of JSON file]”

If using a Windows PC, run the OZO Creator scripting functionality by using the Command Prompt (cmd.exe).

Assuming that the application is installed in its default location, navigate to C:\Program Files\Nokia OZO\OZO

Creator. Type “OZO Creator” [path to and name of JSON file], for example “OZO Creator” C:/Users/”User

Name”/Folder/File. Please note:

The OZO Creator command has inverted commas, “OZO Creator”

A forward slash is used for the path name

Where the file path has two words, inverted commas are used, “User Name”

Scripted Command Line Usage Examples The following documentation is displayed by typing the command line syntax ./OZO\ Creator -–examples on

Mac or “OZO Creator” –-examples on Windows in the install location. However, the easiest way to obtain an

example of the script is to simply write out a JSON file from the Export function in OZO Creator.

OZO Creator command line usage examples The JSON file is the primary interface. The command line arguments can be used instead if there is only one input file. At the moment there can be only one output. Refer to --help text for descriptions of the properties. === Capture -> Fisheye DPX or EXR === JSON file: { "inputs": [ { "file": "/path/to/capture.mov" } ], "outputs": [ { "file": "/path/to/output_exr/exr/cam%d/%%07d.exr", "metadataFile": "/path/to/output_exr/metadata/foo.Metadata", "metadataTextFile": "/path/to/output_exr/metadata/first_frame_metadata.txt" } ]

58

} The output file type is detected from output file. Command line arguments for similar operation would be: --input.file /path/to/capture.mov \ --output.file /path/to/output_exr/exr/cam%d/%%07d.dpx === Capture -> Editorial === { "inputs": [ { "file": "/path/to/capture.mov", "in": 0, "out": 5 } ], "process": { "editorial": true }, "outputs": [ { "file": "/path/to/output_editorial.mov" } ] } "in" and "out" for an input can be used to select in and out points for a capture. === Capture -> Stitched DPX === { "inputs": [ { "file": "/path/to/capture.mov", "in": 0, "out": 3 } ], "process": { "outputWidth": 1920, "outputHeight": 1080, "stitch": { "type": "Separated" } }, "outputs": [ { "file": "/path/to/output_stitched/%s_%%07d.exr" } ] } === Fisheye DPX -> mp4 === { "inputs": [ { "file": "/path/to/output_dpx/dpx/cam%d/%%07d.dpx", "metadataFile": "/path/to/output_dpx/metadata/foo.metadata", "frameCount": 5 } ], "process": { "outputWidth": 1920, "outputHeight": 1080, "stitch": { "type": "Separated" }

59

}, "outputs": [ { "file": "/path/to/output_stitched.mp4" } ] } For image sequence inputs the frameCount has to be specified. This is because reading huge image sequence from a

network disk can be slow.

60

Documents

OZO Post Production Workflow - docs.ozo.nokia.com · PDF fileaudio, and as Nokia’s OZO Audio Interchange format. The camera records both picture and sound either to its own internal