Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Porting After Effects to the Mercury GPU EngineJamie Gjerde | Senior Computer Scientist

Scott Carver | Computer Scientist


How We Got Started

▪ Developer kitchen: third parties already using CUDA inside Effects

▪ Customers want more performance (native AE rendering on the GPU)

▪ Mercury GPU Engine: Premiere has full GPU rendering pipeline, many effects already ported

2


Now….

3

• After Effects now supports GPU rendering in its effects pipeline

• Work is underway to port existing AE effects to GPU: >10 effects have been ported


Why GPU is Important for After Effects

▪ Significantly more compute/processing power for image processing tasks

▪ GPUs are improving at a faster rate than CPUs

▪ Doing more work on the GPU frees up CPU time

▪ In a mobile / non-desktop future, most compute power is GPU-based

4


Initial Expectations / Requirements

▪ Faster renders

▪ Performance will scale with faster GPUs

▪ Leverage Mercury Engine and existing effect kernels

▪ Untie AE internals from CPU pixels, CPU implementations

▪ Visual parity is critical!

5


Architectural challenges

▪ Many rendering tasks will be CPU for a long time.

▪ Render graph must support composite GPU and CPU rendering

▪ Must handle upload / readback transparently.

▪ CPU code cannot be replaced, must co-exist.

▪ Solutions?

▪ Render graph refactored to handle buffers independent of hardware: CPU, CUDA, OpenCL, Metal

▪ If an input buffer is located on one hardware, but rendering is happening on different hardware, buffer is transferred lazily.

6


Architectural challenges: composite render graph

7



▪ Adapt old rendering architecture

▪ Downside: Most code assumes CPU buffers, direct access, not built for pixel-level parallelization

▪ Downside: AE render graph itself is still basically single threaded / synchronous

▪ Upside: Render graph is very memory conservative (designed for 90’s memory limits)

▪ Upside: Single threaded render means simple GPU interaction, no async spaghetti

▪ Solutions?

▪ GPU and CPU paths share pre-render, frame dependency calculation, and setup code.

▪ GPU-specificity hidden from most components.

▪ Invoke GPU kernels only at lowest possible level.

8



▪ Visual Parity is HARD

▪ Bit depth (8, 16, 32-float)

▪ Algorithm match between SSE implementation and OpenCL/CUDA/Metal code

▪ Sometimes, algo's need some rewrite on CPU as well.

▪ Even with effort: 8 and 16 won't always match due to rounding: float differences can accumulate.

▪ Solutions?

▪ Never mix GPU and CPU frames for an effect in a single comp.

▪ Downside: if we run out of memory for a frame, we can't transparently fall back to CPU

▪ On 8 and 16 bpc: clip to 0..1 after each GPU render node

▪ In some cases, clipping/rouning in kernel is also required

9



▪ Memory

▪ AE frame sizes are large (30000 x 30000 x 32bpc maximum)

▪ Render sizes can vary from frame to frame, not predictable

▪ GPU’s cannot recover easily from out-of-memory (e.g. swap)

▪ Solutions?

▪ AE has existing fractional resolution workflows that can help.

▪ Tiled rendering is extremely complex problem for AE's render graph.

▪ However: some kernels could be tiled at a lower level for GPU renders.

10


Native GPU Plug-in SDK

▪ Current internal effects are based on existing Premiere GPU SDK

▪ AE render graph handles allocation and CPU <> GPU transfers

▪ Effect receives GPU buffer via SDK call, with correct pixels and parameters for render.

▪ Future: Premiere SDK isn't rich enough to support all AE render features – expansion is a must for both internal effects and third parties.

▪ Internal effect development has illuminated areas where SDK needs iteration

▪ 3rd party developers need an SDK with richer API’s and tools to write efficient plugins for a variety of GPU platforms

11

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 12

Documents

Porting After Effects to the Mercury GPU Engineon-demand.gputechconf.com/gtc/2017/presentation/s7609...Significantly more compute/processing power for image processing tasks GPUs are