ProxImaL | SIGGRAPH 2016

ProxImaL: Efficient Image Optimization using Proximal Algorithms

Steven Diamond1Felix Heide1,2

Wolfgang Heidrich3,2 Gordon Wetzstein1

2University of British Columbia 3KAUST1Stanford University

www.proximal-lang.org

Matthias Nießner1 Jonathan Ragan-Kelley1

Low-Light Burst Imaging

Pelican Color Array

Interlaced HDR and RGB-IR

Light.co Array Camera Kinect ToF Depth

Imaging

Formal Optimization

Zoran and Weiss 2011 Levin et al. 2004Krishnan and Szeliski 2011

Krishnan and Fergus 2009Heide et al. 2015

Deconvolution Denoising Inpainting + Colorization Camera Image Processing

Schmidt et al. 2015 Chen et al. 2015

Demosaic Denoise

Bad Pixel Correction

Image Enhancing

Tone Mapping

Lens Correction

Black Level

Meteringerror error

errorerror

Formal Optimization

Image Processing Pipeline

Formal Optimization

Formal Optimization

Brooke et al. 1988 Grant and Boyd. 2014 Lofberg 2004

DSLs for convex optimization:

Formal Optimization

Brooke et al. 1988Grant and Boyd. 2014 Lofberg 2004

DSLs for convex optimization:

Infeasible for Imaging problems:• Millions of Variables• Large-Scale Operators

ProxImaL

ProxImaLAndroid HDR+First Frame

ProxImaL Code:

ProxImaLAndroid HDR+

Objective:

An example:

Proximal Code:

OriginalBlurredSubsampled

Translation “by Hand”:

Objective:

or: with either:

ADMM:

Objective:

or: with either:

100 sec 10 sec

Blurred

Blurred + Subsampled

Result

Ambiguous translations drastically affect solver performance !

Translation “by Hand”:

Sum of “proxable” functions:

General Problem Representation:

• . are “proxable” penalty functions with the proximal operator:

are linear transforms on the unknowns.• .

Proximal algorithms:

• ADMM [Boyd 2011]• Linearized ADMM [Boyd 2011]• PC [Chambolle and Pock 2011]• (HQS [Geman and Yang 1995])

Proximal Compiler:

Objective:

Algorithm Implementation:

Halide

Function Numpy [ms] Halide [ms]

sum_of_squares 246 42

dot product 97 16

subsample 356 73

grad 1188 95

conv 7791 121

warp 458 153

norm1 202 27

group_norm1 1037 68

FFT 23 9

Runtime of TV-Deconvolution:

Runtime of TV-Deconvolution:

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

ProxImaL

ProxImaL Code:

ProxImaLKrishnan and Fergus 2009

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

ProxImaL Code:

40 dB 34 dB

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

ProxImaL Code:

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

ProxImaL Code:

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

Applications:

Demosaicking Interlaced HDR Low-Light Burst Imaging

Poisson Deconvolution

Phase Retrieval

Please see paper !

ProxImaLwww.proximal-lang.org

Open Source !