Embed Size (px)
Citation preview
ProxImaL: Efficient Image Optimization using Proximal Algorithms
Steven Diamond1Felix Heide1,2
Wolfgang Heidrich3,2 Gordon Wetzstein1
2University of British Columbia 3KAUST1Stanford University
www.proximal-lang.org
Matthias Nießner1 Jonathan Ragan-Kelley1
Low-Light Burst Imaging
Pelican Color Array
Interlaced HDR and RGB-IR
Light.co Array Camera Kinect ToF Depth
Imaging
Formal Optimization
Zoran and Weiss 2011 Levin et al. 2004Krishnan and Szeliski 2011
Krishnan and Fergus 2009Heide et al. 2015
Deconvolution Denoising Inpainting + Colorization Camera Image Processing
Schmidt et al. 2015 Chen et al. 2015
Demosaic Denoise
Bad Pixel Correction
Image Enhancing
Tone Mapping
Lens Correction
Black Level
Meteringerror error
errorerror
Formal Optimization
Image Processing Pipeline
Formal Optimization
Formal Optimization
Brooke et al. 1988 Grant and Boyd. 2014 Lofberg 2004
DSLs for convex optimization:
Formal Optimization
Brooke et al. 1988Grant and Boyd. 2014 Lofberg 2004
DSLs for convex optimization:
Infeasible for Imaging problems:• Millions of Variables• Large-Scale Operators
ProxImaL
ProxImaLAndroid HDR+First Frame
ProxImaL Code:
ProxImaLAndroid HDR+
Objective:
An example:
Proximal Code:
OriginalBlurredSubsampled
Translation “by Hand”:
Objective:
or: with either:
ADMM:
Objective:
or: with either:
100 sec 10 sec
Blurred
Blurred + Subsampled
Result
Ambiguous translations drastically affect solver performance !
Translation “by Hand”:
Sum of “proxable” functions:
General Problem Representation:
• . are “proxable” penalty functions with the proximal operator:
are linear transforms on the unknowns.• .
Proximal algorithms:
• ADMM [Boyd 2011]• Linearized ADMM [Boyd 2011]• PC [Chambolle and Pock 2011]• (HQS [Geman and Yang 1995])
Proximal Compiler:
Objective:
Algorithm Implementation:
Halide
Function Numpy [ms] Halide [ms]
sum_of_squares 246 42
dot product 97 16
subsample 356 73
grad 1188 95
conv 7791 121
warp 458 153
norm1 202 27
group_norm1 1037 68
FFT 23 9
Runtime of TV-Deconvolution:
Runtime of TV-Deconvolution:
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
ProxImaL
ProxImaL Code:
ProxImaLKrishnan and Fergus 2009
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
ProxImaL Code:
40 dB 34 dB
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
ProxImaL Code:
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
ProxImaL Code:
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
Applications:
Demosaicking Interlaced HDR Low-Light Burst Imaging
Poisson Deconvolution
Phase Retrieval
Please see paper !
ProxImaLwww.proximal-lang.org
Open Source !
