Representation and Compression of Multi-Dimensional Piecewise Functions

Dror BaronSignal Processing and Systems (SP&S) SeminarJune 2009

Joint work with:Venkat ChandrasekaranMichael WakinRichard Baraniuk

The Challenge of Multi-D Horizon Functions• Signals have edges

– images (2D) – video (3D)– light field imaging (4D, 5D)

• Horizon class model– multidimensional– discontinuities– smooth areas

• Main challenge: sparse representation• Related applications: approximation, compression,

denoising, classification, segmentation…

N = 2 N = 3

Existing tool: 1D Wavelets

• Advantages for 1D signals:– efficient filter bank implementation– multiresolution framework– sparse representation for smooth signals

• Success motivates application to 2D, but…

2D Signal Representations

• Challenge: geometry - discontinuities along 1D contours – separable 2D wavelets (squares) fail to capture

geometric structure

• Response:– tight frames: curvelets [Candés & Donoho],

contourlets [Do & Vetterli], bandelets [Mallat]

– geometric tilings: wedgelets [Donoho], wedgeprints [Wakin et al.]

13 26 52

Wedgelet Dictionary [Donoho]

wedgelet decomposition

• Piecewise linear, multiscale representation

– supported over a square dyadic block

• Tree-structured approximation

• Intended for C2 discontinuities

Non-Separable Representations have Potential to be Sparse

Non-separable geometric tiling

13 26 52

Separable wavelets

Signal Representations in Higher Dimensions

• Failure of separable wavelets more pronounced in N>2 dimensions

• Similar problems exist– smooth regions separated by discontinuities– discontinuities often smooth functions in N-1

dimensions

• Shortcomings of existing work– not yet extended to higher dimensions– intended for efficient (sparse) representations for

C2 discontinuities

Goals

• Develop representation for higher-dimensional data containing discontinuities– smooth N-dimensional function– (N-1)-dimensional smooth discontinuity

• Optimal rate-distortion (RD) performance– metric entropy – order of RD function

• Flow of research:– From N=2 dimensions, C2-smooth

discontinuities– To N¸2 dimensions, arbitrary smoothness

Piecewise Constant Horizon Functions [Donoho]

• f: binary function in N dimensions

• b: CK smooth (N-1)-dimensional horizon/boundary discontinuity

• Let x 2 [0,1]N and y = {x1,…,xN-1} 2 [0,1](N-1)

Example Horizon Class Functions

N = 2 N = 3

Compression Problem

• Approximate f with R bits !

• Squared L2 error metric (energy)

• Need optimal tradeoff between rate and L2 distortion

Compression via Implicit Approximation

• Edge detection: – estimate horizon discontinuity b – encode using (N-1)-dimensional

wavelets [Cohen et al.]

• Implicitly approximate f from b

• Theorem [Kolmogorov & Tihomirov; Clements]: Metric entropy for CK smooth (N-1)-D function:

L1 distortionO(¢) lower bound

Metric Entropy for Horizon Functions• Problems with edge detection:

– edge detection often impractical– want to approximate f (not b) require solution that provides estimate in N-

dimensions, without explicit knowledge of b

• Theorem: Metric entropy for N-D horizon function f with CK smooth (N-1)-D discontinuity:

• Converse result – our algorithms achieve this RD performance

Motivation for Solution: Taylor’s Theorem

• For a CK function b in (N-1) dimensions,

• Key idea: order (K-1) polynomial approximation on small regions

• Challenge: organize tractable discrete dictionary for piecewise polynomial approximation

derivatives

Surflets: Piecewise Polynomial Approximations on Dyadic Hypercubes

• Surflet at scale j– N-dimensional atom

– defined on hypercube Xj of size 2-j£2-j££2-j

– horizon function with order K-1 polynomial discontinuity (“surface”-let)

• Tile to form multiscale approximation to f

K = 2 K = 3 K = 4

Wedgelet

3D Surflets

K = 2

K = 3

Discrete Surflet Dictionary

• Describe surflet using polynomial coefficients

K = 2 K = 3 K = 4K = 2 K = 3

Wedgelet

Quantization• Challenge: with naïve quantization of coefficients, dictionary size blows up with K and N• Surflet coefficients approximate Taylor coefficients

• Higher-order coefficients quantized with lesser precision same order error for all coefficients

• Response: for order-l coefficient, use step-size

~ O(2-(K-2)j)

~ O(2-2j)

~ O(2-Kj)

~ O(2-Kj)

Approximation without Edge Detection

• “Taylor surflets” – obtained by quantizing derivatives of b – requires knowledge/estimation of b

• “L2-best surflets”

– obtained by searching dictionary for best fit– requires no explicit knowledge of b– fast search algorithm via manifolds

• Theorem: Taylor or L2–best surflets have same asymptotic performance

Tree-structured Surflet Approximation

• Arrange surflets on 2N-tree– each node is either a leaf or has 2N children– all nodes labeled with surflets– leaf nodes provide approximation– interior nodes useful for predictive coding

Tree-structured Surflet Encoder

• Surflet leaf encoder achieves near-optimal RD performance

• Top-down predictive encoder– code all nodes in surflet tree– use parent surflets to predict children – constant # bits per surflet regardless of scale– layered coarse-scale approximation in early bits

• Theorem: Top-down predictive encoder achieves

Discretization

• Signals often acquired discretely (pixels/voxels) Pixelization artifacts at fine scales

• Approach to discrete data– discretize continuous surflet dictionary– coarse scales: use regular dictionary– smaller dictionary at fine scales

• Theorem: Predictive encoder achieves same RD performance at low rate with discretized dictionary

Numerical Example

• N=2,K=3 • 1024£1024 pixels• Scale-adaptive dictionaries

Wedgelets: 482 bits, 29.9 dB Surflets: 275 bits, 30.2 dB

RD Results

• Dictionary 1: fixed-scale wedgelets• Dictionary 2: wedgelets + scale-adaptive• Dictionary 3: surflets + scale-adaptive

Piecewise Smooth Horizon Functions

• g1,g2: real-valued smooth functions

– N dimensional– CKs smooth

• b: CKd smooth (N-1)-dimensional horizon/boundary discontinuity

• Theorem: Metric entropy for CKs smooth N-D horizon function f with CKd smooth discontinuity:

b(x1)

g1([x1, x2])

g2([x1, x2])

Surfprints

• Challenge: – wavelets good in smooth regions– wavelets wasteful near

discontinuity

• Surflets good near edges

• Response: surfprints project surflets to wavelet subspace

13 26 52

Tree-structured Surprint Encoder

• Discontinuity information needed at finer scales• Top-down encoder

• Prediction not used

• Theorem: Top-down encoder achieves near-optimal

w

w w

w w w w

w w

surfprint

w

coarse

intermediate

maximal

– coarse: keep wavelet nodes– intermediate: nodes with

discontinuity– maximal depth: surfprints

Conclusions and Future Work

• Metric entropy (converse) – piecewise constant/smooth horizon functions– arbitrary dimension & arbitrary smoothness

• Multiresolution compression framework (achievable)– quantization scheme tractable dictionary size – predictive top-down coding optimal performance– scale-adaptive approach to discretization– surfprints at maximal depth near-optimal

• Future research: algorithms

THE END