Computer Vision Laboratory 1 B-Spline Channels & Channel Smoothing Michael Felsberg Computer...

1

Computer Vision Laboratory

B-Spline Channels & Channel Smoothing

Michael FelsbergComputer Vision Laboratory

Linköping UniversitySWEDEN

2

Computer Vision Laboratory

General Idea of Channels

• Encode single value (linear or modular) in N-D coefficient vector (channel vector)

• Locality of encoding– Similar values in same coefficients– Dissimilar values in different coefficients

• Stability by smooth, monopolar basis functions– Small changes of value lead to small changes of

coefficients– Non-negative coefficients

3

Computer Vision Laboratory

Example for Single Value

4

Computer Vision Laboratory

Example for Multiple Values

5

Computer Vision Laboratory

Overview

• Encoding with quadratic B-splines

• Decoding strategies

• Relation to kernel-density estimation

• Relation to robust M-estimation

• Channel smoothing

• Applications

6

Computer Vision Laboratory

Quadratic B-Splines

€

B0 f( ) =

1 f <1

21

2f =

1

2

0 f >1

2

⎧

⎨

⎪ ⎪

⎩

⎪ ⎪

Bk f( ) =k + 2 f +1

2kBk−1 f +

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟+

k − 2 f +1

2kBk−1 f −

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟

€

B2 f( ) =

3

4− f 2 f ∈ 0,

1

2

⎡ ⎣ ⎢

⎞

⎠ ⎟

1

2

3

2− f

⎛

⎝ ⎜

⎞

⎠ ⎟2

f ∈1

2,3

2

⎡ ⎣ ⎢

⎞

⎠ ⎟

0 f ∈3

2,∞

⎡ ⎣ ⎢

⎞

⎠ ⎟

⎧

⎨

⎪ ⎪ ⎪

⎩

⎪ ⎪ ⎪

7

Computer Vision Laboratory

• The value of the nth channel at x is obtained by

• Encoding in practice:– m=round(f)– c[m-1]=(f-m-0.5)2 /2– c[m]=0.75-(f-m)2

– c[m+1]=(m-f-0.5)2 /2

B-Splines Encoding

€

cn f ,x( ) = B2 f x( ) − n( ) n =1... N

We assume f to be shiftedand rescaled such that

€

f ∈ 1.5,N − 0.5[ ]

8

Computer Vision Laboratory

Example

QuickTime™ and a decompressor

are needed to see this picture.

9

Computer Vision Laboratory

Linear Decoding

• Normalized convolution of the channel vector

• Choice of n by heuristics– Largest denominator (3-box filter)– Additional: local maximum

€

fn0=

ncn f( )n= n0 −1

n0 +1

∑

cn f( )n= n0 −1

n0 +1

∑=

cn0 +1( f ) − cn0 −1( f )

cn0 −1( f ) + cn0( f ) + cn0 +1( f )

+ n0

0

10

Computer Vision Laboratory

Quantization Effect

11

Computer Vision Laboratory

Quadratic Decoding I

• Idea: detect local maximum of B-spline interpolated channel vector– Step 1: recursive filtering to obtain

interpolation coefficients:

€

cn+ = cn + zcn−1

+

cn− = z(cn +1

− − cn+)

′ c n = 8cn−

€

c1+ = c1

cN− =

z

z2 −1cN

+

z = 2 2 − 3

12

Computer Vision Laboratory

Quadratic Decoding II

– Step 2: detect zeros

€

0 = λβ 2 + μβ + ν

λ = − ′ c n0 −2 + 2 ′ c n0 −1 − 2 ′ c n0 +1 + ′ c n0 +2( ) 2

μ = − ′ c n0 −2 + 2 ′ c n0− ′ c n0 +2( ) 2

ν = − ′ c n0 −2 − 6 ′ c n0 −1 + 6 ′ c n0 +1 + ′ c n0 +2( ) 8

13

Computer Vision Laboratory

Quadratic Decoding III

– Step 3: compute energy

– Step 4: sort the decoded values according to their energy(the energy represents the confidence)

€

E(n0) = λβ 3 3− μβ 2 2 + νβ +

′ c n0 −2 + 24 ′ c n0 −1 + 46 ′ c n0+ 24 ′ c n0 +1 + ′ c n0 +2( ) 48

The decoded values must be shiftedand rescaled to the original interval

14

Computer Vision Laboratory

Quantization Effect

15

Computer Vision Laboratory

Kernel Density Estimation I

• Given: several realizations of a stochastic variable (samples of the pdf)

• Goal: estimate pdf from samples

• Method: convolve samples with a kernel function

€

˜ p f( ) =1

Nk f − fn( )

n=1

N

∑

16

Computer Vision Laboratory

Kernel Density Estimation II

• Requirements for kernel function:– Non-negative– Integrates to one

• Expectation of estimate:

€

Ε ˜ p f( )[ ] = k f − ′ f ( ) p ′ f ( )∫ d ′ f = k∗p( ) f( )

17

Computer Vision Laboratory

Relation to C.R.

• Adding channel representation of several realizations corresponds to a sampled kernel density estimation

• Ideal interpolation with B-splines possible! €

1

Mcn fm( )

m=1

M

∑ = ˜ p f( )f = n

≈ B2 ∗p( ) f( )f = n

n =1...N

18

Computer Vision Laboratory

L2 vs. Robust Optimization

• Outliers are critical for L2 optimization:

• Idea of robust estimation:– error norm is saturated for outliers– Influence function becomes zero for

outliers

€

f0 = argmin f − f0( )∫2p f( ) df

f0 = f p f( ) df∫Error norm

Influence function

19

Computer Vision Laboratory

Robust Error Norm

E

f - f0

20

Computer Vision Laboratory

Robust Influence Function

E’

f - f0

21

Computer Vision Laboratory

Influence Function of C.R.

€

Ψ f( ) = B2 f −1( ) − B2 f +1( )

Obtained from lineardecoding:

22

Computer Vision Laboratory

Error Norm of C.R.

€

ρ f( ) = 2B3

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟− B3 f +

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟− B3 f −

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟

Obtained by integrating the influence function:

23

Computer Vision Laboratory

Channel Smoothing

24

Computer Vision Laboratory

Channel Smoothing Example

• Discontinuity is preserved

• Constant and linear regions are correctly estimated

25

Computer Vision Laboratory

Stochastic Signals

• Stochastic signal: single realization of a stochastic process

• Ergodicity assumption:– averaging over several realizations at a

single point

can be replaced with– averaging over a neighborhood of a single

realization

26

Computer Vision Laboratory

Ergodicity & C.S.

• Ergodicity often not fulfilled for signals / features, but trivial for channels

• Ergodicity of channels implies that averaging of channels corresponds to (sampled) kernel density estimation

€

cn f ,x( )x

∑ ≈ B2 ∗p( ) f( )f = n

n =1...N

27

Computer Vision Laboratory

Quantization Effect and C.S.

28

Computer Vision Laboratory

Outlier Rejection in C.S.

29

Computer Vision Laboratory

Applications

• Image denoising

• Infilling of information

• Orientation estimation

• Edge detection

• Corner detection

• Disparity estimation

30

Computer Vision Laboratory

Image Denoising

31

Computer Vision Laboratory

Infilling of Information

32

Computer Vision Laboratory

Orientation Estimation

33

Computer Vision Laboratory

Corner Detection

34

Computer Vision Laboratory

Corner Detection

35

Computer Vision Laboratory

Corner Detection

36

Computer Vision Laboratory

Disparity Estimation

37

Computer Vision Laboratory

Disparity Estimation

38

Computer Vision Laboratory

Further Reading

• B-Spline Channel Smoothing for Robust EstimationFelsberg, M., Forssén, P.-E., Scharr, H. LiTH-ISY-R-2579January, 2004