L. Yaroslavsky

FAST SIGNAL SINC-INTERPOLATIONAND ITS APPLICATIONS IN IMAGE

PROCESSING

OUTLINE

Topicality and principles of signal/imageresamplingSinc-interpolation as a gold standard forconvolution based interpolationFast SDFT based sinc-inerpolationExamples of applicationsSinc-interpolation in DCT domainSinc-interpolation in sliding window

Signal/Image Resampling Applications

Signal/image sampling standard conversionSignal “fractional” delayImage co-ordinate conversion; nonuniform-to-uniformraster resamplingSignal/image localization and alignment with sub-pixelaccuracyRadon Transform and tomographic reconstruction3D imaging

Image resampling: basic principle

OutputimageInput

image

Resampling

Signal interpolation computationalmethods

• Nearest neighbor (zero order) interpolation:

( ) ( ) ( )[ ]Lones,,akronconva Lint δrr=0 ; ( ) [ ]

→←=

L,..,,,L 0001δ

• Linear (bilinear) interpolation: ( ) ( ) ( )[ ] ( )[ ]Lones,Lones,,akronconvconva Lint δrr

=1 .

• R-th order spline interpolation: ( ) ( ) ( )[ ] ( )[ ][ ]Lones,Lones,,akronconvconvconva LR

int δrLr

= .

• Sinc-interpolation:

( ) ( )[ ]x/xkxxak

k ∆∆−= ∑∞

−∞=

πα sinc

TWO APPROACHES:Convolution based and Fourier based

For a long time, sinc interpolation has been the holy grail of geometrical operations. ... The sinc-function provides error-free interpolation of bandlimited functions. There are two difficultiesassociated with this statement. The first one is that the class of bandlimited functions represents buta tiny fraction of all possible functions; moreover, they often give a distorted view of the physicalreality – think of transition air/matter in CT scan: ... this transition is abdrupt and cannot beexpressed as a bandlimited function.... The second difficulty is that the support of the sinc-functionis infinite. An infinite support is not too bothering, provided an efficient algorithm can be found toimplement interpolation with another equivalent basis function that has a finite support. This isexactly the trick we used with B-splines and o-Moms” (Ph. Thevenaz, Th. Blu, M. Unser,Interpolation Revisited, IEEE Trans. on Medical Imaging, V. 19, No. 7, July 2000)

Sinc-interpolation versus spline interpolation: an opinion

The opinion that sinc-interpolation assumes band-limited functions is afallacy. The meaning of the sampling theorem is this:

Given samples of a continuous function , there is no betterapproximation of the function with convolution bases functions than bya bandlimited function obtained by sinc-interpolation of its samples.

Sinc-interpolation: the gold standard for discretelinear interpolation of sampled data

Discrete Sampling Theorem: Let a signal of LN samples is sampled with sampling interval of L samples. For

101 −= N,...,k ; 102 −= L,...,k the sampled signal can be written as ( )21kak δ .

Compute its DFT:

( ) ( ) ( )=

+

→← ∑ ∑− −

=

1 1

0

2122

2 111

21 L

k

N

kk

DFTk r

LNkLkika

LNka πδδ exp

( ) ( )=

+∑ ∑

−

=

−

=

1

0

1

0

212

1 21

21 N

k

L

kk r

LNkLkiexpka

LNπδ

( ) Nmodr

N

kk L

rNkiexpa

LNαπ 121 1

0

1

11

=

∑

−

=

This means that sampling discrete signal results in periodical replication of its spectrum with the number of replicas equal to sampling interval.

Discrete sinc-interpolation: the gold standard forconvolution based interpolation of sampled data

Signal interpolation by (periodical) digital convolution:

( ) ( )∑ ∑−

=

−

=

+−=1

0

1

0212

1 21

N

k

L

kNLkk kNkkhkaa modint

~ δ

In DFT domain:{ }( ) ( )( ) { }( )kKrrk hDFTaDFT ⋅==

11 mod~~ αα

Therefore, the only way to avoid aliazing and signal distortions issignal ideal low pass filtering. For N-odd number:

( )( ) ( )( )∑

−

=

→←

−−+−

−1

01

11211

N

kn

IDFTNr La

LNLNNrrect 1mod k-kN;N;sincd/ α

( ) ( )( )NxN

NKx/sin/sinxN;K;sincd

ππ

=

Problem: bounary effects due to the periodical convolution

Discrete sinc-interpolation: a gold standard forinterpolation of sampled data

50 100 150 200 2500

2

4

6

8

10

50 100 150 200 2500

2

4

6

8

10

Low pass filtering

5 0 1 0 0 1 5 0 2 0 0 2 5 0

-1 . 5

-1

-0 . 5

0

0 . 5

1

1 . 5

2

0 50 100 150 200 250

-2

-1

0

1

2

Sincd-Interpolated signal

5 10 15 20 25 30

-1.5

-1

-0.5

0

0.5

1

1.5

2

5 10 15 20 25 300

1

2

3

4

5

6

7

8

9

10

Discrete signal

Its DFT spectrum

Nearest neighbor, Bilinear, Bicubic spline andSinc-image interpolation

Nearest neighborinterpolation

Bicubic splineinterpolation

Bilinearinterpolation

Sinc-interpolation

Computationl algorithms for discteresinc-interpolation: Zero Padding Method

Interpolation functions:

( )( )∑−

=

=1

01

1 N

k1kk Lk-kN;M;sincda

La~

1;

( ) ( )( )N/xsinN

N/KxsinxN;K;sincdπ

π= ; 110 −= LN,...,,k

Shifted Discrete Fourier Transforms as discreterepresentations of the Fourier integral:

x

a(x) Sampled signal

x∆u

f

( )fαSampled signal spectrum

f∆

v

( ) ( )( )xukxaxaN

kreconstr_signk ∆+−= ∑

−

=

1

0ϕ

( ) ( )( )fvrffN

kreconstr_spnr ∆+−= ∑

−

=

1

0ϕαα

Continuos signal

Continuous signalspectrum

( ) ( ) ( )dxfxiexpxaf ∫∞

∞−

= πα 2 ( )( )∑−

=

++

=1

021 N

kk

v,ur N

vrukiexpaN

πα

Fourier integral Shifted DFT (canonic form)

fx/N ∆∆= 1

Signal and spectrum sampling

( )∑−

=

+

=

1

0221 N

kk

v,ur N

rukiexpNkviexpa

Nππα ( )∑

−

=

+

−=

1

0221 N

kk

v,ur N

vrkiexpNruiexpa

Nππα

Direct and inverse Shifted DFTs (reduced form)

L.P. Yaroslavsky, Shifted Discrete Fourier Transforms, In: Digital Signal Processing, Ed. by V. Cappellini, and A. G.Constantinides, Avademic Press, London, 1980, p. 69- 74.

Signal resampling using SDFTs

( )=

+

−

−= ∑

−

= Nqrniexp

Nrpiexp

Na~

K

r

v,uq/v,p/un r

ππα 221 1

0

( )( ) ×

+

+

−∑−

=

1

021N

kk p-un-kN;K;sincdv

NKkiexpa π

( )

−

−

+

−− pu

NKiexpnq

NKiexp 121 ππ

Signal SDFT(u,v) ISDFT(p,q) Sinc-intrerpolated signalshifted by (u-p)

L. Yaroslavsky, M. Eden, Fundamentals of Digital Optics, Birkhauser, Boston, 1996

An algorithm of SDFT based signal sinc-interpolationL. P. Yaroslavsky, Signal sinc-interpolation: a fast computer algorithm, Bioimaging, 4, p. 225-231, 1996

Discrete sinc interpolation versus otherconvolution based interpolation methods

SeparableSeparable & nonseparable2-Dinterpolation

Requires analytical and numericaloptimization

Simple: direct spectrumshaping in DFT domain

Design

Non-invertibleCompletely invertibleInvertibility ofresampling

Zero interpolation error

O( log(Size of the signal frame))

SDFT based discrete sincinterpolation

Non-zero interpolation error:1/ O(Interpolation function support)

Interpolationaccuracy

O(Interpolation function support)Computationalcomplexity (peroutput sample)

Convolution basedinterpolation methodsPh. Thevenaz, T. Blu, M. Unser, Interpolation Revisited,IEE Trans. on Medical Imaging, v. 19, No. 17, July 2000

SDFT versus zero padding sinc-interpolation

No intermediate buffer isrequired

Requires an intermediate buffer for NLsamples

Memory usage

Arbitrary signal shift;Rational zoom-factor

Power of 2 for the most widely used FFTalgorithms

Zoom factor

O(logN)Arbitrary shifts are possible

O(LlogNL)unless FFT pruned algorithms are used.Shift only by (power of 2)-th fraction of thediscretization interval are possible when themost wide spread FFT algorithms are used.

Computational complexity (generaloperations, per output sample) forsignal shift by a fraction of thediscretization interval

O(N)O(NLlogNL)unless FFT pruned algorithms are used

Computational complexity (generaloperations, per output sample) of L-fold zooming signal of N samples inthe vicinity of an individual sample

O(logN)O(logNL)Computational complexity (generaloperations, per output sample) ofL-fold zooming signal of N sampleswith the use of FFT

SDFT based methodZero padding method

Applications:Spectrum analysis with sub-pixel resolution

Applications:Image rotation: a three-pass algorithm

Image rotation by an angle θ as a geometrical transformation of signal co-

ordinates can be described as a multiplication of signal coordinates ( )y,x vector

by a rotation matrix:

( )

−=

θθθθ

θcossinsincos

ROT

In order to simplify computations, one can factorize rotation matrix ( )θROT intoa product of three matrices each of which modifies only one co-ordinate:

(X-shearing Y-shearing X-shearing)

( ) ( ) ( )

−

−=

−=

1021

101

1021 /tan

sin/tan

cossinsincos

ROTθ

θθ

θθθθ

θ

Fast signal sinc-interpolation algorithm is ideally suited for signal translationneeded for image shearing

M. Unser, P. Thevenaz, L. Yaroslavsky, Convolution-based Interpolation for Fast,High-Quality Rotation of Images, IEEE Trans. on Image Processing, Oct. 1995, v. 4, No. 10, p. 1371-1382

Applications: Three pass image rotation withsinc-interpolation

Initial image First pass

Second pass Third pass: rotated image

Applications:Radon Transform and tomosynthesis:Filtered back projection reconstruction

Radon transform: rotationand directional summation

Tomographicreconstruction: ramp-filtering projections, backprojecting, rotation andsummation

Applications:Radon Transform and tomosynthesis:Fourier reconstruction method

Object

Projections

Projection spectra

Interpolated 2-Dobject spectrum

Reconstructedobject

Image geometrical transformations by means ofsinc-interpolated image zooming (oversampling)

Sinc-interpolatedsubsampling

Initial image

Magnified image

Transformed image copies

Image geometrical transformations with sinc-interpolation

Radius

Angle

Cartesianto polar

Polar toCartesian

Angle

Radius

Measuring image profile with sub-pixel resolution

Sincd-interpolation in DCT domain

Purpose: minimization of boundary effects

( )=

+

−= ∑−

=

12

0 2212

21 N

rrk N

rkiN

b /exp πβ

( ) ( )

+

+

+

−+ ∑−

=

∗1

100

212121 N

rrr

DCTr

DCT rN

kirN

kiN

/exp/exp πηπηαηα

−+=

−=

−= ∑

−

=

1210

11021 1

0

NNNk

NkNksi

NhN

ss

k

,...,;

,...,;exp πϕ

For u-shift( )( )

−+==

−==

∗− 112

2212102

NNsNsNus

NsNusi

sN

s

,...,/;/;/cos

/,...,,;/exp

ϕππ

ϕ

=

Nurir πη 22 exp

( )=

+

−= ∑∑

−

=

−

=+

1

0

1

012 2

122221 N

k

N

ssr N

rkiNksi

Nππϕη expexp ( )∑∑

−

=

−

=

+−1

0

1

0 21222

21 N

k

N

ss N

srkiN

πϕ exp

Pruned DFT is useful: first half of the sequence is zero, only odd terms ofDFT are to be computed

Symmetricallyextended signal

Interpolationfilter PSF

50 100 150 200 250

0

5

10

15

20

25

30

35FFT sinc-interpolationDCT sinc-interpolation

Sincd-interpolation in DCT domain

10 20 300

5

10

15

20

25

30

Initial signal8x sinc-interpolated signals

2-D separable sincd-interpolation in DCT domain

FFT interpolation DCT interpolation

Signal resampling in sliding window:Summary

Good approximation to sincd-interpolationSimple design in DFT domainIrregular -to-regular sampling raster resampling is possibleCan naturally be combined with signal denoising and

restorationLocal adaptive interpolation is possibleImplementation of inseparable interpolation kernel is easyComputational complexity O(WindowSize) per output

signal sample (comparable to that of spline interpolation)

Signal sinc-interpolation in sliding window:Impulse and frequency responses

1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

Window11 Window15

Interpolation functions for window size 11 and 15 samples Interpolation filter frequency responses: signal 3x zoom

Image zoom: global versus sliding window sinc-interpolation

Global sinc-interpolation Sliding window sinc-interpolation

Arbitrary image mapping in sliding window

lcmapping_arbitr(len,(mapX+i*mapY)/1.5,8,8)

MapX

MapY

100 200 300 400 500 600 700 800 900 1000

0

0.2

0.4

0.6

0.8

1

Ada ptive ve rs us non a da ptive s inc a nd ne a re s t ne ighbor inte rpola tion

non a da ptivene a re s t ne ighbora da ptive

Sliding window sinc-interpolation:non-adaptive vesrus adaptive

10 20 30 40 50 600

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Initial signal Interpolated signal (8xZoom)

Sliding window sinc-interpolation:non-adaptive vesrus adaptive

non-adaptive adaptive

Irregular-to-regular raster image resampling anddenoising in sliding window

Additional bibliography on FFT based and spline basedinterpolation

• D. Fraser, Interpolation by the FFT revisited – An experimental evaluation, IEEE Trans. ASSP-37, p. 665-676, May1989

• T.J. Cavichi, DFT time domain Interpolation, Proc. Inst.Elec.Eng.-F, vol. 139, pp. 207-211, 1992

• M. Unser, A. Aldroubi, M. Eden, Plynomial spline signal approximations: filter design and asymptotic equivalencewith Shannonäs sampling theorem, IEEE Trans. IT, v. 38, pp. 95-103, Jan. 1992

• Ph. Thevenaz, Th. Blu, M. Unser, Interpolation Revisited, IEEE Trans. on Medical Imaging, V. 19, No. 7, July 2000