Signal Processing and Representation Theory Lecture 4

Signal Processingand

Representation Theory

Lecture 4

Outline:• Review• Fourier Transforms• Applications• Invariant Descriptors


ReviewCircle:

If G is the group of 2D rotations / reflections, acting on the space of functions on the circle, the irreducible sub-representations of G are the 1D subspaces spanned by the complex exponentials:

Vk=Span{eik}


ReviewCircle:

Given a function f defined on the circle, we can obtain a rotation / reflection invariant representation by computing the Fourier decomposition:

and storing the norms of the Fourier coefficients:

k

ikkeaf )(

,...,,...,,,)( 110 nn aaaaaf

Representation TheoryReviewCircle:

Given two functions f and h defined on the circle, we can correlate the functions by computing the Fourier decompositions:

multiplying the Fourier coefficients:

and computing the inverse Fourier transform.

k

ikk

k

ikk ebheaf )()(

k

ikkk ebahf )(),(


ReviewSphere:

If G is the group of 3D rotations / reflections, acting on the space of functions on the sphere, the irreducible sub-representations of G are the (2d+1)-dimensional subspaces spanned by the spherical harmonics of frequency d:

),(),,(,),,(),,( 11 dd

dd

dd

ddd YYYYV Span


ReviewSphere:

Given a function f defined on the sphere, we can obtain a rotation / reflection invariant representation by computing the spherical harmonic decomposition:

and storing the norms of the frequency components:

0

),(),(l

l

lm

ml

ml Yaf

,,,,)(

22211

201

211

00

dd

dd aaaaaaf

Representation TheoryReviewSphere:

Given two functions f and h defined on the sphere, we can correlate the functions by computing the spherical harmonic decompositions:

multiplying the spherical harmonic coefficients:

and computing the inverse Wigner-D transform.

00

),(),(),(),(l

l

lm

ml

ml

l

l

lm

ml

ml YbhYaf

0 ',

',' )()(,

l

l

lmm

lmm

ml

ml RDbahRf



Fourier Transforms2D:

If we have the space of functions in the plane, we can consider the representation obtained by the group of translations.

Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.



The irreducible representations are the sub-spaces spanned by the functions:

Translating each function by (x0,y0) we get:

)(, ),( lykxilk eyxw

),(

),(

,)(

)()(

))()((00,

00

00

00

yxwe

ee

eyyxxw

lklykxi

lykxilykxi

yylxxkilk


Fourier Transforms2D (Invariance):

If f(x,y) is a function defined on the plane, we can express the function in terms of its Fourier decomposition:

and obtain a rotation invariant representation by storing the energy in each frequency:

lk

lykxilk eayxf

,

)(,),(

,,,

,,,

,,,

)(1,11,10,1

1,11,10,1

1,01,00,0

aaa

aaa

aaa

f


Fourier Transforms2D (Correlation):

If f and g are functions defined on the plane whose Fourier decompositions are:

the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:

lk

lykxilk

lk

lykxilk ebyxheayxf

,

)(,

,

)(, ),(),(

lk

lykxilklk ebayyxxhyxf

,

)(,,00

00),(),,(



If we have the space of functions in 3D, we can consider the representation obtained by the group of translations.

Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.



The irreducible representations are the sub-spaces spanned by the functions:

Translating each function by (x0,y0,z0) we get:

)(,, ),,( mzlykximlk ezyxw

),,(

),,(

,,)(

)()(

))()()((000,,

000

000

000

zyxwe

ee

ezzyyxxw

mlkmzlykxi

mzlykximzlykxi

zzmyylxxkimlk


Fourier Transforms3D (Invariance):

If f(x,y,z) is a function defined in 3D, we can express the function in terms of its Fourier decomposition:

and obtain a rotation invariant representation by storing the energy in each frequency:

mlk

mzlykximlk eazyxf

,,

)(,,),,(

,,,

,,,

,,,

,,,

,,,

,,,

,,,

,,,

,,,

)(1,1,11,1,10,1,1

1,1,11,1,10,1,1

1,0,11,0,10,0,1

1,1,11,1,10,1,1

1,1,11,1,10,1,1

1,0,11,0,10,0,1

1,1,01,1,00,1,0

1,1,01,1,00,1,0

1,0,01,0,00,0,0

aaa

aaa

aaa

aaa

aaa

aaa

aaa

aaa

aaa

f


Fourier Transforms3D (Correlation):

If f and g are functions defined in 3D whose Fourier decompositions are:

the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:

mlk

mzlykximlk

mlk

mzlykximlk ebzyxheazyxf

,,

)(,,

,,

)(,, ),,(),,(

mlk

mzlykximlkmlk ebazzyyxxhzyxf

,,

)(,,,,000

000),,(),,,(

Outline:• Review• Fourier Transforms• Applications

– Circle– 2D– Sphere– 3D

• Invariant Descriptors


ApplicationsCircle:

If we have a real-valued function f on the circle, we can express the function in terms of its Fourier decomposition:

where ak,bkℝ and bk=-b-k.

k

ikkk eibaf )(


ApplicationsCircle:

Given the space of real-valued functions on the circle, the Fourier decomposition can be used for correlation and invariants-extraction with respect to the group of 2D rotations / reflections.


ApplicationsCircle:

What if we consider the smaller group of axial flips:

that arise due to the ambiguity in PCA alignment?

10

01

G


ApplicationsCircle:

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)

10

01

10

01

10

01

10

01


ApplicationsCircle:

Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.


ApplicationsCircle:

k

ikkeaf )(

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)


ApplicationsCircle:

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)

k

ikk

k

ikk

k

ikk

ea

ea

eaf

)(kk aa


ApplicationsCircle:

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)

k

ikk

kk

ikk

kk

ikikk

k

ikk

ea

ea

eea

eaf

)1(

)1(

)( )(

kk aa kk aa


ApplicationsCircle:

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)

k

ikk

kk

ikikk

k

ikk

ea

eea

eaf

)1(

)( )(kk aa kk aa k

kk aa )1(


ApplicationsCircle:

Initialf()

y-flipf(π-)

x-flipf(-)

x,y-flipf(π+)

kk aa kk aa kk

k aa )1( kk

k aa )1(


ApplicationsCircle:

If f is a real-valued function on the circle expressed in terms of its Fourier decomposition as:

an axial-flip invariant representation can be obtained by storing the norms of the real and imaginary components:

k

ikkk eibaf )(

,...,,,,,)( 111100 bababaf


ApplicationsCircle:

If f and h are real-valued functions on the circle, we could compute the correlation of f with h over all axial flips by comparing at each axial flip independently.

This would take four times as long.


ApplicationsCircle:

Instead, if we express f and h in terms of their Fourier decomposition:

then the correlation of f with h becomes:

k

ikkk

k

ikkk eidcheibaf )()(

k

kkk

kkk

kkk

kk dbcadbcahf 121212122222)(),(


ApplicationsCircle:

By computing the different summations independently:

we can compute the correlation at the different axial flips more efficiently.

k

kkk

kkk

kkk

kk dbcadbcahf 121212122222)(),(

α(real, even)

β(imaginary, even)

γ(real, odd)

δ(imaginary, odd)


ApplicationsCircle:

α(real, even)

β(imaginary, even)

γ(real, odd)

δ(imaginary,odd)

Initial α β γ δ

x- flip α -β γ -δ

y- flip α -β -γ δ

x,y-flip α β -γ -δ

Initial y-flipx-flip x,y-flip

kk aa kk aa kk

k aa )1( kk

k aa )1(


ApplicationsCircle:

So by pre-computing the values of α, β, γ, and δ, we can compute the correlation at each of the four axial flips by summing the values of α, β, γ, and δ with the appropriate sign.

Instead of taking 4 times as long, it takes 16 extra arithmetic ops. to compute the correlation values.

Initial x-flip y-flip x,y-flip

Correlation α+β+γ+δ α-β+γ-δ α-β-γ+δ α+β-γ-δ


Applications2D (Rotation):

If we are given a function f defined on the set of points inside the unit disk (x2+y21), we can express the function in terms of radius and angle:

)sin,cos(),(~

rrfrf

rr



If we hold the radius fixed, we get a function defined on a circle:

and we can apply methods from functions on a circle to obtain rotation invariants and to correlate.

rrrffr )sin,cos()(

rr



To get rotation invariants, we can express the initial function f as a collection of circular functions, obtained by restricting f to different radii:

Nk

rfff krr N )(,),(

1

rr



Computing the Fourier decomposition of each circular restriction:

we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different circular restrictions:

ik

kkrr eaf ,)(

,...,,,...,,, 1,1,0,1,1,0, 222111 rrrrrr aaaaaaf



To correlate two functions f and h, we can express the initial functions as collections of circular functions, obtained by restricting to different radii:

)(,),()(,),(11

NN rrrr hhhfff



Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions:

Complexity:1. 2N forward Fourier Transforms: O(N2 logN)

2. Frequency multiplication: O(N2)

3. One inverse Fourier Transform: O(N logN)

2/

2/ 1,,

1

2/

2/,,),(),,(

N

Nk

ikN

lkrkr

N

l

N

Nk

ikkrkr ebaebarhrf

llll


Applications2D (Axial Flips):

Given two functions f and h defined on the plane, we can express the two functions in terms of the Fourier decomposition of their radial restrictions:

with ar,k,br,k,cr,k,dr,kℝ.

k

ikkrkrr

k

ikkrkrr eidcheibaf ,,,, )()(



By computing the different summations independently:

the correlation at all four axial flips can be computed with only 16 extra arithmetic operations.

k

N

lkrkr

k

N

lkrkr

k

N

lkrkr

k

N

lkrkr llllllll

dbcadbcahf1

12,12,1

12,12,1

2,2,1

2,2,)(),(

α(real, even)

β(imaginary, even)

γ(real, odd)

δ(imaginary, odd)


ApplicationsSphere:

If we have a real-valued function f on the sphere, we can express the function in terms of its spherical harmonic decomposition:

where alm,blmℝ, al-m=(-1)malm, and bl-m=(-1)mblm.

l

l

lm

ml

ml

ml Yibaf ),(),(


ApplicationsSphere:

Given the space of real-valued functions on the sphere, the spherical harmonic decomposition can be used for correlation and invariants-extraction with respect to the group of 3D rotations / reflections.


ApplicationsSphere:

What if we consider the smaller group of axial flips:

that arise due to the ambiguity in PCA alignment?

100

010

001

G


ApplicationsSphere:

Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.


ApplicationsSphere:

Using the facts that:

and the harmonic of frequency l are even (resp. odd) when l is even (resp. odd), we get:

ml

ml

l

l

lm

ml

ml

ml

mlml

l

l

lm

ml

ml

l

ml

mml

l

l

lm

ml

ml

ml

ml

l

l

lm

ml

ml

aaYay

aaYax

aaYaxy

aaYa

flip-z

flip-z

flip-

Initial

),(

)1(),()1(

)1(),(

),(

imml

ml eP

mlmll

Y cos)!(

)!(

4

12,


ApplicationsSphere:

An axial flip invariant representation can be obtained by computing the spherical harmonic decomposition:

and separately storing the norms of the real and imaginary components of the harmonic coefficients:

l

l

lm

ml

ml

ml Yibaf ),(),(

22

22

12

12

02

02

12

12

22

22

11

11

01

01

11

11

00

00

,,,,,,,,,

,,,,,

,

)(bababababa

bababa

ba

f


ApplicationsSphere:

and in a similar manner as before, we can compute the correlation at all eight axial flips with only 64 extra arithmetic operations.



If we are given a function f defined on the set of points inside the unit disk (x2+y2+z21), we can express the function in terms of radius and angle:

)sinsin,cos,cossin(),,(~

rrrfrf



If we hold the radius fixed, we get a function defined on a sphere:

and we can apply methods from functions on a sphere to obtain rotation invariants and to correlate.

rrrrffr )sinsin,cos,cossin(),(



To get rotation invariants, we can express the initial function f as a collection of spherical functions, obtained by restricting f to different radii:

Nk

rfff krr N )(,),(

1



Computing the spherical harmonic decomposition of each spherical restriction:

we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different spherical restrictions.

l

l

lm

ml

mlrr Yaf ),(),( ,



To correlate two functions f and h, we can express the initial functions as a collection of spherical functions, obtained by restricting to different radii: ),(,),,(),(,),,(

11

NN rrrr hhhfff



Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions:

Complexity:1. 2N forward spherical harmonic transforms: O(N3 log2N)

2. N intra-frequency multiplication: O(N4)

3. One inverse Wigner-D transform: O(N4)

2/

2/',

', 1

',, )()(,

N

Nl

lmm

l

lmm

N

l

mlr

mlr RDbahRf

ll



Given two functions f and h defined in 3D, we can express the two functions in terms of the spherical harmonic decomposition of their radial restrictions and obtain a method that computes the correlation at each axial flip with only 64 extra arithmetic operations.


ApplicationsSummary:

Translation Rotation Axial Flip

Circle 1D Fourier Transform Real/Imaginary Even/Odd Fourier Transform

2D 2D Fourier Transform Circular Restrictions1D Fourier Transform

Circular RestrictionsReal/Imaginary Even/Odd Fourier Transform

Sphere Spherical Harmonics Wigner-D Transform

Real/Imaginary Even/Odd Spherical Harmonic Transform

3D 3D Fourier Transform Spherical RestrictionsSpherical HarmonicsWigner-D Transform

Spherical RestrictionsReal/Imaginary Even/Odd Spherical Harmonic Transform


– Shape Histograms (Shells)

– Shape Distributions (D2)

– Extended Gaussian Images (EGI)?


Shape Histograms (Shells)Obtain a rotation invariant by storing the amount of the shape that resides within each spherical shell.

Model Shape Histogram (Shells)


Shape Histograms (Shells)This amounts to storing the constant order component of a spherical restriction.

We could get a more discriminating descriptor by storing other frequency information.


D2 DistributionsObtain a translation and rotation invariant by storing the distribution of distances between pairs of points on the model.

D2 DistributionD2 Distribution3D Model3D Model

p

q

DistanceD

istr

ibut i

on


D2 DistributionsWe can decompose the computation of the D2 descriptor into two steps:

– Voting on displacement

– Computing the average over spherical restrictions


D2 DistributionsStep 1:

For each pair (p,q) of surface points– Vote on bin corresponding to the vector p-q

3D Model3D Model

p

q


D2 DistributionsStep 2:

Compute the distance distribution by counting up the number of points in each spherical restriction.

D2 DistributionD2 DistributionDistance

Dis

trib

ut i

on


D2 DistributionsRotation Invariance:

Obtained by storing the amount of information in the constant order component.

More rotation invariant information can be obtained by storing the norms of the non-constant frequencies as well.


D2 DistributionsTranslation Invariance:

The value in a bin can be obtained by translating the model and counting the amount of overlap between the original model and the translated one.



Set f(x,y,z) to be the function that is equal to 1 at points on the boundary and 0 everywhere else.

Then the function:

is equal to 1 if and only if (x,y,z) and (x-x0,y-y0,z-z0) are both on the boundary, otherwise the value is 0.

),,(),,( 000 zzyyxxfzyxf



For a fixed offset (x0,y0,z0), the number of points which satisfy the property that both the point (x,y,z) and the point (x-x0,y-y0,z-z0) are on the boundary is equal to the number of points at which:

is equal to 1.

),,(),,( 000 zzyyxxfzyxf



Thus the number of points (x,y,z) which satisfy the property that both (x,y,z) and (x-x0,y-y0,z-z0) are on the boundary is equal:

),,(),,,(

),,(),,(),,(

000

000000

zzyyxxfzyxf

dxdydzzzyyxxfzyxfzyx

Bin



If we write out f in terms of its frequency decomposition:

the auto-correlation is:

mlk

mzlykximlk eazyxf

,,

)(,,),,(

mlk

mzlykximlk eazyx

,,

)(2

,,),,(Bin



So translation invariance is obtained by representing the frequency components by their (square) norms.

mlk

mzlykximlk eazyx

,,

)(2

,,),,(Bin

Documents

Signal Processing and Representation Theory Lecture 4