Anti-Aliasing Jian Huang, CS594, Fall 2008 This set of slides references our text book and the...

Anti-Aliasing

Jian Huang, CS594, Fall 2008

This set of slides references our text book and the slides used at Ohio State by Prof. Roger Crawfis.

Aliasing?

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Aliasing

• Aliasing comes from in-adequate sampling rates of the continuous signal

• The theoretical foundation of anti-aliasing has to do with frequency analysis

• It’s always easier to look at 1D cases, so let’s first look at a few of those.

Example of Sampling

Examples of Inadequate Sampling

Cosine Integrations

• Consider these formulas

)3sin(6

1

2

)sin()cos()2cos(

)2sin(4

1

2)(cos

)sin()cos(

2

xx

dxxx

xx

dxx

xdxx

+=

+=

=

∫

∫

∫

Improper Cosine Integrals

• Evaluating from (-,)

0)cos()2cos(

)(cos

0)cos(

2

=

∞=

=

∫

∫

∫

∞

∞−

∞

∞−

∞

∞−

dxxx

dxx

dxx

Fourier Transform

• The Fourier Transform represents a periodic function as a continuous summation of sin’s and cos’s.

)Im()Re()(

)sin()(

),(),(

)()(

)(2

2

uiuuF

xjxcoxe

dxdyeyxfvuF

dxexfuF

jx

vyuxj

uxj

+=+=

=

=+−

−

∫∫∫

π

π

Fourier Transform

• Note:

• This is also called the direct current or DC component.

∫∞

∞−

= dxxfF )()0(

Inverse Fourier Transform

• We can get back our original function f(x) from F(u) using the inverse transform:

∫∞

∞−

= dueuFxf uxj π2)()(

OK – Why do this?

• Fourier space is a very good space for analyzing and understanding our signals.

• Rarely ever want to transform to Fourier space.

• There are some great theories developed in terms of sampling and convolution.

Fourier Analysis

• By looking at F(u), we get a feel for the “frequencies” of the signal.

• We also call this frequency space.• Intuitively, you can envision, the sharper an

edge, the higher the frequencies.• From a numerical analysis standpoint, the

sharper the edge the greater the tangent magnitude, and hence the interpolation errors.

Fourier Analysis

• Bandlimited– We say a function is bandlimited, if F(u)=0

for all frequencies u>c and u<-c.

• Amplitude Spectrum– The magnitude, |F(u)|, is called the

amplitude spectrum or simply the spectrum.

• Phase Spectrum or Phase

))Re(

)Im((tan)( 1

u

uu −=Φ

Fourier Properties

• Linearity

• Scaling

)()()()( ubGuaFxbgxaf +⇔+

)(1

)(a

uF

aaxf ⇔

Convolution

• Definition:

∫∞

∞−

−=⊗ dtxthtfxhxf )()()()(

Convolution

• Pictorially

f(x)

h(x)

Convolution

f(t)

x

h(t-x)

Convolution

• Consider the function (box filter):

⎪⎩

⎪⎨

⎧

>≤≤−

−<=

21

21

21

21

0

1

0

)(

x

x

x

xh

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This function windows our function f(x).

f(t)

Convolution

• This particular convolution smooths out some of the high frequencies in f(x).

f(x)g(x) f(t)

Another Look At Convolution

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Filtering and Convolution

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Different functions

achieve different

Results.

Impulse Function

• Consider the special function (called impulse):

such that,

⎩⎨⎧

=≠

=0

00)(

xa

xxδ

1)( =∫∞

∞−

dxxδ

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Impulse and Convolution

• Then, if we take the convolution of f(x) with δ(x), we get:

f(x)δ(x) = f(x)

Sampling Function

• A Sampling Function or Impulse Train is defined by:

where T is the sample spacing.

∑∞

−∞=

−=k

T kTxxS )()( δ

T

Sampling Function

• The Fourier Transform of the Sampling Function is itself a sampling function.

• The sample spacing is the inverse.

)()( 1 uSxST

T ⇔

Convolution Theorem

• The convolution theorem states that convolution in the spatial domain is equivalent to multiplication in the frequency domain, and vica versa.

)()()()(

)()()()(

uGuFxgsf

uGuFxgxf

⊗⇔⇔⊗

Convolution Theorem

• This powerful theorem can illustrate the problems with our point sampling and provide guidance on avoiding aliasing.

• Consider: f(x) ST(x)

f(t)

T

Convolution Theorem

• What does this look like in the Fourier domain?

F(u)

T1

S(u)

Convolution Theorem

• In Fourier domain we would convolve

F(u)

T1

S(u)

S(u)F(u)

Aliasing

• What this says, is that any frequencies greater than a certain amount will appear intermixed with other frequencies.

• In particular, the higher frequencies for the copy at 1/T intermix with the low frequencies centered at the origin.

Aliasing and Sampling

• Note, that the sampling process introduces frequencies out to infinity.

• We have also lost the function f(x), and now have only the discrete samples.

• This brings us to our next powerful theory.

Sampling Theorem

• The Shannon Sampling TheoremA band-limited signal f(x), with a cutoff frequency of

, that is sampled with a sampling spacing of T may be perfectly reconstructed from the discrete values f[nT] by convolution with the sinc(x) function, provided:

is called the Nyquist limit.T2

1<

Sampling Theory

• Why is this?

• The Nyquist limit will ensure that the copies of F(u) do not overlap in the frequency domain.

• I can completely reconstruct or determine f(x) from F(u) using the Inverse Fourier Transform.

Sampling Theory

• In order to do this, I need to remove all of the shifted copies of F(u) first.

• This is done by simply multiplying F(u) by a box function of width 2.

F(u)

T1

S(u)

Sampling Theory

• In order to do this, I need to remove all of the shifted copies of F(u) first.

• This is done by simply multiplying F(u) by a box function of width 2.

F(u)S(u)

T1

General ProcessOriginal function Sampled function

ReconstructedFunction

Acquisition

Reconstructio

n

Re-sampled function

Resampling

How? - ConvolutionSpatial Domain:

Mathematically:f(x)*h(x) =

( ) ( )dttxgtf −∫∞

∞−

Frequency Domain:

( ) ( )ω⋅ω HF

Evaluated at discrete points (sum)

Multiplication:Convolution:

ReconstructionMathematically:f(x)*h(x) = (f[i])*h(x)

f[i] h(x)

General Process - Frequency Domain

Acquisition

Reconstructio

n

Resampling

Original function Sampled function

ReconstructedFunction

Re-sampled function

Sampling

• What we have in computer graphics is a point sampling of our scene, or:– I(x) = f(x)•ST(x)

• What we would like is more of an integration across the pixel (or larger area):– I(x) = f(x) h(x)

• What should h(x) be?

Pre-Filtering

Pre-Filtering

Acquisition

Reconstruction

Original function Band-limited function

SampledFunction

Reconstructed function

Ideal Reconstruction with Sinc function

Spatial Domain:• convolution is exact

Frequency Domain:cut off freq. replica

( ) ( ) 0=− xfxfr ( ) ( )x

xx

π

π=

sinSinc

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-25 -20 -15 -10 -5 0 5 10 15 20 250.65

0.7

0.75

0.8

0.85

0.9

0.95

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Reconstructing DerivativesSpatial Domain:

• convolution is exactFrequency Domain:cut off freq. replica

( ) ( ) 0=′− xfxf dr ( ) ( ) ( )2

sincosCosc

x

x

x

xx

π

π−

π=

-1.5

-1

-0.5

0

0.5

1

1.5

-25 -20 -15 -10 -5 0 5 10 15 20 25

Cosc(t)

0.65

0.7

0.75

0.8

0.85

0.9

0.95

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Possible Errors• Post-aliasing

– reconstruction filter passes frequencies beyond the Nyquist frequency (of duplicated frequency spectrum) => frequency components of the original signal appear in the reconstructed signal at different frequencies

• Smoothing– frequencies below the Nyquist frequency are attenuated

• Ringing (overshoot)– occurs when trying to sample/reconstruct discontinuity

• Anisotropy– caused by not spherically symmetric filters

How Good? = ErrorSpatial Domain:

• local error• asymptotic error• numerical error

Frequency Domain:global errorvisual appearanceblurringaliasingsmoothing

ApproximationTheory/Analysis Signal

Processing

Sources of Aliasing• Non-bandlimited signal

Low sampling rate (below Nyquist)

Non perfect reconstruction

Reconstruction Kernels

stop bandpass band

Smoothing error

Postaliasing error

Ideal filter

filter

The spatial extent of reconstruction kernels, or interpolation basis functions, depend on the cut-off frequency as well.

Reconstruction Kernels• Nearest Neighbor (Box)

00.20.40.60.8

1

-6 -4 -2 0 2 4 60

0.20.40.60.8

1

-6 -4 -2 0 2 4 6

Triangular func

Sinc

Gaussian

+ many others Spatial

d.Frequency d.

Higher Dimensions• An-isotropic Filters• (radially symmetric)

separable filters

( ) ( ) ( )yhxhyxh ⋅=,( ) ( )22, yxhyxh +=

Interpolation (an example)• Very important; regardless of algorithm• expensive => done very often for one image• Requirements for good reconstruction

– performance– stability of the numerical algorithm– accuracy

Nearestneighbor

Linear

Put Things in Perspective• In graphics, need to use continuous space functions.

But can only work with discrete data. So, let’s reconstruct from discrete data to continuous space (convolution) and resample

• Interpolation is doing the same thing. Computing one data point in the resulting function, say, at x1.

• So, which reconstruction kernel (basis function) does linear/bilinear/tri-linear interpolations use?

A Cosine Example

• Consider the function f(x)=cos(2πx).

x

u

F(u)

f(x)

Sampling Theory

• So, given f[nT] and an assumption that f(x) does not have frequencies greater than 1/2T, we can write the formula:

f[nT] = f(x) ST(x) F(u) ST(u)

F(u) = (F(u) ST(u)) Box1/2T(u)

therefore,f(x) = f[nT] sinc(x)

A Cosine Example

• Now sample it at T=½

x

u

f(x)

1

F(u)

A Cosine Example

• Problem:– The amplitude is now wrong or undefined.

• Note however, that there is one and only one cosine with a frequency less than or equal to 1 that goes through the sample pts.

u1

F(u)

A Cosine Example

• What if we sample at T=¾?

x

u

f(x)

1

F(u)

)3

2cos( x

π

Supersampling

• Supersampling increases the sampling rate, and then integrates or convolves with a box filter, which is finally followed by the output sampling function.

x

f(x)

Sampling and Anti-aliasing

• If you can not get rid of it, convert it to noise.

Basic checkerboard Checkerboard with noise

Sampling and Anti-aliasing

• The images were calculated as follows:– A 2Kx2K image was constructed

and smoothly rotated into 3D.– For Uniform Sampling, it was

downsampled toa 512x512 image.

– Noise was added to the image, sharpened and thendownsampled for the other one.

– Both were converted to B&W.

Sampling and Anti-aliasing

• The problem:– The signal is not band-limited.– Uniform sampling can pick-up higher

frequency patterns and represent them as low-frequency patterns.

F(u)

T1

S(u)

Sampling and Anti-aliasing

• Turning the high-frequencies into noise.

• Recall, multiplication by the sampling function is equivalent to convolution in the Fourier domain.

• Fourier transform of noise.

Non-uniform Sampling

Quality considerations

• So far we just mapped one point• results in bad aliasing (resampling

problems)• we really need to integrate over

polygon• super-sampling is not a very good

solution (slow!)• most popular (easiest) - mipmaps

Quality considerations

• Pixel area maps to “weird” (warped) shape in texture space

pixel

u

v

xs

ys

Quality considerations

• We need to:– Calculate (or approximate) the

integral of the texture function under this area

– Approximate:• Convolve with a wide filter around the

center of this area• Calculate the integral for a similar (but

simpler) area.

Quality considerations

• the area is typically approxiated by a rectangular region (found to be good enough for most applications)

• filter is typically a box/averaging filter - other possibilities

• how can we pre-compute this?

Mip-maps

• An image-pyramid is built.256 pixels 128 64 32 16 8 4 2 1

Mip-maps

• Find level of the mip-map where the area of each mip-map pixel is closest to the area of the mapped pixel.

pixel

u

v

xs

ys

2x2 pixels level selected

Mip-maps

• Pros– Easy to calculate:

• Calculate pixels area in texture space• Determine mip-map level• Sample or interpolate to get color

• Cons– Area not very close – restricted to square

shapes (64x64 is far away from 128x128). – Location of area is not very tight.

Summed Area Table (SAT)

• Use an axis aligned rectangle, rather than a square

• Precompute the sum of all texels to the left and below for each texel location– For texel (u,v), replace it with:

sum (texels(i=0…u,j=0…v))

Summed Area Table (SAT)

• Determining the rectangle:– Find bounding box and calculate its aspect

ratio

pixel

u

v

xs

ys

Summed Area Table (SAT)

• Determine the rectangle with the same aspect ratio as the bounding box and the same area as the pixel mapping.

pixel

u

v

xs

ys

Summed Area Table (SAT)

• Center this rectangle around the bounding box center.

• Formula:• Area = aspect_ratio*x*x• Solve for x – the width of the rectangle

• Other derivations are also possible using the aspects of the diagonals, …

Summed Area Table (SAT)

• Calculating the color– We want the average of the texel colors

within this rectangle

u

v+

+ -

-

(u3,v3)

(u2,v2)(u1,v1)

(u4,v4)

+ -

+-

Summed Area Table (SAT)

• To get the average, we need to divide by the number of texels falling in the rectangle.– Color = SAT(u3,v3)-SAT(u4,v4)-SAT(u2,v2)+SAT(u1,v1)– Color = Color / ( (u3-u1)*(v3-v1) )

• This implies that the values for each texel may be very large:– For 8-bit colors, we could have a maximum SAT value of

255*nx*ny– 32-bit pixels would handle a 4kx4k texture with 8-bit values.– RGB images imply 12-bytes per pixel.

Summed Area Table (SAT)

• Pros– Still relatively simple

• Calculate four corners of rectangle• 4 look-ups, 5 additions, 1 mult and 1 divide.

– Better fit to area shape– Better overlap

• Cons– Large texel SAT values needed– Still not a perfect fit to the mapped pixel.

Elliptical Weighted Average (EWA) Filter

• Treat each pixel as circular, rather than square.

• Mapping of a circle is elliptical in texel space.

pixel

u

v

xs

ys

EWA Filter

• Precompute?

• Can use a better filter than a box filter.

• Heckbert chooses a Gaussian filter.

EWA Filter

• Calculating the Ellipse

• Scan converting the Ellipse

• Determining the final color (normalizing the value or dividing by the weighted area).

EWA Filter

• Calculating the ellipse– We have a circular function defined in (x,y).– Filtering that in texture space h(u,v).– (u,v) = T(x,y)– Filter: h(T(x,y))

EWA Filter

• Ellipse: (u,v) = Au2 + Buv + Cv2 = F– (u,v) = (0,0) at center of the ellipse

• A = vx2 +vy

2

• B = -2(uxvy + uyvx)

• C = ux2 +uy

2

• F = uxvy + uyvx

EWA Filter

• Scan converting the ellipse:– Determine the bounding box– Scan convert the pixels within it,

calculating (u,v).– If (u,v) < F, weight the underlying texture

value by the filter kernel and add to the sum.

– Also, sum up the filter kernel values within the ellipse.

EWA Filter

• Determining the final color– Divide the weighted sum of texture values

by the sum of the filter weights.

EWA Filter

• What about large areas?– If m pixels fall within the bounding box of the

ellipse, then we have O(n2m) algorithm for an nxn image.

– m maybe rather large.

• We can apply this on a mip-map pyramid,rather than the full detailed image.– Tighter-fit of the mapped pixel– Cross between a box filter and gaussian filter.– Constant complexity - O(n2)