Adaptive DenoisingAdaptive Denoisingfor Video Compressionfor Video Compression

Eren SoyakEECS 463

Winter 2006

Northwestern University

Video Compression and You

Demand for video where no video has gone before

Source You

Source

Source

Video Compression and You

Source You

Source

Source

Encode Medium

Decode

Demand for video where no video has gone before

Video Compression and You

Demand for video where no video has gone before

Source You

Source

Source

Encode Channel

Decode

Pos t proces sing

Preproces sing

Video and Compression Video compression works by identifying and

exploiting redundancy in source video The more information there is in the source,

the more difficult it is to compress into a smaller form

Foreman

Foreman.264

Noise and Compression Noise is usually present in source video due to

various reasons (capture, film grain, quantization, transmission errors etc)

Wide spectrum noise is very difficult to compress

The ever-popular AWGN-type noise

Deprecated old analog-type noise

Dealing with Noise Pre/post filtering methods very useful Simple denoising method: averaging filter

3 pels 5 pels 7 pels

Good, Bad and Ugly Denoising Denoising must distinguish between original signal

and noise, filter out only the noise. Prediction of the noise and/or the original video is usually required for this.

Smoothing, edge loss and blurring are all undesirable

Despeckle

“Smart” blur

10 pel average

Case Study: AWGN Additive White Gaussian Noise (AWGN) can be

introduced by capture devices, especially due to poor lighting and sometimes weather.

AWGN breaks most compression algorithms. Consider signal independent AWGN.

Foreman + AWGN

Advanced Denoising (Wiener) The Wiener filter is commonly used by the

ambitious for generic denoising. Requires little information about noise. Few “catastrophic” corner cases.

Wiener(Foreman + AWGN)

Global Denoising Issues The visibility (and usually compression hindrance)

of noise is a function of the source even if the severity of the noise itself is not – noise is more visible on smooth regions as opposed to texture.

It would be highly desirable to filter noise such that the final video retains local shape/texture characteristics as well.

Adaptive methods begin to suggest themselves.

LMMSE Filtering Linear Minimum Mean Squared Error filter (IIR)

1 2i i212121 = ˆ -i)-i, n)g(n, ih(i), n(ns

Noisy image

Impulse response

LMMSE estimate of ideal image s(n1, n2)

(1)

The Unrealizable Wiener Filter The principle of orthogonality states that the

estimation error s(n1, n2)- (n1, n2) should be orthogonal to every sample of the observed image.

s

0

ˆ =

, ˆ

212121

212121

), kg(k), n(ns), ns(nE

), kg(k), n(ns), ns(n(2)

The Impossible Wiener IR Substituting (1) into (2) and simplifying we can

express the the impulse response of the filter as a 2D convolution

Is impossible to realize since infinite time is required before an output sample is computed.

), n(nR), n(nR), nh(n sggg 212121 =**

“Discrete Wiener-Hopf equation”

autocorrelation of observations

cross correlation between ideal and observed image

Adaptive LMMSE Kuan et al. proposed in ’85:

), n(n)-μ, ng(nσ), n(nσ

) , n(nσ) + , n(n) = μ, n(ns g

vs

sg 21212

212

212

2121ˆ

), n(ns 21ˆ

), n(nμg 21

) , n(nσs 212

), ng(n 21

2vσ

= observation

= filtered output = local variance

= estimated noise variance

= local mean

Adaptive LMMSE Performance Given its adaptive nature to local image properties

the filter is better at preserving edges/texture while removing noise.

It is very process-intensive and sensitive to misestimation of noise variance.

Adaptive LMMSE(Foreman + AWGN)

Comparing Filter Outputs

Comparing Filter Outputs

Adaptive LMMSEWiener

Comparing Compressed Video Compressed at 512 kbps at H.264 Main Profile

Wiener

Adaptive LMMSE

Weighed Adaptive LMMSE Directionally weighed variance matrix

May better account for edges due to 2D direction component

Choice of weight matrix could be optimized

1 2 1

2 3 2

1 2 1

Weighed Adaptive LMMSE Prone to blurring if matrix weights poorly chosen..

Poorly Weighed Adaptive LMMSE(Foreman + AWGN)

Bibliography A. Murat Tekalp, Digital Video Processing, ‘95 J. S. Lim, Two Dimensional Signal and Image

Processing, ‘90 D.T. Kuan, A.A. Sawchuk, T.C. Strand, P. Chavel,

Adaptive noise smoothing filter for images with signal-dependent noise, ‘85