Image ProcessingImage ProcessingIB Paper 8 – Part AIB Paper 8 – Part A

Ognjen ArandjeloviOgnjen Arandjelovićć

http://mi.eng.cam.ac.uk/~oa214/

Lecture RoadmapLecture Roadmap

Face geometry Lecture 1:

Geometric image transformations

Lecture 2:

Colour and brightness enhancement

Lecture 3:

Denoising and image filtering

Lecture 4:

Cross-section through out-of-syllabus techniques

Filter Design – Matched FiltersFilter Design – Matched Filters

Consider the convolution sum of a discrete signal with a particular filter:

When is the filter response maximal?

… 234 233 228 240 241 241 …

1 2 2 1 0 01 2 2 1 0 0

228+ 480+482+ 241 + …

Filter Design – Matched FiltersFilter Design – Matched Filters

The summation is the same as for vector dot product:

The response is thus maximal when the two vectors are parallel i.e. when the filter matches the local patch it overlaps.

… 234 233 228 240 241 241 …

1 2 2 1 0 0

Filter Design – Intensity DiscontinuitiesFilter Design – Intensity Discontinuities

Using the observation that maximal filter response is exhibited when the filter matches the overlapping signal, we can start designing more complex filters:

Kernel with maximal response to intensity edges

0.5 0.0 -0.5

Filter Design – Intensity DiscontinuitiesFilter Design – Intensity Discontinuities

Better yet, perform Gaussian smoothing to suppress noise first:

Noise suppressing kernel with high response to intensity edges

Gaussian kernel

Unsharp Masking EnhancementUnsharp Masking Enhancement

The main principle of unsharp masking is to extract high frequency information and add it onto the original image to enhance edges:

image

HPF

+ output

Original edge Enhanced

Laplacian of Gaussian (LoG) FilterLaplacian of Gaussian (LoG) Filter

The Laplacian of Gaussian is an isotropic kernel that responds maximally to changes in the 2nd derivative:

1D Laplacian of Gaussian 2D LoG as a surface 2D LoG as an image

2D Laplacian of Gaussian:

Laplacian of Gaussian (LoG) FilterLaplacian of Gaussian (LoG) Filter

The response of the 1D Laplacian of Gaussian filter to an edge:

-

Signal (edge) LoG filter Filter output

Unsharp Masking EnhancementUnsharp Masking Enhancement

Unsharp mask filtering performs noise reduction and edge enhancement in one go, by combining a Gaussian LPF with a Laplacian of Gaussian kernel:

Gaussian smoothing Convolution with –ve Laplacian of Gaussian

+ =

Result

Unsharp Masking – ExampleUnsharp Masking – Example

Consider the following synthetic example:

Gaussian smoothed then corrupted with Gaussian noise

Unsharp Masking – ExampleUnsharp Masking – Example

After unsharp masking:

Gaussian smoothed then corrupted with Gaussian noise

– – Distance Transform –Distance Transform –

Motivation – Toy ProblemMotivation – Toy Problem

The problem: produce output image with higher pixel value indicating higher level of belief that a roughly square polygon of edge 225 is centered at it:

225

Matched Filtering May Be?Matched Filtering May Be?

Given the material covered in the previous lecture, you may be tempted to create a matched filter:

225

Matched Filtering May Be?Matched Filtering May Be?

Here is the output of convolving the filter with the example image:

A rather ugly looking result with too sharp discontinuities (i.e. low robustness to small deformations in shape, angle or thickness)

Distance TransformDistance Transform

Rather, compute the distance transformed image – each pixel value indicates the minimal distance of that pixel to the nearest edge in the original image:

Original image Distance transformed

The result is far better looking!

Result after Distance TransformResult after Distance Transform

Consider now the result of convolving our matched filter with the distance transformed image:

– – Nonlinear Denoising –Nonlinear Denoising –

Salt and Pepper NoiseSalt and Pepper Noise

Consider an image synthetically corrupted with salt and pepper noise:

“Salt”(bright)

“Pepper”(dark)

Salt and Pepper NoiseSalt and Pepper Noise

Here is the result of denoising attempt using a Gaussian low-pass filter:

Median FilterMedian Filter

Median filter replaces the old pixel value by the median of its neighbourhood:

0 91 92 93 93 97 108

± 3 pixel neighbourhood (sorted)

MedianOriginal value

Median Filter – 1D ExampleMedian Filter – 1D Example

The result of applying the median filter (with neighbourhood of size 7) on the corrupted 1D signal:

Median Filter – 2D ExampleMedian Filter – 2D Example

The result of applying the median filter (mask size 3 х 3) on the synthetically corrupted image:

No edge smoothing

Noise virtually entirely removed

Filter ComparisonFilter Comparison

The advantages of the median filter are easily seen when considering the difference to the ground truth:

Gaussian denoising Median filtering

RMS difference: 15 (from 20) RMS difference: 5 (from 20)

– – De-Convolution –De-Convolution –

An Example ProblemAn Example Problem

Consider an image of a car plate acquired by a speed control camera:

The plate is entirely unreadable due to motion blur

Is it possible to somehow enhance this image to the level that the plate number can be read off?

Image Formation ModelImage Formation Model

Assuming constant car velocity* the motion blur is caused by simple spatial averaging in the direction of apparent velocity.

As before, this is equivalent to convolving the original, sharp image with a simple pulse function.

* A reasonable assumption, given that the exposure is relatively short.

Recovery AlgorithmRecovery Algorithm

Given an estimate of the car velocity and our image formation model suggests the following algorithm:

2D FourierTransform

2D FourierTransform/De-blurred

result

Degradation model

The ResultThe Result

Using the pulse width of 15 pixels produces the following de-blurred result:

RE03 TGZ

– – Super-Resolution –Super-Resolution –

What is Image Super-ResolutionWhat is Image Super-Resolution

Given one or more low-resolution (LR) images, produce an enhanced, high-resolution (HR) image.

Observation model:

Observed LR image“True” image

Noise

Transformation (geometric, photometric…)

SR via Non-Uniform InterpolationSR via Non-Uniform Interpolation

One of the simplest forms of super-resolution takes on the form of interpolation from non-aligned samples:

Not quite the

same

Example – 2x Sampling FrequencyExample – 2x Sampling Frequency

Non-uniforminterpolation

Simplescaling

– – That is All for Today –That is All for Today –