• Home

  • Documents

  • Introduction and Overview - University of California, San DiegoScaling function, multiresolution...
128
Final Review Image Processing CSE 166 Lecture 18

Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Final Review

Image Processing

CSE 166

Lecture 18

Page 2: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Topics covered

• Basis vectors• Matrix-based transforms• Basis images• Wavelet transform• Image compression• Image watermarking• Morphological image processing• Edge detection• Segmentation

CSE 166, Spring 2019 2

Page 3: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms

CSE 166, Spring 2019 3

Forward transform

Inverse transform

where

Page 4: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

General inverse transform using basis vectors

CSE 166, Spring 2019 4

Page 5: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms using orthonormal basis vectors

• In vector form

CSE 166, Spring 2019 5

Page 6: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms using orthonormal basis vectors

• In matrix form

CSE 166, Spring 2019 6

Page 7: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms using biorthonormal basis vectors

• In matrix form

CSE 166, Spring 2019 7

Page 8: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transform

CSE 166, Spring 2019 8

Example: 8-point DFT of f(x) = sin(2πx)

real part + imaginary part

Page 9: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms

• Discrete Fourier transform (DFT)• Discrete Hartley transform (DHT)• Discrete cosine transform (DCT)• Discrete sine transform (DST)• Walsh-Hadamard (WHT)• Slant (SLT)• Haar (HAAR)• Daubechies (DB4)• Biorthogonal B-spline (BIOR3.1)

CSE 166, Spring 2019 9

Page 10: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis vectors of matrix-based 1D transforms

CSE 166, Spring 2019 10

N = 16

real part

imaginary part

Standard basis(for reference)

Page 11: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis vectors of matrix-based 1D transforms

CSE 166, Spring 2019 11

N = 16

basis dual Standard basis(for reference)

Page 12: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transformsin two dimensions

CSE 166, Spring 2019 12

Forward transform

Inverse transform

where

Page 13: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transformsin two dimensions

• If r and s are separable and symmetric, and M = N, then

– For orthonormal basis vectors

– For biorthonormal basis vectors

CSE 166, Spring 2019 13

Page 14: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Matrix-based transforms in two dimensions using basis images

• Inverse transform

CSE 166, Spring 2019 14

where

Each Su,v is a basis image

Page 15: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 15

Standard basis images (for reference)

8-by-8 array of8-by-8

basis images

Page 16: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 16

Discrete Fourier transform (DFT) basis images

real part imaginary part

Page 17: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 17

Discrete Hartley transform (DHT) basis images

Page 18: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 18

Discrete cosine transform (DCT) basis images

Page 19: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 19

Discrete sine transform (DST) basis images

Page 20: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 20

Walsh-Hadamard transform (WHT) basis images

Page 21: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 21

Slant transform (SLT) basis images

Page 22: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basis images of matrix-based 2D transforms

CSE 166, Spring 2019 22

Haar transform (HAAR) basis images

Page 23: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Wavelet transforms

• A scaling function is used to create a series of approximations of a function or image, each differing by a factor of 2 in resolution from its nearest neighboring approximations.

• Wavelet functions (wavelets) are then used to encode the differences between adjacent approximations.

• The discrete wavelet transform (DWT) uses those wavelets, together with a single scaling function, to represent a function or image as a linear combination of the wavelets and scaling function.

CSE 166, Spring 2019 23

Page 24: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Scaling functions and set of basis vectors

• Father scaling function

• Set of basis functions

– Integer translation k

– Binary scaling j

• Basis of the function space spanned by

CSE 166, Spring 2019 24

Page 25: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Scaling function, multiresolution analysis

1. The scaling function is orthogonal to its integer translates.

2. The function spaces spanned by the scaling function at low scales are nested within those spanned at higher scales.𝑉−∞⊂⋯⊂ 𝑉−1⊂ 𝑉0⊂ 𝑉1⊂⋯⊂ 𝑉∞

3. The only function representable at every scale (all 𝑉𝑗) is f(x) = 0.

4. All measureable, square-integrable functions can be represented as 𝑗 → ∞

CSE 166, Spring 2019 25

Page 26: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

• Given father scaling function , there exists a mother wavelet function whose integer translations and binary scalings

span the difference between any two adjacent scaling spaces

• The orthogonal compliment of

Wavelet functions

CSE 166, Spring 2019 26

Page 27: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Relationship between scaling and wavelet function spaces

CSE 166, Spring 2019 27

Union

Page 28: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Scaling function coefficients and wavelet function coefficients

• Refinement (or dilation) equation

where are scaling function coefficients

• And

where are wavelet function coefficients

• Relationship

CSE 166, Spring 2019 28

Page 29: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

1D discrete wavelet transform

• Forward

• Inverse

CSE 166, Spring 2019 29

Approximation

Details

Page 30: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

2D discrete wavelet transform

• Forward

CSE 166, Spring 2019 30

Approximation

Details

where

Directionalwavelets

Page 31: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

2D discrete wavelet transform

• Inverse

CSE 166, Spring 2019 31

Page 32: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

2D discrete wavelet transform

CSE 166, Spring 2019 32

Decomposition

Page 33: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

2D discrete wavelet transform

CSE 166, Spring 2019 33

3-levelwavelet

decomposition

Page 34: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Wavelet-based edge detection

CSE 166, Spring 2019 34

Zero horizontal

details

Zero lowest scale

approximation

Vertical edges

Edges

Page 35: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Wavelet-based noise removal

CSE 166, Spring 2019 35

Noisy imageThreshold

details

Zero highest

resolution details

Zero details for all levels

Page 36: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Data compression

• Data redundancy

where compression ratio

CSE 166, Spring 2019 36

where

Page 37: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Data redundancy in images

CSE 166, Spring 2019 37

Coding redundancy

Spatial redundancy

Irrelevant information

Does not need all 8 bits

Information is unnecessarily

replicated

Information is not useful

Page 38: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image information

• Entropy

– It is not possible to encode input image with fewer than bits/pixel

CSE 166, Spring 2019 38

where

Page 39: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Fidelity criteria,objective (quantitative)

• Total error

• Root-mean-square error

• Mean-square signal to noise ratio (SNR)

CSE 166, Spring 2019 39

Page 40: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Fidelity criteriasubjective (qualitative)

CSE 166, Spring 2019 40

Page 41: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Approximations

CSE 166, Spring 2019 41

5.17 15.67 14.17

(a) (b) (c)

Objective (quantitative) qualityrms error (in intensity levels)

Subjective (qualitative) quality, relative

Lower is better

(a) is better than (b).

(b) is better than (c)

Page 42: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Compression system

CSE 166, Spring 2019 42

Page 43: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Compression methods

• Huffman coding• Golomb coding• Arithmetic coding• Lempel-Ziv-Welch (LZW) coding• Run-length coding• Symbol-based coding• Bit-plane coding• Block transform coding• Predictive coding• Wavelet coding

CSE 166, Spring 2019 43

Page 44: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Block-transform coding

CSE 166, Spring 2019 44

Encoder

Decoder

Page 45: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Block-transform coding

• Example: discrete cosine transform

• Inverse

CSE 166, Spring 2019 45

where

Page 46: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Block-transform coding

CSE 166, Spring 2019 46

Discrete cosine transformWalsh-Hadamard transform

4x4 subimages (4x4 basis images)

Page 47: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Block-transform coding

CSE 166, Spring 2019 47

Fourier transform

cosine transform

Walsh-Hadamardtransform

2.32 1.78 1.13rms error

Retain 32 largest

coefficients

Error image

8x8 subimages

Lower is better

Page 48: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Block-transform coding

CSE 166, Spring 2019 48

2x2 4x4 8x8

Reconstruction error versus subimage size

DCT subimage size:

Page 49: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

JPEG uses block DCT-based coding

CSE 166, Spring 2019 49

Compression reconstruction

Scaled error image

Zoomed compression

reconstruction

25:1

52:1

Compression ratio

Page 50: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Predictive coding model

CSE 166, Spring 2019 50

Encoder

Decoder

Page 51: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Predictive coding

CSE 166, Spring 2019 51

Input image

Prediction error image

Example: previous pixel coding

Histograms

Page 52: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Wavelet coding

CSE 166, Spring 2019 52

Encoder

Decoder

Page 53: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Wavelet coding

CSE 166, Spring 2019 53

Detail coefficients below 25 are truncated to zero

Page 54: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

JPEG-2000 uses wavelet-based coding

CSE 166, Spring 2019 54

Compression reconstruction

Scaled error image

Zoomed compression

reconstruction

25:1

52:1

Compression ratio

Page 55: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

JPEG-2000 uses wavelet-based coding

CSE 166, Spring 2019 55

Compression reconstruction

Scaled error image

Zoomed compression

reconstruction

75:1

105:1

Compression ratio

Page 56: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Visible watermark

CSE 166, Spring 2019 56

Original image minus watermark

Watermarked image

Watermark

Page 57: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Invisible image watermarking system

CSE 166, Spring 2019 57

Encoder

Decoder

Page 58: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Invisible watermark

CSE 166, Spring 2019 58

Two least significant

bits

JPEGcompressed

Fragile invisible

watermark

Originalimage Extracted

watermark

Example: watermarking using two least significant bits

Page 59: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Invisible watermark

CSE 166, Spring 2019 59

Watermarked images

Example: DCT-based watermarking

Extracted robust

invisible watermark

Page 60: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Reflection and translation

CSE 166, Spring 2019 60

Translation

Reflection

Page 61: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Sets of pixels: objects and structuring elements (SEs)

CSE 166, Spring 2019 61

Border of background

pixels around objects

Tight border around SE

Page 62: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Reflection about the origin

CSE 166, Spring 2019 62

Origin

Don’t care elements

Page 63: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Erosion

CSE 166, Spring 2019 63

Example: square SE

Complement of A (i.e., set of elements not in A)

Page 64: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Erosion

CSE 166, Spring 2019 64

Example: elongated SE

Page 65: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Erosion

CSE 166, Spring 2019 65

11x11

45x4515x15

Shrinks

Page 66: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Dilation

CSE 166, Spring 2019 66

Square SEExamples

Elongated SE

Page 67: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Dilation

CSE 166, Spring 2019 67

Expands

Page 68: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Opening

CSE 166, Spring 2019 68

Structuring element rolls along inner boundary

Page 69: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Closing

CSE 166, Spring 2019 69

Structuring element rolls along outer boundary

Page 70: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Opening and closing

CSE 166, Spring 2019 70

Opening

Closing

Dilation

Erosion

Page 71: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Morphological image processing

CSE 166, Spring 2019 71

Noisy input

Dilation

Erosion

Opening

Dilation

Erosion

Closing

Page 72: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Boundary extraction

CSE 166, Spring 2019 72

Erosion Set difference

Page 73: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Boundary extraction

CSE 166, Spring 2019 73

Page 74: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Hole filling

CSE 166, Spring 2019 74

Given point in hole

Page 75: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Hole filling

CSE 166, Spring 2019 75

Given points in holes All holes filled

Page 76: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Connected components

CSE 166, Spring 2019 76

Given point in A

Page 77: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Connected components

CSE 166, Spring 2019 77

15 connected

components

X-ray image

Threshold (negative)

Page 78: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image segmentation

• General approach

1. Spatial filtering

2. Additional processing

3. Thresholding

CSE 166, Spring 2019 78

Page 79: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image segmentation

CSE 166, Spring 2019 79

Edge-based

Region-based

Input Edges Segmentation

Page 80: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Derivatives in 1D

• Forward difference

• Backward difference

• Central difference

CSE 166, Spring 2019 80

Page 81: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image derivatives

CSE 166, Spring 2019 81

Page 82: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Detection of isolated points

CSE 166, Spring 2019 82

Laplacian(second derivative)

Input Segmentation

Threshold absolute

value

Page 83: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Line detection

CSE 166, Spring 2019 83

Threshold absolute

value

Threshold value

Laplacian(second

derivative)Input

Double lines

Page 84: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Line detection, specific directions

CSE 166, Spring 2019 84

Spatial filters

Page 85: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Line detection, specific directions

CSE 166, Spring 2019 85

+45°

Negative values set

to zero

Threshold

Page 86: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edge models

CSE 166, Spring 2019 86

Step Ramp Roof edge

Page 87: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edge models

CSE 166, Spring 2019 87

Step

RampRoofedge

Page 88: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Ramp edge

CSE 166, Spring 2019 88

First derivative

Second derivative

One point

Two points

Page 89: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Noise and image derivatives

CSE 166, Spring 2019 89

First derivative

Second derivativeInput

Noise

Page 90: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edge detection

1. Image smoothing for noise reduction

2. Detection of image points (edge point candidates)

3. Edge localization (select from candidates, set of edge points)

CSE 166, Spring 2019 90

Page 91: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Gradient and edge direction

CSE 166, Spring 2019 91

Gradient direction is orthogonal to edge direction

Page 92: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Gradient operators

CSE 166, Spring 2019 92

Forward difference

Page 93: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Gradients

CSE 166, Spring 2019 93

Magnitude of

vertical gradient

Magnitude of

horizontal gradient

Magnitude of

gradient vector

Input

Page 94: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Gradients

CSE 166, Spring 2019 94

Magnitude of

vertical gradient

Magnitude of

horizontal gradient

Magnitude of

gradient vector

Input

Smooth image prior to computing gradients.Results in more selective edges

Page 95: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edge detection

1. Smooth the input image

2. Compute the gradient magnitude image

3. Apply nonmaximal suppression to the gradient magnitude image

4. Threshold the resulting image

CSE 166, Spring 2019 95

Page 96: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edge detection

CSE 166, Spring 2019 96

Without smoothing With smoothing

Threshold magnitude of gradient vector

Page 97: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Advanced edge detection

CSE 166, Spring 2019 97

Magnitude of gradient vector

(with smoothing)Input

Marr-Hildreth CannyFigure 10.25 in textbook looks better

Page 98: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Canny edge detector

1. Smooth the input image with a Gaussian filter

2. Compute the gradient magnitude and angle images

3. Apply nonmaximal suppression to the gradient magnitude image

4. Use double thresholding and connectivity analysis to detect and link edges

CSE 166, Spring 2019 98

Page 99: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Double thresholding

• Use a high threshold to start edge curves and a low threshold to continue them– Define two thresholds TH and TL– Starting with output of nonmaximal suppression, find a point q0,

which is a local maximum greater than TH– Start tracking an edge chain at pixel location q0 in one of the two

directions– Stop when gradient magnitude is less than TL

CSE 166, Spring 2019 99

TH

TL

q0

Page 100: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Double thresholding

CSE 166, Spring 2019 100

TH = 15 and TL = 5T = 5T = 15

Single threshold Double threshold

Page 101: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Thresholding

CSE 166, Spring 2019 101

Single threshold Dual thresholds

Histograms

Page 102: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Thresholding

• Single threshold

• Dual thresholds

CSE 166, Spring 2019 102

where

where

Page 103: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Noise and thresholding

CSE 166, Spring 2019 103

Noise

Page 104: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Varying background and thresholding

CSE 166, Spring 2019 104

Intensity ramp Product of input and intensity ramp

Input

Page 105: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basic global thresholding

CSE 166, Spring 2019 105

Intensity ramp ThresholdInput

Page 106: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Basic global thresholding

CSE 166, Spring 2019 106

Value used to stop iterating

Page 107: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Optimum global thresholding

CSE 166, Spring 2019 107

Histogram

Basic global thresholding

Optimum global

thresholding using Otsu’s

method

Input

Page 108: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Otsu’s method

CSE 166, Spring 2019 108

Page 109: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image smoothing to improve global thresholding

CSE 166, Spring 2019 109

Without smoothing

With smoothing

Otsu’s method

Page 110: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Image smoothing does not always improve global thresholding

CSE 166, Spring 2019 110

Without smoothing

With smoothing

Otsu’s method

Page 111: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edges to improve global thresholding

CSE 166, Spring 2019 111

Mask image (thresholded

gradient magnitude)

Input

Masked input

Optimum global

thresholding using Otsu’s

method

Page 112: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Edges to improve global thresholding

CSE 166, Spring 2019 112

Input

Masked input

Optimum global

thresholding using Otsu’s

method

Mask image (thresholded absolute Laplacian)

Page 113: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Variable thresholding

CSE 166, Spring 2019 113

Global thresholding

Local standard

deviations

Local thresholding

using standard

deviations

Input

Page 114: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Variable thresholding

CSE 166, Spring 2019 114

Global thresholding

Local thresholding using moving

averages

Input(spot shading)

Page 115: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Variable thresholding

CSE 166, Spring 2019 115

Global thresholding

Local thresholding using moving

averages

Input(sinusoidal shading)

Page 116: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Segmentation by region growing

CSE 166, Spring 2019 116

Initial seed

image

InputX-ray image

Final seed

image

Output image

Page 117: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Segmentation by region growing

CSE 166, Spring 2019 117

Page 118: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Segmentation by region growing

CSE 166, Spring 2019 118

Difference image thresholded with the smallest of the dual

thresholds

Segmentation by region growing

Difference image thresholded using

dual thresholds

Difference image

Page 119: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Advanced segmentation methods

• k-means clustering

• Superpixels

• Graph cuts

CSE 166, Spring 2019 119

Page 120: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Segmentation using k-means clustering

CSE 166, Spring 2019 120

Input Segmentation using k-means, k = 3

Page 121: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Segmentation using k-means clustering

CSE 166, Spring 2019 121

Page 122: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Superpixels

CSE 166, Spring 2019 122

Image of 4,000 superpixels with

boundaries

Image of 4,000 superpixels

Input image of 480,000 pixels

Page 123: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Superpixels

CSE 166, Spring 2019 123

500 superpixels 250 superpixels1,000 superpixels

Page 124: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Superpixels for image segmentation

CSE 166, Spring 2019 124

Segmentation using k-means, k = 3

Superpixel image (100 superpixels)

Input image of 301,678 pixels

Segmentation using k-means, k = 3

Page 125: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Images as graphs

CSE 166, Spring 2019 125

Simple graph with edges only between

4-connected neighbors

Stronger (greater weight) edges are darker

Page 126: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Graph cuts for image segmentation

CSE 166, Spring 2019 126

Cut the weak edges

Page 127: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

Graph cuts for image segmentation

CSE 166, Spring 2019 127

Smoothed input Graph cut segmentation

Input

Page 128: Introduction and Overview - University of California, San DiegoScaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function

CSE 166

• Image acquisition

• Geometric transformations and image interpolation

• Intensity transformations

• Spatial filtering

• Fourier transform and filtering in the frequency domain

• Image restoration

• Color image processing

• Basis vectors

• Matrix-based transforms

• Basis images

• Wavelet transform

• Image compression

• Image watermarking

• Morphological image processing

• Edge detection

• Segmentation

CSE 166, Spring 2019 128


The multiresolution image format

The multiresolution image format

Software
Wavelets and Multiresolution Processing (Multiresolution ...lkon/imageprocessing/Chapter_07b_Wavelets_and... · Wavelets and Multiresolution Processing (Multiresolution Analysis)

Wavelets and Multiresolution Processing (Multiresolution ...lkon/imageprocessing/Chapter_07b_Wavelets_and... · Wavelets and Multiresolution Processing (Multiresolution Analysis)

Documents
Wavelets and Multiresolution Processing (Multiresolution

Wavelets and Multiresolution Processing (Multiresolution

Documents
Illumination Dependent Refinement of Multiresolution Meshes

Illumination Dependent Refinement of Multiresolution Meshes

Documents
Wavelet and multiresolution process

Wavelet and multiresolution process

Documents
Multiresolution Coding Techniques for Digital

Multiresolution Coding Techniques for Digital

Documents
A Theory for Multiresolution Signal Decomposition: The ...ee225b/sp10/handouts/mallat-good... · A Theory for Multiresolution Signal Decomposition: The Wavelet ... A THEORY FOR MULTIRESOLUTION

A Theory for Multiresolution Signal Decomposition: The ...ee225b/sp10/handouts/mallat-good... · A Theory for Multiresolution Signal Decomposition: The Wavelet ... A THEORY FOR MULTIRESOLUTION

Documents
Isosurfaces Over Simplicial Partitions of Multiresolution ... · assume that the function is piecewise smooth and does not have to be a distance function, generating accurate isosur-faces

Isosurfaces Over Simplicial Partitions of Multiresolution ... · assume that the function is piecewise smooth and does not have to be a distance function, generating accurate isosur-faces

Documents
Wavelet based multiresolution expectation maximization ...raheja/publications/wmrem_cmig.pdf · Wavelet based multiresolution expectation maximization image reconstruction algorithm

Wavelet based multiresolution expectation maximization ...raheja/publications/wmrem_cmig.pdf · Wavelet based multiresolution expectation maximization image reconstruction algorithm

Documents
Mesh Processing Course : Multiresolution

Mesh Processing Course : Multiresolution

Education
Wavelets and Multiresolution Processingweb.engr.oregonstate.edu/~thinhq/teaching/ece499/spring06/wavelet.pdfMultiresolution Analysis (MRA) A scaling function is used to create a series

Wavelets and Multiresolution Processingweb.engr.oregonstate.edu/~thinhq/teaching/ece499/spring06/wavelet.pdfMultiresolution Analysis (MRA) A scaling function is used to create a series

Documents
Multiresolution Frequency Domain Technique Domain ...inside.mines.edu/~aelsherb/pdfs/books/2012/multiresolution_frequency_domain_technique...Multiresolution analysis techniques have

Multiresolution Frequency Domain Technique Domain ...inside.mines.edu/~aelsherb/pdfs/books/2012/multiresolution_frequency_domain_technique...Multiresolution analysis techniques have

Documents
Multiresolution Green’s Function Methods for Interactive …djames/papers/JamesPai02TOG.pdf · 2002. 7. 11. · DINESH K. PAI Department of Computer Science University of British

Multiresolution Green’s Function Methods for Interactive …djames/papers/JamesPai02TOG.pdf · 2002. 7. 11. · DINESH K. PAI Department of Computer Science University of British

Documents
DJOieeetsp97Generalized Sampling Theorems in Multiresolution Subspaces

DJOieeetsp97Generalized Sampling Theorems in Multiresolution Subspaces

Documents
Multiresolution Convolutional Autoencoders

Multiresolution Convolutional Autoencoders

Documents
A Theory for Multiresolution Signal Decomposition

A Theory for Multiresolution Signal Decomposition

Documents
Multiresolution Detection of Spiculated

Multiresolution Detection of Spiculated

Documents
MULTIRESOLUTION FLOW VISUALIZATION - univ-pau.fr

MULTIRESOLUTION FLOW VISUALIZATION - univ-pau.fr

Documents
MULTIRESOLUTION MOLECULAR MECHANICS: DYNAMICS, …d-scholarship.pitt.edu/26277/1/biyiklie_etd2015.pdf · MULTIRESOLUTION MOLECULAR MECHANICS: DYNAMICS, ADAPTIVITY, AND IMPLEMENTATION

MULTIRESOLUTION MOLECULAR MECHANICS: DYNAMICS, …d-scholarship.pitt.edu/26277/1/biyiklie_etd2015.pdf · MULTIRESOLUTION MOLECULAR MECHANICS: DYNAMICS, ADAPTIVITY, AND IMPLEMENTATION

Documents
Multiresolution Analysis of Spatial Environmental ... - ima.ro file1. Discrete Data. Grid Points. 2. 2D Bi-cubic Extension 3. Multiresolution Analysis 4. Numerical Results Multiresolution

Multiresolution Analysis of Spatial Environmental ... - ima.ro file1. Discrete Data. Grid Points. 2. 2D Bi-cubic Extension 3. Multiresolution Analysis 4. Numerical Results Multiresolution

Documents
The Multiresolution Analysis Software

The Multiresolution Analysis Software

Documents
Multiresolution Matrix Compression - Directorypeople.cs.uchicago.edu/~risi/papers/MMCaistats.pdf · Multiresolution Matrix Factorization (MMF) is a recently introduced method for

Multiresolution Matrix Compression - Directorypeople.cs.uchicago.edu/~risi/papers/MMCaistats.pdf · Multiresolution Matrix Factorization (MMF) is a recently introduced method for

Documents
WWavelets & Multiresolution Analysis

WWavelets & Multiresolution Analysis

Documents
The Incremental Multiresolution Matrix Factorization Algorithmopenaccess.thecvf.com/content_cvpr_2017/papers/Ithapu... · 2017-05-31 · The Incremental Multiresolution Matrix Factorization

The Incremental Multiresolution Matrix Factorization Algorithmopenaccess.thecvf.com/content_cvpr_2017/papers/Ithapu... · 2017-05-31 · The Incremental Multiresolution Matrix Factorization

Documents
Multiresolution analysis and wavelet bases Outline : Multiresolution analysis The scaling function and scaling equation Orthogonal wavelets Biorthogonal

Multiresolution analysis and wavelet bases Outline : Multiresolution analysis The scaling function and scaling equation Orthogonal wavelets Biorthogonal

Documents
Multiresolution spline

Multiresolution spline

Documents
Mesh-Based Methods for Multiresolution Representations

Mesh-Based Methods for Multiresolution Representations

Documents
Isosurfaces Over Simplicial Partitions of Multiresolution ...faculty.cs.tamu.edu/schaefer/research/iso_simplicial.pdf · Isosurfaces Over Simplicial Partitions of Multiresolution

Isosurfaces Over Simplicial Partitions of Multiresolution ...faculty.cs.tamu.edu/schaefer/research/iso_simplicial.pdf · Isosurfaces Over Simplicial Partitions of Multiresolution

Documents
Multiresolution analysis & wavelets (quick tutorial)

Multiresolution analysis & wavelets (quick tutorial)

Documents
Wavelets and Multiresolution Processing (Background) › ~cnikou › Courses › Digital_Image... · Wavelets and Multiresolution Processing (Background) Christophoros Nikou cnikou@cs.uoi.gr

Wavelets and Multiresolution Processing (Background) › ~cnikou › Courses › Digital_Image... · Wavelets and Multiresolution Processing (Background) Christophoros Nikou [email protected]

Documents