Presenter: Tzu-Heng Henry Lee Research Advisor: Jian-Jiun Ding, Ph. D. Assistant professor Digital Image and Signal Processing Lab Graduate Institute of

Presenter:Tzu-Heng Henry Lee

Research Advisor:Jian-Jiun Ding , Ph. D.Assistant professor

Digital Image and Signal Processing LabGraduate Institute of Communication Engineering

National Taiwan University

National Taiwan National Taiwan UniversityUniversity

Introduction to Shape-Adaptive Image Compression

Morphological Segmentation Using Erosion

Shape-Adaptive Transform Algorithm Quantization Coding Technique of the Image Segment Simulations Conclusion and Future Work

April 19, 2023Digital Image and Signal Processing Lab

Graduate Institute of Communication EngineeringNational Taiwan University

2

The idea is to exploit high correlation between the color values in the neighboring pixels within the same image segment.

Characteristics in an image segment usually share the similar color values(the color intensity variations are low).

The arbitrarily-shaped image segment can be completely represented by its shape and internal contents [1].



3

JPEG images normally display various kinds of undesired distortion artifacts such as blocking, blurring, and Ringing.

Compressions with low bit-ratesLossy quantization process is used to

compress the DCT coefficients.



4

key features that distinguish the improved algorithm are built around two central components:• Morphological segmentation, and• Shape-adaptive DCT with orthogonal bases.



5

Input ImageShape-Adaptive

DCT

Quantization with Flexible

Quantization ArrayEntropy Coding

Morphological Segmentation

(Erosion Operation)

Compressed Data Stream

Q: Why do we include this stage in our algorithm?

A: The color values at the edge of an

segmented object usually vary significantly. The contour region of a segment contains a

great portion of the high frequency components

Q: Why do we include this stage in our algorithm?

A: The color values at the edge of an

segmented object usually vary significantly. The contour region of a segment contains a

great portion of the high frequency components



6

This allows us to compress the contour sub-region and the interior sub-region of an arbitrarily image segment separately.

So we can minimize quantization noise and enhance overall quality of image compression.



7

Contour sub-region

Interior sub-region

The overall internal region



8

Traditional Method:fill zeros outside the contour of the

arbitrarily image and treat the whole image block as a traditional image block [2].

Drawback:This increases the high-order transform

coefficients which are later truncated.Leads to performance degradation.



9

Based on the concept of KLT(Karhunen-Loeve).

Generic transform that does not need to be computed for each image can be derived.

Lower compuational complexity. Provides a good compromise between

information packing ability and computational complexity [A1].



10

DCT produces less blocking artifacts compared to DFT.

1-D point of view. The implicit n-point periodicity of the DFT

boundary discontinuities High freq Truncation Gibb’s phenomenon The DCT which has the implicit 2n-point

periodicity does not produce such discontinuities [1].



11

DCT-based. Since the height and width of an

arbitrarily-shaped image segment are usually not the same,

we redefine the forward DCT as

for and



12

1 1

0 0

2 ( ) ( ) (2 1) (2 1)( , ) ( , ) cos cos

2 2*

W H

x y

C u C v x u y vF u v f x y

W HH W

0,..., 1u W 0,..., 1v H 1 / 2 for 0( )

1 otherwise

kC k

The inverse DCT can also be re-written as

and the DCT basis is expressed as



13

2 ( ) ( ) (2 1) (2 1)( , ) cos cos,

2 2*

C u C v x u y vu vx y

W HH W

1 1

0 0

2 (2 1) (2 1)( , ) ( ) ( ) ( , ) cos cos

2 2*

W H

u v

x u y vf x y C u C v F u v

W HH W

The DCT bases are not yet customized for a particular arbitrarily-shaped image segment.

Since we are using the traditional DCT bases, we can simply project these basis functions into subspace SB:

A linear combination of can be used to describe the arbitrarily segment vector P(x, y).

This operation removes the components of outside subspace SB [2].



14

ˆ Project( , )i i Bf f S

An example of the projection operation:

shape matrix:formed by filling 1’s in the pixel position inside the contour of the arbitrary shape. Zeroes are filled in the region outside the contour.



15

75 96

105 98 99 101 73 85 66 60

100 97 89 94 87 64 55

84 94 90 81 71 66

93 86 94 81 70

86 86 81 72

98 97 78

105 104

0 0 0 0 1 1 0 0

1 1 1 1 1 1 1 1

0 1 1 1 1 1 1 1

0 0 1 1 1 1 1 1

0 0 1 1 1 1 1 0

0 0 0 1 1 1 1 0

0 0 0 1 1 1 0 0

0 0 0 1 1 0 0 0

The 88 DCT bases with the shape of our example.



16

The number of orthogonal bases M is less than HW

The same basis function could be repetitively chosen.

Generally the HW shape-projected bases are not orthogonal because the number of transform coefficients may exceed the image segment size [2].



17

One of the methods to obtain orthogonal basis functions in the subspace SB is to use the Gram-Schmidt algorithm [2], [3], [4], [5].

We use the Gram-Schmidt process to reduce the bases to M orthogonal ones.



18

Before we use the Gram-Schmidt process to reduce the bases to M orthogonal ones, we reorder the HW shape-projected bases by the zig-zag reordering matrix [6]:



19

make the low frequency components to concentrate on the top-left corner

move the less important high frequency components to concentrate on the bottom-right corner of the matrix.

This is because the low frequency components contain a significant fraction of the total image energy [7].



20

Low-frequency AC coefficients are placed before high-frequency AC coefficients.

Makes upcoming entropy coding process much easier.

By keeping higher frequency coefficients (which are more likely to be zero after quantization) together, we can form long runs of zeros [8].



21

JPEG – fixed quantization matrix for 8X8 blocks

Our method – The length of the quantization array corresponding to the arbitrary-shape DCT coefficients is not fixed.

We define an extendable and segment shape-dependent quantization array Q(k) as a linearly increasing line:

for k = 1, 2,…, M.



22

slope intercept

( ) a cQ k Q k Q

Need to encode the quantized arbitrary-shape DCT coefficients to bit stream.

We use the same coding technique that is used in JPEG.

The quantized coefficients are a series of integer values with large values at the beginning(DC terms) of the series followed by a large amount of zeros at the back(AC terms).



23

The DC coefficient is treated separately from the AC coefficients.

Difference Encoding: It is encoded as the difference between the present DC term and the one from the previous block.

The AC-terms are encoded by zero-run length coding(ZRL) and the Huffman coding [6], [7].



24

We directly combine all the encoded bit streams of all image segments.

In the ZRL coding process, we truncate the successive zeroes in the end of the coefficients, and replace them with an end-of-bit (EOB) symbol.

We can divide the bit stream to each image segment by the EOB symbol in the decoding process.



25



26



27



28

Shape-Adaptive CompressionShape-Adaptive Compressionwith Morphological Segmentation JPEG



29

編碼前

解碼後

(Top) Original image (11080 bytes). (Left) JPEG compressed image (RMSE: 3.9931 and data size: 1128 bytes). (Right) Compressed image using our proposed algorithm (RMSE: 2.7502 and data size: 410 bytes)

編碼前

解碼後

(Top) Original image (11080 bytes). (Left) JPEG compressed image (RMSE: 4.2714 and data size: 1428 bytes). (Right) Compressed image using our proposed algorithm (RMSE: 2.9509 and data size: 419 bytes)

The complexity of Gram-Schmidt orthogonal process: O(n2)

n - the number of points of an image segment



30

What if an image segment is large? It would cost a lot of computational time

Solutions:1. Segment the image in more detail such

that the number of points of an image segment is confined in an acceptable range.

2. A number of bases smaller than the dimension of the image segment can be chosen to avoid n being too large



31

The JPEG method has a poorer because it cannot utilize the characteristics of the image.

Significant improvements on the distortion artifacts caused by the quantization process are made by using the shape-adaptive compression algorithm with morphological segmentation.

A higher compression rate with a comparable RMSE.



32

Improvements on Huffman Coding algorithm.

More efficient ways to segment the image.

Improvements on compression efficiency.

Elimination of the flaws on the erosion operation(small segment problems )



33

[1] R. C. Gonzalez and R. E. Woods, Digital Image Processing Second Edition, Prentice Hall, New Jersey, 2002.

[2] S. F. Chang and D. Messerschmitt, “Transform coding of arbitrarily shaped image segments,” Proc. 1st ACM Int. Conf. Multimedia Anaheim, CA, pp. 83- 90, 1993.

[3] M. Gilge, T. Engelhardt, and R. Mehlan, “Coding of arbitrarily shaped image segments based on a generalized orthonormal transform,” Signal Process: Image Commun., vol. 1, pp. 153–180, Oct. 1989.

[4] J. Apostolopoulos and J. Lim, “Coding arbitrarily-shaped regions,” Proc. SPIE Visual Commun. Image Process., pp. 1713-1726, May 1995.

[5] R. Stasinski and J. Konrad, “A new class of fast shape-adaptive orthogonal transforms and their application to region-based image compression,” IEEE Trans. on Circuits and systems for Video Technology, vol. 9, pp. 16–34, 1999.

[6] W. B. Pennebaker and J. L. Mitchell, JPEG Still Image Data Compression Standard. New York: Van Nostrand Reinhold, 1993.

[7] C. K. Wallace. The JPEG still picture compression standard. Communications of the ACM, 34(4):31-44, 1991.

[8] T. Acharya amd A. K. Ray, Image Processing Principles and Applications, John Wiley & Sons, New Jersey.



34



35

Duh?

Documents

Presenter: Tzu-Heng Henry Lee Research Advisor: Jian-Jiun Ding, Ph. D. Assistant professor Digital Image and Signal Processing Lab Graduate Institute of