Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
International Conference on Image Processing ICIP-2018October 9, 2018 1
Deep Image Compression with Iterative Non-uniform Quantization
Jianrui Cai and Lei Zhang
Dept. of Computing
The Hong Kong Polytechnic University
International Conference on Image Processing ICIP-2018October 9, 2018 2
Introduction
Proposed Method
Experimental Results
Conclusions
Overview
International Conference on Image Processing ICIP-2018October 9, 2018 3
Introduction: motivationFa
ceb
oo
kIn
stag
ram
International Conference on Image Processing ICIP-2018October 9, 2018 4
Introduction : image compression
Image compression is a must to reduce the storage space and provide aneconomic solution to a wide range of image storage and transmission systems.
Image QualityHigh Low
Bit Per Pixel
International Conference on Image Processing ICIP-2018October 9, 2018 5
Introduction : Pipeline
Image Compression Pipeline
ForwardTransform
Quantization Encoder
InverseTransform
De-QuantizationDecoder
0101011...110000Bitstream
Rate
Distortion
International Conference on Image Processing ICIP-2018October 9, 2018 6
Introduction : formulation
Rate-Distortion curve
Loss = argmin [R(z) + λ D(X, Y)]
• X: the original image;
• Z: the latent image;
• Y: the reconstructed image;
• R(): the entropy of the latent image;
• D(): the distortion between two images.
International Conference on Image Processing ICIP-2018October 9, 2018 7
Introduction: JPEG
DCT
QuantizationHuffmanEncoder
IDCT
De-Quantization
HuffmanDecoder
0101011...110000Bitstream
Pre-processing
Processing
Pipeline of JPEG
Pros & Cons: Simple but often with artifact (e.g., blocking)
International Conference on Image Processing ICIP-2018October 9, 2018 8
Introduction: JPEG 2000
DWT
QuantizationEBCOT
Encoder
DWT
De-Quantization
EBCOTDecoder
0101011...110000Bitstream
Pre-processing
Processing
Pipeline of JPEG 2000
Pros & Cons: Much improved but still with visual artifact (e.g., rings)
International Conference on Image Processing ICIP-2018October 9, 2018 9
Introduction: deep neural networks
0101011...110000Bitstream
Encoding Network Quantization Decoding Network
Entropy Network
CABAC
probability
International Conference on Image Processing ICIP-2018October 9, 2018 10
Deep Neural Network
• Pros: end-to-end training; adaptively learn an effective encoder-decoder from a large amount of image data and in a larger context torepresent more complex image structures; reduce the visual artifactsin the decompressed image.
• Cons: time-consuming, uniform quantization
Introduction: deep neural networks
International Conference on Image Processing ICIP-2018October 9, 2018 11
Proposed Method: formulation
Formulation
Reconstructed image
Decoder
Quantizer
Encoder
Original image
Network parameters
International Conference on Image Processing ICIP-2018October 9, 2018 12
Inp
ut
25
6×
25
6×
3
Co
nv
64×3×3 (S2)
Co
nv
+ R
eLU
64×3×3 (S1)
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
Co
nv
128×3×3 (S2)
Co
nv
+ R
eLU
64×3×3 (S1)
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
3 Concat Blocks3 Concat Blocks
Co
nv
128×3×3 (S2)
Co
nv
+ R
eLU
128×3×3 (S1)
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
3 Concat Blocks
Conv+
Sigmoid32×3×3 (S1)
Quantizer
Ou
tpu
t
25
6×
25
6×
3
Co
nv
Co
nv
+ R
eLU
64×3×3 (S1)
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
64×3×3 (S1)
3 Concat Blocks 3 Concat Blocks
Sub
pix
els
128×3×3 (S1)
3 Concat Blocks
Sub
pix
els
Sub
pix
els
×2×2×2
… … …
………
Co
nv
12×3×3 (S1)
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nv
+ R
eLU
Co
nca
t
Proposed Method: network architecture
International Conference on Image Processing ICIP-2018October 9, 2018 13
Proposed Method: loss function
Loss function
International Conference on Image Processing ICIP-2018October 9, 2018 14
Proposed Method: alternative network and quantizer training
With the encoder, we have:
z: the latent image feature
The probability density function (PDF) of z, denoted by P(z):
International Conference on Image Processing ICIP-2018October 9, 2018 15
Proposed Method: alternative network and quantizer training
The optimal quantizer:
quantized values zq:
decision boundaries bq:
International Conference on Image Processing ICIP-2018October 9, 2018 16
Proposed Method: alternative network and quantizer training
Fine-tune the network with the updated quantizer:
Updated PDF of z:
Updated latent image feature z:
International Conference on Image Processing ICIP-2018October 9, 2018 17
Proposed Method: entropy network
Current symbol
Context
Unknow-context
32
32
32 MC
on
v+
ReL
U
64×3×3 (S1)
MC
on
v+
ReL
U
MC
on
v+
ReL
U
MC
on
v+
ReL
U
SUM
Residual Block64×3×3 (S2)
FC1
FC2
1024 64 4
P( z=0 | context )
P( z=1 | context )
P( z=2 | context )
P( z=3 | context )
International Conference on Image Processing ICIP-2018October 9, 2018 18
Experimental results: Kodak dataset
International Conference on Image Processing ICIP-2018October 9, 2018 19
Experimental results: McMaster dataset
International Conference on Image Processing ICIP-2018October 9, 2018 20
Experimental results: BSD300 dataset
International Conference on Image Processing ICIP-2018October 9, 2018 21
Experimental results: visual examples
International Conference on Image Processing ICIP-2018October 9, 2018 22
Experimental results: visual examples
(a) original (b) JPEG (c) JPEG 2000 (d) Uniform (e) Proposed
International Conference on Image Processing ICIP-2018October 9, 2018 23
Conclusions
• We presented an iterative non-uniform quantizationscheme for deep image compression network.
• The quantizer and the encoder-decoder network areupdated alternatively.
• Compared with previous deep compressors, our methodexhibits better PSNR based rate-distortion curves, as wellas better visual quality.
International Conference on Image Processing ICIP-2018October 9, 2018 24