Upload
hoangnhu
View
239
Download
0
Embed Size (px)
Citation preview
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 1
Arithmetic coding
Elias coding
Arithmetic coding as finite-precision Elias
coding
Redundancy due to finite precision
Multiplication-free arithmetic coding
Context-adaptive coding
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 2
Elias coding
Entropy coding algorithm for sequences of
symbols x with general (conditional) probabilities
Representation of x by a subinterval of the unit interval [0,1)
Width of the subinterval is approximately equal to the probability fX(x)
Subinterval for x can be determined by recursive subdivision algorithm
Represent x by shortest binary fraction in the subinterval
Subinterval of width fX(x) is guaranteed to contain
one number that can be represented by L binary digits, with
fX x
0.1111
0.111
0.1101
0.11
0.1011
0.101
0.1001
0.1
0.0111
0.011
0.0101
0.01
0.0011
0.001
0.0001
0 0
1
2logL f X x
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 3
Example: Elias coding of memoryless binary source
1 3
0 14 4
X Xf f
0
1
1
4
nc
n nc a
0
1
4
0
1
1
16
4
16
1
7
64
16
64
0
37
256
28
256
1
121
1024
148
1024
1 1
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 4
Example: Elias coding of memoryless ternary source
Courtesy: T. Wiegand
f
Xa f
Xb
1
4 f
Xc
1
2
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 5
Elias coding: choose binary fraction in subinterval
Uniquely identify interval [cn,cn+an) by a binary fraction
Length of bit-string
Bit-string
bits
0. [ , )
n
n n n
L
bbbbbb b c c a
Ln
log2
1
an
1 h
X0:n1
x0:n1
1
ˆ 2 2 1n nL L
n n nc c c
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 6
Successive decoding
Elias bit string can be decoded symbol by symbol, starting with the
first symbol
For each symbol n=0,1,2, . . .
• Calculate the intervals [cn,cn+an) corresponding to all possible xn
• Determine the interval for which
• Emit the corresponding xn
ˆ ,n n n nc c c a
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 7
Elias coding for memoryless source
Elias coding algorithm
0 0
1
1
Initialize 0 and 1.
For each 0,1,
Update
Update
n n X n
n n n X n
c a
n
a a f x
c c a F x
Interval
width
Interval
lower limit
Start with unit
interval
n-th symbol
to be
encoded
Cumulative distribution
(excluding current symbol)
1
0
0 1 2 1
where { , , ,..., }
i
X i X j
j
X K
F f
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 8
Elias coding for changing probabilities
Elias coding algorithm
0 0
1
1
Initialize 0 and 1.
For each 0,1,
Update
Update
n
n
n n X n
n n n X n
c a
n
a a f x
c c a F x
Interval
width
Interval
lower limit
Start with unit
interval
n-th symbol
to be
encoded
Cumulative distribution
(excluding current symbol)
1
0
0 1 2 1
where { , , ,..., }
n n
n
i
X i X j
j
X K
F f
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 9
Elias coding for Markov random processes
Elias coding algorithm for Markov-p sources
: 1
: 1
0 0
1 | : 1
1 | : 1
Initialize 0 and 1.
For each 0,1,...
Set min ,
Update ,
Update ,
n n p n
n n p n
n n X n n p n
n n n X n n p n
c a
n
p p n
a a f x
c c a F x
X
X
x
x
Interval
width
Interval
lower limit
Start with unit
interval
n-th symbol
to be
encoded
Cumulative conditional
distribution (excluding
current symbol)
Up to p
previous
symbols
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 10
Arithmetic coding
Elias coding not practical for long symbol strings:
required arithmetic precision grows with string length
Finite precision implementations: “arithmetic coding”
Widely used in modern image and video compression algorithms:
JBIG, JPEG, JPEG-2000, H.263, H.264/AVC
nxArithmetic
coder
Arithmetic
decoder
nx
PMF nX nf x PMF
nX nf x
bit stream c
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 11
Finite precision for arithmetic coding
0 0
1
1
Initialize 0 and 1.
For each 0,1,
Update
Update
n n X n
n n n X n
c a
n
a a f x
c c a F x
N-bit precision is
permissible, if
rounding down
P-bit approximation of
probabilities
At most N+P LSBs change,
except for a possible carry
affecting MSBs.
Each bit can be affected by
at most one carry.
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 12
Represent by
state variable Cn
Represent by
state variable rn
Send
Finite precision for arithmetic coding (cont.)
Output code string
Maximum value of rn can be limited by bit stuffing
MSBs that will 1 bits LSBs thatno longer change represent lowerthat might be
interval boundary foraffected by a further computationcarry
. 01111111 1
n
n
r N P
c xxxxxxxxx xxx cccccccc c
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 13
Inefficiency due to interval rounding
Recall: Subinterval of width fX(x) is guaranteed to contain one
number that can be represented by Ln binary digits, with
Hence, rounding one interval value an+1 increases bit string by
L
n log
2f
X0:n1
x0:n1
log2
an
fX
xn
an1
log2
2N 1 1
2N 1 log
21 21 N
1
ln221 N
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 14
2 2 2log 1 log log 1
( )1
ln 2
0
X X
X X
X X
X
X
X X
X X
x xX X
f x f xE f x E f x E
f x f xH X
f xE
f x
f x f xE f x f x
f x f x
Limited precision probabilities
Efficiency loss
provided that rounded
probabilities still add to 1
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 15
Example: inefficiency for binary source
due to representing smaller probability by 2-P
log
2p
rela
tive
bit
-rat
e in
crea
se
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 16
Interval width is normalized to by binary shift after
each symbol
Approximation, with :
Corresponds to approximating of p(L) within a factor of 2:
Factor can be lumped into external probabilities
Optimum depends somewhat on data, typically
JBIG
Multiplication-free binary arithmetic coding
2 3 or 0.708 or
3 4
( ) ( )
( ) (1 ( )) ( )
Ap L p L p L
Ap M A p L A p L A p L
1/ 2 1A
p(L) p(M)
p L p LA
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 17
MPS-LPS switch
Problem: interval width for symbol M can become negative
Solution: switch the role of M and L, if
Problem: interval for symbol M can be smaller than interval for L if
Solution: “Conditional exchange” of symbols, when problem occurs
Improves compression efficiency, implemented in JPEG and JBIG
1
0 if ,12
A p L p L A
12
p L
1 14 2
1 ,
2p L A p L p L A
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 18
Sufficiency of binary coders
Consider r.v. equivalent to a r.v.
No loss by conditionally encoding bits (in any order)
Supply arithmetic coder with symbol/probability pairs
Total number of (conditional) probabilities to be estimated per symbol x . . . same as with K-ary encoding of x
0,1, ,2 1KX
0
1
1K
B
B
B
B
MSB
LSB
0 1 0 1 0 1 2| | , ,K K
H X H
H B H B B H B B B B
B
0: 10 0 1 1 1 1 | 0: 1, , , , , , with 0,
k kK K k B kb p b p b p p f B b
11 2 2 2 1K K
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 19
Consider stationary binary Markov-k process with probabilities
Context labeling function, enumerating all possible context vectors
Backward probability estimation separately for each context
Adaptive probability estimation
nX
0: 1 ( ): 1| 0: 1 | ( ): 10, 0,
k k n n k nX k X n k nf f X Xx x “Context vector,”
2k possible
combinations
: 0,1 0,1, ,2 1k k x x
0 ( ): 1
0
0 ( ): 1 1 ( ): 1
,
, ,
n k n
n k n n k n
C np n
C n C n
x
x x
Count of zeroes that
have previously
occured in that context
Probability
of symbol
x[n]=0
Bias for low counts,
typically 1
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 20
Adaptive probability estimation (cont.)
Tracking changing statistics, compromise between
• Rapid adaptation (small past sample counts)
• Accurate estimates (large past sample counts)
Scaled count estimation algorithm
0 1
1 1
0 0
0 1 min 0 1 max
0 10 1
00
0 1
Initialize 0For 0,1, If 1 C 1 else C 1
If min C , or max C ,
C ; C2 2
Estimate 2
n
C Cn
xC
C
C C C C
C C
Cp n
C C
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 21
Context adaptive coding
2-d extension of Markov model
Markov conditional independence property
n“causal half-plane”
x n
“Coding
context”
n
fX n X
n
X n ,X n
fX n X
n
X n ,Xn
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 22
Context adaptive coding (cont.)
Coding context vector can be mapped into context label directly
without loss
• Feasible for binary images
• Example: JBIG uses 10 binary pixels as context label
For 8-bit images, number of different contexts
• Context might have to be clustered
• Combine with prediction
256
Bernd Girod: EE398A Image and Video Compression Arithmetic Coding no. 23
Reading
W. B. Pennebaker, J. L. Mitchell, G. G. Langdon, Jr., R. B.
Arps, “An overview of the basic principles of the Q-Coder
adaptive binary arithmetic coder,” IBM J. Res. Develop., vol.
32, no. 6, November 1988.
Witten, Radford, Neal, Cleary, “Arithmetic Coding for Data
Compression” Communications of the ACM, vol. 30, no. 6,
pp. 520-540, June 1987.
Moffat, Neal, Witten, “Arithmetic Coding Revisited,” ACM
Transactions on Information Systems, vol. 16, vo. 3, pp.
256–294, July 1998.