View
7
Download
0
Category
Preview:
Citation preview
LECTURE VI
LOSSLESS COMPRESSION
ALGORITHMS
DR OUIEM BCHIR
1
STORAGE SPACE
Uncompressed graphics audio and video data require
substantial storage capacity
Storing uncompressed video is not possible with todayrsquos
technology (CD amp DVD)
Data transmission of uncompressed video over digital
networks require very high bandwidth
To be cost-effective and feasible multimedia systems must use
compressed video and audio streams
2
INTRODUCTION
Compression the process of coding that will effectively reduce the total
number of bits needed to represent certain information
3
General Data Compression Scheme
INTRODUCTION
If the compression and decompression processes induce no
information loss then the compression scheme is lossless
otherwise it is lossy
Compression ratio
4
COMPRESSION STEPS
5
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
STORAGE SPACE
Uncompressed graphics audio and video data require
substantial storage capacity
Storing uncompressed video is not possible with todayrsquos
technology (CD amp DVD)
Data transmission of uncompressed video over digital
networks require very high bandwidth
To be cost-effective and feasible multimedia systems must use
compressed video and audio streams
2
INTRODUCTION
Compression the process of coding that will effectively reduce the total
number of bits needed to represent certain information
3
General Data Compression Scheme
INTRODUCTION
If the compression and decompression processes induce no
information loss then the compression scheme is lossless
otherwise it is lossy
Compression ratio
4
COMPRESSION STEPS
5
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
INTRODUCTION
Compression the process of coding that will effectively reduce the total
number of bits needed to represent certain information
3
General Data Compression Scheme
INTRODUCTION
If the compression and decompression processes induce no
information loss then the compression scheme is lossless
otherwise it is lossy
Compression ratio
4
COMPRESSION STEPS
5
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
INTRODUCTION
If the compression and decompression processes induce no
information loss then the compression scheme is lossless
otherwise it is lossy
Compression ratio
4
COMPRESSION STEPS
5
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
COMPRESSION STEPS
5
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
TYPES OF COMPRESSION
Symmetric Compression
bull Same time needed for decoding and encoding phases
Asymmetric Compression
bull Compression process is performed once and enough
bull time is available hence compression can take longer
bull Decompression is performed frequently and must be
bull done fast
6
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
STATISTICAL ENCODING
(FREQUENCY DEPENDENT)
Fixed length coding
bull Use equal number of bits to represent each symbol - message of N
bull symbols requires L gt= log_2(N) bits per symbol
bull Good encoding for symbols with equal probability of occurrence Not
efficient if probability of each symbol is not equal
Variable length encoding
bull frequently occurring characters represented with shorter strings than
bull seldom occurring characters
bull Statistical encoding is dependant on the frequency of occurrence of a
bull character or a sequence of data bytes
bull You are given a sequence of symbols S1 S2 S3 and the probability
bull of occurrence of each symbol P(Si) = Pi
7
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
BASICS OF
INFORMATION THEORY
The entropy η of an information source with alphabet S = s1s2hellipsn
is
pi probability that symbol si will occur in S
Log2(1pi) amount of information contained in si which
corresponds to the number of bits needed to encode si
8
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
EXAMPLE
Uniform distribution pi=1256 hence the entropy of the image is
log2256=8
9
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
ENTROPY AND CODE LENGTH
Entropy and Code Length
The entropy η is weighted-sum of terms log2(1pi)
It represents the average amount of information
contained per symbol in the source S
The entropy η specifies the lower bound for the
average number of bits to code each symbol in S ie
The average length (measured in bits) of the codewords
produced by the encoder
10
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
RUN-LENGTH CODING
Memoryless Source An information source that is
independently distributed the value of the current
symbol does not depend on the values of the
previously appeared symbols
Run-Length Coding (RLC) (not memoryless)
exploits memory present in the information source
bull Rational for RLC If the information source has the property
that symbols tend to form continuous groups then such
symbol and the length of the group can be coded
11
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
RUN-LENGTH CODING
(RLC)
Content dependent coding
1048698 RLC replaces the sequence of same consecutive
bytes with the number of occurrences
bull The number of occurrences is indicated by a special flag - ldquordquo
1048698 RLC Algorithm
bull If the same byte occurred at least 4 times then count the number of occurrences
bull Write compressed data in the following format ldquothe counted bytenumber of occurrencesrdquo
1048698 Example
bull Uncompressed sequence - ABCCCCCCCCCDEFFFFGGG
bull Compressed sequence - ABC4DEF0GGG (from 20 to 13 bytes)
12
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
VARIABLE-LENGTH
CODING (VLC)
1048698 Shannon-Fano Algorithm a top-down approach
1 Sort the symbols according to the frequency count of
their occurrences
2 Recursively divide the symbols into two parts each
with approximately the same number of counts until all
parts contain only one symbol
1048698 Example coding of ldquoHELLOrdquo
13
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
14
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
15
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
Another coding tree for ldquoHELLOrdquo by Shannon-Fano
16
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
17
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
HUFFMAN CODING
ALGORITHM
Characters are stored with their probabilities
1048698 Number of bits of the coded characters differs Shortest
code is assigned to most frequently occurring character
bull 1048698 To determine Huffman code we construct a binary tree
bull 1048698 Leaves are characters to be encoded
bull 1048698 Nodes contain occurrence probabilities of the characters
bull belonging to the subtree
bull 1048698 0 and 1 are assigned to the branches of the tree arbitrarily -
bull therefore different Huffman codes are possible for the same
bull data
bull 1048698 Huffman table is generated
1048698 Huffman tables must be transmitted with compressed
data
18
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
EXAMPLE OF
HUFFMAN CODING
19
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
PROPERTIES OF
HUFFMAN CODING
1048698 Unique prefix property No Huffman code is a
prefix of any other Huffman code ndash precludes any
ambiguity in decoding
1048698 Optimality Minimum redundancy code ndash proved
optimal for a given data model (ie a given
accurate probability distribution)
1048698 The two least frequent symbols will have the same length
for their Huffman codes differing only at the last bit
1048698 Symbols that occur more frequent will have shorter
Huffman codes than symbols that occur less frequent
1048698 The average code length for an information source S is
strictly less than
20
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
ARITHMETIC CODING
Each symbol is coded by considering prior
data
1048698 encoded sequence must be read from beginning
no random access possible
1048698 Each symbol is a portion of a real number
between 0 and 1
1048698 When the message becomes longer the length of
the interval shortens and the number of bits
needed to represent the interval increases
21
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
ARITHMETIC VS
HUFFMAN
1048698 Arithmetic encoding does not encode each symbol separately
Huffman encoding does
1048698 Arithmetic encoding transmits only length of encoded string
Huffman encoding transmits the Huffman table
1048698 Compression ratios of both are similar
22
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
23
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
ARITHMETIC CODING
ENCODER
24
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
Example
Encode Symbols ldquoCAEE$rdquo
25
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
26
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
27
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
1048698 The final step in Arithmetic encoding calls for the generation
of a number that falls within the rang [low high)
1048698 The above algorithm will ensure that the shortest binary
codeword is found
28
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
29
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
ARITHMETIC CODING
DECODER
30
1048698 Decoding symbols ldquoCAEE$rdquo
31
1048698 Decoding symbols ldquoCAEE$rdquo
31
Recommended