EFFICIENT AND LOW-COMPLEXITY SURVEILLANCE VIDEO COMPRESSION USING

BACKWARD-CHANNEL AWARE WYNER-ZIV VIDEO CODING

Limin Liu, Member, IEEEZhen Li, Member, IEEE

Edward J. Delp, Fellow, IEEECSVT 2009

Outline Introduction Low-complexity Video Encoding Approaches

for SVC SVC Using Wyner-Ziv Video Coding SVC Using Backward-Channel Aware Wyner-

Ziv Video Coding Experimental Results Conclusion

Basic Diagram of Video Surveillance Systems

Introduction Video surveillance system has been widely used to

enhance public safety and privacy protection. Needs efficient transmission and storage. H.264/AVC is high computational complexity at the

encoder. This can increase the cost of a surveillance system.

Video Compression Design Considerations for Surveillance Video

General case Movie studio only needs

to encode once to put on DVD but decoded by the consumers many times.

Heavy encoder, light decoder.

Surveillance systems Encoder is implemented in a

simple and low-cost video surveillance camera.

The video is decoded and analyzed at central server.

Light encoder, heavy decoder.

Other requirements High coding efficiency Error resilience requirements

Surveillance Video Compression(SVC) Low-complexity encoder based on Wyner-Ziv

coding principles to address the tradeoff between computational complexity and coding efficiency.

Backward-Channel Aware Wyner-Ziv (BCAWZ) video coding to improve coding efficiency while maintaining low-complexity at the encoder.

Error resilience scheme for BCAWZ to keep reliable transmission in the backward-channel.

Low-complexity Video Encoding Approaches for SVC

Slepian-Wolf Theorem Construct correlation

between X and Y RX ≥ H(X|Y )

RY ≥ H(Y |X)RX + RY ≥ H(X, Y )

Regardless of its access to side information Y , encoder A can encode X with high fidelity as long as the decoder A has access to Y.

Lower bound

Low-complexity Video Encoding Approaches for SVC Wyner-Ziv Video Coding (WZVC)

Temporal correlation among frames is exploited at the decoder instead of encoder.

Each frame is independently encoded at the encoder.Computational intensive job of motion estimation is shifted

to decoder. WZVC is an ideal choice for SVC due to the light

encoder characteristics.

SVC Using Wyner-Ziv Video Coding

videoInitial estimateTurbo code ->parity bits

LDPC code ->syndrome bits

SVC Using Wyner-Ziv Video Coding Video sequence is divided

into two groups.Key frames

○ Encoded by H.264 INTRA encoder.

○ Serve to side information at decoder

Wyner-Ziv frames ○ Encoded by channel coding

method.○ Turbo code parity bits○ Low-density-parity-check(LDPC)

code syndrome bits

SVC Using Wyner-Ziv Video Coding Key frames

Decoded by the H.264 INTRA decoder. Supply side information to other frames by

find initial estimate. Initial estimate is derived from previously

reconstructed key frames.○ Use co-located pixel value in (n-1)th

frame.○ Take average of the co-located pixel value

at (n-1)th and (n+1)th frames.

Wyner-Ziv frames Quality is low, temporal correlation is not

fully exploited. To obtain higher quality, motion estimation

can be done at decoder.

SVC Using Wyner-Ziv Video Coding Higher quality initial estimate

method1○ Search for the motion vector MVn-1

of the co-located block in (n-1)th frame as the predictor of current block.

○ The reference block in (n-1)th frame is the initial estimate.

method2○ Obtain MVF and MVB

○ Use to find PF1

Use to find PF2

Use to find PB1

Use to find PB2

○ Initial estimate

SVC Using Wyner-Ziv Video Coding The channel decoder uses initial

estimate and the incoming parity or syndrome bits to decode the frame.

Assume decoder can communicate with the encoder to request more bits until correctly decoded. Wyner-Ziv video codec formulates the

video decoding problem as an error correction problem.

If the parity or syndrome bits are lost or corrupted during transmission, it will add complexity.

Standard video codec may not be correctly decoded when bits are lost and error can propagate to the following frame.

SVC Using Backward-Channel Aware Wyner-Ziv Video Coding WZVC with INTRA key frames coding efficiency is higher

than INTRA but much lower than INTER. The distance between two neighboring key frames too far

or too close will result in lower coding efficiency and degrading the quality of side information.

Use Backward-Channel Aware Motion estimation (BCAME). The basic idea BCAME is to perform motion estimation at

decoder and send the motion information back to encoder through backward-channel.

May increase the latency and cause problem for some SVC.

SVC Using Backward-Channel Aware Wyner-Ziv Video Coding

video

Even frames

Odd frames(excluding 1, 3)

1, 3 frames H.264 INTRA encoder H.264 INTRA decoder

Mode selection

SVC Using Backward-Channel Aware Wyner-Ziv Video Coding Encode first and third frame as INTRA

frames. All the other odd frames are encoded with

BCAME called BP frames. All the even frames are encoded as a

Wyner-Ziv frame.

SVC Using Backward-Channel Aware Wyner-Ziv Video Coding BP frames

Mode1 Mode2

Motion vectors are sent back to encoder. Encoder do mode selection by MSE or

SAD. Encoder use received motion vector with

previous reconstructed BP frames to generate MC reference for current BP frame.

Residual between current BP and reference is than transformed and entropy-coded.

Error Resilience In Backward-Channel Aware Wyner-Ziv Video Coding Problem

If there is error or delay at backward-channel. Motion vector is not updated and encoder continues to use it.

Encoder and decoder use different motion vectors in motion compensation for the same frame.

Error Resilience In Backward-Channel Aware Wyner-Ziv Video Coding

Two-stage error-resilient procedure

Insert the index of the frame as header info before sending

Report error when receive index not match the frame number at the encoder

Encodes the key frames as INTRA and send frame type to decoder

Coding key frames as BP

frames

decoder encoder

De-synchronization

OK

Synchronization

channel

Experimental Results H.264/AVC JM8.0 WZVC with INTRA, INTER

or BP key frames Odd-frames -> key frame Even-frame ->Wyner-Ziv frame

Block-size -> 8*8 WZVC with INTER key

frames First-frame -> INTRA Other key frames -> P frame

H.264/AVC INTER coding First-frame -> INTRA Other frames -> IBPBP GOP

structure Quarter-pel motion search Search range -> 32 Number of reference frames -> 3

Experimental Results Comparison

WZVC with INTRA key frames○ 3-4.5 dB gain for low motion

sequences○ 1.5-2 dB gain for high motion

sequences H.264 INTRA coding

Discussions Low motion sequence is more

continuous. Hence motion vectors derived from previous reconstructed frame is good.

Many surveillance video are low motion.

WZVC encoder complexity is comparable to H.264 INTRA coding.

WZVC can provide significant rate reduction compared to H.264 INTRA coding.

Experimental Results Comparison

WZVC with INTRA key frames○ 7-8 dB loss○ 10-13 dB loss

H.264 INTER coding Discussions

INTER coding is much more efficient than INTRA coding especially in low motion sequences.

Experimental Results Comparison

BCAWZ○ 5-7 dB gain

WZVC with INTRA key frames Comparison

BCAWZ○ 1.5-2 dB loss

H.264 INTER coding Discussions

This gain is achieved with marginal complexity increase at the encoder.

BCAWZ○ Two motion vector send from decoder

H.264 INTER coding○ 4*2M*2M candidate motion vector for a

search range of M. (in this paper is 32)

Experimental Results Comparison

BCAWZ○ 2-3 dB loss

WZVC with INTER key frames Discussions

Use INTER key frames significantly increase the encoder complexity.

Experimental Results Comparison

WZVC with INTRA key frames BCAWZ (BP key frame)

Discussions The visual quality of BP frame is

significantly higher than WZVC with INTRA key frames.

Experimental Results Backward-channel /forward-

channel bandwidth Lower rate : 10-15% Higher rate : 5% This usage can be satisfied in

SVC.

Error resilience performance for BCAWZ Without error resilience

○ Sharply drops With error resilience

○ Effectively recover from backward-channel delay or erasure.

Conclusion Present a low-complexity video encoding framework for surveillance video

compression and transmission to address the tradeoff between computational complexity and coding efficiency.

BCAWZ can achieve significantly higher coding efficiency than H.264/AVC INTRA coding as well as existing Wyner–Ziv video coding methods and is close to H.264/AVC INTER coding, while maintaining similar coding complexity with INTRA coding.

Error resilience scheme for BCAWZ to address the concern of reliable transmission in the backward-channel, which is essential to the quality of video data for real-time and reliable activity and event analysis.