Error Control and Concealment for Video Communication

Error Control and Concealment for Video Communication

CMPT820

Summer 2008

Michael Jia

“Error Control and Concealment for Video Communication : A

Review”

YAO WANG, Member, IEEEQIN-FAN ZHU, Member, IEEE

Reference

Introduction

Error Detection

Error Concealment at Coder

Error Concealment at Decoder

Interactive Error Concealment

Conclusion

Outline

2 Types of Transmission Errors

Random Bit Errors

Bit inversion, bit insertion, bit deletion

Erasure Errors

Packet loss, burst errors, system failures

More destructive

Common usage of VLC makes them no differences in video

streaming

Introduction

Lossless recovery

FEC (Forward Error Correction)

ECC (Error Control Coding)

ARQ (Automatic Retransmission Request)

Not necessary for video transmission, human eyes can

tolerate a certain degree of distortion

Focus on signal-reconstruction and error-concealment

techniques

Introduction

Typical structure of a video communication system

Introduction

3 groups of error-concealment techniques Forward error concealment

Encoder plays the primary role Error concealment by post-processing

Decoder fulfills the task Interactive error concealment

Main concerns Effectiveness Required delay Bit-rate overhead Processing complexity

Introduction

Introduction

Error Detection




Conclusion

Outline

At transport layer

Adding header information (sequence number)

H.223

FEC (Forward Error Correction)

H.223, H.261

More reliable

Need more bandwidth

Error Detection

At decoder Based on characteristics of natural video signals

Pixel value differences of neighboring lines (compare to a threshold) Differences between boundary pixels in a block and its four neighbor

blocks Obvious false value of quantization step size or DCT coefficients Insert synchronization code word at the end of line of blocks

No additional bits or very few Rely on smoothness of signal

Error Detection

Introduction

Error Detection




Conclusion

Outline

Layered Coding with Transport Prioritization Most popular and effective (MPEG-2)

Error Concealment at Coder (1)

Layered Coding Base layer – most important layer, with acceptable quality

Transport prioritization Deliver base layer with higher degree of error protection High priority channel Re-transmission and/or FEC

No explicit bit-rate overhead Complicate structure and coding overhead H.264 AVC/SVC

Redundant pictures Data Partitioning


Multiple-Description Coding (MDC)

Several parallel channels Independent error events Small probability of “all channels down”

Multiple “Descriptions” Several coded bit streams Transmitted over separate channels Any one will work


Multiple-Description Coding (MDC)


Joint Source and Channel Coding

Source-channel interaction at a lower level

Given channel error characteristics, design quantizer and entropy

coder for to minimize the effect of errors

For general sources, noisy channel coarse quantizer is better

For image signals, noisy channel

fewer bits to high-frequency coefficients

more bits to low-frequency coefficients


Robust Waveform Coding

Intentionally keep some redundancy in source-coding

Layered coding and MDC both belong to this category

Adding auxiliary information in waveform coder

MPEG-2: sending motion vectors for microblocks in I-frames

Restricting prediction domain

H.263/H.264: prediction is confined within each slice


Robust Entropy Coding

Add redundancy in entropy-coding

To help detect bit errors and prevent error propagation

Self-Synchronizing entropy coding

Add a synchronization code word

H.261, H.263, MPEG-4

Error-Resilient entropy coding (EREC)

MPEG-4 uses RVLC (reversible VLC)


Transport-Level Control

Add redundancies at transport level

Prioritized transport for layered coding

Robust packetization

Spatial block interleaving

Dual transmission of important information

H.264 AVC/SVC

NAL unit syntax structure

Parameter Sets


Error Concealment at Coder (Summary)

Introduction

Error Detection




Conclusion

Outline

Preview

Perform error concealment at the decoder

Can be used in conjunction with the auxiliary information provided

by the source coder

Low frequency components dominate images of natural scenes

Color values of adjacent pixels vary smoothly except sharp edges

Human eyes tolerate more distortion to high-frequency

components


Motion-Compensated Temporal Prediction

Replace damaged MB with the motion compensated block

Very effective when all the motion information in the base layer

Widely used (MPEG-2)

What if motion information or coding mode is lost?

Will discuss in next slide

Error Concealment at Decoder (1)

Recovery of Motion Vectors and Coding Modes What if we lost motion vectors or coding modes?

Interpolated from spatially and temporally adjacent blocks Estimation of coding modes

Simple: treat as ‘intracoded’ More sophisticate: MPEG-2 (See tables)


Estimation of motion vectors

Set to zeros (works well for low motion video)

Use MV of the corresponding block in the previous frame

Use the average of MVs from spatially adjacent blocks

Use the median of MVs from spatially adjacent blocks

Select one of the above methods depending on least boundary

matching error


Maximally Smooth Recovery

A constrained energy minimization approach

Minimize a measure of spatial and temporal variation between

adjacent pixels in this block and its spatially and temporally

neighboring blocks

Measure differences in 3 domains

Spatial – adjacent blocks

Temporal – prediction block in previous frame

Frequency – received coefficients for this block


Maximally Smooth Recovery 2 sample spatial smoothness measures


Spatial- and Frequency-Domain Interpolation

A coefficient in a damaged block is likely to be close to the

corresponding coefficients in spatially adjacent blocks

Interpolate from four neighbor blocks

Maybe not accurate (8-pixel is too far)

Interpolate from four 1-pixel-wide boundaries

2 pixels in 2 nearest boundaries

4 pixels in all 4 boundaries

See picture in next slide


Spatial- and Frequency-Domain Interpolation Interpolate from four 1-pixel-wide boundaries


Error Concealment at Decoder (Summary)

Introduction

Error Detection




Conclusion

Outline

Preview

If a backward channel is available, can achieve better performance

by cooperation

Based on the feedback

At source coder – coding parameters can be adapted

At transport level – adjust the portion of bandwidth used for error control

Decoding delay issue


Selective Encoding for Error Concealment

Simple – code next frame in intramode

Error stopped in about one round-trip time

Will cause bit-rate increase

Send identity info back, perform error concealment at the same

time

Continue encode without using the affected area

Perform same error concealment procedure (need a prediction frame

buffer)

See picture in next slide

Interactive Error Concealment (1)





H.263 uses more prediction frame buffers (reference picture selection mode)


Adaptive Transport for Error Concealment

Retransmission is unacceptable for real-time video applications?

Not always

For one-way video broadcast, we may tolerate a few seconds delay

For multipoint video conferencing, use MCU (multipoint control unit)

If retransmission is controlled properly, end-to-end quality can be

improved

Both H.323 and H.324 defined such mechanisms


Retransmission Without Waiting

Wait for the retransmission data

Not good, may freeze the display

Cause transit delay and accumulation delay

Without waiting

Request the retransmission

Conceal the error

Track the affected pixels

Correct them upon the arrival of the retransmission data


Prioritized, Multi-copy Retransmission

Effective in very lossy channels

Video streaming via PSTN

Are you kidding me?

Send multiple copies of a lost packet

Use layered coding

# of retransmission trials and # of copies are proportional to the

importance of the layer


Interactive Error Concealment (Summary)

Introduction

Error Detection




Conclusion

Outline

Conclusion

Real-time video communication doesn’t require lossless

delivery; signal-reconstruction and error-concealment

techniques are more effective.

Add redundancy when encoding or delivering

Estimate missing information when decoding

Inform sender what is lost

Burstiness has a significant impact on the choice of algorithms

Questions?

Thank You

Documents

Error Control and Concealment for Video Communication