High Efficient Distributed Video Coding with Parallelized Design for Cloud Computing

  • View
    26

  • Download
    0

Embed Size (px)

DESCRIPTION

High Efficient Distributed Video Coding with Parallelized Design for Cloud Computing. . Cheng, Han-Ping Advisor: Prof. Wu, Ja -Ling 2010/6/2. Outline. Introduction DISPAC video codec RD performance of DISPAC Parallelizing DISPAC decoder - PowerPoint PPT Presentation

Transcript

komod Master Thesis Defense

High Efficient Distributed Video Coding with Parallelized Design for Cloud ComputingCMLab, CSIE, NTU1Cheng, Han-Ping Advisor: Prof. Wu, Ja-Ling 2010/6/21OutlineIntroductionDISPAC video codecRD performance of DISPACParallelizing DISPAC decoderDecoding speed of DISPACConclusions and future workCMLab, CSIE, NTU2Distributed Video Coding?Cloud ComputingDISPAC codecDISPACRD performanceDISPAC decoderdecodingconclusionfuture work2Trends of Cloud ComputingCloud Computing makes Clients slimmer&thinnerCMLab, CSIE, NTU3

cloud computingclientinternetclientthin client3Video Coding in Cloud ComputingOnly need low complexity encoder and decoder at client sideConventional video coding (e.g. H.264)Encode once, decode many timesLow complexity decoder

Distributed Video Coding (DVC)e.g. Video surveillance, wireless sensor networkLow complexity encoder

CMLab, CSIE, NTU4video codingclientdecoderencoder

standard codecH.264decoderencoderencode oncedecode many timesapplication

applicationvideo surveillance, wireless sensor networkencoderdecoder

DVCDVCenocder4Distributed Video CodingSlepian-Wolf Theorem (1973)

Wyner-Ziv Theorem (1976)CMLab, CSIE, NTU5RX H(X)Source XSource YDependency exists but is not exploitedJoint DecoderXYEncoder XEncoder YRY H(X)RX + RY?RX + RYH(X, Y) Source XSource YStatistical dependencyJoint EncoderRX H(X)Joint DecoderXYConventional video coding paradigmRY H(Y)Slepian&Wolf : H(X, Y) !!DVCbased onSlepianWolfSlepian-Wolf TheoremWynerZivWyner-Ziv Theorem

Slepian-Wolf Theorem

information theorysource X,H(X)bitsource, H(X) + H(Y)bitXYjoint encodersourcevideo codingintra prediction,motion estimationxycorrelationredundancysourcejoint encodebitH(X,Y)X,Y

Slepian Wolf encodejoint encodeH(X,Y)sourcedecodejoint decodeencodesourceencode encode

5Distributed Video CodingWyner-Ziv Theorem (1976)Extend to lossy codingCMLab, CSIE, NTU6Dependency exists but is not exploitedJoint DecoderEncoder XSource XSource EncoderXSource DecoderVirtual channelEncoder YSource YYSource EncoderSource DecoderSide information estimation

XDVC is also called Wyner-Ziv (WZ) video codingQuantizerQuantizerChannel EncoderChannel DecoderYChannel EncoderChannel DecoderNoisy Channel XXX+P(X+P)Channel coding (Error Control Code):RX + RYWyner&Ziv : H(X, Y) !RY H(Y)RX H(X|Y)Correlation is exploitedPWyner-Ziv TheoremSlepian-Wolf

sourcequantizeH(X,Y)XY

sourcedecoderYH(Y)decoderdecoderYdecodeYdecode decoderY Xside informationXH(X|Y)information theory H(X|Y)+H(Y)H(X,Y)

channel codingWyner Zivchannel codingsource Xnoise channelchannel encoderXchannelredundant bitparity bitPsourcechannelnoiseX+PnoisechannelerrorChannel decodersource Xchannel codeerror control code

DVCchannel codingXchannel encoderXparity bitdecoderDecoderchannel decoderXparityXside informationXXdecodeside informationXnoisevirtual channelerrorXside informationXXparityparity bitsourceparitychannel codechannel codeside informationparity

DVCDVCWyner-Ziv video codingWZ video coding6Video Coding in Cloud ComputingWZ to H.264 video transcoderCMLab, CSIE, NTU7 WZ to H.264 Transcoder

CloudComputational Resource

WZ encoder(Low Complexity) H.264 decoder(Low Complexity)

WZ encoded bitstreamH.264 encoded bitstreamCloud ComputingWZ encoderH.264 decoderWZ to H.264video transcodervideo coding7MotivationThere is still a gap between Wyner-Ziv video coding and conventional video coding (e.g. H.264/AVC)

Most reported WZ codecs have a high time-delay in the decoderTrends of parallel computinge.g. Multi-core CPU, GPUParallelizability of the decoder is essentialCMLab, CSIE, NTU8Wyner-ziv video codingvideo codecH.264

WZ codecdecoder time delaydecoder8DISPAC Video CodecDIStributed video coding with PArallelized design for Cloud computing (DISPAC)To better rate-distortion (RD) performanceCombine coding tools developed in recent literatures with some newly developed modules.

To reduce decoding time-delayHighly parallelized decoder.

CMLab, CSIE, NTU9codecDISPAC codecDIStributed video coding with PArallelized design for Cloud computing distributed parallelized cloudDISPAC

codecDVCcoding toolmoduleDVC

DISPAC codecdecoding moduledecodertime delay

9OutlineIntroductionDISPAC video codecRD performance of DISPACParallelizing DISPAC decoderDecoding speed of DISPACConclusions and future workCMLab, CSIE, NTU10state-of-the-art WZ codec10DISPAC Video CodecCombine coding tools of two state-of-the-art WZ codec:DISCOVER codec (Distributed coding for video services)X. Artigas et al., The DISCOVER codec: architecture, techniques and evaluation, PCS, 2007

MLWZ codec (Motion-learning based Wyner-Ziv video coding)R. Martin et al., Statistical motion learning for improved transform domain Wyner-Ziv video coding, IET Image Processing, 2010CMLab, CSIE, NTU11DVCstate-of-the -art codecDISCOVER codecDVC2007codecrelease

MLWZ codecMotion-learning based Wyner-Ziv video codingIETJournal paperDISCOVER codecRD performance

DISPAC codeccodeccodec

11

DISCOVER Video CodecCMLab, CSIE, NTU12Ref. X. Artigas et al., PCS, 2007GOP 2WZKeyWZKeyKeyGOP 4WZDISCOVER codec12QuantizationCMLab, CSIE, NTU13Eight quantization matricesQ116800800000000000Q232800800000000000Q332840840040000000Q43216841684084004000Q53216841684484404400Q66416881688488448440Q76432168321684168448440Q812864321664321683216841684032 = 25=> use 5 bits8 = 23=> use 3 bits0 bits ()quantization tableQ1,Q2,Q3Q8QindexqualityQindexQ1coefficient13QuantizationCMLab, CSIE, NTU14DCT coefficient bandBlock1S11S12S16S17S13S15S18S113S14S19S112S114S110S111S115S116Block2S21S22S26S27S23S25S28S213S24S29S212S214S210S211S215S216Block3S31S32S36S37S33S35S38S313S34S39S312S314S310S311S315S316DCT coefficient band b1: { S11, S21, S31, SN1 }DCT coefficient band b2: { S12, S22, S32, SN2 }DCT coefficient band b16: { S116, S216, S316, SN16 }DC bandAC bandsDISCOVERquantizeframefrequencycoefficientgroupDCT coefficeint bandblockdc coefficientbandAC bandgroupbandzig zigorder

14Bit plane ExtractionCMLab, CSIE, NTU1500100000010000011110Bit planes of DC band:Bit plane 1:Bit plane 2:Bit plane 3:Bit plane 4:Bit plane 5:Channel Encode(LDPCA)4670631773015For each DCT coefficient bandMSBLSBQ43216841684084004000Quantization DISCOVERbandcoefficientbit planeQ4quantization tableDC bandbitplaneDC32DC5bit41blockdc coefficientMSBbit planebitbit plane5bit planebit planechannel encode

15

DISCOVER Video CodecCMLab, CSIE, NTU16Ref. X. Artigas et al., PCS, 2007Dependency exists but is not exploitedJoint DecoderEncoder XSource XXVirtual channelEncoder YSource YYSource EncoderSource DecoderSide information estimation

XQuantizerQuantizerChannel EncoderChannel DecoderYRY H(Y)RX H(X|Y)Pchannel codeLDPCA codeDVCchannel codeworkLDPCAWZ framebit planeparity bitbufferdecodeside information

side informationsourcenoisevirtual channelerrorChannel decoderside informationencoderparity bitsource

DISCOVERside informationside informationframe nblockside informationDecoderfuture frameco-located blockpast framesearchblockMVfmotionlinearblockmotion vectorframeblockblockPF1PF2past frameblockfuture framemotion vectormotion vectorblockblockside information

side informationDVCside informationWZ frameresidual modellaplacian distribution

laplacian distributionparameter

laplacian parameterside informationsourcedistribution

sourcedistributionsourcebit01channel decoderinput

Soft inputinputside informationbitplaneinputbitplanebit

16Side Information CreationCMLab, CSIE, NTU17

XFXBLow pass filter (3x3 Mean filter)Divide frame to 16x16 non-overlapped blocksMotion estimation (search window: 32)

Side Information CreationCMLab, CSIE, NTU18

XFXB

Side Information CreationCMLab, CSIE, NTU19

XFXB(xL, yL )(xu, yu )Adaptive search range:

NNNN(xR yR )(xB, yB )Side Information CreationCMLab, CSIE, NTU20

XFXBHalf pixel motion estimationSide Information CreationCMLab, CSIE, NTU21

XFXB

Weighted vector median filter:

x1x2x3x4x5x6x7x8x9Spatial motion smoothingMSE2Side Information CreationCMLab, CSIE, NTU22

XFXB

Weighted vector median filter:

x1x2MSE1Side Information CreationCMLab, CSIE, NTU23

XFXB

Weighted vector median filter:

x1

The result of x6 is minimumxwvmf = x6 (Final motion vector ! )Side Information CreationCMLab, CSIE, NTU24

XFXB

Weighted vector median filter:

x6Side Information CreationCMLab, CSIE, NTU25

XFXB

Block interpolation ( 0.75*XB + 0.25*XF )

Bidirectional motion compensation

DISCOVER Video CodecCMLab, CSIE, NTU26Ref. X. Artigas et al., PCS, 2007Laplacian Distributionchannel codeLDPCA codeDVCchannel codeworkLDPCAWZ framebit planeparity bitbufferdecodeside information

side informationsourcenoisevirtual channelerrorChannel decoderside informationencoderparity bitsource

DISCOVERside informationside informationframe nblockside informationDecoderfuture frameco-located blockpast framesearchblockMVfmotionlinearblockmotion vectorframeblockblockPF1PF2past frameblockfuture framemotion vectormotion vectorblockblockside information

side informationDVCside informationWZ frameresidual modellaplacian distribution

laplacian distributionparameter

laplacian parameterside informationsourcedistribution

sourcedistributionsourcebit01channel decoderinput

Soft inputinputside informationbitplaneinputbitplanebit

26CNM Parameter EstimationCMLab, CSIE, NTU27

XFXB

Residual frame generation:

R

CNM Parameter EstimationCMLab, CSIE, NTU28

Residual frame DCT transform : (4x4)

RTz

25810-30

120