Video Codec Comparison Report

8/8/2019 Video Codec Comparison Report

1/17

Video compression: Performance of

available codec software

1 Introduction1.1 DigitalVideoA digital video is a collection of images presented sequentially to produce the effect of continuous

motion. Ittakesadvantageofthespatiotemporalpropertiesofthehumaneyetosimulatecontinuityin

motion. Thepersistenceofthehumaneyeissuchthatnanosecondsofexposuretoanimageresultsin

millisecondsofimageontheretina.Hence, imagesplayedataspeedgreaterthanamillisecondwould

appeartobecontinuous.Ingeneral,theeyecannotdifferentiatebetweenindividualimageswhenthey

areplayedatarateof25persecondorhigher. Severalstandardsfortelevisionexist,whichdefinethe

framerateofthevideobeingdisplayed.SomeofthemaretheNTSC,PALetc.Theframeratevariesfrom

25fpsto60fpsdependingonthestandard.Thevideofileconsistsoftheindividualimages(alsoknownasframes)andthesequencinginformation.

1.2 TheSizeBarrierConsideravideothatisbeingplayedoutattherateof30 imagespersecond.Fora640x480grayscale

video represented in the raw lossless format, thatwouldbe640x480x30bytesper second. Fora30

minutevideo,thiswouldbeapproximately16GB.Foracolourvideo,usingthreebytesperpixel,that

wouldbe48GBnotevenincludingtheaudioandthesequencinginformation. Thisisalmostthesizeof2

blueraydiscsforasmallsizedSDvideo.ForsomemodernHDtransmissions,theframesizesare ashigh

as1920x1080whichwouldworkouttovideosizesbegreaterthan300GBwhenuncompressed.

1.3 VideoCompressionItisimpossibletoevenimaginetransmittingvideosofsuchhugesizes.Toreducethesizeofthevideo

tomanageableproportions,thevideosareusuallyneverstoredortransmittedintherawformat.Even

in situations where compression is not required, the video is still compressed. This is because, the

human eye is insensitive tohigher frequenciesandminute variations in colour and transmitting this

informationwouldbeawasteofresources.Everyvideoissubjectedtosomekindofcompression.The

compressionmethodisbasedupontheapplicationandbandwidthconstraintswherethevideoisused.

Compressionmaybeclassified

I. Baseduponthereproducibilityintoa. Losslesscompression


2/17

As the name indicates, videos compressed using thismethod can be reproduced to the

original content without any change in data. Some methods which perform lossless

compression are Huffman coding, Run Length Coding etc. The amount of compression

achievedusingthesemethodsisverylesscomparedtothoseachievedusinglossmethods(

whicharediscussednext).Further, theamountof compression isalsogreatlydependent

uponthecontentofthevideo.

b. LossycompressionThiscompression isperformedbydropping informationwhichdoesnotsignificantlyaffect

thevisualizationofthevideo.Forexample,thehumaneyeisinsensitivetohighfrequencies

and also does not recognizeminor variations in colours. Hence this information can be

droppedwhileencodingthevideo.MethodssuchasJPEGperformlossycompression.

II. Baseduponwherethecompressionisperformedintoa. Intraframecompression

Thismethod

takes

advantage

of

the

spatial

redundancy

present

in

each

frame

of

the

video and compresses each frame based upon one of the compressionmethods. In

general, indoorvideoshaveauniform,nonchangingbackgroundandhaveaveryhigh

spatialredundancywhichcanbegreatlycompressed.

b. InterframecompressionThis method identifies the temporal redundancies between consecutive frames in a

videoandattemptstoremovethem.Usually,videosdonothavemuchscenechanges

andhencewillhavealotoftemporalredundancy.

UsuallygoodvideoformatsimplementbothInterandintraframecompressiontechniques.

1.4 VideoencodingformatsA video encoding format is a representation for compressed video. Such a format specifies the

representation of each frame, the sequencing information between frames and compression and

decompressionmethodsforinterandintraframeredundancy.

Althoughmaximumcompressionistargeted,usually,allformatshaveacertainamountofredundancy

inthem.Thisistomaintainperformanceinenvironmentswherethereisframedroppinganddataloss.

Errorpropagation resistancemechanismsarepartof thespecificationsofallvideoencoding formats.Thesealsoassistinseekingofdata.Withoutthese,everytimewewatchamove,wewouldhavetostart

fromthebeginningwithoutbeingabletocueforward.


3/17

Somepopularformatsforvideoencodingare:

wmv Mpeg1 Mpeg4 Asf

1.5 ContainerformatsContainerformatsaredifferentfromencodingformats.Theyholdcombinationsofthevideoandaudio

encoded formats. They specify the bitrates of the audio and video and help maintain the

synchronizationbetween the audio and video. Some containers aredesigned to holdonly a specific

combinationofaudioandvideowhilesomearecapableofholdingseveralcombinations(butonlyone

combinationata time).Twopopular container formatsare .aviand .wmv. .avi canbeused tohold

severalvideoformatsincludingmpeg4andmpeg1

Somevideoencodingformatsarecontainersinthemselvesandarecapableofholdingbothaudioand

video.Forexample,MPEG1,MPEG4

1.6 CodecAcodec isanacronym forCoderDecoder. It iscapableofencodinga setof images intoavideoand

decodingavideo intoasetof images.Each imageusuallyconstitutesa frame in thevideo.However,

severaladditionalframesareaddedforthereasonsdiscussedearlier.

Eachcodeciscapableofworkingwithonlyaspecificvideoformat.However,severalcodecscanexistfor

asingleformat.Usually,eachmultimediacompanyformathasitsowncodecforitsplayerforaformat.

Forexample, theora,movareallcodecs for thempeg4 format.Codecs canbe ineither softwareor

hardware.Thesoftwarecodecsareslowerandinexpensiveascomparedtothehardwarecodecswhich

aremuchfaster.

The specifications for a format are not rigid and provide for some variations. Although codecs

implement a specified format, theymay vary in theirmethodof operation resulting in variations in

qualityandperformance.

2 CodecEvaluationWith the ever increasing need for bandwidth, codec designers tend to be over greedy and design

algorithmswhichmightbadlyaffecttheaesthecityofthevideocontent.Hence,evaluationcriteriafor

codecperformancesarerequiredtoverifythequalityofthecompressedvideos.

2.1 CriteriaforComparisonThecodecsarecomparedbasedonthefollowingcriteria


4/17

1. QualityofVideo2. Performanceofthecodec

2.2 QualityofvideoQualityofvideocorrespondstothelookandfeelofthevideo,theresolution,theartifacts,theblurring

andothervisualaestheticcomponents.Thequalityofvideodependsonboththe formatofthevideoand the codec used to encode to that format. Usually, several codecs implement a single format.

However,eachonediffers fromtheother.Qualityalsodependsontheamountof informationonthe

video being encoded. Also, the performancewill not be constant throughout the video. Clipswith

higherinformationhavemoreartifactsthansceneswithlittlemovementandscenechanges.Qualitycan

bemeasuredasobjectiveorsubjective.

2.2.1 ObjectiveQualityObjectivequalityistomeasurethequalityinmathematicaltermswhichmakesitveryeasytocompare

andevaluate.Someofthemetricsavailabletomeasureobjectivequalityare:

a. MeanSquareError(MSE):Itisthesecondmomentofthedifferenceanddescribesthevariancebetweentheoriginalframeandtheencodedframe.

b. PeakSignaltoNoiseRatio(PSNR):Theratiobetweenthemaximumsignallevelandthenoise.Mathematically,itisgivenby:

c. ColourDifference:Thisistheabsolutedifferenceoftheindividualcolourcomponentsbetweentheinputframeandtheoutputframe.Itiscalculatedby

d. StructuralSimilarity(SSIM)[2]Thisisusedtomeasurethesimilaritybetweentwoimages.Itisanumberbetween0and1. Itisafunctionofluminance,contrastandstructuralsimilarity.Itis

independentofthecolourcomponents.

2.2.2 SubjectiveQualitySubjectivequality ismeasuredbyvisually inspectingtheencodedvideo forartifacts,blurring,blocking

andoverallquality.


5/17

2.3 PerformanceoftheCodecTheperformanceofthecodecismeasuredasafunctionofthreequantities

1. Compressionratioofthecodec(Filesize)2. Speedofencoding(compression)3. Speedofdecoding(decompression)

2.3.1 CompressionratioofthecodecThecompressionratioofthecodecismeasuredbyencodingarepetitivesetofframesusingthecodecs

toyieldvideosofdifferentformats.Thefilesizeoftheencodedvideototheuncompressedvideowill

actasameasureofthecapacityforcompression.Byselectingappropriateframestocompress,wecan

measureboththebestandworstcasescenarios.

2.3.2 EncodingandDecodingspeedTheencodinganddecoding speedvary fromcodec tocodecandwithin the samecodec fordifferentframes.Higher the redundancy, slower the encoding and smaller is the size of the file. By selecting

appropriateframestocompress,wecanmeasureboththebestandworstcasescenarios.

2.4 BitRatesBitRateismeasuredinKiloBitspersecondandrepresentstheamountofdataflowperunittime.Itis

animportantfactorthatdecidesthequalityofthevideo.Forexample,consideravideowhichhasabit

rateof1000KbPS.Fora standarddefinitionvideo, thiswouldmean that therewouldbeabout29.8

framesinthe1000KiloBitsi.eabout33KiloBitsperframeor4Kilobytesperframe.Thisrestrictsthe

amountofdatathatcanbeusedtorepresentaframe.Lowerbitratesmeanhighercompressionand

lowerqualityofvideo,morenoise,blocking,discolourationetc.

Application BitRatesa Videostreaming 100500KbPS

b SDvideo 5002000KbPS

c HDvideo >2000KbPS

Bymeasuringeachofthequantitiesdiscussedin2.3and2.4,wewillbeabletoidentifytheappropriate

codecforaspecificapplication.

3 Implementation3.1 CodecsThefollowingcodecsarebeingevaluatedinthisstudy


6/17

Sl.No Codec Designer/Developer Format Container1 WMV2 Microsoft wmv wmv2 Theora Xiph.org MPEG4 avi3 Asf Microsoft asf asf4 MPEG4 MPEG MPEG4 mp45 Quicktime Apple MPEG4 mov6 MPEG1 MPEG MPEG1 mpeg

Allcodecsarepartoftheffmpeglibrary.

3.2 DatasetThefollowingvideosareusedforevaluationofthecodecs.Thereasonforselectionofthevideoisalso

described.Allvideosareof352x288pixeldimension,butmayappearstretchedinthisdocument.

3.2.1 QualityMeasurement3.2.1.1Akiyo

Figure1 AframefromtheAkiyovideosequenceThis isa300framevideointheuncompressedYUVformat.Thisvideoshowsanewsreader.It hasno

backgroundchangesandalmostnegligibleforegroundchanges.


7/17

3.2.1.2Foreman

Figure2AframefromtheForemanvideosequenceThisisa300framevideointheuncompressedYUVformat.Thisvideohasasuddenscenechangeatthe

end.Other

than

that,

there

is

no

background

change.

Only

the

face

shows

rich

emotions

which

can

be

hardtocompress

3.2.1.3Football

Figure3AframefromtheFootballvideosequenceThisisa125framevideointheuncompressedYUVformat.Thisvideohasaconstantbackgroundanda

veryrapidandlargechangeintheforegroundasplayerkeepcominginandgoingoutoftheframes.


8/17

3.2.1.4Stephan

Figure4 AframefromtheStephanvideosequence

ThisisarecordingofStephanEdbergstennismatch.Thisis300framesinlengthandisalsointhe

uncompressedYUV

format.

This

video

has

a

fast

foreground

change

as

the

player

runs

about,

and

a

backgroundchangeasthecamerafollowshim.Thiswouldbethehardestkindofnaturalvideoto

encode.

Video ForegroundChange BackgroundChangeAkiyo

Foreman Football Stephan

3.2.2 PerformanceMeasurementIn order tomeasure the performance in terms of compression ratio and speed of encoding, I have

proposedasetofframesasshownbelow.Theseframeswilltogetherallowustomeasurethebestand

worstcasescenarios.


9/17

Alternateframes SpatialRedundancy TemporalRedundancy3.2.2.1

100% 100%

3.2.2.2100% 0%

3.2.2.3=0% 100%

3.2.2.4=0% =0%

Thesepairsofalternatingframesincorporatethebestandworstcasescenariosforcompression.

3.2.3 BitRatesInordertocovertheentirerangeofapplications,thevideoswillbeencodedtothefollowingBitrates:

a. 600KbPSThisistherangeatwhichyoutubeplaysitsvideos.

b. 1,000KbPSThisisthebitratesgenerallyusedinvideoconferencing

c. 3,000KbPSThesebitratesaregenerallyusedinopticaldiscplaybacks.

4 ResultsandDiscussionAspartoftheexercise,Iwasabletomesuremostoftheevaluationparameters.However,duetoissues

withtheffmpeglibrary,Ididnotgetanaccuratemeasureofthecodinganddecodingtimes.

4.1 Akiyo


10/17

Akio MSE

1.5

2

2.5

3000

wmv2

theora

asf

mpeg41

qt

0.5 mpeg1

0

600 1000

Figure5MeanSquaredErrorforAkiyo

Akio PSNR

48

wmv247

theora

43

44

45

46

1000 3000

asfmpeg4

qt

mpeg1

600

Figure6PSNRforAkiyo

Akio Absolute Color Distance

0

0.2

0.4

0.6

0.8

1

2

3000

1.

wmv2

theora

asf

mpeg4

qt

mpeg1

600 1000

Figure7AbsoluteColourdistanceforAkiyo

Akio SSIM

0.9965

0.997

0.9975

0.998

0.9985

0.999

0.9995

1

3000

wmv2

theora

asf

mpeg4

qt

mpeg1

0.996

600 1000

Figure8StructuralSimilarityforAkiyo


11/17

F ootballMS E

2

3

4

5

6

7

8

9

10

3000

wmv2

Theora

as f

mpeg4

Qt

mpeg1

1

0

600 1000

Figure9MeanSquaredErrorforFootball

F ootballPSNR

36

600

37

38

39

40

41

42

43

1000 3000

wmv

Theora

as f

mpeg4

Qt

mpeg1

F ootballAbs oluteColorDistance

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

600 1000 3000

wmv2

theora

as f

mpeg4

Qt

mpeg1

F ootballSS IM

0.88

600 1000

0.9

0.92

0.94

0.96

0.98

1

3000

wmv2

Theora

as f

mpeg4

Qt

mpeg1

Figure10PSNRforFootball

Figure11AbsoluteColourDistanceforFootball

Figure12StructuralSimilarityforFootball


12/17

ForemanMSE

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

600 1000 3000

wmv2

Theora

as f

mpeg4

Qt

mpeg1

Figure13MeanSquaredErrorforforemanForemanPS NR

39

40

41

42

43

44

45

46

47

600 1000 3000

wmv2

Theora

as f

mpeg4

Qt

mpeg1

Figure14PSNRforforemanForemanAbsoluteColorDistance

0

0.5

1

1.5

2

2.5

3

600 1000 3000

wmv2

theora

as f

mpeg4

Qt

mpeg1

Figure15AbsoluteColourDistanceforforeman

F oremanSS IM

0.975

0.98

0.985

0.99

0.995

1

1.005

600 1000 3000

wmv2

Theora

as f

mpeg4

Qt

mpeg1

Figure16StructuralSimilarityforforeman


13/17

Stephan MSE

0

2

4

6

8

10

600 1000 3000

wmv2

theora

asf

mpeg4

qt

mpeg1

Figure17MeanSquaredErrorforStephan

Stephan PSNR

36

37

38

39

40

41

42

43

44

45

600 1000 3000

wmv2

theora

asf

mpeg4

qt

mpeg1

Figure18PSNRforStephan

Stephan Absolute Color Distance

0

1

2

3

4

5

600 1000 3000

wmv2theora

asf

mpeg4

qt

mpeg1

Figure19AbsoluteColourDistanceforStephan

Stephan SSIM

0.88

0.9

0.92

0.94

0.96

0.98

11.02

600 1000 3000

wmv2

theora

asf

mpeg4

qt

mpeg1

Figure20StructuralSimilarityforStephan


14/17

0 200 400 600 800 1000 1200 1400 1600

Size in KB

1

B-W File Sizes

0 100 200 300 400

Size in KB

1

W-W File Sizes

mpeg-1

Qt

mpeg-4

asf

theora

w mv2

Figure21 Filesizeswithhighspatialandlowtemporalredunda

Figure22Filesizeswithhighspatialandtemporalredundancy

Figure24 FilesizeswithlowspatialandlowtemporalredundaFigure23Filesizeswithlowspatialandhightemporalredundancy

0 500 1000 1500 2000 2500 3000

Size in KB

1

C-C File Sizes

mpeg-1

Qt

mpeg-4

asf

theora

w mv2

0 5000 10000 15000 20000

Size in KB

1

C-N File Sizes


15/17

Followingaresomesampleframesfromtheencodedvideos

Figure25 CounterClockwisefromthetopaframefromtheStephanvideooriginalframe,wmvencodedat600kbpsandwmvencodedat3000kbps

Figure26 CounterClockwisefromthetopaframefromtheAkiyovideooriginalframe,wmvencodedat600kbpsandwmvencodedat1000kbps


16/17

InFigure25,thedistortionisclearlyvisiblewhenencodedat600kbps,butat3000kbps,itisalmost

negligible. However,inFigure26,thereisnovisibledistortionevenat600kbps.Thisimpliesthatthe

encodingprocessissensitivetothecontentofthevideoalso.

5 ConclusionSelection of a format for encoding or representation depends upon the applicationwhich uses the

video.Thevariouscriteriatobeconsideredbeforeselectingaformatare:

Applicationo Transmission

Videos used for transmission and viewing over the internet require a high

compression ratio.They can compromiseon thequalityas suchvideosare rarely

usedforimportantapplications.

o VideoConferencingVideoconferencingapplicationshavespecificcriteriawhenitcomestoquality.They

need thevideostobeclear,butthe frameratecanbecompromised.Surveillance

videos also fall into this category. The encoding and decoding speed are of

significancehere.

o ArchivingVideos used for this purpose do not have significant demands on encoding or

decodingspeed.Theyrequirehigherresolutionandqualitywithlowerfilesizes.

PerformanceRequirementso RealtimevideoprocessingforUAVsetc.

Therequirementhereisforfasterencodingspeedandverylittleblurring

o VideoViewingVideo viewing, in general, does not have much processing requirements. This is

because of the availability of sufficient processing capability and non real time

natureoftheapplication.

Qualityrequirementso Entertainmento Conferencingo Surgicalprocedures

6 FutureWorkPossiblefutureworkincludes


17/17

a. Measuringblurringeffectsofthecodecsb. Measuringblockingeffectsandimpactonedgedetectionalgorithmsc. Evaluatingcodinganddecodingtimes.d. Identifyingimpactofframesizeoncodingspeedandcompressionratio.

7 References[1] Madhuri Khambete, and Madhuri Joshi, Blur and Ringing Artifact Measurement Image Compression

using Wavelet Transform, PROCEEDINGS OF WORLD ACADEMY OF SCIENCE, ENGINEERING AND

TECHNOLOGY VOLUME 20 APRIL 2007 ISSN 1307-6884

[2] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli, Image Quality

Assessment: From Error Visibility to Structural Similarity , IEEE TRANSACTIONS ON IMAGEPROCESSING, VOL. 13, NO. 4, APRIL 2004

Documents

Video Codec Comparison Report