87
AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of Ch nstitute of Digital Media Technology, Peking Univer Oct.12, 2010

AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Embed Size (px)

Citation preview

Page 1: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS – Technology, IPR and Applications

Prof. Huang TiejunAudio and Video coding Standard Working Group of China

Institute of Digital Media Technology, Peking UniversityOct.12, 2010

Page 2: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Video coding…

• Compress high bit-rate digital video signal as low bit-rate stream– SD: from ~250Mbps to < 5Mbps– HD: for >1Gbps to < 20Mbps

• is the reason makes DTV possible at 1994

• and always a core concern for Digital TV system and any other digital video applications

Page 3: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

3

Video coding Standards

• ISO/IEC recommended MPEG series standards– MPEG-1, MPEG-2, MPEG-4

• ITU-T recommended H.26X series standards– H.261, H.262, H.263, H.263+, H.263++

MPEG-China AVS

H.264 /MPEG-4 AVC

China

1G standards1G standards 2G standards2G standards

Page 4: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

MPEG-2: the 1st Generation

• ISO/IEC 13818 / ITU-T H.262 @ 1994– SDTV at ~5Mbps– HDTV at ~20Mbps

• Makes DTV became reality– Europe DVB at 1997– USA ATSC at 1997– Japan ISDB at 1998– China DTMB at 2006

Page 5: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

55

AVS – one of the 2nd Generation• In Feb2006, AVS video

issued as Chinese national standard– GB/T 20090.2-2006– Effective from Mar2006

• In Aug2006, DTMB issued as Chinese national standard– GB/T 20600-2006– Mandatory from Aug2007

Page 6: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 6

2ndG Video – ITU-T IPTV Selection

• ITU-T IPTV FG decided to recommended the following video format at May2007– H.264/ MPEG-4 AVC– MPEG-2 (for Legacy)– AVS – GB/T 20090.2– SMPTE VC-1

• ITU-T Q6-WP3-SG16 meeting at last week (10July09) decided to release upper document as the ITU Technical Paper on “Media coding toolbox for IPTV”

Page 7: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

HD: 20 MbpsSD:4-6Mbps

HD: 6-8MbpsSD:1-2.5Mbps

7

Video Compression Performance Improvement

19911994 2003 2030

150

250

50

Compression Ratio

AVSMPEG-4 AVC/H.264VC-1

MPEG-1

AVS-?/MPEG-?/others-?

MPEG-2

Page 8: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 9: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Higher Performance, High Cost

• Gold age is far away…– MPEG-1,MPEG-2– H.261,H.262, H.263,H.263+/++

• Face to complicated situation nowadays– H.264,VC-1

Page 10: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

10

MPEG Standards

• 1991 : MPEG-1– Royalty Free

• 1994 : MPEG-2– DVD, DTV, Emmy Award– 2.5$ per device

• 1996 : MPEG LA—A licensing agency• 1999 : MPEG-4 SP (Part 2)

– 0.25 $/device, coding time based charge (2 cents/ hours)– Objected by AOL-Times Warner

• 2003 : H.264/MPEG-4 Part 10 AVC– Nov. 17,2003: Licensing policy – title based licensing– Objected by EBU ( EBU Technical Statement-D 96-2003)– May.20,2004: Policy finalized, little revision

Page 11: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

• Leonardo Chiariglione 《 Patents and standards 》– http://www.chiariglione.org/ride/patents_and_standards.htm

• If Gutenberg had lived in a culturally more advanced city like Venice or Florence, which had already started issuing litterae patentes ("open letters", whence the word "patent" comes from) since a few decades, he could have protected his invention with a patent.

• Some time later, MPEG had to deal with MPEG-2 patents. Many companies had participated in the MPEG-1 development not so much because they necessarily had plans to make products in that space, but because they were keen to be around and help tune up the MPEG machine for the big MPEG-2 deal.

• In January 1993 I took the initiative to invite to the Rome meeting Mike Smith, head of the Information Technology Task Force (ITTF), the group inside the ISO Central Secretariat in Geneva looking after JTC1 matters. Mike provided advice on how this difficult issue could be treated without trespassing the limits set by the ISO/IEC directives.

• In the event, an MPEG-2 Patent Pool was established, managed by MPEG LA. This organisation acts as a "one-stop shop" for MPEG-2 Video and Systems patents (Audio is not part of the licensing), but there are a few companies licensing their technology separately.

First Effort to deal with patent issue for MPEG-2

The result is reasonable (4$ - 2.5$/device)

Page 12: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

网址 http://www.mpegla.com

Page 13: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 13

AVC/H.264 licensing structure

Source:AVC/H.264 license terms

CodecManufacturers

Subscription

Title-by-Title

Internet Broadcast

Free Television

Participation FeesWhere End User pays

Participation Fees whereRemuneration is fromOther sources

Page 14: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 14

AVC/H.264 License Terms:Participation Fees

• Where End User pays for AVC Video– Subscription (not limited by title) – 100,000 or fewer subscribers/yr = no

royalty; > 100,000 to 250,000 subscribers/yr = $25,000; >250,000 to 500,000 subscribers/yr = $50,000; >500,000 to 1M subscribers/yr = $75,000; >1M subscribers/yr = $100,000

– Title-by-Title - 12 minutes or less = no royalty; >12 minutes in length = lower of (a) 2% or (b) $0.02 per title. 

• Where remuneration is from other sources– Free Internet Broadcast (not title-by-title, not subscription) – no royalty

during first term (through 2010; not greater than free television thereafter)– Free Television - (a) one-time $2,500 per transmission encoder or (b)

annual fee starting at $2,500 for > 100,000 HH rising to maximum $10,000 for > 1,000,000 HH

• Enterprise cap: $3.5M/yr 2006-07, $4.25M/yr 2008-09, $5M/yr 2010

• Royalties begin January 1, 2006Source:AVC/H.264 license terms

Page 15: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Effort for 2G: RF JVT baseline

AVS Strategy turned in 2003For JVT baseline + essential enhanced tools

To a dependent, compact, patent clear standard

Page 16: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

MPEG LA 专利池构成与风险分析

Page 17: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,
Page 18: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 18

AVS Mission

• To develop a 2nd Generation video coding standard with same/better coding performance than others

• Avoids licensing risk based on clearly analysis of related patents in last 50 years

• To help DTV, IPTV and new media operators in China and out of China leave from more and more dangerous licensing mire

• AVS was born at Mar.2002

Page 19: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 20: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS Working Group

• Set up by MII (Ministry of Information Industry) in June of 2002

• In charge of national standardization of audio-video coding for Chinese A/V industry

• Also in charge of organizing the delegation of China National Body to MPEG(ISO/IEC JTC1/SC29/WG11)

Page 21: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS membership

0

20

40

60

80

100

120

140

160

180

2002 2003 2004 2005 2006 2007

TotalNewEnd

Page 22: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

国内企业 Local Members

Page 24: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS Subgroups

• System

• Video

• Audio

• Digital Media Rights Management

• Requirement

• Conformance

• IPR

• Implementation

Page 25: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS IPR Experts Group formation

• Late 2003 the President of AVS asked that a group of experts be assembled to create a recommendation for an IPR policy for AVS

• Create policies that:– reflect and respect China law and culture– reflect WTO requirements– reflect successful global practices– balance the rights of the inventor and needs of the implementers– innovative and forward looking– evolves with practice and law in China

Page 26: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

Mission of AVS IPR Group

• Make a friendly environment for technical society (standard body), industry, and end user

• Key issue to work out– Reforming the IPR policy making by only IP

holder, to by IP holder + industry + consumer

Page 27: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS IPR Experts Group

• Experienced members: 经验丰富的成员– Patent Attorneys 专利律师– Technical Experts 技术专家– Contract Attorneys 处理合同的律师– Previous experience in setting up patent pools 具有建立专利池相关

经验的成员• Members are from: 成员来自

– Asia 亚洲– Europe 欧洲– North America 北美

• Members represent: 成员代表了– Consumer Electronics 消费电子产品界– Information Technology 信息技术界– Microelectronics 微电子产业界

Page 28: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS IPR Experts group

• Members– Broadcom– IBM– Intel– Matsushita/Panasonic– Microsoft– Nokia– Sony– Sun Microsystems

Page 29: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

Licensing Obligations

Join AVS• Sign Member Agreement•Declare default option

Participate in Subgroup• Commit to license• License options

Submit Contribution• Disclose patents •Declare license option

Non-Participant: • RAND RF• POOL• RAND• No License

• RAND RF• POOL• RAND

China Patent:• RAND RF• POOL

Worldwide Patent: • RAND RF• POOL• RAND

Participant: • RAND RF• POOL• RAND

Page 30: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 31

Disclosure Obligations

• When making a contribution– Unpublished, published and granted patents

• Ongoing– When a patent is discovered– When a Patent Application is published

• During Final Draft Standard Review– 90 days to declare licensing option, OR:– Default applies

Page 31: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 32

AVS Licensing

• 1 RMB concept– Only one charge, to device/software/chip– No charge to operator– No charge to end-user

• Will be handled by AVS PPA– PPA is a non-profit organization– Founded By #1 institute of MII and ICT of

CAS, in 2005

Page 32: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 33

Patent :-Royalty Free-AVS Pool-RAND

Public knowledge

AVS Patent Pool

AVS Standards

AVSAVSWorking GroupWorking Group

AVSAVSPatent Pool CouncilPatent Pool Council

AVSAVSIndustry AllianceIndustry Alliance

•AVS Industry Chain•ICs,software•Device,System•Operating,Service

•AVS Industry Cluster•HDTV•IPTV•DTH•MobileTV•……

R&D IPR IndustryR&D IPR Industry

EssentialEssentialevaluationevaluation

One-StopOne-Stop

licenselicense

AVS LicensingAVS LicensingTechnical EvaluationTechnical EvaluationLicensing principle Licensing principle

IPRIPRPrinciplePrinciple

Cooperation &Cooperation & DevelopmentDevelopment

ApplicatioApplicatioStandardStandard

BetterBetterImplem.Implem.

Three Independent Organizations of AVS

Page 33: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 34

AVS Industry Alliance

– Amoisonic– Broadcom– Changhong– Haier– Huaju– Huawei– Langchao– NSCC

– Panstar Semi– SVA– Skyworth– Spectrum

Communication– Sunnywell– TCL– USstarcom– ZTE

Page 34: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 35: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21

AVS1 FamilyAVS Parts 国家标准计划号

STD Number小组草案( WD)

工作组草案(CD)

最终草案 (FCD)

标准送审稿( FD)

国家标准发布实施( GB)

AVS1-P1(系统 -广播) 20051304-T-339 2003.10 2003.12 2006.10 2007.03  

AVS1-P2 ( J-profile) 20032265-T-339 2003.10 2003.12 2004.4 2004.8  2006.2

AVS1-P2 ( X-profile) 20032265-T-339 2005.9 2006.3 2007.12  

AVS1-P2 ( S-profile) 20051305-T-339 2007.12 2008.3 2008.6 2008.9  

AVS1-P3 (Audio-Stereo) 20051305-T-339 2004.12 2005.3 2005.12 2006.4  

AVS1-P3 ( M-profile) 20051305-T-339 2005.9 2005.12 2005.12 2006.4

AVS1-P4 ( Conformance)

20051306-T-339 2006.9 1006.12 2007.3

AVS1-P5 ( Ref. Software)

20051307-T-339 2007.6

AVS1-P6 ( DRM) 20051308-T-339 2005.3 2005.12 2006.12 2007.3  

AVS1-P7 ( Mobility Video)

20051309-T-339 2004.6 2004.9 2004.12 2006.4  

AVS1-P8.1 ( System-IP) 20074554-T469 2005.3 2005.9 2005.12 2006.3

AVS1-P8.2 ( System-IP) 20074554-T469 2006.1 2006.3 2007.6

AVS1-P9.1 ( File Format)

20074555-T-469 2005.3 2005.9 2005.12 2007.12  

AVS1-P9.2 ( File Format)

20074555-T-469 2006.1 2006.3 2007.9 2007.12

AVS1-P10 ( Mobile Speech and Audio)

TBD 2007.3 2007.11 2008.3  

Page 36: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Tutorial Oct.20, 2004 37

MPEG-2 Video

Entropy Coding

IQ/IT

Motion compensation

Control Data

Coeff.

MV

Inter/Intra

Control

Decoder

Motionestimation

Trans./ Quant-

0

Page 37: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Tutorial Oct.20, 2004 38

Entropy Coding

IQ/IT

Motion compensation

Control Data

Coeff.

MV

Inter/Intra

Control

Decoder

Motionestimation

Trans./ Quant-

Deblock

Intra-frame

Prediction

AVS Video

Page 38: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Tutorial Oct.20, 2004 39

Entropy Coding

IQ/IT

Motion compensation

Control Data

Coeff.

MV

Inter/Intra

Control

Decoder

Motionestimation

Trans./ Quant-

Deblock

Intra-frame

Prediction

AVS Video

Page 39: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 40

Periods of key video coding tools contributed

~1988 19991950

DPCM

Interlace

B-framesP-frames

Scene Adaptive Coder

Block Motion Estimation

DCT

Macroblocks

Hybrid Coding

Huffman Coding

Transform Coding

Motion Vector Prediction

Video Object Planes

Object-Based Scalability

Error Resilience

Generic B-pictures

Advanced Deblocking Filter

Modified from C Reader, 2002

Multiple Reference

Integer Transform

Variable Block-size Motion Compensation

Page 40: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 41

Coding tools and its contribution

Color space

Conversion

Transform

and bit allocation

Entropy

coding

ME

and M

C

2 * 3 * 3 * 3 ≈ 501st G

2 * 3 * 6 * 3 ≈ 1002nd G

Page 41: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Tutorial Oct.20, 2004 42

AVS Video Tools• AVS1 video accepted 42 technical proposals from more than 200.

• Major tools – Transform – 16bit-implemented 8x8 integer transform– Quantization and scaling - scaling only in encoder– Intra prediction – 5 modes– Motion compensation – 16x16/16x8/8x16/8x8 modes– Quarter-pel interpolation – 4-taps interpolation filter– Deblocking– Entropy coding

• Minor tools– Motion vector prediction– Skipped mode and Coded block pattern– Adaptive scan

Page 42: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 43

Spatial Coding Tools

Fourier Transform

Block Quantization

Adaptive Block Coding

Slant Transform

Hadamard Transform

Scene Adaptive Coder

Zonal Coding

Threshold Coding

Zigzag Scan

ZZ-Run-Length Coding

Non-Uniform Quantizer

Uniform Quantizer

Block Classification

Cosine Transform

Dead-Zone Quantizer

Rate Buffer

1960s 1970s

Statistical Coding

Andrews, 1968 Ahmed, 1974Pratt, 1969

Max, 1960

Tescher, 1975

VLC VideoVariable-Length Coding VLC

Huffman 1952

Tescher, 1977

Fano 1949

Chen, 1973

Habibi, 1971

Chen 1981

Tasto 1971Huang 1963

Seyler 1962

Pratt 1969

Pratt 1969

Woods 1969

Tescher, 1976

Huang 1966

Run-Length Coding

Pratt, 1970

C Reader, 2002

Page 43: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

128

128

0

0813.0419.0500.0

500.0332.0169.0

114.0587.0299.0

B

G

R

V

U

Y

DCT

2-3-14-10-1-45

036-402-43

020-2200-3

-14-1-1-2-88-12

-1-215-3-1637

0-40-6-63-70

-3-30-1-4-9-264

-20-44-1-4-321430

C(h,v)=C(v)C(h)ΣΣ[P(x,y)COS{πv(2y+1)/16}COS{πh(2x+1)/16}]

C(0)=1/2√2 C(v)=C(h)=1/2 [v,h≠0]

x=0y=0

7 7

8369564638352927

6956463834292726

5848403532292726

4840353229272622

4037342927262222

3834342927262219

3734292724221616

342927262219168

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

9910310011298959272

10112012110387786449

921131048164553524

771031096856372218

6280875129221714

5669574024161314

5560582619141212

6151402416101116

9999999999999999

9999999999999999

9999999999999999

9999999999999999

9999999999996647

9999999999562624

9999999966262118

9999999947241817

Quant

00000000

00000000

00000000

0000000-1

00000-100

00000000

00000004

000000-2179

011010001011101...

VLC/CABAC

175172169162158163163162

181175171174168165167165

177179181178176170167173

181172175185184176171170

177175186180175174174178

199196189184182181180180

196194191193189182187180

192192192194198190184173

Y

U

V

RGB

YUV

8369564638352927

6956463834292726

5848403532292726

4840353229272622

4037342927262222

3834342927262219

3734292724221616

342927262219168

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

1616161616161616

9910310011298959272

10112012110387786449

921131048164553524

771031096856372218

6280875129221714

5669574024161314

5560582619141212

6151402416101116

9999999999999999

9999999999999999

9999999999999999

9999999999999999

9999999999996647

9999999999562624

9999999966262118

9999999947241817

DCT-based Compression

Page 44: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 45

AVS Innovation Example(1)DCT Transform Matrix in AVS

Differs from the one in MPEG-2 (floating value) and H.264 (4X4 and 8X8) that was patentedwith similar performance (or even better in some cases )

Page 45: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

46

Temporal Coding Tools

Conditional Update(Spatial; Pixel)

ConditionalUpdate(Transform)

3D (Temporal)DPCM

TransformCoding(Intra)

DPCM

HybridCoding

HybridCoding(Transform)

Harrison 1952

Pratt 1968 Reader 1973

Schroeder 1970

Roese 1975

TemporalStatistics

Seyler 1962

MotionModeling

Brofferio 1970

InterlacedField/FrameCoding

Jones 1978

Transform inDPCM Loop with MC

Jain 1979

PRA MotionEstimation

Netravali 1979

Variable –Size BMAMotion Est.

Giorda 1975

MotionCompensatedDPCM (Pixel)

MotionMeasurement

Limb 1975

Rocca 1969

Conditional Update(Spatial; Block)

Seyler 1962 Candy 1971

MotionCompensatedTransform

MotionCompensatedDPCM (Block)

Koga 1981

Netravali 1979

Netravali 1979

BMAMotionEstimation

Taki 1974

MPEG2

H.261MPEG1MPEG2Etc.

H.120

C Reader, 2002

1960s 1970s

Page 46: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

47

Skip, Direct and Symmetry Modes in B-frame

• Same:– Use both forward and backward

reference frame to predict

• Different:– Skip: no coded MVD, no coded

coefficient– Direct: no coded MVD, with

coded coefficient– Symmetry: with 1 coded MVD

and coded coefficient

Time tTime t-1 Time t+1

MV forwardMV backward

Page 47: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

48

AVS B-frame Compared with Other Standards

• AVS– Bi-prediction

• Skip, direct– No motion vector

• Symmetric– 1 coded motion

vector

• Higher compression ratio

• Other Standards– Bi-prediction

• Skip, direct– No motion vector

• Other cases– 2 coded motion

vector

Page 48: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS vs. H.264 vs. MPEG-2

tools AVS1-P2 H.264 MPEG-2

Intra-interpolation

8x8, 5 modes for Y prediction, 4 modes for UV prediction

4x4, 9 modes for Y prediction, 4 modes for UV prediction

Prediction only to DC coefficient

Reference frame

<=2 <=16 <=2

Block-size for MC

16x16, 16x8, 8x16

8x8

16x16, 16x8, 8x16

8x8,8x4,4x8,4x4

16x16,

16x8(interlace)

B frame micro-block bi-direction prediction

Coding front motion vector, and computing back motion vector

Coding two motion vectors

Coding two motion vectors

Page 49: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

tools AVS1-P2 H.264 MPEG-2¼ pixel MC ½ pixels 4-tap

¼ pixels 4-tap

½ pixels 6-tap

¼ pixels 2-tap

½ pixels 2-tap

Transform and quantization

8x8 integer transform ,encoding site normalization only

4x4 integer transform, both encoding and decoding sites need to normalize

8x8 float DCT

Entropy coding

Adaptive 2D VLC CAVLC

CABACVLC

Loop filter 8x8 based

Less boundaries

Less BS-levels (0..2), Less pixels filtered (p0, p1,q0, q1)

4x4 basedMore boundaries

More BS-levels (0..4), More pixels filtered (p0..p3,q0..q3)

N/A

AVS vs. H.264 vs. MPEG-2 (cont’d)

Page 50: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Tutorial 51

AVS Video Profile and Level

• JiZhun ( 基准 ) Profile (Dec. 2003)– 4 levels

• 4.0 : up to Standard Definition with 4:2:0• 4.2 : up to Standard Definition with 4:2:2• 6.0 : up to High Definition with 4:2:0• 6.2 : up to High Definition with 4:2:2

• AVS Zengqiang Profile (2008)– for Storage media such as movie

• AVS Shensuo Profile (2008)– for video surveillance

Page 51: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS1 Video Subjective Testing

• April-Sept., 2005, AVS subjective testing has been executed by SARFT testing center

• The testing is based on GY/T 134-1998 standard as well as related ITU testing standard

Compressed testing video stream bit-rate

SD resolution (625/50i)

HD resolution (1125/50i)

MPEG-2 typical bit-rate (Mbps)

4~6 20

AVS bit-rate (Mbps) 3 1.5 10 6

Testing result A B A A-

Page 52: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Performance- HD (1280 x720)

Harbour

29313335373941

1000 6000 11000 16000 21000 26000 31000 36000

Bi trate(kbi ts/ s)

PSNR

(dB)

H. 264 Mai nH. 264 Basel i neAVSMPEG- 4MPEG- 2

Page 53: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Fl ami ngo

26

28

30

32

34

36

38

40

10000 40000 70000 100000 130000

Bi trate(kbi ts/ s)

PSNR

(dB)

H. 264 Mai nAVSMPEG2

Performance- HD (1280 x720)

Page 54: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Performance – SD (720x576)

Basketbal l

23

26

29

32

35

38

4000 9000 14000 19000 24000 29000 34000

Bi trate(kbi ts/ s)

PSNR

Y(dB

)

H. 264 Mai nAVSMPEG-2

Page 55: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23

26

29

32

35

38

1900 4900 7900 10900 13900

PSNR

Y(dB

)

Bi trate(kbi ts/ s)

Fl ower garden

H. 264 Mai n

AVS

MPEG-2

Performance – SD (720x576i)

Page 56: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Performance – CIF (352x288)

News

28

30

32

34

36

38

55 105 155 205 255 305 355

Bi t rate(kbi ts/ s)

PSNR

Y(dB

)

H. 264 BaseLi ne

H. 264 Mai n

MPEG- 4

AVS

Page 57: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Mobi l e

25

28

31

34

37

40

0 1000 2000 3000 4000 5000 6000

Bi t rate(kbi ts/ s)

PSNR

(dB)

H. 264 BaseLi neH. 264 Mai nMPEG- 4AVS

Performance – CIF (352x288)

Page 58: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 http://www.avs.org.cn 59

Complexity analysis

Tools Estimated increasing

AVS H.264

Multiple reference 1 2

Variable block-size MC 1 2

Quarter pixel 3 3

Entropy coding 0.5 1

Deblocking filter 0.5 1

Total 6 9

Page 59: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Review: MPEG-4 AVC/ H.264(2001~09)

Page 60: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Review: AVS ( 2002 ~)• At Jan.26-Feb.5 , Geneva, ITU-T SG16

Q6 ( VCEG ) Meeting• During the discussion on the requirements

of H.265/H.NGC, Multiple MNC insisted on the new standard show be compact at technology side and lower complexity at implement side. Their proposal on new standard is very similar with AVS

• This is the road AVS selected 8 years ago.

Page 61: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 62: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS DTV End-End Solution

Page 63: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Professional AVS Encoders in the Market

• PowerCoder AE200MC• PowerCoder AE100SD/HD

• CoderStar AE1001• CoderStar AC1001

• 4Caster C4• SUMAVISION 9210• Telarity

Page 64: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

SVA Coderstar SeriesSVA Coderstar Series

Codestar AC1001for SDTV (1,2,4 channel)

Codestar AC1001-Lfor MobileTV (1,2,4 channels)

Codestar AE1001-Hfor HDTV

Professional AVS Encoders from SVA (Shanghai)

Page 65: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

UniAV (Beijing)UniAV (Beijing)

Powercoder AVS Encoders

for CIF,SD and HD

Envivio (USA)Envivio (USA)

4casterFor CIF and SD

Professional AVS Encoders

Page 66: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS decoding Chips in the Market

• China Companies– Spreadtrum SV6111 / SV6100 / SV6600V– Penstar DS-1000 HD / DS-1000 SD– NationalChip GX3203 / GX3101– Celestial CSM1208– Haier Hi2830(HD)/Hi1260(SD)

• MNC– Broadcom BCM7405B / BCM7466– ST STi520x/710x– NXP STB222– SigmaDesign SMP8654

Page 67: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS decoder Chips in the Market

AVS + MPEG-2 SD&HD (Beijing)

AVS SD&HD(Shanghai)

AVS + AVC + MPEG-2 SD&HD (Beijing)

AVS + MPEG-2 SD& CTTB (Hangzhou)

AVS+MPEG-2+AVC+VC-1 SD&HD (USA)

AVS+MPEG-2+AVC+VC-1 SD&HD (Europe)

AVS+MPEG-2+AVC+VC-1 SD (USA)

Page 68: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS decoding Chips in the Market

Page 69: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

USB Dangle + PC

STB and TV Set

AVS End Devices

Page 70: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS STBs

Page 71: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS receivers in car and bus

AVS End Devices

Page 72: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

SVA AVS TV Set获得 2008 年上海国际工业博览会创新奖

AVS TV Set

Page 73: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

车载移动终端 >7” 屏

AVS in Car

Page 74: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS in CBHD(China Blu-ray High-definition Disc player)

AVS in CBHD(China Blu-ray High-definition Disc player)

Page 75: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

双拼双视( Stereo Interleaving Video )

• 时分双拼– 左视和右视以奇偶场双拼

• 空分双拼– 左右– 上下

• 与现有标准关系– 现有视频编码标准可以支持这种双拼视频– 增加信令,沟通采集表示环节与立体显示器

* Images from www.stereo3d.com

Page 76: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS 3D TVPlan to live broadcast the Asia Games at Guangzhou at Nov. of this year

Page 77: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 78: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Terrestrial TV Deployment

Commercial OperatingTesting before Operating

Page 79: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Terrestrial TV in Operating

Shanghai Oriental Pearl digital TV Co., Ltd. 16 SD in 1 Channel

Sichuan Province StarMedia Broadcasting Co.,LTD 36 SD in 3 Channels

Hang Zhou City Culture Radio Television Group 21 CIF in 3 Channels

Shanxi Province Mobile Television 20 SD & 10 CIF in 2 Channels

Qingdao City Mobile Digital TV Co., LTD, 9 CIF in 1 Channel

Shaanxi Province Mobile Digital Television 20 CIF in 1 Channel

Hebei Province Mobile Digital Television Co. LTD 20 CIF in 1 Channel

Page 80: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

23/4/21 81

The State Development and Reform Commission(Responsible for DTV Policy)

Digital TV Development Project (2009)

(1) Digital Terrestrial TV Standards Family(2) Single Frequency Digital Terrestrial TV System Deployment(3) Key DTV Products: AVS, DRA and DTMB Chips, STB, Encoder, etc

Page 81: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Outline

• Licensing problem for AV standards

• AVS approach for dealing with IPR

• AVS Video technologies

• AVS Products

• AVS Applications nowadays

• Conclusion and Discussion

Page 82: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Conclusion

• 2nd Generation Video coding standards can double the efficiency of terrestrial frequency

• There are three choices for 2ndG video coding standards– ISO/IEC MPEG-4 AVC/ITU-T H.264 (2004)– AVS (China, Feb2006) – SMPTE VC-1 (leads by Microsoft, Apr2006)

Page 83: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

ISO/IEC MPEG-4 AVC/ITU-T H.264

• Already being adopted in some countries and area

• But high royalty cost and risk– Operators should pay MPEG LA for

commercial services (e.g. PPV), about 5 million USD per year

– Patent holders outside of MPEG LA Patent Pool are waiting for collecting money (the licensing policy still under table)

Page 84: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Technologies• Much compact designed 2ndG standard

– based on analysis on method & patents in last 50 yrs

– without many complicated but small performance contribution tools in international standards

– Technologies• Free methods and tools in public domain

• Self-innovation patents (~50 Chinese patents for video)

– Similar performance with lower complexity

• Very low cost– 1 RMB ( 1/6 USD) per device

– Royalty free for operators

Page 85: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

AVS Products and Deployments

• Most international video decoder chip vendors provide AVS-compatible SD&HD chips

• 4 China local companies provide low cost chips– e.g. Current price for a AVS+MPEG-2 SD Terrestrial Box is about

30USD

• Enough AVS products in the market• Shanghai deploys first AVS based Terrestrial

broadcasting system with 16 AVS SD programs, 7 provinces/cities provide commercial broadcasting

• AVS will be the unique mandatory format for all TV set and STB from next year

Page 86: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Comments

• 2nd G video coding standards is correct choice, but please consider – MPEG LA licensing policy @ www.mpegla.com– Potential patent holders especially who in MPEG-2 pool

but not in MPEG-4 AVC pool

• AVS is a better choice for long-term commercial operating– Commercial operating experience from multiple cities

and provinces– Cooperation on development on AVS1 and research on

AVS2

Page 87: AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology,

Thanks !AVS Working Group : http://www.avs.org.cnTiejun Huang : [email protected]