Internet Video Streaming

Video Communications over the Internet

Jozsef Vass

1/16/2000

Multimedia Communications and Visualization LaboratoryDept. of Computer Engineering and Computer Science

University of Missouri-ColumbiaColumbia, MO 65211, USA

http://meru.cecs.missouri.edu

2

Presentation Overview

References

Multimedia Requirements

Internet Protocol

Real-Time Transport Protocol

Resource Reservation Protocol

Multicast

Multiresolution Coding

Packetization

Error Control

Flow Control

Conclusions

3

References

S. McCanne, M. Vetterli, and V. Jacobson, “Low-complexity video coding for

receiver driven layered multicast,” IEEE Journal on Selected Areas in

Communications, vol. 15, no. 6, pp. 983-1001, Aug. 1997.

W.-T. Tan and A. Zakhor, “Real-time Internet video using error resilient

scalable compression and TCP-friendly transport protocol,” IEEE Transaction

on Multimedia, vol. 1, no. 2, pp. 172-186, June 1999.

T. Turletti, S.F. Parisis, and J.-C. Bolot, “Experiments with a layered

transmission scheme over the Internet,” INRIA Technical Report, Nov. 1997.

F. Kuo, W. Effelsberg, and J.J. Gracia-Luna-Aceves, Multimedia

Communications, Prentice Hall, 1998.

A.S. Tanenbaum, Computer Networks, Prentice Hall, 1998.

4

Multimedia Requirements

High bandwidth

Delay (critical for interactive applications)

Variation of delay (jitter)

Reliability

Commonly referred as quality-of-service (QoS) parameters

Low complexity codecs: Software only encoder/decoder

Multicasting: Bandwidth reduction

Support for heterogeneous system

5

Connection Type

Connectionless (Datagram): Each packet is routed individually

Currently used in the Internet

Efficient for short connections

Robustness

Easy internetworking

Connection-oriented (Virtual Circuit): Route is established at

connection setup

Currently used in X.25, Frame Relay, ATM

Efficient routing at runtime

Call acceptance to avoid congestion

6

Internet Protocol version 4

Provide reliable transmission in the case of link failure

Datagram service, each packet is routed independently

Best effort service, does not support QoS (fair)

Unicast (multicast addresses are reserved)

No provisions for supporting continuous media

No flow control

No error control

7


Source Address

Destination Address

Header Checksum

Type of Service

TransportProtocol

Identification

Time to Live

HeaderLength

Version(4) Total Length

MF

DF

Fragment Offset

Options (optional)

20 byte header

8


Packets can be segmented as traversing through the network

Identification: Fragmentation information

DF: Do not fragment. Each router shall handle packets of 576 bytes

MF: More fragments

Fragment Offset: Tells the place of the fragment

9


New version of Internet Protocol

New features

Introduction of streams: Flow => Packet stream between sender and receiver

Larger address space: 128 bit

Improved multicast addressing

Priority

Security (authentication, integrity, and encryption)

Backward compatible to IPv4

Connectionless

No flow control

No error control

10


Version(6) Priority

Payload Length

Source Address

Next Header Hop Limit

Source Address

Source Address

Source Address

Destination Address

Destination Address

Destination Address

Destination Address

Flow Label

40 byte header

11


Flow label: Multimedia flow can be identified

Special handling by the router

Soft state: Keep the state at the router until a time-out is reached

12

Migration to IPv6

Requires infrastructure changes

Internet providers work with small profit margins

IPv4 is good enough

May take several years

13

User Datagram Protocol

Source Port

UDP ChecksumUDP Length

Application interface to IP Send datagrams without establishing connection

Eight byte header

Destination Port

14

Real-Time Transport Protocol (RTP)

Transport protocol for multimedia streams over the Internet

Main functionality: Timing and synchronization

Used over UDP

Lightweight protocol No error correction No flow control No packet reordering No retransmission of lost packets

Does not support either resource reservation or QoS

Multicast capable

Associated control protocol: Real-Time Control Protocol (RTCP) Measure connection parameters and send transmission records (participants can

monitor the quality of the media flow) Parameter negotiation

15


Timestamp

Sequence NumberPayload TypeMCCV P

Contributing Source (CSRC) Identifier(Optional, up to 15 fields)

Synchronization Source (SSRC) Identifier

V Version NumberP Padding On/OffCC CSRC CounterM Mark

12 byte header

16


Timestamp: Intrastream and interstream synchronization

Synchronization Source (SSRC) Identifier: Random number generated

by the source, unique identification of the stream

Contributing Source (CSRC) Identifier: If a router (mixer) combines

media streams, the SSRC of contributing sources are recorded

17


Implemented as part of the application

Relatively new protocol

Used in vic and vat

Expected to be used in next generation WWW browsers

18


Earlier protocols ST-II (Internet Stream Protocol): Parallel to IP

Tenet Protocol Suit (UC Berkeley)

Resource Reservation Protocol (RSVP)

To be used with IPv6 (may also be used with IPv4)

Reservations are made for flows (specified in IP header)

Based on the flow label, the router schedules transmission in accordance with the reservation setup

RSVP keeps soft states Large amount of information

Must be refreshed within a given period

19


Receiver oriented: Each receiver may join or leave session

Each receiver decides based on its own characteristics and

requirements => Heterogeneous reservation (multicast)

The path may change: Cannot provide hard QoS guarantees

Tunneling: Traffic may flow through routers that do not support RSVP

=> no guarantees can be given

Broken links are not handled

20

Multicast Distribution

Unicast: Send a copy of each packet to each receiver individually

Multicast: Send each packet simultaneously to all the interested

receivers

Multicast Backbone (MBone) of Internet

Multicast IP address space was reserved (group addresses)

Routers must be extended

Understand group addresses

New routing mechanism

Duplicate incoming packets if necessary

Forward packets on the correct links

21

Multicast Distribution

IGMP: Internet Group Management Protocol

Multicast routing

Joining and leaving multicast groups

Multicasting still considered experimental

Multicasting will be fully integrated to IPv6

Experimental tools: vat, vic, wb, etc.

22


Network heterogeneity Local network Dial up network (PSTN, ISDN, etc.) Wide area network

Earlier video coding techniques (H.261, MPEG 1, H.263, etc.) do not

support scalable coding

Scalability in recent video coding standards

MPEG-2: Spatial and quality scalability

H.263+: Spatial, quality, and temporal scalability

MPEG-4: Spatial, quality, temporal, and object scalability

23


Simulcast: Set of independent encoders each producing a different

output rate => Low compression ratio since cross stream correlation is

not exploited

Multiresolution coding: Correlation across streams is exploited

resulting in high compression efficiency

The more layer the decoder receives the better the visual quality

Each layer may be carried by a different multicast group =>

Multiresolution-multicast framework

24


Hierarchical encoding:

Used in MPEG-2, H.263+, etc.

Base layer => Coarsest resolution signal, received by all receivers

Enhancement layers continuously refine the quality

Only layers received that the physical link can support

Layers must be received in importance order

25


Hierarchical encoding: Some layers are mandatory to reconstruct the

signal

The network must provide prioritized transmission => In the case of

packet loss, always the lowest priority layer must be dropped

ATM supports two levels of priority

IPv4 does not support priorities

IPv6 support priorities

26


Solutions for IPv4

Produce substreams that are of equal importance. No matter which

substream is received, the quality can be continuously improved (Multiple

Description Coding)

Ensure that higher priority streams are more reliably transmitted => Use

unequal error protection (UEP)

27


Audio coding: Subsampling

Each flow is separately encoded

Each flow has the same importance

The larger number of flows received, the better the quality

1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 8 kHz

2.7 kHz

2.7 kHz

2.7 kHz

28

Multiresolution Video Coding Block-Based Techniques

High coding performance

Most standards (H.261, H.263, MPEG-1, MPEG-2, MPEG-4, etc.)

based on this technique

Temporal correlation is exploited by block-based motion estimation

and compensation

Remaining spatial redundancy is exploited by transform coding

High complexity of encoder

Error sensitivity

Error propagation

Scalability is hard to support

29

Multiresolution Video Coding Block-Based Techniques

Conditional replenishment

Only blocks are encoded that change from previous frame

Blocks are encoded in intra mode, no prediction is applied

Lower performance than prediction

Higher error resilience

Only good for videoconferencing (large portion of the scene is

unchanged)

30

Multiresolution Video Coding Three-Dimensional Techniques

Block-based techniques does not naturally support scalability

Wavelet decomposition naturally provides multiresolution

representation

Wavelets can be extended to 3-D for video coding

Error resilience

No recursive loop: Reduces temporal error propagation

No spatial prediction: Reduces spatial error propagation

31

Multiresolution Video Coding Three-Dimensional Techniques

Nine substreams: Each substream is of approximately equal importance

32

Packetization

Before transmission, the stream must be packetized

Due to significant overhead of IP, large packet size is preferred

IPv4+UDP+RTP=20+8+12=40 bytes

IPv6+UDP+RTP=40+8+12=60 bytes

Packetization delay: Time to wait for data to fill the packet

Critical for low bit rate interactive applications (especially for speech)

Multilayer transmission increases the packetization delay

Error protection increases the packetization delay

33

Packetization

Speech example:

Sampling rate 8 kHz, eight bits/sample: 64 kbps

It is not beneficial to use high compression efficiency algorithms

Buffer sizeProtocol

20 ms 40 ms 80 ms

IPv4 16 kbps 8 kbps 4 kbpsIPv6 24 kbps 12 kbps 6 kbps

34

Error Control

Type of errors Bit errors (rare due to high quality fiber)

Packet loss due to congestion

Retransmission Most widely used in the current infrastructure (TCP, etc.) Receiver send ACK or NACK Computationally inexpensive Introduces delay: May not be tolerated in real-time application Requires backchannel Does not work for multicast

Eventually all the packets need to be transmitted NACK impulsion problem

35

Error Control

Forward Error Correction (FEC) Requires processing at the host Increases the bandwidth => May worsen the problem during congestion Interlaced protection for packet loss (introduces packetization delay)

Reed-Solomon codes Maximum error correction capability Low computational complexity

Information Packet

Parity Packet

Information Packet

Information Packet

Parity Packet

36

Flow Control

Rate of insertion packets into the network

Each coded layer is transported by a different flow

How many flow to subscribe

On congestion => Drop a layer (easy to detect)

On spare capacity => Add a layer (hard to detect)

Join experiments => May overload the network for large multicast groups

Layer 1

Layer 2

Layer 3

Layer 4

37

Flow Control

The most dominant traffic of Internet is TCP (WWW)

TCP is adaptive flow: Reduces rate when network is congested

Fair sharing with TCP is required: Match the rate of TCP

k: [0.7,1.3]

MSS: Maximum segment size

RTT: Round trip time

p: Packet loss rate

These parameters must be measured

pRTT

MSSkT

38

Conclusions

Internet video is very challenging

Large overhead of Internet protocols

Best effort service

Low delay for interactive applications

Error control

Multicasting

Lack of prioritization

The situation is improving

IPv6 enables prioritization


Documents

Internet Video Streaming