HTTP Live Streaming

(Apple) HTTP Live Streaming

(HLS) Dynamically adapting video

streaming to network conditions

Auro Tripathy

Outline

• The basics of adaptive streaming

• A Wireshark view

• New tags defined by the HLS protocol

• A comparison with other adaptive

streaming protocols

November 8, 2011 Auro 2

HLS Streaming Architecture


Content Preparation

Media Encoder

Media Segmenter

HTTP Content Server

Index File

Segment

HTTP Cache

The Basics of HLS

• Protocol for transmitting alternate versions (i.e., bitrates) of the stream

• Media can be served soon after its created – Not actually “live”, but near real-time

• Specify a presentation with the URI of a playlist file

• A playlist file is an ordered list of media URI and info tags

• Each media URI refers to a media file which is a segment of a single contiguous file

4 Auro November 8, 2011

How does the client play the

stream?

• Client first obtains the playlist file

– Initial playlist file may contain alternates (see

next slide)

• Client decides which alternate playlist file

to play

• Gets and plays each media file in the

playlist.


Server

Server Provides Alternate

Playlists to Clients


Alternate 1 Index File TS Segment



Index File

w/alternates

M3U format – the History

• M3U stands for MP3 URL.

• Files that contain the .M3U file-extension are MP3 playlist files – commonly referred to as Winamp playlist files

• Important to remember, this file does not contain media, only references to media

• Files whose name end with .m3u are encoded with US-ASCII.

• The unicode version of "m3u" is "m3u8", which uses UTF-8 unicode characters


Extended M3U

• Defines two tags: EXTM3U & EXTINF

• First line must be #EXTM3U

– This distinguishes it from a M3U file

• EXTINF is a record marker that describes the media file identified by the URI that follows it.

• Example #EXTM3U

#EXTINF:<duration>, <title>

http://media.example.com/entire.ts


Server Tasks to Achieve HLS

• Encode video in H.264 format and Single Program Transport Stream “container” – Encode in different bitrates

– taken from live feed or from a file

• Use a stream segmenter to generate short “chunks” of content – typically 10 seconds each - and generate a playlist file (m3u or m3u8) indicating from where to download the chunks

• Serve via HTTP server, and provide appropriate caching for performance


Who supports HLS?

• Apple, of course

– Authored the protocol

http://tools.ietf.org/html/draft-pantos-http-live-

streaming-07

• Support available in Android 3.0

– http://developer.android.com/sdk/android-3.0-

highlights.html


http://developer.android.com/sdk/android-3.0-highlights.html





Deconstructing HLS with

Wireshark

Server Offering Multiple Playlist Files Different encodings of the same presentation


Client request

Server offers four

alternative playlists at

200 Kbps, 311.1 Kbps,

484.4 Kbps & 737.8 Kbps

Client fetches the first Playlist File


Client request 200 Kbps

playlist

Duration of segment in

seconds; appears once

Segment URLs

Note - Playlist file ends with a #EXT-X-ENDLIST


Indicates no more media files will be added to the Playlist file

Client fetches the second Playlist File


Client fetches the 3rd Playlist File


Client fetches the 4th Playlist File


Client requests a Media File to Play


Summarizing the Sequence


Server Client

Respond with four alternate playlists

Location of all segments in Playlist

Location of all segments in Playlist

Respond with A/V data

VLC Plays HLS Test File http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8


New Tags defined in HLS

HLS defines the following new tags Each with it own attribute list

• EXT-X-TARGETDURATION

• EXT-X-MEDIA-SEQUENCE

• EXT-X-KEY

• EXT-X-PROGRAM-DATE-TIME

• EXT-X-ALLOW-CACHE

• EXT-X-PLAYLIST-TYPE

• EXT-X-STREAM-INF

• EXT-X-ENDLIST

• EXT-X-DISCONTINUITY

• EXT-X-VERSION


Tag – EXT-X-TARGETDURATION

• EXT-X-TARGETDURATION:<s>

– Specifies the maximum media file duration

– Appears only once in the playlist file

– <s> is an integer indicating the duration in

seconds


Tag – EXT-X-MEDIA-SEQUENCE

• Each media file URI in a playlist must have

a unique integer number

• Sequence # := Sequence # of URI that

preceded it + 1

• Tag indicates the sequence number of the

first URI that appears in the playlist file

• Format

EXT-X-MEDIA-SEQUENCE:<number>


Tag – EXT-X-KEY

• Contains information to decrypt media files that follow it

• Format #EXT-X-KEY:<attribute-list>

• METHOD attribute specifies encryption method (AES-128 or NONE)

• URI attribute identifies the Key file – Key file contains the cipher key to decrypt subsequent

media files in the Playlist

– Quoted string with URI

• IV attribute specifies the initialization vector to be used with the key


Tag – EXT-X-PROGRAM-DATE-

TIME

• Associated with the next media file

• Should indicate a time zone

• Format EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZone>


Tag – EXT-X-ALLOW-CACHE

• Indicates whether a client MAY or MUST-

NOT cache downloaded media files for

later playback.

• Tag applies to all segments in the playlist

• Format

#EXT-X-ALLOW-CACHE:<YES|NO>


Tag - EXT-X-PLAYLIST-TYPE

• Optional tag provides mutability

information about the playlist file

• Format:

#EXT-X-PLAYLIST-FILE:<Event|VoD>


Tag – EXT-X-ENDLIST

• Indicates no more media files will be

added to the Playlist file

• Occurs only once

• Format:

#EXT-X-ENDLIST


Tag – EXT-X-STREAM-INF

• Tag indicates that the next URI in the playlist file indentifies another playlist file

• Format/example #EXT-X-STREAM-INF:<attribute-list>

<URI of playlist file>

• Defined attributes – BANDWIDTH

– PROGRAM-ID

– CODECS

– RESOLUTION


Tag – EXT-X-DISCONTINUITY

• Tag indicates an encoding discontinuity between the media file that follows it and the one that preceded it

• Reason – File format change

– Number and type of tracks

– Encoding parameters

– Encoding sequence

– Timestamp sequence

• Format – #EXT-X-DISCONTINUITY


Tag – EXT-X-VERSION

• Tag indicates the compatibility version of

the Playlist file

• One occurrence

• Format

#EXT-X-VERSION:<n>


Media Files

• Media files are formatted as MPEG-2

transport streams

• It’s a singe program transport stream

• Should have a PAT/PMT at the start of

each files

• Client should be able to handle any

encoding parameter change such as a

resolution change


Key Files

• The EXT-X-KEY tag has the URI and the

IV attributes to identify the key file and the

Initialization Vector

• If the IV is not present, client must use the

sequence number of the media file as the

IV

• Note, varying the IV increases the strength

of the cipher


Conclusion – Lots of upside with

the protocol

• Uses the HTTP protocol

– Firewall-friendly

– HTTP server are ubiquitous thus reducing operational complexity

– Unifies the delivery for three screens (TV/Mobile/Tablet)

– Scales, no session state on the HTTP server

• Chunked transfers allow the client to compensate/optimize in near real-time for varying network conditions


Conclusion – Potential

Downsides

• Choice of MPEG-2 single program

transport streams

– 188 byte packet structure with 4-byte header

creates non-negligible overhead

• VCR-like trick-modes (seek to a time

forward or backward) can cause

noticeable delays if the segment size is

large


Documents

HTTP Live Streaming