Embed Size (px)
Citation preview
11/16/11
1
Bringing Internet Video to Prime Time Aditya Ganjam Vice President of Engineering Conviva
Background
! Aditya Ganjam – Vice President of Engineering at Conviva ! Conviva
! Startup founded by Hui Zhang (Prof. at CMU) and Ion Stoica in 2006 ! Conviva optimizes video quality for premium content properties such as HBO, ESPN, ABC, Disney, and
Turner through network wide real-time visibility and real-time actions. Conviva's technology has powered some of the world's largest on-line events such as Olympics, FIFA World Cup, NCAA College Basketball March Madness, Major League Baseball, and Academy Awards.
2005: Beginning of the Internet Video Era
Launch of YouTube 100M streams first year
Premium Sports Webcast on Line
Live8 concert online
First popular mobile video device
2006 – 2011: Internet Video Going Prime Time
2006 2007 2008 2009 2010 2011
11/16/11
2
2011 Internet Traffic Distribution
Source: Akamai
66% Internet Traffic is Video
IPTV VOD
Internet Video
P2P
Video Calling
Web, email, data
File transfer
Online gaming
VoIP
Business Internet
2011 and Beyond: A World Full of Elephants
Video (100x traffic growth)
Other Applications (10 x traffic growth)
2011
What Does It Mean For the Internet If 95% Traffic is Video?
2016
Macro Changes in Internet Video Technology and Business
! Technology enablers in place ! Broadband penetraTon ! Standard soUware & hardware plaXorms
! Real business model emerging ! Premium content (ESPN, HBO) ! AdverTsing & subscripTon (mlb.com, neXlix, Hulu)
! Online audiences rival broadcast for major events ! Olympics, InauguraTon, Michael Jackson, World Cup
! Convergence of TV, Internet, Mobile ! Internet connected TVs (Xbox, PlaystaTon, AppleTV, Roku, Sony, Samsung)
! Internet connected smart mobile devices (iPad, iPhone, Android) ! “TV Everywhere” over the top (HBO, ESPN, Turner, Comcast)
This Talk
! Delivering high quality video over the Internet and Conviva’s approach to addressing this challenge
! Three sections …
! Understanding today’s Internet video eco-system ! State of the art of Internet video quality ! A strategy for delivering high quality video
11/16/11
3
Understanding Today’s Internet Video Eco-system
Internet Video Ecosystem : Video Data-plane
Video Source
Encoders & Video Servers
CMS and HosTng Content Delivery
Networks (CDN)
ISP & Home Net
Screen
Video Player
Key components: Video Player & CDN
Content Delivery Networks
Video Player
Content Origin
-‐ Major CDNs are Akamai, Limelight, and Level3 -‐ Many ISPs are also building their own CDNs
Edge Servers
How a CDN works … -‐ A CDN is a large distributed content cache acTng as an overlay mulTcast network -‐ Data flows from the content origin through mid-‐Ter servers to edge servers -‐ Content is cached along the way to achieve scale -‐ Many CDNs use DNS to abstract away the complexity of server selecTon
DNS
Video Uses CDNs A Little Differently
Screen
Video Player
500Kbps
800Kbps
1Mbps
1.5Mbps
2Mbps
3Mbps
Live streaming MulT-‐bitrate streaming MulT-‐CDN streaming
11/16/11
4
Player & Device Eco-System
PC Mobile Game Consoles Set-‐top Boxes Connected TVs
Significant adopTon, especially Flash
Rapid growth in market share
Rapid growth and significant market share
Moderate growth Moderate growth, but will pick up quickly
Sophistication of the Video Player 3rd Party Ad Networks
CDN Verification
2 3
CDN
4 5
6 7
8
1
Targeting – ie.
Ad CDN
Video CDN
Tokenization / License
server
CMS
11
12
9
10
13
14
Ad Proxy Video Player
What is the quality of Internet video today ?
! What is high video quality? ! Prevent video startup failures ! Start the video quickly ! Play the video smoothly and without interruptions ! Play the video at the highest bit rate possible
! What is the best way to measure Internet video quality? ! Claim: Collecting statistics from the video player is the best way to
measure video quality ! Reason 1: The video player interacts with multiple services owned by multiple
companies and is the only single point that has state across all interactions ! Reason 2: With multi-bit rate and multi-CDN technologies, a single server or CDN
does not have the complete quality information for a client
.
Video Player Monitoring : Player Model
Stopped/
Exit
Player
States Joining Playing Buffering Playing
Network/
stream
connection
established
Video
buffer
filled upVideo
buffer
empty
Buffer
replenished
sufficiently
time
User
action
Events
Player
Monitoring
Video download rate,
Available bandwidth,
Dropped frames,
Frame rendering rate, etc.
JoinTime (JT) BufferingRa2o(BR) RateOfBuffering(RB)
AvgBitrate(AB)
RenderingQuality(RQ)
11/16/11
5
Video Player Monitoring : Data Collection
Automa4c Monitoring
• AutomaTc and consistent monitoring of default streaming modules – Flash: NetStream, VideoElement – Silverlight: MediaElement, SmoothStreamingMediaElement – iOS: MPMoviePlayerController
Streaming Module
UI Controller
Content Manager
Messaging & Serializa4on
Player Insight
To backend
HTTPS
Player App
licaT
on
Video Player Monitoring : Cross-platform Challenges
AcTonScript
JavaScript
ObjecTve C
C++
NetStream, …
<video/>, <audio/>
MPMoviePlayerController
SmoothStreaming, …
Flash
HTML5
iOS
Silverlight
PlaXorm Independent PlaXorm Specific
• Consistency in metric computaTon across languages and plaXorms
• Benefit from stronger type checking from C# • Readable output: preserve comments, white space, formarng • Provides control for language-‐specific fragments • Compiles real code and unit tests • Uniform tracing and debugging
Comm. Timer Storage C#
Comm. Timer Storage
Comm. Timer Storage
Comm. Timer Storage
Language Translator
Video Player Monitoring : Data Model
State of The Art of Internet Video Quality
11/16/11
6
We’ve seen patterns across many sites … and billions of streams
Example Video Site Quality Summary
! 31.68% of views had quality issues
! 32.2% of viewers had recurring quality issues
! Viewers with good quality watched 1.5X more video than viewers with poor quality
! Good video quality for all viewers can add 10.9% more minutes of viewed video
! Good video quality can add $120K more revenue per month*
68.3
25.3
5.7 0.68
Total Views
!!"#$%&!'()%*+!
!"video did not start
video buffered
video had low resoluTon
video had good quality
67.8
32.2
100.9
62.2
0 20 40 60 80 100 120
Minutes per Viewer
Poor Quality Viewers Good Quality Viewers
* Assumes $30 CPM ad every 8 minutes
Total Views = 66,44,79,19 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395
Example Video Site Quality Summary
! 31.68% of views had quality issues
! 32.2% of viewers had recurring quality issues
! Viewers with good quality watched 1.5X more video than viewers with poor quality
! Good video quality for all viewers can add 10.9% more minutes of viewed video
! Good video quality can add $120K more revenue per month*
68.3
25.3
5.7 0.68
Total Views
!!"#$%&!'()%*+!
!"video did not start
video buffered
video had low resoluTon
video had good quality
100.9
62.2
0 20 40 60 80 100 120
Minutes per Viewer
Poor Quality Viewers Good Quality Viewers
* Assumes $30 CPM ad every 8 minutes
Total Views = 66,44,79,19 Total Viewers = 3,291,204 Total Minutes Viewed = 290,260,395
0
10
20
30
40
50
1 2 3 4 5 6 7 8 9 Poor experience on 9 different sites
Opportunity of Going Higher Speed
<100 100-‐300 300-‐500 500-‐700 700-‐1000 1000-‐1600 1600-‐2200 2200-‐3000 >3000
Actual Bit Rate Consumed
Available Bandwidth
Untapped Brand Enhancement and Viewer Experience
Actual Bit Rate Consumed
Available Bandwidth
11/16/11
7
Viewers Watch Longer When Video is Not Interrupted by Buffering
Live VoD
Good Views Impacted
Avg Min 6min
22 min 261% more
0 3 6 9 12 15 18 21 24 27 30
Good Views Impacted
Avg Min 11 min
32 min 191% more
0 3 6 9 12 15 18 21 24 27 30
Jan 2011
Concert
Sports
TV1.com
TV2.com
Good Views Impacted
Avg Min 16 min
27 min 69% more
0 3 6 9 12 15 18 21 24 27 30
Good Views Impacted
Avg Min 13 min
17 min 31% more
0 3 6 9 12 15 18 21 24 27 30
Jan 2011
Viewers Return More When Video Is Not Interrupted by Buffering
Even 1% increase in buffering leads to more than 60% loss in audience
1% difference in buffering between two ISPs
68% monthly loss in uniques for ISP with poor performance
Engagement vs Join Time
Join Tme is criTcal for user retenTon
0 10 20 30 40 50 60 70Join time (s)
0
10
20
30
40
50
60
Tota
lpla
ytim
e(m
in)
Correlation coefficient (kendall): -0.74
Engagement vs Buffering Ratio
1% increase in buffering reduces engagement by 3 minutes
!
!
!
!
!
!
!
!
!
!
!!
!!
!!!
!!!!!!!!
!!
!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
0 20 40 60 80 100
010
2030
40
Buffering ratio (%)
Play
time (
min)
Correlation coefficient (kendall): −0.96, slope: −3.25
11/16/11
8
CDNs Vary in Performance over Geographies and Time
• Used one month aggregated data-‐set
• Considered 31,744 DMA-‐ASN-‐hours with > 100 views in each CDN
25%
50%
25%
There is no single best CDN across geographies, networks, and time!
CDNs Vary in Performance over Geographies and Time
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Washington DC
(Hagerstow
n):CMCS(33657)
Milw
aukee:RO
ADRU
NNER
-‐CEN
TRAL(20231)
Green Ba
y -‐ A
ppleton:SCRR
-‐7015(7017)
Denver:ASN
-‐QWEST(20
9)
Charloze
:SCR
R-‐11426(11426)
Washington DC
(Hagerstow
n):ASN
-‐CXA
-‐ALL-‐CCI-‐22773-‐
Philade
lphia:VZ
GNI-‐T
RANSIT(19262)
San Diego:SBIS-‐AS(7132)
Las V
egas:ASN
-‐CXA
-‐LV-‐13432-‐CB
S(13432)
Madiso
n:CH
ARTER-‐NET-‐HKY-‐NC(20115)
Indianapolis:SBIS-‐AS(7132)
Providen
ce -‐ New
Bed
ford:ASN
-‐CXA
-‐ALL-‐CCI-‐22773-‐RDC
Washington DC
(Hagerstow
n):VZG
NI-‐T
RANSIT(19262)
HarXord & New
Haven
:SBIS-‐AS
(7132)
Houston:SBIS-‐AS(7132)
Grand Ra
pids -‐ Kalamazoo
-‐ Ba
zle Creek:SBIS-‐AS(7132)
Atlanta:BE
LLSO
UTH
-‐NET-‐BLK(6389)
Hono
lulu:HAW
AIIAN-‐TELCO
M(36149)
Atlanta:CO
MCA
ST-‐7725(7725)
Washington DC
(Hagerstow
n):SPC
S(10507)
Orla
ndo -‐ D
aytona Beach -‐ Melbo
urne
:EMBA
RQ-‐W
NPK
Denver:CMCS(33652)
Norfolk -‐ Po
rtsm
outh -‐ New
port New
s:AS
N-‐CXA
-‐ALL-‐
Saint Lou
is:CH
ARTER-‐NET-‐HKY-‐NC(20115)
CincinnaT:FU
SE-‐NET(6181)
Phoe
nix:AS
N-‐QWEST(209)
Dallas -‐ Fort W
orth:VZG
NI-‐T
RANSIT(19262)
Pizs
burgh:CC
CH-‐3(7016)
Saint Lou
is:SBIS-‐AS(7132)
San Francisco -‐ O
akland
-‐ San Jose:SBIS-‐AS
(7132)
San Diego:RO
ADRU
NNER
-‐WEST(20001)
Greenville -‐ Spartansburg -‐ A
sheville -‐
Kansas City
:SBIS-‐AS
(7132)
Columbu
s -‐ OH:SCRR
-‐10796(10796)
Louisville:INSIGH
T-‐CO
MMUNICAT
IONS-‐CO
RP-‐AS1(36727)
Chicago:AT
T-‐INTERN
ET3(6478)
Kansas City
:ASN
-‐CXA
-‐ALL-‐CCI-‐22773-‐RDC
(22773)
Miami -‐ Fort Laude
rdale:BE
LLSO
UTH
-‐NET-‐BLK(6389)
Cleveland:NEO
-‐RR-‐CO
M(11060)
Chicago:VZ
GNI-‐T
RANSIT(19262)
Columbia -‐ SC:SCRR
-‐11426(11426)
Dallas -‐ Fort W
orth:SBIS-‐AS
(7132)
Detroit:S
BIS-‐AS
(7132)
Kansas City
:SCR
R-‐11955(11955)
Los A
ngeles:CHA
RTER
-‐NET-‐HKY-‐NC(20115)
Miami -‐ Fort Laude
rdale:CO
MCA
ST-‐20214(20214)
Cleveland:SCRR
-‐10796(10796)
West P
alm Beach -‐ Fort Pierce:CO
MCA
ST-‐20214(20214)
San Diego:AS
N-‐CXA
-‐ALL-‐CCI-‐22773-‐RDC
(22773)
Seaz
le -‐ Tacoma:AS
N-‐QWEST(209)
New
York:CA
BLE-‐NET-‐1(6128)
Portland
-‐ OR:AS
N-‐QWEST(209)
Oklahom
a City:SBIS-‐AS
(7132)
Phoe
nix:AS
N-‐CXA
-‐ALL-‐CCI-‐22773-‐RDC
(22773)
Chicago:SBIS-‐AS(7132)
Greensbo
ro -‐ High Point -‐ Winston
-‐Salem
:SCR
R-‐11426
Albany -‐ Sche
nectady -‐ T
roy:RR
-‐NYSRE
GION-‐ASN
-‐01
Tyler -‐ Lon
gview (Lu}
in & Nacogdo
ches):S
UDD
ENLINK-‐
Grand Ra
pids -‐ Kalamazoo
-‐ Ba
zle Creek:CM
CS(33668)
Houston:CM
CS(33662)
HarXord & New
Haven
:COMCA
ST-‐7015(7015)
Eugene
:ASN
-‐QWEST(209)
San Francisco -‐ O
akland
-‐ San Jose:CMCS(33651)
CDN Streaming Failures Are Common Events Summary of Results
! On most sites 15-30% of viewers do not get an uninterrupted high quality stream
! Quality has a substantial impact on viewer engagement ! Buffering ratio is most critical across genres (for live event: 1% increase in buffering
reduces 3min play time)
! Join time impacts engagement at viewer level
! CDN performance varies minute by minute and region by region
11/16/11
9
A Strategy for Delivering High Quality Video Over the Internet
Refresh: What is high quality?
! Prevent startup failures ! Start the video quickly ! Play the video smoothly and without interruptions ! Play the video at the highest bit rate possible
Three Concepts for High Quality Video Delivery
! Continuous measurement and optimization using a control infrastructure decoupled from the delivery infrastructure
! Multi-bit rate streams delivered using multiple CDNs ! Optimization algorithms based on individual client and
aggregate statistics working at multiple time scales
One-time DNS Re-direction vs Continuous Optimization
May still incur connection or streaming failure or missing asset
Select a server intelligently at start time
Select the best CDN to successfully access the asset without any failure
• Continuously monitor video quality from client side • Use global audience based diagnostics to predict quality issues • Switch bit-‐rates or sources
! CDN
! Continuous optimization
11/16/11
10
Possible Optimization Architecture
Real-‐Tme Alerts
ConTnuous real-‐Tme measurements from every client
Real-‐Tme and historical Insights
Real-‐Tme global opTmizaTons
Inference Engine
Bit Rates
CDNs
Decision Engine
OpTmize viewer performance by selecTng the best opTon within the set of bit rates and CDNs
Akamai
DMA
ASN
DMA
ASN DMA
ASN
Limelight
Level3 Time of day
Localize issues by region, network, CDN, and Tme
Real-‐Tme Global Data AggregaTon and
CorrelaTon (Streaming Map-‐reduce)
Historical Data AggregaTon and Analysis
(Hadoop+Hive+Spark) Global Inference, Decision & Policy
Engine
Example Optimization Using Aggregate Statistics
Akamai Geo
ASN
Geo
ASN
Geo
ASN
Limelight
Level3
This ASN/Geo is saturated on all three CDNs è Don’t switch CDN. Reduce bit rates and maintain
0
10
20
30
40
0 5000 10000 15000 20000 25000 30000 35000
0
20
40
60
80
100
0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000
Band
width FluctuaTo
n Ba
ndwidth FluctuaTo
n
Peak Concurrent Viewers
Peak Concurrent Viewers
Network SaturaTon Point
Unsaturated Network
Conviva Optimization in the Wild
… increased average bit-rate from 1.7 Mbps to 2.1 Mbps…
Reduced views impacted by buffering from 16.13% to 5.56% …
… and raised engagement by 36%
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
1600
1700
1800
1900
2000
2100
2200
Views Impacted by Buffering
Average Bit Rate
Concluding Remarks
! All indications show that we are in the middle of a key transition of main-stream video to the Internet
! Video quality presents opportunity and challenge ! Follow the traffic: 60% Internet traffic today, will be more than 95% in the
next 2-3 years ! Premium video will be consumed by lean-back experience on big screens
à zero tolerance for poor quality
! Video player continuous monitoring and optimization driven by
player-level and global algorithms has the best chance of delivering high quality video
