Gathering Data on

Broadband Traffic

and

Congestion Management

Steven BauerMIT CSAIL Research Affiliate

Broadband networks

Little visibility

Complex ecology

Broadband data today

1) Limited probing from the edges2) “Screen-scraped” data3) Japanese ISP’s project

Broadband data tomorrow

1) Validated data supplied by broadband providers

2) End user traces and data

Project methodology

1) Experimental design (3 months)

2) Initial data collection and analysis (8 months)

3) Project reporting and workshop (1 month)

Anticipated outcomes

• Traffic and congestion characterization• Towards network management best

practices• Measurement design

Provider benefits

• Mobilize public and industry awareness of broadband challenges

• Research progress in network architectures and technology

• Improved insight into how to model broadband traffic

Existing data sets and experimental design

Aggregate traffic

What is the rate of growth?Where is it happening?What is driving growth?

Aggregate trafficWhat is the rate of growth?Where is it happening?What is driving it?

Aggregate trafficWhat is the rate of growth?Where is it happening?What is driving it?

Congestion management

Security

Network management

QoS

Userexperience

Routing

Better data Better research

Better policy

Traffic modeling

Questions heard so far…• Growth rates…• Difference in growth rates in different parts of the network…• Characterize where in the network congestion occurs…• Correlation between traffic volume and congestion volume…• External events causing large swings in traffic…• Classes of users…• Distribution of usage…• Peak period and off-peak period usage/congestion…• Packet size distributions…• Effects of consumption billing changes on peak usage…• Data needed to convince operators and OS vendors to turn on

ECN…

Ongoing cooperative study of broadband in Japan

1

Residential broadband users in Japan

Project challenges:

Mostly political, not technical!

Data collection

• Aggregate data from multiple providers• Extremely careful to preserve user privacy • Sensitive to business concerns of

providers

1. Per interface byte counts2. Netflow samples (from one ISP)

Traffic growth: customer traffic

1l

Bytes downloaded

Byt

es u

ploa

ded

1l

Daily bytes downloaded

Dai

lyby

tes

uplo

aded

May 2005

May 2008

Residential broadband customer traffic

Cumulative distribution of daily

trafficper user

2005

2008

“The observed trends, however, suggest that video content is unlikely to

disastrously overflow the Internet, at least not anytime soon.”

Broadband data tomorrow

1) Validated data supplied by broadband providers

2) End user traces and data

For programmers…

For networkers…

Netwatching.orgTo be developed!

Objectives

• Easy to share• Easy to visualize• Easy to discuss• Easy to annotate

Fostering a healthy dialog

• Voluntary participation• Data-centered discussions • Community of stakeholders• Disclosure• Promote communicating with operators

Data will include1) Measurement errors2) Configuration errors3) Effects of deliberate policies

Real applications and real users generate the best probes

RSTRSTRSTRST

RSTRST

“Everyone’s entitled to their own opinions, but not their own facts.”

Backup

2

Academic Network at Sao Paolo (ANSP)Amsterdam Internet Exchange (AMS-IX)Athens Internet Exchange (AIX)Australian National University (ANU)BIT ISP, Netherlands (BIT)Bangladesh Internet Exchange (BDIX)Belgian National Internet Exchange (BNIX)Boca Network (Boca)Budapest Internet Exchange (BIX)California Research and Education Network at University of California, Santa Cruz (CALREN-UCSC)Catalonia Neutral Internet Exchange (CATNIX)Cern Internet Exchange Point (CIXP)Chiang Mai University (CHIANGMAIU)Cornell University (CORNELL)Creighton University (CREIGHTON)DTUNet (DTU)Deutsche Commercial Internet Exchange (DE-CIX)Espananix Punto Neutro Espanol de Internet (ESPANIX)Finnish Communication and Internet Exchange (FICIX)Hong Kong Internet Exchange (HKIX)Internet Neutral Exchange Point (INEX)Israeli Internet Exchange (IIX)

Japan Internet Exchange (JPIX)Japan Network Access Point (JPNAP)Johannesburg Internet Exchange (JINX)KleyRex Internet Exchange (KLEYREX)Korea Internet Exchange (KINX)London Internet Exchange (LINX)London Internet Providers Exchange (LIPEX)London Network Access Point (LoNAP)Los Angeles International Internet Exchange (LAIIX)Luxembourg Internet Exchange (LIX)Milan Internet Exchange (MIX)Moroccan Academic and Research Wide Area Network (MARWAN)Moscow Internet Exchange (MSK-IX)Nautilus Mediteranean Exchange Point (NAMEX)Nepal Internet Exchange (NPIX)Netherlands Internet Exchange (NL-IX)Network Access Point Peru (NAPPERU)Neutral Internet Exchange (NIX-CZ)New York Internet Exchange (NYIIX)Norwegian Internet Exchange (NIX)Nuremburg Internet Exchange (N-IX)Ottawa Internet Exchange (OTTIX)PTTMetro (PTTMetro)Pacific Wave (PACIFICWAVE)Paris Network Access Point (PANAP)

Pittsburgh Internet Exchange (PITX)Polish Internet Exchange (PL-IX)Porto Federal de Interconnexao de Redes (PTT-FIX)Princeton University (PRINCETON)Punto Neutro Vasco de Internet (EUSKONIX)Reykjavik Internet Exchange (RIX)Romanian Network for Internet Exchange (RoNIX)Seattle Internet Exchange (SeattleIX)Slovakian Internet Exchange (SIX)Swedish Internet Exchange Netnod (NETNOD-IX)Swiss Internet Exchange (SWISSIX)Taiwan Internet Exchange (TWIX)Taiwan Internet Gateway (TWGate)Tallinn Internet Exchange (TLLIX)Tallinn Internet Exchange (2) (TIX.EE)TelX Internet Exchange (ATLANTAIX)Torino Piedmonte Internet Exchange (TOP-IX)Tuscany Internet Exchange (TIX.IT)University of Toronto (UTORONTO)Vienna Internet Exchange (VIX)WORK Internet Exchange (WORK-IX)Warsaw Internet Exchange (WIX)XchangePoint (PacketExchange) (XCHANGEPOINT)Yang Ming University (YANGMING)

Odlyzko’s estimates of traffic growth

Early 1990’s…………... 2x per year

1995 and 1996……….. 10x per year

1997 to 2003…………. 2x (70 to 150%) per year

2003 to present………. 50% per year

Mean growth: 1.665Volume weighted mean annual growth rate: 1.314Median growth: 1.167