Institute of Computer Science Chair of Communication Networks
Prof. Dr.-Ing. P. Tran-Gia
Internet Access Traffic Measurement and Analysis
Steffen Gebert1, Rastin Pries1, Daniel Schlosser1, Klaus Heck2
1University of Würzburg, Germany 2Hotzone GmbH, Berlin, Germany
COST TMA Workshop 12.03.2012
Vienna, Austria
Internet Access Traffic Measurement and Analysis Steffen Gebert
2 2
Motivation
u Internet traffic changes continuously - frequent observation required
u Characteristics vary from core to access, by clients, by country, etc.
u Traffic statistics as input parameters for § Bandwidth estimation § Dimensioning of network equipment § Simulation of networks and devices
u Example use case: Modelling of OpenFlow devices § Forwarding on Layer 2 § Forwarding decisions on a per-flow level
Internet Access Traffic Measurement and Analysis Steffen Gebert
3 3
Motivation (contd.)
u Previous measurements by Wamser, Pries, Heck1 in 2007 and 2008
u We present measurement from 2010 u Compare results and changes
1 On Traffic Characteristics of a Broadband Wireless Internet Access NGI’09, Aveiro, Portugal
other (< 1%) instant messaging (3%) unknown (9%)
streaming media (22%)
web traffic (25%)
P2P traffic (40%)
Internet Access Traffic Measurement and Analysis Steffen Gebert
4 4
Agenda
u Measurement Setup
u Measurement Results § Internet Usage Statistics § Application Statistics § Flow Characteristics
u Conclusion
Imag
e by
cem
a ht
tp://
ww
w.s
xc.h
u/ph
oto/
1215
187
Internet Access Traffic Measurement and Analysis Steffen Gebert
5 5
MEASUREMENT DESCRIPTION
Internet Access Traffic Measurement and Analysis Steffen Gebert
10 10
Network and Measurement Setup
30 Mbps full-duplex
250 households
350 households
Class-based traffic shaping
Class-based traffic shaping
Non-‐whitelisted traffic thro2eled to
1 Mbps full-‐duplex
Internet Access Traffic Measurement and Analysis Steffen Gebert
11 11
Software PaLM (Packet Level Measurements)
u Custom C#-based tool
u Captures network traffic
u Applies OpenDPI Deep-Packet-Inspection (in real-time)
Prefers safe classification over vague heuristics
u Anonymized Packet / Flow data stored in
MySQL database
Pal
m Im
age
base
d on
„Ban
ana
Tree
“ by
ahyl
ton
http
://w
ww
.sxc
.hu/
phot
o/13
7407
5
Internet Access Traffic Measurement and Analysis Steffen Gebert
12 12
MEASUREMENT RESULTS
Internet Access Traffic Measurement and Analysis Steffen Gebert
13 13
Measurement Statistics
u 14 days in June/July 2010
u 600 households
u 4.95 billion packets
u 202 million flows
u 3.23 TB data observed
Imag
e by
zea
fons
o ht
tp://
ww
w.s
xc.h
u/ph
oto/
6742
43
Internet Access Traffic Measurement and Analysis Steffen Gebert
14 14
Internet Usage
(retrieved from the ISP‘s billing system)
u Monthly traffic per user
18% of users > 40 GB
4% of users > 100 GB
60% of users > 10 GB
Digital economy rankings 2010 - Beyond e-readiness Average traffic per household (in 2009): • 12 GB for Germany • 19 GB for US We measured • 21.9 GB on average • 135 GB maximum
Internet Access Traffic Measurement and Analysis Steffen Gebert
15 15
Classification Success
u We prefer safe classification over vague heuristics
u Large number of failed connection attempts
19.0%
43.7%
3.1%
34.1%
#Flows Classified
Unclassified with inverse flow
No inverse flow TCP
No inverse flow UDP
35.8%
63.6%
0.6%
Traffic amount
Internet Access Traffic Measurement and Analysis Steffen Gebert
16 16
The unkown Share
u Port-based exploration of unclassified traffic
u Majority on port 80 § Likely to be web traffic § Cross-validation shows that Bittorrent also uses Port 80
33.0%
Traffic Amount
TCP Port 80
TCP+UDP Port 13838
TCP Port 443
Other
57.7%
7.7% 1.6%
Internet Access Traffic Measurement and Analysis Steffen Gebert
17 17
Application Usage
608 GB
19.6 M 13.1 M
796 MB 500 GB
4.2 M
1 GB
2.3 M
33.1 GB 13.5 GB
u HTTP and Bittorrent dominate amount of traffic
u 2008 (share of total traffic) § 25% HTTP § 40% P2P
u 2010 (of classified traffic) § 47% HTTP § 38% P2P
u HTTP and DNS cause majority of flows
u Byte / Flow ratio depends on application
Internet Access Traffic Measurement and Analysis Steffen Gebert
18 18
Categorization of Protocols
Category Protocols (idenDfied by OpenDPI)
HTTP HTTP, HTTPS, DirectDownload
IM Jabber, Yahoo, ICQ, MSN, IRC
MAIL IMAP, PO3, SMTP
P2P BiSorrent, Gnutella, PANDO
MULTIMEDIA Flash, RTP, SHOUTcast, PPStream, MPEG, WindowsMedia
OTHER IPsec, DHCP, SNMP, NETBIOS, NTP, SSH, DNS, ICMP
UNKNOWN unknown
u Intention: Permit evaluation by application type
Internet Access Traffic Measurement and Analysis Steffen Gebert
19 19
Flow Sizes per Category
• Unclassified traffic contributes to smaller flows
• Flow size variation differs by application type
Internet Access Traffic Measurement and Analysis Steffen Gebert
20 20
Concurrent Flows
u Concurrently active flows at gateway u Calculated once per minute for one day
90% of time < 10 k
max: 16.3 k
mean: 6.9 k
• Peak rate more than double of average rate
• Estimate avg ~10 active flows per contracted user
Internet Access Traffic Measurement and Analysis Steffen Gebert
21 21
Flow Inter-Arrivals
60% within < 10ms
Overall avg. every 3.23 ms
Internet Access Traffic Measurement and Analysis Steffen Gebert
22 22
Conclusion
u Traffic observation of a german 600 househould access network
u Average traffic volume of 21.9 GB per month per user
u HTTP is dominant protocol § In number of bytes and flows § Increased share compared to 2008, overruled P2P
u Flow-based devices have to cope with § More than twice as much flows during peak times as on average § Roughly 10 active flows per user
Internet Access Traffic Measurement and Analysis Steffen Gebert
23 23
QUESTIONS, please!
Thanks to our shepherd Antonio Pescape'