View
37
Download
2
Category
Preview:
DESCRIPTION
“ MEHARI: A System for Analysing the Use of the Internet Services ” Presented by: Arturo Azcorra, Josep Solé-Pareta MEHARI Partners: UC3M, UPC and UPM. MEHARI Project Objectives. Traffic Capture Subsystem High Speed AAL5 Reasembly Modular and scalable Low cost - PowerPoint PPT Presentation
Citation preview
MEHARI - 1UC3M / UPC / UPM
M e h a r i
““MEHARI: A System for Analysing MEHARI: A System for Analysing the Use of the Internet Servicesthe Use of the Internet Services””
Presented by: Arturo Azcorra, Josep Solé-ParetaPresented by: Arturo Azcorra, Josep Solé-Pareta
MEHARI Partners: UC3M, UPC and UPMMEHARI Partners: UC3M, UPC and UPM
MEHARI - 2UC3M / UPC / UPM
M e h a r i
MEHARI Project ObjectivesMEHARI Project Objectives Traffic Capture SubsystemTraffic Capture Subsystem
High SpeedHigh Speed AAL5 ReasemblyAAL5 Reasembly Modular and scalableModular and scalable Low costLow cost
Support for many Traffic Analysis tools:Support for many Traffic Analysis tools: Detailed analyisis (including contents for AUP audits)Detailed analyisis (including contents for AUP audits) Identification and agreggation of bidirectional flows Identification and agreggation of bidirectional flows Traffic classification by usageTraffic classification by usage Traffic classification by origin / destinationTraffic classification by origin / destination Internet header verificationInternet header verification ......
MEHARI - 3UC3M / UPC / UPM
M e h a r i
MEHARI System Analysis Subsystem
TrafficSamples
Analysis Platform(s)
MEHARI Functional Architecture MEHARI Functional Architecture M e h a r i
Capture Subsystem
ATM 1
ATM 0ATM 1
ATM 0
Capture Platform(s)
ATM 1
ATM 0
ATMBackbone
ATMCells
Capturepoint
PPSPPSApplicationModules
IP Biflows+ symptoms
Statisticsand Reports
Auto-regulation
PreprocessingModule
Data base -patterns- addresses- ...
Operator
MEHARI - 4UC3M / UPC / UPM
M e h a r i
Capture SubsystemCapture Subsystem Modular and scalableModular and scalable
N units over the same or different trunk linksN units over the same or different trunk links Requires high speed connection to the analysis Requires high speed connection to the analysis
subsystemsubsystem Senses ALL VPI/VCI in the fiberSenses ALL VPI/VCI in the fiber
Captures in promiscuous or filtered mode over VPI/VCI Captures in promiscuous or filtered mode over VPI/VCI listlist
Capture capacity for each unit Capture capacity for each unit Sustained Average of 8 Mbit/s for a 6,000 Euros unitSustained Average of 8 Mbit/s for a 6,000 Euros unit 3,000% better price/performance than commercial 3,000% better price/performance than commercial
protocol analyzersprotocol analyzers Capture rate controled by analysis rateCapture rate controled by analysis rate
MEHARI - 5UC3M / UPC / UPM
M e h a r i
Information RegisteredInformation Registered
frameseq_num
timestamp UNIX(seg.µseg) VPI/VCI
length (bytes) Truncated AAL5info field
0:893083746.654070:100/1:1064 :45000428E81B40002F062E36C600B...1:893083746.654090:100/1:44:4500002C00AC400037069CF5CC4B3C...2:893083746.654101:100/1:40:45000028455840003606052FCF4F2C1...3:893083746.654280:103/224:1500:450005DC6C4B4000FD06142640...4:893083746.654288:103/224:40:45000028240440007B06401E829FD...5:893083746.654517:103/224:400:45000190B30340001D06B516238A.........1668:893083746.813551:100/1:281:4500011976710000FB04BFFCE40...#init_time=893083746.652986final_time=893083746.813582cap_time=0.160596
Files with programmable granularity
MEHARI - 6UC3M / UPC / UPM
M e h a r i
Pre-processing ModulePre-processing Module
Main functionsMain functions pseudo-packet agreggation to flowspseudo-packet agreggation to flows pseudo-packet analysispseudo-packet analysis count of symptoms associated to each flowcount of symptoms associated to each flow
Produces flow list with associated information:Produces flow list with associated information: flow desc with packet and byte countflow desc with packet and byte count weighted list of symptomsweighted list of symptoms
Highly configurable:Highly configurable: symptom definition and inter-relationsymptom definition and inter-relation aggregation periodaggregation period
MEHARI - 7UC3M / UPC / UPM
M e h a r i
Classification ModuleClassification Module
Current categories:Current categories: LEISURE, COMMERCIAL, ACADEMIC, UNKNOWNLEISURE, COMMERCIAL, ACADEMIC, UNKNOWN
Current heuristics (human auditing):Current heuristics (human auditing): 1º ‘known’ addresses1º ‘known’ addresses
- e.g.: banks (COM), playboy (LEI), sports newspapers (LEI)e.g.: banks (COM), playboy (LEI), sports newspapers (LEI) 2º dominant symptoms2º dominant symptoms
- e.g.: HTTP=2, PASSWD=3, VISA=1 (COM)e.g.: HTTP=2, PASSWD=3, VISA=1 (COM)- e.g.: MAIL=1, CHAT=4, SEX=3 (LEI)e.g.: MAIL=1, CHAT=4, SEX=3 (LEI)
3º non standard ports3º non standard ports- e.g.: ftp over ports other than 20/21 (UNK)e.g.: ftp over ports other than 20/21 (UNK)
4º ‘known’ ports 4º ‘known’ ports - e.g.: 6969 (LEI) e.g.: 6969 (LEI)
Academic by default
MEHARI - 8UC3M / UPC / UPM
M e h a r i
Traffic origin/destination analysis Traffic origin/destination analysis modulemodule
IP Biflows
Identificationof AS
Subnetwork,CIDR, ASs, ...
Databases
Traffic Origin/Destination Analysis Module (TODM)
Pre-processing Module (TCM)
Processor
SummaryReport Files
Official IRRData Bases
NRN BGP
other...
MEHARI - 9UC3M / UPC / UPM
M e h a r i
Internet headers analysis moduleInternet headers analysis module M e h a r i
Internet Headeranalysis
(session oriented)
Capture Files
Pre-processing
Unknown TrafficProcessor
Data basewith header patterns
Summary Report Files
Internet Header Analysis Module (IHM)
- % Verified traffic- % Pending traffic
Summary Report Files(unknown traffic)
Summary Report Files
-Remote and local servers
MEHARI - 10UC3M / UPC / UPM
M e h a r i
Modularity and Scalability of MEHARIModularity and Scalability of MEHARI
P 1.1
P 1.2
P 1.3
P 1.1.2
P 1.1.1
P 1.3.1
P 1.1.3
Process tree structure for information flowProcess tree structure for information flow Interprocess Comunication using shared filesInterprocess Comunication using shared files May be distributed among several machines using NFSMay be distributed among several machines using NFS
MEHARI - 11UC3M / UPC / UPM
M e h a r i
Some applications of these toolsSome applications of these tools
Traffic monitoringTraffic monitoring Billing and charging models for NRN and Corporate Billing and charging models for NRN and Corporate
NetworksNetworks Network configurationNetwork configuration
- Resources dimensioning Resources dimensioning - Placing Proxies, ... Placing Proxies, ...
Service usage controlService usage control Control that the services are used responsibly, i. e. Control that the services are used responsibly, i. e.
auditing the academic networks AUP (Acceptable auditing the academic networks AUP (Acceptable Use Policy)Use Policy)
SecuritySecurity
MEHARI - 12UC3M / UPC / UPM
M e h a r i
ConclusionsConclusions Modular, scalable and extensible architectureModular, scalable and extensible architecture Capture systems with excelent price/performanceCapture systems with excelent price/performance Flow information aggregation with symptoms and Flow information aggregation with symptoms and
bidirectional flow correlationbidirectional flow correlation Intermediate data base of patterns and addressesIntermediate data base of patterns and addresses Application modules currently implemented:Application modules currently implemented:
Classification by usage (AUP)Classification by usage (AUP) Classification by origin/destinationClassification by origin/destination Internet header analysisInternet header analysis
MEHARI - 13UC3M / UPC / UPM
M e h a r i
Future workFuture work Further improvements in capture capacityFurther improvements in capture capacity Applications to detect security attacksApplications to detect security attacks Graphical user interfaceGraphical user interface Automatic reaction to incidents:Automatic reaction to incidents:
Alarms (mail, pager, SNMP, ...)Alarms (mail, pager, SNMP, ...) Flow blocking or re-routingFlow blocking or re-routing Flow logging for off-line human analyisisFlow logging for off-line human analyisis
Other type of statistics:Other type of statistics: Traffic statistics, as those provided by the NetFowTraffic statistics, as those provided by the NetFow Top 100 lists of hosts/serversTop 100 lists of hosts/servers Main origins/destinations of trafficMain origins/destinations of traffic Most popular sites (webs, ftps, chat servers, ...)Most popular sites (webs, ftps, chat servers, ...)
MEHARI - 14UC3M / UPC / UPM
M e h a r i
Trial on Spanish NRN: RedIrisTrial on Spanish NRN: RedIris
GIGACOMTelefónica ATM
Network
RedIRISRegionalNodes
Splitters
ATM Access Switch
Analysis PC(LINUX)
100 BaseTEthernet
NFSInternet(RedIris)
Remote AccessTraffic Capture PC
(FreeBSD)
1
0
STM-1 ATMOptical
Interfaces
RedIRISCore
Router
RedIRIS: the Spanish NRN
MEHARI - 15UC3M / UPC / UPM
M e h a r i
Sample of Results: Sample of Results: Traffic classification by usage (I)Traffic classification by usage (I)
% Bytes (Input traffic)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Academic Leisure Commercial Unknown
User Groups (17)
MEHARI - 16UC3M / UPC / UPM
M e h a r i
Sample of Results:Sample of Results:Traffic classification by usage (II)Traffic classification by usage (II)
Total Input traffic to RedIRIS (% Bytes)
Academic78%
Leisure17%
Unknown2%Commercial
3%Leisure
12%
Commercial2%
Total Output traffic to RedIRIS (% Bytes)
Unknown2%
Academic84%
MEHARI - 17UC3M / UPC / UPM
M e h a r i
Sample of Results: Sample of Results: Main traffic origin/destination (I)Main traffic origin/destination (I)
RedIRISTEN-34/155IbernetRest of Internet (through USA)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%% Bytes (Input traffic)
User Groups (17)
MEHARI - 18UC3M / UPC / UPM
M e h a r i
Sample of Results: Sample of Results: Main traffic origin/destination (II)Main traffic origin/destination (II)
Total Input traffic to RedIRIS
26%
21%
12%
41%36%
16%21%
27%
RedIRISTEN-34/155IbernetRest of Internet (through USA)
Total Output traffic from RedIRIS
MEHARI - 19UC3M / UPC / UPM
M e h a r i
Sample of Results: Sample of Results: % of a% of academic traffic in the link with USA cademic traffic in the link with USA
(according with the IRR description)(according with the IRR description)
0 %
10 %
20 %
30 %
40 %
50 %
60 %Input traffic
% o
f cap
ture
d tr
affic
User Groups (17)
MEHARI - 20UC3M / UPC / UPM
M e h a r i
Sample of Results: Sample of Results: Top 25 most visited Top 25 most visited commercial sites incommercial sites in
one of the user groups one of the user groups
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
TSA
I
RE
DE
STB
OLE
ES
-CTV
-980
527
ES
-TTD
-951
020
CA
IXA
-RE
D
CA
NA
L-P
LUS
-SP
AIE
S
AB
F
ICTN
ET
IBE
RN
ETC
OM
ES
-FC
R-9
5060
7
JETN
ET
GR
N
CO
NE
XIS
IP-M
ULT
IME
DIA
SE
RV
ICO
M2-
NE
TS
AB
CTE
LEM
ATI
C
SP
RIT
EL
SE
RV
ICO
M1-
NE
TS
DA
UC
OM
2ME
G-E
S
RA
N
FUT
INFA
SE
RS
INTE
RC
OM
Oth
er S
ub-N
etw
orks
: 958
% Bytes (Input traffic to one of the user groups)
MEHARI - 21UC3M / UPC / UPM
M e h a r i
Sample of Results (January-February´99): Sample of Results (January-February´99): Top 25 most visited TEN-155 ASs inTop 25 most visited TEN-155 ASs in
one of the user groups one of the user groups
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%
AS
1275
DFN
-IP s
ervi
ce a
nd D
FN c
usto
mer
net
wor
ks
AS
786
The
JA
NE
T IP
Ser
vice
AS
1653
SU
NE
T S
wed
ish
Uni
vers
ity N
etw
ork
AS
1103
SU
RFn
et
AS
2856
BTn
et U
K R
egio
nal n
etw
ork
AS
224
UN
INE
TT, T
he N
orw
egia
n U
nive
rsity
& R
esea
rch
Net
wor
k
AS
1717
RE
NA
TER
AS
3301
Tel
iaN
et S
wed
en
AS
2852
CE
SN
ET
z.s.
p.o.
- TE
N34
-CZ
AS
513
CE
RN
AS
1853
AC
One
t Bac
kbon
e
AS
1239
AS
559
SW
ITC
H, S
wis
s A
cade
mic
and
Res
earc
h N
etw
ork
AS
8761
RE
TEN
ET
Aut
onom
ous
Sys
tem
AS
1741
FU
NE
T au
tono
mou
s sy
stem
AS
8743
Hig
hway
One
Aut
onom
us S
yste
m
AS
1835
DE
Net
- D
anis
h N
etw
ork
for R
esea
rch
and
Edu
catio
n
AS
3269
TE
LEC
OM
ITA
LIA
AS
6805
med
iaW
ays
Aut
onom
ous
Sys
tem
AS
3215
RA
IN
AS
5470
AU
TH-N
ET-
AS
AS
5556
Tel
enor
dia
AB
AS
8209
A20
00 /
Kab
elte
levi
sie
Am
ster
dam
bv
AS
2529
Dem
on In
tern
et L
td
AS
1290
PS
INet
UK
Ltd
.
Oth
er A
ss: 4
33
% Bytes (Input traffic to one of the user groups)
MEHARI - 22UC3M / UPC / UPM
M e h a r i
Sample of Results : Sample of Results : Internet Headers Verification Internet Headers Verification
0.1 %
84.9 %
13.5 % 1.5 %
PendingVerifiedUnknownRejected
Recommended