View
220
Download
3
Category
Tags:
Preview:
Citation preview
Standards Activities on Traffic Measurement
2
Outline
• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups
3
Applications Requiring Traffic Measurement
(1)• Usage-based accounting
– input to charging and billing– various business model
• time-based, volume-based, QoS class-based• per application, per user, per user group
• Traffic engineering– optimizing network usage– traffic analysis on congested links
• origin of traffic• type of traffic• dynamic behavior (bursty, adaptive, …)
• Traffic profiling
4
Applications Requiring Traffic Measurement
(2)
• QoS monitoring– (passive) measurement of QoS properties– validating Service Level Agreements
• Attack detection and analysis– detecting (high volume) traffic patterns– investigation of origin of attacks
• Intrusion detection– detecting unexpected or illegal packets
• …
5
Outline
• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups
6
The General Traffic Flow Measurement Process
Classification &Flow Recording Store
(TCPdump)Observation
Point
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PacketCapturing
Filtering
Display(Ethereal)
Sampling
Visualize(FlowScan)
Analysis by applications
(TE, attack detect., QoS monitoring,
accounting, …)
… other …
packets
Filtering
Samplingflow records
flow records
packets
packets
flow recordsflow
records
7
The General Traffic FlowMeasurement Process
• Packet capturing at observation point• Packet sampling and filtering
– both steps may be trivial (1:1 sampling, no filtering) – both steps may be applied repeatedly
• Packet classification, mapping to flow records, maintaining of flow records
• Flow record sampling and filtering– both steps may be trivial (1:1 sampling, no filtering) – both steps may be applied repeatedly
• Processing flow records in application
8
Filtering
Samplingpackets
packets
Capturingpackets
Packet Capturing Protocols
• Packet Capturing Protocols– Capture packets at
observation point– Optionally: Sample and
filter packets– Export packets or parts of
packets (e.g., first 100 bytes)
– Packet classification, flow recording and processing after transfer
• Proprietary: sFlow• Standard (to be): PSAMP
packet transfer
Classification &Flow Recording
Application
flow records
rout
er
or p
robe
9
Filtering
Sampling
packets
Capturing
Flow Monitoring Protocols
• Flow Monitoring Protocols– Capture packets at
observation point– Optionally: Sample and
filter packets– Classify packets and
update flow records– Export flow records– Flow record processing
after transfer
• Proprietary: NetFlow, LFAP, CRANE
• Standard: Meter MIB, IPFIX flow record transfer
Classification &Flow Recording
Application
flow records
Filtering
Sampling
rout
er
or p
robe
10
Comparison
• Packet Capturing Protocolso simple function on router
or probe+ low cost on router or probe
– high data volume for packet transfer orunreliable recording because of sampling
– packet classification required after data transfer
• Flow Monitoring Protocolso more complex functions
on router or probe– high resource
requirement on router or probe: fast memory for flow records
+ low data volume for flow record transfer
+ flow records available after data transfer
11
Outline
• Applications requiring traffic measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working groups
12
Protocols
• Packet Capturing– sFlow (InMon Corporation, HP spin-off)– PSAMP (under standardization at IETF)
• Flow Monitoring– LFAP (Riverstone)– CRANE (XACCT)– NetFlow (Cisco)– IPFIX (under standardization at IETF)– RTFM Meter MIB (IETF standard)– RMONMIB (IETF standard)
13
Data collector
sFlow
• By InMon Corporation• Includes packet capturing,
sampling and packet transmission• Statistical packet sampling• Timestamping at data collector• Configuration by sFlow MIB• RFC 3176, www.sflow.org• Applicable to high speed links
when sampling is used• Adopted by many vendors (HP, Hitachi,
Alaxaia - by Hitachi and NEC, Extreme and more)
sMon Meter
Application
Packets
14
Data collector
PSAMP
• Under standardization at IETF Packet Sampling WG
• Very similar to sFlow• Time stamping by meter• Configuration by PSAMP MIB• Intention to use IPFIX protocol
for packet transferPSAMP Meter
Application
Packets
15
LFAP• Light-weight Flow Accounting Protocol• Proprietary by Riverstone (Cabletron)• Just data transfer protocol• Meter at Connection Control Entity (CCE)
communicates to Flow Accounting Server (FAS)
• Tight and reliable interaction between CCE and FAS
• Reliable data transport• Flexible TLV coding of transferred data• Larger overhead than NetFlow • More cost-intensive at meter/CCE
and at data collector/FAS
CCE
Application
FAS
Flow records
16
CRANE
• Common Reliable Accounting for Network Element (CRANE) Protocol
• Proprietary by XACCT• Just data transfer protocol• Template-based data model• Focus on reliability• Not yet in extensive commercial use
17
Data collector
IPFIX
• Under standardization at IETF IP Flow Information eXport WG
• Very similar to NetFlow version 9• Will not use UDP, but use TCP or
SCTP (Stream Control Transmission Protocol)
• Standardization close to completion• Close collaboration with PSAMP WG
IPFIX Meter
Application
Flow records
18
Router
NetFlow• Proprietary by Cisco, but de-facto standard• Fast and efficient, implemented for IOS• Configurable measurement per 5-tuple• Unreliable data transport (UDP)• Hardware-supported on some models• Not well documented
– re-engineered by Juniper• Versions 1, 3, 5, 7, 8
– fixed data model– no support of IPv6 flows
• Version 9 (starting point for IPFIX standard)– data model templates– can report IPv6 flows– optional reliable transport– not related to older versions!– RFC 3954
Meter
Data collector
Application
Flow records
19
Real-Time Flow Measurement (RTFM)
• Very flexible and powerful meter– programmable rule sets– can serve several readers– programmable overload behavior
• Reader polls meter• Realization by SNMP Meter MIB• Free software implementation
NeTraMet• No acceptance at manufacturers• Complicated to use (too powerful)• Specified by RFCs 2720 - 2724
Meter
Manager
Reader
Application
Flow records
Config.
20
Remote Network Monitoring MIB (RMON)
• Very flexible and powerful• Serves more general goals (analysis on layers 2-4)
• Just a monitoring tool, no measurement architecture defined
• Suited for very specific analysis tasks• High (hardware) performance requirements• Too complicated and too expensive for massive
usage in routers• Specified by RFCs 2021(RMON2), 2613,
2819(RMON), 2895, 2896, 3144, 3287, 3273, 3395, 3434, 3577, 3729, 3737, 3919, 4149, 4150
21
Outline
• Applications requiring traffic
measurement
• Packet capturing and flow measurement
• Existing protocols
• IETF IPFIX and PSAMP working group
22
IETF IPFIX Working Group
• IP Flow Information eXport (IPFIX)– BoF sessions 12/00 and 08/01– active since 10/01
• Successor of RTFM (Real-Time Flow Measurement) WG
• Target (official): standardizing current practice– Target (unofficial): standardizing (something like) Cisco
NetFlow• Chairs
– Nevil Brownlee, CAIDA– David Plonka, University of Wisconsin
23
IPFIX Scope and General Requirements
Goal: Find or develop a basic common IP Traffic Flow measurement technology to be available on (almost) all future routers
• Fulfilling requirements of many applications• Low hardware/software costs• Simple and scalable• Metering to be integrated in general purpose IP
routers and other devices (probes, middleboxes)• Data processing to be integrated into various
applications• Interoperability by openness or standardization
24
Application
IPFIX Architecture
Flow Record
Observation Point
Flow Information Export
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
MeteringProcess
ExportingProcess
CollectingProcess
25
IPFIX Devices
OM
E
Probe
OM
E
Simple Router
O OO OM
E
Complex Router
O OO OMO OO O
M
E
Multiple Exporters
O OO OM
E
O OO
OM
E
Protocol Converter
(Meter MIB)
OM
E
OM
E
OM
E
M E
Concen-trator
C E
Proxy
C …M: MeterE: ExporterC: Collector
26
IPFIX WG: Expected Output
• Planned documents– Requirements RFC (completed, RFC 3917)– Evaluation RFC (completed, RFC 3955)– Protocol specification (in progress)– Data Model (in progress)– Architecture RFC (in progress)– Information model RFC (in-progress)– Applicability RFC (in-progress)
• No new protocol development in working group• Instead: protocol selection and refinement• Selected protocol: NetFlow version 9• Configuration of measurements
will not (yet?) be standardized
27
IPFIX WG: Current Status
• Good support from IESG (Internet Engineering Steering Group)
• High interest from equipment manufacturers– Cisco designed NetFlow v9 compliant to IPFIX
requirements and contributes to documents– Riverstone/Enterasys contributing actively– Juniper is closely monitoring progress
• Several accounting and billing system providers are monitoring and contributing– HP, XACCT, InMon, ...
• More information at http://ipfix.doit.wisc.edu
28
IETF PSAMP Working Group
• Packet SAMPling (PSAMP)– BoF session 03/02– active since 07/02
• Initiated by Nick Duffield, AT&T• Target: standardizing new technology for sampling,
filtering and exporting packets– can be interpreted as a component of the IPFIX
measurement process– but different to IPFIX, there is no current practice
• Chairs– Juergen Quittek, NEC
29
PSAMP Scope and General Requirements
Goal: Develop effective but low-cost packet sampling technology
• Allowing measurements at high-speed links• Fulfilling requirements of applications using per
packet measurement– QoS analysis, traffic profiling
• Very low hardware/software costs• Much simpler than IPFIX• Will use subset of IPFIX protocol• Metering to be integrated in general purpose IP
routers and other devices (probes, middleboxes)• Configuration of sampling included (different than
IPFIX)
30
Application
PSAMP Architecture
Packet Record
Observation Point
Packet Information Export
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
Sampling & Filtering Process
ExportingProcess
CollectingProcess
31
PSAMP WG: Expected Output
• Planned documents– Architecture RFC (in progress)– Packet Sampling and Filtering Spec. RFC (in progress)– Report Format and Protocol specification (close to final
document)– PSAMP MIB RFC (close to final document)– Applicability RFC (not started)
• Dependencies on IPFIX protocol development
32
PSAMP WG: Current Status
• Good support from IESG (Internet Engineering Steering Group)
• Growing interest from equipment manufacturers– Main drivers are AT&T, Cisco and NEC– Avaya is actively contributing– Alcatel, Avici, InMon, Lucent are monitoring and joining
discussions– Cisco shows strong interest in having PSAMP close to
IPFIX in order to re-use their existing IPFIX software
Recommended