Merit’s CALEA Compliance Merit’s CALEA Compliance Architecture and Platform,Architecture and Platform,

“OpenCALEA”“OpenCALEA”

Mary Eileen McLaughlin,Merit - Director Technical Operations

Manish Karir, Merit - Research and Development

Agenda

Merit’s CALEA decision Technical compliance experiment goals Merit’s approach Experiments to test software and

network functionality Results OpenCALEA Toolset description Case studies for data integrity Next steps

Merit’s CALEA Decision

Merit believes it will need to be “Gateway compliant” for CALEA– Will need to have a device at the ingress/egress points of our

network, to/from the public Internet– In other words, where traffic enters or leaves AS-237– About 9 sites including private peering points

Rationale for compliance at the gateways:– Merit is interconnected to the public Internet at various places. – Merit “supports its connection to the Internet” because it owns

connectivity equipment as well ;– Merit purchases commodity Internet service from various public

Internet providers, that is delivered over its facilities .– Merit’s interconnected network is a “private” network, however,

because Merit limits the availability of its services to only its Members and Affiliate Members.

cont.

Merit’s CALEA Decisioncont.

LEAs can, under CALEA, request surveillance of traffic where it connects to public Internet– Not within a private network, i.e., between

two universities on our network This presentation isn’t about the legal

pros/cons, or the expectations of law, or the challenges– It’s about what are we doing relative to the

above conditions

Experimentation Goals

1. Develop an experimental reference architecture as a model for CALEA compliance

2. Determine what level of compliance is possible at a reasonable price point

3. Experiment with simple hardware/software in order to determine suitability for compliance

4. How well will this solution scale (10G cards, multiple sites) compared to price/performance of commercial solutions

5. Gain a technical understanding of what is required to be CALEA compliant

Approach1. Build and deploy a packet capture platform

– Experimental Architecture 1 -- Dell Precision GX260 Workstation, 2 GIGE interfaces for management and sampling, Pentium 4 3GHz, 1GB RAM, Linux

– Experimental Architecture 2 -- Dell PowerEdge860 1U server, Dual Pentium 2.8GHz, 1 GIGE interface(mgmt), 1myricom 10GIGE adapter, 1GB RAM, Linux

– Tcpdump/tethereal for packet capture -- both depend on pcap library,

– Iperf as the traffic generator

2. Test ability to capture a single data stream in the presence of varying amounts of live background network traffic

3. Metrics: packet loss, cost

Experiment 1 Architecture

Experiment 1 Methodology

1. Background traffic for the duration of the test: ~ 190-225Mbps (Sunday evening load)

2. Repeat for higher traffic load ~400Mbps (Monday afternoon)

3. Test– Send data from source to sink using iperf– Attempt to capture traffic stream at capture

device (full packet captures not just headers)– Measure actual number of packets

transmitted at the source and compare with number of full packets captured

– Measure for Small/Medium/Large UDP flow

Experiment 1 Results

Experiment Network Load

Avg Packet Loss %

10 sec UDP- 390kbps

200Mbps < 1.0

5 min UDP - 390kbps

200Mbps < 1.0

30 min UDP - 390kbps

200Mbps < 1.0

5 min UDP - 390kbps

400Mbps < 1.0

Experiment 1 Conclusions

1. Less than 1% (0.6 - 0.7%) of the packets are missing at the capture device (at a load of roughly 200Mbps).– This appears to hold at least to an aggregate load level of

400Mbps (bidirectional traffic mirrored onto a single port)

2. Losses are NOT in the packet capture process but in the datapath itself.– A UDP stream along the same path at 380Kbps

experienced roughly the same packet loss, implying that the simple hardware/software solution holds promise for at least the lower rate uplink capacities (definitely for OC-3, sub-GIGE type rates) .

3. Total cost of hardware/software: ~$1000

Experiment 2 Architecture

Experiment 2 Methodology1. Scale up experiment 1 architecture to links that

carry over 2Gbps of traffic– Use of better hardware platform: Dell 1U server– 10GiGE Myricom Ethernet Adapter

2. Test ability to deliver the captured packets to LEA– Simple custom software which operates similar to

tcpdump but additionally can transmit packets to LEA

3. Test ability to operate in the presence of complications. (Such as VLANS ~40vlans mirrored on single interface)

4. Measure ability to capture higher bitrate streams in presence of higher background traffic

Experiment 2 ResultsUDP stream with average background network

load of 2.3-2.4 Gbps

Experiment Stream Bitrate

Avg Packet Loss %

5min UDP - 25K packets

1Mbps ~0.0

5 min UDP - 127K packets

5 Mbps ~0.0

5 min UDP - 255K packets

10Mbps < 1.0

5 min UDP - 636K packets

25 Mbps < 1.0

Experiment 2 ResultsUDP stream with average background network load of >

2.5Gbps

Experiment Packet Loss at Tap

Packet Loss at LEA

5min UDP - 100kbps

< 1% < 1%

5min UDP - 200kbps

< 1% < 1%

5min UDP - 400kbps

< 1% < 1%

5 min UDP - 1Mbps

< 1% < 1%

Experiment 2 Conclusions1. Return Path Characteristics are Important -

otherwise there can be packet loss on path to LEA.

2. Check for MTU -- Encapsulation can lead to packet size > 1,500Bytes. (MTU should be able to support jumbo frames on the path to LEA).

3. Packet capture at > 2Gbps network load appears to be feasible.

4. Hardware/software cost: ~ $2,500 (server $1300 + 10Gige I/F card, $1200)

5. Need to Verify: Is there any data impairment during the capture/transfer/writing process?

(See final slides for partial answer.)

OpenCALEA Software ToolsetTap Tool:1. Tap: Perform packet capture

– Receive packets via libpcap interface– Create new UDP packet in appropriate format– Encapsulate captured packet into new packet – Timestamp information to UDP packet– Send to LEA collection IP address– Send the packet header information on

separate UDP port2. Example Usage:

./tap -d 198.108.62.77 -i any -c -f "host 198.108.62.77 and port 5001"

OpenCALEA Software ToolsetLEA Receiver Tool (Consistent with standard):

3. Example of LEA collection function implementation: lea_collect– Receive UDP packets sent by tap– Remove encapsulation– Create standard libpcap packet based on

timestamps and encapsulated packet– Write packet to file– Write packet header information sent by tap

4. Example Usage:

./lea_collect -f capture-file.pcap

OpenCALEA Software Toolset

User Front End (in development):

5. calea_controller:

Responsible for initiating a tap on remote tap devices but issuing the appropriate command

6. calea_collector:

Responsible for listening for commands from calea_controller and initiating the tap with the appropriate filters

Case Study: Capturing Web Browsing Traffic

Question: Is there any data impairment during the capture/transfer/writing process?

1. Web Browsing:– http://www.opencalea.org– Google search example

2. Capture traffic to/from IP address

3. Background network traffic load ~2.4Gbps

4. Tap is to filter IP-address and port 80

5. Tap forwards stream to LEA Collector where it is saved to disk

6. Analyze saved file using tools, e.g., tcpxtract in order to examine accessed web pages

Capturing Web Browsing Traffic

Web Page Reconstructed from Intercepted Packets

Test performed to validate integrity of packets captured.

Capturing Web Browsing Traffic

Web Page Reconstructed from Intercepted Packets

Test performed to validate integrity of packets captured.

Case Study: Capturing Instant Messenger Conversations

1. Capture traffic to/from IP address

2. Background network traffic load > 2.5 Gbps

3. Tap is to filter IP-address and AIM port

4. Tap forwards stream to LEA Collector where it is saved to disk

5. This saved file is then analyzed using tcpdump in order to extract the ASCII text within

Case Study: Capturing Instant Messenger Traffic

Case Study: Capturing Instant Messenger Traffic

Conclusions1. A cost-effective CALEA solution was

developed and tested

2. The solution has performed well in initial testing

3. The solution appears to be- Consistent with technical requirements- Cost effective- Practical

4. Merit plans to use this solution for CALEA compliance

Next Steps Merit will file its Compliance document by

February 12th Continue to fine-tune “OpenCALEA”

software, and develop user interface

– Software release in mid-February Draft SSI document March 1 and release

to community (Quilt, StateNets, etc.)

– Commentary welcomed SSI to be filed by March 14th

Compliance by May 14th