Efficient & Robust TCP Stream Normalization

Mythili Vutukuru

Joint work with Hari Balakrishnan and Vern Paxson

2

Network Intrusion Detection Systems

IDS

attack

attack

Evasion Attacks

3

Evasion by Fragmentation

at tack

at tack

IDS must parse data stream in order.

4

Evasion by Ambiguity: Inconsistent Retransmissions

at junk

tack

at junk

tack

Low TTL

Inconsistent TCP segments

Tools to create such attacks exist.

Makes IDS ineffective.

5

TCP Stream Normalizer

at junk

at junk

tack

Removes ambiguity from network traffic.

Detects inconsistent TCP segments.

6

Existing Normalizer Designs

Buffer all unacknowledged

data.

Buffer content hashes of

unacknowledged data.

at junk

tack

H(at) H(junk)

H(tack)

7

Problems With Existing Designs

Too much memory.

Partial overlaps.

• 20–30% of retransmits

in 5 real-world traces.

• Caused by repacketization.

State exhaustion attacks on the normalizer.

at junk

tack

H(at) H(junk)

H(tack)?

8

Related Work1. Evasion attacks. [Ptacek and Newsham, 1998]

2. Concept of normalization by storing all unacked data. [Malan et al., 2000] [Handley et al., 2001]

3. Buffering hashes of data (without handling partial overlaps). [Sugawara et al., 2005] [Commercial normalizers]

4. Reassembling data streams robustly. [Dharmapurikar and Paxson, 2005].

5. Normalization for signature matching only. [Varghese et al., 2006]

9

RoboNorm

Detects inconsistent TCP retransmissions.

Memory Efficiency: stores only hashes.

Robustness:

• Handles partial overlaps correctly.

• Withstands memory exhaustion attacks.

10

RoboNorm: Basic Mechanism

1-100

H(1-100)

101-200

H(101-200)

101-200

H(101-200)EqualNot equal

51-150

51-100

101-150

Hash store

Partial retransmits held back ...

... until fitting segments arrive.

1-50 151-200H(1-100)

Fitting segments

11

Will segments be held forever?

H(1-100) H(101-200)

51-100

101-150

Hash store

1-50

ACK:1

ACK:101

101-150

Sender TCP stalls!!

Partial retransmits held back.

~2 in thousand connections prone.

12

TCP Stalling: Fixing The Problem

H(1-100) H(101-200)

101-150

Hash store

ACK:101

151-200

Partial retransmits held back.

ACK:151

ACK promotion

Necessary to check partial overlaps.

13

Putting it all together...

RoboNorm

Connection Tuple Ptr

Connection TableHash Store

Held retransmits

DATA ACK

Suitable for hardware implementation.

14

Memory Footprint

Connection Tuple Ptr

Connection TableHash Store

Held retransmits

Segment arrival rate& holding time

Max concurrent partial overlaps

Max # concurrent connections

15

Memory Footprint – Trace Analysis

Connection Tuple Ptr

Connection TableHash Store

Held retransmits

375 KB

100 KB

2 MB

2.5 MB on a Gbps link.

10 X less than storing all content.

Up to 66 X in practice.

16

RoboNorm

Detects inconsistent TCP retransmissions.

Memory efficiency: stores only hashes.

Robustness:

• Handles partial overlaps correctly.

• Withstands memory exhaustion attacks.

17

Memory Exhaustion

Connection Tuple Ptr

Connection TableHash Store

Held retransmits

Goal: should not consume RoboNorm memory “cheaply”.

No new vulnerability.

18

Connection Table

Init state on first data. Reclaim space for

inactive conns. Timeout Bloom Filter.

Inactive Connection Bloom Filter

1 0 1 1 0

SYN Flood. Keep conns idle.

Unterminated conns.

1 0 1 1 0

SYNACK Bloom FilterConnection Tuple Ptr

Connection Table

SYN ACK

Inactive connSave ~50% space with 5 min inactivity timer.

48 bytes 1 byte.

19

Connection Table

Exhaust connection table memory only by:• Opening large number of conns.

• Actively sending data on all of them.

Inactive Connection Bloom Filter

1 0 1 1 0

1 0 1 1 0

SYNACK Bloom FilterConnection Tuple Ptr

Connection Table

No new vulnerability.

20

H(XY)

Hash Store

Pick conn with largest

Coalesce hashes. Or evict connection if

avg segment size large.

Small segments. Segments stored for

long time.

avg segment holding time

avg segment size

H(X) H(Y)

21

Hash Store

Exhaust hash store memory only by:• Sending data in large packets.

• Clearing packets fast.

Fill hash store only by consuming link bandwidth.

22

Hash Function

Hn(X) = (an.X + bn) mod pn

pn = n-bit prime

an in {1,...,pn-1} bn in {0,...,pn-1}

Hn(XY) = { Hn(Y) + 2k [Hn(X) – bn] } mod pn

n = 64 provides sufficient security.

23

Conclusion

TCP Stream Normalizer design that:

• Is memory efficient.

• Detects all inconsistent retransmissions.

• Is robust to state-exhaustion attacks.

Backup Slides

25

Connection table• # peak conns = 34,000

• Bytes per conn = 48

Hash store• Avg sgmt hold time = 200 ms

• Sgmt arrival rate = (1 Gbps / 1000 B)

• Bytes per hash = 15

Held retransmits = 100 KB

Memory Footprint: Trace Analysis

2 MB

375 KB

100 KB

~2.5 MB

26

Eviction Policy of Hash Store

λi avg rate of segment arrival

δi avg hold time of segments.

si avg segment size.

λi δi hash memory consumed (cost)

λi si bandwidth consumed (benefit)

δi / si cost-to-benefit ratio.

Evict conn with largest δi / si