Loss and Delay Accountability for the Internet

Defense by

Jiazhen Chen & Theodore Brockly

Motivation

• Internet provide no guarantees about when or if packets will be delivered– Enabling flexibility of technologies– Simplify forwarding path

• However– To make an informed decision about paths with poor service

• Needs to know which domains are currently dropping or delaying its packets

– Form of performance feedback would help establish whether providers are performing their duty

Probing vs. AudIT

• Ping and Traceroute– Reveal how network reacts to probe packets– Fail to capture low-rate or sporadic misbehavior– Hard to be allowed to inspect the internal

infrastructure of ISPs

• AudIT– Explicit accountability interface– ISPs report their performance through it

• ISPs can’t lie regarding their business• Easily done with modification on NetFlow

The Goals

• Determine which administrative domains are losing or delaying their packets– Provide a traffic source with enough feedback to

determine

– Guarantee an upper bound on the error that a malicious AD can induce

– Tampering should be attributable to a specific link

– Do no harm

• This idea has contribution to the design of next generation Internet.

Threat model

• Our threat model does not allow a malicious AD to modify or otherwise tamper with traffic reports from other Ads– They sign legally binding service-level

agreements. [Business relationship]

Accountability Interface

• AudIT definition– Feedback packet

• Fields:

Accountability Interface

• Each AD can choose its own granularity of reporting aggregates– The only restriction is that two aggregates

• no packets in common • one is a subset of the other

Informal Aggregate and Feedback algebra

• Assume two aggregates

• If they are combinable:

• For a feedback entry – use vector notation

Using AudIT

• AudIT can be used to provide traffic sources with performance feedback

• Honest Reporters– Combine collection of feedback entries on

aggregate from multiple Ads to determine AD-level path.

Using AudIT with -Honest Reporters

Using AudIT with On-Path Lies

• On-Path Lies: an AD misrepresents its performance

• Detection:– If two ADs’ Feedback entries on the same degree

disagree on• How many packets the earlier AD delivered• When it delivered them

– Conclude that either one of them is lying, or the inter-link between them is in trouble

Using AudIT with On-Path Lies

• “Consistent”

Using AudIT with On-Path Lies

• “Blames”– When feedback entries on the same degree

are consistent, an AD can only lie about its performance by implicating one or more of its peers

• Y drops the packet but claim it never got them• Y tries to hide delay by claim it received it ms later

than it actually did

Using AudIT with On-Path Lies

• Lemma– If AD X produces correct feedback entries on traffic

aggregate , then none of X’s peers can blame any loss or average delay on X with respect to without causing a feedback inconsistency (we omit the straightforward proof for lack of space).

• The more information an AD generates about its performance, the more difficult it is for its peers to blame their faults on it.

Using AudIT with On-Path Lies

• Localization– Exposing lying Ads to the peers they

implicated• If an AD responds with a signed entry that differs

from the original one, then S concludes that AD is lying

• If both Ads insist on their original reports, then S sends both signed entries to both X and Y.

– If both tell the truth, they might investigate their inter-AD link

– Otherwise, the lying AD is exposed to the peer it implicated

Using AudIT with On-Path Lies

• Strong Incentive for truth telling– If lying means implicating a peer who will

learn that, then lying means entering a dispute with that peer and damaging the corresponding business relationship.

Using AudIT with On-Path Lies

• Role of Inter-AD links– Feedback inconsistencies between two

peering Ads are impossible to properly ascribe when in fact it is the inter-AD link between them that has failed

Using AudIT with Off-Path Lies

• A malicious A lie about having seen an aggregate when it in fact has not.– When all Ads on an aggregate’s path provide

feedback, the AD liar is situated off the actual path of the aggregate

Basic Feedback on TCP traffic

• AudIT reporting TCP-flow statistics in the number of packets lost and the average delay.

• Checkpoints– Inter-AD links / Inside a router / separate box– Places on all links that traffic enters and exits

• Roles: Entry Point and Exit Point

– Need clocks synchronized• To use NTP or use a GPS receiver with less cost

Accountability Center

• Part of its network management platform– A Module / Cluster

• Depends on the size of AD and the amount of traffic it generates and forwards

– Two Interface:• Feedback receiver interface• Follow up interface

Packet Classification

• A Check point consider packets belong to the same aggregate type if:– All packets have the same Identity Tuple– Any FIN and RST packet is the last one

• Flow Records– Maintain

short-term

state

Determine Where to Report

• To send feedback to the corresponding source Ads– Feedback-receiver map

• First it compiles an IP-prefix-to-origin-AD map using BGP data from border router

• Then it looks up the feedback-receiver address for each origin AD and adds that information to the map

Determine Where to Report

• Tricky scenarios– ASes do not always advertise the address of their

router interfaces– Some interfaces are located at exchange points that

may be advertised by multiple AS– Non-BGP speaking network has multiple providers

• The first two are not relevant to our mechanism• For the third one:

– Either do not receive feedback or receive feedback through their BGP speaking providers

Long-term StateStatistics Reporting

• Each Checkpoints reads its short-term state every Ts seconds– Create feedback entries from its closed records– Packages them per source AD– Copy them to local storage– Send them to the corresponding feedback via UDP

• Lost packets can be recovered through the follow-up channel.

Follow-up Channel

• If receiver misses expected feedback or identifies a inconsistent feedback– Use follow-up interface of the involve ADs to

resolve the issue as described previously

Limitations

• There are two cases, in which providing per-TCP flow statistics is insufficient to characterize AD performance– Split TCP flows– Long, delay-sensitive TCP flows

TCP Delay Feedback

• A checkpoint considers a sequence of packets to be a single-path TCP flow if:– The packets belong to the same flow– The packets have the same entry and exit

points of this AD

TCP Delay Feedback

• This granularity allows the flow time between ADs, as well as the average packet time per flow to be measured

• Need 2 fields to accomplish this:– entryPoint (16)– entryTime (32)

Statistics Collection

• Although conceptually Simple, most ISPs routers aren't equipped to upload time and IP to a capability

• Need to find a less intensive solution

Statistics Reporting

• Workaround Solution:– Each router builds a table of source-

destination pairs on a central server, updated periodically

– Every Tm seconds, an entry point sends a marker packet to it's destinations

– Upon receiving, the end point timestamps and sends the marker to the accountability server

Limitations

• Multiple entry point flows– Exit points must make an educated guess at

the entry point using the lookup table– Cannot compute exact path time for each

packet, but can give the general performance of the flow

– Reordered packets may cause incorrect time data, may be alleviated by packet annotation at the cost of performance

Software Prototype

• Emulates Traffic modelling– Netflow-style Statistics

• Implements State processing– produces/sends feedback

• Two Checkpoints– 3.8 Ghz processors– Emulating 100-checkpoint topology

Results

• Single Processor handles 10Gbps link– Handles > 250k flows/sec– Assumes 5,000 byte flows

• 1 GB memory, 200 GB Storage– Handles 20 million concurrent flows– Long term state for 5h at 1 billion flows/hour

• <2% bandwidth overhead– Using 5,000 bytes per flow and 4 ADs

Further Discussion

• Feedback Tampering– Routers with deceiving timestamps may be a

problem in the future– Looking into combating it with Message

Authentification Codes, to find corrupt routers– Also looking into the ability to force hop-by-

hop feedback. Expensive, but useful for investigation.

Further Discussion

• Looking into implementing Flow Sampling as a first stage implementation

• Reflector Attacks and Spoofing:– Looking into an in depth way to combat

Reflector Attacks– Audit can trace the attack to the base AD

level, once identified.

Conclusion

• AudIt: loss/delay accountability– Sources learn per-flow loss/delay within each AD

• Accurate in the face of lies– Liars exposed to their peers– Deploying ISPs cannot be falsely accused

• Practical to deploy– Reasonable processing/memory req.– <2% bandwidth overhead

Thanks~

AnyAnyQuestion?Question?