Network Tomography (A presentation for STAT 593E) Mingyan Li Radha Sampigethaya

Network Tomography

(A presentation for STAT 593E)

Mingyan Li

Radha Sampigethaya

Outline of Talk

Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

Introduction to Networks

Network is a set of interconnected hosts (PC, server, router) Internet is a network of networks Each host is modeled as functional layers for the purpose of

networking Each host is identified by an IP address (Ex: 128.95.196.98) TCP is the reliable communication protocol for networking;

UDP is non-reliable Internet traffic unit is assumed to be packets

NOTE: TCP – Transmission Control Protocol; IP – Internet Protocol UDP – User Datagram Protocol

Application(Email, HTTP)

Transport(TCP/UDP)

Network(IP)

NetworkInterface

(NIC, modem)

Simplified host layer model

Introduction to Networks (2) TCP is most common on the Internet and provides reliable end-to-end

(application-to-application) communication

Automatic Repeat Request (ARQ): - Packets received are acknowledged by receiver - Packets are retransmitted by sender if:

1. Packets are received but with errors, at the receiver2. ACK’s are not received by the sender indicating loss or

delay of packets- TCP retransmits after waiting for the ACK for a deterministic time

(round-trip-time) Flow Control mechanisms Congestion Control mechanisms (due to finite network bandwidth)

Unicast communication? One user sends data to another user

Multicast communication? One user sends data to many users

Introduction to Networks (3)

IP (Internet Protocol) layer Each host identified by IP address (XXX.XXX.XXX.XXX)

IP layer routes packets from source towards destination (using routing algorithm)

Maintains Routing table: list of next-hop nodes (static or dynamic list)

- Destination IP address (Net ID portion) determines the next-hop node chosen from table

0

1 1 1 0

Net id

Multicast address

Host id

0 31

Class A

Class D

For Internet Multicast: 224.0.0.0 through 239.255.255.255

1 8


Internet Router or Internet Gateway Interconnects 2 networks and passes packets from one

to other

Router has an IP address, routing table, and handles traffic coming in and going out of network

For multicast communication, router needs to be specifically enabled

network1 network2


Multicast Communication- One sender and multiple receivers

Internet Applications: Video conferencing, internet gaming IP Multicast group

- Each group has an IP Address- Hosts need to notify local routers about the multicast group they belong to, and routers will update tables- A host may belong to more than one multicast group!- Routers will forward multicast group packets to appropriate next-hop nodes (refers to routing table) - Any host can send packets to the multicast group by sending to the group IP address - Only members of multicast group receive the packets

Multicast communication is normally best-effort; uses UDP


Internet

ISP

A

C

D

BUnicast Example(Sender-Receiver)

A is senderB, C, D are receivers of the same messageA, B,C,D belong to same multicast group

Link Route to:LINK1 128.95.X.XLINK2 DefaultRouting table for Router 1

128.95.X.X

LINK1

LINK2

Introduction to Networks (7)Multicast Example

(One Sender-Multiple Receiver)

Internet

ISP

A

C

D

E

B

Multicast group:A is senderB, C, D, E are receivers of same message

Link Route to:LINK1, LINK2 multicast group IPLINK1 128.95.X.XLINK2 Default Routing table for Router 1

Introduction to Networks (8) Multicast Routing Tree

Constructed by multicast routing algorithms Rooted at the source, with the receivers of the multicast

group at the leaves Intermediate tree nodes are routers which forward packets Links constitute edges of the tree A physical topology (tree) would consist of all the nodes and

links encountered in the multicast communication

• A logical topology (tree) would consist of subsets of links and nodes (but all the receivers and the sender)Link(s)

1

Router1E

D C B

Source A

A Logical Multicast Topology

ISP Router Router 2

B,C,D,E are receivers

Where are we?


Network Monitoring Background

Networks: set of nodes, links- delays, losses affect performance

Networks normally are interconnected (not isolated)

- hence interdependent for performance

Network monitoring and management- involves collection of network performance

statistics (link delay, link loss, traffic rate) - easier for isolated network compared to inter-network (such as Internet)

- Challenges for Inter-network monitoring and management:

Increased overhead cost, complexity, confidentiality of company network statistics

1

2

34

5 67

8

Link or group of links

Node or network

A Logical Network

Network Monitoring Background contd….

Inter-network monitoring requires:Timely, accurate, localized measurements of performance metrics without any special cooperation amongst the different networks

Measurement Methods: 1. Active (sending probe packets) Adds to normal data traffic2. Passive (traffic analysis) Temporal and spatial dependence might bias measurement

Performance metrics: - Link-level: loss rate, delay; Path-level: traffic matrix- May not be directly measurable- Metrics are then inferred from other easily monitored

network statistics (count of packets, time delay between received packets)

Where are we?


Network Tomography

Inferential network monitoring and does not require any special cooperation between networks

Two forms of network tomography: - link-level metric estimation based on end-to-end, traffic measurements (counts of sent/received packets, time delays between sent/received packets)

- path-level (sender-receiver path) traffic intensity estimation based on link-level traffic measurements (counts of packets through nodes)

Using end-to-end measurements we can infer a network’s link delay/loss statistics, and also the network connectivity or topology

Where are we?


Problem Addressed Logical Multicast Topology Inference using end-to-

end point measurements We consider the problem of determining the

connectivity structure or topology of a network and relate this to the problem of hierarchical clustering.

Active measurement method (multicast probes) is used

Performance statistic measured is loss rate

Problem Addressed (2)Multicast Topology Inference

Given • Sender: probes• Receiver: traces (loss, delay)

Goal: Identify multicast tree topology(Interconnectivity from sender to receivers)

Source

Receivers

= orRouters

Problem Addressed (3)Motivation

Topology: first step to infer other characteristics: link delay, link loss, and this knowledge aids multicast application design

Reliable multicast has been shown to benefit from the knowledge of underlying multicast topology, and it helps scalable local recovery.

MTRACE: is a tool that requires cooperation from routers.

Problem Addressed (4) General Approach to Multicast Topology Inference Given end-to-end measurements: loss, delay Exploit correlation in measurement to group nodes

High correlation, high probability of sharing a parent node

Find correlation function Increases along the path from root to leaves Can be estimated from measurements at leaves

Example Prob. of probe loss Delay

Build topology by recursively grouping nodes to maximize correlation function

Clustering Solutions Loss Based Topology Inference

Why loss based? Delay based required synchronization

Topology inference procedure (1) Multicast probes, record receiver loss (2) Form groups of one receiver initially (3) Merge the 2 groups that have the highest

correlation in loss until there is one hierarchy group

101

110

110

100

Clustering Solutions (2)Multicast Topology Inference Example

Real Topology Inferred Topology

a b c d e f g

2%

2%

3% 2%2%

2%1%

5%

3%

3%

1% 1%

Clustering Solutions (3)Improvement in approach

Approach shown builds only binary treeIn practice, routers may have more

than 2 childrenConsider:

Real: Inferred:

Clustering Solutions (4)Loss Rate Inference

A sender, 2 receivers: a & b Probe lost at link 1 won’t reach a &b Probe lost at link 2 (3) won’t reach a (b)

Define Pab : Prob{ both a & b lose the probe} Pa : Prob{ a loses the probe but not b} Pb : Prob{ b loses the probe but not a} P1 : Prob {probe loss seen by parent node of a&b}

shared loss rate of a & b P2 : loss rate of link 2 P3 : loss rate of link 3

From Pab, Pa, Pb, compute P1, P2, P3

p2

p1

p3

a bpa pb pab

Shared loss rate

Link 1

Link 2 Link 3

Clustering Solutions (5)Loss Rate Inference (Cont.)

Pab = P1 + (1-P1) P2P3

Pa = (1- P1 ) P2 (1- P3 ) Pb = (1- P1 ) (1- P2 ) P3

Solve P2 = Pb / (1- Pab - Pa) P3 = Pa / (1- Pab - Pb) P1 = 1-Pa / (P2(1-P3))

P2

p1

p3

a bpa pb

pab

Link 1

Clustering Solutions (6)Multicast Topology Inference Example

Real Topology Inferred Topology

a b c d e f g

2%

2%

3% 2%2%

2%1%

5%

3%

3%

1% 1%

a b c d e f g

3.1%

1.9%

2.9%

1.9%

2.0%1.8% 5.0%

2.9%

3.7%

1.0% 1.0%

0.1%

Clustering Solutions (7)Multicast Topology Inference Approaches

Two approaches Binary Loss Tree Pruning Algorithm (BLTP)

• Infer binary tree • Estimate link loss rate • Prune the link with loss rate < α (threshold)

General Loss Tree Algorithm (GLT)• Use P1 (shared loss rate) as correlation function, • group nodes to maximize P1, and merge nodes

based on inferred link loss rate

Clustering Solutions (8)Comparison of BLTP & GLT

Both proved to converge to real topology with prob. 1 for sufficient large # of probes

Performance (in terms of accuracy vs. # of probes) are similar

Threshold (α) applied only at the end in BLTP : facilitate adaptive selection of (α)

Clustering Solutions (9)Limitations of the approaches

Only Infer logical (not physical) topology

Only identify links with significant loss rates ( 1%)

Only identify routers with more than one child

Clustering Solutions (10)Other Approaches

Maximum likelihood clusteringSlow convergence, poor performance

Bayesian clusteringOptimal provided topology drawn from

prior distributionMuch more computationally complex

Clustering Solutions (11)Comparison of BLTP & Bayesian

Bayesian can identify links with arbitrarily small loss rates

BLTP (and GLT) require a parameter (α) as a threshold to account for statistical fluctuations

In practice, identification of high loss links is more important and selection of (α) is application-specific.

Where are we?


Conclusions

BLTP (Binary-based clustering with pruning) offers the best combination of accuracy and computational simplicity

A challenging problem remains to combine the estimated logical topology and other tools (mtrace) to infer physical network topology

References

S. Ratnasamy and S. McCanne “Inference of multicast routing trees and bottelneck bandwidths using end-to-end measurements,” INFOCOM’99

N.G.Duffield, J.Horowitz, F.Lo Presti, D. Towsley, “Multicast Topology Inference from Measured end-to-end Loss,” IEEE Info. Theory, 2002

Documents

Network Tomography (A presentation for STAT 593E) Mingyan Li Radha Sampigethaya