Overview of Mesh Networking Research @ MSR

Jitendra PadhyeMicrosoft Research

January 23, 2006

What are mesh networks?

• Multi-hop wireless networks

• Mostly static nodes

• Unplanned node placement

• Applications: Disaster relief, Backhaul for city-wide wireless networks, Meeting mesh, Neighborhood Meshes, internet connection sharing

• Many startups ….

Three main problems in mesh networking

• Capacity

• Capacity

• Capacity

Why is capacity a problem?

SourceMesh Router Destination

With a single radio, a node can not transmit and receive simultaneously.

A two-hop path has half the capacity of a one-hop path. Other interference patterns also possible.

Seminal Result by Gupta and Kumar (2000):Capacity = O(1/sqrt(n))

MSR’s research on Mesh Network Capacity

• Capacity estimation

• Capacity improvement using multiple radios and other techniques

• Feasibility study using realistic traffic

Mesh Network Capacity Estimation

• New framework for estimating capacity of multi-hop wireless networks– Gupta-Kumar result is asymptotic

– Our framework calculates optimal capacity of a given mesh network for given set of flows

MobiCom 2003 (Jain, Padhye, Padmanabhan and Qiu).

• Our framework requires knowledge of which links interfere with one another– Problem of “conflict graph” estimation

– N nodes O(N^2) links O(N^4) pairs!

– We developed an approximation technique that takes O(N^2) time IMC 2005 (Padhye, Agarwal, Padmanabhan, Qiu, Rao and Zill)

Key Insight: Multiple radios necessary to improve capacity

Improving capacity using Multiple Radios

• Select best radio to send each packet using locally available information– Multi-radio unification protocol

IEEE BroadNets 2004: Adya, Bahl, Padhye, Wolman and Zhou)

– Problem: sub-optimal in many cases

• Optimize entire path for a given flow– Take into account interference and link capacity along entire path

– Implemented in Mesh Connectivity Layer (MCL) MobiComm 2004: Padhye, Draves, Zill

• If second radio has very low bandwidth, can we use it to offload signaling?– Simulation-based study of separating control and data into different

frequency bands IEEE BroadNets 2005 (Kyasanur, Padhye, Bahl)

How do we know how much capacity is “enough”?

Feasibility study using realistic traffic

• Collect traffic traces from Microsoft’s wired network

• Replay on mesh testbed

• Study delay characteristics of replayed traffic

• Conclusions: – Factors such as specific card brands, placement of servers have

significant impact, routing metrics have less impact.

– 2-radio mesh network likely sufficient for supporting normal office traffic

– Some large delay spikes.

• MobiSys 2006 (Eriksson, Agarwal, Bahl, Padhye)

Ongoing work related to capacity:

• Capacity improvement using network coding

• Use of directional antennas to reduce interference

• Use of spectrum etiquettes and cognitive radios to improve spectrum utilization

Other challanges:

• Self-management– Network without administrator – is it possible?

– Engineering challenges such as automatic address assignment

• Security and Fairness– Freeloaders

– Information leakage by observing traffic

– Malicious nodes can disrupt routing

Backup slides

Mesh Connectivity Layer (MCL)Design & Implementation

Design ChoiceMulti-hop networking at layer 2.5

Framework– NDIS miniport – provides virtual adapter on virtual link– NDIS protocol – binds to physical adapters that provide next-hop

connectivity– Inserts a new L2.5 header

Why Layer 2.5?– Works over heterogeneous links (e.g. wireless, powerline)– Transparent to higher layer protocols.

• works equally well with IPv4 and IPv6– ARP etc. continue to work without any changes

Features– DSR-like routing with optimizations at virtual link layer

– Link Quality Source Routing (LQSR)– Incorporates 5 different link selection metrics:

– Hop count, RTT, Packet Pair, ETX, WCETT

Scope: Technical Problems we looked at

Range and Capacity– Off-the-shelf wireless hardware Is severely range limited – Throughput of 802.11 MAC degrades rapidly with the number of hopsOur Solution: multi-radio meshbox, directional ant., NLDP, Interference management, Capacity-cal

Routing– Network connectivity is highly dynamic– Classical single path & shortest path routing perform poorly in a dense network

Our Solution: LQSR & MR-LQSR, WCETT (ETX, PacketPair, RTT,..)

Security and Fairness– Mesh is susceptible to freeloaders and malicious users– Achieving “fairness” without topological and traffic information is difficult

Our Solution: “Windows certificate", greedy behavior detection, watchdog mechanism, intrusion detection

Self Management– End users are non-technical– A no-network operator model is challengingOur Solution: M3, watchdog mechanism, data cleaning, liar detection, on-line network simulation, beacon stuffing,

server placement

Spectrum Management– Tragedy of the commons– Exploit spectrum white space

Our Solution: Control channel, dual-frequency meshes, 700-900 MHz, Spectrum etiquettes

Impact of path length on throughput

Experimental Setup

• 23 node testbed

• One IEEE 802.11a radio per node (NetGear card)

• Randomly selected 100 sender-receiver pairs (out of 23x22 = 506)

• 3-minute TCP transfer, only one connection at a time

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 1 2 3 4 5 6

Byte-Averaged Path Length (Hops)

Th

rou

gh

pu

t (K

bp

s)

If a connection takes multiple paths over lifetime, lengths are byte-averaged

Total 506 points.Solution: Multi-Radio Meshes

Link Selection Metrics

Many metrics have been studied in literature– Hop count– Round trip time– Packet pair– Expected data transmission count incl. retransmission– Weighted cumulative expected transmission time– Signal strength stability– Energy related– Link error rate– Location related– …

The ones in red are implemented in MCL

Link Selection Metric for Single Radio: ETX

• Each node periodically broadcasts a probe

• The probe carries information about probes received from neighbors

• Each node can calculate loss rate on forward (Pf) and reverse (Pr) link to each neighbor

• Selects the path with least total ETX

Advantages– Explicitly takes loss rate into

account– Implicitly takes interference

between successive hops into account

– Low overhead

Disadvantages– PHY-layer loss rate of broadcast

probe packets is not the same as PHY-layer loss rate of data packets

Broadcast probe packets are smaller

Broadcast packets are sent at lower data rate

– Does not take data rate or link load into account)P(1*)P(1

1ETX

rf

Developed by De Couto et al @ MIT (2003)

Baseline comparison of Metrics Single Radio Mesh

Experimental Setup

• 23 node testbed

• One IEEE 802.11a radio per node (NetGear card)

• Randomly selected 100 sender-receiver pairs (out of 23x22 = 506)

• 3-minute TCP transfer, only one connection at a time

ETX performs the best

0

200

400

600

800

1000

1200

1400

1600

HOP ETX RTT PktPair

Med

ian

Th

rou

gh

pu

t (K

bp

s)

Median path length:HOP: 2, ETX: 3.01, RTT: 3.43, PktPair: 3.46

Link Selection Metric for Multiple Radios: WCETT

State-of-art metrics (shortest path, Packet Pair, RTT, ETX) do not leverage channel, range, data rate diversity

Multi-Radio Link Quality Source Routing (MR-LQSR)– Link metric: Expected Transmission Time (ETT)

Takes bandwidth and loss rate of the link into account

– Path metric: Weighted Cumulative ETTs (WCETT) Combine link ETTs of links along the path Takes channel diversity into account

– Incorporates into source routing

Developed by Draves, Padhye et al @ MSR(2004)

Expected Transmission Time (ETT)

Given:– Loss rate p– Bandwidth B– Mean packet size S– Min backoff window CWmin

Takes bandwidth and loss rate of the link into account

7i

0i

i1)(i

min

backoffxmit

backoffxmit

p21f(p)

p)2(1

f(p)CWET

p)B(1

SET

where,

ETETETT

WCETT = Combines link ETTs

Need to avoid unnecessarily long paths - bad for TCP performance - bad for global resources

All hops on a path on the same channel interfere– Add ETTs of hops that are on

the same channel

– Path throughput is dominated by the maximum of these sums

Given a n hop path, where each hop can be on any one of k channels, and two tuning parameters, a and b:

j channelon is i hopij

jkj1

n

1ii

ETTX

ba

Xmaxb*ETTa*WCETT

where

Select the path with min WCETT

Experimental Setup

• 23 node testbed

• Randomly selected 100 sender-receiver pairs (out of 23x22 = 506)

• 3-minute TCP transfer

• Two scenarios:– Baseline (Single radio):

802.11a NetGear cards

– Two radios 802.11a NetGear cards 802.11g Proxim cards

Median Throughput of 100 transfers

16011379

1155

2989.5

1508

844

0

500

1000

1500

2000

2500

3000

3500

WCETT ETX Shortest PathT

hro

ug

hp

ut

(Kb

ps)

Single Radio

Two Radios

WCETT utilizes 2nd radio betterthan ETX or shortest path

Baseline Comparison of Metrics Two Radio Mesh

Median path length:HOP: 2, ETX: 2.4, WCETT: 3

Path Length and ThroughputWhich metric is best?

Experimental Setup

• 23 node testbed

• Randomly selected 100 sender-receiver pairs (out of 23x22 = 506)

• 3-minute TCP transfer (transmit as many bytes as possible in 2 minutes, followed by 1 minute of silence)

0

500

1000

1500

2000

2500

3000

3500

4000

A C D E F

Testbed Configuration

Th

rou

gh

pu

t (K

bp

s)

WCETT ETX HOP

0

0.5

1

1.5

2

2.5

3

3.5

A C D E F

Testbed Configuration

Hop

Len

gth

WCETT ETX HOP

For 1 or 2 hop the choice of metric doesn’t matter

Comparison of MetricsWireless Office Scenario

4 4 3 3

89120

82

474

116 5

8

2

179

6

1

10

100

1000

10000

WCETT ETX HOP PKTPAIR RTT

Ad

dit

ion

al D

ela

y (

ms)

4 3 3

862 943

27 31 30

590

1

10

100

1000

10000

WCETT ETX HOP PKTPAIR RTT

Ad

dit

ion

al D

ela

y (

ms)

23 node indoor testbed. Two radios (both 802.11a) per node. 11 active clients, 4 servers.

Heavy Office Traffic1 hour, 308 sessions, 587.5 MB total

Light Office Traffic1 hour, 415 sessions, 19.72 MB total

Relatively light traffic means performance is okay for all metrics. WCETT does better under heavy load (worst case delay)

Management: Resiliency against Liars/Lossy Links

Problem• Identify nodes that report incorrect

information (liars)• Detect lossy links

Assume• Nodes monitor neighboring traffic, build

traffic reports and periodically share info.• Most nodes provide reliable information

Challenge Wireless links are error prone and unstable

Approach• Watchdogs• Find the smallest number of lying nodes to

explain inconsistency in traffic reports• Use the consistent information to estimate

link loss rates

Detect liars

0

0.2

0.4

0.6

0.8

1

NL=1 NL=2 NL=5 NL=8 NL=10 NL=15 NL=20

Fra

ctio

n o

f ly

ing

no

des

id

enti

fied

coverage false positive

Detect lossy links

0

0.2

0.4

0.6

0.8

1

NL=1 NL=2 NL=5 NL=8 NL=10 NL=15 NL=20

Fra

ctio

n o

f lo

ssy

links

id

enti

fied

coverage false positive

Simulation Results