Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2 nd round)

Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2nd round)Richard Rouil, Nada Golmie, and David Griffith

National Institute of Standards and Technology

http://www.antd.nist.gov/seamlessandsecure.shtml

Outline

• MIH transport issues investigated

• Simulation scenarios

• Performance evaluation results– UDP without MIH ACK (for reference)– UDP with MIH ACK– TCP– SCTP

• Conclusions

MIH Transport Issues Investigated• Transport protocol type

– UDP– TCP

• PoS Location– RTT between the MIH nodes

• Message parameters– Size– Rate

• Network conditions– Link congestion

• Priority queuing• Retransmission timeout• Fragmentation• Congestion & rate control

Mobility scenariosVarying the RTT between the MN and the MoS

addresses all 4 scenarios identified in draft-xxx-mipshop-mstp-solution-00:– S1: Home Network MoS, the MN and the services are

located in the home network. – S2: Visited Network MoS, MN is in the visited network

and mobility services are also provided by the visited network.

– S3: Roaming MoS, MN is in the visited network and all services are provided by the home network.

– S4: Third party MoS, MN is in home or visited network and services are provided by a 3rd party network.

Network Topology

MN

AP(co-located PoS)

PoS (not co-located) CN

Variable link delay,

packet loss, and

bandwidth

Background traffic to CN for congestion

MIH message exchange to PoS

Simulation parametersIEEE 802.11

Data rate (Mb/s) 11Mb/s

Coverage area – radius (m) 50

Links

Speed (Mb/s) 100

Delay (s) 0.01

UDP

Max packet size (byte) 1000

Header size (bytes) 8

TCP

Maximum segment size (bytes) 1280

Min RTO (s) 0.2

Max retransmission Unlimited

Queue size Unlimited


IP header

IPv6 header (bytes) 40

MIH Function

Transaction timeout (s) none

Maximum number of retransmission

2

Background traffic

Traffic type cbr over udp

Packet size (bytes) 500

Inter-arrival rate (s) 0.005

Simulation variables

Transport protocol UDP no ACK, UDP ACK, TCP

PoS Location Co-located, Remote

Variable link Delay (s) 0.01-0.5

Variable network packet loss (%) 0-50

MIH message Indication, Request/Response

MIH packet size (byte) 50-1000

MIH packet inter-arrival (s) Exp distribution w/ varying mean in [ 0.1 2]

MIH request processing time (s) 0-0.2

MIH message timeout (xRTT) (s) 0.5-2

Number of background traffic nodes 0-10

Rate limiting bucket size (packet) 1-20

Rate limiting token rate (packet/s) 1-20

Rate limiting queue size (packet) 10-50

Transport protocol performance evaluation

Performance metrics:– Transaction success rate (i.e. indication or

response received)– Delay to complete a transaction– Network load– Overhead created by the MIH

acknowledgement mechanisms and the transport layer

– Transport throughput

Impact of RTT

• Impact of the RTT on the transport of MIH messages:– Vary the RTT between 0.1s and 0.5s– For different packet loss (0%, 10%, 20%)

• The following transport protocol are used:– TCP– UDP without MIH ACK– UDP with MIH ACK

UDP (no ACK) Performance

Transaction delay for indications

• The transaction delay equals half the RTT regardless of the packet loss.

Transaction delay for requests

•The transaction delay equals the RTT regardless of the packet loss.

Transaction success rate for indications

•Success rate is equal to 1-p with p the packet loss in the network.

Transaction success rate for requests

•Success rate is equal to (1-p)2 with p the packet loss in the network.

MIH overhead

UDP (with ACK) Performance


•When there is no loss, the delays are identical to the UDP without retransmission.•Packet loss causes retransmission thus increasing the delays.

•Delay= )Re*2

)(1(2

0

n

ttn TxnR

pp


The results can be sectioned in three parts:•RTT < ReTx: In this case, retransmissions are only due to packet loss. As the packet loss increases, the delay increases.•ReTx < RTT < 2*ReTx: We observe that the transaction delays are converging. This reason is that the second retransmission is ineffective.•RTT > 2*ReTx: Regardless of packet loss, the MIH will timeout twice sending and both retransmissions are ineffective as the ACK would arrive after the timeout of the second retransmission (Transaction state is invalid, and response is ignored).

RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx

Closer look at retransmissionSender Receiver Sender Receiver Sender Receiver

Timeout 1

Timeout 2

Timeout 3Transaction

fails

Timeout 1

Timeout 2


fails

Timeout 1

Timeout 2


fails

RTT < timeout timeout < RTT < 2* timeout RTT > 2*timeout

All retransmissions are effective Second retransmission is ineffective.Two packets lost on the same transaction causes it to fail.

All retransmissions are ineffective. Any loss will cause the transaction to fail.


•Success rate equals 1-p3 with p the packet loss in the network.


The results can be sectioned in three parts:•RTT < ReTx: success rate equals (1-p3)2.•ReTx < RTT < 2*ReTx: success rate equals (1-p2)2. The second retransmission is ineffective.•RTT > 2*ReTx: success rate equals (1-p)2.. Both retransmissions are ineffective.


MIH overhead for indications

The results can be sectioned in three parts:•RTT < ReTx: The retransmissions are due to packet loss only. Overhead =

•ReTx < RTT < 2*ReTx: There is always one retransmission due to late ACK and some additional one due to packet loss. Overhead = 2(2-p) + p(2-p)2.•RTT > 2*ReTx: success rate equals (1-p)2. There are 2 retransmissions due to late ACK. As the packet loss increases, late responses are ignored and therefore no ACK is being generated. Overhead = 3 (2-p).

12

0

2*

n

n

n pp


MIH overhead for requests

The overhead for request/response is following a similar pattern as the overhead for indications.Note: the number of MIH Packets sent is divided by both the number of MIH requests and MIH responses sent.


UPD with MIH ACK summary

• The location of the PoS (i.e. RTT) impacts the performance of the MIH ACK mechanisms.

• If the retransmission timeout is not adequately set, the retransmission may become ineffective:– ReTx < RTT < 2* ReTx: the results are identical to an

MIH using one retransmission.– RTT > 2*ReTx: the results are similar to UDP without

MIH ACK.

TCP Performance

Transaction delays for indications and requests/responses

•TCP retransmits the packets using an exponential backoff leading to high retransmission delays.

Transaction success rate for indications and requests/responses

•TCP’s reliability allows for a success rate of 100%.

MIH overhead

•MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

SCTP Performance

Network Topology

MN

AP IEEE 802.11(co-located PoS)

PoS (not co-located) CN

Variable link delay,

packet loss, and

bandwidth

Background traffic to CN for congestion

MIH message exchange to PoS

BSIEEE 802.16

Interface 2Interface 1

Added Access Network for multi-homing support

Second interface added for multi-homing support

Simulation parametersIEEE 802.16

Coverage area – radius (m) 500

Frame duration (s) 0.005

Modulation 64_QAM_3_4

Scheduling Best Effort

Links

Speed (Mb/s) 100

Delay (s) 0.01 except link to PoS interface 2 with delays of 0.05

SCTP

Max packet size (byte) 1500


Initial RTO (s) 3

Min RTO (s) 1

Max RTO (s) 60

Number of stream 1

Delayed ACK Yes

SACK delay (s) 0.2

Simulation variablesTransport protocol SCTP with and without

retransmission using alternate path


•When multi-homing is not available, a packet loss increases the backoff time for retransmission and generates high delays. The behavior is similar to TCP.•Using multi-homing, retransmissions are done on an alternate path which may provide lower loss. This is can be favorable if the primary path is using an interface ready for handover.



•SCTP provides reliable transport of MIH messages by retransmitting on the same interface or using an alternate when available.


MIH overhead for indications

•MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

MIH overhead for requests

Congestion control

• One of the backbone network link between the MN and PoS provides limited capacity (1Mb/s).

• MN sends messages at a rate between 50kb/s and 1500kb/s for 30s (defined as offered load).

• Packet sizes are 100 bytes and 500 bytes.• The following congestion control are used:

– TCP– UDP with no rate limiter– UDP with Token bucket (200 bucket size, 200

token/s, 200 queue size)– UDP with Token bucket (200 bucket size, 50 token/s,

200 queue size)

Transaction delay for Indications

•While there is no congestion (Offered load less than 1Mb/s), TCP uses all the available bandwidth. During congestion, TCP queues the packets and the delays to receive the indication increase. When the offered load increases, the size of the TCP segments increases to carry multiple MIH messages thus reducing the overhead.•When using UDP without rate limiter, the packets delays are low when there is no congestion. After an offered load of 600kb/s, delays appear due to the overhead generated by UDP (see throughput graph).•The rate limiter generates additional delays even for low offered loads. We observe that it acts later for larger MIH messages as it generates less packets for the same offered load. Furthermore, we can note that the indication is considered successful for indications even with high delays since the receiver is not aware of when it was sent. The sender on the other side may not receive the ACK on time, may retransmit the packet, and may consider the operation has failed.

Transaction delay for Request/Response

•Similar observation for TCP and UDP without rate limiter.•When using rate limiter, the delays incurred due to the queuing causes the responses to be ignored. Therefore only the initial messages succeeded with low transaction delay.

Transaction success rate for Indications

•TCP’s reliability allows for a success rate of 100%. •When UDP is used without rate control, an offered load of 600kb/s or more shows decreasing success rate. This is due to the packets being drop at the congested link in the network.•When control rate is used, the smaller the token rate, the lower the success rate. This is due to the queue limit of the token bucket implementation.

Transaction success rate for Request/Response

•Observation similar to indications but lesser success rate for UDP since delayed responses are ignored.

MIH overhead for Indications

•There is no MIH overhead created when using TCP since there is no retransmission at the MIH level.•When using UDP, there are 2 MIH packets corresponding to the Indication and the ACK for low offered load.•When no rate control is in place and the load is more than 600kb/s, increased in packet delay causes retransmissions, thus increasing the overhead.•The overhead is decreasing when rate control is in place due to packets being drop at the MIH level. The smaller the token rate, the faster the queue fills up and MIH message are dropped without trying to be sent.

MIH overhead for Request/Response

•We observe an increase of overhead before it decreases again. This is happens when the sending rate of MIH packets caused by retransmissions is close to the optimum sending rate allowed by the token configuration. After this rate is passed, the packets are queued and dropped without trying to be sent.

Network layer load for Indications

•The network load measures the traffic generated by the MN’s transport layer to send MIH packets.•TCP’s congestion control can be observed as the sending rate saturates close to 1Mb/s. •When UDP is not using rate control, all the retransmissions increase the network load. This leads to additional congestion in the network. In our scenario, it could saturate the wireless network if other nodes are present.•The rate control allows to limit the load though adequate values of token rate must be configured. We observe for example that in the case of packet size of 500 Bytes and token rate of 200, the rate control performs almost identically to TCP.

Network layer throughput for Indications

•The network throughput is the data rate measured by the receiver side, in this case the PoS.•For TCP and UDP using rate control, the throughput measured is identical to the load since the load is lesser than the available bandwidth on the congested link.•When no rate control is in place, the throughput saturates.

Summary of results on Congestion Control

• Congestion control is needed congestion spreads in the network due to a significant number of retransmissions.

• TCP’s embedded congestion control and packing provides less overhead for low congestion. When the link saturates, the exponential backoff mechanism and Head of Line situation leads to very high delays.

• The token bucket parameters must be adjusted to consider the congestion level and the average MIH packet size.

Documents

Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2 nd round)