Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2nd round)Richard Rouil, Nada Golmie, and David Griffith
National Institute of Standards and Technology
http://www.antd.nist.gov/seamlessandsecure.shtml
Outline
• MIH transport issues investigated
• Simulation scenarios
• Performance evaluation results– UDP without MIH ACK (for reference)– UDP with MIH ACK– TCP– SCTP
• Conclusions
MIH Transport Issues Investigated• Transport protocol type
– UDP– TCP
• PoS Location– RTT between the MIH nodes
• Message parameters– Size– Rate
• Network conditions– Link congestion
• Priority queuing• Retransmission timeout• Fragmentation• Congestion & rate control
Mobility scenariosVarying the RTT between the MN and the MoS
addresses all 4 scenarios identified in draft-xxx-mipshop-mstp-solution-00:– S1: Home Network MoS, the MN and the services are
located in the home network. – S2: Visited Network MoS, MN is in the visited network
and mobility services are also provided by the visited network.
– S3: Roaming MoS, MN is in the visited network and all services are provided by the home network.
– S4: Third party MoS, MN is in home or visited network and services are provided by a 3rd party network.
Network Topology
MN
AP(co-located PoS)
PoS (not co-located) CN
Variable link delay,
packet loss, and
bandwidth
Background traffic to CN for congestion
MIH message exchange to PoS
Simulation parametersIEEE 802.11
Data rate (Mb/s) 11Mb/s
Coverage area – radius (m) 50
Links
Speed (Mb/s) 100
Delay (s) 0.01
UDP
Max packet size (byte) 1000
Header size (bytes) 8
TCP
Maximum segment size (bytes) 1280
Min RTO (s) 0.2
Max retransmission Unlimited
Queue size Unlimited
Header size (bytes) 20
IP header
IPv6 header (bytes) 40
MIH Function
Transaction timeout (s) none
Maximum number of retransmission
2
Background traffic
Traffic type cbr over udp
Packet size (bytes) 500
Inter-arrival rate (s) 0.005
Simulation variables
Transport protocol UDP no ACK, UDP ACK, TCP
PoS Location Co-located, Remote
Variable link Delay (s) 0.01-0.5
Variable network packet loss (%) 0-50
MIH message Indication, Request/Response
MIH packet size (byte) 50-1000
MIH packet inter-arrival (s) Exp distribution w/ varying mean in [ 0.1 2]
MIH request processing time (s) 0-0.2
MIH message timeout (xRTT) (s) 0.5-2
Number of background traffic nodes 0-10
Rate limiting bucket size (packet) 1-20
Rate limiting token rate (packet/s) 1-20
Rate limiting queue size (packet) 10-50
Transport protocol performance evaluation
Performance metrics:– Transaction success rate (i.e. indication or
response received)– Delay to complete a transaction– Network load– Overhead created by the MIH
acknowledgement mechanisms and the transport layer
– Transport throughput
Impact of RTT
• Impact of the RTT on the transport of MIH messages:– Vary the RTT between 0.1s and 0.5s– For different packet loss (0%, 10%, 20%)
• The following transport protocol are used:– TCP– UDP without MIH ACK– UDP with MIH ACK
UDP (no ACK) Performance
Transaction delay for indications
• The transaction delay equals half the RTT regardless of the packet loss.
Transaction delay for requests
•The transaction delay equals the RTT regardless of the packet loss.
Transaction success rate for indications
•Success rate is equal to 1-p with p the packet loss in the network.
Transaction success rate for requests
•Success rate is equal to (1-p)2 with p the packet loss in the network.
MIH overhead
UDP (with ACK) Performance
Transaction delay for indications
•When there is no loss, the delays are identical to the UDP without retransmission.•Packet loss causes retransmission thus increasing the delays.
•Delay= )Re*2
)(1(2
0
n
ttn TxnR
pp
Transaction delay for requests
The results can be sectioned in three parts:•RTT < ReTx: In this case, retransmissions are only due to packet loss. As the packet loss increases, the delay increases.•ReTx < RTT < 2*ReTx: We observe that the transaction delays are converging. This reason is that the second retransmission is ineffective.•RTT > 2*ReTx: Regardless of packet loss, the MIH will timeout twice sending and both retransmissions are ineffective as the ACK would arrive after the timeout of the second retransmission (Transaction state is invalid, and response is ignored).
RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx
Closer look at retransmissionSender Receiver Sender Receiver Sender Receiver
Timeout 1
Timeout 2
Timeout 3Transaction
fails
Timeout 1
Timeout 2
Timeout 3Transaction
fails
Timeout 1
Timeout 2
Timeout 3Transaction
fails
RTT < timeout timeout < RTT < 2* timeout RTT > 2*timeout
All retransmissions are effective Second retransmission is ineffective.Two packets lost on the same transaction causes it to fail.
All retransmissions are ineffective. Any loss will cause the transaction to fail.
Transaction success rate for indications
•Success rate equals 1-p3 with p the packet loss in the network.
Transaction success rate for requests
The results can be sectioned in three parts:•RTT < ReTx: success rate equals (1-p3)2.•ReTx < RTT < 2*ReTx: success rate equals (1-p2)2. The second retransmission is ineffective.•RTT > 2*ReTx: success rate equals (1-p)2.. Both retransmissions are ineffective.
RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx
MIH overhead for indications
The results can be sectioned in three parts:•RTT < ReTx: The retransmissions are due to packet loss only. Overhead =
•ReTx < RTT < 2*ReTx: There is always one retransmission due to late ACK and some additional one due to packet loss. Overhead = 2(2-p) + p(2-p)2.•RTT > 2*ReTx: success rate equals (1-p)2. There are 2 retransmissions due to late ACK. As the packet loss increases, late responses are ignored and therefore no ACK is being generated. Overhead = 3 (2-p).
12
0
2*
n
n
n pp
RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx
MIH overhead for requests
The overhead for request/response is following a similar pattern as the overhead for indications.Note: the number of MIH Packets sent is divided by both the number of MIH requests and MIH responses sent.
RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx
UPD with MIH ACK summary
• The location of the PoS (i.e. RTT) impacts the performance of the MIH ACK mechanisms.
• If the retransmission timeout is not adequately set, the retransmission may become ineffective:– ReTx < RTT < 2* ReTx: the results are identical to an
MIH using one retransmission.– RTT > 2*ReTx: the results are similar to UDP without
MIH ACK.
TCP Performance
Transaction delays for indications and requests/responses
•TCP retransmits the packets using an exponential backoff leading to high retransmission delays.
Transaction success rate for indications and requests/responses
•TCP’s reliability allows for a success rate of 100%.
MIH overhead
•MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.
SCTP Performance
Network Topology
MN
AP IEEE 802.11(co-located PoS)
PoS (not co-located) CN
Variable link delay,
packet loss, and
bandwidth
Background traffic to CN for congestion
MIH message exchange to PoS
BSIEEE 802.16
Interface 2Interface 1
Added Access Network for multi-homing support
Second interface added for multi-homing support
Simulation parametersIEEE 802.16
Coverage area – radius (m) 500
Frame duration (s) 0.005
Modulation 64_QAM_3_4
Scheduling Best Effort
Links
Speed (Mb/s) 100
Delay (s) 0.01 except link to PoS interface 2 with delays of 0.05
SCTP
Max packet size (byte) 1500
Header size (bytes) 12
Initial RTO (s) 3
Min RTO (s) 1
Max RTO (s) 60
Number of stream 1
Delayed ACK Yes
SACK delay (s) 0.2
Simulation variablesTransport protocol SCTP with and without
retransmission using alternate path
Transaction delay for indications
•When multi-homing is not available, a packet loss increases the backoff time for retransmission and generates high delays. The behavior is similar to TCP.•Using multi-homing, retransmissions are done on an alternate path which may provide lower loss. This is can be favorable if the primary path is using an interface ready for handover.
Transaction delay for requests
Transaction success rate for indications
•SCTP provides reliable transport of MIH messages by retransmitting on the same interface or using an alternate when available.
Transaction success rate for requests
MIH overhead for indications
•MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.
MIH overhead for requests
Congestion control
• One of the backbone network link between the MN and PoS provides limited capacity (1Mb/s).
• MN sends messages at a rate between 50kb/s and 1500kb/s for 30s (defined as offered load).
• Packet sizes are 100 bytes and 500 bytes.• The following congestion control are used:
– TCP– UDP with no rate limiter– UDP with Token bucket (200 bucket size, 200
token/s, 200 queue size)– UDP with Token bucket (200 bucket size, 50 token/s,
200 queue size)
Transaction delay for Indications
•While there is no congestion (Offered load less than 1Mb/s), TCP uses all the available bandwidth. During congestion, TCP queues the packets and the delays to receive the indication increase. When the offered load increases, the size of the TCP segments increases to carry multiple MIH messages thus reducing the overhead.•When using UDP without rate limiter, the packets delays are low when there is no congestion. After an offered load of 600kb/s, delays appear due to the overhead generated by UDP (see throughput graph).•The rate limiter generates additional delays even for low offered loads. We observe that it acts later for larger MIH messages as it generates less packets for the same offered load. Furthermore, we can note that the indication is considered successful for indications even with high delays since the receiver is not aware of when it was sent. The sender on the other side may not receive the ACK on time, may retransmit the packet, and may consider the operation has failed.
Transaction delay for Request/Response
•Similar observation for TCP and UDP without rate limiter.•When using rate limiter, the delays incurred due to the queuing causes the responses to be ignored. Therefore only the initial messages succeeded with low transaction delay.
Transaction success rate for Indications
•TCP’s reliability allows for a success rate of 100%. •When UDP is used without rate control, an offered load of 600kb/s or more shows decreasing success rate. This is due to the packets being drop at the congested link in the network.•When control rate is used, the smaller the token rate, the lower the success rate. This is due to the queue limit of the token bucket implementation.
Transaction success rate for Request/Response
•Observation similar to indications but lesser success rate for UDP since delayed responses are ignored.
MIH overhead for Indications
•There is no MIH overhead created when using TCP since there is no retransmission at the MIH level.•When using UDP, there are 2 MIH packets corresponding to the Indication and the ACK for low offered load.•When no rate control is in place and the load is more than 600kb/s, increased in packet delay causes retransmissions, thus increasing the overhead.•The overhead is decreasing when rate control is in place due to packets being drop at the MIH level. The smaller the token rate, the faster the queue fills up and MIH message are dropped without trying to be sent.
MIH overhead for Request/Response
•We observe an increase of overhead before it decreases again. This is happens when the sending rate of MIH packets caused by retransmissions is close to the optimum sending rate allowed by the token configuration. After this rate is passed, the packets are queued and dropped without trying to be sent.
Network layer load for Indications
•The network load measures the traffic generated by the MN’s transport layer to send MIH packets.•TCP’s congestion control can be observed as the sending rate saturates close to 1Mb/s. •When UDP is not using rate control, all the retransmissions increase the network load. This leads to additional congestion in the network. In our scenario, it could saturate the wireless network if other nodes are present.•The rate control allows to limit the load though adequate values of token rate must be configured. We observe for example that in the case of packet size of 500 Bytes and token rate of 200, the rate control performs almost identically to TCP.
Network layer throughput for Indications
•The network throughput is the data rate measured by the receiver side, in this case the PoS.•For TCP and UDP using rate control, the throughput measured is identical to the load since the load is lesser than the available bandwidth on the congested link.•When no rate control is in place, the throughput saturates.
Summary of results on Congestion Control
• Congestion control is needed congestion spreads in the network due to a significant number of retransmissions.
• TCP’s embedded congestion control and packing provides less overhead for low congestion. When the link saturates, the exponential backoff mechanism and Head of Line situation leads to very high delays.
• The token bucket parameters must be adjusted to consider the congestion level and the average MIH packet size.