Upload
trinhdieu
View
216
Download
0
Embed Size (px)
Citation preview
© James P.G. SterbenzITTCHigh-Performance Networking
The University of Kansas EECS 881End-to-End Transport
© 2004–2010 James P.G. Sterbenz18 November 2010
James P.G. Sterbenz
Department of Electrical Engineering & Computer ScienceInformation Technology & Telecommunications Research Center
The University of Kansas
http://www.ittc.ku.edu/~jpgs/courses/hsnets
rev. 10.0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-2
© James P.G. SterbenzITTC
End-to-End TransportOutline
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-3
© James P.G. SterbenzITTC
End-to-End TransportOutline
TL.1. Functions & mechanismsTL.2. State managementTL.3. Framing and multiplexing
TL.4. Error controlTL.5. Transmission controlTL.6. TCP optimisations
network
application
session
transport
network
link
end system
network
link
node
network
link
nodenetwork
link
node
application
session
transport
network
link
end system
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-4
© James P.G. SterbenzITTC
Transport LayerService and Interfaces1
• Transport layer (L4) service to application layer (L7)• Transfer PDU E2E (end-to-end)
– sender: encapsulate ADU into TPDU and transmit– receiver: receive TPDU and decapsulate into ADU
• Multiplexing to application on end system• Optional reliability:
– error checking/correction/retransmission– need depends on application– may be done application-to-application (A2A)
• Flow control between end systems– assistance for network congestion control
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-5
© James P.G. SterbenzITTC
Transport LayerService and Interfaces2
• Transport layer uses service of network layer (L3)– network layer establishes a path through the network
• Transport mechanisms depend on L3 service– datagrams vs. connections– best effort vs. QOS– error characteristics– strength and symmetry of connectivity
What are the Internet assumptions & service model?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-6
© James P.G. SterbenzITTC
Transport LayerTypes of Transport Protocols
• Spectrum of transport protocol specialisation– general purpose– functionally partitioned– application-oriented– special purpose
• Software engineering and implementation choice
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-7
© James P.G. SterbenzITTC
Transport LayerService and Interfaces
• Transport PDU encapsulates application data unit– ADU – application data unit– TPDU – transport protocol data unit
• TPDU = header + ADU + optional trailer (protocol dependent)
application layer
network layer
transport layer
application layer
network layer
transport layer
ADU
ADU TH
ADU
NPDU TH
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-8
© James P.G. SterbenzITTC
Transport LayerFunctional Placement
• Transport layer functionality only in end system– host software or– network interface (NIC)
switch
TPDUs
end system
network interface
mem CPU
frames
end system
network interface
memCPU
framespacket
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-9
© James P.G. SterbenzITTC
Transport LayerEnd-to-End vs. Hop-by-Hop
• Transport layer is E2E analog of HBH link layer– link layer (L2) transfers packets HBH– network layer (L3) determines path of concatenated links
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-10
© James P.G. SterbenzITTC
Ideal End-to-End Network ModelBandwidth and Delay
• Zero end-to-end delay• Unlimited end-to-end bandwidth
M app
M app
D = 0
R = ∞
ES1
CPU
ES2
CPU
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-11
© James P.G. SterbenzITTC
End-to-End TransportTL.1 Functions and Mechanisms
TL.1 Functions and mechanismsTL.1.1 End-to-end semanticsTL.1.2 End-to-end mechanisms
TL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-12
© James P.G. SterbenzITTC
E2E Functions and MechanismsEnd-to-End Semantics
• Hop-by-hop functions– link layer protocols– link compression and FEC– network forwarding– link / subnet error control– embedded protocols
(e.g. protocol boosters)
• End-to-end functions– transport protocols– source routing– end-to-end encryption– session protocols– application protocols
Hop-by-Hop
End-to-End
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-13
© James P.G. SterbenzITTC
E2E Functions and Mechanisms End-to-End Argument
• Hop-by-hop function composition ≠ end-to-end• Examples
– HBH encryption has data in clear inside network nodes– HBH link error control doesn’t cover network layer errors
End-to-End Argument T-3
Functions required by communicating applications can be correctly and completely implemented only with the knowledge and help of the applications themselves. Providing these functions as features within the network itself is not possible.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-14
© James P.G. SterbenzITTC
E2E Functions and Mechanisms Hop-by-Hop Performance Enhancement
• E2E functions replicated HBH if performance benefit• Example
– per link error control in high bandwidth-×-delay networksreduce E2E control loop when error
Hop-by-Hop Performance Enhancement Corollary T-3A
It is beneficial to duplicate an end-to-end function hop-by-hop if the result is an overall (end-to-end) improvement in performance.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-15
© James P.G. SterbenzITTC
End-to-End vs. Hop-by-HopPerformance Enhancement Example
• E2E vs. HBH error control for reliable communication– E2E argument says error control must be done E2E
• e.g. E2E ARQ (error check code and retransmit if necessary– but should HBH error control also be done?
100 mwireless LANUniv. Kansas
100 mfiber LAN
Univ. Sydney
15 000 kmfiber WAN
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-16
© James P.G. SterbenzITTC
End-to-End vs. Hop-by-HopPerformance Enhancement Example
• E2E vs. HBH error control for reliable communication– E2E argument says error control must be done E2E
• e.g. E2E ARQ (error check code and retransmit if necessary– but should HBH error control also be done?
• Effect of high loss rate on wireless link– ~250 ms RTT retransmission for every corrupted packet
• Error control on wireless link reduces to ~1μs– shorter control loop results in dramatically lower E2E delay
100 mwireless LANUniv. Kansas
100 mfiber LAN
Univ. Sydney
15 000 kmfiber WAN
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-17
© James P.G. SterbenzITTC
E2E Functions and Mechanisms E2E Argument Misinterpretations
• E2E-only– do not replicate E2E services or features HBH– violated HBH performance enhancement corollary
• Everything E2E– implement as many services or feature E2E as possible– misstatement of Internet design philosophy:
simple stateless network for resilience and survivability
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-18
© James P.G. SterbenzITTC
E2E Functions and MechanismsMeaning of “In the Network”
• Functional (general use in this tutorial)– layers 1 – 3
• physical• MAC• link• network
• Topological– colocated with network nodes
• Administrative– owned by network service provider
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-19
© James P.G. SterbenzITTC
End-to-End TransportTL.1.2 End-to-End Mechanisms
TL.1 Functions and mechanismsTL.1.1 End-to-end semanticsTL.1.2 End-to-end mechanisms
TL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-20
© James P.G. SterbenzITTC
E2E Functions and MechanismsEnd-to-End Mechanisms
• Framing• Multiplexing• End-to-end state and connection management• Error control• Flow control• Congestion control
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-21
© James P.G. SterbenzITTC
E2E Functions and MechanismsTransport Protocol Service Categories
• Transfer mode– connectionless datagram– connection-oriented– transaction– continuous media streaming
• Reliability– loss tolerance
• Delivery order– ordered– unordered
• Traffic characteristics
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-22
© James P.G. SterbenzITTC
E2E Functions and MechanismsTypes of Transport Protocols
• Spectrum of transport protocol specialisation– general purpose– functionally partitioned– application-oriented– special purpose
• Software engineering and implementation choiceTransport Protocol Functionality T-IV
Transport protocols must be organised to deliver the set of end-to-end high-bandwidth, low latency services needed by applications. Options and service models should be modularly accessible, without unintended performance degradation and feature interactions.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-23
© James P.G. SterbenzITTC
E2E Functions and MechanismsExample7.2 End-to-End Internet Protocols
• Transport protocolsProtocol Name Function Status Ref
TCP transmission control protocol
reliable data transfer with congestion control
socket access to unreliable IP datagrams
file and document transfer(typically over UDP)
remote login
reliable data transfer with no congestion control
signallingproposed for wireless
RFC 793STD 007
UDP user datagram protocol
standard
standard
standards track
experimental
experimental
RFC 768STD 006
RTP real-time protocol RFC 1889
T/TCP TCP for transactions RFC 1644
RDP reliable data protocol RFC 908
proposed standardSCTP stream control
transmission protocol RFC 2960
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-24
© James P.G. SterbenzITTC
E2E Functions and MechanismsExample7.2 End-to-End Internet Protocols
“application layer” only because they run over TCP or UDP
Protocol Name Function/use
HTTP hypertext transfer protocol Web browsing
SMTP simple mail transfer protocol email relay and delivery
FTP file transfer protocol file and document transfer
Telnet telnet remote login
NFS network file system remote access to files
RTSP real-time streaming protocol control of multimedia streaming
• “Application layer” protocols
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-25
© James P.G. SterbenzITTC
E2E Functions and MechanismsControl of State
• Open loop– control parameters– no feedback
• Closed loop– initial parameters– feedback
• dynamic adaptation
• Closed loopwith HBH feedback
• Nested control loops
Open loop
parameters
data
Nested control loops
initial parameters
data
E2E feedback
receiversender
Closed loop with HBH feedback
HBH feedback
initial parameters
data
E2E feedback
Closed loop
initial parameters
data
E2E feedback
HBH loop
HBH loop
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-26
© James P.G. SterbenzITTC
E2E Functions and MechanismsControl of State
Open- vs. Closed-loop Control T-6D
Use open-loop control based on knowledge of the network path to reduce the delay in closed loop convergence. Use closed-loop control to react to dynamic network and application behaviour; intermediate per hop feedback can sometimes reduce the time to converge.
• Open-loop control– use when possible to eliminate control loop latency
• particularly for high bandwidth-×-delay product networks
• Closed-loop feedback control– use when necessary, e.g. reliability
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-27
© James P.G. SterbenzITTC
E2E Functions and MechanismsControl of State
Aggressiveness of Closed-Loop Control T-6Da
The response to closed-loop feedback must be rapid enough so the system converges quickly to a new stable state – but not so aggressive that oscillations occur due to overreaction to transients.
• Aggressiveness of closed-loop control– rapid enough to system converges– not too aggressive avoid instability and oscillations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-28
© James P.G. SterbenzITTC
End-to-End TransportTL.2 State Management
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-29
© James P.G. SterbenzITTC
End-to-End ProtocolsState Management
7.1 Functions and mechanisms7.2 State management
7.2.1 Impact of high speed7.2.2 Transfer modes7.2.3 State establishment and maintenance7.2.4 Assumed initial conditions
7.3 Framing and multiplexing7.4 Error control7.5 Flow and congestion control7.6 Security and information assurance
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-30
© James P.G. SterbenzITTC
State ManagementTransport and Network Connections
• L4 connections– required over
• connectionless L3• path with any
connectionlesssubnetwork
– may exploit• connection-oriented L3
SETUP CONNECT
network layer
end system
transport layer
connection-orientednetwork
network layer
end system
transport layer
setup-req
network layer connection
connectionless network
network layer
transport layer SETUP CONNECT
network layer
transport layer
transport layer connection
network layer
transport layer SETUP CONNECT
network layer
transport layer
connection-oriented subnetwork
connectionlesssubnetwork
heterogeneous subnetworks
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-31
© James P.G. SterbenzITTC
E2E State ManagementImpact of High Speed
• Connection shortening– transmission delay decreases– signalling delay constant
• E2E signalling– significant fraction of time
slow high-speedfast
~1.5tRTT
~2.5tRTT ACK
RELEASEACK
RELEASE
tRTT
SETUP
RELEASE
ACK
tp
td
tp
td→0
SETUP SETUP
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-32
© James P.G. SterbenzITTC
E2E State ManagementImpact of Long Latency
• Connection lengthening– transmission delay increases– signalling delay increases
• E2E signalling– open state longer– denial of resource to others
short links long links
SETUP
ACK
RELEASE
SETUP
ACK
RELEASE
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-33
© James P.G. SterbenzITTC
Transfer ModesDatagram
• Independent packets– each has fully self-describing header– no network state needed to forward
tp+td
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-34
© James P.G. SterbenzITTC
Transfer Modes Connection
• Explicit connection setup– 3-way handshake
SETUP / CONNECT / ACK• Data flow
– packets need connection or flow identifier• used by nodes to look up state
• Release of resources and state– explicit RELEASE– time out state
~tRTT
ACK
RELEASE
CONNECT
SETUP
~1.5tRTT
~tRTT
tr
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-35
© James P.G. SterbenzITTC
Transfer Modes Connection
• ConnectionStateMachine– idle– establishing
• install state– initialising
• stateconvergence
– steady state– closing
• release state
initialising
idle
steady-state
closing closing
establishing
CONNECTfrom net
requested
receive SETUP from net
originate SETUP request
issue CONNECT
ACK from net
RELEASEfrom net
RELEASEfrom app
issue SETUP
issue RELEASE
issue RELEASE
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-36
© James P.G. SterbenzITTC
Transfer Modes Stream
• Various mechanisms to start stream– explicit client request– server push– may or may not establish connection state
• Data flow – synchronisation and control
• embedded or• out-of-band
REQUEST
RELEASE
CONNECT
SETUP
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-37
© James P.G. SterbenzITTC
Transfer Modes Stream
• Playback buffer to reorder and absorb jitter– adds delay
maximum playout point
1
application
2346
5 67
playout buffer
receiving end system
Continuous Media Streaming T-I.2
Timeliness of delivery should not be traded for reliable transfer in real-time continuous media streams. Use a playback buffer to reorder and absorb jitter.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-38
© James P.G. SterbenzITTC
Transfer Modes Transaction
• Transaction request– may or may not establish connection state– explicit release of connection state optional
• Data returned• Overlap to reduce latency
– request/response with control
REQUEST
RELEASE
tr
RESPONSE
Minimise Transaction Latency T-6A
Combine connection establishment with request and state transfer to reduce end-to-end latency for transactions.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-39
© James P.G. SterbenzITTC
E2E State ManagementHard vs. Soft
• Hard state– explicitly established and released– deterministic– delay to establish and memory to maintain
• Soft state– established and removed as needed and memory allows– robust to failures– if not explicitly established, delay to accumulate
Hard vs. Soft State T-5A
Balance the determinism and stability of hard state against the adaptability and robustness to failure of soft state.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-40
© James P.G. SterbenzITTC
E2E State ManagementState Aggregation and Sharing
• Sharing to reduce state– temporal: reuse past state– spatial: e.g. end system state
max link rate link characteristics
(sub)network characteristics
path MTU RTT estimate per E2E path
rate/window ACKs/NAKs
global (per interface)
per network
per connection/stream
appid1
appid2 per application
per application transfer
p e r t r a f f I c c l a s s
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-41
© James P.G. SterbenzITTC
E2E State ManagementState Aggregation and Sharing
• Sharing of state– reduces granularity of individual entities– state shared is fate shared state
State Aggregation T-5B
Spatially aggregate state to reduce complexity, but balance against loss in granularity. Temporally aggregate to reduce or eliminate establishment and initialisation phase.State shared is fate shared.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-42
© James P.G. SterbenzITTC
E2E State ManagementAssumed Initial Conditions
• Rapid convergence to steady state– assume initial conditions
• normal or common case• past behaviour• current network state
Use Initial Assumptions to Converge Quickly T-5E
The time to converge to steady state depends on the accuracy of the initial conditions, which depends on the currency of state information and the dynamicity of the network. Use past information to make assumptions that will converge quickly.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-43
© James P.G. SterbenzITTC
End-to-End TransportTL.3 Framing and Multiplexing
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexing
TL.3.1 Framing and fragmentationTL.3.2 Application layer framingTL.3.3 Multiplexing
TL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-44
© James P.G. SterbenzITTC
E2E Framing and MultiplexingPacket Size and Control Overhead
• Small packets– large overhead– short interarrival
• Grouped• Blocked / bursts• Large
– increased delay• Jumbogram
Balance Packet Size T-8A
Trade between control overhead and fine enough grained control. Choose size and aggregation methods that make sense end-to-end.
tpkt
tpkt
tgrp
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-45
© James P.G. SterbenzITTC
E2E Framing and MultiplexingPacket Size and Control Overhead
• Large packets– impose jitter and delay on one another
interference
in1
in2
out delayed
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-46
© James P.G. SterbenzITTC
E2E State ManagementMTU and per Hop Fragmentation
• Subnetworks have limits on packet size– MTU: maximum transfer unit
• If |TPDU| > min(MTU)– fragmentation occurs in the network if permitted
• significant impact on packet processing rate and delay
– most modern networks drop• fragmentation disabled to prevent overwhelming slow path
Avoid Hop-by-Hop Fragmentation T-5F
The transport layer should be aware of or explicitly negotiate the maximum transfer unit of the path to avoid the overhead of fragmentation and reassembly.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-47
© James P.G. SterbenzITTC
E2E Framing and MultiplexingPacket Size and Control Overhead
Application Layer Framing T-8A
To the degree possible, match ADU and TPDU structure. In cases where the application can better determine structure and react to lost and misordered ADUs, ALF should be employed.
transport layer fragmentation
ADU
MTU
TPDU
ADU ADU
TPDU TPDU TPDU
TPDU
ADU ADU
TPDU TPDU TPDU
ADU
application layer framing
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-48
© James P.G. SterbenzITTC
E2E Framing and MultiplexingLayered Multiplexing
• Multiplexing– application
• interapplicationcoordination
– transport• applications
sharing network interface
– network• end systems
sharing switch– MAC and link
• interfaces sharing medium
a4 pb ADU a4 pc ADU
host4 TPDU TPDU
host3
a3 pa ADUTPDU
NPDUs
network muxing
a3 pa ADU a4 pc ADUa4 pb ADU
host2
pa ADUappx
pb ADUappy
a4 pb ADU
pb ADUappb
pc ADUappc
pc ADUappz
network demuxing
pa ADUappa
transport muxing
transport demuxing
a4 pc ADU
host1
a3 pa ADU TPDU TPDU
TPDU
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-49
© James P.G. SterbenzITTC
E2E Framing and MultiplexingIntegrated Multiplexing
• Multiplexing– significant overhead
• processing limits• delay
– undesirable fate sharing• QOS
– perform all layers at once• network layer for wired• MAC layer for wireless
Limit Layered Multiplexing T-4A
Layered multiplexing should be minimised and performed in a single integrated manner for all layers at the same time.
network muxing
a3 pa ADU a4 pc ADUa4 pb ADU
host2
appx a3 pa ADU
appy a4 pb ADU
host1
appb
appc
host3 a3 pa ADU
a4 pb ADU
a4 pc ADUhost4
a4 pc ADUappx
network demuxingappa
NPDUs
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-50
© James P.G. SterbenzITTC
End-to-End TransportTL.4 Error Control
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error control
TL.4.1 Types and causes of errorsTL.4.2 Impact of high speedTL.4.3 Closed-loop retransmissionTL.4.4 Open-loop error control
TL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-51
© James P.G. SterbenzITTC
E2E Error ControlTypes of Errors0
What are the types of errors?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-52
© James P.G. SterbenzITTC
E2E Error ControlTypes of Errors1
• Bit errors• Packet errors
types?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-53
© James P.G. SterbenzITTC
E2E Error ControlTypes of Errors2
• Bit errors• Packet errors
– packet loss– fragment loss– burst loss– packet misordering– packet insertion– packet duplication
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-54
© James P.G. SterbenzITTC
E2E Error ControlCauses of Bit Errors0
• Bit errorscauses?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-55
© James P.G. SterbenzITTC
E2E Error ControlCauses of Bit Errors1
• Bit errors– flaky hardware– wireless channels
• noise and interference
– optical channels• long-haul fiber links engineered on margin with FEC• control of amplifier and regenerator cost
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-56
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Loss0
• Packet losscauses?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-57
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Loss1
• Packet loss– network buffer overflow or intentional drop– transport protocol packet discard when bit error
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-58
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Loss2
• Packet loss– network buffer overflow or intentional drop– transport protocol packet discard when bit error
• Packet fragment losscauses?impact?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-59
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Loss3
• Packet loss– network buffer overflow or intentional drop– transport protocol packet discard when bit error
• Packet fragment loss– loss of a piece of a fragmented packet– effect: error multiplication
• loss of a fragment in net generally results in entire packet loss
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-60
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Burst Loss0
• Packet burst losscauses?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-61
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Burst Loss1
• Packet burst loss– network buffer overflow during congestion– wireless channel fades
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-62
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Misordering0
• Packet misorderingcauses?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-63
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Misordering1
• Packet misordering– multiple paths through switch or network– non-deterministic paths through switch or network– path reconfiguration
• after failure• due to mobility• due to load balancing
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-64
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Insertion0
• Packet insertionmeaning?causes?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-65
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Insertion1
• Packet duplication– undetectable header error
• packet appears that was destined elsewhere
– long packet life• packet with same destination and sequence number still in net
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-66
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Duplication0
• Packet duplicationcauses?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-67
© James P.G. SterbenzITTC
E2E Error ControlCauses of Packet Duplication1
• Packet duplication– retransmission but original still arrives
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-68
© James P.G. SterbenzITTC
E2E Error ControlImpact of High Speed
• Bandwidth-×-delay product increases– fixed duration error event larger relative impact– closed-loop reaction occurs later in terms of # bits sent
• Sequence numbers– sequence numbers wrap if sequence space too small
• causes packet insertion
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-69
© James P.G. SterbenzITTC
End-to-End TransportTL.4.3 Closed-Loop Retransmission
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error control
TL.4.1 Types and causes of errorsTL.4.2 Impact of high speedTL.4.3 Closed-loop retransmissionTL.4.4 Open-loop error control
TL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-70
© James P.G. SterbenzITTC
E2E Error ControlARQ Closed Loop Retransmission
• Automatic repeat request (ARQ)– PDUs are retransmitted for reliable transfer
• Acknowledgments are used to request retransmission– ACK: positive acknowledgement– NAK (or NACK): negative acknowledgement
which is better?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-71
© James P.G. SterbenzITTC
E2E Error ControlARQ Closed Loop Retransmission
• Automatic repeat request (ARQ)– PDUs are retransmitted for reliable transfer
• Acknowledgments are used to request retransmission– ACK: positive acknowledgement
• required for full reliability by E2E arguments
– NAK (or NACK): negative acknowledgement– tradeoff between error rate and predictability
• hybrid: NAKs supplemented by ACKs when necessary for E2E
Simplest ARQ mechanism?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-72
© James P.G. SterbenzITTC
0
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged
1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-73
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged0
0
2
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-74
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
disadvantages?
0
0
3
A0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-75
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment
0
0
A0
1tack
4
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-76
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment
– sender timer tack fires if ACK not receivedissues?
0
0
A0
1tack
5
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-77
© James P.G. SterbenzITTC
1
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment
– sender timer tack fires if ACK not received• too conservative: issues?• too aggressive: issues?
0
0
A0
1tack
6
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-78
© James P.G. SterbenzITTC
1
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment
– sender timer tack fires if ACK not received• too conservative: unnecessary delay• too aggressive: spurious retransmissions
0
0
A0
1tack
7
1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-79
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Stop-and-Wait
• Each packet is acknowledged– subsequent packet waits on previous ACK
• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment
– sender timer tack fires if ACK not received• too conservative: unnecessary delay• too aggressive: spurious retransmissions
How can we do better?
0
0
A0
1
1
A1
tack
1
8
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-80
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions 0
1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-81
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight 0
0
2
1
A0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-82
© James P.G. SterbenzITTC
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledged
0
0
3
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-83
© James P.G. SterbenzITTC
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedhow is packet loss detected?
0
0
4
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-84
© James P.G. SterbenzITTC
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
0
0
5
1
A0
1
A1
tack
A1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-85
© James P.G. SterbenzITTC
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not receivedimplication?
0
0
6
1
A0
1
A1
tack
A1
A1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-86
© James P.G. SterbenzITTC
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
0
0
7
1
A0
1
A1
tack
A1
A1
A1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-87
© James P.G. SterbenzITTC
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
0
0
8
1
A0
1
A1
tack
A1
A1
A1 2
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-88
© James P.G. SterbenzITTC
4
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
0
0
9
1
A0
1
A1
tack
A1
A1
A1 2
3
A3
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-89
© James P.G. SterbenzITTC
5
4
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
0
0
10
1
A0
1
A1
tack
A1
A1
A1 2
3
A3
A4
4
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-90
© James P.G. SterbenzITTC
6
5
4
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
0
0
11
1
A0
1
A1
tack
A1
A1
A1 2
3
A3
A4
4
5
A5
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-91
© James P.G. SterbenzITTC
6
5
4
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
disadvantages?
0
0
12
1
A0
1
A1
tack
A1
A1
A1 2
3
A3
A4
4
5
A56
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-92
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight
• Packets are sequentially acknowledgedo previous ACK retransmitted for
• subsequent packets after loss• out of sequence packet
o sender timer fires if ACK not received• reset transmission beginning at lost packet
– significant loss penalty for high bw-×-delay• go back and retransmit all since loss• many unneeded retransmissions• significant additional delay
A0
A1
A1
0
1
2
3
4
5
6
2
3
4
5
6
A1
A2
A3
A4
A5
A1
0
1
2
3
5
4
tack
13
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-93
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Optimisation possible for go-back-n?
A0
A1
A1
0
1
2
3
4
5
6
2
3
4
5
6
A1
A2
A3
A4
A5
A1
0
1
2
3
5
4
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-94
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Optimisation go-back-ntimer the only way to detect loss?
A0
A1
A1
0
1
2
3
4
5
6
2
3
4
5
6
A1
A2
A3
A4
A5
A1
0
1
2
3
5
4
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-95
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
0
1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-96
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n 0
0
2
1
A0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-97
© James P.G. SterbenzITTC
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n 0
0
3
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-98
© James P.G. SterbenzITTC
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n 0
0
4
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-99
© James P.G. SterbenzITTC
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are loss
0
0
5
1
A0
1
A1
tack
A1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-100
© James P.G. SterbenzITTC
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired
0
0
6
1
A0
1
A1
tack
A1
A1
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-101
© James P.G. SterbenzITTC
3
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly
0
0
7
1
A0
1
A1
tack
A1
A1
A12
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-102
© James P.G. SterbenzITTC
43
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly
0
0
8
1
A0
1
A1
tack
A1
A1
A123
A3
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-103
© James P.G. SterbenzITTC
543
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly
0
0
9
1
A0
1
A1
tack
A1
A1
A123
A3A4
4
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-104
© James P.G. SterbenzITTC
6
543
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly
0
0
10
1
A0
1
A1
tack
A1
A1
A123
A3A4
45
A5
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-105
© James P.G. SterbenzITTC
6
543
A2
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly
0
0
11
1
A0
1
A1
tack
A1
A1
A123
A3A4
45
A56
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-106
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Fast Retransmit
• Fast retransmit– optimisation for go-back-n
• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly– still suffers from go-back-n inefficiencies
12
6
543
A2
2
5
4
3
2
0
0
1
A0
1
A1
tack
A1
A1
A123
A3A4
45
A56
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-107
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Delayed ACK
• Delayed ACKo optimisation for go-back-n
• Aggregate several ACKs+ reduces ACK traffic+ allows some receiver resequencing
• within ACK aggregation quantum
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-108
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Go-Back-n
• Go-back-n̶ fast retransmit and delayed ACK help slightly̶ but still significant delay penalty for losses
Alternative?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-109
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Go-back-n̶ fast retransmit and delayed ACK help slightly̶ significant delay penalty for losses
• Alternative− don’t go back: selective repeat
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-110
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
1
0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-111
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged 0
0
2
1
A0
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-112
© James P.G. SterbenzITTC
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged 0
0
3
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-113
© James P.G. SterbenzITTC
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged 0
0
4
1
A0
1
A1
tack
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-114
© James P.G. SterbenzITTC
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
– Missequenced packetso acknowledged and held at receiver
0
0
5
1
A0
1
A1
tack
A3
3
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-115
© James P.G. SterbenzITTC
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
0
0
6
1
A0
1
A1
tack
A3
3
A4
43
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-116
© James P.G. SterbenzITTC
2
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
• Lost packetso selectively retransmitted
0
0
7
1
A0
1
A1
tack
A3
3
A4
43
543
A5
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-117
© James P.G. SterbenzITTC
62
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
• Lost packetso selectively retransmitted+ no go-back-n latency penalty– requires more receiver buffer space
0
0
8
1
A0
1
A1
tack
A3
3
A4
43
543
A5
A2
5432
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-118
© James P.G. SterbenzITTC
762
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
• Lost packetso selectively retransmitted+ no go-back-n latency penalty– requires more receiver buffer space
0
0
9
1
A0
1
A1
tack
A3
3
A4
43
543
A5
A2
5432
A6
6
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-119
© James P.G. SterbenzITTC
8762
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
• Lost packetso selectively retransmitted+ no go-back-n latency penalty– requires more receiver buffer space
0
0
10
1
A0
1
A1
tack
A3
3
A4
43
543
A5
A2
5432
A6
6
A7
7
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-120
© James P.G. SterbenzITTC
9
8762
5
4
3
2
E2E Error ControlClosed Loop Retransmission: Selective Repeat
• Selective repeato all packets acknowledged
• Missequenced packetso packets held and reordered at receiver– increases receiver complexity
• Lost packetso selectively retransmitted+ no go-back-n latency penalty– requires more receiver buffer space
• Latency and Bandwidth reduced
0
0
11
1
A0
1
A1
tack
A3
3
A4
43
543
A5
A2
5432
A6
6
A7
87
A8
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-121
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Periodic State
• Selective ACK blockso bit vectors can aggregate ACKs+ significant reduction in control packets+ simple receiver implementation possible
5
4
6
7
0
1
A01_3
0
1
2
3
2
8
9
7–2
8
9A4567
~1.5tRTT
3
43
543
6543
765543
A2
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-122
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Comparison
0
0
A0
1
1
A1
tack
1
A0
A1
A1
0
1
2
3
4
5
6
2
3
4
5
6
A1
A2
A3
A4
A5
A1
0
1
2
3
5
4
tack
9
8762
5
4
3
2
0
0
1
A0
1
A1
tack
A3
3
A4
43
543
A5
A2A6
6
A7
87
A8
5
4
6
7
0
1
A01_3
0
1
2
3
2
8
9
7–2
8
9A4567
~1.5tRTT
3
43
543
6543
765543
A2
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-123
© James P.G. SterbenzITTC
E2E Error ControlImpact of High Speed
• Bandwidth-×-delay product increases– fixed duration error event larger relative impact– reaction to errors decreases in terms of number of bits sent
• Sequence numbers– sequence numbers wrap if sequence space too small
• causes packet misinsertion
Packet Control Field Values T-8C
Optimise header control field values to trade efficiency againstexpected future requirements. Fields that are likely to be insufficient for future bandwidth-×-delay products should contain a scale factor.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-124
© James P.G. SterbenzITTC
E2E Error ControlDecoupling of Control Mechanisms
• Error control mechanisms– difficult to optimise for high performance
when entangled with flow/congestion control(more later )
Decouple Error, Flow, and Congestion Control T-6E
Exploit selective acknowledgements and open-loop rate control to decouple error, flow, and congestion control mechanisms.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-125
© James P.G. SterbenzITTC
E2E Error ControlClosed Loop Retransmission: Multicast
• ACK implosion: ACK suppression neededo NAKs (negative ACKs) with liveness pollingo localised hop-by-hop buffering and retransmission– significant reduction of traffic on relatively reliable links
R
6
ACK
ACK×2ACK×2
ACK×4
ACKACK×6
R
6
NAK
NAK
NAK
NAK×2
31 2 4 31 2 4
ACKACK
ACK
5ACK
NAK
5NAK
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-126
© James P.G. SterbenzITTC
Error ControlExample:7.6 SNA Error Control
• SNA (IBM Systems Network Architecture)
• Not in agreement with end-to-end arguments, but…– engineered reliability of store-and-forward hosts
• no part of data path not covered by ECC: processor + memory
Scope Layer Name Reliability
hop-by-hop link data link control yes
end-to-end transport path control no
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-127
© James P.G. SterbenzITTC
End-to-End TransportTL.4.4 Open-Loop Error Control
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error control
TL.4.1 Types and causes of errorsTL.4.2 Impact of high speedTL.4.3 Closed-loop retransmissionTL.4.4 Open-loop error control
TL.5 Transmission controlTL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-128
© James P.G. SterbenzITTC
E2E Error ControlOpen Loop
• Open loop error controlo some applications do not need guaranteed reliability
• applications that are tolerant to some loss• real-time or interactive delay requirements
– eliminates control loop latency for recovery+ requires additional end-systems bandwidth & processing+ requires additional network bandwidth
• Forward error correction– block codes: per block recovery– convolutional codes: continuous over window
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-129
© James P.G. SterbenzITTC
E2E Error ControlHybrid Open/Closed-Loop Control
• Hybrid open loop with closed loop when necessary– statistical error bound with FEC– retransmission only when FEC doesn’t allow correction– appropriate for
• high latency and bandwidth-×-delay paths• asymmetric bandwidth paths
Forward Error Correction T-6De
Use FEC for low-latency, open-loop flow control when bandwidth is available and statistical loss tolerance bounds areacceptable to the applications.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-130
© James P.G. SterbenzITTC
End-to-End TransportTL.5 Transmission Control
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission control
TL.5.1 Impact of high speedTL.5.2 Open-loop rate controlTL.5.3 Closed-loop flow controlTL.5.4 Closed-loop congestion controlTL.5.5 Hybrid flow and congestion control
TL.6 TCP optimisations
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-131
© James P.G. SterbenzITTC
E2E Transmission ControlDefinitions
• Flow control– control transmission to avoid overwhelming receiver– end-to-end control
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-132
© James P.G. SterbenzITTC
E2E Transmission ControlDefinitions
• Flow control– control transmission to avoid overwhelming receiver– end-to-end control
• Congestion control– control transmission to avoid overwhelming network paths
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-133
© James P.G. SterbenzITTC
E2E Transmission ControlDefinitions
• Flow control– control transmission to avoid overwhelming receiver– end-to-end control
• Congestion control– control transmission to avoid overwhelming network paths– network control with end-to-end assistance (throttling)
• Types– explicit relies on congestion information from nodes
• allows decoupling of error, flow, and congestion mechanisms
– implicit assumes loss is congestion• bad assumption in wireless networks (see [Krishnan 2004])
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-134
© James P.G. SterbenzITTC
E2E Transmission ControlImpact of High Speed
• Bandwidth-×-delay product increases– response to an event happens after more bits transferred– it may be too late to react
• all bits causing problem may already be transmitted• other cause of problem may have gone away
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-135
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control
• Initial negotiation– admission control limits traffic to available resources
admissioncontrol
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-136
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control
• Initial negotiation– admission control limits traffic to available resources– receiver negotiates what it can accept (flow control)
admissioncontrol
receivernegotiation
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-137
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control
• Initial negotiation– admission control limits traffic to available resources– receiver negotiates what it can accept (flow control)
• Steady state– policing enforces traffic contract from transmitter
admissioncontrol
receivernegotiation
ratescheduling
ratepolicing
conforming traffic
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-138
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control
• Initial negotiation– admission control limits traffic to available resources– receiver negotiates what it can accept (flow control)
• Steady state– policing enforces traffic contract from transmitter
• excess traffic may be marked and passed if capacity available
admissioncontrol
receivernegotiation
ratescheduling
ratepolicing
conforming trafficexcess traffic
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-139
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control
Use Knowledge of Network Paths for Open-Loop T-6Do
Control Exploit open-loop rate and congestion control based on a priori knowledge to the degree possible to reduce the need for feedback control.
• Requires connection establishment– overhead and delay for connection setup
• optimistic establishment or fast reservations ameliorate
+ QOS guarantees possible in steady state+ feedback control loops not needed during transmission+ network stability less dependent on congestion control
• unless resources significantly overbooked
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-140
© James P.G. SterbenzITTC
E2E Transmission ControlOpen-Loop Rate Control: Parameters
• Main parameters– peak rate rp
– average rate ra
– burstiness (peak to average ratio or max burst size)
• Derived parameters– jitter
rp
ra bursty
uniform
time
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-141
© James P.G. SterbenzITTC
E2E Transmission ControlClosed-Loop Flow and Congestion Control
Use Closed-Loop Congestion Control to Adjust to T-6Dc
Network Conditions Closed-loop feedback is needed to adjust to network conditions when there are no hard reservations.
networkcongestion control
packet scheduling
flow control
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-142
© James P.G. SterbenzITTC
E2E Transmission ControlClosed-Loop Flow Control
• Feedback from receiver to limit rate• Window to limit amount of unacknowledged data
– static window– dynamic window
• combined with congestion control
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-143
© James P.G. SterbenzITTC
Congestion ControlIdeal Network
offered load
ideal
carr
ied
load
offered load
ideal
1.0
dela
y
throughput delay
• Ideal network:– throughput: carried load = offered load (45° line)– zero delay (flat line)
Why isn’t this possible?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-144
© James P.G. SterbenzITTC
Congestion ControlReal Network
• Delay can’t be zero: speed-of-light– delay through network channels– delay along paths in network nodes
• Throughput can’t be infinite– channel capacity limits bandwidth– switching rate of components limits bit rate
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-145
© James P.G. SterbenzITTC
Congestion ControlDesired Real Network Behaviour
knee
offered load
ideal1.0
1.0
carr
ied
load
offered load 1.0
dela
y
knee
throughput
dmin
delay
• Desired network:– carried load = offered load up to share of capacity– what happens to delay?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-146
© James P.G. SterbenzITTC
Congestion ControlCauses of Congestion
• Congestion when offered load → link capacity• Best effort
– occurs whenever load is high
• Probabilistic guarantees– occurs when load exceeds resource reservations
• Absolute guarantees– can’t happen (in theory)
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-147
© James P.G. SterbenzITTC
Congestion ControlInfinite Buffers / No Retransmission
• Congestion when offered load → link capacity• Infinite buffers
– queue length builds to infinity– d → ∞
todo: figures
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-148
© James P.G. SterbenzITTC
Congestion ControlFinite Buffers with Retransmission
• Finite buffers cause packet loss– retransmissions contribute to offered load
• assuming reliable protocol
– throughput curve levels off
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-149
© James P.G. SterbenzITTC
Congestion ControlMultihop Network
• Multiple hops– downstream packet loss– upstream transmission wasted
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-150
© James P.G. SterbenzITTC
Congestion ControlConsequences of Congestion
• Delay increases– due to packet queuing in network nodes– due to retransmissions when packets overflow buffers
• finite buffers must drop packets when full
• Throughput– levels off gradually (with real traffic)– then decreases
• due to retransmissions when packets dropped• particularly over multiple hops
– congestion collapse• “cliff” of the throughput curve
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-151
© James P.G. SterbenzITTC
Congestion ControlLocal Action
• Local action at point of congestion• Drops packets: discard policy
– tail drop simplest: drop packets that overflow buffers– more intelligent policies possible
• need flow or connection state in switches• discriminate which flows are causing congestion and penalise
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-152
© James P.G. SterbenzITTC
Congestion ControlEnd-to-End Action
• End-to-end action• Throttle source
– reduce transmission rates– prevent unnecessary retransmissions
• Types– explicit– implicit
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-153
© James P.G. SterbenzITTC
Congestion ControlCongestion Avoidance and Control
knee cliff
offered load
ideal1.0
1.0
carr
ied
load
offered load
ideal
1.0
dela
y
knee
cliff
CA CC
throughput
dmin
delay
CA
CC
Congestion Avoidance T-II.4c
Congestion should be avoided before it happens. Keep queues from building and operate just to the left of the knee to maximise throughput and minimise latency.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-154
© James P.G. SterbenzITTC
E2E Transmission ControlClosed-Loop Congestion Control
• Feedback from network to limit rate• Window to limit amount of unacknowledged data
– dynamic window– conservation of packets in the network– self clocking
• Required unless open loop with hard reservations
Use Closed-Loop Congestion Control to Adjust to T-6Dc
Network Conditions Closed-loop feedback is needed to adjust to network conditions when there are no hard reservations.
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-155
© James P.G. SterbenzITTC
Congestion ControlWindow Size
• Window size critical– big enough to fill pipe
A0
A1
0
1
A2
5
slow path fast path
0
1
2
3
A0A1A2A3
A4A5A6A7
4
5
6
A3
A4
A5
2
3
4
5
6
0123
4567
8 9 1011
4 5 6 7
0 1 2 3
891011
12
A8A9
A10A11
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-156
© James P.G. SterbenzITTC
Congestion ControlSlow-Start Initialisation
A0
A1
A2
A3
A4
A5
A6
• Slow start initialisation– increase window until path loaded
• Critical parameters– initial window size– rate of increase
Tradeoffs?
0
3
4
5
6
0
1
2
1
2
3
4
5
6
7
8
7
8
9
10
9
10
11
12
11
12
13
14
13
14
15
A7
A8
A9
A10
A11
A12
13
14
15
16
17
18
19
20
21
20
21
18
19
10
11
12
13
14
15
16
17
A4
A5
A6
A0
A1
A2
A3
0
1
2
3
0
1
2
3
4
5
4
5
6
7
6
7
8
9
8
9
10
11
12
A7
A8
A9
A10
A11
A12
A10
A13
A14
A15
A16
A17
tRTT
tRTT
tRTT
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-157
© James P.G. SterbenzITTC
Congestion ControlSlow-Start Initialisation
A0
A1
A2
A3
A4
A5
A6
• Slow start initialisation– increase window until path loaded
• Critical parameters– initial window size– rate of increase
• Tradeoffs– conservative on high bw-×-delay:
• multiple round trips• never get to full rate for transactions
– aggressive:• induced congestions
0
3
4
5
6
0
1
2
1
2
3
4
5
6
7
8
7
8
9
10
9
10
11
12
11
12
13
14
13
14
15
A7
A8
A9
A10
A11
A12
13
14
15
16
17
18
19
20
21
20
21
18
19
10
11
12
13
14
15
16
17
A4
A5
A6
A0
A1
A2
A3
0
1
2
3
0
1
2
3
4
5
4
5
6
7
6
7
8
9
8
9
10
11
12
A7
A8
A9
A10
A11
A12
A10
A13
A14
A15
A16
A17
tRTT
tRTT
tRTT
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-158
© James P.G. SterbenzITTC
Congestion ControlAIMD Steady State
• AIMD steady state– additive-increase slowly increases rate
• increment window
– multiplicative-decrease quickly throttles with congestion• divide window
– RED attempts to reduce when congestion impending
additive increase
multiplicative decrease
time
rate
congestion detection
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-159
© James P.G. SterbenzITTC
Congestion ControlAIMD Fairness and Stability
• AIMD– R = available bandwidth– initial bandwidth share I– AI: both increase
• until loss beyond R– MD: both decrease
• half way to origin• halve window size
– trajectory toward ideal• intersection of R and equal
[based on Kurose fig. 3.53]
equal bandwidth
shareR
Co n
nec t
ion
2 th
roug
hput
RConnection 2 throughput
I
ideal
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-160
© James P.G. SterbenzITTC
Congestion ControlSlow Start + AIMD
• Initialisation phase: slow start• Steady-state phase: AIMD
AI
time
MD
send
ing
rate
slow start
steady state phase initialisation
congestion detection
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-161
© James P.G. SterbenzITTC
Congestion ControlOptimisation for High-Speed
• Initialisation– start with bigger window– increase more rapidly than “slow start” doubling– danger: too aggressive can cause congestion collapse
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-162
© James P.G. SterbenzITTC
Congestion ControlOptimisation for High-Speed
• Initialisation– start with bigger window– increase more rapidly than “slow start” doubling– danger: too aggressive can cause congestion collapse
• Steady state– better estimate capacity to avoid MD halving
• TCP BIC and CUBIC binary search for faster convergance
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-163
© James P.G. SterbenzITTC
Congestion ControlImplicit Control
• Implicit control– missing ACK assumed to be congestion
good assumption?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-164
© James P.G. SterbenzITTC
Congestion ControlImplicit Control
• Implicit control– missing ACK assumed to be congestion– reasonable when all losses are due to congestion
• fiber optic channels connected by reliable switches
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-165
© James P.G. SterbenzITTC
Congestion ControlImplicit Control
• Implicit control– missing ACK assumed to be congestion– reasonable when all losses are due to congestion
• fiber optic channels connected by reliable switches
– performs poorly when significant losses in channelwhy?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-166
© James P.G. SterbenzITTC
Congestion ControlImplicit Control
• Implicit control– missing ACK assumed to be congestion– reasonable when all losses are due to congestion
• fiber optic channels connected by reliable switches
– performs poorly when significant losses in channel• mobile wireless links• under-provisioned CDMA (including optical CDMA)
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-167
© James P.G. SterbenzITTC
Congestion ControlImplicit Control
• Implicit control– missing ACK assumed to be congestion– reasonable when all losses are due to congestion
• fiber optic channels connected by reliable switches
– performs poorly when significant losses in channel• mobile wireless links• under-provisioned CDMA (including optical CDMA)
Alternatives?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-168
© James P.G. SterbenzITTC
Congestion ControlExplicit Control
• Explicit control– explicit congestion notification (ECN)
• throttle with ECN message
– some decoupling of error and congestion control• throttle before packet loss• not sufficient for lossy wireless link
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-169
© James P.G. SterbenzITTC
Congestion ControlExplicit Control
• Explicit control– explicit congestion notification (ECN)
• throttle with ECN message
– some decoupling of error and congestion control• throttle before packet loss• not sufficient for lossy wireless link
– not sufficient for discrimination of corrupted packets• ELN (explicit loss notification) necessary
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-170
© James P.G. SterbenzITTC
Congestion ControlSteady State Comparison
• Implicit control– standard– with fast retransmit
• Explicit control– ECN
tack
implicit explicit fast retransmit
A7
A7
A7
A7
A7
13
14
15
16
8
9
10
11
12
11
12
9
10
8
A4
A5
A6
A0
A1
A2
A3
0
1
2
3
0
1
2
3
4
5
4
5
6
7
6
7
9
10
11
12
A7
A7
A7
A7
A8
A9
8
A71
A72
A7313
14
8
9
10
11
12
13
14
11
12
9
10
8
A4
A5
A6
A0
A1
A2
A3
0
1
2
3
0
1
2
3
4
5
4
5
6
7
6
7
9
10
11
12
A7
A7
A7
A8
A9
8
12
13
13
14
15
16
17
18
19
20
21
A4
A5
A6
A0
A1
A2
A3
0
1
2
3
0
1
2
3
4
5
4
5
6
7
6
7
9
10
11
12 11
12
9
10
8
13
14
ECN8
18
19
16
17
15
20
21
A7
A8
A9
A10
A11
A12
A13
A14
A15
A16
A17
A18
A10
A11
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-171
© James P.G. SterbenzITTC
End-to-End TransportTL.6 TCP Optimisations
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
TL.6.1 TCP overviewTL.6.2 TCP optimisations for high performance
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-172
© James P.G. SterbenzITTC
Transmission Control ProtocolOverview
• TCP: transmission control protocolRFC 0793 / STD 0007+ RFC 1122 implementation requirements+ RFC 1323 high performance extensions+ RFC 2018 SACK (selective acknowledgements)+ RFC 5681 congestion control+ RFC 3465 appropriate byte counting+ RFC 3168 ECN (explicit congestion notification)
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-173
© James P.G. SterbenzITTC
TCP OverviewTransfer Mode and Characteristics
• Connection-oriented transfer mode– latency of connection setup– connection state must be maintained on each end system
• Point-to-point full-duplex– reliable byte stream: no message boundaries– pipelined transfer (not just stop-and-wait)
• Flow control– will not overwhelm receiver
• Congestion control– attempt to avoid congesting network– implicit and optional ECN
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-174
© James P.G. SterbenzITTC
TCP OverviewSegment Format: Overview
• Relatively large header– fixed length & position fields– variable length options– header length4b in Bytes
• = 40 + |options| B
• Variable length payload• Options
– may not be present• Checksum on entire segment
– same algorithm as UDP
checksum urg data pointerreceive window
ack numbersequence number
dest port #source port #
[options (variable length)]
applicationpayload
(variable length)
flagsHL
32 bits
HL
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-175
© James P.G. SterbenzITTC
TCP OverviewSegment Format: Header Flags
• Control flags (1 bit each)– CWR cong. win. reduced– ECE ECN echo– URG urgent data– ACK acknowledgement– PSH push data– RST reset connection– SYN connection setup– FIN connection teardown
checksum urg data pointerreceive window
ack numbersequence number
dest port #source port #
options (variable length)
applicationpayload
(variable length)
HL
32 bits
CEUAPRSF
URG
ACK
PSH
RST
SYN
FIN
ECE
CWR
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-176
© James P.G. SterbenzITTC
TCP OverviewConnection Management: Segment Format
• Flags define packet type– ACK segment
• ACK for SYN or FIN• SYNACK may be
piggybacked• ACK for data
– RST (reset) segment• e.g. bad port received
– SYN (synchronise) segment• connection setup
– FIN (finish) segment• connection release
checksum urg data pointerreceive window
ACK numbersequence number
dest port #source port #
options (variable length)
applicationpayload
(variable length)
CEUAPRSFHL
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-177
© James P.G. SterbenzITTC
data segments
data segments
TCP OverviewConnection Management: Setup
• Three way handshake– “client” and “server”1. SYN (init seq#) →
• randomised for security2. ← SYNACK (init seq#)
• randomised for security• after buffer allocation• SYN flood vulnerability
3. ACK →– buffer allocation– segment may include data– client may send more
4. server may return data
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
FIN
ACK
FIN
ACK
bufferalloc
bufferalloc
estab.state
C S
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-178
© James P.G. SterbenzITTC
data segments
data segments
TCP OverviewConnection Management: Teardown
• Two two-way handshakes– each is a half-close– may occur simultaneously
• full close– ACK can’t be piggybacked
• Client closes socket1. FIN →2. ← ACK– timed wait before close
• Server closes socket1. ← FIN2. ACK →
net
SYN
ACK
SYNACK
FIN
ACK
FIN
ACK
bufferfree
bufferfree
C S
twait
closed
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-179
© James P.G. SterbenzITTC
TCP OverviewRFC 0793 State Machine
+---------+ ---------\ active OPEN | CLOSED | \ -----------+---------+<---------\ \ create TCB | ^ \ \ snd SYN
passive OPEN | | CLOSE \ \------------ | | ---------- \ \create TCB | | delete TCB \ \
V | \ \+---------+ CLOSE | \| LISTEN | ---------- | | +---------+ delete TCB | |
rcv SYN | | SEND | | ----------- | | ------- | V
+---------+ snd SYN,ACK / \ snd SYN +---------+| |<----------------- ------------------>| || SYN | rcv SYN | SYN || RCVD |<-----------------------------------------------| SENT || | snd ACK | || |------------------ -------------------| |+---------+ rcv ACK of SYN \ / rcv SYN,ACK +---------+| -------------- | | -----------| x | | snd ACK | V V | CLOSE +---------+ | ------- | ESTAB | | snd FIN +---------+ | CLOSE | | rcv FIN V ------- | | -------
+---------+ snd FIN / \ snd ACK +---------+| FIN |<----------------- ------------------>| CLOSE || WAIT-1 |------------------ | WAIT |+---------+ rcv FIN \ +---------+| rcv ACK of FIN ------- | CLOSE | | -------------- snd ACK | ------- | V x V snd FIN V
+---------+ +---------+ +---------+|FINWAIT-2| | CLOSING | | LAST-ACK|+---------+ +---------+ +---------+| rcv ACK of FIN | rcv ACK of FIN | | rcv FIN -------------- | Timeout=2MSL -------------- | | ------- x V ------------ x V \ snd ACK +---------+delete TCB +---------+------------------------>|TIME WAIT|------------------>| CLOSED |
+---------+ +---------+
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-180
© James P.G. SterbenzITTC
TCP OverviewActual Connection State Machine
[Sewell] http://www.cl.cam.ac.uk/users/pes20/Netsem
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-181
© James P.G. SterbenzITTC
TCP OverviewSegment Format: Sequence and Window
• Sequence #– forward data transfer– 1st byte # in segment
• ACK number– reverse acknowledgements– seq # of next byte expected– if ACK flag set– data segment may
piggyback ACK• Receive window
– # unACKed bytes
checksum urg data pointerreceive window
ACK numbersequence number
dest port #source port #
options (variable length)
applicationpayload
(variable length)
HL CEUAPRSF
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-182
© James P.G. SterbenzITTC
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
1
• Data is sequence of bytes– number by byte #
(not segment #)
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-183
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=?
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
2
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-184
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=?
3
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-185
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
data segment (MSS bytes)SN=?
4
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-186
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=?
5
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected– cumulative ACKs
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-187
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=ISNs+2MSS
6
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected– cumulative ACKs
what next?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-188
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs+2MSS
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=ISNs+2MSS
7
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected– cumulative ACKs
what next?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-189
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs+2MSS
data segment (MSS bytes)SN=ISNs
TCP OverviewUnidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=ISNs+2MSS
8ACK AN=ISNs+3MSS
• Data is sequence of bytes– number by byte #
(not segment #)– each segment ≤ MSS B
• Sequence numbers– byte # in byte stream
of 1st byte in segment
• ACK numbers– seq # of next byte expected– cumulative ACKs
• Out of order arrival– implementation specific
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-190
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNs
TCP OverviewBidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
1
• Data both directions– ACKs returned both ways
what next?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-191
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNc
data segment (MSS bytes)SN=ISNs
TCP OverviewBidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
2
• Data both directions– ACKs returned both ways– may be piggybacked
what next?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-192
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNc
data segment (MSS bytes)SN=ISNs
TCP OverviewBidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
3
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=ISNc+MSS
what next?
• Data both directions– ACKs returned both ways– may be piggybacked– delayed ACK waits briefly
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-193
© James P.G. SterbenzITTC
data segment (MSS bytes)SN=ISNc
data segment (MSS bytes)SN=ISNs
TCP OverviewBidirectional Data Transfer
net
SYN (ISNs)
ACK
SYN(ISNc)ACK
C S
ACK AN=ISNs+MSS
4
data segment (MSS bytes)SN=ISNs+MSS
ACK AN=ISNc+MSS
• Data both directions– ACKs returned both ways– may be piggybacked– delayed ACK waits briefly
• Bidirectional stream– data segments– piggybacked ACKs
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-194
© James P.G. SterbenzITTC
End-to-End TransportTL.6.2 TCP Optimisations for High Performance
TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations
TL.6.1 TCP overviewTL.6.2 TCP optimisations for high performance
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-195
© James P.G. SterbenzITTC
TCP OptimisationsExample7.8 Congestion Control
• Fast recovery– 1/2 congestion window after 3dupACK rather than slow start
• Partial acknowledgement response [Hoe 1996]– 1/2 window reduction only once with partial retransmit ACK– rather than per packet
• ABC: appropriate byte counting for cwnd [Allman 2003]
• RED: random early detection (discard) [Floyd 1993]– discard packets when router queue threshold exceeded– throttle TCP source earlier before congestion occurs
• ECN: explicit congestion notification [Floyd 1994]– use IP ECN to trigger multiplicative decrease
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-196
© James P.G. SterbenzITTC
TCP OptimisationsExample7.8 Explicit Congestion Notification
• Network nodes setcongestion indication– hist. ICMP source quench– ECN bits in IP header
• TCP reacts– uses flags for signalling
checksum urg data pointerreceive window
ACK numbersequence number
dest port #source port #
options (variable length)
applicationpayload
(variable length)
CEUAPRSFHL
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-197
© James P.G. SterbenzITTC
TCP OptimisationsExample7.8 Congestion Control Research1
• Research optimisations [jury still out]– Pacing [Visweswaraiah 1997]
• spread segment transmissions• rather than burst
– Rate control [Padhye 1998]• rate based on equations that describe TCP behaviour• can also make UDP transmission “TCP friendly”
– ELN: explicit loss notification [Samaraweera 1997]ETEN: explicit transport error notification [Krishnan 2004]
• signal loss due to channel bit errors• loss of ACK is not misconstrued as congestion
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-198
© James P.G. SterbenzITTC
TCP OptimisationsExample7.8 Congestion Control Research2
• Research optimisations [jury still out]– Vegas [Brakmo 1995]
• RTT estimate rather than AIMD
– HSTCP (high-speed TCP) [RFC 3649 Floyd 2003]STCP (scalable TCP) [Kelly 2003]
• adaptive congestion window in response to LFP
– XCP (explicit control protocol) [Katabi 2003]• congestion header with cwnd, RTT estimate, feedback field
– and many others…
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-199
© James P.G. SterbenzITTC
TCP OptimisationsExample7.8 TCP Implementations
• Tahoe– slow start and congestion avoidance algorithms– fast retransmit after triple duplicate ACKs
• Reno – widely implemented– Tahoe + fast recovery
• NewReno [Floyd 1999]– Reno + partial ACK recovery
• BIC (binary-increase TCP) [Xu 2004]– binary search of window size during steady state– CUBIC variant is now default in Linux kernel > 2.6.19
• CTCP (compound TCP) [Tan 2005]– maintains dual AIMD and Vegas-style delay windows– default in MS-Windows since Vista
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-200
© James P.G. SterbenzITTC
TCP OptimisationsHybrid Control
• Dynamic rate control– open-loop rate control modified by network feedback
• example: ATM ABR
• Pacing to reduce burstiness– sender base rate control augments closed-loop control– window transmission spread over RTT– options
• pace initial window, allow ACK self clocking to take over• periodic repacing to compensate for ACK compression• continuous pacing
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-201
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 Header Fields
• TCP/IP headers• bw-×-delay
– fields that limit• sequence space• timer related• window size
• Field predictability– use template for
• constant• predictable
– must compute• highly variable
payload
TCP header 40B
IP header 40B
payload
source port #
rcv window size
destination port sequence number
acknowledgement number
checksum urgent pointer
total length
destination address
frag offset
source address
identifier ver TOS hl
flags TTL header checksum
source port #
rcv window
destination port # sequence number
acknowledgement number
checksum urgent pointer hdr len
total length
destination address
frag offset
source address
identifier ver TOS hl
flags TTL header checksum
UAPRSF hdr len UAPRSF
protocol protocol
constant predictable bw-×-delay sensitive
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-202
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 High-Bandwidth-×-Delay
• High bandwidth-×-delay paths (long fat pipes)problem?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-203
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 High-Bandwidth-×-Delay
• High bandwidth-×-delay paths (long fat pipes)– problem: large number of bits in flight
Implications to TCP?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-204
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 High-Bandwidth-×-Delay
• High bandwidth-×-delay paths (long fat pipes)– problem: large number of bits in flight
• Implications to TCP– max window size is 64KB problem? – packet loss has significant impact on goodput– only one RTT measurement per window– sequence numbers limited to 32 bit problem?
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-205
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 High-Bandwidth-×-Delay
• Common optimisations [RFC 1323]– Window scale option [RFC 1323]
• SYN option power-of-2 multiplier to initial window
– RTTM (round-trip time measurement) [RFC 1323]• timestamp to allow RTT measurement
– PAWS (protection against wrapped seq. nos.) [RFC 1323]• 32-bit timestamp augments 32-bit sequence number
– SACK (selective acknowledgement) [RFC 2018]– byte range header options for 3 – 4 selective ACK ranges
– Fast retransmit [RFC 2581]• retransmit on triple duplicate ACKs without waiting for timer
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-206
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 SACK
• TCP provides reliable byte stream– cumulative ACKs perform poorly if loss rate not very low– cumulative ACKs limit ability to reorder
• Selective ACKs [RFC2018]– selective repeat ARQ mechanism
• used in conjunction with cumulative ACK mechanisms– SACK byte range in TCP header option
• Retransmissions triggered by:– holes in SACK ranges (SACK is positive ACK)
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-207
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 SACK Header
• SACK permitted negotiation– kind=4 in SYN
• SACK ranges– kind = 05
• after kind=01 NOP pads– length = 8n+2 B
for n SACK blocks• max of 3 SACK blocks
– assuming another option
– each block:• seq num of 1st byte• seq num of last byte +1
– ack semantics unchanged
checksum urg data pointerreceive window
ack numbersequence number
dest port #source port #
applicationpayload
(variable length)
flagsHL
32 bits
seq num after last byteseq num of 1st byte
05 len0101
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-208
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 SACK Operation
• Capability announced with SYN– SACK permitted option 5
• Cumulative ACKs as before– ACK byte in last segment – with no missing prior segments
• Selective ACKs– sent only when non-contiguous
segments received– indicate byte range received– holes between SACK blocks
indicate missing byte ranges
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-209
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 Research Optimisations
• Experimental optimisations– larger initial window ≈ 4KB [RFC 3390]– discussion of initial cwnd of 10 segments on TCPM list– concern about aggressiveness
• Protocol Research– trailer checksum [Bridges 1994]
• hotly debated ID• dismissed by IETF community• question of performance gain vs. implementation pain
– pacing [Partridge 2001]• spreads window transmission within RTT• bandwidth estimation and RTT measurement
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-210
© James P.G. SterbenzITTC
TCP OptimisationsExample7.9 Research Optimisations
• Implementation optimisations– header prediction [Jacobson 1990, Pink 1994]
• Implementation research– TCP/IP ILP [Clark 1990]
• complex interactions with cache
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-211
© James P.G. SterbenzITTC
End-to-End TransportAdditional Sources
• Michael WelzlNetwork Congestion Control:Managing Internet TrafficWiley 2005http://www.welzl.at/congestion
18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-212
© James P.G. SterbenzITTC
End-to-End TransportAcknowledgements
Some material in these foils comes from the textbook supplementary materials:
• Kurose & RossComputer Networking:A Top-Down Approach Featuring the Internethttp://wps.aw.com/aw_kurose_network_3
• Sterbenz & TouchHigh-Speed Networking:A Systematic Approach toHigh-Bandwidth Low-Latency Communicationhttp://hsn-book.sterbenz.org