Large (UDP) Packets in IPv6

Geoff HustonAPNIC

What’s the problem?

What’s the problem?

So what?

Packet Networks like variable packet sizes

• Therangeofpacketsizessupportedinanetworkrepresentsasetofengineeringtrade-offs:• Biterrorrateoftheunderlyingmedia• Desiredcarriageefficiency• Transmissionspeedvspacketswitchingspeed

IPv4 Packet Design

FORWARD fragmentation• Ifaroutercannotforwardapacketonitsnexthopduetoapacketsizemismatchthenitispermittedtofragmentthepacket,preservingtheoriginalIPheaderineachofthefragments

IPv4 and the “Don’t Fragment” bit

IfFragmentationisnotpermittedbythesource,thentherouterdiscardsthepacket.TheroutermaysendanICMPtothepacketsourcewithanUnreacahble code(Type3,Code4)

LaterIPv4implementationsaddedaMTUsizetothisICMPmessage

BUT:ICMPmessagesareextensivelyfilteredintheInternetsoapplicationsshouldnotcountonreceivingthesemessages!

Trouble at the Packet Mill

• Lostfragsrequirearesendoftheentirepacket–thisisfarlessefficientthanrepairingalostpacket• Fragmentsrepresentasecurityvulnerabilityastheyareeasilyspoofed• Fragmentsrepresentaproblemtofirewalls–withoutthetransportheadersitisunclearwhetherfragsshouldbeadmittedordenied• Packetreassemblyconsumesresourcesatthedestination

The thinking at the time…

FragmentationwasaBadIdea!

Kent,C.andJ.Mogul,"FragmentationConsideredHarmful",Proc.SIGCOMM'87WorkshoponFrontiersinComputerCommunicationsTechnology,August1987

IPv6 Packet Design

• AttempttorepairtheproblembyeffectivelyjammingtheDON’TFRAGMENTbittoalwaysON• IPv6usesBACKWARDsignalling• WhenapacketistoobigforthenexthoparoutershouldsendanICMP6TYPE2(PacketTooBig)messagetothesourceaddressandincludetheMTUofthenexthop.

IPv6 Source Fragmentation

What changed? What’s the same?

• Bothprotocolsmayfragmentapacketatthesource• BothprotocolssupportaPacketTooBigsignalfromtheinteriorofthenetworktothesource• OnlyIPv4routersmaygeneratefragmentson-the-fly• IPv6reliesonsupportforExtensionHeaderstosupportitsimplementationofIPpacketfragmentation• Butthathasitsownsetofimplications(Seeslide3)!

What does “Packet Too Big” mean anyway?

errrrr

It’s a Layering Problem

• FragmentationwasseenasanIPlevelproblem• Itwasmeanttobeagnosticwithrespecttotheupperlevel(transport)protocol

• Butwedon’ttreatitlikethat• Andweexpectdifferenttransportprotocolstoreacttofragmentationnotificationindifferentways

What does “Packet Too Big” mean anyway?

ForTCP itmeansthattheactivesessionreferredtointheICMPpayload*shoulddropitssessionMSStomatchtheMTU**

InanidealnetworkyoushouldneverseeIPv6fragmentsinTCP!

*assumingthatthepayloadcontainstheoriginalend-to-endIPheader**assumingthattheICMP isgenuine

What does “Packet Too Big” mean anyway?

ForUDP itsnotclear:• Theoffendingpackethasgoneaway!• SomeIPimplementationsappeartoignoreit• OthersaddahostentrytothelocalIPForwardingtablethatrecordstheMTU• OthersperformarudimentaryformofMTUreductioninalocalMTUcache

Problems

ICMPisreadilyspoofed• ICMPmessagescanconsumehostresources• AnattackermayspoofahighvolumestreamofICMPPTBmessageswithrandomIPv6sourceaddresses• AnattackermayspoofICMPPTBmessageswithverylowMTUvalues

Problems

ICMPiswidelyfiltered• leadingtoblackholesinTCPsessions

• GETisasmallHTTPpacket• Theresponsecanbearbitrarilylarge,andifthereisapathMTUmismatchtheresponsecanwedge

Get Response

Problems

LeadingtoambiguityinUDP• Isthislackofaresponseduetonetworkcongestion,routing&addressingissues,orMTUmismatch?• Shouldthereceiverjustgiveup,resendthetriggerquery,orreverttoTCP?(assumingthatitcan)

What did IPv6 do differently?

IPv6definedaminimumunfragmented packetsizeof1,280bytes:

IPv6 Specification: RFC2460

What did IPv6 do differently?

IPv6definedaminimumunfragmented packetsizeof1,280bytes:

IPv6 Specification: RFC2460

Bewteen 1280 and 1500

WhatshouldanIPv6hostuseasalocalMTUvalue?

• Ifyousetitat1280thenyouinvitefragmentationifyouneedtosendlargerpackets,whichwillriskEHlossonfragmentedpackets• Ifyousetitat1500thenyoumayencounterriskswithMTUmismatchandPTBnotificationlosswhentalkingwithahostwithasmallerMTUandencounterMTUBlackHoles

1280 1500

Lets look

• SoiftheissueisthecombinationofIPv6,UDPandlargerpacketsthenperhapswecanexperimentwiththis• It’scalled“theDNS”!

• Sowesetupanexperiment…• Response1:131octets• Response2:1400octets• Response3:1700octets

AndsetupanameserverreachableonlyonIPv6andonlyonUDP

What we expect to see

SizeFetchedFailedReason

Small(150octets) 99% 1% Noise1160octets 99% 1% Noise1400octets ? ? PTB1700octets <52% >48% EHLoss,

Fragloss,PTB

What we saw

Tested AlwaysFetched Both AlwaysMissed

150 11,719 8,792(75.02%) 377(3.22%) 2,550(21.76%)

1,160 2,004 1,353(67.51%) 5(0.25%) 646(32.24%)

1,400 9,789 7,374(75.33%) 385(3.93%) 2,030(20.74%)

1,425 1,977 1,313(66.41%) 7(0.35%) 657(33.23%)

1,453 1,987 1,298(65.32%) 9(0.45%) 680(34.22%)

1,70011,170 5,859(52.45%) 172(1.54%) 5,139(46.01%)

1280

1500

What we saw

Tested AlwaysFetched Both AlwaysMissed

150 11,719 8,792(75.02%) 377(3.22%) 2,550(21.76%)

1,160 2,004 1,353(67.51%) 5(0.25%) 646(32.24%)

1,400 9,789 7,374(75.33%) 385(3.93%) 2,030(20.74%)

1,425 1,977 1,313(66.41%) 7(0.35%) 657(33.23%)

1,453 1,987 1,298(65.32%) 9(0.45%) 680(34.22%)

1,70011,170 5,859(52.45%) 172(1.54%) 5,139(46.01%)

??

1280

1500

Thereisquitesomenoiseinthisdata– thesmall-sizeresponseshowsa21%lossrate,whichislikelytobeduetoacombinationof:

DNSmulti-slavequeryengine farmsIPv6LinkLayeraddressmanipulationICMPv6AddressunreachableDNStimeouts

Unreachables

• DualStackconfigurationshideamultitudeofsins• AndoneoftheseistheuseofunreachableIPv6addressesforDNSresolvers• 11,077distinctunreachableIPv6addresses!• Outof22,000distinctIPv6/128addresses

• Whichisnotquiteasbadasitlooks– anumberofresolversare“aggressive”intheiruseof/64interfaceidentifiers

Filtering the results

• Joinindividualresolver/128addressesintocommon/64’s• Onlylookatresolver/64’sthatfetcheitherofthetwolow-sizecontrols• WhichmeansthattheIPv6addressisreachable• Andtheresolverwillsuccessfullyresolveagluelessdelegation

What we saw:Size Tested AlwaysFetched Both AlwaysMissed

1505,4335,290(97%)143(3%)0

1,160654651(99%)3(1%)0

14004,6584,495(96%)133(3%)30(1%)

1425636619(97%) 5(1%) 12(2%)

1453638609(95%) 6(1%) 23(4%)

17004,6863,464(74%)79(1%)1,143(25%)

1280

1500

What we saw:Size Tested AlwaysFetched Both AlwaysMissed

1505,4335,290(97%)143(3%)0

1,160654651(99%)3(1%)0

14004,6584,495(96%)133(3%)30(1%)

1425636619(97%) 5(1%) 12(2%)

1453638609(95%) 6(1%) 23(4%)

17004,6863,464(74%)79(1%)1,143(25%)

1280

1500

Between 1280 and 1500 the failure rate rises as the packet size rises.

PTB MTU size distribution

1280

1500

1480

1460 1492

What we saw:Size Tested AlwaysFetched Both AlwaysMissed

1505,4335,290(97%)143(3%)0

1,160654651(99%)3(1%)0

14004,6584,495(96%)133(3%)30(1%)

1425636619(97%) 5(1%) 12(2%)

1453638609(95%) 6(1%) 23(4%)

17004,6863,464(74%)79(1%)1,143(25%)

1280

1500

There is a visible signal here for packets > 1500 octets.

It is not a 48% drop rate, but it is certainly more than 20% over and above the other packet sizes. There is a definite problem here with large IPv6 packets.

EH drop? Or something more mundane?

1,143 IPv6/64sconsistentlycannotfetcha1,700octetUDPresponse• 331/64’s generatedICMPFragmentationreassemblyICMPmessages• Firewallfrontenddiscardingtrailingfragments

• 61 /64’sgeneratedPacketTooBigmessages

• 751 failing/64’sgeneratednoICMPmessagesi.e.EHpacketdropwasamaximumof16%inthisexperiment

What we saw with a 1280 MTU:

Size Tested AlwaysFetched Both AlwaysMissed

1504,7774,600(96%)177(4%)0

14004,6623,695(79%)80(2%)887(19%)

17004,6353,429(74%)95(2%)1,111(24%)

1280

1500

Dropping the local MTU pushes a further 18% fragmentation drop into the 1,400 Byte packet

What are we seeing?

WhetheritsEHdropoffragfiltering,thereissomethingdeeplyconcerninginthesenumbers:• Aprotocolthatsuffersa~20%packetdroprateonfragmentedpacketspresentsaproblem!• HostsshouldusethelargestlocallysupportedMTUforUDP(andusea1,220MSSforTCP)• ApplicationsshouldassumethatlargeIPv6fragmentedpacketsmaysilentlydieintransit.TheyshouldbepreparedtoperformarapidcutovertoTCPintheeventofsuspectedpacketlossinUDP

• Shouldwerevivedraft-bonica-6man-frag-deprecate?

Thanks!