Why eBPF and XDP in Suricata matters · 2019. 1. 9. · Leblond (OISF) Why eBPF and XDP in Suricata matters Nov. 15, 20181/31. 1Problem Packet loss impact Elephant ﬂow Work less

Why eBPF and XDP in Suricata matters

É. Leblond

OISF

Nov. 15, 2018

É. Leblond (OISF) Why eBPF and XDP in Suricata matters Nov. 15, 2018 1 / 31

1 ProblemPacket loss impactElephant flowWork less to get more

2 BypassIntroducing bypassBypass strategy

3 Hipster technologies to the rescueeBPFAF_PACKET bypass via eBPFeBPF supportXDP supportIt’s just the beginning

4 Conclusion





4 Conclusion


Impact of loosing packets

MethodologyUse a sample trafficModify the pcap file to have specified random packet lossDo it 3 times par packet lossGet graph out of that

Test dataUsing a test pcap of 445Mo.Real traffic but lot of malicious behaviorsTraffic is a bit old


Alert loss by packet loss

Some numbers10% missed alerts with 3% packets loss50% missed alerts with 25% packets loss


The case of file extraction

Some numbers10% failed file extraction with 0.4% packets loss50% failed file extraction with 5.5% packets loss





4 Conclusion


The elephant flow problem (1/2)


The elephant flow problem (2/2)

Ring buffer overrunLimited sized ring bufferOverrun cause packets lossthat cause streaming malfunction

Ring size increaseWork aroundUse memoryFail for non burst

Dequeue at NQueue at speed N+M





4 Conclusion


Stream depth method

Attacks characteristicIn most cases attack is done at start of TCP sessionGeneration of requests prior to attack is not commonMultiple requests are often not even possible on same TCPsession

Stream reassembly depthReassembly is done till stream.reassembly.depth bytes.Stream is not analyzed once limit is reachedIndividual packet continue to be inspected





4 Conclusion





4 Conclusion


Introducing bypass

Stop packet handling as soon as possibleTag flow as bypassedMaintain table of bypassed flowsDiscard packet if part of a bypassed flow

Bypass methodLocal bypass: Suricata discard packet after decodingCapture bypass: capture method maintain flow table and discardpackets of bypassed flows


Bypassing big flow: local bypass


Bypassing big flow: capture bypass


Implementation

Suricata updateAdd callback functionCapture method register itself and provide a callbackSuricata calls callback when it wants to offload

NFQ bypass in Suricata 3.2Update capture register functionWritten callback function

Set a mark with respect to a mask on packetMark is set on packet when issuing the verdict





4 Conclusion


Stream depth bypass

Stop all treatment after bypassGo beyond what is currently doneDisable individual packet treatment once stream depth is reached

Activating stream depth bypassSet stream.bypass to yes in YAML

TLS bypassencrypt-handling: bypass


Selective bypass

Ignore some trafficIgnore intensive traffic like NetflixCan be done independently of stream depthCan be done using generic or custom signatures

The bypass keywordA new bypass signature keywordTrigger bypass when signature matchExample of signature

pass h t t p any any −> any any ( content : " s u r i c a t a . i o " ; \ \h t tp_hos t ; bypass ; s id :6666; rev : 1 ; )





4 Conclusion





4 Conclusion


Extended Berkeley Packet Filter

Berkeley Packet FilterVirtual machine inside kernelArithmetic operations and tests on the packet dataFilters are injected by userspace in kernel via syscall

Extended BPFExtended virtual machine: more operators, data and functionaccessVarious attachment points

SocketSyscallTraffic control

Kernel and userspace shared structuresHash tablesArrays


LLVM backend

From C file to eBPF codeWrite C codeUse eBPF LLVM backend (since LLVM 3.7)Use libbpf

Get ELF fileExtract and load section in kernel

BCC: BPF Compiler collectionInject eBPF into kernel from high level scripting languageTrace syscalls and kernel functionshttps://github.com/iovisor/bcc


https://github.com/iovisor/bcc




4 Conclusion


And now AF_PACKET

What’s neededSuricata to tell kernel to ignore flowsKernel system able to

Maintain a list of flow entriesDiscard packets belonging to flows in the listUpdate from userspace

eBPF filter using mapseBPF introduce mapsDifferent data structures

Hash, array, . . .Update and fetch from userspace

Looks good!





4 Conclusion


Filtering

from BPF to eBPFForget about or joined list: not (1.2.3.4 or 2.3.4.5 or12.3.34.4 or ...)

Maintain list in mapsSearch in list in constant time

More on mapsPinningAccess from external tool

Available example filtersfilter.c: drop IPv6vlan_filter.c: accept packet for a set of VLANs


Pinned maps

Expose maps to systemRead and update map from external toolsUpdate BPF filter dynamically

DemoOn the wings of Murphy


Murphy will decide if I need to pass this slide fast


Load balancing

Custom load balancerReturn integerReadig socket determined by taking modulo

Available example filterlb.c: IP pair load balancing


Bypass

eBPF bypassSuricata specialized filterFlow tables for IPv4 and IPv6Bypass function add entry to flow table

Flow handlingDedicated thread in SuricataDump table and handle cleaning





4 Conclusion


eXtreme Data Path

Reaching bare metal performanceAnswer to high performance need

DDoS fightCustom protocol implementation

Run userspace codeWhen Linux network stack do too much

MotivationAvoid cost of skb creation"Kill" DPDK

Universal solution and APIsAvoid non Linux application on Linux


A recent Linux kernel feature

Run a eBPF code the earliest possiblein the driverin the cardbefore the regular kernel path

Act on dataDrop packet (eXtreme Drop Performance)Transmit to kernelRewrite and transmit packet to kernelRedirect to another interfaceCPU load balance


Implementation in Suricata

Similar to eBPF filterSame logic for bypassOnly verdict logic is different

But annoying differenceeBPF code does the parsingNeed to bind to an interface


IPS and bypass

What about IPS bypass ?XDP_DROP is droppingBypassing imply dropping

To light speed and beyondXDP_REDIRECT to send packet to TX queue of other NICDirect transmit from hardware to hardware


CPU redirect

Non symetric RSSNon symetric hash functionLow entropy key not always supportedRSS=1 and burn one CPU

CPU Redirect to the rescueLoad balance in XDP eBPF codeskb creation is done in all CPUs


Stripping tunnels

Big TunnelCan be an elephant flowTunnelized flows can be non elephantTreating ad load balancing on internal flows can save the day

Strip tunnel headerDecode tunnel headerFind offsetMove pointer to new start





4 Conclusion


Complete hardware offload

Join work with Netronome teamAlmost thereTest to start soon


AF_XDP

New capture methodGet packet at XDP stageFully skip the Linux network stack

New architectureShared memoryUser and Kernel lists





4 Conclusion


Conclusion

Suricata, eBPF and XDPAvailable in Suricata 4.1, need Linux 4.16Network card bypass for Netronome comingAF_XDP capture is now in Linux vanilla

More informationSeptun II: https://github.com/pevma/SEPTun-Mark-II/Suricata doc: http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html


https://github.com/pevma/SEPTun-Mark-II/

http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html

http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html

Questions ?

Thanks toJesper Dangaard BrouerAlexei StarovoitovDaniel Borkmann

Contact [email protected]: @regiteric


Documents

Why eBPF and XDP in Suricata matters · 2019. 1. 9. · Leblond (OISF) Why eBPF and XDP in Suricata matters Nov. 15, 20181/31. 1Problem Packet loss impact Elephant ﬂow Work less