Suricata Performance with a S like Security · Filters are injected by userspace in kernel via syscall Extended BPF Extended virtual machine: more operators, data and function

Suricata Performance with a S like Security

É. Leblond

Stamus Networks

July. 03, 2018

É. Leblond (Stamus Networks) Suricata Performance with a S like Security July. 03, 2018 1 / 31

1 IntroductionFeaturesReconstruction work

2 ProblemPacket loss impactElephant flowWork less to get more

3 BypassIntroducing bypassBypass strategy

4 Hipster technologies to the rescueeBPFAF_PACKET bypass via eBPFXDP support

5 Conclusion






5 Conclusion


What it is not ?

https://twitter.com/randomuserid/status/1012474246503845888



A signature based IDS

From individual traffic todetection

Get packet per packetReconstruct to applicationlayerRun detection engine

IdentityGPLv2owned by OISF (non for profitfoundation)Scalability via multithreadingWritten in C and Rust

Example signature


Suricata (Bro) NSM features

Supported protocolsProtocol analysis: http, ftp, smtp, tls, ssh smb, dcerpc, dns, nfs,ntp, ftp-data, tftp, ikev2, krb5, dhcpProtocol recognition: imap, msn

Log example


What it is ? or how to please developers




File related features

File analysisMagic computation and in file data matchChecksum computation and file extraction to diskSupported protocols: http, smtp, smb, ftp, nfs

Fileinfo example






5 Conclusion


Suricata reconstruction and normalization






5 Conclusion






5 Conclusion


Impact of loosing packets

MethodologyUse a sample trafficModify the pcap file to have specified random packet lossDo it 3 times par packet lossGet graph out of that

Test dataUsing a test pcap of 445Mo.Real traffic but lot of malicious behaviorsTraffic is a bit old


Alert loss by packet loss

Some numbers10% missed alerts with 3% packets loss50% missed alerts with 25% packets loss


The case of file extraction

Some numbers10% failed file extraction with 0.4% packets loss50% failed file extraction with 5.5% packets loss






5 Conclusion


The elephant flow problem (1/2)







Ring buffer overrunLimited sized ring bufferOverrun cause packets lossthat cause streaming malfunction

Ring size increaseWork aroundUse memoryFail for non burst

Dequeue at NQueue at speed N+M






5 Conclusion


Stream depth method

Attacks characteristicIn most cases attack is done at start of TCP sessionGeneration of requests prior to attack is not commonMultiple requests are often not even possible on same TCPsession

Stream reassembly depthReassembly is done till stream.reassembly.depth bytes.Stream is not analyzed once limit is reachedIndividual packet continue to be inspected






5 Conclusion






5 Conclusion


Introducing bypass

Stop packet handling as soon as possibleTag flow as bypassedMaintain table of bypassed flowsDiscard packet if part of a bypassed flow

Bypass methodLocal bypass: Suricata discard packet after decodingCapture bypass: capture method maintain flow table and discardpackets of bypassed flows


Bypassing big flow: local bypass


Bypassing big flow: capture bypass


Bypassing big flow: capture bypass


Implementation

Suricata updateAdd callback functionCapture method register itself and provide a callbackSuricata calls callback when it wants to offload

NFQ bypass in Suricata 3.2Update capture register functionWritten callback function

Set a mark with respect to a mask on packetMark is set on packet when issuing the verdict


Implementation

Suricata updateAdd callback functionCapture method register itself and provide a callbackSuricata calls callback when it wants to offload

NFQ bypass in Suricata 3.2Update capture register functionWritten callback function

Set a mark with respect to a mask on packetMark is set on packet when issuing the verdict






5 Conclusion


Stream depth bypass

Stop all treatment after bypassGo beyond what is currently doneDisable individual packet treatment once stream depth is reached

Activating stream depth bypassSet stream.bypass to yes in YAML

TLS bypassencrypt-handling: bypass


Selective bypass

Ignore some trafficIgnore intensive traffic like NetflixCan be done independently of stream depthCan be done using generic or custom signatures

The bypass keywordA new bypass signature keywordTrigger bypass when signature matchExample of signature

pass h t t p any any −> any any ( content : " s u r i c a t a . i o " ; \ \h t tp_hos t ; bypass ; s id :6666; rev : 1 ; )


Selective bypass

Ignore some trafficIgnore intensive traffic like NetflixCan be done independently of stream depthCan be done using generic or custom signatures

The bypass keywordA new bypass signature keywordTrigger bypass when signature matchExample of signature

pass h t t p any any −> any any ( content : " s u r i c a t a . i o " ; \ \h t tp_hos t ; bypass ; s id :6666; rev : 1 ; )






5 Conclusion






5 Conclusion


Extended Berkeley Packet Filter

Berkeley Packet FilterVirtual machine inside kernelArithmetic operations and tests on the packet dataFilters are injected by userspace in kernel via syscall

Extended BPFExtended virtual machine: more operators, data and functionaccessVarious attachment points

SocketSyscallTraffic control

Kernel and userspace shared structuresHash tablesArrays


LLVM backend

From C file to eBPF codeWrite C codeUse eBPF LLVM backend (since LLVM 3.7)Use libbpf

Get ELF fileExtract and load section in kernel

BCC: BPF Compiler collectionInject eBPF into kernel from high level scripting languageTrace syscalls and kernel functionshttps://github.com/iovisor/bcc


https://github.com/iovisor/bcc





5 Conclusion


And now AF_PACKET

What’s neededSuricata to tell kernel to ignore flowsKernel system able to

Maintain a list of flow entriesDiscard packets belonging to flows in the listUpdate from userspace

eBPF filter using mapseBPF introduce mapsDifferent data structures

Hash, array, . . .Update and fetch from userspace

Looks good!


And now AF_PACKET

What’s neededSuricata to tell kernel to ignore flowsKernel system able to

Maintain a list of flow entriesDiscard packets belonging to flows in the listUpdate from userspace

eBPF filter using mapseBPF introduce mapsDifferent data structures

Hash, array, . . .Update and fetch from userspace

Looks good!


Test methodology

Test setupIntel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHzIntel Corporation 82599ES 10-Gigabit SFI/SFP+Live traffic:

Around 1Gbps to 2GbpsReal users so not reproducible

TestsOne hour long runDifferent stream depth valuesCollected Suricata statistics counters (JSON export)Graphs done via Timelion(https://www.elastic.co/blog/timelion-timeline)


https://www.elastic.co/blog/timelion-timeline

Results: stream bypass at 512kb


A few words on graphics

Tests at 512kbWe have on big flow thatkill the bandwidthCapture get almost nullEven number of closedbypassed flows is low


Results






5 Conclusion


A Linux kernel feature

Run a eBPF code the earliest possiblein the driverin the cardbefore the regular kernel path

Act on dataDrop packet (eXtreme Drop Performance)Transmit to kernelRewrite and transmit packet to kernelRedirect to another interfaceCPU load balance


Implementation

Similar to eBPF filterSame logic for bypassOnly verdict logic is different

But annoying differenceeBPF code does the parsingNeed to bind to an interface


Results

TODO: Ask OISF marketing for some fake numbers to show






5 Conclusion


Conclusion

Suricata, eBPF and XDPA fresh but interesting methodNetwork card bypass for Netronome comingAF_XDP capture is now in Linux vanilla

More informationStamus Networks: https://www.stamus-networks.com/Septun II: https://github.com/pevma/SEPTun-Mark-II/Suricata doc: http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html


https://www.stamus-networks.com/

https://github.com/pevma/SEPTun-Mark-II/

http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html

http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html

Questions ?

Thanks toJesper Dangaard BrouerAlexei StorovoitovDaniel Borkmann

Contact [email protected]: @regiteric

Want more fun ?Come to Suricata and SELKSworkshop !Suricon:https://suricon.net/


https://suricon.net/

Documents

Suricata Performance with a S like Security · Filters are injected by userspace in kernel via syscall Extended BPF Extended virtual machine: more operators, data and function