This change makes it possible to generate the eBPF programs even if
Suricata itself is built a different C compiler. It also simplifies
how the correct llc program is detected.
Implements Feature https://redmine.openinfosecfoundation.org/issues/2789
As pointed by Victor Julien, it is not a good idea to use the
nexthdr value, as init key for the hash as it could contain some
other headers and can be changed for a session.
If vlan is not use for tracking in Suricata this result in vlan not
being used in the flow key in Suricata and we need to adjust that
in the XDP filter to avoid any problem.
Tests have shown that when we bypass encrypted traffic a non
neglicteable amount of encrypted packets of the session are already in the
capture ring buffer. Result is that Suricata is doing unnecessary
work on these packets.
These packets can be identified via the first bytes of their payloads
so we can bypass them directly in the XDP code. This is done here
for application data packets on port 443 and for TLS 1.2.
Add a switch to allow to bypass all traffic if the switch is on.
Concept is to use a persistant script and pinned maps, so an
external tool can be used to trigger global bypass in case Suricata
is dead.
Kernel time is not available (and/or costly) on NIC such as
Netronome so we update the logic to detect dead flows based on a
lack of update of packets counters. This way, the XDP filter will
be usable by network card.
This patch also updates the ebpf code to support per CPU and
regular mapping. Netronome is not supporting it and the structure
is using atomic for counter so the cost of simultaneous update
is really low.
This patch also updates the xdp_filter to be able to select if the
flow table is per CPU on shared. Second option will be used for
hardward offload. To deactivate the per cpu hash, you need to set
USE_PERCPU_HASH to 0.
This patch also adds an new option to af-packet named no-percpu-hash
If this option is set to yes then the Flow bypassed manager thread
will use one CPU instead of the number of cores. By doing that
we are able to handle the case where USE_PERCPU_HASH is unset (so
hardware offload for Netronome).
This patch also remove aligment indications in the eBPF filter. This
was not really needed and it seems it is causing problem with
some recent version of LLVM toolchain.
The current ebpf/Makefile.am have the problem that clang compile
errors still result in an ELF .bpf output file. This is obviously
problematic as the problem is first seen runtime when loading
the bpf-prog. This is caused by the uses of a pipe from
clang to llc.
To address this problem, split up the clang and llc invocations
up into two separate commands, to get proper reaction based on
the compiler exit code. The clang compiler is used as a
frontend (+ optimizer) and instructed (via -S -emit-llvm) to
generate LLVM IR (Intermediate Representation) with suffix .ll.
The LLVM llc command is used as a compiler backend taking IR and
producing BPF machine bytecode, and storing this into a ELF
object. In the last step the IR .ll suffix code it removed.
The official documentation of the IR language:
http://llvm.org/docs/LangRef.html
Also fix the previous make portability warning:
'%-style pattern rules are a GNU make extension'
I instead use some static pattern rules:
https://www.gnu.org/software/make/manual/html_node/Static-Usage.html
Signed-off-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
Enable compiling eBPF programs with clang -target bpf.
This is mostly to workaround a bug in libbpf, where clang > ver 4.0.0
generates some ELF sections (.eh_frame) when -target bpf is NOT specified,
and libbpf fails loading such files.
Notice libbpf is provided by the kernel, and in kernel v4.16 the library
will contain the needed function for attaching to the XDP hook.
Kernel commit 949abbe88436 ("libbpf: add function to setup XDP")
https://git.kernel.org/torvalds/c/949abbe88436
The library fix has reached kernel v4.16 but the workaround for Suricata
is interesting anyway in case people use a kernel v4.15.
Signed-off-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
This patch prepares code before enabling the clang -target bpf.
The clang compiler does not like #include <stdint.h> when
using '-target bpf' it will fail with:
fatal error: 'gnu/stubs-32.h' file not found
This is because using clang -target bpf, then clang will have '__bpf__'
defined instead of '__x86_64__' hence the gnu/stubs-32.h include
attempt as /usr/include/gnu/stubs.h contains, on x86_64:
#if !defined __x86_64__
# include <gnu/stubs-32.h>
#endif
#if defined __x86_64__ && defined __LP64__
# include <gnu/stubs-64.h>
#endif
#if defined __x86_64__ && defined __ILP32__
# include <gnu/stubs-x32.h>
#endif
This can be worked around by installing the 32-bit version of
glibc-devel.i686 on your distribution.
But the BPF programs does not really need to include stdint.h,
if converting:
uint64_t -> __u64
uint32_t -> __u32
uint16_t -> __u16
uint8_t -> __u8
This patch does this type syntax conversion.
The build of a ebpf files had an issue for system like Debian
because they don't have a asm/types.h in the include path if the
architecture is not defined which is the case due to target bpf.
This results in:
clang-5.0 -Wall -Iinclude -O2 \
-D__KERNEL__ -D__ASM_SYSREG_H \
-target bpf -S -emit-llvm vlan_filter.c -o vlan_filter.ll
In file included from vlan_filter.c:19:
In file included from include/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not
found
#include <asm/types.h>
^~~~~~~~~~~~~
1 error generated.
Makefile:523: recipe for target 'vlan_filter.bpf' failed
This patch fixes the issue by adding a include path setting the
architecture to the one of the builder.
Signed-off-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
Sidned-off-by: Eric Leblond <eric@regit.org>
The XDP CPU destination array/set, configured via xdp-cpu-redirect,
will always be fairly small. My different benchmarking showed that
the current modulo hashing into the CPU array can easily result in bad
distribution, expecially if the number of CPU is an even number.
This patch uses a proper hashing function on the input key. The key
used for hashing is inspired by the ippair hashing code in
src/tmqh-flow.c, and is based on the IP src + dst.
An important property is that the hashing is flow symmetric, meaning
that if the source and destintation gets swapped then the selected CPU
will remain the same. This is important for Suricate.
That hashing INITVAL (15485863 the 10^6th prime number) was fairly
arbitrary choosen, but experiments with kernel tree pktgen scripts
(pktgen_sample04_many_flows.sh +pktgen_sample05_flow_per_thread.sh)
showed this improved the distribution.
Signed-off-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
Adjusted function call API to take an initval. This allow the API
user to set the initial value, as a seed. This could also be used for
inputting the previous hash.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This patch adds a boolean option "xdp-cpu-redirect" to af-packet
interface configuration. If set, then the XDP filter will load
balance the skb creation on specified CPUs instead of doing the
creation on the CPU handling the packet. In the case of a card
with asymetric hashing this will allow to avoid saturating the
single CPU handling the trafic.
The XDP filter must contains a set of map allowing load balancing.
This is the case of xdp_filter.bpf.
Fixed-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
eBPF has a data type which is a per CPU array. By adding one element
to the array it is in fact added to all per CPU arrays in the kernel.
This allows to have a lockless structure in the kernel even when doing
counter update.
In userspace, we need to update the flow bypass code to fetch all
elements of the per CPU arrays.
Eric Leblond
Hilko Bengen
Jesper Dangaard Brouer
Jesper Dangaard Brouer