When the packet load is low, Suricata can run in interrupt
mode. This more resembles the classic approach of processing
packets - CPU cores run low and only fetch packets
on interrupt.
Ticket: #5839
If a user doesn't allocate/allocates too little hugepages,
Suricata fails to start and outputs a hint to increase
number of hugepages (if enabled).
Ticket: #5966
Multi-tenancy uses loader threads that initialize detection engines. During
this, esp the AC family of MPM implementations, there is significant stack
usage. In most OS' threads have a lower stack size by default. In Linux, when
using the Musl C library, a thread by default gets 128KiB.
This patch does 2 things:
1. it centralizes the handling of the `threading.stack-size`. It it is not
longer handled by the runmodes, but called from the global initialization
logic.
2. it sets a minimum per thread stack size of 512k, unless `threading.stack-size`
is set.
Ticket: #6265.
DPDK apps can specify multiple arguments of the same
type. YAML format only allows unique keys within a single
node. This commit adds support for multiple EAL arguments
of the same type to be used within suricata.yaml.
Ticket: #5964
runmode-dpdk.c:204:18: warning: Result of 'calloc' is converted to a pointer of type 'char *', which is incompatible with sizeof operand type 'ptrdiff_t' [unix.MallocSizeof]
args->argv = SCCalloc(capacity, sizeof(ptrdiff_t)); // alloc array of pointers
^~~~~~~~ ~~~~~~~~~~~~~~~~~
./util-mem.h:36:18: note: expanded from macro 'SCCalloc'
#define SCCalloc calloc
^~~~~~
runmode-dpdk.c:278:16: warning: Result of 'malloc' is converted to a pointer of type 'char *', which is incompatible with sizeof operand type 'char **' [unix.MallocSizeof]
eal_argv = SCMalloc(args.argc * sizeof(args.argv));
^~~~~~~~ ~~~~~~~~~~~~~~~~~
./util-mem.h:35:18: note: expanded from macro 'SCMalloc'
#define SCMalloc malloc
^~~~~~
2 warnings generated.
Commit contains prototype changes of RunModeSetLiveCaptureAutoFp and
RunModeSetLiveCaptureWorkers functions to move the IPS enable logic
out of suricata.c file.
Tested on Fedora 37 with clang 15.
app-layer.c:1055:27: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerSetupCounters()
^
void
app-layer.c:1176:29: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerDeSetupCounters()
^
void
2 errors generated.
src/runmode-dpdk.c:202:11: warning: Size of pointer 'argv' used instead of size of its data. This is likely to lead to a buffer overflow. You probably intend to write 'sizeof(*argv)'. [pointerSize]
args->argv = SCCalloc(capacity, sizeof(args->argv));
^
src/runmode-dpdk.c:777:23: error: Shifting 32-bit value by 63 bits is undefined behaviour [shiftTooManyBits]
if (bits & (1 << i))
^
src/runmode-dpdk.c:776:23: note: Assuming that condition 'i<64' is not redundant
for (int i = 0; i < 64; i++) {
^
src/runmode-dpdk.c:777:23: note: Shift
if (bits & (1 << i))
^
Each thread had its own version of the `machine_start_time`, which
lead to slight time differences. This became apparent mostly in IPS,
where 2 threads each process a side of the flow.
This patch makes the `machine_start_time` global.
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
Set RSS hash function according to Intel ICE PMD available hash functions
Set hash functions according to the support by the ICE PMD, so that no warning
regarding RSS setting is issued.
Set RSS hash function according to Intel IXGBE PMD available hash functions.
During configuration, a warning appeared stating that RSS hash function
has been changed from one value to the other. This has meant that
the supported hash functions did not cover all required hash functions
by the configuration. This commit solves the warning.
Due to peculiar behavior of i40e PMD driver, the RSS is required to be set
via rte_flow rules or a hash filter as compared to other NICs where RSS is
configured through port configuration structure.
RTE_FLOW rules are created on 5-tuples (as opposed to 3-tuple configured
on the other NICs). Fragmented traffic have been tested with this setup
and it has been proven that fragmented packets of the same flow are
received on the same queue. At the same time, setting 3-tuple on rte_flow
rules have not yield in the expected results.
Notes from the experiments:
- Configuration of 5-tuple (as is in the commit):
fragmented and nonfragmented packets are received by the same workers
even when I applied seed to alter them via tcpreplay-edit (option --seed)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_IPV4 (i.e. setting 3-tuple):
when setting ETH_RSS_IPV4, the PMD driver says that pctype is not
supported (generally this means that the "type" of traffic is not
a valid configuration for the i40e)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_NONFRAG_IPV4_OTHER:
this doesn't work well, packets of the same flow are received on
the different workers (my explanation is that the fragmented packets are
matched with ETH_RSS_FRAG_IPV4 but the other UDP packets are not matched
with ETH_RSS_NONFRAG_IPV4_OTHER rte_flow rule (they would be matched with
ETH_RSS_NONFRAG_IPV4_UDP).
Register a new runmode - DPDK. This enables a new flag on Suricata start
(--dpdk).
With the flag given, DPDK runmode is enabled.
Runmode loads the configuration and then initializes EAL.
If successful, it configures the physical NICs according to the configuration
file. After that, worker threads are initialized and then are in continuous
receive loop.
Lukas Sismis
Victor Julien
liaozhiyuan
Philippe Antoine
Lukas Sismis