Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.
This patch reimplements this logic by using a new return code instead.
Split content inspection into public and private version, so that
common initialization can be done in a single place.
Update the callsites.
Signal threads while holding lock. This should make the signalling
more reliable.
From PTHREAD_COND(3):
"Unlocking the mutex and suspending on the condition variable is done
atomically. Thus, if all threads always acquire the mutex before
signaling the condition, this guarantees that the condition cannot be
signaled (and thus ignored) between the time a thread locks the
mutex and the time it waits on the condition variable."
Ticket: #6569.
Problem:
In pcap autofp mode, there is one threads reading packets (RX). These packets
are then passed on to worker threads. When these workers are done with a
packet, they return packets to the pcap reader threads packet pool, which is
the owner of the packets. Since this requires expensive synchronization between
threads, there is logic in place to batch this operation.
When the reader thread depletes its pool, it notifies the other threads that
it is starving and that a sync needs to happen asap. Then the reader enters
a wait state. During this time no new packets are read.
However, there is a problem with this approach. When the reader encountered
an empty pool, it would set an atomic flag that it needed a sync. The first
worker to return a packet to the pool would then set this flag, sync, and
unset the flag. This forced sync could result in just a single packet being
synchronized, or several. So if unlucky, the reader would just get a single
packet before hitting the same condition again.
Solution:
This patch updates the logic to use a new approach. Instead of using a
binary flag approach where the behavior only changes when the reader is
already starved, it uses a dynamic sync threshold that is controlled by
the reader. The reader keeps a running count of packets it its pool,
and calculates the percentage of available packets. This percentage is
then used to set the sync threshold.
When the pool is starved, it sets the threshold to 1 (sync for each packet).
After each successful get/sync the threshold is adjusted.
Ticket: #6104
And failures should be handled to say that the rule failed to load
Reverts the fix by 299ee6ed55
that was simple, but not complete (memory leak),
to have this bigger API change which simplifies code.
Especially fix setup-app-layer script to not forget this part
This allows, for simple loggers, to have a unique definition
of the actual logging function with the jsonbuilder.
This way, alerts, files, and app-layer event can share the code
to output the same data.
Ticket: #3827
To mitigate a bug with AF_XDP sockets in high traffic scenarios, the XDP program must be detatched before
the sockets are closed. This issue happens when large ammounts of traffic are sent to suricata and
the XDP program is not removed before AF_XDP sockets are closed. I believe this is a race
condition bug as detailed here: https://bugzilla.kernel.org/show_bug.cgi?id=217712
Further investigation shows this may be a bug exclusive to the driver/AMD processor combination.
This commit addresses the bug by ensuring the first thread to run the deinit function
removes the XDP program, which fixes the bug as detailed in the bugzilla link.
Bug #6238
The hot reload results in large chunks of memory being freed as the
as the old signature tables are discarded. Help the memory management
system along by telling to release as much memory as it can at this
point.
Bug: #6454.
This is an example of what adding plugin examples to the Suricata repo
could look like.
This plugin is an example plugin for an EVE filetype. It could be
extended to support outputs like Redis, syslog, etc.
There is one issue with adding plugins like this to an autotools
project, the project can't be built with --disable-shared, which is
more of an autotools limitation, and not really a Suricata issue.
Suricata built with --disable-shared will load plugins just fine.
Note that the examples directory was added as DIST_SUBDIRS as we don't
want normal builds to recurse into it and attempt to build the plugin,
its just an example, but we still need to keep distcheck happy.
Our tls fields not_after and not_before are actually logged as
`notafter` and `notbefore`, but were documented with the underscore.
Update the documentation, since updating the log format itself would be
a breaking change.
Task #5494
Victor Julien
Jason Ish
Philippe Antoine
Jeff Lucovsky
Philippe Antoine
Joseph Reilly
Eric Leblond
jason taylor
Thomas Winter
Juliana Fajardini