In certain conditions, it can take a long time for threads to start up.
For example in af-packet, setting up the socket, rings, etc has been
observed to take close to half a second per thread, and since the
threads go one by one in a preset order, this means the start up can
take a lot of time if there are many threads. The old logic would just
allow a hard coded 60s. This was not always enough when the number of
threads was high.
This patch makes the wait time take the number of threads into account.
It adds a second of time budget to the base 60s for each thread.
So as an example, if a system has 112 af-packet threads, it would wait
172 seconds (60 + 112) for the threads to get ready.
Ticket: #7048.
When starting a large amount of threads, the loop was inefficient. It
would loop over the threads and if one wasn't yet ready it would sleep a
bit and then reevaluate all the threads. This reevaluation of threads
already checked was inefficient, and could lead to the time budget
running out.
This patch splits the check, and keeps track of the threads that have
already passed. This avoids the rescanning of already checked threads.
Update the Lua allocated to set a code on memory allocation limit
exceeded errors so an appropriate error message can be logged and a
state incremented.
Fixes the tracking of the allocated size by using the difference
between original size, and new size and toss in some debug
validations.
Distinguish between a generic Lua script error and an error created by a
function being blocked, so each is logged once respective of each other.
Also add a stat that is incremented when a script fails due to a
blocked function.
NOTE: This does not catch calls to functions that are blocked by not
having the library loaded, such as "io.open", as they are blocked by
not even loading the "io" library.
The Lua library surface area is small enough to manage an allow list,
which is generally better than a deny list, as we'll explicitly need
to opt-in to new functions provided by the Lua runtime.
Remove maintainer-clean-local, this is not needed.
In distclean-local, remove "rust/dist" and "rust/vendor" as they are
created during "make dist".
In "clean-local", remove "rust/target" and "rust/gen" as they are
created during a normal "make".
Modify the CentOS 9 Stream build to not have cbdingen available, as
its already building from the dist. But add a "make clean" followed
by a "make" to test that it still builds after a clean.
Move prototypes for functions that exist in util-port-interval-tree.c
from detect-engine-port.h to util-port-interval-tree.h.
Fix header guard names while there.
Ticket: 6575
Multi buffers keywords now use a single registration function
DetectAppLayerMultiRegister with a GetBuffer argument.
This GetBuffer function pointer is similar to the ones used by
single-buffer keyword, except that it takes an additional
parameter which is the index of the buffer to get.
Under the hood, an anonymous union between these 2 functions
pointers types is used.
In the end, this deduplicates code, especially the calls to
DetectEngineContentInspection
Bug: https://redmine.openinfosecfoundation.org/issues/6782
Callers to these allocators often use ``sc_errno`` to provide context of
the error. And in the case of the above bug, they return ``sc_errno``,
but as it has not been set ``sc_errno = 0; == SC_OK``.
This patch simply sets this variable to ensure there is context provided
upon error.
The on-disk pcap pkthdr is 16 bytes. This was calculated using
`sizeof(struct pcap_pkthdr)`, which is 24 bytes on 64 bit Linux. On
Macos, it's even worse, as a comment field grows the struct to 280
bytes.
Address this by hardcoding the value of 16.
Bug: #7037.
In offline mode, a timestamp is kept per thread, and the lowest
timestamp of the active threads is used. This was also considering the
non-packet threads, which could lead to the used timestamp being further
behind that needed. This would happen at the start of the program, as
the non-packet threads were set up the same way as the packet threads.
This patch both no longer sets up the timestamp for non-packet threads
as well as not considering non-packet threads during timestamp
retrieval.
Fixes: 6f560144c1 ("time: improve offline time handling")
Bug: #7034.
Victor Julien
Jason Ish
Jo Johnson
jason taylor
Philippe Antoine
Richard McConnell
Shivani Bhardwaj