There is currently a build failure with macos-latest (recently updated)
to 11 in the libhtp test suite code. Not sure if there are other
build issues in libhtp or Suricata at this time.
(cherry picked from commit 8b9721b265)
To take TCP window into account
And to actually bail out if we received too much data
where the limit is configured by stream.reassembly.depth
(cherry picked from commit f77b027ada)
For size limit checks consider only available data at the stream start
and before any GAPS.
The old check would consider too much data if there were temporary gaps,
like when a data packet was in-window but (far) ahead of the expected
segment.
(cherry picked from commit 7a114e506a)
When tracking data track the size of the blocks so that in case
of gaps we can still know how much data we hold.
(cherry picked from commit be1baa8cab)
The flows exceeding the spare pools config setting would be freed
per at max 100 flows a second. After a high speed test this would
lead to excessive memory use for a long time.
This patch updates the logic to free 10% of the excess flows per
run, freeing multiple blocks of flows as needed.
Bug: #4731.
(cherry picked from commit fa72a5add8)
In a scenario where there was suddenly no more traffic flowing, flows
in a threads `flow_queue` would not be processed. The easiest way to
see this would be in a traffic replay scenario. After the replay is done
no more packets come in and these evicted flows got stuck.
In workers mode, the capture part handles timeout this was updated to
take the `ThreadVars::flow_queue` into account.
The autofp mode the logic that puts a flow into a threads `flow_queue`
would already wake a thread up, but the `flow_queue` was then ignored.
This has been updated to take the `flow_queue` into account.
In both cases a "capture timeout" packet is pushed through the pipeline
to "flush" the queues.
Bug: #4722.
(cherry picked from commit b788d3345c)
Fix FNs in case of too many prefilter engines. A transaction was tracking
which engines have run using a u64 bit array. The engines 'local_id' was
used to set and check this bit. However the bit checking code didn't
handle int types correctly, leading to an incorrect left shift result of
a u32 to a u64 bit value.
This commit addresses that by fixing the int handling, but also by
changing how the engines are tracked.
To avoid wasting prefilter engine tracking bit space, track what
ran by the progress they are registered at, instead of the individual
engine id's. While we can have many engines, the protocols use far
fewer unique progress values. So instead of tracking for dozens of
prefilter id's, we track for the handful of progress values.
To allow for this the engine array is sorted by tx_min_progress, then
app_proto and finally local_id. A new field is added to "know" when
the last relevant engine for a progress value is reached, so that we
can set the prefilter bit then.
A consquence is that the progress values have a ceiling now that
needs to fit in a 64 bit bitarray. The values used by parsers currently
does not exceed 5, so that seems to be ok.
Bug: #4685.
(cherry picked from commit 932cf0b6a6)
Unify handling of signature matches between various rule types and
between noalert and regular rules.
"noalert" sigs are added to the alert queue initially, but removed
from it after handling their actions. This way all actions are applied
from a single place.
Make sure flow drop and pass are mutually exclusive.
The above addresses issue with pass and drops not getting applied
correctly in various cases.
Bug: #4663
Bug: #4670
(cherry picked from commit aa93984b7e)
LuaCallbackStatsPushToStackFromFlow tuple is composed of integer values
not all of them had been converted to lua_pushinteger yet.
(cherry picked from commit 8b53468d32)
The callback for FileInfo was returning the wrong value, resulting
in loss of some tuple values for one calling SCFileInfo in a script.
The documentation comment wasn't mentioning the sha items that are
pushed.
(cherry picked from commit 1315cb793b)
If an HTTP2 file was within only ont DATA frame, the filetracker
would open it and close it in the same call, preventing the
firther call to incr_files_opened
Also includes rustfmt again for all HTTP2 files
(cherry picked from commit bb98a18b3d)
After a file has been closed (CLOSE, COMMIT command or EOF/SYNC part of
READ/WRITE data block) mark it as such so that new file commands on that
file do not reuse the transaction.
When a file transfer is completed it will be flagged as such and not be
found anymore by the NFSState::get_file_tx_by_handle() method. This forces
a new transaction to be created.
(cherry picked from commit 67759795c6)
Storing too early can lead to files being considered TRUNCATED if the
TCP state is not yet CLOSED when logging is triggered. This has been
observed with FTP-DATA and might also be an issue with simple HTTP.
(cherry picked from commit ca124b033e)
Philippe Antoine
Shivani Bhardwaj
Victor Julien
Jason Ish
Juliana Fajardini