If the next PDU is already in the slice next, do not use it and
restrict ourselves to the length of this PDU.
Avoids overconsumption of memory by quadratic complexity, when
having many small PDUS in one big chunk being parsed
Ticket: #6411
Ticket: #6441
This keyword and the response one use a multiple inspection buffer.
But the different instances point to the same memory address
that comes from HttpHeaderGetBufferSpace and is not owned
by the transaction, and is rebuilt, which is a functional
bug in itself.
As it gets crafted, it can get reallocated if one header
is over 1024 bytes, while the previous freed pointer will still get
used for the previous headers.
Ticket: 6477
So as to avoid ending up with too many empty transactions.
This happens when Suricata sees a DATA command in the current
transaction but did not have a confirmation response for it.
Then, if Suricata receives another DATA command, it will
create another new transaction, even if the previous one
is empty. And so, a malicious client can create many empty
transactions by just sending a repeated amount of DATA commands
without having a confirmation code for them.
Suricata cannot use state->current_command == SMTP_COMMAND_DATA
to prevent this attack and needs to resort to a new boolean
is_data because the malicious client may send another dummy command
after each DATA command.
This patch leaves only one call to SMTPTransactionCreate
Ticket: 5926
HTTP2 continuation frames are defined in RFC 9113.
They allow header blocks to be split over multiple HTTP2 frames.
For Suricata to process correctly these header blocks, it
must do the reassembly of the payload of these HTTP2 frames.
Otherwise, we get incomplete decoding for headers names and/or
values while decoding a single frame.
Design is to add a field to the HTTP2 state, as the RFC states that
these continuation frames form a discrete unit :
> Field blocks MUST be transmitted as a contiguous sequence of frames,
> with no interleaved frames of any other type or from any other stream.
So, we do not have to duplicate this reassembly field per stream id.
Another design choice is to wait for the reassembly to be complete
before doing any decoding, to avoid quadratic complexity on partially
decoding of the data.
Present scenario
----------------
Currently, as a part of setting signature count per SGH, a max_idx is
passed which could be as high as the highest signature number (internal
ID).
Issue
-----
Not every SGH needs to evaluate all the signatures while setting
the signature count or while creating the match_array.
In a nonideal scenario, when say, there are 2 SGHs and one SGH has 2
signatures and the other one has 60k, given the current scheme of
evaluating max_idx, the max_idx will be set to 60k, and this shall
later be passed on to SigGroupHeadSetSigCnt or
SigGroupHeadBuildMatchArra which shall traverse over all the 60k sigs
for either SGHs.
Other info
----------
This is a very fast operation as the internal arithmetic is done
bitwise.
Patch
-----
The functions SigGroupHeadSetSigCnt and SigGroupHeadBuildMatchArray can
be optimized by storing the max signature id (internal) per SGH (which
also seemed to be the initial intention as per fn comments).
As a result of this, the sig_array is only walked up until the max sig
id of that respective SGH.
port is not used and logically makes sense to not be in this struct as
this struct is already referenced by DetectPort itself as a part of
SigGroupHead.
Our code handles Uint ranges as exclusive, but for bsize, our
documentation stated that they're inclusive.
Cf. from uint.rs:
DetectUintMode::DetectUintModeRange => {
if val > x.arg1 && val < x.arg2 {
return true;
}
}
Task #6708
Especially sets transactions to complete when we get a response
without having seen the request, so that the transactions
end up getting cleaned (instead of living/leaking in the state).
Also try to set the event on the relevant transaction, instead
of creating a new transaction just for the purpose of having
the event.
Ticket: #6299
If flags are zero, there is nothing to store and remember.
Stored signatures will be reused on a later packet, and
qsorted (which may be expensive), with newer matches candidates.
Avoiding to store, leads to avoid the call to qsort.
When a TCP flow packet has not led to app-layer updates,
it is useless to run DetectRunTx, as there cannot be new
matches.
This happens for instance, when one side sends in a row multiple
packets which are not acked (and thus not parsed in IDS mode).
Doing so requires to move up the call to
AppLayerParserSetTransactionInspectId
so that it is run the same times DetectRunTx is run, and not in the
case where the transaction was not updated.
Ticket: 6299
"sigerror_ok" and "sigerror_requires" were not being reset after each
rule which could lead to a rule load error being incorrectly tracked
as skipped rather than failed.
Also initialize "skippedsigs" to 0 along with "goodsigs" and
"badsigs", while not directly related to this issue, could also throw
off some stats.
Ticket: #6710
Remove references that are mentioning Suricata 3 or less
As a note - only one Suricata 4 reference found:
(suricata-yaml.rst:"In 4.1.x")
Fast pattern selection criteria can be internally found by inspecting
SupportFastPatternForSigMatchList and SigTableSetup functions.
Ticket: #6570
When the packet load is low, Suricata can run in interrupt
mode. This more resembles the classic approach of processing
packets - CPU cores run low and only fetch packets
on interrupt.
Ticket: #5839
Previous integration of hugepage analysis only fetched data
from /proc/meminfo. However this proved to be often
deceiving mainly for providing only global information and
not taking into account different hugepage sizes (e.g. 1GB
hugepages) and different NUMA nodes.
Ticket: #6419
sig_array_size can easily be calculated with length and is only used at
one place for debugging purposes. Remove it from the DetectEngineCtx
struct to avoid making it unnecessarily heavy.
When running Suricata in XDP bypass mode (bypass: yes),
Suricata started up with error:
Error: threads: thread "FB" failed to start in time: flags 0003
"FB" thread does not transition from THV_INIT_DONE to THV_RUNNING.
Set "FB" thread THV_RUNNING state in BypassedFlowManager().
Bug: #6254
Signed-off-by: Vincent Li <vincent.mc.li@gmail.com>
Philippe Antoine
Philippe Antoine
Shivani Bhardwaj
Juliana Fajardini
Jason Ish
jason taylor
Eloy Pérez González
Lukas Sismis
Stephen Donnelly
Jeff Lucovsky
Vincent Li