In firewall mode the alproto logic when building prefilter and rule
engines is strict, but was out of sync between prefilter and rule app
inspect engines.
For SIGNATURE_HOOK_TYPE_APP rules the logic is strict, with an exception
for HTTP/2 and DOH2. DOH2 is not a full protocol implementation, but
rather HTTP/2 with a different alproto.
Fixes: d64954a873 ("detect: don't register unrelated inspect engines")
Support for per transaction sub states: different state machines per
transaction type.
Skip engines not belonging to our substate.
Store tx_type in DetectTransaction.
Each protocol supporting sub states will register states from 1 and up.
To support:
Ticket: #8386.
buffer_type_id is a u32
We have a list per buffer of a list of AppProto
But the list of AppProto can grow maximum for the case of file.data
to all the alprotos supporting files.
Ticket: 8001
Ticket: 8001
Will allow algorithms such as ac to use the MpmCtx, especially
the number of patterns, to preallocate some structures.
For detect engine PatternMatchThreadPrepare, create a dummy
MpmCtx out of all the MpmCtx in the DetectEngineCtx,
and get the max value for the number of patterns
Make `Signature::proto` an optional member, meaning that if it is
NULL we can skip the check. This can be done for `alert ip`, as no check
is needed, and for `alert tcp` and `alert udp` as having a rule in a sgh
for those means that the protocol matches.
Some exceptions are rules that require:
- ipv4/ipv6 specific matching
- frames, due to sharing prefilter between tcp and udp
- ip-only rules, due to those not being per sgh
detect-engine-mpm.c: In function ‘BuildBasicPname’:
detect-engine-mpm.c:197:43: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
197 | snprintf(pname, sizeof(pname), "%s", name);
| ^
detect-engine-mpm.c:197:9: note: ‘snprintf’ output 1 or more bytes (assuming 2) into a destination of size 1
197 | snprintf(pname, sizeof(pname), "%s", name);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
detect-engine-mpm.c: In function ‘AppendTransformsToPname’:
detect-engine-mpm.c:231:61: error: ‘%s’ directive output may be truncated writing up to 1023 bytes into a region of size 7 [-Werror=format-truncation=]
231 | snprintf(xforms_print, sizeof(xforms_print), " (%s)", xforms);
| ^~ ~~~~~~
detect-engine-mpm.c:231:13: note: ‘snprintf’ output between 4 and 1027 bytes into a destination of size 9
231 | snprintf(xforms_print, sizeof(xforms_print), " (%s)", xforms);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ticket: #7905.
- Declare the profiling name variable once we know the actual size allowed
for the name instead of before. Prevents an snprintf truncation warning.
- Account for the null byte when calculating how much space is left to
avoid an snprintf format string truncation warning.
Introduce DetectGetSingleData which does the generic wrapping,
including the transforms, using a new callback prototype
DetectTxGetBufferPtr
The goal is to replace most InspectionBufferGetDataPtr.
For this commit, we do not change every callback to keep the
change relatively small.
Focus here is to remove DetectHelperGetData as its functionality is
provided more directly by the new DetectTxGetBufferPtr.
Generic:
<app_proto>:request_started and <app_proto>:response_started
<app_proto>:request_complete and <app_proto>:response_complete
Per protocol, it uses the registered progress (state) values. E.g.
tls:client_hello_done
A rule ruleset could be:
pass tls:client_hello_done any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)
drop tls:client_hello_done any any -> any any (sid:22;)
The pass rule is evaluated when the client hello is parsed, and if it
doesn't match the drop rule will be evaluated.
Registers each generic lists as "<alproto>:<progress state>:generic"
(e.g. "tls:client_hello_done:generic").
Ticket: #7485.
Ticket: 5053
The names are now dynamically registered at runtime.
The AppProto alproto enum identifiers are still static for now.
This is the final step before app-layer plugins.
Ticket: 2224
It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.
For multi buffers, absent matches if there are 0 buffers.
For file keywords, absent matches if there is no file.
Ticket: 6575
Multi buffers keywords now use a single registration function
DetectAppLayerMultiRegister with a GetBuffer argument.
This GetBuffer function pointer is similar to the ones used by
single-buffer keyword, except that it takes an additional
parameter which is the index of the buffer to get.
Under the hood, an anonymous union between these 2 functions
pointers types is used.
In the end, this deduplicates code, especially the calls to
DetectEngineContentInspection
The old DetectAppLayerMpmRegister has not been around since 4.1.x.
Rename the v2 of this function to a versionless function as there is no
documentation referring to what the 2 means.
Multi buffer matching is implemented as a way for a rule to match
on multiple buffers within the same transaction.
Before this patch a rule like:
dns.query; content:"example"; dns.query; content:".com";
would be equivalent to:
dns.query; content:"example"; content:".com";
If a DNS query would request more than one name, e.g.:
DNS: [example.net][something.com]
Eeach would be inspected to have both patterns present. Otherwise,
it would not be a match. So the rule above would not match, as neither
example.net and somthing.com satisfy both conditions at the same time.
This patch changes this behavior. Instead of the above, each time the
sticky buffer is specified, it creates a separate detection unit. Each
buffer is a "multi buffer" sticky buffer will now be evaluated against
each "instance" of the sticky buffer.
To continue with the above example:
DNS: [example.net] <- matches 'dns.query; content:"example";'
DNS: [something.com] <- matches 'dns.query; content:".com"'
So this would now be a match.
To make sure both patterns match in a single query string, the expression
'dns.query; content:"example"; content:".com";' still works for this.
This patch doesn't yet enable the behavior for the keywords. That is
done in a follow up patch.
To be able to implement this the internal storage of parsed rules
is changed. Until this patch and array of lists was used, where the
index was the buffer id (e.g. http_uri, dns_query). Therefore there
was only one list of matches per buffer id. As a side effect this
array was always very sparsely populated as many buffers could not
be mixed.
This patch changes the internal representation. The new array is densely
packed:
dns.query; content:"1"; dns.query; bsize:1; content:"2";
[type: dns_query][list: content:"1";]
[type: dns_query][list: bsize:1; content:"2";]
The new scheme allows for multiple instances of the same buffer.
These lists are then translated into multiple inspection engines
during the final setup of the rule.
Ticket: #5784.
Instead of a shared mpm context for just "file.data" or "file.magic"
use per alproto mpms. This way http file.data rules won't affect smb
file.data performance.
Ticket: #4378.
Dmitry Uryvchikov
Victor Julien
Philippe Antoine
Jason Ish
Shivani Bhardwaj
Lukas Sismis
Jeff Lucovsky