Move pcre2 data structures used for parsing into the detect engine
context, so that multiple tenant loading threads don't use the same
data structures.
Bug: #4797.
Instead of using flags to indicate a rule type, use an explicit `type`
field.
This will make it more clean in code paths what paths a rule is taking,
and will allow easier debugging as well as analyzer output.
Define the following fields:
- SIG_TYPE_IPONLY: sig meets IP-only criteria and is handled by the IP-only
engine.
- SIG_TYPE_PDONLY: sig inspects protocol detection results only.
- SIG_TYPE_DEONLY: sig inspects decoder events only.
- SIG_TYPE_PKT: sig is inspected per packet.
- SIG_TYPE_PKT_STREAM: sig is inspected against either packet payload or
stream payload.
- SIG_TYPE_STREAM: sig is inspected against the reassembled stream
- SIG_TYPE_APPLAYER: sig is inspected against an app-layer property, but not
against a tx engine.
- SIG_TYPE_APP_TX: sig is inspected the tx aware inspection engine(s).
Ticket: #6085.
Performance measurement of rules is important on live Suricata
as bad rules can cause severe performance regression. This patch
introduces the --enable-profiling-rules that activate profiling
for the rules. This reduces the performance impact of full
profiling and provide visiblity on the rules performance at
the same time.
Multi buffer matching is implemented as a way for a rule to match
on multiple buffers within the same transaction.
Before this patch a rule like:
dns.query; content:"example"; dns.query; content:".com";
would be equivalent to:
dns.query; content:"example"; content:".com";
If a DNS query would request more than one name, e.g.:
DNS: [example.net][something.com]
Eeach would be inspected to have both patterns present. Otherwise,
it would not be a match. So the rule above would not match, as neither
example.net and somthing.com satisfy both conditions at the same time.
This patch changes this behavior. Instead of the above, each time the
sticky buffer is specified, it creates a separate detection unit. Each
buffer is a "multi buffer" sticky buffer will now be evaluated against
each "instance" of the sticky buffer.
To continue with the above example:
DNS: [example.net] <- matches 'dns.query; content:"example";'
DNS: [something.com] <- matches 'dns.query; content:".com"'
So this would now be a match.
To make sure both patterns match in a single query string, the expression
'dns.query; content:"example"; content:".com";' still works for this.
This patch doesn't yet enable the behavior for the keywords. That is
done in a follow up patch.
To be able to implement this the internal storage of parsed rules
is changed. Until this patch and array of lists was used, where the
index was the buffer id (e.g. http_uri, dns_query). Therefore there
was only one list of matches per buffer id. As a side effect this
array was always very sparsely populated as many buffers could not
be mixed.
This patch changes the internal representation. The new array is densely
packed:
dns.query; content:"1"; dns.query; bsize:1; content:"2";
[type: dns_query][list: content:"1";]
[type: dns_query][list: bsize:1; content:"2";]
The new scheme allows for multiple instances of the same buffer.
These lists are then translated into multiple inspection engines
during the final setup of the rule.
Ticket: #5784.
As part of 6d8b50b748, the settings of
THV_CAPTURE_INJECT_PKT ended up in a location unreachable by capture
methods that did not have PktAcqBreakLoop.
Instead, always call TmThreadsCaptureBreakLoop which handles the logic
for how the read loop should be broken.
This fixes the case where read threads won't "break" for rule reloads
until packets are seen.
Ticket: #6021
Add a new command line option, --include. This will merge additional
configuration files into the configuration specified in the main
suricata.yaml. It can be provided multiple times and the files will be
included in the order they appear on the command line.
Ticket: 3912
Tested on Fedora 37 with clang 15.
app-layer.c:1055:27: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerSetupCounters()
^
void
app-layer.c:1176:29: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerDeSetupCounters()
^
void
2 errors generated.
Use the lzma-rs crate for decompressing swf/lzma files instead of
the lzma decompressor in libhtp. This decouples suricata from libhtp
except for actual http parsing, and means libhtp no longer has to
export a lzma decompression interface.
Ticket: #5638
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
Issue: #4554
This commit modifies the workflow to propagate classification parsing
errors when in test mode.
When not in test mode, errors continue to be displayed but they do not
halt Suricata execution.
alert_queue_overflow and alerts_suppressed were not being
reinitialized when there was a reload of Suricata rules, leading to
non-valid stats counters if that happened.
Bug #5457
cppcheck:
src/detect-engine.c:3643:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
^
src/detect-engine.c:3707:5: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
snprintf(prefix, sizeof(prefix), "multi-detect.%d.reload.%d", tenant_id, reload_cnt);
^
src/detect-engine.c:4086:17: warning: %d in format string (no. 1) requires 'int' but the argument type is 'unsigned int'. [invalidPrintfArgType_sint]
snprintf(prefix, sizeof(prefix), "multi-detect.%d", tenant_id);
^
Bug: #5291.
A lot of time was spent in `SigMatchListSMBelongsTo` for the `mpm_sm`.
Optimize this by keeping the value at hand during Signature parsing and
detection engine setup.
Instead of storing a name and description as a pointer in DetectBufferType
store them in fixed size arrays. This is in preparation of runtime registration
of buffer types, where a constant name/desc is not available.
In preparation of more dynamic logic in rule loading also doing
some registration, allow for buffers to be registered as fast_patterns
during rule parsing.
Leaves the register time registrations mostly as-is, but copies the
resulting list into the DetectEngineCtx and works with that onwards.
This list can then be extended.
Victor Julien
Philippe Antoine
Eric Leblond
Jason Ish
Justin Azoff
Todd Mortimer
Philippe Antoine
Jeff Lucovsky
Jufajardini Reichow
Juliana Fajardini
Victor Julien