16384 is used as the max, but a configuration parameter has been
provided. The reason for setting an upper bound is that bit flags can
create a memory amplification as we parse them into individual data
structures.
Ticket: #8181
Lower the number of unreplied requests from 500 to 32 to consider a
flood. At the very least this is an anomaly given the DNP3 spec mentions
that DNP3 should only have one outstanding request at a time, with an
exception for unsolicited responses, so in practice no more than 2
should be seen.
Additionally make this value configurable by introducing the max-tx
parameter.
Ticket: #8181
The functionality of "both" can already be easily achieved by using both
"toclient" and "toserver" in a rule. This just adds the ease of
expression in rule. As it is added now, check the direction for the
pre-existing users of direction as well.
In corner cases, we assume that a midstream exception policy could be
triggered by a prior exception policy in effect. Explain this in the
docs.
Task #5830
Ticket: 3220
DetectSslVersionMatch did not handle properly negation.
It could never match on a signatrue with ssl_version: !tls1.3
That is because, if we had such a signature and network traffic
with tls1.1, we were looking into DetectSslVersionData field
for tls1.1, which was not set, instead of looking at field
for tls1.3 which was set with negated flag.
Previous DetectSslVersionData was holding redundant information.
It did not need to have it for each ssl version, but just globally.
Also, it did not need to hold the version as a value in the array,
as it was redundant with the index of the array.
Issue: 8035
The `luaxform` transform doesn't support the `init` function. This
commit removes that from the example and clarifies how functions in the
Lua script are used.
Ticket: 6724
Allows to use numerical values for example
Also fixes some unit tests that were returning 1 after goto error
FlagsTestParse05 especially took this path as
de->ignored_flags != (TH_SYN|TH_RST) was false
we had de->ignored_flags == 0xff ^ (TH_SYN|TH_RST)
And then, we had a match, instead of what the not-run code
was supposing.
Ticket: 6724
Allows sugar syntax for bitflags keywords.
While the expressivity does not increase, because we could already
use numerial values with all generic integer modes, this modifier
prefix is used with the strings, and follows the syntax
that is already used for fragbits and tcp.flags keyword.
Issue: 7855
Highlight the change to how ether_type values are displayed. Previously,
they were displayed in network order as a decimal value.
They are now displayed in host order as a decimal value.
Add support for two VXLAN reserved bits check modes:
- strict: validate all reserved bits for standard VXLAN format
- permissive: skip all reserved bits validation (allows extensions)
Configuration added to suricata.yaml.in with 'strict' as default.
Includes comprehensive unit tests and documentation updates.
Ticket: 7753
email.body_md5 matches on md5 hash generated from email body
This keyword maps to the EVE field email.body_md5
It is a sticky buffer
Supports prefiltering
Ticket: #7587
Following commit 5379b52af2
rules that use multiple times the keyword tls.cert_subject
will result in
Warning: detect: duplicate instance for tls.cert_subject
These rules likely meant to use a multi-buffer which is not the
case for tls.cert_subject (even if it was documented so).
Ticket: 7890
This is put in a new section of upgrade notes for
upgrading to 8.0.1
Added a page that explains how rules are prioritized by Suri, as well
as what main different types of inspection happen and what elements are
involved when ordering rules.
Task #5449
Explain briefly the internals of inspection of raw data in the following order:
- Stream Engine
- Stream reassembly
- Role of Detection Engine and Applayer Parsers
- High level communication between Stream and Detection Engine
- Relevant suricata.yaml settings
alongwith some diagrams.
Ticket 4351
Add information about:
- available tables, default policies and rule ordering
- Packet layer and applayer tables and hooks
- engine analysis output
- commandline options available
- how to load firewall rules
Also, reorganize sections and content to assist the definitions.
Since userguide/conf.py uses the deprecated get_html_theme_path(),
sphinx emits a warning which breaks the build as warnings are treated
as errors.
Issue: 7859
Remove unused rule prefilter-related stats counters that aren't in use.
94644ac960 (detect: move non-pf rules into special prefilter engines)
removed the logic that made use of and incremented the stats counters:
- det_ctx->counter_fnonmpm_list
- det_ctx->counter_nonmpm_list
Some code was left, registering them, and mentioning them in the
json schema.
Ticket #7834
- cleanup usage and documentation around needs
- mentiond that rule hooks are used instead of "needs" keywords with
link with rule hooks (which is still in the firewall-design doc)
Break out RPM, Debian, and Ubuntu package installation into their own
pages.
Also break out other distributions like "Arch" into an "Other" section
with a note about how those packages are not supported by the OISF.
Ticket: #6252
Ticket: #6069
Using the new configuration format, it is now possible to set CPU affinity
settings per interface.
The threading.autopin option has been added to automatically use CPUs from the
same NUMA node as the interface. The autopin option requires
hwloc-devel / hwloc-dev to be installed and --enable-hwloc flag in configure
script.
Ticket: 7036
Provide backward compatibility with the previous configuration
format to allow smooth transition to the new format.
The commit adds docs about the new format and the introduced changes.
We used `copy_column_count`, while just `columns` is more accurate with
what PostgreSQL describes, and what Wireshark shows.
Related to
Task #7644
Task #7645
This sub-protocol inspects messages sent mainly from the frontend to
the backend after a 'COPY FROM STDIN' has been processed by the
backend.
Parses new messages:
- CopyInResponse -- initiates copy-in mode/sub-protocol
- CopyData (In) -- data transfer message, from frontend to backend
- CopyDone -- signals that no more CopyData messages will be seen from
the frontend, for the current transaction
- CopyFail -- used by the frontend to signal some failure to proceed
with sending CopyData messages
Task #7645
Move the SCThreadInfo global function into the suricata.util library as
thread_info().
This is the last global function to be registered, so remove the
supporting functions.
Setup the init function to simply return:
{streaming = "tcp"}
or
{streaming = "http"}
The returned table can have a lot of parameters that don't make sense
together, this should simplify this one case.
When setting up a Lua output script for streaming data, we're
explicitly requesting stream data. Just pass the streaming data as
arguments, rather than requiring the script to make an extra call to
get the stream data.
The streaming data will be passed in the "stream" field of the args
passed to the log function.
Eliminates the SCStreamingBuffer Lua function.
Similar to flowvars and flowints, but a byte var cannot be registered
from a Lua script, but it still needs to be setup. Instead provide an
"map" function that sets it up, or errors out if the byte var is
unknown.
This also required passing the signature into the Lua init method, as
the state of the Signature object and the time of loading the Lua
keyword is required.
Add section label and doc reference, add another term to Common terms
section.
Tried to also improve readability for the Midstream behavior tables:
- Highlight key-words when differences are only in `do` vs `no`.
- Change order of sentences in certain descriptions, to align with the
steps those happen for the engine.
The terms 'inspection' and 'detection' were being used to signify
different engine actions in this document, while throughout the
documentation and code they're many times interchangeable.
Replace 'inspection' with 'parsing' or even 'decoding and parsing' as
more appropriate.
Add a small glossary to clarify what we mean with those terms.
Following the discussion on #7396, use those symbols for to keep the
standard.
Remove the repeting `Exception Policies` section header, too.
Related to
Task #7396