Introduce the tcp.session: keyword that accepts a comma-separated subset
of {setup, established, closing} and matches packets whose TCP session
state falls within the named phases.
This lets a rule writer cover the full TCP lifecycle in one rule instead
of separate flow:not_established + flow:established rules.
Changes:
- New detect-tcp-session.{c,h} with parser, match function, unit tests
- Register DETECT_TCP_SESSION in detect-engine-register.{c,h}
- Add to Makefile.am source list
- Extend engine-analyzer text output for tcp.session:
- Extend firewall.json keyword_info with tcp_session phase values
- Update flow-keywords.rst documentation
- Require TCP protocol and SIG_FLAG_REQUIRE_PACKET at setup
Ticket: 7704
(cherry picked from commit 398687d5d4)
Extend the app-layer-protocol keyword to accept a pipe-separated list of
protocol values, so a single rule can match any of several protocols:
app-layer-protocol:[!]<proto1>|<proto2>[|...][,<qualifier>]...;
A non-negated list matches when the flow's protocol equals any listed value
(OR); a negated list matches when it equals none of them (NOR). The
single-value form and the trailing mode qualifier are unchanged.
Matching keeps the historical AppProtoEquals() equivalences by default
(dns/doh2, http/http1/http2, dcerpc/smb, ...). An `exact` qualifier selects
strict identity matching with no equivalences and no http umbrella; it
combines with a direction mode in any order. Because a flow is never the
generic ALPROTO_HTTP, `http,exact` is rejected at load.
Values are expanded once at rule load into an effective match-set bitmask, so
the per-packet match is a single bitmask test. Single-value rules remain
prefilterable; multi-value rules are excluded from prefiltering and an
explicit prefilter on them is rejected. Conflicting keyword combinations
(duplicate or overlapping negations, mixed positive/negated) are rejected at
load. Engine-analysis reports the effective match set.
Ticket: 7705
(cherry picked from commit 43bc2db41e)
Ticket: 8731
Adds a configurable limit to string lengths to avoid to retain
too much memory for too long, and avoid producing log events
that are too big
(cherry picked from commit f9515dc71d)
Rust website has changed the location of the installer.
Additionally their installer now includes automatic append to
.bashrc to source the correct path to Rust binaries.
Cbindgen as an instruction is also removed as it is not needed
when building from the downloaded and packaged Suricata.
Ticket: 8344
(cherry picked from commit d7bf52e71c)
Record beside the existing note about division by zero that << and >>
give 0 when rvalue is 64 or more, that a rule giving such an rvalue as
a number still loads with a warning, and that --strict-rule-keywords
makes it fail to load instead. rvalue can name a byte_extract or
byte_math variable rather than a number, and then the count is not
known until the rule runs, so the rule text does not say whether it
will reach 64.
The note on main stops at the result, because 89d09c457c rejects such
a rule there instead of warning about it.
Issue: 8845
Issue: 8902
(cherry picked from commit 4a647784c7)
Every hook has a built-in default policy, but expressing anything other
than the built-in meant naming each hook explicitly.
This commit adds a `default-policy` setting that covers all hooks below it.
For any hook the most specific setting present wins.
Ticket: 8770
(cherry picked from commit 0aaa80d4f3)
Add a minimal, non-intrusive config option firewall.policies.accept-arp
to allow ARP packets to be accepted in firewall mode without requiring
ARP detection rules. Default is false to preserve current behavior.
Ticket: #8314.
The policy config was a flat map mixing packet hooks and app-layer
protocols: `packet-filter` next to `dns`. There was no node that meant
"the packet hooks" or "the app-layer hooks", so a setting could not be
scoped to one group.
Move each group under its own node:
packet-filter -> packet.filter
packet-pre-flow -> packet.pre-flow
packet-pre-stream -> packet.pre-stream
<proto>.<hook> -> app.<proto>.<hook>
Ticket: 8770
(cherry picked from commit b82b4496af)
AppProtoToString(ALPROTO_HTTP1) returns "http", so an HTTP/1 policy had to
be written as `http:` while its rule hooks were already spelled `http1:`.
Use the same name in both places.
Ticket: 8770
(cherry picked from commit 04c9fd9e22)
Ticket: 8592
Fixes: 5ddd808e9b ("ftp: don't halt the flow when raising
too_many_transactions")
In the case we receive a big chunk of TCP data, we end up
creating much more than max-tx transactions, and have
quadratic complexity on this packet, even if all
these transactions get cleaned up at the end of the processing
of this packet.
(cherry picked from commit 82c4190558)
Support STOU on the FTP data channel.
With STOU a filename is optional. If no filname is provided, use
"<stou>". The server does respond with the filename to use, but there
can be a race condition in getting this filename and setting up file
storage before data starts flowing.
Ticket: #8664
(cherry picked from commit f59960d42b)
Support the APPE (append) command on the ftp data channel. Like STOR,
APPE uploads file content on the data channel.
Ticket: #8664
(cherry picked from commit e65bf4c905)
Reading a pcap from /dev/stdin or a named pipe currently fails with "failed to get first packet timestamp. pcap_next_ex(): -1" because InitPcapFile calls setvbuf on the FILE* underlying the pcap handle after libpcap has already consumed the pcap header. On a non-seekable fd glibc cannot recover from that and the next read returns -1.
Detect non-regular files via fstat and skip setvbuf in that case so the read keeps working on pipes, fifos and stdin.
Accept pcap-file.buffer-size values of 0, which disables setvbuf buffering as an explicit opt-out, or PCAP_FILE_BUFFER_SIZE_MIN (4 KiB) to PCAP_FILE_BUFFER_SIZE_MAX (64 MiB). Treat any non-zero setvbuf return value as an error, not just negative values.
When pcap-file.buffer-size fails to parse, retain the default buffer size instead of falling through and setting it to 0. The branches are now mutually exclusive so only one of the parse-error, accepted, or out-of-range messages is logged.
Update the user guide: --pcap-file-buffer-size now documents valid values of 0 (disables setvbuf buffering) or 4 KiB to 64 MiB, and pcap-file.rst notes that 0 is the opt-out for non-seekable sources such as stdin and named pipes.
Bug: #8464.
(cherry picked from commit 565e138754)
Store the NTP reference ID as raw network-order bytes so it can be
exposed as a sticky buffer and matched with payload keywords. The
reference ID is often a 4 character string, or an IP address and not
just an integer identifier.
Updates the log reference ID to be a string of colon separated hex
digits as this matches what tshark does.
Ticket: #8488
(cherry picked from commit 713e4eb900)
Adds basic NTP transaction logging for the current supported message
types.
Includes small cleanups around reference ID.
Ticket: #8425
(cherry picked from commit 31b967b089)
When an alert is generated from firewall context, add an engine value of
"fw", otherwise "td" (for threat detect).
The engine field is only added when firewall mode is enabled.
Ticket: #8456
(cherry picked from commit 029fd1be59)
Update output flushing description to reflect EVE based approach in
documentation and config template.
Added: Provide update callout for out-of-tree output plugins.
Issue: 8286
(cherry picked from commit e7dc0d885b)
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.
Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).
Ticket: 7893
(cherry picked from commit 15c83be61a)
hs: suppress TOCTOU stat use
To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.
Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.
In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.
Additional comment in the code explains problems of using fstat and potential
issues on Windows.
Ticket: 8244
(cherry picked from commit 0fe0390a2f)
hs/cache: cleaner and more detailed output
Reduce logging level of a minor informational message.
Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.
Ticket: 8323
(cherry picked from commit 569ba3d26f)
hs: remove redundant file handle in HSLoadCache
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.
Removed the unused fopen/fclose pair and flattened the control flow.
Ticket: 8326
(cherry picked from commit d754b28717)
hs: use binary mode for cache file I/O
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.
Ticket: 8326
(cherry picked from commit 0cdc77b707)
hs: warn about the same cache directory
This is especially relevant for multi-instance simultaneous setups
as we might risk read/write races.
(cherry picked from commit 56c1552c3e)
hs: validate cached database against current HS installation
After deserializing a cached Hyperscan database, verify that its
version, CPU features, and mode match the current Hyperscan
installation by comparing hs_database_info output against a
reference database. Reject loading incompatible caches.
Ticket: 8326
(cherry picked from commit 2e7b12dda4)
hs: include HS platform info in cache file hash
Hash Hyperscan installation info (version, CPU features, mode)
into the cache filename. A Hyperscan upgrade or platform change
would now produce a different filename, so stale caches from an
older installation are never opened.
Ticket: 8326
(cherry picked from commit d640719413)
hs: address coverity warning in a reference string
Move the locking mechanism outside of the getter function and hold the
lock until the reference string is no longer reused.
** CID 1682023: Concurrent data access violations (MISSING_LOCK)
/src/util-mpm-hs-cache.c: 139 in HSGetReferenceDbInfo()
(cherry picked from commit 6ec9e5c957)
Docs were claiming greater than, less than, etc. support even though this was never supported.
Fixes: 51be8f0238 ("doc/dcerpc: add proto keywords")
Ticket: #8330.
(cherry picked from commit 2480f8c038)
The Deprecation note on Syslog was partly conveying the opposite
message, and we were missing a deprecation notice on the Syslog output
section.
(cherry picked from commit 926fde859e)
Adding the directory "install" to EXTRA_DIST, actually triggers make
to run "make install", which is not what we want. Instead, avoid this
magic keyword and list the files in the install directory
individually.
If the user doesn't have permission to install files to the prefix,
like "/usr", then "make dist" can fail. Worse, even they do have
permission to write into the prefix, a "make dist" will install files
there when it shouldn't.
Ticket: #8279
(cherry picked from commit 2defd5ae5a)
Adds a chapter indicating what are the main steps when adding exception
policies, how is it possible to extend them, as well as main aspects and
files to consider when doing so.
Task #5612
(cherry picked from commit 48b15ac152)
Yash Datre
Philippe Antoine
Lukas Sismis
Jeff Lucovsky
Victor Julien
Antoine Abou Faysal
Jason Ish
Samaresh Kumar Singh
Shivani Bhardwaj
Andreas Dolp
Juliana Fajardini
Sven Cuyt