nDPI 6.0 requires the application to declare under which license the
library is used and, for for-profit use without an agreement with
ntop, does not load its dual-licensed dissectors (DHCP, DNS, QUIC and
TLS as of 6.0), so rules on those protocols never fire.
Expose this as "ndpi.license" with the values not-for-profit (the
default), for-profit and for-profit-dual. An invalid value is fatal.
Older nDPI releases accept and ignore the option with a warning.
Ticket: 8077
Suricata knows the direction of every packet, so tell nDPI instead of
letting it guess. Whether the flow beginning was seen is left unknown
as the flow API exposes no reliable flag for it.
Ticket: 8077
nDPI correlates flows through LRU caches (DNS to TLS, STUN to RTP, TLS
certificates, ...) that are private to each worker's detection module.
As Suricata spreads a host's flows over all workers, these correlations
mostly missed.
Create a global context at plugin init, attach the worker modules to it
and make every cache global. Failures are logged and non-fatal. nDPI
creates a shared cache on first finalization without locking, which is
fine as thread init callbacks run sequentially from TmThreadCreate().
Ticket: 8077
nDPI 5.0 removed the protocol bitmask, ndpi_extra_dissection_possible()
and the third argument of ndpi_detection_giveup(), and introduced a
classification state; nDPI 6.0 added a license argument to
ndpi_init_detection_module(). The plugin only built against nDPI 4.x.
Wrap these differences in a small header of static inline helpers
selected on NDPI_MAJOR so that ndpi.c stays version agnostic. On 5.0
and later a flow is final once classified with no extra dissection
pending, as in ndpiReader, or in monitoring state, where the
classification can no longer change. The packet cap now also applies
to classified flows still under extra dissection.
Ticket: 8077
Pygments 2.21 lexes "x < 0 or x > y" in Lua as a Lua 5.4 variable
attribute (e.g. "<const>"), producing an error token on the "0". Sphinx
reports this as a highlighting failure warning, and as the PDF build
treats warnings as errors, "make dist" fails while building the PDF.
Reorder the comparisons so the Lua example is lexed correctly. The logic
of the example is unchanged.
Example of warning by new clang:
runmodes.c:537:12: warning: variable 'file_logger_count' set but not used [-Wunused-but-set-global]
537 | static int file_logger_count = 0;
The internal type set for datasets of type IP is IPv6, however, the
expectation is that it'll support IPv6, standard IPv4 as well as IPv4 in
IPv6 notation. A regression happened during the conversion of datasets
parsing to rust wherein standard IPv4 was rejected with IP type. Fix
that by handling IPv4 standard notation in IPv6 handler.
Fixes: bc79300dfa ("datasets: move initial file reading to rust")
Ticket: 9006
Ticket: 9096
In case of a 100-continue response, we used to
- parse the response line, set progress to headers
- parse headers, set progress back to line
This caused FPs as the engine expects the progress to just increase
so that it skips signatures that did not match yet, and whose
progress is less than the tx progress.
Solution is to keep progress at line when we see a 100 response,
until we get to the body, or after a next response line which
is not 100.
The functions in util-ioctl.c use an AF_INET datagram socket to
perform interface ioctl operations. Most of them returned silently
when socket creation failed, while GetIfaceRSSQueuesNum() logged only
a warning.
Report socket creation failures as errors in all of these functions.
Such failures can indicate a system-level problem, such as file
descriptor exhaustion, and otherwise make startup and interface
reconfiguration failures difficult to diagnose.
Signed-off-by: Petr Vaganov <petrvaganoff@gmail.com>
Previously, on request failure, the prep stage fall-backed to the
default upstream branches of SV/SU due to not finding the PR body
in the error message.
This commit changes the behavior to fail the prep stage when
the description cannot be read.
Unauthenticated calls to api.github.com are limited to 60 requests per
hour, counted against the originating IP address. A GitHub-hosted
runner's egress IP is shared with jobs from unrelated repositories, so
that allowance is non-deterministic.
By authenticating the request with github.token we increase the limit
to 1000+ requests per hour counted against this repository.
The filedata inspect engine returned NO_MATCH for every non-matching
file, also when the tx had already moved past the engine's progress.
For streaming registrations such as the http2 request body this meant a
firewall LTE rule never became definitive, so the fail-closed default
policy was never applied.
Return CANT_MATCH_FILES once the tx progress moved beyond the engine,
mirroring the existing no file case. The per file state reset keeps the
rule matchable if the tx grows another file.
Ticket: #9122.
Success was decided by matching git-clang-format's stdout against two
known strings. When it fails, e.g. on an invalid revision, it writes to
stderr only and leaves stdout empty, so the check reported a formatting
violation where there is an error.
The reformatting commands had the opposite problem. git-clang-format
exits 1 when it did reformat files, which they treated as a failure, so
branch, commit, cached and rewrite-branch all reported an error whenever
they had work to do.
Keep stderr in its own file and compare against the exit code instead.
Exit code 1 means formatting changes were found or made, 2 and up are
errors.
Ticket: #9097
When a branch has no commits of its own, e.g. when it is the baseline
branch itself, git rev-list returns nothing and first_commit is empty.
The empty value was still used to build "$first_commit^", which git
rejects as a bad revision.
git-clang-format then failed without writing anything to stdout, which
the output check reads as "formatting required". Pushing the baseline
branch therefore failed the CI formatting check, reporting a style
violation that does not exist.
Report that there is nothing to format instead.
git rev-list also returns nothing when the baseline ref does not exist
as it only reports that on stderr, so verify the baseline separately to
keep that case a loud failure rather than a silent pass.
Ticket: #9097
-fsanitize=implicit-conversion flags the frag_offset + data_len
narrowing into the uint16_t frag_end. For both IPv4 and IPv6, compute
frag_end after the too-large check so the sum always fits, and make the
narrowing an explicit cast.
For IPv6 this moves the existing IPV6_MAXPACKET check above the
frag_end assignment. It used to sit at the end of the branch, after
frag_end was already computed. Nothing reads frag_end in between, so
behaviour is the same. Oversize fragments are still dropped with
IPV6_FRAG_PKT_TOO_LARGE, and the wrap can no longer happen, which
matches the IPv4 path.
Also drop the vestigial int32 casts when assigning the uint32_t
tracker id.
Ticket: 8232
The seek into the fragment tree uses frag_offset - 1 as the RB_NFIND
key. frag_offset is uint16_t, so the subtraction promotes to int, and
for the first fragment of a datagram (offset 0) the -1 narrows back to
65535 when it is stored in the key. -fsanitize=implicit-conversion
flags this.
The - 1 has to stay. DefragRbFragCompare never returns 0, so RB_NFIND
is strictly greater rather than inclusive. With key = frag_offset it
returns the fragment after an existing node at the same offset and
leaves prev on that node, so the preceding fragment, which may overlap
from the left, is skipped. With key = frag_offset - 1 the same-offset
node is included and prev ends up on the real predecessor.
So guard only the offset 0 case:
.offset = frag_offset ? frag_offset - 1 : 0,
For frag_offset == 0 the seek starts at RB_MIN either way, so this
keeps the current behaviour and drops the wrap to 65535.
Add two unit tests. DefragOutOfOrderFirstFragmentTest covers offset 0
arriving into a populated tree. DefragSameOffsetPrecedingOverlapTest
pins down the same-offset plus left-overlap case that tells
frag_offset - 1 and frag_offset apart.
Ticket: 8232
Tickete: 8265
The traffic "USER CAPABILITY" could be interpreted as both ftp
or imap.
Add a probing parser to imap after detection of the " CAPABILITY"
pattern to try do disambiguate.
While the pgsql parser is disabled if its setting is present in the yaml
config file, it is actually enabled if truly defaulted, that is, if not
present in the settings.
Actually disable pgsql parsing and logging if not present in the
suricata.yaml config. This implies that eve output enabling isn't enough
to enable the parser, now.
Bug: #8349
EngineAnalysisFP(), CleanupFPAnalyzer(), and CleanupRuleAnalyzer() were writing
to and closing the wrong FILE* pointers. Fast pattern output was written to
rules_analysis.txt instead of rules_fast_pattern.txt, leaving the latter
effectively empty.
Restore the behavior from before c8615bcd4, which inadvertently swapped the
pointers.
Bug: #9007
The byte backstop leaves many tiny OOO chunks unbounded, and the scan
only inspected the in-progress file, which an interleaved second
handle could hide. Add a chunk-count backstop (configured count, 1024
hard cap when disabled) and check every open file tx.
Ticket: #8791.
Replace the hardcoded 2 MiB gap backstop, which silently truncated
OOO file data, with the configured queue size per direction (1 GiB
hard cap when disabled), raising SMBEvent::TruncatedFileData. It runs
ahead of the chunk_left == 0 early return, and a gap reaching the
tracker tags the session gapped even when fully consumed.
Ticket: #8791.
TripleA150
Jason Ish
Juliana Fajardini
Philippe Antoine
Shivani Bhardwaj
Petr Vaganov
Lukas Sismis
Victor Julien
Samaresh Kumar Singh
Giuseppe Longo
William Bland