Issue: 7672
The subslice transform creates a slice of the input buffer.
Specify the subslice desired -- nbytes and truncate are optional:
subslice: offset <,nbytes> <,truncate>
offset: Specifies the starting offset for the new subslice. When
negative, expresses how far from the end of the input buffer to begin.
nbytes: Specifies the size of the subslice. When negative, specifies
that the subslice will end that many bytes from the end of the input
buffer. Nbytes must be non-zero.
When nbytes is not specified, the size of the subslice will be the size
of the input buffer - offset.
truncate: Specify behavior when offset + nbytes exceeds buffer length.
When present, trims nbytes such that offset + nbytes equals buffer
length. When not present, an empty buffer is produced.
Examples:
subslice: 1; - The subslice will be a copy of the input
buffer but omits the input buffer's first byte
"This is Suricata" -> "his is Suricata"
subslice: 0, 13; - The slice is created from the first 13 bytes
of the input buffer
"This is Suricata" -> "This is Suric"
subslice: 10, -5; - The subslice is created starting at offset 10
and continues to 5 bytes before the end of the input buffer
"This is Suricata" -> "r"
subslice: -3; - The subslice will be the last 3 bytes of the
input buffer.
"This is Suricata" -> "ata"
Ticket: 8501
(m)dns.*.rrname keywords required the rules to start with
alert (m)dns
and refused to load if the rule started with generic alert ip or
alert udp, with error message
Error: detect-parse: invalid alproto 0
When using ASAN, flto=auto and compiling on a platform that supports
AVX-512, this memcpy is optimized as its exactly 64 bytes. However, when
using ASAN, there is a fake stack that may not be 64 bytes aligned, and
this AVX write lands on an unaligned byte, causing a crash.
Use strlcpy avoids this optimization as it needs to find the NUL byte.
Ticket: 6813
So, in the case we have a ftp-data flow with data to client as
expected, but after to_client side completes (with timeout for
example), the client sends some data to server,
In this case, we want to check first the direction, and bail out,
then do the assertion check about the finished state.
As a legacy from Suricata 7, mempool size was kept even internally
as a global size. Since in Suricata 8, mempool allocation moved to
per queue model, it makes sense to calculate the correct size right
when loading the configuration.
The previous implementation misscalculated in some scenarios the size
of the mempool which prevented Suricata starting successfully.
This happened, e.g. when there were 32k RX descriptors and 64 TX descs.
The resulting mempool was incorrectly deduced as 32767 (2^n-1), which
didn't even hold all RX descriptors. Instead, it should result to 65535.
Ticket: 8370
If the last signature in the candidates list is also the last for the
current progress value, it needs to be considered as the last for that
progress, so that if it doesn't match the default policy is applied.
Ticket: 8395
For protocols where tx are complete at start, that is
the `tx_get_progress` function returns 1 inconditionnally,
we can use the progress 1, not losing time on detection in ips
mode, and getting ready for firewall mode
For the other protocols (where a tx needs both a request and response
for instance), keep progress 0 as was the case before
Fixes warning:
conf-yaml-loader.c: In function ‘ConfYamlSetConfDirname’:
conf-yaml-loader.c:87:8: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
87 | ep = strrchr(filename, '\\');
| ^
Add --plugin <PATH> to load an additional plugin from the command
line. This is more convenient than "--set plugins.X" especially when
you may already have a plugins loaded and you want to load an
additional one.
Ticket: 8463
CID 1645613
Ticket: 7664
We assign and dereference next_s like
next_s = *match_array++;
next_sflags = next_s->flags;
So, there is no need to check if next_s is NULL, as it can never
be (otherwise, we could crash before)
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream inspection which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
FTP parser can handle multiple command lines per direction. Appropriate calls
to trigger raw stream inspection have been added on succesful parsing of each
request line and response line.
Bug 7742
`alert pkthdr` was initially just an alias for `alert ip`, as that was
really just a way of stating that "any" should be matched. However with
the Ethernet matching in place, it no long makes sense to treat `alert
ip` as "any". Since `pkthdr` is used to match on decoder events, also
for packets that completely failed to parse, it should no longer be
treated as `alert ip` but rather as it's own distinct logic.
Sticky buffer to inspect the ethernet header.
Example rule:
alert ether any any -> any any ( \
ether.hdr; content:"|08 06|"; offset:12; depth:2; \
sid:1;)
Ticket: #8327.
Make `Signature::proto` an optional member, meaning that if it is
NULL we can skip the check. This can be done for `alert ip`, as no check
is needed, and for `alert tcp` and `alert udp` as having a rule in a sgh
for those means that the protocol matches.
Some exceptions are rules that require:
- ipv4/ipv6 specific matching
- frames, due to sharing prefilter between tcp and udp
- ip-only rules, due to those not being per sgh
Add flushing logic driven off of the file contexts. This is a simpler
solution that removes the need for logger registration changes.
Overview:
Use the heartbeat-driven thread to periodically flush all registered EVE
contexts via a global flush list.
The global flush list is a mutex-protected TAILQ of LogFileFlushEntry
nodes; each node points to a LogFileCtx. Mutex = log_file_flush_mutex
Periodic flushing performed by a thread according to the
heartbeat.output-flush-interval [1,60]. LogFileFlushAll() is invoked to
initiate flushing of registered LogFileCtx structs; each struct's
fp_mutex is obtained while the flush occurs to synchronize with
LogFileWrite activity.
Interacts with file-rotation via the fp_mutex.
Deadlock prevention: the log_file_flush_mutex must be obtained before
the fp_mutex.
Issue: 8286
Ticket: 8333
Allows to pass on pseudo packets
So, that a ruleset like
pass ssh any any -> any any (geoip:any,US,IN; sid: 1;)
drop ip any any -> any any (flow:established, to_server; sid:2;)
Does not lead to a drop on the pseudo-packet when we flush
because ssh traffic gets encrypted
Simplify statements where there's a conditional statement followed by
return true or return false.
Instead of
if (conditional)
return true;
else
return false;
Use:
return conditional
In some timezones, LogCustomFormatTest01 was failing as we were
specifying the wrong setting for is_dst. Instead set it to -1 so mktime
can make the decision based on the environment. Appears to fix this test
for "Brazil/East" in the summer.
Add GitHub CI tests to test this specific test with a few different
timezones.
When during raw reassembly it is detected that last ack is moved beyond
the progress and also beyond the data retrieved, update progress to the
last ack value.
Bug: #8272.
When mempool size was configured really low (<32), Suricata exhausted
the mempool with the rx_burst call, which led to undefined behavior.
The current fix ensures the burst size is at most the size of the mempool,
if the mempool is smaller than BURST_SIZE macro.
Based on review it seems p->proto should always be set to the same
value as what `PacketGetIPProto` returns. Add a check to validate
in a decoder that the fuzzer executes.
Move the locking mechanism outside of the getter function and hold the
lock until the reference string is no longer reused.
** CID 1682023: Concurrent data access violations (MISSING_LOCK)
/src/util-mpm-hs-cache.c: 139 in HSGetReferenceDbInfo()
This run-mode does not allow for capture ThreadVars to be created at
the appropriate place in the application life cycle. Instead,
applications should register their own custom run-mode.
Ticket: #8259
If the key `app-layer.protocols.PROTO.enabled` is present, the protocol
is enabled for all carrier protocols. This is not ideal.
Only if the key `app-layer.protocols.PROTO.enabled` is missing, an
attempt is made to look for a setting specific to the ipproto passed
at the time of registration e.g.
`app-layer.protocols.PROTO.udp.enabled`.
By default, check for carrier proto specific setting. If it is not
found, then fall back to the generic setting.
Issue a warning in case an inconsistent combination of global and
ipproto specific setting is found.
Bug 8205
DPDK interface which were added to bond and had auto-calculated
mempool size didn't have enough space as the configured
descriptors and the derived mempool only considered one interface
per YAML node entry.
DPDK Bonding PMD allows for multiple interface to cooperate and
therefore, needs more space.
Ticket: 8125
Following DPDK's recommendation to size mempool to 2^n - 1,
the current implementation was flawed for certain combos,
e.g. mp_size 1023 and 1 queue, which resulted to 1022.
Hash Hyperscan installation info (version, CPU features, mode)
into the cache filename. A Hyperscan upgrade or platform change
would now produce a different filename, so stale caches from an
older installation are never opened.
Ticket: 8325
After deserializing a cached Hyperscan database, verify that its
version, CPU features, and mode match the current Hyperscan
installation by comparing hs_database_info output against a
reference database. Reject loading incompatible caches.
Ticket: 8325
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.
Ticket: 8325
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.
Removed the unused fopen/fclose pair and flattened the control flow.
Ticket: 8325
Reduce logging level of a minor informational message.
Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.
Ticket: 8322
Ticket: 7717
Allows for instance to process/log ARP packets over VXLAN.
That means we need to decode the ethernet layer above vxlan
instead of skipping it as part of the vxlan, even if the vxlan
decoder still checks the ethernet layer to avoid FPs.
Instead of directly accessing the field
Will allow PacketTunnelType to hold the precise tunnel type like
DECODE_TUNNEL_ERSPANII with a modification of PacketIsTunnelChild
Refactor pcap file deletion to use a single delete-when-done option
with three values instead of separate boolean options:
- false (default): No deletion
- true: Always delete files
- "non-alerts": Delete only files with no alerts
Also account for alerts produced by pseudo packets (flow timeout / shutdown flush):
- Introduce small capture hooks and invoke on pseudo-packet creation so the
capture layer can retain references and observe alerts emitted after the last
live packet
- Call the hook from both TmThreadDisableReceiveThreads and TmThreadDrainPacketThreads
Key changes:
- Replace should_delete/delete_non_alerts_only bools with enum
- Move alert counter from global to per-file PcapFileFileVars
- Relocate alert counting from PacketAlertFinalize to pcap module
- Ensure thread safety for both single and continuous pcap modes
- Add unit tests for configuration parsing and pseudo-packet alert path
The --pcap-file-delete command line option overrides YAML config
and forces "always delete" mode for backward compatibility.
Documentation updated to reflect the new three-value configuration.
Fixes OISF#7786
Bindgen the Rust bindings to the C JsonBuilder API along with the rest
of the Rust bindings to C. Breaking it out was probably the wrong
idea.
This should make it easier, and more correct to bindgen C functions
that use SCJsonBuilder types.
There is an unfortunate side-affect that one has to read
output-eve-bindgen.h for the documentation on this type, however, I
think we can resolve that in time.
As output-eve-bindgen.h exists to support bindgen, its odd to see
other Suricata C files using it. Instead Suricata C code should import
"output-eve.h", which itself includes "output-eve-bindgen.h", only
broken out to support the external tool bindgen.
Replace duplicated SCFree() calls in error paths with a single
cleanup label using goto pattern. This reduces code duplication
and ensures consistent resource cleanup.
Additional improvements:
- Fix misleading error message when xstats table size changes
between calls (was passing positive value to rte_strerror)
- Use unsigned int for length/index to match DPDK API semantics
- Initialize xstats_names to NULL at declaration for safe cleanup
- Added "not supported" case when the first call to xstats_get returns 0
Ticket: 8273
... to account for midstream sessions.
Commit 497394e removed inspection of app-proto txs for packets
without an established TCP connection. But this meant that the
first packet seen in a session pick mid-stream could go without
inspection (previous bug 5510 seemed to point towards this behavior,
too).
If a flow has more packets, the stream will be inspected as part of
the upcoming packets and this would go unnoticed. In a single-packet
flow, however, the inspection for the packed would be skipped. Although
this might not affect alerts -- as they could be processed as part of
the flow timeout logic, the actual traffic could be evaded in IPS, in
case of a drop rule.
From the above, the most visible scenario is when there is only one packet on the flow,
as then the engine doesn't have "more time" to pick-up real-packets to
inspect for that given flow. But certain tests show that this can also
happen for more than one packet scenarios: there will be one less drop
event, or traffic from a packet that should have been already dropped
will be logged.
This led to the possibility of a real packet not being blocked, in IPS,
or matched against rules, as the corresponding portion of the stream
was only inspected later, as part of the stream/flow-timeout logic.
To ensure that we correctly flag the first packet seen for a given mid-stream
session, we must check for the session state and existance *after* we
have dealt with TCP flags and state.
Related to
Bug #5510
As part of
Bug #5180
During initialization, the engine reports how many rules were loaded, as
well as which types. Pkt-only or stream-pkt rules would cause a "hole"
in such stats, as they're not counted.
and bindgen it to rust
Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerTxData
Move also the free function to C SCAppLayerTxDataCleanup
As suricata-sys crate defines AppLayerTxData for rust,
It must itself implement the Drop trait, and thus,
We need to define a feature surest
and bindgen it to rust
Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerResult
Keep From<> impl in sys crate that defines it
and bindgen it to rust, and use default trait instead of new
Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerStateData
Makes leak sanitizer work without adding LSAN_OPTIONS=use_unaligned=1
Otherwise, leak sanitizer may report the read and black tree pointers
as leaked when it is still owned by some global variable.
Follow-up on d046e82db6
Add authentication support to the Redis logging output.
It introduces `username` and `password` configuration options for Redis,
allowing Suricata to authenticate with Redis servers that require it.
Ticket: 7062
Some drivers (e.g. BNXT) fail to report stats after the device is
stopped. Move stats collection (DPDKDumpCounters and PrintDPDKPortXstats)
to run before rte_eth_dev_stop() in HandleShutdown.
Also change PrintDPDKPortXstats error handling from FatalError to
graceful return since stats collection failures during shutdown
should not crash the application.
The commit removes ThreadExitPrintStats callback as the function had no
useful features after the stats were moved.
Ticket: 8251
To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.
Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.
In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.
Additional comment in the code explains problems of using fstat and potential
issues on Windows.
Ticket: 8243
Add optional unique_on {src_port|dst_port} to detection_filter for
exact distinct port counting within the seconds window.
Features:
- Runtime uses a single 64k-bit (8192 bytes) union bitmap per
threshold entry with O(1) updates.
- Follows detection_filter semantics: alerting starts after the
threshold (> count), not at it.
- On window expiry, the window is reset and the current packet's
port is recorded as the first distinct of the new window.
Validation:
- unique_on requires a ported transport protocol; reject rules
that are not tcp/udp/sctp or that use ip (protocol any).
Memory management:
- Bitmap memory is bounded by detect.thresholds.memcap.
- New counters: bitmap_memuse and bitmap_alloc_fail.
Tests:
- C unit tests for parsing, distinct counting, window reset, and
allocation failure fallback.
- suricata-verify tests for distinct src/dst port counting.
Task #7928
Makes leak sanitizer work without adding LSAN_OPTIONS=use_unaligned=1
Otherwise, leak sanitizer may report rule_id as leaked
when it is still owned by some global variable
When not sandboxed, a script can get access to the metatable and call
`.__gc` with an invalid value like nil, causing a NULL pointer dereference
in Suricata.
Ticket: #8248
This should make it possible to catch invalid combinations in the same
signature early. This patch covers checking and erroring on the following
invalid cmd combinations:
- set + isset
- unset + isnotset
- set + toggle
- set + unset
- isset + isnotset
- unset + toggle
the same flowbit in the same signature which is basically an unnecessary
operation at runtime.
This also helps bring down the difficulty of handling of actual complex
flowbit chains.
Bug 7772
Bug 7773
Bug 7774
Bug 7817
Bug 7818
Bug 8166
Code refactor to gather all PCAP-related structure members
under one structure.
New pcap_v structure guards protect the union variables from
other capture modes trying to access the packet number incorrectly.
Ticket: 7835
For an easier review process, this is a two-step change process,
in which pcap_cnt is first accessed by functions-to-be, implemented
as simple macros.
In the follow-up commit, the actual refactor is implemented with the new
function. The old macros are deleted.
Ticket: 7835
Use of t_pcapcnt is only relevant when compiled in debug mode only.
This patch adds additional debug guard to also shield the declaration
and assignment.
It does not return anything.
The only known implementation is in dns where it does not return.
And the C code does not bother to check a return value anyways.
Add basic derive counter support, where a counters value is derived from
2 other counters.
Add DERIVE_DIV that divides the first counter value by the second
counters value.
Convert `decoder.avg_pkt_size` to be derived from `decoder.bytes` and
`decoder.pkts`.
Ticket: #5615.
When in a `base64_decode`-`base64_data` pair the decode was depending
on another match through the relative option, the `buffer_offset` would
be updated to the relative position of the previous match. During the
`base64_data` phase, a relative match would use that offset even though
the match happened in a new buffer.
Example::
http.request_body; content:"|27|"; \
base64_decode:relative; \
base64_data; content:"|ff ff ff ff|"; within:16;
This use of the `buffer_offset` is incorrect as that value is relative
to a buffer and the `base64_data` points to a new buffer.
This patch addresses this by resetting DetectEngineThreadCtx::buffer_offset
before inspecting `base64_data`.
Bug: #7842.
Adds a chapter indicating what are the main steps when adding exception
policies, how is it possible to extend them, as well as main aspects and
files to consider when doing so.
Task #5612
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.
Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).
Ticket: 7830
As a intermediary step for Hyperscan (MPM) caching,
the MPM config initialization should be part of the default
detect engine context for later dynamic retrieval.
Ticket: 7830
To have a system-level overview of when was the last time the file was
used, update the file modification timestamp to to the current time.
This is needed to remove stale cache files of the system.
Access time is not used as it may be, on the system level, disabled.
Ticket: 7830
If recursive call to DetectPortInsert (in case DetectPortCut returned
a c != NULL) failed, we had successfullt inserted `new` but
we return error, leading to freeing DetectPort `new` in
DetectPortParseInsertString but it
will be freed again by DetectPortCleanupList from DetectPortParse
Ticket: 6187
Wrapper around `memmem`.
The case sensitive search is implemented by directly calling `memmem`.
As there is no case insensitieve variant available, a wrapper around
memmem is created, that takes a sliding window approach:
1. take a slice of the haystack
2. convert it to lowercase
3. search it using memmem
4. move window forward
Firewall rules were supposed to also load when using a config like
##
## Suricata as a Firewall options (experimental)
##
firewall:
# toggle to enable firewall mode
enabled: yes
# Firewall rule file are in their own path and are not managed
# by Suricata-Update.
rule-path: /etc/suricata/firewall/
# List of files with firewall rules. Order matters, files are loaded
# in order and rules are applied in that order (per state, see docs)
rule-files:
- firewall.rules
But they were not because of a missing check.
Bug: #8206.
16384 is used as the max, but a configuration parameter has been
provided. The reason for setting an upper bound is that bit flags can
create a memory amplification as we parse them into individual data
structures.
Ticket: #8181
Lower the number of unreplied requests from 500 to 32 to consider a
flood. At the very least this is an anomaly given the DNP3 spec mentions
that DNP3 should only have one outstanding request at a time, with an
exception for unsolicited responses, so in practice no more than 2
should be seen.
Additionally make this value configurable by introducing the max-tx
parameter.
Ticket: #8181
Complete is a flag used to tell if the message was completely parsed,
as not all messages may be completely parsed if we don't know all
their objects. However, they are still "done".
In the alstate-progress callback, check the done flag, not the
complete flag.
Ticket: #8181
So far, the alert queue was expanded by doubling in size w/o any
boundary checks in place. This led to situations where doubling
the alert_queue_capacity meant overflow of the very same value
stored in det_ctx.
This led to heap-use-after-free in some conditions where
det_ctx->alert_queue_capacity overflowed.
Fix this by capping the max of alert_queue_capacity by checking if its
expansion could result in an overflow.
Security 8190
Ticket: 8156
In case of non-tx alerts, we try to loop over all the txs to find
the xff header. Do not start from tx_id 0, but from min_id
as AppLayerParserTransactionsCleanup to skip txs that were freed
TLS parsers use x509-parser crate which parses X.509 certificates that
use ASN.1 DER encoding that can allow arbitrary byte sequences. An
attacker could inject null byte in a certificate anywhere to stump the
common language parsers terminating the string at a null byte leading to
a bypass of a possibly malicious certificate.
So far, the rust TLS parser for "subjectaltname" used a pattern that involved:
-> Get ASN.1 DER encoded raw data from the x509-parser crate
-> Convert this raw data to a decoded string (Rust)
-> Convert the Rust string to CString
-- The problem lies here. CString only accepts proper strings/byte
buffers and converts it into an owned C-compatible, null-terminated
string. However, if any null byte occurs in the string passed to the
CString then it panics.
In the rust TLS parser, this panic is handled by returning NULL.
This means that the parser will error out during the decoding of the
certificate. However, Suricata must be able to detect the null byte
injection attack being an IDS/IPS.
Hence, replace all such string patterns w.r.t. TLS SAN with a byte
array.
Bug 7887
TLS parsers use x509-parser crate which parses X.509 certificates that
use ASN.1 DER encoding that can allow arbitrary byte sequences. An
attacker could inject null byte in a certificate anywhere to stump the
common language parsers terminating the string at a null byte leading to
a bypass of a possibly malicious certificate.
So far, the rust TLS parser for "issuerdn" used a pattern that involved:
-> Get ASN.1 DER encoded raw data from the x509-parser crate
-> Convert this raw data to a decoded string (Rust)
-> Convert the Rust string to CString
-- The problem lies here. CString only accepts proper strings/byte
buffers and converts it into an owned C-compatible, null-terminated
string. However, if any null byte occurs in the string passed to the
CString then it panics.
In the rust TLS parser, this panic is handled by returning NULL.
This means that the parser will error out during the decoding of the
certificate. However, Suricata must be able to detect the null byte
injection attack being an IDS/IPS.
Hence, replace all such string patterns w.r.t. TLS IssuerDN with a byte
array.
Bug 7887
TLS parsers use x509-parser crate which parses X.509 certificates that
use ASN.1 DER encoding that can allow arbitrary byte sequences. An
attacker could inject null byte in a certificate anywhere to stump the
common language parsers terminating the string at a null byte leading to
a bypass of a possibly malicious certificate.
So far, the rust TLS parser for "Subject" used a pattern that involved:
-> Get ASN.1 DER encoded raw data from the x509-parser crate
-> Convert this raw data to a decoded string (Rust)
-> Convert the Rust string to CString
-- The problem lies here. CString only accepts proper strings/byte
buffers and converts it into an owned C-compatible, null-terminated
string. However, if any null byte occurs in the string passed to the
CString then it panics.
In the rust TLS parser, this panic is handled by returning NULL.
This means that the parser will error out during the decoding of the
certificate. However, Suricata must be able to detect the null byte
injection attack being an IDS/IPS.
Hence, replace all such string patterns w.r.t. TLS Subject with a byte
array.
Bug 7887
When FlowSwap() reverses the direction of a flow, the MAC address sets
stored in the flow also need to be swapped to maintain consistency with
the new direction. Previously, MAC addresses were not swapped along with
other flow properties like packet/byte counters.
Ticket #8172
Make sure the script can use all bytes configured. So exclude setup like
input buffers that are put on the lua state before script is executed.
Bug #8173.
Expose elephant flow flags on a flow via rule language. The rule keyword
is defined as "flow.elephant: <dir>;" where the <dir> can expand to:
- toserver
- toclient
- either
- both
Feature 8117
The tracking for elephant flow is done per direction however, the flag
was set on the flow whenever either of the directions crossed the
rate-limit defined in the settings. Given that the tracking was already
split, it makes sense to split the detection tracking per direction as
well and allow user to have a better control via rule language.
The functionality of "both" can already be easily achieved by using both
"toclient" and "toserver" in a rule. This just adds the ease of
expression in rule. As it is added now, check the direction for the
pre-existing users of direction as well.
Detection and logging skip a lot of work if PKT_STREAM_EST is not set. When
a TFO packet with data comes in the TCP state is not yet established, but
the data still needs to be considered.
So for this case set the PKT_STREAM_EST flag.
Bug #6744.
Issue: 7389
Follow-up change for verbose handling. Instead of displaying the numeric
verbose level, display the level name.
I.e., instead of 4, display Debug
Ticket: 3220
DetectSslVersionMatch did not handle properly negation.
It could never match on a signatrue with ssl_version: !tls1.3
That is because, if we had such a signature and network traffic
with tls1.1, we were looking into DetectSslVersionData field
for tls1.1, which was not set, instead of looking at field
for tls1.3 which was set with negated flag.
Previous DetectSslVersionData was holding redundant information.
It did not need to have it for each ssl version, but just globally.
Also, it did not need to hold the version as a value in the array,
as it was redundant with the index of the array.
Issue: 7389
Continue to recognize multiple 'v' specifications up to the maximum
recognized. The maximum recognized is the number of slots between the
NOTICE and DEBUG levels.
When debug is configured, `-vvvv` is required to set the log level to
debug. Specifying 4 or more `v` will maintain debug level.
When debug is not configured, `-vvvv` and more `v`'s will cause
config messages to be printed.
If we have
- stream->last_ack 0x40021
Then, we call StreamTcpUpdateLastAck with 0x8000fc21
Then we satisfy SEQ_GT((ack), (stream)->last_ack)
But we do not satisfy SEQ_GT(ack, (stream)->base_seq))
and the new last_ack will be compared to base_seq
So, refuse to make such a big update
Ticket: 6865
To avoid overhead of copies during syncs, compress the local stats table.
Previously all counters had 2 64bit ints: one for the value, one to
track the number of times the counter was updated. The updates field was
only used for counters of type "average" however. Since only a few
average counters were in use, this effectively meant that almost half of
the space used for the table was wasted.
This patch reduces `StatsLocalCounter` to a single i64. Regular counters
simply use this directly by using their id as an index into the table.
For avg counters the logic is similar, however they effectively use 2
indexes, idx and idx+1. The value is at `idx`, the updates are at `idx +
1`.
Ticket: #5613.
Minimize the work done under lock when synchronizing thread stats.
Instead of looping all stats to copy them from the threads private store
to the public store, this patch simply memcpy's the whole array.
When the stats thread then takes the stats from the threads public
store, it now also copies the whole array to a local store before
processing the stats one by one.
This keeps the locking time minimal.
Packet Thread(s)
[ private thread stats ]
| lock
| memcpy
V unlock
[ copy of thread stats ]
Stats Thread
[ copy of thread stats ]
| lock
| memcpy
V unlock
[ private copy of thread stats ]
In `add-hostbit`, `remove-hostbit` and `list-hostbit` commands, the IPv6
address parsing was not using the correct variable:
from /usr/include/dirent.h:25,
from suricata-common.h:73,
from runmode-unix-socket.c:18:
In function ‘inet_pton’,
inlined from ‘UnixSocketHostbitAdd’ at runmode-unix-socket.c:1316:13:
/usr/include/x86_64-linux-gnu/bits/inet-fortified.h:56:10: warning: call to ‘__inet_pton_chk_warn’ declared with attribute warning: inet_pton called with a destination buffer size too small [-Wattribute-warning]
56 | return __glibc_fortify (inet_pton, __sz, sizeof (char),
| ^~~~~~~~~~~~~~~
In function ‘inet_pton’,
inlined from ‘UnixSocketHostbitRemove’ at runmode-unix-socket.c:1403:13:
/usr/include/x86_64-linux-gnu/bits/inet-fortified.h:56:10: warning: call to ‘__inet_pton_chk_warn’ declared with attribute warning: inet_pton called with a destination buffer size too small [-Wattribute-warning]
56 | return __glibc_fortify (inet_pton, __sz, sizeof (char),
| ^~~~~~~~~~~~~~~
In function ‘inet_pton’,
inlined from ‘UnixSocketHostbitList’ at runmode-unix-socket.c:1476:13:
/usr/include/x86_64-linux-gnu/bits/inet-fortified.h:56:10: warning: call to ‘__inet_pton_chk_warn’ declared with attribute warning: inet_pton called with a destination buffer size too small [-Wattribute-warning]
56 | return __glibc_fortify (inet_pton, __sz, sizeof (char),
| ^~~~~~~~~~~~~~~
Bug: #8102.
In file included from decode.h:33,
from host.h:27,
from util-threshold-config.c:34:
util-threshold-config.c: In function 'SCThresholdConfInitContext':
util-debug.h:260:5: warning: '%s' directive argument is null [-Wformat-overflow=]
260 | SCLogErr(SC_LOG_WARNING, __FILE__, __FUNCTION__, __LINE__, _sc_module, __VA_ARGS__)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
util-threshold-config.c:190:9: note: in expansion of macro 'SCLogWarning'
190 | SCLogWarning("Error loading threshold configuration from %s", filename);
| ^~~~~~~~~~~~
util-threshold-config.c:190:66: note: format string is defined here
190 | SCLogWarning("Error loading threshold configuration from %s", filename);
| ^~
Instead of indexing `Packet::alerts` to find out if the action should
be `ACTION_PASS`, add an arguement to make this explicit from the alert
logging logic.
Bug: #7630.
Pass-only rules were previously stored in Packet::alerts, but not
correctly. The `cnt` wasn't incremented leaving it in a weird state.
This patch changes the logic to have a `pass` entry be added to the
queue as well as incrementing the `cnt`. As a consequence of this the
alert logging needs to explicitly check for `PacketAlert::action` having
`ACTION_ALERT` set now.
Update the alert loggers.
Bug: #8021.
ACTION_ALERT is supposed to be set now, but wasn't for rate_filter.
This becomes important in a follow up patch where only PacketAlert
entries with `action` `ACTION_ALERT` are logged in alert logging.
Realloc alp_ctx.ctxs when a dynamic alproto is registered and
g_alproto_max increases. So dynamic alproto can be treated as
real/normal ones. And app-layer switch can be set to any value
of no/deteciton-only/yes.
Ticket: 8000
VarNameStoreRegister can return 0 in case of any error conditions.
Handle this case in all the users of this function. It is an unlikely
event so add branch assistance accordingly.
Bug 8054
Most call sites use a casula SCFree fn to get rid of the DetectXbitsData
in case of an error. However, if a varname is also registered, that must
be unregistered too. Fix all the free calls by replacing them with the
appropriate DetectXbitFree fn.
The fn DetectFlowvarDataFree checks if the pointer (DetectFlowvarData)
passed to it is valid. So, the checks for NULL in the call sites can
safely be removed.
The fn DetectFlowintFree checks if the pointer (DetectFlowintData)
passed to it is valid. So, the checks for NULL in the call sites can
safely be removed.
In case of an error a varname id is set to 0. Ideally, it shouldn't be
found in the hash table lookup but add a check anyway to avoid obtaining
the mutex lock and performing the lookup.
81ee6f5aad ("lua: push correct length back through ScFlowvarGet, work around valgrind warning")
added a workaround for valgrind warnings in pushing a string buffer
into the lua state. This is no longer needed as tested with both
address sanitizer and valgrind.
Fix cases where the wrong free function was being called in error
handlers.
DetectHTTP2sizeUpdateSetup was calling DetectHTTP2settingsFree instead
of DetectHTTP2sizeUpdateFree in error case.
Moving http2.priority and http2.window to multi-integers, instead
of basic integers only modified the Free callback, but the
Setup function was still using the direct call to old obsolete free
function.
Using the callback Free abstration in Setup, allows to be
consistent and have less code to change.
Issue: 7819
DetectEngineReload must hold the `master->lock`; recent changes changed
the locking usages to avoid deadlock when registering/handling tenants.
These changes added the presumption that the master lock is held at a
higher level. Coverity highlighted that the lock is not held
consistently.
Tests better the ignored flags functionality.
This functionality is only used at detection for default mode,
it is not used for any, plus or not mode.
So, have unit tests about igored flags with default mode with
both matching and non matching cases
Ticket: 6724
Allows to use numerical values for example
Also fixes some unit tests that were returning 1 after goto error
FlagsTestParse05 especially took this path as
de->ignored_flags != (TH_SYN|TH_RST) was false
we had de->ignored_flags == 0xff ^ (TH_SYN|TH_RST)
And then, we had a match, instead of what the not-run code
was supposing.
SSL connp maintains all the state and certificate data that was
parsed/decoded successfully and it must retain that for later usage.
There should be just one place to free this object which is SSLStateFree
for both the directions. By freeing the connp data during parsing error,
there is room for memory errors.
This works so far because the field parsed after this cannot error out so
if there's an error parsing this, it anyway does not exist. However, this
is incorrect and leaves scope for mistakes.
Remove this extra free and treat SAN like all other TLS keywords.
Bug 7996
Replace keyword updates a prior content with a heap allocation of the
pattern the content should be replaced with. Make sure this is freed as
well.
Bug: #7997.
Ethertype values are now converted from network format to host format
before display occurs. Displayed values are now in hex instead of
integers.
Without this change, ethertype values such as 0xfbb7 are
displayed in decimal as: 47099 (0xb7fb).
The actual value is 64439 (0xfbb7); all logged ether_type values
will be displayed in host order in decimal format. This example
will log the ether type as 64439
Issue: 7855
Check bail condition before entering conditional wait.
CID 1554236: (#1 of 1): Indefinite wait (BAD_CHECK_OF_WAIT_COND)
dead_wait: A wait is performed without ensuring that the condition is not already satisfied while holding lock ThreadVars_.ctrl_mutex. This can cause unnecessary waiting if the notification happens before the lock is acquired.
CID 1554238: (#1 of 1): Indefinite wait (BAD_CHECK_OF_WAIT_COND)
dead_wait: A wait is performed without ensuring that the condition is not already satisfied while holding lock ThreadVars_.ctrl_mutex. This can cause unnecessary waiting if the notification happens before the lock is acquired.
Ticket: 7973
Files were supported on both TCP and UDP. But file detection keywords
such as file.data made signatures loading fail if the signature
was using an app-layer protocol that enabled on udp only, even
if the signatures could run smoothly.
DPDK Bonding API has been changed in DPDK version 23.11 where
the old *slave* API was marked as deprecated and the new *member*
API was marked as experimental.
This was unfortunately executed by marking both API variants
at the same time. The deprecated version is removed from the follow
up versions while the experimental version will become stable
in the next DPDK releases. This is based on a policy in DPDK where
an API change needs to merged in main for 1 stable release before
removing the experimental flag.
In DPDK 24.11 this has been fixed and warning supression is not
added.
Ticket: 7009
The exclude function incorrectly performs a XOR operation. While it
works when the worker cores occupy all cores, it is not the correct
operation. For example, when a core is affined to only management
and not worker threads, the XOR operation affines it to the worker set.
(1 XOR 0 -> 1, where in fact the desired outcome is 0)
Ticket: 7975
When the decoder finds a packet with wrong IP version for
IPv4|IPv6-in-IPv6 tunnels, it would set an event, but wouldn't flag the
packet invalid. This fixes that.
Bug #7964
Philippe Antoine
Jeff Lucovsky
Jason Ish
Lukas Sismis
Victor Julien
Adam Kiripolsky
Shivani Bhardwaj
Juliana Fajardini
Philippe Antoine
Ofer Dagan
Mingyu Jeon
Fupeng Zhao
Abhijeet Singh
Sergey Zhidkih
Shivani Bhardwaj
Giuseppe Longo
Ali Saidi
Ambre Iooss
James Kaddu
Jhonny Sousa
Olusegun Fajobi
Cheryl Owala
Li Heng
Ezra Adeyinka
Amir Boussejra
Jeff Lucovsky