Reading a pcap from /dev/stdin or a named pipe currently fails with "failed to get first packet timestamp. pcap_next_ex(): -1" because InitPcapFile calls setvbuf on the FILE* underlying the pcap handle after libpcap has already consumed the pcap header. On a non-seekable fd glibc cannot recover from that and the next read returns -1.
Detect non-regular files via fstat and skip setvbuf in that case so the read keeps working on pipes, fifos and stdin.
Accept pcap-file.buffer-size values of 0, which disables setvbuf buffering as an explicit opt-out, or PCAP_FILE_BUFFER_SIZE_MIN (4 KiB) to PCAP_FILE_BUFFER_SIZE_MAX (64 MiB). Treat any non-zero setvbuf return value as an error, not just negative values.
When pcap-file.buffer-size fails to parse, retain the default buffer size instead of falling through and setting it to 0. The branches are now mutually exclusive so only one of the parse-error, accepted, or out-of-range messages is logged.
Update the user guide: --pcap-file-buffer-size now documents valid values of 0 (disables setvbuf buffering) or 4 KiB to 64 MiB, and pcap-file.rst notes that 0 is the opt-out for non-seekable sources such as stdin and named pipes.
Bug: #8464.
DetectEngineThreadCtxInitKeywords returns TM_ECODE_FAILED when a per-thread keyword init fails (for example DetectFilemagicThreadInit), but ThreadCtxDoInit discarded that result and still returned OK. The detect thread then ran with a partially initialized keyword context array, producing indeterminate results. Propagate the failure so the callers abort thread init and clean up.
Add a unit test that registers a keyword whose thread init fails and verifies that DetectEngineThreadCtxInit reports the failure.
Ticket: #8237
For rules that specify an explicit app-layer hook,
e.g. http1:request_headers, don't register inspect engines for
other protocols like HTTP/2. These have their own progress tracking,
so should be excluded from these rules.
EveEmailLogJsonData() freed the provided SCJsonBuilder when
SMTP state was unavailable, even though ownership remains with
the caller. The caller may continue using the builder for
cleanup after the function returns failure.
Return failure without freeing the builder and add defensive
NULL checks in EveSmtpDataLogger().
Signed-off-by: Urval Kheni <urvalkheni777@gmail.com>
SCDetectEngineRegisterRateFilterCallback() dereferences the result of
DetectEngineGetCurrent() unconditionally. Add a NULL guard with
SCLogError and return false so callers can detect registration failure.
Flagged by Svace and confirmed by gcc -fanalyzer.
Ticket: 8560
Add DEBUG_VALIDATE_BUG_ON() at four sites in DetectEngineSignatureIsDuplicate()
where HashListTableLookup() is assumed to return non-NULL. Documents the
invariant that every Signature in sig_list has a dup_sig_hash_table entry
and catches violations in debug builds.
Flagged by Svace static analyzer.
Ticket: 8635
PacketAlertCreate is called from PacketInit on the packet allocation
path. Make PacketInit return bool and propagate the NULL result from
PacketAlertCreate up through PacketGetFromAlloc, which already returns
NULL to signal allocation failure to its callers.
Update the UNITTESTS-only helpers in defrag.c accordingly: helpers
returning Packet * use an explicit NULL check; the one returning int
keeps the existing FAIL_IF style.
Two fixes:
- Remove extra parentheses in existing NULL check: (*ext) -> *ext,
which was causing the cocci script to miss the check as a false negative.
- Simplify SCHSConfigInit to return SCCalloc() directly; the caller
in detect-engine.c already checks the return value for NULL.
The original pointer was overwritten with the SCRealloc result before
checking for NULL, causing a memory leak if reallocation fails.
Check the temporary pointer first before assigning.
When Redis output is configured in stream/xadd mode with a positive
stream-maxlen, SCConfLogOpenRedis() allocates redis_setup.stream_format
and immediately passes it to snprintf().
If SCCalloc() fails, snprintf() receives a NULL destination pointer and
the process can crash during Redis output initialization. Handle this
unrecoverable setup failure with FatalError(), matching the surrounding
Redis initialization error handling.
Ticket: 8588
Track the first DATA chunk's data offset and length during chunk iteration,
then reassign p->payload to point at the user data.
When no DATA chunk is present (INIT, SACK, HEARTBEAT, etc.),
payload_len is set to 0 since there is no application data.
Ticket #4251
Extend the SCTP decoder to parse chunk headers after the 12-byte common
header. Each chunk is validated for minimum header size and length
consistency per RFC 4960 sec 3.2.
Add SCTPChunkHdr and SCTPVars structs to track per-packet chunk metadata
Add five new decoder events for protocol violations:
- SCTP_CHUNK_TOO_SMALL: insufficient data for a chunk header
- SCTP_CHUNK_LEN_INVALID: chunk length < 4 or exceeds packet
- SCTP_INIT_CHUNK_NOT_ALONE: INIT/INIT_ACK bundled (RFC 4960 sec 6.10)
- SCTP_INIT_WITH_NON_ZERO_VTAG: INIT with vtag != 0 (RFC 4960 sec 8.5.1)
- SCTP_DATA_WITH_ZERO_VTAG: DATA chunk with vtag == 0
Ticket #4251
Call a SPARC-specific helper during pre-init so the kernel emulates unaligned accesses for this process instead of terminating on alignment faults. This matches the Solaris/SPARC portability fix that motivated the original patch.
Replace uses of the non-standard sun macro with __sun and use Solaris byte swap helpers from <sys/byteorder.h>. This keeps the Solaris-specific code paths reachable when building with standard-conforming toolchains.
toggle command is not used by any major rulesets and increases the state
complexity of flowbits management. Also, all operations can be carried
out with the combination of other available commands. So, remove it.
Task 8595
At some point, bindgen include ordering changed such that
AppLayerGetFileState was being bindgen'd as opaque, as the definition of
StreamBufferingConfig was not available when bindgen hit
AppLayerGetFileState, and bindgen processes in order.
Move the util includes before the app-layer includes to fix the ordering
problem, but still keep util includes grouped.
The sys diff is large as many things have been re-ordered.
Ticket: 8619
Do not wait to run inspection on the other side of transactions
as we do not run any tx detection on passing flows.
Avoids accumulating txs
Ticket: 8518
Keywords that work for HTTP2 headers match now as soon as possible
A push promise is now considered like a headers frame with regards
to the progress (no dedicated "reserved" progress/state)
http.protocol and http.stat_msg keywords are now registered at
earliest progress, since these are synthetic like "HTTP/2" and
not really seen on the wire.
http.request_line and http.response_line match only on data,
and not on headers, since we must wait the end of headers
to be sure to have the full line
http2.size_update now matches at headers progress as it should
http2.frametype, http2.errorcode, http2.priority now match like
http2.window, when the tx is complete from both sides, as a
half-closed client may still send priority, rst_stream
or window_update frames
Add detection keywords for LLMNR protocol inspection:
- llmnr.queries.rrname
- llmnr.answers.rrname
- llmnr.authorities.rrname
- llmnr.additionals.rrname
- llmnr.response.rrname
LLMNR reuses the existing DNS functions since it follows the same
protocol structure. The implementation registers LLMNR-specific keywords
that leverage the DNS data extraction functions.
Both to_server and to_client directions are supported.
Ticket #8366
This adds an LLMNR protocol logger that reuses existing DNS functions,
following the same approach as the mDNS logger:
- No grouped logging
- Rdata is logged in a field that is named after the rdata type
- Types are logged in lower case
- Flags are logged as an array
Ticket #8366
This adds a parser for LLMNR protocol over both UDP and TCP.
The parser reuses the existing DNS functions since LLMNR shares
the same wire format as DNS, but implements its own protocol-specific
validation:
- LLMNR-specific flag checks (C, TC, T bits)
- Opcode validation (only standard query opcode 0 is valid)
- An Event is set if Z-flag is set, since it's must be zero per RFC4795
LLMNR transactions inherit DNS behavior where each packet creates a new state
with one transaction.
Ticket #8366
Use for IPv6 in the main hash.
Update IPv4 thread cache to also use a seed. For this the thread
initialization is done at thread init instead on the first packet. The
thread ctx is no longer stored in a thread_local, but instead uses the
thread storage API to store in the ThreadVars. This way it will stay
active during rule reloads as well.
Ticket: #8617.
Break out the 3 options: match, partial match, no match for firewall
into separate functions.
Additionally, handle the re-match case for matches on a hook where the
progress value didn't yet progress further. In this case the continue
inspection logic revisits the rule and the accept needs to be
re-applied.
Ticket: 8001
Will allow algorithms such as ac to use the MpmCtx, especially
the number of patterns, to preallocate some structures.
For detect engine PatternMatchThreadPrepare, create a dummy
MpmCtx out of all the MpmCtx in the DetectEngineCtx,
and get the max value for the number of patterns
For the last for progress case we can just break on a firewall drop.
For the accept:flow and accept:tx cases the next sig (if any) will check
the flow/tx flag and manage flow control from there.
Simplify pre-check logic. Pre-check takes care of enforcing
FLOW_ACTION_ACCEPT, APP_LAYER_TX_ACCEPT and default policy enforcement
before the current rule's hook. This should be done in firewall mode for
each rule regardless whether it is a firewall or TD rule.
No need to track verdict state anymore.
More clearly define the relationship between PacketAlerts for firewall
and threat detection events.
Also no longer count firewall_discarded if drop rule came before a
another rule, causing the later rule to not be evaluated/alerted.
When packet:filter and app:filter alert appear in a single alert queue,
handle accept:hook by keeping track of the detect_table.
Support `alert` as a secondary action in packet firewall policies.
To implement this a Signature object is created per policy that uses
alert, and this is stored in a array table. When the policy is applied
the signature is looked up and used in the PacketAlert.
Ticket: #8566.
Support `alert` as a secondary action in app-layer firewall policies.
To implement this a Signature object is created per policy that uses
alert, and this is stored in a hash table. When the policy is applied
the signature is looked up and used in the PacketAlert.
Ticket: #8566.
Allow a single rule to accept a hook and the hooks prior to it.
Example:
accept:flow tls:<client_hello_done ... \
tls.sni; content:"suricata.io"; endswith;
This will evaluate the SNI at the client_hello_done hook, but will
act as if there is a `accept:hook tls:client_in_progress ...` as well.
Implementation is currently specific to this `<` operator. During
parsing the sig gets flagged for this case. During setup this has 3 main
effects:
1. prefilter is disabled as we need to eval this right at the first
state (0)
2. for state 0 a non-PF "prefilter" engine is setup to make sure the
rule is flagged for evaluation
3. In the Signature::app_inspect list a dummy inspect engine is
registered per state before Signature::app_progress_hook
The matching logic is building on the stateful rule handling. The
stateful rule handling can now tell the inspection loop that a partial
match occured. For this rule type the partial match will act as a match
with action accept:hook.
Next app updates will then use the continue detection logic to continue
the stateful match. When that fully matches, the final actions are
applied, like accept:flow or accept:tx.
Ticket: #8472.
In last_for_progress handling set accept only on packet if it was also
triggered on the last tx.
If there are more transactions, the accept can be set later (if policy
allows).
Add a `firewall` stats counter aggregator for all firewall-related
stats.
De-overload "detect.alert_queue_overflow", by adding
"firewall.discarded_alerts" to account for discarded drops in
Firewall mode.
Add Debug statements for tracking corner cases where it can be
difficult to know where a drop is coming from.
Added counters:
- stats.firewall.blocked
- stats.firewall.accepted
- stats.firewall.rejected
- stats.firewall.drop_reason.default_app_policy
- stats.firewall.drop_reason.default_packet_policy
- stats.firewall.drop_reason.flow_drop
- stats.firewall.drop_reason.pre_flow_hook
- stats.firewall.drop_reason.pre_stream_hook
- stats.firewall.drop_reason.rules
- stats.firewall.discarded_alerts
Ticket #7699
Always make sure, that smd is not NULL, as we dereference it a few
lines below and base64_data for instance is forbidden as a fast_pattern
Ticket: 8504
If a runmode is missing but other params are passed on the commandline,
issue an error.
Ticket: #5711
Signed-off-by: Promise Charles <descencybobby@gmail.com>
Support scan-build's taint filtering logic. Adds a yaml config to mark
a dedicated debug validation function as filtering a taint.
Use to suppress warnings in reference, threshold and classification file
handling.
Ticket: #3153.
detect-engine.c:2350:69: warning: Out of bound access to memory after the end of 'new_det_ctx' [security.ArrayBound]
2350 | FlowWorkerReplaceDetectCtx(SC_ATOMIC_GET(s->slot_data), new_det_ctx[i]);
| ^~~~~~~~~~~~~~
1 warning generated.
suricata.c:574:5: warning: Potential out of bound access to the heap area with tainted index [security.ArrayBound]
574 | bpf_filter[nm] = '\0';
| ^~~~~~~~~~~~~~
1 warning generated.
Ticket: 8529
When Suricata handles a HTTP1 response body, it does so with a
file, and tries to get the filename from the Content-Disposition
header if any, then from the uri.
If it failed to find a file name, it tried again every time
there was new data from the response body, even if there was
no new data to find a file name in either the header nor the uri.
This causes a slowdown in the case the Content-Disposition header
is big.
Fix is to set the flag on the first call of the callback, to be
sure that we will parse the Content-Disposition header for a
filename header only once per http1 response.
Decoding the reassembled packet inside DefragInsertFrag while holding
the tracker mutex creates a deadlock situation if the reassembled packet
contains tunneled fragments: two threads could each hold one tracker
mutex and deadlock on the other. Move the decode in Defrag() after
DefragTrackerRelease() so no lock is held during decode (and possibly
re-entering the defrag system).
Make sure to fetch tracker->ip_hdr_offset after DefragInsertFrag since
it may be written during that call if the first fragment arrives last.
Issue: 8550
Ticket: 8526
As some C protocols accepted case-insensitive frame names,
a rule using the same frame twice with different cases,
ended up using 2 different buffers,
which ended up in triggering a buffer overflow
Ticket: 8510
Without this check, in case of a hash collision, and the other
fields are equal, we could end up consider adding an IPv6 packet
to a IPv4 tracker (or vice versa).
Doing so, we end up interpreting an IPv6 packet as an IPv4 one,
and we do not benefit from the bounds checks from ipv4/ipv6 decoder.
Allow configurable policies, including accept. For app-layer this
requires looping all available hooks to apply the policies.
Support configurable policies for packet-filter, pre-stream, pre-flow.
If there are no rules there is also no rule group (sgh). Make sure
the app hooks policies are correctly handled in this case by allowing
a NULL sgh to be handled as well.
For tx rule match actually apply drop directly. Previously this was
always handled by the default drop:flow policy.
Ticket: #7701.
Previously a `accept:flow` action would act as both a firewall "accept" and
a threat detection "pass" for the rest of the flow.
This patch changes that. The `accept:flow` action now only accepts the
rest of the packets for the firewall ruleset, but does still continue
threat detection rule evaluation.
Ticket: #8444.
For firewall rules, allow multiple actions to be specified in a list
accept:flow,pass:flow,alert
accept:flow,alert
accept:flow,pass:flow
It is mandatory to make the first action the primary firewall policy
action: accept, drop, reject.
Ticket: #8480.
If a ruleset would use `dns:request_complete` but not have a rule for
`dns:request_started`, the `request_started` hook default policy would
not get invoked.
Add a check to make sure it is invoked.
Ticket: #8495.
Clean up host mode tracking, which is used by reject to control how
rejects are sent. Before this patch there were 2 modes: sniffer only
and router. This patch introduces a bridge mode that is automatically
set by the bridge modes. In bridge mode the `Packet::livedev` is used.
Ticket: #8390.
Ticket: 8336
At the end of a TLS handshake, in IDS mode, the client acks,
and we parse the server hello and use tls.encryption-handling
to know what to do next (for example bypass)
Everything is parsed, but we have not run detection yet on neither
side.
So, in IDS mode, we need to first flush the client side, as the
comment on the function already stated.
Ticket: 8336
When a packet has flag PKT_PSEUDO_DETECTLOG_FLUSH, we do not
expect to rerun detection on the same tx and direction again
So, do not set mpm_in_progress whose purpose is to not store
the state as we will run again.
Allows transactional bidirectional signatures to work on
thse log+flush pair of packets
When an alert is generated from firewall context, add an engine value of
"fw", otherwise "td" (for threat detect).
The engine field is only added when firewall mode is enabled.
Ticket: #8456
The first version of the event set done=true on state->curr_tx and
returned NULL when the limit hit. curr_tx is usually the active
command just received — marking it done mid-request throws off
response matching, and subsequent commands in the flow stop getting
logged.
SMB behavior is mirrored here:
Walk the tx list, find the oldest tx that isn't done, mark
it done and tag it with the event, then fall through and
create the new tx so the flow parsing continues. One stale tx gets
reaped per overflow so memory stays bounded.
Issue: 8489
Other parsers (SMB, POP3, ENIP) already raise a too_many_transactions
event when a flow exceeds its per-flow transaction limit. FTP had
the limit wired up through app-layer.protocols.ftp.max-tx but just
dropped the excess transaction on the floor with a "FTP does not set
events yet..." TODO, so no event was raised when the limit was
reached.
Add the FtpEventTooManyTransactions variant, a stock rule at
sid 2232002, and raise the event from FTPTransactionCreate when the
live-tx count goes past ftp_config_maxtx.
Issue: 8489
when engine-analysis is turned off in suricata.yaml
and we run suricata with --engine-analysis
suricata will try to use data from de_ctx->ea wich is
free'd in SetupEngineAnalysis() because fp_analysis and
rule_analysis is turned off and will receive SIGSEGV
with something like:
0x00007ffff7b4ac7b in _IO_new_fclose (fp=0xad96ab1cf5b38042) at ./libio/iofclose.c:48
Ticket: 8505
Issue: 7672
The subslice transform creates a slice of the input buffer.
Specify the subslice desired -- nbytes and truncate are optional:
subslice: offset <,nbytes> <,truncate>
offset: Specifies the starting offset for the new subslice. When
negative, expresses how far from the end of the input buffer to begin.
nbytes: Specifies the size of the subslice. When negative, specifies
that the subslice will end that many bytes from the end of the input
buffer. Nbytes must be non-zero.
When nbytes is not specified, the size of the subslice will be the size
of the input buffer - offset.
truncate: Specify behavior when offset + nbytes exceeds buffer length.
When present, trims nbytes such that offset + nbytes equals buffer
length. When not present, an empty buffer is produced.
Examples:
subslice: 1; - The subslice will be a copy of the input
buffer but omits the input buffer's first byte
"This is Suricata" -> "his is Suricata"
subslice: 0, 13; - The slice is created from the first 13 bytes
of the input buffer
"This is Suricata" -> "This is Suric"
subslice: 10, -5; - The subslice is created starting at offset 10
and continues to 5 bytes before the end of the input buffer
"This is Suricata" -> "r"
subslice: -3; - The subslice will be the last 3 bytes of the
input buffer.
"This is Suricata" -> "ata"
Ticket: 8501
(m)dns.*.rrname keywords required the rules to start with
alert (m)dns
and refused to load if the rule started with generic alert ip or
alert udp, with error message
Error: detect-parse: invalid alproto 0
When using ASAN, flto=auto and compiling on a platform that supports
AVX-512, this memcpy is optimized as its exactly 64 bytes. However, when
using ASAN, there is a fake stack that may not be 64 bytes aligned, and
this AVX write lands on an unaligned byte, causing a crash.
Use strlcpy avoids this optimization as it needs to find the NUL byte.
Ticket: 6813
So, in the case we have a ftp-data flow with data to client as
expected, but after to_client side completes (with timeout for
example), the client sends some data to server,
In this case, we want to check first the direction, and bail out,
then do the assertion check about the finished state.
As a legacy from Suricata 7, mempool size was kept even internally
as a global size. Since in Suricata 8, mempool allocation moved to
per queue model, it makes sense to calculate the correct size right
when loading the configuration.
The previous implementation misscalculated in some scenarios the size
of the mempool which prevented Suricata starting successfully.
This happened, e.g. when there were 32k RX descriptors and 64 TX descs.
The resulting mempool was incorrectly deduced as 32767 (2^n-1), which
didn't even hold all RX descriptors. Instead, it should result to 65535.
Ticket: 8370
If the last signature in the candidates list is also the last for the
current progress value, it needs to be considered as the last for that
progress, so that if it doesn't match the default policy is applied.
Ticket: 8395
For protocols where tx are complete at start, that is
the `tx_get_progress` function returns 1 inconditionnally,
we can use the progress 1, not losing time on detection in ips
mode, and getting ready for firewall mode
For the other protocols (where a tx needs both a request and response
for instance), keep progress 0 as was the case before
Fixes warning:
conf-yaml-loader.c: In function ‘ConfYamlSetConfDirname’:
conf-yaml-loader.c:87:8: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
87 | ep = strrchr(filename, '\\');
| ^
Add --plugin <PATH> to load an additional plugin from the command
line. This is more convenient than "--set plugins.X" especially when
you may already have a plugins loaded and you want to load an
additional one.
Ticket: 8463
CID 1645613
Ticket: 7664
We assign and dereference next_s like
next_s = *match_array++;
next_sflags = next_s->flags;
So, there is no need to check if next_s is NULL, as it can never
be (otherwise, we could crash before)
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream inspection which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
FTP parser can handle multiple command lines per direction. Appropriate calls
to trigger raw stream inspection have been added on succesful parsing of each
request line and response line.
Bug 7742
`alert pkthdr` was initially just an alias for `alert ip`, as that was
really just a way of stating that "any" should be matched. However with
the Ethernet matching in place, it no long makes sense to treat `alert
ip` as "any". Since `pkthdr` is used to match on decoder events, also
for packets that completely failed to parse, it should no longer be
treated as `alert ip` but rather as it's own distinct logic.
Sticky buffer to inspect the ethernet header.
Example rule:
alert ether any any -> any any ( \
ether.hdr; content:"|08 06|"; offset:12; depth:2; \
sid:1;)
Ticket: #8327.
Make `Signature::proto` an optional member, meaning that if it is
NULL we can skip the check. This can be done for `alert ip`, as no check
is needed, and for `alert tcp` and `alert udp` as having a rule in a sgh
for those means that the protocol matches.
Some exceptions are rules that require:
- ipv4/ipv6 specific matching
- frames, due to sharing prefilter between tcp and udp
- ip-only rules, due to those not being per sgh
Add flushing logic driven off of the file contexts. This is a simpler
solution that removes the need for logger registration changes.
Overview:
Use the heartbeat-driven thread to periodically flush all registered EVE
contexts via a global flush list.
The global flush list is a mutex-protected TAILQ of LogFileFlushEntry
nodes; each node points to a LogFileCtx. Mutex = log_file_flush_mutex
Periodic flushing performed by a thread according to the
heartbeat.output-flush-interval [1,60]. LogFileFlushAll() is invoked to
initiate flushing of registered LogFileCtx structs; each struct's
fp_mutex is obtained while the flush occurs to synchronize with
LogFileWrite activity.
Interacts with file-rotation via the fp_mutex.
Deadlock prevention: the log_file_flush_mutex must be obtained before
the fp_mutex.
Issue: 8286
Ticket: 8333
Allows to pass on pseudo packets
So, that a ruleset like
pass ssh any any -> any any (geoip:any,US,IN; sid: 1;)
drop ip any any -> any any (flow:established, to_server; sid:2;)
Does not lead to a drop on the pseudo-packet when we flush
because ssh traffic gets encrypted
Simplify statements where there's a conditional statement followed by
return true or return false.
Instead of
if (conditional)
return true;
else
return false;
Use:
return conditional
In some timezones, LogCustomFormatTest01 was failing as we were
specifying the wrong setting for is_dst. Instead set it to -1 so mktime
can make the decision based on the environment. Appears to fix this test
for "Brazil/East" in the summer.
Add GitHub CI tests to test this specific test with a few different
timezones.
When during raw reassembly it is detected that last ack is moved beyond
the progress and also beyond the data retrieved, update progress to the
last ack value.
Bug: #8272.
When mempool size was configured really low (<32), Suricata exhausted
the mempool with the rx_burst call, which led to undefined behavior.
The current fix ensures the burst size is at most the size of the mempool,
if the mempool is smaller than BURST_SIZE macro.
Based on review it seems p->proto should always be set to the same
value as what `PacketGetIPProto` returns. Add a check to validate
in a decoder that the fuzzer executes.
Move the locking mechanism outside of the getter function and hold the
lock until the reference string is no longer reused.
** CID 1682023: Concurrent data access violations (MISSING_LOCK)
/src/util-mpm-hs-cache.c: 139 in HSGetReferenceDbInfo()
This run-mode does not allow for capture ThreadVars to be created at
the appropriate place in the application life cycle. Instead,
applications should register their own custom run-mode.
Ticket: #8259
If the key `app-layer.protocols.PROTO.enabled` is present, the protocol
is enabled for all carrier protocols. This is not ideal.
Only if the key `app-layer.protocols.PROTO.enabled` is missing, an
attempt is made to look for a setting specific to the ipproto passed
at the time of registration e.g.
`app-layer.protocols.PROTO.udp.enabled`.
By default, check for carrier proto specific setting. If it is not
found, then fall back to the generic setting.
Issue a warning in case an inconsistent combination of global and
ipproto specific setting is found.
Bug 8205
DPDK interface which were added to bond and had auto-calculated
mempool size didn't have enough space as the configured
descriptors and the derived mempool only considered one interface
per YAML node entry.
DPDK Bonding PMD allows for multiple interface to cooperate and
therefore, needs more space.
Ticket: 8125
Following DPDK's recommendation to size mempool to 2^n - 1,
the current implementation was flawed for certain combos,
e.g. mp_size 1023 and 1 queue, which resulted to 1022.
Hash Hyperscan installation info (version, CPU features, mode)
into the cache filename. A Hyperscan upgrade or platform change
would now produce a different filename, so stale caches from an
older installation are never opened.
Ticket: 8325
After deserializing a cached Hyperscan database, verify that its
version, CPU features, and mode match the current Hyperscan
installation by comparing hs_database_info output against a
reference database. Reject loading incompatible caches.
Ticket: 8325
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.
Ticket: 8325
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.
Removed the unused fopen/fclose pair and flattened the control flow.
Ticket: 8325
Reduce logging level of a minor informational message.
Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.
Ticket: 8322
Ticket: 7717
Allows for instance to process/log ARP packets over VXLAN.
That means we need to decode the ethernet layer above vxlan
instead of skipping it as part of the vxlan, even if the vxlan
decoder still checks the ethernet layer to avoid FPs.
Instead of directly accessing the field
Will allow PacketTunnelType to hold the precise tunnel type like
DECODE_TUNNEL_ERSPANII with a modification of PacketIsTunnelChild
Refactor pcap file deletion to use a single delete-when-done option
with three values instead of separate boolean options:
- false (default): No deletion
- true: Always delete files
- "non-alerts": Delete only files with no alerts
Also account for alerts produced by pseudo packets (flow timeout / shutdown flush):
- Introduce small capture hooks and invoke on pseudo-packet creation so the
capture layer can retain references and observe alerts emitted after the last
live packet
- Call the hook from both TmThreadDisableReceiveThreads and TmThreadDrainPacketThreads
Key changes:
- Replace should_delete/delete_non_alerts_only bools with enum
- Move alert counter from global to per-file PcapFileFileVars
- Relocate alert counting from PacketAlertFinalize to pcap module
- Ensure thread safety for both single and continuous pcap modes
- Add unit tests for configuration parsing and pseudo-packet alert path
The --pcap-file-delete command line option overrides YAML config
and forces "always delete" mode for backward compatibility.
Documentation updated to reflect the new three-value configuration.
Fixes OISF#7786
Bindgen the Rust bindings to the C JsonBuilder API along with the rest
of the Rust bindings to C. Breaking it out was probably the wrong
idea.
This should make it easier, and more correct to bindgen C functions
that use SCJsonBuilder types.
There is an unfortunate side-affect that one has to read
output-eve-bindgen.h for the documentation on this type, however, I
think we can resolve that in time.
As output-eve-bindgen.h exists to support bindgen, its odd to see
other Suricata C files using it. Instead Suricata C code should import
"output-eve.h", which itself includes "output-eve-bindgen.h", only
broken out to support the external tool bindgen.
Replace duplicated SCFree() calls in error paths with a single
cleanup label using goto pattern. This reduces code duplication
and ensures consistent resource cleanup.
Additional improvements:
- Fix misleading error message when xstats table size changes
between calls (was passing positive value to rte_strerror)
- Use unsigned int for length/index to match DPDK API semantics
- Initialize xstats_names to NULL at declaration for safe cleanup
- Added "not supported" case when the first call to xstats_get returns 0
Ticket: 8273
... to account for midstream sessions.
Commit 497394e removed inspection of app-proto txs for packets
without an established TCP connection. But this meant that the
first packet seen in a session pick mid-stream could go without
inspection (previous bug 5510 seemed to point towards this behavior,
too).
If a flow has more packets, the stream will be inspected as part of
the upcoming packets and this would go unnoticed. In a single-packet
flow, however, the inspection for the packed would be skipped. Although
this might not affect alerts -- as they could be processed as part of
the flow timeout logic, the actual traffic could be evaded in IPS, in
case of a drop rule.
From the above, the most visible scenario is when there is only one packet on the flow,
as then the engine doesn't have "more time" to pick-up real-packets to
inspect for that given flow. But certain tests show that this can also
happen for more than one packet scenarios: there will be one less drop
event, or traffic from a packet that should have been already dropped
will be logged.
This led to the possibility of a real packet not being blocked, in IPS,
or matched against rules, as the corresponding portion of the stream
was only inspected later, as part of the stream/flow-timeout logic.
To ensure that we correctly flag the first packet seen for a given mid-stream
session, we must check for the session state and existance *after* we
have dealt with TCP flags and state.
Related to
Bug #5510
As part of
Bug #5180
Samaresh Kumar Singh
Philippe Antoine
Victor Julien
Urval
Denis Balashov
Giuseppe Longo
Martin Rehak
Antoine Abou Faysal
Shivani Bhardwaj
Jason Ish
William Ling
Giuseppe Longo
Juliana Fajardini
Sergey Pinaev
Promise Charles
Jeff Lucovsky
Jeff Lucovsky
Lukas Sismis
Adam Kiripolsky
Philippe Antoine
Ofer Dagan