Main runs the backstop only when the stream has a gap. Out-of-order file
data on a stream that never gaps therefore grows the queue past the
bound the backstop exists to hold, while a call that carried no data at
all still walked the transactions. Main's chunk-count bound and its scan
of every open file transaction stay as they are.
Gate the scan on the call carrying data and the state holding at least
one file transaction, tracked by a counter maintained with the
transactions list, and resolve the file handle after the chunk_left == 0
return so a call with nothing left does not consume one.
Ticket: #9263.
A queue rejection arms a skip, but every SMB1 and SMB2 read and write
handler still called set_file_left at its tail: a partial record left
rec_size - data_size bytes armed under a guid whose chunk the rejection
never armed. Nothing is misattributed today - with no chunk armed the
next update consumes nothing and drops the promise without skipping a
byte - but the armed value describes data that was never handed over,
and a future path that parses while a chunk is in flight would spend it
on the following record. Guard the trailing set_file_left with the
rejected flag so the promise only ever describes queued data, and keep
the SMB1 WRITE_AND_CLOSE close handling outside the guard so the close
is still processed when its data is rejected. Unit tests pin both.
Read the SMB queue and size settings through u32::try_from so an
over-u32 value logs and keeps its default instead of silently truncating,
and refuse a zero smb.max-tx, which otherwise set a live-transaction
limit that evicted every transaction on insert.
Ticket: #9263.
The claim the scanner skips as a word plus a name was NULL(0) and UNIQUE,
a value no version defines under that name. It is CLAIM_DELEGATE_PREV(3)
whose claim is a component4 name, the same one the parser reads after the
claim word, while 2 is CLAIM_DELEGATE_CUR: its stateid is what the name
length is then read from. So the scanner walked off the end of a compound
that parses fine, and rejected a claim whose record it could have
followed to the op behind it.
Accept 0 and 3, where the claim word is followed by the name the parser
consumes, and reject the others as a malformed compound within the
record's bounds, never as incomplete and never buffering toward the
claim. The test that pins the rejected set loses 3 and gains 2, and a
DELEGATE_PREV claim ahead of another op is now reached.
Ticket: #9263.
An OPEN reply that grants a delegation desynced the parser and the record
scanner that has to mirror it. The nfsace4's who was consumed as a length
and that many bytes, without the XDR pad that follows them, so what came
after the delegation was read from inside the name. The write arm counted
nfs_space_limit4 as two words, four bytes short of the word and uint64 it
is, and lost the same four one field further along. A SEQUENCE result was
skipped as 36 unlabeled bytes.
Consume a who in one helper that takes the length, the bytes and the pad,
used by both parser arms and by the scanner's skip, and read
nfs_space_limit4 as its 12 bytes. The write delegation struct loses the
who field no caller read. SEQUENCE4resok is read as the fields it is made of,
which is what the scanner's 36-byte skip stands for. The fixtures are
laid out per field and parsed to the last byte, and the change_info4 they
carry is no longer all zeros, so a skip that comes up short desyncs
instead of matching. The scanner's op helpers name the element type they
extend with.
Ticket: #9265.
The record scanners unwrapped the (0, 2) combination and took every other
one as a bare compound. A control procedure or a privacy compound was
then scanned as if it were a COMPOUND: the envelope length was read as
the first tag, so the scan either stayed incomplete and buffered toward
the claim, or rejected a record that has nothing to reject.
Scan by the combination the call recorded: hand the credential pair to
gss_compound_scan_data, which leaves (0, 1) raw because rpc_gss_svc_none
wraps nothing, unwraps (0, 2), and reports a privacy (0, 3) compound as
opaque. There is nothing to scan in an encrypted compound, so the record
is skipped to its boundary and neither parsed nor scored: the request
path keeps the call in the request map for its reply, the reply path
drops it. Any other combination is one the full record path rejects as
well, so it is malformed, boundedly and without buffering. The v4
request path records the combination whatever it is, not only when it
unwraps.
Ticket: #9263.
The args of a WRITE request and of a READ reply were trusted to fit the
record that carries them. Where the claim fell below the extent the parse
covers, the file data ran past the record boundary into the next RPC,
which was then read as a record. A max-tx of 0 was accepted and evicted
every transaction on insert.
Compare the claim against that extent: for a WRITE the handle, offset,
count, stable flag and its data, for a READ the status, the optional
attributes, the count, the eof flag and its data. A record whose claim is
below the extent gets a transaction with the malformed data event and a
skip to the record boundary, as an oversized claim already does. Only the
v3 records arrive in pieces, so process_read_record now learns which
reply it is given and the check is v3's. Reject a max-tx of 0 at config
load.
Ticket: #9263.
A partial v3 reply that carries no file data is skipped rather than
buffered toward its claim, which for LOOKUP, READDIRPLUS and CREATE drops
the handle to name mapping their replies establish. The exception to that
reused max-read-size, so those procedures depended on a knob that
documents READ data, and lowering it for READs silently dropped the
mapping of any directory reply above it; the mapping depends on
segmentation, as the same reply parses when it arrives in one piece.
Give them a limit of their own, max-metadata-size, with a 16 MiB default,
and use it to decide whether such a reply may still buffer toward its
record claim. A value of 0 falls back to the default, so the buffering
stays bounded and the limit cannot be disabled. The WRITE and READ limits
share that default through NFS_MAX_RW_SIZE_DEFAULT. Documented in the
userguide.
Ticket: #9263.
The queue-limit work computed the record tail and the chunk promise from
the declared size, and added a count or a data length to it. A padded or
segmented record therefore left a promise armed past the end of its
record: a continuation could run on into the next RPC, an input that
carries only record slack grew the out-of-order queue and truncated a
file whose queued data fits, and the backstop walked the transactions of
a call that holds no file transaction at all.
Take the promise from the record itself, with record_tail(): what its
marker claims minus what the parser holds. Keep its two halves apart:
the stream side is the record tail, the queue side is the file data the
tracker still misses, so a continuation queues no more than its own
bytes. A pad a record carries is inside its claim, so the tail drains
it; it is neither file data nor fill. Release a promise whose
transaction is gone instead of consuming toward the claim, run the
backstop only when the call carries data and the state holds a file
transaction, and count a chunk only for input that carries file data. A
consume past the end of an input is now a debug assertion, with the
advance clamped to the input.
Ticket: #9263.
A WRITE request or READ reply with a count of 0 has no file data, and
processing it returned early. For a partially received record this left the
rest of the record unaccounted for, while the input was reported as consumed.
The rest of the record was then parsed as the next RPC record, which failed
and disabled the parser for the flow.
Like the complete record path, skip the rest of the record and continue with
the next record. For a READ reply this includes removing its request from the
request map, as is done for a complete reply.
Ticket: #9188
When a partially received record failed to parse, for example a WRITE with an
invalid stable value or a READ reply with an invalid eof, the parser returned
an error, disabling the parser for the flow. Yet the RPC record marker had
already given the size of the record.
Like the complete record path already does, set the malformed data event,
skip the rest of the record and continue with the next record. For a READ
reply this includes removing its request from the request map. A record
marker that cannot be parsed is still a fatal error, as the boundary of the
next record is unknown.
Ticket: #9190
The continuation of a partial WRITE request or READ reply only covered the
file data, not the XDR padding that follows it in the record. When a delivery
of stream data ended on the last byte of file data, or inside the padding,
the remaining padding was parsed as the start of a new RPC record, which
failed and disabled the parser for the flow.
Keep the chunk active while its record still has unconsumed bytes, as tracked
by the record tail, and consume exactly those bytes: from the input that
completes the file data and, for what is not there yet, from the next
deliveries. If the file is still open the padding is passed to its file
tracker, which consumes it as fill bytes, never as file data.
A partial READ reply that already holds all of its file data now also has its
request removed from the request map, as is done when a chunk completes.
Builds on the file handle of the in-progress READ chunk added for #9192.
Ticket: #9194
The continuation of a partially received READ reply looked up its file handle
in the request map by the reply's xid. As the request map is bounded and
evicts the least recently used entries, enough other requests on the flow
could evict the entry while the reply was still in progress. The continuation
then consumed nothing and its file data was parsed as a new RPC record, which
failed and disabled the parser for the flow.
Keep the file handle of the in-progress to-client chunk in the state, like
is done for the to-server WRITE chunk, and clear it along with the rest of
the chunk state.
The file transaction itself can also go away while its chunk is in progress,
for example when the max-tx limit forces it closed, after which a READ request
for the same handle creates a new transaction. A transaction found for the
handle whose file is not open, which is also the case after a complete READ
reply with eof set, cannot take the chunk either. In these cases consume the
chunk up to its known record boundary instead of parsing its data as RPC
records or passing it to that transaction.
Ticket: #9192
A NFSv3 WRITE with stable set to FILE_SYNC is assumed to be the last chunk of
its file, and its file transaction was closed as soon as the record was seen.
If the record was only partially received, the continuation could no longer
find the transaction by its file handle. Its file data was then not consumed
but parsed as a new RPC record, which failed and disabled the parser for the
flow, and the file was truncated.
Like the READ reply path already does, only close the transaction right away
if the record holds all of its file data. Otherwise filetracker_update closes
it once the file tracker is done or, as a truncated tracker is never done,
once the data of the last chunk is complete. The latter also applies to a
partial READ reply with eof set.
The NFSv4 WRITE path makes the same FILE_SYNC assumption, but it only handles
complete records, so it is not affected.
This is also the root cause of Redmine #8009.
Ticket: #9196
Rust unit tests link without libsuricata, so a C reference anywhere in
the reachable code makes the test binary unlinkable. The ffi logger
calls SCLogMessage even under cfg(test), unlike the suricata crate's
own, which has a print fallback. Any test that reaches
AppLayerTxData::update_file_flags therefore breaks cargo test --lib
whenever the debug feature is on: it fails the GitHub jobs that build
with --enable-debug or --all-features while our own CI never sees it.
Give log_message the same fallback behind cfg(any(test, feature =
"testing")), the pattern applayer.rs already uses for set_event.
Windows has no SIGHUP and no unix socket, so Suricata could not be told to
reopen its log files after external log rotation. When running as a service,
this can now be requested with the user-defined control code 129, for
example:
sc.exe control Suricata 129
The service control handler only records the request; the main loop then
handles it the same as SIGHUP.
Ticket: 7757
On Windows, fopen() does not open files with FILE_SHARE_DELETE, so a log file
can not be renamed or deleted while Suricata has it open. This prevents
external tools from rotating log files.
Add SCWin32Fopen(), an fopen() replacement for the "a" and "w" modes that
opens the file with CreateFile() and FILE_SHARE_DELETE, and use it for log
files, including suricata.log.
Ticket: 7757
Windows has no SIGUSR2 and no unix socket, so rules could not be reloaded
without restarting Suricata. When running as a service, a rule reload can now
be requested with the user-defined control code 128, for example:
sc.exe control Suricata 128
The service control handler only records the request; the main loop then
handles it the same as SIGUSR2.
Ticket: 7756
By default, a Windows service allows interactive users and service accounts
to send it user-defined control codes (128-255). When installing the Suricata
service, set a DACL that is the default DACL without this right for those
users, so only SYSTEM and Administrators can send them.
This prepares for using user-defined control codes to control Suricata at
runtime, such as reloading rules.
Ticket: 7756
So far the SSL parser returned an error if there were less than 4 bytes
available for the handshake header. This meant that a fragmented
handshake header could result in the SSL parser erroring out and thus
disabling any further inspection for the protocol.
The parser follows a multi-record parsing scheme which means that the
fragment of the header may exist in the next record. Use internal
buffering to assess and process the records just like the handshake
buffer.
Ticket: 9167
Ticket: 8684
The behavior of the rule
alert ip any any -> any any (pcre:"/.*mc|\H+$/"; sid:4;)
depended upon the presence of another rule such as
alert ip any any -> any any (content:"*mcH"; sid:5;)
With the sid 5 content rule, the code went through PrefilterPktStream
which updated raw_stream_progress, which ended up updating raw_progress_rel
which was used on later packets to know where to start.
Without the sid 5 content rule, there was no update of these progresses.
It led to quadratic complexity, as for each new packet, we ran
the pcre engine on the whole tcp stream from the beginning, with
the newly added payload data.
Fix is to process sid 4 pcre rule the same way whether there is
or there is not a sid 5 content rule.
So, in case we do not have a mpm store for a signature group head,
we register a dummy stream prefilter, whose goal will just be
to update raw_stream_progress
DCERPC REQUEST/RESPONSE records can have their stub data fragmented
across multiple PDUs. The flags that convey the state of the current PDU
are as follows:
PFC_FIRST_FRAG: first fragment of (fragmented) data
PFC_LAST_FRAG: last fragment of (fragmented) data
PFC_FIRST_FRAG && PFC_LAST_FRAG: no fragmentation
Consider a REQUEST/RESPONSE complete only when the PFC_LAST_FRAG flag is
found to be set.
Ticket: 9240
Serialize service arguments with Windows quoting and backslash escaping
for both installation and parameter updates. Reject truncated executable
paths and command lines before accessing the Service Control Manager.
Keep the existing service-option prefix matching unchanged. Limit the
fix to command-line construction, without changing the command-line
parser, service function interfaces or SCInstance layout.
Ticket: 9241
With rotate-interval set to minute, hour or day, rotations were not
aligned to the local time boundaries. The first rotation was computed
by SCGetSecondsUntil(), which overshoots the next hour by one minute
(hh:01:00) and the next day by one hour and one minute (01:01:00).
Every following rotation was re-armed with a fixed interval
(e.g. 86400 s) from the time of the rotation.
Local days are not always 86400 s long, so the day rotation also moved
by the DST shift on every DST transition and stayed there until the
next one. A file named after a date then held events of the next day.
Instead, this commit rotates logs when the local minute, hour or date
changes. The daily rotation now happens at local midnight.
On SIGHUP, the period is recorded before the file is reopened, so a boundary
(the next minute/hour/day) passing during the reopen leads to an extra
reopen instead of a missed rotation.
Ticket: #9139
In continuous pcap-file directory mode, a file that makes InitPcapFile()
fail (malformed content, not a pcap, unsupported link type, truncated
header) was skipped without folding its mtime into last_time_seen.
This caused the file(s) to be reinspected on every follow-up poll until
a newer valid file was processed.
Ticket: 9129
The passed poll-interval should determine how many seconds to wait
in-between polls. The previous code passed the value through localtime
function which understood the input integer as an epoch time and
converted it to time structure, of which only seconds were kept.
This kind of worked for sub-60 values but wrapped for values 60+ to
the usual seconds range 0-59. Additionally, setting poll-interval to
60 made it a busy loop with no sleep between polls.
With intervals of 60+ seconds now honored, a single sleep blocked the
shutdown (or a unix socket pcap-interrupt) until the interval elapsed,
and the engine was killed after the 60 second receive thread shutdown
timeout. The wait is now done in one second steps, checking the run
status in-between.
Ticket: 9129
Change the byte order of vlan TCI when extracting it
from vlan_hdr in xdp_filter.c. This fixes xdp bypass for vlan tagged
packets, as without the fix flows that are supposed to be bypassed by
the xdp filter are not bypassed.
Ticket: 8699
Victor Julien
Jason Ish
Shivani Bhardwaj
Philippe Antoine
Lukas Sismis
Philippe Antoine
Adam Kiripolsky