The error arm of a COMPOUND that is fully in hand only set a state event.
set_event marks the newest transaction, and there is none when the
record's first op is the one that fails: the record was then consumed,
nothing was logged, and nothing asked for the missing bytes. A v4 WRITE
whose count runs past the end of its record is that case - the parser
gives up at the data, so the size check in the WRITE handling, which
needs a parsed count, is never reached.
Reject what cannot be walked the way a partial record is rejected: scan
the bytes the parser was given, and log an oversized WRITE as a
write_request_too_large file transaction or another un-walkable record as
a malformed one, on a transaction of its own. What the parser skips is
not in that set: a defined op this parser does not implement keeps base's
event on the newest tx, and an RPCSEC_GSS privacy compound, whose result
is opaque by RFC 2203 on the call and the reply alike, is left unparsed
with no event at all. Either way the record is bounded by its own marker,
and a flow that carries un-parseable v4 traffic leaves a trace in EVE.
Ticket: #9265.
The claim is read as a word followed by one nfsstring whatever its type,
so only CLAIM_NULL and CLAIM_DELEGATE_PREV land on the next op. CLAIM_FH
and CLAIM_DELEG_PREV_FH carry nothing: their OPEN takes the following
op's tag as a name length. CLAIM_PREVIOUS carries an
open_delegation_type4, CLAIM_DELEG_CUR_FH a stateid and CLAIM_DELEGATE_CUR
a stateid then a name: all of them leave the parse in the middle of the
record, so every later op of the compound misparses and the filename is
whatever the read landed on. The open owner's XDR pad and the fattr4 that
follows an EXCLUSIVE4_1 verifier go missing the same way, and an
EXCLUSIVE4_1 OPEN loses its name to the attributes it never consumed.
Read the union by its type: a name for 0, 2 and 3, a word for 1, a
stateid for 2 and 5, nothing for 4 and 6, and an error for a type no
version defines. The filename stays empty where the claim carries none,
as the file is named by the current filehandle. The compound scanner can
then follow the parser through all seven claims instead of rejecting the
record, so a WRITE behind a filehandle-based OPEN is still reached, and an
unknown claim stays a bounded reject.
Ticket: #9265.
Main runs the backstop only when the stream has a gap. Out-of-order file
data on a stream that never gaps therefore grows the queue past the
bound the backstop exists to hold, while a call that carried no data at
all still walked the transactions. Main's chunk-count bound and its scan
of every open file transaction stay as they are.
Gate the scan on the call carrying data and the state holding at least
one file transaction, tracked by a counter maintained with the
transactions list, and resolve the file handle after the chunk_left == 0
return so a call with nothing left does not consume one.
Ticket: #9263.
A queue rejection arms a skip, but every SMB1 and SMB2 read and write
handler still called set_file_left at its tail: a partial record left
rec_size - data_size bytes armed under a guid whose chunk the rejection
never armed. Nothing is misattributed today - with no chunk armed the
next update consumes nothing and drops the promise without skipping a
byte - but the armed value describes data that was never handed over,
and a future path that parses while a chunk is in flight would spend it
on the following record. Guard the trailing set_file_left with the
rejected flag so the promise only ever describes queued data, and keep
the SMB1 WRITE_AND_CLOSE close handling outside the guard so the close
is still processed when its data is rejected. Unit tests pin both.
Read the SMB queue and size settings through u32::try_from so an
over-u32 value logs and keeps its default instead of silently truncating,
and refuse a zero smb.max-tx, which otherwise set a live-transaction
limit that evicted every transaction on insert.
Ticket: #9263.
The claim the scanner skips as a word plus a name was NULL(0) and UNIQUE,
a value no version defines under that name. It is CLAIM_DELEGATE_PREV(3)
whose claim is a component4 name, the same one the parser reads after the
claim word, while 2 is CLAIM_DELEGATE_CUR: its stateid is what the name
length is then read from. So the scanner walked off the end of a compound
that parses fine, and rejected a claim whose record it could have
followed to the op behind it.
Accept 0 and 3, where the claim word is followed by the name the parser
consumes, and reject the others as a malformed compound within the
record's bounds, never as incomplete and never buffering toward the
claim. The test that pins the rejected set loses 3 and gains 2, and a
DELEGATE_PREV claim ahead of another op is now reached.
Ticket: #9263.
An OPEN reply that grants a delegation desynced the parser and the record
scanner that has to mirror it. The nfsace4's who was consumed as a length
and that many bytes, without the XDR pad that follows them, so what came
after the delegation was read from inside the name. The write arm counted
nfs_space_limit4 as two words, four bytes short of the word and uint64 it
is, and lost the same four one field further along. A SEQUENCE result was
skipped as 36 unlabeled bytes.
Consume a who in one helper that takes the length, the bytes and the pad,
used by both parser arms and by the scanner's skip, and read
nfs_space_limit4 as its 12 bytes. The write delegation struct loses the
who field no caller read. SEQUENCE4resok is read as the fields it is made of,
which is what the scanner's 36-byte skip stands for. The fixtures are
laid out per field and parsed to the last byte, and the change_info4 they
carry is no longer all zeros, so a skip that comes up short desyncs
instead of matching. The scanner's op helpers name the element type they
extend with.
Ticket: #9265.
The record scanners unwrapped the (0, 2) combination and took every other
one as a bare compound. A control procedure or a privacy compound was
then scanned as if it were a COMPOUND: the envelope length was read as
the first tag, so the scan either stayed incomplete and buffered toward
the claim, or rejected a record that has nothing to reject.
Scan by the combination the call recorded: hand the credential pair to
gss_compound_scan_data, which leaves (0, 1) raw because rpc_gss_svc_none
wraps nothing, unwraps (0, 2), and reports a privacy (0, 3) compound as
opaque. There is nothing to scan in an encrypted compound, so the record
is skipped to its boundary and neither parsed nor scored: the request
path keeps the call in the request map for its reply, the reply path
drops it. Any other combination is one the full record path rejects as
well, so it is malformed, boundedly and without buffering. The v4
request path records the combination whatever it is, not only when it
unwraps.
Ticket: #9263.
The args of a WRITE request and of a READ reply were trusted to fit the
record that carries them. Where the claim fell below the extent the parse
covers, the file data ran past the record boundary into the next RPC,
which was then read as a record. A max-tx of 0 was accepted and evicted
every transaction on insert.
Compare the claim against that extent: for a WRITE the handle, offset,
count, stable flag and its data, for a READ the status, the optional
attributes, the count, the eof flag and its data. A record whose claim is
below the extent gets a transaction with the malformed data event and a
skip to the record boundary, as an oversized claim already does. Only the
v3 records arrive in pieces, so process_read_record now learns which
reply it is given and the check is v3's. Reject a max-tx of 0 at config
load.
Ticket: #9263.
A partial v3 reply that carries no file data is skipped rather than
buffered toward its claim, which for LOOKUP, READDIRPLUS and CREATE drops
the handle to name mapping their replies establish. The exception to that
reused max-read-size, so those procedures depended on a knob that
documents READ data, and lowering it for READs silently dropped the
mapping of any directory reply above it; the mapping depends on
segmentation, as the same reply parses when it arrives in one piece.
Give them a limit of their own, max-metadata-size, with a 16 MiB default,
and use it to decide whether such a reply may still buffer toward its
record claim. A value of 0 falls back to the default, so the buffering
stays bounded and the limit cannot be disabled. The WRITE and READ limits
share that default through NFS_MAX_RW_SIZE_DEFAULT. Documented in the
userguide.
Ticket: #9263.
The queue-limit work computed the record tail and the chunk promise from
the declared size, and added a count or a data length to it. A padded or
segmented record therefore left a promise armed past the end of its
record: a continuation could run on into the next RPC, an input that
carries only record slack grew the out-of-order queue and truncated a
file whose queued data fits, and the backstop walked the transactions of
a call that holds no file transaction at all.
Take the promise from the record itself, with record_tail(): what its
marker claims minus what the parser holds. Keep its two halves apart:
the stream side is the record tail, the queue side is the file data the
tracker still misses, so a continuation queues no more than its own
bytes. A pad a record carries is inside its claim, so the tail drains
it; it is neither file data nor fill. Release a promise whose
transaction is gone instead of consuming toward the claim, run the
backstop only when the call carries data and the state holds a file
transaction, and count a chunk only for input that carries file data. A
consume past the end of an input is now a debug assertion, with the
advance clamped to the input.
Ticket: #9263.
A WRITE request or READ reply with a count of 0 has no file data, and
processing it returned early. For a partially received record this left the
rest of the record unaccounted for, while the input was reported as consumed.
The rest of the record was then parsed as the next RPC record, which failed
and disabled the parser for the flow.
Like the complete record path, skip the rest of the record and continue with
the next record. For a READ reply this includes removing its request from the
request map, as is done for a complete reply.
Ticket: #9188
When a partially received record failed to parse, for example a WRITE with an
invalid stable value or a READ reply with an invalid eof, the parser returned
an error, disabling the parser for the flow. Yet the RPC record marker had
already given the size of the record.
Like the complete record path already does, set the malformed data event,
skip the rest of the record and continue with the next record. For a READ
reply this includes removing its request from the request map. A record
marker that cannot be parsed is still a fatal error, as the boundary of the
next record is unknown.
Ticket: #9190
The continuation of a partial WRITE request or READ reply only covered the
file data, not the XDR padding that follows it in the record. When a delivery
of stream data ended on the last byte of file data, or inside the padding,
the remaining padding was parsed as the start of a new RPC record, which
failed and disabled the parser for the flow.
Keep the chunk active while its record still has unconsumed bytes, as tracked
by the record tail, and consume exactly those bytes: from the input that
completes the file data and, for what is not there yet, from the next
deliveries. If the file is still open the padding is passed to its file
tracker, which consumes it as fill bytes, never as file data.
A partial READ reply that already holds all of its file data now also has its
request removed from the request map, as is done when a chunk completes.
Builds on the file handle of the in-progress READ chunk added for #9192.
Ticket: #9194
The continuation of a partially received READ reply looked up its file handle
in the request map by the reply's xid. As the request map is bounded and
evicts the least recently used entries, enough other requests on the flow
could evict the entry while the reply was still in progress. The continuation
then consumed nothing and its file data was parsed as a new RPC record, which
failed and disabled the parser for the flow.
Keep the file handle of the in-progress to-client chunk in the state, like
is done for the to-server WRITE chunk, and clear it along with the rest of
the chunk state.
The file transaction itself can also go away while its chunk is in progress,
for example when the max-tx limit forces it closed, after which a READ request
for the same handle creates a new transaction. A transaction found for the
handle whose file is not open, which is also the case after a complete READ
reply with eof set, cannot take the chunk either. In these cases consume the
chunk up to its known record boundary instead of parsing its data as RPC
records or passing it to that transaction.
Ticket: #9192
A NFSv3 WRITE with stable set to FILE_SYNC is assumed to be the last chunk of
its file, and its file transaction was closed as soon as the record was seen.
If the record was only partially received, the continuation could no longer
find the transaction by its file handle. Its file data was then not consumed
but parsed as a new RPC record, which failed and disabled the parser for the
flow, and the file was truncated.
Like the READ reply path already does, only close the transaction right away
if the record holds all of its file data. Otherwise filetracker_update closes
it once the file tracker is done or, as a truncated tracker is never done,
once the data of the last chunk is complete. The latter also applies to a
partial READ reply with eof set.
The NFSv4 WRITE path makes the same FILE_SYNC assumption, but it only handles
complete records, so it is not affected.
This is also the root cause of Redmine #8009.
Ticket: #9196
Rust unit tests link without libsuricata, so a C reference anywhere in
the reachable code makes the test binary unlinkable. The ffi logger
calls SCLogMessage even under cfg(test), unlike the suricata crate's
own, which has a print fallback. Any test that reaches
AppLayerTxData::update_file_flags therefore breaks cargo test --lib
whenever the debug feature is on: it fails the GitHub jobs that build
with --enable-debug or --all-features while our own CI never sees it.
Give log_message the same fallback behind cfg(any(test, feature =
"testing")), the pattern applayer.rs already uses for set_event.
DCERPC REQUEST/RESPONSE records can have their stub data fragmented
across multiple PDUs. The flags that convey the state of the current PDU
are as follows:
PFC_FIRST_FRAG: first fragment of (fragmented) data
PFC_LAST_FRAG: last fragment of (fragmented) data
PFC_FIRST_FRAG && PFC_LAST_FRAG: no fragmentation
Consider a REQUEST/RESPONSE complete only when the PFC_LAST_FRAG flag is
found to be set.
Ticket: 9240
SCFTPFreeResponseLine gated the release of the code buffer on a null
test of the response pointer, so the code buffer had no check of its
own. Neither check could fire: parse_response_line sets both pointers
from Box::into_raw on a boxed slice, which never returns null, and C
code obtains an FTPResponseLine only from SCFTPParseResponseLine. The
comment on the code field claimed the pointer may be null; a line with
no status code gets an empty buffer with code_length 0 instead.
Remove both null checks, note why they are unnecessary, and correct the
field comment.
Ticket: 8803
parse_response_line allocates the code buffer with Box::into_raw on a
boxed slice, but SCFTPFreeResponseLine released it by rebuilding a
Vec<u8> with capacity code_length. Vec::from_raw_parts accepts any
global-allocator pointer whose size and alignment match, and both paths
compute the same layout for a byte slice, so the old free was sound. It
differed only from how the response buffer is released.
Release the code buffer through Box::from_raw on a slice pointer, so
both buffers are freed the same way they were allocated.
Ticket: 8803
In case a keyword should work in firewall mode, with firewall rules
only. The engine errors out if threat detection rules use the given keyword.
Part of
Ticket #9041
Victor Julien
Jason Ish
Shivani Bhardwaj
Philippe Antoine
Jeff Lucovsky
Juliana Fajardini