parse_t123_tpkt runs the X.224 and X.223/MCS sub-parsers over the
length-bounded tpkt payload, but they use streaming nom parsers.
An inner length field overrunning the payload, such as an oversized
CS_CORE block, escaped as Err::Incomplete although the tpkt was fully
present.
This patch wraps the child parsers in complete() so Incomplete from within the
bounded payload becomes an Error.
The tpkt is then returned as an unrecognized Raw child, which the caller skips,
and parsing resumes at the next tpkt.
Ticket #8838
Tx ID handling did not take the required + 1 into account.
From a report:
RDP can skip cleanup because its id convention does not match the
generic Rust iterator. The generic iterator in applayer.rs returns
tx.id() - 1, and cleanup trusts that id when calling StateTransactionFree
in app-layer-parser.c. RDP registers that iterator in rdp.rs, but
RdpTransaction::id() returns the stored id unchanged in rdp.rs, while
free_tx also compares against the raw stored id in rdp.rs. For a single
freeable RDP tx with stored id 1, the iterator returns C id 0; cleanup
calls free_tx(0), nothing is removed, then has_next == false allows
min_id to advance to total_txs in app-layer-parser.c. That leaves the
tx live but now below min_id, so later cleanup will not revisit it.
This patch brings the handling in line with the other parsers.
Bug: #8717.
Mark the direction into RDP transactions at creation time,
so the tx carries SKIP_INSPECT for the direction it is
never seen in, matching DHCP and the other single-direction
parsers. This lets cleanup free completed transactions and
stops a tx from being inspected (and alerting) twice, once per
direction.
RDP bounds its transactions to connection setup and stops
parsing once bypass_parsing is set.
Issue: 8621
and bindgen it to rust, and use default trait instead of new
Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerStateData
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream inspection which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
RDP parser creates a transaction per request or response. Appropriate calls
to trigger raw stream inspection have been added on completion of each request
and response.
Task 7026
Bug 7004
Have bindgen generate bindings for app-layer-protos.h, then use the
generated definitions of AppProto/AppProtoEnum instead if defining
them ourselves.
This header was chosen as its used by Rust, and its a simple header
with no circular dependencies.
Ticket: #7341
Both the macros export_tx_data_get and export_state_data_get can
generate non-pub functions as the function they generate is only used
as a pointer during registration.
Remove "pub" and "no_mangle" from the generated functions and update
the names of the generated functions to follow Rust rules as they are
no longer exported into the global C namespace.
Ticket: 7498
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.
However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup
and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.
Bug 7044
This flag is no longer needed as a parser can now create a transaction
as unidirectional.
Setting this flag also doesn't make sense on parsers that may have
request/reply and some unidirectional messaging.
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
Instead of a method that is required to return a slice of transactions,
use 2 methods, one to return the number of transactions in the
collection, and another to get a transaction by its index in the
collection.
This allows for the transaction collection to not be a contiguous array
and instead can be a VecDeque, or possibly another collection type that
supports retrieval by index.
Ticket #5278
Giuseppe Longo
Victor Julien
Jeff Lucovsky
Philippe Antoine
Jason Ish
Shivani Bhardwaj
Philippe Antoine
Daniel Olatunji
Alice Akaki
Victor Julien
Pierre Chifflier