ENIP parser makes use of nom's complete macros which by design expect
full data to be passed along to them. In case there is incomplete data
sent to them, they result in an error. This disables the ENIP parser and
further detection and could happen with an incomplete/fragmented
request/response.
Use AppLayerResult API to ask for more data from the stream engine in
case the stream slice currently at hand does not match the length
parameters received from the header or is not long enough to parse the
header itself.
Ticket: 8809
Ticket: 8395
For protocols where tx are complete at start, that is
the `tx_get_progress` function returns 1 inconditionnally,
we can use the progress 1, not losing time on detection in ips
mode, and getting ready for firewall mode
For the other protocols (where a tx needs both a request and response
for instance), keep progress 0 as was the case before
and bindgen it to rust
Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerTxData
Move also the free function to C SCAppLayerTxDataCleanup
As suricata-sys crate defines AppLayerTxData for rust,
It must itself implement the Drop trait, and thus,
We need to define a feature surest
The functions around TriggerRawStreamReassembly are misnomers in the
current layout of the code. The functions were named appropriately when
they were created as per the structural and logical layout of the code
at the time.
These functions in today's code are being used to track, update and
trigger progress of inspection in the raw stream only. Hence, rename them
to TriggerRawStreamInspection.
Introduce DetectGetSingleData which does the generic wrapping,
including the transforms, using a new callback prototype
DetectTxGetBufferPtr
The goal is to replace most InspectionBufferGetDataPtr.
For this commit, we do not change every callback to keep the
change relatively small.
Focus here is to remove DetectHelperGetData as its functionality is
provided more directly by the new DetectTxGetBufferPtr.
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
ENIP has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.
Optimization 7026
Bug 7004
Ticket: 7667
And prefix SCDetectBufferSetActiveList to be exported
Allows less use of suricata crate in plugin as we get the functions
prototypes from suricata_sys and they are more correct.
Deranged v0.4.1 (a dependency of the time crate) has implemented
PartialOrd for some integer types that conflict with the
implementation in the standard library creating an ambiguity as such
implementation are global. For more info see
https://github.com/jhpratt/deranged/issues/18.
To fix, use "::from" directly, instead of using .into() which is where
we run into amgibuity.
Have bindgen generate bindings for app-layer-protos.h, then use the
generated definitions of AppProto/AppProtoEnum instead if defining
them ourselves.
This header was chosen as its used by Rust, and its a simple header
with no circular dependencies.
Ticket: #7341
Both the macros export_tx_data_get and export_state_data_get can
generate non-pub functions as the function they generate is only used
as a pointer during registration.
Remove "pub" and "no_mangle" from the generated functions and update
the names of the generated functions to follow Rust rules as they are
no longer exported into the global C namespace.
Ticket: 7498
Allow `set_uint` to accept any number value that can be converted to a
u64. Prevents callers from having to do `as u64`.
This required fixing up any callers that used `.into()` to just pass in
their value without the into conversion.
Most calls using `as u64` can have that cast removed, with the exception
of `usize` values which must still be cast is conversion can't be
guaranteed to be non-fallible.
To optimize detection, and logging, to avoid going through
all the live transactions when only a few were modified.
Two boolean fields are added to the tx data: updated_tc and ts
The app-layer parsers are now responsible to set these when
needed, and the logging and detection uses them to skip
transactions that were not updated.
There may some more optimization remaining by when we set
both updated_tc and updated_ts in functions returning
a mutable transaction, by checking if all the callers
are called in one direction only (request or response)
Ticket: 7087
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.
However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup
and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.
Bug 7044
Shivani Bhardwaj
Philippe Antoine
Jason Ish
Victor Julien
Alice Akaki