With the introduction of AppLayerResult::incomplete API, fragmented data
is no longer handled fully in the dcerpc code. Given that these code
paths are already covered by the following s-v tests, these tests can now be
safely removed.
- dce-gap-handling
- dcerpc-dce-iface-*
Ticket 5699
Instead of own internal mechanism of buffering in case of fragmented
data, use AppLayerResult::incomplete API to let the AppLayer Parser take
care of it. This makes the memory use more efficient.
Remove any unneeded variables and code with the introduction of this
API.
Ticket 5699
TCP data can be presented to the protocol parser in any way e.g. one
byte at a time, single complete PDU, fragmented PDU, multiple PDUs at
once. A limit of 1MB can be easily reached in some of such scenarios.
Remove the check that rejects data that is more than 1MB.
To optimize detection, and logging, to avoid going through
all the live transactions when only a few were modified.
Two boolean fields are added to the tx data: updated_tc and ts
The app-layer parsers are now responsible to set these when
needed, and the logging and detection uses them to skip
transactions that were not updated.
There may some more optimization remaining by when we set
both updated_tc and updated_ts in functions returning
a mutable transaction, by checking if all the callers
are called in one direction only (request or response)
Ticket: 7087
DCERPC/TCP tends to return the same values for invalid and incomplete
headers. As a result of this, invalid headers and any traffic following
it is buffered and processed later on assumed to be valid DCERPC traffic.
Fix this by clearly defining error and incomplete data and taking
appropriate actions.
Bug 7230
In the DCERPC over TCP pcap, logging and rule matching is disrupted by adding a simple rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
Works: alert + 3 dcerpc records.
But when adding a trivial rule:
alert tcp any any -> any any (flow:to_server,established; \
dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
alert tcp any any -> any any (dsize:3; sid:2; rev:1; )
The alert for sid:1 disappears and also there is one dcerpc event less.
In the single rule case we can aggressively free the transactions, as there
is only an sgh in the toserver direction.
This means that when we encounter the 2nd REQUEST, the first 2 transactions
have already been processed and freed. So for the 2nd REQUEST we open a new
TX and run inspection and logging on it.
When the 2nd rule is added, it adds toclient sgh as well. This means that we
will now slightly delay the freeing of the transactions.
As a consequence we still have the TX for the first REQUEST when the 2nd REQUEST
is parsed. This leads to the 2nd REQUEST re-using the TX. Since the TX is
already marked as inspected, it means the toserver rule now no longer matches.
Also we're not logging this TX correctly now.
This commit fixes the issue by not "finding" a TX that as already been
marked complete in the search direction.
Bug #7187.
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.
However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup
and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.
Bug 7044
Ticket: 7013
Done consistently for all protocols
This may change some protocols behaviors which failed early
if they found there was not enough data...
So far, if only the starting request was a DCERPC request, it would be
considered DCERPC traffic. Since ALTER_CONTEXT is a valid request type,
it should be accepted too.
Reported and patch proposed in the following Redmine ticket by
InterNALXz.
Bug 6191
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
The bug:
The dcerpc dce_iface keyword just match the packet following the bind. Only the
next request after the rpc is sent will match. However the expected behaviour it
that all the rpc requests/responses sent under the context of the given
interface would match.
In the Open Group c706 the following is indicated:
In 2.2.1 Binding-related Operations, indicates that one category of binding
operations are those that "operations that establish internal call routing
information for the server." (The other are to establish the protocol which is
not relevant here). And the following statement can be found:
Operations in the second category establish a set of mappings that the server
can use to route calls internally to the appropriate manager routine. This
routing is based on the interface and version, operation and any object
requested by the call.
It indicates that server routes (to call methods) are based on the operation,
interface and object.
- Operation: To indicate the method to call, and operation number is
specified as indicated in the second step of 2.3.3.2 (Client
Binding Steps).
- Interface: An interface is a set of remotely callable operations offered by a
server and invokable by clients. (2.1.1.1)
- Object: Is the manager that implements the interface, as stated in section
Interface and Manager Selection of 2.3.3.3. It is not mandatory, can
be nil.
To call a method, a client must send a request message as defined in 2.6.4.9,
that contains these identifiers:
- opnum: The opnum field identifies the operation being invoked within the
interface.
- p_cont_id (Context ID in Wireshark): The p_cont_id field holds a presentation
context identifier that identifies the
data representation and interface, as
defined in 12.6.3.4 (Context Identifiers).
- object: The object field is contained if the PFC_OBJECT_UUID is set. (Could be
interesting to create a keyword dce_object for matching this UUID)
Therefore, to get the correct method to invoke, the server must map the context
to the correct interface. This is negotiated by the bind request
Interfaces are first negotiated using the bind message (12.6.4.3), contained in
the p_context_elem array. Then they are accepted or rejected using the bind_ack
message (12.6.4.4).
Once these contexts are established, both client and server can use the context
id, which is the index of the p_context_elem array, to refer the interface they
are using.
Moreover, in the middle of the connection, the context can be changed with the
alter_context message.
This is way suricata shouldn't delete the bindack attribute, that contains
the contexts, used by match_backuuid. This is the only way to know the interface
a request message is referring to.
ticket: 4769
https://redmine.openinfosecfoundation.org/issues/4769
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.
This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
Calling a function in unwrap_or causes that function to always
be called even when not needed. Instead use unwrap_or_else with
a closure which will only be called when needed.
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.
This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.
Then fix all not_unsafe_ptr_arg_deref warnings from clippy.
Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
Shivani Bhardwaj
Jason Ish
Philippe Antoine
Victor Julien
Philippe Antoine
Victor Julien
Pierre Chifflier
Eloy Pérez González
Shivani Bhardwaj
Sam Muhammed