Commit Graph

1500 Commits (51b742738cceb2863d2bc23ff50892e03f1635e1)

Author SHA1 Message Date
Philippe Antoine d3eb6569d2 rust: pin serde version
because oss-fuzz does not support newer rust nightly
and newer serde does not support oss-fuzz old nightly
11 months ago
Jason Ish 85ee2f882f lua: use lua from crates.io rather than github
This allows the source override in .cargo/config.toml to apply to the
suricata-lua-sys crate as well fixing offline builds.

Ticket: #7226
11 months ago
Shivani Bhardwaj fbb97c51e4 dcerpc: return error on invalid header
DCERPC/TCP tends to return the same values for invalid and incomplete
headers. As a result of this, invalid headers and any traffic following
it is buffered and processed later on assumed to be valid DCERPC traffic.
Fix this by clearly defining error and incomplete data and taking
appropriate actions.

Bug 7230
11 months ago
Philippe Antoine dc3c048b49 rust/detect: fix too_long_first_doc_paragraph clippy warning
warning: first doc comment paragraph is too long
  --> src/detect/iprep.rs:57:1
   |
57 | / /// value matching is done use `DetectUintData` logic.
58 | | /// isset matching is done using special `DetectUintData` value ">= 0"
59 | | /// isnotset matching bypasses `DetectUintData` and is handled directly
60 | | /// in the match function (in C).
   | |_
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_long_first_doc_paragraph
   = note: `#[warn(clippy::too_long_first_doc_paragraph)]` on by default
help: add an empty line
11 months ago
Philippe Antoine 2a984e3b13 rust/dcerpc: fix single_match clippy warning
warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let`
  --> src/dcerpc/log.rs:36:33
   |
36 |               DCERPC_TYPE_BIND => match &state.bind {
   |  _________________________________^
37 | |                 Some(bind) => {
38 | |                     jsb.open_array("interfaces")?;
39 | |                     for uuid in &bind.uuid_list {
...  |
51 | |                 None => {}
52 | |             },
   | |_____________^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_match
   = note: `#[warn(clippy::single_match)]` on by default
11 months ago
Philippe Antoine 0ebb84538e http2: add frames support
Ticket: 5743

Why ? To add detection capabilities
11 months ago
Victor Julien 65392c02f5 dcerpc: don't reuse completed tx
In the DCERPC over TCP pcap, logging and rule matching is disrupted by adding a simple rule:

        alert tcp any any -> any any (flow:to_server,established; \
                dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
                dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
                content:!"|00|"; within:100; distance:97; sid:1; rev:1; )

Works: alert + 3 dcerpc records.

But when adding a trivial rule:

        alert tcp any any -> any any (flow:to_server,established; \
                dce_iface:5d2b62aa-ee0a-4a95-91ae-b064fdb471fc; dce_opnum:1; \
                dce_stub_data; content:"|42 77 4E 6F 64 65 49 50 2E 65 78 65 20|"; \
                content:!"|00|"; within:100; distance:97; sid:1; rev:1; )
        alert tcp any any -> any any (dsize:3; sid:2; rev:1; )

The alert for sid:1 disappears and also there is one dcerpc event less.

In the single rule case we can aggressively free the transactions, as there
is only an sgh in the toserver direction.

This means that when we encounter the 2nd REQUEST, the first 2 transactions
have already been processed and freed. So for the 2nd REQUEST we open a new
TX and run inspection and logging on it.

When the 2nd rule is added, it adds toclient sgh as well. This means that we
will now slightly delay the freeing of the transactions.

As a consequence we still have the TX for the first REQUEST when the 2nd REQUEST
is parsed. This leads to the 2nd REQUEST re-using the TX. Since the TX is
already marked as inspected, it means the toserver rule now no longer matches.
Also we're not logging this TX correctly now.

This commit fixes the issue by not "finding" a TX that as already been
marked complete in the search direction.

Bug #7187.
11 months ago
Shivani Bhardwaj e93743a094 rust/base64: upgrade crate to latest
base64 crate is updated to the latest version 0.22.1. This came with
several API changes which are applied to the code. The old calls have
been replaced with the newer calls.

This was done following the availability of better fns to directly
decode into slices/vectors as needed and also that previous version was
too old.
Along with this change, update the Cargo.lock.in to reflect all changes
in the package versions.

Task 7219
11 months ago
Shivani Bhardwaj 599a451e44 rust: update Cargo.lock.in
to include the latest packages available.
11 months ago
Jason Ish 080681aff5 pgsql: don't expose PgsqlTransactionState to C
PgsqlTransactionState has a variant named "Init" which is a little too
generic to export to C. Fortunately this method doesn't need to be
exposed to C, instead remove it as it was only called by
rs_pgsql_tx_get_alstate_progress which also doesn't need to be public
or expose to C.

Ticket: #7227
12 months ago
Philippe Antoine 304271e63a rust: compatibility with cbindgen 0.27
Ticket: 7206

Cbindgen 0.27 now handles extern blocks as extern "C" blocks.
The way to differentiate them is to use a special comment
before the block.
1 year ago
Giuseppe Longo 564a6c9a20 rust/ldap: handle GAPs
Following the same logic as for PGSQL, if there is a gap in an LDAP request or
response, the parser tries to sync up again by checking if the message can be
parsed and effectively parses it on the next call.

Ticket #7176
1 year ago
Giuseppe Longo 6a606ff21e rust/ldap: add pdu frames
This adds a pdu frame for both request and response, and removes invalid
returns in SCLdapParseRequest and SCLdapParseResponse.

Ticket #7202
1 year ago
Giuseppe Longo edf70276d6 rust/ldap: enable parser for udp
This introduces a new parser registration function for LDAP/UDP, and update
ldap configuration in order to be able to enable/disable a single parser
independently (such as dns).
Also, GAPs are accepted only for TCP parser and not for UDP.

Ticket #7203
1 year ago
Philippe Antoine ede77bc4db rfb: move app-layer registration code to rust
Ticket: 7178
1 year ago
Philippe Antoine 62a186ceef detect/rfb: move keywords to rust
Ticket: 7178

On the way, convert rfb.secresult to a generic integer with enumeration
cf ticket 6723
1 year ago
Philippe Antoine a673e1913b ssh/frames: avoid unsigned integer overflow
Fixes: 0b2ed97f36 ("ssh: frames support")
1 year ago
Philippe Antoine 42e5e556e5 rust/ike: fix collapsible_match clippy warning
warning: this `match` can be collapsed into the outer `match`
help: the outer pattern can be modified to include the inner pattern
1 year ago
Philippe Antoine 564f685eea rust: fix byte_char_slices clippy warnings
warning: can be more succinctly written as a byte str
   --> src/mime/smtp.rs:762:37
    |
762 |     mime_smtp_find_url_strings(ctx, &[b'\n']);
    |                                     ^^^^^^^^ help: try: `b"\n"`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#byte_char_slices
    = note: `#[warn(clippy::byte_char_slices)]` on by default
1 year ago
Philippe Antoine 089d2b11fd frames: remove unneeded comments
Used by documentation with the SIP frames only
1 year ago
Philippe Antoine ef42f835eb ssh: avoid panic in packet path
use debug_validate_bug_on instead
1 year ago
Philippe Antoine 0b2ed97f36 ssh: frames support
Ticket: 5734

Adds frames for SSH records, that come after banner, and before
the data is encrypted.
These records may contain cipher lists for instance.
1 year ago
Victor Julien da1645b3e1 rust: bump time to most recent
Fixes build on rustc 1.80.

Bumps the MSRV to 1.67.1.

Bug: #7130.
1 year ago
Victor Julien 058ad87089 rust: set MSRV to 1.67.1
This is needed for updating the ``time`` crate.
1 year ago
Victor Julien a0bf282963 rust: address clippy errors 1 year ago
Victor Julien 5bda7b5017 ssh/hassh: fix clippy warning 1 year ago
Philippe Antoine 7617fe5ab0 ldap: reset tx_index_completed on tx removal
So, that this index does not overflow
1 year ago
Philippe Antoine 7f6c963ac4 doh2: log like dns v3 1 year ago
Philippe Antoine 8aa2964e73 doh: move fields into dedicated Optional struct
So as to consume less memory for HTTP2Transaction
1 year ago
Philippe Antoine 6e12475f48 doh2: handle dns message in POST requests
Ticket: 5773

Handles both directions the same way for data if content type is
application/dns-message
1 year ago
Philippe Antoine 0ccad8fd88 doh: make dns and http keywords for doh2
Ticket: 5773
1 year ago
Philippe Antoine 1e82e20c65 doh: implement dns over http2 app-proto
Ticket: 5773
1 year ago
Philippe Antoine 46d98ae81c http2: log dns if DoH is recognized
Ticket: 5773
1 year ago
Philippe Antoine 29d9dc2729 http2: rustfmt 1 year ago
Philippe Antoine b5f55b5b1f dns: prepare for dns over http2 support
by making tx parsing and creation more easily available,
without needing a dns state.

Dns event NotResponse is now set on the right tx, and not the one
before.

Also debug log for Z-flag on request says "request" instead of
"response"

Also rustfmt dns.rs
1 year ago
Victor Julien 6598a6953e rust/ldap: add ldap-parser to Cargo.lock.in 1 year ago
Giuseppe Longo 910a5b226c rust/ldap: implement logger 1 year ago
Giuseppe Longo 93da339975 rust/ldap: implement app-layer 1 year ago
Giuseppe Longo ce7e190501 rust/ldap: implement types and filters
This implementation adds types and filters specified in the LDAP RFC to
work with the ldap_parser.
Although using the parser directly would be
best, strange behavior has been observed during transaction logging.
It appears that C pointers are being overwritten, leading to incorrect
output when LDAP fields are logged.
1 year ago
Philippe Antoine cc3dde8ada smtp: adds server side detection
Ticket: #1125
1 year ago
Philippe Antoine 0a1062fad2 detect/mqtt: move keywords to rust
Ticket: 4863

On the way, convert some keywords to use the first-class integer
support.
And helpers for pure rust the support for multi-buffer.

Move the C unit tests about keyword mqtt.protocol_version
to unit tests for generic integer parsing, and test version 5
instead of testing twice version 3.

Also iterate all tx's messages for reason code as is done for other
keywords.

And allow detection on empty topics.
1 year ago
Philippe Antoine ad08309c75 mqtt: parse and store raw connect flags
for easier later matching
1 year ago
Philippe Antoine 9adf4224e4 rust/derive: string enumeration become case insensitive
As needed for MQTTTypeCode which accepts both CONNECT uppercase
and unassigned lowercase
1 year ago
Philippe Antoine 3c5ad7a23d rust/derive: transform all uppercase names the right way
So that MQTTTypeCode::CONNECT does not become c_o_n_n_e_c_t
1 year ago
Jason Ish fcc1b1067b eve/dns: make version required
The "eve.version" field is not always logged. Update the schema to
enforce that it is, and fix it for records that don't log it.

Ticket: #7167
1 year ago
Shivani Bhardwaj f2de3e01cb src: remove truncate fn and glue code
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.

However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup

and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.

Bug 7044
1 year ago
Shivani Bhardwaj 80159eb519 applayer: remove truncation logic
as its functionality is already covered by the generic code.
This removes APP_LAYER_PARSER_TRUNC_TC and APP_LAYER_PARSER_TRUNC_TS
flags as well as FlowGetDisruptionFlags sets STREAM_DEPTH flag in case
the respective stream depth was reached. This flag tells that whether
all the open files should be truncated or not.

Bug 7044
1 year ago
Jason Ish 575e5b471f dns: add v3 dns logging
DNS v3 logging fixes the discrepancies between request and response
logging with the main difference being queries always being placed in an
array.

Bug: #6281
1 year ago
Jason Ish df656324ba dns: new v3 style logging for alerts
V3 style DNS logging fixes the discrepancies between request and
response logging better dns records and alert records.

The main change is that queries and answers are always logged as
arrays, and header fields are not logged in array items.

For alerts this means that answers are now logged as arrays, queries
already were.

DNS records will get this new format as well, but with a configuration
parameter.

Bug: #6281
1 year ago
Nathan Scrivens 9ecc3573a7 dns: parse and populate OPT rdata struct
Feature: 7017
Add DNSRDataOPT struct and DNSRData enum type OPT.
Add OPT parsing function and test function.
Add DNSRData OPT type to lua.rs match.
Log OPT rdata.
1 year ago