Commit Graph

1574 Commits (c587e90ebccfe8883ea951e723be461dc58dcc06)

Author SHA1 Message Date
Philippe Antoine ae72376ebe detect/snmp: move keywords to rust
Ticket: 4863

On the way, convert unit test DetectSNMPCommunityTest to a SV test.

And also, make snmp.pdu_type use a generic uint32 for detection,
allowing operators, instead of just equality.
1 year ago
Philippe Antoine 4bbe7d92dc detect: helper to have pure rust keywords
detect: make number of keywords dynamic

Ticket: 4683
1 year ago
Philippe Antoine 08c511f1bf enip: remove unnecessary unsafe
As the function SCEnipRegisterParsers is already marked as unsafe
1 year ago
Victor Julien 37be66eef9 detect/iprep: update function naming
Bring in line with new Rust code naming for FFI functions.
1 year ago
Victor Julien 83976a4cd4 detect/iprep: implement isset and isnotset
Implement special "isset" and "isnotset" modes.

"isset" matches if an IP address is part of an iprep category with any
value.

It is internally implemented as ">=,0", which should always be true if
there is a value to evaluate, as valid reputation values are 0-127.

"isnotset" matches if an IP address is not part of an iprep category.

Internally it is implemented outside the uint support.

Ticket: #6857.
1 year ago
Victor Julien 539ab3a404 detect/iprep: update keyword parser for extendibility 1 year ago
Jason Ish f0dbfe863d misc: prefix functions with SC not Sc 1 year ago
Philippe Antoine d9d5170ec0 websocket: add data frame
Ticket: 7051
1 year ago
Juliana Fajardini bb45ac71ef dns: allow triggering raw stream reassembly
For TCP streams, app proto stream reassembly can start earlier, instead
of waiting and queueing up data before doing so.

Task #7018
Related to
Bug #7004
1 year ago
Philippe Antoine 82c03f72c3 enip: convert to rust
Ticket: 3958

- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
    enip.product_name, enip.protocol_version, enip.revision,
    enip.identity_status, enip.state, enip.serial, enip.product_code,
    enip.device_type, enip.vendor_id, enip.capabilities,
    enip.cip_attribute, enip.cip_class, enip.cip_instance,
    enip.cip_status, enip.cip_extendedstatus
1 year ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
1 year ago
Philippe Antoine 5167ff6411 smtp/mime: look for urls in base64 message
Ticket: 5185

Previously, it was looked for message in plain text, and base64
encoding was only handled for attachments.

This commit also fixes the buffering got such base64 data streamed
into urls finding, by buffering a beginning non-empty line,
and by ensuring that we run extraction on the last line,
even if it had no EOL.
1 year ago
Juliana Fajardini 0946c213cd pgsql: trigger raw stream reassembly
Expose the raw stream earlier to the detection engine, as Pgsql can have
multiple messages per transaction and usually will have a message
complete within one TCP packet.

Bug #7000

Related to
Bug #7026
1 year ago
Juliana Fajardini 69e26de197 pgsql/logger: open json object from logger function
Before, the JsonBuilder object for the pgsql event was being created
from the C-side function that actually called the Rust logger.

This resulted that if another module - such as the Json Alert called the
PGSQL logger, we wouldn't have the `pgsql` key present in the log output
- only its inner fields.

Bug #6983
1 year ago
Philippe Antoine 7fb10676e7 dns: remove unneeded mut in logger 1 year ago
Philippe Antoine a10c1f1dde smtp: use rust for mime parsing
Ticket: #3487
1 year ago
Philippe Antoine 5f75b9a6e3 http: use rust for mime parsing
Ticket: #3487
1 year ago
Philippe Antoine 5555aa6788 mime: improved token parsing
Accepts escaped quote in escaped string
1 year ago
Jason Ish 936930778c rust/Makefile: cleanup "clean" targets
Remove maintainer-clean-local, this is not needed.

In distclean-local, remove "rust/dist" and "rust/vendor" as they are
created during "make dist".

In "clean-local", remove "rust/target" and "rust/gen" as they are
created during a normal "make".
1 year ago
Jason Ish 2e440169d6 lua: remove lua as a compile time feature
Its always built-in. However, can be disabled at runtime.
1 year ago
Jason Ish 1fd2c1a379 rust/lua: remove lua_int8 feature
Now that we're fixed to Lua 5.4, the integer size is always 8.
1 year ago
Jason Ish bc011f2205 lua: use rust crate to vendor (bundle) lua
Remove lua-dev(el) from all CI tests.
1 year ago
Shivani Bhardwaj 3a1c12414a tls: store list of subject alternative names
So far, the SANs were available as a part of IssuerDN via x509_parser
crate but SANs were not available to the SSLState* to be directly used
to setup and match against a sticky buffer.
Expose it to SSLStateConnp.

Feature 5234
1 year ago
Jason Ish 8560564657 rust: rename .cargo/config to .cargo/config.toml
Addresses this warning from the Rust compiler:

warning: `../rust/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
1 year ago
Philippe Antoine 3b5f1173ab Revert "rust: temporary: disable debug assertions"
This reverts commit 14ab9aa763.
1 year ago
Philippe Antoine 37a9003736 rust/probing: safety check for null input
Ticket: 7013

Done consistently for all protocols

This may change some protocols behaviors which failed early
if they found there was not enough data...
1 year ago
Philippe Antoine 5dc8dea869 rust: return empty slice without using from_raw_parts
As this triggers rustc 1.78
unsafe precondition(s) violated: slice::from_raw_parts requires
the pointer to be aligned and non-null,
and the total size of the slice not to exceed `isize::MAX`

Ticket: 7013
1 year ago
Philippe Antoine 806052d762 websocket: fix opcodes values for ping/pong
And also set close

Ticket: 7025
1 year ago
Philippe Antoine 8b103ae755 dns: set tx id for frames 1 year ago
Philippe Antoine 715bf048ee frames: rust API makes tx_id explicit
And set it right for SIP and websocket,
so that relevant tx app-layer metadata gets logged.

Ticket: 6973
1 year ago
Jeff Lucovsky cb56752bf7 config/ja3: Eliminate warnings when JA3 is disabled
This commit eliminates warnings when either ja3, ja4 or both are
disabled.
1 year ago
Jason Ish 14ab9aa763 rust: temporary: disable debug assertions 1 year ago
Philippe Antoine c53e9ac0dd sdp: fix logging medias
As introduced by bff790b6ac

Also handles errors in the caller

Ticket: 6994
1 year ago
Jason Ish df8568ee30 rust/dns: visibility cleanups
Remove pub from functions that don't require it.
1 year ago
Jason Ish 556cfe56bf rust/dns: ffi naming and visibility cleanups
- Remove no_mangle and pub from FFI functions that are only accessed
  with a function pointer.
- Rename all no_mangle FFI functions to our C naming scheme.
1 year ago
Giuseppe Longo 868493529b rust/sip: parse and log sdp
If SDP payload is found within a SIP message, it will be parsed and then
logged.

Ticket #6627
1 year ago
Giuseppe Longo bff790b6ac rust/sdp: implement logger
This implements a logger for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.

Ticket #6627
1 year ago
Giuseppe Longo 1ccfc35214 rust/sdp: implement protocol parser
This implements a parser for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.

Ticket #6627.
1 year ago
Philippe Antoine d8e0c72644 app-layer: remove unused C wrappers
Ticket: 4083
1 year ago
Philippe Antoine 03442c9071 http2: do not log duplicate headers
Ticket: 6900

And thus avoid DOS by logging a request using a compressed
header block repeated many times and having a long value...
1 year ago
Philippe Antoine 390f09692e http2: use a reference counter for headers
Ticket: 6892

As HTTP hpack header compression allows one single byte to
express a previously seen arbitrary-size header block (name+value)
we should avoid to copy the vectors data, but just point
to the same data, while reamining memory safe, even in the case
of later headers eviction from the dybnamic table.

Rust std solution is Rc, and the use of clone, so long as the
data is accessed by only one thread.
1 year ago
Philippe Antoine 0291d37009 websocket: configurable logging of payload in alerts 1 year ago
Philippe Antoine 44b6aa5e4b app-layer: websockets protocol support
Ticket: 2695
1 year ago
Philippe Antoine f83ec543e3 http2: add settings from newer RFCs
Including the one for websocket over HTTP/2
1 year ago
Juliana Fajardini ce1556cefd pgsql: check for eol when parsing response
It was brought to my attention by GLongo that Pgsql parser handled eof
diffrently for requests and responses, and apparently there isn't a good
reason for such a difference therefore, apply same logic used for
rs_pgsql_parse_request for checking for eof when parsing a response.
1 year ago
Sascha Steinbiss 120313f4da ja4: implement for TLS and QUIC
Ticket: OISF#6379
1 year ago
Sascha Steinbiss 9d0db71ebf ja3: make feature compile time configurable 1 year ago
Philippe Antoine 49caf005a4 detect/analyzer: create tojson function for generic integers
As will be needed for tcp.mss
1 year ago
Philippe Antoine 3643b6ed4b output: generic simple tx json logger
Ticket: 3827
1 year ago
Jason Ish 1657b6ff3c cargo: add description and license
Required for publishing to crates.io.
1 year ago
Jason Ish b9127e8b96 automake/rust: remove path.lib
Remove the path.lib parameter that is substituted into the output
Cargo.toml by autoconf. Instead, as part of the build, "cd" into the
source directory. We already set the Rust target directory to the
external build directory.

This makes the Cargo.toml more generic, and in a format suitable for
publishing to crates.io. It also makes it easier to pull in external
crates without needing to patch up their Cargo.toml, for example, it
might make pulling libhtp-rs easier.
1 year ago
Jason Ish 71f59e529c jsonbuilder: fix serialization of nan and inf
When outputting a float, check if its infinity, or not a number and
output a null instead.

Using a null was chosen as this is what serde_yaml, Firefox, Chrome,
Node, etc. do.

Ticket: #6921
1 year ago
Philippe Antoine ee50fe4c30 sip: convert transaction list to vecdeque
Ticket: 6891

So as to avoid quadratic complexity on tx cleanup with SIP/TCP
that can create many transactions in one go.
1 year ago
Philippe Antoine f7cde8f00e rust/smb: fix clippy nightly warning
error: unnecessary use of `to_vec`
    --> src/smb/smb.rs:1048:62
     |
1048 |         let (name, is_dcerpc) = match self.guid2name_map.get(&guid.to_vec()) {
     |                                                              ^^^^^^^^^^^^^^ help: replace it with: `guid`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_to_owned
     = note: `#[deny(clippy::unnecessary_to_owned)]` implied by `#[deny(warnings)]`

And also other uses of to_vec() on already Vec
1 year ago
Philippe Antoine 02f2fb8833 rust: fix clippy 1.77 warning
Ticket: 6883

error: field `0` is never read
  --> src/asn1/mod.rs:36:14
   |
36 |     BerError(Err<der_parser::error::BerError>),
   |     -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     field in this variant
   |
1 year ago
Victor Julien a2c817243f rust: add MSRV as rust-version
Update github-actions to use it for the MSRV check.
1 year ago
Philippe Antoine c4b8fb7aca ssh: limit length for banner logs
Ticket: 6770
1 year ago
Philippe Antoine 271ed2008b ssh: avoid quadratic complexity from long banner
Ticket: 6799

When we find an overlong banner, we get into the state just
waiting for end of line, and we just want to skip the bytes
until then.
Returning AppLayerResult::incomplete made TCP engine retain
the bytes and grow the buffer that we parsed again and again...
1 year ago
Hadiqa Alamdar Bukhari 3aa313d0c5 dns: add dns.rcode keyword
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621
1 year ago
Victor Julien 1d3a156179 rust: update parser dependencies
Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.
1 year ago
Hadiqa Alamdar Bukhari 4b81851097 dns: add dns.rrtype keyword
It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666
1 year ago
Giuseppe Longo fe77def816 rust/sip: register pattern matching
This permits to detect the SIP protocol using pattern matching instead of
probing parser.

Since it is no longer used, the respective probing functions have been removed.
1 year ago
Giuseppe Longo 9c9b1a4230 rust/sip: add direction to transaction
This patch permits to set a direction when a new transaction is created in order
to avoid 'signature shadowing' as reported by Eric Leblond in commit
5aaf50760f
1 year ago
Giuseppe Longo c9d309219e rust/sip: register parser for tcp
This patch lets the parser to work over tcp protocol, taking care of handling
data before calling the request/response parsers.

Ticket #3351.
1 year ago
Giuseppe Longo 69f841c998 sip/parser: enforce valid chars for sip version
The `is_version_char` function incorrectly allowed characters that are not
part of the valid SIP version "SIP/2.0".

For instance, 'HTTP/1.1' was mistakenly accepted as a valid SIP version,
although it's not.

This commit fixes the issue by updating the condition to strictly
check for the correct version string.
1 year ago
Giuseppe Longo 7e993d5081 sip/parser: accept valid chars
Accepts valid characters as defined in RFC3261.
1 year ago
Giuseppe Longo 8ff80cb84d rust/sip: rustfmt sip module 1 year ago
Jeff Lucovsky f9a20dafc6 mqtt: Improve frame parsing w/mult. PDUs
This commit improves the mqtt parsing of frames to handle multiple PDUs.

Issue: 6592
1 year ago
Philippe Antoine c99d93c257 app-layer/template: use a max number of txs
Ticket: 6773
1 year ago
Philippe Antoine 68b0052018 rust: fix clippy ptr_arg warnings
error: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
   --> src/dns/log.rs:371:29
    |
371 | pub fn dns_print_addr(addr: &Vec<u8>) -> std::string::String {
    |                             ^^^^^^^^ help: change this to: `&[u8]`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
1 year ago
Philippe Antoine 80abc22f64 http2: limit number of concurrent transactions
Ticket: 6481

Instead of just setting the old transactions to a drop state so
that they get later cleaned up by Suricata, fail creating new ones.

This is because one call to app-layer parsing can create many
transactions, and quadratic complexity could happen in one
single app-layer parsing because of find_or_create_tx
1 year ago
Philippe Antoine 86de7cffa7 pgsql: parse only PDU when type is unknown
A next PDU may already be in the slice to parse.
Do not skip its parsing, ie do not use rest, but take just
the length of the pdu
1 year ago
Philippe Antoine f52c033e56 pgsql: parse auth message within its bound
If the next PDU is already in the slice next, do not use it and
restrict ourselves to the length of this PDU.
Avoids overconsumption of memory by quadratic complexity, when
having many small PDUS in one big chunk being parsed

Ticket: #6411
1 year ago
Philippe Antoine aff54f29f8 http2: handle reassembly for continuation frames
Ticket: 5926

HTTP2 continuation frames are defined in RFC 9113.
They allow header blocks to be split over multiple HTTP2 frames.
For Suricata to process correctly these header blocks, it
must do the reassembly of the payload of these HTTP2 frames.
Otherwise, we get incomplete decoding for headers names and/or
values while decoding a single frame.

Design is to add a field to the HTTP2 state, as the RFC states that
these continuation frames form a discrete unit :
> Field blocks MUST be transmitted as a contiguous sequence of frames,
> with no interleaved frames of any other type or from any other stream.
So, we do not have to duplicate this reassembly field per stream id.

Another design choice is to wait for the reassembly to be complete
before doing any decoding, to avoid quadratic complexity on partially
decoding of the data.
1 year ago
Philippe Antoine f6e1a20215 detect: dns.opcode as first-class integer
Ticket: 5446

That means it can accept ranges
1 year ago
Philippe Antoine d05f3ac791 detect: integer keywords now accept bitmasks
Ticket: 6648

Like &0x40=0x40 to test for a specific bit set
2 years ago
Philippe Antoine 370ac05419 detect/integer: rust derive for enumerations
Ticket: 6647

Allows keywords using integers to use strings in signature
parsing based on a rust enumeration with a derive.
2 years ago
Philippe Antoine 06c5dd3133 detect: integer keywords now accept negated ranges
Ticket: 6646
2 years ago
Philippe Antoine 3b65a2bb61 detect: integer keywords now support hexadecimal
So that we can write enip.revision: 0x203

Ticket: 6645
2 years ago
Philippe Antoine 38db51b878 rust: make cargo clippy clean
Fixing single_match and manual_find intertwined with SCLogDebug
2 years ago
Philippe Antoine 89936b6530 mqtt: fix logic when setting event
Especially sets transactions to complete when we get a response
without having seen the request, so that the transactions
end up getting cleaned (instead of living/leaking in the state).

Also try to set the event on the relevant transaction, instead
of creating a new transaction just for the purpose of having
the event.

Ticket: #6299
2 years ago
jason taylor 3cb7112aa5 detect: update smb.version keyword
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
jason taylor bfc0790d87 rust: fix rustfmt warnings for smb detect
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Eloy Pérez González 415722dab2 smb: add smb.version keyword
Ticket: #5075

Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Philippe Antoine 6896a93d87 rust: update test_case crate
fixes unused_unit

warning: unneeded unit expression
   --> src/bittorrent_dht/parser.rs:590:5
    |
590 | /     #[test_case(
591 | |         b"",
592 | |         "Error: discovered Dict but expected EOF" ;
593 | |         "test parse bittorrent dht packet err 1"
594 | |     )]
    | |______^
2 years ago
Philippe Antoine 259cdf169e rust: fix single_binding
error: this match could be written as a `let` statement
   --> src/nfs/nfs3_records.rs:747:9
    |
747 | /         match result {
748 | |             (r, request) => {
749 | |                 assert_eq!(r.len(), 0);
750 | |                 assert_eq!(request.handle, expected_handle);
751 | |                 assert_eq!(request.name_vec, br#"bln"#);
752 | |             }
753 | |         }
    | |_________^
2 years ago
Philippe Antoine b141eb9f11 rust: fix single_match
warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let`
   --> src/http2/parser.rs:882:17
    |
882 | /                 match ctx.value {
883 | |                     Some(_) => {
884 | |                         panic!("Unexpected value");
885 | |                     }
886 | |                     None => {}
887 | |                 }
    | |_________________^
2 years ago
Philippe Antoine 9a84681bd9 rust: fix vec_init_then_push
warning: calls to `push` immediately after creation
    --> src/pgsql/parser.rs:1179:9
     |
1179 | /         let mut database_param: Vec<PgsqlParameter> = Vec::new();
1180 | |         database_param.push(database);
     | |______________________________________^
help: consider using the `vec![]` macro: `let database_param: Vec<PgsqlParameter> = vec![..];`
2 years ago
Philippe Antoine 85329f5351 rust: fix zero_prefixed_literal
warning: this is a decimal constant
   --> src/mqtt/parser.rs:888:19
    |
888 |             0x00, 06, /* Topic Length: 6 */
    |                   ^^
    |
2 years ago
Philippe Antoine a8199bf2ca rust: fix assertions_on_constants for assert!(false)
using panic! instead with a string message
2 years ago
Philippe Antoine c49463c86f rust: fix assertions_on_constants for assert!(true)
Which will be optimized away by the compiler
2 years ago
Juliana Fajardini 8d3de85edd pgsql: fix u16 overflow in query data_row
Found by oss-fuzz with quadfuzz.

Cf https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63113

According to PostgreSQL documentation the maximum number of rows can be
the maximum of tuples that can fit onto max u32 pages - 4,294,967,295 (cf
https://www.postgresql.org/docs/current/limits.html). Some rough
calculations for that indicate that this could go over max u32, so
updating the data_row data type to u64.

Bug #6389
2 years ago
Philippe Antoine 673d13d445 rust: allow clippy::items_after_test_module
As clippy began to complain about jsonbuilder.rs
2 years ago
Jeff Lucovsky f12e026696 mqtt: Move conf code to rust
Issue: 6387

This commit moves the configuration logic to Rust.
2 years ago
Jason Ish 5d5b0509a5 requires: add requires keyword
Add a new rule keyword "requires" that allows a rule to require specific
Suricata versions and/or Suricata features to be enabled.

Example:

  requires: feature geoip, version >= 7.0.0, version < 8;
  requires: version >= 7.0.3 < 8
  requires: version >= 7.0.3 < 8 | >= 8.0.3

Feature: #5972

Co-authored-by: Philippe Antoine <pantoine@oisf.net>
2 years ago
Jason Ish 15ed51f9b8 feature: provide a Rust binding to the feature API
As the feature module is not available for Rust unit tests, a mock
version is also provided.
2 years ago
Juliana Fajardini 1afb485dfa pgsql: remove unused msg field
The `ConsolidatedDataRow` struct had a `length` field that wasn't truly
used.

Related to
Bug #6389
2 years ago
Juliana Fajardini 30ac77ce65 pgsql: add cancel request message
A CanceldRequest can occur after any query request, and is sent over a
new connection, leading to a new flow. It won't take any reply, but, if
processed by the backend, will lead to an ErrorResponse.

Task #6577
2 years ago
Juliana Fajardini 7fa8bbfe43 pgsql: extract length validation into function
This is called so many times that it seems to make sense that we use a
function for this.
2 years ago
Victor Julien b8440a0917 jsonbuilder: add set_int for signed ints
Bug: #6615
2 years ago
Jason Ish f91122e0e8 dns: replace usage of rs_dns_tx_get_query_name with SCDnsTxGetQueryName
SCDnsTxGetQueryName was introduced to allow for getting the query name
in responses as well as requests, so covers the functionality of
rs_dns_tx_get_query_name.
2 years ago
Jason Ish 482325e28b dns: add dns.query.name sticky buffer
This buffer is much like dns.query_name but allows for detection in both
directions.

Feature: #6497
2 years ago
Jason Ish 5f99abb0cb dns: add dns.answer.name keyword
This sticky buffer will allow content matching on the answer names.
While ansers typically only occur in DNS responses, we allow the buffer
to be used in request context as well as the request message format
allows it.

Feature: #6496
2 years ago
Jason Ish 9464d0b14a dns: consolidate DNSRequest and DNSResponse to DNSMessage
DNS request and response messages follow the same format so there is
no reason not to use the same data structure for each. While its
unlikely to see fields like answers in a request, the message format
does not disallow them, so it might be interesting data to have the
ability to log.
2 years ago
Jason Ish e2d7a7f877 dns: rustfmt with latest stable 2 years ago
Jason Ish 4620776a30 rustfmt: replace deprecated fn_args_layout with fn_params_layout 2 years ago
Philippe Antoine 1b5e04bee3 http2: do not have leading space for response line
Ticket: 6547
2 years ago
Juliana Fajardini bdec2d8ea8 pgsql: don't log password msg if password disabled
If the logging of the password is disabled, there isn't much point in
logging the password message itself.
2 years ago
Juliana Fajardini 9aeeac532e pgsql: remove probe_ts function
With the changes in the probing_ts function, this other one could become
obsolete. Remove it, and directly call `parser::parse_request` when
checking for gaps, instead.
2 years ago
Juliana Fajardini 53d29f652a pgsql: remove unused error handling call 2 years ago
Juliana Fajardini afd6e4dc41 pgsql: don't log unknown message type 2 years ago
Juliana Fajardini 4f85d06192 pgsql: fix probing functions
Some non-pgsql traffic seen by Suricata is mistankenly identified as
pgsql, as the probing function is too generic. Now, if the parser sees
an unknown message type, even if it looks like pgsql, it will fail.

Bug #6080
2 years ago
Juliana Fajardini 1ac5d97259 pgsql: add unknonwn frontend message type
We had unkonwn message type for the backend, but not the frontend
messages. It's important to better identify those to improve pgsql
probing functions.

Related to
Bug #6080
2 years ago
Victor Julien d3ccff5822 detect/asn1: handle in PMATCH
Since the asn1 keyword is processing payload data, move the handling of
the keyword into the PMATCH with content inspection.

Use u32 as buffer length in the Rust FFI
2 years ago
Victor Julien 132fe57ac6 rust: add copyright header to common.rs 2 years ago
Philippe Antoine e38b9de6a2 output/krb5: have krb5 properties in alerts
Ticket: 5977
2 years ago
Philippe Antoine 8a09bff0aa output/tftp: have tftp properties in alerts
Ticket: 6501
2 years ago
Philippe Antoine 0b6b015e26 output/alert: rewrite code for app-layer properties
Especially fix setup-app-layer script to not forget this part

This allows, for simple loggers, to have a unique definition
of the actual logging function with the jsonbuilder.
This way, alerts, files, and app-layer event can share the code
to output the same data.

Ticket: #3827
2 years ago
Philippe Antoine 90c17652a3 rust: remove unused
Ticket: #4083
2 years ago
Sascha Steinbiss 0c55fe3515 detect: add mqtt.connect.protocolstring
Ticket:  OISF#6396
2 years ago
Philippe Antoine e3cd0d073f http2: app-layer event for userinfo in uri
Ticket: #6426

as per RFC 9113
":authority" MUST NOT include the deprecated userinfo subcomponent
for "http" or "https" schemed URIs.
2 years ago
Philippe Antoine 6249722589 http2: normalize host when there is user info
Ticket: 6479
2 years ago
Philippe Antoine b6cd66f41d http2: update brotli crate
Fixes debug assertion found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63144
2 years ago
Philippe Antoine 46a46e5b1f http2: event on mismatch between authority and host
Ticket: #6425
2 years ago
Philippe Antoine ae72ce77fa detect: parse units for integers
Ticket: #6423

Especially for filesize, instead of just a number, a signature
can use a number and a unit such as kb, mb or Gb
2 years ago
Daniel Olatunji 54de0450f4 rust: remove cbindgen:ignore on frames module
This directive is no longer required, and does
mess up the rustdoc description of the module.
2 years ago
Daniel Olatunji 5c0af0b203 rust/doc: add docstring to rust module files.
Issue: #4584
2 years ago
Philippe Antoine 14a4c6c696 rust: update brotli decompressor crate
cf https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59687
2 years ago
Philippe Antoine 9157070907 quic: v2 support per rfc 9369
Ticket: #4968
2 years ago
Yatin Kanetkar b67ff4badf dhcp: Log Vendor Client Identifier (dhcp option 60)
* Log vendor client identifier (dhcp option 60) if extended dhcp
logging is turned on. This required the `vendor_client_identifier` to
be added to the json schema. Validation done using an SV Test
* Added `requested_ip` to the json schema as well, since it was
missed. My SV test failed without it.

Feature #4587
2 years ago
Philippe Antoine 5bdbc1a313 rdp: do not use zero-bit bitflag
cf https://docs.rs/bitflags/latest/bitflags/#zero-bit-flags

As warned by clippy 1.72.0
2 years ago
Philippe Antoine b235e85c68 rust: fix clippy warnings for version 1.72.0
Includes using the right prototype for C SRepCatGetByShortname
2 years ago
Victor Julien 89f1837625 rust: update cargo.lock 2 years ago
Shivani Bhardwaj 8770431986 dcerpc: accept ALTER_CONTEXT as a valid request
So far, if only the starting request was a DCERPC request, it would be
considered DCERPC traffic. Since ALTER_CONTEXT is a valid request type,
it should be accepted too.

Reported and patch proposed in the following Redmine ticket by
InterNALXz.

Bug 6191
2 years ago
Victor Julien 389f166d78 file: remove FILE_USE_DETECT flag
All implementations were converted to use the logic, so the flag itself
can be removed.
2 years ago
Shivani Bhardwaj d4e674b390 rust: fix clippy warnings 2 years ago
Victor Julien 0068b81269 rust: update cargo.lock 2 years ago
Philippe Antoine 60db5e981c http2: do not append data after closing file
Ticket: #6211

Completes commit 02dece5db5

Once a http2 stream has end of stream flag, we close the file.
If we see new data frames with this stream id, the new_chunk
function should ignore them as the file was already closed.
2 years ago
Jeff Lucovsky 690b65ae88 detect/byte_math: Permit var name for bytes value
Issue: 6145

Modifications to permit a variable name to be used for the byte_math
bytes value.
2 years ago
Philippe Antoine 02dece5db5 http2: file tracker is initialized when file is closed
Ticket: #6130

This avoids quadratic complexity by having http2_range_key_get
looking in a growing number of frames
2 years ago
Sascha Steinbiss 1521b77edd rfb: also set unimplemented auth types 2 years ago
Sascha Steinbiss 1606aca881 rfb: ensure logging of incompletely parsed txs 2 years ago
Sascha Steinbiss 1f8a5874fb rfb: never return error on unknown traffic
We only try to parse a small subset of what is possible in
RFB. Currently we only understand some standard auth schemes
and stop parsing when the server-client handshake is complete.
Since in IPS mode returning an error from the parser causes
drops that are likely uncalled for, we do not want to return
errors when we simply do not understand what happens in the
traffic. This addresses Redmine #5912.

Bug: #5912.
2 years ago
Sascha Steinbiss 836fff3679 rfb: add myself as contributor 2 years ago
Sascha Steinbiss bd1fbf392e rfb: be more strict parsing the version 2 years ago
Philippe Antoine d40dca5e55 dcerpc: maximum number of live transactions also for UDP
Ticket: #6129

Avoids that quadratic complexity gets too bad
2 years ago
Jason Ish 68d0d6ca24 rust: fix unit test link error on Rust 1.70
Rust 1.70 appears to now link code on both branches of `if cfg!(test)`
now causing Rust unit tests to fail as that pattern was used to
disable functions only available when linked with the Suricata C code.

To work-around this issue, provide two versions of the `new` function,
one for unit tests and one when running as an application.
2 years ago
Philippe Antoine 7256ec8a6e detect/http2: do not escape ':' in header name or value
for keywords http.request_header and http.response_header

Ticket: #5780
2 years ago
Philippe Antoine 4c466ec5f4 rust/pgsql: remove unused/unconstructed enum variants 2 years ago
Philippe Antoine f2a18e91c4 rust: define AppLayerEventType only in rust
And detect.h does no longer depend on app-layer-events.h
2 years ago
Philippe Antoine 668501c225 rust: remove unused 2 years ago
Philippe Antoine 7ca43e7e1f output/snmp: log version from tx
and not the one from state

If a SNMP flow starts with a V2 version transaction,
then there is a V3i version transaction,
we will now log V3 for the second transaction
2 years ago
Philippe Antoine 0ec0d8de67 output/rfb: remove unused function parameters 2 years ago
Philippe Antoine 24c2702a05 output/mqtt: remove unused function parameters 2 years ago
Philippe Antoine 09d364b32f output/krb5: remove unused function parameters 2 years ago
Lancer Cheng abc76e27de smb: fix data padding logic in writeAndX parser
Bug: #6008
2 years ago
Lancer Cheng 000eb91078 smb: fix wrong data offset when wct = 12
Bug: #6008
2 years ago
Philippe Antoine 6350736882 http2: avoid quadratic complexity in headers
When adding an element to the dynamic headers table, the oldest
ones may get evicted. When multiple elements get evicted, they
should get evicted all at once with drain, instead of one by one
as there will be a massive move each time.

Ticket: #6103
2 years ago
Philippe Antoine 7d3aa91bf4 mqtt: fix quadratic complexity
get_tx_by_pkt_id loops only over the last transactions
in case there is a transaction flood

Ticket: #6100
2 years ago
Haleema Khan 8e19906afa mqtt: rustfmt mqtt.rs 2 years ago
Haleema Khan e474858e25 mqtt: add mqtt frames
Adds PDU, Header and Data frame to the MQTT parser.
Ticket: 5731
2 years ago
Jason Ish 33827beae5 jsonbuilder: check buffer growth
Use try_reserve before growing the internal buffer, and the internal
state vector. This allows allocation errors to be caught and an error
returned instead of just aborting the process.

Ticket: #6057
2 years ago
Jason Ish 95cfc2b34f jsonbuilder: rustfmt
Some very minor changes to formatting.
2 years ago
Jason Ish 039c27789b rust: use 2021 edition
With the MSRV being bumped to 1.62 for 7.0, we can move the edition up
to 2021.
2 years ago
Jason Ish c30fff8bcb rust/doc: restore comment with code example, but ignore
Use backticks for proper markdown processing. As Rust code in
backticks is compiled, and this is a non-complete example, tag the
code sample to be ignored.
2 years ago
Philippe Antoine 5391f0a8a0 detect: http_response_line for HTTP2
Ticket: #4067

Synthetized as HTTP/2 <STAT>\r\n
2 years ago
Philippe Antoine 0dca8cc796 detect: http_request_line support for HTTP2
Ticket: #4067

Synthetized as <METHOD> <URI> HTTP/2\r\n
2 years ago
Jason Ish 13fe957b7e rust/doc: wrap some code examples in backticks 2 years ago
Victor Julien d4c60924f1 rust/doc: fix doc compile issues 2 years ago
Eloy Pérez González ed91d689f2 krb5: use req_type instead of msg_type to get request type 2 years ago
Eloy Pérez González a9b7241417 krb5: set msg_type for KRB-ERROR messages to MessageType::KRB_ERROR 2 years ago
Eloy Pérez González 511dbfe171 krb5: add AS-REQ and TGS-REQ transactions
Fix bug in ticket #4529
2 years ago
Victor Julien f9276fdf00 rust: spelling fixes
Thanks to Josh Soref.
2 years ago
Victor Julien cdd3251982 snmp: fix spelling
Thanks to Josh Soref.
2 years ago
Victor Julien ee7ed99b6f rust: spelling 2 years ago
Victor Julien d630f0fa34 rust: rustfmt files with recent new tests 2 years ago
Victor Julien 77f1658c2a rust: fix new clippy warnings 2 years ago
Lancer Cheng 0cf742a9ca smb: add unit tests
Issue: 4865
2 years ago
tianjinshan 2c0c6cb0a5 smb/ntlmssp: fix parsing of negotiate flags
Ticket: #5783
2 years ago
Jason Ish 0e55307c1d app-layer: remove APP_LAYER_PARSER_OPT_UNIDIR_TXS
This flag is no longer needed as a parser can now create a transaction
as unidirectional.

Setting this flag also doesn't make sense on parsers that may have
request/reply and some unidirectional messaging.
2 years ago
Jason Ish f8ec993401 rust/time: add note why this needs to be pinned 2 years ago
Jason Ish 5925b63d82 rust: update x509-parser to 0.15.0 2 years ago
William Correia e378aa8d15 modbus: bump crate version
sawp 0.12 is available and addresses future compilation failures in
dependent crates.
Updated modbus test case to expect 12 bytes needed instead of 15. This
aligns with expectations as the test case slices 3 bytes off the end of
a 12 byte message so needing 12 bytes is correct.

Ticket #5989
2 years ago
Jason Ish b4f0d3c741 rust: update der-parser to 8.2.0
Minimal modifications required on the Suricata side, mainly for fields
becoming private and needing an accessor instead.

Note: As the kerberos parser still depends on der-parser 6.0, we still
have to depend on that so it is depended on, but renamed to
der-parser6. There is not an udpated kerberos-parser yet that uses
der-parser 8.2.0.

Ticket: #5991
2 years ago
Jason Ish 3d9fc3bf1d rust: update snmp-parser to 0.9.0
Updating snmp-parser required directly depending on the asn1-rs crate
for the Oid type, as snmp-parser does not re-export this type anymore.

Ticket: #5992
2 years ago
Jason Ish 0d6628c64e rust: update cargo.lock
Update Cargo.lock, most importantly the Nom 5.1.3 update which will
prevent future breakage by Rustc.
2 years ago
Jason Ish d2fb958e28 rust: fix clippy lint for assert
Fix done automatically by clippy --fix
2 years ago
Haleema Khan 3531a4abaa rfb: rustfmt rfb.rs 2 years ago
Haleema Khan 3eee311350 rfb: add rfb frames, update tests
Adds a PDU frame to the RFB parser.
Update function signature in tests to reflect frames

Ticket: 5717
2 years ago
Victor Julien 0bbc411743 nfs: fix newline in debug messages 2 years ago
Philippe Antoine 9adb59bcdb http2: faster when reducing dynamic headers size
avoid quadratic complexity from removing the first element
and copying all the contents a big number fo times.

Ticket: #5909
2 years ago
Jason Ish 8ef410e284 app-layer: add direction to transaction creation where needed
Build on Eric's but set the direction on transaction creation when
needed. I think this makes it a little more clear, and easier to
document when creating single direction transactions.

This also somewhat abstracts the inner-workings of a directional
transaction from the implementation.

Ticket: #4759
2 years ago
Eric Leblond 9f4ca26962 sip: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond 8b0d56c414 nfs: TX are not unidirectional
NFS transactions are not unidirectional so we should not declare
them as such.
2 years ago
Eric Leblond 19174de4f3 quic: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond 44482a68b0 ntp: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond c64e4526cd krb: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond 322f3896ba mqtt: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond 7ce557a44c ike: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago
Eric Leblond 8926d82465 dns: add TX orientation
Set no inspection in the opposite side of the transaction.

Ticket: #5799
2 years ago