Tests better the ignored flags functionality.
This functionality is only used at detection for default mode,
it is not used for any, plus or not mode.
So, have unit tests about igored flags with default mode with
both matching and non matching cases
Ticket: 6724
Allows to use numerical values for example
Also fixes some unit tests that were returning 1 after goto error
FlagsTestParse05 especially took this path as
de->ignored_flags != (TH_SYN|TH_RST) was false
we had de->ignored_flags == 0xff ^ (TH_SYN|TH_RST)
And then, we had a match, instead of what the not-run code
was supposing.
Ticket: 6724
Allows sugar syntax for bitflags keywords.
While the expressivity does not increase, because we could already
use numerial values with all generic integer modes, this modifier
prefix is used with the strings, and follows the syntax
that is already used for fragbits and tcp.flags keyword.
So that http2.frametype and http2.errorcode can use EnumString
without changing the format, as they used UPPERCASE instead of
the default EnumString snake_case
IKE attributes are an array of TLV style objects, this means there can
be duplicate types seen on the wire. However, Suricata logs these as a
mapping with the type as the key. This can result in the JSON
containing duplicate keys.
To address this, log the attributes as an array of objects, allow
duplicates to exist, for example:
"client": {
"proposals": [
{
"sa_life_duration": "Unknown",
"sa_life_duration_raw": 86400,
}
}
}
is now logged as:
"client": {
"proposals": [
{"key": "sa_life_duration", "value": "Unknown", "raw": 86400}
]
}
Also adds `"version": 2` to each IKE record to note the change of
format from previous versions.
Ticket: #7902
email.body_md5 matches on md5 hash generated from email body
This keyword maps to the EVE field email.body_md5
It is a sticky buffer
Supports prefiltering
Ticket: #7587
To support alternative cargo and rustc programs (such as cargo-1.82),
respect CARGO and RUSTC environment variables during ./configure much
like CC.
RUSTFMT is also respected as that is required for the tests, and Cargo
can't figure this out like it can for rustc (perhaps a bug in the
packaging).
For cbindgen, we have also have to make sure the cargo environment
variable is set for each invocation.
To build with Ubuntu's Rust 1.82 packaging:
CARGO=cargo-1.82 RUSTC=rustc-1.82 RUSTDOC=rustdoc-1.82 \
./configure
Note that setting RUSTDOC is only required for commands like "make
check" to pass.
Ticket: #7877
Issue: 7853
Support the use of `from_base64` with no optional values. In this case,
the default values for:
- mode RFC4648
- offset: 0
- bytes: buffer size
will be used.
Fix new warning present in Rust 1.89.
warning: hiding a lifetime that's elided elsewhere is confusing
--> src/ldap/types.rs:191:30
= help: the same lifetime is referred to in inconsistent ways, making the signature confusing
= note: `#[warn(mismatched_lifetime_syntaxes)]` on by default
help: use `'_` for type paths
Ticket: 7658
Suricata will not handle well if we open a file for this tx,
do not close it, but set the transaction state to completed.
RFC 9113 section 6.1 states:
If a DATA frame is received whose Stream Identifier field is 0x00,
the recipient MUST respond with a connection error (Section 5.4.1)
of type PROTOCOL_ERROR.
Have htp follow Suricata versioning so we don't have to worry about
version updates as it changes.
For example, between 8.0.0-beta1 and 8.0.0-rc1 there were changes to
the htp, however the version stayed at 2.0.0 making it impossible to
publish these changes to crates.io.
This patch introduces new option to dataset keyword.
Where regular dataset allows match from sets, dataset with json
format allows the same but also adds JSON data to the alert
event. This data is coming from the set definition it self.
For example, an ipv4 set will look like:
[{"ip": "10.16.1.11", "test": "success","context":3}]
The syntax is a JSON array but it can also be a JSON object
with an array inside. The idea is to directly used data coming
from the API of a threat intel management software.
The syntax of the keyword is the following:
dataset:isset,src_ip,type ip,load src.lst,format json, \
enrichment_key src_ip, value_key ip;
Compare to dataset, it just have a supplementary option key
that is used to indicate in which subobject the JSON value
should be added.
The information is added in the even under the alert.extra
subobject:
"alert": {
"extra": {
"src_ip": {
"ip": "10.6.1.11",
"test": "success",
"context": 3
},
The main interest of the feature is to be able to contextualize
a match. For example, if you have an IOC source, you can do
[
{"buffer": "value1", "actor":"APT28","Country":"FR"},
{"buffer": "value2", "actor":"APT32","Country":"NL"}
]
This way, a single dataset is able to produce context to the
event where it was not possible before and multiple signatures
had to be used.
The format introduced in datajson is an evolution of the
historical datarep format. This has some limitations. For example,
if a user fetch IOCs from a threat intel server there is a large
change that the format will be JSON or XML. Suricata has no support
for the second but can support the first one.
Keeping the key value may seem redundant but it is useful to have it
directly accessible in the extra data to be able to query it
independantly of the signature (where it can be multiple metadata
or even be a transformed metadata).
In some case, when interacting with data (mostly coming from
threat intel servers), the JSON array containing the data
to use is not at the root of the object and it is ncessary
to access a subobject.
This patch implements this with support of key in level1.level2.
This is done via the `array_key` option that contains the path
to the data.
Ticket: #7372
If the connection is lost (for example, Suricata is restarted), try to
re-open the connect and re-execute the command.
This was the behavior of the Python implementation.
Ticket: #7746
For example:
error: lifetime flowing from input to output with different syntax can be confusing
--> htp/src/headers.rs:475:16
|
475 | fn null(input: &[u8]) -> IResult<&[u8], ParsedBytes> {
| ^^^^^ ----- ----------- the lifetimes get resolved as `'_`
| | |
| | the lifetimes get resolved as `'_`
| this lifetime flows to the output
|
note: the lint level is defined here
--> htp/src/lib.rs:3:9
This currently only happens when using the Rust nightly compiler, which
we use for our fuzz builds.
Update all deps with cargo update. Additionally, apply the updated
versions to the Cargo.toml, which while not stricly required, does
make it more clear what the version in use is.
There shouldn't be duplicated messages in the requests Vec. And thus
the parser shouldn't log duplicated keys nor messages. Add debug
validations to ensure this.
With PGSQL's current state machine, most frontend/ client messages will
lead to the creation of a new transaction - which would prevent
duplicated messages being pushed to the requests array and reaching the
logger.
The current exceptions for that are:
- CopyDataIn
- CopyDone
- CopyFail
Thus, debug statements were added for those cases.
CopyDone and CopyFail, per the documentation, shouldn't be seen
duplicated on the wire for the same transaction. CopyDataIn -- yes, but
we consolidate those, so the expectation is that they won't be
duplicated in the requests array or when reaching the logger either.
Related to
Task #7645