Ticket: 7973
Files were supported on both TCP and UDP. But file detection keywords
such as file.data made signatures loading fail if the signature
was using an app-layer protocol that enabled on udp only, even
if the signatures could run smoothly.
This also exposed a difference between the handling of TD alerts in
firewall vs non-firewall mode. In firewall mode the table/hook is also
part of the alert ordering to make sure actions from packet:td are
applied before app:td. Handle that explicitly for now.
Allow keywords to specify in which detect table they can function.
E.g. the pre_flow table will not support flow keywords, as no flow is
availble at this time.
Ticket: 7667
Currently no functions are exported.
DetectFile* struct are moved to detect-file-data.h where
they make more sense.
ifndef SURICATA_BINDGEN_H is used for bindgen to exclude
pcre2 related code
Ticket: 7665
Instead of each keyword calling DetectSetupDirection, use a
new flag SIGMATCH_SUPPORT_DIR so that DetectSetupDirection gets
called, before parsing the rest of the keyword.
Allows to support filesize keyword in transactional signatures
For firewall mode, set the pseudo table in the rule and use this
in alert queue ordering, so that rule actions are applied in the
expected order:
packet:filter -> packet:td -> app:filter -> app:td
This makes sure that a packet:td drop is applied before a app:filter
accept.
Config:
Firewall rules are like normal rule, with some key differences.
They are loaded separate, and first, from:
```yaml
firewall-rule-path: /etc/suricata/firewall/
firewall-rule-files:
- fw.rules
```
Can also be loaded with --firewall-rules-exclusive: Mostly for QA purposes.
Allow -S with --firewall-rules-exclusive, so that firewall and threat detection
rules can be tested together.
Rules:
Differences with regular "threat detection" rules:
1. these rules are evaluated before threat detection rules
2. these rules are evaluated in the order as they appear in the rule file
3. currently only rules specifying an explicit hook at supported
a. as a consequence, no rules will be treated as (like) IP-only, PD-only or
DE-only
Require explicit action scope for firewall rules. Default policy is
drop for the firewall tables.
Actions:
New action "accept" is added to allow traffic in the filter tables.
New scope "accept:tx" is added to allow accepting a transaction.
Tables:
Rulesets are per table.
Table processing order: `packet:filter` -> `packet:td` -> `app:*:*` -> `app:td`.
Each of the tables has some unique properties:
`packet:filter`:
- default policy is `drop:packet`
- rules are process in order
- action scopes are explicit
- `drop` or `accept` is immediate
- `accept:hook` continues to `packet:td`
`packet:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `app:*:*` or `alert/action finalize`
`app:*:*`:
- default policy is `drop:flow`
- rules are process in order
- action scopes are explicit
- `drop` is immediate
- `accept` is conditional on possible `drop` from `packet:td`
- `accept:hook` continues to `app:td`, `accept:packet` or `accept:flow`
continues to `alert/action finalize`
`app:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `alert/action finalize`
Implementation:
During sigorder, split into packet:filter, app:*:* and general td.
Allow fw rules to work when in pass:flow mode. When firewall mode is enabled,
`pass:flow` will not skip the detection engine anymore, but instead
process the firewall rules and then apply the pass before inspecting threat
detect rules.
New facility to allow a user to specify where to hook a rule
into the engine. This patch adds this for packets, adding two
hooks:
- `all`: to let a rule be evaluated by all rules
- `flow_start`: to have a rule be evaluated only for the first
packet in both directions
Implemented by adding a hook flags field in the packet.
Generic:
<app_proto>:request_started and <app_proto>:response_started
<app_proto>:request_complete and <app_proto>:response_complete
Per protocol, it uses the registered progress (state) values. E.g.
tls:client_hello_done
A rule ruleset could be:
pass tls:client_hello_done any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)
drop tls:client_hello_done any any -> any any (sid:22;)
The pass rule is evaluated when the client hello is parsed, and if it
doesn't match the drop rule will be evaluated.
Registers each generic lists as "<alproto>:<progress state>:generic"
(e.g. "tls:client_hello_done:generic").
Ticket: #7485.
For some rules the values are always 0, for others they get set, so not
consistent. Also not documented and not tested and should be replaced by
a "suricata.rule" lib.
For example, these globals would be set for this rule:
alert tcp any any -> any any (flow:to_server;
flowbits:isnotset,dataset_added; lua:dataset.lua;
flowbits:set,dataset_added;sid:1;rev:99;)
But not this rule:
alert dns any any -> any any (msg:"TEST DNS LUA dns.rrname"; \
content:"org"; lua:test-rrname.lua; sid:1; rev:1;)
Related to ticket
https://redmine.openinfosecfoundation.org/issues/7490 (but not a fix)
This does not affect the functions with similar names for getting rule
details in output context.
such as ja4.
Why ?
We do not want to see hard-coded protocol constants such as
ALPROTO_QUIC directly used in generic code in detect-parse.c
How ?
From the keyword point of view, this commit adds the function
DetectSignatureSetMultiAppProto which is similar to
DetectSignatureSetAppProto but takes multiple alprotos.
It restricts the signature alprotos to a set of possible alprotos
and errors out if the interstion gets empty.
The data structure SignatureInitData gets extended with
a fixed-length array, as the use case is a sparse number of protocols
Ticket: 7304
Flowbits can make a rule such as a packet rule be treated as a stateful
rule, without actually changing the rule type.
Add a flag to allow reporting such cases via engine analysis.
Task #7456
If `prefilter` is used it should override automatic fast pattern
selection.
Fixes: d6b56929d3 ("detect: set mpm/prefilter during signature parsing")
Ticket: #7523.
Because some alprotos will remain static and defined as a constant,
such as ALPROTO_UNKNOWN=0, or ALPROTO_FAILED.
The regular already used protocols keep for now their static
identifier such as ALPROTO_SNMP, but this could be made more
dynamic in a later commit.
ALPROTO_FAILED was used in comparison and these needed to change to use
either ALPROTO_MAX or use standard function AppProtoIsValid
Ticket: 2224
It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.
For multi buffers, absent matches if there are 0 buffers.
For file keywords, absent matches if there is no file.
Victor Julien
Philippe Antoine
Lukas Sismis
Philippe Antoine
Shivani Bhardwaj
Jason Ish
Jeff Lucovsky
Juliana Fajardini