Commit Graph

13303 Commits (a7f4fd12d5d4e6f8c7f987cd6cdc9b79f8847735)

Author SHA1 Message Date
Philippe Antoine a7f4fd12d5 detect: remove never set SIGMATCH_NOT_BUILT 2 years ago
Philippe Antoine 794f991ad6 unittests: more realistic packet from UTHBuildPacketReal
So that its contents can be reused when translating unit tests
to SV tests
2 years ago
Philippe Antoine 9814b698c8 detect/dns: move keywords to rust
Ticket: 7529
Ticket: 3725

Adds url for dns.opcode on the way
2 years ago
Philippe Antoine bb9b8d2460 detect: new helper to register multi-buffer with progress
This allows to use these engines for hook rules needing exact
progress (checked in SigValidate)
2 years ago
Philippe Antoine 128ee9ba46 output: fix leak in case of alloc error
CID: 1638290
2 years ago
Philippe Antoine 85f2f597f1 defrag: remove unnecessary NULL check
CID: 727861
2 years ago
Philippe Antoine 9dac5ec23c util/mpm: prevents double free
CID: 1645545

PatternDatabaseGetCached frees cd on success
So, we should NULL it, so that in case PatternDatabaseGetSize fails
and we goto error, we do not free cd again.
2 years ago
Philippe Antoine e301e038ef detect: explicitly skip check on SCConfGet
CID: 1644571
2 years ago
Eric Leblond 5cf6459f3f eve/flow: log tx_cnt
This patch adds a `tx_cnt` field to `netflow` events to give some
context about the underlying protocol activity.

Ticket: #7635
2 years ago
Eric Leblond 668c6d646e eve/netflow: add tx_cnt
This patch adds a `tx_cnt` field to `netflow` events to give some
context about the underlying protocol activity.

Ticket: #7635
2 years ago
Eric Leblond db11078315 eve/smb: add tx_id to event
As SMB protocol is using heavily transactions, getting the transaction
ID in SMB events can be really useful for automated analysis.
2 years ago
Jeff Lucovsky d59f5d6db6 output/rotate: Remove extra rotation flag register
Issue: 3436

Remove duplicate register of the rotation flag. Eventually, this will
cause corruption when the file context has been freed and the rotation
flag is deregistered.
2 years ago
Jeff Lucovsky 33445d01b3 output/rotate: Serialize rotation flag handling
Issue: 3436

Serialize rotation flag handling to avoid corruption.
2 years ago
Alice Akaki bda0890834 detect: add email.received keyword
email.received matches on MIME EMAIL Received
This keyword maps to the EVE field email.received[]
It is a sticky buffer
Supports multiple buffer matching
Supports prefiltering

Ticket: #7599
2 years ago
Victor Julien e3c6554ee6 detect/app-layer-protocol: allow matching on 'unknown' 2 years ago
Victor Julien 8f9c05243c firewall: detect: set firewall support flag on select keywords 2 years ago
Victor Julien f96e97205c firewall: detect: add feature flag for keywords supporting firewall 2 years ago
Victor Julien e6bd69b419 firewall: detect: set per rule table
For firewall mode, set the pseudo table in the rule and use this
in alert queue ordering, so that rule actions are applied in the
expected order:

        packet:filter -> packet:td -> app:filter -> app:td

This makes sure that a packet:td drop is applied before a app:filter
accept.
2 years ago
Victor Julien 1643b017b6 detect: don't set conflicting packet/flow actions
If for the same a packet a drop rule and a pass rule would match,
the applying of actions could be contradictionary:

- the drop would be applied to the packet
- the pass rule would also be considered, not overriding the drop,
  but still setting the flow pass flag.

This would lead to the packet being dropped, but the rest of the
flow getting passed, including retransmissions of the dropped
packet.

This patch only sets drop/pass actions if no conflicting action
has been set on the packet before. It respects the action-order.

Bug: #7653.
2 years ago
Victor Julien d6e61b6690 firewall: detect: add explanation 2 years ago
Victor Julien 49979587ce detect/alert: minor cleanup 2 years ago
Alice Akaki ca429ef5e3 detect: add email.url keyword
email.url matches on URLs extracted from an email
This keyword maps to the EVE field email.url[]
Supports multiple buffer matching
Supports prefiltering

Ticket: #7597
2 years ago
Victor Julien 88e96c9d49 detect/domain: move transform fully to rust 2 years ago
Victor Julien 679d25ccd2 detect/prefilter: fix coverity issue
** CID 1646448:  Null pointer dereferences  (NULL_RETURNS)
/src/detect-engine-prefilter.c: 944 in SetupNonPrefilter()

________________________________________________________________________________________________________
*** CID 1646448:  Null pointer dereferences  (NULL_RETURNS)
/src/detect-engine-prefilter.c: 944 in SetupNonPrefilter()
938                 const DetectBufferType *buf = DetectEngineBufferTypeGetById(de_ctx, list_id);
939                 /* for now, exclude app-layer-events, as they are not tied to a specific
940                  * progress value like other keywords. */
941                 SCLogDebug("list_id %d buf %p", list_id, buf);
942                 if (list_id == app_events_list_id)
943                     continue;
>>>     CID 1646448:  Null pointer dereferences  (NULL_RETURNS)
>>>     Dereferencing "buf", which is known to be "NULL".
944                 if (buf->packet) {
945                     SCLogDebug("packet buf");
946                     /* packet is handled below */
947                     pkt_non_pf = true;
948                 } else if (buf->frame) {
949                     for (DetectEngineFrameInspectionEngine *f = de_ctx->frame_inspect_engines;
2 years ago
Jason Ish 1f57bd2c1b log-pcap: error out if malloc fails for bpf filter 2 years ago
Victor Julien 57c73880db lua: enable lua rules by default
Now that sandboxing is in place, lua rule support is enabled by default.
2 years ago
Jeff Lucovsky fdc43e5ac4 app/ftp: Eliminate scan-build warning
Scan-build reports that FTPRealloc could be called with size=0. Modify
the logic so it's never passed 0.
2 years ago
Jeff Lucovsky e0131f6098 detect/ftp: Add ftp.reply rule keyword
Issue: 7508

Add the rule detection keyword "ftp.reply". This is a multi-buffer
command.
2 years ago
Jeff Lucovsky d674ce2510 app/ftp: Use Rust FTP response line handling
Use the Rust logic to parse FTP response lines with the goal to support
multi-buffer matches better.

A side effect is that the completion codes are no longer strings; the
schema update reflects this.

Issue: 4082
2 years ago
Alex Savage 778053876b pop3: app-layer parser using sawp-pop3
This module uses the sawp-pop3 crate to parse POP3 requests and responses
Features:
- eve logging
- events for parsable but non-RFC-compliant messages

Ticket: 3243
2 years ago
Eric Leblond 585c624482 tld: add new transform
Extract the TLD from a buffer.

Ticket: #5639
2 years ago
Eric Leblond 0b031a33c6 domain: add new transformation
Ticket: #5639
2 years ago
Victor Julien 98372b6599 firewall: beginning of ruleset analyzer
Output a `firewall.json` with a per table list of rules.

Also output a sorted list of the threat detection rules.
2 years ago
Victor Julien 31d048ed4b firewall: start of firewall rules support
Config:

Firewall rules are like normal rule, with some key differences.

They are loaded separate, and first, from:

```yaml
firewall-rule-path: /etc/suricata/firewall/
firewall-rule-files:
  - fw.rules
```

Can also be loaded with --firewall-rules-exclusive: Mostly for QA purposes.

Allow -S with --firewall-rules-exclusive, so that firewall and threat detection
rules can be tested together.

Rules:

Differences with regular "threat detection" rules:

1. these rules are evaluated before threat detection rules

2. these rules are evaluated in the order as they appear in the rule file

3. currently only rules specifying an explicit hook at supported

   a. as a consequence, no rules will be treated as (like) IP-only, PD-only or
      DE-only

Require explicit action scope for firewall rules. Default policy is
drop for the firewall tables.

Actions:

New action "accept" is added to allow traffic in the filter tables.

New scope "accept:tx" is added to allow accepting a transaction.

Tables:

Rulesets are per table.

Table processing order: `packet:filter` -> `packet:td` -> `app:*:*` -> `app:td`.

Each of the tables has some unique properties:

`packet:filter`:
- default policy is `drop:packet`
- rules are process in order
- action scopes are explicit
- `drop` or `accept` is immediate
- `accept:hook` continues to `packet:td`

`packet:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `app:*:*` or `alert/action finalize`

`app:*:*`:
- default policy is `drop:flow`
- rules are process in order
- action scopes are explicit
- `drop` is immediate
- `accept` is conditional on possible `drop` from `packet:td`
- `accept:hook` continues to `app:td`, `accept:packet` or `accept:flow`
  continues to `alert/action finalize`

`app:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `alert/action finalize`

Implementation:

During sigorder, split into packet:filter, app:*:* and general td.

Allow fw rules to work when in pass:flow mode. When firewall mode is enabled,
`pass:flow` will not skip the detection engine anymore, but instead
process the firewall rules and then apply the pass before inspecting threat
detect rules.
2 years ago
Victor Julien 10dcc383ae detect/xbits: implement tx bits
Implement `xbits:set,mybit,track tx;` and `xbits:isset,mybit,track tx;`.

Store these in the AppLayerTxData.

Ticket: #6455.
2 years ago
Victor Julien 7c8a55de54 eve/alert: log ts_progress/tc_progress
This is mostly to help with debugging firewall rules, but
can be useful in other places.
2 years ago
Victor Julien 6f5fd77cb9 detect/app-layer-state: keyword for protocol state
Allow matching on the app-layer protocol state.
2 years ago
Victor Julien fff65c5e40 detect/prefilter: allow special engine for each app update
Allow registering the progress as -1, which means it will be invoked
each time the app prefilters are called.
2 years ago
Victor Julien f60e1efc8a detect: start of pkt hooks
New facility to allow a user to specify where to hook a rule
into the engine. This patch adds this for packets, adding two
hooks:

- `all`: to let a rule be evaluated by all rules
- `flow_start`: to have a rule be evaluated only for the first
  packet in both directions

Implemented by adding a hook flags field in the packet.
2 years ago
Victor Julien 95ed975cca detect/tls.version: allow using rule hook
tls.version isn't hooked to a specific state by default. Allow it
to register at the rule hook.
2 years ago
Victor Julien 4071acd659 detect: explicit action scopes
For drop and pass allow setting the scope explicitly:

        drop:flow
        drop:packet

        pass:flow
        pass:packet

Ticket: #7481.
2 years ago
Victor Julien 9539002b39 detect/tls: more precise state registration for keywords 2 years ago
Victor Julien 3485e57953 detect: reuse hook based generic lists 2 years ago
Victor Julien 592444cdc4 detect/lua: use rule hook instead of init() 'needs' table
For registration of app-layer inspection, no longer use the 'needs'
table from the script, but instead use the rule hook setting.

Ticket: #4783.
2 years ago
Victor Julien 67df6ad94e detect: introduce explicit hooks
Generic:
        <app_proto>:request_started and <app_proto>:response_started
        <app_proto>:request_complete and <app_proto>:response_complete

Per protocol, it uses the registered progress (state) values. E.g.

        tls:client_hello_done

A rule ruleset could be:

        pass tls:client_hello_done any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)
        drop tls:client_hello_done any any -> any any (sid:22;)

The pass rule is evaluated when the client hello is parsed, and if it
doesn't match the drop rule will be evaluated.

Registers each generic lists as "<alproto>:<progress state>:generic"
(e.g. "tls:client_hello_done:generic").

Ticket: #7485.
2 years ago
Victor Julien 8c674c8647 http1: register progress state names
Use `request_started` for HTP_REQUEST_NOT_STARTED as when data is
received, the request parsing has started. The request line isn't
complete yet.

Similar for `response_started`.
2 years ago
Victor Julien 6795b4bbb0 tls: expose progress by name 2 years ago
Victor Julien add7d3fcf5 app-layer: API for mapping progress name vs id 2 years ago
Victor Julien 780aeee55c tls: introduce per direction progress tracking
Per direction track progress to be able to have more fine grained
control over where the detection engines and logging hooks in.
2 years ago
Victor Julien b5bc00382d detect: allow longer buffer names
To support hook based buffer names.
2 years ago