Commit Graph

132 Commits (fc23e54c6d2b98421c53a2516dac2d1b36fbb341)

Author SHA1 Message Date
Philippe Antoine c0d6747e82 app-layer: GetTxIteratorFn uses a AppLayerGetTxIterState
Even if every current rust parser uses the u64 variant of the C union
9 months ago
Philippe Antoine c960b7d7c1 app-layer: AppLayerParserFPtr uses a mut pointer to local storage
So, fix the fn prototype in rust ParseFn
9 months ago
Philippe Antoine 1a546936c7 rust: AppLayerTxConfig uses Default impl 9 months ago
Philippe Antoine 4e64de3b3b rust: remove wrapper LoggerFlags struct definition
just use an u32 out of the box.
Will make bindgen like easier
9 months ago
Philippe Antoine f663be5983 rust: define every RustParser fn pointer unsafe
That is what bindgen will do
9 months ago
Philippe Antoine 455b7d1b3b rust: use snake case for applayer_register_protocol_detection 11 months ago
Philippe Antoine b4d8aea0ae rust: bindgen SCAppLayerRegisterProtocolDetection
Ticket: 7662

Changing the struct passed to it to have the minimal number
of useful fields.
11 months ago
Philippe Antoine cc845dccdb rust: bindgen GenericVarFree
Ticket: 7667
1 year ago
Philippe Antoine c52121586a rust: move AppLayerEvents funcs out of RustContext
Ticket: 7667

just use it directly, with bindgen, and cfg not test
1 year ago
Philippe Antoine bf6b8510fe rust: move DetectEngineStateFree out of RustContext
Ticket: 7667

just use it directly, with bindgen, and cfg not test
1 year ago
Philippe Antoine 6fa85d18f0 rust: reenable lint unused_doc_comment
With just local exceptions
1 year ago
Philippe Antoine c391451f19 rust: begin to bindgen app-layer-register.h
Ticket: 7667
1 year ago
Philippe Antoine 6dbc421825 rust: bindgen AppLayerParserConfParserEnabled
Ticket: 7667
1 year ago
Philippe Antoine 49b2a2be5d rust: bindgen SCAppLayerParserRegisterLogger
Ticket: 7667
1 year ago
Philippe Antoine 7bfef2e1e8 rust: bindgen AppLayerParserRegisterParserAcceptableDataDirection
Ticket: 7667
1 year ago
Philippe Antoine 1ff4dbfc24 rust: bindgen AppLayerParserSetStreamDepth
Ticket: 7667
1 year ago
Philippe Antoine b29d46d81f rust: bindgen SCAppLayerParserStateIssetFlag
Ticket: 7667
1 year ago
Philippe Antoine 6d56beffef rust: bindgen AppLayerParserStateSetFlag
Ticket: 7667
1 year ago
Philippe Antoine 7bb0c94ae9 rust: bindgen app-layer-detect-proto.h
Ticket: 7667
1 year ago
Shivani Bhardwaj 2da5bc8039 applayer: add bool status check fns 1 year ago
Philippe Antoine bbc007b4d4 rust: derive for AppLayerState
To enable easily hooks for rust app-layers such as SSH
2 years ago
Victor Julien 31d048ed4b firewall: start of firewall rules support
Config:

Firewall rules are like normal rule, with some key differences.

They are loaded separate, and first, from:

```yaml
firewall-rule-path: /etc/suricata/firewall/
firewall-rule-files:
  - fw.rules
```

Can also be loaded with --firewall-rules-exclusive: Mostly for QA purposes.

Allow -S with --firewall-rules-exclusive, so that firewall and threat detection
rules can be tested together.

Rules:

Differences with regular "threat detection" rules:

1. these rules are evaluated before threat detection rules

2. these rules are evaluated in the order as they appear in the rule file

3. currently only rules specifying an explicit hook at supported

   a. as a consequence, no rules will be treated as (like) IP-only, PD-only or
      DE-only

Require explicit action scope for firewall rules. Default policy is
drop for the firewall tables.

Actions:

New action "accept" is added to allow traffic in the filter tables.

New scope "accept:tx" is added to allow accepting a transaction.

Tables:

Rulesets are per table.

Table processing order: `packet:filter` -> `packet:td` -> `app:*:*` -> `app:td`.

Each of the tables has some unique properties:

`packet:filter`:
- default policy is `drop:packet`
- rules are process in order
- action scopes are explicit
- `drop` or `accept` is immediate
- `accept:hook` continues to `packet:td`

`packet:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `app:*:*` or `alert/action finalize`

`app:*:*`:
- default policy is `drop:flow`
- rules are process in order
- action scopes are explicit
- `drop` is immediate
- `accept` is conditional on possible `drop` from `packet:td`
- `accept:hook` continues to `app:td`, `accept:packet` or `accept:flow`
  continues to `alert/action finalize`

`app:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `alert/action finalize`

Implementation:

During sigorder, split into packet:filter, app:*:* and general td.

Allow fw rules to work when in pass:flow mode. When firewall mode is enabled,
`pass:flow` will not skip the detection engine anymore, but instead
process the firewall rules and then apply the pass before inspecting threat
detect rules.
2 years ago
Victor Julien 10dcc383ae detect/xbits: implement tx bits
Implement `xbits:set,mybit,track tx;` and `xbits:isset,mybit,track tx;`.

Store these in the AppLayerTxData.

Ticket: #6455.
2 years ago
Victor Julien add7d3fcf5 app-layer: API for mapping progress name vs id 2 years ago
Philippe Antoine 3b271b3f5b detect: rename with prefix SCSigTablePreRegister
to make it available to rust via bindgen
2 years ago
Philippe Antoine d24a3eb5f6 snmp: register protocol dynamically
Ticket: 4103
2 years ago
Philippe Antoine e4f71621b7 rust: fix clippy missing_abi warning
warning: extern declarations without an explicit ABI are deprecated
  --> src/core.rs:72:1
   |
72 | extern {
   | ^^^^^^ help: explicitly specify the "C" ABI: `extern "C"`
   |
   = note: `#[warn(missing_abi)]` on by default
2 years ago
Victor Julien 834378ff88 detect: per tx detect flags to a u8 progress value
Reduce per tx space for tracking detection/prefilter progress. Instead
of a per direction u64 of flags, where each bit reflected a progress
value, use a simple u8 to track the linear progression through the
progress values. Use an offset to allow 0 to mean no value.

Add flags field as well to track "skip detect" and "inspect complete".
2 years ago
Victor Julien 1542bcdc67 app-layer: add helper for AppLayerTxData cleanup 2 years ago
Jason Ish 4554c4778d rust: use AppProto from generated bindings instead of duplicating
Have bindgen generate bindings for app-layer-protos.h, then use the
generated definitions of AppProto/AppProtoEnum instead if defining
them ourselves.

This header was chosen as its used by Rust, and its a simple header
with no circular dependencies.

Ticket: #7341
2 years ago
Philippe Antoine b11381d952 rust: AppLayerRegisterParser out of SuricataContext
Just use a regular compile time rust export, instead of having
a runtime definition through the SuricataContext structure
2 years ago
Jason Ish c86b2608aa rust/applayer: clean visibility of export macros
Both the macros export_tx_data_get and export_state_data_get can
generate non-pub functions as the function they generate is only used
as a pointer during registration.

Remove "pub" and "no_mangle" from the generated functions and update
the names of the generated functions to follow Rust rules as they are
no longer exported into the global C namespace.

Ticket: 7498
2 years ago
Jason Ish f62be374ea rust/direction: move direction to own file (cleanup)
Move the implementation of Direction to its own file, direction.rs.
2 years ago
Jason Ish 7ef4caf90e rust/flow: move flow support to its own file (cleanup)
Move the Rust Flow support from core.rs to flow.rs.
2 years ago
Philippe Antoine f426ee3ee2 detect: rename stream_log variables
to better reflect their true meaning
2 years ago
Philippe Antoine b02557ac7d app-layer: track modified/processed txs
To optimize detection, and logging, to avoid going through
all the live transactions when only a few were modified.

Two boolean fields are added to the tx data: updated_tc and ts
The app-layer parsers are now responsible to set these when
needed, and the logging and detection uses them to skip
transactions that were not updated.

There may some more optimization remaining by when we set
both updated_tc and updated_ts in functions returning
a mutable transaction, by checking if all the callers
are called in one direction only (request or response)

Ticket: 7087
2 years ago
Jason Ish ab6dcb3fc1 app-layer: use uint8_t consistent for event IDs
Introduce a common function for mapping names to IDs that performs
bounds checking.

Note: For event IDs in the enum that are larger than a uint8_t, -1
will be returned instead of -4. -4 has special meaning during
signature parsin that means requirements were not met. -4 has no
special handling prior to requirements, or the meaning has been lost.
2 years ago
Jason Ish 45384ef969 rust/applayer: use c_int as return type for get_info_by_id
Rust was using i8 as the return type, while C uses int. As of Rust
1.82, the return value is turned to garbage over the FFI boundary.

Ticket: #7338
2 years ago
Philippe Antoine de9413c654 detect: safety for app-layer logging of stream-only rules
If a stream-only rule matches, and we find a tx where we
want to log the app-layer data, store into the tx data that
we already logged, so that we do not log again the app-layer metadata

Ticket: 7085
2 years ago
Philippe Antoine 304271e63a rust: compatibility with cbindgen 0.27
Ticket: 7206

Cbindgen 0.27 now handles extern blocks as extern "C" blocks.
The way to differentiate them is to use a special comment
before the block.
2 years ago
Philippe Antoine ede77bc4db rfb: move app-layer registration code to rust
Ticket: 7178
2 years ago
Philippe Antoine 1e82e20c65 doh: implement dns over http2 app-proto
Ticket: 5773
2 years ago
Shivani Bhardwaj f2de3e01cb src: remove truncate fn and glue code
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.

However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup

and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.

Bug 7044
2 years ago
Shivani Bhardwaj 80159eb519 applayer: remove truncation logic
as its functionality is already covered by the generic code.
This removes APP_LAYER_PARSER_TRUNC_TC and APP_LAYER_PARSER_TRUNC_TS
flags as well as FlowGetDisruptionFlags sets STREAM_DEPTH flag in case
the respective stream depth was reached. This flag tells that whether
all the open files should be truncated or not.

Bug 7044
2 years ago
Philippe Antoine 5f35035928 filestore: do not try to store a file set to nostore
Ticket: 6390

This can happen with keyword filestore:both,flow
If one direction does not have a signature group with a filestore,
the file is set to nostore on opening, until a signature in
the other direction tries to set it to store.
Subsequent files will be stored in both directions as flow flags
are now set.
2 years ago
Philippe Antoine 82c03f72c3 enip: convert to rust
Ticket: 3958

- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
    enip.product_name, enip.protocol_version, enip.revision,
    enip.identity_status, enip.state, enip.serial, enip.product_code,
    enip.device_type, enip.vendor_id, enip.capabilities,
    enip.cip_attribute, enip.cip_class, enip.cip_instance,
    enip.cip_status, enip.cip_extendedstatus
2 years ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
2 years ago
Philippe Antoine 5dc8dea869 rust: return empty slice without using from_raw_parts
As this triggers rustc 1.78
unsafe precondition(s) violated: slice::from_raw_parts requires
the pointer to be aligned and non-null,
and the total size of the slice not to exceed `isize::MAX`

Ticket: 7013
2 years ago
Philippe Antoine 3643b6ed4b output: generic simple tx json logger
Ticket: 3827
3 years ago
Daniel Olatunji 5c0af0b203 rust/doc: add docstring to rust module files.
Issue: #4584
3 years ago