Commit Graph

1470 Commits (88d2f67a3b5cae367cb69c6aec3d9cb77e46dadd)

Author SHA1 Message Date
Philippe Antoine 54c02e1301 stats: replace dashes by underscores in app-layer protocols
Ticket: 6502

Forbid dashes in json keys for better use by processing tools
3 months ago
Philippe Antoine d030ea7f29 output: rename reject-target to reject_target
Ticket: 6502

Forbid dashes in json keys for better use by processing tools
3 months ago
Jeff Lucovsky 1721ba1ba5 doc/subslice: Document the subslice transform
Add documentation for the subslice transform.

Issue: 7672
3 months ago
Philippe Antoine 64f003190d doc: move more rules to dedicated css container
Ticket: 8372

Also remove dead code from script checking the rules
3 months ago
Philippe Antoine 5f9e436c3f detect/dcerpc: support generic integer for opnum keyword
Ticket: 8179
4 months ago
Jason Ish 713e4eb900 ntp: convert reference_id to buffer and add keyword
Store the NTP reference ID as raw network-order bytes so it can be
exposed as a sticky buffer and matched with payload keywords. The
reference ID is often a 4 character string, or an IP address and not
just an integer identifier.

Updates the log reference ID to be a string of colon separated hex
digits as this matches what tshark does.

Ticket: #8488
4 months ago
Jason Ish 991e7f3b1d ntp: add ntp.mode keyword
This keyword also accepts strings for known mode names.

Ticket: #8429
4 months ago
Jason Ish c10c482290 ntp: add ntp.stratum keyword
Ticket: #8431
4 months ago
Jason Ish ec344fe68d ntp: add ntp.version keyword
SNMP was used as a template.

Ticket: #8430
4 months ago
Jason Ish 31b967b089 ntp: add transaction logging
Adds basic NTP transaction logging for the current supported message
types.

Includes small cleanups around reference ID.

Ticket: #8425
4 months ago
Victor Julien 670fdabd32 detect/snmp: add snmp.trap_type keyword
Implemented as a U8 integer keyword.

Ticket: #8482.
4 months ago
Jason Ish eb46d0129e rust/ffi: add eve callback handler
Wrap the EVE callback handler with a Rust friendly variant that allows
the user to register a callback as a closure which is provided an
already wrapped JsonBuilder object.

Ticket: #8477
4 months ago
Philippe Antoine bcbec8d615 doc: fix eol content in http rules 4 months ago
Philippe Antoine e65643909a doc: move more examples to container:: example-rule
Ticket: 8372

And fix another bad rule
4 months ago
Philippe Antoine 72e13c9774 doc: do not highlight bad transactional rule
As stated, it will refuse to load.

Ticket: 8372
4 months ago
Philippe Antoine 15f45be672 doc: fix rules
Ticket: 8372

Somes rules in the doc had typos, fix them so suricata can load
them when you copy/paste the doc
4 months ago
Jason Ish a11aaadd86 plugins: add --plugin command line option to load plugins
Add --plugin <PATH> to load an additional plugin from the command
line. This is more convenient than "--set plugins.X" especially when
you may already have a plugins loaded and you want to load an
additional one.

Ticket: 8463
4 months ago
Victor Julien 6c3169cee0 doc/userguide: add ether and arp to intro 4 months ago
Victor Julien 6298c47145 doc/userguide: improve protocol docs 4 months ago
Victor Julien e6381a3c22 doc/userguide: add note on rule reloads
Cannot be combined with --firewall-rules-exclusive
4 months ago
Victor Julien f99b86beab doc/userguide: document L2 firewall handling of ARP 4 months ago
Victor Julien 2e2132a16f doc/userguide: improve pkthdr docs 4 months ago
Victor Julien 61a7f47a69 detect: add ether.hdr keyword
Sticky buffer to inspect the ethernet header.

Example rule:

        alert ether any any -> any any ( \
                ether.hdr; content:"|08 06|"; offset:12; depth:2; \
                sid:1;)

Ticket: #8327.
4 months ago
Jeff Lucovsky e7dc0d885b doc/config: Update flushing description
Update output flushing description to reflect EVE based approach in
documentation and config template.

Issue: 8286
5 months ago
Juliana Fajardini c790a9f774 userguide/exceptions: update for qa-simulation use
The docs still referred to ``debug`` mode.

Related to
Task #7885
5 months ago
Juliana Fajardini 56917cf8f3 rules: fix typos & missing keywords in examples
Task #8372
5 months ago
Juliana Fajardini 8c29c471b4 userguide: add references to rules/README to docs
To make it more visible/easier to find.
5 months ago
Jason Ish 8c7172e707 lib: remove the built-in library run-mode
This run-mode does not allow for capture ThreadVars to be created at
the appropriate place in the application life cycle. Instead,
applications should register their own custom run-mode.

Ticket: #8259
5 months ago
Philippe Antoine 1e0f344ab7 doc: be more precise for decompress transforms
Ticket: 7846
5 months ago
Jeff Lucovsky 5a3a2d5fd1 doc/output: Document the IPv6 display behavior
Issue: 7399

Document the IPv6 display behavior and how to display IPv6 addresses in
their compressed form (per RFC-5952).
5 months ago
Shivani Bhardwaj 0e345c93ca app-layer: check ipproto enabled setting first
If the key `app-layer.protocols.PROTO.enabled` is present, the protocol
is enabled for all carrier protocols. This is not ideal.
Only if the key `app-layer.protocols.PROTO.enabled` is missing, an
attempt is made to look for a setting specific to the ipproto passed
at the time of registration e.g.
`app-layer.protocols.PROTO.udp.enabled`.

By default, check for carrier proto specific setting. If it is not
found, then fall back to the generic setting.
Issue a warning in case an inconsistent combination of global and
ipproto specific setting is found.

Bug 8205
5 months ago
Victor Julien 2480f8c038 doc: fix dcerpc.opnum documentation
Docs were claiming greater than, less than, etc. support even though this was never supported.

Fixes: 51be8f0238 ("doc/dcerpc: add proto keywords")

Ticket: #8330.
5 months ago
Victor Julien fe3cce895c doc/userguide: add igmp keyword docs 5 months ago
Victor Julien 7e6fd09e90 doc/userguide: add missing ipopts values 5 months ago
Juliana Fajardini 90823fa90e devguide: add policy on patches w unasigned ticket 5 months ago
Jason Ish 66473f6ad7 doc: add more public datasets
Add:
- UNSW-NB15
- CIC-IDS2017
- CSE-CIC-IDS2018
- CIC-DDoS2019

Update formatting for better display.
5 months ago
Juliana Fajardini 926fde859e userguide: fix Deprecations, warn about syslog
The Deprecation note on Syslog was partly conveying the opposite
message, and we were missing a deprecation notice on the Syslog output
section.
6 months ago
Philippe Antoine d5ea973791 detect/ike: move ike.ike.chosen_sa_attribute keyword to rust
Ticket: 8310

And increases expressivity on the way by supporting other modes
than equality
6 months ago
Sven Cuyt 1246f1cf8c doc/userguide: add link to differences-from-snort
Ticket: #8031.
6 months ago
Sven Cuyt a0cc534471 doc/userguide: fix table showing example payload
Ticket: #8031.
6 months ago
Sven Cuyt 2d662538e5 doc/userguide: add example with non-zero offset for relative isdataat
Ticket: #8031.
6 months ago
Victor Julien 7c5285e5e6 doc/userguide: explain isdataat absolute vs relative difference
Ticket: #8031.
6 months ago
Philippe Antoine d378610e93 stream: non infinite default value for reassembly.depth
So that empty config have a safer default.

Ticket: 8290
6 months ago
Ofer Dagan e1f9d8a067 pcap: refactor delete-when-done to support non-alerts
Refactor pcap file deletion to use a single delete-when-done option
with three values instead of separate boolean options:
- false (default): No deletion
- true: Always delete files
- "non-alerts": Delete only files with no alerts

Also account for alerts produced by pseudo packets (flow timeout / shutdown flush):
- Introduce small capture hooks and invoke on pseudo-packet creation so the
  capture layer can retain references and observe alerts emitted after the last
  live packet
- Call the hook from both TmThreadDisableReceiveThreads and TmThreadDrainPacketThreads

Key changes:
- Replace should_delete/delete_non_alerts_only bools with enum
- Move alert counter from global to per-file PcapFileFileVars
- Relocate alert counting from PacketAlertFinalize to pcap module
- Ensure thread safety for both single and continuous pcap modes
- Add unit tests for configuration parsing and pseudo-packet alert path

The --pcap-file-delete command line option overrides YAML config
and forces "always delete" mode for backward compatibility.

Documentation updated to reflect the new three-value configuration.

Fixes OISF#7786
6 months ago
Philippe Antoine 539e4ee665 detect/transforms: add zlib_deflate transform
Ticket: 7846
6 months ago
Philippe Antoine dbea660729 detect/transforms: add gunzip transform
Ticket: 7846
6 months ago
Jason Ish 1b182025bc output-eve: expose SCEveFileType to Rust with bindgen
There is an unfortunate side-affect that one has to read
output-eve-bindgen.h for the documentation on this type, however, I
think we can resolve that in time.
6 months ago
Jason Ish 2defd5ae5a userguide/Makefile: don't add "install" to EXTRA_DIST
Adding the directory "install" to EXTRA_DIST, actually triggers make
to run "make install", which is not what we want. Instead, avoid this
magic keyword and list the files in the install directory
individually.

If the user doesn't have permission to install files to the prefix,
like "/usr", then "make dist" can fail. Worse, even they do have
permission to write into the prefix, a "make dist" will install files
there when it shouldn't.

Ticket: #8279
6 months ago
Philippe Antoine a079d1c68a src: remove unused enum values 6 months ago
Philippe Antoine 06f78b2a22 rust: move StreamSlice definition to C
and bindgen it to rust

Will make easier the bindgen of RustParser structure which uses
a callback which uses StreamSlice
6 months ago
Andreas Dolp 69eb567dea doc/userguide: fix within-distance pointer graphics in payload-keywords doc
Redmine ticket: #8261

According to [1], the within pointer (if combined with distance)
includes the distance pointer, which is not clearly visible in the
graphic.

Fixed this in a new graphic by some GIMP arts.

PS: Special thanks to one of our team members Annika C. for initially
spotting this!

[1] https://forum.suricata.io/t/is-within-affected-by-distance/1688
6 months ago
Mingyu Jeon c98112eb67 doc: update tls_cert_notafter/before
refs #3065

* add explanation on omitted values
6 months ago
Jeff Lucovsky 45a36e961f doc/byte_jump: Clarify bitmask operation
Issue: 6693

Clarify how the bitmask value is used for byte_jump

Snort compatibility says:
- The bitmask value is applied to the extracted value before the
  multiplier is applied.
- The result of the bitmask operation is to be right shifted by the
  number of trailing 0's in the bitmask value.
6 months ago
Fupeng Zhao f7ec1c8813 redis: Add authentication support
Add authentication support to the Redis logging output.
It introduces `username` and `password` configuration options for Redis,
allowing Suricata to authenticate with Redis servers that require it.

Ticket: 7062
6 months ago
Ofer Dagan 7627756360 detect/detection_filter: add unique_on option
Add optional unique_on {src_port|dst_port} to detection_filter for
exact distinct port counting within the seconds window.

Features:
- Runtime uses a single 64k-bit (8192 bytes) union bitmap per
  threshold entry with O(1) updates.
- Follows detection_filter semantics: alerting starts after the
  threshold (> count), not at it.
- On window expiry, the window is reset and the current packet's
  port is recorded as the first distinct of the new window.

Validation:
- unique_on requires a ported transport protocol; reject rules
  that are not tcp/udp/sctp or that use ip (protocol any).

Memory management:
- Bitmap memory is bounded by detect.thresholds.memcap.
- New counters: bitmap_memuse and bitmap_alloc_fail.

Tests:
- C unit tests for parsing, distinct counting, window reset, and
  allocation failure fallback.
- suricata-verify tests for distinct src/dst port counting.

Task #7928
6 months ago
Juliana Fajardini 2c216e2b73 devguide/exceptions: minor fixes
- cleaner YouTube link
- fix weird sentence

Follow-up of
Task #5612
7 months ago
Philippe Antoine 54cc752cf9 ssh: hassh can be truly disabled
Rules will not re-enable it

Ticket: 8223
7 months ago
Philippe Antoine c960b7d7c1 app-layer: AppLayerParserFPtr uses a mut pointer to local storage
So, fix the fn prototype in rust ParseFn
7 months ago
Juliana Fajardini 48b15ac152 devguide: add chapter about exception policies
Adds a chapter indicating what are the main steps when adding exception
policies, how is it possible to extend them, as well as main aspects and
files to consider when doing so.

Task #5612
7 months ago
Lukas Sismis 56c1552c3e hs: warn about the same cache directory
This is especially relevant for multi-instance simultaneous setups
as we might risk read/write races.
7 months ago
Lukas Sismis 15c83be61a hs: prune stale MPM cache files
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.

Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).

Ticket: 7830
7 months ago
Victor Julien 73a873ecfe doc/userguide: endswith can be mixed with offset/distance/within
Bug: #5030.
7 months ago
Philippe Antoine dd6baccabd src: doc: remove more double-space typos
Found with git grep -E '[a-z]  [a-z]'
7 months ago
Juliana Fajardini a1f088cc65 docs: use correct suricata version mentions
Found with 'git grep suricata-7'

Task #8113
7 months ago
Philippe Antoine 7422c97bf2 quic: encryption-handling config parameters
Ticket: 8116
7 months ago
Shivani Bhardwaj 6702791a9c doc: add dcerpc.max-stub-size config param 7 months ago
Jason Ish 2c95f1ff44 dnp3: bound the maximum number of objects per tx
Default to 2048, but provide a user configuration value.

Ticket: #8181
7 months ago
Jason Ish 3a32bb5743 dnp3: set a bound on the number of points per message
16384 is used as the max, but a configuration parameter has been
provided. The reason for setting an upper bound is that bit flags can
create a memory amplification as we parse them into individual data
structures.

Ticket: #8181
7 months ago
Jason Ish a16f087b93 dnp3: reduce flood threshold to 32 and make configurable
Lower the number of unreplied requests from 500 to 32 to consider a
flood. At the very least this is an anomaly given the DNP3 spec mentions
that DNP3 should only have one outstanding request at a time, with an
exception for unsolicited responses, so in practice no more than 2
should be seen.

Additionally make this value configurable by introducing the max-tx
parameter.

Ticket: #8181
7 months ago
Lukas Sismis b596073a79 src: doc: remove double-space typos 7 months ago
Shivani Bhardwaj 30bdaa44e1 doc: add elephant.flow usage section 8 months ago
Shivani Bhardwaj 06eafb79d6 detect/flow-pkts: check for both dir
The functionality of "both" can already be easily achieved by using both
"toclient" and "toserver" in a rule. This just adds the ease of
expression in rule. As it is added now, check the direction for the
pre-existing users of direction as well.
8 months ago
Juliana Fajardini 0ca874b678 userguide: highlight exceptions interactions
In corner cases, we assume that a midstream exception policy could be
triggered by a prior exception policy in effect. Explain this in the
docs.

Task #5830
8 months ago
Ambre Iooss 91c9e34419 doc/userguide: update windivert instructions
The IPS documentation for Windows had a dead link
to a PDF. Update the Windows build page to include
WinDivert instructions.

Documentation: #8137.
8 months ago
Victor Julien 365ed2a2ff af-packet: disable hwtimestamp by default
Add enable-hwtimestamp option to replace disable-hwtimestamp.

Ticket: #7585.
8 months ago
Jeff Lucovsky 845544aad3 doc/luaxform: Clarify luaxform calling convention
Issue: 8135

Clarify the calling convention for the Lua transform's `transform`
function.
8 months ago
Juliana Fajardini 6c06ab6144 devguide: update backports policy for Suricata 7.0
Also remove mentions to `master` and `6.0x`.

Task #7937
8 months ago
Juliana Fajardini 907f4faff8 doc/thresholding: minor fix for backoff subsection 8 months ago
Philippe Antoine c93e69830a detect/ssl: properly handle negation in ssl_version keyword
Ticket: 3220

DetectSslVersionMatch did not handle properly negation.
It could never match on a signatrue with ssl_version: !tls1.3
That is because, if we had such a signature and network traffic
with tls1.1, we were looking into DetectSslVersionData field
for tls1.1, which was not set, instead of looking at field
for tls1.3 which was set with negated flag.

Previous DetectSslVersionData was holding redundant information.
It did not need to have it for each ssl version, but just globally.
Also, it did not need to hold the version as a value in the array,
as it was redundant with the index of the array.
8 months ago
Philippe Antoine 1329786f84 detect: new command line option : list-rule-protos
To list the protocols we can use a in a rule header

Ticket: 635
8 months ago
Victor Julien 08d625bb10 mpm: remove remaining ac-bs references 8 months ago
Victor Julien be2c40bde7 doc/af-packet: document disable-hwtimestamp option
Ticket: #1954.
8 months ago
Victor Julien 5194c877f2 threads: remove thread group name logic
Once used to merge stats. Stats are tracked per thread and merge by counter
name in the current code.

It was set only for autofp modes.
8 months ago
Juliana Fajardini 1647081b29 doc: remove remaining references to pruned flows
These were removed with a5587fec2e but these mentions went under the
radar.
9 months ago
Philippe Antoine af8390f2ca doc: upgrade note for engine-analysis breaking change
For keywords that now use the generic integer framework

Ticket: 7889
9 months ago
Philippe Antoine d94aea8331 detect: tcp.window is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine 68bb783e97 detect: tcp.seq is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine ed907bbd78 detect: tcp.ack is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine 7a8eb28cae detect: id is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine bff7413488 detect: icmp_seq is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine 79e66f76e6 detect: icmp_id is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine 09531447d1 detect: fragoffset is now a generic integer
Ticket: 7889
9 months ago
Philippe Antoine 85fa894425 detect: dnp3.func is now a generic integer
Ticket: 7889
9 months ago
Jeff Lucovsky b02d9bb4f1 doc/luaxform: Remove init function from example
Issue: 8035

The `luaxform` transform doesn't support the `init` function. This
commit removes that from the example and clarifies how functions in the
Lua script are used.
9 months ago
Victor Julien 558ab73a7f doc/devguide: explain alert logging changes 9 months ago
Philippe Antoine c61f1cb6d0 detect/integers: rename index all1 to all
And all to all_or_absent

Ticket: 7929
9 months ago
Victor Julien ec65fd430e exception-policy: rename 'reject-both' to 'rejectboth'
To align it with the rule action.
9 months ago
Jason Ish cdd4ea0f11 doc/devguide: document eve callback
Document the callback for adding additional data to EVE.

Ticket: #4708
9 months ago
Jason Ish 9fffc09ad7 doc/devguide: document eve file types
Ticket: #4708
9 months ago
Victor Julien 0c4a8fd183 doc/userguide: document reject-both expection policy
Ticket: #5974.
9 months ago