Commit Graph

1470 Commits (main)

Author SHA1 Message Date
Philippe Antoine 646c78269a doc/devguide: section with conceptualized steps for adding app-layer
Ticket: 6840
11 months ago
Tommy Wang fc6b96fb85 doc/lualib: fix wrong tuple section markdown in flowlib
Sections had wrong levels due to wrong markdown.
11 months ago
Tommy Wang a10053e62c doc/lualib: fix flow timestamps return value order
Task #7854
11 months ago
Philippe Antoine d0a513df6a detect/integers: support kibibyte unit
Ticket: 7869
11 months ago
Philippe Antoine be9858d3aa detect/integers: document usage of units
Ticket: 7190
11 months ago
Theo Buehler 315844ccd8 docs: fix deprecated inclusion of rtd theme path
Since userguide/conf.py uses the deprecated get_html_theme_path(),
sphinx emits a warning which breaks the build as warnings are treated
as errors.

Issue: 7859
11 months ago
Juliana Fajardini ec1da6fd3a doc/exceptions: fix wrong section markdown
Sections had wrong levels due to wrong markdown.
11 months ago
Jeff Lucovsky 21707ab26c doc/from_base64: Emphasize keyword only values
Emphasize that specifying the keyword only will result in the defaults
for each option to be used.

Issue: 7853
11 months ago
Juliana Fajardini a8453d73cd detect: remove unused non-pf stats counters
Remove unused rule prefilter-related stats counters that aren't in use.

94644ac960 (detect: move non-pf rules into special prefilter engines)
removed the logic that made use of and incremented the stats counters:
- det_ctx->counter_fnonmpm_list
- det_ctx->counter_nonmpm_list

Some code was left, registering them, and mentioning them in the
json schema.

Ticket #7834
11 months ago
Lukas Sismis 897cdb8571 doc/dpdK: update RX/TX descriptor note for Connect-X 4
Ticket: 7639
11 months ago
Jason Ish eaef74af05 lua: document the dnp3 lib
Ticket: #7631
11 months ago
Jason Ish 3b7b908fe2 doc/upgrade: mention that lua rules are enabled by default 11 months ago
Jason Ish 7a65ca10e2 doc/lua-detection: fix example script; remove most buffers
- Reference rule hooks instead

Ticket: #7728
11 months ago
Jason Ish f56bd4db75 doc/lua-output: fix example script for new apis
Ticket: #7728
11 months ago
Jason Ish 7535b5aa1d doc/lua-functions: update lua-function documentation
- cleanup usage and documentation around needs
- mentiond that rule hooks are used instead of "needs" keywords with
  link with rule hooks (which is still in the firewall-design doc)
11 months ago
Jason Ish decf795e94 doc/install: remove reference to --enable-lua
This configure command no longer exists.
11 months ago
Jason Ish 4791f37ca2 doc/lua-detection: update note to mention rules are enabled by default
In 8.0, Lua rules are enabled by default.
11 months ago
Alexandre Iooss 57e0ff0ee0 doc/lua: fix typo in stream toserver and toclient 11 months ago
Jeff Lucovsky 1030e4fa92 doc/suricatasc: Mentioned get-flow-stats-by-id cmd
Add get-flow-stats-by-id to the list of commands supported by suricatasc

Issue: 7081
11 months ago
Jeff Lucovsky 07b7f36748 doc/reload: Expand rule-reload discussion
Clarify the resources involved in a rule reload.

Issue: 5078
1 year ago
Jeff Lucovsky c0d54d838e gen/typo: Misc. typo fixes 1 year ago
Jeff Lucovsky 97b03b4076 doc/netflow: Discuss netflow
Add discussion for netflow configuration, event type and fields
contained in netflow records.

Issue: 5139
1 year ago
Shivani Bhardwaj 58367149cc doc: add upgrade note about change in inspection 1 year ago
Philippe Antoine f4378eb306 doc/devguide: document app-layer protocol detection
Ticket: 6022
1 year ago
Philippe Antoine 4d4eb84eca doc: document krb5 event type
Ticket: 6566
1 year ago
Philippe Antoine f907216e1a doc: do not have bittorrent in the middle of SMB events 1 year ago
Jason Ish 580a4445cd doc/install: use our recommended header order 1 year ago
Jason Ish 6b94689a44 doc/userguide: remove example with CentOS 7
CentOS 7 is EOL.

Ticket: #7749
1 year ago
Jason Ish 8ed506659d doc/userguide: break out package installation
Break out RPM, Debian, and Ubuntu package installation into their own
pages.

Also break out other distributions like "Arch" into an "Other" section
with a note about how those packages are not supported by the OISF.

Ticket: #6252
Ticket: #6069
1 year ago
Jason Ish 1de19ee94c doc/userguide: add appendix item on eve schema
Add some basic documentation on our EVE schema, mainly to show users
that it exists.
1 year ago
Jeff Lucovsky a300df4c4d detect/entropy: Clarify when entropy is logged
Clarify when entropy values are logged and associated with non-alert log
records.
1 year ago
Jason Ish 4a0f278502 doc/install: windows build documentation
Ticket: #5911
1 year ago
Jason Ish f35a56fa65 doc/code-style: add rust; minor cleanups
- Add small section on Rust code, and Rust code exposed to FFI.
- Other minor cleanups.

Ticket: #7078
Ticket: #6955
1 year ago
Jason Ish 249bd32a9d doc/userguide/code-style: update header ordering
To match our recommended header ordering.
1 year ago
Eric Leblond 751f3eef3b doc/userguide: fix some typos 1 year ago
Eric Leblond 6236574b9c doc/userguide: enrichment_key is now context_key 1 year ago
Eric Leblond 20a0575d96 doc/userguide: fix some typos
Suggestions from Juliana.

Co-authored-by: Juliana Fajardini Reichow <jufajardini@gmail.com>
1 year ago
Eric Leblond 40c545f8d9 doc/userguide: jsonline is now standard ndjson 1 year ago
Eric Leblond f724c75cc9 doc/userguide: improve datajson doc 1 year ago
Eric Leblond a652eee508 doc/userguide: remove left over datajson reference 1 year ago
Eric Leblond 7d28758a54 doc/userguide: improve datajson doc
Patch adds ``remove_key`` option and clarifies the text.
1 year ago
Eric Leblond 0ae88a408a doc/userguide: basic doc for jsonline format 1 year ago
Eric Leblond 9873c5d2e1 doc/userguide: add dataset with json 1 year ago
Jason Ish 8e8c3040e7 doc/upgrade: note about dns address swap on responses
Document the change in DNS addresses for ticket 6400.

Ticket: https://redmine.openinfosecfoundation.org/issues/6400
1 year ago
Philippe Antoine b29d46d81f rust: bindgen SCAppLayerParserStateIssetFlag
Ticket: 7667
1 year ago
Victor Julien f2faba5a23 detect/config: add flow tracking doc 1 year ago
Victor Julien ecbcccf355 detect: add tcp.wscale keyword
Allows matching on wscale option value in TCP header options.

Ticket: #7713.
1 year ago
Lukas Sismis 8817a959e8 threading: support thread autopinning and interface-specific affinity
Using the new configuration format, it is now possible to set CPU affinity
settings per interface.

The threading.autopin option has been added to automatically use CPUs from the
same NUMA node as the interface. The autopin option requires
hwloc-devel / hwloc-dev to be installed and --enable-hwloc flag in configure
script.

Ticket: 7036
1 year ago
Lukas Sismis 1a1789eb29 doc: remove title in threading section with no content 1 year ago
Lukas Sismis 6c654e30ac threading: support previous threading configuration format
Provide backward compatibility with the previous configuration
format to allow smooth transition to the new format.
The commit adds docs about the new format and the introduced changes.
1 year ago
Jeff Lucovsky a8a3780276 doc/entropy: Document the entropy log output 1 year ago
Jeff Lucovsky 62e3c02914 doc: Add missing contributors to ack file
Add missing contributors as identified by
    git shortlog -s -n --no-merges -- .
1 year ago
Juliana Fajardini 9614770483 doc/lua: document request_host lua lib
Seems that we missed bringing this one, when documenting HTTP lua lib
functions.
1 year ago
Jason Ish 62df7c8e81 doc/lua: remove reference to removed functions
These have all been replaced by libs and already documented in their
lib format.

Ticket: #7728
1 year ago
Juliana Fajardini f3aa0085a0 pgsql: rename copy in/out response field
We used `copy_column_count`, while just `columns` is more accurate with
what PostgreSQL describes, and what Wireshark shows.

Related to
Task #7644
Task #7645
1 year ago
Juliana Fajardini 2086f99d6b pgsql: add initial support to CopyIn mode/subproto
This sub-protocol inspects messages sent mainly from the frontend to
the backend after a 'COPY FROM STDIN' has been processed by the
backend.

Parses new messages:
- CopyInResponse -- initiates copy-in mode/sub-protocol
- CopyData (In) -- data transfer message, from frontend to backend
- CopyDone -- signals that no more CopyData messages will be seen from
  the frontend, for the current transaction
- CopyFail -- used by the frontend to signal some failure to proceed
  with sending CopyData messages

Task #7645
1 year ago
Jason Ish e5faedf7e6 lua/util: move SCThreadInfo into suricata.util lib
Move the SCThreadInfo global function into the suricata.util library as
thread_info().

This is the last global function to be registered, so remove the
supporting functions.
1 year ago
Jason Ish 778a699622 lua: simplify streaming output setup
Setup the init function to simply return:

   {streaming = "tcp"}

or

   {streaming = "http"}

The returned table can have a lot of parameters that don't make sense
together, this should simplify this one case.
1 year ago
Jason Ish 02bdea2bce lua/streaming: provide streaming buffer as argument
When setting up a Lua output script for streaming data, we're
explicitly requesting stream data. Just pass the streaming data as
arguments, rather than requiring the script to make an extra call to
get the stream data.

The streaming data will be passed in the "stream" field of the args
passed to the log function.

Eliminates the SCStreamingBuffer Lua function.
1 year ago
Jason Ish 11b3ebcb5f lua/bytevar: convert SCByteVar to Lua lib
Similar to flowvars and flowints, but a byte var cannot be registered
from a Lua script, but it still needs to be setup. Instead provide an
"map" function that sets it up, or errors out if the byte var is
unknown.

This also required passing the signature into the Lua init method, as
the state of the Signature object and the time of loading the Lua
keyword is required.
1 year ago
Jason Ish ce7cdd6f9a lua: create suricata.config lua lib
Currently only provides "log_path" as a replacement for SCLogPath.
1 year ago
Juliana Fajardini c5b9277474 doc/payload: fix typo, minor formatting changes 1 year ago
Juliana Fajardini 95560f0966 docs/exceptions: minor improvements
Add section label and doc reference, add another term to Common terms
section.

Tried to also improve readability for the Midstream behavior tables:
- Highlight key-words when differences are only in `do` vs `no`.
- Change order of sentences in certain descriptions, to align with the
  steps those happen for the engine.
1 year ago
Juliana Fajardini 0256ce51eb doc/exceptions: clarify ambiguous terminology
The terms 'inspection' and 'detection' were being used to signify
different engine actions in this document, while throughout the
documentation and code they're many times interchangeable.

Replace 'inspection' with 'parsing' or even 'decoding and parsing' as
more appropriate.

Add a small glossary to clarify what we mean with those terms.
1 year ago
Juliana Fajardini e5c1e5f792 doc/exceptions: standardize page section markdown
Following the discussion on #7396, use those symbols for to keep the
standard.
Remove the repeting `Exception Policies` section header, too.

Related to
Task #7396
1 year ago
Juliana Fajardini 627b8900ef doc/rule-types: fix typo 1 year ago
Jason Ish 278a9c3806 lua: convert log functions to suricata.log lib
Convert the Lua global functions for logging (SCLogInfo, etc) to a Lua
lib names "suricata.log".

Ticket: #7727
1 year ago
Jason Ish 4a655053e8 mdns: add mdns parser, logger and detection
The mDNS support is based heavily on the DNS support, reusing the
existing DNS parser where possible. This meant adding variations on
DNS, as mDNS is a little different. Mainly being that *all* mDNS
traffic is to_server, yet there is still the concept of request and
responses.

Keywords added are:
- mdns.queries.rrname
- mdns.answers.rrname
- mdns.additionals.rrname
- mdns.authorities.rrname
- mdns.response.rrname

They are mostly in-line with the DNS keywords, except
mdns.answers.rdata which is a better than that mdns.response.rrname,
as its actually looking at the rdata, and not rrnames.

mDNS has its own logger that differs from the DNS logger:

- No grouped logging

- In answers/additionals/authorities, the rdata is logged in a field
  that is named after the rdata type. For example, "txt" data is no
  longer logged in the "rdata" field, but instead a "txt" field. We
  currently already did this in DNS for fields that were not a single
  buffer, like SOA, SRV, etc. So this makes things more consistent. And
  gives query like semantics that the "grouped" object was trying to
  provide.

- Types are logged in lower case ("txt" instead of "TXT")

- Flags are logged as an array: "flags": ["aa", "z"]

Ticket: #3952
1 year ago
Juliana Fajardini 404bb53ce9 pgsql: add query keyword
Add the `pgsql.query` rule keyword to match on PGSQL's query
request message contents. This currently matches on the EVE field:

pgsql.request.simple_query

`pgsql.query` is a sticky buffer and can be used as a fast_pattern.

Task #6259
1 year ago
Shivani Bhardwaj bdb8713ffd doc: move upgrade note to correct section
The PR for the behavior change of dealing with spaces in http URI and
protocol was started in 2019 and merged in 2024. When the PR was
created, it belonged to the correct upgrade section, however, by the
time it was merged, it was 8.0.x branch already.
Move it to upgrade notes from 7 to 8 for correctness.
1 year ago
jason taylor ca9b29c2d0 doc: update http.header_names normalization info 1 year ago
Lukas Sismis eb52e337da pcap-file: document capture method options 1 year ago
Lukas Sismis e780a20f82 doc: update available options in the example config 1 year ago
Jeff Lucovsky 1a13244b4b doc/tls: Update Lua TLS functions
Issue: 7608

Update the documentation to reflect the new and expanded functions
available form the Lua TLS library

There are now "server" and "client" versions of most functions. The TLS
object getter is now "get_tx"
1 year ago
Jeff Lucovsky 77139e0cb1 doc/ftp: Document ftp.completion_code sticky buffer
This commit adds documentation for the ftp.completion_code sticky
buffer. This is a multi-buffer match.

Issue: 7507
1 year ago
Jeff Lucovsky 53c8a0f8f1 doc: Document luaxform transform
Issue: 2290
1 year ago
Jeff Lucovsky aec2513799 doc/ftp: Document ftp.reply_received
Add documentation for the ftp.reply_received keyword.
1 year ago
Richard McConnell d81b76d852 output/tls: Allow logging of sv-handshake params
Ticket: 6695

"server_handshake" which logs the following:
1. TLS version used during handshake
2. The chosen cipher suite, excluding GREASE
3. TLS extensions, excluding GREASE
1 year ago
Richard McConnell 94c8be22d4 output/tls: Allow logging of cl-handshake params
Ticket: 6695

Add new custom log fields:

"client_handshake" which logs the following:
1. TLS version used during handshake
2. TLS extensions, excluding GREASE, SNI and ALPN
3. All cipher suites, excluding GREASE
4. All signature algorithms, excluding GREASE

The use-case is for logging TLS handshake parameters in order to survey
them, and so that JA4 hashes can be computed offline (in the case that
they're not already computed for the purposes of rule matching).
1 year ago
Jeff Lucovsky 0b02b1d2d1 doc/ftp: Document ftp.mode keyword
Document the ftp.mode keyword
Fixup a typo in the ftp.reply keyword section.

Issue: 7505
1 year ago
Philippe Antoine 030493c4a8 lua: better doc for ja3 lib
Completes commit 7e78ad944c

Tickt: 7605
1 year ago
Jason Ish c13f85f18d lua: convert file functions to lib suricata.file
This also breaks out the fileinfo function into a method per file info
item. And likewise for state, just return the state and add a new method
for checking if the file is stored.

Ticket: #7491
1 year ago
Juliana Fajardini 62949b3815 pgsql: remove unused "password_message" code
``Password message`` is actually logged just as ``Password``.
Remove related dead code.
1 year ago
Juliana Fajardini 6f81caf8d4 pgsql: clearly indicate redacted password message
If a password message was seen while logging passwords was disabled
for pgsql, this would lead to an empty request being logged.
Instead of simply not logging anything when there is a password message
and this is disabled, however, log instead that said password is
redacted.

Bug #7647
1 year ago
Jason Ish 97eaeef7d8 lua: convert SMTP functions to lib: suricata.smtp
Ticket: #7606
1 year ago
Philippe Antoine 7e78ad944c lua: convert ja3 function into suricata.ja3 lib
Ticket: 7605
1 year ago
Jason Ish 4c695b1075 doc/devguide: document the rate filter callback
Ticket: #7673
1 year ago
Jason Ish f968a7bfb0 doc: add rst header order recommendation
Ticket: #7396
1 year ago
Jason Ish 8102fbfd0c doc/userguide: fix typo in lua flowvar lib
Was importing suricata.flow, not suricata.flowvar.
1 year ago
Jason Ish 8a5bc12156 lua: convert lua flowint functions to lib: suricata.flowintlib
Ticket: #7487
1 year ago
Philippe Antoine 22abad746a lua: convert hassh function into suricata.hassh lib
Ticket: 7603

We use suricata.ssh lib but also enable hassh.
1 year ago
Jason Ish 35b03b4077 lua: convert flowvar functions to lib
New Lua lib, "suricata.flowvar" for working with flowvars from Lua.

Replaces functions:
- SCFlowvarGet (and ScFlowvarGet)
- SCFlowvarSet (and SCFlowvarSet)

Of note, the DetectLuaData has been made available to the init and
thread_init methods, instead of just the match. This is due to an
issue that if a flow variable is not registered in init, it will not
be logged, registering in thread_init is too late.

Ticket: #7486
1 year ago
Philippe Antoine daabab7381 doc/ssh: document hooks
Ticket: 7607
1 year ago
Philippe Antoine 76d7ab5418 lua: convert ssh function into suricata.ssh lib
Ticket: 7607
1 year ago
Jeff Lucovsky ff59f215d6 doc/ftp: Document ftp.dynamic_port keyword
Document the sticky buffer for ftp.dynamic_port
1 year ago
Jason Ish be483dc873 doc/userguide: document that lua dns rules need hooks
And remove the old "keywords" that a lua Rule can register with for
DNS.
1 year ago
Jason Ish b99f254105 lua: add suricata.rule library
Add a "suricata.rule" library for accessing rule information from a
Lua rule, or a Lua output script.

This lib replaces the following global Lua functions:
- SCRuleIds
- SCRuleAction
- SCRuleMsg
- SCRuleClass

Ticket: #7490
1 year ago
Jason Ish a5e662cb8a doc/lua/dns: fix typo 1 year ago
Philippe Antoine 8757ad5fd3 detect/dns: support string for dns.rrtype
Ticket: 6723
1 year ago
Philippe Antoine 44a6f7f8ca detect/dns: support string for dns.rcode
Ticket: 6723
1 year ago
Eric Leblond adfa46ab1c dox/userguide: add tx_cnt documentation 1 year ago
Alice Akaki bda0890834 detect: add email.received keyword
email.received matches on MIME EMAIL Received
This keyword maps to the EVE field email.received[]
It is a sticky buffer
Supports multiple buffer matching
Supports prefiltering

Ticket: #7599
1 year ago
James deb761367d doc: Update bypass docs to use new keyword format
Ticket: #7143

Update documentation to reflect new sticky buffer keyword format
1 year ago
Victor Julien e3c6554ee6 detect/app-layer-protocol: allow matching on 'unknown' 1 year ago
Alice Akaki ca429ef5e3 detect: add email.url keyword
email.url matches on URLs extracted from an email
This keyword maps to the EVE field email.url[]
Supports multiple buffer matching
Supports prefiltering

Ticket: #7597
1 year ago
Jason Ish b23c96198d doc/userguide: type in pfring upgrade notes 1 year ago
Victor Julien 57c73880db lua: enable lua rules by default
Now that sandboxing is in place, lua rule support is enabled by default.
1 year ago
Alice Akaki d4ec5b9765 detect: add ldap.responses.attribute_type
ldap.responses.attribute_type matches on LDAP attribute type/description
This keyword maps the eve field ldap.responses[].search_result_entry.attributes[].type
It is a sticky buffer
Supports multiple buffer matching
Supports prefiltering

Ticket: #7533
1 year ago
Alice Akaki 75fb352bde detect: add ldap.request.attribute_type
ldap.request.attribute_type matches on LDAP attribute type/description
This keyword maps the following eve fields:
ldap.request.search_request.attributes[]
ldap.request.modify_request.changes[].modification.attribute_type
ldap.request.add_request.attributes[].name
ldap.request.compare_request.attribute_value_assertion.description
It is a sticky buffer
Supports multiple buffer matching
Supports prefiltering

Ticket: #7533
1 year ago
Jeff Lucovsky 88c38fc4a0 doc/ftp: Document the ftp.reply keyword
Issue: 7508
1 year ago
Juliana Fajardini 63adfc6b1e userguide/pop3: minor typo and formatting fixing 1 year ago
Alex Savage 778053876b pop3: app-layer parser using sawp-pop3
This module uses the sawp-pop3 crate to parse POP3 requests and responses
Features:
- eve logging
- events for parsable but non-RFC-compliant messages

Ticket: 3243
1 year ago
Eric Leblond ed20e7cfe4 doc/userguide: doc domain and tld transforms 1 year ago
Philippe Antoine 8cb6a4b82b plugin: document app-layer plugins
Ticket: 7149
Ticket: 7150
Ticket: 7153
1 year ago
Victor Julien 5c7173c6df doc/firewall: start of documentation 1 year ago
Victor Julien 6f5fd77cb9 detect/app-layer-state: keyword for protocol state
Allow matching on the app-layer protocol state.
1 year ago
Philippe Antoine 706a0bd99b output: option to add suricata version in eve logs
Ticket: 4853
1 year ago
Jason Ish 2fce106aec pcap-log: add bpf filter for packets that are logged
Add an optional bpf filter to pcap-log. If set, packets must match the
filter to be logged, otherwise they will be ignored.

This allows a user to limit what is logged to disk if they have pcap-log
enabled, but still inspect all data captured.

Ticket: #6832
1 year ago
Lukas Sismis fbed416e26 doc: stylistic and grammar fixes 1 year ago
Dean Balandin 3cdb0ceb7e stream: decouple stream.bypass dependency from tls bypass
Decouple app.protocols.tls.encryption-handling and stream.bypass.
There's no apparent reason why encrypted TLS bypass traffic should
depend on stream bypass, as these are unrelated features.

Ticket: 6788
1 year ago
Lukas Sismis 185123a130 tls: deprecate "default" as a encryption-handling option
The default value for TLS encryption is generic and not precise.
The new keyword, "track-only" substitues the old keyword, being
more on point what action is actually taken when the TLS flow
reaches an encrypted state.
1 year ago
Philippe Antoine 6afad1af51 smb/log: configuration option for types logging
suricata.yaml output section for smb now parses a types list
and will restrict logging of transactions to these types.

By default, everything still gets logged

Remove unused rs_smb_log_json_request on the way

Ticket: 7620
1 year ago
Juliana Fajardini f06ec5336e doc/pgsql: document CopyOut-related response msgs
Related to
Task #4854
1 year ago
Jeff Lucovsky 590c8f6131 doc/ftp: Add ftp.command_data
This commits adds documentation for the ftp.command_data rule keyword
that includes usage examples.
1 year ago
Philippe Antoine 90666c7da1 doc/sdp: fix doc to match real keywords names
Ticket: 7291
1 year ago
Todd Mortimer 9c324b796e http: Use libhtp-rs.
Ticket: #2696

There are a lot of changes here, which are described below.

In general these changes are renaming constants to conform to the
libhtp-rs versions (which are generated by cbindgen); making all htp
types opaque and changing struct->member references to
htp_struct_member() function calls; and a handful of changes to offload
functionality onto libhtp-rs from suricata, such as URI normalization
and transaction cleanup.

Functions introduced to handle opaque htp_tx_t:
- tx->parsed_uri => htp_tx_parsed_uri(tx)
- tx->parsed_uri->path => htp_uri_path(htp_tx_parsed_uri(tx)
- tx->parsed_uri->hostname => htp_uri_hostname(htp_tx_parsed_uri(tx))
- htp_tx_get_user_data() => htp_tx_user_data(tx)
- htp_tx_is_http_2_upgrade(tx) convenience function introduced to detect response status 101
  and “Upgrade: h2c" header.

Functions introduced to handle opaque htp_tx_data_t:
- d->len => htp_tx_data_len()
- d->data => htp_tx_data_data()
- htp_tx_data_tx(data) function to get the htp_tx_t from the htp_tx_data_t
- htp_tx_data_is_empty(data) convenience function introduced to test if the data is empty.

Other changes:

Build libhtp-rs as a crate inside rust. Update autoconf to no longer
use libhtp as an external dependency. Remove HAVE_HTP feature defines
since they are no longer needed.

Make function arguments and return values const where possible

htp_tx_destroy(tx) will now free an incomplete transaction

htp_time_t replaced with standard struct timeval

Callbacks from libhtp now provide the htp_connp_t and the htp_tx_data_t
as separate arguments. This means the connection parser is no longer
fetched from the transaction inside callbacks.

SCHTPGenerateNormalizedUri() functionality moved inside libhtp-rs, which
now provides normalized URI values.
The normalized URI is available with accessor function: htp_tx_normalized_uri()
Configuration settings added to control the behaviour of the URI normalization:
- htp_config_set_normalized_uri_include_all()
- htp_config_set_plusspace_decode()
- htp_config_set_convert_lowercase()
- htp_config_set_double_decode_normalized_query()
- htp_config_set_double_decode_normalized_path()
- htp_config_set_backslash_convert_slashes()
- htp_config_set_bestfit_replacement_byte()
- htp_config_set_convert_lowercase()
- htp_config_set_nul_encoded_terminates()
- htp_config_set_nul_raw_terminates()
- htp_config_set_path_separators_compress()
- htp_config_set_path_separators_decode()
- htp_config_set_u_encoding_decode()
- htp_config_set_url_encoding_invalid_handling()
- htp_config_set_utf8_convert_bestfit()
- htp_config_set_normalized_uri_include_all()
- htp_config_set_plusspace_decode()
Constants related to configuring uri normalization:
- HTP_URL_DECODE_PRESERVE_PERCENT => HTP_URL_ENCODING_HANDLING_PRESERVE_PERCENT
- HTP_URL_DECODE_REMOVE_PERCENT => HTP_URL_ENCODING_HANDLING_REMOVE_PERCENT
- HTP_URL_DECODE_PROCESS_INVALID => HTP_URL_ENCODING_HANDLING_PROCESS_INVALID

htp_config_set_field_limits(soft_limit, hard_limit) changed to
htp_config_set_field_limit(limit) because libhtp didn't implement soft
limits.

libhtp logging API updated to provide HTP_LOG_CODE constants along with
the message. This eliminates the need to perform string matching on
message text to map log messages to HTTP_DECODER_EVENT values, and the
HTP_LOG_CODE values can be used directly. In support of this,
HTP_DECODER_EVENT values are mapped to their corresponding HTP_LOG_CODE
values.

New log events to describe additional anomalies:
HTP_LOG_CODE_REQUEST_TOO_MANY_LZMA_LAYERS
HTP_LOG_CODE_RESPONSE_TOO_MANY_LZMA_LAYERS
HTP_LOG_CODE_PROTOCOL_CONTAINS_EXTRA_DATA
HTP_LOG_CODE_CONTENT_LENGTH_EXTRA_DATA_START
HTP_LOG_CODE_CONTENT_LENGTH_EXTRA_DATA_END
HTP_LOG_CODE_SWITCHING_PROTO_WITH_CONTENT_LENGTH
HTP_LOG_CODE_DEFORMED_EOL
HTP_LOG_CODE_PARSER_STATE_ERROR
HTP_LOG_CODE_MISSING_OUTBOUND_TRANSACTION_DATA
HTP_LOG_CODE_MISSING_INBOUND_TRANSACTION_DATA
HTP_LOG_CODE_ZERO_LENGTH_DATA_CHUNKS
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD_NO_PROTOCOL
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD_INVALID_PROTOCOL
HTP_LOG_CODE_REQUEST_LINE_NO_PROTOCOL
HTP_LOG_CODE_RESPONSE_LINE_INVALID_PROTOCOL
HTP_LOG_CODE_RESPONSE_LINE_INVALID_RESPONSE_STATUS
HTP_LOG_CODE_RESPONSE_BODY_INTERNAL_ERROR
HTP_LOG_CODE_REQUEST_BODY_DATA_CALLBACK_ERROR
HTP_LOG_CODE_RESPONSE_INVALID_EMPTY_NAME
HTP_LOG_CODE_REQUEST_INVALID_EMPTY_NAME
HTP_LOG_CODE_RESPONSE_INVALID_LWS_AFTER_NAME
HTP_LOG_CODE_RESPONSE_HEADER_NAME_NOT_TOKEN
HTP_LOG_CODE_REQUEST_INVALID_LWS_AFTER_NAME
HTP_LOG_CODE_LZMA_DECOMPRESSION_DISABLED
HTP_LOG_CODE_CONNECTION_ALREADY_OPEN
HTP_LOG_CODE_COMPRESSION_BOMB_DOUBLE_LZMA
HTP_LOG_CODE_INVALID_CONTENT_ENCODING
HTP_LOG_CODE_INVALID_GAP
HTP_LOG_CODE_ERROR

The new htp_log API supports consuming log messages more easily than
walking a list and tracking the current offset. Internally, libhtp-rs
now provides log messages as a queue of htp_log_t, which means the
application can simply call htp_conn_next_log() to fetch the next log
message until the queue is empty. Once the application is done with a
log message, they can call htp_log_free() to dispose of it.

Functions supporting htp_log_t:
htp_conn_next_log(conn) - Get the next log message
htp_log_message(log) - To get the text of the message
htp_log_code(log) - To get the HTP_LOG_CODE value
htp_log_free(log) - To free the htp_log_t
1 year ago
Shivani Bhardwaj 9409a5a49a doc: update configuration with flow rate-tracking
Feature 5647
1 year ago
Alice Akaki 341369f203 detect: add email.x_mailer keyword
email.x_mailer matches on MIME EMAIL X-Mailer
This keyword maps to the EVE field email.x_mailer
It is a sticky buffer
Supports prefiltering

Ticket: #7598
1 year ago
Alice Akaki 52e12410ed detect: add email.message_id keyword
email.message_id matches on MIME EMAIL Message-Id
This keyword maps to the EVE field email.message_id
It is a sticky buffer
Supports prefiltering

Ticket: #7593
1 year ago
Philippe Antoine 1851030010 detect: generic callback for md5-like keywords
Ticket: 5634
1 year ago
Giuseppe Longo 330241b162 doc: add sdp sticky buffers 1 year ago
Jason Ish 22b77b0c56 conf: prefix conf API with SC 1 year ago
Jeff Lucovsky ed2a81dc05 doc/entropy: Add documentation for the entropy keyword
This commits adds documentation for the entropy keyword.
The entropy keyword calculates the Shannon entropy value for content
with the calculated value used to determine whether an alert occurs.
1 year ago
Shivani Bhardwaj 65349442f8 doc: add http library supported fns
Task 7604
1 year ago
Shivani Bhardwaj 5ba0ccaf4b doc: remove http cookie ua from list of buffers
as they are available via library using the tx
- tx:request_header("Cookie")
- tx:response_header("Cookie")
- tx:request_header("User-Agent")
1 year ago
Juliana Fajardini d8c6a56a62 doc/exceptions: change stats counters names
As we've changed them for more search-friendly ones.

Related to
Task #7185
1 year ago
Lukas Sismis 4f2ce17dc5 dpdk: allow zero TX queues when running in IDS mode
When running in non-forwarding (IDS) mode, it is not required
to create TX queues for the interface.
This can be acheived by setting tx-descriptors configuration
field to 0.

Ticket: 7633
1 year ago
Lukas Sismis 1be1c65b6e docs: double quote technical terms in DPDK section 1 year ago
Lukas Sismis fbe5ce7a2b dpdk: document vlan stripping offload
Ticket: 5838
1 year ago
Lukas Sismis 640d0985c2 dpdk: check for link up before full startup
ICE card (Intel E810) was not receiving packets immediatelly
after startup, Suricata workers would act as processing while
it was not. This eliminates the problem by only continuing
in the initialization if the link is already up.

The setting can be turned off manually from the configuraiton
file.

Ticket: 7381
1 year ago
Lukas Sismis cb997a64dc dpdk: replace global with per-thread mempools
It turned out that having global (interface-specific) mempool
that is shared by the threads of the interface is slower than
having individual mempools per queue for each interface.

The commit brings this change and should be user-invisible,
the config setting remains still as a number of objects of
all mempools summed (of that interface).

Ticket: 7382
1 year ago
Lukas Sismis 2ef2a9e26f dpdk: auto configure Rx/Tx descriptors and mempool size
Ticket: 7380
Ticket: 7373
1 year ago
Philippe Antoine 3a092f3027 detect: allow rule which need both directions to match
Ticket: 5665

This is done with `alert ip any any => any any`
The => operator means that we will need both directions
1 year ago
Jason Ish 66eb29affd doc/ndpi: move ndpi docs to new plugins section
Moves the nDPI documentation to an nDPI page in the plugins
section. Remove the duplication of installation and setup
documentation.

Includes some minor cleanups.
1 year ago
Alfredo Cardigliano dfd9ef5784 ndpi: initial implementation of nDPI plugin
Ticket: #7231
1 year ago
Alice Akaki ce2e7aed74 detect: add email.date keyword
email.date matches on MIME EMAIL DATE
This keyword maps to the EVE field email.date
It is a sticky buffer
Supports prefiltering

Ticket: #7591
1 year ago
Victor Julien 8c9dfafc6d doc/tls: add more detail on tls.random 1 year ago
Lukas Sismis 7dc65c2f8a hyperscan: add caching mechanism for hyperscan contexts
Cache Hyperscan serialized databases to disk to prevent compilation
of the same databases when Suricata is run again with the same
ruleset.
Hyperscan binary files are stored per rulegroup in the designated
folder, by default in the cached library folder.
Since caching is per signature group heads,
some chunk of the ruleset can change and it still can reuse part of
the unchanged signature groups.

Loading *fresh* ET Open ruleset:  19 seconds
Loading *cached* ET Open ruleset: 07 seconds

Ticket: 7170
1 year ago
Philippe Antoine 879a733c12 doc/http2: explicit behavior for some http keywords
HTTP/2 does not define a way to carry the version or reason phrase
that is included in an HTTP/1.1 status line.

Ticket: 6548
1 year ago
Juliana Fajardini cd69955d7f doc/userguide: add lua flowlib docs
Task #7489
1 year ago
Juliana Fajardini 9480272509 doc: remove old lua flow methods
Task #7489
1 year ago
Alice Akaki 7ba4ebdc2c detect: add email.cc keyword
email.cc matches on MIME EMAIL Carbon Copy
This keyword maps to the EVE field email.cc[]
It is a sticky buffer
Supports prefiltering

Ticket: #7588
1 year ago
Alice Akaki 9e7d23d73f doc: add keywords to the multi-buffer-matching list 1 year ago
Jason Ish 1a47fdfd46 doc/userguide: group af-packet upgrade notes together
Also fix the rendering of the sip nest list.
1 year ago
Jason Ish 080d48ba29 doc/userguide: upgrade note about defrag now off for inline use
Ticket: #7617
1 year ago
Jason Ish 8fe526006d doc/userguide: upgrade note about tpacket-v3 default for ids
Ticket: #4798
1 year ago
Alice Akaki 5d6a072e35 detect: add email.to keyword
email.to matches on MIME EMAIL TO
This keyword maps to the EVE field email.to[]
It is a sticky buffer
Supports prefiltering

Ticket: #7596
1 year ago
Alice Akaki 09db7c7ac1 detect: add mime email.subject keyword
email.subject matches on MIME EMAIL SUBJECT
This keyword maps to the EVE field email.subject
It is a sticky buffer
Supports prefiltering

Ticket: #7595
1 year ago
Jason Ish 374762d202 af-packet: remove use-mmap option
This option is obsolete and was not used in 7.0 as tpacket-v1 support
was removed (see ticket #4796).
1 year ago
Juliana Fajardini a9b2a62ee4 userguide/exceptions: clarify when stats are logged
The stats for exception policies are only logged/ present when any of
the exception policies are enabled (which means any value other than
"auto" or "ignore" in IDS mode, or "ignore" in IPS mode).

This wasn't clearly stated in the docs.
1 year ago
Juliana Fajardini 08e928988f flow/output: log triggered exception policies
To accompany the Exception Policy stats, also add information about any
Exception Policy triggered and for which target to the flow log event.

Task #6215
1 year ago
Jason Ish a6b116bcbe lua: document new suricata.dns lua library
Ticket: #7602
1 year ago
Alice Akaki 90aab0d62f detect: add email.from
email.from matches on MIME EMAIL FROM
This keyword maps to the EVE field email.from
It is a sticky buffer
Supports prefiltering

Ticket: #7592
1 year ago
Joyce Yu dac0d6371e Doc: update eve-json-output ethernet description
Document getting mac addresses from flow when flow timeout.
1 year ago
Shivani Bhardwaj be372ce39d doc: explain priority port setting
Ticket 7329
1 year ago
Shivani Bhardwaj 040c694256 doc: format and align suricata.yaml section 1 year ago
Juliana Fajardini 3985b24e1b upgrade: list inspection recursion default limit
As the yaml indicated before that if no value was specified there were
no limits, and now there will be one.
1 year ago
Juliana Fajardini e1f9e66af0 doc/upgrade: add datasets hash size limit note 1 year ago
Jason Ish c6d18fc871 doc/userguide: af-packet upgrade notes
Add note about increased block size and how to change it back to old
defaults if needed.

Ticket: #7458
1 year ago
Philippe Antoine 32d0bd2bbb detect: limit base64_decode `bytes` to 64KiB
Ticket: 7613

Avoids potential large per-thread memory allocation. A buffer with the
size of the largest decode_base64 buffer size setting would be allocated
per thread. As this was a u32, it could mean a per-thread 4GiB memory
allocation.

64KiB was already the built-in default for cases where bytes size wasn't
specified.
1 year ago
Juliana Fajardini d8523d9d97 userguide/header-keywords: fix typos, adjust format 1 year ago
Juliana Fajardini 28407b2fb8 doc/rule-types: remove trailing underscore
And other minor fixes that were overseen.
1 year ago
Juliana Fajardini 4a8da8c448 userguide/suricatactl: use suricata community page
We were mentioning "Suricata Support" page, which could be a bit
misleading -- and also used a link that is actually redirected to the
Suricata Community page, anyways.
1 year ago
Jason Ish 814e9ffb7a dns: add keywords for additionals and authorities rrnames
Add keywords dns.additionals.rrname and dns.authorities.rrname. Along
the way, consolidate dns.query.name and dns.answer.name into a single file
and register them altogether since there is a lot of common code.
1 year ago
Jason Ish a026293b42 dns: rename dns.response keyword to dns.response.rrname
This is a better name as the keyword is looking at all rrname type
fields in the response.
1 year ago
Nathan Scrivens d3953dee8b doc/userguide: document dns.response
Feature: 7012
1 year ago
Alice Akaki 137f7fe652 detect: add ldap.responses.message
ldap.responses.message matches on LDAPResult error message
This keyword maps the following eve fields:
ldap.responses[].bind_response.message
ldap.responses[].search_result_done.message
ldap.responses[].modify_response.message
ldap.responses[].add_response.message
ldap.responses[].del_response.message
ldap.responses[].mod_dn_response.message
ldap.responses[].compare_response.message
ldap.responses[].extended_response.message
It is a sticky buffer
Supports prefiltering

Ticket: #7532
1 year ago
Alice Akaki 84605db01d detect: add ldap.responses.result_code
ldap.responses.result_code matches on LDAP result code
This keyword maps the following eve fields:
ldap.responses[].bind_response.result_code
ldap.responses[].search_result_done.result_code
ldap.responses[].modify_response.result_code
ldap.responses[].add_response.result_code
ldap.responses[].del_response.result_code
ldap.responses[].mod_dn_response.result_code
ldap.responses[].compare_response.result_code
ldap.responses[].extended_response.result_code
It is an unsigned 32-bit integer
Doesn't support prefiltering

Ticket: #7532
1 year ago
Alice Akaki 599d33c5bf ldap: return empty buffer in ldap_tx_get_responses_dn
Funciton ldap_tx_get_responses_dn returns empty buffer in case
the response doesn't contain the distinguished name field

Fixes: 73ae6e997f ("detect: add ldap.responses.dn")
1 year ago
Alice Akaki 82ca3e667b ldap: fix LDAPDN nits
Change variable name 'req' to 'resp' in function ldap_tx_get_responses_dn and documentation nits

Fixes:
73ae6e997f ("detect: add ldap.responses.dn")
16dcee46fc ("detect: add ldap.request.dn")
1 year ago
Jason Ish f1d305b373 doc: add upgrade note about suricatasc and suricatactl 1 year ago
Jason Ish 1aa47649ca dist: include generate-evedoc.sh
Without this script `make distcheck` fails on a system with
documentation tooling installed, as its required to build the EVE
appendix.
1 year ago
Jason Ish 11a589f633 doc: remove python references related to suricatasc
These should probably be removed even without the rewrite, and
suricatasc has been installed as a proper program for many releases.
1 year ago
Jeff Lucovsky a3a3ad8968 doc/output: EVE output buffering related settings 1 year ago
Jeff Lucovsky dd344bd07c ftp: Move config file handling to Rust
Issue: 4082

Move the configuration file handling to Rust.

These changes will no longer terminate Suricata when there's an invalid
value for ftp.memcap. Like earlier Suricata releases, an error message
is logged "Invalid value <value> for ftp.memcap" but Suricata will no
longer terminate execution. It will use a default value of "0" instead.
1 year ago
Alice Akaki 73ae6e997f detect: add ldap.responses.dn
ldap.responses.dn matches on LDAPDN from responses operations
This keyword maps the following eve fields:
ldap.responses[].search_result_entry.base_object
ldap.responses[].bind_response.matched_dn
ldap.responses[].search_result_done.matched_dn
ldap.responses[].modify_response.matched_dn
ldap.responses[].add_response.matched_dn
ldap.responses[].del_response.matched_dn
ldap.responses[].mod_dn_response.matched_dn
ldap.responses[].compare_response.matched_dn
ldap.responses[].extended_response.matched_dn
It is a sticky buffer
Supports prefiltering

Ticket: #7471
1 year ago
Alice Akaki 16dcee46fc detect: add ldap.request.dn
ldap.request.dn matches on LDAPDN from request operations
This keyword maps the following eve fields:
ldap.request.bind_request.name
ldap.request.add_request.entry
ldap.request.search_request.base_object
ldap.request.modify_request.object
ldap.request.del_request.dn
ldap.request.mod_dn_request.entry
ldap.request.compare_request.entry
It is a sticky buffer
Supports prefiltering

Ticket: #7471
1 year ago
Alice Akaki 8f807fcfcf doc: use the ldap protocol in rule examples in the LDAP keywords documentation 1 year ago
Alice Akaki 31ee18b5be doc: replace 'eve' with 'EVE' in the LDAP keywords documentation 1 year ago
Jason Ish 97e01a8cc8 doc/userguide: upgrade notes for Lua
- Sandboxed Lua for rules
- Search path changes for Lua output scripts
1 year ago
Jason Ish c6d038b8a4 doc/userguide: document Lua base64 library
Ticket: #7074
1 year ago
Victor Julien 38318438d1 doc/userguide: add lua packetlib docs 1 year ago
Victor Julien cdd2f56cfe doc: remove old lua packet methods 1 year ago
AlirezaPourchali 60dd0ec8a5 doc/userguide: fix typo
Issue: #7540

fixed doc/userguide/performance/hyperscan.rst
fixed doc/userguide/performance/runmodes.rst
1 year ago
Alice Akaki 73455179d7 detect/integers: add support for negated strings when enum is used
function detect_parse_uint_enum can parse strings like !bind_request

Ticket: #7513
1 year ago
Jeff Lucovsky beec1eac2f doc/decode-events: new: unknown event description
Issue: 7129

Document the unknown ethertype event.
1 year ago
Jeff Lucovsky e9128e66e6 doc/threshold: Threshold keyword clarifications
Issue: 7129
1 year ago
Jeff Lucovsky cfbf8fda94 doc/csum: Stream checksum validation change
Describe the change of behavior between the stream.checksum-validation
setting and checksum-based rule keywords.
1 year ago
Juliana Fajardini a2905ae5d4 userguide: explain rule types and categorization
Add documentation about the rule types introduced by commit
2696fda041.

Add doc tags around code definitions that are referenced in the docs.

Task #https://redmine.openinfosecfoundation.org/issues/7031
1 year ago
Cole Dishington c46308957f flow: optionally use pkt recursion for hash
If a Suricata inline IPS device is routing traffic over a
non-encrypted tunnel, like IPv6 tunnels, packets in a flow
will be dropped and not be matched. e.g.

The following example is a Suricata inline IPS with an IPv6 tunnel:
request: IPv4]ICMP] -> |IPS| -> IPv6]IPv4]ICMP]
reply:              <- |IPS| <- IPv6]IPv4]ICMP]
Both the IPv4 request and IPv6 reply will be seen by Suricata on
ingress. The flows will not be matched due to flow recursion level.

Optionally use pkt recursion level in flow hash. Excluding recursion
level in flow hash allows matching of packet flows and defrag on an
inline IPS Suricata scenario where the IPS device is a tunnel
terminator.

Feature: 6260
1 year ago
Jeff Lucovsky 53abe1e5d7 doc: Add ftp.command sticky buffer
Issue: 7502

This commit documents the new FTP sticky buffer "ftp.command".
1 year ago
Philippe Antoine c5f3d33e51 detect/smtp: smtp.rcpt_to keyword
Ticket: 7516

It is a sticky buffer mapping to the smtp.rcpt_to[] log field
It is a multi-buffer
1 year ago
Philippe Antoine 32594766b7 detect/smtp: smtp.mail_from keyword
Ticket: 7517

It is a sticky buffer mapping to the smtp.mail_from log field
1 year ago
Philippe Antoine 3d3b1ade9d detect/smtp: smtp.helo keyword
Ticket: 7515

It is a sticky buffer mapping to the smtp.helo log field
1 year ago
Jason Ish c8b28b1512 doc/userguide: document lua hashlib 1 year ago
Alice Akaki 8416289752 detect: add ldap.responses.count
ldap.responses.count matches on the number of LDAP responses
This keyword maps to the eve field len(ldap.responses[])
It is an unsigned 32-bit integer
Doesn't support prefiltering

Ticket: #7453
1 year ago
Alice Akaki da593abd99 detect: add ldap.responses.operation
ldap.responses.operation matches on Lightweight Directory Access Protocol response operations
This keyword maps to the eve field ldap.responses[].operation
It is an unsigned 8-bit integer
Doesn't support prefiltering

Ticket: #7453
1 year ago
Alice Akaki cdb043810f detect: add ldap.request.operation
ldap.request.operation matches on Lightweight Directory Access Protocol request operations
This keyword maps to the eve field ldap.request.operation
It is an unsigned 8-bit integer
Doesn't support prefiltering

Ticket: #7453
1 year ago
Alice Akaki 078c6469a0 detect: add vlan.layers keyword
vlan.layers matches on the number of VLAN layers per packet
It is an unsigned 8-bit integer
Valid range = [0-3]
Supports prefiltering

Ticket: #1065
2 years ago
Alice Akaki b1c2643c87 detect: add vlan.id keyword
vlan.id matches on Virtual Local Area Network IDs
It is an unsigned 16-bit integer
Valid range = [0-4095]
Supports prefiltering

Ticket: #1065
2 years ago
Shivani Bhardwaj ad7ff1c91b flow/pkts: allow matching on either direction
For flow.bytes and flow.pkts keywords, allow matching in either
direction.

Feature 5646
2 years ago
Shivani Bhardwaj 52fd695e5a doc: update syntax for flow.pkts & flow.bytes 2 years ago
Philippe Antoine a499529477 doc: improve documentation about guess-applayer-tx
Ticket: 7199
2 years ago
Shivani Bhardwaj 6f937c7545 doc: add guide for ticket title
Explain with examples what a good ticket title looks like and
why is it important to have ticket titles convey the correct issues.
2 years ago
Jeff Lucovsky 91d5b77316 doc/commit
Describe how to use the git commit template. The template helps ensure
that the information needed for evaluation and context is included in
the commit message.

Ticket: <Redmine ticket number>
2 years ago
Victor Julien d11e8a8ee7 doc/userguide: document TCP urgent policy 2 years ago
Philippe Antoine f426ee3ee2 detect: rename stream_log variables
to better reflect their true meaning
2 years ago
Philippe Antoine f2c3776314 detect: log app-layer metadata in alert with single tx
Ticket: 7199

Uses a config parameter detect.guess-applayer-tx to enable
this behavior (off by default)

This feature is requested for use cases with signatures not
using app-layer keywords but still targetting application
layer transactions, such as pass/drop rule combination,
or lua usage.

This overrides the previous behavior of checking if the signature
has a content match, by checking if there is only one live
transaction, in addition to the config parameter being set.
2 years ago
Juliana Fajardini 6e4a501e7c flowint: add isnotset support
Similar keywords use `isnotset`, while `flowint` only accepted `notset`
Opted to change the code, not only the regex, to keep the underlying
code also following the same patterns.

Task #7426
2 years ago
Jason Ish 289ff25f5b requires: support requires check for keyword
For example:

    requires: keyword foo;

Will require that Suricata supports the "foo" keyword.

Ticket: #7403
2 years ago
Jason Ish 820a3e51b7 requires: treat unknown requires keywords as unmet requirements
For example, "requires: foo bar" is an unknown requirement, however
its not tracked, nor an error as it follows the syntax. Instead,
record these unknown keywords, and fail the requirements check if any
are present.

A future version of Suricata may have new requires keywords, for
example a check for keywords.

Ticket: #7418
2 years ago
Philippe Antoine 4ec90bd227 detect: absent keyword to test absence of sticky buffer
Ticket: 2224

It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.

For multi buffers, absent matches if there are 0 buffers.

For file keywords, absent matches if there is no file.
2 years ago
Victor Julien 278dc24cd0 doc/userguide: document smb cache size limit options
Ticket: #5672.
2 years ago
Sascha Steinbiss 285cc29ec0 redis: add automatic trimming support for streams 2 years ago
Sascha Steinbiss d3d9f1c395 redis: implement XADD stream support
Ticket: #7082
2 years ago
Juliana Fajardini 1860aa81e6 userguide: fix integer keyword matches list format
List wasn't being properly rendered.
2 years ago
Jason Ish cc519beb91 suricata.yaml: add missing custom tls fields
Also update the suricata.yaml in the userguide.
2 years ago
Juliana Fajardini 55b922ceed tls/conf: clarify usage of custom vs extended logs
Since enabling custom logging will replace the extended logging, thus
possibly leading to certain fields disappearing from the logs, mention
this aspect.

Related to
Bug #7333
2 years ago
Jeff Lucovsky 1e0d3435db doc: add napatech plugin upgrade notes
Issue: 7165
2 years ago
Jason Ish 6ae5ae701b doc/userguide: generate eve documentation
Add EVE documentation for QUIC and Pgsql to their respective sections of
the userguide.

Also add a complete EVE reference as an appendix.

Other protocols can be done, but its a manual process to document in the
schema, then add the glue to pull them into the documentation.

The documentation is generated during "make dist", or if it doesn't
exist, "conf.py" will attempt to generate the eve documentation for
building on Readthedocs.
2 years ago
Philippe Antoine bb714c9178 http: have a headers limit
Ticket: 7191

So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
2 years ago
Philippe Antoine e47598110a detect/datasets: implement unset command
Ticket: 7195

Otherwise, Suricata aborted on such a rule
2 years ago
Juliana Fajardini 18e0d23ed3 docs: remove mentions to Suricata-6
Task #7262
2 years ago
Juliana Fajardini d1d1c8cdac doc/conf/yaml: replace underscore with dashes
Use sed + regex to replace all occurrences of suricata.yaml terms that
used underscore for their up-to-date dash version.

Also search for such terms in the eve-log.yaml partials file, as that
is referenced in the configuration section.

commands used:

sed -i 's/\(^ *[a-z]*\)_\([a-z]*:\)/\1-\2/g'
sed -i 's/\(^ *[a-z]*\)_\([a-z]*\)_\([a-z]*:\)/\1-\2-\3/g'

Some other instances were found manually.

Task #7260
2 years ago
Philippe Antoine 7ab833471e doc/rfb: mention accidental fix for security_result log
Ticket: 7198
2 years ago
Giuseppe Longo 036b68b0a9 doc: add new sip keywords 2 years ago
Juliana Fajardini ef63aa50e2 doc/configuration: improve emergency-recovery docs
When removing mentions to `prune-flows` a few inconsistencies for how
we write and refer to `emergency-recovery` were left behind, still.
2 years ago
Philippe Antoine de9413c654 detect: safety for app-layer logging of stream-only rules
If a stream-only rule matches, and we find a tx where we
want to log the app-layer data, store into the tx data that
we already logged, so that we do not log again the app-layer metadata

Ticket: 7085
2 years ago
jason taylor f46a8776ec doc: add note about big endian for icmp_seq match 2 years ago
Juliana Fajardini 1420c83a87 doc/configuration: remove mention to prune-flows
Although the `prune-flows` option was removed with a5587fec2e,
when documentation for the suricata.yaml config file was added with
b252b0d, this option was also included - as has remained until now.
2 years ago
Jeff Lucovsky 8064847fc6 doc: Document reference config setting
Issue: 4974
2 years ago
Philippe Antoine 0ebb84538e http2: add frames support
Ticket: 5743

Why ? To add detection capabilities
2 years ago
Jason Ish 685baa9680 output-filedata: rename and document registration function
Prefix registration function and pointer function type with SC, as
well as document.

Ticket: #7227
2 years ago
Jason Ish b51eeb3ab5 output-file: rename and document registration function
Rename OutputRegisterFileLogger to SCOutputRegisterFileLogger, add
function documentation and include in userguide.

Ticket: #7227
2 years ago
Jason Ish 14b648f286 output-streaming: rename and document registration
Prefix the registration function and types with "SC", and add function
documentation.

Ticket: #7227
2 years ago
Jason Ish 1ebf33b3c9 output-tx: rename and document transaction logger registration
Rename OutputRegisterTxLogger to SCOutputRegisterTxLogger to make it
part of the public API as well as document.

Ticket: #7227
2 years ago
Jason Ish bb128e3959 devguide: more on low level logging
Use the extending/output section to introduce the low level logging
API.

Ticket: #7227
2 years ago
Juliana Fajardini f3e1095244 userguide: update Security Onion docs reference
They have updated their docs domain, leading to the link we had
returning a 404.

Also checked the other links. Although some seem to only contain old
traffic, they all still work.
2 years ago
Sascha Steinbiss cb14e44780 userguide: fix spelling of `security_result` EVE field
This ensures that the correct spelling of the `security_result` EVE
field for RFB (as opposed to `security-result`) is also reflected in the
documentation.

Ticket: #7210
2 years ago
Shivani Bhardwaj 1345c6d1cb doc/file-extraction: fix highlight syntax 2 years ago