The configuration of this rule is the same as for ixgbe driver except
the hash function is not RTE_ETH_HASH_FUNCTION_DEFAULT but
RTE_ETH_HASH_FUNCTION_TOEPLITZ.
The syntax in dpdk-testpmd for this rule with attributes:
port index == 0
used rx queue indices == 0 1 2 3
<hash_key> == 6d5a symmetric hash key
is as follows:
"flow create 0 ingress pattern eth / end actions rss types ipv4 ipv6
end queues 0 1 2 3 end key <hash_key> key_len 40 func toeplitz / end"
Ticket: 7337
Move and adjust the base of RSS configuration from util-dpdk-i40e.c to
a new file that can be later utilized by other cards.
RSS configuration can be configured via rte_flow rules. This is useful
for possible future features such as specific header offload
(vxlan, nvgre) also implemented via rte_flow rules, as rte_flow
rules can be chained via groups and priorities.
i40e uses multiple different rte_flow rules to setup RSS. At first,
function DeviceSetRSSFlowQueues() is used to setup rx queues.
This rule matches all types of traffic, so the equivalent
to dpdk-testpmd pattern would be "pattern end"
This rule can not contain hash types (ipv4, ipv6 etc.) nor hash key.
The hash function used here is RTE_ETH_HASH_FUNCTION_DEFAULT.
The syntax in dpdk-testpmd for this rule with attributes:
port index == 0
used rx queue indices == 0 1 2 3
is as follows:
"flow create 0 ingress pattern end actions rss queues 0 1 2 3 end
func default / end"
The other rules configured by i40eDeviceSetRSSFlowIPv4() and
i40eDeviceSetRSSFlowIPv6() match specific type of traffic by l4 protocol
(none, TCP, UDP, SCTP). For example, pattern to match l3 ipv4 with l4
tcp traffic in dpdk-testpmd syntax would be equivalent of
"pattern eth / ipv4 / tcp / end".
These rules can not have rx queues configured, but have hash types
(l3 src and dst address). This means that the traffic distribution
is affected only by l3 addresses, independent of the l4 specifics.
Also these pattern matching rules have symmetric 6d5a
hash key configured. The length of the key is dependent on DPDK version.
The hash function (either RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ or
RTE_ETH_HASH_FUNCTION_TOEPLITZ, depending on DPKD version) used
in these rules hashes symmetricaly due to the symmetric hash key.
The syntax in dpdk-testpmd for rule to match ipv4-tcp traffic with
attributes:
port index == 0
<hash_key> == 52 bytes long 6d5a symmetric hash key
is as follows:
"flow create 0 ingress pattern eth / ipv4 / tcp / end actions rss types
ipv4-tcp l3-src-only l3-dst-only end queues end key <hash_key>
key_len 52 func toeplitz / end"
(queues need to be set to NULL)
Ticket: 7337
Use a single array of built-ins and provide 2 functions for
registering them:
- SCLuaLoadBuiltIn: for loading built-in modules in sandboxed
environments.
- SCLuaRequirefBuiltIns: registers built-in modules with the standard
package tool, allows built-ins to be loaded by output scripts that are
not restricted
I hope to refactor the sandbox so they can use SCLuaRequirefBuiltIns
as well.
This is mainly for header sanitization to avoid pulling in detect
modules into the Lua sandbox definition.
Plus if we namespace modules with names like "suricata.dataset", it
probably makes sense to keep those modules in their own files.
Implement a more compact set of trees specifically for IPv4
and IPv6 addresses. This allows for more compact data structures
and fewer memory allocations.
Based on the existing radix tree implementation.
rust-bindings.h was not being installed with "make install-headers",
and its now pulled in by a header used for plugin support, so make
sure its installed.
We first attempt to install the "dist" version if exists, otherwise
install the "gen" one. Also install the "gen" even if the "dist" one
exists, as its going to be newer.
Adds user registerable callbacks for flow initialization, flow
update and flow finish.
Some plugins, such as other DPI libraries like nDPI need a way to hook
into these flow lifecycle events.
Ticket: #7319
Ticket: #7320
Add a pure rust base64 decoder. This supports 3 modes of operation just
like the C decoder as follows.
1. RFC 2045
2. RFC 4648
3. Strict
One notable change is that "strict" mode is carried out by the rust
base64 crate instead of native Rust. This crate was already used for
encoding in a few places like datasets of string type. As a part of this
mode, now, only the strings that can be reliably converted back are
decoded.
The decoder fn is available to C via FFI.
Bug 6280
Ticket 7065
Ticket 7058
Ticket: 4863
On the way, convert some keywords to use the first-class integer
support.
And helpers for pure rust the support for multi-buffer.
Move the C unit tests about keyword mqtt.protocol_version
to unit tests for generic integer parsing, and test version 5
instead of testing twice version 3.
Also iterate all tx's messages for reason code as is done for other
keywords.
And allow detection on empty topics.
Issue: 6487
Implement the from_base64 transform:
[bytes value] [offset value] [mode strict|rfc4648|rfc2045]
The value for bytes and offset may be a byte_ variable or an
unsigned integer.
Ticket: 4863
On the way, convert unit test DetectSNMPCommunityTest to a SV test.
And also, make snmp.pdu_type use a generic uint32 for detection,
allowing operators, instead of just equality.
Ticket: 3958
- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
enip.product_name, enip.protocol_version, enip.revision,
enip.identity_status, enip.state, enip.serial, enip.product_code,
enip.device_type, enip.vendor_id, enip.capabilities,
enip.cip_attribute, enip.cip_class, enip.cip_instance,
enip.cip_status, enip.cip_extendedstatus
Decode file needed ExceptionPolicy types and exception-policy file
needed Decode types, rendering some works quite difficult to work
around.
ExceptionPolicyToStr is useful for registering exception policy
counters, so make that public.
Part of
Task #5816
One of the lessons of the XZ backdoor story was that just linking to
libsystemd to call sd_notify is discouraged by the systemd project:
Lennart Poettering:
"PSA: In context of the xzpocalypse we now added an example reimplementation
of sd_notify() to our man page:
https://www.freedesktop.org/software/systemd/man/devel/sd_notify.html#Notes
It's pretty comprehensive (i.e. uses it for reload notification too), but
still relatively short.
In the past, I have been telling anyone who wanted to listen that if all you
want is sd_notify() then don't bother linking to libsystemd, since the
protocol is stable and should be considered the API, not our C wrapper
around it. After all, the protocol is so trivial"
From: https://mastodon.social/@pid_eins/112202687764571433
This commit takes the example code and uses it to reimplement the notify
logic.
The code is enabled if Linux is detected in configure. Since the code
won't do anything if the NOTIFY_SOCKET env var isn't set, this should
also work fine on systems w/o systemd.
Ticket: #6913.
Remove EVE filetypes from plugin context as they are not only used
from plugins. Plugins allow user code to register filetypes, but we
also have internal file types that use this api including the null
output and syslog. Additionally library users can use this API to
register filetypes, and they are not plugins.
Ideally this code would go in "output-json.[ch]" as the "primary" eve
API, however there are currently some include circular include issues
there, so start new cleaned up EVE API in "output-eve.[ch]" which is
"clean" with respect to includes, and as we cleanup existing EVE API for
"public" use, it can be moved here.
Ticket: #6838
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions
Task #6621
Add new utility files to deal with the interval trees. These cover the
basic ops:
1. Creation/Destruction of the tree
2. Creation/Destruction of the nodes
It also adds the support for finding overlaps for a given set of ports.
This function is used by the detection engine is the Stage 2 of
signature preparation.
Ticket 6792
Bug 6414
Co-authored-by: Victor Julien <vjulien@oisf.net>
An interval tree uses red-black tree as its base data structure and
follows all the properties of a usual red-black tree. The additional
params are:
1. An interval such as [low, high] per node.
2. A max attribute per node. This attribute stores the maximum high
value of any subtree rooted at this node.
At any point in time, an inorder traversal of an interval tree should
give the port ranges sorted by the low key in ascending order.
This commit modifies the IRB_AUGMENT macro and it's call sites to make
sure that on every insertion, the max attribute of the tree is properly
updated.
Ticket 6792
Bug 6414
It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666
We were including "autoconf.h" in our dist releases, however this file
is generated by ./configure. This can be problematic in out of tree
builds as the one included in the dist, not the generated one may be
used.
Also using "autoconf.h" instead of <autoconf.h> so it is searched for
relative to the file including it which is necessary if a library/plugin
does something like "#include "suricata/foo.h".
Previous integration of hugepage analysis only fetched data
from /proc/meminfo. However this proved to be often
deceiving mainly for providing only global information and
not taking into account different hugepage sizes (e.g. 1GB
hugepages) and different NUMA nodes.
Ticket: #6419
Add a new rule keyword "requires" that allows a rule to require specific
Suricata versions and/or Suricata features to be enabled.
Example:
requires: feature geoip, version >= 7.0.0, version < 8;
requires: version >= 7.0.3 < 8
requires: version >= 7.0.3 < 8 | >= 8.0.3
Feature: #5972
Co-authored-by: Philippe Antoine <pantoine@oisf.net>
This sticky buffer will allow content matching on the answer names.
While ansers typically only occur in DNS responses, we allow the buffer
to be used in request context as well as the request message format
allows it.
Feature: #6496
This commit adds the null output device; to use, set the filetype
to "nullsink" for each output that should discard and never persist
logs/alerts/etc.
This is implemented as an "internal eve output plugin" just like the
syslog eve output type.
Debug facility to get a per packet view into the stream engine's state.
Logs after a packet has been processed in the stream engine, so the view
into the state includes the updates based on the current packet.
Marked as experimental so it can be changed w/o notice.
Bug: #5876.
Remove the app-layer-PROTO stub for Rust based parsers. It is no longer
needed as Rust parsers now contain the registration function in Rust.
Ticket: 4939
AF_XDP support is a recent technology introduced that aims at improving
capture performance. With this update, Suricata now provides a new
capture source 'af-xdp' that attaches an eBPF program to the network
interface card. Packets received in the NIC queue are forwarded to
a RX ring in user-space, bypassing the Linux network stack.
Note, there is a configuration option (force-xdp-mode) that forces the
packet through the normal Linux network stack.
libxdp and libbpf is required for this feature and is compile time
configured.
This capture source operates on single and multi-queue NIC's via
suricata.yaml. Here, various features can be enabled, disabled
or edited as required by the use case.
This feature currently only supports receiving packets via AF_XDP,
no TX support has been developed.
Ticket: https://redmine.openinfosecfoundation.org/issues/3306
Additional reading:
https://www.kernel.org/doc/html/latest/networking/af_xdp.html
This patch introduce the IPv4 type for dataset so Suricata commandmatch
on a set of IPv4 addresses. This is meant to complement iprep feature
for people that needs more flexibility such as settings the IP on
the packet path.
Feature: #5383
Add thread local cache to avoid locking overhead for ssns and segments.
A thread will return segments/ssns to a local cache first, and if that
is full, to a return queue where the actual return to the pool returns
a batch, to amortize locking overhead.
Adds segment and session pool/cache counters to see where how effective
the cache is.
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
This patch is adding support for Landlock, a Linux
Security Module available since Linux 5.13.
The concept is to prevent any file operation on directories where
Suricata is not supposed to access.
Landlock support is built by default if the header is present. The
feature is disabled by default and need to be activated in the YAML
to be active.
Landlock documentation: https://docs.kernel.org/userspace-api/landlock.html
Feature: #5479
Add tls.random keyword that matches on the 32 bytes of the TLS
random field for client as well as server.
Add tls.random_time keyword that matches on the first 4 bytes of the TLS
random field for client as well as server.
Add tls.random_bytes keyword that matches on the last 28 bytes of the TLS
random field for client as well as server.
All these are sticky buffers.
Feature 5190
Victor Julien
Adam Kiripolsky
Jeff Lucovsky
Philippe Antoine
Jason Ish
Alice Akaki
Shivani Bhardwaj
Giuseppe Longo
mmaatuq
Philippe Antoine
Jo Johnson
Sascha Steinbiss
Juliana Fajardini
Hadiqa Alamdar Bukhari
Arne Welzel
Eloy Pérez González
Lukas Sismis
Richard McConnell
Aaron Bungay
Eric Leblond