Remove the app-layer-PROTO stub for Rust based parsers. It is no longer
needed as Rust parsers now contain the registration function in Rust.
Ticket: 4939
AF_XDP support is a recent technology introduced that aims at improving
capture performance. With this update, Suricata now provides a new
capture source 'af-xdp' that attaches an eBPF program to the network
interface card. Packets received in the NIC queue are forwarded to
a RX ring in user-space, bypassing the Linux network stack.
Note, there is a configuration option (force-xdp-mode) that forces the
packet through the normal Linux network stack.
libxdp and libbpf is required for this feature and is compile time
configured.
This capture source operates on single and multi-queue NIC's via
suricata.yaml. Here, various features can be enabled, disabled
or edited as required by the use case.
This feature currently only supports receiving packets via AF_XDP,
no TX support has been developed.
Ticket: https://redmine.openinfosecfoundation.org/issues/3306
Additional reading:
https://www.kernel.org/doc/html/latest/networking/af_xdp.html
This patch introduce the IPv4 type for dataset so Suricata commandmatch
on a set of IPv4 addresses. This is meant to complement iprep feature
for people that needs more flexibility such as settings the IP on
the packet path.
Feature: #5383
Add thread local cache to avoid locking overhead for ssns and segments.
A thread will return segments/ssns to a local cache first, and if that
is full, to a return queue where the actual return to the pool returns
a batch, to amortize locking overhead.
Adds segment and session pool/cache counters to see where how effective
the cache is.
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
This patch is adding support for Landlock, a Linux
Security Module available since Linux 5.13.
The concept is to prevent any file operation on directories where
Suricata is not supposed to access.
Landlock support is built by default if the header is present. The
feature is disabled by default and need to be activated in the YAML
to be active.
Landlock documentation: https://docs.kernel.org/userspace-api/landlock.html
Feature: #5479
Add tls.random keyword that matches on the 32 bytes of the TLS
random field for client as well as server.
Add tls.random_time keyword that matches on the first 4 bytes of the TLS
random field for client as well as server.
Add tls.random_bytes keyword that matches on the last 28 bytes of the TLS
random field for client as well as server.
All these are sticky buffers.
Feature 5190
Adds a framework for setting exception policies. These would be called
when the engine reaches some kind of exception condition, like hitting
a memcap or some traffic processing error.
The policy gives control over what should happen next: drop the packet,
drop the packet and flow, bypass, etc.
Implements the policy for:
stream: If stream session or reassembly memcaps are hit call the
memcap policy on the packet and flow.
flow: Apply policy when memcap is reached and no flow could be
freed up.
defrag: Apply policy when no tracker could be picked up.
app-layer: Apply ppolicy if a parser reaches an error state.
All options default to 'ignore', which means the default behavior
is unchanged.
Adds commandline options: add simulation options for exceptions. These
are only exposed if compiled with `--enable-debug`.
Ticket: #5214.
Ticket: #5215.
Ticket: #5216.
Ticket: #5218.
Ticket: #5194.
As Suricata is not supporting pcap-ng we have to stick with one single
datalink type for the capture if ever we want to do pcap logging.
Assuming this, this patch introduces a function to set the link
type globally. This will be used with pcap conditional logging
to get the logging of TCP segments with the correct link type.
- add nom parsers for decoding most messages from StartupPhase and
SimpleQuery subprotocols
- add unittests
- tests/fuzz: add pgsql to confyaml
Feature: #4241
Implement a special sticky buffer to select frames for inspection.
This keyword takes an argument to specify the per protocol frame type:
alert <app proto name> ... frame:<specific frame name>
Or it can specify both in the keyword:
alert tcp ... frame:<app proto name>.<specific frame name>
The latter is useful in some cases like http, where "http" applies to
both HTTP and HTTP/2.
alert http ... frame:http1.request;
alert http1 ... frame:request;
Examples:
tls.pdu
smb.smb2.hdr
smb.smb3.data
Consider a rule like:
alert tcp ... flow:to_server; content:"|ff|SMB"; content:"some smb 1 issue";
this will scan all toserver TCP traffic, where it will only be limited by a port,
depending on how rules are grouped.
With this work we'll be able to do:
alert smb ... flow:to_server; frame:smb1.data; content:"some smb 1 issue";
This rule will only inspect the data portion of SMB1 frames. It will not affect
any other protocol, and it won't need special patterns to "search" for the
SMB1 frame in the raw stream.
The idea of stream frames is that the applayer parsers can tag PDUs and
other arbitrary frames in the stream while parsing. These frames can then
be inspected from the rule language. This will allow rules that are more
precise and less costly.
The frames are stored per direction in the `AppLayerParserState` and will only
be initialized when actual frames are in use. The per direction storage has a
fixed size static portion and dynamic support for a larger number. This is done
for effeciency.
When the Stream Buffer slides, frames are updated as they use offsets relative
to the stream. A negative offset is used for frames that started before the
current window.
Frames have events to inspect/log parser errors that don't fit the TX model.
Frame id starts at 1. So implementations can keep track of frame ids where 0
is not set.
Frames affect TCP window sliding. The frames keep a "left edge" which
signifies how much data to keep for frames that are still in progress.
Set RSS hash function according to Intel ICE PMD available hash functions
Set hash functions according to the support by the ICE PMD, so that no warning
regarding RSS setting is issued.
Set RSS hash function according to Intel IXGBE PMD available hash functions.
During configuration, a warning appeared stating that RSS hash function
has been changed from one value to the other. This has meant that
the supported hash functions did not cover all required hash functions
by the configuration. This commit solves the warning.
Due to peculiar behavior of i40e PMD driver, the RSS is required to be set
via rte_flow rules or a hash filter as compared to other NICs where RSS is
configured through port configuration structure.
RTE_FLOW rules are created on 5-tuples (as opposed to 3-tuple configured
on the other NICs). Fragmented traffic have been tested with this setup
and it has been proven that fragmented packets of the same flow are
received on the same queue. At the same time, setting 3-tuple on rte_flow
rules have not yield in the expected results.
Notes from the experiments:
- Configuration of 5-tuple (as is in the commit):
fragmented and nonfragmented packets are received by the same workers
even when I applied seed to alter them via tcpreplay-edit (option --seed)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_IPV4 (i.e. setting 3-tuple):
when setting ETH_RSS_IPV4, the PMD driver says that pctype is not
supported (generally this means that the "type" of traffic is not
a valid configuration for the i40e)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_NONFRAG_IPV4_OTHER:
this doesn't work well, packets of the same flow are received on
the different workers (my explanation is that the fragmented packets are
matched with ETH_RSS_FRAG_IPV4 but the other UDP packets are not matched
with ETH_RSS_NONFRAG_IPV4_OTHER rte_flow rule (they would be matched with
ETH_RSS_NONFRAG_IPV4_UDP).
Register a new runmode - DPDK. This enables a new flag on Suricata start
(--dpdk).
With the flag given, DPDK runmode is enabled.
Runmode loads the configuration and then initializes EAL.
If successful, it configures the physical NICs according to the configuration
file. After that, worker threads are initialized and then are in continuous
receive loop.
adds a container, ie a thread safe hash table whose
key is the filename
keep a tree of unordered ranges, up to a memcap limit
adds HTPFileOpenWithRange to handle like HTPFileOpen
if there is a range : open 2 files, one for the whole reassembled,
and one only for the current range
Including the whole directory results in .deps files ending up
in the distribution archive which shouldn't be there. Instead
we have to list all the test sources individually.
Jason Ish
Richard McConnell
Shivani Bhardwaj
Aaron Bungay
Eric Leblond
Victor Julien
Philippe Antoine
Juliana Fajardini
Philippe Antoine
Eric Leblond
Juliana Fajardini
Emmanuel Thompson
Lukas Sismis
Jeff Lucovsky
Shivani Bhardwaj