Adds a framework for setting exception policies. These would be called
when the engine reaches some kind of exception condition, like hitting
a memcap or some traffic processing error.
The policy gives control over what should happen next: drop the packet,
drop the packet and flow, bypass, etc.
Implements the policy for:
stream: If stream session or reassembly memcaps are hit call the
memcap policy on the packet and flow.
flow: Apply policy when memcap is reached and no flow could be
freed up.
defrag: Apply policy when no tracker could be picked up.
app-layer: Apply ppolicy if a parser reaches an error state.
All options default to 'ignore', which means the default behavior
is unchanged.
Adds commandline options: add simulation options for exceptions. These
are only exposed if compiled with `--enable-debug`.
Ticket: #5214.
Ticket: #5215.
Ticket: #5216.
Ticket: #5218.
Ticket: #5194.
As Suricata is not supporting pcap-ng we have to stick with one single
datalink type for the capture if ever we want to do pcap logging.
Assuming this, this patch introduces a function to set the link
type globally. This will be used with pcap conditional logging
to get the logging of TCP segments with the correct link type.
- add nom parsers for decoding most messages from StartupPhase and
SimpleQuery subprotocols
- add unittests
- tests/fuzz: add pgsql to confyaml
Feature: #4241
Implement a special sticky buffer to select frames for inspection.
This keyword takes an argument to specify the per protocol frame type:
alert <app proto name> ... frame:<specific frame name>
Or it can specify both in the keyword:
alert tcp ... frame:<app proto name>.<specific frame name>
The latter is useful in some cases like http, where "http" applies to
both HTTP and HTTP/2.
alert http ... frame:http1.request;
alert http1 ... frame:request;
Examples:
tls.pdu
smb.smb2.hdr
smb.smb3.data
Consider a rule like:
alert tcp ... flow:to_server; content:"|ff|SMB"; content:"some smb 1 issue";
this will scan all toserver TCP traffic, where it will only be limited by a port,
depending on how rules are grouped.
With this work we'll be able to do:
alert smb ... flow:to_server; frame:smb1.data; content:"some smb 1 issue";
This rule will only inspect the data portion of SMB1 frames. It will not affect
any other protocol, and it won't need special patterns to "search" for the
SMB1 frame in the raw stream.
The idea of stream frames is that the applayer parsers can tag PDUs and
other arbitrary frames in the stream while parsing. These frames can then
be inspected from the rule language. This will allow rules that are more
precise and less costly.
The frames are stored per direction in the `AppLayerParserState` and will only
be initialized when actual frames are in use. The per direction storage has a
fixed size static portion and dynamic support for a larger number. This is done
for effeciency.
When the Stream Buffer slides, frames are updated as they use offsets relative
to the stream. A negative offset is used for frames that started before the
current window.
Frames have events to inspect/log parser errors that don't fit the TX model.
Frame id starts at 1. So implementations can keep track of frame ids where 0
is not set.
Frames affect TCP window sliding. The frames keep a "left edge" which
signifies how much data to keep for frames that are still in progress.
Set RSS hash function according to Intel ICE PMD available hash functions
Set hash functions according to the support by the ICE PMD, so that no warning
regarding RSS setting is issued.
Set RSS hash function according to Intel IXGBE PMD available hash functions.
During configuration, a warning appeared stating that RSS hash function
has been changed from one value to the other. This has meant that
the supported hash functions did not cover all required hash functions
by the configuration. This commit solves the warning.
Due to peculiar behavior of i40e PMD driver, the RSS is required to be set
via rte_flow rules or a hash filter as compared to other NICs where RSS is
configured through port configuration structure.
RTE_FLOW rules are created on 5-tuples (as opposed to 3-tuple configured
on the other NICs). Fragmented traffic have been tested with this setup
and it has been proven that fragmented packets of the same flow are
received on the same queue. At the same time, setting 3-tuple on rte_flow
rules have not yield in the expected results.
Notes from the experiments:
- Configuration of 5-tuple (as is in the commit):
fragmented and nonfragmented packets are received by the same workers
even when I applied seed to alter them via tcpreplay-edit (option --seed)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_IPV4 (i.e. setting 3-tuple):
when setting ETH_RSS_IPV4, the PMD driver says that pctype is not
supported (generally this means that the "type" of traffic is not
a valid configuration for the i40e)
- Setting only ETH_RSS_FRAG_IPV4 and ETH_RSS_NONFRAG_IPV4_OTHER:
this doesn't work well, packets of the same flow are received on
the different workers (my explanation is that the fragmented packets are
matched with ETH_RSS_FRAG_IPV4 but the other UDP packets are not matched
with ETH_RSS_NONFRAG_IPV4_OTHER rte_flow rule (they would be matched with
ETH_RSS_NONFRAG_IPV4_UDP).
Register a new runmode - DPDK. This enables a new flag on Suricata start
(--dpdk).
With the flag given, DPDK runmode is enabled.
Runmode loads the configuration and then initializes EAL.
If successful, it configures the physical NICs according to the configuration
file. After that, worker threads are initialized and then are in continuous
receive loop.
adds a container, ie a thread safe hash table whose
key is the filename
keep a tree of unordered ranges, up to a memcap limit
adds HTPFileOpenWithRange to handle like HTPFileOpen
if there is a range : open 2 files, one for the whole reassembled,
and one only for the current range
Including the whole directory results in .deps files ending up
in the distribution archive which shouldn't be there. Instead
we have to list all the test sources individually.
Move tests in a seperate commit so that we can use the previous one for
regression testing. This also gets rid of the temporary glue that made
the C tests work with the rust implementation.
Renaming was done with shell commands, git mv for moving the files and content like
find -iname '*.c' | xargs sed -i 's/ikev1/ike/g' respecting the different mixes of upper/lower case.
Split the headers and source into 2 variables. Headers are
marked noinst so they don't get automatically installed on
"make install". Instead they will be installed by a custom
Makefile target, "make install-headers".
Fix another issue with library ordering when breaking apart
LDFLAGS from LIBS for outputting usable command lines for
users of a Suricata library.
RUST_LDADD should just contain the extra libs required by
Rust, not the actual Suricata Rust library.
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.
If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
Building the shared library on Linux is not something by default.
Instead a user must opt-in to building by running the
"make libsuricata.so" target in the src/ directory.
Currently shared library support is only available on Linux. More
OSs will be supported as we can test them.
With the circular reference gone, we can now make use
of a convenience library for the Suricata program
as well as any other programs that depend on the same
source such as the fuzzer.
While its not a libtool convenience library, it serves
the same purpose and is a common idiom in Make and CMake
projects whereas the COMMON_SOURCES approach was more
of a hack we had to resort to until the circular
reference was resolved.
Expose the "SuricataContext" required by Rust as a function. During
normal startup we register this context with the Rust code, but
plugins written in Rust will need to get the same registration
done, but to do this in a plugin, the plugin code must
call and set the context within its address space.
A filetype plugin is a plugin that implements an eve filetype. Most
of the current filetypes could likely be implemented as such a plugin.
Such a plugin must implement Open, Close and Write, where Write
is provided the formatted JSON to be logged.
This commit also includes the plumbing for plugin loading. Example
plugin to come.
Plugins are loaded by the "plugin" section in the configuration
file:
plugins:
- /path/to/directory/plugins
- /path/to/plugin_file.so
This can also be done on the command line with:
--set plugins.0=/path/plugin_file.so
Juliana Fajardini
Philippe Antoine
Philippe Antoine
Victor Julien
Eric Leblond
Juliana Fajardini
Emmanuel Thompson
Jason Ish
Lukas Sismis
Jeff Lucovsky
Shivani Bhardwaj
Simon Dugas
Sascha Steinbiss
frank honza
Carl Smith
Ali Jad Khalil