Issue: 2816
This commit increase the number of VLAN layers supported by Suricata
from 2 to 3. 3-layers are dubbed "Q-in-Q-in-Q".
Note that 3 layers are not compliant with any existing standard but are
often seen in larger deployments.
Issue: 5836
This commit removes the duplicate threaded file tracking from the log
file mechanisms.
Tracking is now consolidated with the threaded hash table.
Issue: 5718
This commit switches the majority of time handling to a new type --
SCTime_t -- which is a 64 bit container for time:
- 44 bits -- seconds
- 20 bits -- useconds
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
The internal flag is not really used. This also reverts the behaviour of
the plugin refactor of passing a ConfNode based on the plugin name
instead of the eve ConfNode.
Register known eve output file types during eve registration. This
removes the function to load internal plugins as they are not really
plugins and moves the registration of them into their respective
subsystem.
With internal code using the plugin API to register an Eve
filetype, the name plugin no longer makes sense. This is
part of my idea that internal plugins aren't plugins at all,
and the plugin interface should be an abstraction over
internal APIs.
Along that idea, this is the start of a refactor of the plugin
file types to be internal, where the plugin API is just an external
interface to that internal API.
The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
Required including NSS header in places that depended on
util-file.h including it.
All filestore suricata-verify tests now pass without libnss.
Also enabled detect-file{md5,sha1,sha256} without NSS support.
Base64Encode function requires the maximum length of the output string
as its last parameter. Use the macro BASE64_BUFFER_SIZE to calculate it
correctly.
We must make sure not to access the flow storage (e.g. keeping a
MacSet) before making sure we have a flow to begin with, We can,
for example, run into an alert without a flow with `ip` rules,
in which case the flow might be NULL. See Redmine issue #4109.
A filetype plugin is a plugin that implements an eve filetype. Most
of the current filetypes could likely be implemented as such a plugin.
Such a plugin must implement Open, Close and Write, where Write
is provided the formatted JSON to be logged.
This commit also includes the plumbing for plugin loading. Example
plugin to come.
Plugins are loaded by the "plugin" section in the configuration
file:
plugins:
- /path/to/directory/plugins
- /path/to/plugin_file.so
This can also be done on the command line with:
--set plugins.0=/path/plugin_file.so
This commit adds MAC address output to the EVE-JSON format. We follow the
remarks made in Redmine ticket #962: for packets, log MAC src/dst as a
scalar field in EVE; for flows, log MAC src/dst as lists in EVE. Field names
are different between flow and packet context to avoid type confusion
(src_mac vs. src_macs). Configuration approach and JSON representation is
taken from previous GitHub PR #2700.
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.
Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
Replaces all patterns of SCLogError() followed by exit() with
FatalError(). Cocci script to do this:
@@
constant C;
constant char[] msg;
@@
- SCLogError(C,
+ FatalError(SC_ERR_FATAL,
msg);
- exit(EXIT_FAILURE);
Closes redmine ticket 3188.
This commit creates a common file output helper function based on the
logic in output-file-info.c:BuildBuildFileInfoRecord
The refactored helper will be used to create "fileinfo" information
during the alert output path.
Jeff Lucovsky
Jeff Lucovsky
Victor Julien
Philippe Antoine
Eric Leblond
Philippe Antoine
Jeff Lucovsky
Jason Ish
Victor Julien
Justin Ossevoort
Shivani Bhardwaj
Sascha Steinbiss