The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
Required including NSS header in places that depended on
util-file.h including it.
All filestore suricata-verify tests now pass without libnss.
Also enabled detect-file{md5,sha1,sha256} without NSS support.
Base64Encode function requires the maximum length of the output string
as its last parameter. Use the macro BASE64_BUFFER_SIZE to calculate it
correctly.
We must make sure not to access the flow storage (e.g. keeping a
MacSet) before making sure we have a flow to begin with, We can,
for example, run into an alert without a flow with `ip` rules,
in which case the flow might be NULL. See Redmine issue #4109.
A filetype plugin is a plugin that implements an eve filetype. Most
of the current filetypes could likely be implemented as such a plugin.
Such a plugin must implement Open, Close and Write, where Write
is provided the formatted JSON to be logged.
This commit also includes the plumbing for plugin loading. Example
plugin to come.
Plugins are loaded by the "plugin" section in the configuration
file:
plugins:
- /path/to/directory/plugins
- /path/to/plugin_file.so
This can also be done on the command line with:
--set plugins.0=/path/plugin_file.so
This commit adds MAC address output to the EVE-JSON format. We follow the
remarks made in Redmine ticket #962: for packets, log MAC src/dst as a
scalar field in EVE; for flows, log MAC src/dst as lists in EVE. Field names
are different between flow and packet context to avoid type confusion
(src_mac vs. src_macs). Configuration approach and JSON representation is
taken from previous GitHub PR #2700.
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.
Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
Replaces all patterns of SCLogError() followed by exit() with
FatalError(). Cocci script to do this:
@@
constant C;
constant char[] msg;
@@
- SCLogError(C,
+ FatalError(SC_ERR_FATAL,
msg);
- exit(EXIT_FAILURE);
Closes redmine ticket 3188.
This commit creates a common file output helper function based on the
logic in output-file-info.c:BuildBuildFileInfoRecord
The refactored helper will be used to create "fileinfo" information
during the alert output path.
JB_SET_TRUE(jb, key), and JB_SET_STRING(string, key, val) are C macros
around jb_set_formatted to set static string and true values as a
(micro) optimization.
Convert alert Eve logging JsonBuilder. Currently
makes heavy use of JsonBuilder being able to log Jansson's json_t
which is a temporary measure until all protocols loggers can be
converted to JsonBuilder.
New functions that replace Jansson versions with JsonBuilder
variations use "Eve" instead of "JSON".
Update the source/target logging to use the cached address info
instead of fetching it from the constructed json_t object.
This is required for migration to JsonBuilder which does not
have the ability to retrieve already set fields.
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.
As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
This change ensures that each anomaly is tagged with an
event type to support querying.
Each anomalous event will include `"event_type": "anomaly"`
in the log record.
Add support for community flow id, meant to give a records a
predictable flow id that can be used to match records to
output of other tools.
Takes a 'seed' that needs to be same across sensors and tools
to make the id less predictable.
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.
rust modification will encode any non-printable character during its
conversion in to_cstring.
Jason Ish
Victor Julien
Justin Ossevoort
Shivani Bhardwaj
Sascha Steinbiss
Jeff Lucovsky
Mark Janssen
Maurizio Abba
Jason Ish
Eric Leblond