- mqtt
- dnp3
- smtp
- ike
- dns
- alert
- tls
- anomaly
- drop
- file
- http
- http2
- templates
- dhcp
The idea is to factor out the commom code for setting
up the output file objects, which is repetitive, and
often done wrong when it comes to threading.
The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.
Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
This commit creates a common file output helper function based on the
logic in output-file-info.c:BuildBuildFileInfoRecord
The refactored helper will be used to create "fileinfo" information
during the alert output path.
JB_SET_TRUE(jb, key), and JB_SET_STRING(string, key, val) are C macros
around jb_set_formatted to set static string and true values as a
(micro) optimization.
Clang's build in travis-ci is actually failing because of this error:
output-json-alert.c:476:40: error: missing field 'state_index' initializer
[-Werror,-Wmissing-field-initializers]
JsonBuilderMark mark = { 0 };
This is to prepare for JsonBuilder conversion where we can't
overwrite an already set value. Here we prepare the addresses
to be logged in a struct, overwite with XFF if needed, then
log.
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.
As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.
rust modification will encode any non-printable character during its
conversion in to_cstring.
Fix Coverity issue:
** CID 1435535: Null pointer dereferences (REVERSE_INULL)
/src/output-json-file.c: 212 in JsonBuildFileInfoRecord()
Where we check a variable for being NULL, when all paths to the
code show that it can't be NULL.
XFF configuration is already set in app-layer-htp-xff, and in
output-json-alert. Extending XFF configuration to files and HTTP allow
to get the same behavior as for alerts.
Extend the configuration of filestore json to let filestore metafile
dump be aware of xff. This is available only if write-fileinfo is set
to yes and file-store version is 2.
Split the building of the fileinfo record from the writing
of the record so the building can be called from other code.
Specifically the new filestore output which uses fileinfo
records as the metadata.
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.
Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.
Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
In normal records it will try to continue parsing.
GAP 'data' will be passed to file api as '0's. New call is used
so that the file API does know it is dealing with a GAP. Such
files are flagged as truncated at the end of the file and no
checksums are calculated.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
A warning log is already emitted if eve-log is enabled in the
configuration but json support is not built so the logger
registration functions can be silent.
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.
Also, move the registration to the output.c which will handle
registration of the loggers.
Philippe Antoine
Jason Ish
Jeff Lucovsky
Shivani Bhardwaj
Victor Julien
Giuseppe Longo
Jason Ish
magenbluten
Maurizio Abba
Eric Leblond
Duarte Silva