Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.
As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.
rust modification will encode any non-printable character during its
conversion in to_cstring.
Fix Coverity issue:
** CID 1435535: Null pointer dereferences (REVERSE_INULL)
/src/output-json-file.c: 212 in JsonBuildFileInfoRecord()
Where we check a variable for being NULL, when all paths to the
code show that it can't be NULL.
XFF configuration is already set in app-layer-htp-xff, and in
output-json-alert. Extending XFF configuration to files and HTTP allow
to get the same behavior as for alerts.
Extend the configuration of filestore json to let filestore metafile
dump be aware of xff. This is available only if write-fileinfo is set
to yes and file-store version is 2.
Split the building of the fileinfo record from the writing
of the record so the building can be called from other code.
Specifically the new filestore output which uses fileinfo
records as the metadata.
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.
Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.
Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
In normal records it will try to continue parsing.
GAP 'data' will be passed to file api as '0's. New call is used
so that the file API does know it is dealing with a GAP. Such
files are flagged as truncated at the end of the file and no
checksums are calculated.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
A warning log is already emitted if eve-log is enabled in the
configuration but json support is not built so the logger
registration functions can be silent.
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.
Also, move the registration to the output.c which will handle
registration of the loggers.
Make the file storage use the streaming buffer API.
As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
When the string representation of a JSON message grew bigger than
64k, the JSON record would just be truncated. This lead to errors
in the parser(s) of the JSON stream.
This patch changes the buffer logic to grow the buffer on demand.
By adding the key in the root of *flow and fileinfo events it
will be possible to get all events for one application layer by
using a 'event_type:proto OR app_proto:proto' filter. This will
permit to the analyst to get a good view of events related to
one protocol.
This patch also fixes a regression in file logging where app_proto
was available before 94dbd303e4 create
the regression.