This is to avoid the tx logging code that doesn't support LoggerId
values over 31 at this time. The simplest fix for now is to just have
all JSON (eve) loggers use the same ID.
DNP3 is left as-is for now as it needs some extra support in the parser.
- mqtt
- dnp3
- smtp
- ike
- dns
- alert
- tls
- anomaly
- drop
- file
- http
- http2
- templates
- dhcp
The idea is to factor out the commom code for setting
up the output file objects, which is repetitive, and
often done wrong when it comes to threading.
DNS no longer requires a logger to be registered for to-client and
to-server directions. This has not been required with the stateless
design of the Rust DNS parser.
The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.
Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.
As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
If the DNS log version is not set, we default to v2. This should
not be warning, but better logged at the config level.
A warning will still be logged if the value is set but is not
1 or 2.
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.
rust modification will encode any non-printable character during its
conversion in to_cstring.
This adds two new output formats that permits to reduce
the number of line logged for a dns answer because
actually an event is logged for each answer.
With this patch, only an event that contains all the answers
is logged.
The formats are named 'detailed' and 'grouped'.
The first format provides a list of answers with
the following fields:
- rrname
- rrdata
- ttl
- rdata
The second format provides a list of record data grouped
by their type.
The output below is an example of the formats:
{
"timestamp": "2017-11-29T10:27:18.148282+0100",
"flow_id": 268864910185905,
"in_iface": "wlp2s0",
"event_type": "dns",
"src_ip": "192.168.1.254",
"src_port": 53,
"dest_ip": "192.168.1.176",
"dest_port": 52609,
"proto": "UDP",
"dns": {
"type": "answer",
"id": 3654,
"rcode": "NOERROR",
"answers": [
{
"rrname": "wordpress.org",
"rrtype": "A",
"ttl": 544,
"rdata": "66.155.40.249"
},
{
"rrname": "wordpress.org",
"rrtype": "A",
"ttl": 544,
"rdata": "66.155.40.250"
}
],
"grouped": {
"A": [
"66.155.40.249",
"66.155.40.250"
]
}
}
}
This patch adds a new configuration for dns,
introducing a "version" that permits to switch
between the new and old format to provide
backward compatibility.
The new configuration is made up of these new fields:
- version
- requests (query)
- response (answer)
- types (custom)
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.
Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.
Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
For loggers that register once per direction, use unique id's per
direction.
Reshuffle id's to keep tx log id's low so we can use u32 for tracking
logged loggers.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
Victor Julien
Philippe Antoine
Haleema Khan
Jason Ish
Jeff Lucovsky
Danny Browning
Jason Ish
Victor Julien
Maurizio Abba
Giuseppe Longo
Clément Galland