As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.
Also, move the registration to the output.c which will handle
registration of the loggers.
Currently the default configuration file contains a "sensor-name"
at the root of the configuration file, however, eve-log will only
use it if its specified under eve-log.
Now we will look for it at the eve-log, if present we'll use it
but log a deprecation warning, if its not present we'll look
for sensor-name at the root of the configuration.
When the string representation of a JSON message grew bigger than
64k, the JSON record would just be truncated. This lead to errors
in the parser(s) of the JSON stream.
This patch changes the buffer logic to grow the buffer on demand.
Even though the json output callback is called with a null terminated
string, it's not useable directly. The size parameter to the callback
might be a lot smaller than the string size. Libjansson gives the size
up to the first point that needs escaping.
We will now output the sensor name independantly of the output
method if it is set in the YAML file. In the case of redis we are
using the hostname value if unset.
Introduce a function to realize the parsing and config file and
opening of connection to the database. Only used by output-json
for now it will be usable by other logging modules.
Introduce a function LogFileWrite that will handle the writing with
respect of the type defined in the configuration. This is used in
this patch to remove the write complexity from output-json.
When using redis output, we are loosing the host key (added by
logstash or logstash-forwarder) and we can't find anymore what
Suricata did cause the alert.
This patch is adding this key during message generation using the
'sensor-name' variable or the hostname is 'sensor-name' is not
defined.
To be able to correlate between events, it is better to have the
tx_id information in the root object. This function adds a new
function to automate the addition of the field.
It uses the new type field in the LogFileCtx instead.
This fixes the problem of not being able to use two eve-json
instance with different logging methods.
Now that we use 'filetype' instead of 'type', we should also
use 'regular' instead of 'file'.
Added fallback to make sure we stay compatible to old configs.
Instead of error phrone externs with macro's, use functions with a local
static enum var instead.
- EngineModeIsIPS(): in IPS mode
- EngineModeIsIDS(): in IDS mode
To set the modes:
- EngineModeSetIDS(): IDS mode (default)
- EngineModeSetIPS(): IPS mode
Bug #1177.
This patch updates the timestamp format used in eve loggin.
It uses a ISO 8601 comptatible string. This allow tools parsing
the output to easily detect adn/or use the timestamp.
In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.
Logstash configuration is simple:
input {
file {
path => [ "/usr/local/var/log/suricata/eve.json" ]
codec => json
type => "suricata-log"
}
}
filter {
if [type] == "suricata-log" {
date {
match => [ "timestamp", "ISO8601" ]
}
}
}
In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:
[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0
Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:
[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata
In both cases the timestamp are correctly imported by
the tools.
Eve-log would call GET_VLAN_ID on the packets vlan header if p->vlan_idx
was bigger than 0. GET_VLAN_ID would then unconditionally dereference
p->vlanh[0] or [1]. However, there are a number of cases in which these
pointers are not set. Defrag pseudo packets, AF_PACKET and in the future
PF_RING, do set the id's, but not the header pointers.
This patch adds 2 new macro's which are wrappers around a function:
VLAN_GET_ID1 and VLAN_GET_ID2 get the id's by calling DecodeVLANGetId.
This function will return the correct id.
Bug #1120.
Victor Julien
Jason Ish
maxtors
Eric Leblond
Zachary Rasmor
Christophe M
Ken Steele
Alexander Gozman
Jason Ish
Tom DeCanio