Commit Graph

151 Commits (c4d9cb02ec5521bf051ebef6d0fcd986b6cc3b51)

Author SHA1 Message Date
Victor Julien ad5a753dde output-json: don't alloc for JSON to string 10 years ago
Eric Leblond 2ea4bbc492 util-logopenfile: move sensor_name to filectx
We will now output the sensor name independantly of the output
method if it is set in the YAML file. In the case of redis we are
using the hostname value if unset.
10 years ago
Eric Leblond f11b269ef1 redis-output: fix sensor-name code
The sensor-name was not freed at exist and the result of SCStrdup
was not checked.
10 years ago
Eric Leblond f953fdfbac util-logopenfile: introduce SCConfLogOpenRedis
Introduce a function to realize the parsing and config file and
opening of connection to the database. Only used by output-json
for now it will be usable by other logging modules.
10 years ago
Eric Leblond a13be67b5e util-logopenfile: add write function
Introduce a function LogFileWrite that will handle the writing with
respect of the type defined in the configuration. This is used in
this patch to remove the write complexity from output-json.
10 years ago
Eric Leblond 60ea49c777 output-json: add sensor-name config variable
When using redis output, we are loosing the host key (added by
logstash or logstash-forwarder) and we can't find anymore what
Suricata did cause the alert.

This patch is adding this key during message generation using the
'sensor-name' variable or the hostname is 'sensor-name' is not
defined.
10 years ago
Eric Leblond 31c91d53bb output-json: improve hiredis define
Use #ifdef instead of #if and don't include the header which is
not needed anymore.
10 years ago
Eric Leblond eef5678e5e output-json: add redis support
This patch adds redis support to JSON output.
10 years ago
Eric Leblond 113d6a3950 output-json: add create header with tx function
To be able to correlate between events, it is better to have the
tx_id information in the root object. This function adds a new
function to automate the addition of the field.
10 years ago
Jason Ish b512580bbe logging: integrate rotation into SCConfLogOpenGeneric.
Addresses issue 1492, and will make it harder to omit
rotation on new outputs.
10 years ago
Zachary Rasmor 0edf28a4f8 Add Feature #1454. Generic eve-log prefix support. 10 years ago
Victor Julien 1e8142c699 logfile: rename ALERT_ types to LOGFILE_TYPE_ 10 years ago
Eric Leblond 4c6a7bea30 output-json: suppress global variable
It uses the new type field in the LogFileCtx instead.

This fixes the problem of not being able to use two eve-json
instance with different logging methods.
10 years ago
Zachary Rasmor f0c659f82f Fix Bug #1204
Fix typo that causes eve syslog settings code to be unreachable.
10 years ago
Victor Julien a3de4ecd97 Suppress debug statements 11 years ago
Christophe M 6c2ae469be Fix to output a JSON buffer to an Unix domain socket.
Create the JSON buffer and write to it like regular file.

Upper function SCConfLogOpenGeneric already handle it properly.

Closes issue #1246.
11 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
11 years ago
Alexander Gozman a6dbf627b2 Add input interface's name to JSON log 11 years ago
Victor Julien f1185d051c flow id: quick and dirty first stab at a flow id
Add a 'flow_id' that is the same for all records produced for packets
belonging to the same flow.

This patch simply takes the flow's memory address.
11 years ago
Victor Julien eaf01449e3 json: add tcp flags to json utility function
Turns a flags bitfield into a set of json bools.
11 years ago
Victor Julien fdd407751e Fix eve 'filetype' parsing
Now that we use 'filetype' instead of 'type', we should also
use 'regular' instead of 'file'.

Added fallback to make sure we stay compatible to old configs.
11 years ago
Alexander Gozman 8048eebd39 Fix handling filetype for eve log 11 years ago
Jason Ish fc2014ab40 Unregister for file rotation notification when a context is
de-initialized.  Required for unix-socket mode where
contexts come and go.
11 years ago
Jason Ish e1b97fed70 Add signal based file rotation for:
- alert debug log
- fast log
- stats log
- dns log
- drop log
- file log
- http log
- tls log
- eve/json log
11 years ago
Victor Julien 96adcf6829 refactor IDS/IPS engine mode logic
Instead of error phrone externs with macro's, use functions with a local
static enum var instead.

- EngineModeIsIPS(): in IPS mode
- EngineModeIsIDS(): in IDS mode

To set the modes:

- EngineModeSetIDS(): IDS mode (default)
- EngineModeSetIPS(): IPS mode

Bug #1177.
11 years ago
Victor Julien bc1c06b9e4 eve-log: fix mem leak at shutdown
Make sure we free all memory in the shutdown function.
11 years ago
Tom DeCanio 4085f08602 json: somewhere along the way IP/port pairs had gotten swapped in
http and ssh eve logs
11 years ago
Eric Leblond 6c3c234ca5 output-json: update timestamp format
This patch updates the timestamp format used in eve loggin.
It uses a ISO 8601 comptatible string. This allow tools parsing
the output to easily detect adn/or use the timestamp.

In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.

Logstash configuration is simple:

input {
   file {
      path => [ "/usr/local/var/log/suricata/eve.json" ]
      codec =>   json
      type => "suricata-log"
   }
}

filter {
   if [type] == "suricata-log" {
      date {
        match => [ "timestamp", "ISO8601" ]
      }
   }
}

In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:

[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0

Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:

[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata

In both cases the timestamp are correctly imported by
the tools.
12 years ago
Victor Julien 88e9c85e36 json output: fix vlan byte order in output
VLAN functions/macros return vlan id in host byte order, so no need
to convert them in output functions.
12 years ago
Victor Julien 872bb5664e Fix null dereference in eve-log
Eve-log would call GET_VLAN_ID on the packets vlan header if p->vlan_idx
was bigger than 0. GET_VLAN_ID would then unconditionally dereference
p->vlanh[0] or [1]. However, there are a number of cases in which these
pointers are not set. Defrag pseudo packets, AF_PACKET and in the future
PF_RING, do set the id's, but not the header pointers.

This patch adds 2 new macro's which are wrappers around a function:

VLAN_GET_ID1 and VLAN_GET_ID2 get the id's by calling DecodeVLANGetId.

This function will return the correct id.

Bug #1120.
12 years ago
Victor Julien d8c486231c output-json: fix minor memory leak on error
If the json file couldn't be opened, a minor memory leak would occur.

Coverity 1166039
12 years ago
Eric Leblond 6fd9b4b255 json: add event_type key
This patch adds an event_type key to the generated events. Current
value is one of "dns", "alert, "file", "tls", "http", "drop". It is
then easy to differentiate in log analysis tools the events based on
source inside Suricata.
12 years ago
Eric Leblond 7a9efd74e4 json: sync key name with CIM
This patch is synchronizing key name with Common Information Model.
It updates key name following what is proposed in:
 http://docs.splunk.com/Documentation/PCI/2.0/DataSource/CommonInformationModelFieldReference
The interest of these modifications is that using the same key name
as other software will provide an easy to correlate and improve
data. For example, geoip setting in logstash can be applied on
all src_ip fields allowing geoip tagging of data.
12 years ago
Victor Julien 31a024c9b5 Various fixes for scan-build warnings 12 years ago
Victor Julien cd7a5ff0ca output: cleanups
Preparation of making output type for json logs configurable.
12 years ago
Victor Julien 5874f52ec6 json: rename output-httplog -> output-json-http 12 years ago
Victor Julien b5ef269b03 json outputs: cleanups
Clean up header files and improve memory handling.
12 years ago
Victor Julien 3fc63d3656 jansson file log: make file log module
Turn the libjansson based file logger into a file module, as a child
of eve-log.
12 years ago
Victor Julien 039f7b3e5f tls json: turn into packet logger
Like log-tls, turn the json tls logger into a packet logger as the
protocol parser is not tx aware.

Make it a child of eve-log as well.
12 years ago
Victor Julien f0aa2ed240 json drop log: move into packet module
Move JSON drop log into a full packet module.
12 years ago
Victor Julien 85335d9cbe alert json: make child of eve-log
Enable alert json for eve-log by registering the module as a sub-
module of eve-log.
12 years ago
Victor Julien 42858647e2 alert-json: make full module out of json alert
Make a full module out of the json alert code in output-json-alert.[ch].
12 years ago
Victor Julien 8c3e71559a dns-json: turn logger to tx api
Convert Json DNS logger into a Tx Logger API logger.
12 years ago
Victor Julien bc71a43e08 http-json: separate module using tx api
Turn HTTP json logger into a Tx Logger API logger.
12 years ago
Victor Julien 4874d5abbb Various compile fixes after rebase with master 12 years ago
Tom DeCanio 6fd1b31c57 Remaining JSON output pull request comment edits 12 years ago
Tom DeCanio a3d86594dc address most initial JSON pull request comments 12 years ago
Tom DeCanio 55df2d5cdb add "united" drop JSON log 12 years ago
Tom DeCanio 0c067646a8 Add "united" JSON files output 12 years ago
Tom DeCanio 730ee3d721 First cut at "united" file log output in JSON 12 years ago
Tom DeCanio 3241732e27 rename alert-json.[ch] output-json.[ch] 12 years ago