All loggers were wrapping just the write in a lock with some
updating a counter. This moves the lock into the write function.
The log_ctx alerts counter was also removed as many modules have
stopped using this and the alert count is available elsewhere.
Should satisfy Coverity CID 1400798:
CID 1400798 (#1 of 1): Data race condition (MISSING_LOCK) 2.
missing_lock: Accessing log_ctx->rotation_flag without holding lock
LogFileCtx_.fp_mutex. Elsewhere, "LogFileCtx_.rotation_flag" is accessed
with LogFileCtx_.fp_mutex held 4 out of 5 times.
Which appears to be a false positive as all calls to SCLogFileWrite
were done under lock, but this will make it more explicit.
Give unified2 a nostamp option which will create the file
without the timestamp suffix (like Snort's nostamp option).
Also register for rotation notification on SIGHUP so the file
will be recreated if it is removed by an external rotation
program (only when nostamp is used).
We will now output the sensor name independantly of the output
method if it is set in the YAML file. In the case of redis we are
using the hostname value if unset.
This patch implements reconnection handling for the redis output.
A reconnect limitation has been implemented with a limitation of
one connection per second.
This patch implements redis pipelining. This consist in contacting
the redis server every N events to minimize the number of TCP
exchange. This is optional and setup via the configuration file.
Introduce a function to realize the parsing and config file and
opening of connection to the database. Only used by output-json
for now it will be usable by other logging modules.
Introduce a function LogFileWrite that will handle the writing with
respect of the type defined in the configuration. This is used in
this patch to remove the write complexity from output-json.
When using redis output, we are loosing the host key (added by
logstash or logstash-forwarder) and we can't find anymore what
Suricata did cause the alert.
This patch is adding this key during message generation using the
'sensor-name' variable or the hostname is 'sensor-name' is not
defined.
When running on a TILEncore-Gx PCIe card, setting the filetype of fast.log
to pcie, will open a connection over PCIe to a host application caleld
tile-pcie-logd, that receives the alert strings and writes them to a file
on the host. The file name to open is also passed over the PCIe link.
This allows running Suricata on the TILEncore-Gx PCIe card, but have the
alerts logged to the host system's file system efficiently. The PCIe API that
is used is the Tilera Packet Queue (PQ) API which can access PCIe from User
Space, thus avoiding system calls.
Created util-logopenfile-tile.c and util-logopen-tile.h for the TILE
specific PCIe logging functionality.
Using Write() and Close() function pointers in LogFileCtx, which
default to standard write and close for files and sockets, but are
changed to PCIe write and close functions when a PCIe channel is
openned for logging.
Moved Logging contex out of tm-modules.h into util-logopenfile.h,
where it makes more sense. This required including util-logopenfile.h
into a couple of alert-*.c files, which previously were getting the
definitions from tm-modules.h.
The source and Makefile for tile-pcie-logd are added in contrib/tile-pcie-logd.
By default, the file name for fast.log specified in suricata.yaml is used as
the filename on the host. An optional argument to tile-pcie-logd, --prefix=,
can be added to prepend the supplied file path. For example, is the file
in suricata.yaml is specified as "/var/log/fast.log" and --prefix="/tmp",
then the file will be written to "/tmp/var/log/fast.log".
Check for TILERA_ROOT environment variable before building tile_pcie_logd
Building tile_pcie_logd on x86 requires the Tilera MDE for its PCIe libraries
and API header files. Configure now checs for TILERA_ROOT before enabling
builing tile_pcie_logd in contrib/tile_pcie_logd
util-logopenfile.[ch] implements the abstraction; util-error.[ch]
modified to include a socket-specific error code; output.h adds a
default filetype for logs ("regular").
Jason Ish
Mats Klepsland
Eric Leblond
Victor Julien
Jeff Barber
Jason Ish
Ken Steele
Mike Pomraning