Commit Graph

15 Commits (00b6e628d17f16789bfba1c4e8ca3029cef4f021)

Author SHA1 Message Date
Jason Ish 00b6e628d1 logging: hook into flow worker thread 9 years ago
Jason Ish 1b4ba4496c logging: rename registration functions to not have tmm
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.

Also, move the registration to the output.c which will handle
registration of the loggers.
9 years ago
Jason Ish fc35a78ba1 logging: use a single entry point for all loggers
Introduces a new thread module, TMM_LOGGER, which is the
root most logger.

Only handles loggers in the packet path, stats and flow
logging are not included.

The loggers are made up of a hierarchy of loggers. At the top we
have the root logger which is the main entry point to
logging. Under the root there exists parent loggers that are the
entry point for specific types of loggers such as packet logger,
transaction loggers, etc. Each parent logger may have 0 or more
loggers that actual handle the job of producing output to something
like a file.
9 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers
The loggers moved away from a TMM required a new
profiling support.
9 years ago
Jason Ish 60b6ccc3c4 logging: convert file data logging to non-thread module 9 years ago
Victor Julien f302a6cf86 output: fix debug messages 9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
Victor Julien c19412243a file logging: fix bad end of file check
File storage could end file storing when the logger was invoked
while no new data chunks were available.
9 years ago
Eric Leblond 2866f87c6d output-filedata: fix memleak
This fixes:

Direct leak of 31792 byte(s) in 3974 object(s) allocated from:
    #0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
    #1 0xd86ce2 in OutputFiledataLogThreadInit /home/pmanev/sandnet-qa/stage/oisf/src/output-filedata.c:308:34
    #2 0x106c255 in TmThreadsSlotPktAcqLoop /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:295:17
    #3 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
10 years ago
Victor Julien d67289b60e output-filedata: close files even w/o data
If there is no data chunk but the file is closed/truncated anyway,
logging is still required.
11 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
11 years ago
Victor Julien c27304451e output api: complete shutdown functions
Add missing function for Filedata API. Clean up list in all functions.
11 years ago
Victor Julien b166e2f0e7 profiling: support log api
The log API calls thread modules directly, so the TMM profiling logic
can be applied to it. This patch does so.

The "Thread Module" out now again lists the individual loggers. As the
module are normally called much less frequently the numbers are hard to
compare to pre-log-api numbers.
12 years ago
Victor Julien 52c3d3ad7c log api: convert all names to const
Instead of strdupping all names w/o a need, use const ptrs.
12 years ago
Victor Julien 9ff6608668 Introduce Filedata Logger API
A new logger API for registering file storage handlers. Where the
FileLog handler is called once per file, this handler will be called
for each data chunk so that storing the entire file is possible.

The logger call in the API is as follows:
    typedef int (*FiledataLogger)(ThreadVars *, void *thread_data,
        const Packet *, const File *, const FileData *, uint8_t flags);

All data is const, thus should be read only. The final flags field
is used to indicate to the caller that the file is new, or if it's
being closed.

Files use an internal unique id 'file_id' which can be used by the
loggers to create unique file names. This id can use the 'waldo'
feature of the log-filestore module. This patch moves that waldo
loading and storing logic to this API's implementation. A new
configuration directive 'file-store-waldo: <filename>' is added,
but the existing waldo settings will also continue to work.
12 years ago