Commit Graph

79 Commits (be9cd0fd843f1e412bceb3a86b0ad4e5cb7b0782)

Author SHA1 Message Date
Victor Julien 093ecf4798 logging: clean up at shutdown 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
9 years ago
Victor Julien 3f3ed71fe7 jansson: cleanup JSON_ESCAPE_SLASH use 10 years ago
Andreas Herz 8bd9f3d072 json: fix missing includes in disable unix socket case 10 years ago
Eric Leblond 6572725a7a util-debug: don't colorize if a redirect is used
It is better to disable the color mode when a redirect of stderr
is done to avoid getting colorized output in the generated file.
10 years ago
Victor Julien 45fc619f79 logging: json output
Make JSON output optional.

Cleanups.
10 years ago
Victor Julien 126ecb3ebf logging: fix per output log formats 10 years ago
Victor Julien b51c4e608f logging: optional colors output
Construct message per output method.
10 years ago
Victor Julien b13de5bf08 logging: change newline handling 10 years ago
Victor Julien 1927b3000c output: cleanup 10 years ago
Victor Julien b30bdc21b5 logging: cleanup output API
Make SCLogMessage master of the logging. Reduces complexity
of the SCLog macro's.
10 years ago
Victor Julien d6fc6e874f log: reorganize SCLogOPIfaceCtx to make it more efficient 10 years ago
Victor Julien b9aaf5a9ab Fix potential deadlock in output
Coverity:
** CID 1296115:  Program hangs  (ORDER_REVERSAL)
/src/tm-threads.c: 1670 in TmThreadClearThreadsFamily()

The problem is with the by default unused '%m' output parameter.
To get the thread vars it takes the tv_root_lock. This may already
be locked by the calling thread. Also, it could lead to a case of
wrong lock order between the tv_root_lock and the thread_store_lock.

Very unlikely to happen though.

As the %m param isn't really used (by default) this patch just
disables it.
10 years ago
Victor Julien 9ef9a14315 Fix util-debug scan-build warnings
util-debug.c:461:12: warning: Potential leak of memory pointed to by 'substr'
    return SC_ERR_SPRINTF;
           ^~~~~~~~~~~~~~
util-debug.c:856:31: warning: Potential leak of memory pointed to by 's'
                op_ifaces_ctx = SCLogInitFileOPIface(s, NULL, SC_LOG_LEVEL_MAX);
                ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
util-debug.c:1349:9: warning: Potential leak of memory pointed to by 's'
    if (log_level >= 0 && log_level < SC_LOG_LEVEL_MAX)
        ^~~~~~~~~
3 warnings generated.
12 years ago
Eric Leblond 7561da4b87 debug: default logging level is notice
Update the string in message because default logging level is
now notice and not info.
12 years ago
Eric Leblond 28c5c68192 error checking: add missing alloc error treatment
The return of some malloc like functions was not treated in some
places of the code.
12 years ago
Victor Julien 35298a0146 Use %u for unsigned int in (console) output 12 years ago
Eric Leblond 2be194d03f suricata: add -v[v] option to increase verbosity
This patch adds a -v option to suricata. It increases the log level
defined in the YAML.
12 years ago
Eric Leblond 7bcacc712a log: change default log level to notice
This patch updates the log level of meaningful start messages to
notice. It also sets the default log level to notice.
12 years ago
Eric Leblond 34abd818dd Prefix util-conf function with Config 12 years ago
Eric Leblond 54006de40c Use new function GetLogDirectory() 12 years ago
Eric Leblond 4726e02afb logging: add warning if no output module is selected
If no daemon compatible logging module is selected, a message is
displayed to avoid the user to look like mad for messages.
13 years ago
Victor Julien 18ecd4b287 Don't use SCStrdup in SCLogMessage as we call it on OOM condition, leading to endless recursion. SCStrdup failure calling SCLogMessage... 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
pi-rho 0df4c5838d spelling corrections documented in redmine bug#533 13 years ago
Eric Leblond a0e57f58e5 OpenBSD: introduce SCLocalTime function.
This function is a wrapper to localtime_r. It is needed to avoid
a compilation warning on OpenBSD. I'm forced to type the function
to a non pointer first parameter. If not we will have to use two
differents functions in OpenBSD where tv->tv_sec is a long
(different from time_t).
13 years ago
Anoop Saldanha bff2866aed more coverity fixes 13 years ago
Eric Leblond 1bebb9831d logging: don't display debug message before setting params. 14 years ago
Anoop Saldanha 420befb180 Changed my email address to anoopsaldanha at gmail dot com from my current one 14 years ago
Eric Leblond 9f7ee03deb log: read output filter from config file.
The output filter was not read from configuration file and thus
not used in this case.
14 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Anoop Saldanha 797b1a44c7 Replace all strdup with SCStrdup 14 years ago
Anoop Saldanha 13ea299ee0 Replace all mallocs with SCMallocs 14 years ago
Eileen Donlon 89599d3b9b fixed bug 288; corrected config boolean parsing problems 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Anoop Saldanha b819643635 coverity - logging system buffer overrun fix 14 years ago
Victor Julien 1c7b7a01a6 Add option to set the syslog level for the alerts. Minor cleanups. 15 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 15 years ago
Anoop Saldanha dc2c8be583 always read config.h header file first 15 years ago
Victor Julien 40de0b0270 Have each output use the global log format if none is specified for that specific output. 15 years ago
Victor Julien 22f770f3bf Better handle low memory conditions. 15 years ago
Pablo Rincon 76af1b049b Make malloc errors on initialization stage a fatal error, resulting on a exit() call 15 years ago
Victor Julien 718fecb6fc Better handle low memory conditions. 15 years ago
Victor Julien 4dd0169499 Fix detection_filter issue. 15 years ago
Ondrej Slanina 1357914d23 added support for synchronous log output on WIN32 15 years ago
Gurvinder Singh cda664a8c4 memroy leaks fixes in detection module, app layer and counters 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 2dd28ea7fd Use threadsafe time functions. 15 years ago
Victor Julien 8ec51fce57 Remove leftover debug print. 15 years ago