Commit Graph

49 Commits (c3ba9926522ddd33ab43f24710d027cd3f9e035d)

Author SHA1 Message Date
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Anoop Saldanha 797b1a44c7 Replace all strdup with SCStrdup 14 years ago
Anoop Saldanha 13ea299ee0 Replace all mallocs with SCMallocs 14 years ago
Eileen Donlon 89599d3b9b fixed bug 288; corrected config boolean parsing problems 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Anoop Saldanha b819643635 coverity - logging system buffer overrun fix 14 years ago
Victor Julien 1c7b7a01a6 Add option to set the syslog level for the alerts. Minor cleanups. 15 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 15 years ago
Anoop Saldanha dc2c8be583 always read config.h header file first 15 years ago
Victor Julien 40de0b0270 Have each output use the global log format if none is specified for that specific output. 15 years ago
Victor Julien 22f770f3bf Better handle low memory conditions. 15 years ago
Pablo Rincon 76af1b049b Make malloc errors on initialization stage a fatal error, resulting on a exit() call 15 years ago
Victor Julien 718fecb6fc Better handle low memory conditions. 15 years ago
Victor Julien 4dd0169499 Fix detection_filter issue. 15 years ago
Ondrej Slanina 1357914d23 added support for synchronous log output on WIN32 15 years ago
Gurvinder Singh cda664a8c4 memroy leaks fixes in detection module, app layer and counters 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 2dd28ea7fd Use threadsafe time functions. 15 years ago
Victor Julien 8ec51fce57 Remove leftover debug print. 15 years ago
Victor Julien d3b573348e Make sure that the SC_LOG_OP_FILTER env var overrides config as well. 15 years ago
Victor Julien 4875c2daf4 Console logging settings are now overridden by env vars. 15 years ago
Gurvinder Singh cf2d254cc6 fixed the memory leaks and buffer overflows reported by parfait 15 years ago
Anoop Saldanha 9e94768385 fix for bug 108 15 years ago
Victor Julien 16b6f536a0 Fixup Linux compilation after applying win32 patches. 16 years ago
Jan Jezek fe6a72befc Code is now compilable on the Win32 platform 16 years ago
Pablo Rincon d0404d8447 Renaming errors with naming conventions 16 years ago
Pablo Rincon ad2c136e8f Renaming errors (naming conventions) 16 years ago
Jason Ish c72d6be58b Making logging configurable. If no logging outputs are defined the default will be used. - Currently per output log formatting is not available. 16 years ago
Nick Rogness 2b7b78f1bf Intial IPFW support FreeBSD and OSX 16 years ago
Anoop Saldanha 06a640e794 fix for bug #47 16 years ago
Anoop Saldanha a83f7abcc1 logging module bug 6 fix 16 years ago
Victor Julien b7bac14040 Fixup code to compile with -Wall -Werror -Wextra -Wno-unused-parameter compiler options. 16 years ago
Jason Ish 749647a69d use const 16 years ago
Jason Ish 527d735500 Suppress these debug lines. 16 years ago
Gurvinder Singh 542a43437e bug19 patch 16 years ago
Victor Julien 4c79e6d5b6 compile fixes 16 years ago
Jason Ish 28cad3429c An example of how logging could be configured from the log file. 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Anoop Saldanha bfc8200768 Fix for the broken test from logging module 16 years ago
Anoop Saldanha a767a28c70 fix for the test bug in the logging module 16 years ago
William Metcalf 0d13505f0e change debug code around to use global log dir 16 years ago
Anoop Saldanha bc42aebdd5 Logging module optimization changes 16 years ago
Victor Julien 968d8df12b More logging API usage. Changed logging macro's slightly so the vars inside them won't conflict with vars used by the calling function. 16 years ago
Victor Julien 91bc83e5c6 More logging API usage changes. 16 years ago
Anoop Saldanha b3a7f4c015 Update for the logging module and symbol renaming 16 years ago
Victor Julien f4acd5a27e First batch of fixes for new debug and logging API usage. 16 years ago
Anoop Saldanha 157d5e8113 Implementation of the logging module 16 years ago