Commit Graph

33 Commits (18da4a8b73ea2d94b7cd61c4d07f1c610cac179c)

Author SHA1 Message Date
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Anoop Saldanha 13ea299ee0 Replace all mallocs with SCMallocs 14 years ago
pilcrow ed69eeab14 Safer macro parenthesization and do/while use 14 years ago
Anoop Saldanha 6c95526423 Introduce a new wrapper macro that wait loops till the flag(s) in question have been set 14 years ago
Anoop Saldanha b0a588beeb Introduce another solution to solve stream timeout shutdown issue using thread flags. No more MSSyncPts 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Victor Julien b753ecce50 Implement a pkt acq loop infra with support for pcap-file. 14 years ago
Anoop Saldanha 4f7df1029d Unify the use of slots to a single struct for threading API. Remove separate slot append functions for 1slot and varslot 14 years ago
Victor Julien 43b2e63c1e Fix minor compiler comments in CUDA code. 14 years ago
Martin Beyer 49d66430bc build cuda modules with make 14 years ago
Eric Leblond 66a15e2d6d Fix some Packet initialisation.
This patch fixes Packet initialisation. In some place the pkt field
was not set after a memset used to zero the structure and this could
lead to some problems.
15 years ago
Eric Leblond 1db4aadd16 Supress usage of Packet declaration in tests.
For convenience, a massive usage of 'Packet p;' declaration has
been done in the tests function. Although this was completely
legal, this is not possible anymore because of the new Packet
allocation structure. This massive patch modifies all suricata
files to use a SCMalloc allocated pointer to Packet instead.

This patch has been done using coccinelle (http://coccinelle.lip6.fr)
which is a semantic patching tool. This ensures that things like call
to SCFree() should have not been forget because the semantic patch
explicitly forces the call to SCFree(p) before each return. With this
patch all unittests are running fine with a small and a big default
packet size.
15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
15 years ago
Anoop Saldanha 42c1287028 renintroduce g_u8_lowercase_table for b2g cuda 15 years ago
Martin Beyer b1c577f829 cuda streams support in b2g-cuda MPM 15 years ago
Martin Beyer 0ce86efe40 cuda handlers support multiple CUmodules per context 15 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 15 years ago
Martin Beyer 8adff3c63c use of pagelocked memory for CUDA 15 years ago
Anoop Saldanha ce4bc5a63b some minor modifications to the b2g cuda tests 15 years ago
Anoop Saldanha 3a0dadc0f3 Fix seg fault while running cuda tests. Don't set the alarm while running unittests, inside cuda-packet-batcher.c. Will result in a seg while the sig handler for ALRM in invoked 15 years ago
Victor Julien 3c1ae607cf Fix cuda compilation. 15 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Anoop Saldanha 35bd0c6b39 compiled and added a 64 bit version of the cuda b2g kernel 16 years ago
Anoop Saldanha 606516911b added x86_64 for the b2g cuda code 16 years ago
Anoop Saldanha 53e8120c9d adapt b2g cuda code for the mpm architecture change 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Anoop Saldanha 8cf60d6645 Changed the way cuda dispatcher passes back results. Now each detection thread has it's own queue to which the dispatcher can pump packets back to the detect thread. Also, with cuda enabled and a non-cuda mpm being used, we won't create a dispatcher and instead call the b2g scan/search funtions directly instead of using the dispatcher. 16 years ago
Anoop Saldanha c26e92733d handle the cuda cleanup at shutdown. should get rid of any errors from the call to SigGroupCleanup 16 years ago
Victor Julien d281a6b8ac CUDA build system support & compile fixes
- add configure support for CUDA
- make sure all code compiles if CUDA is disabled
- fix compiler warnings
16 years ago
Anoop Saldanha a2948fc25c valgrind fixes for b2g cuda mpm 16 years ago
Anoop Saldanha 41e6735b92 mpm b2g cuda support added 16 years ago