aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,260 Commits
7 Branches
161 Tags
210 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '2e878c2024'
${ noResults }
Commit Graph

61 Commits (2e878c202468ea7866de136048191d4d82771589)

Author SHA1 Message Date
Victor Julien 8798bf48b2 profiling: support prefilter engines 9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject 
This keyword is a replacement for tls.subject.
 9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer 
This keyword is a replacement for tls.issuerdn.
 9 years ago
Jason Ish 04a44a077d logging: convert pcap log to non-thread module 9 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers 
The loggers moved away from a TMM required a new
profiling support.
 9 years ago
Victor Julien b3bf7a5729 output: introduce config and perf output levels 
Goal is to reduce info output
 9 years ago
Victor Julien e09643c396 flow worker: profiling 
Previously the detect and stream code lived in their own thread
modules. This meant profiling showed their cost as part of the
thread module profiling logic. Now that only the flow worker is
a thread module this no longer works.

This patch introduces profiling for the 3 current flow worker
steps: flow, stream, detect.
 9 years ago
Victor Julien 4a96820320 stream-tcp: more cleanups 9 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni 
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code 
1 pass, 0 is fail.
 9 years ago
Victor Julien cc4f7a4b96 detect: add profiling for non-mpm list build & filter 11 years ago
Victor Julien 485f34134e unix socket: support profiling 11 years ago
Ken Steele 8f1d75039a Enforce function coding standard 
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
 11 years ago
Victor Julien 2646edc129 Profiling: fix compilation on CentOS5 
Bug #1207
 11 years ago
Victor Julien 6dd10443ce profiling: output log api modules separately 
Skip log api thread modules in the regular 'thread modules' list,
instead print them in a separate list.
 12 years ago
Victor Julien a37a1d9de7 profiling: fix percentage of detect phases 
Use proper 'total' to calculate weigth of each detection phase.
 12 years ago
Victor Julien 8a735a9b90 profiling: add sample-rate yaml option 
Add option "profiling.sample-rate":

  # Run profiling for every xth packet. The default is 1, which means we
  # profile every packet. If set to 1000, one packet is profiled for every
  # 1000 received.
  #sample-rate: 1000

This allows for configuration of the sample rate.
 12 years ago
Victor Julien 2c3a92a1c9 profiling: conditional rule profiling 
Add support for conditional rule profiling. Currently only simple
rate limiting is supported, but hardcoded to inspecting rules for
each packet.
 12 years ago
Victor Julien 13d491f577 profiling: lower overhead when disabled 
Instead of a large (6k+) structure in the Packet, make the profiling
storage dynamic. To do this the Packet->profile is now a pointer.

Initial support for selective sampling, e.g. only profile every
1000th packet.
 12 years ago
Anoop Saldanha 127ef8f903 Use a typdef AppProto <-> uint16_t for representing app layer protocol. 
Some minor refactoring/cleanup, including renaming functions.
 12 years ago
Eric Leblond 34abd818dd Prefix util-conf function with Config 12 years ago
Eric Leblond 54006de40c Use new function GetLogDirectory() 12 years ago
Victor Julien 43ba5a677e DNS: enable mpm/fast_pattern support for dns_query 12 years ago
Victor Julien ffffe6c10e profiling: add formatted totals, percents to packet stats 12 years ago
Victor Julien 82769a1b37 profiling: fix missing profile names 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment. 
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
 13 years ago
Eric Leblond 655577cbbc Add some missing checks of SCMalloc return. 13 years ago
Victor Julien 7babb35aeb profiling: remove obsolete unit test 13 years ago
Victor Julien 2343ff8950 profiling: fix memory error in case of rule reload. 13 years ago
Victor Julien ec7e79c748 Rule profiling update 
- Remove usage of counters api.
- Store stats in detect engine thread ctx to remove locking
- Support rule reloads
 13 years ago
Eric Leblond 7af9fd7735 freebsd: fix warning about redeclaration. 13 years ago
Eric Leblond 0227a87fcb cleaning: fix warning when building with clang. 
clang was issuing some warnings related to unused return in function.
This patch adds some needed error treatment and ignore the rest of the
warnings by adding a cast to void.
 13 years ago
Eric Leblond a0e57f58e5 OpenBSD: introduce SCLocalTime function. 
This function is a wrapper to localtime_r. It is needed to avoid
a compilation warning on OpenBSD. I'm forced to type the function
to a non pointer first parameter. If not we will have to use two
differents functions in OpenBSD where tv->tv_sec is a long
(different from time_t).
 13 years ago
Victor Julien d908e707d7 profiling: add per lock location profiling 
Add profiling per lock location in the code. Accounts how often a
lock is requested, how often it was contended, the max number of
ticks spent waiting for it, avg number of ticks waiting for it and
the total ticks for that location.

Added a new configure flag --enable-profiling-locks to enable this
feature.
 14 years ago
Victor Julien 979edf0b97 Add way to profile mutex/spin locks per thread module. 14 years ago
Victor Julien 0b9038b971 Add atomics to ticks unittests. 14 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword. 
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
 14 years ago
Victor Julien af51493da2 Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm. 14 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Anoop Saldanha 13ea299ee0 Replace all mallocs with SCMallocs 14 years ago
Victor Julien cc4e89fbe1 Profiling: convert all packet profile counters/variables to u64. Improve output for larger numbers. 14 years ago
Eileen Donlon 89599d3b9b fixed bug 288; corrected config boolean parsing problems 14 years ago
Victor Julien 63f834d9a7 Add profiling to various HTTP buffer MPM calls. 14 years ago
Victor Julien 3693a7a9ee Profiling: add accounting for several detection phases. 14 years ago
Victor Julien e8e392fb1f Profiling: add per packet accounting of how much ticks are spend in protocol detection. 14 years ago
Victor Julien 21f387d2c7 profiling: fix stream ticks miscalculation on stream end pseudo packets. 14 years ago
Victor Julien fca541f40e Add per app layer parser profiling 
Per packet per app layer parser profiling. Example summary output:

Per App layer parser stats:

App Layer              IP ver   Proto   cnt        min      max          avg
--------------------   ------   -----   ------     ------   ----------   -------
ALPROTO_HTTP            IPv4       6    163394        126     38560320     42814
ALPROTO_FTP             IPv4       6       644        117        26100      2566
ALPROTO_TLS             IPv4       6       670        117         7137       799
ALPROTO_SMB             IPv4       6    114794        126       225270       957
ALPROTO_DCERPC          IPv4       6      5207        126        25596      1266

Also added to the csv out.

In the csv out there is a new column "stream (no app)" that removes the
app layer parsers from the stream tracking. So raw stream engine performance
becomes visible.
 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Victor Julien 1bd1a62526 Rename profile macro's and variables to reflect that they are for rule profiling. 14 years ago
William bca8fbc79e Add Num, Rev, and Gid columns to rule perf output 14 years ago