Commit Graph

73 Commits (e5edcfaca8105e71f647daf9cb952e63831a02c3)

Author SHA1 Message Date
Anoop Saldanha e5edcfaca8 add unittest for atomic operation with void * 13 years ago
Anoop Saldanha ecad4a24fa live rule support added
To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup
13 years ago
Anoop Saldanha 5878d83174 byte_extract_id var now a non-global de_ctx specific var 13 years ago
Anoop Saldanha 7acf5ad38e clean reference config API 13 years ago
Anoop Saldanha 6003c7cb6b clean classification config API 13 years ago
Anoop Saldanha f2dd61868d variable names global vars, global no more. Moved to detection engine ctx, a place it belongs 13 years ago
Anoop Saldanha 55d4e9518e Kill engine during init stage if it fails to load valid value for sgh-mpm-context 13 years ago
Anoop Saldanha 09ec7ec728 bug 456 fix for byte_extract to have array of the right size to update values with 13 years ago
Victor Julien d378b76c04 http: body inspection improvement
Improve http_client_body and file_data performance when request and
response body limits are set to high values.
13 years ago
Victor Julien da3c5bf84d Minor error message cleanups 14 years ago
Nikolay Denev 139768dd58 Do not use underscored config vars internally. 14 years ago
Anoop Saldanha db859cc56e treate ac-bs auto as single context 14 years ago
Anoop Saldanha 4d38a571cc smtp reply code mpm phase support added 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Anoop Saldanha 35f3eafa5e byte extract added to the engine. Detection support added for packet payload, uri and dce detection engines 14 years ago
Eric Leblond 49adc264bc Don't print message after SCMalloc failure.
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
15 years ago
Victor Julien b24ccf8c80 Clean up stream pmqs in the detect thread ctx. 15 years ago
Victor Julien 6ebe7b7cd3 Change the way the request body limit is enforced. 15 years ago
Anoop Saldanha 778ec0939c make client body buffer limit configurable. Also some minor changes 15 years ago
Victor Julien 07ec1ee10e Slightly cleanup detect-engine.sgh-mpm-context option parsing. 15 years ago
Anoop Saldanha c89507836b if sgh-mpm-context is not available in conf, alias the auto case inside the engine 15 years ago
Victor Julien 275bd3b7d7 Switch back to defaulting to full for detect-engine.sgh-mpm-context as it broke many tests. 15 years ago
Victor Julien 7e6f01765f Change default of detect-engine.sgh-mpm-context to auto. 15 years ago
Anoop Saldanha 59923316bc change the default recursion limit in the code to 3000, the value which we currently have in the conf file. Also change print modifier for printing timeval 15 years ago
Anoop Saldanha bc99328ec8 define a new conf paramter detect-engine:inspection-recursion-limit; Defines a recursion limit for content inspection code 15 years ago
Victor Julien 3bd7441ea5 Default to 'single' ctx for ac-gfbs as well. 15 years ago
Anoop Saldanha a2d04a94b5 selecting auto for detect-engine.sgh_mpm_context now uses single if the mpm is ac, full otherwise 15 years ago
Anoop Saldanha 0ef684705c support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup 15 years ago
Anoop Saldanha b367c37ae6 suricata.yaml conf update to support single mpm context distribution over multiple sghs + code to parse this conf 15 years ago
Victor Julien 87f88867f4 Further improve B2gc. Add B2gm. Improve memory layout. 15 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 15 years ago
Anoop Saldanha 9ecade76b9 in case of duplicate signatures used the one with the latest revision 15 years ago
Gurvinder Singh 8852b83fa7 flowbits, flowvars, pktvars, flow flags and app layer info added to alert-debug.log 15 years ago
Pablo Rincon eed0ef6e69 Adding tag keyword support 15 years ago
Anoop Saldanha bbb5bf5c51 allow counters clubbing for detect TM 15 years ago
Victor Julien 83b2c8abdb Improve stateful uri detection code. 15 years ago
Victor Julien a0c1209a44 Inspect the reassembled stream together with the packet payload in the same direction. 15 years ago
Victor Julien 2fd31a1a11 Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory. 15 years ago
Gurvinder Singh cda664a8c4 memroy leaks fixes in detection module, app layer and counters 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
Victor Julien 7a427ec7f4 Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach. 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Anoop Saldanha 47037ef9ec fix for bug 115 15 years ago
Anoop Saldanha c54b91ed94 fix for bug 113 16 years ago
Victor Julien 50e41817a7 Share content id's between identical patterns. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Victor Julien 60685f8b3c Make unittests run more quiet. 16 years ago
Pablo Rincon 38dc7ffebc Adding settings for detect engine group config 16 years ago