aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,572 Commits
7 Branches
161 Tags
209 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '07df1ce6af'
${ noResults }
Commit Graph

72 Commits (07df1ce6afffb35c0acd25b3b787ce04643306b1)

Author SHA1 Message Date
Victor Julien 4dff903b35 detect: introduce pkt mpm engines 
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.

Share part of the registration code with the AppLayer.

Implement for the tcp.hdr and udp.hdr keywords.
 6 years ago
Victor Julien ddfcf76c57 detect/engine: make DetectAppLayerMpmRegister decprecated 6 years ago
Victor Julien 91296d1eec detect/prefilter: add de_ctx to registration 8 years ago
Victor Julien efbd901385 detect: move mpm engines into detect engine ctx 
This allows safe registration at runtime.
 8 years ago
Victor Julien a499a44f7a detect: move buffer type map into detect ctx 
Move previously global table into detect engine ctx. Now that we
can register buffers at rule loading time we need to take concurrency
into account.

Move DetectBufferType to detect.h and update DetectBufferCtx API calls
to include a detect engine ctx reference.
 8 years ago
Victor Julien f6e5cb1db6 detect: prefilter/inspect API v2, with transforms 
Introduce InspectionBuffer a structure for passing data between
prefilters, transforms and inspection engines.

At rule parsing time, we'll register new unique 'DetectBufferType's
for a 'parent' buffer (e.g. pure file_data) with its transformations.
Each unique combination of buffer with transformations gets it's
own buffer id.

Similarly, mpm registration and inspect engine registration will be
copied from the 'parent' (again, e.g. pure file_data) to the new id's.

The transforms are called from within the prefilter engines themselves.

Provide generic MPM matching and setup callbacks. Can be used by
keywords to avoid needless code duplication. Supports transformations.

Use unique name for profiling, to distinguish between pure buffers
and buffers with transformation.

Add new registration calls for mpm/prefilters and inspect engines.

Inspect engine api v2: Pass engine to itself. Add generic engine that
uses GetData callback and other registered settings.

The generic engine should be usable for every 'simple' case where
there is just a single non-streaming buffer. For example HTTP uri.

The v2 API assumes that registered MPM implements transformations.

Add util func to set new transform in rule and add util funcs for rule
parsing.
 8 years ago
Victor Julien 26abf5337c detect/mpm: minor cleanup: remove unused function arg 8 years ago
Victor Julien dbd2d7c058 detect: more gracefully handle mpm prepare failure 
Exit with error instead of using the detection engine in a broken state.

Bug #2187
 8 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings 
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
 8 years ago
Victor Julien a1465bc4fa detect-engine-mpm: api cleanup 9 years ago
Victor Julien 779d40cedf detect: remove hardcoded sm_list logic from setup 
Introduce utility functions to aid this.
 9 years ago
Victor Julien 960461f4db fast_pattern: register app layer mpms automatically 
Allow for duplicate registrations for the same list. After the first
registration new calls will be ignored.
 9 years ago
Victor Julien 6dd4dff7b2 mpm: remove empty app_mpms table 9 years ago
Victor Julien 5b2e36a1b0 mpm: add App Layer MPM registery 
Register keywords globally at start up.

Create a map of the registery per detection engine. This we need because
the sgh_mpm_context value is set per detect engine.

Remove APP_MPMS_MAX.
 9 years ago
Victor Julien 9ff5703c49 packet/stream: mpm prefilter engine 9 years ago
Victor Julien ac2c206359 mpm: clean up builtin mpm setup, enable single/full 9 years ago
Victor Julien 6bb2b001a3 mpm: cleanup: move mpm funcs into buffer specific files 9 years ago
Victor Julien e57e7d1b96 mpm: cleanup, remove unused structs and prototypes 9 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 9 years ago
Victor Julien c87fcb29ff detect mpm: fast_pattern assignment cleanup 9 years ago
Victor Julien 26517b8b61 detect: mpm store frees mpm_ctx' it owns 9 years ago
Victor Julien 102a82fc7b detect: use mpm store for app layer mpms 
Rework app-layer mpm setup and registration to make this possible.
 9 years ago
Victor Julien fac2cc0560 detect: mpm deduplication 
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
 9 years ago
Victor Julien 66b3dba676 detect: remove dead code 9 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 9 years ago
Victor Julien 0d3f671b55 detect: constify mpm/detect funcs 9 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 10 years ago
Anoop Saldanha cd7b4fac40 remove unused pattern id assignment functions. Goodbye 12 years ago
Victor Julien 43ba5a677e DNS: enable mpm/fast_pattern support for dns_query 12 years ago
Anoop Saldanha a3212f6a0f Minor fixes against the last set of patches for #564, 565, 581 + fp automation. 
Rename struct DetectFigureFPAndId_t_ to DetectFPAndItsId_ and move it's
definition from inside the function where it's used to the global namespace,
as requested on #suricata.

Rename DetectEngineContentModifiedBufferSetup to DetectEngineContentModifierBufferSetup.

Also rename DetectFigureFPAndId() to DetectSetFastPatternAndItsId().

Updated DetectSetFastPatternAndItsId() to not exit on failure and return error.
 13 years ago
Anoop Saldanha 6de8b1ed53 fix for #564. 
Get rid of the hash table, and use a single-one_time_alloc'ed array for
pattern id assignment.
 13 years ago
Anoop Saldanha 4c6efa2d40 Update content id assignment. 
All fp id assignment now happens in one go.
Also noticing a slight perf increase, probably emanating from improved cache
perf.
Removed irrelevant unittests as well.
 13 years ago
Anoop Saldanha 601836d831 Fast pattern setup now configurable in our code. 
You can either enable/disable fp for a particular type + set priority.
 13 years ago
Anoop Saldanha 0b5d277254 code cleanup for all content based keywords. 13 years ago
Anoop Saldanha 3511f91bba Add support for the new keyword - http_raw_host header. 
The corresponding pcre modifier would be 'Z'.
 13 years ago
Anoop Saldanha c4ce19a1be Add support for a new keyword to inspect http_host header. 
The corresponding content keyword would now be - http_host.
The corresponding pcre modifier would be W.
 13 years ago
Anoop Saldanha 7a7cd6999e feature #558. 
Print FP info in rule analysis + other cleanup.
 13 years ago
Anoop Saldanha 988c92f71c http user agent keyword + mpm + inspection + fast pattern support added 13 years ago
Anoop Saldanha 09313cf9bd Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S 14 years ago
Anoop Saldanha 2007c2711c Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword 14 years ago
Anoop Saldanha 419cdc8558 support splitting mpm ctxs based on direction v2 14 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword. 
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
 14 years ago
Anoop Saldanha 17f3f36d38 packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Anoop Saldanha 966119b6aa support for http_raw_uri keyword + mpm engine 15 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 15 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 15 years ago
Anoop Saldanha d241e51b32 adapt fast pattern engine analysis to reflect the new changes made to your mpm design 15 years ago
Anoop Saldanha 72b0fcf419 modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set 15 years ago
Anoop Saldanha 7ec0382774 support fast pattern for http raw header. Also support relative modifiers for http raw header 15 years ago