Commit Graph

325 Commits (823f6b35d05b7ecc0218d2de788ba8970096891c)

Author SHA1 Message Date
Victor Julien ad3c97f470 detect-mpm: cleanup 10 years ago
Victor Julien 960461f4db fast_pattern: register app layer mpms automatically
Allow for duplicate registrations for the same list. After the first
registration new calls will be ignored.
10 years ago
Victor Julien 6dd4dff7b2 mpm: remove empty app_mpms table 10 years ago
Victor Julien e68b2214e5 tls: register mpm from keywords 10 years ago
Victor Julien 57ae3c43e5 dns_query: register mpm from keyword 10 years ago
Victor Julien a1a2187a0c http_cookie: register mpm from keyword 10 years ago
Victor Julien 74661449e0 http_raw_host: register mpm from keyword 10 years ago
Victor Julien b5cd4889ae http_host: register mpm from keyword 10 years ago
Victor Julien 91695c81aa http_client_body: register mpm from keyword 10 years ago
Victor Julien 644d4dc61b http_stat_code: register mpm from keyword 10 years ago
Victor Julien cf96db095a http_stat_msg: register mpm from keyword 10 years ago
Victor Julien 43b281a510 file_data: register mpm from keyword 10 years ago
Victor Julien 6d0632a9c6 http_method: register mpm from keyword 10 years ago
Victor Julien e4ea38a8de http_raw_header: register mpm from keyword 10 years ago
Victor Julien 7813a834d0 http_user_agent: register mpm from keyword 10 years ago
Victor Julien 7b98c0073f http_header: register mpm from keyword 10 years ago
Victor Julien 38e018e2d3 http_raw_uri: register mpm from keyword 10 years ago
Victor Julien 7289d12f1b http_uri: register mpm from keyword 10 years ago
Victor Julien 5b2e36a1b0 mpm: add App Layer MPM registery
Register keywords globally at start up.

Create a map of the registery per detection engine. This we need because
the sgh_mpm_context value is set per detect engine.

Remove APP_MPMS_MAX.
10 years ago
Victor Julien 732921922a detect mpm: consider sgh direction when adding rules 10 years ago
Victor Julien 9187c20782 detect mpm: negated setup fix 10 years ago
Victor Julien 56239690d0 prefilter: implement prefilter keyword
Introduce prefilter keyword to force a keyword to be used as prefilter.

e.g.
alert tcp any any -> any any (content:"A"; flags:R; prefilter; sid:1;)
alert tcp any any -> any any (content:"A"; flags:R; sid:2;)
alert tcp any any -> any any (content:"A"; dsize:1; prefilter; sid:3;)
alert tcp any any -> any any (content:"A"; dsize:1; sid:4;)

In sid 2 and 4 the content keyword is used in the MPM engine.
In sid 1 and 3 the flags and dsize keywords will be used.
10 years ago
Victor Julien 85cb749e8b detect cleanup: remove sgh mpm_ctx pointers 10 years ago
Victor Julien 82d3c0b520 sgh: remove unused flags 10 years ago
Victor Julien 08407b6d47 tls: mpm prefilter engines 10 years ago
Victor Julien 7acdc66061 smtp file_data: mpm prefilter engine 10 years ago
Victor Julien 0019a7bd9f http_raw_header: mpm prefilter engine
Register for both regular headers and trailer.
10 years ago
Victor Julien cef12ed80f http_server_body / file_data: mpm prefilter engine 10 years ago
Victor Julien 5646dd9ecf http_client_body: mpm prefilter engine 10 years ago
Victor Julien 9b6fd6bb48 http_headers: mpm prefilter engines
Register for both regular headers and trailers.
10 years ago
Victor Julien 9cab3ea2cd http_stat_code: mpm prefilter engine 10 years ago
Victor Julien 4d57b2fc63 http_stat_msg: mpm prefilter engine 10 years ago
Victor Julien 86d303e32b http_raw_host: mpm prefilter engine 10 years ago
Victor Julien 5218849213 http_host: mpm prefilter engine 10 years ago
Victor Julien 61c3748fc4 http_user_agent: mpm prefilter engine 10 years ago
Victor Julien a43a69305d http_cookie: mpm prefilter engine 10 years ago
Victor Julien 7a46364e42 http_raw_uri: mpm prefilter engine 10 years ago
Victor Julien 746a169127 dns_query: mpm prefilter engine 10 years ago
Victor Julien 9ff5703c49 packet/stream: mpm prefilter engine 10 years ago
Victor Julien 72f2a78b1f http_method: mpm prefilter engine 10 years ago
Victor Julien b62c4cc359 http_uri: mpm prefilter engine
Inspect partial request line as well.
10 years ago
Victor Julien 4c0ab681f2 mpm: remove Cleanup API call
It's unused by all of the implementations.
10 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject
This keyword is a replacement for tls.subject.
10 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer
This keyword is a replacement for tls.issuerdn.
10 years ago
Victor Julien ec0217f52c detect: minor style fixes 10 years ago
Victor Julien b3bf7a5729 output: introduce config and perf output levels
Goal is to reduce info output
11 years ago
Victor Julien 371113e21e ac-ks: don't allow use on big-endian 11 years ago
Justin Viiret c9d0d6f698 mpm: add "auto" default for mpm-algo
Setting mpm-algo to "auto" will use "hs" if Suricata was built against
Hyperscan, and "ac" otherwise (or "ac-tile" on Tilera platforms).
11 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
11 years ago
maxtors 9d3fd82849 Removed duplicate include statements. 11 years ago
Victor Julien d085362e61 detect: fix error handling in mpm setup
*** CID 1358124:  Null pointer dereferences  (REVERSE_INULL)
/src/detect-engine-mpm.c: 940 in MpmStoreSetup()
934                     PopulateMpmHelperAddPatternToPktCtx(ms->mpm_ctx,
935                             cd, s, 0, (cd->flags & DETECT_CONTENT_FAST_PATTERN_CHOP));
936                 }
937             }
938         }
939
>>>     CID 1358124:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "ms->mpm_ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
940         if (ms->mpm_ctx != NULL) {
941             if (ms->mpm_ctx->pattern_cnt == 0) {
942                 MpmFactoryReClaimMpmCtx(de_ctx, ms->mpm_ctx);
943                 ms->mpm_ctx = NULL;
944             } else {
945                 if (ms->sgh_mpm_context == MPM_CTX_FACTORY_UNIQUE_CONTEXT) {
11 years ago
Victor Julien 5b1d75f0bd detect: suppress output 11 years ago
Victor Julien ac2c206359 mpm: clean up builtin mpm setup, enable single/full 11 years ago
Victor Julien 6ef27c9f92 mpm: allow app buffer shared/unique
Allow setting of shared or unique setting per app buffer type:
e.g. detect.mpm.http_uri.shared=true
11 years ago
Victor Julien 79a96b2b90 mpm: refactor 'single' setup handling 11 years ago
Victor Julien 157ca89dd7 mpm: remove useless flag from factory 11 years ago
Victor Julien 87f3adbe4c detect/mpm: unify packet/stream mpm_ctx pointers
SGH's for tcp and udp are now always only per proto and per direction.
This means we can simply reuse the packet and stream mpm pointers.

The SGH's for the other protocols already used a directionless catch
all mpm pointer.
11 years ago
Victor Julien 6bb2b001a3 mpm: cleanup: move mpm funcs into buffer specific files 11 years ago
Victor Julien c880b79f45 detect: shrink sgh
Turn list of mpm_ctx pointers into a union so that we don't waste
space. The sgh's for tcp and udp are in one direction only, so the
ts and tc ones are now in the union.
11 years ago
Victor Julien c804102a9a detect: move app_mpms array to init data 11 years ago
Victor Julien 9b3d4f7e24 mpm: unify & localize mpm pattern (id) handling
So far, the patterns as passed to the mpm's would use global id's that
were shared among all buffers, directions. This would lead to a fairly
large pattern id space. As the mpm algo's use the pattern id's to
prevent duplicate matching through a pattern id based bitarray,
shrinking this space will optimize performance.

This patch implements this. It sets a flag before adding the pattern
to the mpm ctx, instructing the mpm to ignore the provided pid and
handle pids management itself. This leads to a shrinking of the
bitarray size.

This is made possible by the previous work that removes the pid logic
from the code.

Next to this, this patch moves the pattern setup stage to common util
functions. This avoids code duplication.

Update ac, ac-bs and ac-ks to use this.
11 years ago
Victor Julien ba9d43cce5 mpm: improve negated mpm
The idea is: if mpm is negated, it's both on mpm and nonmpm sid lists
and we can kick it out in that case during the merge sort.

It only works for patterns that are 'independent'. This means that the
rule doesn't need to only match if the negated mpm pattern is limited
to the first 10 bytes for example.

Or more generally, an negated mpm pattern that has depth, offset,
distance or within settings can't be handled this way. These patterns
are not added to the mpm at all, but just to to non-mpm list. This
makes sense as they will *always* need manual inspection.

Similarly, a pattern that is 'chopped' always needs validation. This
is because in this case we only inspect a part of the final pattern.
11 years ago
Victor Julien 9e71ef4c3b detect: remove signature pattern id reference 11 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 11 years ago
Victor Julien 4e8e591715 detect mpm: mpm store cleanup
Move all rule modification to the fast_pattern assigment.
11 years ago
Victor Julien c87fcb29ff detect mpm: fast_pattern assignment cleanup 11 years ago
Victor Julien 7c94077892 detect mpm: remove unused mpm flags 11 years ago
Victor Julien a34be23002 detect: simplify negated mpm handling 11 years ago
Victor Julien e48d745ed7 mpm: constify search func args 11 years ago
Victor Julien 26517b8b61 detect: mpm store frees mpm_ctx' it owns 11 years ago
Victor Julien 102a82fc7b detect: use mpm store for app layer mpms
Rework app-layer mpm setup and registration to make this possible.
11 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
11 years ago
Victor Julien f0ba00e51d detect: remove old unused code 11 years ago
Victor Julien d82df4eb8b detect-mpm: make sgh setup proto aware
Allow multi-proto, multi-direction sgh's.
11 years ago
Victor Julien 66b3dba676 detect: remove dead code 11 years ago
Victor Julien b3dcdb10be detect mpm: remove dead code 11 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 11 years ago
Victor Julien 0d3f671b55 detect: constify mpm/detect funcs 11 years ago
Justin Viiret c37195c95f mpm: pass offset, depth args to add functions
MpmAddPatternCI and MpmAddPatternCS had arguments for offset and depth,
but these were not being passed in by the caller.
11 years ago
Victor Julien 9c5ee76455 tcp: fix unlikely NULL-ptr dereference
If a TCP packet could not get a flow (flow engine out of flows/memory)
and there were *only* TCP inspecting rules with the direction
explicitly set to 'to_server', a NULL pointer deref could happen.

PacketPatternSearchWithStreamCtx would fall through to the 'to_client'
case which was not initialized.
11 years ago
Victor Julien c0b16fa2bb detect: allow for more than 64k mpm rules 11 years ago
Victor Julien da7bad7c1b mpm: improve debug output 11 years ago
Victor Julien 977074930b mpm: use IPPROTO_TCP for readability 11 years ago
Victor Julien a559c41295 mpm: optimize & debug validate
Wrappers are called only if a mpm_ctx is available. So remove the test
for a null ctx and replace it by a debug validation BUG_ON.
11 years ago
Victor Julien 0dd3b73db2 mpm: assume we'll likely have a mpm_ctx 11 years ago
Victor Julien 7c336f4190 mpm: indent fix, no functional change 11 years ago
Victor Julien a00d83f1f5 mpm: change direction checking in mpm wrappers
Instead of having reachable assertions, use DEBUG_VALIDATE_BUG_ON
11 years ago
Victor Julien e755913b4b mpm: minor fixes and cleanups 11 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 11 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 12 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
12 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
12 years ago
Ken Steele 3f3481e4d2 Fix indentation 12 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
12 years ago
Ken Steele 970f22c752 Move memcpy_lower() into new util-memcpy.h
Remove local copies from each MPM file and use include file instead.
Might be better to also add util-memcpy.c rather than inlining it each time,
to get smaller code, since only seems to be used at initialization.
13 years ago
Ken Steele cd1c18d981 Store case-insensitive patterns as lowercase.
This is required because SCMemcmpLowercase() expects it first argument
to be already lowercase for the comparison. This is done by using
memcpy_tolower() for NO_CASE patterns.

This addresses code review comments from Victor.
13 years ago
Ken Steele c41041a9c7 When assigning Pattern IDs pids, check Case flags
This fixes bug 1110. When assigning PIDs, use the NO_CASE flag when comparing
for duplicates. The state of the flag must be the same, but also use the same
type of comparisons when checking for duplicates.

Previously, "foo":CS would match with "foo":CI when it should not.
and "foo":CI would not match "FoO":CI when it should. Both of those
cases are fixed with this change.

This then allows simplifying the use of pid in MPMs because now if they
pids match, then so do the flags, so checking the flags is not required.
13 years ago
Victor Julien 0ec375d95a stream msg: remove structure 13 years ago
Anoop Saldanha a49cbf8a49 Code cleanup.
Use the MpmAddPattern[CS|CI] wrapper to add patterns to the mpm context.

Also use MpmInitCtx() to init the mpm context.
13 years ago
Eric Leblond 79fcf1378a Use unlikely in malloc failure test.
This patch is a result of applying the following coccinelle
transformation to suricata sources:

  @istested@
  identifier x;
  statement S1;
  identifier func =~ "(SCMalloc|SCStrdup|SCCalloc|SCMallocAligned|SCRealloc)";
  @@

  x = func(...)
  ... when != x
  - if (x == NULL) S1
  + if (unlikely(x == NULL)) S1
13 years ago