Commit Graph

223 Commits (725d6c37395f21e62c93da34530a383500c07e67)

Author SHA1 Message Date
Victor Julien ac2c206359 mpm: clean up builtin mpm setup, enable single/full 9 years ago
Victor Julien 6ef27c9f92 mpm: allow app buffer shared/unique
Allow setting of shared or unique setting per app buffer type:
e.g. detect.mpm.http_uri.shared=true
9 years ago
Victor Julien 79a96b2b90 mpm: refactor 'single' setup handling 9 years ago
Victor Julien 157ca89dd7 mpm: remove useless flag from factory 9 years ago
Victor Julien 87f3adbe4c detect/mpm: unify packet/stream mpm_ctx pointers
SGH's for tcp and udp are now always only per proto and per direction.
This means we can simply reuse the packet and stream mpm pointers.

The SGH's for the other protocols already used a directionless catch
all mpm pointer.
9 years ago
Victor Julien 6bb2b001a3 mpm: cleanup: move mpm funcs into buffer specific files 9 years ago
Victor Julien c880b79f45 detect: shrink sgh
Turn list of mpm_ctx pointers into a union so that we don't waste
space. The sgh's for tcp and udp are in one direction only, so the
ts and tc ones are now in the union.
9 years ago
Victor Julien c804102a9a detect: move app_mpms array to init data 9 years ago
Victor Julien 9b3d4f7e24 mpm: unify & localize mpm pattern (id) handling
So far, the patterns as passed to the mpm's would use global id's that
were shared among all buffers, directions. This would lead to a fairly
large pattern id space. As the mpm algo's use the pattern id's to
prevent duplicate matching through a pattern id based bitarray,
shrinking this space will optimize performance.

This patch implements this. It sets a flag before adding the pattern
to the mpm ctx, instructing the mpm to ignore the provided pid and
handle pids management itself. This leads to a shrinking of the
bitarray size.

This is made possible by the previous work that removes the pid logic
from the code.

Next to this, this patch moves the pattern setup stage to common util
functions. This avoids code duplication.

Update ac, ac-bs and ac-ks to use this.
9 years ago
Victor Julien ba9d43cce5 mpm: improve negated mpm
The idea is: if mpm is negated, it's both on mpm and nonmpm sid lists
and we can kick it out in that case during the merge sort.

It only works for patterns that are 'independent'. This means that the
rule doesn't need to only match if the negated mpm pattern is limited
to the first 10 bytes for example.

Or more generally, an negated mpm pattern that has depth, offset,
distance or within settings can't be handled this way. These patterns
are not added to the mpm at all, but just to to non-mpm list. This
makes sense as they will *always* need manual inspection.

Similarly, a pattern that is 'chopped' always needs validation. This
is because in this case we only inspect a part of the final pattern.
9 years ago
Victor Julien 9e71ef4c3b detect: remove signature pattern id reference 9 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 9 years ago
Victor Julien 4e8e591715 detect mpm: mpm store cleanup
Move all rule modification to the fast_pattern assigment.
9 years ago
Victor Julien c87fcb29ff detect mpm: fast_pattern assignment cleanup 9 years ago
Victor Julien 7c94077892 detect mpm: remove unused mpm flags 9 years ago
Victor Julien a34be23002 detect: simplify negated mpm handling 9 years ago
Victor Julien e48d745ed7 mpm: constify search func args 9 years ago
Victor Julien 26517b8b61 detect: mpm store frees mpm_ctx' it owns 9 years ago
Victor Julien 102a82fc7b detect: use mpm store for app layer mpms
Rework app-layer mpm setup and registration to make this possible.
9 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
9 years ago
Victor Julien f0ba00e51d detect: remove old unused code 9 years ago
Victor Julien d82df4eb8b detect-mpm: make sgh setup proto aware
Allow multi-proto, multi-direction sgh's.
9 years ago
Victor Julien 66b3dba676 detect: remove dead code 9 years ago
Victor Julien b3dcdb10be detect mpm: remove dead code 9 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 9 years ago
Victor Julien 0d3f671b55 detect: constify mpm/detect funcs 9 years ago
Justin Viiret c37195c95f mpm: pass offset, depth args to add functions
MpmAddPatternCI and MpmAddPatternCS had arguments for offset and depth,
but these were not being passed in by the caller.
9 years ago
Victor Julien 9c5ee76455 tcp: fix unlikely NULL-ptr dereference
If a TCP packet could not get a flow (flow engine out of flows/memory)
and there were *only* TCP inspecting rules with the direction
explicitly set to 'to_server', a NULL pointer deref could happen.

PacketPatternSearchWithStreamCtx would fall through to the 'to_client'
case which was not initialized.
9 years ago
Victor Julien c0b16fa2bb detect: allow for more than 64k mpm rules 10 years ago
Victor Julien da7bad7c1b mpm: improve debug output 10 years ago
Victor Julien 977074930b mpm: use IPPROTO_TCP for readability 10 years ago
Victor Julien a559c41295 mpm: optimize & debug validate
Wrappers are called only if a mpm_ctx is available. So remove the test
for a null ctx and replace it by a debug validation BUG_ON.
10 years ago
Victor Julien 0dd3b73db2 mpm: assume we'll likely have a mpm_ctx 10 years ago
Victor Julien 7c336f4190 mpm: indent fix, no functional change 10 years ago
Victor Julien a00d83f1f5 mpm: change direction checking in mpm wrappers
Instead of having reachable assertions, use DEBUG_VALIDATE_BUG_ON
10 years ago
Victor Julien e755913b4b mpm: minor fixes and cleanups 10 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 10 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 10 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
10 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
11 years ago
Ken Steele 3f3481e4d2 Fix indentation 11 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
11 years ago
Ken Steele 970f22c752 Move memcpy_lower() into new util-memcpy.h
Remove local copies from each MPM file and use include file instead.
Might be better to also add util-memcpy.c rather than inlining it each time,
to get smaller code, since only seems to be used at initialization.
12 years ago
Ken Steele cd1c18d981 Store case-insensitive patterns as lowercase.
This is required because SCMemcmpLowercase() expects it first argument
to be already lowercase for the comparison. This is done by using
memcpy_tolower() for NO_CASE patterns.

This addresses code review comments from Victor.
12 years ago
Ken Steele c41041a9c7 When assigning Pattern IDs pids, check Case flags
This fixes bug 1110. When assigning PIDs, use the NO_CASE flag when comparing
for duplicates. The state of the flag must be the same, but also use the same
type of comparisons when checking for duplicates.

Previously, "foo":CS would match with "foo":CI when it should not.
and "foo":CI would not match "FoO":CI when it should. Both of those
cases are fixed with this change.

This then allows simplifying the use of pid in MPMs because now if they
pids match, then so do the flags, so checking the flags is not required.
12 years ago
Victor Julien 0ec375d95a stream msg: remove structure 12 years ago
Anoop Saldanha a49cbf8a49 Code cleanup.
Use the MpmAddPattern[CS|CI] wrapper to add patterns to the mpm context.

Also use MpmInitCtx() to init the mpm context.
12 years ago
Eric Leblond 79fcf1378a Use unlikely in malloc failure test.
This patch is a result of applying the following coccinelle
transformation to suricata sources:

  @istested@
  identifier x;
  statement S1;
  identifier func =~ "(SCMalloc|SCStrdup|SCCalloc|SCMallocAligned|SCRealloc)";
  @@

  x = func(...)
  ... when != x
  - if (x == NULL) S1
  + if (unlikely(x == NULL)) S1
12 years ago
Victor Julien af311aee4e Minor fix for detection engine setup error check
cppcheck said:
[detect-engine-mpm.c:2075] -> [detect-engine-mpm.c:2075]: (style) Same expression on both sides of '||'.
12 years ago
Anoop Saldanha cfa2cda42b fix for bug #973.
An alternative solution for bug #970.

For chopped patterns, which in it's whole is a duplicate of another
pattern we assign an unique content id.
12 years ago