aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,557 Commits
7 Branches
155 Tags
197 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '38f67d88ea'
${ noResults }
Commit Graph

6557 Commits (38f67d88ea996e6af00398deb8facb5e7c2b4a05)
 

Author SHA1 Message Date
Victor Julien 38f67d88ea tcp: reduce TCP options storage in packets 
Until now, the TCP options would all be stored in the Packet structure.
The commonly used ones (wscale, ts, sack, sackok and mss*) then had a
pointer to the position in the option array. Overall this option array
was large. About 360 bytes on 64bit systems. Since no part of the engine
would every access this array other than through the common short cuts,
this was actually just wasteful.

This patch changes the approach. It stores just the common ones in the
packet. The rest is gone. This shrinks the packet structure with almost
300 bytes.

* even though mss wasn't actually used
 9 years ago
Torgeir Natvig 7ef8558e79 lua_close() segfaults on null pointers 9 years ago
Victor Julien d085362e61 detect: fix error handling in mpm setup 
*** CID 1358124:  Null pointer dereferences  (REVERSE_INULL)
/src/detect-engine-mpm.c: 940 in MpmStoreSetup()
934                     PopulateMpmHelperAddPatternToPktCtx(ms->mpm_ctx,
935                             cd, s, 0, (cd->flags & DETECT_CONTENT_FAST_PATTERN_CHOP));
936                 }
937             }
938         }
939
>>>     CID 1358124:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "ms->mpm_ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
940         if (ms->mpm_ctx != NULL) {
941             if (ms->mpm_ctx->pattern_cnt == 0) {
942                 MpmFactoryReClaimMpmCtx(de_ctx, ms->mpm_ctx);
943                 ms->mpm_ctx = NULL;
944             } else {
945                 if (ms->sgh_mpm_context == MPM_CTX_FACTORY_UNIQUE_CONTEXT) {
 9 years ago
Victor Julien dc008206e3 detect: fix scan-build warning 
detect-engine-siggroup.c:700:38: warning: Call to 'malloc' has an allocation size of 0 bytes
    sgh->non_mpm_other_store_array = SCMalloc(non_mpm * sizeof(SignatureNonMpmStore));
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./util-mem.h:177:14: note: expanded from macro 'SCMalloc'
    ptrmem = malloc((a)); \
             ^~~~~~~~~~~
1 warning generated.
 9 years ago
Victor Julien 2ee3e726a9 detect: don't set unused variable 
detect.c:3801:13: warning: Value stored to 'tmplist2_tail' is never read
            tmplist2_tail = joingr;
            ^               ~~~~~~
detect.c:3804:13: warning: Value stored to 'tmplist2_tail' is never read
            tmplist2_tail = joingr;
            ^               ~~~~~~
2 warnings generated.
 9 years ago
Victor Julien 9b6e292a28 mpm: remove unused max pattern len field 9 years ago
Victor Julien f1ee125810 detect-flowvar: shrink mem structure by 8 bytes 9 years ago
Victor Julien e2789a87c2 detect: shrink IPOnlyCIDRItem with 8 bytes 9 years ago
Victor Julien adc453eec6 detect-port: improve comment about sgh pointer 9 years ago
Victor Julien ef6f347f84 detect-address: remove sgh pointer as it's unused 9 years ago
Victor Julien d5c6d08bc8 detect-port: remove debug mem counters 9 years ago
Victor Julien 9bd8197009 detect-address: remove debug mem counters 9 years ago
Victor Julien e0111fbb90 detect grouping: remove debug mem counters 9 years ago
Victor Julien 5f676167a3 detect grouping: make json dump configurable 
Make the rule grouping dump to rule_group.json configurable.

detect:
  profiling:
    grouping:
      dump-to-disk: false
      include-rules: false      # very verbose
      include-mpm-stats: false
 9 years ago
Victor Julien d6ba01b1b7 detect: make port whitelisting configurable 
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.

detect:
  grouping:
    tcp-whitelist: 80, 443
    udp-whitelist: 53, 5060

No portranges are allowed at this point.
 9 years ago
Victor Julien 5b1d75f0bd detect: suppress output 9 years ago
Victor Julien 725d6c3739 yaml: convert detect-engine to just detect 
Instead of detect-engine which used a list for no good reason, use a
simple map now.

detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
  # If set to yes, the loading of signatures will be made after the capture
  # is started. This will limit the downtime in IPS mode.
  #delayed-detect: yes
 9 years ago
Victor Julien ac2c206359 mpm: clean up builtin mpm setup, enable single/full 9 years ago
Victor Julien 1dd135d512 mpm: always cleanup factory 9 years ago
Victor Julien 6ef27c9f92 mpm: allow app buffer shared/unique 
Allow setting of shared or unique setting per app buffer type:
e.g. detect.mpm.http_uri.shared=true
 9 years ago
Victor Julien 79a96b2b90 mpm: refactor 'single' setup handling 9 years ago
Victor Julien 157ca89dd7 mpm: remove useless flag from factory 9 years ago
Victor Julien fdd05e8fb4 mpm: remove unused app proto factory 9 years ago
Victor Julien 4e91f6b1e6 mpm: in factory register, consider name const 9 years ago
Victor Julien 2b84387ea4 detect: work around cocci limitation 9 years ago
Victor Julien 0311f01b97 rule grouping: speed up port based grouping 
Create a hash table of unique DetectPort objects before trying to
create a unique list of these objects. This safes a lot of cycles
in the creation of the list.
 9 years ago
Victor Julien a2223bb066 mpm: consify packet/stream search 9 years ago
Victor Julien 87f3adbe4c detect/mpm: unify packet/stream mpm_ctx pointers 
SGH's for tcp and udp are now always only per proto and per direction.
This means we can simply reuse the packet and stream mpm pointers.

The SGH's for the other protocols already used a directionless catch
all mpm pointer.
 9 years ago
Victor Julien 30755265ee http_raw_header: improve mpm progress handling 9 years ago
Victor Julien af3bf3dc7a detect: optimize sgh layout 9 years ago
Victor Julien de273d88cc detect: remove unused content minlen tracking 9 years ago
Victor Julien e43c4f3ea2 mpm: optimize calls 
For all mpm wrapper functions, check minlen vs the input buffer to see
if we can bypass the mpm search.

Next to this, make all the function inline. Also constify the input and
do other minor cleanups.
 9 years ago
Victor Julien 5857660568 http_uri: mpm cleanup. Use mpm_ctx's minlen 9 years ago
Victor Julien 6bb2b001a3 mpm: cleanup: move mpm funcs into buffer specific files 9 years ago
Victor Julien e57e7d1b96 mpm: cleanup, remove unused structs and prototypes 9 years ago
Victor Julien eb19fc4c7b mpm: remove unused structure 9 years ago
Victor Julien caea596ce5 profiling: output post-prefilter matches 
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:

 --set detect.profiling.inspect-logging-threshold=200

A file called packet_inspected_rules.json is created in the default
log dir.
 9 years ago
Victor Julien 92c571b26d detect: move sm_list to string funcs to parser code 9 years ago
Victor Julien 722e2dbf7c profiling: initial rulegroup tracking 
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.

Logs SGH id so it can be compared with the rule_group.json output.

Implemented both in a human readable text format and a JSON format.
 9 years ago
Victor Julien e310a033be detect: assign id to sgh 9 years ago
Victor Julien c880b79f45 detect: shrink sgh 
Turn list of mpm_ctx pointers into a union so that we don't waste
space. The sgh's for tcp and udp are in one direction only, so the
ts and tc ones are now in the union.
 9 years ago
Victor Julien c804102a9a detect: move app_mpms array to init data 9 years ago
Victor Julien 9b3d4f7e24 mpm: unify & localize mpm pattern (id) handling 
So far, the patterns as passed to the mpm's would use global id's that
were shared among all buffers, directions. This would lead to a fairly
large pattern id space. As the mpm algo's use the pattern id's to
prevent duplicate matching through a pattern id based bitarray,
shrinking this space will optimize performance.

This patch implements this. It sets a flag before adding the pattern
to the mpm ctx, instructing the mpm to ignore the provided pid and
handle pids management itself. This leads to a shrinking of the
bitarray size.

This is made possible by the previous work that removes the pid logic
from the code.

Next to this, this patch moves the pattern setup stage to common util
functions. This avoids code duplication.

Update ac, ac-bs and ac-ks to use this.
 9 years ago
Victor Julien ba9d43cce5 mpm: improve negated mpm 
The idea is: if mpm is negated, it's both on mpm and nonmpm sid lists
and we can kick it out in that case during the merge sort.

It only works for patterns that are 'independent'. This means that the
rule doesn't need to only match if the negated mpm pattern is limited
to the first 10 bytes for example.

Or more generally, an negated mpm pattern that has depth, offset,
distance or within settings can't be handled this way. These patterns
are not added to the mpm at all, but just to to non-mpm list. This
makes sense as they will *always* need manual inspection.

Similarly, a pattern that is 'chopped' always needs validation. This
is because in this case we only inspect a part of the final pattern.
 9 years ago
Victor Julien 9e71ef4c3b detect: remove signature pattern id reference 9 years ago
Victor Julien 46734ec41b mpm: remove unused pmq merge function 9 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 9 years ago
Victor Julien 4e8e591715 detect mpm: mpm store cleanup 
Move all rule modification to the fast_pattern assigment.
 9 years ago
Victor Julien c87fcb29ff detect mpm: fast_pattern assignment cleanup 9 years ago
Victor Julien 7c94077892 detect mpm: remove unused mpm flags 9 years ago