Commit Graph

414 Commits (85cb749e8b62f0a665aeec29d534f7727d4bfb20)

Author SHA1 Message Date
Victor Julien 85cb749e8b detect cleanup: remove sgh mpm_ctx pointers 9 years ago
Victor Julien 82d3c0b520 sgh: remove unused flags 9 years ago
Victor Julien 9ff5703c49 packet/stream: mpm prefilter engine 9 years ago
Victor Julien 5bcdbe3922 prefilter: introduce prefilter engines
Introduce abstraction layer for prefilter engines.
9 years ago
Victor Julien 3dad824fb2 detect: rename SignatureNonMpmStore
New name is SignatureNonPrefilterStore to reflect that it's not just
about MPM anymore.
9 years ago
Victor Julien 17bc0299fe detect: rename non_mpm lists/vars to non_pf
Rename to non_pf: non prefilter.
9 years ago
Victor Julien bb0cd0e883 prefilter: rename PatternMatcherQueue datatype
In preparation of the introduction of more general purpose prefilter
engines, rename PatternMatcherQueue to PrefilterRuleStore. The new
engines will fill this structure a similar way to the current mpm
prefilters.
9 years ago
Giuseppe Longo 07564c4e41 detect: add bypass keyword
This adds a new keyword which permits to call the
bypass callback when a sig is matched.

The callback must be called when the match of the sig
is complete.
9 years ago
Mats Klepsland dc8e0b3cf2 detect: add detect engine for tls validity keywords
Add detect engine for tls validity keywords (tls_cert_notbefore and
tls_cert_notafter).
9 years ago
Mats Klepsland ea5696812f detect: add tls_cert_notbefore and tls_cert_notafter keywords
Detection plugin for TLS certificate fields notBefore and notAfter.

Supports equal to, less than, greater than, and range operations
for both keywords. Dates can be represented as either ISO 8601 or
epoch (Unix time).

Examples:
alert tls [...] tls_cert_notafter:1445852105; [...]
alert tls [...] tls_cert_notbefore:<2015-10-22T23:59:59; [...]
alert tls [...] tls_cert_notbefore:>2015-10-22; [...]
alert tls [...] tls_cert_notafter:2000-10-22<>2020-05-15; [...]
9 years ago
Duarte Silva 89eb935f73 file-hashing: added support for SHA-256 file hashing 9 years ago
Duarte Silva a6d928e269 file-hashing: added support for SHA-1 file hashing 9 years ago
Duarte Silva 188b382c46 file-hashing: common code added
Moved and adapted code from detect-filemd5 to util-detect-file-hash,
generalised code to work with SHA-1 and SHA-256 and added necessary
flags and other constants.
9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject
This keyword is a replacement for tls.subject.
9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer
This keyword is a replacement for tls.issuerdn.
9 years ago
Victor Julien 215d0d54c7 detect: optimize rule address parsing
Many rules have the same address vars, so instead of parsing them
each time use a hash to store the string and the parsed result.

Rules now reference the stored result in the hash table.
9 years ago
Victor Julien 99dce740ef detect: mark alproto in keyword reg deprecated
No existing code uses it, and it had been useless for some time.
9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
Victor Julien 46e55f1e34 http body handling: use streaming buffer API
Convert HTTP body handling to use the Streaming Buffer API. This means
the HtpBodyChunks no longer maintain their own data segments, but
instead add their data to the StreamingBuffer instance in the HtpBody
structure.

In case the HtpBodyChunk needs to access it's data it can do so still
through the Streaming Buffer API.

Updates & simplifies the various users of the reassembled bodies:
multipart parsing and the detection engine.
9 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
9 years ago
Justin Viiret cce2d114e8 spm: add and use new SPM API
This new API allows for different SPM implementations, using a function
pointer table like that used for MPM.

This change also switches over the paths that make use of
DetectContentData (which previously used BoyerMoore directly) to the new
API.
9 years ago
Justin Viiret ce408c4d10 spm: add SinglePatternMatchDefaultMatcher
Allows selecting SPM algorithm with the 'spm-algo' value in the YAML
config file.
9 years ago
Victor Julien f1ee125810 detect-flowvar: shrink mem structure by 8 bytes 9 years ago
Victor Julien e2789a87c2 detect: shrink IPOnlyCIDRItem with 8 bytes 9 years ago
Victor Julien adc453eec6 detect-port: improve comment about sgh pointer 9 years ago
Victor Julien ef6f347f84 detect-address: remove sgh pointer as it's unused 9 years ago
Victor Julien d6ba01b1b7 detect: make port whitelisting configurable
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.

detect:
  grouping:
    tcp-whitelist: 80, 443
    udp-whitelist: 53, 5060

No portranges are allowed at this point.
9 years ago
Victor Julien fdd05e8fb4 mpm: remove unused app proto factory 9 years ago
Victor Julien 0311f01b97 rule grouping: speed up port based grouping
Create a hash table of unique DetectPort objects before trying to
create a unique list of these objects. This safes a lot of cycles
in the creation of the list.
9 years ago
Victor Julien 87f3adbe4c detect/mpm: unify packet/stream mpm_ctx pointers
SGH's for tcp and udp are now always only per proto and per direction.
This means we can simply reuse the packet and stream mpm pointers.

The SGH's for the other protocols already used a directionless catch
all mpm pointer.
9 years ago
Victor Julien af3bf3dc7a detect: optimize sgh layout 9 years ago
Victor Julien de273d88cc detect: remove unused content minlen tracking 9 years ago
Victor Julien 5857660568 http_uri: mpm cleanup. Use mpm_ctx's minlen 9 years ago
Victor Julien e57e7d1b96 mpm: cleanup, remove unused structs and prototypes 9 years ago
Victor Julien caea596ce5 profiling: output post-prefilter matches
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:

 --set detect.profiling.inspect-logging-threshold=200

A file called packet_inspected_rules.json is created in the default
log dir.
9 years ago
Victor Julien 722e2dbf7c profiling: initial rulegroup tracking
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.

Logs SGH id so it can be compared with the rule_group.json output.

Implemented both in a human readable text format and a JSON format.
9 years ago
Victor Julien e310a033be detect: assign id to sgh 9 years ago
Victor Julien c880b79f45 detect: shrink sgh
Turn list of mpm_ctx pointers into a union so that we don't waste
space. The sgh's for tcp and udp are in one direction only, so the
ts and tc ones are now in the union.
9 years ago
Victor Julien c804102a9a detect: move app_mpms array to init data 9 years ago
Victor Julien 9e71ef4c3b detect: remove signature pattern id reference 9 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 9 years ago
Victor Julien 7c94077892 detect mpm: remove unused mpm flags 9 years ago
Victor Julien a7d126738a detect address: remove unused features 9 years ago
Victor Julien cbf80de6fe detect-port: cleanup 9 years ago
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 9 years ago
Victor Julien 1f7e33a4b0 detect: remove unused flag 9 years ago
Victor Julien e8c9598035 detect sgh: remove unused field 9 years ago
Victor Julien a34be23002 detect: simplify negated mpm handling 9 years ago
Victor Julien b84d6d402f detect grouping: multiple whitelist conditions
Instead of the binary yes/no whitelisting used so far, use different
values for different sorts of whitelist reasons. The port list will
be sorted by whitelist value first, then by rule count.

The goal is to whitelist groups that have weak sigs:

 - 1 byte pattern groups

 - SYN sigs

    Rules that check for SYN packets are mostly scan detection rules.
    They will be checked often as SYN packets are very common.

    e.g. alert tcp any any -> any 22 (flags:S,12; sid:123;)

    This patch adds whitelisting for SYN-sigs, so that the sigs end up
    in as unique groups as possible.

 - negated mpm sigs

    Currently negated mpm sigs are inspected often, so they are quite
    expensive. For this reason, try to whitelist them.

These values are set during 'stage 1', rule preprocessing.
9 years ago
Victor Julien 3c184c19cd detect grouping: port based group whitelisting
Whitelist some ports in grouping to make sure they get their own group.
9 years ago