Commit Graph

264 Commits (a00629ab555ce666b4a0e08e0bc96b8523c8d6fd)

Author SHA1 Message Date
Victor Julien a00629ab55 http_host: register inspect engine from keyword 9 years ago
Victor Julien edb2936998 http_user_agent: register inspect engine from keyword 9 years ago
Victor Julien fc857c5455 http_raw_uri: register inspect engine from keyword 9 years ago
Victor Julien b1adea6eee http_cookie: register inspect engine from keyword 9 years ago
Victor Julien cd8b1b0b4c http_method: register inspect engine from keyword 9 years ago
Victor Julien b314829614 http_raw_header: register inspect engine from keyword 9 years ago
Victor Julien eb19eb3fe4 http_header: register inspect engine from keyword 9 years ago
Victor Julien 4096f76b1b http_client_body: register inspect engine from keyword 9 years ago
Victor Julien b96c2c5db5 http_uri: register inspect engine from keyword 9 years ago
Victor Julien cc96fedb90 http_response_line: register inspect engine from keyword 9 years ago
Victor Julien 0feeb8d538 http_request_line: register inspect engine from keyword 9 years ago
Victor Julien 5bde86b0e8 detect-engine: new registration call
Make it more in line with MPM registration.
9 years ago
Victor Julien 5f994756e6 detect-engine: improved inspect engines
Inspect engines are called per signature per sigmatch list. Most
wrap around DetectEngineContentInspection, but it's more generic.

Until now, the inspect engines were setup in a large per ipproto,
per alproto, per direction table. For stateful inspection each
engine needed a global flag.

This approach had a number of issues:
1. inefficient: each inspection round walked the table and then
   checked if the inspect engine was even needed for the current
   rule.
2. clumsy registration with global flag registration.
3. global flag space was approaching the need for 64 bits
4. duplicate registration for alprotos supporting both TCP and
   TCP (DNS).

This patch introduces a new approach.

First, it does away with the per ipproto engines. This wasn't used.

Second, it adds a per signature list of inspect engine containing
only those engines that actually apply to the rule.

Third, it gets rid of the global flags and replaces it with flags
assigned per rule per engine.
9 years ago
Victor Julien f1e3840516 http_response_body: implement keyword with mpm
Implemented as 'stickybuffer'.
9 years ago
Victor Julien 4c98b6cef3 http_request_line: implement keyword and mpm
Implemented as 'stickybuffer'.

Move all logic into the keyword file and remove bad tests that tested
URI instead of request line.
9 years ago
Victor Julien 5b2e36a1b0 mpm: add App Layer MPM registery
Register keywords globally at start up.

Create a map of the registery per detection engine. This we need because
the sgh_mpm_context value is set per detect engine.

Remove APP_MPMS_MAX.
9 years ago
Victor Julien 125603871b detect: config opt to enable keyword prefilters 9 years ago
Victor Julien 17bc0299fe detect: rename non_mpm lists/vars to non_pf
Rename to non_pf: non prefilter.
9 years ago
Mats Klepsland dc8e0b3cf2 detect: add detect engine for tls validity keywords
Add detect engine for tls validity keywords (tls_cert_notbefore and
tls_cert_notafter).
9 years ago
Mats Klepsland d91664d67a detect-dns: move DetectEngineInspectGenericList to detect-engine.c
Move DetectEngineInspectGenericList from detect-engine-dns.c to
detect-engine.c to enable it to be used other places as well.
9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject
This keyword is a replacement for tls.subject.
9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer
This keyword is a replacement for tls.issuerdn.
9 years ago
Victor Julien 215d0d54c7 detect: optimize rule address parsing
Many rules have the same address vars, so instead of parsing them
each time use a hash to store the string and the parsed result.

Rules now reference the stored result in the hash table.
9 years ago
Eric Leblond f2d1e93e65 unix-socket: add auto mode
When running in live mode, the new default 'auto' value of
unix-command.enabled causes unix-command to be activated. This
will allow users of live capture to benefit from the feature and
result in no side effect for user running in offline capture.
9 years ago
Victor Julien b3b78d4326 detect: log earlier that rule reload is happening 9 years ago
Victor Julien a88359dcf0 detect: get proper legacy custom values. Issue #1804 9 years ago
Victor Julien b3bf7a5729 output: introduce config and perf output levels
Goal is to reduce info output
9 years ago
Victor Julien 72d3ea6552 detect: make pattern matcher messages less verbose 9 years ago
Victor Julien 36535cbc61 yaml: remove conf_filename global
conf_filename was a global pointer to the filename of the yaml.

Move into SCInstance. This reduces it's scope and cleans up the code.
9 years ago
Justin Viiret d807bf4e8a detect-engine: log MPM/SPM matchers being used 9 years ago
Justin Viiret c9d0d6f698 mpm: add "auto" default for mpm-algo
Setting mpm-algo to "auto" will use "hs" if Suricata was built against
Hyperscan, and "ac" otherwise (or "ac-tile" on Tilera platforms).
9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
Victor Julien 46e55f1e34 http body handling: use streaming buffer API
Convert HTTP body handling to use the Streaming Buffer API. This means
the HtpBodyChunks no longer maintain their own data segments, but
instead add their data to the StreamingBuffer instance in the HtpBody
structure.

In case the HtpBodyChunk needs to access it's data it can do so still
through the Streaming Buffer API.

Updates & simplifies the various users of the reassembled bodies:
multipart parsing and the detection engine.
9 years ago
Victor Julien 52d500c670 flowworker: initial support
Initial version of the 'FlowWorker' thread module. This module
combines Flow handling, TCP handling, App layer handling and
Detection in a single module. It does all flow related processing
under a single flow lock.
9 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
9 years ago
Justin Viiret cce2d114e8 spm: add and use new SPM API
This new API allows for different SPM implementations, using a function
pointer table like that used for MPM.

This change also switches over the paths that make use of
DetectContentData (which previously used BoyerMoore directly) to the new
API.
9 years ago
Justin Viiret ce408c4d10 spm: add SinglePatternMatchDefaultMatcher
Allows selecting SPM algorithm with the 'spm-algo' value in the YAML
config file.
9 years ago
maxtors 9d3fd82849 Removed duplicate include statements. 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
9 years ago
Victor Julien d6ba01b1b7 detect: make port whitelisting configurable
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.

detect:
  grouping:
    tcp-whitelist: 80, 443
    udp-whitelist: 53, 5060

No portranges are allowed at this point.
9 years ago
Victor Julien 725d6c3739 yaml: convert detect-engine to just detect
Instead of detect-engine which used a list for no good reason, use a
simple map now.

detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
  # If set to yes, the loading of signatures will be made after the capture
  # is started. This will limit the downtime in IPS mode.
  #delayed-detect: yes
9 years ago
Victor Julien 1dd135d512 mpm: always cleanup factory 9 years ago
Victor Julien caea596ce5 profiling: output post-prefilter matches
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:

 --set detect.profiling.inspect-logging-threshold=200

A file called packet_inspected_rules.json is created in the default
log dir.
9 years ago
Victor Julien 722e2dbf7c profiling: initial rulegroup tracking
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.

Logs SGH id so it can be compared with the rule_group.json output.

Implemented both in a human readable text format and a JSON format.
9 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 9 years ago
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 9 years ago
Victor Julien fa885e1d85 mpm: remove pattern id logic 9 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
9 years ago
Victor Julien f0ba00e51d detect: remove old unused code 9 years ago
Victor Julien 10b049304f detect: set new defaults for grouping 9 years ago