Commit Graph

248 Commits (ae5846b4deed24cd3150d87465f088503ead29b7)

Author SHA1 Message Date
Victor Julien 125603871b detect: config opt to enable keyword prefilters 9 years ago
Victor Julien 17bc0299fe detect: rename non_mpm lists/vars to non_pf
Rename to non_pf: non prefilter.
9 years ago
Mats Klepsland dc8e0b3cf2 detect: add detect engine for tls validity keywords
Add detect engine for tls validity keywords (tls_cert_notbefore and
tls_cert_notafter).
9 years ago
Mats Klepsland d91664d67a detect-dns: move DetectEngineInspectGenericList to detect-engine.c
Move DetectEngineInspectGenericList from detect-engine-dns.c to
detect-engine.c to enable it to be used other places as well.
9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject
This keyword is a replacement for tls.subject.
9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer
This keyword is a replacement for tls.issuerdn.
9 years ago
Victor Julien 215d0d54c7 detect: optimize rule address parsing
Many rules have the same address vars, so instead of parsing them
each time use a hash to store the string and the parsed result.

Rules now reference the stored result in the hash table.
9 years ago
Eric Leblond f2d1e93e65 unix-socket: add auto mode
When running in live mode, the new default 'auto' value of
unix-command.enabled causes unix-command to be activated. This
will allow users of live capture to benefit from the feature and
result in no side effect for user running in offline capture.
9 years ago
Victor Julien b3b78d4326 detect: log earlier that rule reload is happening 9 years ago
Victor Julien a88359dcf0 detect: get proper legacy custom values. Issue #1804 9 years ago
Victor Julien b3bf7a5729 output: introduce config and perf output levels
Goal is to reduce info output
9 years ago
Victor Julien 72d3ea6552 detect: make pattern matcher messages less verbose 9 years ago
Victor Julien 36535cbc61 yaml: remove conf_filename global
conf_filename was a global pointer to the filename of the yaml.

Move into SCInstance. This reduces it's scope and cleans up the code.
9 years ago
Justin Viiret d807bf4e8a detect-engine: log MPM/SPM matchers being used 9 years ago
Justin Viiret c9d0d6f698 mpm: add "auto" default for mpm-algo
Setting mpm-algo to "auto" will use "hs" if Suricata was built against
Hyperscan, and "ac" otherwise (or "ac-tile" on Tilera platforms).
9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
Victor Julien 46e55f1e34 http body handling: use streaming buffer API
Convert HTTP body handling to use the Streaming Buffer API. This means
the HtpBodyChunks no longer maintain their own data segments, but
instead add their data to the StreamingBuffer instance in the HtpBody
structure.

In case the HtpBodyChunk needs to access it's data it can do so still
through the Streaming Buffer API.

Updates & simplifies the various users of the reassembled bodies:
multipart parsing and the detection engine.
9 years ago
Victor Julien 52d500c670 flowworker: initial support
Initial version of the 'FlowWorker' thread module. This module
combines Flow handling, TCP handling, App layer handling and
Detection in a single module. It does all flow related processing
under a single flow lock.
9 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
9 years ago
Justin Viiret cce2d114e8 spm: add and use new SPM API
This new API allows for different SPM implementations, using a function
pointer table like that used for MPM.

This change also switches over the paths that make use of
DetectContentData (which previously used BoyerMoore directly) to the new
API.
9 years ago
Justin Viiret ce408c4d10 spm: add SinglePatternMatchDefaultMatcher
Allows selecting SPM algorithm with the 'spm-algo' value in the YAML
config file.
9 years ago
maxtors 9d3fd82849 Removed duplicate include statements. 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
10 years ago
Victor Julien d6ba01b1b7 detect: make port whitelisting configurable
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.

detect:
  grouping:
    tcp-whitelist: 80, 443
    udp-whitelist: 53, 5060

No portranges are allowed at this point.
10 years ago
Victor Julien 725d6c3739 yaml: convert detect-engine to just detect
Instead of detect-engine which used a list for no good reason, use a
simple map now.

detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
  # If set to yes, the loading of signatures will be made after the capture
  # is started. This will limit the downtime in IPS mode.
  #delayed-detect: yes
10 years ago
Victor Julien 1dd135d512 mpm: always cleanup factory 10 years ago
Victor Julien caea596ce5 profiling: output post-prefilter matches
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:

 --set detect.profiling.inspect-logging-threshold=200

A file called packet_inspected_rules.json is created in the default
log dir.
10 years ago
Victor Julien 722e2dbf7c profiling: initial rulegroup tracking
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.

Logs SGH id so it can be compared with the rule_group.json output.

Implemented both in a human readable text format and a JSON format.
10 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 10 years ago
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 10 years ago
Victor Julien fa885e1d85 mpm: remove pattern id logic 10 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
10 years ago
Victor Julien f0ba00e51d detect: remove old unused code 10 years ago
Victor Julien 10b049304f detect: set new defaults for grouping 10 years ago
Victor Julien 2ee9bf2aef detect: rename groupings vars 10 years ago
Victor Julien 1f70ccfc23 detect: remove unused grouping settings 10 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 10 years ago
Victor Julien 4f8e1f59a6 mpm: remove obsolete mpm algos
Remove: ac-gfbs, wumanber, b2g, b3g.
10 years ago
Victor Julien 58e533858b detect mpm: ac-tile/ac-ks default to single
Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.
10 years ago
Victor Julien ffb5498228 detect: fix potential deadlock during reload
If interrupted during the BreakLoop stage during reload, a deadlock
could happen.
10 years ago
Justin Viiret 13b87f5aff mpm: add Hyperscan integration
This adds an MPM implementation that uses the Hyperscan regex engine
library from Intel, accessible as the "hs" mpm-algo.
10 years ago
Victor Julien 11099cfa42 detect reload: generic packet injection for capture
Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.

This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.

The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.

Implementations for pcap, AF_PACKET and PF_RING.
10 years ago
Victor Julien eafd212661 detect reload: call 'breakloop' on capture method
Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used

Interupt capture more than once if needed

Move packet injection into util func
10 years ago
Victor Julien 8394b38941 cppcheck: work around snprintf warning
Cppcheck 1.72 gives a warning on the following code pattern:

    char blah[32] = "";
    snprintf(blah, sizeof(blah), "something");

The warning is:

    (error) Buffer is accessed out of bounds.

While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.
10 years ago
Victor Julien c91546022d smtp: clean up thread local memory 10 years ago
Victor Julien b9ee86fdb4 detect-engine: free memory in error conditions (CID 1351210) 10 years ago
Victor Julien 0dd81b85d4 multi-tenants: improve error handling (CID 1312702) 10 years ago
Victor Julien 4dfbc0effa multi-detect: fix and simplify config
instead

mappings:
  - vlan:
    vlan-id: 1
    tenant-id: 2

we'll now use:

mappings:
  - vlan-id: 1
    tenant-id: 2

For YAML it pretty much means the same thing.

Ticket: 1517
10 years ago
Victor Julien 07d8617b3e multi-detect: improve error handling 10 years ago
Victor Julien 906b95eed3 multi-detect: handle missing mappings
Notify/warn user about missing mappings depending on other settings
like unix socket and init errors fatal.
10 years ago