Commit Graph

484 Commits (f4ce9011d22d2323b1029ca907f4578fe4b99be3)

Author SHA1 Message Date
Anoop Saldanha 93fa7ea828 modify detection engine to run hrhd mpm before building the match array 15 years ago
Anoop Saldanha b140ed1c9c modify detection engine to run hhd mpm before building the match array 15 years ago
Anoop Saldanha 4e273f2c8b modify detection engine to carry out hcbd mpm run before build match array if alproto is http and if sgh has atleast one sig with hcbd mpm set 15 years ago
Anoop Saldanha 72b0fcf419 modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set 15 years ago
Victor Julien 6a5d2cb40d Fix potential locking issue in out of memory conditions in the http_header, http_raw_header code. Fix other potential small issues in http_ code. 15 years ago
Anoop Saldanha 7ec0382774 support fast pattern for http raw header. Also support relative modifiers for http raw header 15 years ago
Victor Julien 0c806f70bb Fix --enable-debug compilation, just unittest with --enable-debug-validation enabled. 15 years ago
Victor Julien 1a32d9b5ec Fix printing unprintable characters in the engine-analysis fast_pattern mode. 15 years ago
Anoop Saldanha c61c68fd36 mpm and fast pattern support for http_header. Also support relative modifiers for http_header 15 years ago
Anoop Saldanha 778ec0939c make client body buffer limit configurable. Also some minor changes 15 years ago
Anoop Saldanha 0aa5cffb12 fast pattern support for http_client_body keyword added. Also mpm support for http_client_body added 15 years ago
Anoop Saldanha c227aeeacb remove support for skipping reinspecting fast pattern contents once again during packet payload inspection. Also make some changes to our detection engine 15 years ago
Anoop Saldanha bbd0c5056b store the content added for mpm inside Signature. also carry out an unconditional cleanup of packet pattern matcher pmq det_ctx->pmq 15 years ago
Anoop Saldanha 6df051321f fix fp when content is negated and also added to mpm 15 years ago
Anoop Saldanha 5c6a65dc58 support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch] 15 years ago
Anoop Saldanha 3d2f81d978 replace all Signature->dmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_DMATCH] 15 years ago
Anoop Saldanha a7353be20d replace all Signature->amatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_AMATCH] 15 years ago
Anoop Saldanha e0476242c6 replace all Signature->umatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_UMATCH] 15 years ago
Anoop Saldanha e54358a9e1 replace all Signature->pmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_PMATCH] 15 years ago
Anoop Saldanha 82fd581b64 replace all sm lists (match, pmatch, dmatch, umatch, amatch, tmatch) with an array Signature->sm_lists[]. Replace all Signature->match instances in the engine with Signature->sm_lists[DETECT_SM_LIST_MATCH] 15 years ago
Victor Julien 001f91056e Add http_raw_header as an alias to the http_header keyword as that actually inspects the raw headers (see issue #243). Closes issue #242. 15 years ago
Gurvinder Singh b7da115e6d support for http_stat_code keyword has been added to detection module 15 years ago
Gurvinder Singh 1deae70cf7 added http_stat_msg keyword support for detection module 15 years ago
Anoop Saldanha 2cdb5be391 Print out file name for fast_pattern engine_analysis. Also add some info logs 15 years ago
Anoop Saldanha a2d04a94b5 selecting auto for detect-engine.sgh_mpm_context now uses single if the mpm is ac, full otherwise 15 years ago
Anoop Saldanha 174048544d fix hash generation in b2g and ac addpattern. Brings down the no of patterns added from close to a million to a couple of thousands 15 years ago
Anoop Saldanha 0ef684705c support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup 15 years ago
Anoop Saldanha a85fa6b792 support for fast_pattern only and fast_pattern:offset,length. Also support the new option for engine-analysis 15 years ago
Anoop Saldanha 0d741b9a55 fix for bug 227. For negated contents that have been added to mpm we might have pmq.pattern_id_array_cnt as 0. We can't ignore inspecting sigs if this is 0, in case the content added is negated 15 years ago
Victor Julien cbd4c298ed Initial version of a new bitmask based signature pre-filtering method. 15 years ago
Victor Julien 94898a91cc Reorganize SigMatchSignatures. 15 years ago
Victor Julien fc248ca7a1 Many small performance updates. 15 years ago
Pablo Rincon 5c43db85ce Drop streams on inline mode when a drop rule match from a reassembled stream and/or app layer inspection 15 years ago
Anoop Saldanha f094523eb1 clang fix - some minor fixes for unittests 15 years ago
Pablo Rincon 9d7baa7a9f Adding ssh app layer module with two new keywords: ssh.protoversion and ssh.softwareversion 15 years ago
Victor Julien 04d3832d8f Remove ports check and fix small typo. 15 years ago
Victor Julien a492518e7a Properly detect detect-event-only sigs. 15 years ago
Pablo Rincon 21d79b05ad Fix for bug221 (avoid considering sig as "decoder event only" if ports are specified). Now the sig gets grouped to get a sgh at SigMatchSignatures 15 years ago
Victor Julien 6299fbfb0f Fix stream msg content inspection not inspecting the correct id. 15 years ago
Victor Julien 1071a53210 Fix unittests after ip_proto keyword change. 15 years ago
Pablo Rincon 70bda6506d Fix for bug 180 (check proto specified at the IP hdr) 15 years ago
Victor Julien 7acb97da9d Use same mpm prepare procedure for uricontent as for normal content. More cleanups. 15 years ago
Victor Julien 9ba11dbfbd Clean up detection engine mpm initialization phase. 15 years ago
Victor Julien 0d008c8135 Change stateful detection engine to be able to start the stateful detection separate from other sigs. Fixes bugs #213, #214, #215. 15 years ago
Victor Julien 689d05b10b Add missing protocol check in the sig matching process. This prevents FP's such as the one reported in bug #209. 15 years ago
Victor Julien 0219b767b8 Fix a content pattern matching bug related to signature grouping and mpm_ctx sharing. In certain conditions (signature combinations) the mpm_stream_ctx (the ctx that handles stream pattern scanning) wasn't properly setup. 15 years ago
Victor Julien 102092a89c Make signature address matching more cache efficient. 15 years ago
Victor Julien 1eec149f5e Use Address structure in DetectAddress struct. 15 years ago
Victor Julien c6ddcda7f8 Improve out of memory handling during initialization. 15 years ago
Victor Julien bfd167521e Fix DCERPC over SMB/SMB2 detection issues. Fix not updating transaction id in a stream direction if there was no sgh. 15 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 15 years ago
Victor Julien b3c22cd512 Improve app layer proto check. 15 years ago
Victor Julien 39cb1bdbda Fix app layer sigs being recognized as decoder event only or ip only. 15 years ago
Victor Julien d41b5645ef Make sure decoder event rules are inspected even if the packet is invalid and has no addesses or proto. Update fast log and alert debug log to display the alerts. Fixes #179. 15 years ago
Victor Julien e685579231 Add optional structure validation code. 15 years ago
Victor Julien 393acd77d2 Detection improvements: uricontent escaping now working, better negated pattern (content) handling. 15 years ago
Anoop Saldanha 9ecade76b9 in case of duplicate signatures used the one with the latest revision 15 years ago
Gurvinder Singh 8852b83fa7 flowbits, flowvars, pktvars, flow flags and app layer info added to alert-debug.log 15 years ago
Victor Julien 580b09c2b8 Make sure we inspect all outstanding reassembled stream chunks (smsg) if the stream is shutting down. Make sure to do inspect signatures that use dsize against the tcp packet payload, even if that payload was already added to the stream. Likewise, the dsize signatures are not inspected against the reassembled stream. 15 years ago
Victor Julien a3ff0e7210 Don't scan TCP packet payload if it was added to the stream. Inspect the tcp stream with the correct packet. Should fix #184 and #185. 15 years ago
Pablo Rincon a8cb8d830b Fix for bug 186 and thresholding issue handling ip versions 15 years ago
Pablo Rincon eed0ef6e69 Adding tag keyword support 15 years ago
Victor Julien ca7f54de25 Make sure ICMP unreach packets are not inspected against the flow sgh as it's for the original protocol, not for the ICMP packet. Fixes #174. 15 years ago
Victor Julien b8fec77f37 Fix tcp connections that are reset (RST packet) not always inspecting the reassembled stream. Update transaction id code to make sure both directions of a transaction are inspected before incrementing the inspect_id. 15 years ago
Victor Julien cdc9570f0e Have the detect.alerts counter count actual alerts. 15 years ago
William Metcalf 0e4235cc94 FLOW_DESTROY added to clean-up UT's that init flow 15 years ago
Victor Julien 2f29b8a724 Improve detection of app layer, making sure we only handle app layer on 'established' packets. Should really fix #166. 15 years ago
Victor Julien 37442a8a84 Prefilter signatures before fully scanning them. 15 years ago
Victor Julien d6709b0961 Fix a segv caused by invalidly accessing the smsg_pmq array. 15 years ago
Victor Julien 8cea3779fa Move dce payload inspection to stateful detection engine. 15 years ago
Anoop Saldanha 45ea0d914e dce stub content keywords support using dcepayload.c support for all dce related content keywords 15 years ago
Victor Julien 83b2c8abdb Improve stateful uri detection code. 15 years ago
Victor Julien 9dd753b5f3 Scan uricontent mpm on demand. 15 years ago
Victor Julien e8fce5f7fa Convert uricontent scanning to use the detect engine state. 15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
William Metcalf cc76aa4bc6 properly init flows inside of unit-tests caused lock-up when falling back to using mutex locks 15 years ago
Victor Julien a24f288074 Moving the stream content scanning to have it's own mpm ctx. 15 years ago
Victor Julien 9a08d6c11c Fixes to stream pattern matching. 15 years ago
Victor Julien a0c1209a44 Inspect the reassembled stream together with the packet payload in the same direction. 15 years ago
Victor Julien 81f2499834 Store stream msgs processed by the app layer in the tcp session so they can be inspected by the detection module as well. The detection module returns them to the pool. 15 years ago
Victor Julien c26434fef1 Move flow use cnt to atomic and outside of the flow mutex protection. 15 years ago
Victor Julien 2fd31a1a11 Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory. 15 years ago
Victor Julien dff6795df5 Detect cleanups. 15 years ago
Gerardo Iglesias Galvan 55dfa36963 Add support for http_uri keyword 15 years ago
Jason Ish ea4b7cc33b add profiling to stateful detection engine + other fixups. 15 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 15 years ago
Pablo Rincon 3fa3229e01 ASN1 decoder and keyword implementation 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
15 years ago
Jason Ish 18e5ac8cde Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules. 15 years ago
Victor Julien 42eeb84c9a Properly lock flow before setting IP only action flags. Small alert api cleanups. 15 years ago
Pablo Rincon 9bae6a8628 Moving alert logic to detect-engine-alert.c 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf 8d66323f62 clang fixes for null derefrences 15 years ago
Victor Julien e27cefa6f7 Complete conversion of pattern id mpm storage vs sig id storage. 15 years ago
Victor Julien 46831e0f8f Fix signature grouping bug for protocols without ports. Add debugging code. 15 years ago
Victor Julien 7a427ec7f4 Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach. 15 years ago
Pablo Rincon 46187bfe73 Fix action logic after last pass changes 15 years ago
Gurvinder Singh 3721037de5 unittests for bug 134&139 and some typo correction 15 years ago
Victor Julien a372c1d14e Fix/workaround a strange detection issue. 15 years ago
Victor Julien ce90e87304 Fix failing thresholding unittests 15 years ago
Pablo Rincon e18e2ec998 Changing threshold logic 15 years ago
Pablo Rincon 8bcdf29ab7 Small fix on pass action handling and added more unittests 15 years ago
Pablo Rincon 1238668961 Adding actions order and suport for rule action "pass" 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 15 years ago
Pablo Rincon ab02ab9ead adding http_header keyword support 15 years ago
Pablo Rincon 224a33f19e Moving inline functions to the .h files, so gcc can inline them correctly 15 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 15 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 15 years ago
Gurvinder Singh 69a4fee757 fixed the API and logic error reported by clang tool 15 years ago
Victor Julien 78e15ea7fa Explicitly test for ipv6 in the htp personalities code. Update all affected unittests to set addr family to the flow. 15 years ago
Anoop Saldanha 47037ef9ec fix for bug 115 15 years ago
Victor Julien fe7ece997a Different approach to the reference keyword. Lots of cleanups, bug fixes in reference keyword code and tests. 15 years ago
Breno Silva 89baf93a40 Reference Support 15 years ago
Pablo Rincon e7a989e305 IP Only Engine using radix trees 15 years ago
Anoop Saldanha 97d49d8f5e support for http_client_body keyword 15 years ago
Victor Julien 26e8a0a06a Cleanup global threshold code. 15 years ago
Breno Silva 67f2026279 Global Threshold config 15 years ago
Victor Julien 08600df6b1 Small uri cleanups. 15 years ago
Pablo Rincon c7350a8ac6 Fixing some naming convention issues and incorrect error messages 15 years ago
Pablo Rincon b708d7f65d Adding Uricontent inspection with spm. Modifiers for uricontent are now supported 15 years ago
Anoop Saldanha c54b91ed94 fix for bug 113 15 years ago
Victor Julien 6be0778532 Comment SigMatchSignatures a bit. 15 years ago
Gerardo Iglesias Galvan ef2ae76c42 Add support for detection_filter keyword 16 years ago
Victor Julien 297001c6d9 Only process a app layer sig if it has the proper state. Make sure a sig can't have conflicting sigmatches, such as ftpbouce and uricontent. 16 years ago
Victor Julien ec47f840f3 Remove more scan references. 16 years ago
Victor Julien bee4e04664 More scan/search related cleanups. 16 years ago
Victor Julien 6990d9c91b Fix thresholding signature unittests. Because of the bug fix that made thresholding compatible to ip-only sigs the test sigs needed to be made non-ip-only. 16 years ago
Victor Julien b259e362cd Convert uricontent to use new scanning methods as well. Move http_method and http_cookie keywords out of pmatch list for now. 16 years ago
Victor Julien bef70a04ce First stage of detect engine redesign: equal patterns share id's, search phase no longer used, new match verification phase. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Pablo Rincon ebcbc859e3 Fix on IPOnly match at flows, for inline mode 16 years ago
Gurvinder Singh 50f7d0a887 app layer htp logging and better htp request handling. removed recent_in_tx. 16 years ago
Gurvinder Singh 4768e42159 bug 95 patch 16 years ago
Pablo Rincon 38dc7ffebc Adding settings for detect engine group config 16 years ago
Anoop Saldanha 1a5ee37bd3 Added cuda logs for the engine, which shows device info and memory usage 16 years ago
Anoop Saldanha c26e92733d handle the cuda cleanup at shutdown. should get rid of any errors from the call to SigGroupCleanup 16 years ago
Anoop Saldanha 41e6735b92 mpm b2g cuda support added 16 years ago
Pablo Rincon 34216fd3e2 bug87 Fix IPOnly veredicts on flows 16 years ago
Pablo Rincon 0165b3f0d8 pcre P modifier support (pcre match over http body requests) 16 years ago
Pablo Rincon ad2c136e8f Renaming errors (naming conventions) 16 years ago
Breno Silva 7e299834d2 FragOffset Rule Keyword 16 years ago
Victor Julien 6a53ab9c5a Stream engine memory handling update
The stream engine memory handling needed updating as it didn't scale. Changes:

- pools can now be initialized to size 0, meaning unlimited
- stream engine uses a memcap setting. Sessions, segments and aldata is part
  of this, app layer state isn't.
- memory is accounted using a global int that is spinlocked.
- a counter for sessions that have not been picked up because of memcap was
  added.
- all reassembly errors are converted to debug msgs.
16 years ago
Pablo Rincon 51dc773eec Changing the veredict actions to flags to allow simultaneous veredict 16 years ago
Victor Julien b3bcba077f Only inspect http flows against uri sigs, clean up uri scanning code. 16 years ago
Gurvinder Singh 0cb43d27e9 uricontent new design 16 years ago
Gurvinder Singh 356a8bf385 applayer uri match and modified http handling 16 years ago
Victor Julien 0d34990d7f Add OpenBSD's strlcpy and strlcat and replace all strcat/strcpy/strncat/strncpy by those calls. 16 years ago
Gurvinder Singh f6b0c481b0 urilen support for engine 16 years ago
Victor Julien 4284276b11 Merge applayer detect function into normal match function. Should speed up detection. 16 years ago
Victor Julien bcd0682150 Make engine startup a little less verbose. 16 years ago
Breno Silva 1d055b0e09 ICMP Seq Rule Keyword 16 years ago
Pablo Rincon c80160b96d More examples of unittest helper functions usage reference 16 years ago
Pablo Rincon b6a3395c08 Adding unittest helper functions for building generic packets, checking arrays of expected match results, perform generic tests, etc. Look at util-unittest-helper.c and detect-ipproto.c for references 16 years ago
Victor Julien eb67bb442e Fixup unittests that use buffers that simulate configuration files. They now include the YAML header. 16 years ago
Gerardo Iglesias Galvan 40c514f295 Fix bug in logging msg when using --init-errors-fatal 16 years ago
Gerardo Iglesias Galvan 5eb819b0f4 Add signature line no. to error message when parsing fails 16 years ago
Gerardo Iglesias Galvan fae92f8d7b Fix bug#30. Fix logging call from prev patch 16 years ago
Gerardo Iglesias Galvan 988dc5520b Improve output when loading rules 16 years ago
Victor Julien 6b36e23e45 Fix not decreasing the flow use_cnt reference counter in some cases from the app layer detection code. This caused some streams to never fully time out and thus clutter up the flow table and session pool. 16 years ago
Brian Rectanus c22d42693a Added http_method rule keyword. 16 years ago
Victor Julien b7bac14040 Fixup code to compile with -Wall -Werror -Wextra -Wno-unused-parameter compiler options. 16 years ago
Victor Julien 2b66667a76 Make sure we can't overflow our packet alert storage 16 years ago
Anoop Saldanha f684989f98 dce_iface, dce_opnum, dce_stub_data keyword support 16 years ago
Victor Julien ecab1fae36 Remove contents of VRT classification.config. 16 years ago
Anoop Saldanha 011b74df63 Modify the classification config tests to use the buffer than a temp file and also fix an invalid free 16 years ago
Anoop Saldanha bc4df59414 Support for Classtype keyword and Classification Config file 16 years ago
Victor Julien d5c732f1f9 Add tag keyword stub 16 years ago
Jason Ish ce20c33634 multiline rule support. 16 years ago
Will Metcalf 23aa6cf642 more fixes for exit on sig init failure 16 years ago
Victor Julien 778228d1c5 Flags keyword fix. Fatal init fix. 16 years ago
Victor Julien 35e884f303 Make sure offset modifies depth. 16 years ago
William Metcalf c63b1e0f67 failing unit test depth doesn't take into account offset 16 years ago
Gurvinder Singh 8cfdf6c666 bug 18 patch update 16 years ago
Gurvinder Singh b92886a79a bug#18 and some minor changes 16 years ago
Victor Julien 10cc9d5b6a Add icmp flow handling. 16 years ago
Victor Julien 71ed2d38f5 Fix scan patterns sometimes not being added to the scan ctx. Should fix bug #9. 16 years ago
William Metcalf c40e81d42b failing unittest rules with same content match fail 16 years ago
Victor Julien 4824868766 Application layer detection improvements
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
16 years ago
Pablo Rincon f2f9b83280 Adding FTP app layer parser and ftpbounce detection at L7 16 years ago
Gerardo Iglesias Galvan 7e87f373b9 Add icmp_id keyword support 16 years ago
Victor Julien 01976a0b74 Support for sigs with both pkt and applayer detect
Sets a flowbit with the sig id if the packet matches match. Checks
on that if the app layer matches match. Currently misuses the
flowbits api for this in a way that needs fixing.
16 years ago
Victor Julien 493715c0d2 Implement alert sid storage in the flow so we can check previous alerts in the flow. 16 years ago
Victor Julien 9fd46e9425 Support for sigs with both pkt and applayer detect
Sets a flowbit with the sig id if the packet matches match. Checks
on that if the app layer matches match. Currently misuses the
flowbits api for this in a way that needs fixing.
16 years ago
Victor Julien 4f843ff8e9 Improve matching of packet and app layer sigs. 16 years ago
Pablo Rincon 992aaa3d79 Small fix on SigMatchSignaturesAppLayer() and SigMatchSignatures() 16 years ago
Breno Silva 69eb869cc9 Threshold Rule 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Gurvinder Singh a0f184866c http_cookie keywork support 16 years ago
Pablo Rincon a67bd2457b Loading rules from config support 16 years ago
Pablo Rincon c816af822e Adding support to load rule files from config 16 years ago
Pablo Rincon 1ad6d75dfe Added rpc keyword support at packet level 16 years ago
Pablo Rincon a8d7b71490 First version of flowints 16 years ago
Gerardo Iglesias 991d421394 Changed printf's to logging API functions 16 years ago
Victor Julien 2cfa284999 Fix app layer detect to actually work. 16 years ago
Victor Julien a1d33e2d0b Compilation fixes after merge. 16 years ago
Victor Julien f1f7df0766 First iteration of doing app layer detection. 16 years ago
Anoop Saldanha 7a10ddc07b Fixes for the fast-pattern tests and a couple of other minor changes 16 years ago
Anoop Saldanha 6ca5dbc9e9 Support fast_pattern modifier keyword for content 16 years ago
Victor Julien 07bcc8cc9a Fix signatures with ports and/or addresses but without sigmatches. 16 years ago