Commit Graph

330 Commits (1a2ad059a12910120cd997e76ea75e688747ff7d)

Author SHA1 Message Date
Victor Julien 67989e7e4e rule parsing: reduce mallocs and clean up
Reduce mallocs during rule parsing. Also, no longer recursively
call the option parse function.
13 years ago
Victor Julien e7f6107e79 signature address parsing improvements and tests
Fix sigatures not supporting [10.0.0.0/24, !10.1.1.1] notation when
used directly in a rule instead of through a variable.

Add tests for Bugs #815 and #920.
13 years ago
Anoop Saldanha d0c5f51293 Update rule engine relationship with regard to setting ip protocol between specifying protocol after action, ip_proto and app-layer-protocol.
Now we can specify alproto, ip_proto combinations this way

alert dns (ip_proto:[tcp/udp];)
alert ip (app-layer-protocol:dns;)
alert ip (app-layer-protocol:dns; ip_proto:tcp;)
alert tcp (app-layer-protocol:dns:)

so on.  Neater than using dnstcp/dnsudp.

This is related to feature #424.
13 years ago
Anoop Saldanha f592c481dc Introduce a separate inspection engine for app events. 13 years ago
Anoop Saldanha b1dffdfbe0 Add app layer protocol packet event detection support. 13 years ago
Anoop Saldanha 1077acecd7 validate dns sigs that are reported as plain dns and not dnsudp or dnstcp. 13 years ago
Anoop Saldanha 94e40907e2 feature #727 - Add support for app-layer-protocol:<protocol> keyword 13 years ago
Anoop Saldanha ddde572fba Introduce new options into the conf file to enable/disable -
1. Proto detection
2. Parsers

For app layer protocols.

libhtp has now been moved to the section under app-layer.protocols.http,
but we still provide backward compatibility with older conf files.
13 years ago
Anoop Saldanha d9686fae57 Now supports accepting port addresses as strings, like the ones accepted in our rules. As a consequence we now accept port range, and other such combination. Support PP for ports based on ipproto as well. 13 years ago
Victor Julien 4827a4dcef Coverity 400477: pcre_get_substring retval
Add missing return code check to pcre_get_substring call.
13 years ago
Eric Leblond c5bd04f102 unittest: recycle packet before exit
To avoid an issue with flow validation, we need to recycle the packet
before cleaning the flow.
13 years ago
Anoop Saldanha 48cf0585fb Suricata upgrade to libhtp 0.5.x.
Remove the support for now unsupported personalities from libhtp -
TOMCAT_6_0, APACHE and APACHE_2_2.  We instead use the APACHE_2
personality.
13 years ago
Victor Julien 73e27c1fb7 Generate proper errors if sid,gid,rev values are out of range. Bug #779. 13 years ago
Victor Julien 7292998a58 Content: set up sticky buffers like file_data and dce_stub_data w/o flags, but with a list variable 13 years ago
Victor Julien f10dd603ff DNS: adding dns_request content modifier 13 years ago
Victor Julien 59780ca770 Hacks to enable alert dns even though we have dnstcp and dnsudp parsers. Needs proper solution later. 13 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
13 years ago
Victor Julien 19511cda97 Remove obsolete DetectParseContentString function, it has been replaced by DetectContentDataParse 13 years ago
Victor Julien 4165de4771 Minor SigValidate cleanup 14 years ago
Anoop Saldanha 0d7305dfc7 Update the way we handle http_host keywords.
Previously we would have forced all users to use nocase with http_host
keywords(since the hostname buffer is lowercase).

We now error out on sigs that has nocase set with http_host set.  Also if
the http_host pattern or http_host pcre has an uppercase character set, we
invalidate such sigs.  Unittests also updated to reflect the above change.
14 years ago
Anoop Saldanha 4c6efa2d40 Update content id assignment.
All fp id assignment now happens in one go.
Also noticing a slight perf increase, probably emanating from improved cache
perf.
Removed irrelevant unittests as well.
14 years ago
Anoop Saldanha 60be1751d5 Figure out sig fp during validation stage, instead of staging stage. 14 years ago
Anoop Saldanha f8ae53ac02 Further customize content modifier buffer registration.
Allow modifier setups functions to have CustomCallbacks to enable their
internal conditions.
14 years ago
Anoop Saldanha a304a98d1d http_* setup unified. 14 years ago
Anoop Saldanha 0b5d277254 code cleanup for all content based keywords. 14 years ago
Anoop Saldanha a308d718ae Allow the use of relative without the presence of a related previous keyword. 14 years ago
Anoop Saldanha 3511f91bba Add support for the new keyword - http_raw_host header.
The corresponding pcre modifier would be 'Z'.
14 years ago
Anoop Saldanha c4ce19a1be Add support for a new keyword to inspect http_host header.
The corresponding content keyword would now be - http_host.
The corresponding pcre modifier would be W.
14 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
14 years ago
Victor Julien 84bad6db77 Silence compiler warnings found by clang 14 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
14 years ago
pi-rho 0df4c5838d spelling corrections documented in redmine bug#533 14 years ago
Victor Julien e0bfcb7dde Only set SIG_FLAG_REQUIRE_STREAM if signature inspects TCP. 14 years ago
Anoop Saldanha bf6cd48259 if a sig's set as stream sig only, don't updated it as both stream and pkt sig if offset/depth's present
bug #495 - update rule analyzer to not warn on offset_depth-tcp_pkt update if sig is stream only

bug #497 - rule_warnings fixed
14 years ago
Anoop Saldanha 960d421f9d Update SigValidate() to allow http keywords to be specified in the right flow direction 14 years ago
Victor Julien ab421978f0 Free all sig match structs when freeing a signature. 14 years ago
Victor Julien af97c36c08 Properly clean signature's ip only data. 14 years ago
Victor Julien 19e3348cae Fix a reload memleak in the duplicate sig detection hash. 14 years ago
Anoop Saldanha 0d602d9cde we now support offset, depth inspection against all packet payloads and stream messages 14 years ago
Anoop Saldanha a34f91358d tests to highlight that
- suricata treates sigs with offset/depth without any packet keywords as stream sigs
- as a consequence suricata will FN on such sigs

The tests introduced here will fail, displaying the issues.  The
next patch in the series would fix the said issues.
14 years ago
Anoop Saldanha 988c92f71c http user agent keyword + mpm + inspection + fast pattern support added 14 years ago
Victor Julien 073ce64786 Fix parsing of tcp-pkt and tcp-stream sigs, add unittest. 15 years ago
Eileen Donlon da633d490b fix misleading comment 15 years ago
Victor Julien da3c5bf84d Minor error message cleanups 15 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 15 years ago
Anoop Saldanha 4810ee9c5f All uricontent modified patterns now are DETECT_CONTENT and not DETECT_URICONTENT. Step towards unifying all content based patterns. Makes way for easier management of patterns 15 years ago
Anoop Saldanha 93d7a6e671 code cleanup. Remove unused functions 15 years ago
Anoop Saldanha eb07c345b8 code cleanup - replace SigMatchAppendThreshold with SigMatchAppendSMToList 15 years ago
Anoop Saldanha dd7e710f35 code cleanup - replace SigMatchAppendPostMatch with SigMatchAppendSMToList 15 years ago
Anoop Saldanha a4638fb0ad code cleanup - replace SigMatchAppendPacket with SigMatchAppendSMToList 15 years ago
Anoop Saldanha ff38d42bf1 code cleanup - replace SigMatchAppendTag with SigMatchAppendSMToList 15 years ago
Anoop Saldanha ac68c3f893 code cleanup - replace SigMatchAppendDcePayload with SigMatchAppendSMToList 15 years ago
Anoop Saldanha 6cab663bf0 code cleanup - replace SigMatchAppendPayload with SigMatchAppendSMToList 15 years ago
Anoop Saldanha c4cb37b8da code cleanup - replace SigMatchAppendUricontent with SigMatchAppendSMToList 15 years ago
Anoop Saldanha c9af50ea0c code cleanup - replace SigMatchAppendAppLayer with SigMatchAppendSMToList 15 years ago
Anoop Saldanha bbb9f35f26 code cleanup - replace SigMatchGetLastSM with SigMatchGetLastSMFromLists 15 years ago
Anoop Saldanha 09313cf9bd Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S 15 years ago
Anoop Saldanha 2007c2711c Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword 15 years ago
Anoop Saldanha 3b5d95547d bug #412 - Remove the commented out SigInitReal() 15 years ago
Anoop Saldanha 6cbd3a1046 bug #412 - Unify SigInit() and SigInitReal(). Remove any use of SigInitReal() 15 years ago
Anoop Saldanha 88ad3691d1 bug #405 - fix bug where raw uri inspection sigs were not treated as stateful sigs 15 years ago
Victor Julien c8c4a76dc6 Move threshold to it's own sig match list. 15 years ago
Victor Julien 02e1229565 Enforce flow direction for http_raw_header sigs. Fix unittests that missed the flow direction. 15 years ago
Victor Julien e81f94cd83 Fix flowbits sigmatch structure added to the match and post-match list, causing corruption of the prev ptr. This lead to an endless loop condition in the thresholding code. Thanks to Chris Wakelin for reporting the issue. 15 years ago
Victor Julien 6eeab37ab3 Add post-match list, move flowbits set, etc functions to it. Move flowint set, etc functions to it as well. 15 years ago
Victor Julien aac2d91bcc Set DROP flag for reject action so in addition to sending the rst, in IPS mode also drop the offending packet. 15 years ago
Victor Julien e55390e4e7 Add check to invalidate signatures that inspect raw http headers in the to_client direction (response headers) if libhtp hasn't been patched yet. Also add hack to disable the test for unittests, many tests fail and we'll fix those ASAP. 15 years ago
Victor Julien 21ee59e6f3 Add signature direction (flow:toserver/flow:toclient) as a signature flag. 15 years ago
Victor Julien 298289f43f Let flow:only_stream and flow:no_stream set the require packet and require stream flags. Toss out sigs with conflicting settings. Rename flow:stream_only to flow:only_stream. Fixes #261. 15 years ago
Victor Julien e0cf2ccb91 Fix invalid direction error message. 15 years ago
Victor Julien 07e560b137 file-data: initial file_data support
Support file_data for: content, pcre (relative), byte_test, byte_jump,
byte_extract, isdataat.

File_data support is handled at signature parsing time, all matches
occurring after the file_data in the rule are converted to http_server_body
matches.

Content matches relative to the file_data are converted. Within to depth,
distance to offset. Relative to the start of the body buffer.
15 years ago
Victor Julien d5ed28b065 Remove SIG_FLAG_MPM flag. 15 years ago
Victor Julien 4992f7c417 Remove SIG_FLAG_MPM_URI flag. It was checked but never set. 15 years ago
Victor Julien 2650551192 Rename signature init flags to indicate they are init flags. 15 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword.
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
15 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 15 years ago
Victor Julien 70f0d3d2e7 Add negation to filename and fileext, use same syntax as with content. 15 years ago
Eileen Donlon a92d15ed37 Fixed duplicate signature check 15 years ago
Anoop Saldanha 7433d92dd2 undo this commit -
commit eff08f93d8
Author: Anoop Saldanha <poonaatsoc@gmail.com>
Date:   Thu Nov 3 14:31:24 2011 +0530

    update failing unittest to reflect the mpm design update

Fixed a bug in the mpm code that would make all the changes in the commit just undone wrong.
15 years ago
Victor Julien d070869c48 Reinstate replace validation check. 15 years ago
Anoop Saldanha eff08f93d8 update failing unittest to reflect the mpm design update 15 years ago
Victor Julien af51493da2 Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm. 15 years ago
Anoop Saldanha 9887084370 support multiple ipprotos in the same sig + unittest 15 years ago
Anoop Saldanha 432c3317d2 app layer probing parser updates 15 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 15 years ago
Eric Leblond a85dc9b0e2 Add support for replace keyword.
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.

One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.

Replace is only activated in IPS mode and apply only to raw match.
15 years ago
Gerardo Iglesias Galvan 44692c83aa Properly check retval for config and conversion function calls 15 years ago
Gerardo Iglesias Galvan 73dd5562c3 Fix potential crash in signature parsing code 15 years ago
Victor Julien 681f8329a6 Make error on <- direction operation use more explicit. 16 years ago
Anoop Saldanha 966119b6aa support for http_raw_uri keyword + mpm engine 16 years ago
Victor Julien b9fd978253 Fix compiler warnings in two unittests. 16 years ago
Victor Julien 153f9298e7 Fix priority handling during the signature parsing stage. Fixes #275. 16 years ago
Gurvinder Singh 27f67c97de log error on duplicate sig and also for dup sig with newer revision 16 years ago
Eric Leblond 2c80f18dc9 detect: Add sctp detection and parsing.
This patch adds the support of SCTP in signature subsystem.
16 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 16 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 16 years ago
Anoop Saldanha 2321a4dd58 support isdataat negation. Also fix addiing isdataat to appropriate lists 16 years ago
Eric Leblond 1db4aadd16 Supress usage of Packet declaration in tests.
For convenience, a massive usage of 'Packet p;' declaration has
been done in the tests function. Although this was completely
legal, this is not possible anymore because of the new Packet
allocation structure. This massive patch modifies all suricata
files to use a SCMalloc allocated pointer to Packet instead.

This patch has been done using coccinelle (http://coccinelle.lip6.fr)
which is a semantic patching tool. This ensures that things like call
to SCFree() should have not been forget because the semantic patch
explicitly forces the call to SCFree(p) before each return. With this
patch all unittests are running fine with a small and a big default
packet size.
16 years ago
Anoop Saldanha 2b781f00d7 support relative pcre for client body. All pcre processing for client body moved to hcbd engine 16 years ago
Victor Julien 435d0fb327 Clean up signature flags creating room for merging flags and mpm_flags. Merge flags and mpm_flags. Move new mpm id's into signature header. Get rid of full signature access in signature prefiltering. 16 years ago
Victor Julien 169aa5581a Rename SIG_FLAG_AMATCH flag to SIG_FLAG_STATE_MATCH to better reflects its purpose. 16 years ago
Victor Julien d7b92d9bfe Consolidate several signature flags into one. 16 years ago
Anoop Saldanha 07f20674ac allow sigs for http client body of the form content:one; content:two; distance:0; http_client_body; 16 years ago
Anoop Saldanha 5c6a65dc58 support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch] 16 years ago
Anoop Saldanha 4883efd0f6 unifying content structure - uricontent now uses DetectContentData 16 years ago
Anoop Saldanha 6eaba8941c Use new flags to indicate uricontent has a mpm set 16 years ago
Anoop Saldanha 46b4806d8e use a single populatempm() function to add the right content for mpm 16 years ago
Anoop Saldanha 4a038511ff Change the struct members uricontent and uricontent_len in DetectUricontentData to content and content_len. Make replacements everywhere else in the codebase to accomodate these changes 16 years ago
Anoop Saldanha ede7be34b5 replace all Signature->tmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_TMATCH] 16 years ago
Anoop Saldanha 3d2f81d978 replace all Signature->dmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_DMATCH] 16 years ago
Anoop Saldanha a7353be20d replace all Signature->amatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_AMATCH] 16 years ago
Anoop Saldanha e0476242c6 replace all Signature->umatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_UMATCH] 16 years ago
Anoop Saldanha e54358a9e1 replace all Signature->pmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_PMATCH] 16 years ago
Anoop Saldanha 82fd581b64 replace all sm lists (match, pmatch, dmatch, umatch, amatch, tmatch) with an array Signature->sm_lists[]. Replace all Signature->match instances in the engine with Signature->sm_lists[DETECT_SM_LIST_MATCH] 16 years ago
Victor Julien afdb39e5f6 Print an error if the protocol field of a signature contains a unknown/invalid value. 16 years ago
Anoop Saldanha 88d94b136d Support for reference.config file 16 years ago
Anoop Saldanha fe700737a3 fix null dereference in detect parse test - clang fix 16 years ago
Anoop Saldanha f094523eb1 clang fix - some minor fixes for unittests 16 years ago
Pablo Rincon bbab0f9987 Set default gid to 1 on Sig init 16 years ago
Victor Julien c62a3d995e Fix signatures with trailing spaces being rejected by the regex. Add test. 16 years ago
Victor Julien 05ae4f99d8 Kick out invalid signature with uricontent and flow:to_client or flow:from_server. 16 years ago
Pablo Rincon c6e090f72c App layer proto specific sigs (use the app layer to match proto) 16 years ago
Victor Julien 102092a89c Make signature address matching more cache efficient. 16 years ago
Anoop Saldanha 89e3d92cdb fix creating a static array of length 0 in SigMatchGetLastSMFromLists - clang fix 16 years ago
Pablo Rincon 693d4f54eb Load signatures with incompatible fast_pattern option (due to design differences for optimization) 16 years ago
Anoop Saldanha 673322f01f unittests for dce_stub_data content based singature parsing + fixes 16 years ago
Anoop Saldanha ce8d27425d fix signature parsing to how snort does it for content based keywords along with dce_stub_data 16 years ago
Anoop Saldanha 5fb6981e9e content handling changes in detect-engine-payload.c for multiple relative matches 16 years ago
Victor Julien 393acd77d2 Detection improvements: uricontent escaping now working, better negated pattern (content) handling. 16 years ago
Victor Julien 7454336ef5 Make SigWrapper private to detect-parse.c and rename to SigDuplWrapper to reflect it's use and purpose. 16 years ago
Anoop Saldanha 9ecade76b9 in case of duplicate signatures used the one with the latest revision 16 years ago
Pablo Rincon b931895901 Fixing flow cleanup and ctx initialization 16 years ago
Pablo Rincon eed0ef6e69 Adding tag keyword support 16 years ago
Victor Julien 37442a8a84 Prefilter signatures before fully scanning them. 16 years ago
Anoop Saldanha 45ea0d914e dce stub content keywords support using dcepayload.c support for all dce related content keywords 16 years ago
Victor Julien 0a607fce3d Finish http_uri keyword, fix invalid read issue in one of the tests. 16 years ago
Gurvinder Singh cda664a8c4 memroy leaks fixes in detection module, app layer and counters 16 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
16 years ago
Victor Julien 7a427ec7f4 Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach. 16 years ago
Gurvinder Singh 3721037de5 unittests for bug 134&139 and some typo correction 16 years ago
William Metcalf 5fb405335e Small wrapper fixes to allow for windows compilation 16 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 16 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 16 years ago
Victor Julien 565b44a7e5 Improve sig parsing unittest error handling. 17 years ago
Gurvinder Singh 69a4fee757 fixed the API and logic error reported by clang tool 17 years ago
Victor Julien fe7ece997a Different approach to the reference keyword. Lots of cleanups, bug fixes in reference keyword code and tests. 17 years ago
Breno Silva 89baf93a40 Reference Support 17 years ago
Victor Julien 01c0e316b9 Cleanup of libnet patch. 17 years ago
William Metcalf 3678dda185 libnet now optional 17 years ago
Pablo Rincon e7a989e305 IP Only Engine using radix trees 17 years ago