Commit Graph

111 Commits (b6e2a6f5255cd98d98f9e5dceb934d1ad5b3e434)

Author SHA1 Message Date
Victor Julien b6e2a6f525 detect: locking update continued
Make DeStateDetectContinueDetection get it's own alstate pointer instead
of using the one that was passed to it. We now get and use it only
inside a flow lock.
11 years ago
Victor Julien cf31e2cc74 detect: locking update
Make DeStateDetectStartDetection get it's own alstate pointer instead
of using the one that was passed to it. We now get and use it only
inside a flow lock.
11 years ago
Victor Julien 6e0112d737 detect: modify AMATCH locking
This is an intrusive change. This patch modifies the way AMATCH
inspection uses locking.

So far, each keyword did it's own locking. This lead to a situation
where a 'alstate' pointer was passed around that was not always
protected by a lock.

This patch moves the locking to the Stateful detection functions.
11 years ago
Victor Julien a96446d39e detect state: fix indent
AMATCH block was indented too far.
11 years ago
Victor Julien 6c8ca76942 detect state: remove alproto check for AMATCH
Not all AMATCHes set a alproto.
11 years ago
Victor Julien c9436a6aef Fix app-layer-protocol FP on multi TX flow
In case of multiple transactions, the stored AMATCH list would not have
been reset, but it would still be reconsidered. Even though none would
match, the engine would still conclude that the rule matched.
11 years ago
Victor Julien 2c3a92a1c9 profiling: conditional rule profiling
Add support for conditional rule profiling. Currently only simple
rate limiting is supported, but hardcoded to inspecting rules for
each packet.
12 years ago
Victor Julien 8dbf7a0d78 Update tests to use AppLayerParserThreadCtx ptr instead of void. Fix a few bugs uncovered by this. 12 years ago
Victor Julien fdefb65be4 app-layer: rename AppLayerThreadCtx funcs
AppLayerParserGetCtxThread -> AppLayerParserThreadCtxAlloc
AppLayerParserDestroyCtxThread -> AppLayerParserThreadCtxFree
12 years ago
Victor Julien f5f148805c app layer: uint16_t alproto -> AppProto alproto
This conversion was missing in a couple of places.
12 years ago
Anoop Saldanha 429c6388f6 App layer API rewritten. The main files in question are:
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].

Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
  parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
  "dnsudp" in the rules.  A user who previously wrote a rule like this -

  "alert dnstcp....." or
  "alert dnsudp....."

  would now have to use,

  alert dns (ipproto:tcp;) or
  alert udp (app-layer-protocol:dns;) or
  alert ip (ipproto:udp; app-layer-protocol:dns;)

  The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
  callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
  STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
  functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0.  This is
  needed by unittests, which would try to clean the flow, and that would
  call the api, AppLayerParserCleanupParserState(), which would try to
  clean the app state, but the app layer now needs an ipproto to figure
  out which api to internally call to clean the state, and if the ipproto
  is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
  they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
12 years ago
Victor Julien 7b687da726 profiling: add tracking of missing keywords 12 years ago
Victor Julien 5686c673ec profiling: per buffer profiling 12 years ago
Victor Julien 97bfcac444 profiling: introduce per keyword profiling
Initial version of per keyword profiling. Prints stats about
how ofter a keyword was checked and what the costs were.
12 years ago
Victor Julien edeeb7ed44 Store TX id with alerts
When generating an alert and storing it in the packet, store the tx_id
as well. This way the output modules can log the tx_id and access the
proper tx for logging.

Issue #904.
12 years ago
Anoop Saldanha c5cd3562d0 Stateful detection inspection continuation API call should update per
signature's Sigmatch entry as well.
12 years ago
Anoop Saldanha 94e40907e2 feature #727 - Add support for app-layer-protocol:<protocol> keyword 12 years ago
Eric Leblond cd3e32ce19 unittests: some functions needs a flow lock.
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
12 years ago
Anoop Saldanha 48cf0585fb Suricata upgrade to libhtp 0.5.x.
Remove the support for now unsupported personalities from libhtp -
TOMCAT_6_0, APACHE and APACHE_2_2.  We instead use the APACHE_2
personality.
12 years ago
Victor Julien 43ba5a677e DNS: enable mpm/fast_pattern support for dns_query 12 years ago
Victor Julien f10dd603ff DNS: adding dns_request content modifier 12 years ago
Eric Leblond 3f107fa130 decode: Packet action start with PACKET
Rename all Packet action macro to have them prefixed by PACKET.
12 years ago
Eric Leblond efaa9a7302 action handling: define and use macros
The action field in Packet structure should not be accessed
directly as the tunneled packet needs to update the root packet
and not the initial packet.

This patch is fixing issue #819 where suricata was not able to
drop fragmented packets in AF_PACKET IPS mode. It also fixes
drop capability for tunneled packets.
12 years ago
Victor Julien 6e18ed0489 luajit flowvar support
This patch adds flowvar support to luajit. It does so by exposing two special
C functions to the luajit scripts: ScFlowvarGet and ScFlowvarSet.
12 years ago
Anoop Saldanha 9219079e1a Allow protocols to have both app layer keywords, as well as transaction
based ones.

Our general logic and assumption is protocols either support one of the
above and not have both.
12 years ago
Anoop Saldanha a490176c8a More lock fixes for the transaction update. Issues reported by Coverity. 12 years ago
Anoop Saldanha 7cf4042337 Fix luajit compilation failure introduced by the transaction update.
Fix coverity lock issues reported by transaction update as well.
12 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
12 years ago
Anoop Saldanha 3511f91bba Add support for the new keyword - http_raw_host header.
The corresponding pcre modifier would be 'Z'.
13 years ago
Anoop Saldanha c4ce19a1be Add support for a new keyword to inspect http_host header.
The corresponding content keyword would now be - http_host.
The corresponding pcre modifier would be W.
13 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Anoop Saldanha fdab6f2ab1 fix flow deadlock issue in detection engine state introduced by tx api.
Issue discovered by coverity.
13 years ago
Anoop Saldanha b99f9fe890 New app inspection engine introduced. Moved existing inspecting engines to use it. 13 years ago
Anoop Saldanha 7b4eac3e8d Change all inspect callbacks to accept TV and a tx_id param. 13 years ago
Anoop Saldanha 10a6e6a3eb Engine cleanup. Remove all old engine inspection and mpm functions. 13 years ago
Anoop Saldanha b0e20a486c update client/server/http_header to use a different form of
buffering/buffer_retrieval.

Now it happens per tx, based on tx id.  Also notice a perf improvement with
this.
13 years ago
Victor Julien 1598425a40 detect: properly store a stateful match if it happens at the start of inspection 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien ec7e79c748 Rule profiling update
- Remove usage of counters api.
- Store stats in detect engine thread ctx to remove locking
- Support rule reloads
13 years ago
Victor Julien 8f71333e12 file: implement filesize keyword. #489. 13 years ago
Victor Julien 4cde2355bd Simplify flow resetting on de_ctx update. Detect ctx id starts at 1. So in a flow 0 means uninitialized (thus set) and if we detect flow is not equal to detect id, we reset the sgh storage and de_state. 13 years ago
Anoop Saldanha ecad4a24fa live rule support added
To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup
13 years ago
Anoop Saldanha 988c92f71c http user agent keyword + mpm + inspection + fast pattern support added 13 years ago
Victor Julien 19a7e7f395 flow: create a flow lock macro API, implement it for mutex and rwlocks. Mutex remains the default. 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Victor Julien 5ba41c7890 Fix locking error in filestore handling. Add debug validate check for asserting a flow is locked. 14 years ago
Anoop Saldanha 09313cf9bd Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S 14 years ago
Anoop Saldanha 2007c2711c Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword 14 years ago
Anoop Saldanha 30247dce8c bug 389 - support http response header inspection + fix bug with stateful inspection for sigs that would have both request/response inpection 14 years ago
Victor Julien 026a4efc57 Make sure that continued stateful detection only inspects sigs in the proper direction. 14 years ago