Commit Graph

45 Commits (1fb7c0dddc97b20c0ac03d6810f4e76bce523a57)

Author SHA1 Message Date
Anoop Saldanha 429c6388f6 App layer API rewritten. The main files in question are:
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].

Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
  parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
  "dnsudp" in the rules.  A user who previously wrote a rule like this -

  "alert dnstcp....." or
  "alert dnsudp....."

  would now have to use,

  alert dns (ipproto:tcp;) or
  alert udp (app-layer-protocol:dns;) or
  alert ip (ipproto:udp; app-layer-protocol:dns;)

  The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
  callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
  STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
  functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0.  This is
  needed by unittests, which would try to clean the flow, and that would
  call the api, AppLayerParserCleanupParserState(), which would try to
  clean the app state, but the app layer now needs an ipproto to figure
  out which api to internally call to clean the state, and if the ipproto
  is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
  they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
12 years ago
Ken Steele d12834769a Add const for Packet * in flow functions.
By moving FlowReference() out of FlowGetFlowFromHash() and into the one
function that calls it, all the flow functions take const Packet * instead
of Packet *.
12 years ago
Ken Steele c1b9f0e1f4 Formatting and comment updates in flow files
Some reformatting to meet coding standards.

Added a few comments to make it more clear where p->flow gets set.
12 years ago
Eric Leblond 4db2fc2cbb Add per-flow generic storage
This patch adds a per-flow storage that can be created via the functions
available in flow-storage.c.
12 years ago
Victor Julien 58ed1f2411 flow: take vlan_id's into account in the flow hash
In VLAN we can have 2 layers of encapsulation. In this patch both
layers are used in the flow hash to distinguish between encapsulated
traffic.
12 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Eric Leblond 0227a87fcb cleaning: fix warning when building with clang.
clang was issuing some warnings related to unused return in function.
This patch adds some needed error treatment and ignore the rest of the
warnings by adding a cast to void.
13 years ago
Victor Julien 4157d9408d Various small flow and host table fixes. 14 years ago
Victor Julien 0150e66ede flow engine: improve scalability
Major redesign of the flow engine. Remove the flow queues that turned
out to be major choke points when using many threads. Flow manager now
walks the hash table directly. Simplify the way we get a new flow in
case of emergency.
14 years ago
Eric Leblond 5a63662766 Flow: use condition system instead of short sleep
Short sleep can lead to some really annoying performance issue in
some environnement like virtual systems. This technic was used in
the flow manager. This patch uses an alternate approach based on
a timed condition which is triggered each time a new flow has to
be created. This avoid to run out of flow. A counter is also done
to be able not to run the cleaning code at each new flow.
14 years ago
Victor Julien 262a7300d7 flow: shrink Flow datatype
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.

Add macro's to check the family, copy the address, etc.

Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.

On 64 bit this shrinks the flow with 8 bytes.
14 years ago
Victor Julien 06904c9024 App Layer cleanup
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
14 years ago
Victor Julien 9baa16af63 Convert flow memcap to u64. Bug #332. 14 years ago
Anoop Saldanha 49e2b580cb Removed FLOW_AL_PROTO_UNKNOWN. We don't need this flag 14 years ago
Eric Leblond 674b0bfae7 flow: Add basic SCTP support
This patch adds a basic flow support to SCTP. SCTP specificities
like the verification tag are not taken into account.
15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
Victor Julien c26434fef1 Move flow use cnt to atomic and outside of the flow mutex protection. 15 years ago
Victor Julien 87345e5c60 Switch flow memuse counter to the atomic api. 15 years ago
Victor Julien 3484e2abde Fix flow engine memory handling. 15 years ago
Pablo Rincon b491ef2be0 Removing FlowAllocDirect since it's not needed anymore 15 years ago
Victor Julien 32e3fea9e6 Exclude parts of a flow that are not changing after init from the flow mutex. Cleanup flow-hash function. 15 years ago
Pablo Rincon 29a6fc2f03 Adding some flow improvements and recovery on emergency mode 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 38f7479af2 Make sure we set the address family (AF_INET & AF_INET6) in the flow's address structures. Needed by HTP personalities code. 15 years ago
Victor Julien 5e3f072fa6 Clean up flow mutexes after use. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Victor Julien 6b36e23e45 Fix not decreasing the flow use_cnt reference counter in some cases from the app layer detection code. This caused some streams to never fully time out and thus clutter up the flow table and session pool. 16 years ago
Victor Julien 10cc9d5b6a Add icmp flow handling. 16 years ago
Victor Julien 4824868766 Application layer detection improvements
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Victor Julien b102ea2123 Big update:
- Implement "closing" state in flow.
- Add protocol specific timeouts.
- Lots of stream tracking updates, fixing a lot of out of window issues.
- Stream reassembly fixes.
- Implement a new IDS runmode with 4 stream and detect threads.
- Added a BUG_ON macro that aborts the engine if the expression is true.
- Better balance the flow queue handler for traffic that doesn't have flow (like icmp currently).
- Simplify application level protocol in the Tcp Session.
- Add some debugging memory counters.
16 years ago
Victor Julien b4f0e82463 Small flow updates. 16 years ago
Gurvinder Singh e3b76dcaad efficient protocol specific timeouts 16 years ago
Gurvinder Singh b46d9dc2b4 part2 and part3.1 of timout task 16 years ago
Gurvinder Singh 59e2232211 setting timeout on first packet in the flow 16 years ago
Gurvinder Singh a31a261737 Another and right way i guess for timeouts 16 years ago
Gurvinder Singh 0e64fa78b4 Protocol Specific Timeouts 16 years ago
Brian Rectanus fa5939ca91 64 bit cleanup part2 16 years ago
Victor Julien ff4b5a5db7 Add support for flowbits. 16 years ago
Victor Julien efb10fc0d6 big update 16 years ago
Victor Julien bab4b62376 Initial add of the files. 16 years ago