Commit Graph

43 Commits (84696ebe2a67339f654c21a0e8564b03590aef14)

Author SHA1 Message Date
Victor Julien 84696ebe2a stream: configurable stream chunk prealloc
The stream chunk pool contains preallocating stream chunks (StreamMsg).
These are used for raw reassembly, used in raw content inspection by
the detection engine. The default setting so far has been 250, which
was hardcoded. This meant that in setups that needed more, allocs and
frees would be happen constantly.

This patch introduces a yaml option to set the 'prealloc' value in the
pool. The default is still 250.

stream.reassembly.chunk-prealloc

Related to feature #1093.
12 years ago
Victor Julien 4049c2f74c Packet logging API: convert unified2
Convert unified2 alert to new logging API.
12 years ago
Victor Julien bf6ab333ff stream: use reassembly.memcap for stream chunks
Use the stream.reassembly.memcap for stream chunks (StreaMsg) as well.
12 years ago
Victor Julien b159c1714c stream: remove flow reference from StreamMsg
StreamMsg' flow reference was used mostly to make sure a flow would
not get removed from the hash before inspection. For this it needed
to reference the flow use_cnt reference counter. Nowadays we have
more advanced flow timeout handling. This will make sure that if
there still are pending smsgs' in a flow, these will still be
processed.
12 years ago
Ken Steele 5532af4621 Create SCMUTEX_INITIALIZER to abstract out PTHREAD_MUTEX_INITIALIZER
This allows replacing pthread mutexes with other types of mutex.
12 years ago
Victor Julien 00948882e7 Suppress warnings when StreamSegmentForEach is called for UDP or SCTP, unless debug is compiled in. 13 years ago
Anoop Saldanha 67981d1c5c Update suricata to use FlowReference/FlowDeReference for the ones left out
from last update.
13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien 20d2db085e reintroduce pool free func for cases where block alloc is not used. 13 years ago
Eric Leblond fa079c1da0 pool: realize a block allocation for preallocated item.
This patch required a evolution of Pool API as it is needed to
proceed to alloc or init separetely. The PoolInit has been changed
with a new Init function parameter.
13 years ago
Victor Julien ff8755af5c Make sure stream debug code is only used in debug mode. 14 years ago
Victor Julien 0150e66ede flow engine: improve scalability
Major redesign of the flow engine. Remove the flow queues that turned
out to be major choke points when using many threads. Flow manager now
walks the hash table directly. Simplify the way we get a new flow in
case of emergency.
14 years ago
Eric Leblond 628bfcc1b9 stream: Change return of StreamSegmentForEach
The function now returns the number of segment where the callback
has ben runned successfully.
14 years ago
Eric Leblond 4f0cdf28a3 Introduce StreamSegmentForEach function
This patch introduces a function called StreamMsgForEach which
can be used to run a callback on all segments of a stream. This
is currently only supported for TCP as this is the only streaming
aware protocol.
14 years ago
Victor Julien 4f5aad1476 Enforce configurable minimum chunk size in raw stream reassembly. Minor stream cleanups, unittest updates. 15 years ago
Victor Julien 936b34ddf6 Remove minimum init chunk length code, set a default limit of 2560 to the minimum chunk size, allow toclient raw reassembly to start even if toserver hasn't started yet. 15 years ago
Victor Julien 864c8718e1 Store matching stream msg (ptr) in packets alert structure so it's available to the output plugins. 15 years ago
Victor Julien 61a9936d55 Inspect a pseudo packet upon receiving a RST so that we are sure both sides of the TCP session are inspected. 15 years ago
Victor Julien a3ff0e7210 Don't scan TCP packet payload if it was added to the stream. Inspect the tcp stream with the correct packet. Should fix #184 and #185. 15 years ago
Victor Julien ba12f3c109 Applayer to flow fixes and cleanups. 15 years ago
Victor Julien a0c1209a44 Inspect the reassembled stream together with the packet payload in the same direction. 15 years ago
Victor Julien 81f2499834 Store stream msgs processed by the app layer in the tcp session so they can be inspected by the detection module as well. The detection module returns them to the pool. 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 16 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 16 years ago
Gurvinder Singh 8e444f1772 stream and application layer improvements 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Victor Julien bcc5bbef93 Yet more logging api usage changes. 16 years ago
Victor Julien 91bc83e5c6 More logging API usage changes. 16 years ago
Victor Julien 3a28171fbd Another round of logging api usage updates. 16 years ago
Victor Julien be3bbe0a85 Fix segv in reassembly. Fix sequence gap handling tests. 16 years ago
Victor Julien 5ecd187b6f Tie app layer parsing to the stream engine. 16 years ago
Victor Julien edfddcb282 Clean up stream tests memory handling. Remove counters in the address handling that were thread unsafe. 16 years ago
Victor Julien b102ea2123 Big update:
- Implement "closing" state in flow.
- Add protocol specific timeouts.
- Lots of stream tracking updates, fixing a lot of out of window issues.
- Stream reassembly fixes.
- Implement a new IDS runmode with 4 stream and detect threads.
- Added a BUG_ON macro that aborts the engine if the expression is true.
- Better balance the flow queue handler for traffic that doesn't have flow (like icmp currently).
- Simplify application level protocol in the Tcp Session.
- Add some debugging memory counters.
16 years ago
Victor Julien 84da1e9dc4 Add seg_list integrity testing to reassemly. Remove all debug output but some. Better deal with packets before the point that we already reassembled. 16 years ago
Brian Rectanus fa5939ca91 64 bit cleanup part2 16 years ago
Victor Julien 54ae12b1f9 Fixes for the stream reassembly. It turned out that using both a prev_seg pointer and a list_seg->prev pointer at the same time was not the best of ideas. So removed the prev_seg ptr. Cleaned up some copy functions too. Added some more debug statements. Made sure the L7 stuff doesn't kick in when running the unittests for reassembly. 16 years ago
Victor Julien 1f995361dd Switch to pthread_cond_timedwait in streammsg queue. 16 years ago
Victor Julien c957dc7775 Updates & cleanups to stream & l7 stuff 16 years ago
Victor Julien 1c2240cfeb Stream reassembly update and WIP code for L7 modules. 16 years ago