Anoop Saldanha
51c9955c79
fix for bug #577 .
...
If a pattern has matched on mpm, don't re-inspect it later, subject to certain
conditions met by the pattern - namely, not negated, right chop, no replacet
attached to it.
13 years ago
Eric Leblond
e176be6fcc
Use unlikely for error treatment.
...
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Anoop Saldanha
21f92c0a89
Give priority to non stream content over stream content when selecting fast
...
pattern.
13 years ago
Anoop Saldanha
64fad5b36e
Update fast_pattern engine to not use negated content as fast_pattern if we have non-negated content in the sig.
...
Noticing a good spike in perf with et_pro ruleset.
Thanks to Will Metcalf for the suggestion.
13 years ago
Anoop Saldanha
0eaf0b0129
mpm engine and ac mem free fixes
13 years ago
Anoop Saldanha
f4ce9011d2
make mpm ctx container de_ctx specific. Also introduce global variable in mpm_ctx. this is a workaround for cleaning non global mpm_ctx's since we now don't supply the de_ctx around the detection engine API
13 years ago
Anoop Saldanha
fc15cc7de1
some more mpm engine cleanup
13 years ago
Anoop Saldanha
f9612f3b83
mpm engine cleanup. Remove unnecessary flags
13 years ago
Anoop Saldanha
0d602d9cde
we now support offset, depth inspection against all packet payloads and stream messages
13 years ago
Anoop Saldanha
db8500bb26
fast pattern cleanup - Remove FastPatternSupportEnabledForSigMatchList() and all it's associated structures
13 years ago
Anoop Saldanha
988c92f71c
http user agent keyword + mpm + inspection + fast pattern support added
13 years ago
Anoop Saldanha
2995867328
b2g cuda up, compiling and running
14 years ago
Victor Julien
28d88746e4
Fix compiler warning and silence complaining unittests.
14 years ago
Victor Julien
8e48a2edfd
Fix NULL dereference in PacketPatternSearchWithStreamCtx code.
14 years ago
Anoop Saldanha
4810ee9c5f
All uricontent modified patterns now are DETECT_CONTENT and not DETECT_URICONTENT. Step towards unifying all content based patterns. Makes way for easier management of patterns
14 years ago
Anoop Saldanha
dcb2afb02f
Use sm_list to differentiate between different content types while retrieving pattern ids instead of sm_type
14 years ago
Anoop Saldanha
83d9439877
DetectPatternGetId() cleanup. Remove separate search element creation for uricontent. We don't need this now since we have unified content structures for content and uricontent
14 years ago
Anoop Saldanha
0677190960
rebase commit for hscd and hsmd patches
14 years ago
Anoop Saldanha
09313cf9bd
Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S
14 years ago
Anoop Saldanha
2007c2711c
Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword
14 years ago
Anoop Saldanha
9a665e035b
code cleanup over last 2 commits
14 years ago
Anoop Saldanha
55c4e419fd
if a signature is non-tcp, it's always a packet sig
14 years ago
Anoop Saldanha
419cdc8558
support splitting mpm ctxs based on direction v2
14 years ago
Anoop Saldanha
42bc22cfa5
indendation fix
14 years ago
Anoop Saldanha
ecc7a769a7
reclaim mpm contexts if no patterns are added to it, even in non-full mode
14 years ago
Anoop Saldanha
1389cf6913
update cuda mpm to support per proto mpm contexts. Fix faulty stream mpm usage of cuda
14 years ago
Anoop Saldanha
92643f6110
introduce separate mpm ctxs for tcp/udp/other_protos
14 years ago
Anoop Saldanha
a5dec3cb2e
refactor all http mpm engine code
14 years ago
Anoop Saldanha
34cf557abf
fix indentation
14 years ago
Anoop Saldanha
5b91cec4ae
remove unnecessary if/else checks
14 years ago
Victor Julien
dd9da1a56f
Merge all http mpm related signature flags into a single set: SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_HTTP_NEG.
14 years ago
Victor Julien
d5ed28b065
Remove SIG_FLAG_MPM flag.
14 years ago
Victor Julien
fe48920514
Remove per sgh mpm_streamcontent_maxlen variable. It was checked but never set.
14 years ago
Victor Julien
291ddd95f2
Detection engine -- mpm
...
Each signature is in one mpm ctx at max, but there were 3 separate
id's in use: packet, stream, http. Merged them all into one.
Could shrink the SignatureHeader structure with 8 bytes because of this,
should lead to better caching performance.
14 years ago
Victor Julien
89f83e714c
Introduce http_server_body keyword.
...
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".
To facilitate this change the signature flags field was changed to be 64 bit.
14 years ago
Anoop Saldanha
17f3f36d38
packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same
14 years ago
Victor Julien
09b5dca343
Consider signatures with the flags keyword to be packet inspecting only, not stream.
14 years ago
Eric Leblond
0c34a1c5e7
rewrite constants and add flag for replace
...
This patch make use of bit shift to rewrite some of the mask constants.
It also delete an unused flag value and suppress the associated dead code.
The numeric value of the flag is now used by the flag needed for replace
code.
14 years ago
Anoop Saldanha
966119b6aa
support for http_raw_uri keyword + mpm engine
14 years ago
Anoop Saldanha
c9897a44a4
fast pattern support for http_cookie. Also support relative modifiers
15 years ago
Anoop Saldanha
bbbedaf963
fast pattern support for http_method. Also support relative modifiers
15 years ago
Eric Leblond
49adc264bc
Don't print message after SCMalloc failure.
...
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
15 years ago
Victor Julien
18b4e3380f
Make mpm-algo use the mpm_table that has the actual mpm's registered. Clean up dead code.
15 years ago
Victor Julien
435d0fb327
Clean up signature flags creating room for merging flags and mpm_flags. Merge flags and mpm_flags. Move new mpm id's into signature header. Get rid of full signature access in signature prefiltering.
15 years ago
Anoop Saldanha
25588b6910
comment out hrhd flags that we were using previously. Also remove the de_mpm_ based flags inside detect.h used by uri|hcbd|hhd|hrhd mpms. indentation fix as well
15 years ago
Anoop Saldanha
72b0fcf419
modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set
15 years ago
Anoop Saldanha
7ec0382774
support fast pattern for http raw header. Also support relative modifiers for http raw header
15 years ago
Anoop Saldanha
c61c68fd36
mpm and fast pattern support for http_header. Also support relative modifiers for http_header
15 years ago
Anoop Saldanha
778ec0939c
make client body buffer limit configurable. Also some minor changes
15 years ago
Anoop Saldanha
302011dbca
fix compilation issues with debug enabled.
15 years ago
Anoop Saldanha
0aa5cffb12
fast pattern support for http_client_body keyword added. Also mpm support for http_client_body added
15 years ago
Anoop Saldanha
bbd0c5056b
store the content added for mpm inside Signature. also carry out an unconditional cleanup of packet pattern matcher pmq det_ctx->pmq
15 years ago
Anoop Saldanha
68b78664fa
Add unittests for checking content flags. Fix indentation in PopulateMpmAddPatternToMpm(). Also fix DETECT_CONTENT_IS_SINGLE
15 years ago
Anoop Saldanha
b15ada8102
set content_packet_mpm and content_stream_mpm flag for content to prevent double check inside inspection code
15 years ago
Anoop Saldanha
67aecc73c2
set content_uri_mpm flag for uri content to prevent double check inside inspection code
15 years ago
Anoop Saldanha
6df051321f
fix fp when content is negated and also added to mpm
15 years ago
Victor Julien
234656e5f6
Fix compilation in --enable-debug mode.
15 years ago
Anoop Saldanha
eade60f0fd
make some name changes. break PopulateMpm(). Set the avoid mpm double check flags
15 years ago
Anoop Saldanha
4883efd0f6
unifying content structure - uricontent now uses DetectContentData
15 years ago
Anoop Saldanha
3c73854d2d
completely remove populate_mpm_flags. Some indentation changes. Also disable support to avoid double checks inside payload inspection for patterns added to mpm. Also add support to MpmFactory to reclaim a mpm_ctx
15 years ago
Anoop Saldanha
a6899218fc
remove populate_mpm_flags from inside PatternMatchPreparePopulateMpm()
15 years ago
Anoop Saldanha
6eaba8941c
Use new flags to indicate uricontent has a mpm set
15 years ago
Anoop Saldanha
46b4806d8e
use a single populatempm() function to add the right content for mpm
15 years ago
Anoop Saldanha
4a038511ff
Change the struct members uricontent and uricontent_len in DetectUricontentData to content and content_len. Make replacements everywhere else in the codebase to accomodate these changes
15 years ago
Anoop Saldanha
e0476242c6
replace all Signature->umatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_UMATCH]
15 years ago
Anoop Saldanha
e54358a9e1
replace all Signature->pmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_PMATCH]
15 years ago
Anoop Saldanha
0c5b82d891
provide separate ids for content, uricontent, http_(client_body_data|cookie|header|method|uri), when they share the same pattern
15 years ago
Anoop Saldanha
e072841e93
hash fix in staging to differentiate nocase duplicate patterns from case-senstive ones
15 years ago
Anoop Saldanha
29b5cb9abd
respect content flags in hash compare function during staging. For example, we would end up ignoring a nocase version of a duplicate content from another sig in the same sgh
15 years ago
Anoop Saldanha
0ef684705c
support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup
15 years ago
Anoop Saldanha
658ff5753d
aho-corasick for the cpu. We have 2 versions of ac. The first MPM_AC uses the delta table and the secone one MPM_AC_GFBS uses the goto-failure table
15 years ago
Victor Julien
5a7efe5f97
Add comment and layout update to new fast_pattern code.
15 years ago
Anoop Saldanha
a85fa6b792
support for fast_pattern only and fast_pattern:offset,length. Also support the new option for engine-analysis
15 years ago
Victor Julien
1859ed54c7
Add memcmp api with a plain memcmp function and a SSE3 accelerated memcmp.
15 years ago
Victor Julien
fc248ca7a1
Many small performance updates.
15 years ago
Victor Julien
87f88867f4
Further improve B2gc. Add B2gm. Improve memory layout.
15 years ago
Victor Julien
9dfbab42f8
WIP B2gc
15 years ago
Pablo Rincon
76af1b049b
Make malloc errors on initialization stage a fatal error, resulting on a exit() call
15 years ago
Victor Julien
7acb97da9d
Use same mpm prepare procedure for uricontent as for normal content. More cleanups.
15 years ago
Victor Julien
9ba11dbfbd
Clean up detection engine mpm initialization phase.
15 years ago
Victor Julien
0219b767b8
Fix a content pattern matching bug related to signature grouping and mpm_ctx sharing. In certain conditions (signature combinations) the mpm_stream_ctx (the ctx that handles stream pattern scanning) wasn't properly setup.
15 years ago
Anoop Saldanha
33f4beb0bc
batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms
15 years ago
Victor Julien
393acd77d2
Detection improvements: uricontent escaping now working, better negated pattern (content) handling.
15 years ago
Victor Julien
a3ff0e7210
Don't scan TCP packet payload if it was added to the stream. Inspect the tcp stream with the correct packet. Should fix #184 and #185 .
15 years ago
Victor Julien
b8fec77f37
Fix tcp connections that are reset (RST packet) not always inspecting the reassembled stream. Update transaction id code to make sure both directions of a transaction are inspected before incrementing the inspect_id.
15 years ago
Victor Julien
83b2c8abdb
Improve stateful uri detection code.
15 years ago
Victor Julien
9dd753b5f3
Scan uricontent mpm on demand.
15 years ago
Victor Julien
a24f288074
Moving the stream content scanning to have it's own mpm ctx.
15 years ago
Victor Julien
9a08d6c11c
Fixes to stream pattern matching.
15 years ago
Victor Julien
a0c1209a44
Inspect the reassembled stream together with the packet payload in the same direction.
15 years ago
Gurvinder Singh
cda664a8c4
memroy leaks fixes in detection module, app layer and counters
15 years ago
Gurvinder Singh
5fe1dc1d24
support for sslv2/sslv3 their unit tests and better stream no reassembly flag handling
15 years ago
Gerardo Iglesias Galvan
9f4fae5b1a
Fix inconsistent use of dynamic memory allocation
15 years ago
Victor Julien
7a427ec7f4
Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach.
15 years ago
Victor Julien
a372c1d14e
Fix/workaround a strange detection issue.
15 years ago
William Metcalf
ce01927515
Import of GPLv2 Header 050410
15 years ago
Anoop Saldanha
53e8120c9d
adapt b2g cuda code for the mpm architecture change
16 years ago
Pablo Rincon
b708d7f65d
Adding Uricontent inspection with spm. Modifiers for uricontent are now supported
16 years ago
Victor Julien
ec47f840f3
Remove more scan references.
16 years ago
Victor Julien
f0d68b633e
Remove nosearch flag from pattern api and add a generic bitwise flags field.
16 years ago
Victor Julien
1e01fd613c
Remove all references to the scan phase from the pattern matchers and it's api.
16 years ago
Victor Julien
dd846c9b0e
Remove all search code from the pattern matchers, cleanup mpm api, remove unused http code, more cleanups.
16 years ago
Victor Julien
b259e362cd
Convert uricontent to use new scanning methods as well. Move http_method and http_cookie keywords out of pmatch list for now.
16 years ago
Victor Julien
bef70a04ce
First stage of detect engine redesign: equal patterns share id's, search phase no longer used, new match verification phase.
16 years ago
Pablo Rincon
25a3a5c6d8
Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks.
16 years ago
Anoop Saldanha
30940c9a94
pack all the packet pattern scan and search packet setup for cuda into a function inside util-cuda-handlers.[ch]
16 years ago
Anoop Saldanha
8cf60d6645
Changed the way cuda dispatcher passes back results. Now each detection thread has it's own queue to which the dispatcher can pump packets back to the detect thread. Also, with cuda enabled and a non-cuda mpm being used, we won't create a dispatcher and instead call the b2g scan/search funtions directly instead of using the dispatcher.
16 years ago
Anoop Saldanha
c26e92733d
handle the cuda cleanup at shutdown. should get rid of any errors from the call to SigGroupCleanup
16 years ago
Anoop Saldanha
41e6735b92
mpm b2g cuda support added
16 years ago
Gurvinder Singh
999a200bc9
pattern matcher options support
16 years ago
Victor Julien
b3bcba077f
Only inspect http flows against uri sigs, clean up uri scanning code.
16 years ago
Gurvinder Singh
356a8bf385
applayer uri match and modified http handling
16 years ago
Steve Grubb
60ad9d29c5
Memory leak cleanup in detectors
...
Hello,
I ran the code through an analysis program and found several leaks that
should be cleaned up.
*In src/detect-engine-address-ipv4.c at line 472, the test for ag == NULL
will never be true since that is the loop entry test.
*In src/detect-engine-port.c at line 1133, the test for p == NULL will
never be true since that is the loop entry test.
*In src/detect-engine-mpm.c at line 263 is a return without freeing
fast_pattern
*In src/detect-ack.c at line 80 and 85, data catches the return from malloc.
One of them should be deleted.
*In src/detect-seq.c at line 81 and 86, data catches the return from malloc.
One of them should be deleted.
*In src/detect-content.c at line 749, many of the paths that lead to the error
exit still has temp pointing to allocated memory. To clean this up, temp
should be set to NULL if not immediately assigning and new value.
*In src/detect-uricontent.c at line 319, both cd and str needto be freed. At
lines 344, str needs to be freed. And at line 347 str and temp need to be
freed.
*In src/detect-flowbits.c at line 231 and 235, str was not being freed. cd was
not being freed at line 235.
*In src/detect-flowvar.c at line 127, str was not being freed. At line 194, cd
and str were not being freed.
*In src/detect-flowint.c at line 277, sfd was not being freed. At line 315, str
was not being freed.
*In src/detect-pktvar.c at line 121, str was not being freed. At line 188, str
and cd was not being freed.
*In src/detect-pcre.c at line 389, there is an extra free of "re" that should
be deleted.
*In src/detect-depth.c at line 42 & 48, str has not been freed.
*In src/detect-distance.c at line 49 and 55, str has not been freed
*In src/detect-offset.c at line 45, str has not been freed.
The patch below fixes these issues.
-Steve
16 years ago
Victor Julien
71ed2d38f5
Fix scan patterns sometimes not being added to the scan ctx. Should fix bug #9 .
16 years ago
Victor Julien
ecf86f9c23
Rename to Suricata.
16 years ago
Anoop Saldanha
45acb64a61
Bug fix for fast_pattern - bug #8
16 years ago
Anoop Saldanha
8c9df4cd6b
modifications to PatternMatchPreprarePopulateMpm to fasten fast_pattern processing
16 years ago
Anoop Saldanha
7a10ddc07b
Fixes for the fast-pattern tests and a couple of other minor changes
16 years ago
Anoop Saldanha
6ca5dbc9e9
Support fast_pattern modifier keyword for content
16 years ago
Victor Julien
4f2164677a
- Fix pattern matchers b2g and b3g not being able to deal with a single pattern of the max pattern length (32 bytes by default).
...
- Fix the setting of the correct pattern matcher when it was set in the detection ctx.
- Add tests for the fixes.
16 years ago
Victor Julien
2d0e9658f8
Speed up per sgh content maxlen calc. Remove mpm ptrs from mpm ctx. Add unittests testing the detection engine internals.
16 years ago
Victor Julien
7a7bb7a390
Get rid of global mpm_ctx.
16 years ago
Victor Julien
fbe87a3ad5
Bunch of mostly unittest related memleak fixes.
16 years ago
Victor Julien
0d0ffb9963
Reorganize header inclusions.
16 years ago
Victor Julien
6eaff4be12
Fixes for issues found by static code analyzer.
16 years ago
Victor Julien
1132ab635a
Rename all pmt->det_ctx.
16 years ago
Victor Julien
b9972a9d2c
Cleanups
16 years ago
Victor Julien
4369816cdd
Improvements to content keyword memory handling.
...
First version of a simple pattern based L7 proto detection engine. Currently just works by matching a single pattern in the initial data. Implemented HTTP, SSL, MSN, JABBER, SMTP and a few more.
Couple of pattern matcher cleanups.
16 years ago
Anoop Saldanha
244f5d547a
new registration functions for the stats api, with local thread storage for counter ids
16 years ago
Brian Rectanus
fa5939ca91
64 bit cleanup part2
16 years ago
Victor Julien
689bbfdc45
Rename all structure definitions in the "typedef struct _SomeStruct" format to "typedef struct SomeStruct_" to make the Doxygen output more useful.
...
Remove the Trie multi pattern matcher code. It wasn't used anymore.
16 years ago
Victor Julien
8397413942
Comment updates.
16 years ago
Victor Julien
84aa365a3b
Fix iponly matching.
16 years ago
Victor Julien
657be002d1
Big detection engine update: scan improvements, b2g/b3g updates, bloom fixes, iponly detection implementation, dsize/flow grouping.
16 years ago
Victor Julien
5df5b35e90
Put all globals in the detection engine ctx. Add HashList type, a hash that also stores the items ina list to it can be traversed. Many cleanups.
16 years ago
Victor Julien
b2eb954099
Add b3g 3gram BNDM pattern matcher. Fix multi queue nfq initialization. Improve speed of b2g and wumanber.
16 years ago
Victor Julien
1c0ad1d415
Add implementation of the Simple BNDM 2gram pattern matcher algorithm.
16 years ago
Victor Julien
efb10fc0d6
big update
16 years ago
Victor Julien
21364b34dc
Fix uricontent scan for copied siggroupheads.
16 years ago
Victor Julien
69e056e33f
Add the scanning to uricontent as well.
16 years ago
Victor Julien
fedcc397de
Detection engine improvement: don't run pattern matcher on packets with payload sizes less that the biggest content we need to match. Add some extra stats.
16 years ago
Victor Julien
dce2c12915
Add Scan before Search to the detection engine.
16 years ago
Victor Julien
4c4862d838
Improve logging, add alert-output module, at module exit stats, add HTTP POST uri capture.
16 years ago
Victor Julien
54ffe2053e
Large detection engine update.
16 years ago
Victor Julien
8b3d06fd92
Rename some detection engine related files.
16 years ago