Anoop Saldanha
4810ee9c5f
All uricontent modified patterns now are DETECT_CONTENT and not DETECT_URICONTENT. Step towards unifying all content based patterns. Makes way for easier management of patterns
14 years ago
Anoop Saldanha
e682796d03
feature #414 - support listing supported keywords. Remove support for dummy keywords __address__, __proto__, __port__. Remove support for recursive keyword and all references to it
14 years ago
Anoop Saldanha
35f1f7e8d9
unify payload detection engines + fix other bugs in pcre init
14 years ago
Anoop Saldanha
0677190960
rebase commit for hscd and hsmd patches
14 years ago
Anoop Saldanha
09313cf9bd
Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S
14 years ago
Anoop Saldanha
2007c2711c
Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword
14 years ago
Victor Julien
9dc153c8f4
Fix path handling for including rule files on win32.
14 years ago
Anoop Saldanha
716afac5a2
fix debug messages that have references to the old mpm contexts
14 years ago
Anoop Saldanha
9a665e035b
code cleanup over last 2 commits
14 years ago
Anoop Saldanha
419cdc8558
support splitting mpm ctxs based on direction v2
14 years ago
Victor Julien
28e15be526
Clean up default output. Use simpler output format for releases.
14 years ago
Victor Julien
6eeab37ab3
Add post-match list, move flowbits set, etc functions to it. Move flowint set, etc functions to it as well.
14 years ago
Victor Julien
aac2d91bcc
Set DROP flag for reject action so in addition to sending the rst, in IPS mode also drop the offending packet.
14 years ago
Victor Julien
f084874998
Fix HTTP state and raw stream not being inspected at the same time. Adds an exception to transaction id handling for HTTP.
14 years ago
Anoop Saldanha
9c4954ff42
support http response raw header inspection + carry out hrhd mpm on both request/response headers + add unittests for the same
14 years ago
Anoop Saldanha
30247dce8c
bug 389 - support http response header inspection + fix bug with stateful inspection for sigs that would have both request/response inpection
14 years ago
Victor Julien
21ee59e6f3
Add signature direction (flow:toserver/flow:toclient) as a signature flag.
14 years ago
Victor Julien
d5402d33d4
Simplify detection loop. Inspect packet keywords before the state.
14 years ago
Anoop Saldanha
eea5ab4a7a
Support for app layer decoder events added + app_layer_event keyword added
14 years ago
Anoop Saldanha
517040c4af
indentation fix
14 years ago
Anoop Saldanha
37b223645a
fix detection engine for alert stability. Fix cases where we have multiple rules having same pattern. We should see good perf increase(~5%) with this change, now that we avoid unnecessary inspection"
14 years ago
Anoop Saldanha
92643f6110
introduce separate mpm ctxs for tcp/udp/other_protos
14 years ago
Victor Julien
18d79c4215
file store: respect flowbits and other keywords
...
The filestore keyword until now flagged a file, tx or ssn for storage as soon
as the keyword was inspected. This happens before flowbits and some other
keywords, so files were stored that weren't supposed to.
This patch makes the filestore keyword fill an array in the detect engine
thread ctx. Then if the full signature matches, a post-match filestore
function makes the store final.
14 years ago
Victor Julien
7adac3048d
file-data: create initial keyword registration.
14 years ago
Victor Julien
dd9da1a56f
Merge all http mpm related signature flags into a single set: SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_HTTP_NEG.
14 years ago
Victor Julien
2650551192
Rename signature init flags to indicate they are init flags.
14 years ago
Victor Julien
291ddd95f2
Detection engine -- mpm
...
Each signature is in one mpm ctx at max, but there were 3 separate
id's in use: packet, stream, http. Merged them all into one.
Could shrink the SignatureHeader structure with 8 bytes because of this,
should lead to better caching performance.
14 years ago
Victor Julien
7db72bce75
Optimize detection engine prefiltering logic.
14 years ago
Victor Julien
89f83e714c
Introduce http_server_body keyword.
...
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".
To facilitate this change the signature flags field was changed to be 64 bit.
14 years ago
Anoop Saldanha
b970273163
fix broken unittest
14 years ago
Anoop Saldanha
d40fb5b933
Remove unnecessary flow NULL check
14 years ago
Anoop Saldanha
cc7db6315c
Move setting packet iponly flags from decode section to stream section
14 years ago
Victor Julien
117d51c965
Fix a compile warning when debug is enabled.
14 years ago
Victor Julien
042fd850fc
Make sure we check the sgh for no magic and no store once per flow direction.
14 years ago
Victor Julien
d59ca75e46
file extract: split toserver and toclient tracking
...
Split toserver and toclient file tracking for the http state.
14 years ago
Victor Julien
04ea70ccf7
file extract: pruning
...
Add pruning of files in memory so we keep only memory what we really need.
Fix magic logic.
Reset file part of the de_state on receiving another file in the same tx.
14 years ago
Victor Julien
e1022ee5ae
file-extraction: Disconnect file handling from flow and move into the app layer state.
14 years ago
Victor Julien
9b62ec65ab
Make sure filemagic works properly regardless of filestore being in use for a flow.
14 years ago
Victor Julien
5945e652d6
Initial implementation of filemagic keyword.
14 years ago
Victor Julien
23e01d23d3
Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored.
14 years ago
Pablo Rincon
6d60b3a747
filename and fileext keywords
14 years ago
Anoop Saldanha
9a6aef459e
modify all relevant app layer API calls to accomodate passing parser local storage argument
14 years ago
Anoop Saldanha
2b356dadff
Support for tos keyword added
14 years ago
Victor Julien
262a7300d7
flow: shrink Flow datatype
...
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.
Add macro's to check the family, copy the address, etc.
Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.
On 64 bit this shrinks the flow with 8 bytes.
14 years ago
Victor Julien
06904c9024
App Layer cleanup
...
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
14 years ago
Victor Julien
3d845b6c77
Consider Windows new line chars as well when parsing rule files. Bug #374 .
14 years ago
Victor Julien
8cc82c7241
Add -S commandline option that loads a rule file exclusively. Issue #338 .
14 years ago
Victor Julien
2d16abcf8b
Minor code cleanups fixing all GCC 4.6 compiler warnings for default, debug and unittests mode.
14 years ago
Anoop Saldanha
0acfcc206c
fix unittests. fix replace unittests that allow alproto keywords with replace
14 years ago
Anoop Saldanha
a0eec3d846
fix detection code that handles cases when we use recursion(from recursive keyword)
14 years ago
Anoop Saldanha
7433d92dd2
undo this commit -
...
commit eff08f93d8
Author: Anoop Saldanha <poonaatsoc@gmail.com>
Date: Thu Nov 3 14:31:24 2011 +0530
update failing unittest to reflect the mpm design update
Fixed a bug in the mpm code that would make all the changes in the commit just undone wrong.
14 years ago
Anoop Saldanha
1b1332fff0
fix mpm bug on running stream mpm for packets not added to stream mpm
14 years ago
Anoop Saldanha
eff08f93d8
update failing unittest to reflect the mpm design update
14 years ago
Victor Julien
af51493da2
Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm.
14 years ago
Anoop Saldanha
17f3f36d38
packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same
14 years ago
Anoop Saldanha
c7b9d3fecb
Remove broken dsize_sm in SigMatch used by dsize in detection engine
14 years ago
Eric Leblond
60a99915c1
doc: create http support group
...
This patch create an httplayer group and adds related files to
it. It also fixes some typo in documentation string and format.
14 years ago
Eric Leblond
6bf15bac31
Fix various packet access.
...
The coccinelle based tests have detected invalid uses of access to
Packet data. This patch fixes the detected problems.
14 years ago
Victor Julien
7ef34b7bcc
Exlcude DSIZE LT case from setting the 'need payload' mask bit as it can include 0, which means no payload.
14 years ago
Victor Julien
2e2e80b812
Add packet alert flag to indicate a match happened (partly) in the app layer state. Make unified2 use this flag.
14 years ago
Victor Julien
a1f68bf411
Fix detection engine informational message misrepresenting decoder only signatures.
14 years ago
Eric Leblond
3f75b10f79
Suppress useless code.
...
This code was making a warning for some time now. This patch kills
it.
14 years ago
Eric Leblond
a85dc9b0e2
Add support for replace keyword.
...
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.
One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.
Replace is only activated in IPS mode and apply only to raw match.
14 years ago
Anoop Saldanha
8028392e9a
fix mpm segv. Use sgh flags to check if the sgh has packet or stream mpm set or not
14 years ago
Victor Julien
e13181496c
ip-only: added support for matching on ports.
14 years ago
Victor Julien
751a77a9be
Make sure stream/engine-event signatures are recognized as such.
14 years ago
Victor Julien
63f834d9a7
Add profiling to various HTTP buffer MPM calls.
14 years ago
Victor Julien
3693a7a9ee
Profiling: add accounting for several detection phases.
14 years ago
Eric Leblond
e3a6d8955e
Introduce engine-event keyword
...
This patch renames the 'decode-event' keyword to 'engine-event' and
keep it for backword compatibility of rulesets. All *DecodeEvent*
references in the code are replaced by EngineEvent version.
14 years ago
Eric Leblond
2ac8755382
Rename detect-decode-event to detect-engine-event
...
This patch does a simple renaming of detect-decode-event file to
the more global detect-engine-event name.
14 years ago
Victor Julien
820b0ded82
Add per packet profiling.
...
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.
Stats per packet include:
1) total ticks spent
2) ticks spent per individual thread module
3) "threading overhead" which is simply calculated by subtracting (2) of (1).
A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.
Code depends on --enable-profiling just like the rule profiling code.
New yaml parameters:
profiling:
# packet profiling
packets:
# Profiling can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: yes
filename: packet_stats.log
append: yes
# per packet csv output
csv:
# Output can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: no
filename: packet_stats.csv
Example output of summary stats:
IP ver Proto cnt min max avg
------ ----- ------ ------ ---------- -------
IPv4 6 19436 11448 5404365 32993
IPv4 256 4 11511 49968 30575
Per Thread module stats:
Thread Module IP ver Proto cnt min max avg
------------------------ ------ ----- ------ ------ ---------- -------
TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770
TMM_DETECT IPv4 6 19436 1107 137241 1504
TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155
TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138
TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154
TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944
TMM_STREAMTCP IPv4 6 19434 828 1299159 19438
The proto 256 is a counter for handling of pseudo/tunnel packets.
Example output of csv:
pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337
First line of the file contains labels.
2 example gnuplot scripts added to plot the data.
14 years ago
Victor Julien
1bd1a62526
Rename profile macro's and variables to reflect that they are for rule profiling.
14 years ago
Anoop Saldanha
7e5c52c80b
add flowbits:set; only sigs to be treated as ip only
14 years ago
Victor Julien
16b41a5eff
Use p->proto in detect to determine TCP/UDP/SCTP.
14 years ago
Victor Julien
f7f037c1d1
Make sure stateful detection engine inspecting HTTP streams works well for to_client rules as well.
14 years ago
Anoop Saldanha
35f3eafa5e
byte extract added to the engine. Detection support added for packet payload, uri and dce detection engines
14 years ago
Victor Julien
73efb4c70f
Add a app layer state and stateful detection engine counter that makes sure the stateful inspection is only done when the state changes.
14 years ago
Victor Julien
0768ca9806
Fix SIMD mask checking on 64 bit systems.
14 years ago
Victor Julien
350215966b
Fix signature mask bitorder.
14 years ago
Victor Julien
aa822c0ac1
Always reset alert cnt and always increment det_ctx->pkts.
14 years ago
Victor Julien
1e0b050a54
Add more mask flags.
14 years ago
Victor Julien
4b52823ab6
Use 64 bit mask on 64-bit systems.
14 years ago
Victor Julien
e5b6c0f518
Check 32 masks per run instead of 16 in the SIMD code.
14 years ago
Victor Julien
2dbfdd40af
Clean up new SIMD mask checking code, improve non-SIMD checks.
14 years ago
Victor Julien
b421019cef
Match packet mask against 16 signature masks at once using SIMD instructions for SSE3 and up.
14 years ago
Victor Julien
e1d4e16645
Simplify packet decoding macro's.
14 years ago
Victor Julien
75439863ed
Shrink PacketAlerts structure so that Packet structure is a lot smaller. Reduce max events per packet from 256 to 15.
14 years ago
Victor Julien
6aa551c558
Small optimizations to IPV4 and TCP header parsing.
15 years ago
Anoop Saldanha
966119b6aa
support for http_raw_uri keyword + mpm engine
15 years ago
Victor Julien
864c8718e1
Store matching stream msg (ptr) in packets alert structure so it's available to the output plugins.
15 years ago
Victor Julien
140eb4fde8
Fix decode-event keyword parsing. Fix code that indicates a signature is decode-event only. Add 'pkthdr' protocol as an alias for any/ip to be used by decode-event signatures.
15 years ago
Victor Julien
e197f50727
Fix IP-Only unittests failing on Big Endian.
15 years ago
Eric Leblond
9beebf621a
Add support for 'nfq_set_mark' keyword
...
This patch introduces 'nfq_set_mark' which is new rules option. If a packet
matches a rule using nfq_set_mark in NFQ mode, it is marked with the mark/mask
specified in the option during the verdict.
It is thus possible to trigger different behaviour on the packet inside
Linux/Netfilter.
15 years ago
Anoop Saldanha
c105a739e9
support for ssl_state keyword added
15 years ago
Victor Julien
b9fd978253
Fix compiler warnings in two unittests.
15 years ago
Eric Leblond
005dc599a6
detect.c: Fix usage of integer standing for protocol
...
This patch fixes direct usage of integer to code protocol value.
15 years ago
Eric Leblond
2c80f18dc9
detect: Add sctp detection and parsing.
...
This patch adds the support of SCTP in signature subsystem.
15 years ago
Gurvinder Singh
8f8b1212af
support for ssl_version keyword
15 years ago
Victor Julien
ec5b622553
Inspect all stream msgs at any time when running in stream-inline mode. Skip detection for packets flagged for dropping before detect.
15 years ago
Victor Julien
1d971b53a6
Update all unittests
15 years ago
Victor Julien
f606621e8c
Fix the pseudo packet having the wrong proto set, causing massive fp's. Flag packets to be part of the established phase of a tcp session, so we won't prematurely inspect the app layer state.
15 years ago
Victor Julien
24f071cabb
Make sure http_cookie inspects all HTTP transactions. Clean up error messages. Get rid of unused code and dead comments.
15 years ago
Anoop Saldanha
c9897a44a4
fast pattern support for http_cookie. Also support relative modifiers
15 years ago
Anoop Saldanha
bbbedaf963
fast pattern support for http_method. Also support relative modifiers
15 years ago
Eric Leblond
49adc264bc
Don't print message after SCMalloc failure.
...
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
15 years ago
Eric Leblond
1db4aadd16
Supress usage of Packet declaration in tests.
...
For convenience, a massive usage of 'Packet p;' declaration has
been done in the tests function. Although this was completely
legal, this is not possible anymore because of the new Packet
allocation structure. This massive patch modifies all suricata
files to use a SCMalloc allocated pointer to Packet instead.
This patch has been done using coccinelle (http://coccinelle.lip6.fr )
which is a semantic patching tool. This ensures that things like call
to SCFree() should have not been forget because the semantic patch
explicitly forces the call to SCFree(p) before each return. With this
patch all unittests are running fine with a small and a big default
packet size.
15 years ago
Eric Leblond
dd038c1906
Modify files to avoid direct pckt payload access
...
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
15 years ago
Victor Julien
18b4e3380f
Make mpm-algo use the mpm_table that has the actual mpm's registered. Clean up dead code.
15 years ago
Victor Julien
3f47eade6b
Fix couple of cases where incorrect handling of keyword parsing errors would lead to access of uninitialized memory. Found by clang.
15 years ago
Victor Julien
3d60e9bfeb
Clean up output.
15 years ago
Anoop Saldanha
d241e51b32
adapt fast pattern engine analysis to reflect the new changes made to your mpm design
15 years ago
Anoop Saldanha
c734cd1bdd
make cuda mpm parameters configurable
15 years ago
Victor Julien
0e8e8e3728
Don't stop stream reassembly if protocol detection failed, only stop/prevent app layer parsing.
15 years ago
Victor Julien
39dea56a84
Remove flowbits as a mask prefilter as they are dynamic. Add a dynamic check.
15 years ago
Anoop Saldanha
8c21511c99
don't buffer raw headers. Retrieve them individually from htp_state during mpm stage and content valiadation stage
15 years ago
Victor Julien
55ca988222
Change locking of http_header, http_raw_header and http_client_body so that flow isn't accessed without lock anywhere.
15 years ago
Victor Julien
435d0fb327
Clean up signature flags creating room for merging flags and mpm_flags. Merge flags and mpm_flags. Move new mpm id's into signature header. Get rid of full signature access in signature prefiltering.
15 years ago
Victor Julien
169aa5581a
Rename SIG_FLAG_AMATCH flag to SIG_FLAG_STATE_MATCH to better reflects its purpose.
15 years ago
Victor Julien
d7b92d9bfe
Consolidate several signature flags into one.
15 years ago
Victor Julien
2102a54c26
Cleanup and rearrange detection code slightly.
15 years ago
Anoop Saldanha
25588b6910
comment out hrhd flags that we were using previously. Also remove the de_mpm_ based flags inside detect.h used by uri|hcbd|hhd|hrhd mpms. indentation fix as well
15 years ago
Anoop Saldanha
e50503e820
cleanup/remove dead code
15 years ago
Anoop Saldanha
93fa7ea828
modify detection engine to run hrhd mpm before building the match array
15 years ago
Anoop Saldanha
b140ed1c9c
modify detection engine to run hhd mpm before building the match array
15 years ago
Anoop Saldanha
4e273f2c8b
modify detection engine to carry out hcbd mpm run before build match array if alproto is http and if sgh has atleast one sig with hcbd mpm set
15 years ago
Anoop Saldanha
72b0fcf419
modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set
15 years ago
Victor Julien
6a5d2cb40d
Fix potential locking issue in out of memory conditions in the http_header, http_raw_header code. Fix other potential small issues in http_ code.
15 years ago
Anoop Saldanha
7ec0382774
support fast pattern for http raw header. Also support relative modifiers for http raw header
15 years ago
Victor Julien
0c806f70bb
Fix --enable-debug compilation, just unittest with --enable-debug-validation enabled.
15 years ago
Victor Julien
1a32d9b5ec
Fix printing unprintable characters in the engine-analysis fast_pattern mode.
15 years ago
Anoop Saldanha
c61c68fd36
mpm and fast pattern support for http_header. Also support relative modifiers for http_header
15 years ago
Anoop Saldanha
778ec0939c
make client body buffer limit configurable. Also some minor changes
15 years ago
Anoop Saldanha
0aa5cffb12
fast pattern support for http_client_body keyword added. Also mpm support for http_client_body added
15 years ago
Anoop Saldanha
c227aeeacb
remove support for skipping reinspecting fast pattern contents once again during packet payload inspection. Also make some changes to our detection engine
15 years ago
Anoop Saldanha
bbd0c5056b
store the content added for mpm inside Signature. also carry out an unconditional cleanup of packet pattern matcher pmq det_ctx->pmq
15 years ago
Anoop Saldanha
6df051321f
fix fp when content is negated and also added to mpm
15 years ago
Anoop Saldanha
5c6a65dc58
support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch]
15 years ago
Anoop Saldanha
3d2f81d978
replace all Signature->dmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_DMATCH]
15 years ago
Anoop Saldanha
a7353be20d
replace all Signature->amatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_AMATCH]
15 years ago
Anoop Saldanha
e0476242c6
replace all Signature->umatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_UMATCH]
15 years ago
Anoop Saldanha
e54358a9e1
replace all Signature->pmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_PMATCH]
15 years ago
Anoop Saldanha
82fd581b64
replace all sm lists (match, pmatch, dmatch, umatch, amatch, tmatch) with an array Signature->sm_lists[]. Replace all Signature->match instances in the engine with Signature->sm_lists[DETECT_SM_LIST_MATCH]
15 years ago
Victor Julien
001f91056e
Add http_raw_header as an alias to the http_header keyword as that actually inspects the raw headers (see issue #243 ). Closes issue #242 .
15 years ago
Gurvinder Singh
b7da115e6d
support for http_stat_code keyword has been added to detection module
15 years ago
Gurvinder Singh
1deae70cf7
added http_stat_msg keyword support for detection module
15 years ago
Anoop Saldanha
2cdb5be391
Print out file name for fast_pattern engine_analysis. Also add some info logs
15 years ago
Anoop Saldanha
a2d04a94b5
selecting auto for detect-engine.sgh_mpm_context now uses single if the mpm is ac, full otherwise
15 years ago
Anoop Saldanha
174048544d
fix hash generation in b2g and ac addpattern. Brings down the no of patterns added from close to a million to a couple of thousands
15 years ago
Anoop Saldanha
0ef684705c
support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup
15 years ago
Anoop Saldanha
a85fa6b792
support for fast_pattern only and fast_pattern:offset,length. Also support the new option for engine-analysis
15 years ago
Anoop Saldanha
0d741b9a55
fix for bug 227. For negated contents that have been added to mpm we might have pmq.pattern_id_array_cnt as 0. We can't ignore inspecting sigs if this is 0, in case the content added is negated
15 years ago
Victor Julien
cbd4c298ed
Initial version of a new bitmask based signature pre-filtering method.
15 years ago
Victor Julien
94898a91cc
Reorganize SigMatchSignatures.
15 years ago
Victor Julien
fc248ca7a1
Many small performance updates.
15 years ago
Pablo Rincon
5c43db85ce
Drop streams on inline mode when a drop rule match from a reassembled stream and/or app layer inspection
15 years ago
Anoop Saldanha
f094523eb1
clang fix - some minor fixes for unittests
15 years ago
Pablo Rincon
9d7baa7a9f
Adding ssh app layer module with two new keywords: ssh.protoversion and ssh.softwareversion
15 years ago
Victor Julien
04d3832d8f
Remove ports check and fix small typo.
15 years ago
Victor Julien
a492518e7a
Properly detect detect-event-only sigs.
15 years ago
Pablo Rincon
21d79b05ad
Fix for bug221 (avoid considering sig as "decoder event only" if ports are specified). Now the sig gets grouped to get a sgh at SigMatchSignatures
15 years ago
Victor Julien
6299fbfb0f
Fix stream msg content inspection not inspecting the correct id.
15 years ago
Victor Julien
1071a53210
Fix unittests after ip_proto keyword change.
15 years ago
Pablo Rincon
70bda6506d
Fix for bug 180 (check proto specified at the IP hdr)
15 years ago
Victor Julien
7acb97da9d
Use same mpm prepare procedure for uricontent as for normal content. More cleanups.
15 years ago
Victor Julien
9ba11dbfbd
Clean up detection engine mpm initialization phase.
15 years ago
Victor Julien
0d008c8135
Change stateful detection engine to be able to start the stateful detection separate from other sigs. Fixes bugs #213 , #214 , #215 .
15 years ago
Victor Julien
689d05b10b
Add missing protocol check in the sig matching process. This prevents FP's such as the one reported in bug #209 .
15 years ago
Victor Julien
0219b767b8
Fix a content pattern matching bug related to signature grouping and mpm_ctx sharing. In certain conditions (signature combinations) the mpm_stream_ctx (the ctx that handles stream pattern scanning) wasn't properly setup.
15 years ago
Victor Julien
102092a89c
Make signature address matching more cache efficient.
15 years ago
Victor Julien
1eec149f5e
Use Address structure in DetectAddress struct.
15 years ago
Victor Julien
c6ddcda7f8
Improve out of memory handling during initialization.
15 years ago
Victor Julien
bfd167521e
Fix DCERPC over SMB/SMB2 detection issues. Fix not updating transaction id in a stream direction if there was no sgh.
15 years ago
Anoop Saldanha
33f4beb0bc
batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms
15 years ago
Victor Julien
b3c22cd512
Improve app layer proto check.
15 years ago
Victor Julien
39cb1bdbda
Fix app layer sigs being recognized as decoder event only or ip only.
15 years ago
Victor Julien
d41b5645ef
Make sure decoder event rules are inspected even if the packet is invalid and has no addesses or proto. Update fast log and alert debug log to display the alerts. Fixes #179 .
15 years ago
Victor Julien
e685579231
Add optional structure validation code.
15 years ago
Victor Julien
393acd77d2
Detection improvements: uricontent escaping now working, better negated pattern (content) handling.
15 years ago
Anoop Saldanha
9ecade76b9
in case of duplicate signatures used the one with the latest revision
15 years ago
Gurvinder Singh
8852b83fa7
flowbits, flowvars, pktvars, flow flags and app layer info added to alert-debug.log
15 years ago
Victor Julien
580b09c2b8
Make sure we inspect all outstanding reassembled stream chunks (smsg) if the stream is shutting down. Make sure to do inspect signatures that use dsize against the tcp packet payload, even if that payload was already added to the stream. Likewise, the dsize signatures are not inspected against the reassembled stream.
15 years ago
Victor Julien
a3ff0e7210
Don't scan TCP packet payload if it was added to the stream. Inspect the tcp stream with the correct packet. Should fix #184 and #185 .
15 years ago
Pablo Rincon
a8cb8d830b
Fix for bug 186 and thresholding issue handling ip versions
15 years ago
Pablo Rincon
eed0ef6e69
Adding tag keyword support
15 years ago
Victor Julien
ca7f54de25
Make sure ICMP unreach packets are not inspected against the flow sgh as it's for the original protocol, not for the ICMP packet. Fixes #174 .
15 years ago
Victor Julien
b8fec77f37
Fix tcp connections that are reset (RST packet) not always inspecting the reassembled stream. Update transaction id code to make sure both directions of a transaction are inspected before incrementing the inspect_id.
15 years ago
Victor Julien
cdc9570f0e
Have the detect.alerts counter count actual alerts.
15 years ago
William Metcalf
0e4235cc94
FLOW_DESTROY added to clean-up UT's that init flow
15 years ago
Victor Julien
2f29b8a724
Improve detection of app layer, making sure we only handle app layer on 'established' packets. Should really fix #166 .
15 years ago
Victor Julien
37442a8a84
Prefilter signatures before fully scanning them.
15 years ago
Victor Julien
d6709b0961
Fix a segv caused by invalidly accessing the smsg_pmq array.
15 years ago
Victor Julien
8cea3779fa
Move dce payload inspection to stateful detection engine.
15 years ago
Anoop Saldanha
45ea0d914e
dce stub content keywords support using dcepayload.c support for all dce related content keywords
15 years ago
Victor Julien
83b2c8abdb
Improve stateful uri detection code.
15 years ago
Victor Julien
9dd753b5f3
Scan uricontent mpm on demand.
15 years ago
Victor Julien
e8fce5f7fa
Convert uricontent scanning to use the detect engine state.
15 years ago
Pablo Rincon
8cc525c939
UDP support at AppLayer message handling
15 years ago
William Metcalf
cc76aa4bc6
properly init flows inside of unit-tests caused lock-up when falling back to using mutex locks
15 years ago
Victor Julien
a24f288074
Moving the stream content scanning to have it's own mpm ctx.
15 years ago
Victor Julien
9a08d6c11c
Fixes to stream pattern matching.
15 years ago
Victor Julien
a0c1209a44
Inspect the reassembled stream together with the packet payload in the same direction.
15 years ago
Victor Julien
81f2499834
Store stream msgs processed by the app layer in the tcp session so they can be inspected by the detection module as well. The detection module returns them to the pool.
15 years ago
Victor Julien
c26434fef1
Move flow use cnt to atomic and outside of the flow mutex protection.
15 years ago
Victor Julien
2fd31a1a11
Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory.
15 years ago
Victor Julien
dff6795df5
Detect cleanups.
15 years ago
Gerardo Iglesias Galvan
55dfa36963
Add support for http_uri keyword
15 years ago
Jason Ish
ea4b7cc33b
add profiling to stateful detection engine + other fixups.
15 years ago
Victor Julien
4e7df60b2f
Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this.
15 years ago
Pablo Rincon
3fa3229e01
ASN1 decoder and keyword implementation
15 years ago
Victor Julien
70b32f7380
First stab at creating a stateful detection engine.
...
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:
- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.
This commit fixes bug #124 .
15 years ago
Jason Ish
18e5ac8cde
Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules.
15 years ago
Victor Julien
42eeb84c9a
Properly lock flow before setting IP only action flags. Small alert api cleanups.
15 years ago
Pablo Rincon
9bae6a8628
Moving alert logic to detect-engine-alert.c
15 years ago
Gerardo Iglesias Galvan
9f4fae5b1a
Fix inconsistent use of dynamic memory allocation
15 years ago
William Metcalf
8d66323f62
clang fixes for null derefrences
15 years ago
Victor Julien
e27cefa6f7
Complete conversion of pattern id mpm storage vs sig id storage.
15 years ago
Victor Julien
46831e0f8f
Fix signature grouping bug for protocols without ports. Add debugging code.
15 years ago
Victor Julien
7a427ec7f4
Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach.
15 years ago
Pablo Rincon
46187bfe73
Fix action logic after last pass changes
15 years ago
Gurvinder Singh
3721037de5
unittests for bug 134&139 and some typo correction
16 years ago
Victor Julien
a372c1d14e
Fix/workaround a strange detection issue.
16 years ago
Victor Julien
ce90e87304
Fix failing thresholding unittests
16 years ago
Pablo Rincon
e18e2ec998
Changing threshold logic
16 years ago
Pablo Rincon
8bcdf29ab7
Small fix on pass action handling and added more unittests
16 years ago
Pablo Rincon
1238668961
Adding actions order and suport for rule action "pass"
16 years ago
William Metcalf
ce01927515
Import of GPLv2 Header 050410
16 years ago
Victor Julien
070ed778b8
Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added.
16 years ago
Pablo Rincon
ab02ab9ead
adding http_header keyword support
16 years ago
Pablo Rincon
224a33f19e
Moving inline functions to the .h files, so gcc can inline them correctly
16 years ago
Victor Julien
eeb98c6900
Move SCSetThreadName to proper functions.
16 years ago
Gerardo Iglesias Galvan
9f35a24a1f
Set threads name. Fix bug #83
16 years ago
Gurvinder Singh
69a4fee757
fixed the API and logic error reported by clang tool
16 years ago
Victor Julien
78e15ea7fa
Explicitly test for ipv6 in the htp personalities code. Update all affected unittests to set addr family to the flow.
16 years ago
Anoop Saldanha
47037ef9ec
fix for bug 115
16 years ago
Victor Julien
fe7ece997a
Different approach to the reference keyword. Lots of cleanups, bug fixes in reference keyword code and tests.
16 years ago
Breno Silva
89baf93a40
Reference Support
16 years ago
Pablo Rincon
e7a989e305
IP Only Engine using radix trees
16 years ago
Anoop Saldanha
97d49d8f5e
support for http_client_body keyword
16 years ago
Victor Julien
26e8a0a06a
Cleanup global threshold code.
16 years ago
Breno Silva
67f2026279
Global Threshold config
16 years ago
Victor Julien
08600df6b1
Small uri cleanups.
16 years ago
Pablo Rincon
c7350a8ac6
Fixing some naming convention issues and incorrect error messages
16 years ago
Pablo Rincon
b708d7f65d
Adding Uricontent inspection with spm. Modifiers for uricontent are now supported
16 years ago
Anoop Saldanha
c54b91ed94
fix for bug 113
16 years ago
Victor Julien
6be0778532
Comment SigMatchSignatures a bit.
16 years ago
Gerardo Iglesias Galvan
ef2ae76c42
Add support for detection_filter keyword
16 years ago
Victor Julien
297001c6d9
Only process a app layer sig if it has the proper state. Make sure a sig can't have conflicting sigmatches, such as ftpbouce and uricontent.
16 years ago
Victor Julien
ec47f840f3
Remove more scan references.
16 years ago
Victor Julien
bee4e04664
More scan/search related cleanups.
16 years ago
Victor Julien
6990d9c91b
Fix thresholding signature unittests. Because of the bug fix that made thresholding compatible to ip-only sigs the test sigs needed to be made non-ip-only.
16 years ago
Victor Julien
b259e362cd
Convert uricontent to use new scanning methods as well. Move http_method and http_cookie keywords out of pmatch list for now.
16 years ago
Victor Julien
bef70a04ce
First stage of detect engine redesign: equal patterns share id's, search phase no longer used, new match verification phase.
16 years ago
Pablo Rincon
25a3a5c6d8
Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks.
16 years ago
Pablo Rincon
ebcbc859e3
Fix on IPOnly match at flows, for inline mode
16 years ago
Gurvinder Singh
50f7d0a887
app layer htp logging and better htp request handling. removed recent_in_tx.
16 years ago
Gurvinder Singh
4768e42159
bug 95 patch
16 years ago
Pablo Rincon
38dc7ffebc
Adding settings for detect engine group config
16 years ago
Anoop Saldanha
1a5ee37bd3
Added cuda logs for the engine, which shows device info and memory usage
16 years ago
Anoop Saldanha
c26e92733d
handle the cuda cleanup at shutdown. should get rid of any errors from the call to SigGroupCleanup
16 years ago
Anoop Saldanha
41e6735b92
mpm b2g cuda support added
16 years ago
Pablo Rincon
34216fd3e2
bug87 Fix IPOnly veredicts on flows
16 years ago
Pablo Rincon
0165b3f0d8
pcre P modifier support (pcre match over http body requests)
16 years ago
Pablo Rincon
ad2c136e8f
Renaming errors (naming conventions)
16 years ago
Breno Silva
7e299834d2
FragOffset Rule Keyword
16 years ago
Victor Julien
6a53ab9c5a
Stream engine memory handling update
...
The stream engine memory handling needed updating as it didn't scale. Changes:
- pools can now be initialized to size 0, meaning unlimited
- stream engine uses a memcap setting. Sessions, segments and aldata is part
of this, app layer state isn't.
- memory is accounted using a global int that is spinlocked.
- a counter for sessions that have not been picked up because of memcap was
added.
- all reassembly errors are converted to debug msgs.
16 years ago
Pablo Rincon
51dc773eec
Changing the veredict actions to flags to allow simultaneous veredict
16 years ago
Victor Julien
b3bcba077f
Only inspect http flows against uri sigs, clean up uri scanning code.
16 years ago
Gurvinder Singh
0cb43d27e9
uricontent new design
16 years ago
Gurvinder Singh
356a8bf385
applayer uri match and modified http handling
16 years ago
Victor Julien
0d34990d7f
Add OpenBSD's strlcpy and strlcat and replace all strcat/strcpy/strncat/strncpy by those calls.
16 years ago
Gurvinder Singh
f6b0c481b0
urilen support for engine
16 years ago
Victor Julien
4284276b11
Merge applayer detect function into normal match function. Should speed up detection.
16 years ago
Victor Julien
bcd0682150
Make engine startup a little less verbose.
16 years ago
Breno Silva
1d055b0e09
ICMP Seq Rule Keyword
16 years ago
Pablo Rincon
c80160b96d
More examples of unittest helper functions usage reference
16 years ago
Pablo Rincon
b6a3395c08
Adding unittest helper functions for building generic packets, checking arrays of expected match results, perform generic tests, etc. Look at util-unittest-helper.c and detect-ipproto.c for references
16 years ago
Victor Julien
eb67bb442e
Fixup unittests that use buffers that simulate configuration files. They now include the YAML header.
16 years ago
Gerardo Iglesias Galvan
40c514f295
Fix bug in logging msg when using --init-errors-fatal
16 years ago
Gerardo Iglesias Galvan
5eb819b0f4
Add signature line no. to error message when parsing fails
16 years ago
Gerardo Iglesias Galvan
fae92f8d7b
Fix bug#30. Fix logging call from prev patch
16 years ago
Gerardo Iglesias Galvan
988dc5520b
Improve output when loading rules
16 years ago
Victor Julien
6b36e23e45
Fix not decreasing the flow use_cnt reference counter in some cases from the app layer detection code. This caused some streams to never fully time out and thus clutter up the flow table and session pool.
16 years ago
Brian Rectanus
c22d42693a
Added http_method rule keyword.
16 years ago
Victor Julien
b7bac14040
Fixup code to compile with -Wall -Werror -Wextra -Wno-unused-parameter compiler options.
16 years ago
Victor Julien
2b66667a76
Make sure we can't overflow our packet alert storage
16 years ago
Anoop Saldanha
f684989f98
dce_iface, dce_opnum, dce_stub_data keyword support
16 years ago
Victor Julien
ecab1fae36
Remove contents of VRT classification.config.
16 years ago
Anoop Saldanha
011b74df63
Modify the classification config tests to use the buffer than a temp file and also fix an invalid free
16 years ago
Anoop Saldanha
bc4df59414
Support for Classtype keyword and Classification Config file
16 years ago
Victor Julien
d5c732f1f9
Add tag keyword stub
16 years ago
Jason Ish
ce20c33634
multiline rule support.
16 years ago
Will Metcalf
23aa6cf642
more fixes for exit on sig init failure
16 years ago
Victor Julien
778228d1c5
Flags keyword fix. Fatal init fix.
16 years ago
Victor Julien
35e884f303
Make sure offset modifies depth.
16 years ago
William Metcalf
c63b1e0f67
failing unit test depth doesn't take into account offset
16 years ago
Gurvinder Singh
8cfdf6c666
bug 18 patch update
16 years ago
Gurvinder Singh
b92886a79a
bug#18 and some minor changes
16 years ago
Victor Julien
10cc9d5b6a
Add icmp flow handling.
16 years ago
Victor Julien
71ed2d38f5
Fix scan patterns sometimes not being added to the scan ctx. Should fix bug #9 .
16 years ago
William Metcalf
c40e81d42b
failing unittest rules with same content match fail
16 years ago
Victor Julien
4824868766
Application layer detection improvements
...
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
16 years ago
Pablo Rincon
f2f9b83280
Adding FTP app layer parser and ftpbounce detection at L7
16 years ago
Gerardo Iglesias Galvan
7e87f373b9
Add icmp_id keyword support
16 years ago
Victor Julien
01976a0b74
Support for sigs with both pkt and applayer detect
...
Sets a flowbit with the sig id if the packet matches match. Checks
on that if the app layer matches match. Currently misuses the
flowbits api for this in a way that needs fixing.
16 years ago
Victor Julien
493715c0d2
Implement alert sid storage in the flow so we can check previous alerts in the flow.
16 years ago
Victor Julien
9fd46e9425
Support for sigs with both pkt and applayer detect
...
Sets a flowbit with the sig id if the packet matches match. Checks
on that if the app layer matches match. Currently misuses the
flowbits api for this in a way that needs fixing.
16 years ago
Victor Julien
4f843ff8e9
Improve matching of packet and app layer sigs.
16 years ago
Pablo Rincon
992aaa3d79
Small fix on SigMatchSignaturesAppLayer() and SigMatchSignatures()
16 years ago
Breno Silva
69eb869cc9
Threshold Rule
16 years ago
Victor Julien
ecf86f9c23
Rename to Suricata.
16 years ago
Gurvinder Singh
a0f184866c
http_cookie keywork support
16 years ago
Pablo Rincon
a67bd2457b
Loading rules from config support
16 years ago
Pablo Rincon
c816af822e
Adding support to load rule files from config
16 years ago
Pablo Rincon
1ad6d75dfe
Added rpc keyword support at packet level
16 years ago
Pablo Rincon
a8d7b71490
First version of flowints
16 years ago
Gerardo Iglesias
991d421394
Changed printf's to logging API functions
16 years ago
Victor Julien
2cfa284999
Fix app layer detect to actually work.
16 years ago
Victor Julien
a1d33e2d0b
Compilation fixes after merge.
16 years ago
Victor Julien
f1f7df0766
First iteration of doing app layer detection.
16 years ago
Anoop Saldanha
7a10ddc07b
Fixes for the fast-pattern tests and a couple of other minor changes
16 years ago
Anoop Saldanha
6ca5dbc9e9
Support fast_pattern modifier keyword for content
16 years ago
Victor Julien
07bcc8cc9a
Fix signatures with ports and/or addresses but without sigmatches.
16 years ago
Anoop Saldanha
1c7ac13c25
Support for negated content
16 years ago
Victor Julien
241db64dd7
Fix a few memory issues.
16 years ago
Victor Julien
981ca859cb
Fixes for distance and within content modifiers.
16 years ago
William Metcalf
884b1034b4
failed unittest for within distance
16 years ago
Anoop Saldanha
dc44700ce5
Support vars lookup from conf file. Current patch support address and port group vars lookup
16 years ago
Victor Julien
951b4d5cf4
Fix within in some corner cases and add some more tests.
16 years ago
Victor Julien
4f2164677a
- Fix pattern matchers b2g and b3g not being able to deal with a single pattern of the max pattern length (32 bytes by default).
...
- Fix the setting of the correct pattern matcher when it was set in the detection ctx.
- Add tests for the fixes.
16 years ago
Victor Julien
d6be6ceb19
Fixup artifact from automatic renames: rename DetectAddresssHead to DetectAddressHead.
16 years ago
Anoop Saldanha
78db80ea15
Unittests and style fix for detect-engine-siggroup.[ch]
16 years ago
Gurvinder Singh
cacbf31aad
support for ttl keyword
16 years ago
Victor Julien
de8caa7964
Rename DetectAddressGroup* to DetectAddress*
16 years ago
Victor Julien
13dca5e8ce
Cleanup of address functions.
16 years ago
Victor Julien
3c7a038477
Further memory cleanups. Split out init only vars out of the sig group head.
16 years ago
Victor Julien
32e51e5e5a
Replace sgh refcnt by a flag.
16 years ago
Victor Julien
2d0e9658f8
Speed up per sgh content maxlen calc. Remove mpm ptrs from mpm ctx. Add unittests testing the detection engine internals.
16 years ago
Victor Julien
e4c98c562c
Merge DetectAddressData and DetectAddressGroup
16 years ago
Victor Julien
15ab5d7003
More engine init memleaks fixed. HashListTable remove function fixed.
16 years ago
Victor Julien
7a7bb7a390
Get rid of global mpm_ctx.
16 years ago
Victor Julien
fbe87a3ad5
Bunch of mostly unittest related memleak fixes.
16 years ago
Victor Julien
227cc082a1
Fix small memleak at engine init.
16 years ago
Brian Rectanus
ed30067bd7
Ack/Seq Keywords
16 years ago
Brian Rectanus
ec6c5258b6
Sameip Keyword
16 years ago
Victor Julien
f76a740fe2
Small layout fix.
16 years ago
Victor Julien
b470e7b40f
Fixup merge artifact.
16 years ago
Breno Silva
15a8f34d36
Gid Keyword
...
Signed-off-by: Brian Rectanus <brectanu@gmail.com>
16 years ago
Breno Silva
6100a7f610
FragBits Keyword
...
Signed-off-by: Brian Rectanus <brectanu@gmail.com>
16 years ago
Victor Julien
d4733e3a76
Move pcre test out of detect.c
16 years ago
William Metcalf
86380b9d45
split out pcre parsing fixed some errors
16 years ago