Victor Julien
575c87aeba
engine events: prefilter sigs that need a event
13 years ago
Victor Julien
c91c359692
profiling: fix build on older systems
13 years ago
Victor Julien
ec7e79c748
Rule profiling update
...
- Remove usage of counters api.
- Store stats in detect engine thread ctx to remove locking
- Support rule reloads
13 years ago
Victor Julien
ba3260ed38
Thread local ctx for detection keywords
...
Some detection keywords need thread local ctx storage. Example is the
filemagic keyword that has a ctx that is modified with each call. That
is not thread safe. This functionality allows registration of thread
local ctxs so that each detect thread works on it's own copy.
13 years ago
Victor Julien
f58e828c5e
luajit: stub detection keyword
13 years ago
Eric Leblond
71b4257bc2
sig: add l3_proto keyword
...
This patch adds a l3_proto keyword to the signature language. It
can be used to specify if the signature has to match on IPv4, IPv6
or both. For example, one can write:
alert http any any -> any 22 (msg: "HTTP v6"; l3_proto:ip6; sid:14;)
This should close #494 .
13 years ago
Victor Julien
8f71333e12
file: implement filesize keyword. #489 .
13 years ago
Victor Julien
3849588c61
Create separate detect API call (FileMatch) for file detection keywords. #531 .
13 years ago
Eric Leblond
7e09cdc265
Delay Detect threads initialization
...
This patch modifies the init of Detect threads. They are now started
with a dummy function and their initialisation is done after the
signatures are loaded. Just after this, the dummy function is switched
to normal one.
In IPS mode, this permit to route packets without waiting for the
signature to start and should fix #488 .
Offline mode such as pcap file don't use this mode to be sure to
analyse all packets in the file.
The patch introduces a "delayed-detect" configuration variable
under detect-engine. It can be used to activate the feature
(set to "yes" to have signature loaded after capture is started).
13 years ago
Jean-Paul Roliers
c4df7a45ae
tls: adding store option for TLS
...
This patch adds a TLS store option to save certificate in PEM format.
Each time the store action is met, a file and a metafile are created.
Reworked-by: Eric Leblond <eric@regit.org>
13 years ago
Jean-Paul Roliers
00d4357362
tls: adding support for fingerprint rule matching.
...
Add the support for tls.fingerprint keyword in rules.
13 years ago
Victor Julien
c9e93ec52c
filemd5: add support code for md5 handling for signatures.
13 years ago
Anoop Saldanha
0c24bbab0c
code cleanup for live swap
13 years ago
Anoop Saldanha
ecad4a24fa
live rule support added
...
To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup
13 years ago
Anoop Saldanha
5878d83174
byte_extract_id var now a non-global de_ctx specific var
13 years ago
Anoop Saldanha
f4ce9011d2
make mpm ctx container de_ctx specific. Also introduce global variable in mpm_ctx. this is a workaround for cleaning non global mpm_ctx's since we now don't supply the de_ctx around the detection engine API
13 years ago
Anoop Saldanha
f2dd61868d
variable names global vars, global no more. Moved to detection engine ctx, a place it belongs
13 years ago
Anoop Saldanha
f9612f3b83
mpm engine cleanup. Remove unnecessary flags
13 years ago
Anoop Saldanha
988c92f71c
http user agent keyword + mpm + inspection + fast pattern support added
13 years ago
Victor Julien
9f7588a756
Add filemd5 keyword that loads a list of md5's to match a file's md5 against.
13 years ago
Victor Julien
cae46ab5eb
pcre: print filename and line number for JIT warning.
13 years ago
Anoop Saldanha
493c3db413
fix FNs for flow- only_stream and no_stream options
13 years ago
Victor Julien
d378b76c04
http: body inspection improvement
...
Improve http_client_body and file_data performance when request and
response body limits are set to high values.
13 years ago
Eric Leblond
ad0e05a112
TLS app layer: Add tls.issuerdn keyword.
14 years ago
Pierre Chifflier
4be65fd016
TLS handshake: decode the SERVER_CERTIFICATE message
...
Add a decoder for the SERVER_CERTIFICATE during a TLS handshake, extracts the
certificates and keep the subject name.
Add the tls.subject keyword for substring match in rules (TLS layer).
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
14 years ago
Victor Julien
c0a2cbd478
Move over src and dst thresholding to use host table. Fix a bug in threshold 'both' handling.
14 years ago
Victor Julien
a05df345de
Introduce host table, make tag use it
...
Add a host table similar to the flow table. A hash using fine grained
locking. Flow manager for now takes care of book keeping / garbage
collecting.
Tag subsystem now uses this for host based tagging instead of the
global tag hash table. Because the latter used a global lock and the
new code uses very fine grained locking this patch should improve
scalability.
14 years ago
Anoop Saldanha
e682796d03
feature #414 - support listing supported keywords. Remove support for dummy keywords __address__, __proto__, __port__. Remove support for recursive keyword and all references to it
14 years ago
Anoop Saldanha
603d4a719a
remove det_ctx->payload_offset and use det_ctx->buffer_offset. Update hscd and hsmd to use the new generic content inspection engine
14 years ago
Anoop Saldanha
35f1f7e8d9
unify payload detection engines + fix other bugs in pcre init
14 years ago
Anoop Saldanha
09313cf9bd
Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S
14 years ago
Anoop Saldanha
2007c2711c
Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword
14 years ago
Anoop Saldanha
419cdc8558
support splitting mpm ctxs based on direction v2
14 years ago
Victor Julien
c8c4a76dc6
Move threshold to it's own sig match list.
14 years ago
Victor Julien
6eeab37ab3
Add post-match list, move flowbits set, etc functions to it. Move flowint set, etc functions to it as well.
14 years ago
Victor Julien
21ee59e6f3
Add signature direction (flow:toserver/flow:toclient) as a signature flag.
14 years ago
Anoop Saldanha
eea5ab4a7a
Support for app layer decoder events added + app_layer_event keyword added
14 years ago
Anoop Saldanha
92643f6110
introduce separate mpm ctxs for tcp/udp/other_protos
14 years ago
Victor Julien
298289f43f
Let flow:only_stream and flow:no_stream set the require packet and require stream flags. Toss out sigs with conflicting settings. Rename flow:stream_only to flow:only_stream. Fixes #261 .
14 years ago
Victor Julien
18d79c4215
file store: respect flowbits and other keywords
...
The filestore keyword until now flagged a file, tx or ssn for storage as soon
as the keyword was inspected. This happens before flowbits and some other
keywords, so files were stored that weren't supposed to.
This patch makes the filestore keyword fill an array in the detect engine
thread ctx. Then if the full signature matches, a post-match filestore
function makes the store final.
14 years ago
Victor Julien
07e560b137
file-data: initial file_data support
...
Support file_data for: content, pcre (relative), byte_test, byte_jump,
byte_extract, isdataat.
File_data support is handled at signature parsing time, all matches
occurring after the file_data in the rule are converted to http_server_body
matches.
Content matches relative to the file_data are converted. Within to depth,
distance to offset. Relative to the start of the body buffer.
14 years ago
Victor Julien
7adac3048d
file-data: create initial keyword registration.
14 years ago
Victor Julien
fa0152fa80
Shrink signature flags field to 32 bits.
14 years ago
Victor Julien
dd9da1a56f
Merge all http mpm related signature flags into a single set: SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_HTTP_NEG.
14 years ago
Victor Julien
d5ed28b065
Remove SIG_FLAG_MPM flag.
14 years ago
Victor Julien
fe48920514
Remove per sgh mpm_streamcontent_maxlen variable. It was checked but never set.
14 years ago
Victor Julien
4992f7c417
Remove SIG_FLAG_MPM_URI flag. It was checked but never set.
14 years ago
Victor Julien
2650551192
Rename signature init flags to indicate they are init flags.
14 years ago
Victor Julien
6ebd71545b
Fix signature flag definitions on 32 bit.
14 years ago
Victor Julien
291ddd95f2
Detection engine -- mpm
...
Each signature is in one mpm ctx at max, but there were 3 separate
id's in use: packet, stream, http. Merged them all into one.
Could shrink the SignatureHeader structure with 8 bytes because of this,
should lead to better caching performance.
14 years ago
Victor Julien
7db72bce75
Optimize detection engine prefiltering logic.
14 years ago
Victor Julien
89f83e714c
Introduce http_server_body keyword.
...
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".
To facilitate this change the signature flags field was changed to be 64 bit.
14 years ago
Victor Julien
04ea70ccf7
file extract: pruning
...
Add pruning of files in memory so we keep only memory what we really need.
Fix magic logic.
Reset file part of the de_state on receiving another file in the same tx.
14 years ago
Victor Julien
9b62ec65ab
Make sure filemagic works properly regardless of filestore being in use for a flow.
14 years ago
Victor Julien
5945e652d6
Initial implementation of filemagic keyword.
14 years ago
Victor Julien
23e01d23d3
Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored.
14 years ago
Pablo Rincon
6d60b3a747
filename and fileext keywords
14 years ago
Anoop Saldanha
2b356dadff
Support for tos keyword added
14 years ago
Victor Julien
8cc82c7241
Add -S commandline option that loads a rule file exclusively. Issue #338 .
14 years ago
Anoop Saldanha
c7b9d3fecb
Remove broken dsize_sm in SigMatch used by dsize in detection engine
14 years ago
Anoop Saldanha
ba6bada155
change rev field in Signature to u32 and use strotoul to extract the value. Cleanup some dead code/comments
14 years ago
Eric Leblond
a85dc9b0e2
Add support for replace keyword.
...
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.
One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.
Replace is only activated in IPS mode and apply only to raw match.
14 years ago
Eric Leblond
0c34a1c5e7
rewrite constants and add flag for replace
...
This patch make use of bit shift to rewrite some of the mask constants.
It also delete an unused flag value and suppress the associated dead code.
The numeric value of the flag is now used by the flag needed for replace
code.
14 years ago
Eric Leblond
3f153fb0da
Add 'stream-event' keyword.
...
This patch adds an alias to the 'engine-event' keyword. It is now
possible to access to the stream events via the 'stream-event'
keyword. A simple transformation is done:
stream-event:reassembly_segment_before_base_seq
is a shortcut for:
engine-event:stream.reassembly_segment_before_base_seq
14 years ago
Eric Leblond
e3a6d8955e
Introduce engine-event keyword
...
This patch renames the 'decode-event' keyword to 'engine-event' and
keep it for backword compatibility of rulesets. All *DecodeEvent*
references in the code are replaced by EngineEvent version.
14 years ago
Eric Leblond
03c185a3ad
threshold: add suppress keyword
...
This patch adds the suppress keyword to the threshold.config file.
The alerts are suppressed but the other elements like flowbits are
maintained.
14 years ago
Anoop Saldanha
35f3eafa5e
byte extract added to the engine. Detection support added for packet payload, uri and dce detection engines
14 years ago
Victor Julien
1174df9712
Fix passing a uint8_t as an int. Breaks on some args.
14 years ago
Victor Julien
1e0b050a54
Add more mask flags.
14 years ago
Victor Julien
2dbfdd40af
Clean up new SIMD mask checking code, improve non-SIMD checks.
14 years ago
Victor Julien
b421019cef
Match packet mask against 16 signature masks at once using SIMD instructions for SSE3 and up.
14 years ago
Victor Julien
75439863ed
Shrink PacketAlerts structure so that Packet structure is a lot smaller. Reduce max events per packet from 256 to 15.
14 years ago
Anoop Saldanha
966119b6aa
support for http_raw_uri keyword + mpm engine
14 years ago
Eric Leblond
9beebf621a
Add support for 'nfq_set_mark' keyword
...
This patch introduces 'nfq_set_mark' which is new rules option. If a packet
matches a rule using nfq_set_mark in NFQ mode, it is marked with the mark/mask
specified in the option during the verdict.
It is thus possible to trigger different behaviour on the packet inside
Linux/Netfilter.
15 years ago
Anoop Saldanha
c105a739e9
support for ssl_state keyword added
15 years ago
Gurvinder Singh
8f8b1212af
support for ssl_version keyword
15 years ago
Anoop Saldanha
c9897a44a4
fast pattern support for http_cookie. Also support relative modifiers
15 years ago
Anoop Saldanha
bbbedaf963
fast pattern support for http_method. Also support relative modifiers
15 years ago
Victor Julien
39dea56a84
Remove flowbits as a mask prefilter as they are dynamic. Add a dynamic check.
15 years ago
Anoop Saldanha
8c21511c99
don't buffer raw headers. Retrieve them individually from htp_state during mpm stage and content valiadation stage
15 years ago
Victor Julien
435d0fb327
Clean up signature flags creating room for merging flags and mpm_flags. Merge flags and mpm_flags. Move new mpm id's into signature header. Get rid of full signature access in signature prefiltering.
15 years ago
Victor Julien
169aa5581a
Rename SIG_FLAG_AMATCH flag to SIG_FLAG_STATE_MATCH to better reflects its purpose.
15 years ago
Victor Julien
d7b92d9bfe
Consolidate several signature flags into one.
15 years ago
Anoop Saldanha
25588b6910
comment out hrhd flags that we were using previously. Also remove the de_mpm_ based flags inside detect.h used by uri|hcbd|hhd|hrhd mpms. indentation fix as well
15 years ago
Anoop Saldanha
72b0fcf419
modify detection engine to carry out uri mpm run before build match array if alproto is http and if sgh has atleast one sig with uri mpm set
15 years ago
Anoop Saldanha
7ec0382774
support fast pattern for http raw header. Also support relative modifiers for http raw header
15 years ago
Anoop Saldanha
c61c68fd36
mpm and fast pattern support for http_header. Also support relative modifiers for http_header
15 years ago
Anoop Saldanha
778ec0939c
make client body buffer limit configurable. Also some minor changes
15 years ago
Anoop Saldanha
0aa5cffb12
fast pattern support for http_client_body keyword added. Also mpm support for http_client_body added
15 years ago
Anoop Saldanha
bbd0c5056b
store the content added for mpm inside Signature. also carry out an unconditional cleanup of packet pattern matcher pmq det_ctx->pmq
15 years ago
Anoop Saldanha
6df051321f
fix fp when content is negated and also added to mpm
15 years ago
Anoop Saldanha
5c6a65dc58
support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch]
15 years ago
Anoop Saldanha
eade60f0fd
make some name changes. break PopulateMpm(). Set the avoid mpm double check flags
15 years ago
Anoop Saldanha
6eaba8941c
Use new flags to indicate uricontent has a mpm set
15 years ago
Anoop Saldanha
ede7be34b5
replace all Signature->tmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_TMATCH]
15 years ago
Anoop Saldanha
3d2f81d978
replace all Signature->dmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_DMATCH]
15 years ago
Anoop Saldanha
a7353be20d
replace all Signature->amatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_AMATCH]
15 years ago
Anoop Saldanha
e0476242c6
replace all Signature->umatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_UMATCH]
15 years ago
Anoop Saldanha
e54358a9e1
replace all Signature->pmatch instances in the engine with Signature->sm_lists[DETECT_SM_LIST_PMATCH]
15 years ago
Anoop Saldanha
82fd581b64
replace all sm lists (match, pmatch, dmatch, umatch, amatch, tmatch) with an array Signature->sm_lists[]. Replace all Signature->match instances in the engine with Signature->sm_lists[DETECT_SM_LIST_MATCH]
15 years ago
Victor Julien
001f91056e
Add http_raw_header as an alias to the http_header keyword as that actually inspects the raw headers (see issue #243 ). Closes issue #242 .
15 years ago
Gurvinder Singh
b7da115e6d
support for http_stat_code keyword has been added to detection module
15 years ago
Gurvinder Singh
1deae70cf7
added http_stat_msg keyword support for detection module
15 years ago
Anoop Saldanha
bc99328ec8
define a new conf paramter detect-engine:inspection-recursion-limit; Defines a recursion limit for content inspection code
15 years ago
Anoop Saldanha
88d94b136d
Support for reference.config file
15 years ago
Anoop Saldanha
0ef684705c
support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup
15 years ago
Anoop Saldanha
b367c37ae6
suricata.yaml conf update to support single mpm context distribution over multiple sghs + code to parse this conf
15 years ago
Victor Julien
cbd4c298ed
Initial version of a new bitmask based signature pre-filtering method.
15 years ago
Victor Julien
fc248ca7a1
Many small performance updates.
15 years ago
Victor Julien
57b098e98c
Add padding to commonly used data structures.
15 years ago
Pablo Rincon
f225bd1428
Adding modifiers /C /H and /M to pcre (http cookie, header and method)
15 years ago
Pablo Rincon
9d7baa7a9f
Adding ssh app layer module with two new keywords: ssh.protoversion and ssh.softwareversion
15 years ago
Victor Julien
9ba11dbfbd
Clean up detection engine mpm initialization phase.
15 years ago
Anoop Saldanha
b94eaec7c2
implement relative pcre matching in detect-engine-(payload|uri|dcepayload).c. Also fix within/distance handling of RELATIVE_NEXT flag for uricontent
15 years ago
Anoop Saldanha
ae3148aded
fix false positives for a negated content case
15 years ago
Victor Julien
102092a89c
Make signature address matching more cache efficient.
15 years ago
Victor Julien
1eec149f5e
Use Address structure in DetectAddress struct.
15 years ago
Pablo Rincon
693d4f54eb
Load signatures with incompatible fast_pattern option (due to design differences for optimization)
15 years ago
Victor Julien
d41b5645ef
Make sure decoder event rules are inspected even if the packet is invalid and has no addesses or proto. Update fast log and alert debug log to display the alerts. Fixes #179 .
15 years ago
Victor Julien
67429e523f
Fix compiler warning about incomplete prototype.
15 years ago
Victor Julien
7454336ef5
Make SigWrapper private to detect-parse.c and rename to SigDuplWrapper to reflect it's use and purpose.
15 years ago
Anoop Saldanha
9ecade76b9
in case of duplicate signatures used the one with the latest revision
15 years ago
Gurvinder Singh
8852b83fa7
flowbits, flowvars, pktvars, flow flags and app layer info added to alert-debug.log
15 years ago
Victor Julien
580b09c2b8
Make sure we inspect all outstanding reassembled stream chunks (smsg) if the stream is shutting down. Make sure to do inspect signatures that use dsize against the tcp packet payload, even if that payload was already added to the stream. Likewise, the dsize signatures are not inspected against the reassembled stream.
15 years ago
Pablo Rincon
eed0ef6e69
Adding tag keyword support
15 years ago
Pablo Rincon
6950537a76
Move rate_filter rule tracking data from Signature to Threshold context
15 years ago
Victor Julien
37442a8a84
Prefilter signatures before fully scanning them.
15 years ago
Anoop Saldanha
45ea0d914e
dce stub content keywords support using dcepayload.c support for all dce related content keywords
15 years ago
Victor Julien
83b2c8abdb
Improve stateful uri detection code.
15 years ago
Victor Julien
9dd753b5f3
Scan uricontent mpm on demand.
15 years ago
Victor Julien
a24f288074
Moving the stream content scanning to have it's own mpm ctx.
15 years ago
Victor Julien
a0c1209a44
Inspect the reassembled stream together with the packet payload in the same direction.
15 years ago
Victor Julien
2fd31a1a11
Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory.
15 years ago
Victor Julien
dff6795df5
Detect cleanups.
15 years ago
Gerardo Iglesias Galvan
55dfa36963
Add support for http_uri keyword
15 years ago
Victor Julien
7a6d4b57f0
Add support for class id in classification code. Submitted by firnsy@securixlive.com, thanks.
15 years ago
Pablo Rincon
3fa3229e01
ASN1 decoder and keyword implementation
15 years ago
Victor Julien
70b32f7380
First stab at creating a stateful detection engine.
...
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:
- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.
This commit fixes bug #124 .
15 years ago
Jason Ish
18e5ac8cde
Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules.
15 years ago
Victor Julien
e27cefa6f7
Complete conversion of pattern id mpm storage vs sig id storage.
15 years ago
Victor Julien
46831e0f8f
Fix signature grouping bug for protocols without ports. Add debugging code.
15 years ago
Victor Julien
7a427ec7f4
Switch to pattern id based results checking in the mpm. Move app layer proto detection towards a more signature based approach.
15 years ago
Victor Julien
a372c1d14e
Fix/workaround a strange detection issue.
15 years ago
Pablo Rincon
e18e2ec998
Changing threshold logic
15 years ago
William Metcalf
ce01927515
Import of GPLv2 Header 050410
15 years ago
Pablo Rincon
ab02ab9ead
adding http_header keyword support
15 years ago
Anoop Saldanha
47037ef9ec
fix for bug 115
15 years ago
Victor Julien
fe7ece997a
Different approach to the reference keyword. Lots of cleanups, bug fixes in reference keyword code and tests.
15 years ago
Breno Silva
89baf93a40
Reference Support
15 years ago
Pablo Rincon
e7a989e305
IP Only Engine using radix trees
16 years ago