354 Commits (e75a93b1256e604d2528baff76b9c236057dba64)

Author	SHA1	Message	Date
Giuseppe Longo	a1d8439b25	iprep: extends cidr support ... Implements new API to expand the IP reputation to netblocks with CIDR notation A new object 'srepCIDRTree' is kept in the DetectionEngineCtx, which contains two tree (one for ipv4 and one for ipv6) where the reputation values are stored.	11 years ago
Victor Julien	d9c523a332	filestore: fix parsing bug ... Filestore keyword can have options or no options, and the parser was enforcing the NOOPT flag too strictly. Bug #1288	11 years ago
Victor Julien	3b98a1ce66	detect: track current tx_id in det_ctx ... When using the inspection engines, track the current tx_id in the thread storage the detect thread uses. As 0 is a valid tx_id, add a simple bool that indicates if the tx_id field is set.	11 years ago
Victor Julien	f23399d672	Rename Lua code to just Lua ... As we support regular Lua as well as LuaJIT, it makes more sense to call it all Lua.	11 years ago
Victor Julien	1af39d55cf	detect: allow alias registration for rule keywords ... This allows for registering a keyword under another name while keeping the old name active and supported. Do this for 'luajit', which can now also be used as just 'lua'.	12 years ago
Victor Julien	f5f148805c	app layer: uint16_t alproto -> AppProto alproto ... This conversion was missing in a couple of places.	12 years ago
Victor Julien	d397ed94c5	detect: use macro for max rule size	12 years ago
Victor Julien	5686c673ec	profiling: per buffer profiling	12 years ago
Victor Julien	9419b9cbb6	detect: add tostring function for DETECT_SM_LIST_ enum.	12 years ago
Victor Julien	97bfcac444	profiling: introduce per keyword profiling ... Initial version of per keyword profiling. Prints stats about how ofter a keyword was checked and what the costs were.	12 years ago
Victor Julien	7ebd1e6433	Counters: fix delayed-detect counter registration ... Make sure we register the detect.alerts counter before packet runtime starts even in delayed detect mode. The registration of new counters at packet runtime is not supported by the counters api and might lead to crashes as there is no proper locking to allow for this operation. This changes how delayed detect works a bit. Now we call the ThreadInit callback twice. The first call will only register the counter. The 2nd call will do all the other setup. This way the counter is registered before the counters api starts operating in the packet runtime. Fixes the segv reported in ticket #1018.	12 years ago
Victor Julien	8ce38ac8fe	Split Thresholds and Suppression ... Thresholds and suppression can be handled independently. Suppression only suppresses output, and is not related to Threshold state tracking. This simplifies mixing suppression and thresholding rules. Part of the Bug #425 effort.	12 years ago
Ken Steele	85a51638c9	Improve Signature sorting speed ... Changed the signature sorting code to use a a single merge sort instead of the multiple pass sorting that was being used. This reduces startup time on Tile by a factor of 3. Also replace the user array of pointers to ints with a simpler array of ints.	12 years ago
Anoop Saldanha	d0c5f51293	Update rule engine relationship with regard to setting ip protocol between specifying protocol after action, ip_proto and app-layer-protocol. ... Now we can specify alproto, ip_proto combinations this way alert dns (ip_proto:[tcp/udp];) alert ip (app-layer-protocol:dns;) alert ip (app-layer-protocol:dns; ip_proto:tcp;) alert tcp (app-layer-protocol:dns:) so on. Neater than using dnstcp/dnsudp. This is related to feature #424.	12 years ago
Anoop Saldanha	f592c481dc	Introduce a separate inspection engine for app events.	12 years ago
Anoop Saldanha	94e40907e2	feature #727 - Add support for app-layer-protocol:<protocol> keyword	12 years ago
Ken Steele	50f859e9f2	Move SIMD implementations out of detect.c ... Move SIMD the implementations of SigMatchSignaturesBuildMatchArray() for SSE3 and Tile out of detect.c to reduce the size of the file. Also moved SIMD unit tests to detect-simd.c	12 years ago
Ken Steele	eb4f0da97f	Change one more atomic size in detect.h ... Change uint16_t to int for better tile atomic performance. Checked with pahole that it doesn't increase the size of the structure.	12 years ago
Ken Steele	784843b146	Use Tilera SIMD for Signature matching ala SSE3 ... Makes use of 8-wide byte compare instructions in signature matching. For allocating aligned memory, _mm_malloc() is SSE only, so added check for __tile__ to use memalign() instead. Shows a 13% speed up.	12 years ago
Victor Julien	055b422c28	Remove obsolete code: flow alert sid storage	12 years ago
Eric Leblond	2f2916d9ec	ccccinelle: add formatted comment for flag test	12 years ago
Victor Julien	7292998a58	Content: set up sticky buffers like file_data and dce_stub_data w/o flags, but with a list variable	12 years ago
Anoop Saldanha	fba95e9125	Remove mpm ctxs in the wrong direction. ... A lot of http mpm ctxs have now been removed as a result of this.	12 years ago
Victor Julien	6645620c03	Merge SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_DNS into SIG_FLAG_MPM_APPLAYER, do the same for the _NEG variant.	12 years ago
Victor Julien	43ba5a677e	DNS: enable mpm/fast_pattern support for dns_query	12 years ago
Victor Julien	f10dd603ff	DNS: adding dns_request content modifier	12 years ago
Anoop Saldanha	b787da5643	Remove all cuda related code in the engine except for the cuda api wrappers	12 years ago
Victor Julien	6e18ed0489	luajit flowvar support ... This patch adds flowvar support to luajit. It does so by exposing two special C functions to the luajit scripts: ScFlowvarGet and ScFlowvarSet.	12 years ago
Anoop Saldanha	ab1f8afbc3	Removed Signature->order_id and replaced it with Signature->num.	12 years ago
Anoop Saldanha	d4d18e3136	Transaction engine redesigned. ... Improved accuracy, improved performance. Performance improvement noticeable with http heavy traffic and ruleset. A lot of other cosmetic changes carried out as well. Wrappers introduced for a lot of app layer functions. Failing dce unittests disabled. Will be reintroduced in the updated dce engine. Cross transaction matching taken care of. FPs emanating from these matches have now disappeared. Double inspection of transactions taken care of as well.	12 years ago
Victor Julien	4cd736fcc9	flowvar: fix deadlock with http buffers ... Bug #802 Flowvars are set from pcre, and lock the flow when being set. However when HTTP buffers were inspected, flow was already locked: deadlock. This patch introduces a post-match list in the detection engine thread ctx, where store candidates are kept. Then a post-match function is used to finalize the storing if the rule matches. Solves the deadlock and brings the handling of flowvars more in line with flowbits and flowints.	12 years ago
Victor Julien	9ea4d36f7a	Minor reshuffling of Signature struct.	12 years ago
Victor Julien	eb11280888	Use define instead of magic number for pmq's per detect thread	12 years ago
Anoop Saldanha	4c6efa2d40	Update content id assignment. ... All fp id assignment now happens in one go. Also noticing a slight perf increase, probably emanating from improved cache perf. Removed irrelevant unittests as well.	13 years ago
Anoop Saldanha	c63317d02e	Detect sm_list rearranged for performance reasons.	13 years ago
Anoop Saldanha	51dcf19817	turn dce_stub_data into a sticky buffer.	13 years ago
Anoop Saldanha	3511f91bba	Add support for the new keyword - http_raw_host header. ... The corresponding pcre modifier would be 'Z'.	13 years ago
Anoop Saldanha	c4ce19a1be	Add support for a new keyword to inspect http_host header. ... The corresponding content keyword would now be - http_host. The corresponding pcre modifier would be W.	13 years ago
Ignacio Sanchez	d771e08156	Adds support for the geoip keyword ... Adds support for match-on conditions (src, dst, any, both) Uses GEOIP_MEMORY_CACHE for performance reasons Adds support for negation and multiple countries in the same rule Bug fixes Changed to take flow direction from rule, if present Comments addressed. Unit tests added.	13 years ago
Eric Leblond	40891223e9	list-keyword: detect non built keyword ... This patch update the glafs list to be able to indicate that a flag is not supported. This information is used by list-keyword to display information to the user.	13 years ago
Eric Leblond	6842545331	Add documentation url in list-keyword output. ... The output of the list-keyword is modified to include the url to the keyword documentation when this is available. All documented keywords should have their link set. list-keyword can be used with an optional value: no option or short: display list of keywords csv: display a csv output on info an all keywords all: display a human readable output of keywords info $KWD: display the info about one keyword.	13 years ago
Eric Leblond	5e4552fdcd	suricata: update list-keyword command ... This patch update the list-keyword command. Without any option, the previous behavior is conserved. If 'all' is used as option, suricata print a csv formatted output of keyword information: name;features;description If a keyword name is used as argument, suricata print a readable message: tls.subject Features: state inspecting Description: Match TLS/SSL certificate Subject field	13 years ago
Victor Julien	e30b1bfe64	Simple IP reputation implementation	13 years ago
Xavier Lange	234922f3c6	Keyword pkt_data	13 years ago
Victor Julien	70bc9e2494	filestore: fix logic flag in continued stateful detection	13 years ago
Victor Julien	fd6df00684	Bug 585: use per detect thread libmagic ctx	13 years ago
Anoop Saldanha	b0e20a486c	update client/server/http_header to use a different form of ... buffering/buffer_retrieval. Now it happens per tx, based on tx id. Also notice a perf improvement with this.	13 years ago
Victor Julien	10a11b750d	Add dsize check to prefilter stage ... Many sigs with dsize have a weak fast_pattern. Those patterns are likely to match. By filtering on dsize early, we safe a lot of cycles later.	13 years ago
Victor Julien	45cbef0735	For signatures with the dsize option set depth on any content match in that sig.	13 years ago
Victor Julien	98484ffdcc	luajit: prealloc lua states to increases chances of alloc success. Luajit requires them to be in memory <2GB.	13 years ago
Victor Julien	575c87aeba	engine events: prefilter sigs that need a event	13 years ago
Victor Julien	c91c359692	profiling: fix build on older systems	13 years ago
Victor Julien	ec7e79c748	Rule profiling update ... - Remove usage of counters api. - Store stats in detect engine thread ctx to remove locking - Support rule reloads	13 years ago
Victor Julien	ba3260ed38	Thread local ctx for detection keywords ... Some detection keywords need thread local ctx storage. Example is the filemagic keyword that has a ctx that is modified with each call. That is not thread safe. This functionality allows registration of thread local ctxs so that each detect thread works on it's own copy.	13 years ago
Victor Julien	f58e828c5e	luajit: stub detection keyword	13 years ago
Eric Leblond	71b4257bc2	sig: add l3_proto keyword ... This patch adds a l3_proto keyword to the signature language. It can be used to specify if the signature has to match on IPv4, IPv6 or both. For example, one can write: alert http any any -> any 22 (msg: "HTTP v6"; l3_proto:ip6; sid:14;) This should close #494.	13 years ago
Victor Julien	8f71333e12	file: implement filesize keyword. #489.	13 years ago
Victor Julien	3849588c61	Create separate detect API call (FileMatch) for file detection keywords. #531.	13 years ago
Eric Leblond	7e09cdc265	Delay Detect threads initialization ... This patch modifies the init of Detect threads. They are now started with a dummy function and their initialisation is done after the signatures are loaded. Just after this, the dummy function is switched to normal one. In IPS mode, this permit to route packets without waiting for the signature to start and should fix #488. Offline mode such as pcap file don't use this mode to be sure to analyse all packets in the file. The patch introduces a "delayed-detect" configuration variable under detect-engine. It can be used to activate the feature (set to "yes" to have signature loaded after capture is started).	13 years ago
Jean-Paul Roliers	c4df7a45ae	tls: adding store option for TLS ... This patch adds a TLS store option to save certificate in PEM format. Each time the store action is met, a file and a metafile are created. Reworked-by: Eric Leblond <eric@regit.org>	13 years ago
Jean-Paul Roliers	00d4357362	tls: adding support for fingerprint rule matching. ... Add the support for tls.fingerprint keyword in rules.	13 years ago
Victor Julien	c9e93ec52c	filemd5: add support code for md5 handling for signatures.	13 years ago
Anoop Saldanha	0c24bbab0c	code cleanup for live swap	13 years ago
Anoop Saldanha	ecad4a24fa	live rule support added ... To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup	13 years ago
Anoop Saldanha	5878d83174	byte_extract_id var now a non-global de_ctx specific var	13 years ago
Anoop Saldanha	f4ce9011d2	make mpm ctx container de_ctx specific. Also introduce global variable in mpm_ctx. this is a workaround for cleaning non global mpm_ctx's since we now don't supply the de_ctx around the detection engine API	13 years ago
Anoop Saldanha	f2dd61868d	variable names global vars, global no more. Moved to detection engine ctx, a place it belongs	13 years ago
Anoop Saldanha	f9612f3b83	mpm engine cleanup. Remove unnecessary flags	13 years ago
Anoop Saldanha	988c92f71c	http user agent keyword + mpm + inspection + fast pattern support added	13 years ago
Victor Julien	9f7588a756	Add filemd5 keyword that loads a list of md5's to match a file's md5 against.	13 years ago
Victor Julien	cae46ab5eb	pcre: print filename and line number for JIT warning.	13 years ago
Anoop Saldanha	493c3db413	fix FNs for flow- only_stream and no_stream options	13 years ago
Victor Julien	d378b76c04	http: body inspection improvement ... Improve http_client_body and file_data performance when request and response body limits are set to high values.	13 years ago
Eric Leblond	ad0e05a112	TLS app layer: Add tls.issuerdn keyword.	14 years ago
Pierre Chifflier	4be65fd016	TLS handshake: decode the SERVER_CERTIFICATE message ... Add a decoder for the SERVER_CERTIFICATE during a TLS handshake, extracts the certificates and keep the subject name. Add the tls.subject keyword for substring match in rules (TLS layer). Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>	14 years ago
Victor Julien	c0a2cbd478	Move over src and dst thresholding to use host table. Fix a bug in threshold 'both' handling.	14 years ago
Victor Julien	a05df345de	Introduce host table, make tag use it ... Add a host table similar to the flow table. A hash using fine grained locking. Flow manager for now takes care of book keeping / garbage collecting. Tag subsystem now uses this for host based tagging instead of the global tag hash table. Because the latter used a global lock and the new code uses very fine grained locking this patch should improve scalability.	14 years ago
Anoop Saldanha	e682796d03	feature #414 - support listing supported keywords. Remove support for dummy keywords __address__, __proto__, __port__. Remove support for recursive keyword and all references to it	14 years ago
Anoop Saldanha	603d4a719a	remove det_ctx->payload_offset and use det_ctx->buffer_offset. Update hscd and hsmd to use the new generic content inspection engine	14 years ago
Anoop Saldanha	35f1f7e8d9	unify payload detection engines + fix other bugs in pcre init	14 years ago
Anoop Saldanha	09313cf9bd	Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S	14 years ago
Anoop Saldanha	2007c2711c	Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword	14 years ago
Anoop Saldanha	419cdc8558	support splitting mpm ctxs based on direction v2	14 years ago
Victor Julien	c8c4a76dc6	Move threshold to it's own sig match list.	14 years ago
Victor Julien	6eeab37ab3	Add post-match list, move flowbits set, etc functions to it. Move flowint set, etc functions to it as well.	14 years ago
Victor Julien	21ee59e6f3	Add signature direction (flow:toserver/flow:toclient) as a signature flag.	14 years ago
Anoop Saldanha	eea5ab4a7a	Support for app layer decoder events added + app_layer_event keyword added	14 years ago
Anoop Saldanha	92643f6110	introduce separate mpm ctxs for tcp/udp/other_protos	14 years ago
Victor Julien	298289f43f	Let flow:only_stream and flow:no_stream set the require packet and require stream flags. Toss out sigs with conflicting settings. Rename flow:stream_only to flow:only_stream. Fixes #261.	14 years ago
Victor Julien	18d79c4215	file store: respect flowbits and other keywords ... The filestore keyword until now flagged a file, tx or ssn for storage as soon as the keyword was inspected. This happens before flowbits and some other keywords, so files were stored that weren't supposed to. This patch makes the filestore keyword fill an array in the detect engine thread ctx. Then if the full signature matches, a post-match filestore function makes the store final.	14 years ago
Victor Julien	07e560b137	file-data: initial file_data support ... Support file_data for: content, pcre (relative), byte_test, byte_jump, byte_extract, isdataat. File_data support is handled at signature parsing time, all matches occurring after the file_data in the rule are converted to http_server_body matches. Content matches relative to the file_data are converted. Within to depth, distance to offset. Relative to the start of the body buffer.	14 years ago
Victor Julien	7adac3048d	file-data: create initial keyword registration.	14 years ago
Victor Julien	fa0152fa80	Shrink signature flags field to 32 bits.	14 years ago
Victor Julien	dd9da1a56f	Merge all http mpm related signature flags into a single set: SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_HTTP_NEG.	14 years ago
Victor Julien	d5ed28b065	Remove SIG_FLAG_MPM flag.	14 years ago
Victor Julien	fe48920514	Remove per sgh mpm_streamcontent_maxlen variable. It was checked but never set.	14 years ago
Victor Julien	4992f7c417	Remove SIG_FLAG_MPM_URI flag. It was checked but never set.	14 years ago
Victor Julien	2650551192	Rename signature init flags to indicate they are init flags.	14 years ago
Victor Julien	6ebd71545b	Fix signature flag definitions on 32 bit.	14 years ago
Victor Julien	291ddd95f2	Detection engine -- mpm ... Each signature is in one mpm ctx at max, but there were 3 separate id's in use: packet, stream, http. Merged them all into one. Could shrink the SignatureHeader structure with 8 bytes because of this, should lead to better caching performance.	14 years ago