926 Commits (cf2feeecf430bd7b5f3d2ec1fcf4d7e98981d692)

Author	SHA1	Message	Date
Jason Ish	9ff8882cbd	detect: don't consider an empty rule file an error	8 years ago
Jason Ish	f56428d996	tcp/udp: fix checksum validation when 0xffff ... Issue: https://redmine.openinfosecfoundation.org/issues/2041 One approach to fixing this issue to just validate the checksum instead of regenerating it and comparing it. This method is used in some kernels and other network tools. When validating, the current checksum is passed in as an initial argument which will cause the final checksum to be 0 if OK. If generating a checksum, 0 is passed and the result is the generated checksum.	9 years ago
Victor Julien	4f8eacdc69	disable-detect: fix needless file hashing ... When detection is running flags are set on flows to indicate if file hashing is needed. This is based on global output settings and rules. In the case of --disable-detection this was not happening, so all files where hashed with all methods. This has a significant performance impact. This patch adds logic to set the flow flags in --disable-detect mode.	9 years ago
Victor Julien	c066f043a5	detect: remove unused alversion logic	9 years ago
Victor Julien	6c6802b957	detect: simplify state detect code: remove unused params	9 years ago
Victor Julien	c961da2dce	detect: clean up test	9 years ago
Victor Julien	15d26f14e1	file-store: fix force store	9 years ago
Victor Julien	2a5b9eb5fa	stream: remove unused variable	9 years ago
Mats Klepsland	d6508e640a	detect: add (mpm) keyword tls_cert_serial ... Match on TLS certificate serial number using tls_cert_serial keyword, e.g.: alert tls any any -> any any (msg:"TLS cert serial test"; tls_cert_serial; content:"5C:19:B7:B1:32:3B:1C:A1"; sid:12345;)	9 years ago
Victor Julien	0f708d427b	pkt-var: abuse flowvar postmatch logic for pktvars ... Flowvars were already using a temporary store in the detect thread ctx. Use the same facility for pktvars. The reasons are: 1. packet is not always available, e.g. when running pcre on http buffers. 2. setting of vars should be done post match. Until now it was also possible that it is done on a partial match.	9 years ago
Victor Julien	5e39486399	pkt-var: use id instead of name pointer	9 years ago
Victor Julien	e95a0c1344	alert-debug: print flowbit names from VarNameStore	9 years ago
Victor Julien	22f3205664	var-names: expose outside of detect engine ... Until now variable names, such as flowbit names, were local to a detect engine. This made sense as they were only ever used in that context. For the purpose of logging these names, this needs a different approach. The loggers live outside of the detect engine. Also, in the case of reloads and multi-tenancy, there are even multiple detect engines, so it would be even more tricky to access them from the outside. This patch brings a new approach. A any time, there is a single active hash table mapping the variable names and their id's. For multiple tenants the table is shared between tenants. The table is set up in a 'staging' area, where locking makes sure that multiple loading threads don't mess things up. Then when the preparing of a detection engine is ready, but before the detect threads are made aware of the new detect engine, the active varname hash is swapped with the staging instance. For this to work, all the mappings from the 'current' or active mapping are added to the staging table. After the threads have reloaded and the new detection engine is active, the old table can be freed. For multi tenancy things are similar. The staging area is used for setting up until the new detection engines / tenants are applied to the system. This patch also changes the variable 'id'/'idx' field to uint32_t. Due to data structure padding and alignment, this should have no practical drawback while allowing for a lot more vars.	9 years ago
Victor Julien	43cc06eabe	detect: use engine version instead of id ... Use engine version based on global detect engine master. This is incremented between reloads.	9 years ago
Victor Julien	920709fe6f	detect: ssh_software sticky buffer	9 years ago
Victor Julien	f1ab6a6153	detect: ssh_proto stickybuffer	9 years ago
Victor Julien	dfac5276b8	detect: remove unused SIGMATCH_PAYLOAD flag	9 years ago
Victor Julien	fe415ae518	detect: remove DMATCH list	9 years ago
Victor Julien	1c02cf4542	flow: remove unused Flow::de_state	9 years ago
Victor Julien	ad238121e3	detect: remove the AMATCH list	9 years ago
Victor Julien	1087495d6d	detect: http_start sticky buffer ... Matches on the start of a HTTP request or response. Uses a buffer constructed from the request line and normalized request headers, including the Cookie header. Or for the response side, it uses the response line plus the normalized response headers, including the Set-Cookie header. Both buffers are terminated by an extra \r\n.	9 years ago
Victor Julien	36535efa04	detect: add http_protocol sticky buffer ... Matches on protocol field in HTTP.	9 years ago
Victor Julien	f2fc5a255f	http_header: convert to use common code	9 years ago
Victor Julien	5ee68ca2b2	http_header: remove old files	9 years ago
Victor Julien	aaeeae0722	detect: http_header_names sticky buffer keyword ... A sticky buffer that allows content inspection on a contructed buffer of HTTP header names. The buffer starts with \r\n, the names are separated by \r\n and the end of the buffer contains an extra \r\n. E.g. \r\nHost\r\nUser-Agent\r\n\r\n The leading \r\n is to make sure one can match on a full name in all cases.	9 years ago
Victor Julien	cf9678d926	detect: global registery for keyword thread data ... Some keywords need a scratch space where they can do store the results of expensive operations that remain valid for the time of a packets journey through the detection engine. An example is the reconstructed 'http_header' field, that is needed in MPM, and then for each rule that manually inspects it. Storing this data in the flow is a waste, and reconstructing multiple times on demand as well. This API allows for registering a keyword with an init and free function. It it mean to be used an initialization time, when the keyword is registered.	9 years ago
Victor Julien	49fbd28ceb	detect: cleanup built-in list id's	9 years ago
Victor Julien	815120896b	app-layer-events: dynamic list	9 years ago
Victor Julien	6f867c3c73	http_raw_uri: dynamic buffer	9 years ago
Victor Julien	ee55aefa1c	http_client_body: dynamic buffer	9 years ago
Victor Julien	e7d5e845c7	http_header / http_raw_header: dynamic buffers	9 years ago
Victor Julien	b694d96e22	http_stat_msg: dynamic buffer	9 years ago
Victor Julien	7e3ab4f5ea	http_stat_code: dynamic buffer	9 years ago
Victor Julien	128b59d4f6	http_raw_host: dynamic buffer	9 years ago
Victor Julien	296c275e23	http_host: dynamic buffer	9 years ago
Victor Julien	67b7d9734e	http_cookie: dynamic buffer	9 years ago
Victor Julien	54604c7bf2	http_user_agent: dynamic buffer	9 years ago
Victor Julien	6346a074a7	http_uri: dynamic buffer ... Clean up tests	9 years ago
Victor Julien	e34102d67a	http_method: make list dynamic	9 years ago
Victor Julien	6bd37611ee	file_data: dynamic buffer	9 years ago
Victor Julien	779d40cedf	detect: remove hardcoded sm_list logic from setup ... Introduce utility functions to aid this.	9 years ago
Victor Julien	04592efb76	detect: buffer type API ... To replace the hardcoded SigMatch list id's, use this API to register and query lists by name. Also allow for registering descriptions and whether mpm is supported. Registration is only allowed at startup.	9 years ago
Victor Julien	f370e88135	detect: move init only Signature members to init_data	9 years ago
Victor Julien	0a5ae415b8	detect: shrink Signature::sm_arrays ... Signature::sm_arrays now only contains 'built-in' lists, and so is sized appropriately.	9 years ago
Victor Julien	4978a7a133	detect: reorganize id's in prep of dynamic lists	9 years ago
Victor Julien	59303d1fbb	threshold: fix and redo tests	9 years ago
Victor Julien	6f7e4adbe8	detect: improve memory handling & comments	9 years ago
Victor Julien	8edc954e82	detect: get rid of Signature::sm_lists ... Instead use the lists in init_data during setup and the SigMatchData arrays during runtime.	9 years ago
Victor Julien	715ff60087	detect: remove unused SIG_FLAG_INIT_PAYLOAD init_flag	9 years ago
Victor Julien	bfd4bc8233	detect: constify Signature/SigMatch use at runtime	9 years ago
Victor Julien	86222428dd	detect: don't run IP inspection on non-IP packets ... The code to get the rule group (sgh) would return the group for IP proto 0 instead of nothing. This lead to certain types of rules unintentionally matching (False Positive). Since the packets weren't actually IP, the logged alert records were missing the IP header. Bug #2017.	9 years ago
Victor Julien	f91d490d25	detect: remove unused flow_locked hint	9 years ago
Victor Julien	810e43f373	magic: make optional ... Make libmagic optional. If installed it will be enabled by default in configure. Use --disable-libmagic to disable.	9 years ago
Victor Julien	3da7dad514	lua: luajit improvements ... Luajit has a strange memory requirement, it's 'states' need to be in the first 2G of the process' memory. This patch improves the pool approach by moving it to the front of the start up. A new config option 'luajit.states' is added to control how many states are preallocated. It defaults to 128. Add a warning when more states are used then preallocated. This may fail if flow/stream/detect engines use a lot of memory. Add hint at exit that gives the max states in use if it's higher than the default.	9 years ago
Victor Julien	1f670837ac	detect: add missing break (CID 1374301)	9 years ago
Jason Ish	1a31bded4a	DNP3: dnp3_data, dnp3_func, dnp3_ind, dnp3_obj rule keywords	9 years ago
Victor Julien	321fb6463e	vars: small cleanups	9 years ago
Victor Julien	0ed119068d	detect-app-layer-protocol: implement prefilter ... Introduce 'Protocol detection'-only rules. These rules will only be fully evaluated when the protocol detection completed. To allow mixing of the app-layer-protocol keyword with other types of matches the keyword can also inspect the flow's app-protos per packet. Implement prefilter for the 'PD-only' rules.	9 years ago
Victor Julien	43aed70976	detect: during detection sgh is read only so turn into const	9 years ago
Victor Julien	0e31124609	detect: add util func for post-inspect tasks on first sgh	9 years ago
Victor Julien	d3fb4de1b5	detect: move file flags update into it's own function	9 years ago
kwong	a3ffebd835	Adding SCADA EtherNet/IP and CIP protocol support ... Add support for the ENIP/CIP Industrial protocol This is an app layer implementation which uses the "enip" protocol and "cip_service" and "enip_command" keywords Implements AFL entry points	9 years ago
Victor Julien	8321f04ef3	prefilter: clean up setup code	9 years ago
Victor Julien	d36c0c15ea	detect: reshuffle keyword registration order ... The order of keyword registration currently affects inspect engine registration order and ultimately the order of inspect engines per rule. Which in turn affects state keeping. This patch makes sure the ordering is the same as with older releases.	9 years ago
Victor Julien	5f994756e6	detect-engine: improved inspect engines ... Inspect engines are called per signature per sigmatch list. Most wrap around DetectEngineContentInspection, but it's more generic. Until now, the inspect engines were setup in a large per ipproto, per alproto, per direction table. For stateful inspection each engine needed a global flag. This approach had a number of issues: 1. inefficient: each inspection round walked the table and then checked if the inspect engine was even needed for the current rule. 2. clumsy registration with global flag registration. 3. global flag space was approaching the need for 64 bits 4. duplicate registration for alprotos supporting both TCP and TCP (DNS). This patch introduces a new approach. First, it does away with the per ipproto engines. This wasn't used. Second, it adds a per signature list of inspect engine containing only those engines that actually apply to the rule. Third, it gets rid of the global flags and replaces it with flags assigned per rule per engine.	9 years ago
Victor Julien	f1e3840516	http_response_body: implement keyword with mpm ... Implemented as 'stickybuffer'.	9 years ago
Victor Julien	4c98b6cef3	http_request_line: implement keyword and mpm ... Implemented as 'stickybuffer'. Move all logic into the keyword file and remove bad tests that tested URI instead of request line.	9 years ago
Victor Julien	5b2e36a1b0	mpm: add App Layer MPM registery ... Register keywords globally at start up. Create a map of the registery per detection engine. This we need because the sgh_mpm_context value is set per detect engine. Remove APP_MPMS_MAX.	9 years ago
Victor Julien	e1eb481647	prefilter: cleanup and optimization	9 years ago
Victor Julien	dba14b676c	profiling: more prefilter profiling	9 years ago
Victor Julien	125603871b	detect: config opt to enable keyword prefilters	9 years ago
Victor Julien	9bb12ccb27	prefilter: move payload engines into separate list	9 years ago
Victor Julien	14b0537f95	prefilter: implement basic prefilter priority order	9 years ago
Victor Julien	8798bf48b2	profiling: support prefilter engines	9 years ago
Victor Julien	f80623fd73	prefilter: show prefilter capability in --list-keywords	9 years ago
Victor Julien	56239690d0	prefilter: implement prefilter keyword ... Introduce prefilter keyword to force a keyword to be used as prefilter. e.g. alert tcp any any -> any any (content:"A"; flags:R; prefilter; sid:1;) alert tcp any any -> any any (content:"A"; flags:R; sid:2;) alert tcp any any -> any any (content:"A"; dsize:1; prefilter; sid:3;) alert tcp any any -> any any (content:"A"; dsize:1; sid:4;) In sid 2 and 4 the content keyword is used in the MPM engine. In sid 1 and 3 the flags and dsize keywords will be used.	9 years ago
Victor Julien	08407b6d47	tls: mpm prefilter engines	9 years ago
Victor Julien	7acdc66061	smtp file_data: mpm prefilter engine	9 years ago
Victor Julien	0019a7bd9f	http_raw_header: mpm prefilter engine ... Register for both regular headers and trailer.	9 years ago
Victor Julien	cef12ed80f	http_server_body / file_data: mpm prefilter engine	9 years ago
Victor Julien	5646dd9ecf	http_client_body: mpm prefilter engine	9 years ago
Victor Julien	9b6fd6bb48	http_headers: mpm prefilter engines ... Register for both regular headers and trailers.	9 years ago
Victor Julien	9cab3ea2cd	http_stat_code: mpm prefilter engine	9 years ago
Victor Julien	4d57b2fc63	http_stat_msg: mpm prefilter engine	9 years ago
Victor Julien	86d303e32b	http_raw_host: mpm prefilter engine	9 years ago
Victor Julien	5218849213	http_host: mpm prefilter engine	9 years ago
Victor Julien	61c3748fc4	http_user_agent: mpm prefilter engine	9 years ago
Victor Julien	a43a69305d	http_cookie: mpm prefilter engine	9 years ago
Victor Julien	7a46364e42	http_raw_uri: mpm prefilter engine	9 years ago
Victor Julien	746a169127	dns_query: mpm prefilter engine	9 years ago
Victor Julien	9ff5703c49	packet/stream: mpm prefilter engine	9 years ago
Victor Julien	72f2a78b1f	http_method: mpm prefilter engine	9 years ago
Victor Julien	b62c4cc359	http_uri: mpm prefilter engine ... Inspect partial request line as well.	9 years ago
Victor Julien	5bcdbe3922	prefilter: introduce prefilter engines ... Introduce abstraction layer for prefilter engines.	9 years ago
Victor Julien	17bc0299fe	detect: rename non_mpm lists/vars to non_pf ... Rename to non_pf: non prefilter.	9 years ago
Victor Julien	ff70e0cca0	mpm tls: remove unused function args	9 years ago
Victor Julien	ad3a55d938	mpm dns query: remove unused function args	9 years ago
Victor Julien	d647db1775	mpm stat code: remove unused function args	9 years ago
Victor Julien	bd03307921	mpm stat msg: remove unused function args	9 years ago
Victor Julien	6d54b70db4	mpm ua: remove unused function args	9 years ago