7861 Commits (ffbf8cec162c9f6c77663d4df22e17bd5f8b5a01)

 

All Branches   

Search

Author	SHA1	Message	Date
Victor Julien	1ba8c2fe3a	pcre: new way of specifying var names ... Until now the way to specify a var name in pcre substring capture into pkt and flow vars was to use the pcre named substring support: e.g. /(?P<pkt_somename>.*)/ This had 2 drawbacks: 1. limitations of the name. The name could be max 32 chars, only have alphanumeric and the underscore characters. This imposed limitations that are not present in flowbits/ints. 2. we didn't actually use the named substrings in pcre through the API. We parsed the names separately. So putting the names in pcre would actually be wasteful. This patch introduces a new way of mapping captures with names: pcre:"/(.*)/, pkt:somename"; pcre:"/([A-z]+) ([0-9]+)/, pkt:somename,flow:anothername"; The order of the captures and the order of the names are mapped 1 on 1. This method is no longer limited by the pcre API's naming limits. The 'flow:' and 'pkt:' prefixes indicate what the type of variable is. It's mandatory to specify one. The old method is still supported as well.	8 years ago
Victor Julien	0f708d427b	pkt-var: abuse flowvar postmatch logic for pktvars ... Flowvars were already using a temporary store in the detect thread ctx. Use the same facility for pktvars. The reasons are: 1. packet is not always available, e.g. when running pcre on http buffers. 2. setting of vars should be done post match. Until now it was also possible that it is done on a partial match.	8 years ago
Victor Julien	5e39486399	pkt-var: use id instead of name pointer	8 years ago
Victor Julien	a0bd15a1c4	pcre: support multiple captures ... Support up to 8 substring captures into pkt or flow vars.	8 years ago
Victor Julien	017b16d421	detect-pcre: small cleanups	8 years ago
Victor Julien	ac42a44280	alert-debug: print flowvar/int names	8 years ago
Victor Julien	e95a0c1344	alert-debug: print flowbit names from VarNameStore	8 years ago
Victor Julien	22f3205664	var-names: expose outside of detect engine ... Until now variable names, such as flowbit names, were local to a detect engine. This made sense as they were only ever used in that context. For the purpose of logging these names, this needs a different approach. The loggers live outside of the detect engine. Also, in the case of reloads and multi-tenancy, there are even multiple detect engines, so it would be even more tricky to access them from the outside. This patch brings a new approach. A any time, there is a single active hash table mapping the variable names and their id's. For multiple tenants the table is shared between tenants. The table is set up in a 'staging' area, where locking makes sure that multiple loading threads don't mess things up. Then when the preparing of a detection engine is ready, but before the detect threads are made aware of the new detect engine, the active varname hash is swapped with the staging instance. For this to work, all the mappings from the 'current' or active mapping are added to the staging table. After the threads have reloaded and the new detection engine is active, the old table can be freed. For multi tenancy things are similar. The staging area is used for setting up until the new detection engines / tenants are applied to the system. This patch also changes the variable 'id'/'idx' field to uint32_t. Due to data structure padding and alignment, this should have no practical drawback while allowing for a lot more vars.	8 years ago
Victor Julien	43cc06eabe	detect: use engine version instead of id ... Use engine version based on global detect engine master. This is incremented between reloads.	8 years ago
Victor Julien	920709fe6f	detect: ssh_software sticky buffer	8 years ago
Victor Julien	f1ab6a6153	detect: ssh_proto stickybuffer	8 years ago
Victor Julien	dfac5276b8	detect: remove unused SIGMATCH_PAYLOAD flag	8 years ago
Victor Julien	14ced15e36	detect: remove unused state file flag	8 years ago
Victor Julien	fa1ef158b2	detect: small API cleanup	8 years ago
Victor Julien	073fcbeb7f	detect: move file hash common code	8 years ago
Victor Julien	5bafc64c08	detect: unify FileMatch API with other calls	8 years ago
Victor Julien	fe415ae518	detect: remove DMATCH list	8 years ago
Victor Julien	1c02cf4542	flow: remove unused Flow::de_state	8 years ago
Victor Julien	ad238121e3	detect: remove the AMATCH list	8 years ago
Victor Julien	775e182531	detect: remove AppLayerMatch API call	8 years ago
Victor Julien	f018ae94b0	dce: dynamic lists	8 years ago
Victor Julien	84ba9cf9df	smb/dcerpc: use tx api	8 years ago
Victor Julien	d318bfc934	dcerpc: simplify common detect code	8 years ago
Victor Julien	402eb645a0	ftp: parser and ftpbounce update ... Convert parser to TX API. Convert ftpbounce keyword to use that.	8 years ago
Victor Julien	d9a300cd8c	detect: move lua smtp support to dynamic list	8 years ago
Victor Julien	96b8100a51	lua: convert lua output to be tx aware	8 years ago
Victor Julien	a10b2fdecf	detect: make ssh detection use dynamic list	8 years ago
Victor Julien	c412352474	ssh: remove single logger limit	8 years ago
Victor Julien	3ee4989ba7	ssh: convert app-layer parser to be tx aware ... Like with SSL, there is only a single 'tx'.	8 years ago
Victor Julien	4ae4fd0802	lua: use tls_generic list for ssl/tls	8 years ago
Victor Julien	a8975c68e0	detect ssl/tls: use dynamic lists	8 years ago
Victor Julien	1cacba435b	ssl/tls: clean up keywords	8 years ago
Victor Julien	6ef4712cd6	detect: convert old tls keywords to dynamic list	8 years ago
Victor Julien	ef149bed7b	tls.store: convert to postmatch	8 years ago
Victor Julien	8b3a71a285	tls.store: cleanup	8 years ago
Victor Julien	8eac1156c6	app-layer-events: remove unused API options	8 years ago
Victor Julien	1087495d6d	detect: http_start sticky buffer ... Matches on the start of a HTTP request or response. Uses a buffer constructed from the request line and normalized request headers, including the Cookie header. Or for the response side, it uses the response line plus the normalized response headers, including the Set-Cookie header. Both buffers are terminated by an extra \r\n.	8 years ago
Victor Julien	36535efa04	detect: add http_protocol sticky buffer ... Matches on protocol field in HTTP.	8 years ago
Victor Julien	f2fc5a255f	http_header: convert to use common code	8 years ago
Victor Julien	6279ec399e	http_header: common detection code	8 years ago
Victor Julien	5ee68ca2b2	http_header: remove old files	8 years ago
Victor Julien	2bb0cae05a	http_header: move all code into keyword files	8 years ago
Victor Julien	aaeeae0722	detect: http_header_names sticky buffer keyword ... A sticky buffer that allows content inspection on a contructed buffer of HTTP header names. The buffer starts with \r\n, the names are separated by \r\n and the end of the buffer contains an extra \r\n. E.g. \r\nHost\r\nUser-Agent\r\n\r\n The leading \r\n is to make sure one can match on a full name in all cases.	8 years ago
Victor Julien	cf9678d926	detect: global registery for keyword thread data ... Some keywords need a scratch space where they can do store the results of expensive operations that remain valid for the time of a packets journey through the detection engine. An example is the reconstructed 'http_header' field, that is needed in MPM, and then for each rule that manually inspects it. Storing this data in the flow is a waste, and reconstructing multiple times on demand as well. This API allows for registering a keyword with an init and free function. It it mean to be used an initialization time, when the keyword is registered.	8 years ago
Victor Julien	75907fce06	profiling: output all sort options for rules ... Limit the default number of sids to 10.	8 years ago
Victor Julien	7d8a5a75ef	profiling: honor limit in json rule output	8 years ago
Victor Julien	a9a228a289	profiling: fix keyword profiling	8 years ago
Victor Julien	a1465bc4fa	detect-engine-mpm: api cleanup	8 years ago
Victor Julien	8bd1422948	detect: detect engine registration cleanup	8 years ago
Victor Julien	49fbd28ceb	detect: cleanup built-in list id's	8 years ago