Commit Graph

7860 Commits (47a5b493d748e899c478ec7f75c4858fee4552ee)
 

Author SHA1 Message Date
Victor Julien 0f708d427b pkt-var: abuse flowvar postmatch logic for pktvars
Flowvars were already using a temporary store in the detect thread
ctx.

Use the same facility for pktvars. The reasons are:

1. packet is not always available, e.g. when running pcre on http
   buffers.

2. setting of vars should be done post match. Until now it was also
   possible that it is done on a partial match.
8 years ago
Victor Julien 5e39486399 pkt-var: use id instead of name pointer 8 years ago
Victor Julien a0bd15a1c4 pcre: support multiple captures
Support up to 8 substring captures into pkt or flow vars.
8 years ago
Victor Julien 017b16d421 detect-pcre: small cleanups 8 years ago
Victor Julien ac42a44280 alert-debug: print flowvar/int names 8 years ago
Victor Julien e95a0c1344 alert-debug: print flowbit names from VarNameStore 8 years ago
Victor Julien 22f3205664 var-names: expose outside of detect engine
Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.

For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.

This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.

The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.

For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.

After the threads have reloaded and the new detection engine is active,
the old table can be freed.

For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.

This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
8 years ago
Victor Julien 43cc06eabe detect: use engine version instead of id
Use engine version based on global detect engine master. This is
incremented between reloads.
8 years ago
Victor Julien 920709fe6f detect: ssh_software sticky buffer 8 years ago
Victor Julien f1ab6a6153 detect: ssh_proto stickybuffer 8 years ago
Victor Julien dfac5276b8 detect: remove unused SIGMATCH_PAYLOAD flag 8 years ago
Victor Julien 14ced15e36 detect: remove unused state file flag 8 years ago
Victor Julien fa1ef158b2 detect: small API cleanup 8 years ago
Victor Julien 073fcbeb7f detect: move file hash common code 8 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 8 years ago
Victor Julien fe415ae518 detect: remove DMATCH list 8 years ago
Victor Julien 1c02cf4542 flow: remove unused Flow::de_state 8 years ago
Victor Julien ad238121e3 detect: remove the AMATCH list 8 years ago
Victor Julien 775e182531 detect: remove AppLayerMatch API call 8 years ago
Victor Julien f018ae94b0 dce: dynamic lists 8 years ago
Victor Julien 84ba9cf9df smb/dcerpc: use tx api 8 years ago
Victor Julien d318bfc934 dcerpc: simplify common detect code 8 years ago
Victor Julien 402eb645a0 ftp: parser and ftpbounce update
Convert parser to TX API.

Convert ftpbounce keyword to use that.
8 years ago
Victor Julien d9a300cd8c detect: move lua smtp support to dynamic list 8 years ago
Victor Julien 96b8100a51 lua: convert lua output to be tx aware 8 years ago
Victor Julien a10b2fdecf detect: make ssh detection use dynamic list 8 years ago
Victor Julien c412352474 ssh: remove single logger limit 8 years ago
Victor Julien 3ee4989ba7 ssh: convert app-layer parser to be tx aware
Like with SSL, there is only a single 'tx'.
8 years ago
Victor Julien 4ae4fd0802 lua: use tls_generic list for ssl/tls 8 years ago
Victor Julien a8975c68e0 detect ssl/tls: use dynamic lists 8 years ago
Victor Julien 1cacba435b ssl/tls: clean up keywords 8 years ago
Victor Julien 6ef4712cd6 detect: convert old tls keywords to dynamic list 8 years ago
Victor Julien ef149bed7b tls.store: convert to postmatch 8 years ago
Victor Julien 8b3a71a285 tls.store: cleanup 8 years ago
Victor Julien 8eac1156c6 app-layer-events: remove unused API options 8 years ago
Victor Julien 1087495d6d detect: http_start sticky buffer
Matches on the start of a HTTP request or response.

Uses a buffer constructed from the request line and normalized request
headers, including the Cookie header.

Or for the response side, it uses the response line plus the
normalized response headers, including the Set-Cookie header.

Both buffers are terminated by an extra \r\n.
8 years ago
Victor Julien 36535efa04 detect: add http_protocol sticky buffer
Matches on protocol field in HTTP.
8 years ago
Victor Julien f2fc5a255f http_header: convert to use common code 8 years ago
Victor Julien 6279ec399e http_header: common detection code 8 years ago
Victor Julien 5ee68ca2b2 http_header: remove old files 8 years ago
Victor Julien 2bb0cae05a http_header: move all code into keyword files 8 years ago
Victor Julien aaeeae0722 detect: http_header_names sticky buffer keyword
A sticky buffer that allows content inspection on a contructed buffer
of HTTP header names. The buffer starts with \r\n, the names are
separated by \r\n and the end of the buffer contains an extra \r\n.

E.g. \r\nHost\r\nUser-Agent\r\n\r\n

The leading \r\n is to make sure one can match on a full name in all
cases.
8 years ago
Victor Julien cf9678d926 detect: global registery for keyword thread data
Some keywords need a scratch space where they can do store the results
of expensive operations that remain valid for the time of a packets
journey through the detection engine.

An example is the reconstructed 'http_header' field, that is needed
in MPM, and then for each rule that manually inspects it. Storing this
data in the flow is a waste, and reconstructing multiple times on
demand as well.

This API allows for registering a keyword with an init and free function.

It it mean to be used an initialization time, when the keyword is
registered.
8 years ago
Victor Julien 75907fce06 profiling: output all sort options for rules
Limit the default number of sids to 10.
8 years ago
Victor Julien 7d8a5a75ef profiling: honor limit in json rule output 8 years ago
Victor Julien a9a228a289 profiling: fix keyword profiling 8 years ago
Victor Julien a1465bc4fa detect-engine-mpm: api cleanup 8 years ago
Victor Julien 8bd1422948 detect: detect engine registration cleanup 8 years ago
Victor Julien 49fbd28ceb detect: cleanup built-in list id's 8 years ago
Victor Julien 40851eecf0 template: dynamic buffer 8 years ago