Commit Graph

336 Commits (1a2ad059a12910120cd997e76ea75e688747ff7d)

Author SHA1 Message Date
Victor Julien 22f3205664 var-names: expose outside of detect engine
Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.

For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.

This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.

The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.

For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.

After the threads have reloaded and the new detection engine is active,
the old table can be freed.

For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.

This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
10 years ago
Victor Julien 43cc06eabe detect: use engine version instead of id
Use engine version based on global detect engine master. This is
incremented between reloads.
10 years ago
Victor Julien fe415ae518 detect: remove DMATCH list 10 years ago
Victor Julien ad238121e3 detect: remove the AMATCH list 10 years ago
Victor Julien f2fc5a255f http_header: convert to use common code 10 years ago
Victor Julien cf9678d926 detect: global registery for keyword thread data
Some keywords need a scratch space where they can do store the results
of expensive operations that remain valid for the time of a packets
journey through the detection engine.

An example is the reconstructed 'http_header' field, that is needed
in MPM, and then for each rule that manually inspects it. Storing this
data in the flow is a waste, and reconstructing multiple times on
demand as well.

This API allows for registering a keyword with an init and free function.

It it mean to be used an initialization time, when the keyword is
registered.
10 years ago
Victor Julien 8bd1422948 detect: detect engine registration cleanup 10 years ago
Victor Julien 49fbd28ceb detect: cleanup built-in list id's 10 years ago
Victor Julien 40851eecf0 template: dynamic buffer 10 years ago
Victor Julien 815120896b app-layer-events: dynamic list 10 years ago
Victor Julien b68343e372 files: use dynamic list 10 years ago
Victor Julien e4bfdd53c2 cip/enip: dynamic buffer 10 years ago
Victor Julien f2393c1ae0 modbus: dynamic buffer 10 years ago
Victor Julien cfdd934aba dnp3: dynamic buffers/lists 10 years ago
Victor Julien 9ba386a141 tls: dynamic buffers 10 years ago
Victor Julien d9b3ae6cd6 dns: use dynamic buffers 10 years ago
Victor Julien 6f867c3c73 http_raw_uri: dynamic buffer 10 years ago
Victor Julien ee55aefa1c http_client_body: dynamic buffer 10 years ago
Victor Julien e7d5e845c7 http_header / http_raw_header: dynamic buffers 10 years ago
Victor Julien b694d96e22 http_stat_msg: dynamic buffer 10 years ago
Victor Julien 7e3ab4f5ea http_stat_code: dynamic buffer 10 years ago
Victor Julien 128b59d4f6 http_raw_host: dynamic buffer 10 years ago
Victor Julien 296c275e23 http_host: dynamic buffer 10 years ago
Victor Julien 67b7d9734e http_cookie: dynamic buffer 10 years ago
Victor Julien 54604c7bf2 http_user_agent: dynamic buffer 10 years ago
Victor Julien 9262fa3dcf http_response_line: dynamic buffer 10 years ago
Victor Julien 6346a074a7 http_uri: dynamic buffer
Clean up tests
10 years ago
Victor Julien e34102d67a http_method: make list dynamic 10 years ago
Victor Julien 6bd37611ee file_data: dynamic buffer 10 years ago
Victor Julien 7052f9b933 http_request_line: dynamic buffer 10 years ago
Victor Julien 779d40cedf detect: remove hardcoded sm_list logic from setup
Introduce utility functions to aid this.
10 years ago
Victor Julien 04592efb76 detect: buffer type API
To replace the hardcoded SigMatch list id's, use this API to register
and query lists by name.

Also allow for registering descriptions and whether mpm is supported.

Registration is only allowed at startup.
10 years ago
Victor Julien 58e1180efe detect: inspect engine setup cleanup 10 years ago
Victor Julien 0a5ae415b8 detect: shrink Signature::sm_arrays
Signature::sm_arrays now only contains 'built-in' lists, and so is
sized appropriately.
10 years ago
Victor Julien 59303d1fbb threshold: fix and redo tests 10 years ago
Victor Julien 6f7e4adbe8 detect: improve memory handling & comments 10 years ago
Victor Julien 8edc954e82 detect: get rid of Signature::sm_lists
Instead use the lists in init_data during setup and the SigMatchData
arrays during runtime.
10 years ago
Victor Julien f281481b67 detect: use detect list passed to generic funcs
Until now the GenericList users used hardcoded list id's.
10 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 10 years ago
Victor Julien a0fe67a3c0 detect: template list in engine 10 years ago
Victor Julien da7c816c7c detect: enip/cip list in engine 10 years ago
Victor Julien e94a7bddb3 detect: modbus list in engine 10 years ago
Victor Julien 7f7d4296da detect: file list in engine 10 years ago
Victor Julien f5adccba1d detect: app-event list in engine 10 years ago
Victor Julien 747dbf92ce detect: dns & tls lists in engine 10 years ago
Victor Julien 5a2e568385 detect: http lists in engine 10 years ago
Victor Julien 1ee32da2ca detect-engine: memory handling of sm_lists
For lists that are registered multiple times, like http_header and
http_cookie, making the engines owner of the lists is complicated.
Multiple engines in a sig may be pointing to the same list. To
address this the 'free' code needs to be extra careful about not
double freeing, so it takes an approach to first fill an array
of the to-free pointers before freeing them.
10 years ago
Victor Julien cf42fbf51f detect: use InspectEngineFuncPtr in inspect engines
Replace explicit function pointer use by InspectEngineFuncPtr typedef
10 years ago
Victor Julien 5f7e096be4 detect: shrink inspect engine by using 'id' as state flag 10 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 10 years ago