Commit Graph

186 Commits (e19c41a80726f27672f9ba889ab9b0444cc09399)

Author SHA1 Message Date
Victor Julien e19c41a807 multi-detect: hash lookup for tenants
Use hash for storing and looking up det_ctxs.
10 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
10 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant
Generate error if tentant to be added is already loaded.
10 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file
Load tenants and mappings from the suricata.yaml when available.
10 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler
Remove a tenant handler from the list and apply it.
10 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
10 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered
Force user to select the method at startup.
10 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx
Otherwise counters are only registered after the stats api is
already fixed.
10 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 10 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
10 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
10 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry
Limited to Pcap only currently.
10 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 10 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch
At start up we will set this flag based on "multi-detect.enabled".
10 years ago
Victor Julien def2b58725 detect: initial MT lookup logic
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
10 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
10 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 10 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 10 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 10 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 10 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 10 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 10 years ago
Victor Julien 1ef786e7cb counters: rename register API calls
Also remove 'type' parameter which was always the same.
10 years ago
Victor Julien b293a4b7d0 counters: remove unused description 10 years ago
Victor Julien b2da57c827 reference: remove global 10 years ago
Victor Julien 393689ce44 classification: remove global from parsing
Parsing code used a 'fd' global. Remove this.
10 years ago
Giuseppe Longo f0c54d4764 Detect engine for smtp file_data file_data: inspecting smtp attachments
Create a buffer to store reassembled file chunks,
and inspect the content.
10 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
10 years ago
Victor Julien 724c7044e1 detect-reload: 0 detect threads is no error
The reload code would consider 0 detect threads to be an error,
but it's not in case of unix socket mode.
10 years ago
Victor Julien 7c9e015748 unix-socket: implement reload-rules
Implement the reload-rules unix socket command. The unix command
thread signals the main thread to do the reload and it waits for
it to complete.
10 years ago
Victor Julien 71d01f06b9 detect reload: load config
Load the YAML into a prefix "detect-engine-reloads.N" where N is the
reload counter. This way we can load the updated config w/o overwriting
the current one.
10 years ago
Victor Julien b51075e804 detect: remove config at prefix
Remove config at prefix when freeing a detect engine.
10 years ago
Victor Julien 7108085d33 detect: initialize detection engine by prefix
Initalize detection engine by configuration prefix.

    DetectEngineCtxInitWithPrefix(const char *prefix)

Takes the detection engine configuration from:
<prefix>.<config>

If prefix is NULL the regular config will be used.

Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
10 years ago
Victor Julien a80cc696d7 detect: allow det_ctx->de_ctx to be NULL
When freeing det_ctx, allow de_ctx to be NULL.
10 years ago
Victor Julien c9a8262ccf detect: reload thread init cleanup
Rename the thread init function DetectEngineThreadCtxInitForLiveRuleSwap
to DetectEngineThreadCtxInitForReload and change it's logic to take the
new detection engine as argument and let it return the
DetectEngineThreadCtx or NULL on error.

The old approach used the thread init API format, but it wasn't used in
that way.
10 years ago
Victor Julien 55e7370fc5 detect reload: allow master update during reload
Add DetectEngineReference, which takes a reference to a detect engine,
and make DetectEngineThreadCtxInitForLiveRuleSwap use it. This way
reload will not depend on master staying the same. This allows master
to be updated in between w/o affecting the reload that is in progress.
10 years ago
Victor Julien b1c54a8673 detect: remove old live reload implementation
Remove code that ran the reload in it's own thread. Simplify the
signal handling.
10 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
10 years ago
Victor Julien 38b349af1e runmodes: remove DetectEngineCtx passing from API
No longer pass a pointer to the current detection engine to the
runmode API calls.

Note: breaks delayed detect. Will be fixed in a future commit.
10 years ago
Victor Julien b038b6a2f8 unittests: add exception to detect engine setup
Add code to allow for unittests not following the complete api.

Update replace tests as they don't use the unittests runmode that
powers the workaround based on RunmodeIsUnittests().
10 years ago
Victor Julien d66fa1add1 detect: update detect engine management
Update detect engine management to make it easier to reload the detect
engine.

Core of the new approach is a 'master' ctx, that keeps a list of one or
more detect engines. The detect engines will not be passed to any thread
directly, but instead will only be accessed through the detect engine
thread contexts. As we can replace those atomically, replacing a detect
engine becomes easier.

Each thread keeps a reference to its detect context. When a detect engine
is replaced or removed, it's added to a free list. Once its reference
count reaches 0, it is freed.
10 years ago
Victor Julien 49bad2cfba detect: consolidate more setup into DetectEngineCtxInit
Loading of classifications, references and action order was done
unconditionally, so can be done in one place.
10 years ago
Victor Julien 6723d03c7e http: add inspection engine for http request line
No MPM though.
10 years ago
Victor Julien 04e49cea89 Fix live reload detect counter setup
When profiling was compiled in the detect counters were not setup
properly after a reload.
11 years ago
Victor Julien a8c16405fb detect: properly size det_ctx::non_mpm_id_array
Track which sgh has the higest non-mpm sig count and use that value
to size the det_ctx::non_mpm_id_array array.
11 years ago
Victor Julien 62751c8017 Fix live reload detect thread ctx setup
Code failed to setup non_mpm_id_array in case of a live reload.
11 years ago
Victor Julien b5a3127151 detect: add mask check prefilter for non mpm list
Add mask array for non_mpm sigs, so that we can exclude many sigs before
we merge sort.

Shows 50% less non mpm sigs inspected on average.
11 years ago
Ken Steele 904441327c Conditionalize SigMatch performance counters.
Only include the counters when PROFILING.
11 years ago
Victor Julien 30b7fdcb49 Detect perf counters 11 years ago
DIALLO David 55c5081240 Detect-engine: Add Modbus detection engine
Management of Modbus Tx

Based on DNS source code.

Signed-off-by: David DIALLO <diallo@et.esia.fr>
11 years ago