Commit Graph

219 Commits (4e8e5917153d8bf1b06c7a3f56457f540034a8d1)

Author SHA1 Message Date
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 10 years ago
Victor Julien fa885e1d85 mpm: remove pattern id logic 10 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
10 years ago
Victor Julien f0ba00e51d detect: remove old unused code 10 years ago
Victor Julien 10b049304f detect: set new defaults for grouping 10 years ago
Victor Julien 2ee9bf2aef detect: rename groupings vars 10 years ago
Victor Julien 1f70ccfc23 detect: remove unused grouping settings 10 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 10 years ago
Victor Julien 4f8e1f59a6 mpm: remove obsolete mpm algos
Remove: ac-gfbs, wumanber, b2g, b3g.
10 years ago
Victor Julien 58e533858b detect mpm: ac-tile/ac-ks default to single
Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.
10 years ago
Victor Julien ffb5498228 detect: fix potential deadlock during reload
If interrupted during the BreakLoop stage during reload, a deadlock
could happen.
10 years ago
Justin Viiret 13b87f5aff mpm: add Hyperscan integration
This adds an MPM implementation that uses the Hyperscan regex engine
library from Intel, accessible as the "hs" mpm-algo.
10 years ago
Victor Julien 11099cfa42 detect reload: generic packet injection for capture
Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.

This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.

The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.

Implementations for pcap, AF_PACKET and PF_RING.
10 years ago
Victor Julien eafd212661 detect reload: call 'breakloop' on capture method
Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used

Interupt capture more than once if needed

Move packet injection into util func
10 years ago
Victor Julien 8394b38941 cppcheck: work around snprintf warning
Cppcheck 1.72 gives a warning on the following code pattern:

    char blah[32] = "";
    snprintf(blah, sizeof(blah), "something");

The warning is:

    (error) Buffer is accessed out of bounds.

While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.
10 years ago
Victor Julien c91546022d smtp: clean up thread local memory 10 years ago
Victor Julien b9ee86fdb4 detect-engine: free memory in error conditions (CID 1351210) 10 years ago
Victor Julien 0dd81b85d4 multi-tenants: improve error handling (CID 1312702) 10 years ago
Victor Julien 4dfbc0effa multi-detect: fix and simplify config
instead

mappings:
  - vlan:
    vlan-id: 1
    tenant-id: 2

we'll now use:

mappings:
  - vlan-id: 1
    tenant-id: 2

For YAML it pretty much means the same thing.

Ticket: 1517
10 years ago
Victor Julien 07d8617b3e multi-detect: improve error handling 10 years ago
Victor Julien 906b95eed3 multi-detect: handle missing mappings
Notify/warn user about missing mappings depending on other settings
like unix socket and init errors fatal.
10 years ago
Victor Julien 27783f4c66 multi-detect: consider vlan tracking
Refuse to use vlan selector if vlan tracking is disabled.
10 years ago
Victor Julien 04889f154d multi-detect: validate vlan_id 10 years ago
Victor Julien d7d76e7b27 multi-detect: use default tenant
The default detect engine can be used as 'default tenant'.
10 years ago
Victor Julien dc3c1ef01e multi-detect: clean up output 10 years ago
Jason Ish 6b15686fd1 base64_decode, base64_data: decode and match base64 10 years ago
Eric Leblond a4089873c7 rules-reload: fix reload with -s or -S
When using the -S or -s option, the reload was causing the specified
rules file to be forgotten and the default rules to be loaded at
reload time.
10 years ago
Jason Ish 06beca62f5 app-layer: template for application layer content inspection 10 years ago
Victor Julien 979bd35277 detect loader: move to own file 10 years ago
Victor Julien cfeaf42cab detect-loaders: configurable amount of loaders 10 years ago
Victor Julien 99c0a7ad72 multi-detect: improve memory handling of setup code 10 years ago
Victor Julien b7b27684c2 multi-detect: detect loader for unix socket
Move the tenant load and reload commands to be executed by the detect
loader thread(s).

Limitation: no yaml parsing in parallel. The Conf API is currently not
thread safe, so don't load the tenant config (yaml) in parallel.
10 years ago
Victor Julien eb09118d64 detect: create loader threads
To speed up startup with many tenants, tenant loading will be parallelized.
As no tempary threads should be used for these memory allocation heavy
tasks, this patch adds new type of 'command' thread that can be used to
load and reload tenants.

This patch hardcodes the number of loaders to 4. Future work will make it
dynamic.

The loader thread essentially sleeps constantly. When a tasks is sent to
it, it will wake up and execute it.
10 years ago
Victor Julien e19c41a807 multi-detect: hash lookup for tenants
Use hash for storing and looking up det_ctxs.
10 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
10 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant
Generate error if tentant to be added is already loaded.
10 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file
Load tenants and mappings from the suricata.yaml when available.
10 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler
Remove a tenant handler from the list and apply it.
10 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
10 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered
Force user to select the method at startup.
10 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx
Otherwise counters are only registered after the stats api is
already fixed.
10 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 10 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
10 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
10 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry
Limited to Pcap only currently.
10 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 10 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch
At start up we will set this flag based on "multi-detect.enabled".
10 years ago
Victor Julien def2b58725 detect: initial MT lookup logic
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
10 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
10 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 10 years ago