aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,441 Commits
7 Branches
161 Tags
173 MiB
main-7.0.x
master
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '321fb6463e'
${ noResults }
Commit Graph

279 Commits (321fb6463e0e8e6cf20c53dfdc28f83fc1b7e7de)

Author SHA1 Message Date
Justin Viiret cce2d114e8 spm: add and use new SPM API 
This new API allows for different SPM implementations, using a function
pointer table like that used for MPM.

This change also switches over the paths that make use of
DetectContentData (which previously used BoyerMoore directly) to the new
API.
 9 years ago
Justin Viiret ce408c4d10 spm: add SinglePatternMatchDefaultMatcher 
Allows selecting SPM algorithm with the 'spm-algo' value in the YAML
config file.
 9 years ago
maxtors 9d3fd82849 Removed duplicate include statements. 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code 
1 pass, 0 is fail.
 10 years ago
Victor Julien d6ba01b1b7 detect: make port whitelisting configurable 
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.

detect:
  grouping:
    tcp-whitelist: 80, 443
    udp-whitelist: 53, 5060

No portranges are allowed at this point.
 10 years ago
Victor Julien 725d6c3739 yaml: convert detect-engine to just detect 
Instead of detect-engine which used a list for no good reason, use a
simple map now.

detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
  # If set to yes, the loading of signatures will be made after the capture
  # is started. This will limit the downtime in IPS mode.
  #delayed-detect: yes
 10 years ago
Victor Julien 1dd135d512 mpm: always cleanup factory 10 years ago
Victor Julien caea596ce5 profiling: output post-prefilter matches 
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:

 --set detect.profiling.inspect-logging-threshold=200

A file called packet_inspected_rules.json is created in the default
log dir.
 10 years ago
Victor Julien 722e2dbf7c profiling: initial rulegroup tracking 
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.

Logs SGH id so it can be compared with the rule_group.json output.

Implemented both in a human readable text format and a JSON format.
 10 years ago
Victor Julien c1ad08d11e detect: remove stream pmq array 10 years ago
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 10 years ago
Victor Julien fa885e1d85 mpm: remove pattern id logic 10 years ago
Victor Julien fac2cc0560 detect: mpm deduplication 
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
 10 years ago
Victor Julien f0ba00e51d detect: remove old unused code 10 years ago
Victor Julien 10b049304f detect: set new defaults for grouping 10 years ago
Victor Julien 2ee9bf2aef detect: rename groupings vars 10 years ago
Victor Julien 1f70ccfc23 detect: remove unused grouping settings 10 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 10 years ago
Victor Julien 4f8e1f59a6 mpm: remove obsolete mpm algos 
Remove: ac-gfbs, wumanber, b2g, b3g.
 10 years ago
Victor Julien 58e533858b detect mpm: ac-tile/ac-ks default to single 
Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.
 10 years ago
Victor Julien ffb5498228 detect: fix potential deadlock during reload 
If interrupted during the BreakLoop stage during reload, a deadlock
could happen.
 10 years ago
Justin Viiret 13b87f5aff mpm: add Hyperscan integration 
This adds an MPM implementation that uses the Hyperscan regex engine
library from Intel, accessible as the "hs" mpm-algo.
 10 years ago
Victor Julien 11099cfa42 detect reload: generic packet injection for capture 
Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.

This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.

The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.

Implementations for pcap, AF_PACKET and PF_RING.
 10 years ago
Victor Julien eafd212661 detect reload: call 'breakloop' on capture method 
Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used

Interupt capture more than once if needed

Move packet injection into util func
 10 years ago
Victor Julien 8394b38941 cppcheck: work around snprintf warning 
Cppcheck 1.72 gives a warning on the following code pattern:

    char blah[32] = "";
    snprintf(blah, sizeof(blah), "something");

The warning is:

    (error) Buffer is accessed out of bounds.

While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.
 10 years ago
Victor Julien c91546022d smtp: clean up thread local memory 10 years ago
Victor Julien b9ee86fdb4 detect-engine: free memory in error conditions (CID 1351210) 10 years ago
Victor Julien 0dd81b85d4 multi-tenants: improve error handling (CID 1312702) 10 years ago
Victor Julien 4dfbc0effa multi-detect: fix and simplify config 
instead

mappings:
  - vlan:
    vlan-id: 1
    tenant-id: 2

we'll now use:

mappings:
  - vlan-id: 1
    tenant-id: 2

For YAML it pretty much means the same thing.

Ticket: 1517
 10 years ago
Victor Julien 07d8617b3e multi-detect: improve error handling 10 years ago
Victor Julien 906b95eed3 multi-detect: handle missing mappings 
Notify/warn user about missing mappings depending on other settings
like unix socket and init errors fatal.
 10 years ago
Victor Julien 27783f4c66 multi-detect: consider vlan tracking 
Refuse to use vlan selector if vlan tracking is disabled.
 10 years ago
Victor Julien 04889f154d multi-detect: validate vlan_id 10 years ago
Victor Julien d7d76e7b27 multi-detect: use default tenant 
The default detect engine can be used as 'default tenant'.
 10 years ago
Victor Julien dc3c1ef01e multi-detect: clean up output 10 years ago
Jason Ish 6b15686fd1 base64_decode, base64_data: decode and match base64 10 years ago
Eric Leblond a4089873c7 rules-reload: fix reload with -s or -S 
When using the -S or -s option, the reload was causing the specified
rules file to be forgotten and the default rules to be loaded at
reload time.
 10 years ago
Jason Ish 06beca62f5 app-layer: template for application layer content inspection 10 years ago
Victor Julien 979bd35277 detect loader: move to own file 10 years ago
Victor Julien cfeaf42cab detect-loaders: configurable amount of loaders 10 years ago
Victor Julien 99c0a7ad72 multi-detect: improve memory handling of setup code 10 years ago
Victor Julien b7b27684c2 multi-detect: detect loader for unix socket 
Move the tenant load and reload commands to be executed by the detect
loader thread(s).

Limitation: no yaml parsing in parallel. The Conf API is currently not
thread safe, so don't load the tenant config (yaml) in parallel.
 10 years ago
Victor Julien eb09118d64 detect: create loader threads 
To speed up startup with many tenants, tenant loading will be parallelized.
As no tempary threads should be used for these memory allocation heavy
tasks, this patch adds new type of 'command' thread that can be used to
load and reload tenants.

This patch hardcodes the number of loaders to 4. Future work will make it
dynamic.

The loader thread essentially sleeps constantly. When a tasks is sent to
it, it will wake up and execute it.
 10 years ago
Victor Julien e19c41a807 multi-detect: hash lookup for tenants 
Use hash for storing and looking up det_ctxs.
 10 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code 
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
 10 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant 
Generate error if tentant to be added is already loaded.
 10 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file 
Load tenants and mappings from the suricata.yaml when available.
 10 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler 
Remove a tenant handler from the list and apply it.
 10 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml 
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
 10 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered 
Force user to select the method at startup.
 10 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx 
Otherwise counters are only registered after the stats api is
already fixed.
 10 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 10 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler 
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
 10 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants 
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
 10 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry 
Limited to Pcap only currently.
 10 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 10 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch 
At start up we will set this flag based on "multi-detect.enabled".
 10 years ago
Victor Julien def2b58725 detect: initial MT lookup logic 
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
 10 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands 
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
 10 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 10 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 10 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 10 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 10 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 10 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 10 years ago
Victor Julien 1ef786e7cb counters: rename register API calls 
Also remove 'type' parameter which was always the same.
 10 years ago
Victor Julien b293a4b7d0 counters: remove unused description 10 years ago
Victor Julien b2da57c827 reference: remove global 10 years ago
Victor Julien 393689ce44 classification: remove global from parsing 
Parsing code used a 'fd' global. Remove this.
 10 years ago
Giuseppe Longo f0c54d4764 Detect engine for smtp file_data file_data: inspecting smtp attachments 
Create a buffer to store reassembled file chunks,
and inspect the content.
 10 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA 
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
 10 years ago
Victor Julien 724c7044e1 detect-reload: 0 detect threads is no error 
The reload code would consider 0 detect threads to be an error,
but it's not in case of unix socket mode.
 11 years ago
Victor Julien 7c9e015748 unix-socket: implement reload-rules 
Implement the reload-rules unix socket command. The unix command
thread signals the main thread to do the reload and it waits for
it to complete.
 11 years ago
Victor Julien 71d01f06b9 detect reload: load config 
Load the YAML into a prefix "detect-engine-reloads.N" where N is the
reload counter. This way we can load the updated config w/o overwriting
the current one.
 11 years ago
Victor Julien b51075e804 detect: remove config at prefix 
Remove config at prefix when freeing a detect engine.
 11 years ago
Victor Julien 7108085d33 detect: initialize detection engine by prefix 
Initalize detection engine by configuration prefix.

    DetectEngineCtxInitWithPrefix(const char *prefix)

Takes the detection engine configuration from:
<prefix>.<config>

If prefix is NULL the regular config will be used.

Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
 11 years ago
Victor Julien a80cc696d7 detect: allow det_ctx->de_ctx to be NULL 
When freeing det_ctx, allow de_ctx to be NULL.
 11 years ago
Victor Julien c9a8262ccf detect: reload thread init cleanup 
Rename the thread init function DetectEngineThreadCtxInitForLiveRuleSwap
to DetectEngineThreadCtxInitForReload and change it's logic to take the
new detection engine as argument and let it return the
DetectEngineThreadCtx or NULL on error.

The old approach used the thread init API format, but it wasn't used in
that way.
 11 years ago
Victor Julien 55e7370fc5 detect reload: allow master update during reload 
Add DetectEngineReference, which takes a reference to a detect engine,
and make DetectEngineThreadCtxInitForLiveRuleSwap use it. This way
reload will not depend on master staying the same. This allows master
to be updated in between w/o affecting the reload that is in progress.
 11 years ago
Victor Julien b1c54a8673 detect: remove old live reload implementation 
Remove code that ran the reload in it's own thread. Simplify the
signal handling.
 11 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine 
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
 11 years ago
Victor Julien 38b349af1e runmodes: remove DetectEngineCtx passing from API 
No longer pass a pointer to the current detection engine to the
runmode API calls.

Note: breaks delayed detect. Will be fixed in a future commit.
 11 years ago
Victor Julien b038b6a2f8 unittests: add exception to detect engine setup 
Add code to allow for unittests not following the complete api.

Update replace tests as they don't use the unittests runmode that
powers the workaround based on RunmodeIsUnittests().
 11 years ago
Victor Julien d66fa1add1 detect: update detect engine management 
Update detect engine management to make it easier to reload the detect
engine.

Core of the new approach is a 'master' ctx, that keeps a list of one or
more detect engines. The detect engines will not be passed to any thread
directly, but instead will only be accessed through the detect engine
thread contexts. As we can replace those atomically, replacing a detect
engine becomes easier.

Each thread keeps a reference to its detect context. When a detect engine
is replaced or removed, it's added to a free list. Once its reference
count reaches 0, it is freed.
 11 years ago
Victor Julien 49bad2cfba detect: consolidate more setup into DetectEngineCtxInit 
Loading of classifications, references and action order was done
unconditionally, so can be done in one place.
 11 years ago
Victor Julien 6723d03c7e http: add inspection engine for http request line 
No MPM though.
 11 years ago
Victor Julien 04e49cea89 Fix live reload detect counter setup 
When profiling was compiled in the detect counters were not setup
properly after a reload.
 11 years ago
Victor Julien a8c16405fb detect: properly size det_ctx::non_mpm_id_array 
Track which sgh has the higest non-mpm sig count and use that value
to size the det_ctx::non_mpm_id_array array.
 11 years ago
Victor Julien 62751c8017 Fix live reload detect thread ctx setup 
Code failed to setup non_mpm_id_array in case of a live reload.
 11 years ago
Victor Julien b5a3127151 detect: add mask check prefilter for non mpm list 
Add mask array for non_mpm sigs, so that we can exclude many sigs before
we merge sort.

Shows 50% less non mpm sigs inspected on average.
 11 years ago
Ken Steele 904441327c Conditionalize SigMatch performance counters. 
Only include the counters when PROFILING.
 11 years ago
Victor Julien 30b7fdcb49 Detect perf counters 11 years ago
DIALLO David 55c5081240 Detect-engine: Add Modbus detection engine 
Management of Modbus Tx

Based on DNS source code.

Signed-off-by: David DIALLO <diallo@et.esia.fr>
 11 years ago
DIALLO David b3bf2f9939 Detect: Add Modbus keyword management 
Add the modbus.function and subfunction) keywords for public function match in rules (Modbus layer).
Matching based on code function, and if necessary, sub-function code
or based on category (assigned, unassigned, public, user or reserved)
and negation is permitted.

Add the modbus.access keyword for read/write Modbus function match in rules (Modbus layer).
Matching based on access type (read or write),
and/or function type (discretes, coils, input or holding)
and, if necessary, read or write address access,
and, if necessary, value to write.
For address and value matching, "<", ">" and "<>" is permitted.

Based on TLS source code and file size source code (address and value matching).

Signed-off-by: David DIALLO <diallo@et.esia.fr>
 11 years ago
Victor Julien d0357c6169 smtp: add file inspection engine 
Fix file inspection engine.

TODO: test
 11 years ago
Victor Julien 033409a042 iprep: cleanup ctx on shutdown 
~~Dr.M~~ Error #1: LEAK 480 direct bytes 0x0aae7fc0-0x0aae81a0 + 0 indirect bytes
~~Dr.M~~ # 0 replace_malloc                    [/work/drmemory_package/common/alloc_replace.c:2373]
~~Dr.M~~ # 1 SRepInit                          [.../Suricata/src/reputation.c:594]
~~Dr.M~~ # 2 DetectEngineCtxInit               [.../src/detect-engine.c:844]
~~Dr.M~~ # 3 main                              [.../Suricata/src/suricata.c:2230]
 11 years ago
Ken Steele 228abb89ac fixup 11 years ago
Ken Steele 8f1d75039a Enforce function coding standard 
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
 11 years ago
Victor Julien cc54250cf9 Fix live reload segv when startup isn't complete 
If a live reload signal was given before the engine was fully started
up (e.g. pcap file thread waiting for a disk to spin up), a segv could
occur.

This patch only enables live reloads after the threads have been
started up completely.
 11 years ago
Victor Julien c5041d35d5 Fix live reload 
Fix memsets clearing out of bounds memory on live reload, causing
crashes and corrupted backtraces.

Bug #1128.
 12 years ago