lock_acquire: Calling pthread_mutex_lock acquires lock ThreadVars_.ctrl_mutex.
725 SCCtrlMutexLock(th_v->ctrl_mutex);
CID 1554214: (#1 of 1): Indefinite wait (BAD_CHECK_OF_WAIT_COND)
dead_wait: A wait is performed without ensuring that the condition is not already satisfied while holding lock ThreadVars_.ctrl_mutex. This can cause a deadlock if the notification happens before the lock is acquired.
Acquire the lock, then check the wait condition in a loop, without releasing with the lock before the wait. This will prevent deadlocks and failed conditions from spurious wakeups.
CC detect-engine-loader.o
In file included from /usr/include/stdio.h:970,
from suricata-common.h:77,
from detect-engine-loader.c:24:
In function 'fgets',
inlined from 'DetectLoadSigFile' at detect-engine-loader.c:139:11:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:313:12: warning: argument 2 value -1 is negative [-Wstringop-overflow=]
313 | return __fgets_alias (__s, __n, __stream);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/features.h:523,
from /usr/include/dirent.h:25,
from suricata-common.h:73:
/usr/include/x86_64-linux-gnu/bits/stdio2-decl.h: In function 'DetectLoadSigFile':
/usr/include/x86_64-linux-gnu/bits/stdio2-decl.h:96:14: note: in a call to function '__fgets_alias' declared with attribute 'access (write_only, 1, 2)'
96 | extern char *__REDIRECT (__fgets_alias,
| ^~~~~~~~~~
For firewall mode, set the pseudo table in the rule and use this
in alert queue ordering, so that rule actions are applied in the
expected order:
packet:filter -> packet:td -> app:filter -> app:td
This makes sure that a packet:td drop is applied before a app:filter
accept.
Config:
Firewall rules are like normal rule, with some key differences.
They are loaded separate, and first, from:
```yaml
firewall-rule-path: /etc/suricata/firewall/
firewall-rule-files:
- fw.rules
```
Can also be loaded with --firewall-rules-exclusive: Mostly for QA purposes.
Allow -S with --firewall-rules-exclusive, so that firewall and threat detection
rules can be tested together.
Rules:
Differences with regular "threat detection" rules:
1. these rules are evaluated before threat detection rules
2. these rules are evaluated in the order as they appear in the rule file
3. currently only rules specifying an explicit hook at supported
a. as a consequence, no rules will be treated as (like) IP-only, PD-only or
DE-only
Require explicit action scope for firewall rules. Default policy is
drop for the firewall tables.
Actions:
New action "accept" is added to allow traffic in the filter tables.
New scope "accept:tx" is added to allow accepting a transaction.
Tables:
Rulesets are per table.
Table processing order: `packet:filter` -> `packet:td` -> `app:*:*` -> `app:td`.
Each of the tables has some unique properties:
`packet:filter`:
- default policy is `drop:packet`
- rules are process in order
- action scopes are explicit
- `drop` or `accept` is immediate
- `accept:hook` continues to `packet:td`
`packet:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `app:*:*` or `alert/action finalize`
`app:*:*`:
- default policy is `drop:flow`
- rules are process in order
- action scopes are explicit
- `drop` is immediate
- `accept` is conditional on possible `drop` from `packet:td`
- `accept:hook` continues to `app:td`, `accept:packet` or `accept:flow`
continues to `alert/action finalize`
`app:td`:
- default policy is `accept:hook`
- rules are ordered by IDS/IPS ordering logic
- action scopes are implicit
- actions are queued
- continues to `alert/action finalize`
Implementation:
During sigorder, split into packet:filter, app:*:* and general td.
Allow fw rules to work when in pass:flow mode. When firewall mode is enabled,
`pass:flow` will not skip the detection engine anymore, but instead
process the firewall rules and then apply the pass before inspecting threat
detect rules.
Threads are only set to paused upon initialization and never again, we
should only have to wait once, so move the wait before any loop that
was waiting before.
Additionally, if the thread was killed while waiting to be unpaused,
don't enter the loop.
The pattern of checking the pause flag, setting to paused then
waiting to unpause was done enough times to factor out into its own
function. This is also needed by library users who bring their own
packet acquisition threads.
Remove the global "run_mode" var as it was a duplicate of the runmode on
the "instance" struct. For direct access outside of suricata.c, use the
getter function.
Also expose a setter function for unit tests that need to change it.
Rework locking logic to avoid the following coverity warning.
** CID 1591966: Concurrent data access violations (MISSING_LOCK)
/src/detect-engine-loader.c: 475 in DetectLoadersSync()
474 SCCtrlMutexLock(loader->tv->ctrl_mutex);
>>> CID 1591966: Concurrent data access violations (MISSING_LOCK)
>>> Accessing "loader->tv" without holding lock "DetectLoaderControl_.m". Elsewhere, "DetectLoaderControl_.tv" is written to with "DetectLoaderControl_.m" held 1 out of 1 times (1 of these accesses strongly imply that it is necessary).
475 pthread_cond_broadcast(loader->tv->ctrl_cond);
476 SCCtrlMutexUnlock(loader->tv->ctrl_mutex);
The warning itself is harmless.
A dead lock could occur at start up, where a loader thread would
get stuck on it's condition variable, while the main thread was
polling the loaders task results.
The vector to the dead lock is as follows:
main loader
DetectEngineMultiTenantSetup
-DetectLoaderSetupLoadTenant
--DetectLoaderQueueTask
---lock loader
---add task
---unlock loader
lock loader
check/exec tasks
unlock loader
---wake up threads
lock ctrl mutx
cond wait ctrl
unlock ctrl
-DetectLoadersSync
--lock loader
--check tasks
--unlock loader
Between the main thread unlocking the loader and waking up the
threads, it is possible that the loader has already moved ahead
but not yet entered its conditional wait. The main thread sends
its condition signal, but since the loader isn't yet waiting on
it the signal is ignored. Then when the loader does enter its
conditional wait, the signal is not sent again.
This patch updates the logic to send signals much more often.
It also makes sure that the signal is sent under lock, as the
API requires.
Bug: #6766.
"sigerror_ok" and "sigerror_requires" were not being reset after each
rule which could lead to a rule load error being incorrectly tracked
as skipped rather than failed.
Also initialize "skippedsigs" to 0 along with "goodsigs" and
"badsigs", while not directly related to this issue, could also throw
off some stats.
Ticket: #6710
Add a new rule keyword "requires" that allows a rule to require specific
Suricata versions and/or Suricata features to be enabled.
Example:
requires: feature geoip, version >= 7.0.0, version < 8;
requires: version >= 7.0.3 < 8
requires: version >= 7.0.3 < 8 | >= 8.0.3
Feature: #5972
Co-authored-by: Philippe Antoine <pantoine@oisf.net>
Issue: 6239
This commit moves the global variables associated with engine analysis
into the detect engine context. Doing so provides encapsulation of the
analysis components as well as thread-safe operation in a multi-tenant
(context) deployment.
Tested on Fedora 37 with clang 15.
app-layer.c:1055:27: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerSetupCounters()
^
void
app-layer.c:1176:29: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerDeSetupCounters()
^
void
2 errors generated.
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
When reloading rules, respect `--set default-rule-path=...` from the
command line if set.
Previously the rule reload would always take the default-rule-path from
the configuration file, even if overrided on the command line.
Issue: #1911
This commit improves the handling of threshold.config. When used with
"-T", a non-zero return code occurs when the file cannot be validated.
To maintain legacy behavior, when "-T" is not used and threshold.config
contains one or more invalid lines, Suricata continues execution.
Until this point the SC_ATOMIC_ADD macro pointed to a 'add_fetch'
intrinsic. This patch changes it to a 'fetch_add'.
There are 2 reasons for this:
1. C11 stdatomics.h has only 'atomic_fetch_add' and no 'add_fetch'
So this patch prepares for adding support for C11 atomics.
2. It was not consistent with SC_ATOMIC_SUB, which did use 'fetch_sub'
and not 'sub_fetch'.
Most callers are not using the return value, so these are unaffected.
The callers that do use the return value are updated.
The idea of an OK signature parsing error is an error that is
allowed to occur, but still lets test mode pass, unlike
silent errors which will still fail testing.
This is introduced to allow for app-layer event keywords to be
removed, but not have old rules fail out on this case. For example
the Rust DNS parser removes from DNS app-layer events that are
not used anymore.
To signal that an error is OK, -3 is returned. This also implies
silent.
A sigmatches 'Setup' function may indicate it intends to fail
silently after the first error. It will return -2 instead of -1
in this case.
This is tracked in the DetectEngineCtx object, so errors will
be shown again at rule reloads.
EngineAnalysisRules2 was in a strange location where it did not respect
the --engine-analysis flag. It has been moved to the same call location
as EngineAnalysisRules.
Move previously global table into detect engine ctx. Now that we
can register buffers at rule loading time we need to take concurrency
into account.
Move DetectBufferType to detect.h and update DetectBufferCtx API calls
to include a detect engine ctx reference.
according to its man page, sigprocmask has undefined behavior in
multithreaded environments. Instead of explictly blocking the handling
of SIGUSR2 in every thread, direct block handling SIGUSR2 before
creating the threads and enable again the handling of this signal
afterwards. In this way, only the main thread will be able to manage
this signal properly.
The "sig_file" argument of DetectLoadCompleteSigPath() is not checked for NULL-values. If this argument is NULL a SEGV occurs because of a dereferenced NULL-pointer in strlen in PathIsAbsolute. This commit fixes bug #2347. Here is the ASAN-output:
==17170==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fd1afa00646 bp 0x7ffe8398e6d0 sp 0x7ffe8398de58 T0)
0 0x7fd1afa00645 in strlen (/lib/x86_64-linux-gnu/libc.so.6+0x80645)
1 0x7fd1b3242eec (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x3beec)
2 0x5561c8cddf7f in PathIsAbsolute /root/suricata-1/src/util-path.c:40
3 0x5561c8cddfea in PathIsRelative /root/suricata-1/src/util-path.c:65
4 0x5561c89275e4 in DetectLoadCompleteSigPath /root/suricata-1/src/detect.c:264
5 0x5561c8929e75 in SigLoadSignatures /root/suricata-1/src/detect.c:486
6 0x5561c8c0f2b3 in LoadSignatures /root/suricata-1/src/suricata.c:2419
7 0x5561c8c1051d in PostConfLoadedDetectSetup /root/suricata-1/src/suricata.c:2550
8 0x5561c8c12424 in main /root/suricata-1/src/suricata.c:2887
9 0x7fd1af9a02b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
10 0x5561c87b31a9 in _start (/usr/local/bin/suricata+0xc51a9)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libc.so.6+0x80645) in strlen
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
Victor Julien
Philippe Antoine
Jason Ish
Lukas Sismis
Shivani Bhardwaj
Jeff Lucovsky
Jeff Lucovsky
Victor Julien
Jacob Masen-Smith
Maurizio Abba
Wolfgang Hotwagner