A lot of time was spent in `SigMatchListSMBelongsTo` for the `mpm_sm`.
Optimize this by keeping the value at hand during Signature parsing and
detection engine setup.
In preparation of more dynamic logic in rule loading also doing
some registration, allow for buffers to be registered as fast_patterns
during rule parsing.
Leaves the register time registrations mostly as-is, but copies the
resulting list into the DetectEngineCtx and works with that onwards.
This list can then be extended.
Instead of a map that is constantly realloc'd, use 2 hash tables for
DetectBufferType entries: one by name (+transforms), the other by
id. Use these everywhere.
Allow progress values in the range 0-47 so we have 48 bits to track
prefilter engines.
Mark bits 48-62 as reserved explicitly.
Add debug validation checks to make sure the reserved space isn't used.
Buffers with transforms are based on the non-transformed "base"
buffer, with a new ID assigned and the transform callbacks added.
This patch stores the id of the original buffer in the new buffer
inspect and prefilter structures. This way the buffers with and
without transforms can share some of the logic are progression
of file and body inspection trackers.
Related tickets: #4361#4199#3616
Replaces all patterns of SCLogError() followed by exit() with
FatalError(). Cocci script to do this:
@@
constant C;
constant char[] msg;
@@
- SCLogError(C,
+ FatalError(SC_ERR_FATAL,
msg);
- exit(EXIT_FAILURE);
Closes redmine ticket 3188.
This commit adds support for transform-specific options. During Setup,
transforms have the signature string available for options detection.
When a transform detects an option, it should convert the option into an
internal format and supply a pointer to this format as the last argument
to DetectSignatureAddTransform.
Transforms that support options must provide a function in their
Sigmatch table entry. When the transform is freed, a pointer to the
internal format of the option is passed to this function.
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.
Share part of the registration code with the AppLayer.
Implement for the tcp.hdr and udp.hdr keywords.
Prepare MPM part of the detection engine for a new type of per
packet matching, where the L4 header will be inspected.
Preparation for TCP header inspection keyword.
The fast pattern selection logic would truncate a patterns len to
255 leading to assigning the same pid to different patterns.
This in turn would be caught by the hyperscan setup code which would
abort.
Bug #2714.
Move previously global table into detect engine ctx. Now that we
can register buffers at rule loading time we need to take concurrency
into account.
Move DetectBufferType to detect.h and update DetectBufferCtx API calls
to include a detect engine ctx reference.
Introduce InspectionBuffer a structure for passing data between
prefilters, transforms and inspection engines.
At rule parsing time, we'll register new unique 'DetectBufferType's
for a 'parent' buffer (e.g. pure file_data) with its transformations.
Each unique combination of buffer with transformations gets it's
own buffer id.
Similarly, mpm registration and inspect engine registration will be
copied from the 'parent' (again, e.g. pure file_data) to the new id's.
The transforms are called from within the prefilter engines themselves.
Provide generic MPM matching and setup callbacks. Can be used by
keywords to avoid needless code duplication. Supports transformations.
Use unique name for profiling, to distinguish between pure buffers
and buffers with transformation.
Add new registration calls for mpm/prefilters and inspect engines.
Inspect engine api v2: Pass engine to itself. Add generic engine that
uses GetData callback and other registered settings.
The generic engine should be usable for every 'simple' case where
there is just a single non-streaming buffer. For example HTTP uri.
The v2 API assumes that registered MPM implements transformations.
Add util func to set new transform in rule and add util funcs for rule
parsing.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
Now that detect moves the raw progress forward, it's important
to deal with the case where detect don't consider raw inspection.
If no 'stream' rules are active, disable raw. For this the disable
raw flag is now per stream.
The new Hyperscan 4.4 API provides a function to check for SSSE3
presence at runtime. This allows us to fall back to non-Hyperscan
matchers on systems without SSSE3 even when the suricata executable
is built with Hyperscan support. Addresses Redmine issue #2010.
Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
Tested-by: Arturo Borrero Gonzalez <arturo@debian.org>
Register keywords globally at start up.
Create a map of the registery per detection engine. This we need because
the sgh_mpm_context value is set per detect engine.
Remove APP_MPMS_MAX.