Allow progress values in the range 0-47 so we have 48 bits to track
prefilter engines.
Mark bits 48-62 as reserved explicitly.
Add debug validation checks to make sure the reserved space isn't used.
Fix FNs in case of too many prefilter engines. A transaction was tracking
which engines have run using a u64 bit array. The engines 'local_id' was
used to set and check this bit. However the bit checking code didn't
handle int types correctly, leading to an incorrect left shift result of
a u32 to a u64 bit value.
This commit addresses that by fixing the int handling, but also by
changing how the engines are tracked.
To avoid wasting prefilter engine tracking bit space, track what
ran by the progress they are registered at, instead of the individual
engine id's. While we can have many engines, the protocols use far
fewer unique progress values. So instead of tracking for dozens of
prefilter id's, we track for the handful of progress values.
To allow for this the engine array is sorted by tx_min_progress, then
app_proto and finally local_id. A new field is added to "know" when
the last relevant engine for a progress value is reached, so that we
can set the prefilter bit then.
A consquence is that the progress values have a ceiling now that
needs to fit in a 64 bit bitarray. The values used by parsers currently
does not exceed 5, so that seems to be ok.
Bug: #4685.
Unify handling of signature matches between various rule types and
between noalert and regular rules.
"noalert" sigs are added to the alert queue initially, but removed
from it after handling their actions. This way all actions are applied
from a single place.
Make sure flow drop and pass are mutually exclusive.
The above addresses issue with pass and drops not getting applied
correctly in various cases.
Bug: #4663
Bug: #4670
When stateful detection rules, for which detection has already started
for a previous packet, are added to the candidates array, the array
is sorted to mantain the correct inspection order. However, due to a
trivial error in the sort helper the array was sorted in descending
instead of ascending order.
Improve flow file flags and file flags updates. Introduce a mask
that is set at start up to avoid lots of runtime checks.
Disable cocci flags check as it doesn't support the more dynamic
nature of the flag updates.
ICMP unreachable errors are linked to the flow they send an error for.
This would lead to the detection engine calling the TX inspection
engines on them.
The stream inspect engine would default to a match for non-UDP
and non-TCP as for ICMP we're not expected to use a TX inspect engine
for stream data.
This all would lead to a false positive match.
This patch fixes this by making sure the TX engines are not called if
the packet protocol and flow protocol are not the same.
Bug #2769.
Instead of hard coded calls to the inspection logic for
payload inspection and 'MATCH'-list inspection use a callback
approach. This will register a callback per 'sm_list' much like
how app-layer inspect engines are registered.
This will allow for adding more types later without adding
runtime overhead.
Implement the callback for the PMATCH and MATCH logic.
Make sure all file sig mismatches indicate this in their return
code, not just the ones with filestore enabled. This is needed
to tell the stateful detect engine that it is dealing with a file
sig, so it can make sure these are inspected correctly even if
there are possibly multiple files per tx.
Fix match array reset depending on prefilter matches for the
current run. If there were none, the match array of the previous
packet was used. This could lead to inspection of rules from the
wrong rule group.
If a bad RST was received the stream inspection would not happen
for that packet, but it would still move the 'raw progress' tracker
forward. Following good packets would then fail to detect anything
before the 'raw progress' position.
Bug #2770
Reported-by: Alexey Vishnyakov
Fix and Optimize cleanup. For the simple single inspect buffer optimize
the cleanup by keeping track of the actually used buffers. This avoid
looping over unused buffers.
Fix the case of cleaning not being done after a tx if the next tx is
also inspected in the context of the same packet.
Fix cleanup of the multi-inspect buffers. Optimize in 2 ways. First
like with single keep track of which multi-inspect buffers have been
used. Second, keep a max of the buffers within a multi-inspect buffer.
Use this max to limit (nested) looping.
Fix mpm progress being updated by irrelevant engines. Esp in the
case of file_data engines, signature can contain multiple versions
of the same engine, registered for different 'progress' values.
This would lead to signatures being considered 'can't match' even
in cases where they clearly could still match.
Only consider those progress values that apply to the protocol in
use.
Victor Julien
Jason Ish
Victor Julien
Joshua Lumb
Philippe Antoine