firewall: a non-fast-pattern engine does not miss on an incomplete state

An engine that is not the rule's fast-pattern engine is reached on every update
while the transaction is at its hook. While that state is not complete, such an
engine's buffer is missing or partial, yet a miss returned -1, a final
no-match, and the walk applied the hook's default policy: an allow-list rule
such as `http.host; pcre:"/good\.example/";` dropped a benign request whose
headers span two segments.

In effect that changes rules without a prefilter. For a gated rule the first
engine is the fast-pattern one, and its in-progress miss already returned -2
through mpm_in_progress; this arm is only reached when nothing matched with no
flags set, which for a gated rule means the pattern engine already deferred.

A miss on a buffer engine of the transaction's current, incomplete state now
returns -2, as a fast pattern in progress does: no default policy is applied
for it, and the rule is inspected again on the next update. Engines without a
buffer, the hook's generic list (flowbits, xbits), stay final: they read
variables the rules before them already set.

The deferral is bounded by the state: on the update that completes the state,
the engine sits below the transaction's progresses and the miss is final, so
the hook's default policy decides there. A flow that never completes the state
keeps it undecided, as it does for a reset - no default is applied and the flow
is not decided by omission.

Ticket: #9257.

src/detect.c

@@ -1324,6 +1324,9 @@ static int DetectRunTxInspectRule(ThreadVars *tv, DetectEngineCtx *de_ctx,
     uint16_t file_no_match = 0;
     bool mpm_before_progress = false;   // is mpm engine before progress?
     bool mpm_in_progress = false;       // is mpm engine in a buffer we will revisit?
+    /* a firewall rule missed on a buffer of the tx's current state, which is not
+     * complete: the buffer can still grow, so the miss is not final */
+    bool fw_in_progress = false;
 
     TRACE_SID_TXS(s->id, tx, "starting %s", direction ? "toclient" : "toserver");
 
@@ -1476,6 +1479,14 @@ static int DetectRunTxInspectRule(ThreadVars *tv, DetectEngineCtx *de_ctx,
                 file_no_match = 1;
             }
             /* implied DETECT_ENGINE_INSPECT_SIG_NO_MATCH */
+            /* only an engine that reads a buffer: the hook's generic list reads
+             * variables, which the rules before this one already set or not */
+            if (match == DETECT_ENGINE_INSPECT_SIG_NO_MATCH && engine->v2.GetData != NULL &&
+                    (s->flags & (SIG_FLAG_FIREWALL | SIG_FLAG_FW_HOOK_LTE)) == SIG_FLAG_FIREWALL &&
+                    engine->progress == tx->tx_progress && tx->tx_progress < tx->tx_end_state &&
+                    (p->flags & PKT_PSEUDO_DETECTLOG_FLUSH) == 0) {
+                fw_in_progress = true;
+            }
             if (engine->mpm && mpm_before_progress) {
                 inspect_flags |= DE_STATE_FLAG_SIG_CANT_MATCH;
                 inspect_flags |= BIT_U32(engine->id);
@@ -1551,7 +1562,9 @@ static int DetectRunTxInspectRule(ThreadVars *tv, DetectEngineCtx *de_ctx,
         } else {
             if (inspect_flags == 0) {
                 TRACE_SID_TXS(s->id, tx, "no match: inspect_flags %08x", inspect_flags);
-                return -1;
+                /* nothing stored: a rule without a prefilter is in the list on every
+                 * update, so it is revisited like a fast pattern in progress is */
+                return fw_in_progress ? -2 : -1;
             }
         }
     }