detect: extend app-layer-protocol to accept a pipe-separated value list

Extend the app-layer-protocol keyword to accept a pipe-separated list of
protocol values, so a single rule can match any of several protocols:

    app-layer-protocol:[!]<proto1>|<proto2>[|...][,<qualifier>]...;

A non-negated list matches when the flow's protocol equals any listed value
(OR); a negated list matches when it equals none of them (NOR). The
single-value form and the trailing mode qualifier are unchanged.

Matching keeps the historical AppProtoEquals() equivalences by default
(dns/doh2, http/http1/http2, dcerpc/smb, ...). An `exact` qualifier selects
strict identity matching with no equivalences and no http umbrella; it
combines with a direction mode in any order. Because a flow is never the
generic ALPROTO_HTTP, `http,exact` is rejected at load.

Values are expanded once at rule load into an effective match-set bitmask, so
the per-packet match is a single bitmask test. Single-value rules remain
prefilterable; multi-value rules are excluded from prefiltering and an
explicit prefilter on them is rejected. Conflicting keyword combinations
(duplicate or overlapping negations, mixed positive/negated) are rejected at
load. Engine-analysis reports the effective match set.

Ticket: 7705
(cherry picked from commit 43bc2db41e)

doc/userguide/rules/app-layer.rst

@@ -10,7 +10,11 @@ Match on the detected app-layer protocol.
 
 Syntax::
 
-    app-layer-protocol:[!]<protocol>(,<mode>);
+    app-layer-protocol:[!]<protocol>[,<qualifier>]...;
+    app-layer-protocol:[!]<proto1>|<proto2>[|...|<protoN>][,<qualifier>]...;
+
+Each ``<qualifier>`` is either a ``<mode>`` (at most one, see below) or the
+``exact`` option, in any order.
 
 Examples::
 
@@ -21,6 +25,12 @@ Examples::
     app-layer-protocol:http,to_server; app-layer-protocol:tls,to_client;
     app-layer-protocol:http2,final; app-layer-protocol:http1,original;
     app-layer-protocol:unknown;
+    app-layer-protocol:unknown|tls;
+    app-layer-protocol:unknown|tls|http;
+    app-layer-protocol:!tls|http;
+    app-layer-protocol:tls|http,either;
+    app-layer-protocol:dns,exact;
+    app-layer-protocol:tls|dns,either,exact;
 
 A special value 'failed' can be used for matching on flows in which
 protocol detection failed. This can happen if Suricata doesn't know
@@ -42,12 +52,92 @@ By default, (if no mode is specified), the mode is ``direction``.
 .. note:: when negation is used, like ``!http``, it will not match on the
    "unknown" state in the flow.
 
+Protocol equivalences and the ``exact`` option
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default a value matches its related protocols as well as itself. For
+example ``http`` matches ``http1`` and ``http2``, ``dns`` also matches
+``doh2`` (DNS over HTTP/2), and ``dcerpc`` also matches ``smb``. This is the
+long-standing behaviour and keeps existing rules working.
+
+Add the ``exact`` qualifier to match strictly, with no equivalences: the
+flow's protocol must equal the configured value exactly. ``exact`` applies to
+all values in the list and can be combined with a mode::
+
+    app-layer-protocol:dns,exact;          # matches dns only, not doh2
+    app-layer-protocol:tls|dns,either,exact;
+
+Because ``exact`` disables all equivalences, the generic ``http`` value is not
+expanded to ``http1``/``http2`` either. A flow is never classified as the
+generic ``http``, so ``app-layer-protocol:http,exact`` can never match and is
+rejected at rule load; use ``http1`` or ``http2`` instead.
+
 Here is an example of a rule matching non-http traffic on port 80:
 
 .. container:: example-rule
 
     alert tcp any any -> any 80 (msg:"non-HTTP traffic over HTTP standard port"; flow:to_server; app-layer-protocol:!http,final; sid:1; )
 
+Multi-value form
+~~~~~~~~~~~~~~~~
+
+The ``app-layer-protocol`` keyword also accepts a pipe-separated (``|``) list
+of protocol values. A rule matches when the flow's resolved application-layer
+protocol equals **any** value in the list (logical OR).
+
+Syntax::
+
+    app-layer-protocol:[!]<proto1>|<proto2>[|...|<protoN>](,<mode>);
+
+Using ``|`` for the list keeps the optional trailing ``,<mode>`` qualifier
+unambiguous, so the single-value ``<protocol>,<mode>`` form is unchanged.
+
+Examples::
+
+    app-layer-protocol:unknown|tls;
+    app-layer-protocol:unknown|tls|http;
+    app-layer-protocol:tls|http,either;
+    app-layer-protocol:!tls|http;
+
+The ``unknown|<proto>`` detection-window idiom
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When Suricata has not yet classified a flow's protocol (the "detection
+window"), the flow's app-layer protocol is ``unknown``. Once protocol
+detection completes, the protocol transitions to its classified value
+(e.g., ``tls``, ``http``). Including ``unknown`` in a multi-value list
+allows a single rule to cover both the detection window and the confirmed
+protocol::
+
+    app-layer-protocol:unknown|tls;
+
+This rule matches during the detection window (while the protocol is still
+``unknown``) **and** after classification (when the protocol is ``tls``).
+If the flow is classified to a protocol not in the list (e.g., ``http``),
+the rule stops matching once the protocol is classified; in firewall mode the
+flow is then handled by the default policy if no other rule accepts it.
+
+Negated multi-value (NOR semantics)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the multi-value form is negated with ``!``, it implements NOR semantics
+across the entire list: the rule matches when the resolved application-layer
+protocol is **known** AND matches **none** of the listed values.
+
+Example::
+
+    app-layer-protocol:!tls|http;
+
+This matches when the flow's protocol is known and is neither ``tls`` nor
+``http`` (e.g., it matches ``dns``, ``ssh``, ``smtp``, etc.).
+
+.. note:: Negated multi-value rules do not match during the detection window
+   (when the protocol is still ``unknown``). This prevents false positives
+   before protocol classification is complete.
+
+.. note:: The value ``unknown`` cannot appear in a negated list. The parser
+   rejects ``!unknown`` and ``!unknown|tls`` at rule-load time.
+
 .. _proto-detect-bail-out:
 
 Bail out conditions

src/detect-app-layer-protocol.h

@@ -24,6 +24,31 @@
 #ifndef SURICATA_DETECT_APP_LAYER_PROTOCOL__H
 #define SURICATA_DETECT_APP_LAYER_PROTOCOL__H
 
+#include "app-layer-protos.h"
+
 void DetectAppLayerProtocolRegister(void);
+const char *DetectAppLayerProtocolModeName(uint8_t mode);
+struct DetectAppLayerProtocolData_;
+uint16_t DetectAppLayerProtocolGetValues(
+        const struct DetectAppLayerProtocolData_ *data, AppProto *out, uint16_t max);
+
+/**
+ * \brief Per-rule keyword data for `app-layer-protocol:`.
+ *
+ * `alprotos` is the effective match set: a bitmask (one bit per AppProto,
+ * sized g_alproto_max) holding every flow protocol that should match, with the
+ * AppProtoEquals() equivalences (or, with the `exact` option, only the exact
+ * values) already expanded in at rule load. The per-packet match is then a
+ * single bitmask test. `alproto` is the first configured value, used as the
+ * prefilter bucket key for single-value (prefilterable) rules.
+ */
+typedef struct DetectAppLayerProtocolData_ {
+    AppProto alproto; /**< first configured value; single-value prefilter key */
+    bool negated;
+    bool exact;   /**< `exact` option: strict identity, no equivalences/umbrella */
+    bool is_list; /**< more than one value configured (not prefilterable) */
+    uint8_t mode;
+    uint8_t *alprotos; /**< effective match set (g_alproto_max bits) */
+} DetectAppLayerProtocolData;
 
 #endif /* SURICATA_DETECT_APP_LAYER_PROTOCOL__H */

src/detect-engine-analyzer.c

@@ -54,6 +54,7 @@
 #include "util-var-name.h"
 #include "detect-icmp-id.h"
 #include "detect-tcp-window.h"
+#include "detect-app-layer-protocol.h"
 
 static int rule_warnings_only = 0;
 
@@ -975,6 +976,21 @@ static void DumpMatches(RuleAnalyzer *ctx, SCJsonBuilder *js, const SigMatchData
                 SCJbClose(js);
                 break;
             }
+            case DETECT_APP_LAYER_PROTOCOL: {
+                const DetectAppLayerProtocolData *ad = (const DetectAppLayerProtocolData *)smd->ctx;
+                SCJbOpenObject(js, "app_layer_protocol");
+                AppProto vals[256];
+                uint16_t n = DetectAppLayerProtocolGetValues(ad, vals, ARRAY_SIZE(vals));
+                SCJbOpenArray(js, "protocols");
+                for (uint16_t i = 0; i < n; i++) {
+                    SCJbAppendString(js, AppProtoToString(vals[i]));
+                }
+                SCJbClose(js);
+                SCJbSetString(js, "mode", DetectAppLayerProtocolModeName(ad->mode));
+                SCJbSetBool(js, "negated", ad->negated);
+                SCJbClose(js);
+                break;
+            }
         }
         SCJbClose(js);
 

src/detect-prefilter.c

@@ -29,6 +29,7 @@
 #include "detect.h"
 #include "detect-parse.h"
 #include "detect-content.h"
+#include "detect-app-layer-protocol.h"
 #include "detect-engine-mpm.h"
 #include "detect-prefilter.h"
 #include "util-debug.h"
@@ -107,6 +108,16 @@ static int DetectPrefilterSetup (DetectEngineCtx *de_ctx, Signature *s, const ch
             SCReturnInt(-1);
         }
 
+        /* A multi-value app-layer-protocol keyword stores its values in a
+         * bitmask that the single-valued prefilter bucket key can't carry, so
+         * forcing prefilter would bucket the rule under ALPROTO_UNKNOWN and
+         * silently never match. */
+        if (sm->type == DETECT_APP_LAYER_PROTOCOL &&
+                ((const DetectAppLayerProtocolData *)sm->ctx)->is_list) {
+            SCLogError("prefilter is not supported for multi-value app-layer-protocol");
+            SCReturnInt(-1);
+        }
+
         /* make sure setup function runs for this type. */
         de_ctx->sm_types_prefilter[sm->type] = true;
     }