doc: document xor transform variable key and offset syntax

Add user-facing documentation for the xor transform's 'extract <nbytes>
<offset>' syntax and offset parameter in rules/transforms.rst.

Add a devguide section (devguide/extending/detect/transforms.rst)
covering the detection pipeline execution order, how transforms run
before detection-time keywords, and how xor reads key bytes directly
from the raw buffer at a specified position and width. Explains how
the engine deduplicates inspection buffers for rules with variable
keys at different buffer positions.

Issue: 8671

doc/userguide/devguide/extending/detect/index.rst

@@ -1,6 +1,11 @@
 Detection
 #########
 
+.. toctree::
+   :maxdepth: 1
+
+   transforms.rst
+
 Rate Filter Callback
 ********************
 

doc/userguide/devguide/extending/detect/transforms.rst

@@ -0,0 +1,39 @@
+Transforms
+**********
+
+Overview
+========
+
+Transforms modify the contents of an inspection buffer before content
+keywords inspect it. They are applied in the order they appear in the rule,
+with each transform's output becoming the next transform's input.
+
+Execution Order
+===============
+
+Transforms run at two points in the detection pipeline:
+
+1. **Prefilter (MPM):** when the buffer is set up for the multi-pattern
+   matcher. The MPM searches the *transformed* buffer for fast-pattern
+   content.
+2. **Full inspection:** when the rule's keywords are evaluated against the
+   (transformed) buffer.
+
+In both cases, transforms execute before detection-time keywords such as
+``byte_extract`` and ``byte_math``. The transformed buffer is what all
+subsequent keywords — including ``content`` — operate on.
+
+Transform Identity
+==================
+
+The engine deduplicates inspection buffers: rules that use the same buffer
+keyword with the same transform configuration share one pre-computed buffer.
+Equivalence is determined at rule load time by comparing an identity value
+that each transform instance produces from its configuration. Instances with
+identical identity share a buffer; instances with different identity (or where
+a transform produces no identity) get independent buffers.
+
+For ``xor`` with a static key the key bytes are the identity. For a variable
+key the identity is the key's offset and byte count — so two rules reading
+their key from the same buffer location share a buffer, while two rules with
+different key locations each get their own correctly transformed buffer.

doc/userguide/rules/transforms.rst

@@ -192,18 +192,42 @@ xor
 
 Takes the buffer, applies xor decoding.
 
-.. note:: this transform requires a mandatory option which is the hexadecimal encoded xor key.
+The key can be a hexadecimal string or a variable specified inline using
+``extract <nbytes> <offset>``. When a variable key is used, the engine reads
+``<nbytes>`` bytes starting at ``<offset>`` in the raw inspection buffer
+at transform time.
 
+An optional ``offset`` parameter specifies the byte position in the buffer
+where XOR decoding starts. Bytes before this position are left as-is.
+For example, if the first byte of the buffer is the XOR key, use
+``offset 1`` so decoding begins after that key byte.
+
+Syntax::
+
+    xor:"<hex_key>"
+    xor:extract <nbytes> <offset>
+    xor:offset <N>,"<hex_key>"
+    xor:offset <N>,extract <nbytes> <offset>
 
 Quotes around a hex key are optional; ``xor:0d0ac8ff`` and ``xor:"0d0ac8ff"``
 are equivalent.
 
-This example alerts if ``http.uri`` contains ``password=`` xored with 4-bytes key ``0d0ac8ff``
+This example alerts if ``http.uri`` contains ``password=`` xored with 4-bytes key ``0d0ac8ff``:
+
 .. container:: example-rule
 
     alert http any any -> any any (msg:"HTTP with xor"; http.uri; \
         xor:"0d0ac8ff"; content:"password="; sid:1;)
 
+This example reads a 1-byte XOR key from offset 0 of the request body,
+then decodes the buffer starting at offset 1 (skipping the key byte) and
+matches ``infected`` in the decoded data:
+
+.. container:: example-rule
+
+    alert http any any -> any any (msg:"XOR with variable key"; \
+        http.request_body; xor:offset 1,extract 1 0; content:"infected"; sid:2;)
+
 header_lowercase
 ----------------