From 26b9c1d78d32a67ace92d7fc8119576adb7bf41d Mon Sep 17 00:00:00 2001 From: Victor Julien Date: Sun, 27 Sep 2026 23:56:52 +0000 Subject: [PATCH] doc: update the SSH docs for the reworked state machine State table and failure/completion semantics for the per-direction scale (banner/kex/session, done never reported), the removed banner_wait_eol state and legacy hook names, the new eve ssh error/state fields, and the transactions/lua/upgrade examples - all updated to match. Ticket: #9046. --- .../extending/app-layer/transactions.rst | 12 +- doc/userguide/firewall/index.rst | 1 + .../firewall/ssh-state-migration.rst | 138 +++++++++++++++++ doc/userguide/lua/libs/ssh.rst | 2 +- doc/userguide/output/eve/eve-json-format.rst | 6 + doc/userguide/rules/app-layer.rst | 6 +- doc/userguide/rules/ssh-keywords.rst | 142 ++++++++++++++++-- doc/userguide/upgrade.rst | 25 +++ 8 files changed, 315 insertions(+), 17 deletions(-) create mode 100644 doc/userguide/firewall/ssh-state-migration.rst diff --git a/doc/userguide/devguide/extending/app-layer/transactions.rst b/doc/userguide/devguide/extending/app-layer/transactions.rst index 0593652d10..a1bf7c8030 100644 --- a/doc/userguide/devguide/extending/app-layer/transactions.rst +++ b/doc/userguide/devguide/extending/app-layer/transactions.rst @@ -148,10 +148,10 @@ Code snippet from: rust/src/ssh/ssh.rs: .. code-block:: rust pub enum SSHConnectionState { - SshStateInProgress = 0, - SshStateBannerWaitEol = 1, - SshStateBannerDone = 2, - SshStateFinished = 3, + SshStateBanner = 0, + SshStateKex = 1, + SshStateSession = 2, + SshStateDone = 3, } From src/app-layer-ftp.h: @@ -217,8 +217,8 @@ rust/src/ssh/ssh.rs: .. code-block:: rust - tx_comp_st_ts: SSHConnectionState::SshStateFinished as i32, - tx_comp_st_tc: SSHConnectionState::SshStateFinished as i32, + tx_comp_st_ts: SSHConnectionState::SshStateDone as i32, + tx_comp_st_tc: SSHConnectionState::SshStateDone as i32, In C, callback usage would be as follows: diff --git a/doc/userguide/firewall/index.rst b/doc/userguide/firewall/index.rst index a225e8c50d..6b0d097c94 100644 --- a/doc/userguide/firewall/index.rst +++ b/doc/userguide/firewall/index.rst @@ -7,3 +7,4 @@ Firewall Mode firewall-example firewall-stats tls-state-migration + ssh-state-migration diff --git a/doc/userguide/firewall/ssh-state-migration.rst b/doc/userguide/firewall/ssh-state-migration.rst new file mode 100644 index 0000000000..5425a920cb --- /dev/null +++ b/doc/userguide/firewall/ssh-state-migration.rst @@ -0,0 +1,138 @@ +Migrating from the old SSH state names +====================================== + +In this release the SSH parser was rebuilt on per-direction active +phases: a state names the part of the handshake the state progression +of one direction is working through - it is entered when that unit +starts being parsed and left when the unit completes. The two +directions progress independently. The state names used in rules +(``accept:hook ssh:``, ``alert ssh:``) and in the +``firewall.policies.app.ssh`` config keys were renamed accordingly, +and the removed ``banner_wait_eol`` state has no replacement: a long +banner line now keeps its direction in the banner state. Rules using +an old state name fail to load. + +Old and new names +----------------- + +.. list-table:: + :header-rows: 1 + + * - old state (milestone) + - new state (phase) + - note + * - ``request_in_progress`` + - ``request_banner`` + - the banner (version) phase: the first packets of the direction + * - ``request_banner_wait_eol`` + - ``request_banner`` + - the state is gone; a banner line that does not end within 256 + bytes keeps the direction in ``request_banner`` until the + end-of-line arrives (the ``ssh.long_banner`` event) + * - ``request_banner_done`` + - ``request_kex`` + - entered when the banner line completes; ``ssh.proto``, + ``ssh.software`` and the hassh buffers are registered here + * - ``request_finished`` + - ``request_session`` + - a ``new keys`` record was seen in this direction; the old top + state required both directions to reach it + * - ``response_in_progress`` + - ``response_banner`` + - the server-side banner phase + * - ``response_banner_wait_eol`` + - ``response_banner`` + - as ``request_banner_wait_eol`` above, in the other direction + * - ``response_banner_done`` + - ``response_kex`` + - as ``request_banner_done`` above, in the other direction + * - ``response_finished`` + - ``response_session`` + - as ``request_finished`` above, in the other direction + +``request_done`` / ``response_done`` is the registered completion +state. No direction ever reports it: a successful direction tops out +at ``session``, and a failed direction is frozen in the state the +failure occurred in. The completion state is neither hookable nor a +firewall policy key, so ``ssh:request_complete`` (and the +``request-complete`` policy key) matches nothing for a well-formed +flow - a session-admit rule like that no longer fires. Only a +disrupted flow (depth truncation or async reassembly) reports the +completion state, because the engine then returns it as the +transaction progress instead of querying the parser. Failures are +reported through the ``ssh.invalid_banner`` / ``ssh.invalid_record`` +events instead. + +Semantics that changed with the rename +-------------------------------------- + +- The scale is per direction: each direction reports its own raw + phase, where the old progress accessor clamped both directions to + one shared value. A rule hooking the old top state therefore meant + "the flow completed", not "this direction completed the handshake". + ``session`` is now set on the packet's own direction only, when that + direction sees a ``new keys`` record; the two directions are + independent. +- A parse failure freezes the failing direction in the state it + occurred in: the direction neither advances nor regresses, and the + frozen state is the last progress it reports. Nothing after the + failing packet is parsed or delivered, mid-flow or at the flow-end + flush. The failure reason is available through the + ``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer events and + the per-direction ``error`` / ``state`` fields of the eve ``ssh`` + object (``invalid_banner`` / ``invalid_record`` and the state the + failure occurred in; a failed direction is logged even when it + parsed no banner). A keyword registered at the failed state gets no + eof run, so its flow-end evaluation stays non-terminal. +- A banner line of 256 bytes or more without an end-of-line keeps the + direction in the banner state. The line is consumed greedily until + the end-of-line arrives, and the direction then advances to ``kex``. + The condition is observable through the ``ssh.long_banner`` event + (the banner data is published when the long line first parses). If + the line does not parse as a banner when it completes, the direction + fails and is frozen in ``banner``. +- The firewall applies one decision per state, as for TLS: after an + ``accept:hook`` rule matches at a state, further rules at the same + state are not evaluated, and the verdict applies while the + transaction stays at the hooked state. Remapping two old rules onto + one new state therefore leaves the second one dead - keep at most + one rule per state. The old milestones collapsed both directions + onto one progress value; a rule now hooks one direction's phase. +- The state names are also exposed by the ``app-layer-state`` keyword + and the ``ts_progress`` / ``tc_progress`` output fields. The name + table changed with this rename: rules using ``app-layer-state`` + against the old SSH state names no longer match. ``done`` is out of + both name tables, so ``app-layer-state:>request_session`` matches + only a direction disrupted past ``session``; a successful direction + stops at ``request_session`` and a failed one is frozen where it + failed. +- The generic hook lists for the completion state run only at a + disrupted flow end, so Lua hooks on SSH completion no longer fire + for well-formed flows; before this change the accessor reported the + terminal state once both directions had sent NewKeys. + +Migration steps +--------------- + +1. **Rules.** Search the ruleset for the old names + (``request_in_progress``, ``request_banner_wait_eol``, + ``request_banner_done``, ``request_finished``, and the + ``response_`` equivalents) and update them to the new names. A rule + with an unknown state fails to load with a ``does not support + hook`` error and is **not enforced**, so check the load output after + updating. A rule that hooked ``request_banner_wait_eol`` maps to + ``request_banner``; there is no separate long-line state. +2. **Firewall config keys.** Config keys use the **dash** form + (``firewall.policies.app.ssh.request-banner-done``); rules use the + underscore form (``ssh:request_banner_done``). Update the keys, or + the state falls back to the implicit default policy (drop in + firewall mode). Review ``request-complete`` policies: the + completion state is never reported on a clean flow. +3. **``app-layer-state`` and hook users.** Rules and Lua hooks keyed on + the old state names no longer match, and a hook on the completion + state fires only for a disrupted flow. Use the ``ssh.invalid_*`` + events to observe parse failures. +4. **Monitoring.** The eve ``ssh`` object now carries the per-direction + ``error`` / ``state`` fields for a parse failure; a failed + direction is logged even when it parsed no banner. The + ``ts_progress`` / ``tc_progress`` alert fields carry the new names. diff --git a/doc/userguide/lua/libs/ssh.rst b/doc/userguide/lua/libs/ssh.rst index def89024a7..93ba17890c 100644 --- a/doc/userguide/lua/libs/ssh.rst +++ b/doc/userguide/lua/libs/ssh.rst @@ -22,7 +22,7 @@ For example: .. container:: example-rule - alert ssh::example-rule-emphasis:`response_banner_done` any any -> any any (sid: 1;) + alert ssh::example-rule-emphasis:`response_kex` any any -> any any (sid: 1;) Setup ^^^^^ diff --git a/doc/userguide/output/eve/eve-json-format.rst b/doc/userguide/output/eve/eve-json-format.rst index 97f18ce775..0f36559bd7 100644 --- a/doc/userguide/output/eve/eve-json-format.rst +++ b/doc/userguide/output/eve/eve-json-format.rst @@ -1844,6 +1844,12 @@ Fields * "software_version": The software version used by end user * "hassh.hash": MD5 of hassh algorithms of client or server * "hassh.string": hassh algorithms of client or server +* "error": (per direction) unrecoverable parse failure: "invalid_banner" or + "invalid_record". Present only when the direction failed; a failed + direction is logged even when it parsed no banner, so the error field + may be the only content of the direction object. +* "state": (per direction) the state the failure occurred in (banner, + kex or session). Present only alongside the error field. Hassh must be enabled in the Suricata config file (set 'app-layer.protocols.ssh.hassh' to 'yes'). diff --git a/doc/userguide/rules/app-layer.rst b/doc/userguide/rules/app-layer.rst index e2aedf0b77..5d57471152 100644 --- a/doc/userguide/rules/app-layer.rst +++ b/doc/userguide/rules/app-layer.rst @@ -196,10 +196,14 @@ applayer_proto_detection_skipped Protocol detection was skipped because of :ref:`proto-detect-bail-out`. +.. _app-layer-state: + app-layer-state --------------- -Match on the detected app-layer protocol transaction state. +Match on the detected app-layer protocol transaction state. ``<`` and +``>`` are strict comparisons against the current per-direction +transaction progress. Syntax:: diff --git a/doc/userguide/rules/ssh-keywords.rst b/doc/userguide/rules/ssh-keywords.rst index 71e01ed41b..7b4edc649d 100644 --- a/doc/userguide/rules/ssh-keywords.rst +++ b/doc/userguide/rules/ssh-keywords.rst @@ -10,21 +10,145 @@ connections. Hooks ----- -The available hooks for SSH are: +The SSH parser exposes a per-direction state machine. A rule that +hooks one of the states below is evaluated once the transaction's +progress in that direction has reached the state; for firewall rules +the verdict applies while the transaction stays at the hooked state +and stops applying once it advances past it (see :ref:`app-layer-state`). Request (``to_server``) side: -* ``request_in_progress`` -* ``request_banner_wait_eol`` -* ``request_banner_done`` -* ``request_finished`` ++---------------------------+-------------------------------------------------+ +| State | Meaning | ++===========================+=================================================+ +| ``request_banner`` | Version exchange: the banner (version) phase; | +| | the first packets of the direction. A banner | +| | line of 256 bytes or more without an | +| | end-of-line keeps the direction here: the line | +| | is consumed greedily until the end-of-line, and | +| | the condition is observable through the | +| | ``ssh.long_banner`` event (the banner data is | +| | published when the long line first parses). | ++---------------------------+-------------------------------------------------+ +| ``request_kex`` | A valid banner line was parsed; key exchange. | +| | ``ssh.proto``, ``ssh.software`` and the hassh | +| | buffers are registered at this state. A long | +| | banner can publish them while the line is | +| | still open; if the line never completes, the | +| | values are not inspected at a clean flow end | +| | (the direction stays below the registered | +| | state); a flow that ends disrupted (depth | +| | truncation or async reassembly) reads as past | +| | this state, and the values are inspected there | ++---------------------------+-------------------------------------------------+ +| ``request_session`` | A ``new keys`` record was seen in this | +| | direction: the session is established for that | +| | direction; the two directions are tracked | +| | independently (a parse failure in either ends | +| | the flow's app-layer parsing, see Completion | +| | and failure). ``new keys`` is a transition | +| | rather than a phase with its own data: match | +| | the record itself with a frame rule (message | +| | code 21, see the example below). | ++---------------------------+-------------------------------------------------+ Response (``to_client``) side: -* ``response_in_progress`` -* ``response_banner_wait_eol`` -* ``response_banner_done`` -* ``response_finished`` ++---------------------------+-------------------------------------------------+ +| State | Meaning | ++===========================+=================================================+ +| ``response_banner`` | Server banner (version) phase: the same | +| | behavior as ``request_banner`` above, in the | +| | server-to-client direction. | ++---------------------------+-------------------------------------------------+ +| ``response_kex`` | Server key exchange: same as | +| | ``request_kex`` — the direction's | +| | ``ssh.proto`` / ``ssh.software`` and hassh | +| | buffers are registered at this state. | ++---------------------------+-------------------------------------------------+ +| ``response_session`` | A ``new keys`` record was seen in the | +| | server-to-client direction: same as | +| | ``request_session``. | ++---------------------------+-------------------------------------------------+ + +Overly long banner lines +~~~~~~~~~~~~~~~~~~~~~~~~ + +A banner line of 256 bytes or more without an end-of-line keeps the +direction in the banner state. The line is consumed greedily until +the end-of-line arrives, and the direction then advances to +``kex``. The condition is observable through the ``ssh.long_banner`` +app-layer event (the banner data is published when the long line +first parses). If the line does not parse as a banner when it +completes, the direction fails and is frozen in ``banner``. + +Completion and failure +~~~~~~~~~~~~~~~~~~~~~~ + +``request_done`` / ``response_done`` is the registered completion +state. No direction ever reports it: a successful direction tops out +at ``session``, and a direction that hits an unrecoverable error — +an invalid banner or an invalid record (header or payload) — is +frozen in the state the error occurred in. It is the last progress +the direction reports, and the reason is available through +the ``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer +events and the per-direction ``error`` / ``state`` fields of the +eve ``ssh`` object (``invalid_banner`` / ``invalid_record`` +and the state the failure occurred in; a failed flow is logged even when +no banner was parsed, so the error field may be its only content). +A parse failure puts the flow into an error state: nothing after +the failing packet is parsed or delivered, mid-flow or at the +flow-end flush, and the frozen state is what the progress +accessor - and thus the flow-end eve object - reports. The +``ssh.invalid_banner`` / ``ssh.invalid_record`` events are +evaluated as soon as the tx is next inspected - the failing +packet itself unless the error policy skips it - at the latest +the flow-end flush. +A keyword registered at the failed state gets no eof run (eof +requires progress above the registered state), so its flow-end +evaluation stays non-terminal; states before it were already +terminalized when the direction left them. The ``ssh.long_banner`` and +``ssh.long_kex_record`` events are non-fatal: the direction +continues (a long banner keeps it in ``banner`` until the line +completes; an oversized key-exchange record keeps it in ``kex`` until +the record has been buffered in full). + +Consequences worth knowing: + +* ``ssh:request_complete`` (and the firewall policy key + ``request-complete``) is bound to the completion state, which no + direction reports: the hook matches nothing for well-formed + flows. A disrupted flow (depth truncation or async reassembly) + matches, because on a disrupted flow the engine reports the + completion state as the transaction progress instead of + querying the parser. Before this change it matched a flow whose + session was established. Review existing + ``accept:* ssh:request_complete`` rules and ``request-complete`` + firewall policies: a session-admit rule like that no longer fires. + Failures are reported through the ``ssh.invalid_banner`` / + ``ssh.invalid_record`` events instead. +* Rules cannot reference ``done`` (``request_done`` / + ``response_done`` do not load): it is out of both name tables. + The generic hook lists for the completion state are registered + under the built-in completion names + (``ssh:request_complete:generic`` / + ``ssh:response_complete:generic``); the state-name-based lists + are not registered for the completion id (the id-to-name lookup + returns null). They run only at a disrupted flow end, where the + engine reports the completion state as the transaction progress + instead of querying the parser - so Lua hooks on ssh completion + no longer fire for well-formed flows; before this change the + accessor reported the terminal state once both directions had + sent NewKeys. +* ``app-layer-state:>request_session`` matches only directions that + were *disrupted past session (e.g. by depth truncation or async + reassembly)* — a successful request direction stops at + ``request_session``, and a failed one is frozen at the state it + failed in (use the ``ssh.invalid_*`` events for that). + +For firewall rules and ``app-layer-state`` users, +:doc:`../firewall/ssh-state-migration` maps the old hook names to the +new phases and describes the behaviour changes. Frames ------ diff --git a/doc/userguide/upgrade.rst b/doc/userguide/upgrade.rst index a6426b73cc..1673d0db8a 100644 --- a/doc/userguide/upgrade.rst +++ b/doc/userguide/upgrade.rst @@ -92,6 +92,16 @@ Major Changes Logging Changes ~~~~~~~~~~~~~~~ +- The ssh eve record now carries the per-direction fields + ``ssh.client.error`` / ``ssh.client.state`` and + ``ssh.server.error`` / ``ssh.server.state`` (``invalid_banner`` or + ``invalid_record``, and the state the failure occurred in - the + state field is present only alongside the error) for flows that + hit an unrecoverable ssh parse error; a failed direction is + logged even when it parsed no banner. The object is logged at + the failing delivery, carrying the error and the state; + successful flows are logged at the next EOF-flush delivery. + - The format of IKEv1 proposal attributes has been changed to handle duplicate attribute types. See :ref:`IKE logging changes <9.0-ike-logging-changes>` @@ -106,6 +116,7 @@ Logging Changes ``stats.app_layer.*.ftp-data`` becomes ``stats.app_layer.*.ftp_data``, and same for bittorrent_dht + - MQTT user properties are now logged as a new object called ``user_properties`` as an array of key-value pairs like ``[{"key":"mykey", "value":"myvalue"}]`` under ``properties`` object, instead of as ``{key: value}`` pairs as a part @@ -131,6 +142,20 @@ Keyword Changes some rules will have less false negatives, and some rules will trigger once per transaction instead of twice (one time for each direction) +- The SSH app-layer state hooks have been reworked and renamed: + the hooks are now ``request_banner``, ``request_kex`` and + ``request_session`` (and the ``response_`` equivalents); rules + using the old names fail to load. The completion state is never + reported, so ``ssh:request_complete`` and the + ``request-complete`` policy key match nothing for ssh except + disrupted flows (depth truncation or async reassembly) - + session-admit rules or policies no longer fire, review them + before upgrading. Failed key exchanges are reported through the + ``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer events + and the new ``error`` / ``state`` fields of the ssh eve record. + See :doc:`rules/ssh-keywords` for the full state and failure + semantics. + Other Changes ~~~~~~~~~~~~~ - ``engine-analysis`` output has been changed for keywords that now use the generic integers framework