doc: update the SSH docs for the reworked state machine

State table and failure/completion semantics for the
per-direction scale (banner/kex/session, done never reported),
the removed banner_wait_eol state and legacy hook names, the new
eve ssh error/state fields, and the transactions/lua/upgrade
examples - all updated to match.

Ticket: #9046.
pull/16368/head
Victor Julien 4 weeks ago
parent 166841e93f
commit 26b9c1d78d

@ -148,10 +148,10 @@ Code snippet from: rust/src/ssh/ssh.rs:
.. code-block:: rust
pub enum SSHConnectionState {
SshStateInProgress = 0,
SshStateBannerWaitEol = 1,
SshStateBannerDone = 2,
SshStateFinished = 3,
SshStateBanner = 0,
SshStateKex = 1,
SshStateSession = 2,
SshStateDone = 3,
}
From src/app-layer-ftp.h:
@ -217,8 +217,8 @@ rust/src/ssh/ssh.rs:
.. code-block:: rust
tx_comp_st_ts: SSHConnectionState::SshStateFinished as i32,
tx_comp_st_tc: SSHConnectionState::SshStateFinished as i32,
tx_comp_st_ts: SSHConnectionState::SshStateDone as i32,
tx_comp_st_tc: SSHConnectionState::SshStateDone as i32,
In C, callback usage would be as follows:

@ -7,3 +7,4 @@ Firewall Mode
firewall-example
firewall-stats
tls-state-migration
ssh-state-migration

@ -0,0 +1,138 @@
Migrating from the old SSH state names
======================================
In this release the SSH parser was rebuilt on per-direction active
phases: a state names the part of the handshake the state progression
of one direction is working through - it is entered when that unit
starts being parsed and left when the unit completes. The two
directions progress independently. The state names used in rules
(``accept:hook ssh:<state>``, ``alert ssh:<state>``) and in the
``firewall.policies.app.ssh`` config keys were renamed accordingly,
and the removed ``banner_wait_eol`` state has no replacement: a long
banner line now keeps its direction in the banner state. Rules using
an old state name fail to load.
Old and new names
-----------------
.. list-table::
:header-rows: 1
* - old state (milestone)
- new state (phase)
- note
* - ``request_in_progress``
- ``request_banner``
- the banner (version) phase: the first packets of the direction
* - ``request_banner_wait_eol``
- ``request_banner``
- the state is gone; a banner line that does not end within 256
bytes keeps the direction in ``request_banner`` until the
end-of-line arrives (the ``ssh.long_banner`` event)
* - ``request_banner_done``
- ``request_kex``
- entered when the banner line completes; ``ssh.proto``,
``ssh.software`` and the hassh buffers are registered here
* - ``request_finished``
- ``request_session``
- a ``new keys`` record was seen in this direction; the old top
state required both directions to reach it
* - ``response_in_progress``
- ``response_banner``
- the server-side banner phase
* - ``response_banner_wait_eol``
- ``response_banner``
- as ``request_banner_wait_eol`` above, in the other direction
* - ``response_banner_done``
- ``response_kex``
- as ``request_banner_done`` above, in the other direction
* - ``response_finished``
- ``response_session``
- as ``request_finished`` above, in the other direction
``request_done`` / ``response_done`` is the registered completion
state. No direction ever reports it: a successful direction tops out
at ``session``, and a failed direction is frozen in the state the
failure occurred in. The completion state is neither hookable nor a
firewall policy key, so ``ssh:request_complete`` (and the
``request-complete`` policy key) matches nothing for a well-formed
flow - a session-admit rule like that no longer fires. Only a
disrupted flow (depth truncation or async reassembly) reports the
completion state, because the engine then returns it as the
transaction progress instead of querying the parser. Failures are
reported through the ``ssh.invalid_banner`` / ``ssh.invalid_record``
events instead.
Semantics that changed with the rename
--------------------------------------
- The scale is per direction: each direction reports its own raw
phase, where the old progress accessor clamped both directions to
one shared value. A rule hooking the old top state therefore meant
"the flow completed", not "this direction completed the handshake".
``session`` is now set on the packet's own direction only, when that
direction sees a ``new keys`` record; the two directions are
independent.
- A parse failure freezes the failing direction in the state it
occurred in: the direction neither advances nor regresses, and the
frozen state is the last progress it reports. Nothing after the
failing packet is parsed or delivered, mid-flow or at the flow-end
flush. The failure reason is available through the
``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer events and
the per-direction ``error`` / ``state`` fields of the eve ``ssh``
object (``invalid_banner`` / ``invalid_record`` and the state the
failure occurred in; a failed direction is logged even when it
parsed no banner). A keyword registered at the failed state gets no
eof run, so its flow-end evaluation stays non-terminal.
- A banner line of 256 bytes or more without an end-of-line keeps the
direction in the banner state. The line is consumed greedily until
the end-of-line arrives, and the direction then advances to ``kex``.
The condition is observable through the ``ssh.long_banner`` event
(the banner data is published when the long line first parses). If
the line does not parse as a banner when it completes, the direction
fails and is frozen in ``banner``.
- The firewall applies one decision per state, as for TLS: after an
``accept:hook`` rule matches at a state, further rules at the same
state are not evaluated, and the verdict applies while the
transaction stays at the hooked state. Remapping two old rules onto
one new state therefore leaves the second one dead - keep at most
one rule per state. The old milestones collapsed both directions
onto one progress value; a rule now hooks one direction's phase.
- The state names are also exposed by the ``app-layer-state`` keyword
and the ``ts_progress`` / ``tc_progress`` output fields. The name
table changed with this rename: rules using ``app-layer-state``
against the old SSH state names no longer match. ``done`` is out of
both name tables, so ``app-layer-state:>request_session`` matches
only a direction disrupted past ``session``; a successful direction
stops at ``request_session`` and a failed one is frozen where it
failed.
- The generic hook lists for the completion state run only at a
disrupted flow end, so Lua hooks on SSH completion no longer fire
for well-formed flows; before this change the accessor reported the
terminal state once both directions had sent NewKeys.
Migration steps
---------------
1. **Rules.** Search the ruleset for the old names
(``request_in_progress``, ``request_banner_wait_eol``,
``request_banner_done``, ``request_finished``, and the
``response_`` equivalents) and update them to the new names. A rule
with an unknown state fails to load with a ``does not support
hook`` error and is **not enforced**, so check the load output after
updating. A rule that hooked ``request_banner_wait_eol`` maps to
``request_banner``; there is no separate long-line state.
2. **Firewall config keys.** Config keys use the **dash** form
(``firewall.policies.app.ssh.request-banner-done``); rules use the
underscore form (``ssh:request_banner_done``). Update the keys, or
the state falls back to the implicit default policy (drop in
firewall mode). Review ``request-complete`` policies: the
completion state is never reported on a clean flow.
3. **``app-layer-state`` and hook users.** Rules and Lua hooks keyed on
the old state names no longer match, and a hook on the completion
state fires only for a disrupted flow. Use the ``ssh.invalid_*``
events to observe parse failures.
4. **Monitoring.** The eve ``ssh`` object now carries the per-direction
``error`` / ``state`` fields for a parse failure; a failed
direction is logged even when it parsed no banner. The
``ts_progress`` / ``tc_progress`` alert fields carry the new names.

@ -22,7 +22,7 @@ For example:
.. container:: example-rule
alert ssh::example-rule-emphasis:`response_banner_done` any any -> any any (sid: 1;)
alert ssh::example-rule-emphasis:`response_kex` any any -> any any (sid: 1;)
Setup
^^^^^

@ -1844,6 +1844,12 @@ Fields
* "software_version": The software version used by end user
* "hassh.hash": MD5 of hassh algorithms of client or server
* "hassh.string": hassh algorithms of client or server
* "error": (per direction) unrecoverable parse failure: "invalid_banner" or
"invalid_record". Present only when the direction failed; a failed
direction is logged even when it parsed no banner, so the error field
may be the only content of the direction object.
* "state": (per direction) the state the failure occurred in (banner,
kex or session). Present only alongside the error field.
Hassh must be enabled in the Suricata config file (set 'app-layer.protocols.ssh.hassh' to 'yes').

@ -196,10 +196,14 @@ applayer_proto_detection_skipped
Protocol detection was skipped because of :ref:`proto-detect-bail-out`.
.. _app-layer-state:
app-layer-state
---------------
Match on the detected app-layer protocol transaction state.
Match on the detected app-layer protocol transaction state. ``<`` and
``>`` are strict comparisons against the current per-direction
transaction progress.
Syntax::

@ -10,21 +10,145 @@ connections.
Hooks
-----
The available hooks for SSH are:
The SSH parser exposes a per-direction state machine. A rule that
hooks one of the states below is evaluated once the transaction's
progress in that direction has reached the state; for firewall rules
the verdict applies while the transaction stays at the hooked state
and stops applying once it advances past it (see :ref:`app-layer-state`).
Request (``to_server``) side:
* ``request_in_progress``
* ``request_banner_wait_eol``
* ``request_banner_done``
* ``request_finished``
+---------------------------+-------------------------------------------------+
| State | Meaning |
+===========================+=================================================+
| ``request_banner`` | Version exchange: the banner (version) phase; |
| | the first packets of the direction. A banner |
| | line of 256 bytes or more without an |
| | end-of-line keeps the direction here: the line |
| | is consumed greedily until the end-of-line, and |
| | the condition is observable through the |
| | ``ssh.long_banner`` event (the banner data is |
| | published when the long line first parses). |
+---------------------------+-------------------------------------------------+
| ``request_kex`` | A valid banner line was parsed; key exchange. |
| | ``ssh.proto``, ``ssh.software`` and the hassh |
| | buffers are registered at this state. A long |
| | banner can publish them while the line is |
| | still open; if the line never completes, the |
| | values are not inspected at a clean flow end |
| | (the direction stays below the registered |
| | state); a flow that ends disrupted (depth |
| | truncation or async reassembly) reads as past |
| | this state, and the values are inspected there |
+---------------------------+-------------------------------------------------+
| ``request_session`` | A ``new keys`` record was seen in this |
| | direction: the session is established for that |
| | direction; the two directions are tracked |
| | independently (a parse failure in either ends |
| | the flow's app-layer parsing, see Completion |
| | and failure). ``new keys`` is a transition |
| | rather than a phase with its own data: match |
| | the record itself with a frame rule (message |
| | code 21, see the example below). |
+---------------------------+-------------------------------------------------+
Response (``to_client``) side:
* ``response_in_progress``
* ``response_banner_wait_eol``
* ``response_banner_done``
* ``response_finished``
+---------------------------+-------------------------------------------------+
| State | Meaning |
+===========================+=================================================+
| ``response_banner`` | Server banner (version) phase: the same |
| | behavior as ``request_banner`` above, in the |
| | server-to-client direction. |
+---------------------------+-------------------------------------------------+
| ``response_kex`` | Server key exchange: same as |
| | ``request_kex`` — the direction's |
| | ``ssh.proto`` / ``ssh.software`` and hassh |
| | buffers are registered at this state. |
+---------------------------+-------------------------------------------------+
| ``response_session`` | A ``new keys`` record was seen in the |
| | server-to-client direction: same as |
| | ``request_session``. |
+---------------------------+-------------------------------------------------+
Overly long banner lines
~~~~~~~~~~~~~~~~~~~~~~~~
A banner line of 256 bytes or more without an end-of-line keeps the
direction in the banner state. The line is consumed greedily until
the end-of-line arrives, and the direction then advances to
``kex``. The condition is observable through the ``ssh.long_banner``
app-layer event (the banner data is published when the long line
first parses). If the line does not parse as a banner when it
completes, the direction fails and is frozen in ``banner``.
Completion and failure
~~~~~~~~~~~~~~~~~~~~~~
``request_done`` / ``response_done`` is the registered completion
state. No direction ever reports it: a successful direction tops out
at ``session``, and a direction that hits an unrecoverable error —
an invalid banner or an invalid record (header or payload) — is
frozen in the state the error occurred in. It is the last progress
the direction reports, and the reason is available through
the ``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer
events and the per-direction ``error`` / ``state`` fields of the
eve ``ssh`` object (``invalid_banner`` / ``invalid_record``
and the state the failure occurred in; a failed flow is logged even when
no banner was parsed, so the error field may be its only content).
A parse failure puts the flow into an error state: nothing after
the failing packet is parsed or delivered, mid-flow or at the
flow-end flush, and the frozen state is what the progress
accessor - and thus the flow-end eve object - reports. The
``ssh.invalid_banner`` / ``ssh.invalid_record`` events are
evaluated as soon as the tx is next inspected - the failing
packet itself unless the error policy skips it - at the latest
the flow-end flush.
A keyword registered at the failed state gets no eof run (eof
requires progress above the registered state), so its flow-end
evaluation stays non-terminal; states before it were already
terminalized when the direction left them. The ``ssh.long_banner`` and
``ssh.long_kex_record`` events are non-fatal: the direction
continues (a long banner keeps it in ``banner`` until the line
completes; an oversized key-exchange record keeps it in ``kex`` until
the record has been buffered in full).
Consequences worth knowing:
* ``ssh:request_complete`` (and the firewall policy key
``request-complete``) is bound to the completion state, which no
direction reports: the hook matches nothing for well-formed
flows. A disrupted flow (depth truncation or async reassembly)
matches, because on a disrupted flow the engine reports the
completion state as the transaction progress instead of
querying the parser. Before this change it matched a flow whose
session was established. Review existing
``accept:* ssh:request_complete`` rules and ``request-complete``
firewall policies: a session-admit rule like that no longer fires.
Failures are reported through the ``ssh.invalid_banner`` /
``ssh.invalid_record`` events instead.
* Rules cannot reference ``done`` (``request_done`` /
``response_done`` do not load): it is out of both name tables.
The generic hook lists for the completion state are registered
under the built-in completion names
(``ssh:request_complete:generic`` /
``ssh:response_complete:generic``); the state-name-based lists
are not registered for the completion id (the id-to-name lookup
returns null). They run only at a disrupted flow end, where the
engine reports the completion state as the transaction progress
instead of querying the parser - so Lua hooks on ssh completion
no longer fire for well-formed flows; before this change the
accessor reported the terminal state once both directions had
sent NewKeys.
* ``app-layer-state:>request_session`` matches only directions that
were *disrupted past session (e.g. by depth truncation or async
reassembly)* — a successful request direction stops at
``request_session``, and a failed one is frozen at the state it
failed in (use the ``ssh.invalid_*`` events for that).
For firewall rules and ``app-layer-state`` users,
:doc:`../firewall/ssh-state-migration` maps the old hook names to the
new phases and describes the behaviour changes.
Frames
------

@ -92,6 +92,16 @@ Major Changes
Logging Changes
~~~~~~~~~~~~~~~
- The ssh eve record now carries the per-direction fields
``ssh.client.error`` / ``ssh.client.state`` and
``ssh.server.error`` / ``ssh.server.state`` (``invalid_banner`` or
``invalid_record``, and the state the failure occurred in - the
state field is present only alongside the error) for flows that
hit an unrecoverable ssh parse error; a failed direction is
logged even when it parsed no banner. The object is logged at
the failing delivery, carrying the error and the state;
successful flows are logged at the next EOF-flush delivery.
- The format of IKEv1 proposal attributes has been changed to handle
duplicate attribute types. See :ref:`IKE logging changes
<9.0-ike-logging-changes>`
@ -106,6 +116,7 @@ Logging Changes
``stats.app_layer.*.ftp-data`` becomes ``stats.app_layer.*.ftp_data``,
and same for bittorrent_dht
- MQTT user properties are now logged as a new object called ``user_properties``
as an array of key-value pairs like ``[{"key":"mykey", "value":"myvalue"}]``
under ``properties`` object, instead of as ``{key: value}`` pairs as a part
@ -131,6 +142,20 @@ Keyword Changes
some rules will have less false negatives, and some rules will trigger once per transaction
instead of twice (one time for each direction)
- The SSH app-layer state hooks have been reworked and renamed:
the hooks are now ``request_banner``, ``request_kex`` and
``request_session`` (and the ``response_`` equivalents); rules
using the old names fail to load. The completion state is never
reported, so ``ssh:request_complete`` and the
``request-complete`` policy key match nothing for ssh except
disrupted flows (depth truncation or async reassembly) -
session-admit rules or policies no longer fire, review them
before upgrading. Failed key exchanges are reported through the
``ssh.invalid_banner`` / ``ssh.invalid_record`` app-layer events
and the new ``error`` / ``state`` fields of the ssh eve record.
See :doc:`rules/ssh-keywords` for the full state and failure
semantics.
Other Changes
~~~~~~~~~~~~~
- ``engine-analysis`` output has been changed for keywords that now use the generic integers framework

Loading…
Cancel
Save