doc/firewall: start of documentation

doc/userguide/Makefile.am

@@ -10,6 +10,7 @@ EXTRA_DIST = \
 	configuration \
 	devguide \
 	file-extraction \
+	firewall \
 	generate-evedoc.sh \
 	index.rst \
 	upgrade \

doc/userguide/firewall/firewall-design.rst

@@ -0,0 +1,188 @@
+Firewall Mode Design
+====================
+
+.. note:: In Suricata 8 the firewall mode is experimental and subject to change.
+
+The firewall mode in Suricata allows the use of a ruleset that has different
+properties than the default "threat detection" rulesets:
+
+1. default policy is ``drop``, meaning a firewall ruleset needs to specify what
+   is allowed
+2. firewall rules are loaded from separate files
+3. firewall rules use a new action ``accept``
+4. firewall rules are required to use explicit action scopes and rule hooks (see below)
+5. evaluation order is as rules are in the file(s), per protocol state
+
+Concepts
+--------
+
+* ``table`` collection of rules with different properties: ``packet:filter``, ``packet:td``,
+  ``app:<proto>:<state>``, ``app:td``. These are built-in. No custom tables can be created.
+* ``state`` controls a specific protocol state at which a rule is evaluated. Examples are
+  ``tcp.flow_start`` or ``tls.client_body_done``.
+
+The tables for the application layer are per app layer protocol and per protocol state.
+For example, ``app:http:request_line``.
+
+
+Actions and Action Scopes
+-------------------------
+
+Firewall rules require action scopes to be explicitly specified.
+
+accept
+~~~~~~
+
+``accept`` is used to issue an accept verdict to the packet, flow or hook.
+
+* ``packet`` accept this packet
+* ``flow`` accept the rest of the packets in this flow
+* ``hook`` accept rules for the current hook/state, evaluate the next tables
+* ``tx`` accept rules for the current transaction, evaluate the next tables
+
+The ``accept`` action is only available in firewall rules.
+
+.. note:: some protocol implementations like ``dns`` use a transaction per direction.
+   For those ``accept:tx`` will only accept packets that are part of that direction.
+
+drop
+~~~~
+
+``drop`` is used to drop either the packet or the flow
+
+* ``packet`` drop this packet directly, don't eval any further rules
+* ``flow`` drop this packet as with ``packet`` and drop all future packets in this flow 
+
+.. note:: the action ``pass`` is not available in firewall rules due to ambiguity around
+   the existing meaning for threat detection rules.
+
+
+Explicit rule hook (states)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the regular IDS/IPS rules the engine infers from the rule's matching logic where the
+rule should be "hooked" into the engine. While this works well for these types of rules,
+it does lead to many edge cases that are not acceptable in a firewall ruleset. For this
+reason in the firewall rules the hook needs to be explicitly set.
+
+This is done in the protocol field of the rule. Where in threat detection a rule might look like::
+
+    alert http ... http.uri; ...
+
+In the firewall case it will be::
+
+    accept:hook http1:request_line ... http.uri; ...
+
+The application layer states / hooks are defined per protocol. Each of the hooks has its own
+default-``drop`` policy, so a ruleset needs an ``accept`` rule for each of the states to allow
+the traffic to flow through.
+
+general
+^^^^^^^
+
+Each protocol has at least the default states.
+
+Request (``to_server``) side:
+
+* ``request_started``
+* ``request_complete``
+
+Response (``to_client``) side:
+
+* ``response_started``
+* ``response_complete``
+
+http
+^^^^
+
+For the HTTP protocol there are a number of states to hook into. These apply to HTTP 0.9, 1.0
+and 1.1. HTTP/2 uses its own state machine.
+
+Available states:
+
+Request (``to_server``) side:
+
+* ``request_started``
+* ``request_line``
+* ``request_headers``
+* ``request_body``
+* ``request_trailer``
+* ``request_complete``
+
+Response (``to_client``) side:
+
+* ``response_started``
+* ``response_line``
+* ``response_headers``
+* ``response_body``
+* ``response_trailer``
+* ``response_complete``
+
+tls
+^^^
+
+Available states:
+
+Request (``to_server``) side:
+
+* ``client_in_progress``
+* ``client_hello_done``
+* ``client_cert_done``
+* ``client_handshake_done``
+* ``client_finished``
+
+Response (``to_client``) side:
+
+* ``server_in_progress``
+* ``server_hello``
+* ``server_cert_done``
+* ``server_hello_done``
+* ``server_handshake_done``
+* ``server_finished``
+
+Firewall pipeline
+~~~~~~~~~~~~~~~~~
+
+The firewall pipeline works in the detection engine, and is invoked after packet decoding, flow
+update, stream tracking and reassembly and app-layer parsing are all done in the context of a
+single packet.
+
+For each packet rules in the first firewall hook ``packet:filter`` are then evaluated. Assuming
+the verdict of this hook is ``accept:hook``, the next hook is evaluated: ``packet:td`` (packet
+threat detection). In this hook the IDS/IPS rules are evaluated. Rule actions here are not
+immediate, as they can still be modified by alert postprocessing like rate_filter, thresholding, etc.
+
+The default ``drop`` for the ``packet:filter`` table is ``drop:packet``. Thus the ``drop`` is
+only applied to the current packet.
+
+If the packet has been marked internally as a packet with an application layer update, then the
+next table is ``app:*:*``.
+
+In ``app:*:*`` the per application layer states are all evaluated at least once. At each of
+these states an ``accept:hook`` is required to progress to the next state. When all available states
+have been accepted, the pipeline moves to the final table ``app:td`` (application layer threat
+detection). A ``drop`` in the ``app_filter`` table is immediate, however and ``accept`` is
+conditional on the verdict of the ``app:td`` table.
+
+The default ``drop`` in one of the ``app:*:*`` tables is a ``drop:flow``. This means that the
+current packet as well as all future packets from that flow are dropped.
+
+In ``app:td`` the IDS/IPS rules for the application layer are evaluated. ``drop`` actions in this
+table are queued in the alert queue.
+
+When all tables have been evaluated, the alert finalize process orders threat detection alerts
+by ``action-order`` logic. It can then apply a ``drop`` or default to ``accept``-ing.
+
+
+.. image:: fw-pipeline.png
+
+
+Pass rules with Firewall mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In IDS/IPS mode, a ``pass`` rule with app-layer matches will bypass the detection engine for the
+rest of the flow. In firewall mode, this bypass no longer happens in the same way, as ``pass`` rules
+do not affect firewall rules. So the detection engine is still invoked on packets of such a flow,
+but the ``packet_td`` and ``app_td`` tables are skipped.
+
+

doc/userguide/firewall/firewall-example.rst

@@ -0,0 +1,93 @@
+Firewall Ruleset Examples
+=========================
+
+.. note:: In Suricata 8 the firewall mode is experimental and subject to change.
+
+HTTP
+----
+
+In this example a simple HTTP ruleset will be shown. It will allow HTTP to flow
+as long as:
+
+- method is GET or POST
+- User-Agent is "curl"
+- Status code is 200.
+
+It starts by allowing the TCP port 80 traffic.
+
+::
+
+    accept:hook tcp:all any any <> any 80 (sid:10;)
+
+The stream tracking combined with the default exception policy handling will enforce
+a proper TCP handshake, etc.
+
+The HTTP rules need to ``accept`` each state::
+
+    # allow traffic before the request line is complete
+    accept:hook http1:request_started any any -> any any (sid:100;)
+    # allow GET
+    accept:hook http1:request_line any any -> any any ( \
+            http.method; content:"GET"; sid:101;)
+    # or allow POST
+    accept:hook http1:request_line any any -> any any ( \
+            http.method; content:"POST"; sid:102;)
+    # allow User-Agent curl
+    accept:hook http1:request_headers any any -> any any ( \
+            http.user_agent; content:"curl"; sid:103;)
+    # allow the body, if any
+    accept:hook http1:request_body any any -> any any (sid:104;)
+    # allow trailers, if any
+    accept:hook http1:request_trailer any any -> any any (sid:105;)
+    # allow completion
+    accept:hook http1:request_complete any any -> any any (sid:106;)
+
+    # allow traffic before the response line is complete
+    accept:hook http1:response_started any any -> any any (sid:200;)
+    # allow the 200 ok stat code.
+    accept:hook http1:response_line any any -> any any ( \
+            http.stat_code; content:"200"; sid:201;)
+    # allow all other states
+    accept:hook http1:response_headers any any -> any any (sid:202;)
+    accept:hook http1:response_body any any -> any any (sid:203;)
+    accept:hook http1:response_trailer any any -> any any (sid:204;)
+    accept:hook http1:response_complete any any -> any any (sid:205;)
+
+Each state needs an ``accept`` rule. Each state is evaluated at least once.
+
+TLS SNI with complex TCP rules
+------------------------------
+
+In this example the ``packet_filter`` rules will be more opinionated about the traffic::
+
+    # allow 3-way handshake
+    accept:hook tcp:all $HOME_NET any -> $EXTERNAL_NET 443 (flags:S; \
+            flow:not_established; flowbits:set,syn; sid:1;)
+    accept:hook tcp:all $EXTERNAL_NET 443 -> $HOME_NET any (flags:SA; \
+            flow:not_established; flowbits:isset,syn; flowbits:set,synack; sid:2;)
+    accept:hook tcp:all $HOME_NET any -> $EXTERNAL_NET 443 (flags:A; \
+            flow:not_established; flowbits:isset,synack;             \
+            flowbits:unset,syn; flowbits:unset,synack; sid:3;)
+    # allow established
+    accept:hook tcp:all $HOME_NET any <> $EXTERNAL_NET 443 (flow:established; sid:4;)
+
+Then on the TLS level this will be a TLS SNI firewall.
+
+Again all the states need to be accepted. Only in the ``client_hello_done`` state will
+there be additional constraints::
+
+    accept:hook tls:client_in_progress $HOME_NET any -> $EXTERNAL_NET any (sid:100;)
+    # allow the good sites
+    accept:hook tls:client_hello_done $HOME_NET any -> $EXTERNAL_NET any (tls.sni; \
+            pcre:"/^(suricata.io|oisf.net)$/; sid:101;)
+    accept:hook tls:client_cert_done $HOME_NET any -> $EXTERNAL_NET any (sid:102;)
+    accept:hook tls:client_handshake_done $HOME_NET any -> $EXTERNAL_NET any (sid:103;)
+    accept:hook tls:client_finished $HOME_NET any -> $EXTERNAL_NET any (sid:104;)
+
+    accept:hook tls:server_in_progress $EXTERNAL_NET any -> $HOME_NET any (sid:200;)
+    accept:hook tls:server_hello $EXTERNAL_NET any -> $HOME_NET any (sid:201;)
+    accept:hook tls:server_cert_done $EXTERNAL_NET any -> $HOME_NET any (sid:202;)
+    accept:hook tls:server_hello_done $EXTERNAL_NET any -> $HOME_NET any (sid:203;)
+    accept:hook tls:server_handshake_done $EXTERNAL_NET any -> $HOME_NET any (sid:204;)
+    accept:hook tls:server_finished $EXTERNAL_NET any -> $HOME_NET any (sid:205;)
+

doc/userguide/firewall/index.rst

@@ -0,0 +1,7 @@
+Firewall Mode
+=============
+
+.. toctree::
+
+   firewall-design
+   firewall-example

doc/userguide/index.rst

@@ -30,6 +30,7 @@ This is the documentation for Suricata |version|.
    capture-hardware/index.rst
    unix-socket.rst
    plugins/index.rst
+   firewall/index.rst
    3rd-party-integration/index.rst
    manpages/index.rst
    acknowledgements