DOH2 depends on HTTP/2, so it makes no sense to enable it separately.
It would also put the sub state handling in a weird state, as the DOH2
side reuses the registered HTTP/2 callbacks.
Split into 2 sub-states:
- stream, which has the "HTTP" requests and responses, including DOH2
- global, which has the settings and other global or control handling
Introduce a simpler progress tracking for the global sub state:
- HTTP2ProgGlobalStart and HTTP2ProgGlobalComplete.
The stream sub state uses the same state machine as before.
Ticket: #8386.
Support for per transaction sub states: different state machines per
transaction type.
Skip engines not belonging to our substate.
Store tx_type in DetectTransaction.
Each protocol supporting sub states will register states from 1 and up.
To support:
Ticket: #8386.
Allow registration of per substate progress name mappings.
Implement logic for getting sub-state names, id's.
Add transaction type and end of progress values to AppLayerTxData.
Introduce helpers to keep logic clean.
Ticket: 8725
So that multiple HTTP2 DATA frames with EndOfStream flag set,
do not make the buffer grow, while processing it each time,
resulting in quadratic complexity
Ticket: 8629
When we are in async-oneside mode, we see only one direction
of the traffic, and should not wait for the other direction
before cleaning up a transaction.
new_tx() now refuses to create a transaction when the list is already at
SMB_MAX_TX, returning None instead of a transaction. Every creation path --
the new_*_tx helpers and their callers across smb1/smb2/dcerpc/session/
files/ioctl -- propagates that, so no single input can create more than the
limit, including a compound SMB2 request that chains many PDUs in one
record. When the list is full the parser puts the flow into an error state
and stops processing it.
This replaces the previous force-completion of old transactions, which did
not reliably bound the list and could leave transactions unreclaimable on
asymmetric flows. The now-unused tx_index_completed bookkeeping is removed.
Issue: 8629
Ticket: 8694
Otherwise, a flow full of small compression bombs is too slow
to process.
When the threshold is reached, decompression is skipped for the
rest of the flow.
Ticket: 8649
Fully resets all the fields before tackling an ecapsulated message
to avoid evasion, due to the encoding of the upper file
leaking into the next one...
Body lengths were stored as u16, so a SIP body of 65536 bytes truncated
body_len to 0 and the RequestBody/ResponseBody frame was never created,
letting body content evade inspection. Widen the framing fields to u32.
Ticket #8582
dcerpc parser creates a new tx with id 0 and compensates for the 1 based
index handling throughout the code by overriding that value in a trait
implementation. Make this consistent with other applayer parsers.
Task 8720
Tx ID handling did not take the required + 1 into account.
From a report:
RDP can skip cleanup because its id convention does not match the
generic Rust iterator. The generic iterator in applayer.rs returns
tx.id() - 1, and cleanup trusts that id when calling StateTransactionFree
in app-layer-parser.c. RDP registers that iterator in rdp.rs, but
RdpTransaction::id() returns the stored id unchanged in rdp.rs, while
free_tx also compares against the raw stored id in rdp.rs. For a single
freeable RDP tx with stored id 1, the iterator returns C id 0; cleanup
calls free_tx(0), nothing is removed, then has_next == false allows
min_id to advance to total_txs in app-layer-parser.c. That leaves the
tx live but now below min_id, so later cleanup will not revisit it.
This patch brings the handling in line with the other parsers.
Bug: #8717.
Mark the direction into RDP transactions at creation time,
so the tx carries SKIP_INSPECT for the direction it is
never seen in, matching DHCP and the other single-direction
parsers. This lets cleanup free completed transactions and
stops a tx from being inspected (and alerting) twice, once per
direction.
RDP bounds its transactions to connection setup and stops
parsing once bypass_parsing is set.
Issue: 8621
DHCP is a stateless parser where each datagram is its own standalone,
single-direction transaction. It was creating transactions with
AppLayerTxData::new(), which leaves both SKIP_INSPECT bits clear, so the
engine treats every tx as still needing inspection in both directions.
For a flow that only ever carries one direction (broadcast DHCP, or a
relay seeing one side), the never-observed direction's inspect bit can
never be set, so AppLayerParserTransactionsCleanup() never frees the tx.
The per-flow transaction Vec then grows without bound and every packet
re-scans the whole list, giving O(n^2) CPU and unbounded memory on a
busy DHCP aggregation point.
Use AppLayerTxData::for_direction() like every other stateless parser
(DNS, SNMP, NTP, IKE, KRB5, MQTT, QUIC, SIP, WebSocket, bittorrent-dht)
so the tx carries SKIP_INSPECT for the direction it will never be seen
in. This lets cleanup free completed transactions and also stops the tx
from being inspected (and alerting) twice, once per direction.
Issue: 8621
Ticket: 8659
If a client sends a SOTR/RETR command before doing a PASV/PORT
command, ther server may reply
425 Use PORT or PASV first.\r\n
and allow the client to continue sendinf other commands
So, Suricata should not put the whole flow into error, it just
sets an event, and continues to parse further
Extend the xor transform to accept a variable-position key using
the 'extract <nbytes> <offset>' syntax. The engine reads <nbytes> bytes
at buffer position <offset> at transform time.
The optional 'offset' parameter specifies where XOR decoding
starts in the buffer; bytes before that position are left unchanged.
Syntax:
xor:"<hex_key>"
xor:extract <nbytes> <offset>
xor:offset <N>,"<hex_key>"
xor:offset <N>,extract <nbytes> <offset>
Variable-key helpers (VariableKeyLocation, variable_key_bytes,
parse_key_location, strip_keyword_prefix) are imported from the new
varkey module.
The variable key bytes are copied out of the inspection buffer before
decoding, so an in-place transform cannot corrupt the key when the key
region overlaps the decoded range.
Issue: 8671
Introduce rust/src/detect/transforms/varkey.rs to centralise helpers
used by transforms that read key material directly from the inspection
buffer at transform time.
Exported items:
VariableKeyLocation { offset: u16, nbytes: u8 }
variable_key_bytes bounds-checked slice into an inspection buffer
strip_keyword_prefix strip 'keyword<ws>' from option strings
parse_key_location parse '<nbytes> <offset>' into VariableKeyLocation
All four items are pub so any transform can import them. Unit tests
are included in the module.
Issue: 8671
Victor Julien
Philippe Antoine
Jason Ish
Jeff Lucovsky
Giuseppe Longo
Shivani Bhardwaj
Jeff Lucovsky