The Rust time crate used by the x509-parser crate represents dates
before 1970 as negative numbers which do not survive the conversion to
SCTime_t and formatting with the current time formatting functions.
Instead of fixing our formatting functions to handle such dates,
create a Rust function for logging TLS dates directly to JSON using
the time crate that handles such dates properly.
Also add a FFI function for formatting to a provided C buffer for the
legacy tls-log.
Issue: 5817
Make sure to exit the parser early on incomplete header data.
Additionally, make sure to not create duplicated tls frames in this
case.
Add a debug validation check for the header parser parsing too much
data, which should never happen.
Improve unknown record handling. Inspired by Wireshark 'unknown record'
handling, we take a best effort approach for records with unknown content
types in TLS versions 1.0, 1.1 and 1.2.
Improve record length check and set 'invalid_record_length' event instead
of 'invalid_tls_header'.
Parse client cerificates and store them in the state similar to how
this is done for server certificates.
Update "progress" handling to not consider the TLS handshake complete
if the server indicated a client cert was needed.
TCP Buffering is now done in the app-layer using the incomplete API, on
the SSL/TLS record level. TLS level fragmentation will be implemented
separately.
To avoid overhead of stream buffering for records we don't do
much with anyway, pass through application records instead of
buffering the entire record in the stream engine.
The TLS record header is parsed in streaming mode still, but once the
record size is known we tell the app-layer API to give us the full
record.
Ticket: #5481
Add tls.random keyword that matches on the 32 bytes of the TLS
random field for client as well as server.
Add tls.random_time keyword that matches on the first 4 bytes of the TLS
random field for client as well as server.
Add tls.random_bytes keyword that matches on the last 28 bytes of the TLS
random field for client as well as server.
All these are sticky buffers.
Feature 5190
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
With this check, on the first packet of a certificate presenting
a length of 16Mbytes, we only allocate up to 65Kb
When we get to the point where need more than 65Kb, we realloc
to the true size.
With this check, it makes it more expensive for an attacket to use
this allocation as a way to trigger ressource exhaustion...
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.
This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
Required including NSS header in places that depended on
util-file.h including it.
All filestore suricata-verify tests now pass without libnss.
Also enabled detect-file{md5,sha1,sha256} without NSS support.
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.
Update all parsers to take advantage of this.
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2
This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
Support reassembling multi-frag certificates. For this the cert queuing
code is changed to queue just the cert, not entire tls record.
Improve message tracking. Better track where a message starts and ends
before passing data around.
Add wrapper macros to check for 'impossible' conditions that are activate
in debug validation mode. This helps fuzzers find input that might trigger
these conditions, if they exist.
To avoid future memcpy issues introduce a wrapper and check the
result of it.
When compiled with --enable-debug-validation the wrapper will abort if
the input is wrong.
In some error conditions, or potentially in case of multiple 'certificate'
records, the extracted subject, issuerdn and serial could be overwritten
without freeing the original memory.
'trec' buffer was not grown properly when it was checked as too small.
After this it wasn't checked again so that copying into the buffer could
overflow it.
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.
Introduce two macros:
APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)
Update all parsers to use this.
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
When Suricata picks up a flow it assumes the first packet is
toserver. In a perfect world without packet loss and where all
sessions neatly start after Suricata itself started, this would be
true. However, in reality we have to account for packet loss and
Suricata starting to get packets for flows already active be for
Suricata is (re)started.
The protocol records on the wire would often be able to tell us more
though. For example in SMB1 and SMB2 records there is a flag that
indicates whether the record is a request or a response. This patch
is enabling the procotol detection engine to utilize this information
to 'reverse' the flow.
There are three ways in which this is supported in this patch:
1. patterns for detection are registered per direction. If the proto
was not recognized in the traffic direction, and midstream is
enabled, the pattern set for the opposing direction is also
evaluated. If that matches, the flow is considered to be in the
wrong direction and is reversed.
2. probing parsers now have a way to feed back their understanding
of the flow direction. They are now passed the direction as
Suricata sees the traffic when calling the probing parsers. The
parser can then see if its own observation matches that, and
pass back it's own view to the caller.
3. a new pattern + probing parser set up: probing parsers can now
be registered with a pattern, so that when the pattern matches
the probing parser is called as well. The probing parser can
then provide the protocol detection engine with the direction
of the traffic.
The process of reversing takes a multi step approach as well:
a. reverse the current packets direction
b. reverse most of the flows direction sensitive flags
c. tag the flow as 'reversed'. This is because the 5 tuple is
*not* reversed, since it is immutable after the flows creation.
Most of the currently registered parsers benefit already:
- HTTP/SMTP/FTP/TLS patterns are registered per direction already
so they will benefit from the pattern midstream logic in (1)
above.
- the Rust based SMB parser uses a mix of pattern + probing parser
as described in (3) above.
- the NFS detection is purely done by probing parser and is updated
to consider the direction in that parser.
Other protocols, such as DNS, are still to do.
Ticket: #2572
Cipher suites length should always be divisible by two. If it is a
odd number, which should not happen with normal traffic, it ends up
reading one byte too much.