Commit Graph

300 Commits (928ac012156fb8d393ce5ac4a496fde3c2e87b00)

Author SHA1 Message Date
Victor Julien 9a4231d737 src: various comment spelling fixes
Thanks to Josh Soref.
3 years ago
Victor Julien f28459ed78 app-layer: spelling 3 years ago
Jason Ish 6344501dba tls: fix date logging for dates before 1970
The Rust time crate used by the x509-parser crate represents dates
before 1970 as negative numbers which do not survive the conversion to
SCTime_t and formatting with the current time formatting functions.

Instead of fixing our formatting functions to handle such dates,
create a Rust function for logging TLS dates directly to JSON using
the time crate that handles such dates properly.

Also add a FFI function for formatting to a provided C buffer for the
legacy tls-log.

Issue: 5817
4 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 4 years ago
Philippe Antoine b5147189ae tls: fix off by one in supported versions extension
Ticket: #5663
4 years ago
Victor Julien 8f0d820218 ssl: add debug validation checks for recent changes
Make sure the assumptions are correct.
4 years ago
Philippe Antoine 3fbcba8ab7 tls: fix buffer overread
Recently introduced by commit 4bab6e24e5

Ticket: #5564
4 years ago
Philippe Antoine 5a0d07b13f clean: remove useless * sizeof(char) 4 years ago
Victor Julien c27df6304d app-layer: introduce common AppLayerStateData API
Add per state structure for storing flags and other variables.
4 years ago
Philippe Antoine ae6abd8ca3 ssl: fix compiler warning
implicit conversion loses integer precision: 'int' to 'uint16_t'
because C shifts << translates automatically to signed integers
4 years ago
Victor Julien 1701a6b14c tls: handle incomplete header sooner
Make sure to exit the parser early on incomplete header data.

Additionally, make sure to not create duplicated tls frames in this
case.

Add a debug validation check for the header parser parsing too much
data, which should never happen.
4 years ago
Victor Julien 9053c49178 ssl: add debug validation check for incomplete api 4 years ago
Victor Julien 69be41b241 tls: improve record checks
Improve unknown record handling. Inspired by Wireshark 'unknown record'
handling, we take a best effort approach for records with unknown content
types in TLS versions 1.0, 1.1 and 1.2.

Improve record length check and set 'invalid_record_length' event instead
of 'invalid_tls_header'.
4 years ago
Victor Julien c028800ae1 tls: improve versions extension logic
Skip over unusable versions like GREASE.
4 years ago
Victor Julien 599791fa33 tls: make version and size checks stricter
This way bad records won't buffer lots of stream data.
4 years ago
Victor Julien c73d812026 tls: store cert data in heap buffer
Cert chain is a list of pointers into this buffer, so can't use a
stream slice approach.
4 years ago
Victor Julien e817a8f968 tls: parse client certificates
Parse client cerificates and store them in the state similar to how
this is done for server certificates.

Update "progress" handling to not consider the TLS handshake complete
if the server indicated a client cert was needed.
4 years ago
Victor Julien 10f5e6cb66 tls: prepare for client cert parsing 4 years ago
Victor Julien 14b2e04b58 tls: make cert handling more generic
In preparation for client cert handling.
4 years ago
Victor Julien cf4c201acb tls: avoid tls.invalid_handshake_message FP
Don't set TLS_DECODER_EVENT_INVALID_HANDSHAKE_MESSAGE event on encrypted
handshake messages.
4 years ago
Victor Julien e9d63f3355 tls: don't set 2 events for a single exception
Keep the more specific ones.
4 years ago
Victor Julien 214e466b29 tls: remove incomplete tests
These tests are incompatible with the incomplete API usage and should
have been pcap based tests in the first place.
4 years ago
Victor Julien 82e03bd8fc tls: set event if record size exceeds limit 4 years ago
Victor Julien e857c864ca tls: support server hello done message 4 years ago
Victor Julien 4bab6e24e5 tls: support handshake fragmentation
Implement TLS record defrag for handshake messages.

Ticket: #5183.
4 years ago
Victor Julien bcaf0f6f7d tls: remove certificate buffering code
TCP Buffering is now done in the app-layer using the incomplete API, on
the SSL/TLS record level. TLS level fragmentation will be implemented
separately.
4 years ago
Victor Julien 0839317ea7 tls: parse handshake protocol records in single pass 4 years ago
Victor Julien 9f0ea5e70c sslv2: use version from client hello
Remove streaming code that is now unused.

Incomplete handling makes this record parsing work on full data.
4 years ago
Victor Julien c8d79fb81f ssl: implement 'incomplete' handling for SSLv2 4 years ago
Victor Julien 6076a51511 tls: streaming mode for application records
To avoid overhead of stream buffering for records we don't do
much with anyway, pass through application records instead of
buffering the entire record in the stream engine.
4 years ago
Victor Julien 129fcb5c72 tls: use incomplete API to get full TLS records
The TLS record header is parsed in streaming mode still, but once the
record size is known we tell the app-layer API to give us the full
record.

Ticket: #5481
4 years ago
Victor Julien dbf3d1e977 tls: make SSLSetEvent a macro to help debugging 4 years ago
Victor Julien e250ef6402 debug: remove empty header 4 years ago
Shivani Bhardwaj 42c3f418c6 tls: add tls.random* keywords
Add tls.random keyword that matches on the 32 bytes of the TLS
random field for client as well as server.
Add tls.random_time keyword that matches on the first 4 bytes of the TLS
random field for client as well as server.
Add tls.random_bytes keyword that matches on the last 28 bytes of the TLS
random field for client as well as server.

All these are sticky buffers.

Feature 5190
4 years ago
Philippe Antoine 02f2602dde src: rework includes as per cppclean 4 years ago
Victor Julien 84448d3bae tests: remove unnecessary flow locks
Added once to satisfy debug validation, but we don't mix unittests
and debug validation anymore.

    sed -i -E '/.*FLOWLOCK_.*LOCK/d' *.c
4 years ago
Philippe Antoine c4d9cb02ec util: better hex print function
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
4 years ago
Jason Ish adda8801d8 conf: remove ConfGetValue
All uses of ConfGetValue are satisfied by ConfGet
4 years ago
Philippe Antoine 862e84877f ssl: first pass limit when allocating buffer for certificates
With this check, on the first packet of a certificate presenting
a length of 16Mbytes, we only allocate up to 65Kb

When we get to the point where need more than 65Kb, we realloc
to the true size.

With this check, it makes it more expensive for an attacket to use
this allocation as a way to trigger ressource exhaustion...
5 years ago
Juliana Fajardini e5838b8193 applayer/frame: remove output from GetFrame funcs
As these functions can be probed, having output there results in
misleading output.
5 years ago
Victor Julien af797b5926 ssl: implement frames for SSLv3 and TLS 5 years ago
Philippe Antoine c56b1c99d5 ssl: fix int warnings
especially increasing padding_len size
5 years ago
Philippe Antoine e1c0725e05 doc: fix typo lenght/length 5 years ago
Jason Ish 6392216f6b base64: use the Rust base64 encode implementation
Replace our internal base64 implementation with a ffi wrapper
around the Rust implementation provided by an external crate.
5 years ago
Victor Julien c073d5cfbf app-layer: use StreamSlice as input to parsers
Remove input, input_len and flags in favor of stream slice.
5 years ago
Victor Julien 6466296b32 app-layer: add StreamSlice to pass data to parsers
Since object to contain relevant pointer, length, offset, flags to make
it easy to pass these to the parsers.
5 years ago
Jason Ish 7732efbec2 app-layer: include decoder events in app-layer tx data
As most parsers use an events structure we can include it in the
tx_data structure to reduce some boilerplate/housekeeping code
in app-layer parsers.
5 years ago
Jason Ish 9c67c634c1 app-layer: include DetectEngineState in AppLayerTxData
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.

This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
5 years ago
Jeff Lucovsky 1eeb96696b general: Cleanup bool usage 5 years ago
Philippe Antoine 2997be6707 sslv2: precise detection pattern with probing parser 6 years ago
Philippe Antoine eb460cf78d ssl: reset state when breaking out of SSLV3_HANDSHAKE_PROTOCOL
So that we cannot resumt it with corrupted values
6 years ago
Jason Ish 90dcaff41d ssl: replace ComputeSHA1 with Rust SCSha1HashBuffer
Removes the final use of internal ComputeSHA1 function.
6 years ago
Jason Ish d5bf748c90 ja3: remove requirement on NSS 6 years ago
Jason Ish e50ee7eb62 filestore: remove requirement of nss for filestore
Required including NSS header in places that depended on
util-file.h including it.

All filestore suricata-verify tests now pass without libnss.

Also enabled detect-file{md5,sha1,sha256} without NSS support.
6 years ago
Victor Julien efc9a7a398 app-layer: remove callback for completion status
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.

Update all parsers to take advantage of this.
6 years ago
Philippe Antoine 4706b38866 ssl: do not reuse struct session_id_length
As it can be confused between SSLv2 and TLSv13
In SSLv2, this variable is not used after the function scope, so
we can use a temporary variable.
6 years ago
Philippe Antoine 2eacc6a9a8 ssl: adds safety checks
Ensure the client version is valid by checking hello flags

Ensure no integer underflow occurs in SSLv3ParseHandshakeType
6 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
6 years ago
Victor Julien 4f73943df9 app-layer: split EOF flag per direction 6 years ago
Victor Julien fb780c7d92 ssl/tls: support AppLayerTxData 6 years ago
Victor Julien 3ed188e0bc ssl: support multi-frag certificate assembly
Support reassembling multi-frag certificates. For this the cert queuing
code is changed to queue just the cert, not entire tls record.

Improve message tracking. Better track where a message starts and ends
before passing data around.

Add wrapper macros to check for 'impossible' conditions that are activate
in debug validation mode. This helps fuzzers find input that might trigger
these conditions, if they exist.
6 years ago
Victor Julien 4f679fd843 ssl: add asserts for 'impossible' conditions
Wrap in debug validation so that fuzzing can pick them up.
6 years ago
Victor Julien 68d5a9dc2c tls/sni: parsing cleanup
Set proper event on all invalid sni length values.
6 years ago
Victor Julien 61b8c99236 ssl: improve error checking 6 years ago
Victor Julien bb06298102 ssl: unify main parsing routine 6 years ago
Victor Julien 40be9d2219 ssl: improve debug output 6 years ago
Victor Julien f1bf11f716 ssl: record parsing cleanup 6 years ago
Victor Julien ab44b5edac ssl: handshake parsing code cleanup 6 years ago
Victor Julien d1ada2e13c ssl: copy data using a safe memcpy wrapper
To avoid future memcpy issues introduce a wrapper and check the
result of it.

When compiled with --enable-debug-validation the wrapper will abort if
the input is wrong.
6 years ago
Victor Julien cffbdff024 ssl: don't say we consumed bytes if we didn't consume them 6 years ago
Victor Julien 9950ebffe6 ssl: code cleanups 6 years ago
Victor Julien 1578c84605 ssl: bump copyright year 6 years ago
Victor Julien 9a97821c43 ssl: improve 'first cert' check to avoid leaks
In some error conditions, or potentially in case of multiple 'certificate'
records, the extracted subject, issuerdn and serial could be overwritten
without freeing the original memory.
6 years ago
Victor Julien fa2a1385ea ssl: fix handshake cert buffer sizing
'trec' buffer was not grown properly when it was checked as too small.
After this it wasn't checked again so that copying into the buffer could
overflow it.
6 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 7 years ago
Pierre Chifflier 333fcc43e7 ssl/tls: call rs_cstring_free for strings allocated in Rust 7 years ago
Pierre Chifflier 1d9f37a60e DER: remove the C parser for DER 7 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 7 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
7 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
7 years ago
Victor Julien 4164c0bbd6 app-layer: make dns,smb,tls parsers less noisy w/o config 7 years ago
Victor Julien c582fd28d9 tls/ja3: allow 'auto' setting for ja3 7 years ago
Victor Julien 29dcd98ed1 tls/ja3: add way to check active config 7 years ago
Victor Julien 4cd3b84606 tls/ja3: allow dynamic enabling of ja3 7 years ago
Victor Julien 922f4f7d78 ssl: fix bounds checking in version decoding
Reported-by: Sirko Höer -- Code Intelligence for DCSO.

Bug #3169.
7 years ago
Victor Julien 229eccdd04 ssl: minor cleanups 7 years ago
Victor Julien 579cc9f02b const: constify decoder, app-layer, detect funcs 7 years ago
Nick Price d0a85b7550 ja3: Mention LibNSS dependency for JA3 7 years ago
Philippe Antoine 477328f79b ssl: register probing for port 443 if no config 7 years ago
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
7 years ago
Jeff Lucovsky f7b934f83f app-layer/logging: protocol parser updates 7 years ago
Mats Klepsland a4471987ba app-layer-ssl: generate JA3S fingerprints
Generate JA3S fingerprints based on fields in the ServerHello record.
7 years ago
Philippe Antoine 316a411b6b ssl : SSLProbingParser overflow fix
Found by fuzzing
Fixes ssl detection evasion by packet splitting
7 years ago
Victor Julien 422e4892cc proto-detect: improve midstream support
When Suricata picks up a flow it assumes the first packet is
toserver. In a perfect world without packet loss and where all
sessions neatly start after Suricata itself started, this would be
true. However, in reality we have to account for packet loss and
Suricata starting to get packets for flows already active be for
Suricata is (re)started.

The protocol records on the wire would often be able to tell us more
though. For example in SMB1 and SMB2 records there is a flag that
indicates whether the record is a request or a response. This patch
is enabling the procotol detection engine to utilize this information
to 'reverse' the flow.

There are three ways in which this is supported in this patch:

1. patterns for detection are registered per direction. If the proto
   was not recognized in the traffic direction, and midstream is
   enabled, the pattern set for the opposing direction is also
   evaluated. If that matches, the flow is considered to be in the
   wrong direction and is reversed.

2. probing parsers now have a way to feed back their understanding
   of the flow direction. They are now passed the direction as
   Suricata sees the traffic when calling the probing parsers. The
   parser can then see if its own observation matches that, and
   pass back it's own view to the caller.

3. a new pattern + probing parser set up: probing parsers can now
   be registered with a pattern, so that when the pattern matches
   the probing parser is called as well. The probing parser can
   then provide the protocol detection engine with the direction
   of the traffic.

The process of reversing takes a multi step approach as well:

a. reverse the current packets direction
b. reverse most of the flows direction sensitive flags
c. tag the flow as 'reversed'. This is because the 5 tuple is
   *not* reversed, since it is immutable after the flows creation.

Most of the currently registered parsers benefit already:

- HTTP/SMTP/FTP/TLS patterns are registered per direction already
  so they will benefit from the pattern midstream logic in (1)
  above.

- the Rust based SMB parser uses a mix of pattern + probing parser
  as described in (3) above.

- the NFS detection is purely done by probing parser and is updated
  to consider the direction in that parser.

Other protocols, such as DNS, are still to do.

Ticket: #2572
8 years ago
Mats Klepsland d62d33cdff app-layer-ssl: check that cipher suites length is divisible by two
Cipher suites length should always be divisible by two. If it is a
odd number, which should not happen with normal traffic, it ends up
reading one byte too much.
8 years ago
Mats Klepsland b5dfc9ed41 app-layer-ssl: fix coverty error (RESOURCE_LEAK)
Bug #2677
8 years ago
Mats Klepsland 033e756905 app-layer-ssl: add Facebook TLSv1.3 draft versions
Add draft versions for Facebooks custom TLSv1.3 implementation "fizz"
to SSLVersionToString().
8 years ago
Mats Klepsland dd5374c20d app-layer-ssl: change how TLSv1.3 drafts are logged
Change from logging TLSv1.3 drafts as "TLS 1.3 (draft 28)" to
"TLS 1.3 draft-28" instead.
8 years ago
Mats Klepsland a8347e1bc2 app-layer-ssl: fix flow and inspection bypass for TLSv1.3 8 years ago
Mats Klepsland 3b73b7d542 app-layer-ssl: add 0-RTT support for TLSv1.3 8 years ago