Commit Graph

535 Commits (b698f66c2701c19903cdac6ac5a5c370b33a8817)

Author SHA1 Message Date
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 8fe9faecb2 app-layer: remove DetectFlags API. Replaced by AppLayerTxData 5 years ago
Victor Julien 88dd0abb38 rdp: support AppLayerTxData 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien e0debed0b4 tftp: support AppLayerTxData 5 years ago
Victor Julien 4ff51a0e07 sip: support AppLayerTxData 5 years ago
Victor Julien cc1210c956 ntp: support AppLayerTxData 5 years ago
Victor Julien e0f75157a0 ikev2: support AppLayerTxData 5 years ago
Victor Julien 64e2a27512 applayer/template: support AppLayerTxData 5 years ago
Victor Julien a484bbbe1b dhcp: support AppLayerTxData 5 years ago
Victor Julien 11e2434526 snmp: support AppLayerTxData 5 years ago
Victor Julien 5afe4835ad rfb: support AppLayerTxData 5 years ago
Victor Julien 9f29366c7c krb5: support AppLayerTxData 5 years ago
Victor Julien 2aab1938d7 ssh: support AppLayerTxData 5 years ago
Victor Julien a1e06247a6 dcerpc/udp: support AppLayerTxData 5 years ago
Victor Julien 3202d29325 dcerpc: support AppLayerTxData 5 years ago
Victor Julien 77a95eddd9 smb: support AppLayerTxData 5 years ago
Victor Julien 7a7805cde6 nfs: support AppLayerTxData 5 years ago
Victor Julien fb3bdd8cf3 dns: remove detect_flags and logged now that we use AppLayerTxData 5 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
5 years ago
Victor Julien e15995e2d2 detect: store detect flags in AppLayerTxData 5 years ago
Victor Julien c797c9f09c app-layer: add logger flags to AppLayerTxData 5 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
5 years ago
Jason Ish 03efbccfe6 jsonbuilder: set_float, append_float methods
New methods for setting and appending float values.
5 years ago
Philippe Antoine ece29c4210 ssh: fix incomplete return for ssh kex
In the case where we already parsed some records
5 years ago
Philippe Antoine ca6d072297 dcerpc: detect right parsing of empty op version 5 years ago
Emmanuel Thompson 6e5d64f102 detect/asn1: Simplify errors and checks 5 years ago
Emmanuel Thompson 4fc45b5c60 detect/asn1: Update ASN1 struct lifetime
- 'static is only realistic when allocating and leaking it over the
FFI boundary
5 years ago
Emmanuel Thompson 627e90a4bd detect/asn1: Log out errors
- Failure to parse asn1-max-frames
- Failure on asn1 detection checks
5 years ago
Emmanuel Thompson 88601b1993 detect/asn1: Update relative_offset keyword
- To be consistent with recent C version changes
- Add checks for over/underflows
5 years ago
Emmanuel Thompson 7af6cdb7ec detect/asn1: Update asn1 C files to use rust code
Mark rust extern "C" functions as pub in asn1 module to expose via cbindgen
Update detect-asn1.c/h to use rust functions
5 years ago
Emmanuel Thompson 63704fdf13 rust/asn1: Introduce ASN1 rust module
This module uses the `der-parser` crate to parse ASN1 objects in order to replace src/util-decode-asn1.c
It also handles the parsing of the asn1 keyword rules and detection checks performed in src/detect-asn1.c
5 years ago
Emmanuel Thompson 6b8517dc12 rust: Update der, kerberos and snmp parser dependencies
- The update to der-parser allows us to use the latest API changes
5 years ago
Jason Ish 43b9bfaed4 applayer template (rust): convert to JsonBuilder 5 years ago
Vadym Malakhatko 126597144c eve: add Hassh fields to SSH JSON logger and add ssh log condition 5 years ago
Vadym Malakhatko 536cee3ba9 rust/ssh: add hassh generation
Add generation of hassh fingerprints based on fields in the kexinit record
5 years ago
Jeff Lucovsky d5bb41011c output/ikev2: Convert to JsonBuilder
Convert the IKEV2 Json logging to use JsonBuilder.
5 years ago
Victor Julien 65e9a7c31c smb: fix 'dangling' files in lossy sessions
In case of lossy connections the SMB state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the SMB's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.
5 years ago
Victor Julien 25f2efe977 smb: check post-gap timeouts once a second at most 5 years ago
Victor Julien 8aa380600d smb: update ts only if it changed 5 years ago
Jeff Lucovsky 8c5c949cfa output/tftp: Convert to JsonBuilder
This commit converts the TFTP logging mechanisms to JsonBuilder.
5 years ago
Jason Ish 07e88a7479 jsonbuilder: add debug_validate to state
If debug validation is enabled, panic on invalid state errors.

For example, calling close on an already closed jsonbuilder
object.
5 years ago
Jason Ish ca6b70ea1b rust: macro debug_validate_fail to fail with message
Add a new debug_validate macro that unconditionally panics
with a message. Useful in Rust pattern matching.
5 years ago
Shivani Bhardwaj a7535099b4 smb/eve: convert to jsonbuilder
Closes redmine ticket 3712.
5 years ago
Jason Ish 639f3d265e rust: lock to nom 5.1.1
5.1.2 pulls in dependencies that don't build on Rust 1.34.
5 years ago
Victor Julien 79681bf655 app-layer: remove old MPMId API calls
Had been deprecated and non-functional since 2017.
5 years ago
Jason Ish a545cdef6a jsonbuilder: setter for formatted data
Create a method to set preformatted data that contains the key
and the value already formatted.

This is an optimization for static data.
5 years ago
Jason Ish e3b7c58218 jsonbuilder: export {set,append}_string_from_bytes to C 5 years ago
Jason Ish f184bcc10e jsonbuilder: use Box::from_raw instead of transmute to free
I think this is a bad use of transmute, while the end result
is the same, Box::from_raw is more correct as we created this
pointer with Box::into_raw.
5 years ago
Philippe Antoine baf5f52f22 ssh/eve: convert to jsonbuilder 5 years ago
Victor Julien b3b5802c85 eve/nfs: switch output to jsonbuilder 5 years ago
Jason Ish 6ce9b2972b rdp: enable by default
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3255
5 years ago
Jason Ish 5a7ba62493 sip: enable by default
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3256
5 years ago
Jason Ish 36d687580a rfb/eve: convert to jsonbuilder 5 years ago
Jason Ish 60bfbd43fd jsonbuilder: add reset marks
Add methods to get the state of a JsonBuilder (called a mark),
then allow restoring to the mark.
5 years ago
Jason Ish ddb22549be rust: allow some clippy lints without warning
Suppresses some clippy lints that have more to do with style
than anything else, to reduce the amount of noise in the
clippy output.
5 years ago
Jason Ish 03cf3dcd6d dns/eve: convert to jsonbuilder 5 years ago
Jason Ish 6a70d6bb6e sip/eve: convert to jsonbuilder 5 years ago
Jason Ish deed0541bb dhcp/eve: convert to jsonbuilder 5 years ago
Jason Ish 942dd08654 jsonbuilder: new module for generating json
JsonBuilder is a Rust module for creating JSON output. Unlike
Jansson, the final JSON string is built up as items are added,
instead of building up an object tree and rendering it when
done.

The idea is to create a more efficient JSON serializer instead
of a flexible one.
5 years ago
Jason Ish c1f4edc434 rust: bring back libc as a dependency
Its already pulled in by some of other dependencies so adds zero
extra weight, and provides handy definitions for basic functions
like free().
5 years ago
Jason Ish 5513b4ed0b rust/json: expose libjansson json_dumps
This will be temporarily used by JsonBuilder to add the ability
to extend JsonBuilder with Jansson's json_t types.
5 years ago
Victor Julien 9fd326b6c0 ssh: minor cleanups in incomplete handling 5 years ago
Victor Julien 3a2434ed4d app-layer: support Copy and Clone traits in AppLayerResult 5 years ago
Victor Julien b0288da686 app-layer: add methods to get status from AppLayerResult 5 years ago
Philippe Antoine 6373071aa3 ssh: handles incomplete record after banner
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
5 years ago
Philippe Antoine 69b4fffdae parse: move SSH parser from C to Rust 5 years ago
Philippe Antoine cb3c478525 style: compressed function headers for rustfmt 5 years ago
Philippe Antoine 6b2e7dde7d rust: export enums definition to C 5 years ago
Shivani Bhardwaj 80adf7d1cf smb: Import constants from DCERPC
Remove DCERPC constants to avoid duplicate name errors. Import the
required constants from DCERPC implementation.
5 years ago
Shivani Bhardwaj 6457754fd6 dcerpc: Replace C function calls with Rust
All the dead code in C after the Rust implementation is hereby removed.
Invalid/migrated tests have also been deleted.
All the function calls in C have been replaced with appropriate calls to
Rust functions. Same has been done for smb/detect.rs as a part of this
migration.
5 years ago
Shivani Bhardwaj 8036202c7b rust: Add DCERPC parser
This parser rewrites the DCE/RPC protocol implementation of Suricata
in Rust. More tests have been added to improve the coverage and some
fixes have been made to the tests already written in C. Most of the
valid tests from C have been imported to Rust.

File anatomy

src/dcerpc.rs
This file contains the implementation of single transactions in DCE/RPC
over TCP. It takes care of REQUEST, RESPONSE, BIND and BINDACK business
logic before and after the data parsing. DCERPCState holds the state
corresponding to a particular transaction and handles all important
aspects. It also defines any common structures and constants required
for DCE/RPC parsing irrespective of the carrier protocol.

src/dcerpc_udp.rs
This file contains the implementation of single transactions in DCE/RPC
over UDP. It takes care of REQUEST and RESPONSE parsing. It borrows the
Request and Response structs from src/dcerpc.rs.

src/detect.rs
This file contains the implementation of dce_iface and opnum detect
keywords. Both the parsing and the matching is taken care of by
functions in this file. Tests have been rewritten with the test data
from C.

src/parser.rs
This file contains all the nom parsers written for DCERPCRequest,
DCERPCResponse, DCERPCBind, DCERPCBindAck, DCERPCHeader, DCERPCHdrUdp.
It also implements functions to assemble and convert UUIDs. All the
fields have their endianness defined unless its an 8bit field or an
unusable one, then it's little endian but it won't make any difference.

src/mod.rs
This file contains all the modules of dcerpc folder which should be
taken into account during compilation.

Function calls

This is a State-wise implementation of the protocol for single
transaction only i.e. a valid state object is required to parse any
record. Function calls start with the app layer parser in C which
detects the application layer protocol to be DCE/RPC and calls the
appropriate functions in C which in turn make a call to these functions
in Rust using FFI. All the necessary information is passed from C to the
parsers and handlers in Rust.

Implementation

When a batch of input comes in, there is an analysis of whether the
input header and the direction is appropriate. Next check is about the
size of fragment. If it is as defined by the header, process goes
through else the data is buffered and more data is awaited. After this,
type of record as indicated by the header is checked. A call to the
appropriate handler is made. After the handling, State is updated with
the latest information about whatever record came in.
AppLayerResult::ok() is returned in case all went well else
AppLayerResult::err() is returned indicating something went wrong.
5 years ago
Shivani Bhardwaj 6db1f19d62 rust: Add debug_validate_bug_on macro
This macro allows to check if certain parts of the code are reachable
during fuzzing.
5 years ago
Shivani Bhardwaj 57ad609a97 rust: Add new crate uuid 5 years ago
Shivani Bhardwaj ab6b4986ce rust: Add Debug and PartialEq to AppLayerResult 5 years ago
frank honza bbe9137f20 rfb: Update incomplete handling in parser.
This commit adds an updated incomplete handling for the RFB-Parser. If
incomplete data is processed, the successfully consumed position and
length of remainder + 1 is returned. If the next packet is not empty
suricata will call the parser again.

This commit is a result of discussion on https://github.com/OISF/suricata/pull/4792.
5 years ago
Philippe Antoine edcb784f1a dns: improve probe_tcp handling of incomplete data 5 years ago
Sascha Steinbiss 713c379427 rfb: make sure size calculations do not overflow
Addresses #3570 by extra checking of calculated size requests.

With the given input, the parser eventually arrived at
parser::parse_failure_reason() which parsed from the remaining four
bytes (describing the string length) that the failure string to follow
would be 4294967295 bytes long. While calculating the total size of the
data to request via AppLayerResult::incomplete(), adding the four bytes
for the parsed but not consumed string length caused the u32 length to
overflow, resulting in a much smaller value triggering the bug condition.

This problem was addressed by more careful checking of values in each step
that could overflow: one subtraction, one addition (which could overflow
the usize length values), and a final check to determine whether the result
still fit into the u32 values required by AppLayerResult::incomplete().
If so, we would safely convert the values and pass them to the result type.
If not, we simply return AppLayerResult::err() but do not erroneously and
silently request the wrong amount.
5 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 5 years ago
Pierre Chifflier 36d2e257c6 rust/x509: use the raw serial number so leading zeros are not removed 5 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 5 years ago
Pierre Chifflier 10d9deec9f rust: add common function to exchange CString objects from/to C 5 years ago
Sascha Steinbiss 26123e05f2 rfb: use more idiomatic Rust code
Using 'if let Some()...' makes the code in these many checks more
concise and readable.
5 years ago
Frank Honza 1c8943dedd add RFB parser
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
5 years ago
Victor Julien acef21b759 app-layer: extend AppLayerResult to add convenience 5 years ago
Victor Julien 2f5834cdfa rust: merge parser.rs into applayer.rs
Both were serving the same purpose.
5 years ago
Victor Julien 21e6f1f063 app-layer: document return macros 5 years ago
Victor Julien 5b9b0b7226 nfs: switch to new 'incomplete' logic
Remove buffering code in favor of using incomplete API.
5 years ago
Victor Julien 66598f9ce7 nfs: switch to AppLayerResult 5 years ago
Victor Julien 5cd9cfb5a0 smb: update return type of GAP handling 5 years ago
Victor Julien 4bf87d30e4 smb: convert to return AppLayerResult
Support returning 'incomplete' and remove the buffering
code from the parser.
5 years ago
Victor Julien 674b8dc0fb app-layer: add 'incomplete' return logic
Allow app-layer parsers to indicate how much data they need
before being called again.
5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
5 years ago
Victor Julien 6ae66cb2bb nfs: code cleanups
Use 'if let' to replace simple 'match' statements.

Use explicit returns to easy code review.
5 years ago
Victor Julien a729d266c3 smb: fix rustc 1.42 warnings 5 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust
And port the C DNS tests to Rust.
5 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
5 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust
Remove app-layer-dns-common.c as its no longer needed.
5 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
5 years ago