Commit Graph

143 Commits (0a831b5ea2a311c5bdc688d43238a272170c9a29)

Author SHA1 Message Date
Jason Ish 69cf5c9eea rust(lint): remove needless borrows
These are needless borrows (references) as the item is already
a reference.
5 years ago
Jason Ish 363b5f99c3 rust: functions that reference raw pointers are unsafe
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.

This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.

Then fix all not_unsafe_ptr_arg_deref warnings from clippy.

Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
5 years ago
Jason Ish 53413f2d7a rust: remove all usage of transmute
All cases of our transmute can be replaced with more idiomatic
solutions and do no require the power of transmute.

When returning an object to C for life-time management, use
Box::into_raw to convert the boxed object to pointer and use
Box::from_raw to convert back.

For cases where we're just returning a pointer to Rust managed
data, use a cast.
5 years ago
Philippe Antoine 9e7ea631b2 dns: improve probing parser
Checks opcode is valid
Checks additional_rr do not exceed message length
Better logic for incomplete cases
5 years ago
Philippe Antoine 7d0a39412b detect: use u32 for InspectionBufferMultipleForList
So that we do not have an endless loop casting index to
u16 and having more than 65536 buffers in one transaction

Changes for all protocols, even ones where it is impossible
to have such a pattern, so as to avoid bad pattern copy/paste
in the future
5 years ago
Jeff Lucovsky 00167121dc dns: Rework constructs to use Self/Default 5 years ago
Jason Ish c7f44447c9 dns: remove flood protection purging
It doesn't look like flood protection is required with the
stateless parser anymore. It actually can get in the way of TCP
DNS when a large number of requests end-up in the same segment
where a TX can get purged before it has a chance to go through
the normal TX life-cycle.
5 years ago
Philippe Antoine d2d0e0adc9 rust: remove exported unused functions 5 years ago
Jason Ish 6853bf98fb dns: only register a single logger
DNS no longer requires a logger to be registered for to-client and
to-server directions. This has not been required with the stateless
design of the Rust DNS parser.
5 years ago
Sascha Steinbiss a4556c9427 dns: use rest() for NULL parsing 6 years ago
Sascha Steinbiss 81b206a5f4 dns: add test for SRV 6 years ago
Sascha Steinbiss 08a6f1441a dns: parse SRV records 6 years ago
Antti Tönkyrä f3ad7f14fa dns: add rdata logging for NS rrtype 6 years ago
Simon Dugas 4336a0e739 dns: log rdata for NULL record type
Logs the rdata for a NULL record type as a printable string.
6 years ago
Victor Julien efc9a7a398 app-layer: remove callback for completion status
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.

Update all parsers to take advantage of this.
6 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 6 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
6 years ago
Simon Dugas 7f26246ce1 dns: parse and log fields for SOA record type
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.

Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
6 years ago
Simon Dugas af498fd840 dns: use nom's rest to take all remaining rdata
Using nom's `rest` combinator eliminates the need to call the do_parse
macro for parsing a single element.
6 years ago
Simon Dugas 8005f50647 dns: refactor to handle more rdata formats
Represent rdata as `DNSRData` enum variants instead of `Vec<u8>`.
This will allow parsing/logging of more complex formats like SOA.
6 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins
Fix plugin macros so they can be used by external Rust crates
such as plugins.
6 years ago
Philippe Antoine b20a8de646 dns: simply skips zero-sized dns requests/responses 6 years ago
Jason Ish 6cff558663 dns: add tailing data to split tcp unit test
Add trailing data to the complete payload to test the case
where data is consumed, but still incomplete.
6 years ago
Jason Ish cc154ce152 dns: fix subtraction overflow in debug message
The math for the return value wasn't updated in the debug log
to match the logic and could lead to a subtraction overflow
6 years ago
Philippe Antoine 80ccbff28a dns: return status from parsing tcp request 6 years ago
Philippe Antoine 1296740471 dns: fix incomplete results
and messages parsing whose length is >= 32768
6 years ago
Jeff Lucovsky 961b314b86 dns: Remove parser buffering code 6 years ago
Jason Ish d60671d855 rust/dns: use new flags field to set parser option flags 6 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
6 years ago
Jason Ish 9d9a901b68 dns: conditional logging
Apply config to newly created response TX.
6 years ago
Victor Julien ac3cf6ff75 detect/config: set config for special cases
Allow app-layer to declare the txs are uni-directional and special
care is needed for applying config.
6 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 6 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 6 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 6 years ago
Victor Julien fb3bdd8cf3 dns: remove detect_flags and logged now that we use AppLayerTxData 6 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
6 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
6 years ago
Victor Julien 79681bf655 app-layer: remove old MPMId API calls
Had been deprecated and non-functional since 2017.
6 years ago
Jason Ish 03cf3dcd6d dns/eve: convert to jsonbuilder 6 years ago
Philippe Antoine edcb784f1a dns: improve probe_tcp handling of incomplete data 6 years ago
Victor Julien 2f5834cdfa rust: merge parser.rs into applayer.rs
Both were serving the same purpose.
6 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
6 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
6 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust
And port the C DNS tests to Rust.
6 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
6 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust
Remove app-layer-dns-common.c as its no longer needed.
6 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
6 years ago
Jason Ish d809b0959b dns: cleanup: move DnsGetRcode (Lua) to rust
Move the implementation of Lua DnsGetRcode to Rust.
6 years ago
Pierre Chifflier 442500678b rust: use the streaming version of combinators to fix incomplete reads 6 years ago
Pierre Chifflier f3ddd7127c rust: Add types annotation when required
Unfortunately, the transition to nom 5 (and functions instead of macros)
has side-effects, one of them being requiring lots of types annotations
when using a parsing, for ex in a match instruction.
6 years ago
Pierre Chifflier 8664a55ee7 rust/dns: remove unneeded calls to closure! 6 years ago
Pierre Chifflier 5b809f77f1 rust: upgrade all parsers to nom 5 6 years ago
Jason Ish e68dfa46a6 rust: cargo fix for Rust 2018 edition 7 years ago
Jason Ish d79c23baa3 dns/detect: dns.opcode keyword
Add a rule keyword, dns.opcode to match on the opcode flag
found in the DNS request and response headers.

Only exact matches are allowed with negation.

Examples:
  - dns.opcode:4;
  - dns.opcode:!1;
7 years ago
Jason Ish 5f1d21f247 dns: handle mid stream pickup on response packet
Related Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2146
7 years ago
Jason Ish 42e5065ab8 rust: update to Rust 2018 with cargo fix
Migrate to Rust 2018 edition.

Credit to Danny Browning for first demontrating this:
https://github.com/OISF/suricata/pull/3604/commits
7 years ago
Victor Julien 579cc9f02b const: constify decoder, app-layer, detect funcs 7 years ago
Victor Julien 3f6624bf16 rust: remove libc crate dependency
Use std::os::raw instead.
7 years ago
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
7 years ago
Victor Julien b1d4931842 rust: fix warnings about wrong type of comments
"rustdoc does not generate documentation for macro expansions"
7 years ago
Victor Julien bf1bd407dd rust: fix libc deprecation warnings for int types 7 years ago
Pierre Chifflier f22695130b rust: nom4 requires to add complete!() when using many! combinators 8 years ago
Pierre Chifflier 13b7399790 rust: upgrade all parsers to nom4 8 years ago
Jason Ish b7083bc3a8 rust/dns/v2 - log rrtype in response
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2723
8 years ago
Jason Ish b7a58680db dns/rust - if let Some over options instead of loop.
Except in one case where the loop makes more sense for easy break
out.

Also remove one line of non-conforming debug logging.
8 years ago
Jason Ish 4163d5c360 rust/dns/lua - fix call convention to match C.
Also, when requesting the query, if the request doesn't exist,
return the query from the response. This makes it behave
more like C implementation.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2730
8 years ago
Jason Ish 87250da0fc rust/dns: add v1 dns logging
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2704
8 years ago
Jason Ish d83707bef9 rust/dns - remove extra parantheses
Removes rust compiler warning.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2521
8 years ago
Jason Ish 27fd521420 eve/dns/v2: support eve/dns v2 in rust 8 years ago
Victor Julien 73fac478a2 rust/dns: fix nom verbose error mode 9 years ago
Jason Ish c411519605 app-layer: remove has events callback - not used 9 years ago
Victor Julien f815027cdf rust/dns: simplify tx freeing
Now that we no longer need the state when freeing a TX, we can simply
do cleanup from the Drop trait.
9 years ago
Victor Julien 7548944b49 app-layer: remove unused HasTxDetectState call
Also remove the now useless 'state' argument from the SetTxDetectState
calls. For those app-layer parsers that use a state == tx approach,
the state pointer is passed as tx.

Update app-layer parsers to remove the unused call and update the
modified call.
9 years ago
Victor Julien 98eca55241 rust/dns: implement detect_flags API 9 years ago
Victor Julien bca0cd71ae app-layer: use logger bits to avoid looping
Avoid looping in transaction output.

Update app-layer API to store the bits in one step
and retrieve the bits in a single step as well.

Update users of the API.
9 years ago
Jason Ish 5a8537fe4a rust/dns - convert more type values to text
Issue:
https://redmine.openinfosecfoundation.org/issues/2364

Convert more record type and errr code values to text.
Remove duplicate type declarations.
9 years ago
Clément Galland 3396747cd6 Dns logger display flags information 9 years ago
Victor Julien e023ce9aad rust/dns: fix new warning in rustc 1.21 9 years ago
Jason Ish 3063851d85 rust/dns/tcp - probe even if payload is short
As the DNS probe just uses the query portion of a response, don't
require there to be as many bytes as specified in the TCP DNS
header. This can occur in large responses where probe is called
without all the data.

Fixes the cases where the app proto is recorded as failed.

Fixes issue:
https://redmine.openinfosecfoundation.org/issues/2169
9 years ago
Jason Ish 40991cab82 rust/dns: handle multiple txt strings
Fix handling of TXT records when there are multiple strings
in a single TXT record. For now, conform to the C implementation
where an answer record is created for each string in a single
txt record.

Also removes the data_len field from the answer entry. In Rust,
the length is available from actual data, which after decoding
may actually be different than the encoded data length, so just
use the length from the actual data.
9 years ago
Jason Ish 829155b9d5 rust/dns: pass byte arrays directly to rust/json
Using the json.set_string_from_bytes which will
safely convert the bytes printable ascii string
before logging.
9 years ago
Jason Ish c473c56eed rust/dns: fix panic on rrnames with bad chars
Check for erros in the UTF-8 conversion, on error, print the
the printable chars as chars, and print non printable chars
as \xHEX.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2148
9 years ago
Jason Ish ecc63481c6 rust/dns: fix tcp message length verification
And add Rust unit tests to check length validation.

Redmine issue 2144:
https://redmine.openinfosecfoundation.org/issues/2144
9 years ago
Jason Ish 70808a4f1d rust/dns: support gaps in TCP DNS 9 years ago
Jason Ish 4bdb722371 rust/dns: fix unit tests on Rust 1.7.0 9 years ago
Jason Ish 2aebfbce94 rust/dns: support txt records 9 years ago
Jason Ish 33e09a0002 rust dns: fixup for nom 3.0 9 years ago
Jason Ish ba1a67e2cb rust: dns: add log filtering on rrtype
While the filtering is still configured in C, the filtering
flags are passed into Rust so it can determine if a record
should be logged or not.
9 years ago
Jason Ish c54fc7f98f rust: use LoggerFlags type to track logged state 9 years ago
Jason Ish b588b49779 rust: lua support for DNS based Rust
Uses Rust wrappers around Lua to populate Lua
data structures.
9 years ago
Jason Ish 73388042b2 rust: DNS app-layer.
A DNS application layer in Rust. This is different than the
C based one, as it is partially stateless by not matching
up responses to replies.
9 years ago
Jason Ish 9449739dd5 rust: dns: nom DNS parsers 9 years ago
Jason Ish f6f126d53d rust: example of how an app-layer may be initialized
Also shows basic usage of the configuration API from Rust.
9 years ago