If an HTTP2 file was within only ont DATA frame, the filetracker
would open it and close it in the same call, preventing the
firther call to incr_files_opened
Also includes rustfmt again for all HTTP2 files
(cherry picked from commit bb98a18b3d)
After a file has been closed (CLOSE, COMMIT command or EOF/SYNC part of
READ/WRITE data block) mark it as such so that new file commands on that
file do not reuse the transaction.
When a file transfer is completed it will be flagged as such and not be
found anymore by the NFSState::get_file_tx_by_handle() method. This forces
a new transaction to be created.
(cherry picked from commit 67759795c6)
So that lexical-core, needed by nom, and using bitflags
is used with version 0.7.5 instead of version 0.7.0
which fixed the fact that BITS is now a reserved keyword
in nightly version
(cherry picked from commit 0105d4f017)
Due to ef5755338f, the variables
that are only used for debug output now emit unused variable
warnings when Suricata is not built with debug. Prefix these
variables with _ to suppress these warnings.
Before, even if there were no outputs, all the arguments
were evaluated, which could turn expensive
All variables which are used only in certain build configurations
are now prefixed by underscore to avoid warnings
(cherry picked from commit ef5755338f)
So that we do not have an endless loop casting index to
u16 and having more than 65536 buffers in one transaction
Changes for all protocols, even ones where it is impossible
to have such a pattern, so as to avoid bad pattern copy/paste
in the future
(cherry picked from commit 7d0a39412b)
Suppress the warning about an unused variable in the template
parser. As this is just a template I think this is OK, however
master should make sure this variable is used, if only to be
more self documenting.
Adding the already consumed bytes
In case an incomplete tls handshake is handled with/after
a refular rdp t123_tpkt
(cherry picked from commit 6da9a37285)
Lately, Wireguard proto starting w pattern |04 00| is misdetected as
DCERPC/UDP which also starts with the same pattern, add more checks
to make sure that it is the best guess for packet to be dcerpc/udp.
(cherry picked from commit f967a49104)
The book defines transmute as "This is really, truly, the most horribly unsafe
thing you can do in Rust. The guardrails here are dental floss."
Transmute can result into mind boggling undefined behaviors. Get rid of
it wherever possible.
(cherry picked from commit 0ca8591994)
Prior to Rust 1.44, Cargo would name static libs with the .lib
extension. 1.44 changes this extension to .a when running under
a GNU environment on Windows like msys to make it more similar
to other unix environments.
Now assume static library name to be the same on Windows and
unix, but rename the .lib if found to still support older
versions of Rust on Windows.
Implement missing transaction handling.
Fix logging wrongly casting 'state' to DCERPCState instead of
DCERPCUDPState leading to crashes and malformed output.
Remove unused fields from DCERPCUDPState.
warning: getting the inner pointer of a temporary `CString`
this `CString` is deallocated at the end of the statement,
bind it to a variable to extend its lifetime
Expand macros in the do_log macro after checking the log level
instead of each log macro (ie: SCLogDebug) expanding the macros
then passing off to do_log to have the log level check.
Will eliminate any expense of expanding macros if this log level
does not permit the given message to be logged.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4114
Log fields that only are meant to be in a PDU for a particular RPC
version. Since DCERPC/UDP works on RPC version 4 and DCERPC/TCP works on
RPC version 5, there are certain fields that are particular to each
version.
Remove call_id from the logger for UDP.
Add activityuuid and seqnum fields to the logger for UDP.
call_id and (activityuuid + seqnum) fields are used to uniquely pair a
request with response for RPC versions 5 and 4 respectively.
So far, request and response were paired with serial number fields in
the header. This is incorrect. According to
https://pubs.opengroup.org/onlinepubs/9629399/chap12.htm,
"Together, the activity UUID and the sequence number uniquely identify
a remote procedure call."
Hence, add activity uuid and sequence number to the transaction and pair
the request accordingly. Remove incorrect handling of this and fix
tests.
warning: variable does not need to be mutable
--> src/dcerpc/dcerpc.rs:1036:42
|
1036 | let tx = if let Some(mut tx) = self.get_tx_by_call_id(current_call_id, core::STREAM_TOCLIENT) {
| ----^^
| |
| help: remove this `mut`
|
= note: `#[warn(unused_mut)]` on by default
warning: variable does not need to be mutable
--> src/dcerpc/dcerpc.rs:1061:30
|
1061 | Some(mut tx) => {
| ----^^
| |
| help: remove this `mut`
The headers table from client to server
and the one from server to client
may have different maximum sizes
(even if both endpoints have to keep both tables)
Scenario is use of dummy padding in write AndX request
or other similar commands using a data offset.
Parsing skips now these dummy bytes, and generates one event
Evasion scenario is
- a first dummy write of one byte at offset 0 is done
- the second full write of EICAR at offset 0 is then done
and does not trigger detection
The last write had the final value, and as we cannot "cancel"
the previous write, we set an event which is then transformed into
an app-layer decoder alert
This patch addresses issues discovered by redmine ticket 3896. With the
approach of finding latest record, there was a chance that no record was
found at all and consumed + needed became input length.
e.g.
input_len = 1000
input = 01 05 00 02 00 03 a5 56 00 00 .....
There exists no |05 00| identifier in the rest of the record. After
having parsed |05 00|, there was a search for another record with the
leftover data. Current data length at this point would be 997. Since the
identifier was not found in the data, we calculate the consumed bytes at
this point i.e. consumed = current_data.len() - 1 which would be 996.
Needed bytes still stay at a constant of 2. So, consumed + needed = 996
+ 2 = 998 which is lesser than initial input length of 1000 and hence
the assertion fails.
There could be two fixes to this problem.
1. Finding the latest record but making use of the last found record in
case no new record was found.
2. Always use the earliest record.
This patch takes the approach (2). It also makes sure that the gap and
current direction are the same.
When one side of the connection reaches the STREAM_DEPTH condition the
parser should be aware of this. Otherwise transactions will forever be
waiting for data in that direction.
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2
This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
As is documented in RFC 7541, section 6.1
The index value of 0 is not used. It MUST be treated as a decoding
error if found in an indexed header field representation.
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.
Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.
As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
Functions written in Rust will need to suricata::plugin::init()
to bootstrap themselves. This bootstrap process sets the log level
within the Rust address space, and hooks up function pointers
that are expected to be set during normal runs of Suricata.
Make sure the log level is setup with a pure Rust function, so
when it is set, its set within the address space of the caller.
This is important for Rust plugins where the Rust modules are not
in the address space of the Suricata main process.
This commit logs http2 as an http event. The idea is to somewhat
normalize http/http2 so common info can be version agnostic.
This puts the http2 specific fields in an "http2" object inside
the "http" object.
HTTP2 headers/values that are in common with HTTP1 are logged
under the "http" object to be compatible with HTTP1 logging.
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.
When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.
For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.
For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.
The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.
Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
Elastic search didn't accept the 'hassh' and 'hassh.string'. It would
see the first 'hassh' as a string and split the second key into a
object 'hassh' with a string member 'string'. So two different types
for 'hassh', so it rejected it.
This patch mimics the ja3(s) logging by creating a 'hassh' object
with 2 members: 'hash', which holds the md5 representation, and
'string' which holds the string representation.
In case of lossy connections the NFS state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.
This would lead these files staying in the NFS's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.
This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.
Based on 65e9a7c31c
DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.
In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
Uses "cargo doc --no-deps" to build the documentation just for
our Suricata package. Without --no-deps, documentation will be
build for all our dependencies as well.
The generated documentation will end up in target/doc as HTML.