Use .is_some() and .is_none() instead of comparing against None.
Comparing against None requires a value to impl PartialEq, is_none() and
is_some() do not and are more idiomatic.
This patch updates the NT status code definition to use the status
definition used on Microsoft documentation website. A first python
script is building JSON object with code definition.
```
import json
from bs4 import BeautifulSoup
import requests
ntstatus = requests.get('https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55')
ntstatus_parsed = BeautifulSoup(ntstatus.text, 'html.parser')
ntstatus_parsed = ntstatus_parsed.find('tbody')
ntstatus_dict = {}
for item in ntstatus_parsed.find_all('tr'):
cell = item.find_all('td')
if len(cell) == 0:
continue
code = cell[0].find_all('p')
description_ps = cell[1].find_all('p')
description_list = []
if len(description_ps):
for desc in description_ps:
if not desc.string is None:
description_list.append(desc.string.replace('\n ', ''))
else:
description_list = ['Description not available']
if not code[0].string.lower() in ntstatus_dict:
ntstatus_dict[code[0].string.lower()] = {"text": code[1].string, "desc": ' '.join(description_list)}
print(json.dumps(ntstatus_dict))
```
The second one is generating the code that is ready to be inserted into the
source file:
```
import json
ntstatus_file = open('ntstatus.json', 'r')
ntstatus = json.loads(ntstatus_file.read())
declaration_format = 'pub const SMB_NT%s:%su32 = %s;\n'
resolution_format = ' SMB_NT%s%s=> "%s",\n'
declaration = ""
resolution = ""
text_max = len(max([ntstatus[x]['text'] for x in ntstatus.keys()], key=len))
for code in ntstatus.keys():
text = ntstatus[code]['text']
text_spaces = ' ' * (4 + text_max - len(text))
declaration += declaration_format % (text, text_spaces, code)
resolution += resolution_format % (text, text_spaces, text)
print(declaration)
print('\n')
print('''
pub fn smb_ntstatus_string(c: u32) -> String {
match c {
''')
print(resolution)
print('''
_ => { return (c).to_string(); },
}.to_string()
}
''')
```
Bug #5412.
Introduce AppLayerTxData::file_tx as direction(s) indicator for transactions.
When set to 0, its not a file tx and it will not be considered for file
inspection, logging and housekeeping tasks.
Various tx loop optimizations in housekeeping and output.
Update the "file capable" app-layers to set the fields based on their
directional file support as well as on the traffic.
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
This commit creates a module named "detect" for rule parsing logic. As
part of this commit, detect.rs is moved from its toplevel position into
the new module. Thus, use crate::detect::detect to refer to items within
detect.rs (instead of create::detect).
Ticket: 5077
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.
Feature #5413.
As an SMB2 async response does not have a tree id, even if
the request has it.
Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.
Ticket: #5508
On the Rust side, a Frame requires a StreamSlice to be created. We can
derive the direction from the StreamSlice removing the need for callers
to provide the direction when operating on the frame.
Move it away from http2 to generic core crate.
And use it for DCERPC (and SMB)
And remove the C version.
Main change in API is the free function is not free itself, but
a rust wrapper around unbox.
Ticket: #4112
Instead of a method that is required to return a slice of transactions,
use 2 methods, one to return the number of transactions in the
collection, and another to get a transaction by its index in the
collection.
This allows for the transaction collection to not be a contiguous array
and instead can be a VecDeque, or possibly another collection type that
supports retrieval by index.
Ticket #5278
To recognize a protocol, Suricata first looks for
patterns, which can be confirmed by a probing parser.
If this does not work, Suricata can try to run
some probing parsers on some ports.
This is the case for SMB.
This commit makes handling the confirming and the probing
paser differently even if they share much code.
The confirmation parser knows that a pattern has been found.
So, it must not do the midstream case of looking for this
pattern in the whole buffer, but only check it at the beginning.
But it must reverse direction if needed.
Allow limiting in-flight out or order data chunks per size or count.
Implemented for read and writes separately:
app-layer.protocols.smb.max-write-queue-size
app-layer.protocols.smb.max-write-queue-cnt
app-layer.protocols.smb.max-read-queue-size
app-layer.protocols.smb.max-read-queue-cnt
Instead of closing files in both direction when receiving a close request,
close only toserver files for the request and close toclient on receiving
a response.
If an SMB record is seen in the wrong direction, set an event on the PDU
frame and don't process the record in the state.
No error is returned, so the next record will be processed.
The bits were being parsed in the order they're displayed in Wireshark,
rather than the order they were being seen on the wire, resulting in
direction and async being 0 more often than they should be.
Instead of bits, take the 4 bytes as an le_u32 and just use bit masks to
extract what we need into a struct, I think its easier to reason about
this way when comparing to the Microsoft documentation.
The smb dce_iface keyword must match for all those dcerpc requests
and responses sent in the context of the given interface. They are
not matching as the current bind interfaces are deleted by any
non bind message.
Ticket: 4767
The smb dce_iface keyword must match for all those dcerpc requests and
responses sent in the context of the given interface. They are not
matching because in rs_smb_tx_get_dce_iface, x.req_cmd is erroneously
compared with 1. Fix this by comparing with DCERPC_TYPE_REQUEST instead.
Ticket: 4767
The smb dce_opnum matches all the opnums that are higher that the
indicated opnum. This is due the range comparison if was put in the
exact comparison context, and in case the opnum doesn't match exactly,
then the range comparison is triggered (the upper limit is always true).
Move the erroneus if to the outer context, as else option of the block
checks if comparison should be exact or range.
Ticket: 4767
The smb dce_opnum keyword doesn't match the dcerpc requests/responses.
This occurs because in the rs_smb_tx_match_dce_opnum function, the
x.req_cmd is matched against the erroneous code 1. Fix this by using
DCERPC_TYPE_REQUEST for the comparison instead.
Ticket: 4767
SMB1 record parsing code simplification.
Frames:
nbss.pdu
nbss.hdr
nbss.data
smb1.pdu
smb1.hdr
smb1.data
smb2.pdu
smb2.hdr
smb2.data
smb3.pdu
smb3.hdr
smb3.data
The smb* frames are created for valid SMB records.
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.
This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
The stream depth setting was broken since it was moved to Rust because
of a missing parser for memory values in configuration.
Use get_memval fn from conf.rs to calculate and fetch the correct
values.
Calling a function in unwrap_or causes that function to always
be called even when not needed. Instead use unwrap_or_else with
a closure which will only be called when needed.
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.
This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.
Then fix all not_unsafe_ptr_arg_deref warnings from clippy.
Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
All cases of our transmute can be replaced with more idiomatic
solutions and do no require the power of transmute.
When returning an object to C for life-time management, use
Box::into_raw to convert the boxed object to pointer and use
Box::from_raw to convert back.
For cases where we're just returning a pointer to Rust managed
data, use a cast.
Before, even if there were no outputs, all the arguments
were evaluated, which could turn expensive
All variables which are used only in certain build configurations
are now prefixed by underscore to avoid warnings
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.
Update all parsers to take advantage of this.
Jason Ish
Eric Leblond
Victor Julien
Victor Julien
Pierre Chifflier
Jeff Lucovsky
Philippe Antoine
Eloy Pérez González
Shivani Bhardwaj
Modupe Falodun
Sam Muhammed
Jeff Lucovsky
Juliana Fajardini