Commit Graph

1135 Commits (f848e34bcc266a2a4d8f5fc2661d2b430449b190)

Author SHA1 Message Date
Jason Ish f342d4aacd rust: fix clippy lints for clippy::len_zero 3 years ago
Jason Ish 5e5401d3e9 rust: fix clippy lints for clippy::char_lit_as_u8 3 years ago
Jason Ish 29a4a7fddc rust: fix clippy lints for clippy::assign_op_pattern 3 years ago
Jason Ish c4034dafa1 rust: fix clippy lints for clippy::derive_partial_eq_without_eq 3 years ago
Jason Ish 5a10fcd2d8 rust: suppress large enum variant lint at location
And disable the global lint.
3 years ago
Jason Ish 74b7522b6a rust/http2: box decompressor variants
These variants, in particular the Brotli one can be large at over 2500
bytes which is allocated no matter which decompressor is being used.

Gzip comes in at over 500 bytes.  Box deflate for consistency.
3 years ago
Jason Ish 36f8ada305 rust: remove clippy lints that no longer warn 3 years ago
Jason Ish e8c00dd980 rust: sort clippy allow statements 3 years ago
Haleema Khan 6c922e0b98 rust: fix lint warning for clippy::enum's name
Ticket: #4597
3 years ago
Jason Ish 2a42386c28 rust: fix clippy lint for null comparison
Use .is_null() instead of checking for equality against
std::ptr::null().
3 years ago
Jason Ish 45dfea2497 rust/modbus: derive default instead of manual impl
Cleans up a clippy lint for a trivial default impl that can be derived.
3 years ago
Jason Ish 9218da0eb8 rust/frames: cleanup clippy lint for unsafe
Where possible mark the relevant functions unsafe.  Otherwise suppress
the warning for now as this pattern is supposed to be a safe API around
an unsafe one. Might need some further investigation, but in general the
"guarantee" here is provided from the C side.
3 years ago
Jason Ish 105d9a5f02 rust: fix clippy lint for unnecessary_unwrap
Avoid check if not none followed by unwrap.
3 years ago
Jason Ish 85cfa7254b rust: fix clippy lint for single_char_add_str
Idiomatic cleanup and a fix automatically done by `cargo clippy --fix`.
3 years ago
Jason Ish f3e4bcfe23 rust: fix clippy lint for bool_assert_comparison
Checking for is_empty is faster than checking for equality.
3 years ago
Jason Ish f60e1b30f6 rust: fix clippy lint for partialeq_to_none
Use .is_some() and .is_none() instead of comparing against None.
Comparing against None requires a value to impl PartialEq, is_none() and
is_some() do not and are more idiomatic.
3 years ago
Jason Ish 7d623f0854 rust: fix clippy lint for explicit_auto_deref
This adds unnecessary complexity to code.
3 years ago
Jason Ish c503ca62e2 rust: fix clippy lint for needless_late_init 3 years ago
Jason Ish 94dd85baed rust: fix clippy lint for borrow_deref_ref
This type of borrow then reference has no effect.
3 years ago
Jason Ish e9597f3d0c rust: fix clippy lint for redundant_closure
Removes a closure where the function can be directly provided.
3 years ago
Jason Ish c5b26e2043 rust: fix clippy ling for needless borrows
Cleanup needless borrows found by clippy. This fix done automatically by
`cargo clippy --fix`.
3 years ago
Jason Ish 63b3d73ccc rust: allow some more clippy lints
Allow these lints for now until some more investigation can be done, as
--fix attempts to fix these.
3 years ago
Eric Leblond a9519778de rust/smb: avoid allocation in smb status function
Avoid an allocation by returning a static string.
3 years ago
Eric Leblond 9cb06d4376 detect/smb: add smb.ntlmssp_domain keyword
Feature #5411.
3 years ago
Eric Leblond 5debb86cd5 rust/smb1: add a missing command 3 years ago
Eric Leblond 69ef1bc194 detect/smb: add smb.ntlmssp_user keyword
Feature #5411.
3 years ago
Eric Leblond f46f895e8d rust/smb: import NT status code for Microsoft doc
This patch updates the NT status code definition to use the status
definition used on Microsoft documentation website. A first python
script is building JSON object with code definition.

```
import json
from bs4 import BeautifulSoup
import requests

ntstatus = requests.get('https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55')

ntstatus_parsed = BeautifulSoup(ntstatus.text, 'html.parser')

ntstatus_parsed = ntstatus_parsed.find('tbody')

ntstatus_dict = {}

for item in ntstatus_parsed.find_all('tr'):
    cell = item.find_all('td')
    if len(cell) == 0:
        continue
    code = cell[0].find_all('p')
    description_ps = cell[1].find_all('p')
    description_list = []
    if len(description_ps):
        for desc in description_ps:
            if not desc.string is None:
                description_list.append(desc.string.replace('\n ', ''))
    else:
        description_list = ['Description not available']
    if not code[0].string.lower() in ntstatus_dict:
        ntstatus_dict[code[0].string.lower()] = {"text": code[1].string, "desc": ' '.join(description_list)}

print(json.dumps(ntstatus_dict))
```

The second one is generating the code that is ready to be inserted into the
source file:

```
import json

ntstatus_file = open('ntstatus.json', 'r')

ntstatus = json.loads(ntstatus_file.read())

declaration_format = 'pub const SMB_NT%s:%su32 = %s;\n'
resolution_format = '        SMB_NT%s%s=> "%s",\n'

declaration = ""
resolution = ""

text_max = len(max([ntstatus[x]['text'] for x in ntstatus.keys()], key=len))

for code in ntstatus.keys():
    text = ntstatus[code]['text']
    text_spaces = ' ' * (4 + text_max - len(text))
    declaration += declaration_format % (text, text_spaces, code)
    resolution += resolution_format % (text, text_spaces, text)

print(declaration)
print('\n')
print('''
pub fn smb_ntstatus_string(c: u32) -> String {
    match c {
''')
print(resolution)
print('''
        _ => { return (c).to_string(); },
    }.to_string()
}
''')
```

Bug #5412.
3 years ago
Victor Julien db0f9ddc69 files/tx: inspection, logging and loop optimizations
Introduce AppLayerTxData::file_tx as direction(s) indicator for transactions.
When set to 0, its not a file tx and it will not be considered for file
inspection, logging and housekeeping tasks.

Various tx loop optimizations in housekeeping and output.

Update the "file capable" app-layers to set the fields based on their
directional file support as well as on the traffic.
3 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
3 years ago
Victor Julien 01e64d80da app-layer: trunc parser per direction 3 years ago
Victor Julien ff9d1807f9 app-layer: parser flags to u16 3 years ago
Victor Julien c27df6304d app-layer: introduce common AppLayerStateData API
Add per state structure for storing flags and other variables.
3 years ago
Victor Julien 879a46f085 rust: lock to time 0.3.13 to avoid MSRV bump to 1.59
Indirect dependency through x509-parser.
3 years ago
Pierre Chifflier 16db04c1a7 rust: remove nom 5 dependency 3 years ago
Pierre Chifflier 0acf75bff7 rust/applayertemplate: convert to nom7 3 years ago
Pierre Chifflier 378e915846 rust/asn1: convert parsers to nom7 3 years ago
Pierre Chifflier 0ba0572c4a rust/x509: finish transition to nom7 3 years ago
Pierre Chifflier 3ef5121ab0 rust/telnet: convert parsers to nom7 3 years ago
Pierre Chifflier d98b386f36 rust/conf: convert parser to nom7 3 years ago
Pierre Chifflier db9a1e17b6 rust/ssh: finish transition to nom7 3 years ago
Pierre Chifflier b31c72c06a rust/rdp: convert parsers to nom7 3 years ago
Pierre Chifflier 49520b2143 rust/rdp: upgrade dependency on tls-parser 3 years ago
Pierre Chifflier beadd090b8 rust: upgrade versions of BER/DER, Kerberos and SNMP parsers 3 years ago
Jason Ish baab1de735 rust: update x509-parser to 0.14.0
Resolves RustSec issues in time and chrono:
- https://rustsec.org/advisories/RUSTSEC-2020-0071
- https://rustsec.org/advisories/RUSTSEC-2020-0159

Ticket: #5259.

Ammended by Victor Julien to bump to 0.14 instead of 0.13.
3 years ago
Pierre Chifflier 3aace49649 rust/x509: update dependency on x509-parser 3 years ago
Jeff Lucovsky ab4d0f7f4a detect/stream_size: Rename detect.rs to stream_size.rs
This commit renames detect.rs to stream_size.rs to reflect its content.
3 years ago
Jeff Lucovsky 2386f2614f detect/iprep: Move iprep logic into a separate module 3 years ago
Jeff Lucovsky c957882d1c detect/uri: Move uri logic into a separate module 3 years ago
Jeff Lucovsky 484c34bc60 detect/uint: Move uint logic into a separate module
This commit moves the uint logic into its own module.
3 years ago
Jeff Lucovsky 8bfe427a74 rust/detect: Create detect module for rule parsing
This commit creates a module named "detect" for rule parsing logic. As
part of this commit, detect.rs is moved from its toplevel position into
the new module. Thus, use crate::detect::detect to refer to items within
detect.rs (instead of create::detect).

Ticket: 5077
3 years ago
Jeff Lucovsky ccd1063e43 detect/bytemath: convert parser to Rust
Issue: 5077

This commit
- Converts the PCRE based parser to Rust.
- Adds unit tests to the new Rust modules
- Removes the PCRE parser from detect-bytemath.c
- Adjusts the C source modules to refer to the Rust definitions
- Includes the multiply operator (missing from the C parser)
3 years ago
Philippe Antoine af40873127 pgsql: config limit maximum number of live transactions
As is done for other protocols

Ticket: #5527
3 years ago
Philippe Antoine e160917bcf mqtt: remove quadratic time complexity
When having many transactions in a single parsing call...

Fix has overhead of having one more field in the mqtt state.

Completes commit a8079dc978

Ticket: #5399
3 years ago
Philippe Antoine 5ef259722b dhcp: adds renewal-time keyword
Ticket: #5507
3 years ago
Philippe Antoine 6faf6299e0 dhcp: adds rebinding-time keyword
Ticket: #5506
3 years ago
Philippe Antoine 95f0424423 nfs4: fix write record handling
Ticket: #5280
3 years ago
Philippe Antoine bf43011a43 dcerpc: convert transaction list to vecdeque for UDP
As was done for TCP in dfe76bb90 and d745d28d4

Ticket: #5518
3 years ago
Eric Leblond 2cc9152fc9 rust/smb: log uuid of interface in dcerpc
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.

Feature #5413.
3 years ago
Eric Leblond b6f1cf255c rust/smb/dcerpc: parse context id
As context id is used to know to which variant of the endpoint the
request is done, it is interesting to parse it.

Feature #5413.
3 years ago
Philippe Antoine 11f849c3ee protocol-change: sets event in case of failure
Protocol change can fail if one protocol change is already
occuring.

Ticket: #5509
3 years ago
Philippe Antoine e94920b49f smb: do not use tree id to match create request and response
As an SMB2 async response does not have a tree id, even if
the request has it.

Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.

Ticket: #5508
3 years ago
Philippe Antoine 461725a9bf dhcp: adds leasetime keyword
As it is logged

Ticket: #5435
3 years ago
Philippe Antoine 9b4a133777 http2: remove to_vec for comparisons
Ticket: #5454
3 years ago
Philippe Antoine d011b468da http2: fix clippy warning about &Vec<u8>
Using &[u8] instead in function prototype
3 years ago
Philippe Antoine 3de735ae70 ike: log ikev1 tx fields instead of state ones
As state fields can grow abitrarily, and this can lead to DOS
by quadratic complexity (CPU time and disk space)

Adds a direction field to retain all the information in the
transaction.

Also checks array vendor_ids had at least one element before
logging it.

Ticket: #5455
3 years ago
Philippe Antoine d0171d7418 ike: rustfmt 3 years ago
Philippe Antoine 5c7b5c5fb5 krb: detection for ticket encryption
As is done for logging.

Ticket: #5442
3 years ago
Philippe Antoine 64b2385c64 krb: log for ticket encryption
Also logs if the ticket encryption is weak.
It is different from the encryption used for the rest of the
packet, and this allows to detect kerberoasting attack.

Ticket: #5442
3 years ago
Philippe Antoine 7fcc6696cb krb: rustfmt kerberos.rs 3 years ago
Philippe Antoine 675de33405 krb: bump up crate version
kerberos parser crate is also used by other procotols : nfs and
smb. These protocols use an older der_parser crate version.
Upgrading der_parser will simplify the code further.
3 years ago
Philippe Antoine 783dff2c38 krb: rustfmt detect.rs 3 years ago
Jason Ish c862e84c01 rust/frames: cleanups
- Implement the Display trait on Direction to print "toserver" or
  "toclient" which used in a format string.

- Use Direction struct inside Frame instead of a u32.  Requires a helper
  method as there are two representation in C for direction, and the C
  methods for frames don't use the internal representation of the
  Direction enum (some sweeping changes could help here)
3 years ago
Jason Ish f92708b8ca rust/frames: derive direction from StreamSlice
On the Rust side, a Frame requires a StreamSlice to be created. We can
derive the direction from the StreamSlice removing the need for callers
to provide the direction when operating on the frame.
3 years ago
Jason Ish b39d7f46e7 dns/tests: fix StreamSlice to satisfy debug validation 3 years ago
Philippe Antoine f3b6fd3329 quic: update to nom7 3 years ago
Philippe Antoine 95125811b8 quic: reassemble crypto frames and parse it 3 years ago
Philippe Antoine f242fb7f22 quic: events and rules on them 3 years ago
Philippe Antoine b9c1d9e86b quic: parse gquic version Q039
Ticket: #5166
3 years ago
Philippe Antoine 018fef5ef8 quic: ja3 computation and logging and detection
Logging as is done in TLS.

Detection using the generic generic ja3.string keyword

Ticket: #5143
3 years ago
Philippe Antoine c6cf61a39b quic: complete parsing of initial for non gquic
The format of initial packet for quic ietf, ie quic v1,
is described in rfc 9000, section 17.2.2

Parse more frames and logs interesting extensions from crypto frame

Do not try to parse encrypted data, ie after we have seen
a crypto frame in each direction.

Use sni from crypto frame with tls for detection already implemented

Ticket: #4967
3 years ago
Philippe Antoine 7044131c39 quic: rustfmt 3 years ago
Philippe Antoine 0c346af4a9 rust: bump up digest crates
so that we can use hkdf crate for quic
3 years ago
Philippe Antoine 2294e9cdbc rdp: bump up tls-parser crate version
so that we can use new functions in quic parser
3 years ago
Philippe Antoine 11e0eb9c89 quic: do not log empty cyu array
Ticket: #5167
3 years ago
Philippe Antoine 632581ac95 ike: do not log empty notify array
Ticket: #5167
3 years ago
Philippe Antoine 262a93ce18 mqtt: do not log reason_codes if there is none
Ticket: #5167
3 years ago
Philippe Antoine 1621f5e453 detect/nfs: use inclusive ranges 3 years ago
Philippe Antoine ed6955ee98 detect: use generic integer functions for iprep
Ticket: #4112
3 years ago
Philippe Antoine cfb60d0fce detect: use generic integer functions for urilen
Ticket: #4112
3 years ago
Philippe Antoine c57052181c snmp: rustfmt detect.rs 3 years ago
Philippe Antoine c7214be99b snmp: adds usm keyword
as is logged

Ticker: #5416
3 years ago
Philippe Antoine eb1c2a6083 smb: use default stream-depth 0 by default
As broken by commit e5c948df87

Ticket: #5390
3 years ago
Philippe Antoine c585be338c nfs: fix arbitrary allocation
Bug introduced by https://github.com/OISF/suricata/pull/7111

Nom's count begins by allocating a Vector, which leads to arbitrary
allocation due to flavors_cnt coming from network, and not even
being checked against i.len()

Ticket: #5237
3 years ago
Philippe Antoine 26dc70648c dns: remove unused events field from state
found overflowing by oss-fuzz
3 years ago
Philippe Antoine d1a4dae36b detect: use generic integer functions for streamsize
By the way, adds the prefilter feature

Ticket: #2697
Ticket: #4112
3 years ago
Philippe Antoine 35b6dcec7e detect: use generic integer functions for filesize
Ticket: #4112
3 years ago
Philippe Antoine f29b43defd detect: rust generic functions for integers
Move it away from http2 to generic core crate.
And use it for DCERPC (and SMB)

And remove the C version.
Main change in API is the free function is not free itself, but
a rust wrapper around unbox.

Ticket: #4112
3 years ago
Philippe Antoine c4d9cb02ec util: better hex print function
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
3 years ago
Philippe Antoine 6058792bee rust: make suricata context const
So that it is read only and its pointers do not get modified
3 years ago
Philippe Antoine 6224e283fa modbus: bump up rust crate version
So that probing parser is more strict and does not accept unknown
function code as valid modbus.

Ticket: #5377
3 years ago