aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,710 Commits
7 Branches
155 Tags
192 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'd3354555af'
${ noResults }
Commit Graph

1383 Commits (d3354555afdf1bdd0e442a42d002c96b66f80115)

Author SHA1 Message Date
Victor Julien d57510a10f files: remove unused Rust binding for file pruning 2 years ago
Victor Julien a24d7dc45c smb: fix post-trunc chunk behavior 
After a gap in a file transaction, the file tracker is truncated. However
this did not clear any stored out of order chunks from memory or stop more
chunks to be stored, leading to accumulation of a large number of chunks.

This patches fixes this be clearing the stored chunks on trunc. It also
makes sure no more chunks are stored in the tracker after the trunc.

Bug: #5781.
 2 years ago
Philippe Antoine 55c4834e4e smb: configurable max number of transactions per flow 
Ticket: #5753
 2 years ago
Philippe Antoine 1d9183638f smb: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5753
 2 years ago
Philippe Antoine cb89192ec3 smb: fix typo in comment 2 years ago
Haleema Khan cfcb7df9dc mqtt: rustfmt parser.rs 2 years ago
Haleema Khan 23acb89653 mqtt: add unittests for nom7 parsers 
Ticket: #5742
 2 years ago
Haleema Khan cdc5ccd7f7 rfb: rustfmt parser.rs 2 years ago
Haleema Khan b95d7efbd0 rfb: add unittests for nom7 parsers 
Task: #5741
 2 years ago
Philippe Antoine 3979acb5ed smb: set event for ntlmssp unusual order 2 years ago
Philippe Antoine e41c01a483 smb: rustfmt ntlmssp_records.rs 2 years ago
Philippe Antoine 1db8685848 smb/ntlmssp: parse fields independently of order 
Instead of relying on the usual ordering...

Ticket: #5258
 2 years ago
Jason Ish ae192ebae7 rust: sync log levels with C 2 years ago
Jeff Lucovsky f8474344cd log: Add module and subsystem identifiers to log 
Issue: 2497

This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.

The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.
 2 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 2 years ago
Jason Ish bd9adac3ac rust/clippy: comments on why we have specific allows 2 years ago
Jason Ish dfd7abe185 rust/clippy: fix lint: type_complexity 
Convert a DNS sub-parser to use a return type rather than a large
tuple. For mqtt, allow the lint for now, but remove the global allow.
 2 years ago
Jason Ish e49ce49471 rust/clippy: allow result_unit_err in http2 only 
Its the only module making use of this pattern, but we shouldn't let
new modules use this pattern.
 2 years ago
Jason Ish 7ba2dadc7f rust/clippy: fix lint: upper_case_acronyms 2 years ago
Jason Ish 029ac650d7 rust/clippy: fix lint: manual_find 
These get_tx methods look like ideal candidates for generic and/or
derived methods.
 2 years ago
Jason Ish 4940dfb3bd rust/clippy: fix lint: len_without_is_empty 2 years ago
Jason Ish e1cffd348f rust/clippy: fix lint: field_reassign_with_default 2 years ago
Jason Ish 9df7c326b9 rust/clippy: remove allow: collapsible_else_if 2 years ago
Jason Ish 30ee5fc835 rust/clippy: remove allow: collapsible_if 
Already clean.
 2 years ago
Jason Ish da12b77f18 rust/clippy: fix lint: new_without_default 2 years ago
Jason Ish c4cf062a6f rust/clippy: fix lint: redundant_pattern_matching 2 years ago
Jason Ish 7c293ff68f rust/clippy: fix lint: never_loop 2 years ago
Jason Ish e8823644ec rust/clippy: fix lint: nonminimal_bool 2 years ago
Jason Ish 53ae0c8a06 rust/clippy: fix lint: derive_partial_eq_without_eq 2 years ago
Jason Ish 5d62995e26 rust/clippy: fix lint: explicit_counter_loop 2 years ago
Jason Ish f250b92180 rust/clippy: fix lint: extra_unused_lifetimes 2 years ago
Jason Ish 3044565cf4 rust/clippy: fix lint: needless_range_loop 2 years ago
Jason Ish 2ac52d0610 rust/clippy: remove lint: for_loops_over_fallibles 
Already clean.
 2 years ago
Jason Ish c026d8531b rust/clippy: fix lint: match_ref_pats 2 years ago
Jason Ish 359d5fcb7e rust/clippy: fix lint: needless_lifetimes 2 years ago
Jason Ish 4e001688de rust/clippy: remove lint: bool_comparison 
Already clean.
 2 years ago
Jason Ish f15ffbc869 rust/clippy: fix lint: single_match 
Allow this lint in some cases where a match statement adds clarity.
 2 years ago
Jason Ish 925bc74c1f rust/clippy: fix lint: while_let_loop 2 years ago
Jason Ish cf20fa1e67 template: import c_void, c_char, c_int 
These are ffi types that are commonly used, import them so they can be
used by their short names instead of a fully qualified name.
 2 years ago
Jason Ish 4220f18258 template: remove no_mangle and pub where not needed 
Extern functions that are only used as a function pointer do not
require "pub" or "no_mangle".
 2 years ago
Jason Ish 4a7567b3f0 template: rename template-rust to template 
Remove the distinction between the C template protocol "template" and
the Rust template protocol "template-rust" and make the Rust parser
simply template now that we no longer have support to generate a C
protocol template.
 2 years ago
Jason Ish 38321a213f rust/app-layer-template: rustfmt 2 years ago
Jason Ish 50a787a9a3 app-layer-template-rust: remove C app-layer stub 
Remove the app-layer-PROTO stub for Rust based parsers.  It is no longer
needed as Rust parsers now contain the registration function in Rust.

Ticket: 4939
 2 years ago
Jason Ish baa7021ee6 rust/conf: add fn conf_get_node 
A wrapper around ConfGetNode to get a configuration node by name.
 2 years ago
Victor Julien 64c0459d2d rust/lzma: clippy fixup 2 years ago
Jason Ish 35f99d1af7 rust/http2: fix clippy lint for is_empty() 
This snuck through as "cargo clippy" check wasn't finding lints that
were fixed by the previous test for fixable lints.
 2 years ago
Todd Mortimer 7d1a8cc335 file/swf: Use lzma-rs decompression instead of libhtp. 
Use the lzma-rs crate for decompressing swf/lzma files instead of
the lzma decompressor in libhtp. This decouples suricata from libhtp
except for actual http parsing, and means libhtp no longer has to
export a lzma decompression interface.

Ticket: #5638
 2 years ago
Victor Julien 45eb038e63 smb: fix file reopening issue 
Fuzzing highlighted an issue where a command sequence on the same file
id triggered a logging issue:

file data for id N
close id N
file data for id N

If this happened in a single blob of data passed to the parser, the
existing file tx would be reused, the file "reopened", confusing the
file logging logic. This would trigger a debug assert.

This patch makes sure a new file tx is created for the file data
coming in after the first file tx is closed.

Bug: #5567.
 2 years ago
Philippe Antoine 29f40c9e07 dcerpc: fix integer underflow 
as input.len() can be 65536, it cannot be directly cast to u16

Ticket: #5557
 2 years ago
Jason Ish 91617f479a rust: sha-1 is now sha1 
This is the same crate, but renamed to be more consistent with the
RustCrypto project naming. Some recent discussion is available here:

    https://github.com/RustCrypto/hashes/issues/438
 2 years ago
Philippe Antoine af44504550 smb: do not use tree id to match request and response 
Completes commit e94920b49f

This must be true for access to state ssn2vecoffset_map

Ticket: #5161
 2 years ago
Victor Julien cade6046c5 rust/files: open file without trackid as pointer 2 years ago
Victor Julien ad869e1c52 rust/filecontainer: remove unused declaration 2 years ago
Philippe Antoine 086b28da3d http2: fix decompression buffering 
It was not enough to set Cursor position to 0,
also its inner Vec should be cleared.

This way, a new input gets written at the beginning of the
Cursor and its inner Vec...

Ticket: #5691
 2 years ago
Philippe Antoine c6349d3cfc http2: support padded data frames 
Ticket: #5691
 2 years ago
Philippe Antoine e1ee401a12 quic: use VecDeque 
Ticket: #5637
 2 years ago
Philippe Antoine 286bd2a7ed rust: fix cargo clippy --all-features 2 years ago
Philippe Antoine bc287018e5 rust: cargo clippy --all-features --fix --allow-no-vcs 2 years ago
Philippe Antoine cd4bf518f3 rust: fix warnings on rustc 1.67.0-nightly 
warning: for loop over an `Option`. This is more readably written
as an `if let` statement
 2 years ago
Juliana Fajardini a654ef50de pgsql: add test for parameter status parser 
Since we've done some changes to how the parameters are parsed, add one
more test case to check that.

Bug #5579
 2 years ago
Juliana Fajardini c4fbd78770 pgsql: move database into opt parameters list 
For StartupMessages, the database parameter is optional. This moves the
parameter into the optional_parameters list.

Bug #5579
 2 years ago
Philippe Antoine cc68898532 pgsql: support empty parameter values 
Bug #5579
 2 years ago
Philippe Antoine 1e0190bc6b pgsql: support out of order parameters for startup message 
As user can be not the first parameter

Bug #5579
 2 years ago
Jason Ish 1f056f9974 bittorrent-dht: parse and log node6 lists 
Node6 lists are just like node lists, but for IPv6 addresses.
 2 years ago
Jason Ish 86d5ab8644 bittorrent-dht: remove tests that are no longer valid 2 years ago
Jason Ish 2f9eb5d1dd bittorrent-dht: fix values decoding, as a list of peers 
The "values" field is not a string, but instead peer information in
compact format. Decode this properly and then properly format in the
log.
 2 years ago
Jason Ish 4a0859beeb jsonbuilder: add append_hex - add hex to array 
New method, append_hex to add a byte array to a JSON array in hex
encoding.
 2 years ago
Jason Ish 4bc9cf3986 bittorrent-dht: parse token and target as byte values 2 years ago
Jason Ish 5a30ee77a1 bittorrent-dht: only attempt to parse dht messages 
The bittorrent flow is shared with transport messages as well as dht
messages. Only attempt to parse dht message as dht, ignore the rest.
 2 years ago
Jason Ish 98a9391210 bittorrent-dht: decode node data structures 
Instead of decoding the nodes field into a blog of bytes, decode it into
an array of node info objects, each with a node id, IP address and port.
 2 years ago
Jason Ish 3cb50592ed bittorrent-dht: convert some fields to byte arrays 
Some fields that were previously strings are not always value UTF-8
data, instead the protocol specification refers to them as strings of
bytes, so in other words byte arrays.

Currently fields converted are:
- client_version
- info_hash
- response.id
- request.id
- nodes
- token
 2 years ago
Jason Ish 78ba17caa8 bittorrent-dht: register a pattern for protocol detection 
Removes the port based probing which takes a long time to register for
ports 1024-65535 and instead use pattern based protocol detection.
 2 years ago
Jason Ish 350c0723d7 bittorrent-dht: set parser to unidirectional 
This parser does not match up responses with requests so flag it as
unidirectional.
 2 years ago
Jason Ish 06eaec67ac bittorrent: updates for new event handling 
Fixes anomaly events.
 2 years ago
Aaron Bungay 86037885a9 bittorrent-dht: add bittorrent-dht app layer 
Parses and logs the bittorrent-dht protocol.

Note: Includes some compilation fixups after rebase by Jason Ish.

Feature: #3086
 2 years ago
Haleema Khan 8d5c5f24a1 dns/eve: add 'HTTPS' type logging 
Add a new DNS record type to represent HTTPS
Ticket: #4751
 2 years ago
Alice Akaki ccdc992a71 rust: fix lint warnings about mixed case hex literals 
Ticket: #4593
 2 years ago
Gabriel Lima Luz 4e90d17fd9 rust: fix lint warnings about if same then else 
Ticket: 4609
 2 years ago
Kristina Jefferson 9cd00424c3 rust: fix lint warnings about ptr_arg 
Ticket: #4599
 2 years ago
Jason Ish 21bb697bc9 rust: fix clippy lint for unneeded late initialization 2 years ago
Jason Ish 7cca238128 rust: fix clippy lint for cmp_null is debug code 
Ticket: 5577
 2 years ago
Jason Ish f0952aef0d rust: fix clippy lints for unneeded reference in debug code 2 years ago
Jason Ish 6a7439a26b rust: fix clippy lints for is_empty in debug code 2 years ago
Jason Ish 560c4ea125 rust: don't allow fixed up clippy lints 2 years ago
Jason Ish 6db85d6f89 rust: clippy fix for bitwise or with 0 2 years ago
Jason Ish 04f0ee0151 rust: fix clippy lints for clippy::unnecessary_cast 2 years ago
Jason Ish b6cc0e25b1 rust: fix clippy lints for clippy::redundant_static_lifetimes 2 years ago
Jason Ish 13db83274b rust: fix clippy lints for clippy::redundant_pattern_matching 2 years ago
Jason Ish 7ba1d3e300 rust: fix clippy lints for clippy::nonminimal_bool 2 years ago
Jason Ish 6ba0a67143 rust: fix clippy lints for clippy::map_flatten 2 years ago
Jason Ish 7ebdfa539a rust: fix clippy lints for clippy::manual_find 2 years ago
Jason Ish 119e02cf81 rust: fix clippy lints for clippy::collapsible_if 2 years ago
Jason Ish 572505870a rust: fix clippy lints for clippy::collapsible_else_if 2 years ago
Jason Ish 6b71d69356 rust: fix clippy lints for clippy::bool_comparison 2 years ago
Jason Ish e373d9f5e0 rust: fix clippy lints for clippy::crate_in_macro_def 2 years ago
Jason Ish 565da0d0af rust: fix clippy lints for clippy::redundant_field_names 2 years ago
Jason Ish 5f7ba03e63 rust: fix clippy lints for clippy::needless_bool 2 years ago
Jason Ish 3ec435a703 rust: fix clippy lints for clippy::manual_range_contains 2 years ago
Jason Ish f342d4aacd rust: fix clippy lints for clippy::len_zero 2 years ago
Jason Ish 5e5401d3e9 rust: fix clippy lints for clippy::char_lit_as_u8 2 years ago
Jason Ish 29a4a7fddc rust: fix clippy lints for clippy::assign_op_pattern 2 years ago
Jason Ish c4034dafa1 rust: fix clippy lints for clippy::derive_partial_eq_without_eq 2 years ago
Jason Ish 5a10fcd2d8 rust: suppress large enum variant lint at location 
And disable the global lint.
 2 years ago
Jason Ish 74b7522b6a rust/http2: box decompressor variants 
These variants, in particular the Brotli one can be large at over 2500
bytes which is allocated no matter which decompressor is being used.

Gzip comes in at over 500 bytes.  Box deflate for consistency.
 2 years ago
Jason Ish 36f8ada305 rust: remove clippy lints that no longer warn 2 years ago
Jason Ish e8c00dd980 rust: sort clippy allow statements 2 years ago
Haleema Khan 6c922e0b98 rust: fix lint warning for clippy::enum's name 
Ticket: #4597
 2 years ago
Jason Ish 2a42386c28 rust: fix clippy lint for null comparison 
Use .is_null() instead of checking for equality against
std::ptr::null().
 2 years ago
Jason Ish 45dfea2497 rust/modbus: derive default instead of manual impl 
Cleans up a clippy lint for a trivial default impl that can be derived.
 2 years ago
Jason Ish 9218da0eb8 rust/frames: cleanup clippy lint for unsafe 
Where possible mark the relevant functions unsafe.  Otherwise suppress
the warning for now as this pattern is supposed to be a safe API around
an unsafe one. Might need some further investigation, but in general the
"guarantee" here is provided from the C side.
 2 years ago
Jason Ish 105d9a5f02 rust: fix clippy lint for unnecessary_unwrap 
Avoid check if not none followed by unwrap.
 2 years ago
Jason Ish 85cfa7254b rust: fix clippy lint for single_char_add_str 
Idiomatic cleanup and a fix automatically done by `cargo clippy --fix`.
 2 years ago
Jason Ish f3e4bcfe23 rust: fix clippy lint for bool_assert_comparison 
Checking for is_empty is faster than checking for equality.
 2 years ago
Jason Ish f60e1b30f6 rust: fix clippy lint for partialeq_to_none 
Use .is_some() and .is_none() instead of comparing against None.
Comparing against None requires a value to impl PartialEq, is_none() and
is_some() do not and are more idiomatic.
 2 years ago
Jason Ish 7d623f0854 rust: fix clippy lint for explicit_auto_deref 
This adds unnecessary complexity to code.
 2 years ago
Jason Ish c503ca62e2 rust: fix clippy lint for needless_late_init 2 years ago
Jason Ish 94dd85baed rust: fix clippy lint for borrow_deref_ref 
This type of borrow then reference has no effect.
 2 years ago
Jason Ish e9597f3d0c rust: fix clippy lint for redundant_closure 
Removes a closure where the function can be directly provided.
 2 years ago
Jason Ish c5b26e2043 rust: fix clippy ling for needless borrows 
Cleanup needless borrows found by clippy. This fix done automatically by
`cargo clippy --fix`.
 2 years ago
Jason Ish 63b3d73ccc rust: allow some more clippy lints 
Allow these lints for now until some more investigation can be done, as
--fix attempts to fix these.
 2 years ago
Eric Leblond a9519778de rust/smb: avoid allocation in smb status function 
Avoid an allocation by returning a static string.
 2 years ago
Eric Leblond 9cb06d4376 detect/smb: add smb.ntlmssp_domain keyword 
Feature #5411.
 2 years ago
Eric Leblond 5debb86cd5 rust/smb1: add a missing command 2 years ago
Eric Leblond 69ef1bc194 detect/smb: add smb.ntlmssp_user keyword 
Feature #5411.
 2 years ago
Eric Leblond f46f895e8d rust/smb: import NT status code for Microsoft doc 
This patch updates the NT status code definition to use the status
definition used on Microsoft documentation website. A first python
script is building JSON object with code definition.

```
import json
from bs4 import BeautifulSoup
import requests

ntstatus = requests.get('https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55')

ntstatus_parsed = BeautifulSoup(ntstatus.text, 'html.parser')

ntstatus_parsed = ntstatus_parsed.find('tbody')

ntstatus_dict = {}

for item in ntstatus_parsed.find_all('tr'):
    cell = item.find_all('td')
    if len(cell) == 0:
        continue
    code = cell[0].find_all('p')
    description_ps = cell[1].find_all('p')
    description_list = []
    if len(description_ps):
        for desc in description_ps:
            if not desc.string is None:
                description_list.append(desc.string.replace('\n ', ''))
    else:
        description_list = ['Description not available']
    if not code[0].string.lower() in ntstatus_dict:
        ntstatus_dict[code[0].string.lower()] = {"text": code[1].string, "desc": ' '.join(description_list)}

print(json.dumps(ntstatus_dict))
```

The second one is generating the code that is ready to be inserted into the
source file:

```
import json

ntstatus_file = open('ntstatus.json', 'r')

ntstatus = json.loads(ntstatus_file.read())

declaration_format = 'pub const SMB_NT%s:%su32 = %s;\n'
resolution_format = '        SMB_NT%s%s=> "%s",\n'

declaration = ""
resolution = ""

text_max = len(max([ntstatus[x]['text'] for x in ntstatus.keys()], key=len))

for code in ntstatus.keys():
    text = ntstatus[code]['text']
    text_spaces = ' ' * (4 + text_max - len(text))
    declaration += declaration_format % (text, text_spaces, code)
    resolution += resolution_format % (text, text_spaces, text)

print(declaration)
print('\n')
print('''
pub fn smb_ntstatus_string(c: u32) -> String {
    match c {
''')
print(resolution)
print('''
        _ => { return (c).to_string(); },
    }.to_string()
}
''')
```

Bug #5412.
 2 years ago
Victor Julien db0f9ddc69 files/tx: inspection, logging and loop optimizations 
Introduce AppLayerTxData::file_tx as direction(s) indicator for transactions.
When set to 0, its not a file tx and it will not be considered for file
inspection, logging and housekeeping tasks.

Various tx loop optimizations in housekeeping and output.

Update the "file capable" app-layers to set the fields based on their
directional file support as well as on the traffic.
 2 years ago
Victor Julien 79499e4769 app-layer: move files into transactions 
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
 2 years ago
Victor Julien 01e64d80da app-layer: trunc parser per direction 2 years ago
Victor Julien ff9d1807f9 app-layer: parser flags to u16 2 years ago
Victor Julien c27df6304d app-layer: introduce common AppLayerStateData API 
Add per state structure for storing flags and other variables.
 2 years ago
Victor Julien 879a46f085 rust: lock to time 0.3.13 to avoid MSRV bump to 1.59 
Indirect dependency through x509-parser.
 2 years ago
Pierre Chifflier 16db04c1a7 rust: remove nom 5 dependency 2 years ago
Pierre Chifflier 0acf75bff7 rust/applayertemplate: convert to nom7 2 years ago
Pierre Chifflier 378e915846 rust/asn1: convert parsers to nom7 2 years ago
Pierre Chifflier 0ba0572c4a rust/x509: finish transition to nom7 2 years ago
Pierre Chifflier 3ef5121ab0 rust/telnet: convert parsers to nom7 2 years ago
Pierre Chifflier d98b386f36 rust/conf: convert parser to nom7 2 years ago
Pierre Chifflier db9a1e17b6 rust/ssh: finish transition to nom7 2 years ago
Pierre Chifflier b31c72c06a rust/rdp: convert parsers to nom7 2 years ago
Pierre Chifflier 49520b2143 rust/rdp: upgrade dependency on tls-parser 2 years ago
Pierre Chifflier beadd090b8 rust: upgrade versions of BER/DER, Kerberos and SNMP parsers 2 years ago
Jason Ish baab1de735 rust: update x509-parser to 0.14.0 
Resolves RustSec issues in time and chrono:
- https://rustsec.org/advisories/RUSTSEC-2020-0071
- https://rustsec.org/advisories/RUSTSEC-2020-0159

Ticket: #5259.

Ammended by Victor Julien to bump to 0.14 instead of 0.13.
 2 years ago
Pierre Chifflier 3aace49649 rust/x509: update dependency on x509-parser 2 years ago
Jeff Lucovsky ab4d0f7f4a detect/stream_size: Rename detect.rs to stream_size.rs 
This commit renames detect.rs to stream_size.rs to reflect its content.
 2 years ago
Jeff Lucovsky 2386f2614f detect/iprep: Move iprep logic into a separate module 2 years ago
Jeff Lucovsky c957882d1c detect/uri: Move uri logic into a separate module 2 years ago
Jeff Lucovsky 484c34bc60 detect/uint: Move uint logic into a separate module 
This commit moves the uint logic into its own module.
 2 years ago
Jeff Lucovsky 8bfe427a74 rust/detect: Create detect module for rule parsing 
This commit creates a module named "detect" for rule parsing logic. As
part of this commit, detect.rs is moved from its toplevel position into
the new module. Thus, use crate::detect::detect to refer to items within
detect.rs (instead of create::detect).

Ticket: 5077
 2 years ago
Jeff Lucovsky ccd1063e43 detect/bytemath: convert parser to Rust 
Issue: 5077

This commit
- Converts the PCRE based parser to Rust.
- Adds unit tests to the new Rust modules
- Removes the PCRE parser from detect-bytemath.c
- Adjusts the C source modules to refer to the Rust definitions
- Includes the multiply operator (missing from the C parser)
 2 years ago
Philippe Antoine af40873127 pgsql: config limit maximum number of live transactions 
As is done for other protocols

Ticket: #5527
 2 years ago
Philippe Antoine e160917bcf mqtt: remove quadratic time complexity 
When having many transactions in a single parsing call...

Fix has overhead of having one more field in the mqtt state.

Completes commit a8079dc978

Ticket: #5399
 2 years ago
Philippe Antoine 5ef259722b dhcp: adds renewal-time keyword 
Ticket: #5507
 3 years ago
Philippe Antoine 6faf6299e0 dhcp: adds rebinding-time keyword 
Ticket: #5506
 3 years ago
Philippe Antoine 95f0424423 nfs4: fix write record handling 
Ticket: #5280
 3 years ago
Philippe Antoine bf43011a43 dcerpc: convert transaction list to vecdeque for UDP 
As was done for TCP in dfe76bb90 and d745d28d4

Ticket: #5518
 3 years ago
Eric Leblond 2cc9152fc9 rust/smb: log uuid of interface in dcerpc 
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.

Feature #5413.
 3 years ago
Eric Leblond b6f1cf255c rust/smb/dcerpc: parse context id 
As context id is used to know to which variant of the endpoint the
request is done, it is interesting to parse it.

Feature #5413.
 3 years ago
Philippe Antoine 11f849c3ee protocol-change: sets event in case of failure 
Protocol change can fail if one protocol change is already
occuring.

Ticket: #5509
 3 years ago
Philippe Antoine e94920b49f smb: do not use tree id to match create request and response 
As an SMB2 async response does not have a tree id, even if
the request has it.

Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.

Ticket: #5508
 3 years ago
Philippe Antoine 461725a9bf dhcp: adds leasetime keyword 
As it is logged

Ticket: #5435
 3 years ago
Philippe Antoine 9b4a133777 http2: remove to_vec for comparisons 
Ticket: #5454
 3 years ago
Philippe Antoine d011b468da http2: fix clippy warning about &Vec<u8> 
Using &[u8] instead in function prototype
 3 years ago
Philippe Antoine 3de735ae70 ike: log ikev1 tx fields instead of state ones 
As state fields can grow abitrarily, and this can lead to DOS
by quadratic complexity (CPU time and disk space)

Adds a direction field to retain all the information in the
transaction.

Also checks array vendor_ids had at least one element before
logging it.

Ticket: #5455
 3 years ago
Philippe Antoine d0171d7418 ike: rustfmt 3 years ago
Philippe Antoine 5c7b5c5fb5 krb: detection for ticket encryption 
As is done for logging.

Ticket: #5442
 3 years ago
Philippe Antoine 64b2385c64 krb: log for ticket encryption 
Also logs if the ticket encryption is weak.
It is different from the encryption used for the rest of the
packet, and this allows to detect kerberoasting attack.

Ticket: #5442
 3 years ago
Philippe Antoine 7fcc6696cb krb: rustfmt kerberos.rs 3 years ago
Philippe Antoine 675de33405 krb: bump up crate version 
kerberos parser crate is also used by other procotols : nfs and
smb. These protocols use an older der_parser crate version.
Upgrading der_parser will simplify the code further.
 3 years ago
Philippe Antoine 783dff2c38 krb: rustfmt detect.rs 3 years ago
Jason Ish c862e84c01 rust/frames: cleanups 
- Implement the Display trait on Direction to print "toserver" or
  "toclient" which used in a format string.

- Use Direction struct inside Frame instead of a u32.  Requires a helper
  method as there are two representation in C for direction, and the C
  methods for frames don't use the internal representation of the
  Direction enum (some sweeping changes could help here)
 3 years ago
Jason Ish f92708b8ca rust/frames: derive direction from StreamSlice 
On the Rust side, a Frame requires a StreamSlice to be created. We can
derive the direction from the StreamSlice removing the need for callers
to provide the direction when operating on the frame.
 3 years ago
Jason Ish b39d7f46e7 dns/tests: fix StreamSlice to satisfy debug validation 3 years ago
Philippe Antoine f3b6fd3329 quic: update to nom7 3 years ago
Philippe Antoine 95125811b8 quic: reassemble crypto frames and parse it 3 years ago
Philippe Antoine f242fb7f22 quic: events and rules on them 3 years ago
Philippe Antoine b9c1d9e86b quic: parse gquic version Q039 
Ticket: #5166
 3 years ago
Philippe Antoine 018fef5ef8 quic: ja3 computation and logging and detection 
Logging as is done in TLS.

Detection using the generic generic ja3.string keyword

Ticket: #5143
 3 years ago
Philippe Antoine c6cf61a39b quic: complete parsing of initial for non gquic 
The format of initial packet for quic ietf, ie quic v1,
is described in rfc 9000, section 17.2.2

Parse more frames and logs interesting extensions from crypto frame

Do not try to parse encrypted data, ie after we have seen
a crypto frame in each direction.

Use sni from crypto frame with tls for detection already implemented

Ticket: #4967
 3 years ago
Philippe Antoine 7044131c39 quic: rustfmt 3 years ago
Philippe Antoine 0c346af4a9 rust: bump up digest crates 
so that we can use hkdf crate for quic
 3 years ago
Philippe Antoine 2294e9cdbc rdp: bump up tls-parser crate version 
so that we can use new functions in quic parser
 3 years ago
Philippe Antoine 11e0eb9c89 quic: do not log empty cyu array 
Ticket: #5167
 3 years ago
Philippe Antoine 632581ac95 ike: do not log empty notify array 
Ticket: #5167
 3 years ago
Philippe Antoine 262a93ce18 mqtt: do not log reason_codes if there is none 
Ticket: #5167
 3 years ago
Philippe Antoine 1621f5e453 detect/nfs: use inclusive ranges 3 years ago
Philippe Antoine ed6955ee98 detect: use generic integer functions for iprep 
Ticket: #4112
 3 years ago
Philippe Antoine cfb60d0fce detect: use generic integer functions for urilen 
Ticket: #4112
 3 years ago
Philippe Antoine c57052181c snmp: rustfmt detect.rs 3 years ago
Philippe Antoine c7214be99b snmp: adds usm keyword 
as is logged

Ticker: #5416
 3 years ago
Philippe Antoine eb1c2a6083 smb: use default stream-depth 0 by default 
As broken by commit e5c948df87

Ticket: #5390
 3 years ago
Philippe Antoine c585be338c nfs: fix arbitrary allocation 
Bug introduced by https://github.com/OISF/suricata/pull/7111

Nom's count begins by allocating a Vector, which leads to arbitrary
allocation due to flavors_cnt coming from network, and not even
being checked against i.len()

Ticket: #5237
 3 years ago
Philippe Antoine 26dc70648c dns: remove unused events field from state 
found overflowing by oss-fuzz
 3 years ago
Philippe Antoine d1a4dae36b detect: use generic integer functions for streamsize 
By the way, adds the prefilter feature

Ticket: #2697
Ticket: #4112
 3 years ago
Philippe Antoine 35b6dcec7e detect: use generic integer functions for filesize 
Ticket: #4112
 3 years ago
Philippe Antoine f29b43defd detect: rust generic functions for integers 
Move it away from http2 to generic core crate.
And use it for DCERPC (and SMB)

And remove the C version.
Main change in API is the free function is not free itself, but
a rust wrapper around unbox.

Ticket: #4112
 3 years ago
Philippe Antoine c4d9cb02ec util: better hex print function 
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
 3 years ago
Philippe Antoine 6058792bee rust: make suricata context const 
So that it is read only and its pointers do not get modified
 3 years ago
Philippe Antoine 6224e283fa modbus: bump up rust crate version 
So that probing parser is more strict and does not accept unknown
function code as valid modbus.

Ticket: #5377
 3 years ago
Philippe Antoine 2d761810db rust: cbindgen first verifies existing bindings 
So as not to recompile every C file inclusing rust.h
 3 years ago
Juliana Fajardini 6ccc01a79c rust: fix doc comments that trigger rust warnings 
Rust generates warnings that are treated as errors for documentation
blocks before `extern` blocks.
 3 years ago
Philippe Antoine d745d28d4a dcerpc: use vecdeque tx iterator 
Ticket: #5321
 3 years ago
Jason Ish dfe76bb905 dcerpc: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5271
 3 years ago
Jason Ish 8790968281 mqtt, rdp: fix copyright dates 3 years ago
Philippe Antoine c78722a671 rust: RustParser same fields as AppLayerParser 
So that there is no problem when crossing FFI
 3 years ago
Sam Muhammed 323fe1c1ac nfs3/records: Fix typo 
Fix response_lookup unittest name
 3 years ago
Jason Ish b8b6a17a5b dns: add pdu frame 
Adds a PDU frame to the DNS parser. For UDP this is the DNS payload
portion of the DNS packet, for TCP this is the payload minus the leading
legth field.

Ticket: 4984
 3 years ago
Jason Ish 8d1840f595 frames(rust): don't call into C if running Rust unit tests 
Wrap the calls behind frames to C code if a `cfg!(not(test))` so they
don't get compiled when running Rust unit tests.  Linkage to C functions
is not yet available for Rust unit tests, and this will keep the check
out of individual parsers.

Ticket: 4984
 3 years ago
Jason Ish c74ea3840d frames (rust): method to create StreamSlice from slice 
Useful in unit test for function that require a StreamSlice.
 3 years ago
Jason Ish d712a8b29d eve/dns: remove dns v1 logging 
Removal of DNS v1 logging was scheduled to be removed in May 2022.

Ticket: #4157
 3 years ago
Jason Ish e319d31c14 template(rust): convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5298
 3 years ago
Jason Ish 9b0b2beac1 pgsql: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5297
 3 years ago
Jason Ish 2db84726ad http2: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5296
 3 years ago
Jason Ish 4e0ad5e0bd rdp: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5295
 3 years ago
Jason Ish 3b422e25f3 mqtt: convert transaction list to vecdeque 
Allows for more efficient removal from front.

Ticket: #5294
 3 years ago
Jason Ish 3189414788 dns: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5277
 3 years ago
Jason Ish 7b11b4d3a1 app-layer: more generic state trait 
Instead of a method that is required to return a slice of transactions,
use 2 methods, one to return the number of transactions in the
collection, and another to get a transaction by its index in the
collection.

This allows for the transaction collection to not be a contiguous array
and instead can be a VecDeque, or possibly another collection type that
supports retrieval by index.

Ticket #5278
 3 years ago
Juliana Fajardini bbd9a2ff1a pgsql: apply clippy fixes 3 years ago
Juliana Fajardini 0fc9ade7a9 pgsql: fix uint overflow and optimizations 
Fuzzers found a possible integer overflow bug when parsing response
messages. To fix that, removed the case where we incremented the parsed
field length and created a new message type for situations where Suri
parsers an Unknown message. This is good because there may happen that
an unknown message to Suri is valid, and in this case, we would still be
able to log it.

Philippe Antoine found the bug while fuzzing with rust debug assertions.

Bug #5016
 3 years ago
Juliana Fajardini 8daa79513a pgsql: fix clippy is_null usage warning 3 years ago
Philippe Antoine d2f00ac824 dcerpc: use wrappingadd for padding parsing 
As we compute a modulo, we can safely wrap around even if there
is an overflow

Ticket: #5301
 3 years ago
Philippe Antoine 464ff80c6a smb: protocol detection on pattern without midstream 
To recognize a protocol, Suricata first looks for
patterns, which can be confirmed by a probing parser.
If this does not work, Suricata can try to run
some probing parsers on some ports.

This is the case for SMB.

This commit makes handling the confirming and the probing
paser differently even if they share much code.

The confirmation parser knows that a pattern has been found.
So, it must not do the midstream case of looking for this
pattern in the whole buffer, but only check it at the beginning.
But it must reverse direction if needed.
 3 years ago
Victor Julien dc57460427 smb: fix event types for limit exceeded rules 3 years ago
Jason Ish 0623ada24d dns: better error handling when parsing names 
The DNS name parser will error out with an error even if the
error is incomplete. Instead of manually generating errors,
use '?' to let the nom error ripple up the error handling chain.

The reason this wasn't done in the first place is this code
predates the ? operator, or we were not aware of it at the time.

This prevents the case where probing fails when there is enough data to
parse the header, but not enough to complete name parser. In such a case
a parse error is returned (instead of incomplete) resulting in the
payload not being detected as DNS.

Ticket #5034
 3 years ago
Jason Ish 27679a12aa dns: don't parse a full request during probe if not enough data 
If there is more data than a header, but not enough for a complete DNS
message, the hostname parser could return an error causing the probe to
fail on valid DNS messages.

So only parse the complete message if we have enough input data. This is
reliable for TCP as DNS messages are prefixed, but for UDP its just
going to be the size of the input buffer presented to the parser, so
incomplete could still happen.

Ticket #5034
 3 years ago
Victor Julien fc9b65d8d3 smb2: validate negotiate read/write max sizes 
Raise event if they exceed the configured limit.
 3 years ago
Victor Julien 4be8334c9e smb2: allow limiting in-flight data size/cnt 
Allow limiting in-flight out or order data chunks per size or count.

Implemented for read and writes separately:

app-layer.protocols.smb.max-write-queue-size
app-layer.protocols.smb.max-write-queue-cnt
app-layer.protocols.smb.max-read-queue-size
app-layer.protocols.smb.max-read-queue-cnt
 3 years ago
Victor Julien 2c5ad8858e filetracker: track total queued data (in_flight) 
As well as expose number of chunks.
 3 years ago
Victor Julien 90d4b8e438 smb: log max read/write sizes 3 years ago
Victor Julien 5bcc4162f7 smb2: add options for max read/write size 
Add options for the max read/write size accepted by the parser.
 3 years ago
Victor Julien f28888513a smb2: track max read/write size and enforce its values 3 years ago
Victor Julien 594acec5dc smb: minor function cleanup 
Remove used argument from `filetracker_newchunk()`. We're not
using fill_bytes with smb.
 3 years ago
Victor Julien c7a474c725 filetracker: make FileChunk private 3 years ago
Philippe Antoine 3b13008c1b mqtt: fix consumed bytes computation for truncated msg 
Ticket: 5268
 3 years ago
Philippe Antoine 704bc878ea dcerpc: store consumed_bytes as i32 
As it can grow bigger than u16
 3 years ago
Philippe Antoine dfd17e9acc ike: fix integer underflow in parse_proposal 
By not restricting a usize to i16
 3 years ago
Victor Julien 93d5bce0aa rust: update regex & memchr dependencies 
Bug: #5260.
 3 years ago
Victor Julien 053a9d2e68 smb/ntlmssp: add stricter len/offset validation 3 years ago
Philippe Antoine 3e48881b78 smb: prevents integer underflow 
Ticket: 5246

If msg_id is 0, we cannot find the previous request
 3 years ago
Philippe Antoine e72036f12f smb: ntlmssp domain_blob_offset underflow check 
Ticket: 5246
 3 years ago
Philippe Antoine 817a5001a5 smb: check on param parsing 
Ticket: 5246

so as not to overflow u16
 3 years ago
Sascha Steinbiss 7eb279ac53 mqtt: remove redundant "where" keyword 3 years ago
Sascha Steinbiss d63e5b8c51 mqtt: make some functions non-public 3 years ago
Sascha Steinbiss 2a3ed9a6ae mqtt: rustfmt 3 years ago
Sascha Steinbiss 1ba62993d5 mqtt: raise event on parse error 3 years ago
Sascha Steinbiss 5618273ef4 mqtt: ensure we do not request extra data after buffering 
This addresses Redmine bug #5018 by ensuring that the parser
never requests additional data via the Incomplete error, but to
raise an actual parse error, since it is supposed to have all
the data as specified by the message length in the header already.
 3 years ago
Victor Julien 6d30f4442c http2: fix file accounting for ranged files 
Increment files_opened for tx that 'gets' reassembled ranged file
 3 years ago
Victor Julien b336882008 smb1: apply close to direction 
Instead of closing files in both direction when receiving a close request,
close only toserver files for the request and close toclient on receiving
a response.
 3 years ago
Victor Julien b9cd502249 smb: convert 'close' parser to function 3 years ago
Sam Muhammed 3a490fb16c nfs: Implement frames 
Feature #4872

Frames:
  - RPC Frames: Generic over TCP/UDP
     - rpc.pdu
     - rpc.hdr
     - rpc.data
     - rpc.creds -- for rpc calls

  - NFSv2, NFSv3
     - nfs.pdu
     - nfs.status -- for nfs responses

  - NFSv4 Only Frames
     - nfs4.pdu
     - nfs4.hdr
     - nfs4.ops -- for compound request/response operations
     - nfs4.status -- for nfs4 responses

RPC tcp/udp frames created with separate registeration functions e.g:
add_rpc_tcp_tc_frames()
add_rpc_udp_tc_frames()
 3 years ago
Sam Muhammed d090dcbce9 rpc: Improve rpc_record struct 
Add creds_len field to rpc_record
needed for rpc.creds frame length calculation
 3 years ago