Commit Graph

12802 Commits (2fb89f1b79562ca788693c72d63ca0efd39e0ac5)

Author SHA1 Message Date
Jason Ish 15c4eb3d16 threads: move wait for unpause outside of loop
Threads are only set to paused upon initialization and never again, we
should only have to wait once, so move the wait before any loop that
was waiting before.

Additionally, if the thread was killed while waiting to be unpaused,
don't enter the loop.
2 years ago
Jason Ish 7492fb6a91 threads: merge unpause test into wait for pause function
TmThreadTestThreadUnPaused was only being used by
TmThreadsWaitForUnpause and is still enough to just become one
function.
2 years ago
Jason Ish 3f8c3698db threads: helper function TmThreadsWaitForUnpause
The pattern of checking the pause flag, setting to paused then
waiting to unpause was done enough times to factor out into its own
function. This is also needed by library users who bring their own
packet acquisition threads.
2 years ago
Philippe Antoine 96c8470cdd template: move detect keywords to pure rust
Ticket: 3195

Also remove unused src/tests/detect-template-buffer.c

Completes commit 4a7567b3f0
to remove references to template-rust
2 years ago
Zemeteri Kamimizu 87e6e9374f conf: init parser after check with stat()
Commit changes are made to avoid possible memory leaks. If the parser
is initialized before configuration file checking, there was no deinit
call before function return. Do check config file existance and type
before YAML parser initialization, so we don't need to deinit parser
before exiting the function.

Bug: #7302
2 years ago
Jeff Lucovsky d32b28c85e napatech: bring back command line argument
Re-introduce support for command line argument "--napatech"

Issue: 7165
2 years ago
Jeff Lucovsky 330c408930 napatech: remove, to make room for plugin
Issue: 7166
2 years ago
Philippe Antoine 58556b7f8b template: remove -rust references
Ticket: 7315

Completes commit 4a7567b3f0

Allows keyword template.buffer to work properly when template
protocol is enabled
2 years ago
Jeff Lucovsky c9e4524b3c misc: Eliminate compiler warnings
Issue: 7314

Fixup macro usages to eliminate compiler warnings.
2 years ago
Philippe Antoine 76527dde79 http: fix condition check
Ticket: 7309

Do not use a constant expression in a condition
2 years ago
Alice Akaki d5dd549bcd detect/address: convert ipv4 unittests to FAIL/PASS
Ticket: OISF#6318
2 years ago
Philippe Antoine 40e97423b4 transform/base64: check for 0-sized buffer
So as to avoid undefined behavior with a 0-sized variable length
array

Ticket: #7296
2 years ago
Zemeteri Kamimizu adcac9ee0f detect: add new_de_ctx release in case of errors in initialization
Detect engine tenant reloading function hasn't got engine release call
under error label, so it is possible memory leak in case of errors in
further new detect engine initialization.

Bug: #7303
2 years ago
Jason Ish d8f73c9215 tm-modules: minor code cleanups
- includes
- loops
2 years ago
Jason Ish 415c5786de threading: remove TM_FLAG_LOGAPI_TM as its not used
This flag is never set, remove.
2 years ago
Shivani Bhardwaj 501f79c0ed detect/engine: deduplicate fn definitions 2 years ago
Alexey Simakov c72404e554 yaml: Add check of allocation for node object
Fix potential dereference of nullptr in case
of unsuccessful allocation of memory for
list node

Bug: #7270
2 years ago
Jason Ish 3b6f709331 threads: don't drop capabilities for packet threads
Remove the call to SCDropCaps for packet processing threads. This
logic in this function is required to setup packet processing even
when the thread is provided by a library user, in which case Suricata
should not be touching is capabilities.

As SCDropCaps is currently a no-op its clear this feature needs to
be (re)designed properly, taking into consideration library users as
well.

Related ticket: https://redmine.openinfosecfoundation.org/issues/2375
2 years ago
Ilya Bakhtin 976dec7f33 detect: pseudo-packets inherit inspect flags from parent packet
Instead of inheriting from flow, because encrypted protocols like TLS
and SSH may have just set the flow flags to indicate rest of stream is
encrypted and does not need to run stream inspection. But inspection
still needs to be run detection on this last flushing packet.

Ticket: #7235.
2 years ago
Philippe Antoine 26da953f6d util/hash: use randomized hash algorithm
For datasets and http ranges

Ticket: 7209

Prevents abusive hash collisions from known djb2 algorithm
2 years ago
Philippe Antoine bb714c9178 http: have a headers limit
Ticket: 7191

So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
2 years ago
Philippe Antoine 9203656496 defrag: fix off by one
Ticket: 7067

This off by one could lead to an empty fragment being inserted
in the rb tree, which led to integer underflow
2 years ago
Victor Julien 45eb7e4881 stream: improve 3whs completed by ACK with data
If the ACK packet completing the 3whs is received, the stream engine will
transition to "established". However, the packet itself will not be tagged
as "established". This will only happen for the next packet after the 3whs,
so that `flow:established` only matches after the 3whs.

It is possible that the ACK packet completing the 3whs was lost. Since the
ACK packets themselves are not acknowledged, there will be no retransmission
of them. Instead, the next packet can have the expected ACK flag as well as
data.

This case was mishandled in a subtle way. The stream engine state transition
was done correctly, as well as the data handling and app-layer updates.
However, the packet itself was not tagged as "established", which meant
that `flow:established` would not yet match.

This patch detects this case and tags the packet as established if ACK
with data is received that completes the 3whs.

Bug: #7264.
2 years ago
Philippe Antoine c55c7d6c27 detect/dataset: abort only in debug mode
Ticket: 7195
2 years ago
Philippe Antoine e47598110a detect/datasets: implement unset command
Ticket: 7195

Otherwise, Suricata aborted on such a rule
2 years ago
Philippe Antoine 1352ed68c7 datasets: restrict scope of macro/enum 2 years ago
Philippe Antoine 1d32f11745 ssl/ja3: better check for ja3 being enabled
Ticket: 6634

Completes commit 84735251b5

Avoids error log in Ja3BufferAddValue about NULL buffer
2 years ago
Noah Liu 951bcff970 stream/reassembly: optimize GetBlock
Current GetBlock degrees the sbb search from rb tree to
line, which costs much cpu time, and could be replaced by
SBB_RB_FIND_INCLUSIVE. It reduces time complexity from
O(nlogn) to O(logn).

Ticket: 7208.
2 years ago
Shivani Bhardwaj 6ff0f72f4d detect/base64: minor cleanups
1. decode_len can be u32 as it stores min of two u32s.
2. Add defensive check for payload_len calculation underflow.
2 years ago
Shivani Bhardwaj 86eec116eb transform/base64: add explicit mode to test
Without any mode setting, the test would take up the mode to be 0 which
used to be the relax mode for base64 decoder in C. However, there was no
code corresponding to that mode and it was never used so nothing
happened when this test was run.

Add an explicit strict mode as per the expectation of the test from its
comments.
2 years ago
Shivani Bhardwaj cbb571a61c rust/base64: add decoder
Add a pure rust base64 decoder. This supports 3 modes of operation just
like the C decoder as follows.
1. RFC 2045
2. RFC 4648
3. Strict

One notable change is that "strict" mode is carried out by the rust
base64 crate instead of native Rust. This crate was already used for
encoding in a few places like datasets of string type. As a part of this
mode, now, only the strings that can be reliably converted back are
decoded.

The decoder fn is available to C via FFI.

Bug 6280
Ticket 7065
Ticket 7058
2 years ago
Giuseppe Longo 969f4d131f sip: rustify sticky buffers
Ticket #7204
2 years ago
Jeff Lucovsky 8f107c8252 detect: Clear errno before strtoul
Per the notes for strtoul, since 0 or ULONG_MAX is a legitimate return
value, errno must be cleared before the call so an error can be checked
after the call by testing errno.

Issue: 7126
2 years ago
Jeff Lucovsky a115acf208 smtp: Prevent error messages on packet path
Issue: 7126

This commit abandons the use of StringParseUint32 which generates an
error message of there are non-numeric characters.

The SMTP parser had used this function on the packet path; this commit
uses strtoul instead.

An example of the content causing the error message to be emitted:

    3460 LAST
2 years ago
Juliana Fajardini 2c7824a41f pgsql: use new API style for extern C functions 2 years ago
Victor Julien 51392a6af1 eve/flow: turn error into debug assertion
In case this hits logging an error is harmful, better to silently
continue.

Remove use of `default` to help the compiler flag new enum additions.
2 years ago
Victor Julien b3ed752cf1 eve/flow: log flow state directly
No need to first turn it into a flags field.
2 years ago
Philippe Antoine de9413c654 detect: safety for app-layer logging of stream-only rules
If a stream-only rule matches, and we find a tx where we
want to log the app-layer data, store into the tx data that
we already logged, so that we do not log again the app-layer metadata

Ticket: 7085
2 years ago
Jeff Lucovsky 51b742738c output/json: Remove unneeded parameters
Issue: 4974

Remove unused parameters in output path for
- AlertJsonMetadata
- AlertJsonHeader
2 years ago
Jeff Lucovsky 8064847fc6 doc: Document reference config setting
Issue: 4974
2 years ago
Jeff Lucovsky 9bc468e63a alert/reference: Optionally add reference(s)
Issue: 4974

Optionally include rule references with the alert. Since there can be
multiple reference keywords, they are collected into an array.
2 years ago
Philippe Antoine 31bed10ff6 runmode: fix coverity warning
CID 1619284:  Memory - illegal accesses  (OVERRUN)

In ParseAFXDPConfig, a pointer to bool is cast into a pointer
to int.

Also removing the cast pattern when useless
2 years ago
Philippe Antoine 89d229c76f frames: do not rely on FRAME_STREAM_ID
As it is not always the first frame to be created :
if it is not enabled, it does not get created, and other enabled
frames may be created first. see use of FrameConfigTypeIsEnabled

This resulted that this other frame got its length updated
on stream end, which led to false positives.

Resorting on FRAME_STREAM_TYPE is more consistent.

Ticket: 7213
2 years ago
Victor Julien 79aa4861e0 detect/app-layer-proto: don't run detection on ALPROTO_UNKNOWN
The `app-layer-protocol` keyword inconsistently checks whether the
alproto is ALPROTO_UNKNOWN. In the regular match function it isn't
checked, in the prefilter function its checked for all but the "either"
mode.

This leads to false positives for negated matching, as an expression
like "!tls" will match if checked against ALPROTO_UNKNOWN.

This patch adds the checking everywhere. The keyword returns no match as
long as the alproto is ALPROTO_UNKNOWN.

Bug: #7241.
2 years ago
Philippe Antoine 84735251b5 tls/ja3: do not append to ja3 str once ja3 hash is computed
Ticket: 6634

That means take only the first client hello into account.
This way, we do not end with ja3 string with 9 commas...
2 years ago
Jason Ish 685baa9680 output-filedata: rename and document registration function
Prefix registration function and pointer function type with SC, as
well as document.

Ticket: #7227
2 years ago
Jason Ish b51eeb3ab5 output-file: rename and document registration function
Rename OutputRegisterFileLogger to SCOutputRegisterFileLogger, add
function documentation and include in userguide.

Ticket: #7227
2 years ago
Jason Ish 14b648f286 output-streaming: rename and document registration
Prefix the registration function and types with "SC", and add function
documentation.

Ticket: #7227
2 years ago
Jason Ish cba8527f75 output-file: use void *initdata instead of OutputCtx
Avoids leaking a higher level abstraction into a low level logger.

Ticket: #7227
2 years ago
Jason Ish 80d395efbb output-filedata: use void *initdata instead of OutputCtx
Avoids leaking a higher level abstraction into a low level logger.

Ticket: #7227
2 years ago