Some compiler/platform combinations don't like creating one const from
another as it can't guarantee the other is defined, resulting in the
following compile error:
output-json-tls.c:102:5: error: initializer element is not constant
BASIC_FIELDS |
^~~~~~~~~~~~
Fixes commit 377989df6c
Introduce a common function for mapping names to IDs that performs
bounds checking.
Note: For event IDs in the enum that are larger than a uint8_t, -1
will be returned instead of -4. -4 has special meaning during
signature parsin that means requirements were not met. -4 has no
special handling prior to requirements, or the meaning has been lost.
During EVE TLS setup, a broken check for Ja3 being enabled led to Ja3
being disabled, but only in custom mode. This check is not needed, if
Ja3 is disabled, it won't be available, and won't be logged.
This is required to implement "extended" in terms of "custom" fields.
The profiling arrays are incorrectly sized by the number of thread
modules. Since they contain app-layer protocol data, they should be
sized by ALPROTO_MAX.
This was done following the fact that this setting was historically
named incorrectly. The purpose of the setting was always to define the
ports that will be prioritized and have rule groups associated w them on
priority. Rename all occurences of this to correctly reflect the purpose
of the setting.
Threads are only set to paused upon initialization and never again, we
should only have to wait once, so move the wait before any loop that
was waiting before.
Additionally, if the thread was killed while waiting to be unpaused,
don't enter the loop.
The pattern of checking the pause flag, setting to paused then
waiting to unpause was done enough times to factor out into its own
function. This is also needed by library users who bring their own
packet acquisition threads.
Commit changes are made to avoid possible memory leaks. If the parser
is initialized before configuration file checking, there was no deinit
call before function return. Do check config file existance and type
before YAML parser initialization, so we don't need to deinit parser
before exiting the function.
Bug: #7302
Detect engine tenant reloading function hasn't got engine release call
under error label, so it is possible memory leak in case of errors in
further new detect engine initialization.
Bug: #7303
Remove the call to SCDropCaps for packet processing threads. This
logic in this function is required to setup packet processing even
when the thread is provided by a library user, in which case Suricata
should not be touching is capabilities.
As SCDropCaps is currently a no-op its clear this feature needs to
be (re)designed properly, taking into consideration library users as
well.
Related ticket: https://redmine.openinfosecfoundation.org/issues/2375
Instead of inheriting from flow, because encrypted protocols like TLS
and SSH may have just set the flow flags to indicate rest of stream is
encrypted and does not need to run stream inspection. But inspection
still needs to be run detection on this last flushing packet.
Ticket: #7235.
Ticket: 7191
So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
If the ACK packet completing the 3whs is received, the stream engine will
transition to "established". However, the packet itself will not be tagged
as "established". This will only happen for the next packet after the 3whs,
so that `flow:established` only matches after the 3whs.
It is possible that the ACK packet completing the 3whs was lost. Since the
ACK packets themselves are not acknowledged, there will be no retransmission
of them. Instead, the next packet can have the expected ACK flag as well as
data.
This case was mishandled in a subtle way. The stream engine state transition
was done correctly, as well as the data handling and app-layer updates.
However, the packet itself was not tagged as "established", which meant
that `flow:established` would not yet match.
This patch detects this case and tags the packet as established if ACK
with data is received that completes the 3whs.
Bug: #7264.
Current GetBlock degrees the sbb search from rb tree to
line, which costs much cpu time, and could be replaced by
SBB_RB_FIND_INCLUSIVE. It reduces time complexity from
O(nlogn) to O(logn).
Ticket: 7208.
Without any mode setting, the test would take up the mode to be 0 which
used to be the relax mode for base64 decoder in C. However, there was no
code corresponding to that mode and it was never used so nothing
happened when this test was run.
Add an explicit strict mode as per the expectation of the test from its
comments.
Add a pure rust base64 decoder. This supports 3 modes of operation just
like the C decoder as follows.
1. RFC 2045
2. RFC 4648
3. Strict
One notable change is that "strict" mode is carried out by the rust
base64 crate instead of native Rust. This crate was already used for
encoding in a few places like datasets of string type. As a part of this
mode, now, only the strings that can be reliably converted back are
decoded.
The decoder fn is available to C via FFI.
Bug 6280
Ticket 7065
Ticket 7058
Per the notes for strtoul, since 0 or ULONG_MAX is a legitimate return
value, errno must be cleared before the call so an error can be checked
after the call by testing errno.
Issue: 7126
Issue: 7126
This commit abandons the use of StringParseUint32 which generates an
error message of there are non-numeric characters.
The SMTP parser had used this function on the packet path; this commit
uses strtoul instead.
An example of the content causing the error message to be emitted:
3460 LAST
If a stream-only rule matches, and we find a tx where we
want to log the app-layer data, store into the tx data that
we already logged, so that we do not log again the app-layer metadata
Ticket: 7085
CID 1619284: Memory - illegal accesses (OVERRUN)
In ParseAFXDPConfig, a pointer to bool is cast into a pointer
to int.
Also removing the cast pattern when useless
As it is not always the first frame to be created :
if it is not enabled, it does not get created, and other enabled
frames may be created first. see use of FrameConfigTypeIsEnabled
This resulted that this other frame got its length updated
on stream end, which led to false positives.
Resorting on FRAME_STREAM_TYPE is more consistent.
Ticket: 7213
The `app-layer-protocol` keyword inconsistently checks whether the
alproto is ALPROTO_UNKNOWN. In the regular match function it isn't
checked, in the prefilter function its checked for all but the "either"
mode.
This leads to false positives for negated matching, as an expression
like "!tls" will match if checked against ALPROTO_UNKNOWN.
This patch adds the checking everywhere. The keyword returns no match as
long as the alproto is ALPROTO_UNKNOWN.
Bug: #7241.
Remove the ThreadExitPrintStats callback, this is the only logger that
was actually using it, and this logger is marked for deprecation. This
allows us to remove the callback from the registration signature.
Ticket: #7227
Remove the callback to print stats on thread exit. The counter value
was never being used and this helps us get rid of this callback
altogether as their is only one other usage of it.
Ticket: #7227
Using OutputCtx results in the low level output-tx packet logger being
aware of Suricata's higher level loggers that use OutputCtx, for the
low level logger this is purely opaque data that may not be an
OutputCtx for custom loggers.
Ticket: #7227
Rename OutputRegisterFlowLogger to SCOutputRegisterFlowLogger and
document in the header file.
Mark other functions in the header file as part of the internal API.
Ticket: #7227
Rename OutputRegisterPacketLogger to SCOutputRegisterPacketLogger as
its part of the public API and document its parameters.
Comment on the other functions in the header that they are part of the
internal API.
Ticket: #7227
Loggers need an ID uniquely identify them for profiling. To help with
loggers registered at runtime (library, plugins), provide a
LOGGER_USER that can be used. It won't provide per-logger details if
they have more than one, but will provide a total for all their
registered loggers.
Ticket: #7227
The use of OutputCtx as the data type for initdata was leaking Eve
submodule logic into the low level packet logger. Instead use void *,
as the packet logging module is not concerned with the type of data
here.
Also document this initdata parameter.
Ticket: #7227
A library/plugin user wanting to register a custom flow logger must
include "output-flow.h", however that depends on some other includes.
One train of thought with respect to include files in libraries, is
that they should include all their dependencies on behalf of the
user. To make a custom flow logger just a little easier, include
"flow.h" and "decode.h".
Ticket: #7227
The use of OutputCtx as the data type for initdata was leaking Eve
submodule logic into the low level flow logger. Instead use void *, as
the flow logging module is not concerned with the type of data here.
Also document this initdata parameter.
Ticket: #7227
The initdata argument to OutputFlowThreadInit was always NULL, remove
it. Internally the ThreadInit functions still get initdata, but this
is the data provided when that logging instance was registered.
Ticket: #7227
The lookup function was not taking into account that we can have
an IPv4 or an IPv6 address as parameters and that this addresses
need to be converted to Suricata internal storage.
By using the already defined dedicated parsing function, we are
fixing the issue.
Issue: #6969
As it fails to work correctly on FreeBSD and OpenBSD.
On FreeBSD, these are the errors:
Info: pcap: Pcap-file will use 4096 buffer size [PcapFileGlobalInit:source-pcap-file.c:159]
Error: pcap: failed to get first packet timestamp. pcap_next_ex(): -2 [PeekFirstPacketTimestamp:source-pcap-file-helper.c:186]
Warning: pcap: Failed to init pcap file input.pcap, skipping [ReceivePcapFileThreadInit:source-pcap-file.c:299]
Error: pcap: pcap file reader thread failed to initialize [ReceivePcapFileLoop:source-pcap-file.c:185]
This introduces a new parser registration function for LDAP/UDP, and update
ldap configuration in order to be able to enable/disable a single parser
independently (such as dns).
Also, GAPs are accepted only for TCP parser and not for UDP.
Ticket #7203
Ticket: 4863
On the way, convert some keywords to use the first-class integer
support.
And helpers for pure rust the support for multi-buffer.
Move the C unit tests about keyword mqtt.protocol_version
to unit tests for generic integer parsing, and test version 5
instead of testing twice version 3.
Also iterate all tx's messages for reason code as is done for other
keywords.
And allow detection on empty topics.
Ticket: 7172
When parsing an integer for a rule keyword fails, we return error
straight away, without bothering to try to free the NULL pointer.
On the way, remove some one-line wrapper around DetectUxParse
Victor Julien
Jason Ish
Adam Kiripolsky
Nancy Enos
Alice Akaki
Sascha Steinbiss
Jeff Lucovsky
Juliana Fajardini
Shivani Bhardwaj
Philippe Antoine
Zemeteri Kamimizu
Alexey Simakov
Ilya Bakhtin
Noah Liu
Giuseppe Longo
Jeff Lucovsky
Eric Leblond
Lukas Sismis
mmaatuq