src/runmode-dpdk.c:202:11: warning: Size of pointer 'argv' used instead of size of its data. This is likely to lead to a buffer overflow. You probably intend to write 'sizeof(*argv)'. [pointerSize]
args->argv = SCCalloc(capacity, sizeof(args->argv));
^
src/runmode-dpdk.c:777:23: error: Shifting 32-bit value by 63 bits is undefined behaviour [shiftTooManyBits]
if (bits & (1 << i))
^
src/runmode-dpdk.c:776:23: note: Assuming that condition 'i<64' is not redundant
for (int i = 0; i < 64; i++) {
^
src/runmode-dpdk.c:777:23: note: Shift
if (bits & (1 << i))
^
The check meant to see if the ip address part of the ip/cidr combo
was more specific than needed wasn't fully implemented, leading to
warnings being issued on completely valid and correct input.
This patch implements the same logic as in IPv4. If the ip address
as specified is different from the ip after the mask has been applied,
a warning is displayed.
Bug: #5747.
Remove the distinction between the C template protocol "template" and
the Rust template protocol "template-rust" and make the Rust parser
simply template now that we no longer have support to generate a C
protocol template.
Remove the app-layer-PROTO stub for Rust based parsers. It is no longer
needed as Rust parsers now contain the registration function in Rust.
Ticket: 4939
AF_XDP support is a recent technology introduced that aims at improving
capture performance. With this update, Suricata now provides a new
capture source 'af-xdp' that attaches an eBPF program to the network
interface card. Packets received in the NIC queue are forwarded to
a RX ring in user-space, bypassing the Linux network stack.
Note, there is a configuration option (force-xdp-mode) that forces the
packet through the normal Linux network stack.
libxdp and libbpf is required for this feature and is compile time
configured.
This capture source operates on single and multi-queue NIC's via
suricata.yaml. Here, various features can be enabled, disabled
or edited as required by the use case.
This feature currently only supports receiving packets via AF_XDP,
no TX support has been developed.
Ticket: https://redmine.openinfosecfoundation.org/issues/3306
Additional reading:
https://www.kernel.org/doc/html/latest/networking/af_xdp.html
Use the lzma-rs crate for decompressing swf/lzma files instead of
the lzma decompressor in libhtp. This decouples suricata from libhtp
except for actual http parsing, and means libhtp no longer has to
export a lzma decompression interface.
Ticket: #5638
In lossy streams, esp where TcpSession::lossy_be_liberal it is possible
to end up inserting a segment that is out of the expected sequence
number bounds.
While updating the destination pointer, we were also adding the padded
bytes which are not a part of the decoded bytes. This led to running out
of space on the destination buffer.
Fix it by only incrementing destination buffer ptr by the number of
actual bytes that were decoded.
Ticket 5623
Suricata already logs if AF_PACKET or Netmap are running in a mixed IPS
and IDS/TAP mode. As the behavior is undefined when these modes are
mixed, it is best to deprecate and to not allow this behavior. For now
warn that it will be unsupported and fail in Suricata 8.
Ticket: 5587
Instead of returning on the first line that fails to parse, log the
error and continue instead of returning.
The fail fast makes sense in test mode, but not in a normal run mode
where you don't want one bad line to abort processing the whole file.
This will still fail out in test mode.
Related issue: 4554
Issue: 2982
This commit moves bsize validation with respect to content matches to
the post-parse validation stage. This allows bsize to consider all
content-related values, including those that follow the bsize keyword.
This commit adds validation of the bsize value(s) with the available
buffer size. Signatures are flagged if the bsize and buffer size are
incompatible.
Issue: 3682
Ticket: 2982
This commit validates that the content usage in a rule will not exceed
the dsize value.
Values of distance that cause the right edge to be exceeded are
considered an error and the signature will be rejected.
Various parts of ASAN will require creation of threads. At least
LSAN reporting and ASAN stack traces require this. Detect if we
run with ASAN and bypass the noproc setting with a warning.
Bug: #5661.
Update DNP3 to work with a single TX logger, and just register one
logger instead of 2.
This primarily creates a TX per message instead of correlating replies
to requests, which fits the DNP3 model better, but we didn't really have
this concept nailed down when DNP3 was written.
with setrlimit NPROC.
So that, if Suricata wants to execve or such to create a new process
the OS will forbid it so that RCE exploits are more painful to write.
Ticket: #5373
This is to avoid the tx logging code that doesn't support LoggerId
values over 31 at this time. The simplest fix for now is to just have
all JSON (eve) loggers use the same ID.
DNP3 is left as-is for now as it needs some extra support in the parser.
Each module (thread) updates its status to indicate running.
Main thread awaits for all threads to be in a running state
before continuing the initialisation process
Implements feature 5384
(https://redmine.openinfosecfoundation.org/issues/5384)
This patch introduce the IPv4 type for dataset so Suricata commandmatch
on a set of IPv4 addresses. This is meant to complement iprep feature
for people that needs more flexibility such as settings the IP on
the packet path.
Feature: #5383
This patch adds a dataset-dump command to the list of unix socket
commands. Implementation is not optimal as we are locking the
datasets when doing the dump. But if we consider that the current
alternative from an implementation point of view is to stop Suricata
then this is far better than current state.
Ticket: #5184
Lock is only held to update/check ints, so spin lock will be more
efficient.
Place the member of Packet in a new "persistent" area to make it
clear this is not touched by the PacketReinit logic.
Ticket: #5592.
Each thread had its own version of the `machine_start_time`, which
lead to slight time differences. This became apparent mostly in IPS,
where 2 threads each process a side of the flow.
This patch makes the `machine_start_time` global.
Update DROP action handling in tunnel packets. DROP/REJECT action is set
to outer (root) and inner packet.
Check action flags both against outer (root) and inner packet.
Remove PACKET_SET_ACTION macro. Replace with RESET for the one reset usecase.
The reason to remove is to make the logic easier to understand.
Reduce scope of RESET macros.
Rename PacketTestAction to PacketCheckAction except in unittests. Keep
PacketTestAction as a wrapper around PacketCheckAction. This makes it
easier to trace the action handling in the real code.
Fix rate_filter setting actions directly.
General code cleanups.
Bug: #5571.
As per RFC 4648,
Implementations MUST reject the encoded data if it contains characters
outside the base alphabet when interpreting base-encoded data, unless
the specification referring to this document explicitly states
otherwise.
Add a new mode BASE64_MODE_RFC4648, and handle input strictly as per the
specification.
Bug 5223
When looking at an alert event, it was impossible to determine which
side from src or dest IP in the alert was the client and wich side
was the server with regards to the underlying flow. This was a problem
when you try to known who belongs a metadata property such as a HTTP
hostname or a TLS JA3.
This patch updates the code to add src and dest IP in the flow
subobject as well as src and dst port. This way, we can now which
side is the client and which side is the server.
The result is looking like:
{
"event_type": "alert",
"src_ip": "22.47.184.196",
"src_port": 81,
"dest_ip": "192.168.1.47",
"dest_port": 1063,
"proto": "TCP",
"tx_id": 0,
"alert": {
"signature_id": 2018959,
"rev": 3,
},
"app_proto": "http",
"flow": {
"pkts_toserver": 22,
"pkts_toclient": 35,
"bytes_toserver": 1370,
"bytes_toclient": 48852,
"start": "2009-10-28T10:01:46.755232+0100",
"src_ip": "192.168.1.47",
"dest_ip": "22.47.184.196",
"src_port": 1063,
"dest_port": 81
}
}
Add a command to extract the accounting data from a live
flow using the unix socket. It takes the flow_id as param
and return the volume of data seen on the flow as well as
its age.
Previous method was truncating the flow hash value when building
the flow_id. It is interesting not to loose the flow hash value
as it can be used in other tools or to interact with a flow that
is still active.
Worker threads are responsible for final processing of timed out flows.
These are selected by the Flow Manager and inserted into a per thread
queue. The Flow Worker then checks this queue after each packet. Due to
the burstiness of this process, the packet threads would sometimes process
a lot of these flows in the context of a single packet, leading to spike
in latency which might cause packet loss.
This patch changes the behavior to only process at max 2 flows per packet.
This way added processing cost is amortized over many packets.
Add thread local cache to avoid locking overhead for ssns and segments.
A thread will return segments/ssns to a local cache first, and if that
is full, to a return queue where the actual return to the pool returns
a batch, to amortize locking overhead.
Adds segment and session pool/cache counters to see where how effective
the cache is.
Track packets that updated the app-layer, and for those run
the transaction housekeeping and output-tx logging loops.
Do the same of end of flow packets.
This skips needless iterations over the transaction stores.
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
Introduce AppLayerTxData::file_tx as direction(s) indicator for transactions.
When set to 0, its not a file tx and it will not be considered for file
inspection, logging and housekeeping tasks.
Various tx loop optimizations in housekeeping and output.
Update the "file capable" app-layers to set the fields based on their
directional file support as well as on the traffic.
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
Issue: 5077
This commit
- Converts the PCRE based parser to Rust.
- Adds unit tests to the new Rust modules
- Removes the PCRE parser from detect-bytemath.c
- Adjusts the C source modules to refer to the Rust definitions
- Includes the multiply operator (missing from the C parser)
Make sure to exit the parser early on incomplete header data.
Additionally, make sure to not create duplicated tls frames in this
case.
Add a debug validation check for the header parser parsing too much
data, which should never happen.
Improve unknown record handling. Inspired by Wireshark 'unknown record'
handling, we take a best effort approach for records with unknown content
types in TLS versions 1.0, 1.1 and 1.2.
Improve record length check and set 'invalid_record_length' event instead
of 'invalid_tls_header'.
Parse client cerificates and store them in the state similar to how
this is done for server certificates.
Update "progress" handling to not consider the TLS handshake complete
if the server indicated a client cert was needed.
TCP Buffering is now done in the app-layer using the incomplete API, on
the SSL/TLS record level. TLS level fragmentation will be implemented
separately.
To avoid overhead of stream buffering for records we don't do
much with anyway, pass through application records instead of
buffering the entire record in the stream engine.
The TLS record header is parsed in streaming mode still, but once the
record size is known we tell the app-layer API to give us the full
record.
Ticket: #5481
My review of the FreeBSD kernel code reveals that this setting
a) is ignored by the kernel b) is not required. The sending
side of divert(4) never checks so->so_options, but always gives
IP_ALLOWBROADCAST to ip_output().