It turned out that having global (interface-specific) mempool
that is shared by the threads of the interface is slower than
having individual mempools per queue for each interface.
The commit brings this change and should be user-invisible,
the config setting remains still as a number of objects of
all mempools summed (of that interface).
Ticket: 7382
This can happen when the flow is UDP, but an ICMP unreachable is
returned, which gets assigned to the same flow.
Reference: https://github.com/ntop/nDPI/issues/2762
Moves the nDPI documentation to an nDPI page in the plugins
section. Remove the duplication of installation and setup
documentation.
Includes some minor cleanups.
Split DetectHelperKeywordRegister into 2 functions, one for acquiring
a new keyword ID, and another to perform the registration.
This makes it easier to do the traditional C keyword initialization
with a dynamic ID.
Reduce per tx space for tracking detection/prefilter progress. Instead
of a per direction u64 of flags, where each bit reflected a progress
value, use a simple u8 to track the linear progression through the
progress values. Use an offset to allow 0 to mean no value.
Add flags field as well to track "skip detect" and "inspect complete".
Cache Hyperscan serialized databases to disk to prevent compilation
of the same databases when Suricata is run again with the same
ruleset.
Hyperscan binary files are stored per rulegroup in the designated
folder, by default in the cached library folder.
Since caching is per signature group heads,
some chunk of the ruleset can change and it still can reuse part of
the unchanged signature groups.
Loading *fresh* ET Open ruleset: 19 seconds
Loading *cached* ET Open ruleset: 07 seconds
Ticket: 7170
This variant of hashlittle2() ensures that it avoids
accesses beyond the last byte of the string, which will
cause warnings from tools like Valgrind or Address
Sanitizer.
The buffer-size value that controls file output buffering defaults to
8k. To be consistent with previous logic, the default is being changed
to 0 (e.g., needed if there are old config files that don't specifically
enable the new value).
such as ja4.
Why ?
We do not want to see hard-coded protocol constants such as
ALPROTO_QUIC directly used in generic code in detect-parse.c
How ?
From the keyword point of view, this commit adds the function
DetectSignatureSetMultiAppProto which is similar to
DetectSignatureSetAppProto but takes multiple alprotos.
It restricts the signature alprotos to a set of possible alprotos
and errors out if the interstion gets empty.
The data structure SignatureInitData gets extended with
a fixed-length array, as the use case is a sparse number of protocols
Ticket: 7304
There was an implicit limit of 32 app-layer protocols
used by probing parsers through a mask, meaning that
Suricata should not support more than 32 app-layer protocols
in total.
This limit is relaxed to each flow not being able to
run more than 32 probing parsers, meaning that for each source
and destination port combination, the sum of registered
probing parsers should not exceed 32, even if there are more
than 32 in total.
Also sets probing parsers done sooner in the case the other
side of the connection was detected first.
Ticket: 7437
Methods:
`get` creates the flow object.
`id` returns the flow id.
`has_alerts` returns a boolean indicating if the flow triggered alerts.
`app_layer_proto` returns various app-layer related fields as 5 strings:
alproto, alproto_ts, alproto_tc, alproto_orig, alproto_expect.
`stats` returns cnts for bytes and packets to sever and to client, as 4
numbers.
`tuple` -- returns various fields: srcip, dstip, proto, sp, dp.
`timestamps` returns time as 4 numbers: seconds and microseconds, for
first and last packet of the flow.
`timestring_legacy` returns the first packet from the flow's timestring
as a string (like fastlog).
`timestring_iso8601` returns the first packet from the flow's
timestring as an iso8601 compat timestring (like eve).
Example:
```
name = "lua-scflowstats.log"
local flow = require("suricata.flow")
function init(args)
local needs = {}
needs["type"] = "flow"
return needs
end
function setup(args)
filename = SCLogPath() .. "/" .. name
file = assert(io.open(filename, "a"))
SCLogInfo("lua SCFlowStats Log Filename " .. filename)
end
function log(args)
local f = flow.get()
timestring = f:timestring_legacy()
tscnt, tsbytes, tccnt, tcbytes = f:stats()
file:write ("[**] " .. timestring .. "\nSCFlowStats is\nPacket count to server: " .. tscnt .. "\nByte count to server: " .. tsbytes .. "\nPacket count to client: " .. tccnt .. "\nByte count to client: " .. tcbytes .. "\n[**]")
file:flush()
end
function deinit(args)
file:close(file)
end
```
Task #7489
Lukas Sismis
Philippe Antoine
Jason Ish
Alfredo Cardigliano
Alice Akaki
Victor Julien
Shivani Bhardwaj
Jeff Lucovsky
Philippe Antoine
Juliana Fajardini