- remove "rs_" prefix from functions that are not exported
- prefix exported functions with "SC"
- don't export functions that are only used by pointer
Ticket: 7498
Both the macros export_tx_data_get and export_state_data_get can
generate non-pub functions as the function they generate is only used
as a pointer during registration.
Remove "pub" and "no_mangle" from the generated functions and update
the names of the generated functions to follow Rust rules as they are
no longer exported into the global C namespace.
Ticket: 7498
Ticket: 7495
We want to finish also if we tested all the expected protocols
in mask, or if we tested even more.
There can be one more protocol coming from pe0, which can be
the protocol already found in the other direction.
This commit removes the `_AL_` usage in detect keywords for improved
readability.
Some of the HTTP rule keywords already had counterparts without using
"_AL_". These rule keywords are the legacy content modifier keywords
that now have sticky buffer equivalents.
For these, "_AL_" was removed and a suffix was added to the #define:
src/detect-engine-register.h:151: DETECT_HTTP_COOKIE_CM
src/detect-engine-register.h:153: DETECT_HTTP_METHOD_CM
src/detect-engine-register.h:161: DETECT_HTTP_HEADER_CM
src/detect-engine-register.h:173: DETECT_HTTP_RAW_HEADER_CM
src/detect-engine-register.h:175: DETECT_HTTP_URI_CM
src/detect-engine-register.h:179: DETECT_HTTP_STAT_MSG_CM
src/detect-engine-register.h:181: DETECT_HTTP_STAT_CODE_CM
src/detect-engine-register.h:185: DETECT_HTTP_HOST_CM
Ticket: 5053
The names are now dynamically registered at runtime.
The AppProto alproto enum identifiers are still static for now.
This is the final step before app-layer plugins.
Because some alprotos will remain static and defined as a constant,
such as ALPROTO_UNKNOWN=0, or ALPROTO_FAILED.
The regular already used protocols keep for now their static
identifier such as ALPROTO_SNMP, but this could be made more
dynamic in a later commit.
ALPROTO_FAILED was used in comparison and these needed to change to use
either ALPROTO_MAX or use standard function AppProtoIsValid
This is mainly for header sanitization to avoid pulling in detect
modules into the Lua sandbox definition.
Plus if we namespace modules with names like "suricata.dataset", it
probably makes sense to keep those modules in their own files.
Re-work the Lua dataset lib to be required into a user script like:
local dataset = require("suricata.data")
The main difference from loading it into global space is providing a
custom require function (as we removed it in the sandbox) and load it on
demand, returning a table to the module.
dataset.new
create a dataset object in lua
<dataset>:get
gets a reference to an existing dataset
<dataset>:add
returns 1 if a new entry was added
returns 0 if entry was already in the set
Example:
```
function init (args)
local needs = {}
needs["packet"] = tostring(true)
return needs
end
function thread_init (args)
conn_new, dataset.new()
ret, err conn_new:get("conn-seen")
if err ~= nil then
SCLogWarning("dataset warning: " .. err)
return 0
end
end
function match (args)
ipver, srcip, dstip, proto, sp, dp = SCFlowTuple()
str = ipver .. ":<" .. srcip .. ">:<" .. dstip .. ">:" .. dp
ret, err = conn_new:add(str, #str);
if ret == 1 then
SCLogInfo(str .. " => " .. ret)
end
return ret
end
```
Ticket: #7243.
The fix for issue 7447 introduced an error with threaded eve output.
The changes that were committed for that issue mishandled the return
value when a file is being opened for the 2nd or higher time.
Instead of returning the existing file context, null was returned.
Currently, the syntax includes direction as a part of the keyword which
is against how usually keywords are done. By making direction as a
mandatory argument, it is possible to make the syntax cleaner and the
implementation more compact and easily extendable.
Pros:
- Registration table sees lesser entries if newer options are added
- If the options have to be extended, it can be done trivially
- In accordance w existing keyword implementations
Note that this commit also retains the existing direction specific
keywords.
Since forever (1578ef1e3e) a valid RST
would update the internal `last_ack` representation to include all
unack'd data. This was originally done to make sure the unACK'd data was
inspected/processed at flow timeout.
It was observed however, that if GAPs existed in this unACK'd data, a
GAP could be reported in the stats and a GAP event would be raised. This
doesn't make sense, as missing segments in the unACK'd part of the
stream are completely normal. Segments simply do not all arrive in
order.
It turns out that the original behavior of updating `last_ack` to
include all unACK'd data is no longer needed.
For raw stream inspection, the detection engine will already include the
unACK'd data on flow end.
For app-layer updates the unACK'd data is often harmful, as the data
often has GAPs. Parser like the http parser would report these GAPs and
could also get confused about the post-GAP data being a new transaction
including a file. This lead to many reported errors and fantom txs and
files.
Since the GAP detection uses `last_ack` to determine GAPs, not moving
`last_ack` addresses the GAP false positives.
Ticket: #7422.
Some checks can be done w/o holding a lock:
- seeing if the flow matches the packet
- if the hash row needs a timeout check
This patch skips taking a lock in these conditions.
Since `Thread` objects are part of a big allocation, more than one
Thread could be on a single cache line, leading to false sharing. Atomic
updates to one `Thread` could then lead to poor performance accessing
another `Thread`. Align to CLS (cache line size) to avoid this.
Until now many accesses to the Thread structure required taking a global
lock, leading to performance issues. In practice this only happened in
offline mode.
This patch adds a finer grained locking scheme. It assumes that the
Thread object itself cannot disappear, and adds a spinlock to protect
updates to the structure.
Additionally, the `pktts` field is made an atomic, so that it can be
read w/o taking the spinlock. Updates to it are still done under lock.
Jason Ish
Eric Leblond
Philippe Antoine
Jeff Lucovsky
Alice Akaki
Victor Julien
Shivani Bhardwaj