Remove the half finished support for accepting a table returned from a
Lua rule's match function. This is not documented, not tested, and not
really implemented.
Also, use lua_tointeger to get the return value from the match function
as an integer instead of a float.
Ticket: #6941
Move the SCThreadInfo global function into the suricata.util library as
thread_info().
This is the last global function to be registered, so remove the
supporting functions.
Similar to flowvars and flowints, but a byte var cannot be registered
from a Lua script, but it still needs to be setup. Instead provide an
"map" function that sets it up, or errors out if the byte var is
unknown.
This also required passing the signature into the Lua init method, as
the state of the Signature object and the time of loading the Lua
keyword is required.
Adds a new lua script capability to use a script as a buffer transform
keyword.
It uses a `transform` lua function that returns the input buffer after
modifying it.
Issue: 2290
New Lua lib, "suricata.flowvar" for working with flowvars from Lua.
Replaces functions:
- SCFlowvarGet (and ScFlowvarGet)
- SCFlowvarSet (and SCFlowvarSet)
Of note, the DetectLuaData has been made available to the init and
thread_init methods, instead of just the match. This is due to an
issue that if a flow variable is not registered in init, it will not
be logged, registering in thread_init is too late.
Ticket: #7486
For some rules the values are always 0, for others they get set, so not
consistent. Also not documented and not tested and should be replaced by
a "suricata.rule" lib.
For example, these globals would be set for this rule:
alert tcp any any -> any any (flow:to_server;
flowbits:isnotset,dataset_added; lua:dataset.lua;
flowbits:set,dataset_added;sid:1;rev:99;)
But not this rule:
alert dns any any -> any any (msg:"TEST DNS LUA dns.rrname"; \
content:"org"; lua:test-rrname.lua; sid:1; rev:1;)
Related to ticket
https://redmine.openinfosecfoundation.org/issues/7490 (but not a fix)
This does not affect the functions with similar names for getting rule
details in output context.
In addition to
function init (args)
local needs = {}
needs["dnp3"] = tostring(true)
return needs
end
allow for
function init (args)
return {dnp3=true}
end
with the idea that the former will be completely removed for 8.0.
This works with all existing SV tests and as a string value, any value
is considered "truthy".
Update the Lua allocated to set a code on memory allocation limit
exceeded errors so an appropriate error message can be logged and a
state incremented.
Fixes the tracking of the allocated size by using the difference
between original size, and new size and toss in some debug
validations.
Distinguish between a generic Lua script error and an error created by a
function being blocked, so each is logged once respective of each other.
Also add a stat that is incremented when a script fails due to a
blocked function.
NOTE: This does not catch calls to functions that are blocked by not
having the library loaded, such as "io.open", as they are blocked by
not even loading the "io" library.
The Lua library surface area is small enough to manage an allow list,
which is generally better than a deny list, as we'll explicitly need
to opt-in to new functions provided by the Lua runtime.
If a rule script crashed, the return value was treated as a no
match. This would make a negation of the rule match and alert.
Instead cleanup and exit early if the rule script crashed and don't
run negation logic.
A stat, detect.lua.errors has been added to count how many times a
script crashes.
Also consolidates the running of the Lua script and return value
handling to a common function.
Bug: #6940
Rename DetectAppLayerInspectEngineRegister2 to
DetectAppLayerInspectEngineRegister as there is no other variant of
this function, and the versioning with lack of supporting
documentation can lead to confusion.
Ticket: #6104
And failures should be handled to say that the rule failed to load
Reverts the fix by 299ee6ed55
that was simple, but not complete (memory leak),
to have this bigger API change which simplifies code.
Implement a new design for handling var name id's. The old logic
was aware of detection engine versions and generally didn't work
well for multi-tenancy cases. Other than memory leaks and crashes,
logging of var names worked or failed based on which tenant was
loaded last.
This patch implements a new approach, where there is a global store
of vars and their id's for the lifetime of the program.
Overall Design:
Base Store: "base"
Used during keyword registration. Operates under lock. Base is shared
between all detect engines, detect engine versions and tenants.
Each variable name is ref counted.
During the freeing of a detect engine / tenant, unregistration decreases
the ref cnt.
Base has both a string to id and a id to string hash table. String to
id is used during parsing/registration. id to string during unregistration.
Active Store Pointer (atomic)
The "active" store atomic pointer points to the active lookup store. The call
to `VarNameStoreActivate` will build a new lookup store and hot swap
the pointer.
Ensuring memory safety. During the hot swap, the pointer is replaced, so
any new call to the lookup functions will automatically use the new store.
This leaves the case of any lookup happening concurrently with the pointer
swap. For this case we add the old store to a free list. It gets a timestamp
before which it cannot be freed.
Free List
The free list contains old stores that are waiting to get removed. They
contain a timestamp that is checked before they are freed.
Bug: #6044.
Bug: #6201.
To protect against possible supply chain attacks, disable Lua rules by
default. They can be enabled under the "security" section of
suricata.yaml.
Ticket: #6122
Jason Ish
Philippe Antoine
Jeff Lucovsky
Victor Julien
Jo Johnson
Gabriel Lima Luz