Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.
For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.
This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.
The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.
For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.
After the threads have reloaded and the new detection engine is active,
the old table can be freed.
For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.
This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
Luajit has a strange memory requirement, it's 'states' need to be in the
first 2G of the process' memory.
This patch improves the pool approach by moving it to the front of the
start up.
A new config option 'luajit.states' is added to control how many states
are preallocated. It defaults to 128.
Add a warning when more states are used then preallocated. This may fail
if flow/stream/detect engines use a lot of memory. Add hint at exit that
gives the max states in use if it's higher than the default.
To be able to add a transaction counter we will need a ThreadVars
in the AppLayerParserParse function.
This function is massively used in unittests
and this result in an long commit.
To simplify locking, move all locking out of the individual detect
code. Instead at the start of detection lock the flow, and at the
end of detection unlock it.
The lua code can be called without a lock still (from the output
code paths), so still pass around a lock hint to take care of this.
Support TLS in Lua detection scripts.
function init (args)
local needs = {}
needs["tls"] = tostring(true)
return needs
end
function match(args)
version, subject, issuer, fingerprint = TlsGetCertInfo();
if version == nil then
return 0
end
str = string.format("Version %s\nIssuer %s\nSubject %s\nFingerprint %s",
version, issuer, subject, fingerprint)
SCLogInfo(str);
return 1
end
This commit do a find and replace of the following:
- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*
- HSBD by FILEDATA:
sed -i 's/HSBDMATCH/FILEDATA/g' src/*
Initalize detection engine by configuration prefix.
DetectEngineCtxInitWithPrefix(const char *prefix)
Takes the detection engine configuration from:
<prefix>.<config>
If prefix is NULL the regular config will be used.
Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
The Match functions don't need a pointer to the SigMatch object, just the
context pointer contained inside, so pass the Context to the Match function
rather than the SigMatch object. This allows for further optimization.
Change SigMatch->ctx to have type SigMatchCtx* rather than void* for better
type checking. This requires adding type casts when using or assigning it.
The SigMatch contex should not be changed by the Match() funciton, so pass it
as a const SigMatchCtx*.
Allow a script to set the 'stream' buffer type. This will add the
script to the PMATCH list.
Example script:
alert tcp any any -> any any (content:"html"; lua:stream.lua; sid:1;)
function init (args)
local needs = {}
needs["stream"] = tostring(true)
return needs
end
-- return match via table
function match(args)
local result = {}
b = tostring(args["stream"])
o = tostring(args["offset"])
bo = string.sub(b, o);
print (bo)
return result
end
return 0
As there is no inspection engine for request_line, the sigmatch was
added to the AMATCH list. However, no AppLayerMatch function for
lua scripts was defined.
This patch defines a AppLayerMatch function.
Bug #1273.