Commit Graph

47 Commits (22f3205664f28fb382b979f9ddff5695b8ea66e4)

Author SHA1 Message Date
Victor Julien 22f3205664 var-names: expose outside of detect engine
Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.

For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.

This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.

The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.

For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.

After the threads have reloaded and the new detection engine is active,
the old table can be freed.

For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.

This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
9 years ago
Victor Julien 775e182531 detect: remove AppLayerMatch API call 9 years ago
Victor Julien d9a300cd8c detect: move lua smtp support to dynamic list 9 years ago
Victor Julien a10b2fdecf detect: make ssh detection use dynamic list 9 years ago
Victor Julien 4ae4fd0802 lua: use tls_generic list for ssl/tls 9 years ago
Victor Julien cfdd934aba dnp3: dynamic buffers/lists 9 years ago
Victor Julien d9b3ae6cd6 dns: use dynamic buffers 9 years ago
Victor Julien 6f867c3c73 http_raw_uri: dynamic buffer 9 years ago
Victor Julien ee55aefa1c http_client_body: dynamic buffer 9 years ago
Victor Julien e7d5e845c7 http_header / http_raw_header: dynamic buffers 9 years ago
Victor Julien 67b7d9734e http_cookie: dynamic buffer 9 years ago
Victor Julien 54604c7bf2 http_user_agent: dynamic buffer 9 years ago
Victor Julien 6346a074a7 http_uri: dynamic buffer
Clean up tests
9 years ago
Victor Julien 6bd37611ee file_data: dynamic buffer 9 years ago
Victor Julien 7052f9b933 http_request_line: dynamic buffer 9 years ago
Victor Julien 8edc954e82 detect: get rid of Signature::sm_lists
Instead use the lists in init_data during setup and the SigMatchData
arrays during runtime.
9 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 9 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 9 years ago
Victor Julien 3f8ee2afd3 detect-lua: unify on using 'lua' name vs 'luajit' 9 years ago
Victor Julien 0366d47608 luajit: remove unused instance counter 9 years ago
Victor Julien 3da7dad514 lua: luajit improvements
Luajit has a strange memory requirement, it's 'states' need to be in the
first 2G of the process' memory.

This patch improves the pool approach by moving it to the front of the
start up.

A new config option 'luajit.states' is added to control how many states
are preallocated. It defaults to 128.

Add a warning when more states are used then preallocated. This may fail
if flow/stream/detect engines use a lot of memory. Add hint at exit that
gives the max states in use if it's higher than the default.
9 years ago
Jason Ish a59f31a99f DNP3: Lua detect support.
Adds support for access the DNP3 transaction in Lua rules.
9 years ago
Victor Julien 4c98b6cef3 http_request_line: implement keyword and mpm
Implemented as 'stickybuffer'.

Move all logic into the keyword file and remove bad tests that tested
URI instead of request line.
9 years ago
Giuseppe Longo 675fa56497 app-layer: add ThreadVars to AppLayerParserParse
To be able to add a transaction counter we will need a ThreadVars
in the AppLayerParserParse function.
This function is massively used in unittests
and this result in an long commit.
9 years ago
Victor Julien 6530c3d0d8 unittests: replace SCMutex* calls by FLOWLOCK_* 9 years ago
Jason Ish 8865009fca lua: remove flow locking from the lua layer 9 years ago
Victor Julien da8f9c1896 lua: add smtp for detection 9 years ago
Victor Julien 408948815f detect: simplify flow locking
To simplify locking, move all locking out of the individual detect
code. Instead at the start of detection lock the flow, and at the
end of detection unlock it.

The lua code can be called without a lock still (from the output
code paths), so still pass around a lock hint to take care of this.
9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
9 years ago
Nicolas Thill 3c9538b783 lua: set thread vars in DetectLuaMatch
Fix internal error when calling SCThreadInfo from Lua detection modules.

Signed-off-by: Nicolas Thill <ntl@p1sec.com>
10 years ago
Torgeir Natvig af6d1dddfa Bugfix for detect-engine.luajit-states
detect-engine is a list, and luajit-states was looked up as a map.
10 years ago
Mats Klepsland 7eab16c478 lua: SSH support
Support SSH in lua detection scripts (Feature #1569).
10 years ago
Victor Julien 371648a8c6 lua: TLS support
Support TLS in Lua detection scripts.

function init (args)
    local needs = {}
    needs["tls"] = tostring(true)
    return needs
end

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    if version == nil then
        return 0
    end
    str = string.format("Version %s\nIssuer %s\nSubject %s\nFingerprint %s",
                        version, issuer, subject, fingerprint)
    SCLogInfo(str);
    return 1
end
10 years ago
Victor Julien 1e84950017 detect-lua: set direction 10 years ago
Victor Julien 4dd53c8f03 lua: dns support 10 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 10 years ago
Victor Julien 21f9328989 lua: fix error handling 10 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
10 years ago
Victor Julien dea8dda3c1 vars: redo var name <-> idx handling
Can't use sm type anymore as multiple var carriers (host/flow/etc) will
all have xbits.
10 years ago
Victor Julien 7108085d33 detect: initialize detection engine by prefix
Initalize detection engine by configuration prefix.

    DetectEngineCtxInitWithPrefix(const char *prefix)

Takes the detection engine configuration from:
<prefix>.<config>

If prefix is NULL the regular config will be used.

Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
10 years ago
Victor Julien 6723d03c7e http: add inspection engine for http request line
No MPM though.
10 years ago
Ken Steele 923a77e952 Change Match() function to take const SigMatchCtx*
The Match functions don't need a pointer to the SigMatch object, just the
context pointer contained inside, so pass the Context to the Match function
rather than the SigMatch object. This allows for further optimization.

Change SigMatch->ctx to have type SigMatchCtx* rather than void* for better
type checking. This requires adding type casts when using or assigning it.

The SigMatch contex should not be changed by the Match() funciton, so pass it
as a const SigMatchCtx*.
11 years ago
Eric Leblond 667b9a5220 lua: add export of dns.rrname
Add the capability for a lua script to ask for rrname in DNS query.
11 years ago
Victor Julien 944276b988 lua detect: expose stream payload
Allow a script to set the 'stream' buffer type. This will add the
script to the PMATCH list.

Example script:
alert tcp any any -> any any (content:"html"; lua:stream.lua; sid:1;)

    function init (args)
        local needs = {}
        needs["stream"] = tostring(true)
        return needs
    end

    -- return match via table
    function match(args)
        local result = {}

        b = tostring(args["stream"])
        o = tostring(args["offset"])

        bo = string.sub(b, o);
        print (bo)

        return result
    end

    return 0
11 years ago
Victor Julien c152ddf072 lua: fix http.request_line inspection
As there is no inspection engine for request_line, the sigmatch was
added to the AMATCH list. However, no AppLayerMatch function for
lua scripts was defined.

This patch defines a AppLayerMatch function.

Bug #1273.
11 years ago
Victor Julien 22272f6c5b lua: export packet keywords to detect scripts
Set packet pointer, so it's available to the lua keywords that
require it.
11 years ago
Victor Julien f23399d672 Rename Lua code to just Lua
As we support regular Lua as well as LuaJIT, it makes more sense to call
it all Lua.
11 years ago