Commit Graph

7793 Commits (115b3138cc4723090774f68ac5a43fb91bd971ab)
 

Author SHA1 Message Date
Mats Klepsland 115b3138cc output-json-tls: log certificate and chain
Log entire certificate and certificate chain Base64 encoded.
8 years ago
Mats Klepsland 0716199acb output-json-tls: custom tls logging 8 years ago
Jason Ish 89ba5816dc doc: update unified2 section
Remove documentation on older unified formats that have
been removed.
8 years ago
Jason Ish 82f6103149 unified2: nostamp and file rotation
Give unified2 a nostamp option which will create the file
without the timestamp suffix (like Snort's nostamp option).

Also register for rotation notification on SIGHUP so the file
will be recreated if it is removed by an external rotation
program (only when nostamp is used).
8 years ago
Jason Ish 2c01985e73 autoconf - look for stdbool.h 8 years ago
Jason Ish d8e9e1fd77 unified2: minor cleanups
- remove unused arguments and make static
8 years ago
Jason Ish c810748a60 Makefile: fix race condition in make install-full
Use recursive make for the install process so it
is executed in a predictable order.

Addresses issue:
https://redmine.openinfosecfoundation.org/issues/1470
which triggered on OSX/macOS.
8 years ago
Mats Klepsland 234a3a890b output-json-alert: print 'tunnel' JSON object if tunnel
Log src_ip, dst_ip and proto for root packet (p->root) if the
packet that triggered is inside a tunnel, as JSON object
'tunnel'. Also log recursion depth to indicate the depth of
the tunnel.
8 years ago
Mats Klepsland 7293286f2f output-json: move code to get 5-tuple to own function
Move code to get 5-tuple in JSON object to own function 'JsonFiveTuple'.
This enables this code to be reused when printing 'parent' JSON object in
output-json-alert.
8 years ago
Mats Klepsland 8c8db1cf0f output-json-alert: fix wrongful comments 8 years ago
Mats Klepsland 6a382259f8 doc: documentation for custom JSON flags in eve-log 8 years ago
Mats Klepsland 65317ba865 output-json: make JSON flags in eve-log user configurable 8 years ago
Victor Julien d445b4b5fa dce: remove commented out code 8 years ago
Victor Julien 9da7be81f0 flow-worker: clean up thread init 8 years ago
Victor Julien c859d39f4f coverity: suppress CID 1400648 8 years ago
Victor Julien 955c227127 detect-ssh: cleanup duplicate code 8 years ago
Victor Julien 2f30adb08a detect-lua: setup cleanup, fixing a potential int issue 8 years ago
Victor Julien 511e804915 detect: fix missing unlock in error path 8 years ago
Victor Julien f74eff9eac threads: address sleep under lock issue 8 years ago
Victor Julien f380871057 threads: don't sleep under lock 8 years ago
Victor Julien cc4010343d detect: add and use util func for alproto sets 8 years ago
Victor Julien c477c4370e doc: update for unix socket hostbits 8 years ago
Victor Julien 4a49260897 flowvar: shrink flowvar type by using padded space 8 years ago
Victor Julien 99517cbd53 lua: support key/value flowvars in lua 8 years ago
Victor Julien f0af133c5f flowvar: remove unused DETECT_VAR_TYPE_ALWAYS 8 years ago
Victor Julien 71607c905a doc: update unix socket 8 years ago
Victor Julien 8fde6f967f suricatasc: add/list/remove hostbit commands
Syntax:
    add-hostbit <ip> <bit name> <expire>
Example:
    add-hostbit 1.2.3.4 blacklist 3600

Syntax:
    remove-hostbit <ip> <bit name>
Example:
    remove-hostbit 1.2.3.4 blacklist

Syntax:
    list-hostbit <ip>
Example:
    list-hostbit 1.2.3.4
8 years ago
Victor Julien 88888c3d8b unix-socket: add/list/remove hostbit commands
add-hostbit adds a named hostbit with an expire time in seconds.
remove-hostbit removes hostbit by name.

add-hostbit, remove-hostbit return success or failure.

list-hostbit returns a json array of hostbits with their name and
expire time:

    {
        "message": {
            "count": 1,
            "hostbits":
                [{
                    "expire": 3222,
                    "name": "firefox-users"
                }]
        },
        "return": "OK"
    }
8 years ago
Victor Julien b6e4276792 hostbits: add list API 8 years ago
Victor Julien 996112edf5 pktvars: same name pktvars, key-value vars 8 years ago
Victor Julien 5ca4a2e6fe outputs: vars log
EVE addition called 'vars' that logs pkt/flow vars for each packet/flow.
8 years ago
Victor Julien 1a2ad059a1 eve: log pktvars/flowvars/bits/ints
Optionally logs 'vars' into alerts
8 years ago
Victor Julien 1ba8c2fe3a pcre: new way of specifying var names
Until now the way to specify a var name in pcre substring capture
into pkt and flow vars was to use the pcre named substring support:
e.g. /(?P<pkt_somename>.*)/

This had 2 drawbacks:

1. limitations of the name. The name could be max 32 chars, only have
   alphanumeric and the underscore characters. This imposed limitations
   that are not present in flowbits/ints.

2. we didn't actually use the named substrings in pcre through the
   API. We parsed the names separately. So putting the names in pcre
   would actually be wasteful.

This patch introduces a new way of mapping captures with names:

  pcre:"/(.*)/, pkt:somename";
  pcre:"/([A-z]+) ([0-9]+)/, pkt:somename,flow:anothername";

The order of the captures and the order of the names are mapped 1 on 1.
This method is no longer limited by the pcre API's naming limits. The
'flow:' and 'pkt:' prefixes indicate what the type of variable is. It's
mandatory to specify one.

The old method is still supported as well.
8 years ago
Victor Julien 0f708d427b pkt-var: abuse flowvar postmatch logic for pktvars
Flowvars were already using a temporary store in the detect thread
ctx.

Use the same facility for pktvars. The reasons are:

1. packet is not always available, e.g. when running pcre on http
   buffers.

2. setting of vars should be done post match. Until now it was also
   possible that it is done on a partial match.
8 years ago
Victor Julien 5e39486399 pkt-var: use id instead of name pointer 8 years ago
Victor Julien a0bd15a1c4 pcre: support multiple captures
Support up to 8 substring captures into pkt or flow vars.
8 years ago
Victor Julien 017b16d421 detect-pcre: small cleanups 8 years ago
Victor Julien ac42a44280 alert-debug: print flowvar/int names 8 years ago
Victor Julien e95a0c1344 alert-debug: print flowbit names from VarNameStore 8 years ago
Victor Julien 22f3205664 var-names: expose outside of detect engine
Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.

For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.

This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.

The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.

For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.

After the threads have reloaded and the new detection engine is active,
the old table can be freed.

For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.

This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
8 years ago
Victor Julien 43cc06eabe detect: use engine version instead of id
Use engine version based on global detect engine master. This is
incremented between reloads.
8 years ago
Victor Julien 920709fe6f detect: ssh_software sticky buffer 8 years ago
Victor Julien f1ab6a6153 detect: ssh_proto stickybuffer 8 years ago
Victor Julien dfac5276b8 detect: remove unused SIGMATCH_PAYLOAD flag 8 years ago
Victor Julien 14ced15e36 detect: remove unused state file flag 8 years ago
Victor Julien fa1ef158b2 detect: small API cleanup 8 years ago
Victor Julien 073fcbeb7f detect: move file hash common code 8 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 8 years ago
Victor Julien fe415ae518 detect: remove DMATCH list 8 years ago
Victor Julien 1c02cf4542 flow: remove unused Flow::de_state 8 years ago