aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,641 Commits
8 Branches
174 Tags
252 MiB
main
main-8.0.x
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.7
suricata-8.0.6
suricata-7.0.17
suricata-8.0.5
suricata-7.0.16
suricata-8.0.4
suricata-7.0.15
suricata-8.0.3
suricata-7.0.14
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '751f3eef3b'
${ noResults }
Commit Graph

13570 Commits (751f3eef3b9ae9ee262b7507646880428cc8a2cd)

Author SHA1 Message Date
Eric Leblond a297fd029a packet: optimize json context cleaning 
We don't need to recycle the full alert array. This is going to
optimize packet recycle time.
 1 year ago
Eric Leblond 1deca14331 decode: light refactor on packet cleaning 1 year ago
Eric Leblond 8425eed8cd datasets: improve lock handling 
DatasetGetOrCreate should now be called with the dataset lock
hold so the lock management is easier to read.
 1 year ago
Eric Leblond 131bfb9679 detect/dataset: fix locking on datajson 1 year ago
Eric Leblond cf7dda354e detect/dataset: check context_key validity 
As context_key is an user entry and as it is used to build the JSON
string of alert, we could end up with an invalid event if the string
contains improper characters.
 1 year ago
Eric Leblond 803d61c9d1 eve/alert: add comment about string format 1 year ago
Eric Leblond 0f172e721d detect/pcre: fix error handling 1 year ago
Eric Leblond ab5070c0e3 detect/engine: put datajson related code in a func 1 year ago
Eric Leblond e600f40f74 datasets: use SCLogError for errors 1 year ago
Eric Leblond 337472449a detect/engine: remove unused keyword 1 year ago
Eric Leblond cd07db83dc datasets: comment to specify usage of field 1 year ago
Eric Leblond be9e13c6b1 decode: use pointer for PacketContextData 
By adding a pointer instead of the structure in the alert info
we spare some memory.
 1 year ago
Eric Leblond 92f4b36e72 decode: rename ExtraDataJsonList 1 year ago
Eric Leblond b285894f69 datasets: comments about format 1 year ago
Eric Leblond 449d6fe607 datasets: rename function 
DatasetCreateOrGet was in fact doing get or create.
 1 year ago
Eric Leblond e0c5302df1 datasets: fix error handling in StringSet 1 year ago
Eric Leblond 9b6047ab3e datasets: factorize set operation 1 year ago
Eric Leblond 3880b8de22 datasjson: fix include barrier name 1 year ago
Eric Leblond be0f5dac0d detect/pcre: fix copyright date 1 year ago
Eric Leblond 43743b0fd3 datajson: key length fix 
When using type string, we should not limit the size of the string.

This patch also fixes some implicit cast issues.
 1 year ago
Eric Leblond 43a1ef45ca datajson: value_key is necessary in json format 1 year ago
Eric Leblond 62e90206c6 datajson: rename enrichment_key to context_key 1 year ago
Eric Leblond 3e48342248 datajson: factorize code 
Code was duplicated and a build error was reported so factorizing
was the best way to clean it.
 1 year ago
Eric Leblond df99d29ee4 datajson: add sanity check on length 
Also cast to avoid compilation error.
 1 year ago
Eric Leblond cfa7a747c2 datajson: use systematic allocation 1 year ago
Eric Leblond d84d7431e4 datajson: more explicit memory handling 1 year ago
Eric Leblond fed7ff1e76 datasets: remove comment about unused return value 1 year ago
Eric Leblond 79c8f431d0 datajson: simplify memory handling 
DatajsonAdd function is now responsible of the handling of the mem
of datajsontype passed as argument.
 1 year ago
Eric Leblond 606f7ba656 datajson: use wrapper 1 year ago
Eric Leblond 7e7bd92432 datajson: fix potential leak in error handling 1 year ago
Eric Leblond 4a420085b7 datajson: remove unnecessary abstraction 1 year ago
Eric Leblond d57e8731bd datajson: use more broadly supported formatter 1 year ago
Eric Leblond c15bcbe9d6 datasets: factorize dataset creation 
Factorize DatasetGet and DatajsonGet to only have the difference
between the two in the respective function.
 1 year ago
Eric Leblond 28742871d2 datasets: separate DatasetGet in 2 functions 
This will be used to factorize the code with datajson.
 1 year ago
Eric Leblond b766bd9438 datasets/context: rename the datajson files 
They are is renamed to datasets-context-json.* so we see that
it is about context and not about a new datasets type.
 1 year ago
Eric Leblond f7dc2a71db datajson: rename jsonline to ndjson 1 year ago
Eric Leblond b03d4f8e1a datajson: output context to "context" 
Using `alert.extra` was not really reflecting the nature of what
was added. So renaming it to `alert.context`.
 1 year ago
Eric Leblond 0bc21eec48 datajson: fix thread safety violation 1 year ago
Eric Leblond 40f86571d9 datajson: reduce size length 1 year ago
Eric Leblond b8e2791482 datajson: fix string format in error message 1 year ago
Eric Leblond 3dde17fb75 datajson: add remove_key option to dataset 
This option allows to remove the key corresponding to the match
value from the JSON object before creating the JSON object that
will be added to the `extra` data.

For example, matching on the following JSON on the `ip` key:

```json
{"ip": "10.16.1.11", "test": "success", "context":3}
```

with a match like:

```
dataset:isset,src_ip,type ip,load src.lst,format jsonline,enrichment_key src_ip,value_key ip;
```

will produce the following:

```json
"extra": {
  "src_ip": {
    "ip": "10.16.1.11",
    "test": "success",
    "context": 3
  }
```

if we add the `remove_key` option to the match:

```
dataset:isset,src_ip,type ip,load src.lst,format jsonline,enrichment_key src_ip,value_key ip, remove_key;
```

it will produce the following:

```json
"extra": {
  "src_ip": {
    "test": "success",
    "context": 3
  }
```

The option is set to false by default.

Ticket: #7372
 1 year ago
Eric Leblond 3223d8fe59 datajson: implement jsonline format 
This format allows to use a one valid JSON object per line in the
data file.

Ticket: #7372
 1 year ago
Eric Leblond 8058964449 datajson: prepare jsonline format 
There is just a change in the iterator to go from json to jsonline
so let's factorize the parsing functions.

Ticket: #7372
 1 year ago
Eric Leblond 3fbc718728 detect/pcre: add extraction for alert 
With datajson infrastructure in place, it is now possible to
add data in the extra information section. Following an idea
by Jason Ish, this patch adds the feature for pcre extraction.

A PCRE such as pcre:"/(?P<alert_ua>[a-zA-Z]+)\//" will add the
content of the captured group to alert.extra.ua.
 1 year ago
Eric Leblond dd94dc6cc6 datajson: introduce feature 
This patch introduces new option to dataset keyword.
Where regular dataset allows match from sets, dataset with json
format allows the same but also adds JSON data to the alert
event. This data is coming from the set definition it self.
For example, an ipv4 set will look like:

  [{"ip": "10.16.1.11", "test": "success","context":3}]

The syntax is a JSON array but it can also be a JSON object
with an array inside. The idea is to directly used data coming
from the API of a threat intel management software.

The syntax of the keyword is the following:

  dataset:isset,src_ip,type ip,load src.lst,format json, \
       enrichment_key src_ip, value_key ip;

Compare to dataset, it just have a supplementary option key
that is used to indicate in which subobject the JSON value
should be added.

The information is added in the even under the alert.extra
subobject:

  "alert": {
    "extra": {
      "src_ip": {
        "ip": "10.6.1.11",
        "test": "success",
        "context": 3
      },

The main interest of the feature is to be able to contextualize
a match. For example, if you have an IOC source, you can do

 [
   {"buffer": "value1", "actor":"APT28","Country":"FR"},
   {"buffer": "value2", "actor":"APT32","Country":"NL"}
 ]

This way, a single dataset is able to produce context to the
event where it was not possible before and multiple signatures
had to be used.

The format introduced in datajson is an evolution of the
historical datarep format. This has some limitations. For example,
if a user fetch IOCs from a threat intel server there is a large
change that the format will be JSON or XML. Suricata has no support
for the second but can support the first one.

Keeping the key value may seem redundant but it is useful to have it
directly accessible in the extra data to be able to query it
independantly of the signature (where it can be multiple metadata
or even be a transformed metadata).

In some case, when interacting with data (mostly coming from
threat intel servers), the JSON array containing the data
to use is not at the root of the object and it is ncessary
to access a subobject.

This patch implements this with support of key in level1.level2.
This is done via the `array_key` option that contains the path
to the data.

Ticket: #7372
 1 year ago
Eric Leblond 53ac35337a util/byte: add HexToRaw function 1 year ago
Eric Leblond 61e485446a util/ip: add IPv4 and IPv6 length 1 year ago
Victor Julien d5ae9156b9 detect: replace DetectEngineCtx flag with EngineModeIsFirewall 1 year ago
Victor Julien b0b3808a55 detect/config: supported for firewall rules 1 year ago
Victor Julien 1ee9b15a6a help: group and reorder help/usage output 1 year ago