This patch introduce the IPv4 type for dataset so Suricata commandmatch
on a set of IPv4 addresses. This is meant to complement iprep feature
for people that needs more flexibility such as settings the IP on
the packet path.
Feature: #5383
Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
If there is a space following a keyword that does not expect a value,
the rule fails to load due to improper value evaluation.
e.g. Space after "set" command
alert http any any -> any any (http.user_agent; dataset:set ,ua-seen,type string,save datasets.csv; sid:1;)
gives error
[ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] - dataset action "" is not supported.
Fix this by handling values correctly for such cases.
It is now possible to set the memcap and hashsize via suricata.yaml and
rules.
Rule example:
alert http any any -> any any (http.user_agent; dataset:isset,ua-seen,type string,load datasets.csv,memcap 100mb,hashsize 2048; sid:1;)
suricata.yaml example:
datasets:
ua-seen:
type: string
load: datasets.csv
memcap: 20mb
hashsize: 2048
Test the full path instead of just the filename provided in the
rule to see if it exists.
Fixes the case where a rule file is loaded from a directory
other than the default-rule-directory.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3916
Datasets are sets/lists of data that can be accessed or added from
the rule language.
This patch implements 3 data types:
1. string (or buffer)
2. md5
3. sha256
The patch also implements 2 new rule keywords:
1. dataset
2. datarep
The dataset keyword allows matching against a list of values to see if
it exists or not. It can also add the value to the set. The set can
optionally be stored to disk on exit.
The datarep support matching/lookups only. With each item in the set a
reputation value is stored and this value can be matched against. The
reputation value is unsigned 16 bit, so values can be between 0 and 65535.
Datasets can be registered in 2 ways:
1. through the yaml
2. through the rules
The goal of this rules based approach is that rule writers can start using
this without the need for config changes.
A dataset is implemented using a thash hash table. Each dataset is its own
separate thash.
Victor Julien
Eric Leblond
Shivani Bhardwaj
Jason Ish
Victor Julien
Jeff Lucovsky