Detect duplicate instances and use the one with the highest
priority.
Use new priority flag to make the logic around explicit priority
sets easier to follow.
Minor code cleanups. Also clean up unittests.
Introduce Signature init_flag to indicate priority has been set.
This will be needed in a follow-up classtype update.
Detect duplicate priority instances in a keyword, and use the
highest priority in the rule. Do issue a warning in this case.
As the file prune is now moved to the flow worker, the file
prune is run later, meaning the first file has not yet
been pruned from the file container list.
Adjust test to look for a second file, and check the
flags on that file.
For commit addressing bug 2490.
If a keyword like filemd5 was being used without a filestore,
or a file output enabled, it would be pruned before detection
had a chance to match.
Consolidate file pruning to the end of the flow worker so files
are available for detection even when a file output is not
enabled.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2490
At the startup, if the default log dir provided either by command line
options or suricat.yaml is not writable, the error comes quite later.
This patch makes suricata exit if there is such an error in the
beginning itself.
Closes redmine ticket #2386.
In file included from suricata-common.h:471,
from app-layer-enip-common.c:27:
app-layer-enip-common.c: In function ‘DecodeCIPRequestPathPDU’:
util-debug.h:222:31: warning: ‘req_path_class8’ may be used uninitialized in this function [-Wmaybe-uninitialized]
int _sc_log_ret = snprintf(_sc_log_msg, SC_LOG_MAX_LOG_MSG_LEN, __VA_ARGS__); \
^~~~~~~~
app-layer-enip-common.c:589:13: note: ‘req_path_class8’ was declared here
uint8_t req_path_class8;
^~~~~~~~~~~~~~~
app-layer-enip-common.c:607:9: warning: ‘segment’ may be used uninitialized in this function [-Wmaybe-uninitialized]
switch (segment)
^~~~~~
app-layer-enip-common.c: In function ‘DecodeCIPResponsePDU’:
app-layer-enip-common.c:773:13: warning: ‘service’ may be used uninitialized in this function [-Wmaybe-uninitialized]
service &= 0x7f; //strip off top bit to get service code. Responses have first bit as 1
^~
app-layer-enip-common.c: In function ‘DecodeCIPRequestPDU’:
app-layer-enip-common.c:503:25: warning: ‘path_size’ may be used uninitialized in this function [-Wmaybe-uninitialized]
offset += path_size * sizeof(uint16_t); //move offset past pathsize
~~~~~~~~~~^~~~~~~~~~~~~~~~~~
app-layer-enip-common.c:506:5: warning: ‘service’ may be used uninitialized in this function [-Wmaybe-uninitialized]
switch (service)
^~~~~~
Bug #3139.
Improve warnings when eve.stats can't work because of the global config
missing or disabled.
Issue warning if global config is missing but stats are still enabled due
to the legacy stats.log.
Issue clearer warning when stats are disabled and unix socket dump-counters
command is issued.
Warnings include links to docs.
Bug #2465.
bzero(3): The bzero() function is deprecated (marked as LEGACY in
POSIX.1-2001); use memset(3) in new programs. POSIX.1-2008 removes
the specification of bzero().
Use memset instead.
Replace index by strchr and rindex by strrchr.
index(3) states "POSIX.1-2008 removes the specifications of index() and
rindex(), recommending strchr(3) and strrchr(3) instead."
Add index/rindex to banned function check so they don't get reintroduced.
Bug #1443.
OpenSSL uses 30, so this seems a reasonable limit.
Set a smaller limit than before to reduce the resources spent on
specially crafted input designed to be maximally expensive.
Set and Sequence parsers would pass on max available data instead
of the size of their object.
Malformed data could trigger massive recursion this way, leading
to spending much more resources than necessary.
Found using AFL.
Bug #3185.
Field is at data+1 offset, not +3. Also makes sure we always stay
within checked data bounds.
Reported-by: Sirko Höer -- Code Intelligence for DCSO.
Bug #3176.
Before re-assembling, check that the first fragment is large
enough to contain the IPv4 or IPv6 header to prevent
an out of bounds read (IPv4) or write (IPv6).
Reported-by: Sirko Höer -- Code Intelligence for DCSO.
Bug #3171.
Restructure code to make it clearer that either 'basic', 'extended'
or 'custom' is being printed, by creating one function for each of
the possibilities.
Add a rule keyword, dns.opcode to match on the opcode flag
found in the DNS request and response headers.
Only exact matches are allowed with negation.
Examples:
- dns.opcode:4;
- dns.opcode:!1;
This permits to use stream-depth value set for file-store.
Currently if a file is being stored and hits a limit,
such as request or response body, it will be truncated
although file-store.stream-depth is enabled but the file should be
closed and not truncated.
Two unit tests have been added to verify that:
- a file is stored correctly
- chunk's length computation doesn’t cause an underflow
output-json-ftp.c: In function ‘JsonFTPLogger’:
output-json-ftp.c:129:9: warning: ‘js_respcode_list’ may be used uninitialized in this function [-Wmaybe-uninitialized]
129 | json_object_set_new(cjs, "completion_code", js_respcode_list);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
output-json-ftp.c:74:13: note: ‘js_respcode_list’ was declared here
74 | json_t *js_respcode_list;
| ^~~~~~~~~~~~~~~~
output-json-ftp.c:128:9: warning: ‘js_resplist’ may be used uninitialized in this function [-Wmaybe-uninitialized]
128 | json_object_set_new(cjs, "reply", js_resplist);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
output-json-ftp.c:73:13: note: ‘js_resplist’ was declared here
73 | json_t *js_resplist;
| ^~~~~~~~~~~
Datasets are sets/lists of data that can be accessed or added from
the rule language.
This patch implements 3 data types:
1. string (or buffer)
2. md5
3. sha256
The patch also implements 2 new rule keywords:
1. dataset
2. datarep
The dataset keyword allows matching against a list of values to see if
it exists or not. It can also add the value to the set. The set can
optionally be stored to disk on exit.
The datarep support matching/lookups only. With each item in the set a
reputation value is stored and this value can be matched against. The
reputation value is unsigned 16 bit, so values can be between 0 and 65535.
Datasets can be registered in 2 ways:
1. through the yaml
2. through the rules
The goal of this rules based approach is that rule writers can start using
this without the need for config changes.
A dataset is implemented using a thash hash table. Each dataset is its own
separate thash.
Thread safe hash table implementation based on the Flow hash, IP Pair
hash and others.
Hash is array of buckets with per bucket locking. Each bucket has a
list of elements which also individually use locking.
1. Set WARN_UNUSED macro on DetectSignatureSetAppProto.
2. Replace all direct 'sets' of Signature::alproto from keyword registration.
Closes redmine ticket #3006.
This define is used to remove reference to capture bypass in case
no capture method implementing this is active.
This patch also introduces CAPTURE_OFFLOAD_MANAGER that is defined
if we need the flow bypass manager code.
ICMP unreachable errors are linked to the flow they send an error for.
This would lead to the detection engine calling the TX inspection
engines on them.
The stream inspect engine would default to a match for non-UDP
and non-TCP as for ICMP we're not expected to use a TX inspect engine
for stream data.
This all would lead to a false positive match.
This patch fixes this by making sure the TX engines are not called if
the packet protocol and flow protocol are not the same.
Bug #2769.
CC source-netmap.o
source-netmap.c: In function ‘NetmapOpen’:
source-netmap.c:327:56: error: ‘%s’ directive output may be truncated writing up to 15 bytes into a region of size between 10 and 57 [-Werror=format-truncation=]
snprintf(devname, sizeof(devname), "netmap:%s%s%s",
^~
ns->iface, strlen(optstr) ? "/" : "", optstr);
~~~~~~
source-netmap.c:327:9: note: ‘snprintf’ output 8 or more bytes (assuming 70) into a destination of size 64
snprintf(devname, sizeof(devname), "netmap:%s%s%s",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ns->iface, strlen(optstr) ? "/" : "", optstr);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
source-netmap.c:330:59: error: ‘%s’ directive output may be truncated writing up to 15 bytes into a region of size between 8 and 55 [-Werror=format-truncation=]
snprintf(devname, sizeof(devname), "netmap:%s-%d%s%s",
^~
ns->iface, ring, strlen(optstr) ? "/" : "", optstr);
~~~~~~
source-netmap.c:330:9: note: ‘snprintf’ output 10 or more bytes (assuming 72) into a destination of size 64
snprintf(devname, sizeof(devname), "netmap:%s-%d%s%s",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ns->iface, ring, strlen(optstr) ? "/" : "", optstr);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
source-netmap.c:316:54: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
snprintf(devname, sizeof(devname), "%s}%d%s%s",
^
source-netmap.c:316:9: note: ‘snprintf’ output 3 or more bytes (assuming 65) into a destination of size 64
snprintf(devname, sizeof(devname), "%s}%d%s%s",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ns->iface, ring, strlen(optstr) ? "/" : "", optstr);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Gcc 8 with -Wformat-truncation=1
Date makes it even clearer that when was the last commit for the build
that one is running. Add this info alongwith rev. Change inspired by
rustc.
Before
```
$ suricata -V
This is Suricata version 5.0.0-dev (rev 2d217e666)
```
After
```
This is Suricata version 5.0.0-dev (2d217e666 2019-07-12)
```
Closes redmine ticket #3092
This changeset breaks multi-line FTP responses into separate array
entries. Multi-line responses are those with "text-1\r\ntext-2[...]".
Each of \r\n delimited text segments is reported in the `reply` array;
each text segment _may_ include a completion code; completion codes are
reported in the `completion_code` array.
Permit picking up any reply w/o a request. Observed unsolicited server
messages before connection termination.
Previously the code assumed that this could only happen on connection
start when there was no previously recorded command.
This changeset ensures that unknown commands are logged.
Unknown commands are either
- Banner responses when connecting to the FTP port
- Commands not includes in the FtpCommands descriptor table
Modified transaction logic to create a new transaction with each
request; replies location transactions by using the oldest "open"
(unmatched) transaction or the last transaction if none are open.
When a TCP session is picked up from the response the flow is
reversed by the protocol detection code.
This would lead to duplicate logging of the response. The reason this
happened was that the per stream app progress tracker was not handled
correctly by the direction reversing code. While the streams were
swapped the stream engine would continue to use a now outdated pointer
to what had become the wrong direction.
This patches fixes this by making the stream a ptr to ptr that can be
updated by the protocol detection as well.
In addition, the progress tracking was cleaned up and the GAP error
handling in this case was improved as well.
Previously, source-pfring.c would copy the vlan_id from the extended
header only if vlan.use-for-tracking was enabled. This commit removes
that check.
Related to https://redmine.openinfosecfoundation.org/issues/3076
Fill in the vlan_id fields unconditionally. We can now remove the check
for the vlan.use-for-tracking setting in decode.c. The debug log message
is moved to suricata.c.
Since the vlan.use-for-tracking setting is now handled in flow-hash.c,
we can fill in the vlan_id fields unconditionally. This makes the vlanh
fields unnecessary.
Related to https://redmine.openinfosecfoundation.org/issues/3076
If vlan.use-for-tracking is disabled, set the vlan_id fields to 0 when
hashing or comparing flows. This is done using a bitmask as suggested by
Victor Julien in IRC, in order to avoid adding more branches to this
code.
Currently, suricata does not fill in vlan_id fields if
vlan.use-for-tracking is disabled and instead leaves them at the default
0 value, so this commit makes no functional change. This change is in
preparation for future commits where the vlan_ids will be always filled
in.
Related to https://redmine.openinfosecfoundation.org/issues/3076
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.
Share part of the registration code with the AppLayer.
Implement for the tcp.hdr and udp.hdr keywords.
Implement port config handling. Also check both src port and dest
port for tunnels that only set the destination port to the VXLAN
port. At the point of the check we don't know the packet direction
yet.
Implement as Suricata tunnel similar to Teredo.
Cleanups.
Avoid clash by adding a leading underscore to the declaration in the
macro. These temporary vars should never clash with valid variables
from the code where they are called from.