When a TCP DNS flow would start with a GAP on the TS side, the successful
protocol detection on the TC side would trigger 'opposing side' reassembly
and app-layer processing. In this case the stream flags would indicate the
wrong direction and the wrong parser would be called.
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.
Introduce two macros:
APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)
Update all parsers to use this.
The idea of an OK signature parsing error is an error that is
allowed to occur, but still lets test mode pass, unlike
silent errors which will still fail testing.
This is introduced to allow for app-layer event keywords to be
removed, but not have old rules fail out on this case. For example
the Rust DNS parser removes from DNS app-layer events that are
not used anymore.
To signal that an error is OK, -3 is returned. This also implies
silent.
On an unknown app-layer event, return -3 for "silent OK fail". A
warning will still be emitted, but its not considered a rule parse
error. This is to handle app-layer events being removed in a more
graceful manner for the user.
This allows -T to pass with an old app-layer events rule file
that may used removed app-layer event keywords.
This commit places restrictions on the length of the file name specified
in attachments (`name=` or `filename=`) to `NAME_MAX`. Names exceeding
these limits will be truncated and processing will continue with the
truncated name.
This patch addresses two problems.
First, various parts of the engine, but most notably the flow manager (FM),
use a minimum of the time notion of the packet threads. This did not
however, take into account the scenario where one or more of these
threads would be inactive for prolonged times. This could lead to the
time used by the FM could get stale.
This is addressed by keeping track of the last time the per thread packet
timestamp was updated, and only considering it for the 'minimum' when it
is reasonably current.
Second, there was a minor race condition at start up, where the FM would
already inspect the hash table(s) while the packet threads weren't active
yet. Since FM gets the time from the packet threads, it would use a bogus
time of 0.
This is addressed by adding a wait loop to the start of the FM that waits
for 'time' to get ready.
A race condition during the start of pcap file processing could cause
missed alerts and logged events. This race happens between the packet
threads and the flow manager. It was observed on slower hardware, but in
theory could happen on any machine. It required the 'autofp' runmode.
In commit 6f560144c1 ("time: improve offline time handling") the logic
was added to make the flow manager use a minimum of all the packet threads
perception of time.
The race condition was that the flow manager may become active _before_
all of the packet threads have started processing packets and thus setting
their timestamp. The threads that had not yet initialized their timestamp
would not be considered when calculating the minimum.
As a result of this, older packets timestamps would not yet be registered.
This would give the Flow Manager a timestamp too far in the future. While
the FM was running, the packet processing would start and a flow would
be created. This flow would then immediately be considered 'timed out' by
the FM, due to the timestamp too far in the future.
In the observed case, the thread processing packet 1 from the pcap had not
yet started processing while other threads had already started. The FM was
also already active. Due to the timestamps in the pcap this meant that the
time the FM used was about 500 seconds in the future compared to packet 1.
This patch fixes the issue by initializing all of the threads timestamps
with the timestamp value of the first packet. This way the minimum will
always consider this timestamp.
This commit adds logic checking if the sticky buffer in effect provides
the required content.
If the sticky buffer doesn't, the rule will not load and a diagnostic
message with follow-on steps is displayed.
Improve flow file flags and file flags updates. Introduce a mask
that is set at start up to avoid lots of runtime checks.
Disable cocci flags check as it doesn't support the more dynamic
nature of the flag updates.
Using the run-as configuration option with the nflog capture method
results in the following error during the startup of suricata:
[ERRCODE: SC_ERR_NFLOG_BIND(248)] - nflog_bind_pf() for AF_INET failed
This is because SCDropMainThreadCaps does not have any capabilities
defined for the nflog runmode (unlike other runmodes). Therefore, apply
the same capabilities to the nflog runmode that are already defined for
the nfqueue runmode. This has been confirmed to allow suricata start
and drop its privileges in the nflog runmode.
Fixes redmine issue #3265.
Signed-off-by: Timo Sigurdsson <public_timo.s@silentcreek.de>
Previously each 'TmSlot' had it's own packet queue that was passed
to the registered SlotFunc as an argument. This was used mostly for
tunnel packets by the decoders and by defrag.
This patch removes that in favor of a single queue in the ThreadVars:
decode_pq. This is the non-locked version of the queue as this is
only a temporary store for handling packets within a thread.
This patch removes the PacketQueue pointer argument from the API.
The new queue can be accessed directly through the ThreadVars
pointer.
Shrink ThreadVars by removing the queue handler names that are only
used at shutdown. Since this is not performance critical, we can use
the id's to look up the queue handler.
When TmThreadsSlotProcessPkt fails it will return the packet that was
passed to it to the packetpool.
Some of the packet sources were doing this manually as well. This patch
fixes those sources.
Use a single packet queue per thread for flow timeout packet
injection. The per slot queue was unused except for this use
case. Having a single queue makes the logic and implementation
simpler.
In case of 'autofp', the per thread packet queue will actually
use the threads input queue. For workers/single a dedicated
queue will be set up.
Rename TmThreadsSlotHandlePostPQs to TmThreadsHandleInjectedPackets
to reflect the changed logic.
Set the livedev on reassembled packets to that of the parent
packet. Fixes issues with multidetect, specifically a segfault
as reported in issue 3380.
Bug #3380.
This patch splits the list of loggers the tx logging walks into lists per
alproto. The list was getting longer with each eve addition. The result
was that for each tx we would have to loop through multiple loggers that
did not apply to this tx as it was for the wrong protocol.
In some cases a SBB could be seen as overlapping with the requested
offset, when it was in fact precisely before it. In some special cases
this could lead to the stream engine not progressing the 'raw' progress.
Don't assume that the next block after the sbb head is after the
requested offset.
If the next block was before the offset, the returned data_len
would underflow and return a nonsense value to the app-layer.
Bug #2993.
This commit causes Suricata to exit when a buffer from the analyzer
table is not recognized.
Since the table must match what's registered, exiting will bring noticed
to the condition.
Currently all failures in IPv4 option decode are ignore with
respect to continuing to handle the packet.
Change this to fail, and abort handling the packet if the
option length is invalid.
Ticket 3328:
https://redmine.openinfosecfoundation.org/issues/3328
Fix evasion posibility by rejecting packets with a broken ACK field.
These packets have a non-0 ACK field, but do not have a ACK flag set.
Bug #3324.
Reported-by: Nicolas Adba
RST injection during the SYN_SENT state could trick Suricata into marking
a session as CLOSED. The way this was done is: using invalid TSECR value
in RST+ACK packet. The ACK was needed to force Linux into considering the
TSECR value and compare it to the TSVAL from the SYN packet.
The second works only against Windows. The client would not use a TSVAL
but the RST packet would. Windows will reject this, but Suricata considered
the RST valid and triggered the CLOSED logic.
This patch addresses both. When the SYN packet used timestamp support
the timestamp of incoming packet is validated. Otherwise, packet responding
should not have a timestamp.
Bug #3286
Reported-by: Nicolas Adba
Fix address negation detection not resolving variables when
looking for the negation.
This patch makes use of the actual parsing routines to relay this
information to the signature parser.
Bug #3389.
Fixes: 92f08d85aa ("detect/iponly: improve negation handling in parsing")
If file prune is called inspect has already run. So if file is closed
we can just prune. No need to consider a window anymore.
When still in progress, fix the left_edge calculation.
Now that we call the file prune loop very regularly, we can move the
SMTP specific inspection pruning logic into this loop. Helps with
cases there we don't (often) update a files inspection trackers.
If we already know that the boundary exists, we can start looking
there. Otherwise, we can skip trying as the boundary is a subset
of the form end marker.
This commit ensures direction warnings for ICMP v4 and v6
are suppressed and corrects check so that both protocols
are checked (instead of the same protocol being checked twice).
Update the default packet size computation to use LiveDeviceName
instead of LiveDevice as the LiveDevice list is not built when
the default packet size is built.