Commit Graph

10803 Commits (10d9deec9f2293c797df7dbe8bda346d4dc7d600)
 

Author SHA1 Message Date
Victor Julien fe9aeed0f0 threading: fix shutdown race condition
A BUG_ON statement would seemingly randomly trigger during the threading
shutdown logic. After a packet thread reached the THV_RUNNING_DONE state,
it would sometimes still receive flow timeout packets which would then
remain unprocessed.

1 main:   TmThreadDisableReceiveThreads(); <- stop capturing packets
2 worker: -> TmThreadTimeoutLoop (THV_FLOW_LOOP) phase starts
3 main:   FlowForceReassembly();           <- inject packets from flow engine
4 main:   TmThreadDisablePacketThreads();  <- then disable packet threads
5 main:   -> checks if 'worker' is ready processing packets
6 main:   -> sends THV_KILL to worker
7 worker: breaks out of TmThreadTimeoutLoop and changes to THV_RUNNING_DONE.

Part of the problem was with (5) above. When checking if the worker was
already done with its work, TmThreadDisablePacketThreads would not consider
the injected flow timeout packets. The second part of the problem was with (7),
where the worker checked if it was ready with the TmThreadTimeoutLoop in a
thread unsafe way.

As a result TmThreadDisablePacketThreads would not wait long enough for the
worker(s) to finish its work and move the threads to the THV_RUNNING_DONE
phase by issuing the THV_KILL command.

When waiting for packet processing threads to process all in-flight packets,
also consider the 'stream_pq'. This will have received the flow timeout
packets.

Bug #1871.
5 years ago
Victor Julien 825173a2ba threading: fix flow timeout loop race 5 years ago
Victor Julien 56354afd41 threading: improve thread queues checking by dumping more info 5 years ago
Victor Julien 0a809bf577 packet: set unique pkt_src 'flush' packets
Set unique type for capture timeout and for detect reload flush
to assist in debugging.
5 years ago
Victor Julien 6bc7636826 stream: remove unused code
Remove now unused 'pkt_src' type as well.

Remove related unittests.
5 years ago
Victor Julien 1633744fcb nfq: remove unused queue handler type 5 years ago
Victor Julien ab01cbe345 log-pcap: remove stale comments 5 years ago
Victor Julien 6de025bb12 alert-syslog: remove stale comments 5 years ago
Jeff Lucovsky 90c2e3561c Add general purpose `ARRAY_SIZE` macro
This commit adds `ARRAY_SIZE` as an helper for determining the number of
elements in an initialized array. The calculation is the same but the
macro provides a convenient shortcut. The implementation was borrowed
from the kernel sources.
5 years ago
Jeff Lucovsky ae198add6d detect/analyzer: Refactor engine analysis code
This commit changes the analysis code to be table driven to better
identify the rule elements covered by the analysis.
5 years ago
Philippe Antoine 6e63c957ff signature: Fixes memory leak in parsing app layer event 5 years ago
Jason Ish fccbd36d37 dns: log addresses in flow direction, not packet
Ticket #3340.
https://redmine.openinfosecfoundation.org/issues/3340
5 years ago
Victor Julien 0824b04134 filestore: don't assume flow is TCP
Filestore can be used by UDP based protocols as well. NFSv2 is one
that Suricata supports.

Bug #3277.
5 years ago
Victor Julien 2a55afbd89 decode/pppoe: fix potential crash in debug statement 5 years ago
Victor Julien 51ad701d8e version: starting work on 5.0.1 5 years ago
Victor Julien 2ab7fb4b41 version: automate and cleanup ver handling
Create a single function to return the version string, to avoid lots
of ifdefs in multiple places.

Make the version determine the 'release' status. If the version from
autoconf has '-dev' in the name, it is not a release. If it hasn't
it is considered a release version.
5 years ago
Victor Julien 51ec980e80 dataset: fix string length handling in hash 5 years ago
Eric Leblond 1721da91ef dataset: fix hash computation 5 years ago
Victor Julien 64a789bbf6 nfq: clear memory of queue before using it
Avoids using uninitialized memory. Show showed itself
in nonsense values in counters, and in nfq_handle_packet
errors that were likely the result of passing uninitialized
memory to the nfq API.

Bug 3263.
Bug 3120.

Fixes: b2a6c60dee ("source-nfq: increase maximum queues number to 65535")
5 years ago
Victor Julien 2fd1174a56 nfq: micro optimization 5 years ago
Victor Julien 9d6087f7d6 nfq: don't warn on 'handle_packet' error
NFQ can generate warnings/errors with a delay. After Suricata has
succesfully passed a verdict to the kernel, there are still things
that can go wrong for that verdict. This is then passed to the
queue through a netlink error message, which leads to nfq_handle_packet
returning an error code.

Suppress the warning. Also remove the errno/strerror use as
nfq_handle_packet does not set the errno.

Thanks to Florian Westphal.

Bug 3120.
5 years ago
Victor Julien f8acad7fca nfq: code cleanups 5 years ago
Alexander Gozman f280e66f84 nfq: check for EAGAIN after recv() call in NFQRecvPkt() 5 years ago
Victor Julien 4cc90e9a4c nfq: minor code cleanups 5 years ago
Victor Julien 01cea2ec89 datasets: suppress noisy debug statement 5 years ago
Victor Julien 505b2dd256 log-pcap: don't print (null) for compression method 5 years ago
Victor Julien fb26268c6b tcp: don't set event on empty SACK opt
TCP_OPT_INVALID_LEN was set if the opt len was 2. While useless
an empty SACK is not uncommon.

Seen on an iOS device talking to an Apple server.

Bug #3254.
5 years ago
Victor Julien aae76a84cd suricata: use version from autoconf 5 years ago
Eric Leblond 9ef2f81ee7 doc/userguide: fix typo 5 years ago
Eric Leblond 821d590f5b doc/userguide: fix base64 example
Add a sticky buffer example and fix the content modifier one.
5 years ago
Eric Leblond 2d11e9394c detect-base64: fix url in list keywords commands 5 years ago
Pascal Delalande 8e6a2bd42e doc: removal of disable-rust and path typo for suricatasc 5 years ago
Jason Ish 6eada54fc8 eve/dns: don't log warning if dns log version not set
If the DNS log version is not set, we default to v2. This should
not be warning, but better logged at the config level.

A warning will still be logged if the value is set but is not
1 or 2.
5 years ago
Philippe Antoine 989a6461b0 signature: leak fix in DetectAddressParse2 5 years ago
Philippe Antoine c1e41632c1 config: use logging instead of stderr 5 years ago
Wesley van der Ree bf1b65558b mpls: Allow MPLS after vlan.
Fixes #2771
5 years ago
Victor Julien 7cca9005fb dns: minor cleanup 5 years ago
Victor Julien 4164c0bbd6 app-layer: make dns,smb,tls parsers less noisy w/o config 5 years ago
Victor Julien 0526878fee detect/tls: set alternatives for legacy tls keywords 5 years ago
Victor Julien 3019f10ac7 detect/tls: tls.cert_fingerprint is a sticky buffer
Not a content modifier.
5 years ago
Victor Julien d5ae68afc2 doc: fix version in install doc 5 years ago
Victor Julien 1c27a99827 doc: add upgrade page 5 years ago
Victor Julien 697410cbbf changelog: update for 5.0.0 5 years ago
Victor Julien 3b5b71afa0 changelog: update 5.0rc1 section with missing entries 5 years ago
Jason Ish 412ae11bad automake: use tar-ustar for longer filenames
According to the automake manual it should be considered
portable these days.

https://www.gnu.org/software/automake/manual/html_node/List-of-Automake-options.html

Required for the dist generation with Rust vendoring.
5 years ago
Jason Ish 718fcbb682 doc: document eve/dns v2 as the default
Adds eve/dns v2 format documentation. Update legacy format
to require the version field.
5 years ago
Jason Ish 16221c0b33 suricata.yaml/dns: small cleanups, not that default is v2
Note that the eve dns log format is version 2 by default.

Make the value of commented out values their default.

Update the comment on the types to better reflect what it does.
5 years ago
Jeff Lucovsky 42452b327c mem: Use correct len with strlcpy 5 years ago
Fabrice Fontaine b026fbb519 configure.ac: fix static build with pcap
pcap can depends on nl-3 so use pkg-config to find these dependencies
otherwise all AC_CHECK_LIB calls will fail when building statically

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
5 years ago
Jason Ish 8cf4afabc3 Makefile: prefix suricata-update error with @
So only the echo output is displayed. Not the lines themselves.
5 years ago