Commit Graph

10747 Commits (8a643c893c4cae61a22e343f1b910461dd110a54)
 

Author SHA1 Message Date
Victor Julien cd66c37711 http/multipart: use proper progress value to test eof 5 years ago
jason taylor 95237f9894 docs: update datasets examples
Signed-off-by: jason taylor <jtfas90@gmail.com>
5 years ago
Philippe Antoine 08b84e060b fastlog: use icmp type and code instead of port
Fixes #3266
5 years ago
Philippe Antoine 75a7d9641c fastlog: move code to reduce variable scope 5 years ago
Philippe Antoine c2fdd7c969 transform: fixes comment about compress_whitespace 5 years ago
Philippe Antoine 9126fc25c1 transform: updates doc about compress_whitespace
And removes duplicate test from strip_whitespace
5 years ago
Victor Julien 2b581fa836 qa/banned-functions: remove strndup now we have a fall back 5 years ago
Eric Leblond 18a1d93c58 qa/coccinelle: fix make distcheck
We need to use top_srcdir to know where the sources are. Relative
directory is not enough.
5 years ago
Eric Leblond 7a8ddf06de qa/coccinelle: flag check for setter and getter
WHen adding something like
/* coccinelle: AppLayerParserStateIssetFlag():4,2:APP_LAYER_PARSER_ */
the coccinelle check will consider that AppLayerParserStateIssetFlag
is taking 4 parameters and that the second one is a flag that needs
to be checked against APP_LAYER_PARSER_.
5 years ago
Eric Leblond 3ded7f1170 qa/coccinelle: fix false positive in setter getter
Coccinelle test was doing a false positive on the function
AppLayerParserStateSetFlag and AppLayerParserStateIssetFlag.
To address that, this patch adds a new coccinelle markup:

 /* coccinelle: AppLayerParserStateSetFlag():2,2:APP_LAYER_PARSER_ */

It indicates that AppLayerParserStateSetFlag is a setter and getter
and that the checks should be disabled inside the function.

Currently this markup is only used for that but following patch will
add some checks on option value.
5 years ago
Eric Leblond f745f0655b qa/coccinelle: port struct-flags.py to Python3 5 years ago
Eric Leblond b58d508bff qa/coccinelle: reformat struct-flags.py 5 years ago
Eric Leblond ba6b58a0da qa: generate struct-flags.cocci at each run 5 years ago
EmilienCourt 50bb8d4cb2 doc: fix typo on example
Quotes have been forgotten in the dnp3.data example, which throws an
SC_ERR_INVALID_SIGNATURE(39) if used like in the example.
5 years ago
Jeff Lucovsky 8f4f1cb633 detect/analyzer: Improved fast pattern display
When transforms are part of a rule, improve information displayed with
fast patterns to include the original buffer name and whether any
transform(s) are applied.
5 years ago
Jeff Lucovsky c88c1f1e14 detect/analyzer: Suppress direction warnings
This commit suppresses direction warnings by the rules analyzer for ICMP
and ICMPV6 since it's not actionable.
5 years ago
Victor Julien 83bbe287e7 stats: fix stats not always syncing in flow timeout 5 years ago
Jason Ish ba3a2c31bf app-layer: validate TX detect flag callbacks
Check that both are set or unset.
5 years ago
Jason Ish 706558d4d5 enip: add tx detect flags 5 years ago
Jason Ish cb62c8dacf dcerpc: add tx detect flags 5 years ago
Jason Ish e2c846d01f snmp: add tx detect flags 5 years ago
Jason Ish 21f014f5c3 modbus: add tx detect flags 5 years ago
Jason Ish fa4b9d37c2 krb5: register tx detect flags
Related ticker #3345:
https://redmine.openinfosecfoundation.org/issues/3345
5 years ago
Jason Ish 8a232be77e rust: define TxDetectFlag struct and binding macros
Define a TxDetectFlag type and macros to generating C
bindings for getting and settings the tx detect
flags.
5 years ago
Jason Ish cde49ec246 rust: add tx detect flags function to registration struct 5 years ago
Jason Ish 20bc08a722 app-layer: add tx detect functions to register struct 5 years ago
Jason Ish fdb587d2fc detect-engine: check for tx detect flag support
When registing a detection engine, check that the app-layer
protocol supports tx detect flags. Exit with a fatal
error if it does not as this is a code implementation
error that should be resolved during development.
5 years ago
Jason Ish b1beb76fd7 ftpdata: add tx detect flags 5 years ago
Jason Ish 62e4211f04 debug: add SCReturnBool function exit macro 5 years ago
Jason Ish 739df21e2d app-layer: method to see if parser supports tx detect flags
Add method to check if a parser for an app-layer protocol
supports tx detect flags.

This is a bit of a hack for now as where we need to run
this check from we do not have the IP protocol.
5 years ago
Jason Ish 3887f8d1f3 github-actions: builds for our tier one linux distributions
Example of using GitHub actions to perform builds across
CentOS, Ubuntu, Debian and the latest Fedora.
5 years ago
Jeff Lucovsky 218a5c4345 mpm: Fix typos and spelling errors 5 years ago
Jeff Lucovsky aef24bee96 detect: Fix spelling errors 5 years ago
Jeff Lucovsky f318a46d34 detect: Improve handling of variable values
When one of offset/depth/distance is from a variable, adjust the depth
by the offset as is done with scalar values at parse time.
5 years ago
Jeff Lucovsky db8527e7b3 detect/mpm: Improved handling of variable values
This commit removes the offset and depth if either of these values are
dependent upon a byte-extract operation.
5 years ago
Victor Julien 94982ae690 http: split request/response tx id handling
When HTTP pipelining was in use, the transaction id used for events
and files could be off. If the request side was several requests ahead
of the responses, it would use the HtpState::transaction_cnt for events
and files, even though that is only incremented on complete requests.

Split request and response tx id tracking. The response is still handled
by the HtpState::transaction_cnt, but the request side is now handled by
its own logic.
5 years ago
Victor Julien b82e71b95e files: remove FILE_USE_TRACKID flag
Once it was optional but as it no longer is it is no longer useful.

Remove it.
5 years ago
Victor Julien f9155aa121 files: simplify pruning logic
Since ebcc4db84a the flow worker runs
file pruning after parsing, detection and loging. This means we can
simplify the pruning logic. If a file is in state >= CLOSED, we can
prune it. Detection and outputs will have had a final chance to
process it.

Remove the calls to the pruning code from Rust. They are no longer
needed.
5 years ago
Victor Julien ab471c3054 app-layer: don't consider tx flags if not registered
If a protocol does not support TxDetectFlags, don't try to use them.

The consequence of trying to use them was that a TX would never be
considered done, and it would never be freed. This would lead to excessive
memory use and performance problems due to walking an ever increasing
list.
5 years ago
Eric Leblond 54d3620662 source-pcap-file: honor bpf filter on command line
When a BPF filter is given on the command line when reading a
pcap file, the BPF filter is not honored.

The regression has been introduced in:

commit 3ab9120821
Author: Dana Helwig <dana.helwig@protectwise.com>
Date:   Thu Apr 27 11:17:16 2017 -0600

    source-pcap-file: Pcap Directory Mode (Feature #2222)

Reported-By: Tim Colin <tcolin@et.esiea.fr>
5 years ago
Eric Leblond 860f43753c source-pcap-file: fix memory leak on pcap filter 5 years ago
Philippe Antoine 20e06f45c0 util: removes warning about double conversion
From clang 10 :
implicit conversion from 'unsigned long' to 'double' changes value
from 18446744073709551615 to 18446744073709551616
5 years ago
Victor Julien 502a8b5fb3 detect: fix inspection buffer for packet engines
Fix buffers not being reset per inspection round for packet engines.

Bug #3341.
5 years ago
Victor Julien 58b9a2dc21 threading: add debug validation for stale packets 5 years ago
Victor Julien fe9aeed0f0 threading: fix shutdown race condition
A BUG_ON statement would seemingly randomly trigger during the threading
shutdown logic. After a packet thread reached the THV_RUNNING_DONE state,
it would sometimes still receive flow timeout packets which would then
remain unprocessed.

1 main:   TmThreadDisableReceiveThreads(); <- stop capturing packets
2 worker: -> TmThreadTimeoutLoop (THV_FLOW_LOOP) phase starts
3 main:   FlowForceReassembly();           <- inject packets from flow engine
4 main:   TmThreadDisablePacketThreads();  <- then disable packet threads
5 main:   -> checks if 'worker' is ready processing packets
6 main:   -> sends THV_KILL to worker
7 worker: breaks out of TmThreadTimeoutLoop and changes to THV_RUNNING_DONE.

Part of the problem was with (5) above. When checking if the worker was
already done with its work, TmThreadDisablePacketThreads would not consider
the injected flow timeout packets. The second part of the problem was with (7),
where the worker checked if it was ready with the TmThreadTimeoutLoop in a
thread unsafe way.

As a result TmThreadDisablePacketThreads would not wait long enough for the
worker(s) to finish its work and move the threads to the THV_RUNNING_DONE
phase by issuing the THV_KILL command.

When waiting for packet processing threads to process all in-flight packets,
also consider the 'stream_pq'. This will have received the flow timeout
packets.

Bug #1871.
5 years ago
Victor Julien 825173a2ba threading: fix flow timeout loop race 5 years ago
Victor Julien 56354afd41 threading: improve thread queues checking by dumping more info 5 years ago
Victor Julien 0a809bf577 packet: set unique pkt_src 'flush' packets
Set unique type for capture timeout and for detect reload flush
to assist in debugging.
5 years ago
Victor Julien 6bc7636826 stream: remove unused code
Remove now unused 'pkt_src' type as well.

Remove related unittests.
5 years ago
Victor Julien 1633744fcb nfq: remove unused queue handler type 5 years ago