Commit Graph

11275 Commits (d720ead470bcb5dd5a0c0ae7db302ab170205ee6)

Author SHA1 Message Date
Victor Julien 381e9f2bdf mpm/ac: remove dead code 4 years ago
Victor Julien f33784bed3 mpm/ac: use FatalError instead of Critical and exit 4 years ago
Victor Julien dcf2f6b2be detect: more compact message on error 4 years ago
Victor Julien 8b846bdcff af-packet: output cleanups and unification
Make all messages use the same format:

<iface>: <message>
4 years ago
Victor Julien 806104c09a output: issue warning if file can't be opened 4 years ago
Victor Julien 79c84211d5 output: remove all custom errors 4 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 4 years ago
Victor Julien 39f5c7f56a error: use SC_EINVAL for invalid input 4 years ago
Victor Julien e855fa0fe7 error: remove unused alloc error code 4 years ago
Victor Julien e042cd785e error: use SC_ENOMEM for alloc errors 4 years ago
Victor Julien d2c98c3281 logopenfile: errno style error handling 4 years ago
Victor Julien 1e36db9d0e hashlist: use errno style error handling 4 years ago
Victor Julien 0efdab1f5a membuffer: errno style error reporting 4 years ago
Victor Julien 8ef95c10a8 pool: use errno style error reporting 4 years ago
Victor Julien e621416d80 error: bootstrap errno like error code approach 4 years ago
Victor Julien cdd52dc8cd detect/mpm: output cleanups 4 years ago
Victor Julien cdac0425a8 device: special case handling of dpdk devices 4 years ago
Victor Julien d84bcdfd5c device: output tweak 4 years ago
Victor Julien c78ebd1352 dpdk: set RSS flags for igb driver
Tested on i340.
4 years ago
Victor Julien b08e3e7460 dpdk: ips output cleanups 4 years ago
Victor Julien 70b69a3171 output: minor code cleanups 4 years ago
Victor Julien 10c26bb628 unix-socket: don't print unix socket message twice 4 years ago
Jeff Lucovsky 82158f92c1 general: Correct typos 4 years ago
Victor Julien be895d67b6 runmodes: minor output cleanup 4 years ago
Victor Julien cef0a84bd2 detect: turn cleanup message into debug message 4 years ago
Victor Julien b76f6dfa42 device/mtu: clean up mtu message 4 years ago
Victor Julien 7ccfa9b9a6 device: cleanup stats print 4 years ago
Victor Julien fd4e522038 runmode: log test mode later 4 years ago
Victor Julien 181400bfdd pcap-file: output cleanups 4 years ago
Victor Julien ef881c942b af-packet: output cleanups 4 years ago
Victor Julien 393546af07 mpm/hs: turn cleanup messages in to debug 4 years ago
Victor Julien 892b7021b2 stream: debug message cleanup 4 years ago
Victor Julien d973e1021c threading: memset cpu_set_t to address cppcheck warning
src/tm-threads.c:734:5: error: Uninitialized variable: cs [uninitvar]
    CPU_ZERO(&cs);
4 years ago
Victor Julien 58c28d1778 dpdk: fix cppcheck warnings
src/runmode-dpdk.c:202:11: warning: Size of pointer 'argv' used instead of size of its data. This is likely to lead to a buffer overflow. You probably intend to write 'sizeof(*argv)'. [pointerSize]
    args->argv = SCCalloc(capacity, sizeof(args->argv));
          ^
src/runmode-dpdk.c:777:23: error: Shifting 32-bit value by 63 bits is undefined behaviour [shiftTooManyBits]
        if (bits & (1 << i))
                      ^
src/runmode-dpdk.c:776:23: note: Assuming that condition 'i<64' is not redundant
    for (int i = 0; i < 64; i++) {
                      ^
src/runmode-dpdk.c:777:23: note: Shift
        if (bits & (1 << i))
                      ^
4 years ago
Victor Julien 17cd41c887 frames: suppress cppcheck warning
src/app-layer-frames.c:471:5: warning: Identical condition 'stream_slice->input==NULL', second condition is always false [identicalConditionAfterEarlyExit]
    BUG_ON(stream_slice->input == NULL);
    ^
src/app-layer-frames.c:468:29: note: If condition 'stream_slice->input==NULL' is true, the function will return/exit
    if (stream_slice->input == NULL)
                            ^
src/app-layer-frames.c:471:5: note: Testing identical condition 'stream_slice->input==NULL'
    BUG_ON(stream_slice->input == NULL);
    ^
src/app-layer-frames.c:548:5: warning: Identical condition 'stream_slice->input==NULL', second condition is always false [identicalConditionAfterEarlyExit]
    BUG_ON(stream_slice->input == NULL);
    ^
src/app-layer-frames.c:545:29: note: If condition 'stream_slice->input==NULL' is true, the function will return/exit
    if (stream_slice->input == NULL)
                            ^
src/app-layer-frames.c:548:5: note: Testing identical condition 'stream_slice->input==NULL'
    BUG_ON(stream_slice->input == NULL);
    ^
4 years ago
Victor Julien 16c19bd2f6 capture: remove unnecessary mtu check 4 years ago
Victor Julien f45914cd99 classification: avoid duplicate errors 4 years ago
Victor Julien 7d46d4217f unix-socket: avoid duplicate errors 4 years ago
Victor Julien 4dfb584164 dnp3: no error logging in packet path 4 years ago
Haleema Khan 05e87bf32e detect-detection: Convert unittests to new FAIL/PASS API
Ticket: #4019
4 years ago
Haleema Khan 6094aeee9e detect-detection: Apply clang-format 4 years ago
Victor Julien eb69902719 dpdk: NUMA output adjustments
Reduce verbosity of the NUMA logs; switch them to start with the
iface.

Don't print the cpu number.
4 years ago
Haleema Khan 4a98c4bde6 detect/sigorder: Convert unittests to new FAIL/PASS API
Ticket: #4029
4 years ago
Lukas Sismis 563f8bb56b dpdk: decrease intensity of warnings related to NUMA placement
Ticket: #5617
4 years ago
Victor Julien 991f9fde32 radix: fix ipv6 address parsing warning
The check meant to see if the ip address part of the ip/cidr combo
was more specific than needed wasn't fully implemented, leading to
warnings being issued on completely valid and correct input.

This patch implements the same logic as in IPv4. If the ip address
as specified is different from the ip after the mask has been applied,
a warning is displayed.

Bug: #5747.
4 years ago
Bazzan Don b289030696 detect-engine-tag: convert unittests to new FAIL/PASS API
Ticket: #4030
4 years ago
Philippe Antoine a58ffe5b3e fuzz: do not use timestamps at the end of times
so as not to have integer overflows
4 years ago
Philippe Antoine 190c945e67 ftp: disables debug assertion
Ticket: #5443

Until we decide on the right way to fix this.
4 years ago
Philippe Antoine 4daee8bae1 quic: do not use stack for inspection buffer
Make the inspection buffer copy the content in one buffer it owns.

Ticket: #5707
4 years ago
Jason Ish 4a7567b3f0 template: rename template-rust to template
Remove the distinction between the C template protocol "template" and
the Rust template protocol "template-rust" and make the Rust parser
simply template now that we no longer have support to generate a C
protocol template.
4 years ago
Jason Ish 8683154115 templates: remove C app-layer templates 4 years ago
Jason Ish 05e16820de templates: clang format cleanups
Cleanup the trivial clang-formatting issues in templates.  Length of
protocol names may require clang-format after new protocol generation.
4 years ago
Jason Ish 50a787a9a3 app-layer-template-rust: remove C app-layer stub
Remove the app-layer-PROTO stub for Rust based parsers.  It is no longer
needed as Rust parsers now contain the registration function in Rust.

Ticket: 4939
4 years ago
Victor Julien 7bfb63210e mime/base64: unify space handling 4 years ago
Philippe Antoine 0e054af1ac mime: do not skip empty lines for quoted-printable
As these lines are in the file...

Ticket: #5725
4 years ago
Philippe Antoine 04fd2ae78e dns: do not oputput empty array for query 4 years ago
Richard McConnell 6e128f48a2 af-xdp: Add AF_XDP socket support
AF_XDP support is a recent technology introduced that aims at improving
capture performance. With this update, Suricata now provides a new
capture source 'af-xdp' that attaches an eBPF program to the network
interface card. Packets received in the NIC queue are forwarded to
a RX ring in user-space, bypassing the Linux network stack.

Note, there is a configuration option (force-xdp-mode) that forces the
packet through the normal Linux network stack.

libxdp and libbpf is required for this feature and is compile time
configured.

This capture source operates on single and multi-queue NIC's via
suricata.yaml. Here, various features can be enabled, disabled
or edited as required by the use case.

This feature currently only supports receiving packets via AF_XDP,
no TX support has been developed.

Ticket: https://redmine.openinfosecfoundation.org/issues/3306

Additional reading:
https://www.kernel.org/doc/html/latest/networking/af_xdp.html
4 years ago
Todd Mortimer 7d1a8cc335 file/swf: Use lzma-rs decompression instead of libhtp.
Use the lzma-rs crate for decompressing swf/lzma files instead of
the lzma decompressor in libhtp. This decouples suricata from libhtp
except for actual http parsing, and means libhtp no longer has to
export a lzma decompression interface.

Ticket: #5638
4 years ago
Victor Julien 9ed65907a7 fuzz/sigpcap: set pkt_src 4 years ago
Victor Julien f848e34bcc stream: stricter check inserting segments
In lossy streams, esp where TcpSession::lossy_be_liberal it is possible
to end up inserting a segment that is out of the expected sequence
number bounds.
4 years ago
Victor Julien 5b6193f4c4 flow: cleanup and clarify ancient debug messages 4 years ago
Victor Julien 03d049dadc decode: enforce layer limit through tunnel layers
Bug: #5686.
4 years ago
Shivani Bhardwaj f80c999db3 util/base64: fix heap buffer overflow
While updating the destination pointer, we were also adding the padded
bytes which are not a part of the decoded bytes. This led to running out
of space on the destination buffer.
Fix it by only incrementing destination buffer ptr by the number of
actual bytes that were decoded.

Ticket 5623
4 years ago
Victor Julien 9f4dd4fc56 smtp/files: don't modify prev file on open failure 4 years ago
Victor Julien e601ebdfd8 files: always initialize inspect_window and min_inspect_depth
This is to make sure the files buffers are properly managed even
when there are no rules or when there are no file.data rules.

Bug: #5703.
4 years ago
Victor Julien df7d8d96c9 streaming/buffer: set hard limit on buffer size
Don't allow the buffer to grow beyond 1GiB. Add a once per thread
warning if it does reach it.

Bug: #5703.
4 years ago
Victor Julien 2edfff7a0c src: unify how warnings specify ticket id's 4 years ago
Jason Ish 0c00f28ebc afpacket/netmap: warn about mixed ips, ids/tap deprecation
Suricata already logs if AF_PACKET or Netmap are running in a mixed IPS
and IDS/TAP mode.  As the behavior is undefined when these modes are
mixed, it is best to deprecate and to not allow this behavior. For now
warn that it will be unsupported and fail in Suricata 8.

Ticket: 5587
4 years ago
Philippe Antoine 3e4f58e375 detect: fix memory leak when parsing signature
Ticket: #5529
4 years ago
Victor Julien 1e653cc36d profiling: fix includes 4 years ago
Victor Julien 50b858aa49 ipfw: fix missing include 4 years ago
Victor Julien f3f2807202 netmap: fix missing include 4 years ago
Jason Ish dcd9dabc70 classification: continue processing on parse error
Instead of returning on the first line that fails to parse, log the
error and continue instead of returning.

The fail fast makes sense in test mode, but not in a normal run mode
where you don't want one bad line to abort processing the whole file.

This will still fail out in test mode.

Related issue: 4554
4 years ago
Philippe Antoine ad713246a9 src: remove double includes
Keep the unconditional include to be sure it works

git grep '#include "' src/*.c | sort | uniq -c | awk '$1 > 1'
4 years ago
Philippe Antoine 9af0dafbad src: fix some include orders
So as to be able to get include removal right
4 years ago
Philippe Antoine cc23923de1 src: remove obsolete comment
Should have been removed along by commit
82dba07579
4 years ago
Philippe Antoine 62352ad030 src: fix remaining cppclean warnings 4 years ago
Philippe Antoine 1f066cbbe8 unittest: fix unneeded includes as per cppclean
Especially because there is conditional inclusion from a header
4 years ago
Philippe Antoine 662f0ce503 util: fix includes for util-memcmp
u8_tolower is now in suricata-common.h

Fixes commit 19e94e93fa
4 years ago
Philippe Antoine e85f3916e3 src: fix integer warnings
and adds defrag debug validations
4 years ago
Philippe Antoine b5147189ae tls: fix off by one in supported versions extension
Ticket: #5663
4 years ago
Jeff Lucovsky a4239d433a detect/bsize: Validate bsize values after parsing
Issue: 2982

This commit moves bsize validation with respect to content matches to
the post-parse validation stage. This allows bsize to consider all
content-related values, including those that follow the bsize keyword.
4 years ago
Jeff Lucovsky 9d73777a46 bsize/general: Remove unnecessary includes
This commit removes unused/commented out #include lines.
4 years ago
Jeff Lucovsky 8b41754acd add to general: Typo fixup 4 years ago
Jeff Lucovsky 8df6701186 netmap: Fix include file issues 4 years ago
Jeff Lucovsky ecfdc24e08 detect/bsize: Semantic validation of bsize values
This commit adds validation of the bsize value(s) with the available
buffer size. Signatures are flagged if the bsize and buffer size are
incompatible.

Issue: 3682
4 years ago
Jeff Lucovsky 25c0a6ea7c tests/bsize: Test cases with preceding content
This commit adds test cases that validate behavior when "content"
immediately precedes "bsize".
4 years ago
Jeff Lucovsky c91b987732 general: Typo fixup 4 years ago
Jeff Lucovsky 115297c016 detect/content: Use SCCalloc instead of malloc/memset
This commit replaces a SCMalloc/memset with SCCalloc
4 years ago
Jeff Lucovsky 7eb5fb1826 detect/content: Consider distance in validation
Ticket: 2982

This commit validates that the content usage in a rule will not exceed
the dsize value.

Values of distance that cause the right edge to be exceeded are
considered an error and the signature will be rejected.
4 years ago
Shivani Bhardwaj 58e5033a44 util/base64: use padding var for calculations 4 years ago
Victor Julien ca6ce935b5 security: disable setrlimit with asan
Various parts of ASAN will require creation of threads. At least
LSAN reporting and ASAN stack traces require this. Detect if we
run with ASAN and bypass the noproc setting with a warning.

Bug: #5661.
4 years ago
Victor Julien 8f0d820218 ssl: add debug validation checks for recent changes
Make sure the assumptions are correct.
4 years ago
Philippe Antoine 3fbcba8ab7 tls: fix buffer overread
Recently introduced by commit 4bab6e24e5

Ticket: #5564
4 years ago
Philippe Antoine 4706083929 detect: avoids memory leak on ja3 signature parsing
If a later keyword enforces a protocol incompatible with ja3
4 years ago
Philippe Antoine c09ca7d171 clean: use SC_MD5_HEX_LEN instead of magic number 32 4 years ago
Philippe Antoine 7823653a30 clean: replace MD5_STRING_LENGTH with more used SC_MD5_HEX_LEN 4 years ago
Philippe Antoine 5a0d07b13f clean: remove useless * sizeof(char) 4 years ago
Philippe Antoine f75d18b077 quic: ja3 code deduplication
As quic transactions are unidirectional, the same function is
used to get ja3 from client or from server.
4 years ago
Philippe Antoine 7cb40a1dfc quic: allow ja3.hash keyword
Ticket: #5624
4 years ago
Daniel Young aeb0c0e71a detect/parse-port: fix whitespaces bug
Updated by Victor Julien to:
- accept but strip leading whitespaces
- update tests
- cleanup tests
4 years ago
Eric Leblond ea85e2ccf2 detect/filename: fix buffer description 4 years ago
Shivani Bhardwaj 82dba07579 src: remove unneeded header and refs 4 years ago
Haleema Khan df45fc9e97 detect-l3proto: Convert unittests to new FAIL/PASS API
Ticket: #4050
4 years ago
Alice Akaki d46ce170ba detect-icmp-seq: clean-up unittests
Task: #5597
4 years ago
Eric Leblond 4e1ff9a7ff flow: fix FlowSwapFileFlags function
Ticket: #5625
4 years ago
Haleema Khan 160c778a1f detect-filesize: Convert unittests to new FAIL/PASS API
Ticket: 4039
4 years ago
Haleema Khan f25d4cc546 detect-distance: Convert unittests to new FAIL/PASS API
Ticket : #4020
4 years ago
Alice Akaki f251b6af65 detect-icmp-id: convert unittests to FAIL/PASS APIs
Task: #4042
4 years ago
Haleema Khan 1fdd56a61d detect-itype: Convert unittests to new FAIL/PASS API
Bug: #5589
4 years ago
Simeon Miteff 6673846481 profiling: Make rule_perf.log JSON output newline-delimited
Adding a newline makes the output valid JSON when multiple sortings of
the rules profile object are written to the log.
4 years ago
Jason Ish 0d3cfbbe3f bittorrent-dht/eve: log as bittorrent_dht 4 years ago
Aaron Bungay 86037885a9 bittorrent-dht: add bittorrent-dht app layer
Parses and logs the bittorrent-dht protocol.

Note: Includes some compilation fixups after rebase by Jason Ish.

Feature: #3086
4 years ago
Jason Ish 27672c950c dnp3: fixups to work with unified json tx logger
Update DNP3 to work with a single TX logger, and just register one
logger instead of 2.

This primarily creates a TX per message instead of correlating replies
to requests, which fits the DNP3 model better, but we didn't really have
this concept nailed down when DNP3 was written.
4 years ago
Victor Julien 2f9ca8bb34 dpdk: set new running flag at thread startup 4 years ago
Victor Julien 91b7b63ed2 threads: count thread types after all initialized
To avoid double counting in case a thread took longer than
expected to start up.
4 years ago
Alice Akaki 6621b0ec93 detect-icmp-seq: convert unittests to FAIL/PASS APIs
Task: #4043
4 years ago
Haleema Khan 8d5c5f24a1 dns/eve: add 'HTTPS' type logging
Add a new DNS record type to represent HTTPS
Ticket: #4751
4 years ago
Haleema Khan b1972a5c61 detect-itype: remove unittests
Ticket: #5590
4 years ago
Gabriel Lima Luz 4b009eb907 detect-replace: Convert unittests to FAIL/PASS API
Ticket: 4054
4 years ago
Philippe Antoine a003640ecf security: prevents process creation
with setrlimit NPROC.

So that, if Suricata wants to execve or such to create a new process
the OS will forbid it so that RCE exploits are more painful to write.

Ticket: #5373
4 years ago
Jason Ish 2ab3646fad profiling: sort LoggerId's in same order as defined
Sort the LoggerId's in the order they are define in suricata-common.h.
4 years ago
Jason Ish 99cb8c666a loggers: all json tx loggers can share the same loggerid
This is to avoid the tx logging code that doesn't support LoggerId
values over 31 at this time. The simplest fix for now is to just have
all JSON (eve) loggers use the same ID.

DNP3 is left as-is for now as it needs some extra support in the parser.
4 years ago
Richard McConnell 13beba141c source: add THV_RUNNING flag to notify of running state
Each module (thread) updates its status to indicate running.
Main thread awaits for all threads to be in a running state
before continuing the initialisation process

Implements feature 5384
(https://redmine.openinfosecfoundation.org/issues/5384)
4 years ago
Eric Leblond 94664ef565 datasets: don't exit on invalid data 4 years ago
Eric Leblond 3bd48d9336 detect: doc link for ip.src and ip.dst 4 years ago
Eric Leblond 7518204ad4 datasets: introduce new IPv6 type
This patch also simplifies IPv6 parsing.

Feature: #5383
4 years ago
Eric Leblond 6fe9d510cf detect: add ip.dst keyword as sticky buffer
Feature: #5383
4 years ago
Eric Leblond 7e516aad94 detect: add ip.src keyword
It is a sticky buffer matching on src_ip.

Feature: #5383
4 years ago
Eric Leblond b2cdc6c899 datasets: introduce ipv4 type
This patch introduce the IPv4 type for dataset so Suricata commandmatch
on a set of IPv4 addresses. This is meant to complement iprep feature
for people that needs more flexibility such as settings the IP on
the packet path.

Feature: #5383
4 years ago
Eric Leblond a9c05c7d96 datasets: factorize serialised operations
Ticket: #5184
4 years ago
Eric Leblond 843dba0a28 datasets: add dataset-lookup command
Ticket: #5184
4 years ago
Eric Leblond 2f25e48897 datasets: add dataset-clear command
Ticket: #5184
4 years ago
Eric Leblond a480abcdd0 datasets: add dump via unix socket
This patch adds a dataset-dump command to the list of unix socket
commands. Implementation is not optimal as we are locking the
datasets when doing the dump. But if we consider that the current
alternative from an implementation point of view is to stop Suricata
then this is far better than current state.

Ticket: #5184
4 years ago
Victor Julien 1fafb83fed packet: turn tunnel lock into spinlock
Lock is only held to update/check ints, so spin lock will be more
efficient.

Place the member of Packet in a new "persistent" area to make it
clear this is not touched by the PacketReinit logic.

Ticket: #5592.
4 years ago
Victor Julien 57e70841c4 stream/tcp: remove obsolete and commented out tests 4 years ago
Victor Julien e72770c1b2 decode/vxlan/tests: don't memset new packet 4 years ago
Victor Julien edf93ae5b6 decode/mpls/tests: improve pkt handling; cleanups 4 years ago
Victor Julien 0f7fe2a4c3 app-layer/tests: don't memset new packet 4 years ago
Victor Julien 6dc53447f1 decode/geneve/tests: don't memset packet
Packet is already initialized.
4 years ago
Victor Julien b07c7ad14c threading: improve/add thread queues explanations 4 years ago
Victor Julien 951bcde0b2 eve/alert: remove tunnel locking
Tunnel lock is only used to sync verdict logic.
4 years ago
Victor Julien 0e7adc21a6 decode: alloc packets using calloc 4 years ago
Victor Julien 68a9da52ad packetpool: remove PKT_ALLOC flag
Use Packet::pool instead. If Packet::pool is non-NULL the packet is
owned by a pool. Otherwise it is allocated and should be freed after
use.
4 years ago
Victor Julien 3ed7b4473e runmodes: remove dead error check 4 years ago
Victor Julien 9d3c60bde3 smtp/mime: no error logging in packet path 4 years ago
Victor Julien 256f0f2c5c defrag: no error logging in packet path 4 years ago
Victor Julien 8f02a3e415 applayer: make sure to use correct ipproto 4 years ago
Victor Julien dba7103a96 eve/alert: use flow proto to avoid crash on ICMP 4 years ago
Alice Akaki 51c0714d71 detect-filemd5: convert unittests to FAIL/PASS APIs
Task: #4035
4 years ago