Commit Graph

9306 Commits (eef7760870d99beca75cf96262f4721563198a42)

Author SHA1 Message Date
Eric Leblond 6c9d1c0861 app-layer-expectation: limit number of expectations
This patch introduces a limitation in term of number of
expectations attached to one IPPair. This is done using
a circle list so we have a FIFO approach on expectation
handling.

Circleq list code is copied from BSD code like was pre existing code
in queue.h.
7 years ago
Eric Leblond 03e4bfeb02 app-layer-expectation: remove unused parameter 7 years ago
Jeff Lucovsky 0ae6b0b250 tests/bsize: Fuzzing test case added
This commit adds a test case to validate the issue found during fuzz
testing.
7 years ago
Jeff Lucovsky 5b38bc9894 detect/bsize: Ensure numeric values fit
This commit ensures that the numeric values will not exceed the size of
the containers used to hold them.
7 years ago
Victor Julien 095981cb2a detect/parse: fix crash on 'internal' keyword use
When keyword __flowvar__postmatch__, an internal keyword, is used
in a rule the 'Setup' func ptr will be NULL. This caused a crash.
7 years ago
Victor Julien 1e71eecf47 fuzz/siginit: fix leak in case of bidir sig 7 years ago
Victor Julien 5430141f7a fuzz/siginit: minor improvements
Enable detect engine 'quiet' mode to generate less output.

Set a fake filename so that datarep doesn't hit a reachable assert.
7 years ago
Victor Julien 13c9d0ca7e detect/pkt_data: error on unconsumed transforms
If a rule has transforms w/o consuming them (e.g. a content keyword),
don't consider 'pkt_data' valid.
7 years ago
Victor Julien e1c474a1b0 detect/pkt_data: code and test cleanup 7 years ago
Victor Julien 7f19da1cc0 detect: more robust against transform issues
In case of transform issues (transform not consumed before pkt_data
for example), the code would hit an ugly BUG_ON.

Address this by a more graceful error message, that will still
invalidate the sig but not crash the engine.
7 years ago
Jeff Lucovsky 2823bc5aed detect/tls: Use pcre_copy_substring to avoid leak
This commit eliminates a memory leak while parsing TLS version
information. The leak was identified through fuzzing.
7 years ago
Victor Julien 3d969a1c7d build: wrap fuzz targets in guard to fix 'make tags' 7 years ago
Victor Julien 8cbae1371f fuzz/sigpcap: fix FPs due to missing pkt cleanup 7 years ago
Victor Julien e97cdb48f3 decode/teredo: implement port support
Implement support for limiting Teredo detection and decoding to specific
UDP ports, with 3544 as the default.

If no ports are specified, the old behaviour of detecting/decoding on any
port is still in place. This can also be forced by specifying 'any' as the
port setting.
7 years ago
Shivani Bhardwaj 0e4f261224 Use StringParse* for all parsers and configurations 7 years ago
Shivani Bhardwaj c4c734541a Use appropriate ByteExtractString* functions 7 years ago
Shivani Bhardwaj 6b2c7d5be8 util: Add StringParse* functions
StringParse* functions would perform a stricter check compared to
ByteExtractString* functions. These new functions shall also check if
any extra characters follow the extracted numeric value in addition to
the checks performed by ByteExtractString* and return -1 in that case.
This is particularly important in parser, configuration and setup functions.
7 years ago
Philippe Antoine 293eebd999 fuzz: remove obsolete AFL code 7 years ago
Philippe Antoine bf60959d84 fuzz: simpler way to force usage of CXX linker 7 years ago
Philippe Antoine 440bb4d600 fuzz: remove decodeder fuzz target
As we removed decodeder function
7 years ago
Victor Julien e500c59b99 stream/tcp: fix STREAM_HAS_SEEN_DATA macro
The macro would not return true for smaller TCP streams, leading to
cases where the app-layer was not notified of EOF.
7 years ago
Victor Julien 1618fb1b97 stream/tcp: clean up stream flags 7 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 7 years ago
Pierre Chifflier 333fcc43e7 ssl/tls: call rs_cstring_free for strings allocated in Rust 7 years ago
Pierre Chifflier 1d9f37a60e DER: remove the C parser for DER 7 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 7 years ago
Jeff Lucovsky e0bd79670c detect: byte-test convert neg_op flag to a bool
Only 8 flags are permitted so convert one of them to a struct member. I
choose neg_op
7 years ago
Jeff Lucovsky 313c23a26b detect: Add unittests to exercise bitmask 7 years ago
Jeff Lucovsky d12950c9e4 detect: fixup incorrect comments, indentation 7 years ago
Jeff Lucovsky 31ed9786f6 detect: byte_test impl for bitmask
This commit implements byte_test's bitmask feature.
7 years ago
Victor Julien b85539b2ab stream/tcp: fix fast open off by one
With data on SYN the sequence number used for the first data
was off by one, leading to the next segments to appear to come
after a one byte gap.
7 years ago
Philippe Antoine f51d7d8947 fuzz: check tcp splitting evasions in protocol detection 7 years ago
Philippe Antoine 9eddaa038e fuzz: enable AFLFUZZ_PERSISTANT_MODE for libfuzzer targets 7 years ago
Philippe Antoine ac35118ebe fuzz: use env variable to restrict app layer 7 years ago
Philippe Antoine 600b0d7c55 fuzz: adds eight fuzz targets
And ways to compile them with enable-fuzztargets at configure time
Adds utility function in util-unittest-helper
7 years ago
Frank Honza 1c8943dedd add RFB parser
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
7 years ago
Victor Julien b4d75b7448 output/anomaly: minor code cleanups 7 years ago
Victor Julien 4d21b03575 detect/app-layer-event: code cleanups 7 years ago
Jeff Lucovsky f0bd69e7e3 detect/pcre: Correct spelling typos 7 years ago
Jeff Lucovsky 7f6af10fed general: copyright bump 7 years ago
Jeff Lucovsky 4b0085b03c detect: Update to take advantage of PCRE refactor
This commit changes the keyword detectors to use the refactored PCRE
modifications from detect-parse.[ch]
7 years ago
Jeff Lucovsky abe0cdc4ad detect/pcre: Changes to support pcre_jit_exec
This command causes `pcre_jit_exec` to be used when available. If it's
available and there are allocation errors preparing for it, things
fallback to `pcre_exec`.
7 years ago
Jeff Lucovsky aa67a0a236 detect/pcre: Add warning for failed registrations
This commit adds a warning used by the PCRE detect logic when it fails
to register initialization and free functions for per-thread JIT stack
handling.

This error code is only used when the platform has PCRE JIT exec
functionality.
7 years ago
Jeff Lucovsky d19429f7e5 detect/parse: Refactor interfaces/definitions
This commit refactors existing code patterns to reduce code duplication
and to be a base for supporting additional PCRE jit-related actions.
7 years ago
Philippe Antoine 1cd314c500 detect: adds icmpv6.mtu keyword 7 years ago
Philippe Antoine 75ec528384 detect: adds utility file for uint keywords 7 years ago
Philippe Antoine 0355b70f5a detect: define generic PrefilterIsPrefilterableById 7 years ago
Philippe Antoine bb7abbd6ec detect: extend PrefilterPacketHeaderValue to 16 bytes 7 years ago
Philippe Antoine d04d69acbb decode: fix endianness for ICMPv6 MTU 7 years ago
Philippe Antoine 9d2fabcd99 doc: fix typo in DetectTcpmssSetup description
And use lowercase true for boolean
7 years ago
Victor Julien efee458af8 stream: fix direction flags in corner case
When a TCP DNS flow would start with a GAP on the TS side, the successful
protocol detection on the TC side would trigger 'opposing side' reassembly
and app-layer processing. In this case the stream flags would indicate the
wrong direction and the wrong parser would be called.
7 years ago
Victor Julien 960045323d app-layer: minor code cleanup 7 years ago
Victor Julien 9b1f0656d0 app-layer: fix progress tracking
Esp in combination with GAPs and proto detection.
7 years ago
Victor Julien 21e6f1f063 app-layer: document return macros 7 years ago
Victor Julien 66598f9ce7 nfs: switch to AppLayerResult 7 years ago
Victor Julien 5cd9cfb5a0 smb: update return type of GAP handling 7 years ago
Victor Julien 4bf87d30e4 smb: convert to return AppLayerResult
Support returning 'incomplete' and remove the buffering
code from the parser.
7 years ago
Victor Julien 674b8dc0fb app-layer: add 'incomplete' return logic
Allow app-layer parsers to indicate how much data they need
before being called again.
7 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
7 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
7 years ago
Victor Julien 54b2ec0e4d app-layer: minor optimization 7 years ago
Victor Julien fee7b97ec0 stream: fix function style 7 years ago
Victor Julien 157d01e87e ftp: minor code cleanups 7 years ago
Stephen Donnelly 47082dd5df dag: Skip over ERF_TYPE_META records
Suricata generates an error on unrecognised ERF types.
Suricata should ignore ERF 'Provenance' records with ERF_TYPE_META.
7 years ago
Jason Ish 8a643c893c detect/parse: allow for OK signature parsing errors
The idea of an OK signature parsing error is an error that is
allowed to occur, but still lets test mode pass, unlike
silent errors which will still fail testing.

This is introduced to allow for app-layer event keywords to be
removed, but not have old rules fail out on this case. For example
the Rust DNS parser removes from DNS app-layer events that are
not used anymore.

To signal that an error is OK, -3 is returned. This also implies
silent.
7 years ago
Jason Ish 947cfac62e detect/parse: softer error on unknown app-layer event
On an unknown app-layer event, return -3 for "silent OK fail". A
warning will still be emitted, but its not considered a rule parse
error. This is to handle app-layer events being removed in a more
graceful manner for the user.

This allows -T to pass with an old app-layer events rule file
that may used removed app-layer event keywords.
7 years ago
Jason Ish 688874582a dhcp: remove C app-layer-dhcp wrapper
This just wrapped the Rust function to register the parse,
so instead just call the Rust function directly to remove
the C wrapper, and the 2 C files.
7 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust
And port the C DNS tests to Rust.
7 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
7 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust
Remove app-layer-dns-common.c as its no longer needed.
7 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
7 years ago
Jason Ish 4d312b6813 dns: cleanup: remove C DNS type definitions (dead code) 7 years ago
Jason Ish d809b0959b dns: cleanup: move DnsGetRcode (Lua) to rust
Move the implementation of Lua DnsGetRcode to Rust.
7 years ago
Jason Ish d6b3b33f4a dns: cleanup: remove unused function DNSCreateTypeString 7 years ago
Philippe Antoine 8396333493 detect: adds icmpv6.hdr keyword 7 years ago
Philippe Antoine 02d94f5cf2 detect: fix typo for ipv6.hdr description 7 years ago
Philippe Antoine fc48a7cebf detect: use SC macros for IPV6 header 7 years ago
Philippe Antoine 5f95822f8b doc: fix typo in ByteExtractUint32 description 7 years ago
Jeff Lucovsky c20ab53eae detect/threshold: Correct typos 7 years ago
Jeff Lucovsky ff9a01ee1b detect/threshold: Don't allow duplicates
This commit detects duplicate threshold rule options. When duplicates
are found in a rule, an error message is displayed and the rule is
rejected.
7 years ago
Victor Julien f6c77dcdd9 pcap/file: minor code cleanups 7 years ago
Victor Julien faf5beed2f threads/time: minor code cleanup 7 years ago
Jeff Lucovsky c92975e22b mime: Test cases for filename length limit 7 years ago
Jeff Lucovsky 9a33b5d5de smtp/mime: Fix typos 7 years ago
Jeff Lucovsky 130b8d26e7 smtp/mime: Set event when name exceeds limit 7 years ago
Jeff Lucovsky d0d20bd874 smtp/mime: Restrict file name lengths
This commit places restrictions on the length of the file name specified
in attachments (`name=` or `filename=`) to `NAME_MAX`. Names exceeding
these limits will be truncated and processing will continue with the
truncated name.
7 years ago
Victor Julien 072c421e46 pcap/file: improve time handling
This patch addresses two problems.

First, various parts of the engine, but most notably the flow manager (FM),
use a minimum of the time notion of the packet threads. This did not
however, take into account the scenario where one or more of these
threads would be inactive for prolonged times. This could lead to the
time used by the FM could get stale.

This is addressed by keeping track of the last time the per thread packet
timestamp was updated, and only considering it for the 'minimum' when it
is reasonably current.

Second, there was a minor race condition at start up, where the FM would
already inspect the hash table(s) while the packet threads weren't active
yet. Since FM gets the time from the packet threads, it would use a bogus
time of 0.

This is addressed by adding a wait loop to the start of the FM that waits
for 'time' to get ready.
7 years ago
Victor Julien 4977daec32 threads/time: rename ts to pktts to make purpose clear 7 years ago
Victor Julien ae4137937e pcap/file: fix race during pcap processing start
A race condition during the start of pcap file processing could cause
missed alerts and logged events. This race happens between the packet
threads and the flow manager. It was observed on slower hardware, but in
theory could happen on any machine. It required the 'autofp' runmode.

In commit 6f560144c1 ("time: improve offline time handling") the logic
was added to make the flow manager use a minimum of all the packet threads
perception of time.

The race condition was that the flow manager may become active _before_
all of the packet threads have started processing packets and thus setting
their timestamp. The threads that had not yet initialized their timestamp
would not be considered when calculating the minimum.

As a result of this, older packets timestamps would not yet be registered.
This would give the Flow Manager a timestamp too far in the future. While
the FM was running, the packet processing would start and a flow would
be created. This flow would then immediately be considered 'timed out' by
the FM, due to the timestamp too far in the future.

In the observed case, the thread processing packet 1 from the pcap had not
yet started processing while other threads had already started. The FM was
also already active. Due to the timestamps in the pcap this meant that the
time the FM used was about 500 seconds in the future compared to packet 1.

This patch fixes the issue by initializing all of the threads timestamps
with the timestamp value of the first packet. This way the minimum will
always consider this timestamp.
7 years ago
Victor Julien 1b1855c3bd time: remove unused time structure 7 years ago
Victor Julien 13b4ece11d time: minor code cleanup 7 years ago
Victor Julien 6176338d0b time: fix function name typo 7 years ago
Victor Julien 227e2812f6 pcap/file: fix function ptr naming 7 years ago
Jason Ish a2d91d9bf5 log-pcap: fix log message: unified2 -> pcap
Likely a result of copy and pasted code.
7 years ago
Jason Ish 13635dba57 unified2: not supported message if configured 7 years ago
Jason Ish d86973b386 unified2: remove deprecated output unified2
Ticket 2385:
https://redmine.openinfosecfoundation.org/issues/2385
7 years ago
Jeff Lucovsky e73aec150b pcre: Sticky buffer check
This commit adds logic checking if the sticky buffer in effect provides
the required content.

If the sticky buffer doesn't, the rule will not load and a diagnostic
message with follow-on steps is displayed.
7 years ago
Jeff Lucovsky 8279bab8dc general: Wordsmith "no rules loaded" message 7 years ago
Sascha Steinbiss efaa1755c6 snmp-version: make comment more clear 7 years ago
Sascha Steinbiss f02a18e557 snmp: do not set SIGMATCH_NOOPT 7 years ago
Jeff Lucovsky 6d94b096a9 spelling: Fix spelling error 7 years ago
Jeff Lucovsky 6c3503932f detect/ssl_state: Fix memory leaks from pcre_get_substring
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
7 years ago
Jeff Lucovsky 9fe51a8bd2 detect/flowvar: Fix memory leaks from pcre_get_substring
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
7 years ago
Jeff Lucovsky c2071e1c4e detect/filestore: Fix memory leaks from pcre_get_substring
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
7 years ago
Jeff Lucovsky bcea730266 util-error: define SC_ERR_PCRE_COPY_SUBSTRING 7 years ago
Jeff Lucovsky be4c6b85de general: Convert _Bool to bool
This commit addresses task 3167 and changes usages of '_Bool` to `bool`.
The latter is included from `suricata-common.h`
7 years ago
Jeff Lucovsky 1519c1f006 detect/feature: Support --dump-features 7 years ago
Jeff Lucovsky 5e08e6bccf detect: update version comment 7 years ago
Jeff Lucovsky 1cddbc085b detect: Fix typos/spelling errors. 7 years ago
Jeff Lucovsky 8f6e133497 main: feature init and release 7 years ago
Jeff Lucovsky 9631808b04 output/filestore: announce provided features 7 years ago
Jeff Lucovsky cf840c37b3 detect/filestore: Warn if req'd feature missing 7 years ago
Jeff Lucovsky 31e2cff9cb detect/analysis: Warn if required feature missing 7 years ago
Jeff Lucovsky f83b199f2a feature: feature tracking services 7 years ago
Philippe Antoine 5597801ccb perf: optimizes alert fast log
In the case when we have multiple alerts for one packet
We suppose this happens more often than having decoder_event != 0
7 years ago
Phil Young 1c99536945 napatech: add hardware based bypass support
Napatech hardware bypass support enables Suricata to utilize
capabilities of Napatech SmartNICs to selectively bypass flow-based
traffic.
7 years ago
Philippe Antoine 52970d8508 detect/parse: move spaces skip up the stack
Switch to isspace() as well.
7 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 7 years ago
Jeff Lucovsky 427ec4e739 decode: Fix typos/spelling 7 years ago
Jeff Lucovsky aec4e9a032 decode: Handle ERSPAN Type I 7 years ago
Philippe Antoine af2a689a99 init: move main to a separate file
so as to reuse code from suricata.c with fuzz targets
7 years ago
Philippe Antoine bb42be7e08 fuzz: disable signal handlers while fuzzing 7 years ago
Philippe Antoine 47eba33110 init: make PostConfLoadedSetup global
to be used by fuzz targets
7 years ago
Philippe Antoine 913082941c init: fix PostConfLoadedSetup to use local suri ptr 7 years ago
Philippe Antoine d2b3668a68 init: InitGlobal function
To be reused by fuzz targets
7 years ago
Philippe Antoine f5190da67e util: UTHmemsearch to use memmem if defined 7 years ago
Philippe Antoine ce55d06569 check: Using const keyword for some arguments
For ConfigSetLogDirectory and PrintRawLineHexBuf
7 years ago
Philippe Antoine 12a3a24906 log: can use a file set from env variable
Enables the redirection of log to a file set by an environment
variable SC_LOG_FILE when SC_LOG_OP_IFACE=file
7 years ago
vanlink 2456f27d08 stream/reassembly: fix data overlap check
Fix function CheckOverlap bug.
7 years ago
Victor Julien aeefc82eb9 tls: fix missing extern logic for cert_id tracking 7 years ago
Victor Julien 62c0f3d2b4 stats: fix missing extern keyword 7 years ago
Victor Julien 903291f88a defrag: fix use of globals 7 years ago
Victor Julien 5c3c6c609c threading: fix queue handlers globals use 7 years ago
Victor Julien 3ae1854d2f htp: fix globals use for flags 7 years ago
Victor Julien 85289f3283 proto: fix globals use 7 years ago
Victor Julien 5e583f3a12 flow: fix global variable use 7 years ago
Victor Julien 29f54a34ae stream: fix global declaration of the config 7 years ago
Victor Julien 2436daccd9 threading/modules: fix global declarations 7 years ago
Victor Julien c5f4b41881 ippair: fix global declarations 7 years ago
Victor Julien 0a006d2258 host: fix global declarations 7 years ago
Victor Julien 29780d6164 mpm: fix global declarations 7 years ago
Victor Julien b89059bda7 detect: fix global declaration of sigmatch_table 7 years ago
Victor Julien 0118e07d57 spm: fix global declaration of spm_table 7 years ago
Victor Julien a12c0b499d threading: fix global declaration of threading_set_cpu_affinity 7 years ago
Victor Julien 45955d2e58 unix-socket: avoid using global variable w/o extern 7 years ago
Victor Julien a9a522fac3 decode: fix default-packet-size global variable 7 years ago
Victor Julien 7709b90c16 detect/file-data: remove debug abort that wasn't reachable 7 years ago
Victor Julien ac8ceae9bf detect/file-data: fix function doc 7 years ago
Victor Julien 500e8da63a files: tracking flag update
Improve flow file flags and file flags updates. Introduce a mask
that is set at start up to avoid lots of runtime checks.

Disable cocci flags check as it doesn't support the more dynamic
nature of the flag updates.
7 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
7 years ago