aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,603 Commits
7 Branches
161 Tags
209 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c41e64d637'
${ noResults }
Commit Graph

9751 Commits (c41e64d637fc0ce7a95e68c2de08f4d4ea24f49f)

Author SHA1 Message Date
Jason Ish 99f460aa5a eve/fivetuple: use intermediate address struct (jsonbuilder prep) 
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.

As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
 5 years ago
Jason Ish cc4f9d7f3d alert/eve: remove jansson specific feature (jsonbuilder prep) 
Remove the Jansson specific feature of being able to delete
an object from json_t, in prep for refactors to JsonBuilder.

Instead create a new header for each alert to be logged.
 5 years ago
Jason Ish f4f1fdbf86 alert/eve: move logging of rule text (jsonbuilder prep) 
Move the logging of the rule text to where the alert object
is being logged to remove the usage of json_object_get...

Getting previously logged objects will not be possible with
JsonBuilder.
 5 years ago
Victor Julien f8f2a2bbc0 detect/pcre: set app proto correctly when using modifiers 5 years ago
Victor Julien 9fd56e8430 detect/pcre: minor code cleanups 5 years ago
Victor Julien de6c9b9441 detect: clarify and slightly cleanup non-pf logic 5 years ago
Victor Julien 5acfdfcc76 flow/manager: fix management tasks not running 
Fix tasks not running on the first manager, even if there is just
a single manager.
 5 years ago
Eric Leblond ae5650d443 magic: get rid of global lock 
Global magic context was involving a lock that appear to be really
costly for some traffic.
 5 years ago
Victor Julien d8c82d4f39 af-packet: fix warnings by undefined sanitizer 5 years ago
Victor Julien 3957750731 capture: optimize checksum handling 
Don't use a flag in the livedev, but overwrite the config setting after
'auto' mode has determined checksums should be disabled.
 5 years ago
Victor Julien bbdc11842d windows: fix timestring timezone display 
Bug: #3690
 5 years ago
Jeff Lucovsky 12148bc53c detect/pcre: Use the keyword context for JIT stack 
When PCRE `jit` is available, store the JIT stack in the keyword context
instead of on a global id. This ensures proper cleanup and
re-initialization over a rule reload.
 5 years ago
Victor Julien d1e690ccb3 profiling: c11 atomics fixup 5 years ago
Jason Ish ca88e4d0e3 filestore v1: remove 
File store v1 has been deprecated and was scheduled for removal
by June 2020.

Log an error if a file-store configuration is loaded without
version set to 2.
 5 years ago
Philippe Antoine 69b4fffdae parse: move SSH parser from C to Rust 5 years ago
Shivani Bhardwaj 6457754fd6 dcerpc: Replace C function calls with Rust 
All the dead code in C after the Rust implementation is hereby removed.
Invalid/migrated tests have also been deleted.
All the function calls in C have been replaced with appropriate calls to
Rust functions. Same has been done for smb/detect.rs as a part of this
migration.
 5 years ago
Philippe Antoine 2fe82ce0d6 fuzz: do not reuse global variable named suricata 5 years ago
Philippe Antoine 304aedfa95 fuzz: improves sigpcap target 
So that it can cover alert generation
ie in function DetectRun, get past scratch.sgh == NULL condition
 5 years ago
Jeff Lucovsky 690bd14371 napatech: Fix parameters passed to thread-check 
This commit corrects an error introduced earlier: the call to
`TmThreadsCaptureHandleTimeout` is passing too many parameters.
 5 years ago
Jeff Lucovsky 9db8a917a2 dag: Fix parameters passed to thread-check 
This commit corrects an error introduced earlier: the call to
`TmThreadsCaptureHandleTimeout` is passing too many parameters.
 5 years ago
Victor Julien a0392c6027 fuzz/sigpcap: enable protocols, add more outputs 5 years ago
Victor Julien 032f31b7d3 htp: fix test after libhtp changes 5 years ago
Jeff Lucovsky beb45c564e detect/smtp: Refactor command check 
This commit refactors the code that matches reply with command.

Bug: #3677
 5 years ago
Jeff Lucovsky dc7a991bfb app-layer/smtp: Improve RSET handling 
This commit improves how the parser handles the `RSET` command.
Termination of the transaction occurs when the `RSET` ack is seen (reply
code 250).

Bug: #3677
 5 years ago
Philippe Antoine a15e503b7d enip: more precise probing parser 
Bug: #3615
 5 years ago
Victor Julien 049c5fe230 detect/port: limit recursion in port parsing 
Bug: #3586
 5 years ago
Victor Julien 476b5f21f3 detect/address: limit recursion during parsing 
Allow a max depth of 64.

Bug: #3586
 5 years ago
Victor Julien b6658e6269 detect/address: minor cleanups 5 years ago
Victor Julien 41d0dcae99 decode: cleanup packet properly on bad packets 
In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar
structures would not be cleaned up because the cleanup depends on the
'header' pointer being set, but the error handling would unset that.

This could mean these structures were already filled with values before
the error was detected. As packets were recycled, the next packet decoding
would use this unclean structure.

To make things worse these structures are part of unions. IPv4/IPv6 and
TCP/ICMPv4/ICMPv6 share the same memory location.

LibFuzzer+UBSAN found this both locally and in Oss-Fuzz:

decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool'
    #0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9
    #1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13
    #2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9
    #3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25
    #4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
    #5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
    #6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
    #7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5
    #8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6
    #9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
    #10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8)

Bug: #3496
 5 years ago
Victor Julien 3ed188e0bc ssl: support multi-frag certificate assembly 
Support reassembling multi-frag certificates. For this the cert queuing
code is changed to queue just the cert, not entire tls record.

Improve message tracking. Better track where a message starts and ends
before passing data around.

Add wrapper macros to check for 'impossible' conditions that are activate
in debug validation mode. This helps fuzzers find input that might trigger
these conditions, if they exist.
 5 years ago
Victor Julien 4f679fd843 ssl: add asserts for 'impossible' conditions 
Wrap in debug validation so that fuzzing can pick them up.
 5 years ago
Victor Julien 68d5a9dc2c tls/sni: parsing cleanup 
Set proper event on all invalid sni length values.
 5 years ago
Victor Julien 61b8c99236 ssl: improve error checking 5 years ago
Victor Julien bb06298102 ssl: unify main parsing routine 5 years ago
Victor Julien 40be9d2219 ssl: improve debug output 5 years ago
Victor Julien f1bf11f716 ssl: record parsing cleanup 5 years ago
Victor Julien ab44b5edac ssl: handshake parsing code cleanup 5 years ago
Victor Julien d1ada2e13c ssl: copy data using a safe memcpy wrapper 
To avoid future memcpy issues introduce a wrapper and check the
result of it.

When compiled with --enable-debug-validation the wrapper will abort if
the input is wrong.
 5 years ago
Victor Julien cffbdff024 ssl: don't say we consumed bytes if we didn't consume them 5 years ago
Victor Julien 9950ebffe6 ssl: code cleanups 5 years ago
Victor Julien 1578c84605 ssl: bump copyright year 5 years ago
Victor Julien 9a97821c43 ssl: improve 'first cert' check to avoid leaks 
In some error conditions, or potentially in case of multiple 'certificate'
records, the extracted subject, issuerdn and serial could be overwritten
without freeing the original memory.
 5 years ago
Victor Julien fa2a1385ea ssl: fix handshake cert buffer sizing 
'trec' buffer was not grown properly when it was checked as too small.
After this it wasn't checked again so that copying into the buffer could
overflow it.
 5 years ago
Victor Julien 26bcc97515 detect/keywords: dynamic version part of doc URL 5 years ago
Victor Julien 8f2df0f938 logging: fix default log format for release mode 5 years ago
Victor Julien eef7760870 datasets: reputation value validation 5 years ago
Philippe Antoine ae102ca096 detect: refactoring parsing of ip range 
To optimize first netmask
 5 years ago
Philippe Antoine 8ca9c0e8f0 signature: minimizes ip CIDR for ip range 
Example leading to over allocation is 41.232.107.2-43.252.37.6
 5 years ago
Jeff Lucovsky fa082d04dc decode/erspan: Warn on ERSPAN Type I config 
This commit checks whether pre-6.x settings for ERSPAN Type I are
present. ERSPAN Type I is no longer enabled/disabled through a
configuration setting -- it's always enabled.

When a setting exists to enable/disable ERSPAN Type I decoding, a
warning message is logged.

Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x
 5 years ago
Jeff Lucovsky 82da71bbc4 decode/erspan: Add warning ERSPAN Type I config 
This commit adds a warning value when ERSPAN Type I configuration
settings are detected; specifically, when ERSPAN Type I `enabled` is
specified.

Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x
 5 years ago
Victor Julien ad448da3f4 flowbits: fix hang in flowbits 'or' parsing 5 years ago
Victor Julien 07ed0dadae fuzz: suppress too noisy htp errors check 5 years ago
Jeff Lucovsky 2b93898771 napatech: Correct timestamp rounding issue 
This commit fixes the conversion of timestamps. Without the extra
parens, the resulting timestamp value for usecs will be 1 or 0 due to
the operator precedence order (+ takes precedence over ?:)
 5 years ago
Jeff Lucovsky f12adcc58c napatech: Check for out-of-band control operations 
This commit causes the packet source to check for out of band control
operations when there are no packets immediately available.
 5 years ago
Jeff Lucovsky 5b13468bfc dag: Check for out-of-band control operations 
This commit causes the packet source to check for out of band control
operations when there are no packets immediately available.
 5 years ago
Phil Young b48049c51c napatech: Restructure Packet/Hostbuffer release 
The end-of-processing has been restructured so that Packet and Hostbuffer
data structures are now released within the NapatechReleasePacket() callback
function.
 5 years ago
Shivani Bhardwaj e22b345bb6 af-packet: change type of cluster_id to uint16_t 5 years ago
Shivani Bhardwaj e7c0f0ad91 src: remove multiple uses of atoi 
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* functions.

Partially closes redmine ticket #3053.
 5 years ago
Shivani Bhardwaj 92bb52f430 Add wrappers for validating range checks 5 years ago
Victor Julien c2d36ed261 fastlog: copyright year bump and remove stale comments 5 years ago
Victor Julien 28837b203e fastlog: fix unlikely memleak 
Fix memleak is case of alloc error during startup.
 5 years ago
Victor Julien b763885d1b thash: suppress coverity fp's 5 years ago
Victor Julien b0c79c6996 datasets: suppress coverity fp's 5 years ago
Philippe Antoine 053c728871 http: adds debug check against too many warnings 5 years ago
Jeff Lucovsky aa3f784d32 detect/ftp: FTP memory accounting fixes 
This commit continues the work started by @vanlink and corrects the
accounting of FTP memory usage against the memcap limit.
 5 years ago
Victor Julien 7ca94ba0a2 app-layer: fix protocol detection bail conditions for TCP fastopen 5 years ago
Jeff Lucovsky e8ad67fa4f detect/lua: Unregister object during free 
This commit removes the registration for the object being freed.
 5 years ago
Jeff Lucovsky d3a65fe156 detect: Provide `de_ctx` to free functions 
This commit makes sure that the `DetectEngineCtx *` is available
to each detector's "free" function.
 5 years ago
Jeff Lucovsky d1151f3f8e detect: Provide function to clear per-thread ctx 
This commit provides an interface to free previously allocated
per-thread contextual information on the keyword lists.
 5 years ago
Shivani Bhardwaj cf4e4e4ac3 flowbits: Allow support for flowbit ORing 
This patch allows to OR multiple flowbits on isset and isnotset flowbit
actions.

e.g.
Earlier in order to check if either fb1 or fb2 was set, it was required
to write two rules,
```
alert ip any any -> any any (msg:\"Flowbit fb1 isset\"; flowbits:isset,fb1; sid:1;)
alert ip any any -> any any (msg:\"Flowbit fb2 isset\"; flowbits:isset,fb2; sid:2;)
```

now, the same can be achieved with
```
alert ip any any -> any any (msg:\"Flowbit fb2 isset\"; flowbits:isset,fb1|fb2; sid:23;)
```

This operator can be used to check if one of the many flowbits is set
and also if one of the many flowbits is not set.
 5 years ago
Philippe Antoine fef124b92d ftp: use switch for ftp commands for style 5 years ago
Philippe Antoine 6f36403219 ftp: FTPGetAlstateProgress for done port commands 
For a done transaction with command PORT,
we expect FTP_STATE_FINISHED
and we got FTP_STATE_PORT_DONE instead
which prevented logging of these transactions

We change the order of the evaluations to get the right result
 5 years ago
Philippe Antoine 699d6682da ftp: indent FTPParseResponse again 5 years ago
Philippe Antoine a6294d6ec2 ftp: FTPParseResponse bufferizes lines 
Protects against evasion by TCP packet splitting

The problem arised if the FTP response is split on multiple packets

The fix is to bufferize the content, until we get a complete line
 5 years ago
Philippe Antoine cd26fc139e detect: fix insertion in linked list for fast pattern 
Make sure we do not add the same list_id twice
by checking at least all the lists with the current priority
 5 years ago
Victor Julien 0ce489bcc9 conf/datadir: fix possible out of bounds array access 5 years ago
Victor Julien 1d8d03184d datasets: remove useless variables 5 years ago
Victor Julien 7a6269798b datasets: add 'dataset-remove' unix command 5 years ago
Victor Julien af06883f65 datasets: add 'remove' support 5 years ago
Victor Julien 51726e0a0f thash: add 'remove' support 5 years ago
Victor Julien b80ab56d10 datasets: improve 'dataset-add' error checking 5 years ago
Victor Julien ff55a444d4 datasets: fix return values for 'add's 5 years ago
Victor Julien 381bc2dd64 datasets: fix ref cnt handling 
Each 'add' and 'lookup' would increment the use_cnt, without anything
bringing it back down.

Since there is no removal yet, nothing is actually affected by it yet.
 5 years ago
Victor Julien 03dc5d1d74 datasets: silence noisy 'dataset-add' log 5 years ago
Victor Julien f8159bd372 build: default to c11 standard 
Rearrange pcap includes to fix builds on MinGW
 5 years ago
Victor Julien 1893e40e79 build: don't limit C std to c99 (gnu99) 
Now that C11 atomics and thread local storage are supported, the
compiler can figure out what version to use.
 5 years ago
Victor Julien 7691fc4f9e configure: check for u_int and friends 5 years ago
Victor Julien cb4b5296da fuzz: include pcap headers through suricata-common.h 5 years ago
Victor Julien d4f86e3709 threads: remove u_long usage 5 years ago
Victor Julien 3ba4afd40b threads: make thread local storage manadatory 
Support either the __thread GNUism or the C11 _Thread_local.

Use 'thread_local' to point to the one that is used. Convert existing
__thread user to 'thread_local'.

Remove non-thread-local code from the packet pool code.
 5 years ago
Victor Julien 32cfd71f1a atomics: stdatomics.h version of SC_ATOMIC_* wrappers 5 years ago
Victor Julien 7553937a22 detect-engine/tags: avoid confusion over data type 5 years ago
Victor Julien 5b9d17b485 atomics: remove unused macros 5 years ago
Victor Julien c83a607b6a atomics: add SC_ATOMIC_INITPTR macro 
Until now both atomic ints and pointers were initialized by SC_ATOMIC_INIT
by setting them to 0. However, C11's atomic pointer type cannot be
initialized this way w/o causing compiler warnings.

As a preparation to supporting C11's atomics, this patch introduces a
new macro to initialize atomic pointers and updates the relevant callers
to use it.
 5 years ago
Victor Julien 531ff3ddec atomics: change SC_ATOMIC_ADD to 'fetch_add' 
Until this point the SC_ATOMIC_ADD macro pointed to a 'add_fetch'
intrinsic. This patch changes it to a 'fetch_add'.

There are 2 reasons for this:

1. C11 stdatomics.h has only 'atomic_fetch_add' and no 'add_fetch'
   So this patch prepares for adding support for C11 atomics.

2. It was not consistent with SC_ATOMIC_SUB, which did use 'fetch_sub'
   and not 'sub_fetch'.

Most callers are not using the return value, so these are unaffected.
The callers that do use the return value are updated.
 5 years ago
Victor Julien 109b2ae551 atomics: avoid unnecessary (direct) CAS use 5 years ago
Victor Julien c660757153 atomics: remove useless SC_ATOMIC_DESTROY 5 years ago
Victor Julien 1cb7eec52d atomics: remove spinlocked fallback 5 years ago
Victor Julien 967340e901 fuzz: fix applayer eof check segv 5 years ago
Philippe Antoine 4fda7ed4bd fuzz: stop app layer target as Suricata 
Before being overwhelmed by successive errors
 5 years ago
Philippe Antoine fe1d36ec7e conf: returns instead of exiting in ConfYamlParse 
So that we can keep on fuzzing even on too much recursion
 5 years ago
Victor Julien dfdf2eb050 fuzz: add missing debug validation to configure 5 years ago
Victor Julien c76f98073e fuzz: add configure wrapper for oss-fuzz 5 years ago
Victor Julien 5e13816380 includes: don't include sys/types.h twice 5 years ago
Victor Julien df79613fb5 privs: include headers in suricata-common.h 5 years ago
Victor Julien 61c9e01f87 conf/yaml: include yaml.h after suricata-common.h 5 years ago
Victor Julien f6bf86f136 fuzz/sigpcap: enable all of eve 5 years ago
Victor Julien 4d50eb1647 detect/iponly: fix parsing of '0' valued netmask 5 years ago
Victor Julien d4613e5c70 util/mem: reduce scope of win32 specific include 5 years ago
Victor Julien 415c992909 util/mem: cleanup by moving atomic from mem hdr 5 years ago
Victor Julien 3b877929e3 util/mem: move most logic to functions 
Reduce macro use and simplify code. Also reduces compiled code
size.
 5 years ago
Victor Julien 48bb26abe7 util/mem: remove old debug code for counting allocs 5 years ago
Victor Julien 481a1923b4 logging: turn SCLog and SCLogErr into funcs 
Reduces compiled code size.
 5 years ago
Victor Julien 64e307936e common: add ATTR_FMT_PRINTF wrapper 
Wraps around __attribute__((format(printf, (x), (y))))
 5 years ago
Victor Julien a8c8e2d5c9 common: use suricata-common.h in more places 5 years ago
Victor Julien b856caad94 common: use WARN_UNUSED macro 5 years ago
Victor Julien f903766849 detect/mpm: don't process empty store 5 years ago
Victor Julien a95fa3c156 dns/tests: comment typo fixes 5 years ago
Victor Julien d5712efc91 decode: return bool network layer 
So that the caller can set the correct event type on error.
 5 years ago
Victor Julien 328a94206e decode/hdlc: initial support 5 years ago
Victor Julien 136d351e40 decode: single network layer entrypoint 
This way new layers can be added in a single place.
 5 years ago
Victor Julien 88bccfb80e decode: create linklayer entry point 
Make AF_PACKET and PCAP mode use it.
 5 years ago
Victor Julien 685d490d07 decode/ieee8021ah: fix possible packet truncation 5 years ago
Victor Julien 5404dc7f6d fuzz/siginit: cleanup detect engine every 1024 runs 5 years ago
Andreas Herz aaa604b4c6 app-layer-template: fix log typo 5 years ago
Jason Ish 4dc80a6e6f conf/yaml: limit recursion depth while paring YAML 
A deeply nested YAML file can cause a stack-overflow while
reading in the configuration to do the recursive parser. Limit
the recursion level to something sane (128) to prevent this
from happening.

The default Suricata configuration has a recursion level of 128
so there is still lots of room to grow (not that we should).

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/3630
 5 years ago
Victor Julien fc6ada8541 detect/parse: properly free bidir sigs in error path 5 years ago
Victor Julien 5abead9325 detect/parse: fix minor memory leak in error path 
Only reachable on SCMalloc so should be unlikely to be reached.
 5 years ago
Victor Julien 27186778b8 fuzz: allow uninitialized stats api 5 years ago
Victor Julien 794d9eeb83 fuzz: remove UNITTEST dependency 
Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.

This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
 5 years ago
Jason Ish 4639dd7932 source/erf: validate record length before read 
Check the ERF record length before attempting to read it as
a record length less than the size of the record header
is invalid.

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/3593
 5 years ago
Victor Julien 960c52d7ff fuzz/sigpcap: initialize empty packet pool 
Fixes runs with --enable-debug-validation. The target did not init a
packet pool, so for a tunnel packet would try to get a packet from
an uninitialized pool. In non-debug mode, this silently works by
falling back to a packet from alloc.

    (gdb) bt
    #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
    #1  0x00007ffff35a6801 in __GI_abort () at abort.c:79
    #2  0x00007ffff359639a in __assert_fail_base (fmt=0x7ffff371d7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555557fe7260 "!(pool->initialized == 0)",
        file=file@entry=0x555557fe7220 "tmqh-packetpool.c", line=line@entry=253, function=function@entry=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:92
    #3  0x00007ffff3596412 in __GI___assert_fail (assertion=0x555557fe7260 "!(pool->initialized == 0)", file=0x555557fe7220 "tmqh-packetpool.c", line=253,
        function=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:101
    #4  0x00005555577e24be in PacketPoolGetPacket () at tmqh-packetpool.c:253
    #5  0x0000555556914ecd in PacketGetFromQueueOrAlloc () at decode.c:183
    #6  0x00005555569161e1 in PacketTunnelPktSetup (tv=0x555559863980 <tv>, dtv=0x614000068e40, parent=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=72, proto=DECODE_TUNNEL_IPV4) at decode.c:286
    #7  0x00005555569de694 in DecodeIPv4inIPv6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", plen=72) at decode-ipv6.c:59
    #8  0x00005555569e60b5 in DecodeIPV6ExtHdrs (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=112) at decode-ipv6.c:522
    #9  0x00005555569e846f in DecodeIPV6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-ipv6.c:641
    #10 0x0000555556a032f9 in DecodeRaw (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-raw.c:70
    #11 0x0000555557659ba8 in DecodePcapFile (tv=0x555559863980 <tv>, p=0x61e0000fc080, data=0x614000068e40) at source-pcap-file.c:412
    #12 0x0000555556573401 in LLVMFuzzerTestOneInput (data=0x613000000047 "\241\262\315\064", size=339) at tests/fuzz/fuzz_sigpcap.c:158
    #13 0x0000555557a4dc66 in main (argc=2, argv=0x7fffffffdfa8) at tests/fuzz/onefile.c:51

That line:

    BUG_ON(pool->initialized == 0);
 5 years ago
Todd Mortimer 944209592f detect/threshold: Add tests for thresholding by_rule and by_both. 5 years ago
Todd Mortimer 50e5b80463 detect/threshold: Add a common function to (re)allocate the by_rule threshold table. 
Ensure that the by_rule threshold table is initialized if a rule
is thresholded by_rule. Replace manual table reallocaton with calls
to the common function.
 5 years ago
Todd Mortimer 82dc61f4c3 detect/threshold: Refactor threshold calculation to handle by_rule and by_both. 
The only difference between threshold calculations for by_src/by_dst,
by_rule or by_both is which table stores the DetectThresholdEntry.
Refactor the ThresholdHandlePacket* functions to do table lookup and
storage individually, but calculate thresholds in a common function.
 5 years ago
Todd Mortimer 9fafc1031c time: Add TIMEVAL_EARLIER and TIMEVAL_DIFF_SEC macros. 
Make it easy to compare 'struct timeval's and get their difference.
 5 years ago
Todd Mortimer e945dea244 detect/threshold: Parse by_rule and by_both in rules. 
Also add tests for parsing them.
 5 years ago
Victor Julien ed8f48b053 app-layer/proto-detect: minor cleanup 
Make sure the mask calculation is u32.
 5 years ago
Victor Julien aba4e19548 detect/pktvar: fix memory leaks 5 years ago
Philippe Antoine 240df05af5 fuzz: limit input size for protocol detection consistency check 5 years ago
Jeff Lucovsky 6bffe0bd35 detect/ssl: Fix memory leak in version parsing 
This commit fixes a memory leak in the SSL version handling that
manifests when the version identifier is incomplete or incorrect.
 5 years ago
Philippe Antoine 91b2930891 fuzz: build compatibility with oss-fuzz flags 
ie C define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
 5 years ago
Victor Julien 09a21545ce flow: cleanup expectations first 
Make sure to cleanup expectations for a flow as the first step, before
parts of the flow itself are getting cleaned/freed.

Also indicate use unlikely as flows with expectations should be relatively
rare.
 5 years ago
Eric Leblond fcfeeeb694 app-layer-expectation: update copyright date 5 years ago
Eric Leblond 1ddd77fae0 app-layer-expectation: clean expectation at flow end 
When a flow timeout, we can have still existing expectations that
are linked to this flow. Given that there is a delay between the
real ending of the flow and its destruction by Suricata, the
expectation should be already honored so we can assume the risk
to clean the expectations that have been triggered by the
to-be-deleted flow.
 5 years ago
Eric Leblond 6c9d1c0861 app-layer-expectation: limit number of expectations 
This patch introduces a limitation in term of number of
expectations attached to one IPPair. This is done using
a circle list so we have a FIFO approach on expectation
handling.

Circleq list code is copied from BSD code like was pre existing code
in queue.h.
 5 years ago
Eric Leblond 03e4bfeb02 app-layer-expectation: remove unused parameter 5 years ago
Jeff Lucovsky 0ae6b0b250 tests/bsize: Fuzzing test case added 
This commit adds a test case to validate the issue found during fuzz
testing.
 5 years ago
Jeff Lucovsky 5b38bc9894 detect/bsize: Ensure numeric values fit 
This commit ensures that the numeric values will not exceed the size of
the containers used to hold them.
 5 years ago
Victor Julien 095981cb2a detect/parse: fix crash on 'internal' keyword use 
When keyword __flowvar__postmatch__, an internal keyword, is used
in a rule the 'Setup' func ptr will be NULL. This caused a crash.
 5 years ago
Victor Julien 1e71eecf47 fuzz/siginit: fix leak in case of bidir sig 5 years ago
Victor Julien 5430141f7a fuzz/siginit: minor improvements 
Enable detect engine 'quiet' mode to generate less output.

Set a fake filename so that datarep doesn't hit a reachable assert.
 5 years ago
Victor Julien 13c9d0ca7e detect/pkt_data: error on unconsumed transforms 
If a rule has transforms w/o consuming them (e.g. a content keyword),
don't consider 'pkt_data' valid.
 5 years ago
Victor Julien e1c474a1b0 detect/pkt_data: code and test cleanup 5 years ago
Victor Julien 7f19da1cc0 detect: more robust against transform issues 
In case of transform issues (transform not consumed before pkt_data
for example), the code would hit an ugly BUG_ON.

Address this by a more graceful error message, that will still
invalidate the sig but not crash the engine.
 5 years ago
Jeff Lucovsky 2823bc5aed detect/tls: Use pcre_copy_substring to avoid leak 
This commit eliminates a memory leak while parsing TLS version
information. The leak was identified through fuzzing.
 5 years ago
Victor Julien 3d969a1c7d build: wrap fuzz targets in guard to fix 'make tags' 5 years ago
Victor Julien 8cbae1371f fuzz/sigpcap: fix FPs due to missing pkt cleanup 5 years ago
Victor Julien e97cdb48f3 decode/teredo: implement port support 
Implement support for limiting Teredo detection and decoding to specific
UDP ports, with 3544 as the default.

If no ports are specified, the old behaviour of detecting/decoding on any
port is still in place. This can also be forced by specifying 'any' as the
port setting.
 5 years ago
Shivani Bhardwaj 0e4f261224 Use StringParse* for all parsers and configurations 5 years ago
Shivani Bhardwaj c4c734541a Use appropriate ByteExtractString* functions 5 years ago
Shivani Bhardwaj 6b2c7d5be8 util: Add StringParse* functions 
StringParse* functions would perform a stricter check compared to
ByteExtractString* functions. These new functions shall also check if
any extra characters follow the extracted numeric value in addition to
the checks performed by ByteExtractString* and return -1 in that case.
This is particularly important in parser, configuration and setup functions.
 5 years ago
Philippe Antoine 293eebd999 fuzz: remove obsolete AFL code 5 years ago
Philippe Antoine bf60959d84 fuzz: simpler way to force usage of CXX linker 5 years ago
Philippe Antoine 440bb4d600 fuzz: remove decodeder fuzz target 
As we removed decodeder function
 5 years ago
Victor Julien e500c59b99 stream/tcp: fix STREAM_HAS_SEEN_DATA macro 
The macro would not return true for smaller TCP streams, leading to
cases where the app-layer was not notified of EOF.
 5 years ago
Victor Julien 1618fb1b97 stream/tcp: clean up stream flags 5 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 5 years ago
Pierre Chifflier 333fcc43e7 ssl/tls: call rs_cstring_free for strings allocated in Rust 5 years ago
Pierre Chifflier 1d9f37a60e DER: remove the C parser for DER 5 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 5 years ago
Jeff Lucovsky e0bd79670c detect: byte-test convert neg_op flag to a bool 
Only 8 flags are permitted so convert one of them to a struct member. I
choose neg_op
 5 years ago
Jeff Lucovsky 313c23a26b detect: Add unittests to exercise bitmask 5 years ago
Jeff Lucovsky d12950c9e4 detect: fixup incorrect comments, indentation 5 years ago
Jeff Lucovsky 31ed9786f6 detect: byte_test impl for bitmask 
This commit implements byte_test's bitmask feature.
 5 years ago
Victor Julien b85539b2ab stream/tcp: fix fast open off by one 
With data on SYN the sequence number used for the first data
was off by one, leading to the next segments to appear to come
after a one byte gap.
 5 years ago
Philippe Antoine f51d7d8947 fuzz: check tcp splitting evasions in protocol detection 5 years ago
Philippe Antoine 9eddaa038e fuzz: enable AFLFUZZ_PERSISTANT_MODE for libfuzzer targets 5 years ago
Philippe Antoine ac35118ebe fuzz: use env variable to restrict app layer 5 years ago
Philippe Antoine 600b0d7c55 fuzz: adds eight fuzz targets 
And ways to compile them with enable-fuzztargets at configure time
Adds utility function in util-unittest-helper
 5 years ago
Frank Honza 1c8943dedd add RFB parser 
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
 5 years ago
Victor Julien b4d75b7448 output/anomaly: minor code cleanups 5 years ago
Victor Julien 4d21b03575 detect/app-layer-event: code cleanups 5 years ago
Jeff Lucovsky f0bd69e7e3 detect/pcre: Correct spelling typos 5 years ago
Jeff Lucovsky 7f6af10fed general: copyright bump 5 years ago
Jeff Lucovsky 4b0085b03c detect: Update to take advantage of PCRE refactor 
This commit changes the keyword detectors to use the refactored PCRE
modifications from detect-parse.[ch]
 5 years ago
Jeff Lucovsky abe0cdc4ad detect/pcre: Changes to support pcre_jit_exec 
This command causes `pcre_jit_exec` to be used when available. If it's
available and there are allocation errors preparing for it, things
fallback to `pcre_exec`.
 5 years ago
Jeff Lucovsky aa67a0a236 detect/pcre: Add warning for failed registrations 
This commit adds a warning used by the PCRE detect logic when it fails
to register initialization and free functions for per-thread JIT stack
handling.

This error code is only used when the platform has PCRE JIT exec
functionality.
 5 years ago
Jeff Lucovsky d19429f7e5 detect/parse: Refactor interfaces/definitions 
This commit refactors existing code patterns to reduce code duplication
and to be a base for supporting additional PCRE jit-related actions.
 5 years ago
Philippe Antoine 1cd314c500 detect: adds icmpv6.mtu keyword 5 years ago
Philippe Antoine 75ec528384 detect: adds utility file for uint keywords 5 years ago
Philippe Antoine 0355b70f5a detect: define generic PrefilterIsPrefilterableById 5 years ago
Philippe Antoine bb7abbd6ec detect: extend PrefilterPacketHeaderValue to 16 bytes 5 years ago
Philippe Antoine d04d69acbb decode: fix endianness for ICMPv6 MTU 5 years ago
Philippe Antoine 9d2fabcd99 doc: fix typo in DetectTcpmssSetup description 
And use lowercase true for boolean
 5 years ago
Victor Julien efee458af8 stream: fix direction flags in corner case 
When a TCP DNS flow would start with a GAP on the TS side, the successful
protocol detection on the TC side would trigger 'opposing side' reassembly
and app-layer processing. In this case the stream flags would indicate the
wrong direction and the wrong parser would be called.
 5 years ago
Victor Julien 960045323d app-layer: minor code cleanup 5 years ago
Victor Julien 9b1f0656d0 app-layer: fix progress tracking 
Esp in combination with GAPs and proto detection.
 5 years ago
Victor Julien 21e6f1f063 app-layer: document return macros 5 years ago
Victor Julien 66598f9ce7 nfs: switch to AppLayerResult 5 years ago
Victor Julien 5cd9cfb5a0 smb: update return type of GAP handling 5 years ago
Victor Julien 4bf87d30e4 smb: convert to return AppLayerResult 
Support returning 'incomplete' and remove the buffering
code from the parser.
 5 years ago
Victor Julien 674b8dc0fb app-layer: add 'incomplete' return logic 
Allow app-layer parsers to indicate how much data they need
before being called again.
 5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details 
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
 5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes 
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
 5 years ago
Victor Julien 54b2ec0e4d app-layer: minor optimization 5 years ago
Victor Julien fee7b97ec0 stream: fix function style 5 years ago
Victor Julien 157d01e87e ftp: minor code cleanups 5 years ago
Stephen Donnelly 47082dd5df dag: Skip over ERF_TYPE_META records 
Suricata generates an error on unrecognised ERF types.
Suricata should ignore ERF 'Provenance' records with ERF_TYPE_META.
 5 years ago
Jason Ish 8a643c893c detect/parse: allow for OK signature parsing errors 
The idea of an OK signature parsing error is an error that is
allowed to occur, but still lets test mode pass, unlike
silent errors which will still fail testing.

This is introduced to allow for app-layer event keywords to be
removed, but not have old rules fail out on this case. For example
the Rust DNS parser removes from DNS app-layer events that are
not used anymore.

To signal that an error is OK, -3 is returned. This also implies
silent.
 5 years ago
Jason Ish 947cfac62e detect/parse: softer error on unknown app-layer event 
On an unknown app-layer event, return -3 for "silent OK fail". A
warning will still be emitted, but its not considered a rule parse
error. This is to handle app-layer events being removed in a more
graceful manner for the user.

This allows -T to pass with an old app-layer events rule file
that may used removed app-layer event keywords.
 5 years ago
Jason Ish 688874582a dhcp: remove C app-layer-dhcp wrapper 
This just wrapped the Rust function to register the parse,
so instead just call the Rust function directly to remove
the C wrapper, and the 2 C files.
 5 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust 
And port the C DNS tests to Rust.
 5 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust 
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
 5 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust 
Remove app-layer-dns-common.c as its no longer needed.
 5 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events 
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
 5 years ago
Jason Ish 4d312b6813 dns: cleanup: remove C DNS type definitions (dead code) 5 years ago
Jason Ish d809b0959b dns: cleanup: move DnsGetRcode (Lua) to rust 
Move the implementation of Lua DnsGetRcode to Rust.
 5 years ago
Jason Ish d6b3b33f4a dns: cleanup: remove unused function DNSCreateTypeString 5 years ago
Philippe Antoine 8396333493 detect: adds icmpv6.hdr keyword 5 years ago
Philippe Antoine 02d94f5cf2 detect: fix typo for ipv6.hdr description 5 years ago
Philippe Antoine fc48a7cebf detect: use SC macros for IPV6 header 5 years ago
Philippe Antoine 5f95822f8b doc: fix typo in ByteExtractUint32 description 5 years ago
Jeff Lucovsky c20ab53eae detect/threshold: Correct typos 5 years ago
Jeff Lucovsky ff9a01ee1b detect/threshold: Don't allow duplicates 
This commit detects duplicate threshold rule options. When duplicates
are found in a rule, an error message is displayed and the rule is
rejected.
 5 years ago
Victor Julien f6c77dcdd9 pcap/file: minor code cleanups 5 years ago
Victor Julien faf5beed2f threads/time: minor code cleanup 5 years ago
Jeff Lucovsky c92975e22b mime: Test cases for filename length limit 5 years ago
Jeff Lucovsky 9a33b5d5de smtp/mime: Fix typos 5 years ago
Jeff Lucovsky 130b8d26e7 smtp/mime: Set event when name exceeds limit 5 years ago
Jeff Lucovsky d0d20bd874 smtp/mime: Restrict file name lengths 
This commit places restrictions on the length of the file name specified
in attachments (`name=` or `filename=`) to `NAME_MAX`. Names exceeding
these limits will be truncated and processing will continue with the
truncated name.
 5 years ago
Victor Julien 072c421e46 pcap/file: improve time handling 
This patch addresses two problems.

First, various parts of the engine, but most notably the flow manager (FM),
use a minimum of the time notion of the packet threads. This did not
however, take into account the scenario where one or more of these
threads would be inactive for prolonged times. This could lead to the
time used by the FM could get stale.

This is addressed by keeping track of the last time the per thread packet
timestamp was updated, and only considering it for the 'minimum' when it
is reasonably current.

Second, there was a minor race condition at start up, where the FM would
already inspect the hash table(s) while the packet threads weren't active
yet. Since FM gets the time from the packet threads, it would use a bogus
time of 0.

This is addressed by adding a wait loop to the start of the FM that waits
for 'time' to get ready.
 5 years ago
Victor Julien 4977daec32 threads/time: rename ts to pktts to make purpose clear 5 years ago
Victor Julien ae4137937e pcap/file: fix race during pcap processing start 
A race condition during the start of pcap file processing could cause
missed alerts and logged events. This race happens between the packet
threads and the flow manager. It was observed on slower hardware, but in
theory could happen on any machine. It required the 'autofp' runmode.

In commit 6f560144c1 ("time: improve offline time handling") the logic
was added to make the flow manager use a minimum of all the packet threads
perception of time.

The race condition was that the flow manager may become active _before_
all of the packet threads have started processing packets and thus setting
their timestamp. The threads that had not yet initialized their timestamp
would not be considered when calculating the minimum.

As a result of this, older packets timestamps would not yet be registered.
This would give the Flow Manager a timestamp too far in the future. While
the FM was running, the packet processing would start and a flow would
be created. This flow would then immediately be considered 'timed out' by
the FM, due to the timestamp too far in the future.

In the observed case, the thread processing packet 1 from the pcap had not
yet started processing while other threads had already started. The FM was
also already active. Due to the timestamps in the pcap this meant that the
time the FM used was about 500 seconds in the future compared to packet 1.

This patch fixes the issue by initializing all of the threads timestamps
with the timestamp value of the first packet. This way the minimum will
always consider this timestamp.
 5 years ago
Victor Julien 1b1855c3bd time: remove unused time structure 5 years ago
Victor Julien 13b4ece11d time: minor code cleanup 5 years ago
Victor Julien 6176338d0b time: fix function name typo 5 years ago
Victor Julien 227e2812f6 pcap/file: fix function ptr naming 5 years ago
Jason Ish a2d91d9bf5 log-pcap: fix log message: unified2 -> pcap 
Likely a result of copy and pasted code.
 6 years ago
Jason Ish 13635dba57 unified2: not supported message if configured 6 years ago
Jason Ish d86973b386 unified2: remove deprecated output unified2 
Ticket 2385:
https://redmine.openinfosecfoundation.org/issues/2385
 6 years ago
Jeff Lucovsky e73aec150b pcre: Sticky buffer check 
This commit adds logic checking if the sticky buffer in effect provides
the required content.

If the sticky buffer doesn't, the rule will not load and a diagnostic
message with follow-on steps is displayed.
 6 years ago
Jeff Lucovsky 8279bab8dc general: Wordsmith "no rules loaded" message 6 years ago
Sascha Steinbiss efaa1755c6 snmp-version: make comment more clear 6 years ago
Sascha Steinbiss f02a18e557 snmp: do not set SIGMATCH_NOOPT 6 years ago
Jeff Lucovsky 6d94b096a9 spelling: Fix spelling error 6 years ago
Jeff Lucovsky 6c3503932f detect/ssl_state: Fix memory leaks from pcre_get_substring 
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
 6 years ago
Jeff Lucovsky 9fe51a8bd2 detect/flowvar: Fix memory leaks from pcre_get_substring 
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
 6 years ago
Jeff Lucovsky c2071e1c4e detect/filestore: Fix memory leaks from pcre_get_substring 
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
 6 years ago
Jeff Lucovsky bcea730266 util-error: define SC_ERR_PCRE_COPY_SUBSTRING 6 years ago
Jeff Lucovsky be4c6b85de general: Convert _Bool to bool 
This commit addresses task 3167 and changes usages of '_Bool` to `bool`.
The latter is included from `suricata-common.h`
 6 years ago
Jeff Lucovsky 1519c1f006 detect/feature: Support --dump-features 6 years ago
Jeff Lucovsky 5e08e6bccf detect: update version comment 6 years ago
Jeff Lucovsky 1cddbc085b detect: Fix typos/spelling errors. 6 years ago
Jeff Lucovsky 8f6e133497 main: feature init and release 6 years ago
Jeff Lucovsky 9631808b04 output/filestore: announce provided features 6 years ago
Jeff Lucovsky cf840c37b3 detect/filestore: Warn if req'd feature missing 6 years ago
Jeff Lucovsky 31e2cff9cb detect/analysis: Warn if required feature missing 6 years ago
Jeff Lucovsky f83b199f2a feature: feature tracking services 6 years ago
Philippe Antoine 5597801ccb perf: optimizes alert fast log 
In the case when we have multiple alerts for one packet
We suppose this happens more often than having decoder_event != 0
 6 years ago
Phil Young 1c99536945 napatech: add hardware based bypass support 
Napatech hardware bypass support enables Suricata to utilize
capabilities of Napatech SmartNICs to selectively bypass flow-based
traffic.
 6 years ago
Philippe Antoine 52970d8508 detect/parse: move spaces skip up the stack 
Switch to isspace() as well.
 6 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 6 years ago
Jeff Lucovsky 427ec4e739 decode: Fix typos/spelling 6 years ago
Jeff Lucovsky aec4e9a032 decode: Handle ERSPAN Type I 6 years ago
Philippe Antoine af2a689a99 init: move main to a separate file 
so as to reuse code from suricata.c with fuzz targets
 6 years ago
Philippe Antoine bb42be7e08 fuzz: disable signal handlers while fuzzing 6 years ago
Philippe Antoine 47eba33110 init: make PostConfLoadedSetup global 
to be used by fuzz targets
 6 years ago
Philippe Antoine 913082941c init: fix PostConfLoadedSetup to use local suri ptr 6 years ago
Philippe Antoine d2b3668a68 init: InitGlobal function 
To be reused by fuzz targets
 6 years ago
Philippe Antoine f5190da67e util: UTHmemsearch to use memmem if defined 6 years ago
Philippe Antoine ce55d06569 check: Using const keyword for some arguments 
For ConfigSetLogDirectory and PrintRawLineHexBuf
 6 years ago
Philippe Antoine 12a3a24906 log: can use a file set from env variable 
Enables the redirection of log to a file set by an environment
variable SC_LOG_FILE when SC_LOG_OP_IFACE=file
 6 years ago
vanlink 2456f27d08 stream/reassembly: fix data overlap check 
Fix function CheckOverlap bug.
 6 years ago
Victor Julien aeefc82eb9 tls: fix missing extern logic for cert_id tracking 6 years ago
Victor Julien 62c0f3d2b4 stats: fix missing extern keyword 6 years ago
Victor Julien 903291f88a defrag: fix use of globals 6 years ago
Victor Julien 5c3c6c609c threading: fix queue handlers globals use 6 years ago
Victor Julien 3ae1854d2f htp: fix globals use for flags 6 years ago
Victor Julien 85289f3283 proto: fix globals use 6 years ago
Victor Julien 5e583f3a12 flow: fix global variable use 6 years ago
Victor Julien 29f54a34ae stream: fix global declaration of the config 6 years ago
Victor Julien 2436daccd9 threading/modules: fix global declarations 6 years ago
Victor Julien c5f4b41881 ippair: fix global declarations 6 years ago
Victor Julien 0a006d2258 host: fix global declarations 6 years ago
Victor Julien 29780d6164 mpm: fix global declarations 6 years ago
Victor Julien b89059bda7 detect: fix global declaration of sigmatch_table 6 years ago
Victor Julien 0118e07d57 spm: fix global declaration of spm_table 6 years ago
Victor Julien a12c0b499d threading: fix global declaration of threading_set_cpu_affinity 6 years ago
Victor Julien 45955d2e58 unix-socket: avoid using global variable w/o extern 6 years ago
Victor Julien a9a522fac3 decode: fix default-packet-size global variable 6 years ago
Victor Julien 7709b90c16 detect/file-data: remove debug abort that wasn't reachable 6 years ago
Victor Julien ac8ceae9bf detect/file-data: fix function doc 6 years ago
Victor Julien 500e8da63a files: tracking flag update 
Improve flow file flags and file flags updates. Introduce a mask
that is set at start up to avoid lots of runtime checks.

Disable cocci flags check as it doesn't support the more dynamic
nature of the flag updates.
 6 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls 
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
 6 years ago
Victor Julien d369e54f1d app-layer: all protocols are tx aware now 
So remove the runtime check for it.
 6 years ago
Timo Sigurdsson 1262ecbde0 init: Fix dropping privileges in nflog runmode 
Using the run-as configuration option with the nflog capture method
results in the following error during the startup of suricata:
[ERRCODE: SC_ERR_NFLOG_BIND(248)] - nflog_bind_pf() for AF_INET failed

This is because SCDropMainThreadCaps does not have any capabilities
defined for the nflog runmode (unlike other runmodes). Therefore, apply
the same capabilities to the nflog runmode that are already defined for
the nfqueue runmode. This has been confirmed to allow suricata start
and drop its privileges in the nflog runmode.

Fixes redmine issue #3265.

Signed-off-by: Timo Sigurdsson <public_timo.s@silentcreek.de>
 6 years ago
Victor Julien 7810f22413 decode: remove pseudo packet checks 
Bug 1107 checks/hacks should not longer be needed, so remove them.
 6 years ago
Victor Julien 272a5f526b threading/queues: simplify error handling 6 years ago
Victor Julien dce227ec88 threading/queues: remove 256 queue limit 
Convert fixed size array to a dynamic TAILQ so we can
grow as needed.
 6 years ago
Victor Julien 0e3f27a87e threading: remove 'trans_q' array of packet queues 
Let the queues code set up PacketQueues on demand.
 6 years ago
Victor Julien efa8a69923 packet-queue: create alloc and free functions 6 years ago
Victor Julien 550cfdd98d threading: hide 'trans_q' from queue handlers 6 years ago
Victor Julien 45e5e19e6e threading/threadvars: optimize layout 
Make sure StatsPublicThreadContext is on its own cache line.
 6 years ago
Victor Julien e3fbdf1948 flowworker/stream: use no-lock packet queue 
Use smaller structure for temporary packet queues.
 6 years ago
Victor Julien f8aed4ce2d threading: change local packet queue logic 
Previously each 'TmSlot' had it's own packet queue that was passed
to the registered SlotFunc as an argument. This was used mostly for
tunnel packets by the decoders and by defrag.

This patch removes that in favor of a single queue in the ThreadVars:
decode_pq. This is the non-locked version of the queue as this is
only a temporary store for handling packets within a thread.

This patch removes the PacketQueue pointer argument from the API.
The new queue can be accessed directly through the ThreadVars
pointer.
 6 years ago
Victor Julien b8c2b66d33 packet-queue: introduce a non-locked version 
Works exactly like PacketQueue, just does not contain a mutex
and cond var, leading to much reduced memory size.
 6 years ago
Victor Julien 9ed260c489 threading: more efficient TmSlot layout 6 years ago
Victor Julien 18e652309f threading: remove 'id' field from TmSlot 
Field was now unused.
 6 years ago
Victor Julien d7cb0774dd detect: cleanup reload thread handling 6 years ago
Victor Julien 786e697590 threading: simplify flow timeout loop 6 years ago
Victor Julien 261b77742e threading: shrink and reorganize TmSlot 6 years ago
Victor Julien 87c9b11d8c threading/threadvars: rearrange for better cache behavior 6 years ago
Victor Julien 071b753e84 threading/threadvars: remove unused 'prev' field 6 years ago
Victor Julien f53f004917 threading: remove unused 'TmThreadRemove' function 6 years ago
Victor Julien 569a5d985b threading: remove handler names to shink struct 
Shrink ThreadVars by removing the queue handler names that are only
used at shutdown. Since this is not performance critical, we can use
the id's to look up the queue handler.
 6 years ago
Victor Julien 74a6f8d4dd threading/queues: add way to lookup by ID 
In preparation of doing runtime operations by ID instead of by name,
add functions to look up by ID and to convert name to ID.
 6 years ago
Victor Julien d0218696ba threading: shrink threadvars struct size 6 years ago
Victor Julien c029599515 threading: remove unused threadvars field 6 years ago
Victor Julien f1ee176111 threading: clarify threadvars fields 6 years ago
Victor Julien d50492cb20 threading: cleanup packet thread shutdown loop 6 years ago
Victor Julien 8e762f5190 source-pcap: remove unused function 6 years ago
Victor Julien 3a703c84ad threading/modules: declare prototypes static 
Declare registered threading API funcs static where appropriate.
 6 years ago
Victor Julien 7c83cb585e sources: fix pipeline failure handling 
When TmThreadsSlotProcessPkt fails it will return the packet that was
passed to it to the packetpool.

Some of the packet sources were doing this manually as well. This patch
fixes those sources.
 6 years ago
Victor Julien 49599dfe89 threading: use tm_flowworker for pseudo packets 
Pseudo packets don't need to be processed by the decoding layer.
 6 years ago
Victor Julien 9df8e1c984 threading: add shortcut to flowworker 6 years ago
Victor Julien 02004fa547 threading: remove per slot post_pq 
Use a single packet queue per thread for flow timeout packet
injection. The per slot queue was unused except for this use
case. Having a single queue makes the logic and implementation
simpler.

In case of 'autofp', the per thread packet queue will actually
use the threads input queue. For workers/single a dedicated
queue will be set up.

Rename TmThreadsSlotHandlePostPQs to TmThreadsHandleInjectedPackets
to reflect the changed logic.
 6 years ago
Victor Julien 15e3bdb7b8 af-packet: prototypes cleanup 
Remove unused prototype.

Declare other prototypes static.
 6 years ago
Victor Julien 44d7f636f2 threading: remove post_pq argument from 'SlotFunc' 
This was not in use anywhere.
 6 years ago
Victor Julien f5045af3e3 runmodes: code cleanups 6 years ago
Victor Julien 1a8562b3c6 detect: clean up threads handling 
Clean up reload and break loop thread handling.
 6 years ago
Victor Julien e5010d7704 detect: inject packet cleanup 6 years ago
Victor Julien abea227cfc flow-manager: code cleanups 6 years ago
Victor Julien 6fd35fb786 flow-manager: avoid doubly signaling threads 
Don't try to wake up the threads we just flagged and validated that
they changed their state.
 6 years ago
Victor Julien 539c1a275f threading: remove commented out function 6 years ago
Victor Julien b55f617c2f threading: optimize and unify post_pq checks 
TmThreadsSlotProcessPkt did not need to look all 'slots' as only the first
slots post_pq can have been used.

Unify post_pq cleanup handling.
 6 years ago
Victor Julien 2a1ed3ba1b threading: remove wrong unlikely statement; minor cleanups 6 years ago
Victor Julien e5192ae20a threading: TmSlot::SlotFunc does not need to be atomic 6 years ago
Victor Julien 89048d71ad threading: fix flags handling by using uint32_t everywhere 6 years ago
Victor Julien 85cf341189 threading: optimize error handling in main packet loop 6 years ago
Victor Julien 9d8ea3b4fe threading: minor code style cleanups 6 years ago
Victor Julien b1056b3836 threading: simplify packetpool checks 6 years ago
Victor Julien 603b2ced47 threading/queues: add shortcut for packetpool check 
Allows code simplifycation in the threading loops.
 6 years ago
Victor Julien 29cb9d1d52 threading/queues: minor code cleanups 6 years ago
Victor Julien f05c12b70f afl: fix compilation 6 years ago
Victor Julien 01862eae78 afl/decoder: make file dumps optional 6 years ago
Victor Julien 42d112e7b6 detect/address: dead code removal and style cleanups 6 years ago
Jason Ish 80cafb2979 flow: expose last time as a function 
This function returns the individual components
of the timeval in output pointers making it suitable
for use over Rust FFI.
 6 years ago
Jason Ish d1eab5aa46 defrag: set livedev on the reassembled packet (issue-3380) 
Set the livedev on reassembled packets to that of the parent
packet. Fixes issues with multidetect, specifically a segfault
as reported in issue 3380.

Bug #3380.
 6 years ago
Victor Julien 9d0976ea8a output/tx: split list of loggers per alproto 
This patch splits the list of loggers the tx logging walks into lists per
alproto. The list was getting longer with each eve addition. The result
was that for each tx we would have to loop through multiple loggers that
did not apply to this tx as it was for the wrong protocol.
 6 years ago
Victor Julien 5b7aa506c1 output: micro optimization 
LogFunc is always set, so don't check for it at runtime.
 6 years ago
Victor Julien 4f9e4d41e0 output: optimize root logging loop 
Instead of unconditionally looping all the 'root' loggers, loop only
those that are in use.

Root loggers are: packet, tx, file, filedata, streaming.
 6 years ago
Victor Julien 07df1ce6af output: clarify registration 6 years ago
Victor Julien 284c3cf68a output/tx: bail early if no flow 6 years ago
Victor Julien ed99e9204f output: fatal error if root logger alloc fails 6 years ago
Danny Browning b573c16dd5 build: cbindgen 
Rust headers are now generated using cbindgen. If cbindgen is present, they can
be generated during dist, otherwise they will be available for builds.
 6 years ago
Victor Julien 2c050187a3 streaming/api: fix overlap check 
In some cases a SBB could be seen as overlapping with the requested
offset, when it was in fact precisely before it. In some special cases
this could lead to the stream engine not progressing the 'raw' progress.
 6 years ago
Victor Julien 0f41cf3d74 debug/validation: check tcp/app-layer data lengths 6 years ago
Victor Julien a742c86741 stream: improve app-layer data retrieval with GAPs 
Don't assume that the next block after the sbb head is after the
requested offset.

If the next block was before the offset, the returned data_len
would underflow and return a nonsense value to the app-layer.

Bug #2993.
 6 years ago
Jeff Lucovsky ed2f6ac64b modbus: Correct typo 6 years ago
Jeff Lucovsky d4428d94de modbus: Update correct TX flags 6 years ago
Jeff Lucovsky 6c2cdbb5f0 analysis: exit if table entries are stale 
This commit causes Suricata to exit when a buffer from the analyzer
table is not recognized.

Since the table must match what's registered, exiting will bring noticed
to the condition.
 6 years ago
Victor Julien 627cc23769 detect/asn1: fix offset bounds checking 6 years ago
Jason Ish 8609939e60 ipv4: continue parsing options after invalid option 
As long as an option has a valid length, we can continue
parsing the options after an invalid one.
 6 years ago
Jason Ish df8db1ddb0 ipv4: fail packet decoding on bad ipv4 option length 
Currently all failures in IPv4 option decode are ignore with
respect to continuing to handle the packet.

Change this to fail, and abort handling the packet if the
option length is invalid.

Ticket 3328:
https://redmine.openinfosecfoundation.org/issues/3328
 6 years ago
Victor Julien fa692df37a stream: reject broken ACK packets 
Fix evasion posibility by rejecting packets with a broken ACK field.
These packets have a non-0 ACK field, but do not have a ACK flag set.

Bug #3324.

Reported-by: Nicolas Adba
 6 years ago
Victor Julien 9f0294fadc stream: fix SYN_SENT RST/FIN injection 
RST injection during the SYN_SENT state could trick Suricata into marking
a session as CLOSED. The way this was done is: using invalid TSECR value
in RST+ACK packet. The ACK was needed to force Linux into considering the
TSECR value and compare it to the TSVAL from the SYN packet.

The second works only against Windows. The client would not use a TSVAL
but the RST packet would. Windows will reject this, but Suricata considered
the RST valid and triggered the CLOSED logic.

This patch addresses both. When the SYN packet used timestamp support
the timestamp of incoming packet is validated. Otherwise, packet responding
should not have a timestamp.

Bug #3286

Reported-by: Nicolas Adba
 6 years ago
Victor Julien df74f34a62 decode/tcp: accept TCP fast open cookie request 6 years ago
Victor Julien 040aff5197 htp: close request only from request side 
This allows the response side to keep going for just
a bit longer.
 6 years ago
Victor Julien 77539e08fc stream: in IDS mode, call app-layer at EOF 
On stream end call app-layer with empty message in IDS mode.
 6 years ago
Victor Julien eceb7dcba4 eve: support pcap_filename for unix socket mode 
Bug #3390.
 6 years ago
Philippe Antoine 4a2918e6b5 yaml: clarify comment about dump-all-headers 
Logs a warning if the value is unknown
Fixes #2810
 6 years ago
Victor Julien 007a461d69 detect/parse: track negation during address parsing 
Fix address negation detection not resolving variables when
looking for the negation.

This patch makes use of the actual parsing routines to relay this
information to the signature parser.

Bug #3389.

Fixes: 92f08d85aa ("detect/iponly: improve negation handling in parsing")
 6 years ago
Victor Julien 34b7035a0d detect/iponly: debug output improvements 6 years ago
Victor Julien 618ad0d92f app-layer: optimize inspection id tracking 
Increase the inspect id for a completely inspected tx in any case.
This avoids re-evaluating transactions.

Reported-by: Ilya Bakhtin
 6 years ago
Victor Julien f302f3543f files: add call for setting inspect sizes 
The inspect sizes are currently only used during file prune
house keeping for SMTP.
 6 years ago
Victor Julien f9f958d66e smtp: fix and clean up new file handling 
Set tx id on files that were just opened.

Move logic to a small util func.
 6 years ago
Victor Julien 683b22d114 smtp: use FILE_USE_DETECT for raw-extract 6 years ago
Victor Julien 21760bfc76 files: change pruning behavior 
If file prune is called inspect has already run. So if file is closed
we can just prune. No need to consider a window anymore.

When still in progress, fix the left_edge calculation.
 6 years ago
Victor Julien 682014619f files: fix FILE_USE_DETECT with --disable-detection 
Don't set FILE_USE_DETECT flag if detect is disabled.
 6 years ago
Victor Julien 4ac9cd2c70 files: move smtp prune logic to main 
Now that we call the file prune loop very regularly, we can move the
SMTP specific inspection pruning logic into this loop. Helps with
cases there we don't (often) update a files inspection trackers.
 6 years ago
Victor Julien 4b7599af90 http/file: modernize unittests 
Part of ticket #2975.
 6 years ago
Victor Julien 1cdb2182e4 fastlog: apply icmp type logic to icmpv6 too 6 years ago
Victor Julien 5ef05ffad1 http/multipart: small cleanup 6 years ago
Victor Julien aae00df4df http/multipart: use wider type for boundary lengths 
Use uint32_t for a local type instead of uint8_t to avoid casts.

Length should always stay under this regardless.
 6 years ago
Victor Julien 4d0db9cb4a http/multipart: optimize form end search 
If we already know that the boundary exists, we can start looking
there. Otherwise, we can skip trying as the boundary is a subset
of the form end marker.
 6 years ago
Victor Julien 54d93e1eb9 http/multipart: process incomplete file data 
Start processing multipart data as soon as it is available to
allow inspection sooner.
 6 years ago
Jeff Lucovsky fcfb679893 detect/analyzer: Suppress direction warnings 
This commit ensures direction warnings for ICMP v4 and v6
are suppressed and corrects check so that both protocols
are checked (instead of the same protocol being checked twice).
 6 years ago
Eric Leblond 1b9009ea0e suricata: fix computing of default packet size 
Update the default packet size computation to use LiveDeviceName
instead of LiveDevice as the LiveDevice list is not built when
the default packet size is built.
 6 years ago
Victor Julien c010f092e0 detect/replace: fix debug print issue 
Don't print field that will likely not be 0 terminated.
 6 years ago
Victor Julien c3ea5e71e5 detect/file.data: fix buffer reusing id 0 6 years ago
Victor Julien cd66c37711 http/multipart: use proper progress value to test eof 6 years ago
Philippe Antoine 08b84e060b fastlog: use icmp type and code instead of port 
Fixes #3266
 6 years ago
Philippe Antoine 75a7d9641c fastlog: move code to reduce variable scope 6 years ago
Philippe Antoine c2fdd7c969 transform: fixes comment about compress_whitespace 6 years ago
Philippe Antoine 9126fc25c1 transform: updates doc about compress_whitespace 
And removes duplicate test from strip_whitespace
 6 years ago
Eric Leblond 3ded7f1170 qa/coccinelle: fix false positive in setter getter 
Coccinelle test was doing a false positive on the function
AppLayerParserStateSetFlag and AppLayerParserStateIssetFlag.
To address that, this patch adds a new coccinelle markup:

 /* coccinelle: AppLayerParserStateSetFlag():2,2:APP_LAYER_PARSER_ */

It indicates that AppLayerParserStateSetFlag is a setter and getter
and that the checks should be disabled inside the function.

Currently this markup is only used for that but following patch will
add some checks on option value.
 6 years ago
Jeff Lucovsky 8f4f1cb633 detect/analyzer: Improved fast pattern display 
When transforms are part of a rule, improve information displayed with
fast patterns to include the original buffer name and whether any
transform(s) are applied.
 6 years ago
Jeff Lucovsky c88c1f1e14 detect/analyzer: Suppress direction warnings 
This commit suppresses direction warnings by the rules analyzer for ICMP
and ICMPV6 since it's not actionable.
 6 years ago
Victor Julien 83bbe287e7 stats: fix stats not always syncing in flow timeout 6 years ago
Jason Ish ba3a2c31bf app-layer: validate TX detect flag callbacks 
Check that both are set or unset.
 6 years ago
Jason Ish 706558d4d5 enip: add tx detect flags 6 years ago
Jason Ish cb62c8dacf dcerpc: add tx detect flags 6 years ago
Jason Ish 21f014f5c3 modbus: add tx detect flags 6 years ago
Jason Ish 20bc08a722 app-layer: add tx detect functions to register struct 6 years ago
Jason Ish fdb587d2fc detect-engine: check for tx detect flag support 
When registing a detection engine, check that the app-layer
protocol supports tx detect flags. Exit with a fatal
error if it does not as this is a code implementation
error that should be resolved during development.
 6 years ago
Jason Ish b1beb76fd7 ftpdata: add tx detect flags 6 years ago
Jason Ish 62e4211f04 debug: add SCReturnBool function exit macro 6 years ago
Jason Ish 739df21e2d app-layer: method to see if parser supports tx detect flags 
Add method to check if a parser for an app-layer protocol
supports tx detect flags.

This is a bit of a hack for now as where we need to run
this check from we do not have the IP protocol.
 6 years ago
Jeff Lucovsky 218a5c4345 mpm: Fix typos and spelling errors 6 years ago
Jeff Lucovsky aef24bee96 detect: Fix spelling errors 6 years ago
Jeff Lucovsky f318a46d34 detect: Improve handling of variable values 
When one of offset/depth/distance is from a variable, adjust the depth
by the offset as is done with scalar values at parse time.
 6 years ago
Jeff Lucovsky db8527e7b3 detect/mpm: Improved handling of variable values 
This commit removes the offset and depth if either of these values are
dependent upon a byte-extract operation.
 6 years ago
Victor Julien 94982ae690 http: split request/response tx id handling 
When HTTP pipelining was in use, the transaction id used for events
and files could be off. If the request side was several requests ahead
of the responses, it would use the HtpState::transaction_cnt for events
and files, even though that is only incremented on complete requests.

Split request and response tx id tracking. The response is still handled
by the HtpState::transaction_cnt, but the request side is now handled by
its own logic.
 6 years ago
Victor Julien b82e71b95e files: remove FILE_USE_TRACKID flag 
Once it was optional but as it no longer is it is no longer useful.

Remove it.
 6 years ago
Victor Julien f9155aa121 files: simplify pruning logic 
Since ebcc4db84a the flow worker runs
file pruning after parsing, detection and loging. This means we can
simplify the pruning logic. If a file is in state >= CLOSED, we can
prune it. Detection and outputs will have had a final chance to
process it.

Remove the calls to the pruning code from Rust. They are no longer
needed.
 6 years ago
Victor Julien ab471c3054 app-layer: don't consider tx flags if not registered 
If a protocol does not support TxDetectFlags, don't try to use them.

The consequence of trying to use them was that a TX would never be
considered done, and it would never be freed. This would lead to excessive
memory use and performance problems due to walking an ever increasing
list.
 6 years ago
Eric Leblond 54d3620662 source-pcap-file: honor bpf filter on command line 
When a BPF filter is given on the command line when reading a
pcap file, the BPF filter is not honored.

The regression has been introduced in:

commit 3ab9120821
Author: Dana Helwig <dana.helwig@protectwise.com>
Date:   Thu Apr 27 11:17:16 2017 -0600

    source-pcap-file: Pcap Directory Mode (Feature #2222)

Reported-By: Tim Colin <tcolin@et.esiea.fr>
 6 years ago
Eric Leblond 860f43753c source-pcap-file: fix memory leak on pcap filter 6 years ago
Philippe Antoine 20e06f45c0 util: removes warning about double conversion 
From clang 10 :
implicit conversion from 'unsigned long' to 'double' changes value
from 18446744073709551615 to 18446744073709551616
 6 years ago
Victor Julien 502a8b5fb3 detect: fix inspection buffer for packet engines 
Fix buffers not being reset per inspection round for packet engines.

Bug #3341.
 6 years ago
Victor Julien 58b9a2dc21 threading: add debug validation for stale packets 6 years ago
Victor Julien fe9aeed0f0 threading: fix shutdown race condition 
A BUG_ON statement would seemingly randomly trigger during the threading
shutdown logic. After a packet thread reached the THV_RUNNING_DONE state,
it would sometimes still receive flow timeout packets which would then
remain unprocessed.

1 main:   TmThreadDisableReceiveThreads(); <- stop capturing packets
2 worker: -> TmThreadTimeoutLoop (THV_FLOW_LOOP) phase starts
3 main:   FlowForceReassembly();           <- inject packets from flow engine
4 main:   TmThreadDisablePacketThreads();  <- then disable packet threads
5 main:   -> checks if 'worker' is ready processing packets
6 main:   -> sends THV_KILL to worker
7 worker: breaks out of TmThreadTimeoutLoop and changes to THV_RUNNING_DONE.

Part of the problem was with (5) above. When checking if the worker was
already done with its work, TmThreadDisablePacketThreads would not consider
the injected flow timeout packets. The second part of the problem was with (7),
where the worker checked if it was ready with the TmThreadTimeoutLoop in a
thread unsafe way.

As a result TmThreadDisablePacketThreads would not wait long enough for the
worker(s) to finish its work and move the threads to the THV_RUNNING_DONE
phase by issuing the THV_KILL command.

When waiting for packet processing threads to process all in-flight packets,
also consider the 'stream_pq'. This will have received the flow timeout
packets.

Bug #1871.
 6 years ago
Victor Julien 825173a2ba threading: fix flow timeout loop race 6 years ago
Victor Julien 56354afd41 threading: improve thread queues checking by dumping more info 6 years ago
Victor Julien 0a809bf577 packet: set unique pkt_src 'flush' packets 
Set unique type for capture timeout and for detect reload flush
to assist in debugging.
 6 years ago
Victor Julien 6bc7636826 stream: remove unused code 
Remove now unused 'pkt_src' type as well.

Remove related unittests.
 6 years ago
Victor Julien 1633744fcb nfq: remove unused queue handler type 6 years ago
Victor Julien ab01cbe345 log-pcap: remove stale comments 6 years ago
Victor Julien 6de025bb12 alert-syslog: remove stale comments 6 years ago
Jeff Lucovsky 90c2e3561c Add general purpose `ARRAY_SIZE` macro 
This commit adds `ARRAY_SIZE` as an helper for determining the number of
elements in an initialized array. The calculation is the same but the
macro provides a convenient shortcut. The implementation was borrowed
from the kernel sources.
 6 years ago
Jeff Lucovsky ae198add6d detect/analyzer: Refactor engine analysis code 
This commit changes the analysis code to be table driven to better
identify the rule elements covered by the analysis.
 6 years ago
Philippe Antoine 6e63c957ff signature: Fixes memory leak in parsing app layer event 6 years ago
Jason Ish fccbd36d37 dns: log addresses in flow direction, not packet 
Ticket #3340.
https://redmine.openinfosecfoundation.org/issues/3340
 6 years ago
Victor Julien 0824b04134 filestore: don't assume flow is TCP 
Filestore can be used by UDP based protocols as well. NFSv2 is one
that Suricata supports.

Bug #3277.
 6 years ago
Victor Julien 2a55afbd89 decode/pppoe: fix potential crash in debug statement 6 years ago
Victor Julien 2ab7fb4b41 version: automate and cleanup ver handling 
Create a single function to return the version string, to avoid lots
of ifdefs in multiple places.

Make the version determine the 'release' status. If the version from
autoconf has '-dev' in the name, it is not a release. If it hasn't
it is considered a release version.
 6 years ago
Victor Julien 51ec980e80 dataset: fix string length handling in hash 6 years ago
Eric Leblond 1721da91ef dataset: fix hash computation 6 years ago
Victor Julien 64a789bbf6 nfq: clear memory of queue before using it 
Avoids using uninitialized memory. Show showed itself
in nonsense values in counters, and in nfq_handle_packet
errors that were likely the result of passing uninitialized
memory to the nfq API.

Bug 3263.
Bug 3120.

Fixes: b2a6c60dee ("source-nfq: increase maximum queues number to 65535")
 6 years ago
Victor Julien 2fd1174a56 nfq: micro optimization 6 years ago
Victor Julien 9d6087f7d6 nfq: don't warn on 'handle_packet' error 
NFQ can generate warnings/errors with a delay. After Suricata has
succesfully passed a verdict to the kernel, there are still things
that can go wrong for that verdict. This is then passed to the
queue through a netlink error message, which leads to nfq_handle_packet
returning an error code.

Suppress the warning. Also remove the errno/strerror use as
nfq_handle_packet does not set the errno.

Thanks to Florian Westphal.

Bug 3120.
 6 years ago
Victor Julien f8acad7fca nfq: code cleanups 6 years ago
Alexander Gozman f280e66f84 nfq: check for EAGAIN after recv() call in NFQRecvPkt() 6 years ago
Victor Julien 4cc90e9a4c nfq: minor code cleanups 6 years ago
Victor Julien 01cea2ec89 datasets: suppress noisy debug statement 6 years ago
Victor Julien 505b2dd256 log-pcap: don't print (null) for compression method 6 years ago
Victor Julien fb26268c6b tcp: don't set event on empty SACK opt 
TCP_OPT_INVALID_LEN was set if the opt len was 2. While useless
an empty SACK is not uncommon.

Seen on an iOS device talking to an Apple server.

Bug #3254.
 6 years ago
Victor Julien aae76a84cd suricata: use version from autoconf 6 years ago
Eric Leblond 2d11e9394c detect-base64: fix url in list keywords commands 6 years ago