Remove the Jansson specific feature of being able to delete
an object from json_t, in prep for refactors to JsonBuilder.
Instead create a new header for each alert to be logged.
Move the logging of the rule text to where the alert object
is being logged to remove the usage of json_object_get...
Getting previously logged objects will not be possible with
JsonBuilder.
When PCRE `jit` is available, store the JIT stack in the keyword context
instead of on a global id. This ensures proper cleanup and
re-initialization over a rule reload.
File store v1 has been deprecated and was scheduled for removal
by June 2020.
Log an error if a file-store configuration is loaded without
version set to 2.
All the dead code in C after the Rust implementation is hereby removed.
Invalid/migrated tests have also been deleted.
All the function calls in C have been replaced with appropriate calls to
Rust functions. Same has been done for smb/detect.rs as a part of this
migration.
This commit improves how the parser handles the `RSET` command.
Termination of the transaction occurs when the `RSET` ack is seen (reply
code 250).
Bug: #3677
In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar
structures would not be cleaned up because the cleanup depends on the
'header' pointer being set, but the error handling would unset that.
This could mean these structures were already filled with values before
the error was detected. As packets were recycled, the next packet decoding
would use this unclean structure.
To make things worse these structures are part of unions. IPv4/IPv6 and
TCP/ICMPv4/ICMPv6 share the same memory location.
LibFuzzer+UBSAN found this both locally and in Oss-Fuzz:
decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool'
#0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9
#1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13
#2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9
#3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25
#4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
#5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
#6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
#7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5
#8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6
#9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
#10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8)
Bug: #3496
Support reassembling multi-frag certificates. For this the cert queuing
code is changed to queue just the cert, not entire tls record.
Improve message tracking. Better track where a message starts and ends
before passing data around.
Add wrapper macros to check for 'impossible' conditions that are activate
in debug validation mode. This helps fuzzers find input that might trigger
these conditions, if they exist.
To avoid future memcpy issues introduce a wrapper and check the
result of it.
When compiled with --enable-debug-validation the wrapper will abort if
the input is wrong.
In some error conditions, or potentially in case of multiple 'certificate'
records, the extracted subject, issuerdn and serial could be overwritten
without freeing the original memory.
'trec' buffer was not grown properly when it was checked as too small.
After this it wasn't checked again so that copying into the buffer could
overflow it.
This commit checks whether pre-6.x settings for ERSPAN Type I are
present. ERSPAN Type I is no longer enabled/disabled through a
configuration setting -- it's always enabled.
When a setting exists to enable/disable ERSPAN Type I decoding, a
warning message is logged.
Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x
This commit adds a warning value when ERSPAN Type I configuration
settings are detected; specifically, when ERSPAN Type I `enabled` is
specified.
Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x
This commit fixes the conversion of timestamps. Without the extra
parens, the resulting timestamp value for usecs will be 1 or 0 due to
the operator precedence order (+ takes precedence over ?:)
The end-of-processing has been restructured so that Packet and Hostbuffer
data structures are now released within the NapatechReleasePacket() callback
function.
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* functions.
Partially closes redmine ticket #3053.
This patch allows to OR multiple flowbits on isset and isnotset flowbit
actions.
e.g.
Earlier in order to check if either fb1 or fb2 was set, it was required
to write two rules,
```
alert ip any any -> any any (msg:\"Flowbit fb1 isset\"; flowbits:isset,fb1; sid:1;)
alert ip any any -> any any (msg:\"Flowbit fb2 isset\"; flowbits:isset,fb2; sid:2;)
```
now, the same can be achieved with
```
alert ip any any -> any any (msg:\"Flowbit fb2 isset\"; flowbits:isset,fb1|fb2; sid:23;)
```
This operator can be used to check if one of the many flowbits is set
and also if one of the many flowbits is not set.
For a done transaction with command PORT,
we expect FTP_STATE_FINISHED
and we got FTP_STATE_PORT_DONE instead
which prevented logging of these transactions
We change the order of the evaluations to get the right result
Protects against evasion by TCP packet splitting
The problem arised if the FTP response is split on multiple packets
The fix is to bufferize the content, until we get a complete line
Each 'add' and 'lookup' would increment the use_cnt, without anything
bringing it back down.
Since there is no removal yet, nothing is actually affected by it yet.
Support either the __thread GNUism or the C11 _Thread_local.
Use 'thread_local' to point to the one that is used. Convert existing
__thread user to 'thread_local'.
Remove non-thread-local code from the packet pool code.
Until now both atomic ints and pointers were initialized by SC_ATOMIC_INIT
by setting them to 0. However, C11's atomic pointer type cannot be
initialized this way w/o causing compiler warnings.
As a preparation to supporting C11's atomics, this patch introduces a
new macro to initialize atomic pointers and updates the relevant callers
to use it.
Until this point the SC_ATOMIC_ADD macro pointed to a 'add_fetch'
intrinsic. This patch changes it to a 'fetch_add'.
There are 2 reasons for this:
1. C11 stdatomics.h has only 'atomic_fetch_add' and no 'add_fetch'
So this patch prepares for adding support for C11 atomics.
2. It was not consistent with SC_ATOMIC_SUB, which did use 'fetch_sub'
and not 'sub_fetch'.
Most callers are not using the return value, so these are unaffected.
The callers that do use the return value are updated.
A deeply nested YAML file can cause a stack-overflow while
reading in the configuration to do the recursive parser. Limit
the recursion level to something sane (128) to prevent this
from happening.
The default Suricata configuration has a recursion level of 128
so there is still lots of room to grow (not that we should).
Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/3630
Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.
This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
Fixes runs with --enable-debug-validation. The target did not init a
packet pool, so for a tunnel packet would try to get a packet from
an uninitialized pool. In non-debug mode, this silently works by
falling back to a packet from alloc.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff35a6801 in __GI_abort () at abort.c:79
#2 0x00007ffff359639a in __assert_fail_base (fmt=0x7ffff371d7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555557fe7260 "!(pool->initialized == 0)",
file=file@entry=0x555557fe7220 "tmqh-packetpool.c", line=line@entry=253, function=function@entry=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:92
#3 0x00007ffff3596412 in __GI___assert_fail (assertion=0x555557fe7260 "!(pool->initialized == 0)", file=0x555557fe7220 "tmqh-packetpool.c", line=253,
function=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:101
#4 0x00005555577e24be in PacketPoolGetPacket () at tmqh-packetpool.c:253
#5 0x0000555556914ecd in PacketGetFromQueueOrAlloc () at decode.c:183
#6 0x00005555569161e1 in PacketTunnelPktSetup (tv=0x555559863980 <tv>, dtv=0x614000068e40, parent=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=72, proto=DECODE_TUNNEL_IPV4) at decode.c:286
#7 0x00005555569de694 in DecodeIPv4inIPv6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", plen=72) at decode-ipv6.c:59
#8 0x00005555569e60b5 in DecodeIPV6ExtHdrs (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=112) at decode-ipv6.c:522
#9 0x00005555569e846f in DecodeIPV6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-ipv6.c:641
#10 0x0000555556a032f9 in DecodeRaw (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-raw.c:70
#11 0x0000555557659ba8 in DecodePcapFile (tv=0x555559863980 <tv>, p=0x61e0000fc080, data=0x614000068e40) at source-pcap-file.c:412
#12 0x0000555556573401 in LLVMFuzzerTestOneInput (data=0x613000000047 "\241\262\315\064", size=339) at tests/fuzz/fuzz_sigpcap.c:158
#13 0x0000555557a4dc66 in main (argc=2, argv=0x7fffffffdfa8) at tests/fuzz/onefile.c:51
That line:
BUG_ON(pool->initialized == 0);
Ensure that the by_rule threshold table is initialized if a rule
is thresholded by_rule. Replace manual table reallocaton with calls
to the common function.
The only difference between threshold calculations for by_src/by_dst,
by_rule or by_both is which table stores the DetectThresholdEntry.
Refactor the ThresholdHandlePacket* functions to do table lookup and
storage individually, but calculate thresholds in a common function.
Make sure to cleanup expectations for a flow as the first step, before
parts of the flow itself are getting cleaned/freed.
Also indicate use unlikely as flows with expectations should be relatively
rare.
When a flow timeout, we can have still existing expectations that
are linked to this flow. Given that there is a delay between the
real ending of the flow and its destruction by Suricata, the
expectation should be already honored so we can assume the risk
to clean the expectations that have been triggered by the
to-be-deleted flow.
This patch introduces a limitation in term of number of
expectations attached to one IPPair. This is done using
a circle list so we have a FIFO approach on expectation
handling.
Circleq list code is copied from BSD code like was pre existing code
in queue.h.