Update port grouping logic. Previously it would create one consistent
list w/o overlap. It largely still does this, except for the 'catch
all' port group at the end of the list. This port group contains all
the sigs that didn't fit into the other groups.
Replace tree based approach for rule grouping with a per port (tcp/udp)
and per protocol approach.
Grouping now looks like:
+----+
|icmp+--->
+----+
|gre +--->
+----+
|esp +--->
+----+
other|... |
+----->-----+
| |N +--->
| +----+
|
| tcp +----+ +----+
+----->+ 80 +-->+ 139+-->
| +----+ +----+
|
| udp +----+ +----+
+---+----->+ 53 +-->+ 135+-->
| +----+ +----+
|toserver
+--->
|toclient
|
+--->
So the first 'split' in the rules is the direction: toserver or toclient.
Rules that don't have a direction, are in both branches.
Then the split is between tcp/udp and the other protocols. For tcp and
udp port lists are used. For the other protocols, grouping is simply per
protocol.
The ports used are the destination ports for toserver sigs and source
ports for toclient sigs.
Introduce 'ac-ks' or the Kenneth Steele AC implementation. It's
actually 'ac-tile' written by Ken for the Tilera platform. This
patch adds support for it on other architectures as well.
Enable ac-tile for other archs as 'ac-ks'.
Fix a bunch of OOB reads in the loops that triggered ASAN.
*** CID 1358023: Null pointer dereferences (REVERSE_INULL)
/src/util-mpm-hs.c: 860 in SCHSDestroyThreadCtx()
854 if (thr_ctx->scratch != NULL) {
855 hs_free_scratch(thr_ctx->scratch);
856 mpm_thread_ctx->memory_cnt--;
857 mpm_thread_ctx->memory_size -= thr_ctx->scratch_size;
858 }
859
>>> CID 1358023: Null pointer dereferences (REVERSE_INULL)
>>> Null-checking "mpm_thread_ctx->ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
860 if (mpm_thread_ctx->ctx != NULL) {
861 SCFree(mpm_thread_ctx->ctx);
862 mpm_thread_ctx->ctx = NULL;
863 mpm_thread_ctx->memory_cnt--;
864 mpm_thread_ctx->memory_size -= sizeof(SCHSThreadCtx);
865 }
Direct leak of 80 byte(s) in 5 object(s) allocated from:
#0 0x4c673b in __interceptor_malloc (/home/victor/dev/suricata/src/suricata+0x4c673b)
#1 0xb7a425 in DetectEngineSignatureIsDuplicate /home/victor/dev/suricata/src/detect-parse.c:1715:10
#2 0xb79390 in DetectEngineAppendSig /home/victor/dev/suricata/src/detect-parse.c:1836:19
#3 0x86fe56 in DetectLoadSigFile /home/victor/dev/suricata/src/detect.c:357:15
#4 0x815fee in ProcessSigFiles /home/victor/dev/suricata/src/detect.c:419:13
#5 0x8139a8 in SigLoadSignatures /home/victor/dev/suricata/src/detect.c:499:15
#6 0xfe435d in LoadSignatures /home/victor/dev/suricata/src/suricata.c:1979:9
#7 0xfcd87e in main /home/victor/dev/suricata/src/suricata.c:2345:17
#8 0x7fb66bf7cec4 in __libc_start_main /build/eglibc-3GlaMS/eglibc-2.19/csu/libc-start.c:287
By default, hashlittle() will read off the end of the key, up to the
next four-byte boundary, although the data beyond the end of the key
doesn't affect the hash. This read causes uninitialized read warnings
from Valgrind and Address Sanitizer.
Here we add hashlittle_safe(), which avoids reading off the end of the
buffer (using the code inside the VALGRIND-guarded block in the original
hashlittle() implementation).
Capture methods that are non blocking will still not generate packets
that go through the system if there is no traffic. Some maintenance
tasks, like rule reloads rely on packets to complete.
This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
instructs the capture thread to create a fake packet.
The capture implementations can call the TmThreadsCaptureInjectPacket
utility function either with the packet they already got from the pool
or without a packet. In this case the util func will get it's own
packet.
Implementations for pcap, AF_PACKET and PF_RING.
Split wait loop into three steps:
- first insert pseudo packets
- 2nd nudge all capture threads to break out of their loop
- third, wait for the detection thread contexts to be used
Interupt capture more than once if needed
Move packet injection into util func
Simplify handling of USR2 signal. The SCLogInfo usage could lead to
dead locks as the SCLog API can do many complicated things including
memory allocations, syslog calls, libjansson message construction.
If an existing malloc call was interupted, it could lead to the
following dead lock:
0 __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:97
1 0x0000003140c7d2df in _L_lock_10176 () from /lib64/libc.so.6
2 0x0000003140c7ab83 in __libc_malloc (bytes=211543457408) at malloc.c:3655
3 0x0000003140c80ec2 in __strdup (s=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ") at strdup.c:43
4 0x000000000059dd4a in SCLogMessageGetBuffer (tval=0x7fff52b47360, color=1, type=SC_LOG_OP_TYPE_REGULAR, buffer=0x7fff52b47370 "", buffer_size=2048,
log_format=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ", log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:307
5 0x000000000059e940 in SCLogMessage (log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:549
6 0x000000000057e374 in SignalHandlerSigusr2StartingUp (sig=12) at suricata.c:287
7 <signal handler called>
8 _int_malloc (av=0x3140f8fe80, bytes=<value optimized out>) at malloc.c:4751
9 0x0000003140c7ab1c in __libc_malloc (bytes=296) at malloc.c:3657
10 0x0000000000504d55 in FlowAlloc () at flow-util.c:60
11 0x00000000004fd909 in FlowInitConfig (quiet=0 '\000') at flow.c:454
12 0x0000000000584c8e in main (argc=6, argv=0x7fff52b4a3b8) at suricata.c:2300
This patch simply sets a variable and lets the main loop act on that.
Victor Julien
Mats Klepsland
browner87
Jason Ish
Justin Viiret