When the file API is used to do content inspection (currently only
smtp does this), the detection should be considered while pruning
the file chunks.
This patch introduces a new flag for the file API: FILE_USE_DETECT
When it is used, 'FilePrune' will not remove chunks that are (partly)
beyond the File::content_inspected tracker.
When using this flag, it's important to realize that when the detect
engine is disabled or rules are not matching, content_inspected
might not get updated.
Cppcheck 1.72 gives a warning on the following code pattern:
char blah[32] = "";
snprintf(blah, sizeof(blah), "something");
The warning is:
(error) Buffer is accessed out of bounds.
While this appears to be a FP, in most cases the initialization to ""
was unnecessary as the snprintf statement immediately follows the
variable declaration.
We want to add counters in order to track the number of times we hit a
decode event. A decode event is related to an error in the protocol
decoding over a certain packet.
This patch fist modifies the decode-event list, reordering it in order
to separate single packet events from stream-related events and adding
the prefix "decoder" to decode events.
The counters are created during the decode setup and the relative event
counter is increased every time a packet with the flag PKT_IS_INVALID is
finalized in the decode phase
If stream.inline setting was missing it would default to IDS.
This patch changes the default to 'auto', meaning that in IPS mode
the stream engine also uses IPS mode and in IDS mode it's still in
IDS mode.
Bug #1570
When defrag creates a new reassembled IP packet, it then passes this
packet to the IP decoder. If this decoder returns an error the packet
is returned back to the packet pool with a call to TmqhOutputPacketpool
This lead to the first problem. The returned packet had it's p->root
pointer set, and it's PKT_TUNNEL flag set. This could cause problems
in TmqhOutputPacketpool, as this may reference the packet referenced
in p->root.
The second and more glaring problem is that the packet that was
returned to the packetpool, was still returned by the Defrag function
and processed further. It would then at the end of it's processing
be returned to the packet pool, which at this point already had a
reference to this packet.
This patch fixes both issues by unsetting the tunnel references and
returning NULL from Defrag in this case.
When --with-libpcre-libraries is used we skip the libpcre 8.35 check
since pkg-config might still point to the 8.35 version installed
although newer version was passed with --with-libpcre-libraries.
Logic used when adding a new prefix to a node was not correct
as we were allocating a prefix that could be at the end unused.
This patch is updating the code to have a complete creation to
be done if and only if we are needing the complete object. In
the other cases, it was enough to use the function input values.
This fixes:
104 (48 direct, 56 indirect) bytes in 2 blocks are definitely lost in loss record 184 of 327
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x9C2DAD: SCRadixCreatePrefix (util-radix-tree.c:144)
by 0x9AFA5B: SCRadixAddKey (util-radix-tree.c:522)
by 0x9B1A4D: SCRadixAddKeyIPV4Netblock (util-radix-tree.c:897)
by 0x67C824: IPOnlyPrepare (detect-engine-iponly.c:1197)
by 0x55172B: SigAddressPrepareStage2 (detect.c:3534)
by 0x5486F4: SigGroupBuild (detect.c:4671)
by 0x547C87: SigLoadSignatures (detect.c:538)
by 0x8FB5D0: LoadSignatures (suricata.c:1976)
by 0x8F3B32: main (suricata.c:2342)
TmThreadCreate copy string provided as name for threads to
avoid any issue is a non allocated string is used.
This patch also introduce TmThreadSetGroupName function. This
function is used to be sure we have an allocation when
assigning the thread group name. This way we can free allocated
memory at exit.
Both code changes have required some fixes in different parts of
the code to be in sync with the new API.
Good point about these changes is that it fixes an inconsistency
were some names were not allocated and some were.
This fixes:
72 bytes in 3 blocks are definitely lost in loss record 153 of 316
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x9AF041: SCRadixCreateRadixTree (util-radix-tree.c:430)
by 0x50FF5D: DefragPolicyLoadFromConfig (defrag-config.c:138)
by 0x5129F5: DefragInit (defrag.c:962)
by 0x87ECFD: UnixSocketPcapFilesCheck (runmode-unix-socket.c:386)
by 0x90FEC0: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x913C6D: UnixManager (unix-manager.c:980)
by 0x9072F3: TmThreadsManagement (tm-threads.c:602)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
16 bytes in 2 blocks are definitely lost in loss record 69 of 319
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x85955D: OutputStatsLogThreadInit (output-stats.c:118)
by 0x4CAE13: StatsMgmtThread (counters.c:352)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
96 bytes in 4 blocks are definitely lost in loss record 178 of 320
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x8FF88C: TmqhOutputFlowSetupCtx (tmqh-flow.c:163)
by 0x908D5F: TmThreadCreate (tm-threads.c:1097)
by 0x909A62: TmThreadCreatePacketHandler (tm-threads.c:1156)
by 0x8790CF: RunModeFilePcapAutoFp (runmode-pcap-file.c:188)
by 0x8837CB: RunModeDispatch (runmodes.c:372)
by 0x87F2A5: UnixSocketPcapFilesCheck (runmode-unix-socket.c:393)
by 0x910460: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x91420D: UnixManager (unix-manager.c:980)
by 0x907853: TmThreadsManagement (tm-threads.c:602)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Fix cleaning of threads where mutex and condition where not freed.
This fixes:
352 (192 direct, 160 indirect) bytes in 4 blocks are definitely lost in loss record 301 of 327
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x909404: TmThreadInitMC (tm-threads.c:1764)
by 0x908DE7: TmThreadCreate (tm-threads.c:1120)
by 0x90A326: TmThreadCreateMgmtThread (tm-threads.c:1183)
by 0x4CA0AD: StatsSpawnThreads (counters.c:856)
by 0x87F254: UnixSocketPcapFilesCheck (runmode-unix-socket.c:396)
by 0x910330: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x9140DD: UnixManager (unix-manager.c:980)
by 0x9077F3: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Queue name was sometimes allocated and sometimes not. This
patch updates the behavior of creation function so it is
always allocated. This way we can free it at exit and fix
memory leak.
This fixes:
900 bytes in 110 blocks are definitely lost in loss record 322 of 329
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x803E0A9: strdup (in /lib/x86_64-linux-gnu/libc-2.21.so)
by 0x90090E: StoreQueueId (tmqh-flow.c:112)
by 0x8FFEA8: TmqhOutputFlowSetupCtx (tmqh-flow.c:180)
by 0x908C7F: TmThreadCreate (tm-threads.c:1095)
by 0x909982: TmThreadCreatePacketHandler (tm-threads.c:1154)
by 0x87906F: RunModeFilePcapAutoFp (runmode-pcap-file.c:188)
by 0x88376B: RunModeDispatch (runmodes.c:372)
by 0x87F245: UnixSocketPcapFilesCheck (runmode-unix-socket.c:393)
by 0x9102B0: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x91405D: UnixManager (unix-manager.c:980)
by 0x907773: TmThreadsManagement (tm-threads.c:600)
tm-queue: fix logic WIP
This function is causing a memleak because it is necessary to clean
up after usage.
This fixes at least:
37 (32 direct, 5 indirect) bytes in 1 blocks are definitely lost in loss record 104 of 394
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x66D0C7B: ??? (in /usr/lib/x86_64-linux-gnu/libjansson.so.4.7.0)
by 0x943584: LiveDeviceIfaceList (util-device.c:264)
by 0x910889: UnixCommandExecute (unix-manager.c:486)
by 0x910D7E: UnixCommandRun (unix-manager.c:545)
by 0x911193: UnixMain (unix-manager.c:593)
by 0x913CC7: UnixManager (unix-manager.c:961)
by 0x907753: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Victor Julien
Mats Klepsland
Maurizio Abba
Eric Leblond
Andreas Herz