Commit Graph

15548 Commits (6e10c660782044472fd0fb2bdc91c4c102c7fb5b)
 

Author SHA1 Message Date
Victor Julien 6e10c66078 eve/frame: improve frame payload logging
Log using stream callback API, meaning that data will also
be logged if there are GAPs.

Also implement GAP indicators: '[123 bytes missing]'.
12 months ago
Victor Julien a205583269 eve/frames: pass membuffer to API
In preparation of stream logging changes.
12 months ago
Victor Julien 462a6d7913 eve/alert: init membuffer size on missing config
Don't init buffer to 0 size but use the desired default of 4k.
12 months ago
Victor Julien 43858f70ad eve/alert: log payload directly from stream buffer
This avoids looping over partly duplicate segments that cause
output data corruption by logging parts of the stream data multiple
times.

For data with GAPs now add a indicator '[4 bytes missing]' similar
to how Wireshark does it.

Bug: #6553.
12 months ago
Victor Julien 829bab295b eve/frame: implement payload-buffer-size option
Modeled after the same option in eve/alert. Defaults to 4k.
12 months ago
Victor Julien a5a6527d26 stream: const args for StreamReassembleLog
Needed a workaround cast for RBTREE use.
12 months ago
Victor Julien f53c4ab149 yaml: remove newline from error message 12 months ago
Jeff Lucovsky e41c2f15c2 gen/typo: Correct comment typo 12 months ago
Jeff Lucovsky 3c5745978f flow: Swap thread_ids
Issue: 6835

When swapping the flow's direction, also swap the thread_ids.

This should help with the issues identified in
https://redmine.openinfosecfoundation.org/issues/2725
12 months ago
Jeff Lucovsky 9ad73faa0a flow/inject: Ensure initialized thread value used
Issue: 6835

When injecting a flow, ensure that the selected thread_id has been
initialized. When a flow is picked up midstream, the initialized thread
can be the second thread element.
12 months ago
Philippe Antoine 632e52ca2b ci: update ubuntu22.04 builds with clang14+asan
using a workround about ASLR
12 months ago
Lukas Sismis 16c88f2db7 dpdk: fix typo in the struct member name 12 months ago
Lukas Sismis 5592ec079d dpdk: refactor the main packet loop into smaller functions 12 months ago
Lukas Sismis 0b5966c347 dpdk: only close the port when workers are synchronized
When Suricata was running in IPS mode and received a signal to stop,
the first worker of every interface/port stopped the port and
proactively stopped the peered interface as well.
This was done to be as accurate with port stats as possible.
However, in a highly active scenarios (lots of packets moving around)
the peered workers might still be in the process of a packet
release operation. These workers would then attempt to transmit
on a stopped interface - resulting in an errorneous operation.

Instead, this patch proposes a worker synchronization of the given
port. After these workers are synchronized, it is known that no packets
will be sent of the peered interface, therefore the first worker can
stop it. This however cannot be assumed about "its own" port as the
peered workers can still try to send the packets. Therefore, ports
are only stopped by the peered workers.

Ticket: #6790
12 months ago
Philippe Antoine 45bb936187 http: event on request line missing protocol
Ticket: 6856
12 months ago
Jason Ish 8284df3ed4 devguide: add an upgrade section
Add an upgrade section to the devguide. This should cover any changes
to APIs that users might be using from plugins or as a library user.
12 months ago
Jason Ish b7b16fb481 eve/filetype: reorder fields to match lifecycle
Enhances readability.
12 months ago
Jason Ish a3354e55e6 eve/filetypes: use more const 12 months ago
Jason Ish eee9757dba eve/filetype: ThreadDeinit can return void
Change ThreadDeinit to return void instead of an int, there is nothing
to be done on success or failure.
12 months ago
Jason Ish cd85d89f14 output-eve: doxygen docs for SCEveFileType
Add documentation for the SCEveFileType in Doxygen format.
12 months ago
Jason Ish 500d29f356 doxygen: document the examples directory 12 months ago
Jason Ish bd55cd4c55 eve/filetypes: common init for threaded and non-threaded
In 7.0 if EVE was non-threaded, the ThreadInit for the filetype was
not called meaning that the filetype author had to handle the threaded
and non-threaded cases.

To simplify this, if non-threaded, still call ThreadInit (and
ThreadDeinit) once with a thread_id of 0. This should simplify
authoring EVE filetype plugins.
12 months ago
Jason Ish 3ff72d3efa eve: rename plugin to filetypes
EVE filetypes are not always plugins, for example, null and syslog
that are built-in filetypes.
12 months ago
Jason Ish 7c8c9fff32 plugins: remove conf.h from suricata-plugin.h
Remove "conf.h" from suricata-plugin.h as its not needed by that
header. However, some other files became transitively dependent on
through other includes, so fix those up.
12 months ago
Jason Ish ead09c2497 eve/filetypes: remove from plugin context
Remove EVE filetypes from plugin context as they are not only used
from plugins. Plugins allow user code to register filetypes, but we
also have internal file types that use this api including the null
output and syslog.  Additionally library users can use this API to
register filetypes, and they are not plugins.

Ideally this code would go in "output-json.[ch]" as the "primary" eve
API, however there are currently some include circular include issues
there, so start new cleaned up EVE API in "output-eve.[ch]" which is
"clean" with respect to includes, and as we cleanup existing EVE API for
"public" use, it can be moved here.

Ticket: #6838
12 months ago
Jeff Lucovsky 3bf92bb14f example/plugin: Use ThreadId 12 months ago
Jeff Lucovsky 85d321a689 output/plugin: Use Suri thread-id for plugins
Issue: 6408

Use the Suricata thread id for plugin thread initialization to give the
plugin a better correlating factor to the actual Suricata threads.
12 months ago
Victor Julien 1122fa24b6 mpm/ac: minor test cleanups 12 months ago
Victor Julien d47bfbb14d mpm/ac: add endswith test 12 months ago
Victor Julien c312d670d4 mpm/ac: implement endswith
When a pattern is using endswith, only consider it a match when it
is the end of the data.

Ticket: #6852.
12 months ago
Victor Julien b499239ef5 mpm: register algo features
This is so patterns can reply on mpm match meaning a full match.

Not yet used.
12 months ago
Victor Julien dcfa59bb69 mpm/ac: tidy up main search loop 12 months ago
Victor Julien 4fd0745b78 detect: minor cleanup 12 months ago
Jason Ish ece2029b0b requirements.txt: use suricata-update master 12 months ago
Victor Julien e5d3fad5f4 nfq: stricter thread sync
No longer update `Packet::flags` for tracking packet modifications,
as thread safety was not guaranteed.

Clearly separate between various kinds of `Packet::nfq_v` accesses for:
- mark
- mark_modified
- verdicted
These are either done under lock (Packet::persistent.tunnel_lock) or,
if the Packet is not part of a tunnel, not under lock.

This is safe as in all the related logic the Packet's tunnel state
is fixed and can no longer change.
12 months ago
Victor Julien d2f7e89d11 decode/tunnel: split verdict logic
Allows caller to take their own lock.
12 months ago
Victor Julien 6066c4d6e7 decode/tunnel: improve tunnel handling
Give each packet explicit tunnel type `ttype`: none, root, child.

Assigning happens when a (tunnel) packet is set up and is thread
safe.
12 months ago
Victor Julien 9bc42e3b34 decode/tunnel: move tunnel verdicted logic
In preparation of cleaning up thread safety, move "verdicted"
logic out of Packet::flags. Unsafe writes to "flags" can potentially
have side effects.
12 months ago
Victor Julien c31a6f5649 nfq: remove obsolete comment 12 months ago
Victor Julien 85ddeb6546 nfq: minor code cleanup 12 months ago
Victor Julien 792c87af25 nfq: use bool for verdicted packet var 12 months ago
Victor Julien af97316f42 defrag: match up v4 and v6 packet setup
v4 was doing redundant recursion level setup.

v6 was missing PKT_REBUILT_FRAGMENT flag.
12 months ago
Philippe Antoine 59d18b9a68 decode/pppoe: skip useless loop unless debug
Looping over tags has no effects, except debug logging
12 months ago
Philippe Antoine d2db8bfdca decode/pppoe: pointer cast consistency
Ticket: 6787

To do pointer arithmetic, we need to use uint8_t* pointer :
Pointer arithmetic in C is automatically scaled according
to the size of the data type.

Also simplifies the loop condition
12 months ago
Jason Ish 44388f1b69 src: make include guards more library friendly
Include guards for libraries should use a prefix that is meaningful for
the library to avoid conflicts with other user code. For Suricata, use
SURICATA.

Additionally, remove the pattern of leading and trailing underscores as
these are reserved for the language implementation per the C and C++
standards.
12 months ago
Shivani Bhardwaj 632ca75dd3 detect/port: handle single port that is range too
If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.

Bug 6843
12 months ago
Hadiqa Alamdar Bukhari 3aa313d0c5 dns: add dns.rcode keyword
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621
12 months ago
Juliana Fajardini 7b2bef1bc6 devguide: add chapter and short intro to libsuricata
With this, we intend to make more users aware of this use case, and that
we are working towards this.

Related to
Task #2693
12 months ago
Shivani Bhardwaj 2d6708f1ff util/interval-tree: fix coverity warning
Fix Coverity warning

** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()

________________________________________________________________________________________________________
*** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249                      * will be sorted, insert any new ports to the end of the list
250                      * and avoid walking the entire list */
251                     if (*list == NULL) {
252                         *list = new_port;
253                         (*list)->last = new_port;
254                     } else if (((*list)->last->port != new_port->port) &&
>>>     CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
>>>     "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255                                ((*list)->last->port2 != new_port->port)) {
256                         DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257                         (*list)->last->next = new_port;
258                         new_port->prev = (*list)->last;
259                         (*list)->last = new_port;
260                     } else {

The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.

Bug 6839
12 months ago
Jeff Lucovsky af529a56a9 netmap: Avoid HW errors when using pipes
Issue: 6837

When using netmap pipes (with lb, for example), avoid direct hardware
related IOCTLs that will fail (not supported with pipes).
12 months ago