Windows has no SIGHUP and no unix socket, so Suricata could not be told to
reopen its log files after external log rotation. When running as a service,
this can now be requested with the user-defined control code 129, for
example:
sc.exe control Suricata 129
The service control handler only records the request; the main loop then
handles it the same as SIGHUP.
Ticket: 7757
On Windows, fopen() does not open files with FILE_SHARE_DELETE, so a log file
can not be renamed or deleted while Suricata has it open. This prevents
external tools from rotating log files.
Add SCWin32Fopen(), an fopen() replacement for the "a" and "w" modes that
opens the file with CreateFile() and FILE_SHARE_DELETE, and use it for log
files, including suricata.log.
Ticket: 7757
Windows has no SIGUSR2 and no unix socket, so rules could not be reloaded
without restarting Suricata. When running as a service, a rule reload can now
be requested with the user-defined control code 128, for example:
sc.exe control Suricata 128
The service control handler only records the request; the main loop then
handles it the same as SIGUSR2.
Ticket: 7756
By default, a Windows service allows interactive users and service accounts
to send it user-defined control codes (128-255). When installing the Suricata
service, set a DACL that is the default DACL without this right for those
users, so only SYSTEM and Administrators can send them.
This prepares for using user-defined control codes to control Suricata at
runtime, such as reloading rules.
Ticket: 7756
So far the SSL parser returned an error if there were less than 4 bytes
available for the handshake header. This meant that a fragmented
handshake header could result in the SSL parser erroring out and thus
disabling any further inspection for the protocol.
The parser follows a multi-record parsing scheme which means that the
fragment of the header may exist in the next record. Use internal
buffering to assess and process the records just like the handshake
buffer.
Ticket: 9167
Ticket: 8684
The behavior of the rule
alert ip any any -> any any (pcre:"/.*mc|\H+$/"; sid:4;)
depended upon the presence of another rule such as
alert ip any any -> any any (content:"*mcH"; sid:5;)
With the sid 5 content rule, the code went through PrefilterPktStream
which updated raw_stream_progress, which ended up updating raw_progress_rel
which was used on later packets to know where to start.
Without the sid 5 content rule, there was no update of these progresses.
It led to quadratic complexity, as for each new packet, we ran
the pcre engine on the whole tcp stream from the beginning, with
the newly added payload data.
Fix is to process sid 4 pcre rule the same way whether there is
or there is not a sid 5 content rule.
So, in case we do not have a mpm store for a signature group head,
we register a dummy stream prefilter, whose goal will just be
to update raw_stream_progress
DCERPC REQUEST/RESPONSE records can have their stub data fragmented
across multiple PDUs. The flags that convey the state of the current PDU
are as follows:
PFC_FIRST_FRAG: first fragment of (fragmented) data
PFC_LAST_FRAG: last fragment of (fragmented) data
PFC_FIRST_FRAG && PFC_LAST_FRAG: no fragmentation
Consider a REQUEST/RESPONSE complete only when the PFC_LAST_FRAG flag is
found to be set.
Ticket: 9240
Serialize service arguments with Windows quoting and backslash escaping
for both installation and parameter updates. Reject truncated executable
paths and command lines before accessing the Service Control Manager.
Keep the existing service-option prefix matching unchanged. Limit the
fix to command-line construction, without changing the command-line
parser, service function interfaces or SCInstance layout.
Ticket: 9241
With rotate-interval set to minute, hour or day, rotations were not
aligned to the local time boundaries. The first rotation was computed
by SCGetSecondsUntil(), which overshoots the next hour by one minute
(hh:01:00) and the next day by one hour and one minute (01:01:00).
Every following rotation was re-armed with a fixed interval
(e.g. 86400 s) from the time of the rotation.
Local days are not always 86400 s long, so the day rotation also moved
by the DST shift on every DST transition and stayed there until the
next one. A file named after a date then held events of the next day.
Instead, this commit rotates logs when the local minute, hour or date
changes. The daily rotation now happens at local midnight.
On SIGHUP, the period is recorded before the file is reopened, so a boundary
(the next minute/hour/day) passing during the reopen leads to an extra
reopen instead of a missed rotation.
Ticket: #9139
In continuous pcap-file directory mode, a file that makes InitPcapFile()
fail (malformed content, not a pcap, unsupported link type, truncated
header) was skipped without folding its mtime into last_time_seen.
This caused the file(s) to be reinspected on every follow-up poll until
a newer valid file was processed.
Ticket: 9129
The passed poll-interval should determine how many seconds to wait
in-between polls. The previous code passed the value through localtime
function which understood the input integer as an epoch time and
converted it to time structure, of which only seconds were kept.
This kind of worked for sub-60 values but wrapped for values 60+ to
the usual seconds range 0-59. Additionally, setting poll-interval to
60 made it a busy loop with no sleep between polls.
With intervals of 60+ seconds now honored, a single sleep blocked the
shutdown (or a unix socket pcap-interrupt) until the interval elapsed,
and the engine was killed after the 60 second receive thread shutdown
timeout. The wait is now done in one second steps, checking the run
status in-between.
Ticket: 9129
Change the byte order of vlan TCI when extracting it
from vlan_hdr in xdp_filter.c. This fixes xdp bypass for vlan tagged
packets, as without the fix flows that are supposed to be bypassed by
the xdp filter are not bypassed.
Ticket: 8699
A rule whose relative pcre follows a keyword that moves the detection
pointer is refused at load time with "pcre with /R (relative) needs
preceding match in the same buffer":
alert tcp any any -> any any (msg:"x"; byte_jump:4,0; pcre:"/^x/R"; ...)
The search for the preceding match only asked for content and pcre
sigmatches, so a rule that anchors a relative pcre on a byte_jump had
nothing to find even though byte_jump leaves the detection pointer at
the jump target.
Add the keywords that update det_ctx->buffer_offset to the search:
byte_jump, byte_extract and byte_math. byte_test and isdataat only read
that pointer, so they stay rejected; a relative pcre after them alone
would silently match from the start of the buffer.
Ticket: #7987.
A packet that netmap delivers across several ring slots was handed to the
rest of the code as the first slot buffer together with the summed length
of all the slots. When those slot buffers were not next to each other in
memory this read past the first buffer, which could crash the process or
leak bytes from unrelated buffers into inspection, pcap output and IPS or
tap forwarding.
Multi-slot packets are now copied slot by slot into a per-thread buffer so
the caller always gets one addressable frame, and such packets are always
copied into the Packet instead of being referenced zero-copy. Single-slot
packets keep the zero-copy path. The walk over the fragment slots is bounded
by the slots we own and the total length is capped, so a broken or over-long
chain is dropped as one unit and the ring keeps moving forward.
Adds unit tests that build a fake ring and cover single-slot, adjacent,
non-adjacent, ring-wrap, empty-fragment, over-size and never-ending chains.
Ticket: #8777
NetmapReadPackets walked the slots of a single packet inline while also
driving the per ring iteration. Move the slot walk into
NetmapReadOnePacket so the ring loop only picks the next packet and the
slot handling can be changed on its own.
No functional change.
Ticket: #8777
Packets received in segmented mbufs that cannot be linearized are
inspected as a copy. In the copy modes (IPS/TAP), modifications made to
the copy (inline stream normalization, the replace keyword) did not
reach the mbufs, so the unmodified packet was forwarded. Chains larger
than the maximum packet size were truncated for inspection but
forwarded in full, and chains that could not be copied were forwarded
uninspected.
When Suricata modifies a copied packet, it now writes the data back to
the mbuf chain before transmitting it. Chains that cannot be inspected
in full or transmitted are dropped: chains larger than the maximum
packet size, chains with more segments than the copy interface can
transmit, and malformed chains.
Forwarding chains requires the multi-segment TX offload, which replaces
MBUF_FAST_FREE and selects a slower TX path on some NICs. As mbufs sized
from the MTU rarely lead to chains on physical NICs, forwarding them is
opt-in through the new tx-segmented-mbufs interface setting (default
false), which both copy interfaces must share. Without it, chains that
cannot be linearized are dropped in the copy modes.
The drops are counted in capture.dpdk.segmented_drops and
capture.dpdk.segmented_too_large. In IPS mode, they are also counted in
ips.blocked with the new capture_error drop reason, so that the drop
reasons explain the blocked packets.
Ticket: 6012
Especially when running as a secondary DPDK process,
Suricata may receive segmented (chained) mbufs from
the primary application. Previously, these packets would
trigger a warning and potentially be processed
incorrectly since only the first segment was accessible.
Suricata now handles segmented mbufs by:
1. Attempting to linearize in-place with rte_pktmbuf_linearize()
2. If linearization fails (insufficient tailroom), copying all
segments into the packet's internal buffer
A one-time warning is still logged advising users to increase
mbuf size (secondary) or MTU (primary) for better performance.
Ticket: 6012
The replace keyword was only enabled in the NFQ and IPFW runmodes, so in
DPDK IPS mode rules with replace only alerted without modifying
packets. DPDK packets point directly to the mbuf data, so the modified
payload and the recalculated checksums are forwarded as they are.
Enable replace for the DPDK runmode when the engine runs in IPS mode.
Some major drivers (ixgbe, i40e and ice) size their maximum frame for two
VLAN tags (QinQ) in addition to the Ethernet header and CRC.
This change moves Suricata to be in line with their behavior.
Ticket: 6012
The replace keyword documentation said that Suricata recalculates the
checksums after a replacement. Suricata only recalculates the IPv4
header checksum and the TCP and UDP checksums, so for example a replaced
ICMP payload is forwarded with its original ICMP checksum.
Several C loggers are registered through a cast to
EveJsonSimpleTxLogFunc, which has an incompatible signature. Clang's
function sanitizer reports this as a call through a pointer to an
incorrect function type.
Change these loggers to the generic callback signature and register them
without casts.
Ticket: 9226
Lukas Sismis
Jason Ish
Shivani Bhardwaj
Philippe Antoine
Lukas Sismis
Philippe Antoine
Adam Kiripolsky
Victor Julien
Samaresh Kumar Singh
Mahmoud Maatuq