Commit Graph

15323 Commits (7cc86938fc3a7270d763c5d04ce7d2480e75562b)
 

Author SHA1 Message Date
Shivani Bhardwaj 0d05dcbb69 interval-tree: add base data structure
Ticket 6792
Bug 6414

(cherry picked from commit fde4ca5608)
2 years ago
Victor Julien 40d3e1e0cc detect/engine: fix whitelisting check
In the commit 4a00ae607, the whitelisting check was updated in a quest
to make use of the conditional better but it made things worse as every
range would be whitelisted as long as it had any of the default
whitelisted port which is very common.

(cherry picked from commit fb9680bb7b)
2 years ago
Philippe Antoine 930eadddcb detect: log relevant frames app-layer metadata
Ticket: 6973

Completes commit 2b4e10224e

(cherry picked from commit 9e01956e77)
2 years ago
Philippe Antoine d910924787 detect: use direction-based tx for app-layer logging
When we only have stream matches.

Ticket: 6846

This solves the case where another transaction was created
by parsing data in the other direction, before running the
detection.

Like
1. get data in direction 1
2. acked data: parse it, but do not run detection in dir 1
3. other data in direction 2
4. other data acked : parse it and create new tx,
then run detection for direction 1 with data from first packet

(cherry picked from commit 7274ad58aa)
2 years ago
Philippe Antoine a07a0c35fd output/alert: check flag before logging app-layer
Ticket: 6846
(cherry picked from commit 2b4e10224e)
2 years ago
Philippe Antoine 582014b60c output: do not use tx id 0 when there is no tx
Ticket: 6846

This led to packet rules logging irrelevant app-layer data

(cherry picked from commit 910f6af54f)
2 years ago
Victor Julien 8eb461a892 decode/ppp: fix iplen check int handling
** CID 1596376:    (CONSTANT_EXPRESSION_RESULT)
/src/decode-ppp.c: 64 in DecodePPPCompressedProto()
/src/decode-ppp.c: 55 in DecodePPPCompressedProto()

________________________________________________________________________________________________________
*** CID 1596376:    (CONSTANT_EXPRESSION_RESULT)
/src/decode-ppp.c: 64 in DecodePPPCompressedProto()
58             case 0x57: { /* PPP_IPV6 */
59                 if (unlikely(len < (data_offset + IPV6_HEADER_LEN))) {
60                     ENGINE_SET_INVALID_EVENT(p, PPPIPV6_PKT_TOO_SMALL);
61                     return TM_ECODE_FAILED;
62                 }
63                 DEBUG_VALIDATE_BUG_ON(len < data_offset);
>>>     CID 1596376:    (CONSTANT_EXPRESSION_RESULT)
>>>     "65535 /* 32767 * 2 + 1 */ < (uint16_t)(len - data_offset)" is always false regardless of the values of its operands. This occurs as the logical first operand of "?:".
64                 uint16_t iplen = MIN(USHRT_MAX, (uint16_t)(len - data_offset));
65                 return DecodeIPV6(tv, dtv, p, pkt + data_offset, iplen);
66             }
67             case 0x2f: /* PPP_VJ_UCOMP */
68                 if (unlikely(len < (data_offset + IPV4_HEADER_LEN))) {
69                     ENGINE_SET_INVALID_EVENT(p, PPPVJU_PKT_TOO_SMALL);
/src/decode-ppp.c: 55 in DecodePPPCompressedProto()
49             case 0x21: { /* PPP_IP */
50                 if (unlikely(len < (data_offset + IPV4_HEADER_LEN))) {
51                     ENGINE_SET_INVALID_EVENT(p, PPPVJU_PKT_TOO_SMALL);
52                     return TM_ECODE_FAILED;
53                 }
54                 DEBUG_VALIDATE_BUG_ON(len < data_offset);
>>>     CID 1596376:    (CONSTANT_EXPRESSION_RESULT)
>>>     "65535 /* 32767 * 2 + 1 */ < (uint16_t)(len - data_offset)" is always false regardless of the values of its operands. This occurs as the logical first operand of "?:".
55                 uint16_t iplen = MIN(USHRT_MAX, (uint16_t)(len - data_offset));
56                 return DecodeIPV4(tv, dtv, p, pkt + data_offset, iplen);
57             }
58             case 0x57: { /* PPP_IPV6 */
59                 if (unlikely(len < (data_offset + IPV6_HEADER_LEN))) {
60                     ENGINE_SET_INVALID_EVENT(p, PPPIPV6_PKT_TOO_SMALL);

(cherry picked from commit dc5b78ec71)
2 years ago
Victor Julien 59f922a134 decode/ppp: add missing types definitions
Recognize PPP_CCP, PPP_CBCP and PPP_COMP_DGRAM.

Does not implement decoders for these record types, so these
are logged as unsupported types. Was "wrong_type" before.

(cherry picked from commit 516441b600)
2 years ago
Victor Julien c06453722b decode/ppp: clean up ppph pointer use
No users of the pointer anymore, so remove it.

(cherry picked from commit 7e3f071e49)
2 years ago
Victor Julien 4f0709e0ee decode/ppp: remove ppph check in favor of flag
As we now support variable size headers, we can't use the old pointer.

Replace with a flag.

(cherry picked from commit 6067955afd)
2 years ago
Victor Julien 701468d3de decode/ppp: support different header formats
Support compressed proto and optional HDLC header.

Bug: #6942.
(cherry picked from commit 68092ff33c)
2 years ago
Victor Julien 9a74eaa48a hostbits: release use_cnt for unix (add|remove)-hostbit
Commands would leave use_cnt incremented, never decrementing them. This
would lead to a asserting triggering at shutdown.

Bug: #7020.
(cherry picked from commit d02c57bd1f)
2 years ago
Victor Julien 043efd45bf device: don't crash on unix command 'iface-bypassed-stat'
In the default config iface bypass support is not enabled,
and storage API not initialized for it. Using it will lead to a crash.

This commit first checks if the device storage API is initialized.

Bug: #7022.
(cherry picked from commit bc2dfe4c17)
2 years ago
Richard McConnell e87fe32bb1 app-layer: Set sc_errno upon error return
Bug: https://redmine.openinfosecfoundation.org/issues/6782

Callers to these allocators often use ``sc_errno`` to provide context of
the error. And in the case of the above bug, they return ``sc_errno``,
but as it has not been set ``sc_errno = 0; == SC_OK``.

This patch simply sets this variable to ensure there is context provided
upon error.

(cherry picked from commit fc2e49f84a)
2 years ago
Victor Julien 87be155d0f pcap-log: use correct pkthdr size for limit enforcement
The on-disk pcap pkthdr is 16 bytes. This was calculated using
`sizeof(struct pcap_pkthdr)`, which is 24 bytes on 64 bit Linux. On
Macos, it's even worse, as a comment field grows the struct to 280
bytes.

Address this by hardcoding the value of 16.

Bug: #7037.
(cherry picked from commit 6c937a9243)
2 years ago
Victor Julien 151b528470 time: only consider packet threads
In offline mode, a timestamp is kept per thread, and the lowest
timestamp of the active threads is used. This was also considering the
non-packet threads, which could lead to the used timestamp being further
behind that needed. This would happen at the start of the program, as
the non-packet threads were set up the same way as the packet threads.

This patch both no longer sets up the timestamp for non-packet threads
as well as not considering non-packet threads during timestamp
retrieval.

Fixes: 6f560144c1 ("time: improve offline time handling")

Bug: #7034.
(cherry picked from commit 5455799795)
2 years ago
Jeff Lucovsky aa034a6648 profiling/rules: Improve dynamic rule handling
Issue: 6861

Without this commit, disabling rule profiling via suricatasc's command
'ruleset-profile-stop' may crash because profiling_rules_entered becomes
negative.

This can happen because
- There can be multiple rules evaluated for a single packet
- Each rule is profiled individually.
- Starting profiling is gated by a configuration setting and rule
  profiling being active
- Ending profiling is gated by the same configuration setting and
  whether the packet was marked as profiling.

The crash can occur when a rule is being profiled and rule profiling
is then disabled after one at least one rule was profiled for the packet
(which marks the packet as being profiled).

In this scenario, the value of profiling_rules_entered was
not incremented so the BUG_ON in the end profiling macro trips
because it is 0.

The changes to fix the problem are:
- In the profiling end macro, gate the actions taken there by the same
  configuration setting and use the profiling_rues_entered (instead of
  the per-packet profiling flag). Since the start and end macros are
  tightly coupled, this will permit profiling to "finish" if started.
- Modify SCProfileRuleStart to only check the sampling values if the
  packet hasn't been marked for profiling already. This change makes all
  rules for a packet (once selected) to be profiled (without this change
  sampling is applied to each *rule* that applies to the packet.

(cherry picked from commit bf5cfd6ab7)
2 years ago
Philippe Antoine 2bd3bd0e31 http: fix nul deref on memcap reached
HttpRangeOpenFileAux may return NULL in different cases, including
when memcap is reached.
But is only caller did not check it before calling HttpRangeAppendData
which would dereference the NULL value.

Ticket: 7029
(cherry picked from commit fd262df457)
2 years ago
Philippe Antoine b556619213 rust/probing: safety check for null input
Ticket: 7013

Done consistently for all protocols

This may change some protocols behaviors which failed early
if they found there was not enough data...

(cherry picked from commit 37a9003736)
2 years ago
Philippe Antoine e797836b6e rust: return empty slice without using from_raw_parts
As this triggers rustc 1.78
unsafe precondition(s) violated: slice::from_raw_parts requires
the pointer to be aligned and non-null,
and the total size of the slice not to exceed `isize::MAX`

Ticket: 7013
(cherry picked from commit 5dc8dea869)
2 years ago
Philippe Antoine 5a5fe36bfc tests: do not bother to free a null pointer
Ticket: #7013
(cherry picked from commit edd5507ea4)
2 years ago
Victor Julien 84fc3bed2c detect/iprep: update doc about 0 value
A value of 0 was already allowed by the rule parser, but didn't
actually work.

Bug: #6834.
(cherry picked from commit fcca5c7514)
2 years ago
Victor Julien 514f3c5d70 detect/iprep: allow 0 as a reputation value
Rules would allow checking against value 0, but internally the value
was used to indicate "no value". To address this, the internals now
return negative values for not found. This way value 0 can be fully
supported.

Bug: #6834.
(cherry picked from commit 64dc217f9f)
2 years ago
Victor Julien 2f42d89474 detect/iprep: minor code cleanups
(cherry picked from commit 673d27c861)
2 years ago
Philippe Antoine 7d26045cd9 ci: fix macos build
use brew instead of pip
limit the number of jobs for make
set a prefix where we can install
use brew flags for library finding

(cherry picked from commit 47a1502dbb)
2 years ago
Shivani Bhardwaj ed8258cdb3 defrag: apply clang formatting 2 years ago
Jason Ish f30377961d clang-format.sh: prefer clang-format-14
Add clang-format-14 as the preferred version, this is the default on
Ubuntu 22.04.

(cherry picked from commit 5ebae1e8ed)
2 years ago
Jason Ish 358062b1a2 github-ci/formatting: update to Ubuntu 22.04
Update the formatting CI job to Ubuntu 22.04 to get a newer version of
clang-format, in this case clang-format-14.

(cherry picked from commit 93071501b5)
2 years ago
Shivani Bhardwaj f806fbeb55 tls/random: fix incorrect direction handling
The connp objects were incorrectly set per direction leading to
incorrect matches on respective directions.

Bug 6989

(cherry picked from commit 14e2c579f6)
2 years ago
Shivani Bhardwaj 27fbb70a14 version: start development towards 7.0.6 2 years ago
Shivani Bhardwaj c4cf7b09f0 release: 7.0.5; update changelog 2 years ago
Philippe Antoine bcc65a7ffa detect/parse: set limits for pcre2
Ticket: 6889

To avoid regexp dos with too much backtracking.
This is already done on pcre keyword, and pcrexform transform.
We use the same default limits for rules parsing.

(cherry picked from commit 316cc528f7)
2 years ago
Philippe Antoine c0af92295e http2: do not log duplicate headers
Ticket: 6900

And thus avoid DOS by logging a request using a compressed
header block repeated many times and having a long value...

(cherry picked from commit 03442c9071)
2 years ago
Philippe Antoine e68ec4b227 http2: use a reference counter for headers
Ticket: 6892

As HTTP hpack header compression allows one single byte to
express a previously seen arbitrary-size header block (name+value)
we should avoid to copy the vectors data, but just point
to the same data, while reamining memory safe, even in the case
of later headers eviction from the dybnamic table.

Rust std solution is Rc, and the use of clone, so long as the
data is accessed by only one thread.

(cherry picked from commit 390f09692e)
2 years ago
Philippe Antoine 311002baf2 util/base64: fix buffer overflow
Ticket: 6902

In case the caller of DecodeBase64 does not supply a big enough
output buffer.

(cherry picked from commit fd47e67dc6)
2 years ago
Jason Ish e6267758ed defrag: fix check for complete packet
The list of fragments may still contain overlaps, so adding up the
fragment lengths is flawed. Instead track the largest size of
contiguous data that can be re-assembled.

Bug: #6675
(cherry picked from commit d226d0a3fc)
2 years ago
Jason Ish 1e110d0a71 defrag: fix subsequent overlap of start of original (bsd)
Fix the BSD policy case where a subsequent fragment starts before an
original fragment and overlaps the beginning of the original
fragment. In this case the overlapping data from the new fragment is
preferred.

Suricata was preferring the data from the original fragment, but it
should only do that when the original fragment has an offset <= to the
new fragment.

- Adds test for this case

Bug: #6669
(cherry picked from commit f1709ea551)
2 years ago
Jason Ish 2f39ba75f1 defrag: check next fragment for overlap before stopping re-assembly
Instead of breaking the loop when the current fragment does not have
any more fragments, set a flag and continue to the next fragment as
the next fragment may have data that occurs before this fragment, but
overlaps it.

Then break if the next fragment does not overlap the previous.

Bug: #6668
(cherry picked from commit d0fd078250)
2 years ago
Jason Ish 6578f74a95 defrag: use uint8_t in unit tests
(cherry picked from commit bdd17de73d)
2 years ago
Jason Ish 7137d5e7ab defrag: consistent unit test naming
Use a more consistent naming scheme between ipv4 and ipv6.

(cherry picked from commit 2f00b5870a)
2 years ago
Jason Ish 54ffe12f06 defrag: make tests more readable
Make tests more readable for comparing to the paper "Target-Based
Fragmentation Reassembly".

(cherry picked from commit 6339deadce)
2 years ago
Jason Ish bcc7343c0c defrag: minor cleanups
- typo in comment
- remove debug function that is not used and no longer valid

(cherry picked from commit 276d3d6541)
2 years ago
Victor Julien 04b8a7abbe ips: check for livedev.use-for-tracking
For the capture methods that support livedev and IPS,
livedev.use-for-tracking is not supported.

This setting causes major flow tracking issues, as both sides of
a flow would be tracked in different flows.

This patch disables the livedev.use-for-tracking setting if it
is set to true. A warning will be issued.

Ticket: #6726.
(cherry picked from commit 08841f27ff)
2 years ago
Victor Julien b8739ea13f capture: improve IDS + IPS check
Improve it for af-packet, dpdk, netmap. Check would not consider
an interface IDS if the `default` section contained a copy-mode
field.

(cherry picked from commit 58bff9b855)
2 years ago
Victor Julien 8b1dcbd5e3 streaming/buffer: improve integer handling safety
Unsafe handling of buffer offset and to be inserted data's length
could lead to a integer overflow. This in turn would skip growing
the target buffer, which then would be memcpy'd into, leading to
an out of bounds write.

This issue shouldn't be reachable through any of the consumers of
the API, but to be sure some debug validation checks have been
added.

Bug: #6903.
(cherry picked from commit cf6278f95a)
2 years ago
Philippe Antoine 93ac371fe8 detect/http-server-body: avoid FP on toserver direction
Ticket: 6948

http.response_body keyword did not enforce a direction, and thus
could match on files sent with POST requests

(cherry picked from commit e6895b835a)
2 years ago
Lukas Sismis 5c180d81bf dpdk: support 52 byte long key on ice (E810) cards
Ticket: 6872

(cherry picked from commit 10590e6d94)
2 years ago
Lukas Sismis 438f855e2d dpdk: enlarge key length to 52 bytes
(cherry picked from commit d4085fceb6)
2 years ago
Jeff Lucovsky f3a7b0ff28 detect/ipopts: Handle multiple ip options
Issue: 6864

Multiple IP options were not handled properly as the value being OR'd
into the packet's ip option variable were enum values instead of bit
values.

(cherry picked from commit d7026b7b11)
2 years ago
Jeff Lucovsky 2100c346d1 detect/ipopts: Remove unneeded PCRE logic
Issue: 6864

Reduce complexity by eliminating the PCRE logic and adding a unittest to
validate null/empty string handling

(cherry picked from commit ee942391f7)
2 years ago