IPOnly Engine does not support parsing and recording IPv6 ranges. So,
passing a range of ipv6 there just results in non-fatal errors in
parsing the IP addresses which means they fail to be added to the radix
tree struct.
Re-classify the ipv6 ranges as like_ip_only so they fall out of the
IPOnly engine and are processed per packet.
Ticket: 8847
Make `Signature::proto` an optional member, meaning that if it is
NULL we can skip the check. This can be done for `alert ip`, as no check
is needed, and for `alert tcp` and `alert udp` as having a rule in a sgh
for those means that the protocol matches.
Some exceptions are rules that require:
- ipv4/ipv6 specific matching
- frames, due to sharing prefilter between tcp and udp
- ip-only rules, due to those not being per sgh
Simplify statements where there's a conditional statement followed by
return true or return false.
Instead of
if (conditional)
return true;
else
return false;
Use:
return conditional
During initialization, the engine reports how many rules were loaded, as
well as which types. Pkt-only or stream-pkt rules would cause a "hole"
in such stats, as they're not counted.
Ticket: 6724
Allows to use numerical values for example
Also fixes some unit tests that were returning 1 after goto error
FlagsTestParse05 especially took this path as
de->ignored_flags != (TH_SYN|TH_RST) was false
we had de->ignored_flags == 0xff ^ (TH_SYN|TH_RST)
And then, we had a match, instead of what the not-run code
was supposing.
New facility to allow a user to specify where to hook a rule
into the engine. This patch adds this for packets, adding two
hooks:
- `all`: to let a rule be evaluated by all rules
- `flow_start`: to have a rule be evaluated only for the first
packet in both directions
Implemented by adding a hook flags field in the packet.
Generic:
<app_proto>:request_started and <app_proto>:response_started
<app_proto>:request_complete and <app_proto>:response_complete
Per protocol, it uses the registered progress (state) values. E.g.
tls:client_hello_done
A rule ruleset could be:
pass tls:client_hello_done any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)
drop tls:client_hello_done any any -> any any (sid:22;)
The pass rule is evaluated when the client hello is parsed, and if it
doesn't match the drop rule will be evaluated.
Registers each generic lists as "<alproto>:<progress state>:generic"
(e.g. "tls:client_hello_done:generic").
Ticket: #7485.
Flowbits can make a rule such as a packet rule be treated as a stateful
rule, without actually changing the rule type.
Add a flag to allow reporting such cases via engine analysis.
Task #7456
This commit removes the `_AL_` usage in detect keywords for improved
readability.
Some of the HTTP rule keywords already had counterparts without using
"_AL_". These rule keywords are the legacy content modifier keywords
that now have sticky buffer equivalents.
For these, "_AL_" was removed and a suffix was added to the #define:
src/detect-engine-register.h:151: DETECT_HTTP_COOKIE_CM
src/detect-engine-register.h:153: DETECT_HTTP_METHOD_CM
src/detect-engine-register.h:161: DETECT_HTTP_HEADER_CM
src/detect-engine-register.h:173: DETECT_HTTP_RAW_HEADER_CM
src/detect-engine-register.h:175: DETECT_HTTP_URI_CM
src/detect-engine-register.h:179: DETECT_HTTP_STAT_MSG_CM
src/detect-engine-register.h:181: DETECT_HTTP_STAT_CODE_CM
src/detect-engine-register.h:185: DETECT_HTTP_HOST_CM
Ticket: 5053
The names are now dynamically registered at runtime.
The AppProto alproto enum identifiers are still static for now.
This is the final step before app-layer plugins.
This was done following the fact that this setting was historically
named incorrectly. The purpose of the setting was always to define the
ports that will be prioritized and have rule groups associated w them on
priority. Rename all occurences of this to correctly reflect the purpose
of the setting.
Instead of a Host and IPPair table thresholding layer, use a dedicated
THash to store both. This allows hashing on host+sid+tracker or
ippair+sid+tracker, to create more unique hash keys.
This allows for fewer hash collisions.
The per rule tracking also uses this, so that the single big lock is no
longer a single point of contention.
Reimplement storage for flow thresholds to reuse as much logic as
possible from the host/ippair/rule thresholds.
Ticket: #426.
If a signature uses a condition that requires a real packet, filter
out pseudo packets as early as possible. To do this, the SignatureMask
logic is used.
This allows for the removal of checks for pseudo packets in individual
keywords `Match` functions, which will be done in a follow up commit.
Update analyzer to output the new flag.
Ticket: #7002.
So far, if a port was found to be single which was earlier a part of the
range, port + 1 was added to the list to honor the range that it was a
part of. But, this is incorrect in case the port is 65535 or if the port
was found to be of range when it was earlier a single port.
Bug 6896
If a single port happens before a range port, the port groups created
were incorrect. Fix it to use smarter range check.
For example, given,
80:80 - SGH1
100:120 - SGH2
Range created should be
80:80 - SGH1
100:120 - SGH2
Bug 6881
If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.
Bug 6843
Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.
Ticket #6795
To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.
Ticket 6792
Bug 6414
Using the unique port points, create a list of small port ranges which
contain the DetectPort objects and the designated SGHs found by finding
the overlaps with the existing ports and copying the SGHs accordingly.
Ticket 6792
Bug 6414
After all the SGHs have been appropriately copied to the designated
ports, create an interval tree out of it for a faster lookup when later
a search for overlaps is made.
Ticket 6792
Bug 6414
In order to create the smallest possible port ranges, it is convenient
to first have a list of unique ports. Then, the work becomes simple. See
below:
Given, a port range P1 = [1, 8]; SGH1
and another, P2 = [3, 94]; SGH2
right now, the code will follow a logic of recursively cutting port
ranges until we create the small ranges. But, with the help of unique
port points, we get, unique_port_points = [1, 3, 8, 94]
So, now, in a later stage, we can create the ranges as
[1, 2], [3, 7], [8, 8], [9, 94] and copy the designated SGHs where they
belong. Note that the intervals are closed which means that the range
is inclusive of both the points.
The final result becomes:
1. [1, 2]; SGH1
2. [3, 7]; SGH1 + SGH2
3. [8, 8]; SGH1 + SGH2
4. [9, 94]; SGH2
There would be 3 unique rule groups made for the case above.
Group 1: [1, 2]
Group 2: [3, 7], [8, 8]
Group 3: [9, 94]
Ticket 6792
Bug 6414
In the commit 4a00ae607, the whitelisting check was updated in a quest
to make use of the conditional better but it made things worse as every
range would be whitelisted as long as it had any of the default
whitelisted port which is very common.
Shivani Bhardwaj
Victor Julien
Philippe Antoine
Jeff Lucovsky
Juliana Fajardini
Jason Ish
Juliana Fajardini