c9434b3703
detect/frame: fix assert
...
A rule like `frame:http1.response; to_md5;` could have a buffer
bigger than the frame
Ticket: 8763
2 months ago
400328c3c3
pcap-file: prep codebase for pcap_cnt move refactor
...
For an easier review process, this is a two-step change process,
in which pcap_cnt is first accessed by functions-to-be, implemented
as simple macros.
In the follow-up commit, the actual refactor is implemented with the new
function. The old macros are deleted.
Ticket: 7835
9 months ago
439f96dea7
rust: bindgen frames functions
...
Ticket: 7667
12 months ago
ad0f6e388e
src: remove BUG_ON in packet path
...
Transforms them into debug validations
Ticket: 1484
Packet path is here considered FlowWorker and all lines
covered by SV tests are here fixed
1 year ago
3c743529b7
detect/frames: fix -Wshorten-64-to-32 warnings
...
Ticket: #6186
1 year ago
94644ac960
detect: move non-pf rules into special prefilter engines
...
Instead of having a per detection engine list of rule that couldn't be
prefiltered, put those into special "prefilter" engines.
For packet and frame rules this doesn't change much, it just removes
some hard coded logic from the detect engine.
For the packet non-prefilter rules in the "non-prefilter" special prefilter
engine, add additional filtering for the packet variant. It can prefilter on
alproto, dsize and dest port.
The frame non-prefilter rules are added to a single engine, that per
rule checks the alproto and the type.
For app-layer, there is an engine per progress value, per app-layer
protocol and per direction. This hooks app-layer non-prefilter rules
into the app inspect logic at the correct "progress" hook.
e.g. a rule like
dns.query; bsize:1;
Negated MPM rules will also fall into this category:
dns.query; content:!"abc";
Are part of a special "generic list" app engine for dns, at the
same progress hook as `dns.query`.
This all results in a lot fewer checks:
previous:
--------------------------------------------------------------------------
Date: 1/29/2025 -- 10:22:25. Sorted by: number of checks.
--------------------------------------------------------------------------
Num Rule Gid Rev Ticks % Checks Matches Max Ticks Avg Ticks Avg Match Avg No Match
-------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- --------------
1 20 1 0 181919672 11.85 588808 221 60454 308.96 2691.46 308.07
2 50 1 0 223455914 14.56 453104 418 61634 493.17 3902.59 490.02
3 60 1 0 185990683 12.12 453104 418 60950 410.48 1795.40 409.20
4 51 1 0 192436011 12.54 427028 6084 61223 450.64 2749.12 417.42
5 61 1 0 180401533 11.75 427028 6084 61093 422.46 2177.04 397.10
6 70 1 0 153899099 10.03 369836 0 61282 416.13 0.00 416.13
7 71 1 0 123389405 8.04 369836 12833 44921 333.63 2430.23 258.27
8 41 1 0 63889876 4.16 155824 12568 39138 410.01 1981.97 272.10
9 40 1 0 64149724 4.18 155818 210 39792 411.70 4349.57 406.38
10 10 1 0 70848850 4.62 65558 0 39544 1080.70 0.00 1080.70
11 11 1 0 94743878 6.17 65558 32214 60547 1445.19 2616.14 313.92
this commit:
--------------------------------------------------------------------------
Date: 1/29/2025 -- 10:15:46. Sorted by: number of checks.
--------------------------------------------------------------------------
Num Rule Gid Rev Ticks % Checks Matches Max Ticks Avg Ticks Avg Match Avg No Match
-------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- --------------
1 50 1 0 138776766 19.23 95920 418 167584 1446.80 3953.11 1435.83
2 60 1 0 97988084 13.58 95920 418 182817 1021.56 1953.63 1017.48
3 51 1 0 105318318 14.60 69838 6084 65649 1508.04 2873.38 1377.74
4 61 1 0 89571260 12.41 69838 6084 164632 1282.56 2208.41 1194.20
5 11 1 0 91132809 12.63 32779 32214 373569 2780.22 2785.58 2474.45
6 10 1 0 66095303 9.16 32779 0 56704 2016.39 0.00 2016.39
7 70 1 0 48107573 6.67 12928 0 42832 3721.19 0.00 3721.19
8 71 1 0 32308792 4.48 12928 12833 39565 2499.13 2510.05 1025.09
9 41 1 0 25546837 3.54 12886 12470 41479 1982.53 1980.84 2033.05
10 40 1 0 26069992 3.61 12886 210 38495 2023.13 4330.05 1984.91
11 20 1 0 639025 0.09 221 221 14750 2891.52 2891.52 0.00
2 years ago
22ea5ddbb7
detect/transforms: Add engine detect thread ctx to signature
...
Modify the transform function signature to include the detect engine
thread ctx.
2 years ago
7d5c170dd2
gen/bool: Clarify bool checks
...
Remove comparisons with true or false with C bools.
2 years ago
44a8bf463e
detect/rule-header: use bool type
...
Update frame prototype as well, to match already returned true/false values.
3 years ago
b1fa975503
detect/content-inspect: remove const casting
3 years ago
7f42506760
detect: reimplement discontinue matching logic
...
Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.
This patch reimplements this logic by using a new return code instead.
Split content inspection into public and private version, so that
common initialization can be done in a single place.
Update the callsites.
3 years ago
68a2fcaad3
mpm: thread ctx cleanups
...
Remove unused thread ctx' from AC variants
Use single thread store in detection.
Minor cleanups.
3 years ago
da89164ba1
detect: spelling: DetectBufferMpmRegistry
4 years ago
977c5ea719
detect: initialize empty buffers
4 years ago
9e41075d5d
detect/frames: improve IPS and GAP handling
...
Inspect individual chunks in lossy traffic.
Don't use the frame idx as the inspection buffer idx. Engines are running
per frame, so multi inspection can be used for stream chunks instead.
Ticket: #4977 .
4 years ago
a95934b5ee
detect/frames: reduce scope of private function
4 years ago
aa376a3b21
detect/frame: improve frame detection
...
Add a per frame progress tracker.
4 years ago
b43dc5a64a
app-layer/frames: use absolute frame offset
...
Frame offset was already a 64 bit integer, so simplify things by
making it an absolute offset from the start of the stream.
4 years ago
f773b714e9
detect/frames: minor cleanup in buffer handling
...
Don't lookup a buffer twice, even if the lookup should be fast.
4 years ago
040404b093
detect/profiling: track bytes scanned by prefilter engines
4 years ago
d31beba8d4
detect/frames: fix too strict debug check
...
Frame::len is -1 if it is still unknown. Handle that in the debug
check.
4 years ago
af90478167
detect/frames: reduce severity of validation check
5 years ago
3a7d09edfc
detect/frame: get data using stream callback
...
Inspect only data that has already been consumed by the
app-layer parser. This allows for simpler progress tracking.
5 years ago
61df4120da
detect/frame: improve assert accuracy
...
Handle frames of unknown size correctly.
Bug: #5226 .
5 years ago
c96d22e8a1
frames: support UDP frames
...
UDP frames point to the UDP packet payloads.
The frames are removed after each packet.
Ticket: #4983 .
5 years ago
f6f124f283
detect/engine: support frames
...
Implement the low level detect engine support for inspecting frames,
including MPM, transforms and inspect API's.
5 years ago
Philippe Antoine
Lukas Sismis
Victor Julien
Jeff Lucovsky