Commit Graph

72 Commits (b31ffde6f4f9069eaa7f366d3d9a3ab68e97209e)

Author SHA1 Message Date
Victor Julien b31ffde6f4 output: remove error codes from output 3 years ago
Victor Julien e042cd785e error: use SC_ENOMEM for alloc errors 3 years ago
Alice Akaki 8614bff017 detect-filemagic: convert unittests to FAIL/PASS APIs
Task: #4034
3 years ago
Victor Julien 41265a859e detect/files: optimize file.data by skipping non-file txs
As well as 'file' txs not in our direction.

Implement the same logic for file.name and file.magic prefilter engines.
3 years ago
Victor Julien 3263202094 detect/tx: add AppLayerTxData to PrefilterTx
In preparation of some file inspection optimizations, for which we need the
tx data.

Update all users.
3 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
3 years ago
Victor Julien 040404b093 detect/profiling: track bytes scanned by prefilter engines 3 years ago
Victor Julien e250ef6402 debug: remove empty header 3 years ago
Victor Julien af145ad125 detect/file: reduce scope of keyword data structures 3 years ago
Victor Julien 73eb7744d8 detect/file: update copyright years 3 years ago
Philippe Antoine 02f2602dde src: rework includes as per cppclean 3 years ago
Philippe Antoine b8524e70d9 detect: change InspectEngineFuncPtr2 to return uint8_t 3 years ago
Victor Julien 4feb0529a4 detect/file: minor code cleanup
Reduce scope where possible. Suggested by cppcheck.
3 years ago
Victor Julien 47629b7aeb detect/filemagic: don't pass unused pointer 3 years ago
Victor Julien d8d1fbe443 detect/files: fix buffer tracking with multiple files 4 years ago
Victor Julien 3dc50322db detect: fix multi inspect buffer issue; clean up
Fix multi inspect buffer API causing cleanup logic in the single
inspect buffer paths. This could lead to a buffer overrun in the
"to clear" logic.

Multi buffers now use InspectionBufferSetupMulti instead of
InspectionBuffer. This is enforced by a check in debug validation.

Simplify the multi inspect buffer setup code and update the callers.
4 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
4 years ago
Victor Julien 13cebb1857 detect: fix heap overflow issue with buffer setup
In some cases, the InspectionBufferGet function would be followed by
a failure to set the buffer up, for example due to a HTTP body limit
not yet being reached. Yet each call to InspectionBufferGet would lead
to the matching list_id to be added to the
DetectEngineThreadCtx::inspect.to_clear_queue. This array is sized to
add each list only once, but in this case the same id could be added
multiple times, potentially overflowing the array.
4 years ago
Angelo Mirabella c72069c3b2 detect/magic: fix crash on rule reloading
This changseset fixes a bug causing a segmentation fault.
When rules are reloaded and a rule using libmagic matches, suricata
crashes due to an improper reinitialization of the thread contexts.

Bug: #3726
5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Victor Julien 6ab323d323 detect: hide RegisterTests behind ifdef UNITTESTS
Update all callers to more aggressively use UNITTESTS guards as well.
5 years ago
Eric Leblond ae5650d443 magic: get rid of global lock
Global magic context was involving a lock that appear to be really
costly for some traffic.
5 years ago
Victor Julien 26bcc97515 detect/keywords: dynamic version part of doc URL 5 years ago
Jeff Lucovsky d3a65fe156 detect: Provide `de_ctx` to free functions
This commit makes sure that the `DetectEngineCtx *` is available
to each detector's "free" function.
5 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
6 years ago
Victor Julien 4dff903b35 detect: introduce pkt mpm engines
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.

Share part of the registration code with the AppLayer.

Implement for the tcp.hdr and udp.hdr keywords.
6 years ago
Victor Julien 14896365ef detect: remove Threadvars argument from API calls
Remove it as it's (almost) never used. If it is really needed it can
be accessed through DetectEngineThreadCtx::tv as well.
6 years ago
Victor Julien 32fb7d773a detect/content-inspect: turn void arg into Packet
Replace the 'void *data' argument by a 'Packet *p' as this was
the only user left of the data pointer.
6 years ago
Victor Julien 84881bf1b8 detect/file.magic: add sticky buffer
Add sticky buffer to inspect file magic. Includes mpm support.
6 years ago
Eric Leblond 63f2032ed6 doc: fix way to build URL 6 years ago
Victor Julien 164252e381 detect/file: fix minor scan-build warnings 7 years ago
Andreas Herz 2e8678a5ff docs: replace redmine links and enforce https on oisf urls 8 years ago
Victor Julien a636d96b15 detect/file: cleanups
TX id is enfored in the engine, so the keywords don't need to.

Unify detect file engines.
8 years ago
Victor Julien 9b1f74409b magic: fix compile warnings 8 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
8 years ago
Victor Julien 342059835f detect-parse: improve common parser
In preparation of turning input to keyword parsers to const add
options to the common rule parser to enforce and strip double
quotes and parse negation support.

At registration, the keyword can register 3 extra flags:

    SIGMATCH_QUOTES_MANDATORY: value to keyword must be quoted

    SIGMATCH_QUOTES_OPTIONAL: value to keyword may be quoted

    SIGMATCH_HANDLE_NEGATION: leading ! is parsed

In all cases leading spaces are removed. If the 'quote' flags are
set, the quotes are removed from the input as well.
8 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 9 years ago
Victor Julien b68343e372 files: use dynamic list 9 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 9 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 9 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
9 years ago
Victor Julien 810e43f373 magic: make optional
Make libmagic optional. If installed it will be enabled by default in
configure. Use --disable-libmagic to disable.
9 years ago
Eric Leblond a2e2f50fb9 documentation: fix list keywords URLs
Update URLs in keyword definition to point to sphinx documentation.
9 years ago
Victor Julien 9030e89c94 detect: don't set alproto while registering keyword
The field is not used except for some printing, and is wrong for
many keywords.
9 years ago
Victor Julien c957c62824 detect file: enable HTTP inspection from validate func 9 years ago
Victor Julien 621860f5b2 detect file: enforce protocol in single place
Instead of trying to enforce the app layer protocol in each file
function, enforce it in the generic validation function.
9 years ago
Andreas Herz 36e4126227 detect-filemagic: fix heap-use-after-free
This fixes the heap-use-after-free issue with sm being freed without
being removed from the signature (s) list. Move the protocol check for
rules with filemagic before the alloc and make the error log more
precise.
9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
maxtors 9d3fd82849 Removed duplicate include statements. 9 years ago
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
9 years ago