Commit Graph

70 Commits (2fb89f1b79562ca788693c72d63ca0efd39e0ac5)

Author SHA1 Message Date
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
2 years ago
Philippe Antoine 63caa0b40a detect: respect directionality for filestore
Ticket: 6617

So that rules with keyword like `filestore:to_server,flow`
only store the files to server and not the ones to client...

Directionality only worked with the default scope, ie the
current file, and not the scope tx or scope flow.
For non-default scope, tx or flow, both directions were stored
whatever the directionality specified.

For these non-default scopes, this commit keeps a default
of both directions, but use only one direction if specified.

Need to split flag FLOWFILE_STORE per direction, so that Suricata
can retain this (optional) directional info from the filestore
keyword.

Fixes: 79499e4769 ("app-layer: move files into transactions")
3 years ago
Philippe Antoine db99c45d23 detect: errors on 65k filestore signatures
Errors when a detection engine gets 65k filestore signatures to
avoid the hard limit to have 65k filestore per signature group
head

Ticket: #6393
3 years ago
Philippe Antoine 5954a91457 detect/filestore: fix memory leak on sig parsing
Ticket: 6574

Introduced by commit c272a646c5
3 years ago
Jeff Lucovsky 193e0ea1a9 memory/alloc: Use SCCalloc instead of malloc/memset 3 years ago
Philippe Antoine c272a646c5 detect: SigMatchAppendSMToList can fail
Ticket: #6104

And failures should be handled to say that the rule failed to load

Reverts the fix by 299ee6ed55
that was simple, but not complete (memory leak),
to have this bigger API change which simplifies code.
3 years ago
Jeff Lucovsky 3286c3b912 detect/pcre: Use local match variables
pcre2 is not thread-safe wrt match objects so use locally scoped
objects.

Issue: 4797
3 years ago
Victor Julien 9a4231d737 src: various comment spelling fixes
Thanks to Josh Soref.
4 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 4 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 4 years ago
Victor Julien 8f02a3e415 applayer: make sure to use correct ipproto 4 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
4 years ago
Victor Julien e250ef6402 debug: remove empty header 4 years ago
Victor Julien 046287c2b5 detect/filestore: clean up stream flag handling 4 years ago
Philippe Antoine 3de99a214c pcre2: migrate keywords parsing 5 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
6 years ago
Philippe Antoine 93e6401ce0 http: introduces ALPROTO_HTTP_ANY
For any versions of HTTP, both ALPROTO_HTTP and ALPROTO_HTTP2
6 years ago
Philippe Antoine c8dbe24fb6 proto: introduce signature protocol, as extension to flow protocol
AppProtoEquals function allows to check if a flow protocol
matches a signature protocol
6 years ago
Victor Julien 6ab323d323 detect: hide RegisterTests behind ifdef UNITTESTS
Update all callers to more aggressively use UNITTESTS guards as well.
6 years ago
Victor Julien 26bcc97515 detect/keywords: dynamic version part of doc URL 7 years ago
Jeff Lucovsky d3a65fe156 detect: Provide `de_ctx` to free functions
This commit makes sure that the `DetectEngineCtx *` is available
to each detector's "free" function.
7 years ago
Jeff Lucovsky 4b0085b03c detect: Update to take advantage of PCRE refactor
This commit changes the keyword detectors to use the refactored PCRE
modifications from detect-parse.[ch]
7 years ago
Jeff Lucovsky c2071e1c4e detect/filestore: Fix memory leaks from pcre_get_substring
This commit replaces usages of pcre_get_substring with
pcre_copy_substring to avoid leaking memory on error conditions.
7 years ago
Jeff Lucovsky cf840c37b3 detect/filestore: Warn if req'd feature missing 7 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
7 years ago
Victor Julien 0824b04134 filestore: don't assume flow is TCP
Filestore can be used by UDP based protocols as well. NFSv2 is one
that Suricata supports.

Bug #3277.
7 years ago
Giuseppe Longo ed5a439b8e app-layer-parser: flag a tx to use stream depth
This adds a new API that permit to set the stream-depth
file for file-storing when a rule with filestore keyword is matched.
7 years ago
Victor Julien 14896365ef detect: remove Threadvars argument from API calls
Remove it as it's (almost) never used. If it is really needed it can
be accessed through DetectEngineThreadCtx::tv as well.
7 years ago
magenbluten 09a21627d5 filestore: fix dropping of unwanted files (Issue #2853) 8 years ago
Victor Julien 7497c633c4 detect/filestore: use postmatch callback 8 years ago
magenbluten 1378f376a1 filestore v2: print sid in json output 8 years ago
Victor Julien 16100e592d file: use u32 file id everywhere 8 years ago
Victor Julien 9c2c258f2b uricontent: move debug func into unittests
Cleanup header, which lead to the app-layer-htp.h header needing to
be added in a few other places.
8 years ago
Victor Julien 944ab48b20 file: clarify file store id name 10 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
10 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 10 years ago
Victor Julien b68343e372 files: use dynamic list 10 years ago
Victor Julien 8edc954e82 detect: get rid of Signature::sm_lists
Instead use the lists in init_data during setup and the SigMatchData
arrays during runtime.
10 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 10 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 10 years ago
Eric Leblond a2e2f50fb9 documentation: fix list keywords URLs
Update URLs in keyword definition to point to sphinx documentation.
10 years ago
Giuseppe Longo 3f214b506a file-store: add depth setting
When a rules match and fired filestore we may want
to increase the stream reassembly depth for this specific.

This add the 'depth' setting in file-store config,
which permits to specify how much data we want to reassemble
into a stream.
10 years ago
Giuseppe Longo 5b71b5834f filestore: avoid conflict with bypass keyword
If a packet triggers a rule which contains both
bypass and filestore keywords,
it won't be stored since it's not inspected.

To avoid that, when a rule containing filestore keyword
we make sure that also bypass keyword is present.
10 years ago
Victor Julien 9030e89c94 detect: don't set alproto while registering keyword
The field is not used except for some printing, and is wrong for
many keywords.
10 years ago
Victor Julien 621860f5b2 detect file: enforce protocol in single place
Instead of trying to enforce the app layer protocol in each file
function, enforce it in the generic validation function.
10 years ago
Victor Julien 408948815f detect: simplify flow locking
To simplify locking, move all locking out of the individual detect
code. Instead at the start of detection lock the flow, and at the
end of detection unlock it.

The lua code can be called without a lock still (from the output
code paths), so still pass around a lock hint to take care of this.
11 years ago
Victor Julien e67ae0f174 detect keywords: use parse regex util func 11 years ago
Victor Julien 5111aa2ec0 detect-state: handle 'post match' locking
The post match list was called with an unlocked flow until now.
However, recent de_state handling updates changed this. The stateful
detection code can now call the post match functions while keeping
the flow locked. The normal detection code still calls it with an
unlocked flow.

This patch adds a hint to the DetectEngineThreadCtx called
'flow_locked' that is set to true if the caller has already locked
the flow.
12 years ago
Ken Steele 923a77e952 Change Match() function to take const SigMatchCtx*
The Match functions don't need a pointer to the SigMatch object, just the
context pointer contained inside, so pass the Context to the Match function
rather than the SigMatch object. This allows for further optimization.

Change SigMatch->ctx to have type SigMatchCtx* rather than void* for better
type checking. This requires adding type casts when using or assigning it.

The SigMatch contex should not be changed by the Match() funciton, so pass it
as a const SigMatchCtx*.
12 years ago
Victor Julien f7c2c219cd filestore: fix crash if keyword setup fails
SigMatch would be added to list, then the alproto check failed, leading
to freeing of sm. But as it was still in the list, the list now contained
a dangling pointer.
12 years ago