Commit Graph

62 Commits (72829dbaea1d17370c486fa8c7504fb399efb534)

Author SHA1 Message Date
Victor Julien 67e8779659 detect/file: reset smd iterator for each file in multi-file txs
DetectFileInspect() walks the per-signature SigMatchData array with
a pointer (smd) that is incremented inside the inner keyword-matching
loop. Because smd is the function parameter, this mutation persists
across iterations of the outer for file loop and there is no reset
of the cursor between files.

With a single file this works correctly. In any transaction that carries
two or more File objects every subsequent file is evaluated starting at
the already-advanced smd instead of the beginning of the keyword chain,
leading to skipping of keywords.

This patch addresses the issue by using a local iterator instead of
using the function parameter directly, so every file sees the full keyword
list.

Ticket: #8815.
2 months ago
Philippe Antoine ef884b009a detect/dcerpc: move dcepayload unit tests to SV
Ticket: 8391
6 months ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
2 years ago
Jeff Lucovsky 59fea84956 detect/file: Remove centralized proto definition
Issue: 4145

Remove centralized protocol definitions for file handling in favor of
consolidated file access handling.
3 years ago
Philippe Antoine e75956717d detect/files: centralize definition of protocols
Protocols supporting files are only defined in one place, which
gets used by all keywords, which can handle some exceptions
(like HTTP2 not having file names)
3 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 4 years ago
Shivani Bhardwaj 82dba07579 src: remove unneeded header and refs 4 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
4 years ago
Philippe Antoine b8524e70d9 detect: change InspectEngineFuncPtr2 to return uint8_t 4 years ago
Shivani Bhardwaj bac69af7e4 dcerpc: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
6 years ago
Victor Julien e0e4454db7 detect/files: inspect api v2 6 years ago
Victor Julien 48094b05c8 detect/file: minor cleanup 6 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
7 years ago
Victor Julien 14896365ef detect: remove Threadvars argument from API calls
Remove it as it's (almost) never used. If it is really needed it can
be accessed through DetectEngineThreadCtx::tv as well.
7 years ago
Victor Julien a6a0b0aa4a detect/files: fix file sigs state handling
Make sure all file sig mismatches indicate this in their return
code, not just the ones with filestore enabled. This is needed
to tell the stateful detect engine that it is dealing with a file
sig, so it can make sure these are inspected correctly even if
there are possibly multiple files per tx.
8 years ago
Victor Julien f30c05e684 smb: remove C implementation
Now that Rust is mandatory it is obsolete.

Ticket: #2849
8 years ago
Victor Julien 9a8092249e detect/http-client-body: move tests into tests/ 8 years ago
Victor Julien 76fd666cad detect/http_raw_header: move tests into tests/ 8 years ago
Victor Julien ab027cb481 detect/http_cookie: move tests into tests/ 8 years ago
Victor Julien cb332b4cda detect/http_method: move all tests into tests/ 8 years ago
Victor Julien 0a405e27a0 detect/http_raw_uri: code reorganization
Move registration into http_uri logic, move tests into the other uri
tests. Switch to v2 mpm/inspect APIs.
8 years ago
Victor Julien 10e2731f18 detect/http-uri: move tests into tests/ 8 years ago
Victor Julien a636d96b15 detect/file: cleanups
TX id is enfored in the engine, so the keywords don't need to.

Unify detect file engines.
10 years ago
Victor Julien ae99e08396 file: update loops to account for parallel files 10 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
10 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 10 years ago
Victor Julien 5ee68ca2b2 http_header: remove old files 10 years ago
Victor Julien b68343e372 files: use dynamic list 10 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 10 years ago
Victor Julien 7f7d4296da detect: file list in engine 10 years ago
Victor Julien 2f87c975d4 detect: add SigMatch arg to inspect functions 10 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
10 years ago
Duarte Silva 89eb935f73 file-hashing: added support for SHA-256 file hashing 10 years ago
Duarte Silva a6d928e269 file-hashing: added support for SHA-1 file hashing 10 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
11 years ago
Victor Julien c4a9580fce detect file: improve multi file handling
When multiple files were in a tx, the first one(s) closed/complete
and a new open one as well, a match in the former could lead to not
inspecting the latter.

This patch adds a workaround for this case, by allowing the file
inspection code to return a special code for 'match, but more files
available in tx'.

The stateful detection engine will then not make this match final for
the tx. It relies on the file pruning to kick in to make sure the
already complete files are removed from the tx before the next time
the detection engine is called on the tx.
11 years ago
Victor Julien 8cd4405c21 detect file: cleanups 11 years ago
Ken Steele 923a77e952 Change Match() function to take const SigMatchCtx*
The Match functions don't need a pointer to the SigMatch object, just the
context pointer contained inside, so pass the Context to the Match function
rather than the SigMatch object. This allows for further optimization.

Change SigMatch->ctx to have type SigMatchCtx* rather than void* for better
type checking. This requires adding type casts when using or assigning it.

The SigMatch contex should not be changed by the Match() funciton, so pass it
as a const SigMatchCtx*.
12 years ago
Victor Julien d0357c6169 smtp: add file inspection engine
Fix file inspection engine.

TODO: test
12 years ago
David Abarbanel c2dc686742 SMTP MIME Email Message decoder 12 years ago
Victor Julien 3b98a1ce66 detect: track current tx_id in det_ctx
When using the inspection engines, track the current tx_id in the
thread storage the detect thread uses. As 0 is a valid tx_id, add
a simple bool that indicates if the tx_id field is set.
12 years ago
Victor Julien 7b687da726 profiling: add tracking of missing keywords 13 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
14 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
14 years ago
Anoop Saldanha b99f9fe890 New app inspection engine introduced. Moved existing inspecting engines to use it. 14 years ago
Anoop Saldanha 7b4eac3e8d Change all inspect callbacks to accept TV and a tx_id param. 14 years ago
Victor Julien 8f71333e12 file: implement filesize keyword. #489. 14 years ago
Victor Julien 3849588c61 Create separate detect API call (FileMatch) for file detection keywords. #531. 14 years ago
Victor Julien c9e93ec52c filemd5: add support code for md5 handling for signatures. 15 years ago
Victor Julien 9f7588a756 Add filemd5 keyword that loads a list of md5's to match a file's md5 against. 15 years ago