Commit Graph

50 Commits (c41e64d637fc0ce7a95e68c2de08f4d4ea24f49f)

Author SHA1 Message Date
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
5 years ago
Victor Julien 14896365ef detect: remove Threadvars argument from API calls
Remove it as it's (almost) never used. If it is really needed it can
be accessed through DetectEngineThreadCtx::tv as well.
6 years ago
Victor Julien a6a0b0aa4a detect/files: fix file sigs state handling
Make sure all file sig mismatches indicate this in their return
code, not just the ones with filestore enabled. This is needed
to tell the stateful detect engine that it is dealing with a file
sig, so it can make sure these are inspected correctly even if
there are possibly multiple files per tx.
6 years ago
Victor Julien f30c05e684 smb: remove C implementation
Now that Rust is mandatory it is obsolete.

Ticket: #2849
6 years ago
Victor Julien 9a8092249e detect/http-client-body: move tests into tests/ 7 years ago
Victor Julien 76fd666cad detect/http_raw_header: move tests into tests/ 7 years ago
Victor Julien ab027cb481 detect/http_cookie: move tests into tests/ 7 years ago
Victor Julien cb332b4cda detect/http_method: move all tests into tests/ 7 years ago
Victor Julien 0a405e27a0 detect/http_raw_uri: code reorganization
Move registration into http_uri logic, move tests into the other uri
tests. Switch to v2 mpm/inspect APIs.
7 years ago
Victor Julien 10e2731f18 detect/http-uri: move tests into tests/ 7 years ago
Victor Julien a636d96b15 detect/file: cleanups
TX id is enfored in the engine, so the keywords don't need to.

Unify detect file engines.
8 years ago
Victor Julien ae99e08396 file: update loops to account for parallel files 8 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
8 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 9 years ago
Victor Julien 5ee68ca2b2 http_header: remove old files 9 years ago
Victor Julien b68343e372 files: use dynamic list 9 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 9 years ago
Victor Julien 7f7d4296da detect: file list in engine 9 years ago
Victor Julien 2f87c975d4 detect: add SigMatch arg to inspect functions 9 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
9 years ago
Duarte Silva 89eb935f73 file-hashing: added support for SHA-256 file hashing 9 years ago
Duarte Silva a6d928e269 file-hashing: added support for SHA-1 file hashing 9 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
Victor Julien c4a9580fce detect file: improve multi file handling
When multiple files were in a tx, the first one(s) closed/complete
and a new open one as well, a match in the former could lead to not
inspecting the latter.

This patch adds a workaround for this case, by allowing the file
inspection code to return a special code for 'match, but more files
available in tx'.

The stateful detection engine will then not make this match final for
the tx. It relies on the file pruning to kick in to make sure the
already complete files are removed from the tx before the next time
the detection engine is called on the tx.
9 years ago
Victor Julien 8cd4405c21 detect file: cleanups 9 years ago
Ken Steele 923a77e952 Change Match() function to take const SigMatchCtx*
The Match functions don't need a pointer to the SigMatch object, just the
context pointer contained inside, so pass the Context to the Match function
rather than the SigMatch object. This allows for further optimization.

Change SigMatch->ctx to have type SigMatchCtx* rather than void* for better
type checking. This requires adding type casts when using or assigning it.

The SigMatch contex should not be changed by the Match() funciton, so pass it
as a const SigMatchCtx*.
11 years ago
Victor Julien d0357c6169 smtp: add file inspection engine
Fix file inspection engine.

TODO: test
11 years ago
David Abarbanel c2dc686742 SMTP MIME Email Message decoder 11 years ago
Victor Julien 3b98a1ce66 detect: track current tx_id in det_ctx
When using the inspection engines, track the current tx_id in the
thread storage the detect thread uses. As 0 is a valid tx_id, add
a simple bool that indicates if the tx_id field is set.
11 years ago
Victor Julien 7b687da726 profiling: add tracking of missing keywords 12 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
12 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Anoop Saldanha b99f9fe890 New app inspection engine introduced. Moved existing inspecting engines to use it. 13 years ago
Anoop Saldanha 7b4eac3e8d Change all inspect callbacks to accept TV and a tx_id param. 13 years ago
Victor Julien 8f71333e12 file: implement filesize keyword. #489. 13 years ago
Victor Julien 3849588c61 Create separate detect API call (FileMatch) for file detection keywords. #531. 13 years ago
Victor Julien c9e93ec52c filemd5: add support code for md5 handling for signatures. 13 years ago
Victor Julien 9f7588a756 Add filemd5 keyword that loads a list of md5's to match a file's md5 against. 13 years ago
Victor Julien 19a7e7f395 flow: create a flow lock macro API, implement it for mutex and rwlocks. Mutex remains the default. 13 years ago
Victor Julien 2650551192 Rename signature init flags to indicate they are init flags. 14 years ago
Victor Julien 3009429e3c HTTP transaction handling improvement
In some cases AppLayerTransactionGetInspectId can return -1, which is
now handled by all it's callers.

Improve logic of selecting which transactions are inspected by the various
HTTP keywords.
14 years ago
Victor Julien 938e9b3db0 Fix filestore related segv. 14 years ago
Victor Julien 9878eca086 file handling: expand filestore keyword
Filestore keyword by default (... filestore; ... ) marks only the file in the
same direction as the rule match for storing. This makes sense when inspecting
individual files (filemagic, filename, etc) but not so much when looking at
suspicious file requests, where the actual file is in the response.

The filestore keyword now takes 2 optional options:

filestore:<direction>,<scope>;

By default the direction is "same as rule match", and scope is "currently
inspected file".

For direction the following values are possible: "request" and "to_server",
"response" and "to_client", "both".

For scope the following values are possible: "tx" for all files in the current
HTTP/1.1 transation, "ssn" and "flow" for all files in the session/flow.

For the above case, where a suspious request should lead to a response file
download, this would work:

alert http ... content:"/suspicious/"; http_uri; filestore:response; ...
14 years ago
Victor Julien d59ca75e46 file extract: split toserver and toclient tracking
Split toserver and toclient file tracking for the http state.
14 years ago
Victor Julien 04ea70ccf7 file extract: pruning
Add pruning of files in memory so we keep only memory what we really need.
Fix magic logic.
Reset file part of the de_state on receiving another file in the same tx.
14 years ago
Victor Julien b402d97179 File carving -- enable reponse file extraction
- Enable response body tracking
- Enable file extraction for responses
- File store meta file includes magic, close reason.
- Option to force magic lookup for all stored files.
- Fix libmagic calls thead safety.
14 years ago
Victor Julien e1022ee5ae file-extraction: Disconnect file handling from flow and move into the app layer state. 14 years ago
Victor Julien 9b62ec65ab Make sure filemagic works properly regardless of filestore being in use for a flow. 14 years ago
Victor Julien 5945e652d6 Initial implementation of filemagic keyword. 14 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 14 years ago