Commit Graph

327 Commits (032f31b7d38f6be5561f03d10d05a54b2271407f)

Author SHA1 Message Date
Victor Julien 032f31b7d3 htp: fix test after libhtp changes 5 years ago
Victor Julien 07ed0dadae fuzz: suppress too noisy htp errors check 5 years ago
Shivani Bhardwaj e7c0f0ad91 src: remove multiple uses of atoi
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* functions.

Partially closes redmine ticket #3053.
5 years ago
Philippe Antoine 053c728871 http: adds debug check against too many warnings 5 years ago
Victor Julien c660757153 atomics: remove useless SC_ATOMIC_DESTROY 5 years ago
Philippe Antoine 91b2930891 fuzz: build compatibility with oss-fuzz flags
ie C define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
5 years ago
Victor Julien 3ae1854d2f htp: fix globals use for flags 6 years ago
Victor Julien 040aff5197 htp: close request only from request side
This allows the response side to keep going for just
a bit longer.
6 years ago
Victor Julien 5ef05ffad1 http/multipart: small cleanup 6 years ago
Victor Julien aae00df4df http/multipart: use wider type for boundary lengths
Use uint32_t for a local type instead of uint8_t to avoid casts.

Length should always stay under this regardless.
6 years ago
Victor Julien 4d0db9cb4a http/multipart: optimize form end search
If we already know that the boundary exists, we can start looking
there. Otherwise, we can skip trying as the boundary is a subset
of the form end marker.
6 years ago
Victor Julien 54d93e1eb9 http/multipart: process incomplete file data
Start processing multipart data as soon as it is available to
allow inspection sooner.
6 years ago
Victor Julien cd66c37711 http/multipart: use proper progress value to test eof 6 years ago
Victor Julien 94982ae690 http: split request/response tx id handling
When HTTP pipelining was in use, the transaction id used for events
and files could be off. If the request side was several requests ahead
of the responses, it would use the HtpState::transaction_cnt for events
and files, even though that is only incremented on complete requests.

Split request and response tx id tracking. The response is still handled
by the HtpState::transaction_cnt, but the request side is now handled by
its own logic.
6 years ago
Philippe Antoine af4f816204 http: sets compression bomb limit 6 years ago
Philippe Antoine c09ad01836 http: disable lzma decompression from configuration 6 years ago
Philippe Antoine 9cbf9ef7a4 HTTP new parser warning for Ambiguous C-L 6 years ago
Victor Julien c68fbfcfe6 htp: simplify depth check 6 years ago
Giuseppe Longo de904db830 app-layer-htp: use stream depth with filestore
This permits to use stream-depth value set for file-store.

Currently if a file is being stored and hits a limit,
such as request or response body, it will be truncated
although file-store.stream-depth is enabled but the file should be
closed and not truncated.

Two unit tests have been added to verify that:
- a file is stored correctly
- chunk's length computation doesn’t cause an underflow
6 years ago
Jason Ish 61a6eaf330 htp/lzma: set limit from configuration
Also use a default defined in Suricata, not libhtp.
6 years ago
Victor Julien c9c23d5cda htp: set lzma memlimit from config 6 years ago
Philippe Antoine 9665ab0409 http: wait for response line for filename
See http evader case 481
6 years ago
Victor Julien 579cc9f02b const: constify decoder, app-layer, detect funcs 6 years ago
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
6 years ago
Jeff Lucovsky f7b934f83f app-layer/logging: protocol parser updates 6 years ago
Philippe Antoine b5f3e03209 New app layer event for invalid http request line
Handles logs from libhtp even if case of error
6 years ago
Philippe Antoine 8a339e73d3 http: adds an event for double encoded uri 6 years ago
Philippe Antoine e5879650bb http: fixes double-encode configuration keywords 6 years ago
Philippe Antoine 3e12066819 http: adds events for each libhtp log
Fixes #997
6 years ago
Philippe Antoine b6b7778e2d http: adds event for header repetition 6 years ago
Philippe Antoine a1c6e091ac http: new event for auth unrecognized
activates libhtp auth parsing
Fixes #984
6 years ago
Philippe Antoine bef190f767 http: logs content range
Fixes #2485
6 years ago
Victor Julien d34e41068f app-layer: fix tx tracking updates in tx cleanup
Fix min_id not getting updated in all cases.

Reported by: Ilya Bakhtin
7 years ago
Victor Julien 13ea30ef23 spelling: fixing minor spelling mistakes 7 years ago
Victor Julien e02b74dee7 http: implement min size stream logic
Update HTTP parser to set the min inspect depth per transaction. This
allows for signatures to have their fast_pattern in the HTTP body,
while still being able to inspect the raw stream reliably with it.

The inspect depth is set per transaction as it:
- depends on the per personality config for min inspect size
- is set to the size of the actual body if it is smaller

After the initial inspection is done, it is set to 0 which disables
the feature for the rest of the transaction.

This removes the rescanning flush logic in commit
7e004f52c6 and provides an alternative
fix for bug #2522. The old approach caused too much rescanning of
HTTP body data leading to a performance degradation.

Bug #2522
7 years ago
Victor Julien 7186ce7b99 stream: introduce min inspect depth logic
Some rules need to inspect both raw stream data and higher level
buffers together. When this higher level buffer is a streaming
buffer itself, the risk of mismatch exists.

This patch allows an app-layer parser to set a 'min inspect depth'.
The value is used by the stream engine to keep at least this
depth worth of data, so that the detection engine can request
all of it for inspection.

For rules that have the SIG_FLAG_FLUSH flag set, data is inspected
not from offset raw_progress, but from raw_progress minus
min_inspect_depth.

At this time this is only used for sigs that have their fast_pattern
in a HTTP body and have raw stream match as well.
7 years ago
Giuseppe Longo fc38013975 app-layer-htp: close file with TRUNCATE state
When a file in TOSERVER direction is being stored and
libhtp or stream depth limit is reached,
it will be closed by HTPCallbackRequest without setting
any flags so the file state will be set to CLOSED
instead of TRUNCATED.
7 years ago
Victor Julien 83e7ddf41f http: fix setting event on the last tx 7 years ago
Victor Julien d0cded2523 http: set events for too many layers of compression
libhtp would already issue warnings, but these were not mapped
to events yet.
7 years ago
Victor Julien 7bc3c3ac6e app-layer: pass STREAM_* flags to parser
Pass the STREAM_* flags to the app-layer parser functions so that
the parser can know more about how it is called.
7 years ago
Victor Julien 26b61bad90 htp: cleanup and fix test 7 years ago
Victor Julien 08af5ddd88 http: add tests for header folding
To test for https://github.com/OISF/libhtp/issues/159
7 years ago
Victor Julien 953dceece2 http: clean up unittest 7 years ago
Jason Ish c411519605 app-layer: remove has events callback - not used 8 years ago
Victor Julien 7548944b49 app-layer: remove unused HasTxDetectState call
Also remove the now useless 'state' argument from the SetTxDetectState
calls. For those app-layer parsers that use a state == tx approach,
the state pointer is passed as tx.

Update app-layer parsers to remove the unused call and update the
modified call.
8 years ago
Giuseppe Longo d0f92e2a56 app-layer-htp: add swf decompression settings
This adds some settings needed to do swf file decompression
under libhtp section in suricata.yaml
8 years ago
Victor Julien 6f339abdf0 htp: minor debug addition 8 years ago
Victor Julien c63b1ce2c6 htp: remove used body operation field 8 years ago