Victor Julien
129b6a65ca
http: add test for HTTP_DECODER_EVENT_UNKNOWN_ERROR event as a result of a too long request
12 years ago
Anoop Saldanha
9e4eec200f
Update htp event handler to both warning and error events regardless of any conditions.
12 years ago
Anoop Saldanha
5e2d9dbdc3
Add and use EventGetInfo for getting info on an event.
...
Also update existing parsers and app-layer-event Setup to use this.
12 years ago
Anoop Saldanha
6cb0014287
Move app event module registration as a part of app layer proto table.
12 years ago
Anoop Saldanha
0d7159b525
App layer protocol detection updated and improved. We now use
...
confirmation from both directions and set events if there's a mismatch
between the 2 directions.
FPs from corrupt flows have disappeared with this.
12 years ago
Anoop Saldanha
ddde572fba
Introduce new options into the conf file to enable/disable -
...
1. Proto detection
2. Parsers
For app layer protocols.
libhtp has now been moved to the section under app-layer.protocols.http,
but we still provide backward compatibility with older conf files.
12 years ago
Victor Julien
7f8d256e7c
Fix tests that didn't expect radix to be freed
12 years ago
Victor Julien
6f2cb141cf
Http: improve tx data cleanup
12 years ago
Victor Julien
67c12c61d3
Http: fix memory leaks when cleaning up our per-tx storage
12 years ago
Anoop Saldanha
94e2527606
Introduce a saner way to validate the completion of request and
...
response bodies.
Also don't change app state for http from inside inspection.
12 years ago
Anoop Saldanha
dcdcbd9721
Fix creating a backup of htp config. This is used by unittests that
...
changed htp config.
12 years ago
Anoop Saldanha
56143131da
Fix unittests that use chunked encoding.
12 years ago
Ken Steele
5532af4621
Create SCMUTEX_INITIALIZER to abstract out PTHREAD_MUTEX_INITIALIZER
...
This allows replacing pthread mutexes with other types of mutex.
12 years ago
Eric Leblond
cd3e32ce19
unittests: some functions needs a flow lock.
...
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
12 years ago
Victor Julien
9faa4b740d
Add --unittests-coverage option to list how many code modules have tests
12 years ago
Anoop Saldanha
48cf0585fb
Suricata upgrade to libhtp 0.5.x.
...
Remove the support for now unsupported personalities from libhtp -
TOMCAT_6_0, APACHE and APACHE_2_2. We instead use the APACHE_2
personality.
12 years ago
Victor Julien
080c15b3fc
Enable libhtp 0.3.0 compilation and crash free UT run. Still see 5 failed tests.
12 years ago
Victor Julien
0fd9b0c4fa
HTP: free TX from transaction free API call
12 years ago
Victor Julien
f59f90331d
Applayer: remove obsolete StateUpdateTransactionId
...
Also, update StateTransactionFree to take an u64 tx id, so it's
consistant with the rest of the engine.
To reflect these changes, AppLayerRegisterTransactionIdFuncs has
been renamed to AppLayerRegisterTxFreeFunc.
HTP, DNS, SMB, DCERPC parsers updated.
12 years ago
Anoop Saldanha
058e9278c5
Fix wrong casting of htp pointer. Fixed it back to (HTPState *) inside
...
htp utility functions.
12 years ago
Anoop Saldanha
ab4b15c2e7
fix for #788 .
...
Now depth is kept in mind when we inspect chunks in client/server body.
This takes care of FPs originating from inspecting subsequent chunks that
match with depth, but shouldn't.
12 years ago
Anoop Saldanha
d4d18e3136
Transaction engine redesigned.
...
Improved accuracy, improved performance. Performance improvement
noticeable with http heavy traffic and ruleset.
A lot of other cosmetic changes carried out as well. Wrappers introduced
for a lot of app layer functions.
Failing dce unittests disabled. Will be reintroduced in the updated dce
engine.
Cross transaction matching taken care of. FPs emanating from these
matches have now disappeared. Double inspection of transactions taken
care of as well.
12 years ago
Eric Leblond
6d225378e4
Workaround function missing in libhtp include
...
As reported in bug #688 , htp_config_set_path_decode_u_encoding
function is not included in libhtp header before 0.3.0. Result
is that suricata compilation fail with an external htp library.
The following patch detect the issue and adds the missing
declaration.
13 years ago
Victor Julien
cc51eec59d
Use new libhtp query string normalization. Bug #739 .
13 years ago
Victor Julien
9f519e95a2
http: add event for libhtp detection of request port not matching tcp port.
13 years ago
Last G
8ae11f73b2
Added parentheses to fix Eclipse static code analysis
...
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Victor Julien
5cd46433d3
http: now that htp_state has a cfg reference, use it for body limits
13 years ago
Victor Julien
2763a61213
http: allow configuration of request and response body inspection limits. Issue #560 .
13 years ago
Victor Julien
aa4ae98d37
http: fix multipart parsing leading to missing chunks of files in file extraction.
13 years ago
Anoop Saldanha
028c6c1782
Make available custom features of libhtp.
...
The power of libhtp customisation now available to users.
Options available -
path-backslash-separators: yes
path-compress-separators: yes
path-control-char-handling: none
path-convert-utf8: yes
path-decode-separators: yes
path-decode-u-encoding: yes
path-invalid-encoding-handling: preserve_percent
path-invalid-utf8-handling: none
path-nul-encoded-handling: none
path-nul-raw-handling: none
set-path-replacement-char: ?
set-path-unicode-mapping: bestfit
You can use this for your libhtp customisation. Options explained in our
wiki.
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Advanced_libhtp_customization
13 years ago
Anoop Saldanha
340542c44e
refactor htpconfigure()
13 years ago
Eric Leblond
e176be6fcc
Use unlikely for error treatment.
...
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien
fcc21ae4cc
http: fix multipart parsing bug
13 years ago
Victor Julien
869109a6a0
stream/app layer: add Truncate app layer callback that is called if stream depth is reached. Use it to trunc open files in HTTP.
13 years ago
Eric Leblond
92679442ca
Convert to atomic and disable check on HTP config change.
...
This patch converts the series of variable to an atomic.
Furthermore, as the callbacks are now always run, it is not
necessary anymore to refuse a ruleswap if HTP parameters are
changing.
13 years ago
Eric Leblond
66a083dafa
Get rid of AppLayerHtpRegisterExtraCallbacks
...
This patch add a early exit condition to the body handling callback.
This permits to avoid to avoid a complex system to handle htp
object change.
13 years ago
Victor Julien
e737e2dc56
http: after path double decoding, also normalize the path again. #504 .
13 years ago
Victor Julien
e839cea9e5
Http: don't double decode URI path and query by default. Instead add per server options to enable double decoding for both cases. #464 #504 .
13 years ago
Victor Julien
ad827ad030
http: add more decoding unittests.
13 years ago
Victor Julien
43c7fd7585
file inspection: improve logging when stream.depth limit is reached. #493 .
13 years ago
Victor Julien
0c98980e21
http: add unittest to test \r in header line.
13 years ago
Anoop Saldanha
6fa46d7526
If new ruleset requires any htp callbacks that aren't already set, don't load new ruleset; request user to restart suricata + disable setting fileinsepection flags unconditionally in main
13 years ago
Victor Julien
ab3fcb01f9
http: decode double decoded path and query string characters. Bug #464 .
13 years ago
Victor Julien
f2f8dfd8d6
http: add test to make sure a missing space between header name and value is not a problem (ref #474 ).
13 years ago
Victor Julien
00948c86d5
Add debug messages to HTTP error/warning handling.
13 years ago
Anoop Saldanha
64625675ce
set stream_eof flag per stream, only when the stream initiates a close. Fix htp parser to close connection per direction based on this
13 years ago
Anoop Saldanha
608f4fe787
bug 452 - enable http extra callbacks for configs other than the default configs
13 years ago
Victor Julien
d378b76c04
http: body inspection improvement
...
Improve http_client_body and file_data performance when request and
response body limits are set to high values.
13 years ago
Victor Julien
4354434522
Add htp error debug printing.
13 years ago
Victor Julien
18837dce92
http: improve multipart parsing, skip empty records.
14 years ago
Victor Julien
18e81b7ba9
Bail out early if we're in http tunnel mode.
14 years ago
Anoop Saldanha
109662450d
Add new command line option --list-app-layer-protocols to list supported app layer protocols in sigs
14 years ago
Victor Julien
fe9258f0fb
Fix issue discovered by Anoop. Passing u32 ptr to a size_t can caused badness.
14 years ago
Victor Julien
e3935a2af2
Improve http filename parsing.
14 years ago
Victor Julien
e21d8cdf01
file extract: improve multipart parsing and set events on some error conditions.
14 years ago
Victor Julien
cdba2f50d1
Various fixes and improvements based on feedback by Coverity analyzer.
14 years ago
Victor Julien
3702a33ae9
file-inspection: support POST requests that do not use multipart.
14 years ago
Victor Julien
64827e3864
file-inspection: use filename= value from Content-Disposition where available to determine the filename in GET requests.
14 years ago
Victor Julien
93d121bf21
Update app layer events for HTTP now that libhtp has fixes for some response errors.
14 years ago
Victor Julien
16cfae2f51
Trigger raw stream reassembly on receiving a full HTTP request or response.
14 years ago
Victor Julien
f713b653ab
Convert error logging for HTTP to use new app layer event API. Expose libhtp warnings to this as well.
14 years ago
Victor Julien
a6e75aff21
file-extraction: improve handling of complex multipart bodies.
14 years ago
Anoop Saldanha
4b8ebb5c53
set default response body limit for specific http server conf
14 years ago
Anoop Saldanha
6240131a4e
updates to accomodate master rebase
14 years ago
Anoop Saldanha
7c9d1b80fd
Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls
14 years ago
Anoop Saldanha
e0c13434ef
bug 333 - support new Size Parsing API. Update various conf params inside the engine to use this API to parse sizes in the format xxx <-just the no represents bytes, xxxkb <- kilobytes, xxxmb <- megabytes, xxxgb <- gigabytes, where xxx is a \d+
14 years ago
Victor Julien
004b5dde88
Support libhtp's different handling of CONNECT requests.
14 years ago
Victor Julien
33848124d1
Fix a multipart body parsing issue.
14 years ago
Victor Julien
d59ca75e46
file extract: split toserver and toclient tracking
...
Split toserver and toclient file tracking for the http state.
14 years ago
Victor Julien
1c934acc85
Don't store fd per file (too many fd's). Enable IPv6 storing. Close file on receiving stream end flag.
14 years ago
Victor Julien
b402d97179
File carving -- enable reponse file extraction
...
- Enable response body tracking
- Enable file extraction for responses
- File store meta file includes magic, close reason.
- Option to force magic lookup for all stored files.
- Fix libmagic calls thead safety.
14 years ago
Victor Julien
66a3cd96a8
Prepare HTTP response body tracking.
14 years ago
Victor Julien
e1022ee5ae
file-extraction: Disconnect file handling from flow and move into the app layer state.
14 years ago
Victor Julien
23e01d23d3
Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored.
14 years ago
Victor Julien
403b2788d6
Add support for extracting PUT files.
14 years ago
Victor Julien
59cda9a358
Fix not using new htp callback when using the bundled htp. Add indication to --build-info. Fix valgrind warning in test and further improve test.
14 years ago
Victor Julien
ef0536794c
Adding comments, some cleanups.
14 years ago
Victor Julien
21acd72adf
Cleanups to the Multipart parsing code. Fixes to negation in filename and fileext.
14 years ago
Victor Julien
32fb9f375d
log-file log-dir option added, meta file created, fixes.
14 years ago
Victor Julien
a6b7a560f1
Fix a bug in the HTTP file closing.
14 years ago
Victor Julien
7e3d537338
Fix setting libhtp personality.
14 years ago
Victor Julien
cd618e48df
Allow for 0 (unlimited) HTTP request_body_limit, fix option parsing.
14 years ago
Victor Julien
4723f07254
Improve testing and fix some bugs.
14 years ago
Victor Julien
9d5d46c4bb
Implement flow file storage API, create HTP wrappers for it, use it in HTTP parsing.
14 years ago
Victor Julien
a0ee6ade3e
Improve HTTP multipart parsing, add streaming parsing for files.
14 years ago
Victor Julien
4537f889ef
Handle all strings as raw strings in HTTP content-type and content-disposition header parsing.
14 years ago
Pablo Rincon
6d60b3a747
filename and fileext keywords
14 years ago
Anoop Saldanha
9a6aef459e
modify all relevant app layer API calls to accomodate passing parser local storage argument
14 years ago
Victor Julien
262a7300d7
flow: shrink Flow datatype
...
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.
Add macro's to check the family, copy the address, etc.
Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.
On 64 bit this shrinks the flow with 8 bytes.
14 years ago
Victor Julien
06904c9024
App Layer cleanup
...
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
14 years ago
Anoop Saldanha
4650bf7170
minor code cleanup. remove commented out code
14 years ago
Victor Julien
2d16abcf8b
Minor code cleanups fixing all GCC 4.6 compiler warnings for default, debug and unittests mode.
14 years ago
Victor Julien
5395071c11
Make http logging code more robust against cases where the htp state is incomplete (out of memory conditions).
14 years ago
Eric Leblond
60a99915c1
doc: create http support group
...
This patch create an httplayer group and adds related files to
it. It also fixes some typo in documentation string and format.
14 years ago
Eric Leblond
a64eea9628
Fix minor error message.
14 years ago
Eric Leblond
92d74fd480
doc: Add missing params in func description.
14 years ago
Victor Julien
bde55578d6
Override HTP IDS personality normalizing the query string to lowercase. Bug #362 .
14 years ago
Victor Julien
3644e90a2c
Don't set higher transaction id's in HTTP sessions than we have.
14 years ago
Anoop Saldanha
4e44073c79
http logging module should log all txs in the list and not just the last complete tx available on EOF
14 years ago
Anoop Saldanha
b406af451b
updates to http tx id vars. FFR now flags the app layer session for EOF when creating a pseudo packet for a flow
14 years ago