Commit Graph

170 Commits (7e268bd4d4684838559c80948ae2c9932e33eb82)

Author SHA1 Message Date
Victor Julien 9dc04d9fab app layer: add support for per TX decoder events 12 years ago
Victor Julien 59780ca770 Hacks to enable alert dns even though we have dnstcp and dnsudp parsers. Needs proper solution later. 12 years ago
Victor Julien 8e01cba85d DNS TCP and UDP parser and DNS response logger 12 years ago
Anoop Saldanha a490176c8a More lock fixes for the transaction update. Issues reported by Coverity. 12 years ago
Anoop Saldanha 7cf4042337 Fix luajit compilation failure introduced by the transaction update.
Fix coverity lock issues reported by transaction update as well.
12 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
12 years ago
Victor Julien afb2d4eddf Fix stateful inspection not always inspecting at stream end. 13 years ago
Eric Leblond 1fd47cfb96 Remove useless code. 13 years ago
Eric Leblond b3d4285982 fix logic error in sanity check 13 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Eric Leblond 6842545331 Add documentation url in list-keyword output.
The output of the list-keyword is modified to include the url to
the keyword documentation when this is available. All documented
keywords should have their link set.

list-keyword can be used with an optional value:
 no option or short: display list of keywords
 csv: display a csv output on info an all keywords
 all: display a human readable output of keywords info
 $KWD: display the info about one keyword.
13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien 20d2db085e reintroduce pool free func for cases where block alloc is not used. 13 years ago
Eric Leblond 619014a280 pool: rename Free function to Cleanup
This patch renames Free functions to Cleanup as the free is made
by the pool system.
13 years ago
Eric Leblond fa079c1da0 pool: realize a block allocation for preallocated item.
This patch required a evolution of Pool API as it is needed to
proceed to alloc or init separetely. The PoolInit has been changed
with a new Init function parameter.
13 years ago
Victor Julien f93c54136c stream/app layer: call new Truncate callback for data gap case as well. 13 years ago
Victor Julien 869109a6a0 stream/app layer: add Truncate app layer callback that is called if stream depth is reached. Use it to trunc open files in HTTP. 13 years ago
Anoop Saldanha 6c5b596ada coverity fixes 13 years ago
Anoop Saldanha 109662450d Add new command line option --list-app-layer-protocols to list supported app layer protocols in sigs 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Victor Julien 8b1333a277 Add more flow lock assertions to the debug validation code. 14 years ago
Victor Julien 5ba41c7890 Fix locking error in filestore handling. Add debug validate check for asserting a flow is locked. 14 years ago
Victor Julien f084874998 Fix HTTP state and raw stream not being inspected at the same time. Adds an exception to transaction id handling for HTTP. 14 years ago
Victor Julien 16cfae2f51 Trigger raw stream reassembly on receiving a full HTTP request or response. 14 years ago
Victor Julien f773942ce0 Disable printing dreaded app layer error messages to the screen: app layer events are here to safe us. 14 years ago
Anoop Saldanha 5311cd4866 Support for smtp decoder events 14 years ago
Anoop Saldanha eea5ab4a7a Support for app layer decoder events added + app_layer_event keyword added 14 years ago
Victor Julien 3009429e3c HTTP transaction handling improvement
In some cases AppLayerTransactionGetInspectId can return -1, which is
now handled by all it's callers.

Improve logic of selecting which transactions are inspected by the various
HTTP keywords.
14 years ago
Victor Julien 45d86ff58a Stream reassembly / app layer: disable gap errors
Gap errors on the app layer are now silently handled. No longer printed
to the screen.
14 years ago
Victor Julien b74c73309b file handling: improve filestore keyword handling
In stateful detection only inspect the file portion of the rule after all
other conditions matched. This to prevent "filestore" from tagging files
for storage during a partial match.

Add a couple of unittests to test the behaviour change.
14 years ago
Victor Julien 2ccd35c6e4 Fix code after rebase. 14 years ago
Victor Julien d59ca75e46 file extract: split toserver and toclient tracking
Split toserver and toclient file tracking for the http state.
14 years ago
Victor Julien e1022ee5ae file-extraction: Disconnect file handling from flow and move into the app layer state. 14 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 14 years ago
Anoop Saldanha 9a6aef459e modify all relevant app layer API calls to accomodate passing parser local storage argument 14 years ago
Anoop Saldanha 01a35bb604 introduce app layer local storage api support 14 years ago
Victor Julien 262a7300d7 flow: shrink Flow datatype
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.

Add macro's to check the family, copy the address, etc.

Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.

On 64 bit this shrinks the flow with 8 bytes.
14 years ago
Victor Julien 06904c9024 App Layer cleanup
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
14 years ago
Victor Julien 7e3c15e54a stream: improve TCP ssn reuse cleanup. 14 years ago
Anoop Saldanha de9ad02b59 Remove leftover imap and msn toclient alproto PM contents 14 years ago
Victor Julien 8186565240 Fix a number of potential issues found by CLANG and cppcheck. 14 years ago
Anoop Saldanha 0edf053f31 if app layer inspection is disabled, immediately set the eof flag 14 years ago
Anoop Saldanha fe11e02f58 fix inspect id update bug. This should prevent unnecessary FPs for pipelined requests 14 years ago
Anoop Saldanha 4e44073c79 http logging module should log all txs in the list and not just the last complete tx available on EOF 14 years ago
Anoop Saldanha c13ad8c28a Provide a function to set the app layer tx eof flag. Use this in FFR code instead of diretly setting the flag. This cleans up the API as well 14 years ago
Anoop Saldanha b406af451b updates to http tx id vars. FFR now flags the app layer session for EOF when creating a pseudo packet for a flow 14 years ago
Anoop Saldanha d68775d47d introduce bitmasks instead of alproto_masks for use by the probing parser. Remove all alproto_masks we had previouslys for PP 14 years ago
Anoop Saldanha d3989e7cee probing parser updated to always accept u32 buflens. Update all probing parser functions to accomodate this change 14 years ago
Anoop Saldanha 432c3317d2 app layer probing parser updates 14 years ago
Eric Leblond 6b9d1012ff Transform inet_ntop call into PrintInet one. 14 years ago
Anoop Saldanha 576ec7da66 smtp parser support 14 years ago
Victor Julien 73efb4c70f Add a app layer state and stateful detection engine counter that makes sure the stateful inspection is only done when the state changes. 14 years ago
Anoop Saldanha 6e0d98d9c4 fix valgrind issue for SMB test. Small restructuring. probing_parsers global variable now part of AlpProtoDetectCtx 14 years ago
Anoop Saldanha 7f8fb0f00d fix bounds checking in smb probing parser 14 years ago
Anoop Saldanha a40fdc794e Added probing parser for nbss/smb on port 139 14 years ago
Anoop Saldanha 7c31a2327e Add support for port based probing parsers for alproto detection 14 years ago
Anoop Saldanha fe6e41e3ef Removed FLOW_AL_NO_APPLAYER_INSPECTION. Moved it as FLOW_NO_APPLAYER_INSPECTION in Flow->flags. Turned Flow->flags into uint32_t and removed Flow->alflags 14 years ago
Anoop Saldanha 38fe2b9070 Removed FLOW_AL_STREAM_START, EOF and GAP flags. We don't need these. Just use STREAM_* flags 14 years ago
Anoop Saldanha 000ce98cd1 push all proto detection code into their respective app parser register functions for every alproto 14 years ago
Victor Julien 8fa5a2c025 Split applayer and raw stream reassembly
Split stream reassembly in 2 parts: a part that sends ack'd data to the app
layer parsers as soon as it's available, and another part that queues up
data into larger chunks for raw inspection.
15 years ago
Victor Julien dda6d3e07b Add error counters. 15 years ago
Eric Leblond 49adc264bc Don't print message after SCMalloc failure.
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
15 years ago
Victor Julien b8fec77f37 Fix tcp connections that are reset (RST packet) not always inspecting the reassembled stream. Update transaction id code to make sure both directions of a transaction are inspected before incrementing the inspect_id. 15 years ago
William Metcalf 0e4235cc94 FLOW_DESTROY added to clean-up UT's that init flow 15 years ago
Victor Julien 83b2c8abdb Improve stateful uri detection code. 15 years ago
Victor Julien e8fce5f7fa Convert uricontent scanning to use the detect engine state. 15 years ago
Victor Julien ba12f3c109 Applayer to flow fixes and cleanups. 15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
William Metcalf cc76aa4bc6 properly init flows inside of unit-tests caused lock-up when falling back to using mutex locks 15 years ago
Victor Julien 9f95ab7441 Make sure a stream that has a failing app layer inspection module no longer stops reassembly, but only app layer inspection. This way we can continue to inspect the reassembled stream. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien c3392b7c22 Fix checking for the stream GAP after the ssn ptr was initialized. 15 years ago
Victor Julien 9676273e6d Kick out streams with gaps in them in the app layer parser until we add proper support. 15 years ago
Victor Julien 13e6c8035d Make sure we don't leak memory on app layer protocols we detect, but don't parse. Fixed #132. Thanks to Gurvinder Singh for pointing out where the issue was. 15 years ago
Pablo Rincon f862de2ee6 Fixing some code reviews (Thanks to Steve Grubb) 16 years ago
Gurvinder Singh 8e444f1772 stream and application layer improvements 16 years ago
Victor Julien 3d7b882bde Make sure all smsgs are handled every time, even in case or error. The fuzzer found an issue where unhandled messages remained in the queue leading to threading issues. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Pablo Rincon d0404d8447 Renaming errors with naming conventions 16 years ago
Pablo Rincon ad2c136e8f Renaming errors (naming conventions) 16 years ago
Victor Julien 6a53ab9c5a Stream engine memory handling update
The stream engine memory handling needed updating as it didn't scale. Changes:

- pools can now be initialized to size 0, meaning unlimited
- stream engine uses a memcap setting. Sessions, segments and aldata is part
  of this, app layer state isn't.
- memory is accounted using a global int that is spinlocked.
- a counter for sessions that have not been picked up because of memcap was
  added.
- all reassembly errors are converted to debug msgs.
16 years ago
Gurvinder Singh 356a8bf385 applayer uri match and modified http handling 16 years ago
Victor Julien c352bff6fb Remove unused conditional locking code from the app layer parsing code. 16 years ago
Pablo Rincon 705471e4ee Adding single pattern matcher algorithms. If you cannot store a context for the patterns, use SpmSearch() macro. Adding unittests and stats 16 years ago
Victor Julien cae8e06cb9 Properly lock app layer result pool and add some debugging code for memory tracking. 16 years ago
Gurvinder Singh 66cc392177 init b46 16 years ago
Victor Julien 4824868766 Application layer detection improvements
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
16 years ago
Gurvinder Singh d8433c7255 fixed-pool-error-and-tcp-state-transition 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Victor Julien 18fe3818dc Remove need_lock from app layer parsers. 16 years ago
Victor Julien ba7e8012af Add some debugging and simplify locking for app layer slightly. 16 years ago
Gurvinder Singh fc2f7f29fa app layer htp error handling and fixes for memory leaks and segv 16 years ago
Victor Julien d388444ac3 Use updated mutex calls. 16 years ago
Gurvinder Singh ad3e463974 updated error info ouput 16 years ago
Gurvinder Singh 1b39e602d0 fixed port info 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Gurvinder Singh c1e485cc44 app layer error handling 16 years ago