Commit Graph

6084 Commits (77119a31863ee9cc3bedcd8abf007914ba7942cc)
 

Author SHA1 Message Date
Eric Leblond 77119a3186 file-json: output smtp proto info 10 years ago
Eric Leblond 47a199ee97 smtp-json: introduce function to output smtp data 10 years ago
Eric Leblond 94dbd303e4 file-json: log http data using common function 10 years ago
Eric Leblond 4ef12dcf5d alert-json: use new JsonHttpAddMetadata function
This patch uses the newly introduced function to handle the logging
of HTTP data.
10 years ago
Eric Leblond bccabe3813 http-json: introduce JsonHttpAddMetadata function
This function will be usable in other logging components to add
the http data to their messages.
10 years ago
Eric Leblond d7e13c2c03 email-json: output MIME parsing status
If the status is not PARSE_DONE then in that case we may have
imcomplete information. Increasing the stream reassemly depth
in that case would be a good idea.
10 years ago
Eric Leblond a233a982ea decode-mime: add function to get status
This new function return the textual status of MIME parsing.
10 years ago
Eric Leblond 9900558428 smtp: add 'body-md5' mime option
This option will allow the user to select weither or not he wants
to journalize the md5 of the mail body.
10 years ago
Eric Leblond ea311c1594 email-json: export md5sum of body
The body_md5 has been added and contain the value of the md5sum
of the body.

This patch is using the state PARSE_DONE on the MIME state to
detect when a message has been completely parsed.
10 years ago
Eric Leblond d39009ca58 decode-mime: compute body md5
This patch is computing the md5 sum of the body of the MIME message.
This will allow to detect messages with same content and sent to
different people.
10 years ago
Eric Leblond e43eb76abd app-layer-stmp: simplify code
Delete a only used once goto to a point where we only do a return.
10 years ago
Eric Leblond 0f3979cc81 output-json-smtp: output RCPT TO fields
This patch uses an array to output the RCPT TO fields to the
JSON message.
10 years ago
Eric Leblond 752fdba957 app-layer-smtp: parse and extract RCPT TO fields
Add the RCPT TO fields to a linked list stored in the transaction.
10 years ago
Eric Leblond 2abae3f0a1 smtp-json: update SMTP EVE messages
This patch updates SMTP message to have them feature a 'smtp'
section which will contain all fields coming from the smtp
protocol.
10 years ago
Eric Leblond 7bca8268bc app-layer-smtp: extract and store HELO and MAIL FROM
This patch updates the SMTP transaction and SMTP state to be able
to contain the HELO and MAIL FROM fields.
10 years ago
Eric Leblond 5c26a2f2c8 email-json: move email fields to email section
This patch changes the way smtp message are written. It is using
the "email" key to store the email related fields. This will
allow to do the same search through SMTP and IMAP if we implement
this last one.
10 years ago
Victor Julien 77302e5d51 threshold: remove debug message from info loglevel 10 years ago
Victor Julien fc7f090cd3 flow: add missing storage size to checks, output 10 years ago
Victor Julien 37fa4a4876 host: update host size logic
Instead of using (sizeof(Host)+HostStorageSize()) in many places,
create a simple size variable that is set during setup.
10 years ago
Victor Julien 480e91edac ippair: update ippair size logic
Instead of using (sizeof(IPPair)+IPPairStorageSize()) in many places,
create a simple size variable that is set during setup.
10 years ago
Victor Julien ff769b73a7 stream: improve retransmission detection
Consider packets starting before last_ack and ending after it also
to be retransmissions. This way we can see if they are having
different data.
10 years ago
Victor Julien 6b2f831a70 mpm: SGH maxlen was actually minlen, so rename 10 years ago
Victor Julien 2716c78628 mpm: improve SGH content len tracking
SGH's track content length for rule grouping.

This patch changes the logic to only consider the pattern that is
used in the mpm for a sig.
10 years ago
Victor Julien e529ebb50e mpm: redo uri maxlen logic
The mpm_uricontent_maxlen logic was meant to track the shortest
possible pattern in the MPM of a SGH. So a minlen more than a maxlen.

This patch replaces the complicated tracking logic by a simpler
scheme. When the SGH's are finalize, the minlen is calculated.

It also fixes a small corner case where the calculated "maxlen" could
be wrong. This would require a smaller pattern in a rule to be forced
as fast pattern.
10 years ago
Victor Julien df95d375bb detect: improve comments on mpm 10 years ago
Victor Julien 496f9800ac mpm: remove used counter 10 years ago
Victor Julien c53c9b4b20 mpm: remove bloated counting logic
Counters were only used to print debug info.
10 years ago
Victor Julien da7bad7c1b mpm: improve debug output 10 years ago
Victor Julien 977074930b mpm: use IPPROTO_TCP for readability 10 years ago
Victor Julien a559c41295 mpm: optimize & debug validate
Wrappers are called only if a mpm_ctx is available. So remove the test
for a null ctx and replace it by a debug validation BUG_ON.
10 years ago
Victor Julien 0dd3b73db2 mpm: assume we'll likely have a mpm_ctx 10 years ago
Victor Julien 7c336f4190 mpm: indent fix, no functional change 10 years ago
Victor Julien a00d83f1f5 mpm: change direction checking in mpm wrappers
Instead of having reachable assertions, use DEBUG_VALIDATE_BUG_ON
10 years ago
Victor Julien 804f861967 debug validation: introduce DEBUG_VALIDATE_BUG_ON
DEBUG_VALIDATE_BUG_ON(exp) will call BUG_ON(exp) if debug validation
is compiled in. Otherwise it's a no-op.
10 years ago
Victor Julien e755913b4b mpm: minor fixes and cleanups 10 years ago
Victor Julien cacf425bd3 stream: improve handling of GAPs at stream start
Detect and handle gaps at the start of the stream, when there may
be no segments in the list (yet).
10 years ago
Victor Julien 574ef0ad2a stream: RST last_ack update fix
Only use ACK if ACK flag was set and ACK value is valid.
10 years ago
Victor Julien 3ca44219dc proto detect: more bypass conditions
More exceptional cases for protocol detection. In very unbalanced flows,
where just a few bytes are sent toserver and many toclient, proto detect
might not complete in time on the toserver direction. This can lead to
queuing up many segments in the toclient direction.

Another case is that in come cases the stream is flagged as proto detect
done, but the flows proto detect flags are not set. This is now handled
by the ProtoDetectDone() check.
10 years ago
Victor Julien fa8dc77dcc debug validation: add segment list sanity check 10 years ago
Victor Julien e67188e437 detect: fix issue with smsg and seq wraps
Due to a broken sequence number check, detect could fail to process
smsgs in case of a sequence wrap. This could lead to excessive use
of smsg's but also of segments, since these aren't cleared until the
smsg containing them is.
10 years ago
Victor Julien 8ac49d9129 stream: allow next_seq catch up after pkt loss
If next_seq falls behind last_ack, force update it.
10 years ago
Victor Julien 596465b76d stream: use reassembly fast path after proto detect
Use the reassembly fast paths only after protocol detection has completed.
In some corner cases the sending of smaller segments lead to protocol
detection failing.
10 years ago
Victor Julien 34ed15e182 stream: fix protocol detection issue for GAPs
If the protocol required TOSERVER data first, but the SSN started with
a GAP, then the TOCLIENT side would get stuck in an expensive path:

1. it would run detection on TOCLIENT
2. it would try to force reassembly for TOSERVER
3. it would reset the detected protocol as TOSERVER failed
4. it would not evict any segment

This had 2 consequences:
1. on long running sessions this could lead to using lots of memory
   on segments, denying other sessions resources
2. wasted cycles on protocol detection and segment list management

This patch introduces a fix. It checks in the (2) stage above, whether
the opposing stream (that we depend on) it is a NOREASSEMBLY state. If
so, it gives up on this side of the session as well.
10 years ago
Victor Julien 708e80c900 stream: optimize proto detect segment handling
In case of protocol detection not yet being complete, the segment
list was walked unconditionally to unset the app layer processed
flag. Optimize this to bail on the first segment that doesn't have
the flag set.
10 years ago
Jason Ish 84fd28eaed app-layer setup scripts: fix header substitution.
Fixes make distcheck.
10 years ago
Eric Leblond a286715367 host-storage: document host storage API 10 years ago
Eric Leblond f8b8b6f753 configure: use pkg_config for libhtp
It was not possible to simply specify PKG_CONFIG_PATH to build
with an non bundled libhtp. With this patch we don't need anymore
the htp lib and include configure options.
10 years ago
Jason Ish 4a738023d5 app-layer: scripts to setup app-layer templates
setup-app-layer.sh sets up an application layer detector and
parser template.

setup-app-layer-logger.sh sets up a JSON application layer
transaction logger for an application parser that has
already been provisioned.

setup-app-layer-detect.sh sets up a keyword for performing
content inspections on buffers created by the application
layer.
10 years ago
Jason Ish 06beca62f5 app-layer: template for application layer content inspection 10 years ago
Jason Ish bcda92134d app-layer: template for application layer tx logger 10 years ago