Commit Graph

5447 Commits (21db5ee691b349d3da1925b703ea5200d736e40a)

Author SHA1 Message Date
Victor Julien 21db5ee691 counters: reduce global usage 10 years ago
Victor Julien a5168d5977 dce_opnum: improve memory handling on parsing error 10 years ago
Victor Julien b3b7625be5 htp: fix test 10 years ago
Victor Julien c0807c3df5 fast log: clean up tests 10 years ago
Victor Julien 29fbcce50d detect hsbd: simplify resize logic 10 years ago
Victor Julien 4e7cb7b863 app-layer: update all protocols to accept NULL+EOF
Update all non-HTTP protocol parsers to accept a NULL+EOF input.
10 years ago
Victor Julien cf9ff6adbd app-layer: improve EOF handling
On receiving TCP end of stream packets (e.g. RST, but also sometimes FIN
packets), in some cases the AppLayer parser would not be notified. This
could happen in IDS mode, but would especially be an issue in IPS mode.

This patch changes the logic of the AppLayer API to handle this. When no
new data is available, and the stream ends, the AppLayer API now gets
called with a NULL/0 input, but with the EOF flag set.

This allows the AppLayer parser to call it's final routines still in the
context of a real packet.
10 years ago
Victor Julien df79c1019f print: make PrintRawDataFp take a const arg 10 years ago
Victor Julien 7451d33396 stream: update StreamMsg to don't have fixed size
StreamMsg would have a fixed size buffer. This patch replaces the buffer
by a dynamically allocated buffer.

Preparation of allowing bigger and customizable buffer sizes.
10 years ago
Victor Julien 3ffa01d150 stream: remove STREAMTCP_STREAM_FLAG_CLOSE_INITIATED logic 10 years ago
Giuseppe Longo 32563d51d1 http: rework UT
This reworks UT based on
commit 3203555708
10 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 10 years ago
Victor Julien 3163a57577 smtp file_data: fix wrong free 10 years ago
Victor Julien 9f52bdd1e5 flow timeout: prevent dead locks
The flow timeout mechanism called both from the flow manager at run time
and at shutdown creates pseudo packets. For this it has it's own packet
pool, which can be depleted if the timeout logic is faster than the packet
processing threads. In this case the flow timeout would enter a wait loop.
The problem however, is that this wait loop would happen while keeping a
flow locked. This could lead to a race condition when the packet thread(s)
are waiting for the lock that the flow manager has.

This patch introduces a new packet pool call 'PacketPoolWaitForN', meant
to make sure that the thread's packet pool has at least N available
packets. The flow timeout paths use this to make sure enough packets are
available *before* grabbing the flow lock. If there aren't enough packets
available yet, the wait happens before the lock as well.

This still means the wait can happen while the flow hash row is locked, so
we do make sure some more packets are available when entering that. But
perhaps in the future we need a more precise logic there as well.
10 years ago
Victor Julien 423d5fd0de http: improve inline body tracking
Rewrite the sliding window handling for IPS mode for the server body.

The buffer used will have the following properties:
    left edge: inspected data - window_size
    right edge: the most recent data

Due to handling of compressed bodies, the data can be much bigger than
the configured window size.
10 years ago
Victor Julien 79d34f4575 http: body pruning update
Take inspect limits into account. Consider per direction inspect settings.
10 years ago
Giuseppe Longo 8f1998e35f hsbd: inspect buffer depending on the engine mode
Currently, data is buffered up to response-body-minimal size
and response-body-inspect-window before being inspected.
With this, in IPS mode, inspect data as it comes in up.

The sliding window concept is used here,
some data chunks are copied into the window (buffer)
then it's inspected.
10 years ago
Giuseppe Longo a459376d2e app-layer-htp: add http_body_inline setting 10 years ago
Victor Julien 80615f7d43 ips: move counters in common struct and funcs 10 years ago
Victor Julien 813930f027 nfq: add ips stats 10 years ago
Victor Julien 14466a803d decode: create util function for basic counter updates 10 years ago
Jason Ish e3ce29f694 json-stats: log deltas
If "deltas" is yes, log delta values as the name of the value
suffixed with _delta.
10 years ago
Victor Julien 3e14aa729a detect-state: remove/hide BUG_ON statements 10 years ago
Victor Julien b7f1e9e370 file extract: add app_proto to logging 10 years ago
Victor Julien 925aac854e alproto: improve AppProtoToString
Make AppProtoToString compilation fail on missing 'case's.
10 years ago
Eric Leblond 0bafc71689 app-layer: add modbus to AppProtoToString
It was missing causing protocol identified as modbus not to be
displayed in netflow events.
10 years ago
Victor Julien ebb42f831c suppress: add track by_either mode
So far suppress rules would apply to src or dst addresses of a packet.
This meant that if a ip would need to suppressed both as src and as dst,
2 suppress rules would be needed.

This patch introduces track by_either, which means that the ip(s) in the
suppress rule are tested against both the packets source and dest ip's.
If either of them is on the suppress list, the alert is suppressed.
10 years ago
Victor Julien e85a44c383 suppress: support ip-lists
Ticket: 1137

Support supplying a list of IP's to the suppress keyword. Variables from
the address-groups and negation is supported. The same logic (and code) is
used that is also used in parting the IP portions of regular detection
rules.
10 years ago
Jason Ish 26fc5682ad hostbits: ignore leading and trailing white space
Ignore leading and trailing space around the name and
direction tokens.
10 years ago
Jason Ish 7c40c73482 json-stats: reorg threads and totals
Totals are now placed at the top level instead of under a "Total"
object.

Threads are placed under a "threads" object.
10 years ago
Jason Ish 1f2caf78c3 json-stats: log uptime in seconds, instead of a string 10 years ago
Jason Ish 0f1dd0d7ea flowbits: strip leading and trailing spaces in name
Redmine bug 1481. Strip leading and trailing white space. Factor
out parsing from setup while in here.
10 years ago
Zachary Rasmor 0edf28a4f8 Add Feature #1454. Generic eve-log prefix support. 10 years ago
Victor Julien a083513c49 decode: optimize DecodeThreadVars layout
Put common counters on the first cache line. Please the flow output
pointer last as it's use depends on the flow logging being enabled
and even then it's only called very rarely.
10 years ago
Victor Julien fe5a85aea0 decode: add erspan counter 10 years ago
Victor Julien 928957f0a3 decode: add ERSPANv1 decoder
Only allow v1 to be parsed as thats what is tested.

Take vlan_id from the ERSPAN layer.
10 years ago
Victor Julien aa6b24f814 decode: clean up tunnel decode logic
Don't use mix of existing and custom types to indicate the next
layer.
10 years ago
Victor Julien ef7cd043cc detect: various header cleanups 10 years ago
Victor Julien 5483b800c5 detect: remove struct/union tricks from Signature 10 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 10 years ago
Victor Julien 9fa2f85cc7 http: improve body pruning
Take inspect window into account.
10 years ago
Victor Julien 0bbc818b2d http: fix body tracking
In HTTP body tracking for response bodies, pruning body chunks was broken
as the body parsing code wouldn't update HtpBody::body_parsed.
10 years ago
Victor Julien 3203555708 http-client-body: create unittest util func 10 years ago
Eric Leblond d837562441 logging: fix modules ordering during logging
With the previous code the order of the logging modules in the
YAML were determining which module was run first. This was not
wished and a consequences was that the EVE fileinfo module was
not correctly displaying the key 'stored' because it was
depending on a flag set alter by the filestore module.

This patch adds a priority file to the TmModule structure. The
higher the priority is set, the sooner the module is run in the
logging process. The RunModeOutput structure has also been
updated to contain the name of the original TmModule. Thus allowing
to define a priority for a RunModeOutput.

Currently only the filestore has a priority set. The rest of them is
set to the default value of zero.
10 years ago
Eric Leblond be07620a60 output-lua: sync variable name with yaml
'script-dir' was used in the code but we had 'scripts-dir' in the
configuration file. This patch fixes it to 'scripts-dir'.
10 years ago
Jason Ish ae23144b67 --set - handle spaces on either side of '='
Discard spaces when provided as part of --set around the '='. For
example, "val=key", "val = key", "val= key" and "val =key" are
all equivalent now.
10 years ago
Jason Ish d9fe95bc8a conf - function declaration style
Use consistent style - function return type and declaration on
same line.
10 years ago
DIALLO David 0a4fd39f9c modbus: fix heap-buffer-overflow in Modbus parser
Modbus parser does not check length to extract/read data (read or write address,
quantity of data, etc.) that should be present.

In case of malformated data (invalid length in header), Modbus parser reads data
over the input data length.

Add check before extracting/reading data from input buffer to avoid head buffer
overflow.
10 years ago
Victor Julien 07efec550d counters: use ptr to name instead of copy
All counters have hardcoded names, so copies are not needed.
10 years ago
Victor Julien 7e66c70507 counters: don't run if no counters have been registered 10 years ago