Set event on overlapping data segments that have different data.
Add stream-events option stream-event:reassembly_overlap_different_data and
add an example rule.
Issue 603.
It violates thread safety. #601.
Suricata assures thread safety on the flow level for HTTP tracking. Part of the flow is (in case of HTTP) libhtp's htp_connp_t state. At startup the libhtp glue layer, app-layer-htp initializes as many htp_cfg_t instances as there are libhtp server configurations in the yaml. At HTTP session start, we look up the proper htp_cfg_t based on the server ip and pass it to htp_connp_create. A ptr to the relevant htp_cfg_t is part of the htp_connp_t. The htp_cfg_t contains "hooks". The are registered based on yaml config at init time.
The hooks have lists of type list_t. The list is run with a built in iterator. The iterator is reset at the start of each "hook_run_all". Since multiple flows share the same htp_cfg_t flow A can reset the iterator while flow B is using it. The flow lock has no effect as flows share the htp_cfg_t.
This has been observed in real traffic. hook_response_body_data was run on the same data multiple times, leading to corrupt extracted files.
This patch adds a call to close the queue when the acquisition
loop is ending. This way the incoming packets will be accepted
during all the shutdown phase (if the queue-bypass option of
NFQUEUE is used). At the same time the currently processed packets
will be dropped but the time scale are different: suricata will
drop 20 ms of packets and the shutdown can take 0.5 seconds.
Patch based on an idea of Victor Julien.
If a pattern has matched on mpm, don't re-inspect it later, subject to certain
conditions met by the pattern - namely, not negated, right chop, no replacet
attached to it.
The power of libhtp customisation now available to users.
Options available -
path-backslash-separators: yes
path-compress-separators: yes
path-control-char-handling: none
path-convert-utf8: yes
path-decode-separators: yes
path-decode-u-encoding: yes
path-invalid-encoding-handling: preserve_percent
path-invalid-utf8-handling: none
path-nul-encoded-handling: none
path-nul-raw-handling: none
set-path-replacement-char: ?
set-path-unicode-mapping: bestfit
You can use this for your libhtp customisation. Options explained in our
wiki.
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Advanced_libhtp_customization
In FFRv2, dereference flow from a packet using the new reference/dereference
util macros. This allows the decr use_cnt for flow and reseting the flow
pointer to NULL for the pseudo pkt to happen simultaneously, in case there we
fail to retrieve a pseudo_packet and have to return the already obtained
pseudo packets, back to the packetpool.