Commit Graph

14156 Commits (a6723bca7c5f2e9e3a917479706e0e5a7a0c66e3)
 

Author SHA1 Message Date
Victor Julien aa376a3b21 detect/frame: improve frame detection
Add a per frame progress tracker.
2 years ago
Victor Julien 169ee11ead output/frame: log frame type stream 2 years ago
Victor Julien d72bc364de output/frame: improve 'complete' logging 2 years ago
Victor Julien 6cbb5306c6 frame: add debug validation check 2 years ago
Victor Julien b43dc5a64a app-layer/frames: use absolute frame offset
Frame offset was already a 64 bit integer, so simplify things by
making it an absolute offset from the start of the stream.
2 years ago
Victor Julien 39d9b3adbe frames: implement generic <alproto>.stream frames
Add a hard coded <alproto>.stream option for all stream data for
a protocol.

Starts at stream offset 0 or at the point of a protocol upgrade
in case of STARTTLS or CONNECT.
2 years ago
Victor Julien f773b714e9 detect/frames: minor cleanup in buffer handling
Don't lookup a buffer twice, even if the lookup should be fast.
2 years ago
Victor Julien d0f1507c83 htp: simplify streaming buffer config
Use a single static config instead of the per profile config.
2 years ago
Victor Julien c79c0ca347 streaming: remove config pointer from struct 2 years ago
Victor Julien 53d9a1f39f streaming: internal switch to sbcfg by argument 2 years ago
Victor Julien ff882cd73f streaming: add sbcfg to StreamingBufferClear 2 years ago
Victor Julien 6e5f35e0a0 streaming: add sbcfg to StreamingBufferFree 2 years ago
Victor Julien 96f29440cb streaming: add sbcfg to StreamingBufferAppendRaw 2 years ago
Victor Julien 30ee9165b5 streaming: add sbcfg to StreamingBufferAppend 2 years ago
Victor Julien 3b5deb4ec7 streaming: add sbcfg to StreamingBufferSlideToOffset 2 years ago
Victor Julien b9540d1073 streaming: add sbcfg to StreamingBufferInsertAt 2 years ago
Victor Julien 058dc02e81 streaming: add sbcfg to StreamingBufferAppendNoTrack 2 years ago
Victor Julien 355f259b8c output/filedata: trunc file in output again 2 years ago
Victor Julien e3e55406a7 files: update API and callers to take stream config
This is to allow not storing the stream buffer config in each file.
2 years ago
Victor Julien f7dbdb7631 output/filedata: don't call file close
Will be reenabled after file API is updated.
2 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 2 years ago
Victor Julien a1a221066f files: remove filecontainer drop trait
In preparation of it becoming impossible to use due to the free
function getting an cfg argument.
2 years ago
Victor Julien 0320c03f8c http2: explicity free files
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
2 years ago
Victor Julien 4b1e9f7c21 smb: explicity free files
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
2 years ago
Victor Julien 3a24cce289 nfs: explicity free files
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
2 years ago
Victor Julien 4bfeac6591 nfs: file handling cleanups 2 years ago
Victor Julien 33f6a16290 smb: file handling cleanups 2 years ago
Victor Julien d57510a10f files: remove unused Rust binding for file pruning 2 years ago
Victor Julien f19b40a7f6 streaming: set size and max regions defaults 2 years ago
Victor Julien 229b82721d htp: remove duplicate prototypes 2 years ago
Victor Julien f788d31f22 htp/body: minor optimization 2 years ago
Victor Julien b62a513c47 flow: inline commonly used getters 2 years ago
Victor Julien 76a256a8b1 streaming: remove inefficient buffer grow logic 2 years ago
Victor Julien 467234659d streaming: remove unused slide logic 2 years ago
Victor Julien 0b36bde9be streaming: remove unused config member 2 years ago
Victor Julien 2cfbefb6c6 streaming: make minimum region gap size configurable 2 years ago
Victor Julien 8e9dac99f3 stream/reassembly: make max-regions configurable; set default 2 years ago
Victor Julien f896f03b7b streaming: add max regions config option 2 years ago
Victor Julien 42d3cd2061 stream/list: hack around GAP handling in tests 2 years ago
Victor Julien d2001ef94b stream: improve gap tests 2 years ago
Victor Julien 96dfd65b96 eve: log max regions 2 years ago
Victor Julien 1bb6f44ff0 stream: flow timeout improvement
Check continuous data for app-layer and post gap data as well for
stream inspection.
2 years ago
Victor Julien 1dac2467c5 streaming: implement memory regions
In TCP, large gaps in the data could lead to an extremely poor utilization
of the streaming buffer memory. This was caused by the implementation using
a single continues memory allocation from the "stream offset" to the
current data. If a 100 byte segment was inserted for ISN + 20MiB, we would
allocate 20MiB, even if only 100 bytes were actually used.

This patch addresses the issue by implementing a list of memory regions.
The StreamingBuffer structure holds a static "main" region, which can be
extended in the form of a simple list of regions.

    [ main region ] [ gap ] [ aux region ]
    [ sbb ]                 [ sbb ]

On insert, find the correct region and see if the new data fits. If it
doesn't, see if we can expand the current region, or than we need to add
a new region. If expanding the current region means we overlap or get
too close to the next region, we merge them.

On sliding, we free any regions that slide out of window and consolidate
auxilary regions into main where needed.

Bug: #4580.
2 years ago
Victor Julien 61e47ad6f5 stream: reduce streaming buffer internals use 2 years ago
Victor Julien a24d7dc45c smb: fix post-trunc chunk behavior
After a gap in a file transaction, the file tracker is truncated. However
this did not clear any stored out of order chunks from memory or stop more
chunks to be stored, leading to accumulation of a large number of chunks.

This patches fixes this be clearing the stored chunks on trunc. It also
makes sure no more chunks are stored in the tracker after the trunc.

Bug: #5781.
2 years ago
Philippe Antoine e22b59b8c1 fuzz: check libpcap timestamp consistency
That is microseconds should be positive
2 years ago
Philippe Antoine 1c436fe0ca fuzz: fix use of uninitialized value
packet timestamp is not set when function returns error.
also use C positive modulo for microseconds
2 years ago
Philippe Antoine 1660172a8b ftp: completely resets port_line
In the case port_line is first allocated and port_line_len is set,
Then a second request reaches memcap and frees port_line,
port_line_len should also be reset, because both will get used
by the response parsing.

Ticket: #5701
2 years ago
jason taylor 0632233791 userguide: update http.cookie description
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Victor Julien b79c14f710 flow-worker: prune frames and stream for pseudo packets as well 2 years ago