80 Commits (d720ead470bcb5dd5a0c0ae7db302ab170205ee6)

Author	SHA1	Message	Date
Victor Julien	d0f1507c83	htp: simplify streaming buffer config ... Use a single static config instead of the per profile config.	4 years ago
Victor Julien	e3e55406a7	files: update API and callers to take stream config ... This is to allow not storing the stream buffer config in each file.	4 years ago
Victor Julien	39cf5b151a	src: includes cleanup ... Work towards making `suricata-common.h` only introduce system headers and other things that are independent of complex internal Suricata data structures. Update files to compile after this. Remove special DPDK handling for strlcpy and strlcat, as this caused many compilation failures w/o including DPDK headers for all files. Remove packet macros from decode.h and move them into their own file, turn them into functions and rename them to match our function naming policy.	4 years ago
Victor Julien	79499e4769	app-layer: move files into transactions ... Update APIs to store files in transactions instead of the per flow state. Goal is to avoid the overhead of matching up files and transactions in cases where there are many of both. Update all protocol implementations to support this. Update file logging logic to account for having files in transactions. Instead of it acting separately on file containers, it is now tied into the transaction logging. Update the filestore keyword to consider a match if filestore output not enabled.	4 years ago
Victor Julien	e250ef6402	debug: remove empty header	4 years ago
Philippe Antoine	02f2602dde	src: rework includes as per cppclean	4 years ago
Victor Julien	9537d119b9	http: fix reassembled range file accounting	4 years ago
Victor Julien	6d30f4442c	http2: fix file accounting for ranged files ... Increment files_opened for tx that 'gets' reassembled ranged file	4 years ago
Philippe Antoine	bfcd6cb46a	range: validity check when end is bigger than size ... Ticket: 5132 Down the line, HttpRangeOpenFileAux assumes the range has a valid value when doing buflen = end - start + 1;	4 years ago
Philippe Antoine	334b1382e0	http: : fix int warnings ... Explicitly truncate file names to UINT16_MAX Before, they got implicitly truncated, meaning a UINT16_MAX + 1 file name, went to 0 file name (because of modulo 65536)	5 years ago
Jason Ish	7732efbec2	app-layer: include decoder events in app-layer tx data ... As most parsers use an events structure we can include it in the tx_data structure to reduce some boilerplate/housekeeping code in app-layer parsers.	5 years ago
Philippe Antoine	c023116857	range: prevents memory leak of file from HTTP2 ... If a HTTP2 transaction gets freed before the end of the range request, we need to have the files container which is in the state, to transfer owernship of this file to the files container. Ticket: 4811	5 years ago
Philippe Antoine	bba70607e8	range: checks that end is after start ... Otherwise, we end up allocating too much memory	5 years ago
Philippe Antoine	65a6f61004	http: delete obsolete range log ... Commit d776d72711 has been transfering ownership of file container So, we cannot log it	5 years ago
Philippe Antoine	98f84d5a9b	http2: follow range requests ... Move the content-range parsing code to rust	5 years ago
Philippe Antoine	3b9a4f9f19	range: no final zero in key ... for future compatibility with rust	5 years ago
Philippe Antoine	7819f3262f	http: more consistent return values for HTPFileOpenWithRange	5 years ago
Philippe Antoine	bb7cc6f5e3	http: increase tracked files for ranges as well	5 years ago
Philippe Antoine	3ed38d2d5d	http: avoid one lock for range append data ... Better structure design to ensure that one flow maximum is owning and appending into the file, adding fileOwning field. Adds also a gap field in a range buffer, so that we can feed the gap on closing, when we are protected from concurrency by a lock, (lock which got removed in the append path) Fixes memcap when encountering a duplicate while inserting in red and black tree Adds many comments	5 years ago
Victor Julien	7a797631e0	http/range: cleanup and simplification ... Simplify locking by using the THashData lock instead of a separate range lock. Avoid size_t in function arguments. Clean up file handling functions. Implement handling of alloc errors. Rename yaml entry to byterange Unify public api naming	5 years ago
Philippe Antoine	e82416a415	http/range: reassemble files from different flows with range ... adds a container, ie a thread safe hash table whose key is the filename keep a tree of unordered ranges, up to a memcap limit adds HTPFileOpenWithRange to handle like HTPFileOpen if there is a range : open 2 files, one for the whole reassembled, and one only for the current range	5 years ago
Victor Julien	d74c18ee28	http: support per-tx file accounting	5 years ago
Jeff Lucovsky	1eeb96696b	general: Cleanup bool usage	5 years ago
Philippe Antoine	707f027231	protos: renaming ALPROTO_HTTP* constants ... Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP Run with 3 sed commands git grep ALPROTO_HTTP | cut -d: -f1 | uniq | xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g' git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq | xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g' git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq | xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g' and then running clang-format	6 years ago
Victor Julien	b6b317cae6	http: enable and fix content range tests	6 years ago
Victor Julien	4b7599af90	http/file: modernize unittests ... Part of ticket #2975.	7 years ago
Jason Ish	ebcc4db84a	file extraction: always prune files after detect ... If a keyword like filemd5 was being used without a filestore, or a file output enabled, it would be pruned before detection had a chance to match. Consolidate file pruning to the end of the flow worker so files are available for detection even when a file output is not enabled. Redmine issue: https://redmine.openinfosecfoundation.org/issues/2490	7 years ago
Philippe Antoine	2d217e6661	http: fixes overflow in range parsing	7 years ago
Jeff Lucovsky	d568e7fadd	eve/logging: 2991 Optimize logging by TX ... This changeset makes changes to the TX logging path. Since the txn is passed to the TX logger, the TX can be used directly instead of through the TX id.	7 years ago
Philippe Antoine	a3e38861b9	http: adds unit tests for HTPParseContentRange	7 years ago
Philippe Antoine	90ab0b0ec2	http: fix overflow in HTPParseContentRange	7 years ago
Victor Julien	9132e4032a	files: open files with track id only	7 years ago
Philippe Antoine	bef190f767	http: logs content range ... Fixes #2485	7 years ago
Victor Julien	9ca71beb03	htp: remove usused file flags	9 years ago
Victor Julien	5c01b40931	tests: update tests for app-layer changes	9 years ago
Victor Julien	ab1200fbd7	compiler: more strict compiler warnings ... Set flags by default: -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wwrite-strings -Wcast-align -Wbad-function-cast -Wformat-security -Wno-format-nonliteral -Wmissing-format-attribute -funsigned-char Fix minor compiler warnings for these new flags on gcc and clang.	9 years ago
Eric Leblond	569cc5d238	util-file: introduce new functions for file size ... This patch introduces the FileDataSize and FileTrackedSize functions. The first one is just a renaming of the initial FilSize function whereas the other one is using the newly introduced size field as value.	10 years ago
Victor Julien	c81aaeda7b	flow: move file flags into their own variable ... Move FLOW_FILE_* flags into Flow::file_flags. Rename them to FLOWFILE_* so non updated code will break.	10 years ago
Giuseppe Longo	675fa56497	app-layer: add ThreadVars to AppLayerParserParse ... To be able to add a transaction counter we will need a ThreadVars in the AppLayerParserParse function. This function is massively used in unittests and this result in an long commit.	10 years ago
Victor Julien	6530c3d0d8	unittests: replace SCMutex* calls by FLOWLOCK_*	10 years ago
Victor Julien	4426f3ff55	file: introduce common flags handling function	10 years ago
Duarte Silva	89eb935f73	file-hashing: added support for SHA-256 file hashing	10 years ago
Duarte Silva	a6d928e269	file-hashing: added support for SHA-1 file hashing	10 years ago
Victor Julien	e43ce0a9ec	file: switch to streaming buffer API ... Make the file storage use the streaming buffer API. As the individual file chunks were not needed by themselves, this approach uses a chunkless implementation.	10 years ago
Victor Julien	46e55f1e34	http body handling: use streaming buffer API ... Convert HTTP body handling to use the Streaming Buffer API. This means the HtpBodyChunks no longer maintain their own data segments, but instead add their data to the StreamingBuffer instance in the HtpBody structure. In case the HtpBodyChunk needs to access it's data it can do so still through the Streaming Buffer API. Updates & simplifies the various users of the reassembled bodies: multipart parsing and the detection engine.	10 years ago
maxtors	9d3fd82849	Removed duplicate include statements.	10 years ago
Jason Ish	796dd5223b	tests: no longer necessary to provide successful return code ... 1 pass, 0 is fail.	10 years ago
Victor Julien	6b942b6734	htp file: constify name/data args	11 years ago
Victor Julien	136c6440c8	http file: fix txid type	11 years ago
Ken Steele	8f1d75039a	Enforce function coding standard ... Functions should be defined as: int foo(void) { } Rather than: int food(void) { } All functions where changed by a script to match this standard.	12 years ago