7316 Commits (efe11dc37e7f5be575981eefa50cb2891b62bb67)

Author	SHA1	Message	Date
Victor Julien	314516ffe2	stream: don't call app reassembly if disable flag set	10 years ago
Victor Julien	89af036336	stream: app-layer micro optimizations	10 years ago
Victor Julien	2f77302eeb	stream: raw reassembly explicit disable raw handling	10 years ago
Victor Julien	d6d7f65050	stream: mpm inspect micro optimizations	10 years ago
Victor Julien	7bddd0e168	stream: improve --disable-detection GAP handling	10 years ago
Victor Julien	6fefe70196	stream: remove unused StreamTcpGetStreamSize function	10 years ago
Victor Julien	422095668e	stream: optimize session pruning	10 years ago
Victor Julien	ee00a6f2ec	stream: validate code	10 years ago
Victor Julien	e1aba7d6c2	detect: only do flow dependent cleanup if a flow is present	10 years ago
Victor Julien	61c35d3c39	detect: make SigMatchSignatures void ... None of the callers cared for it's retval, so get rid of it.	10 years ago
Victor Julien	f49150ddb9	detect: turn single detect flag into bool	10 years ago
Victor Julien	6f76cbb870	detect: remove unused detect flag	10 years ago
Victor Julien	04b24cf24e	stream: improve needs reassembly code	10 years ago
Victor Julien	55e19bfb89	stream: more aggressive StreamReassembleRawHasDataReady	10 years ago
Victor Julien	bf3f3ce6b2	app-layer: change logic of setting 'no reassembly' ... Instead of killing all reassembly instantly do things slightly more gracefully: 1. disable app-layer reassembly immediately 2. flag raw reassembly not to accept new data This will allow the current data to be inspected still. After detect as run the raw reassembly will be fully disabled and thus all reassembly will be as well.	10 years ago
Victor Julien	de4f4e23a0	stream: new depth / disable raw logic ... Depth reach sets NOREASSEMBLY after detect. No new raw sets NORAW after detect.	10 years ago
Victor Julien	7c56c9ada0	stream: allow raw reassembly catch up ... If raw reassembly falls behind, for example because no raw mpm is active, then we need to sync up to the app progress if that is available, or to the generic tcp tracking otherwise.	10 years ago
Victor Julien	89d0267df2	stream: detect stream GAP also during reassembly	10 years ago
Victor Julien	0c1ec17c92	debug-validation: add stream checks	10 years ago
Victor Julien	69519bda48	stream: StreamTcpReassembleRawCheckLimit cleanup	10 years ago
Victor Julien	b099008b94	stream: handle no stream scanning case ... Now that detect moves the raw progress forward, it's important to deal with the case where detect don't consider raw inspection. If no 'stream' rules are active, disable raw. For this the disable raw flag is now per stream.	10 years ago
Victor Julien	0ef46a8fd2	stream: raw content inspection inline mode ... Implement the inline mode for raw content inspection. Packets are leading, and when a packet's payload has been added to the stream, the packet is inspected in the context of the stream. Reassembly will return a buffer with the packet data with older data in front of it and after it, if available.	10 years ago
Victor Julien	149e324060	flow/stream: reduce/disable pseudo packet injections ... At flow timeout, we no longer need to first run reassembly in one dir, then inspection in the other. We can do both in single packet now. Disable pseudo packets when receiving stream end packets. Instead call the app-layer parser in the packet direction for stream end packets and flow end packets. These changes in handling of those stream end packets make the pseudo packets unnecessary.	10 years ago
Victor Julien	2d223b69cd	stream: set 'trigger raw' per direction	10 years ago
Victor Julien	971ab18b95	detect / stream: new 'raw' stream inspection ... Remove the 'StreamMsg' approach from the engine. In this approach the stream engine would create a list of chunks for inspection by the detection engine. There were several issues: 1. the messages had a fixed size, so blocks of data bigger than ~4k would be cut into multiple messages 2. it lead to lots of data copying and unnecessary memory use 3. the StreamMsgs used a central pool The Stream engine switched over to the streaming buffer API, which means that the reassembled data is always available. This made the StreamMsg approach even clunkier. The new approach exposes the streaming buffer data to the detection engine. It has to pay attention to an important issue though: packet loss. The data may have gaps. The streaming buffer API tracks the blocks of continuous data. To access the data for inspection a callback approach is used. The 'StreamReassembleRaw' function is called with a callback and data. This way it runs the MPM and individual rule inspection code. At the end of each detection run the stream engine is notified that it can move forward it's 'progress'.	10 years ago
Victor Julien	564c0bd2c1	stream: constify StreamTcpReassembleRawCheckLimit	10 years ago
Victor Julien	0bff0de516	unittests: fail if TCP memory still in use ... abort() so test can be analyzed.	10 years ago
Victor Julien	807312320f	stream-tcp: implement thread pool for segments ... Config option: stream: reassembly: segment-prealloc: 2048	10 years ago
Victor Julien	bd821f57f2	stream: implement memory handling functions	10 years ago
Victor Julien	c2a5b9c393	stream: use static instead of dynamic streaming buffer structure	10 years ago
Victor Julien	dd2b8bb298	stream: test cleanups and fixes	10 years ago
Victor Julien	8924653cd4	stream: add insert failure counters	10 years ago
Victor Julien	f02dc377ef	stream: add tcp.overlap and tcp.overlap_diff_data counters	10 years ago
Victor Julien	8c36e52d93	stream: improve no app and no raw case	10 years ago
Victor Julien	5ee36a0c8b	stream: make raw_progress relative to STREAM_BASE_OFFSET	10 years ago
Victor Julien	fa2a832022	stream: make app_progress relative to STREAM_BASE_OFFSET	10 years ago
Victor Julien	187e2381c8	stream: reduce space used for progress tracking ... Instead of the explicit base_seq_offset, use a macro instead. The macro points to the stream buffer offset. The two were always in sync.	10 years ago
Victor Julien	9bd11bcda5	stream: small cleanups	10 years ago
Victor Julien	b3e9d39771	stream: remove unused zero copy setting	10 years ago
Victor Julien	bbb0df14d2	stream: safety check in overlap handling	10 years ago
Victor Julien	8c9f521707	tcp: streaming implementation ... Make stream engine use the streaming buffer API for it's data storage. This means that the data is stored in a single reassembled sliding buffer. The subleties of the reassembly, e.g. overlap handling, are taken care of at segment insertion. The TcpSegments now have a StreamingBufferSegment that contains an offset and a length. Using this the segment data can be retrieved per segment. Redo segment insertion. The insertion code is moved to it's own file and is simplified a lot. A major difference with the previous implementation is that the segment list now contains overlapping segments if the traffic is that way. Previously there could be more and smaller segments in the memory list than what was seen on the wire. Due to the matching of in memory segments and on the wire segments, the overlap with different data detection (potential mots attacks) is much more accurate. Raw and App reassembly progress is no longer tracked per segment using flags, but there is now a progress tracker in the TcpStream for each. When pruning we make sure we don't slide beyond in-use segments. When both app-layer and raw inspection are beyond the start of the segment list, the segments might not be freed even though the data in the streaming buffer is already gone. This is caused by the 'in-use' status that the segments can implicitly have. This patch accounts for that when calculating the 'left_edge' of the streaming window. Raw reassembly still sets up 'StreamMsg' objects for content inspection. They are set up based on either the full StreamingBuffer, or based on the StreamingBufferBlocks if there are gaps in the data. Reworked 'stream needs work' logic. When a flow times out the flow engine checks whether a TCP flow still needs work. The StreamNeedsReassembly function is used to test if a stream still has unreassembled segments or uninspected stream chunks. This patch updates the function to consider the app and/or raw progress. It also cleans the function up and adds more meaningful debug messages. Finally it makes it non-inline. Unittests have been overhauled, and partly moved into their own files. Remove lots of dead code.	10 years ago
Victor Julien	3fa2e8689c	streaming: remove BUG_ON and other improvements ... Can be triggered by memory limits.	10 years ago
Victor Julien	d789dc7e6d	streaming: add blocklist ... Add list of 'blocks'. This list contains offsets and lengths to continuous data blocks. This is useful for TCP tracking where we can have data gaps. The blocks don't contain any data themselves, instead they contain lenght and offsets. This way no extra copying is needed. On inserting new data, existing blocks are expanded instead of having multiple neighbouring blocks.	10 years ago
Victor Julien	9e1470d81c	stream-tcp: StreamTcpUTAddPayload unittest helper	10 years ago
Victor Julien	3411697106	profile: account flow-worker tcp-prune step	10 years ago
Victor Julien	595f6d1f26	detect: implement http_content_len sticky buffer ... This implements inspection of the Content-Length buffer as a content sticky buffer.	10 years ago
Victor Julien	d96cbddbe4	detect: implement http_content_type sticky buffer	10 years ago
Victor Julien	46febef7cd	detect: implement http_accept_enc sticky buffer ... Inspects Accept-Encoding header.	10 years ago
Victor Julien	9d5bbc3af6	detect: implement http_accept_lang sticky buffer ... Inspects Accept-Language header	10 years ago
Victor Julien	1d1176b0d5	detect: implement http_connection sticky buffer	10 years ago
Victor Julien	5714129e32	detect: implement http referer sticky buffer	10 years ago
Victor Julien	88cfb99910	detect: http_accept sticky buffer + common code ... Implement common code to easily add more per HTTP header detection keywords. Implement http_accept sticky buffer. It operates on the HTTP Accept header.	10 years ago
Ray Ruvinskiy	7539973109	tls: logging for session resumption ... We assume session resumption has occurred if the Client Hello message included a session id, we have not seen the server certificate, but we have seen a Change Cipher Spec message from the server. Previously, these transactions were not logged at all because the server cert was never seen. Ticket: https://redmine.openinfosecfoundation.org/issues/1969	10 years ago
Ray Ruvinskiy	0dc57c937e	device: fix warning about NULL device ... Fix '[ERRCODE: SC_ERR_INVALID_VALUE(130)] - Name of device should not be null' warning on start-up with pfring. Ticket: https://redmine.openinfosecfoundation.org/issues/2097	10 years ago
Victor Julien	29c8125758	redis: use SCCalloc to reduce risk of unitialized vars	10 years ago
fooinha	a64e5e77c7	eve: async mode for redis output ... eve: detects libevent for async redis at configure eve: moves redis output code to new file - util-log-redis.{c,h} eve: redis ECHO and QUIT commands for async mode eve: redis output defaults if conf is missing	10 years ago
Victor Julien	c3b4dd5a7d	qa: add --no-random commandline option	10 years ago
Victor Julien	535d9e3529	random: convert stream and htp to new call	10 years ago
Victor Julien	dd70b3fda0	random: improve random logic ... Improve random logic for hash tables. Implement Windows random API if it is available.	10 years ago
Victor Julien	ec964ebf84	mingw: don't use uint type as mingw doesn't have it	10 years ago
Victor Julien	8f15e30a58	common: improve byte order and wordsize detection	10 years ago
Victor Julien	a4dce24151	core dumps: check for sys/resource.h	10 years ago
Victor Julien	618f351210	cleanup: get rid of %llu format specifiers ... Use more explicit types instead.	10 years ago
Jason Ish	9ff8882cbd	detect: don't consider an empty rule file an error	10 years ago
Jason Ish	8436a892f9	logging: only do non-blocking writes if live ... If running against a pcap there is no reason to drop events, a blocking socket is fine here. So only do non-blocking writes when running off a live device.	10 years ago
Jason Ish	673549e5cf	eve: log number of events dropped at exit	10 years ago
Jason Ish	59b98649de	logging: don't block on socket writes ... Writing to a unix socket can cause Suricata to block in the packet path. This could happen if the read-endpoint of the unix socket stays connected, but stops reading, or simply can't read fast enough as part of its event processing. To choose packets over events, do non-blocking socket writes and drop the event if the write would block and update a dropped counter.	10 years ago
Mats Klepsland	ffbf8cec16	logging: create log directories when needed ... Recursively create new log directories when needed. This makes it possible to use date modifiers in the file path to create directories based on date, e.g.: /var/log/suricata/2017/02/14/	10 years ago
Mats Klepsland	47a5b493d7	output-json: rotate log file based on time ... Rotate log file based on time. Support both rotating based on a timer (XXs, XXm, XXd, XXw) and rotating based on a absolute time, like each minute, hour or day.	10 years ago
Mats Klepsland	db6c80fd8e	logging: support date modifiers in log filenames ... Allow log filenames to contain date modifiers, e.g.: - eve-log: filename: eve-%Y-%m-%d-%H:%M:%S.json	10 years ago
Jason Ish	845e18f25a	defrag: (windows) detect more overlaps	10 years ago
Jason Ish	7922f9be1b	defrag: (linux) fix an error in overlapping fragments ... If a subsequent fragment has a lower offset than a previous one and overlaps, trim off the beginning of the previous fragment. Based on an issue reported privately.	10 years ago
Jason Ish	ca92be896e	defrag: use new unit test macros ... Also reformat unit test functions to Suricata style.	10 years ago
Jason Ish	ff30d924db	unix-socket: return failure on failure ... UnixManagerThreadInit needs to return a failure code if the socket fails to initialize to avoid entering the UnixManager loop which will continuously fail on the call to bind, as no listening socket was setup. This can occur when the socket fails to initialize due to a permissions error and fatal init errors is not on.	10 years ago
fooinha	20d4d40051	log: tls custom format log	10 years ago
fooinha	af174c82bb	log: common custom format output	10 years ago
Victor Julien	d5a0eb4b2c	geoip: fix compiler warning ... detect-geoip.c:78:40: error: incompatible pointer types assigning to 'int (*)(ThreadVars *, DetectEngineThreadCtx *, Packet *, const Signature *, const SigMatchCtx *)' from 'int (ThreadVars *, DetectEngineThreadCtx *, Packet *, Signature *, const SigMatchCtx *)' [-Werror,-Wincompatible-pointer-types] sigmatch_table[DETECT_GEOIP].Match = DetectGeoipMatch; ^ ~~~~~~~~~~~~~~~~ 1 error generated. Bug #2045	10 years ago
Victor Julien	b134dfcce7	modbus: fix compiler warnings about alignment ... app-layer-modbus.c:1226:39: warning: taking address of packed member 'transactionId' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member] if (ModbusExtractUint16(modbus, &(header->transactionId), input, input_len, &offset) || ^~~~~~~~~~~~~~~~~~~~~ app-layer-modbus.c:1228:39: warning: taking address of packed member 'protocolId' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member] ModbusExtractUint16(modbus, &(header->protocolId), input, input_len, &offset) || ^~~~~~~~~~~~~~~~~~ app-layer-modbus.c:1230:39: warning: taking address of packed member 'length' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member] ModbusExtractUint16(modbus, &(header->length), input, input_len, &offset) || ^~~~~~~~~~~~~~ 3 warnings generated. Bug #2088	10 years ago
Victor Julien	e1bf48c0ee	pool: fix compiler warning ... clang-4.0 reported: util-pool.c:242:13: warning: logical not is only applied to the left hand side of this bitwise operator [-Wlogical-not-parentheses] if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ~ util-pool.c:242:13: note: add parentheses after the '!' to evaluate the bitwise operator first if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ( ) util-pool.c:242:13: note: add parentheses around left hand side expression to silence this warning if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ( ) util-pool.c:261:13: warning: logical not is only applied to the left hand side of this bitwise operator [-Wlogical-not-parentheses] if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ~ util-pool.c:261:13: note: add parentheses after the '!' to evaluate the bitwise operator first if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ( ) util-pool.c:261:13: note: add parentheses around left hand side expression to silence this warning if (! pb->flags & POOL_BUCKET_PREALLOCATED) { ^ ( ) 2 warnings generated.	10 years ago
Victor Julien	6227d0955f	af-packet: fix cppcheck false positive ... [src/source-af-packet.c:1903]: (error) Resource leak: fd	10 years ago
Victor Julien	3726fd66be	http: fix body tracking corner case ... In some cases, observed with inspect limits 0, the body tracking could get confused. When all chunks were already freed, a new chunk would be considered to be the start of the body. This would overwrite the bodies 'content_len_so_far' tracker, instead of adding to it. This in turn could lead to a assertion abort in the inspection code. This patch redoes the append code to always add the current lenght. It cleans up the code to remove redundant logic. Issue: https://redmine.openinfosecfoundation.org/issues/2078 Reported-By: Jørgen Bøhnsdalen	10 years ago
Jason Ish	3ca1a29bbd	defrag: fix argument used in macro to match signature ... "p" was being used in the macro but was not an argument to the macro, but it worked due to the context of the macro. Use the actual macro argument, d2, instead of p. Results in no change to generated code.	10 years ago
Mats Klepsland	365aa1f3e5	lua: add SCFlowTimestamps function ... Add SCFlowTimestamps() to return startts and lastts as seconds and microseconds from flow. Examples: startts, lastts = SCFlowTimestamps() startts_s, lastts_s, startts_us, lastts_us = SCFlowTimestamps()	10 years ago
Mats Klepsland	005a700e54	logging: support custom file permissions ... Support setting file permissions per logger using 'filemode', e.g.: outputs: - eve-log: enabled: yes filetype: regular filename: eve.json filemode: 660	10 years ago
Jason Ish	63078909d5	ipv4: update checksum function to be like tcp/udp ... Update the IPv4 checksum function to be like the changed TCP/UDP checksum functions for consistency.	10 years ago
Jason Ish	b79a18ea15	tcp/udp: rename checksum functions for better meaning ... The TCP/UDP checksum functions no longer just calculate the checksum, they can validate as well as calculate so use a more generic name.	10 years ago
Jason Ish	f56428d996	tcp/udp: fix checksum validation when 0xffff ... Issue: https://redmine.openinfosecfoundation.org/issues/2041 One approach to fixing this issue to just validate the checksum instead of regenerating it and comparing it. This method is used in some kernels and other network tools. When validating, the current checksum is passed in as an initial argument which will cause the final checksum to be 0 if OK. If generating a checksum, 0 is passed and the result is the generated checksum.	10 years ago
Victor Julien	b58127edcb	print: constify input	10 years ago
Victor Julien	44bab885d6	detect: fix ssl_state test	10 years ago
Victor Julien	4f8eacdc69	disable-detect: fix needless file hashing ... When detection is running flags are set on flows to indicate if file hashing is needed. This is based on global output settings and rules. In the case of --disable-detection this was not happening, so all files where hashed with all methods. This has a significant performance impact. This patch adds logic to set the flow flags in --disable-detect mode.	10 years ago
Victor Julien	e24eb0f2b1	app-layer: remove version logic	10 years ago
Victor Julien	a56b4817ef	flow: remove unused alversion fields	10 years ago
Victor Julien	c066f043a5	detect: remove unused alversion logic	10 years ago
Victor Julien	6c6802b957	detect: simplify state detect code: remove unused params	10 years ago
Victor Julien	c961da2dce	detect: clean up test	10 years ago
Victor Julien	3626ecb474	bytejump: don't print errors when matching ... When bytejump was told to convert some payload data to int from a string it would print an error to the screen if the conversion failed. This is unwanted as the payload is controlled by an attacker and printing is expensive.	10 years ago
Victor Julien	15d26f14e1	file-store: fix force store	10 years ago
Victor Julien	dbbf185173	app-layer: fix gap handling in protocol detection ... A GAP during protocol detection would lead to all reassembly getting disabled, so also the raw reassembly. In addition, it could prevent the opposing side from doing protocol detection. This patch remove the 'disable reassembly' logic. Stream engine will take the stream with GAP and app-layer will make the proto detection as complete.	10 years ago
Victor Julien	f1e19b5dd4	file store: store multiple files if available	10 years ago
Victor Julien	7c0a3b5582	app-layer: fix memleak on bad traffic ... If state was alloc'd after protocol detection, but then the direction turned out to be wrong, the state would not be freed.	10 years ago
Victor Julien	132e0e21f0	ippair: fix xbits unset memleak	10 years ago
Victor Julien	4c597bb9e1	ippair: use both addresses in hash	10 years ago
Victor Julien	1de1413863	stream: validate SACK right edge to be in window	10 years ago
Victor Julien	daf39b422b	stream: remove unused stream config member	10 years ago
Victor Julien	7512949670	stream: make data pointer in StreamSegmentCallback const	10 years ago
Victor Julien	2a5b9eb5fa	stream: remove unused variable	10 years ago
Victor Julien	a86b7b7042	app-layer: minor debug improvement	10 years ago
Jason Ish	6585ac4a9f	logging: remove unused print stats callbacks ... Remove the ThreadExitPrintStats callback from logging modules that weren't doing anything with it.	10 years ago
Jason Ish	0c3f1e2974	logging: move lock into write function ... All loggers were wrapping just the write in a lock with some updating a counter. This moves the lock into the write function. The log_ctx alerts counter was also removed as many modules have stopped using this and the alert count is available elsewhere. Should satisfy Coverity CID 1400798: CID 1400798 (#1 of 1): Data race condition (MISSING_LOCK) 2. missing_lock: Accessing log_ctx->rotation_flag without holding lock LogFileCtx_.fp_mutex. Elsewhere, "LogFileCtx_.rotation_flag" is accessed with LogFileCtx_.fp_mutex held 4 out of 5 times. Which appears to be a false positive as all calls to SCLogFileWrite were done under lock, but this will make it more explicit.	10 years ago
Jason Ish	ddf1bf6518	unified2: unlock using same dereference as lock ... Addresses Coverity CIDs: 1400797 1400796 Note that the mutex was actually being unlocked, but from a different variable pointing to the same mutex.	10 years ago
Mats Klepsland	d6508e640a	detect: add (mpm) keyword tls_cert_serial ... Match on TLS certificate serial number using tls_cert_serial keyword, e.g.: alert tls any any -> any any (msg:"TLS cert serial test"; tls_cert_serial; content:"5C:19:B7:B1:32:3B:1C:A1"; sid:12345;)	10 years ago
Mats Klepsland	95864375f2	lua: add function to print certificate serial number ... Add function LuaGetCertSerial to print serial number from TLS certificate. Example: function log (args) serial = TlsGetCertSerial() if serial then file:write(serial .. "\n"); file:flush() end end	10 years ago
Mats Klepsland	2e339aa791	tls-log: log certificate serial number	10 years ago
Mats Klepsland	2b460b8d06	output-json-lua: log certificate serial number	10 years ago
Mats Klepsland	2c1a36dd6e	app-layer-tls: decode certificate serial number	10 years ago
Alexander Gozman	310b27a196	af-packet: fix parsing packet in TPACKET_V3 mode ... AFPParsePacketV3() saved tpacket_block_desc structure instead of tpacket3_hdr. As a result, reconstructed packets were wrong. Bug #2044.	10 years ago
Alexander Gozman	e41a9d637b	af-packet: get VLAN info for packets in TPACKET_V3 mode	10 years ago
Alexander Gozman	ee7e689b54	af-packet: write VLAN info for both TPACKET_V2 and V3	10 years ago
Jason Ish	92db12c349	pidfile: fail if the pid file exists ... Changes the pidfile check function to fail just on the existence of the file to address issue https://redmine.openinfosecfoundation.org/issues/1335 but log a message if the pid file appears to be stale.	10 years ago
Mats Klepsland	bbd32c1419	output-json-tls: code cleanup	10 years ago
Mats Klepsland	115b3138cc	output-json-tls: log certificate and chain ... Log entire certificate and certificate chain Base64 encoded.	10 years ago
Mats Klepsland	0716199acb	output-json-tls: custom tls logging	10 years ago
Jason Ish	82f6103149	unified2: nostamp and file rotation ... Give unified2 a nostamp option which will create the file without the timestamp suffix (like Snort's nostamp option). Also register for rotation notification on SIGHUP so the file will be recreated if it is removed by an external rotation program (only when nostamp is used).	10 years ago
Jason Ish	2c01985e73	autoconf - look for stdbool.h	10 years ago
Jason Ish	d8e9e1fd77	unified2: minor cleanups ... - remove unused arguments and make static	10 years ago
Mats Klepsland	234a3a890b	output-json-alert: print 'tunnel' JSON object if tunnel ... Log src_ip, dst_ip and proto for root packet (p->root) if the packet that triggered is inside a tunnel, as JSON object 'tunnel'. Also log recursion depth to indicate the depth of the tunnel.	10 years ago
Mats Klepsland	7293286f2f	output-json: move code to get 5-tuple to own function ... Move code to get 5-tuple in JSON object to own function 'JsonFiveTuple'. This enables this code to be reused when printing 'parent' JSON object in output-json-alert.	10 years ago
Mats Klepsland	8c8db1cf0f	output-json-alert: fix wrongful comments	10 years ago
Mats Klepsland	65317ba865	output-json: make JSON flags in eve-log user configurable	10 years ago
Victor Julien	d445b4b5fa	dce: remove commented out code	10 years ago
Victor Julien	9da7be81f0	flow-worker: clean up thread init	10 years ago
Victor Julien	c859d39f4f	coverity: suppress CID 1400648	10 years ago
Victor Julien	955c227127	detect-ssh: cleanup duplicate code	10 years ago
Victor Julien	2f30adb08a	detect-lua: setup cleanup, fixing a potential int issue	10 years ago
Victor Julien	511e804915	detect: fix missing unlock in error path	10 years ago
Victor Julien	f74eff9eac	threads: address sleep under lock issue	10 years ago
Victor Julien	f380871057	threads: don't sleep under lock	10 years ago
Victor Julien	cc4010343d	detect: add and use util func for alproto sets	10 years ago
Victor Julien	4a49260897	flowvar: shrink flowvar type by using padded space	10 years ago
Victor Julien	99517cbd53	lua: support key/value flowvars in lua	10 years ago
Victor Julien	f0af133c5f	flowvar: remove unused DETECT_VAR_TYPE_ALWAYS	10 years ago
Victor Julien	88888c3d8b	unix-socket: add/list/remove hostbit commands ... add-hostbit adds a named hostbit with an expire time in seconds. remove-hostbit removes hostbit by name. add-hostbit, remove-hostbit return success or failure. list-hostbit returns a json array of hostbits with their name and expire time: { "message": { "count": 1, "hostbits": [{ "expire": 3222, "name": "firefox-users" }] }, "return": "OK" }	10 years ago
Victor Julien	b6e4276792	hostbits: add list API	10 years ago
Victor Julien	996112edf5	pktvars: same name pktvars, key-value vars	10 years ago
Victor Julien	5ca4a2e6fe	outputs: vars log ... EVE addition called 'vars' that logs pkt/flow vars for each packet/flow.	10 years ago
Victor Julien	1a2ad059a1	eve: log pktvars/flowvars/bits/ints ... Optionally logs 'vars' into alerts	10 years ago
Victor Julien	1ba8c2fe3a	pcre: new way of specifying var names ... Until now the way to specify a var name in pcre substring capture into pkt and flow vars was to use the pcre named substring support: e.g. /(?P<pkt_somename>.*)/ This had 2 drawbacks: 1. limitations of the name. The name could be max 32 chars, only have alphanumeric and the underscore characters. This imposed limitations that are not present in flowbits/ints. 2. we didn't actually use the named substrings in pcre through the API. We parsed the names separately. So putting the names in pcre would actually be wasteful. This patch introduces a new way of mapping captures with names: pcre:"/(.*)/, pkt:somename"; pcre:"/([A-z]+) ([0-9]+)/, pkt:somename,flow:anothername"; The order of the captures and the order of the names are mapped 1 on 1. This method is no longer limited by the pcre API's naming limits. The 'flow:' and 'pkt:' prefixes indicate what the type of variable is. It's mandatory to specify one. The old method is still supported as well.	10 years ago
Victor Julien	0f708d427b	pkt-var: abuse flowvar postmatch logic for pktvars ... Flowvars were already using a temporary store in the detect thread ctx. Use the same facility for pktvars. The reasons are: 1. packet is not always available, e.g. when running pcre on http buffers. 2. setting of vars should be done post match. Until now it was also possible that it is done on a partial match.	10 years ago
Victor Julien	5e39486399	pkt-var: use id instead of name pointer	10 years ago
Victor Julien	a0bd15a1c4	pcre: support multiple captures ... Support up to 8 substring captures into pkt or flow vars.	10 years ago
Victor Julien	017b16d421	detect-pcre: small cleanups	10 years ago
Victor Julien	ac42a44280	alert-debug: print flowvar/int names	10 years ago
Victor Julien	e95a0c1344	alert-debug: print flowbit names from VarNameStore	10 years ago
Victor Julien	22f3205664	var-names: expose outside of detect engine ... Until now variable names, such as flowbit names, were local to a detect engine. This made sense as they were only ever used in that context. For the purpose of logging these names, this needs a different approach. The loggers live outside of the detect engine. Also, in the case of reloads and multi-tenancy, there are even multiple detect engines, so it would be even more tricky to access them from the outside. This patch brings a new approach. A any time, there is a single active hash table mapping the variable names and their id's. For multiple tenants the table is shared between tenants. The table is set up in a 'staging' area, where locking makes sure that multiple loading threads don't mess things up. Then when the preparing of a detection engine is ready, but before the detect threads are made aware of the new detect engine, the active varname hash is swapped with the staging instance. For this to work, all the mappings from the 'current' or active mapping are added to the staging table. After the threads have reloaded and the new detection engine is active, the old table can be freed. For multi tenancy things are similar. The staging area is used for setting up until the new detection engines / tenants are applied to the system. This patch also changes the variable 'id'/'idx' field to uint32_t. Due to data structure padding and alignment, this should have no practical drawback while allowing for a lot more vars.	10 years ago
Victor Julien	43cc06eabe	detect: use engine version instead of id ... Use engine version based on global detect engine master. This is incremented between reloads.	10 years ago
Victor Julien	920709fe6f	detect: ssh_software sticky buffer	10 years ago
Victor Julien	f1ab6a6153	detect: ssh_proto stickybuffer	10 years ago
Victor Julien	dfac5276b8	detect: remove unused SIGMATCH_PAYLOAD flag	10 years ago
Victor Julien	14ced15e36	detect: remove unused state file flag	10 years ago
Victor Julien	fa1ef158b2	detect: small API cleanup	10 years ago
Victor Julien	073fcbeb7f	detect: move file hash common code	10 years ago
Victor Julien	5bafc64c08	detect: unify FileMatch API with other calls	10 years ago
Victor Julien	fe415ae518	detect: remove DMATCH list	10 years ago
Victor Julien	1c02cf4542	flow: remove unused Flow::de_state	10 years ago
Victor Julien	ad238121e3	detect: remove the AMATCH list	10 years ago
Victor Julien	775e182531	detect: remove AppLayerMatch API call	10 years ago
Victor Julien	f018ae94b0	dce: dynamic lists	10 years ago
Victor Julien	84ba9cf9df	smb/dcerpc: use tx api	10 years ago
Victor Julien	d318bfc934	dcerpc: simplify common detect code	10 years ago
Victor Julien	402eb645a0	ftp: parser and ftpbounce update ... Convert parser to TX API. Convert ftpbounce keyword to use that.	10 years ago
Victor Julien	d9a300cd8c	detect: move lua smtp support to dynamic list	10 years ago
Victor Julien	96b8100a51	lua: convert lua output to be tx aware	10 years ago
Victor Julien	a10b2fdecf	detect: make ssh detection use dynamic list	10 years ago
Victor Julien	c412352474	ssh: remove single logger limit	10 years ago
Victor Julien	3ee4989ba7	ssh: convert app-layer parser to be tx aware ... Like with SSL, there is only a single 'tx'.	10 years ago
Victor Julien	4ae4fd0802	lua: use tls_generic list for ssl/tls	10 years ago
Victor Julien	a8975c68e0	detect ssl/tls: use dynamic lists	10 years ago
Victor Julien	1cacba435b	ssl/tls: clean up keywords	10 years ago
Victor Julien	6ef4712cd6	detect: convert old tls keywords to dynamic list	10 years ago
Victor Julien	ef149bed7b	tls.store: convert to postmatch	10 years ago
Victor Julien	8b3a71a285	tls.store: cleanup	10 years ago
Victor Julien	8eac1156c6	app-layer-events: remove unused API options	10 years ago
Victor Julien	1087495d6d	detect: http_start sticky buffer ... Matches on the start of a HTTP request or response. Uses a buffer constructed from the request line and normalized request headers, including the Cookie header. Or for the response side, it uses the response line plus the normalized response headers, including the Set-Cookie header. Both buffers are terminated by an extra \r\n.	10 years ago
Victor Julien	36535efa04	detect: add http_protocol sticky buffer ... Matches on protocol field in HTTP.	10 years ago
Victor Julien	f2fc5a255f	http_header: convert to use common code	10 years ago
Victor Julien	6279ec399e	http_header: common detection code	10 years ago
Victor Julien	5ee68ca2b2	http_header: remove old files	10 years ago
Victor Julien	2bb0cae05a	http_header: move all code into keyword files	10 years ago
Victor Julien	aaeeae0722	detect: http_header_names sticky buffer keyword ... A sticky buffer that allows content inspection on a contructed buffer of HTTP header names. The buffer starts with \r\n, the names are separated by \r\n and the end of the buffer contains an extra \r\n. E.g. \r\nHost\r\nUser-Agent\r\n\r\n The leading \r\n is to make sure one can match on a full name in all cases.	10 years ago
Victor Julien	cf9678d926	detect: global registery for keyword thread data ... Some keywords need a scratch space where they can do store the results of expensive operations that remain valid for the time of a packets journey through the detection engine. An example is the reconstructed 'http_header' field, that is needed in MPM, and then for each rule that manually inspects it. Storing this data in the flow is a waste, and reconstructing multiple times on demand as well. This API allows for registering a keyword with an init and free function. It it mean to be used an initialization time, when the keyword is registered.	10 years ago
Victor Julien	75907fce06	profiling: output all sort options for rules ... Limit the default number of sids to 10.	10 years ago
Victor Julien	7d8a5a75ef	profiling: honor limit in json rule output	10 years ago
Victor Julien	a9a228a289	profiling: fix keyword profiling	10 years ago
Victor Julien	a1465bc4fa	detect-engine-mpm: api cleanup	10 years ago
Victor Julien	8bd1422948	detect: detect engine registration cleanup	10 years ago
Victor Julien	49fbd28ceb	detect: cleanup built-in list id's	10 years ago
Victor Julien	40851eecf0	template: dynamic buffer	10 years ago
Victor Julien	815120896b	app-layer-events: dynamic list	10 years ago
Victor Julien	b68343e372	files: use dynamic list	10 years ago
Victor Julien	e4bfdd53c2	cip/enip: dynamic buffer	10 years ago