6627 Commits (bac37fc9ae5e3469652fda2ef268de617de485dd)

Author	SHA1	Message	Date
Jason Ish	8b38b9d728	output.[ch]: consistent style ... - Clean up function declaration. - Consistenly use typedefs for function points. No functional changes.	9 years ago
Jason Ish	fa27a76462	logging: add profiling back for non-tmm loggers ... The loggers moved away from a TMM required a new profiling support.	9 years ago
Jason Ish	42b8f30272	logging: convert lua output to non-thread module	9 years ago
Jason Ish	7a0737b9a9	logging: convert tls log to non-thread module	9 years ago
Jason Ish	7cb16bc90d	logging: convert alert debug log to non-thread module	9 years ago
Jason Ish	7a8e8343e5	logging: convert tcp data logging to non-thread module	9 years ago
Jason Ish	4d8b8ca046	logging: convert tls store logging to non-thread module	9 years ago
Jason Ish	60b6ccc3c4	logging: convert file data logging to non-thread module	9 years ago
Jason Ish	f9bb9029c5	logging: convert file logging to non-thread module	9 years ago
Jason Ish	669827ae16	logging: convert unified2 to non-thread module	9 years ago
Jason Ish	b580016c80	logging: convert stats loggers to non-thread module	9 years ago
Jason Ish	9475c83713	logging: convert http log to non-thread module	9 years ago
Jason Ish	e00dcd52a0	logging: convert alert syslog to non-thread module	9 years ago
Jason Ish	869d2eb701	logging: convert drop output to non-thread module	9 years ago
Jason Ish	5bbb4fd134	logging: convert json template output to non-thread module	9 years ago
Jason Ish	b605984f34	tests: setup unit test framework earlier ... Allows tests to be registered early, in support of moving outputs away from thread modules.	9 years ago
Jason Ish	bac65f09e8	logging: convert json drop output to non-thread module	9 years ago
Jason Ish	38354479b7	logging: convert json smtp output to non-thread module	9 years ago
Jason Ish	3fea12d7b3	logging: convert json ssh output to non-thread module	9 years ago
Jason Ish	01cc508257	logging: convert json netflow output to non-thread module	9 years ago
Jason Ish	983a619ff0	logging: convert json flow output to non-thread module	9 years ago
Jason Ish	ad15ac8297	logging: convert json alert output to non-thread module	9 years ago
Jason Ish	aaa65f3d16	logging: convert json tls output to non-thread module	9 years ago
Jason Ish	31663f1627	logging: convert prelude output to non-thread module	9 years ago
Jason Ish	dedda33f01	logging: convert eve http to non-thread module	9 years ago
Jason Ish	687602c0ca	logging: convert eve dns logging to non-thread module	9 years ago
Jason Ish	b1200dba54	logging: convert fast log to a non-thread module	9 years ago
Jason Ish	637aa34610	logging: convert dns log to a non-thread module	9 years ago
Victor Julien	99dce740ef	detect: mark alproto in keyword reg deprecated ... No existing code uses it, and it had been useless for some time.	9 years ago
Victor Julien	9030e89c94	detect: don't set alproto while registering keyword ... The field is not used except for some printing, and is wrong for many keywords.	9 years ago
Victor Julien	c957c62824	detect file: enable HTTP inspection from validate func	9 years ago
Victor Julien	621860f5b2	detect file: enforce protocol in single place ... Instead of trying to enforce the app layer protocol in each file function, enforce it in the generic validation function.	9 years ago
Victor Julien	bcfa484bce	app-layer: add function to check if app-layer supports files	9 years ago
Victor Julien	85db260eed	threads: remove EngineKill & SURICATA_KILL ... EngineStop and EngineKill were effectively doing the same, so removed the kill variant.	9 years ago
Victor Julien	045c10db43	threads: failed thread is a fatal error now	9 years ago
Victor Julien	fb655d5f15	threading: remove thread restart logic ... Thread restarts never worked well and the rest of the engine was never really expecting errors to lead to thread restarts. Either and error is recoverable in the thread, or not at all. So this patch removes the functionality completely.	9 years ago
Victor Julien	54503ef310	Open Suricata 3.2 development branch	9 years ago
Victor Julien	471b61a0e1	magic: fix broken tests after CentOS6 update	9 years ago
Victor Julien	82282a9e68	mpls: add missing event type + rule	9 years ago
Victor Julien	71c8d1f46c	bpf: fix file parsing memory handling ... Fix improper fread string handling. Improve error handling. Skip trailing spaces for slightly more pretty printing. Coverity CID 400763. Thanks to Steve Grubb for helping address this issue.	9 years ago
Victor Julien	519b2970ec	detect: don't print (null) in --list-keywords=all	9 years ago
Eric Leblond	ed90a16e89	detect: fix setup for some keywords ... Fix problems found by siginit.cocci.	9 years ago
Jason Ish	17e70483c5	detect-flowbits: more unittest macro usage ... Also cleanup some tests by removing extra code after a test was determined to fail.	9 years ago
Jason Ish	3c5d8e65d4	hostbits: use new unittest macros	9 years ago
Jason Ish	c4945607e3	hostbits: fail parse on unexpected trailing data ... Address issue https://redmine.openinfosecfoundation.org/issues/1889 for hostbits. This involves updating the regular expresssion to capture any trailing data as the regex already keeps spaces out of the name. A unit test was converted to new macros to find out which line it was failing at after updating regex.	9 years ago
Jason Ish	24f2387b23	flowbits: validate that there are no spaces in the name ... Fixes issue: https://redmine.openinfosecfoundation.org/issues/1889 To catch the issue where the ';' is missing we have to expand the regex to capture the whole name string, not just the leading valid stuff. Then verify that there are no spaces in the name (Snort has the same restriction) and fail if there is.	9 years ago
Eric Leblond	1cdd062dc6	unix-manager: fix output of version command ... Make it consistent with the output of version command line flag.	9 years ago
Andreas Herz	65fd09a399	rule-parsing: reject unescaped double quote within content section	9 years ago
Victor Julien	2997d086be	eve-drop: allow logging all drops ... - drop: alerts: yes # log alerts that caused drops flows: all # start or all: 'start' logs only a single drop # per flow direction. All logs each dropped pkt.	9 years ago
Victor Julien	1cc5f9825d	dns: use nonnull attr for log functions	9 years ago
Victor Julien	bbcc22d2ad	dns: fix coverity warning ... ** CID 1372324: Null pointer dereferences (FORWARD_NULL) /src/output-json-dns.c: 532 in OutputAnswer() ________________________________________________________________________________________________________ *** CID 1372324: Null pointer dereferences (FORWARD_NULL) /src/output-json-dns.c: 532 in OutputAnswer() 526 } 527 } 528 529 /* reset */ 530 MemBufferReset(aft->buffer); 531 json_object_set_new(djs, "dns", js); >>> CID 1372324: Null pointer dereferences (FORWARD_NULL) >>> Dereferencing null pointer "entry". 532 if (likely(DNSRRTypeEnabled(entry->type, aft->dnslog_ctx->flags))) { 533 OutputJSONBuffer(djs, aft->dnslog_ctx->file_ctx, &aft->buffer); 534 } 535 json_object_del(djs, "dns"); 536 537 return; Move checks to the top of the functions. Should be more efficient too.	9 years ago
Victor Julien	b4565004c7	detect-template: modernize	9 years ago
Jason Ish	e878dd2231	app-layer templates: cleanups ... - cleanup file headers - add todo section - convert unit tests to new macros - add markers to remove disabled by default behaviour	9 years ago
Jason Ish	3cf8b4629f	decode-icmpv6: use FAIL macros in tests	9 years ago
Jason Ish	2a42e8be03	unittest: FAIL macro to unconditionally fail a test	9 years ago
Jason Ish	af4085b77b	icmpv6: fix checksum verification if fcs present ... Calculate the length of the ICMPv6 packet from decoded information instead of off the wire length. This will provide the correct length if trailing data like an FCS is present. Fixes issue: https://redmine.openinfosecfoundation.org/issues/1849	9 years ago
Victor Julien	120f59386b	affinity: fix compilation on SunOS	9 years ago
Victor Julien	a2c9b86cdf	byteswap: fix compilation on SunOS	9 years ago
Victor Julien	ef1acdfaee	threads: provide SCGetThreadIdLong for SunOS	9 years ago
Victor Julien	4271d57157	decode: declare IPPROTO_IPIP if OS doesn't have it	9 years ago
Victor Julien	6956c1c749	decode: fix int types	9 years ago
Victor Julien	ec87123339	configure: check for strings.h: used by SunOS	9 years ago
Victor Julien	8600872e02	logfile: resolve name clash on SunOS	9 years ago
Victor Julien	b81ea0d7db	eve: reduce flow_id to 51 bits ... Evebox & ELK couldn't handle the large integers. It looks like (partly) a javascript limitation that doesn't treat 64bit ints as real ints.	9 years ago
Victor Julien	9ca34fa5c9	eve: output more unique flow_id	9 years ago
Victor Julien	78b4db8a95	flow: introduce function to generate flow id ... The flow id itself is not stored in the flow, but generated based on properties that do not change during the lifetime of the flow. As it's meant for use with the json output, it is limited to a signed 64 bit integer (int64_t) because that is the time json_integer_t uses.	9 years ago
Victor Julien	666bba8121	detect: implement continue detect for dcepayload ... Also fix a corner case in start detection. Bug 1853.	9 years ago
Victor Julien	ecf4a2862c	detect: cleanup	9 years ago
Victor Julien	6b078e4f51	detect: fix ICMP error handling issue ... The first packet in both directions of a flow looks up the rule group (sgh) and stores it in the flow. This makes sure the lookup doesn't have to be performed for each packet. ICMPv4 error messages are connected to the TCP or UDP flow they apply to. In the case of such an ICMP error being the first packet in a flow's direction, this would lead to an issue. The packet would look up the rule group based on the ICMP protocol, not based on the embedded TCP/UDP. This makes sense, as the ICMP packet is inspected as ICMP packet. The consequence however, was that this rule group pointer (sgh) would be stored in the flow. This is wrong, as TCP/UDP packets that follow the ICMP packet would have no sgh or the wrong sgh. In normal traffic this shouldn't normally happen, but it could be used to evade Suricata's inspection.	9 years ago
Victor Julien	2eb941f9d9	output dns: fix bit declarations	9 years ago
Victor Julien	5c6ffe5653	common: introduce macro for bit declarations	9 years ago
Tom DeCanio	0f6c8806a0	output-json-dns: dns output filtering.	9 years ago
Jason Ish	1691c10681	eve: make logging of tagged packets optional ... But it is enabled in the default configuration.	9 years ago
Jason Ish	040660556e	eve: log tag packets as packet events ... Create a new eve event type, "packet" for logging packets that are tagged as part of an event. The packet is still at the top level to keep it consistent with alert event types. In addition to the packet being logged, a packet_info object is created to hold the linktype and any future meta data we may want to add about the packet.	9 years ago
Victor Julien	305b1b90fd	detect: minor cleanup	9 years ago
Victor Julien	ec0217f52c	detect: minor style fixes	9 years ago
Victor Julien	7d11af16ef	detect: minor debug output cleanup	9 years ago
Victor Julien	0e2ea4e63b	detect: remove unused debug code	9 years ago
Victor Julien	b79d9cda3a	detect: minor cleanups	9 years ago
Victor Julien	199bb3bae3	stream-tcp: fix ssn returning to wrong thread pool	9 years ago
Jason Ish	cf61472619	app-layer-dcerpc-udp: style cleanups ... - consistent 4 space indent - cleanup file header	9 years ago
Jason Ish	e55334fb37	detect-flowbits: fix misleading indentation ... detect-flowbits.c: In function ‘FlowBitsTestSig02’: detect-flowbits.c:475:4: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if(error_count == 5) ^~ detect-flowbits.c:478:5: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’ SigGroupBuild(de_ctx); ^~~~~~~~~~~~~	9 years ago
Jason Ish	a975fdcfeb	app-layer-dcerpc-udp: fix missleading indentation ... app-layer-dcerpc-udp.c: In function ‘DCERPCUDPParserTest01’: app-layer-dcerpc-udp.c:1105:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if (alp_tctx != NULL) ^~ app-layer-dcerpc-udp.c:1107:2: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’ StreamTcpFreeConfig(TRUE); ^~~~~~~~~~~~~~~~~~~	9 years ago
Jason Ish	95015a3f6d	decode: support Cisco Fabric Path / DCE ... Cisco Fabric Path is ethernet wrapped in an ethernet like header with 2 extra bytes. The ethernet type is in the same location so the ethernet decoder can be used with some validation for the extra length.	9 years ago
Victor Julien	a8da6bbd71	output: use safer logic for fingerprint printing	9 years ago
Jason Ish	d3c0135eec	app-layer-tls: accomodate trailing \0 in hash output	9 years ago
Jason Ish	73a0451070	output-json-dns: allocate correct size hexstring buffer ... The buffer allocated for the hexstring was not large enough for a ':' separated hex string.	9 years ago
Victor Julien	b3b78d4326	detect: log earlier that rule reload is happening	9 years ago
Victor Julien	731d4a7049	dns: fix OOB read on malformed TXT record	9 years ago
Jason Ish	7e6ce01600	unified2: fix logging of tagged packets ... The structure for create the alert preceding each tagged packet was not being initialized, preventing tagged packets from being logged. Note: Snort unified2 does not precede tagged packets with an alert like is done here, so this just fixes what the code intended to do, it does not make it Snort unified2 compatible. Address issue: https://redmine.openinfosecfoundation.org/issues/1854	9 years ago
Victor Julien	26e67400ba	dns: fix name parsing issue leading to events	9 years ago
Victor Julien	884fddf035	packet: remove empty and unused UDPVars struct	9 years ago
Victor Julien	c9756caeef	packet: make tcp/udp/icmp vars union non-anonymous ... Clean the whole thing after use.	9 years ago
Victor Julien	79388df887	commandline: fix strlcpy usage	9 years ago
Jason Ish	2403af5177	pcap: don't fail with --pcap with no device present ... Issue: 1856. A device with the name of "" (empty string) was being added with LiveRegisterDevice which failed to initialize causing Suricata to fail.	9 years ago
Victor Julien	2856dfd119	output lua: improve debugging output	9 years ago
Victor Julien	a26e59cb6d	output lua: set proper logging progress values	9 years ago
Victor Julien	7188c2630f	outputs: small code cleanup	9 years ago
Jason Ish	f397e7bfc2	dns: directional logging ... Register loggers for to server and to client so requests and responses can be logged independently of each other. This results in the request log having the actual timestamp of the request instead of the reply.	9 years ago
Jason Ish	fcad270d96	logging: setup all registered loggers for a name ... When setting up a configured logger, do so for all registered loggers of that name instead of just the first registered one. This allows a logger to register itself more than once, which can allow for independent logging of requests and replies without touching the core transaction handling logic. We do this so just having "dns" in the eve-log can configured multiple "dns" loggers instead of having something like "dns-tc" and "dns-ts" in the configuration file.	9 years ago
Victor Julien	9d01ef58fc	lua smtp: fix SMTPGetMimeField arg checking ... Properly check argument before passing it on: CID 1363385: (NULL_RETURNS)	9 years ago
Victor Julien	2b10b8374c	cmdline: fix --list-keywords and --list-app-layer-protos ... Ticket #1840	9 years ago
Victor Julien	ed483b4e13	output: don't register loggers for disabled protocols	9 years ago
Victor Julien	01913f6a56	app-layer: add AppLayerParserIsTxAware ... This function globally checks if the protocol is registered and enabled by testing for the per alproto callback: StateGetProgressCompletionStatus This check is to be used before enabling Tx-aware code, like loggers.	9 years ago
Victor Julien	f302a6cf86	output: fix debug messages	9 years ago
Victor Julien	b73098e990	smb: style fix in log message	9 years ago
Victor Julien	3bb408940f	af-packet: improve threads selection logic ... Only use RSS queue count when cluster_qm is used. Only use core count when cluster_flow is used. Use a local variable to simplify the check so that we don't have to deal with the extra flags.	9 years ago
Victor Julien	da8f9c1896	lua: add smtp for detection	9 years ago
Victor Julien	928cb1eba9	lua output: expose smtp functions to output scripts	9 years ago
Victor Julien	7501bf744f	lua: SMTPGetRcptList use position as key, not value	9 years ago
tobiass1	7581f5129f	Lua: SMTP support; Addresses feature ticket #1775; v5	9 years ago
Victor Julien	ff3baeee90	lua: support smtp tx logging	9 years ago
Victor Julien	5e4d071b76	lua-output: don't crash on script setup error	9 years ago
Victor Julien	3c59d60049	cuda: make sure we don't use cuda in proto detect	9 years ago
Victor Julien	4111331ab0	af-packet: minor cleanups	9 years ago
Victor Julien	402bdf9b2b	af-packet: test if fanout is supported before use ... Older system may pretend they can support FANOUT but then fail to work at runtime. CentOS6 is an example of this. It would fail to start up with the default configuration with errors like: [15770] 21/6/2016 -- 16:00:13 - (tm-threads.c:2168) <Notice> (TmThreadWaitOnThreadInit) -- all 4 packet processing threads, 4 management threads initialized, engine started. [15785] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available [15785] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error [15770] 21/6/2016 -- 16:00:13 - (suricata.c:2664) <Notice> (main) -- Signal Received. Stopping engine. [15787] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available [15788] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available [15786] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available [15789] 21/6/2016 -- 16:00:13 - (flow-manager.c:693) <Perf> (FlowManager) -- 0 new flows, 0 established flows were timed out, 0 flows in closed state [15787] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error [15788] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error [15786] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error This patch adds a test that if run before the number of threads is determined. If the test fails, only 1 thread is created.	9 years ago
Victor Julien	ab65b6f83b	netmap: fix coverity warning 1362789 ... ** CID 1362789: Null pointer dereferences (FORWARD_NULL) /src/runmode-netmap.c: 247 in ParseNetmapConfig() ________________________________________________________________________________________________________ *** CID 1362789: Null pointer dereferences (FORWARD_NULL) /src/runmode-netmap.c: 247 in ParseNetmapConfig() 241 strlcpy(aconf->iface_name, iface_name, sizeof(aconf->iface_name)); 242 SC_ATOMIC_INIT(aconf->ref); 243 (void) SC_ATOMIC_ADD(aconf->ref, 1); 244 245 /* Find initial node */ 246 netmap_node = ConfGetNode("netmap"); >>> CID 1362789: Null pointer dereferences (FORWARD_NULL) >>> Comparing "netmap_node" to null implies that "netmap_node" might be null. 247 if (netmap_node == NULL) { 248 SCLogInfo("Unable to find netmap config using default value"); 249 } else { 250 if_root = ConfFindDeviceConfig(netmap_node, aconf->iface_name); 251 if_default = ConfFindDeviceConfig(netmap_node, "default"); 252 }	9 years ago
Andreas Herz	e9a2a341ce	util-threshold-config: parse suppress rules with spaces in ip list ... This modified regex allows spaces witihn the ip list for supress rules like [10.0.0.1, 10.0.0.2]	9 years ago
Jason Ish	f0e22c91cb	privs: add capability CAP_SYS_NICE. ... Allows the setting of thread priorities after dropping privileges.	9 years ago
Victor Julien	6045420812	detect: reduce verbosity, don't warn on empty files	9 years ago
Victor Julien	46ac5ed7b7	pfring: move output to 'Perf' level	9 years ago
Victor Julien	18de4c9654	offloading: work around missing TOE support	9 years ago
Victor Julien	9b80c21d78	offloading: distinguish between csum and the rest ... As AF_PACKET handles csum offloading don't check for this type of offloading. Other methods like pcap and netmap do require it to be turned off. Improve disable command suggestion wording.	9 years ago
Victor Julien	03d46f1369	offloading: reduce verbosity	9 years ago
Victor Julien	507027845d	afpacket: update offloading warning	9 years ago
Victor Julien	45fa25eb0c	offloading: improve checks on FreeBSD ... Move FreeBSD specific (but not netmap specific) checks from the netmap code to the general ioctl wrapper code. Warn from the check functions now, so callers no longer need to.	9 years ago
Victor Julien	33f8769001	offloading: move linux specific into their own func	9 years ago
Victor Julien	54bc471810	offloading: check for more offloading on Linux	9 years ago
Victor Julien	b1d191b478	netmap: fix enabling promisc mode on FreeBSD ... In FreeBSD setting the IFF_PROMISC flag has no effect. Instead we need to set the IFF_PPROMISC flag.	9 years ago
Victor Julien	6c7bf006b7	netmap: redo config parsing ... Normally we parse the config per interface only. But to properly setup the bridge, netmap also needs the config of it's peering interface. Instead of using a complicated peering scheme like in afpacket, simply parse the peers config too.	9 years ago
Andreas Herz	d0baa83d2b	util-runmode: pass initdata to runmode workers for nfqueue ... The VerdictNFQ was missing the initdata which results in a segfault within CaptureStatsSetup. This commit adds the passing of the initdata.	9 years ago
Victor Julien	f7124b1149	afpacket: disable tpacket-v3 by default ... It's still considered experimental at this point.	9 years ago
Victor Julien	66346e4632	libnet: work around older libnet type difference ... Older libnet 1.1.x have a non-const type for libnet_init's dev argument.	9 years ago
Victor Julien	a88359dcf0	detect: get proper legacy custom values. Issue #1804	9 years ago
Victor Julien	5c974f92a8	livedev: shorten devname at registration	9 years ago
Victor Julien	b673e14411	afl: fix various --afl-* options	9 years ago
Victor Julien	e76b334f8d	http body: fix compression tests	9 years ago
Victor Julien	5ec885e451	http: set of response body decompress limit ... This is a per personality setting.	9 years ago
Victor Julien	ed7dc0c6b3	unittest: minor cleanup	9 years ago
Victor Julien	b313f8ca7b	http: update compression mismatch test	9 years ago
Andreas Herz	36e4126227	detect-filemagic: fix heap-use-after-free ... This fixes the heap-use-after-free issue with sm being freed without being removed from the signature (s) list. Move the protocol check for rules with filemagic before the alloc and make the error log more precise.	9 years ago
Victor Julien	a309598721	netmap: work around mtu error on iface+ settings	9 years ago
Victor Julien	648a69759b	netmap: don't set more than 1 thread on sw ring	9 years ago
Victor Julien	86d44cea96	netmap: code cleanup	9 years ago
Victor Julien	b5633b9bfd	affinity: small cleanups to output & code	9 years ago
Victor Julien	5f9de1e734	affinity: rename detect -> worker set internall	9 years ago
Victor Julien	723e90a174	affinity: rename detect-cpu-set to worker-cpu-set ... Add fallback for existing configs.	9 years ago
Victor Julien	570b9d06e0	affinity: remove unused settings ... These were never referenced to in the code so they can be removed. Add bypass to config parser in case the settings are still in old yamls.	9 years ago
Victor Julien	bdc2c6e9ce	affinity: type cleanup	9 years ago
Victor Julien	2aac437927	output: reduce verbosity on info level	9 years ago
Victor Julien	4c663bb143	netmap: don't check for offloading twice	9 years ago
Victor Julien	2beb39469b	netmap: output cleanup	9 years ago
Victor Julien	b3bf7a5729	output: introduce config and perf output levels ... Goal is to reduce info output	9 years ago
Victor Julien	cc2ed783c5	output: improve notice and warning/error color handling	9 years ago
Victor Julien	e11753e3f2	profiling: fix minor compiler warning	9 years ago
Victor Julien	4369161b92	netmap: get offloading settings and warn if needed ... Add FreeBSD lookup function and use the existing code on Linux.	9 years ago
Victor Julien	a8a918545b	netmap: get correct RSS queues on Linux as well	9 years ago
Victor Julien	d861bf100b	netmap: reduce verbosity at startup	9 years ago
Victor Julien	d58d02fed5	netmap: handle missing config with better defaults ... Default to 'threads: auto' which uses RSS RX count when no config has been created for a interface.	9 years ago
Victor Julien	f446577412	netmap: implement 'threads: auto' ... Add until function for retrieving RSS RX count from netmap. Use the RSS count to create the threads.	9 years ago
Victor Julien	d39e5754e6	instance: use enum for runmode	9 years ago
Victor Julien	2412681eff	instance: memset to 0 before use	9 years ago
Eric Leblond	4defc5acc2	util-ioctcl: increase header size ... Headers can contain VLAN or Qing so we need to increase the value returned by GetIfaceMaxHWHeaderLength.	9 years ago
Andreas Herz	ed561c73a5	suricata: fix double packet processing threads ... With the additional ParseInterfacesList the packet processing threads were doubled since the Interface was included twice unless the device was passed via the commandline with af-packet=IF. The additonal ParseInterfacesList isn't necessary so remove it again	9 years ago
Victor Julien	371113e21e	ac-ks: don't allow use on big-endian	9 years ago
Victor Julien	181f67ff97	flow-worker: small cleanups	9 years ago
Victor Julien	72d3ea6552	detect: make pattern matcher messages less verbose	9 years ago
Victor Julien	36535cbc61	yaml: remove conf_filename global ... conf_filename was a global pointer to the filename of the yaml. Move into SCInstance. This reduces it's scope and cleans up the code.	9 years ago
Victor Julien	4b9a62d1fe	profiling: fix compilation if libjansson is missing	9 years ago
Victor Julien	661d7c1d09	pfring: cleanup	9 years ago
Victor Julien	6f7740807d	pfring: improve profiling ... Reset packet profiling after pfring_recv. The packet was taken from the packet pool before this call. The packet will already have it's start ticks initialized. To avoid including ticks while pfring_recv waits for traffic, reset the ticks right after it.	9 years ago
Victor Julien	c9159892c7	profiling: allow packet profiling to be reset	9 years ago
Eric Leblond	291af719c6	coverity: fix CID 1362014 ... Error handling was not correct regarding ring buffer memory handling.	9 years ago
Victor Julien	9f7ba07153	af-packet: use better defaults if config is missing	9 years ago
Victor Julien	093ecf4798	logging: clean up at shutdown	9 years ago
Victor Julien	c1f679d3f3	flow worker: move UDP app-layer into main function ... This way it's more clean what happens and we can profile it.	9 years ago
Victor Julien	e09643c396	flow worker: profiling ... Previously the detect and stream code lived in their own thread modules. This meant profiling showed their cost as part of the thread module profiling logic. Now that only the flow worker is a thread module this no longer works. This patch introduces profiling for the 3 current flow worker steps: flow, stream, detect.	9 years ago
Victor Julien	48771c1acf	debug: fix compiler warnings	9 years ago
Victor Julien	7dfdcdc770	thread modules: remove unused id's	9 years ago
Victor Julien	a8f257e05f	detect: no longer a thread module ... Like stream, detect is now invoked directly by the FlowWorker.	9 years ago
Victor Julien	4a96820320	stream-tcp: more cleanups	9 years ago
Victor Julien	8b06badbcf	stream-tcp: no longer register as a thread module ... Now that the FlowWorker handles the TCP Stream directly, having the TCP engine as a thread module is no longer needed. This patch removes the registration.	9 years ago
Victor Julien	eec66c7b4f	smtp: improve thread data use ... The SMTP app layer used a thread local data structure for the mpm in reply parsing, but it only used a pmq. The MpmThreadCtx was actually global. Until now this wasn't really noticed because non of the mpm's used the thread ctx. Hyperscan does use it however. This patch creates a new structure SMTPThreadCtx, which contains both the pmq and the mpm thread ctx. It's passed directly to the reply parsing function instead of storing a pointer to it in the SMTPState. Additionally fix a small memory leak warning wrt the smtp global mpm state.	9 years ago
Justin Viiret	7a0dbc6f9f	app-layer-smtp: free mpm contexts on shutdown ... Adds a cleanup function for the SMTP parser that destroys the MPM context and MPM thread context it uses. Also marks smtp_mpm_thread_ctx static.	9 years ago
Justin Viiret	d807bf4e8a	detect-engine: log MPM/SPM matchers being used	9 years ago
Justin Viiret	c9d0d6f698	mpm: add "auto" default for mpm-algo ... Setting mpm-algo to "auto" will use "hs" if Suricata was built against Hyperscan, and "ac" otherwise (or "ac-tile" on Tilera platforms).	9 years ago
Justin Viiret	8c6deecc55	app-layer-detect-proto: use mpm-algo ... Use the matcher configured by the user rather than hard-coding MPM_AC.	9 years ago
Justin Viiret	88b50d2c34	app-layer-detect-proto: pass mpm_ctx to DestroyCtx ... The MPM DestroyCtx function accepts the MpmCtx, not the ctx pointer inside it.	9 years ago
Justin Viiret	31d8d4b0a1	detect-engine: adjust unit tests for hs mpm ... The Hyperscan MPM does match deduplication internally (using HS_FLAG_SINGLEMATCH) and only returns the number of unique matches, unlike AC.	9 years ago
Justin Viiret	68ddcdccde	app-layer-smtp: init mpm thread ctx after prepare ... This allows the Hyperscan MPM to correctly allocate scratch.	9 years ago
Justin Viiret	24a1488591	mpm-hs: make errors from hs_scan() fatal ... Hyperscan will only return an error at scan time if the database or scratch region are corrupted, which should provoke a fatal error.	9 years ago
Justin Viiret	a765cfde19	mpm-hs,spm-hs: don't call hs_scan() for zero bytes	9 years ago
Aleksey Katargin	2a5f487a16	netmap: close sw ring before hw rings ... Fix issue #1714	9 years ago
Jason Ish	b23d74ac88	tls-json-log: register module as tls-json-log, not dns-json-log ... Fixes issue: https://redmine.openinfosecfoundation.org/issues/1792 where dns-json-log would not log any data.	9 years ago
Victor Julien	5e7f617b7b	isdataat: remove unused code	9 years ago
Arturo Borrero Gonzalez	221cb93024	src/: fix typo: receieved vs received ... Reported by Debian's lintian tool. Signed-off-by: Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com>	9 years ago
Victor Julien	ea23b85776	flow worker: set up decoder thread vars	9 years ago
Victor Julien	6286e70555	ac: allow use of 31bits of pid space instead of 16	9 years ago
Victor Julien	1334859379	dns: add support for sshfp records ... Update parser to process the records. Update json output to log it.	9 years ago
Victor Julien	1df5acb001	coverity: CID 1362011: Control flow issues (DEADCODE)	9 years ago
Victor Julien	213f041c97	coverity: CID 1362012: Incorrect expression (EVALUATION_ORDER)	9 years ago
Victor Julien	3ffd19bdf5	coverity CID 1362013: Control flow issues (NESTING_INDENT_MISMATCH)	9 years ago
Victor Julien	f947539d79	af-packet: CentOS6 build fixes	9 years ago
Eric Leblond	49612128f3	af-packet: use time() instead of GetTime() ... As we only use the second we don't need GetTime() which is slower and get us milliseconds.	9 years ago
Eric Leblond	88f5d7d166	af-packet: print errno on mmap error	9 years ago
Eric Leblond	a40f08a213	af-packet: ask for hardware timestamp	9 years ago
Eric Leblond	8035d83467	af-packet: make mmap options parsing conditional ... Only parse them if mmap is activated.	9 years ago
Eric Leblond	7fea0ec6f9	af-packet: reset stats at start of capture ... We can loose packets during setup because we are reading nothing. So it is logical to discard the counter at start of capture to start from a clean state. This means we don't need to account the drop at start. But the stats call that will reset the drop counts will also return and reset the packets count. So we need to know how many packets we really have. This is in fact the number of packets coming from the stats call minus the number of discarded packets and the drop count. All the other packets will have to be read.	9 years ago
Eric Leblond	876b356bbe	af-packet: use mmap capture by default ... Update the code to use mmap capture by default even in unset in configuration file. mmap capture is now be turned off by using explicitely 'use-mmap: no' in configuration.	9 years ago
Eric Leblond	c2d0d93806	af-packet: detect availability of tpacket_v3 ... If TPACKET_V3 is not defined then it is not available and we should not build anything related to tpacket_v3. This will allow us to activate it dy default and fallback to v2 if not available.	9 years ago
Eric Leblond	f5c2019167	af-packet: add option to use memory locked mmap	9 years ago
Eric Leblond	234aefdff9	af-packet: configurable tpacket_v3 block timeout ... Block timeout defines the maximum filling duration of a block.	9 years ago
Eric Leblond	fa902abedf	af-packet: configurable tpacket_v3 block size ... It is used to set the block size in tpacket_v3. It will allow user to tune the capture depending on his bandwidth. Default block size value has been updated to a bigger value to allow more efficient wlak on block.	9 years ago
Eric Leblond	c7bde9dff6	af-packet: put ring setup in a separate function	9 years ago
Eric Leblond	7fa963718f	af-packet: pack AFPPeer structure	9 years ago
Eric Leblond	5f84b55d98	af-packet: AFPWalkBlock error handling ... Error handling was not done. The implementation is making the choice to consider we must detroy the socket in case of parsing error. The same was done for tpacket_v2.	9 years ago
Eric Leblond	b797fd926c	af-packet: continuing cleaning and hole hunting ... Suppress useless fields in AFPThreadVars. This patch also get rid of bytes counter as it was only used to display a message at exit. Information on livedev and on packet counters are enough.	9 years ago
Eric Leblond	9500d12c9f	af-packet: cleaning and hole hunting ... Reorder fields in AFPThreadVars and suppress some that were not used elsewhere than in the initialization.	9 years ago
Eric Leblond	bae1b03cf5	af-packet: tpacket_v3 implementation ... This patch adds a basic implementation of AF_PACKET tpacket v3. It is basic in the way it is only working for 'workers' runnning mode. If not in 'workers' mode there is a fallback to tpacket_v2. Feature is activated via tpacket-v3 option in the af-packet section of Suricata YAML.	9 years ago
Eric Leblond	d094039600	af-packet: remove useless code ... No need for cooked header in the case of mmap capture.	9 years ago
Eric Leblond	27adbfa868	af-packet: micro optimization	9 years ago
Eric Leblond	5f400785c8	af-packet: avoid test for each packet	9 years ago
Justin Viiret	f77bc5195c	spm: handle null ptrs in destroy funcs gracefully ... This will handle minimal DetectEngineCtx structures (used in delayed detect mode) safely, since they don't get SPM global contexts allocated. Also added BUG_ON checks for valid spm_table entries.	9 years ago
Victor Julien	e43ce0a9ec	file: switch to streaming buffer API ... Make the file storage use the streaming buffer API. As the individual file chunks were not needed by themselves, this approach uses a chunkless implementation.	9 years ago
Victor Julien	e836a750c8	http: improve body inspection ... Enforce inspect window also in IDS mode. Try always to get at least 'inspect win' worth of data. In case there is more new data, take some of the old data as well to make sure there is always some overlap. This unifies IDS and IPS modes, the only difference left is the start of inspection. IDS waits until min_size is available, IPS starts right away.	9 years ago
Victor Julien	feafc838db	http: make htpstate cfg ptr const	9 years ago
Victor Julien	24a2f51569	http: move body settings into per dir struct	9 years ago
Victor Julien	6fb808fc1a	http: add per direction config for body parsing ... The HTPCfgDir structure is meant to contain config for per direction body parsing parameters. This patch stores the streaming API config.	9 years ago
Victor Julien	46e55f1e34	http body handling: use streaming buffer API ... Convert HTTP body handling to use the Streaming Buffer API. This means the HtpBodyChunks no longer maintain their own data segments, but instead add their data to the StreamingBuffer instance in the HtpBody structure. In case the HtpBodyChunk needs to access it's data it can do so still through the Streaming Buffer API. Updates & simplifies the various users of the reassembled bodies: multipart parsing and the detection engine.	9 years ago
Victor Julien	81b2984c4e	streaming: buffer API ... Add a new API to store data from streaming sources, like HTTP body processing or TCP data. Currently most of the code uses a pattern of list of data chunks (e.g. TcpSegment) that is reassembled into a large buffer on-demand. The Streaming Buffer API changes the logic to store the data in reassembled form from the start, with the segments/chunks pointing to the reassembled data. The main buffer storing the data slides forward, automatically or manually. The *NoTrack calls allows for a segmentless mode of operation. This approach has two main advantages: 1. accessing the reassembled data is virtually cost-free 2. reduction of allocations and memory management	9 years ago
Victor Julien	78ecfe8780	autofp: update queue handlers ... Now that the flow lookup is done in the worker threads the flow queue handlers running after the capture thread(s) no longer have access to the flow. This limits the options of how flow balancing can be done. This patch removes all code that is now useless. The only 2 methods that still make sense are 'hash' and 'ippair'.	9 years ago
Victor Julien	61ce05e7ed	flow: remove dead code	9 years ago
Victor Julien	52d500c670	flowworker: initial support ... Initial version of the 'FlowWorker' thread module. This module combines Flow handling, TCP handling, App layer handling and Detection in a single module. It does all flow related processing under a single flow lock.	9 years ago
Victor Julien	408948815f	detect: simplify flow locking ... To simplify locking, move all locking out of the individual detect code. Instead at the start of detection lock the flow, and at the end of detection unlock it. The lua code can be called without a lock still (from the output code paths), so still pass around a lock hint to take care of this.	9 years ago
Victor Julien	6f560144c1	time: improve offline time handling ... When we run on live traffic, time handling is simple. Packets have a timestamp set by the capture method. Management threads can simply use 'gettimeofday' to know the current time. There should never be any serious gap between the two or major differnces between the threads. In offline mode, things are dramatically different. Here we try to keep the time from the pcap, which means that if the packets are recorded in 2011 the log output should also reflect this. Multiple issues: 1. merged pcaps might have huge time jumps or time going backward 2. slowly recorded pcaps may be processed much faster than their 'realtime' 3. management threads need a concept of what the 'current' time is for enforcing timeouts 4. due to (1) individual threads may have very different views on what the current time is. E.g. T1 processed packet 1 with TS X, while T2 at the very same time processes packet 2 with TS X+100000s. The changes in flow handling make the problems worse. The capture thread no longer handles the flow lookup, while it did set the global 'time'. This meant that a thread may be working on Packet 1 with TS 1, while the capture thread already saw packet 2 with TS 10000. Management threads would take TS 10000 as the 'current time', considering a flow created by the first thread as timed out immediately. This was less of a problem before the flow changes as the capture thread would also create a flow reference for a packet, meaning the flow couldn't time out as easily. Packets in the queues between capture thread and workers would all hold such references. The patch updates the time handling to be as follows. In offline mode we keep the timestamp per thread. If a management thread needs current time, it will get the minimum of the threads' values. This is to avoid the problem that T2s time value might already trigger a flow timeout as the flow lastts + 100000s is almost certainly meaning the flow would be considered timed out.	9 years ago
Victor Julien	2f0e0f17db	flow: move flow handling into worker threads ... Instead of handling the packet update during flow lookup, handle it in the stream/detect threads. This lowers the load of the capture thread(s) in autofp mode. The decoders now set a flag in the packet if the packet needs a flow lookup. Then the workers will take care of this. The decoders also already calculate the raw flow hash value. This is so that this value can be used in flow balancing in autofp. Because the flow lookup/creation is now done in the worker threads, the flow balancing can no longer use the flow. It's not yet available. Autofp load balancing uses raw hash values instead. In the same line, move UDP AppLayer out of the DecodeUDP module, and also into the stream/detect threads. Handle TCP session reuse inside the flow engine itself. If a looked up flow matches the packet, but is a TCP stream starter, check if the ssn needs to be reused. If that is the case handle it within the lookup function. Simplies the locking and removes potential race conditions.	9 years ago
Victor Julien	ae7aae81dc	flow: get flow reference during lookup ... Update Flow lookup functions to get a flow reference during lookup. This reference is set under the FlowBucket lock. This paves the way to not getting a flow lock during lookups.	9 years ago
Victor Julien	a81766c046	detect: split detect entry into flow/noflow ... This is a preparation for flow locking updates.	9 years ago
Mats Klepsland	a13df67864	detect: add (mpm) keyword for tls_sni ... Match on server name indication (SNI) extension in TLS using tls_sni keyword, e.g: alert tls any any -> any any (msg:"SNI test"; tls_sni; content:"example.com"; sid:12345;)	9 years ago
Jason Ish	3da79610af	typo: SURCATA -> SURICATA	9 years ago
Victor Julien	b77d307272	ipv6: simplify ext hdr parsing	9 years ago
Victor Julien	abb0a31aed	defrag: work around packet creation issues ... Defrag tests set up packets but don't call Decode on them. Work around failing IPv6 tests.	9 years ago
Victor Julien	68c7fae79f	ipv6: simplify ext hdr parsing and storage ... This reduces size of the IPV6ExtHdr structure part of every packet significantly. Clean up macro's in the ipv6 header.	9 years ago
Victor Julien	64405ae194	detect-ipopts: optimize matching	9 years ago
Victor Julien	3ab7dfd988	detect-ipopts: cleanup	9 years ago
Victor Julien	6ea0db2f60	ipv4: removed unused variables	9 years ago
Victor Julien	8c37906cf9	ipv4: store ipopts as flags, not bools	9 years ago
Victor Julien	be5a5df1f7	ipv4: shrink per packet ipopts storage	9 years ago
Justin Viiret	91011b30a6	spm: add "spm-algo: auto" setting ... This will default to Hyperscan when Suricata is built with Hyperscan support. Otherwise, Boyer-Moore is used by default.	9 years ago
Justin Viiret	6a6d019245	spm: add Hyperscan implementation	9 years ago