151 Commits (285b4dd981c3d62997477f94fe267ccc7e525fc6)

Author	SHA1	Message	Date
Eric Leblond	9ca7257279	nfq: suppress unused functions.	15 years ago
Eric Leblond	aee2e3ddd6	nfq: Add autofp mode support	15 years ago
Eric Leblond	115c3499d2	nfq: factorize auto mode	15 years ago
Eric Leblond	830ca7c2c8	source-nfq: suppress insecable space. ... This patch supresses an insecable space and fixes an indentation.	15 years ago
Eric Leblond	bbd04fde30	NFQ: fix race condition at exit. ... A race condition was observed when leaving NFQ. This was caused by the queue handle being accessed after been nullified. This patch uses the handle mutex to protect the destruction and adds tests on nullity to avoid crashed.	15 years ago
Anoop Saldanha	3f1c4efceb	Add new flags var to tm module. TMs can now set flags to identify special properties. Also use these to identify receive TMs	15 years ago
Victor Julien	c590bba4a4	Undo tunnel reference counting using atomic operations. Revert to mutex based code.	15 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	15 years ago
Eric Leblond	a354034cfc	nfq: Fix deinit phase ... If receive thread is failling, we need to restart it but the code was not restarting the queue (this was done in verdict thread).	15 years ago
Eric Leblond	eddcedba0a	nfq: make thread abort if NFQ verdict fail	15 years ago
Eric Leblond	2ffcef0a8e	nfq: Add iterator on nfq_set_verdict ... This patch adds retry to nfq_set_verdict in case of error.	15 years ago
Eric Leblond	bc68c108a7	NFQ: use per thread allocated data for recv buffer.	15 years ago
Victor Julien	52eb8d2be0	Convert mutex protected tunnel counters to lockless atomic counters.	15 years ago
Eric Leblond	ee6552f25e	nfq_set_mark: handle feature in NFQ. ... This patch implements the nfq_set_mark related modification of verdict handling.	16 years ago
Eric Leblond	04f2afa81b	nfq: fix exit function ... Exit function was trying to close the nfq handler even if it was null. This was causing a crash.	16 years ago
Victor Julien	477bc1d050	Set DROP flag on a packet in addition to the REJECT flags. This makes sure we not only send a reject, but also drop the offending packet. Closes #248.	16 years ago
Victor Julien	d424ac7c61	Fix nfq lockup due to improper handling of PKT_PSEUDO_STREAM_END packets.	16 years ago
Victor Julien	c9f9e3f9a4	Add configure check for signed or unsigned nfq_get_payload, adapt code.	16 years ago
Eric Leblond	c74116949c	source-nfq: improve nfq option system ... This patch modifies the NFQ option system to avoid implicit choice. 'nfq.mode' is now a string which can take a value in the 'accept', 'repeat' and 'route' set. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	94596ff282	source-nfq: add queue redirect support ... This patch adds the support of queue redirect. If 'next_queue' variable is set, the verdict sent to kernel is modified to contain the indication of a queue number (equal to 'next_queue') which will receive the packet after the verdict. This feature can be used to chain easily tools using NFQUEUE. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	aded7b4fae	source-nfq: add detection of already treated packet. ... This patch adds detection of already treated packet. If a packet is coming with an already set mark, it will be accepted and the processing of the packet is aborted. The patch display a message when the problem occurs but the number of message is limited to a fix counter in a way to avoid flooding the log. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	e399e74fc1	source-nfq: Factorize buffer usage ... A big sized buffer was allocated at each packet parsing. This patch uses a per-thread variable to have a persistent memory usage. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	1e600c1054	source-nfq: add simulated non-terminal NFQUEUE verdict ... This patch adds a new mode for NFQ inline mode. The idea is to simulate a non final NFQUEUE rules. This permit to do send all needed packets to suricata via a simple FORWARD rule: iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE And below, we have a standard filtering ruleset. To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and put a mark ($MARK) with respect to a mask ($MASK) on the handled packet. NF_REPEAT verdict has for effect to have the packet reinjected at start of the hook after the verdict. As it has been marked by suricata during the verdict it will not rematch the initial rules and make his way to the following classical ruleset. Mode, mark and mask can be configured via suricata.yaml file with the following syntax: nfq: repeat_mode: (false|true) mark: $MARK mask: $MASK Default is false to preserve backward compatibility. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	72ec56ab23	source-nfq: autodetection of queue max length function ... Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	1375e90030	Prepare multi queue support in NFQ ... This patch prepare support for multiqueue in the source file. The NFQ vars contained in Packet structure has a new member. It is a reference to the NFQ thread var it comes from. The behaviour is modified as a single verdict thread treat packet for all Netfilter queues. Locking is done in the verdict function to ensure that simultaneous modifications of counters can not occur. Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Eric Leblond	96e3852191	source-nfq: add define of SOL_NETLINK ... Signed-off-by: Eric Leblond <eric@regit.org>	16 years ago
Victor Julien	929ce0bb9b	Add a counter to NFQ for modified packets.	16 years ago
Victor Julien	1dca88fe69	Do the actual checksum recalculation and packet replacement on modifing a packet in the stream engine.	16 years ago
Victor Julien	acc38c9ebf	Make sure we don't try to 'verdict' the fake PKT_PSEUDO_STREAM_END packets.	16 years ago
Eric Leblond	89558ab9a4	RFC: modify error treatment in PacketCopyData ... Hello Victor This patch modifies error treatment following our discussion on IRC. It tries to follow the error treatment guideline I've been able to read in the different files. I will merge this patch in the original commit if the error treatment seems ok for you. BR, Eric	16 years ago
Eric Leblond	156b202597	Fix decode part of source-nfq	16 years ago
Eric Leblond	dd038c1906	Modify files to avoid direct pckt payload access ... This patch implements the needed modification of payload access in a Packet structure to support the abstraction introduced by the extended data system.	16 years ago
Eric Leblond	f73c60b4ff	nfq: set some options on netlink socket ... This patch modify nfq system to set some options on the netlink socket. This should improve performances by handling more correctly capacity overrun.	16 years ago
Victor Julien	4cacb1e970	Disable adding to unregistered mbit/s counter.	16 years ago
Victor Julien	6519a86ec7	Move packet pool to ringbuffer, update packet pool api and ringbuffer api. Remove memset usage from PACKET_RECYCLE, add proper cleanup macros.	16 years ago
Victor Julien	dce323b1f4	Fix SCondWait -> SCCondWait typo.	16 years ago
Pablo Rincon	8cc525c939	UDP support at AppLayer message handling	16 years ago
Victor Julien	4e7df60b2f	Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this.	16 years ago
William Metcalf	2eef905c07	GPL and Copyright header updates.	16 years ago
William Metcalf	b629b7c5c1	only show cli opts via help that we have support for	16 years ago
Gerardo Iglesias Galvan	9f4fae5b1a	Fix inconsistent use of dynamic memory allocation	16 years ago
Victor Julien	bb685751d9	Fix NFQ receive/verdict race condition in cases where the packetpool is empty.	16 years ago
Victor Julien	647b1c0eba	Fix NFQ compilation.	16 years ago
William Metcalf	ce01927515	Import of GPLv2 Header 050410	16 years ago
Victor Julien	070ed778b8	Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added.	16 years ago
Victor Julien	591c898f6a	NFQ inline changes ... - split NFQRecvPkt into a Linux and Windows version for readability - Add many function header comments - convert tabs to spaces in Windows code.	17 years ago
Jan Jezek	366671a8ce	Added inline mode support on Windows	17 years ago
Victor Julien	eeb98c6900	Move SCSetThreadName to proper functions.	17 years ago
Gerardo Iglesias Galvan	9f35a24a1f	Set threads name. Fix bug #83	17 years ago
Pablo Rincon	25a3a5c6d8	Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks.	17 years ago
Victor Julien	16b6f536a0	Fixup Linux compilation after applying win32 patches.	17 years ago
Gurvinder Singh	5293681860	b86	17 years ago
Victor Julien	2cb2989ad8	Apply configurable max pending packets to nfq and ipfw	17 years ago
Pablo Rincon	ad2c136e8f	Renaming errors (naming conventions)	17 years ago
Gurvinder Singh	cf5266094d	bug 66 patch	17 years ago
Victor Julien	df4c642c70	Fix weird compile error	17 years ago
Pablo Rincon	51dc773eec	Changing the veredict actions to flags to allow simultaneous veredict	17 years ago
Eric Leblond	51be576a30	nfq: modify queue length computation logic ... This patch modifies max queue length computation logic. The max queue length was set to MAX_PENDING which is the total number of packet processed simultaneously in suricata. This value is correct but this will not permit to take all burst effects into account (read sudden quantity of packet that arrives faster than suricata is enable to parse). Furthermore there is a delaying system when suricata gets overloaded which make necessary to have packet storable into kernel for some time. To improve this situation the patch increases the maximum queue length to NFQ_BURST_FACTOR (4) time the MAX_PENDING packet and it also increase the nfnetlink buffer size to be able to store all packets waiting for suricata in the netlink receive buffer.	17 years ago
Eric Leblond	72d48f6658	nfq: add sanity checking ... This patch adds sanity checking to payload handling. It set length of packet to zero if an error occurs.	17 years ago
Eric Leblond	56cccdfa62	nfq: use switch instead of 'else if' ... This patch convert a 'else if' serie to a switch to increase the readability of the decision related code. Please enter the commit message for your changes. Lines starting	17 years ago
Victor Julien	13e10ccd86	Enable bytes per sec and mbit per sec for nfq as well	17 years ago
Victor Julien	ecf86f9c23	Rename to Suricata.	17 years ago
Victor Julien	aa653157bf	Convert stats printing in nfq to logging api.	17 years ago
Victor Julien	e5301effe6	Make nfq module use logging api.	17 years ago
Pablo Rincon	e26833be3f	Changing mutex/spinlocks/conditions naming types	17 years ago
Pablo Rincon	769022f4be	Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests	17 years ago
Anoop Saldanha	8beef4a9fc	stats upgrade. Added interval counters to the decoder module	17 years ago
Anoop Saldanha	ceb7e495ae	refactoring perf stats code	17 years ago
Gurvinder Singh	e7952b1331	some changes in threading constants	17 years ago
Gurvinder Singh	40b8afdd56	support for thread exit constants	17 years ago
Victor Julien	f0d556b9e3	Suppress nfq debug output.	17 years ago
Victor Julien	a4fe971897	Fix datalink retrieval for pcap file mode and nfq mode for use in unified2.	17 years ago
Victor Julien	0d0ffb9963	Reorganize header inclusions.	17 years ago
William Metcalf	07bb501aca	compile fix for source-nfq.c	17 years ago
Victor Julien	a39108843e	Small tm module API rename to reflect that Init/Deinit/ExitPrintStats are per thread calls.	17 years ago
Victor Julien	e7f8a02484	Improve GRE counting. Actually use the GRE decoder. Register the GRE counter.	17 years ago
Victor Julien	57f71f7e4b	Pass the DecodeThreadVars to all Decoder functions properly. Improve the error handling.	17 years ago
Anoop Saldanha	244f5d547a	new registration functions for the stats api, with local thread storage for counter ids	17 years ago
Victor Julien	e31563f61c	Fix nfq compilation.	17 years ago
Brian Rectanus	fa5939ca91	64 bit cleanup part2	17 years ago
Anoop Saldanha	8af9f902e8	additional support for type qualifier for the stats api	17 years ago
Victor Julien	175eaeca93	Slightly moved around the NFQ define a bit.	17 years ago
William Metcalf	a3510f2025	Made NFQ optional via --enable-nfqueue, --enable-logsigs will now load local.rules in the path other fixes	17 years ago
Victor Julien	51a9e36e10	Remove vips references. Rename to eidps.	17 years ago
Victor Julien	8397413942	Comment updates.	17 years ago
Victor Julien	9c7f5afa79	Large update: pcap support, threading fixes, initial stream tracking, time handling, pool support, runmodes, decoders added, autojunk update.	17 years ago
Victor Julien	657be002d1	Big detection engine update: scan improvements, b2g/b3g updates, bloom fixes, iponly detection implementation, dsize/flow grouping.	17 years ago
Victor Julien	5df5b35e90	Put all globals in the detection engine ctx. Add HashList type, a hash that also stores the items ina list to it can be traversed. Many cleanups.	17 years ago
Victor Julien	b2eb954099	Add b3g 3gram BNDM pattern matcher. Fix multi queue nfq initialization. Improve speed of b2g and wumanber.	17 years ago
Victor Julien	efb10fc0d6	big update	17 years ago
Victor Julien	4c4862d838	Improve logging, add alert-output module, at module exit stats, add HTTP POST uri capture.	17 years ago
Victor Julien	4b6a8715fd	Fix packets getting stuck in NFQ under high load.	17 years ago
Victor Julien	edf8650a7e	Tunnel update.	17 years ago
Victor Julien	8cda70668c	Tunnel fixes.	17 years ago
Victor Julien	28d9415e37	New approach to tunnel decoding.	17 years ago
Victor Julien	a454fe4fb7	Small fixes	17 years ago
William Metcalf	0ffa1c2465	updates for configure.in, added reject code, some decode stuff for tcp	17 years ago
Victor Julien	867d493d7f	Source NFQ update... less hackish, but still needs work as soon as we know how to do configuration.	17 years ago
Victor Julien	6c1f2071be	Add unittest registration to the threading modules api.	17 years ago
William Metcalf	559edc01e3	NFQUE drop support added with ident of 4 :-(	17 years ago