51 Commits (94571c5dd28858ff68c44b648fd41c5d87c0e28d)

Author	SHA1	Message	Date
Eric Leblond	18c7eeefca	packet handling: fix release function ... Extended data were freed before the release function was called. The result was that, in AF_PACKET IPS mode, the release function was only sending void data because it the content of the extended data is the content of the packet. This patch updates the code to have the freeing of extended data done in the cleaning function for a packet which is called by the release function. This improves consistency of the code and fixes the bug.	12 years ago
Victor Julien	55f8672a04	profiling: end profiling tunnel packets inside lock ... End profiling inside the lock for a tunnel packet as otherwise another thread may already free the packet while the profiling code runs. SEGV's observed and now gone.	12 years ago
Eric Leblond	3088b6ac34	Add invalid pkt counter. ... This patch adds and increments a invalid packet counter. It does this by introducing PacketDecodeFinalize function This function is incrementing the invalid counter and is also signalling the packet to CUDA.	12 years ago
Ken Steele	b076a26cdc	Replace ReleaseData function on Packet Structure with ReleasePacket. ... This commit allows handling Packets allocated by different methods. The ReleaseData function pointer in the Packet structure is replaced with ReleasePacket function pointer, which is then always called to release the memory associated with a Packet. Currently, the only usage of ReleaseData is in AF Packet. Previously ReleaseData was only called when it was not NULL. To implement the same functionality as before in AF Packet, a new function is defined in AF Packet to first call the AFP specific ReleaseData function and then releases the Packet structure. Three new general functions are defined for releasing packets in the default case: 1) PacketFree() - To release a packet alloced with SCMalloc() 2) PacketPoolReturnPacket() - For packets allocated from the Packet Pool. Calls RECYCLE_PACKET(p) 3) PacketFreeOrRelease() - Calls PacketFree() or PacketPoolReturnPacket() based on the PKT_ALLOC flag. Having these functions removes the need to check the PKT_ALLOC flag when releasing a packet in most cases, since the ReleasePacket function encodes how the Packet was allocated. The PKT_ALLOC flag is still set and is needed when AF Packet releases a packet, since it replaces the ReleasePacket function pointer with its own function and then calls PacketFreeOfRelease(), which uses the PKT_ALLOC flag.	12 years ago
Ken Steele	9d677ea006	Clear the PKT_ALLOC flag when storing Packets into the Packet pool. ... The PKT_ALLOC flag is set by PacketGetFromAlloc(), which needs to be cleared for Packets in the Packet Pool, so clear the flag here.	12 years ago
Ken Steele	9c7b411a5d	More PacketGetFromMalloc() to allocate packets.	12 years ago
Ken Steele	699d9e01f1	Move memset() out of PACKET_INITIALIZE() ... The memset() inside PACKET_INITIALIZE() is redundant in some cases and it is cleaner to do as part of the memory allocation. This simplifies changes for integrating Tilera mPIPE support because the size of memory cleared in that case is different from SIZE_OF_PACKET. For the cases where Packets are directly allocated and then call PACKET_INITIALIZE() without memset() first, this patch adds memset() calls. A further change would use GetPacketFromAlloc() directly.	12 years ago
Victor Julien	e30b1bfe64	Simple IP reputation implementation	13 years ago
Anoop Saldanha	67981d1c5c	Update suricata to use FlowReference/FlowDeReference for the ones left out ... from last update.	13 years ago
Eric Leblond	e176be6fcc	Use unlikely for error treatment. ... When handling error case on SCMallog, SCCalloc or SCStrdup we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc. This patch has been obtained via coccinelle. The transformation is the following: @istested@ identifier x; statement S1; identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)"; @@ x = func(...) ... when != x - if (x == NULL) S1 + if (unlikely(x == NULL)) S1	13 years ago
Eric Leblond	2011a3f87e	capture: add data release mechanism ... This patch adds a data release mechanism. If the capture module has a call to indicate that userland has finished with the data, it is possible to use this system. The data will then be released when the treatment of the packet is finished. To do so the Packet structure has been modified: + TmEcode (*ReleaseData)(ThreadVars *, struct Packet_ *); If ReleaseData is null, the function is called when the treatment of the Packet is finished. Thus it is sufficient for the capture module to code a function wrapping the data release mechanism and to assign it to ReleaseData field. This patch also includes an implementation of this mechanism for AF_PACKET.	13 years ago
Victor Julien	728c4f9ea0	Clean up packet pool at shut down.	13 years ago
Victor Julien	ea0d172693	No longer pass StreamMsg to output for alert logging, instead use the same callback code as is used for state alerts.	13 years ago
Victor Julien	88a21456e3	stream: keep segments in memory until we are sure the stream/state is inspected.	13 years ago
Eric Leblond	34b3f19465	af-packet: Implement zero copy ... This patch adds support for zero copy to AF_PACKET running mode. This requires to use the 'worker' mode which is the only one where the threading architecture is simple enough to permit this without heavy modification.	14 years ago
Victor Julien	c590bba4a4	Undo tunnel reference counting using atomic operations. Revert to mutex based code.	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Victor Julien	52eb8d2be0	Convert mutex protected tunnel counters to lockless atomic counters.	14 years ago
Victor Julien	3d22713b09	Convert Packet tunnel variables to bit flag checks.	15 years ago
Victor Julien	864c8718e1	Store matching stream msg (ptr) in packets alert structure so it's available to the output plugins.	15 years ago
Victor Julien	4bf4382354	Make sure tunnel packets (and pseudo packets) properly decrement the flow use counter in all cases.	15 years ago
Eric Leblond	e802e1ed16	Modify Packet structure and prepare accessor. ... This patch modifies decode.c and decode.h to avoid the usage by default of a bigger than 65535 bytes array in Packet structure. The idea is that the packet are mainly under 1514 bytes size and a bigger size must be supported but should not be the default. If the packet length is bigger than DFLT_PACKET_SIZE then the data are stored in a dynamically allocated part of the memory. To ease the modification of the rest of the code, functions to access and set the payload/length in a Packet have been introduced. The default packet size can be set at runtime via the default-packet-size configuration variable.	15 years ago
Victor Julien	ffcd512167	Clean up packet pool handler on shutdown.	15 years ago
Pablo Rincon	76af1b049b	Make malloc errors on initialization stage a fatal error, resulting on a exit() call	15 years ago
Victor Julien	6eb7eea705	Fix a data race for packet pool packets when defrag/tunnel code needs a packet.	15 years ago
Victor Julien	6519a86ec7	Move packet pool to ringbuffer, update packet pool api and ringbuffer api. Remove memset usage from PACKET_RECYCLE, add proper cleanup macros.	15 years ago
Victor Julien	dce323b1f4	Fix SCondWait -> SCCondWait typo.	15 years ago
Victor Julien	c26434fef1	Move flow use cnt to atomic and outside of the flow mutex protection.	15 years ago
Victor Julien	f80016530c	Remove unnecessary locking for thread-local packet-queues breaking on Win32.	15 years ago
Victor Julien	a8dd484aba	Fix small potential bug in debug mode found by clang.	16 years ago
Victor Julien	e741bd0202	Cleanup packet recycling code. Fix issues in the packet tunnel/pseudo code.	16 years ago
Victor Julien	19584d0416	Fix tunneled and defrag reassembled packets with the new pending limits.	16 years ago
Victor Julien	ccf22cf563	Move to different way of enforcing max_pending. Should require less locks.	16 years ago
William Metcalf	ce01927515	Import of GPLv2 Header 050410	16 years ago
Victor Julien	27d1dbc342	Add comment explaining how I feel a CLANG related change is unnecessary, but that I might be wrong :-)	16 years ago
Gurvinder Singh	69a4fee757	fixed the API and logic error reported by clang tool	16 years ago
Jason Ish	7142fdb780	quick way to make max_pending configurable.	16 years ago
Victor Julien	ecf86f9c23	Rename to Suricata.	16 years ago
Pablo Rincon	e26833be3f	Changing mutex/spinlocks/conditions naming types	16 years ago
Pablo Rincon	769022f4be	Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests	16 years ago
Anoop Saldanha	f35d9f0437	threading improvements. Replaced the use of slot(2/3) with varslot. Improve error handling in slot functions. Additional helper functions for thread creation	16 years ago
Victor Julien	391e69e699	Fix tunnel packet handling.	16 years ago
Brian Rectanus	fa5939ca91	64 bit cleanup part2	16 years ago
Victor Julien	51a9e36e10	Remove vips references. Rename to eidps.	16 years ago
Victor Julien	9c7f5afa79	Large update: pcap support, threading fixes, initial stream tracking, time handling, pool support, runmodes, decoders added, autojunk update.	16 years ago
Victor Julien	a0b390c541	Fix pktvar and http uri memory leaks.	16 years ago
Victor Julien	b2eb954099	Add b3g 3gram BNDM pattern matcher. Fix multi queue nfq initialization. Improve speed of b2g and wumanber.	16 years ago
Victor Julien	efb10fc0d6	big update	16 years ago
Victor Julien	edf8650a7e	Tunnel update.	16 years ago
Victor Julien	8cda70668c	Tunnel fixes.	16 years ago