48 Commits (fd9a20ffcfd6fcd176efa003493b68f99ec99a63)

Author	SHA1	Message	Date
Stephen Donnelly	879db3dbc3	endace: Fix source-dag timestamps ... Bug: #6618. Fix Endace ERF to SCTime_t timestamp conversion Fix typo preventing compilation with --enable-dag	3 years ago
Jeff Lucovsky	193e0ea1a9	memory/alloc: Use SCCalloc instead of malloc/memset	3 years ago
Jeff Lucovsky	9fbe683642	time: Rework SCTime_t into a struct ... Issue: 5718 This commit changes SCTime_t to a struct with members setup as bitfields.	4 years ago
Jeff Lucovsky	31793aface	time: Replace struct timeval with scalar value ... Issue: 5718 This commit switches the majority of time handling to a new type -- SCTime_t -- which is a 64 bit container for time: - 44 bits -- seconds - 20 bits -- useconds	4 years ago
Victor Julien	b31ffde6f4	output: remove error codes from output	4 years ago
Victor Julien	e042cd785e	error: use SC_ENOMEM for alloc errors	4 years ago
Richard McConnell	13beba141c	source: add THV_RUNNING flag to notify of running state ... Each module (thread) updates its status to indicate running. Main thread awaits for all threads to be in a running state before continuing the initialisation process Implements feature 5384 (https://redmine.openinfosecfoundation.org/issues/5384)	4 years ago
Eric Leblond	1c2fba57f8	suricata: introduce global linktype ... As Suricata is not supporting pcap-ng we have to stick with one single datalink type for the capture if ever we want to do pcap logging. Assuming this, this patch introduces a function to set the link type globally. This will be used with pcap conditional logging to get the logging of TCP segments with the correct link type.	4 years ago
Victor Julien	0025467f90	sources: hide RegisterTests behind ifdef UNITTESTS ... Update callers.	6 years ago
Jason Ish	4639dd7932	source/erf: validate record length before read ... Check the ERF record length before attempting to read it as a record length less than the size of the record header is invalid. Redmine ticket: https://redmine.openinfosecfoundation.org/issues/3593	7 years ago
Victor Julien	7810f22413	decode: remove pseudo packet checks ... Bug 1107 checks/hacks should not longer be needed, so remove them.	7 years ago
Victor Julien	f8aed4ce2d	threading: change local packet queue logic ... Previously each 'TmSlot' had it's own packet queue that was passed to the registered SlotFunc as an argument. This was used mostly for tunnel packets by the decoders and by defrag. This patch removes that in favor of a single queue in the ThreadVars: decode_pq. This is the non-locked version of the queue as this is only a temporary store for handling packets within a thread. This patch removes the PacketQueue pointer argument from the API. The new queue can be accessed directly through the ThreadVars pointer.	7 years ago
Victor Julien	44d7f636f2	threading: remove post_pq argument from 'SlotFunc' ... This was not in use anywhere.	7 years ago
Victor Julien	11be9bd971	mingw: add SCNtohl and SCNtohs macro's ... On MinGW the result of ntohl needs to be casted to uint32_t and the result of ntohs to uint16_t. To avoid doing this everywhere add SCNtohl and SCNtohs macros.	9 years ago
Victor Julien	ab1200fbd7	compiler: more strict compiler warnings ... Set flags by default: -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wwrite-strings -Wcast-align -Wbad-function-cast -Wformat-security -Wno-format-nonliteral -Wmissing-format-attribute -funsigned-char Fix minor compiler warnings for these new flags on gcc and clang.	9 years ago
Victor Julien	85db260eed	threads: remove EngineKill & SURICATA_KILL ... EngineStop and EngineKill were effectively doing the same, so removed the kill variant.	10 years ago
cardigliano	57e0bd39e9	pkt acq: introduce break loop API ... This patch adds a new callback PktAcqBreakLoop() in TmModule to let packet acquisition modules define "break-loop" functions to terminate the capture loop. This is useful in case of blocking functions that need special actions to take place in order to stop the execution. Implement this for PF_RING	11 years ago
Victor Julien	14466a803d	decode: create util function for basic counter updates	11 years ago
Victor Julien	30cce2bd29	counters: s/SCPerfCounterSetUI64/StatsSetUI64/g	11 years ago
Victor Julien	1c0b4ee0ae	counters: s/SCPerfCounterIncr/StatsIncr/g	11 years ago
Victor Julien	8992275b0c	counters: s/SCPerfCounterAddUI64/StatsAddUI64/g	11 years ago
Victor Julien	b34c6dc93a	counters: remove references to SCPerfCounterAddDouble ... They were all in comments anyway.	11 years ago
Victor Julien	e9b067c1eb	counters: make increment call take threadvars ... This hides the implementation from the caller.	11 years ago
Victor Julien	9a8bff7d96	counters: threadvars s/sc_perf_pca/perf_private_ctx/g	11 years ago
Victor Julien	98c88d5170	decode: pass ThreadVars to DecodeThreadVarsFree ... Flow output thread data deinit function which will be called from DecodeThreadVarsFree will need it.	12 years ago
Ken Steele	28ccea51d3	Add error checking for pthread_setspecific() and pthread_key_create().	12 years ago
Ken Steele	a38d5a0135	Implement thread specific data option when __thread is not available.	12 years ago
Ken Steele	3c6e01f653	Replace ringbuffer in Packet Pool with a stack for better cache locality ... Using a stack for free Packet storage causes recently freed Packets to be reused quickly, while there is more likelihood of the data still being in cache. The new structure has a per-thread private stack for allocating Packets which does not need any locking. Since Packets can be freed by any thread, there is a second stack (return stack) for freeing packets by other threads. The return stack is protected by a mutex. Packets are moved from the return stack to the private stack when the private stack is empty. Returning packets back to their "home" stack keeps the stacks from getting out of balance. The PacketPoolInit() function is now called by each thread that will be allocating packets. Each thread allocates max_pending_packets, which is a change from before, where that was the total number of packets across all threads.	12 years ago
Victor Julien	ab503873ca	erf-file: clean up decode thread local storage ... Clean up the thread local data the decode portion of ERF users. Bug #978	13 years ago
Victor Julien	f7b1aefaf4	Bug 1107: decoders: bail out on pseudo packets ... Flow-timeout code injects pseudo packets into the decoders, leading to various issues. For a full explanation, see: https://redmine.openinfosecfoundation.org/issues/1107 This patch works around the issues with a hack. It adds a check to each of the decoder entry points to bail out as soon as a pseudo packet from the flow timeout is encountered. Ticket #1107.	13 years ago
Victor Julien	5f307acace	Pass ThreadVars ptr to various thread init funcs ... To be able to register counters from AppLayerGetCtxThread, the ThreadVars pointer needs to be available in it and thus in it's callers: - AppLayerGetCtxThread - DecodeThreadVarsAlloc - StreamTcpReassembleInitThreadCtx	13 years ago
Eric Leblond	3088b6ac34	Add invalid pkt counter. ... This patch adds and increments a invalid packet counter. It does this by introducing PacketDecodeFinalize function This function is incrementing the invalid counter and is also signalling the packet to CUDA.	13 years ago
Victor Julien	698ff4e4aa	Counters: remove all unused parts of the API	13 years ago
Last G	8bb9c3af35	Added return value to non-void function with "forever"-loop to fit ... Eclipse static code analysis	14 years ago
Anoop Saldanha	b33986c887	Add a packet src for every packet generated inside suricata.	14 years ago
Eric Leblond	e176be6fcc	Use unlikely for error treatment. ... When handling error case on SCMallog, SCCalloc or SCStrdup we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc. This patch has been obtained via coccinelle. The transformation is the following: @istested@ identifier x; statement S1; identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)"; @@ x = func(...) ... when != x - if (x == NULL) S1 + if (unlikely(x == NULL)) S1	14 years ago
Jason Ish	ea020e2be6	Do not trim the FCS, pcaps converted to ERF will have have an FCS.	14 years ago
Eric Leblond	a48d6cb207	erf: fix logical operator usage.	14 years ago
Anoop Saldanha	34581ce902	rx TMs shouldn't return TM_ECODE_FAILED if engine is in shutdown mode + minor cleanup	14 years ago
Anoop Saldanha	bc6cf43840	#482 - use decode_flag for all decode TMs. Use the flag as a way to retrieve decode TMs from ThreadVars	14 years ago
Jason Ish	90548837e3	Update the ERF file runmodes to support autofp and single.	15 years ago
Anoop Saldanha	3f1c4efceb	Add new flags var to tm module. TMs can now set flags to identify special properties. Also use these to identify receive TMs	15 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	15 years ago
Eric Leblond	dd038c1906	Modify files to avoid direct pckt payload access ... This patch implements the needed modification of payload access in a Packet structure to support the abstraction introduced by the extended data system.	16 years ago
Victor Julien	4cacb1e970	Disable adding to unregistered mbit/s counter.	16 years ago
Pablo Rincon	8cc525c939	UDP support at AppLayer message handling	16 years ago
Victor Julien	4e7df60b2f	Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this.	17 years ago
Jason Ish	a93b2e6b84	Support for reading ERF files.	17 years ago