suricata

Commit Graph

Author	SHA1	Message	Date
Victor Julien	a37a1d9de7	profiling: fix percentage of detect phases Use proper 'total' to calculate weigth of each detection phase.	12 years ago
Victor Julien	8a735a9b90	profiling: add sample-rate yaml option Add option "profiling.sample-rate": # Run profiling for every xth packet. The default is 1, which means we # profile every packet. If set to 1000, one packet is profiled for every # 1000 received. #sample-rate: 1000 This allows for configuration of the sample rate.	12 years ago
Victor Julien	2c3a92a1c9	profiling: conditional rule profiling Add support for conditional rule profiling. Currently only simple rate limiting is supported, but hardcoded to inspecting rules for each packet.	12 years ago
Victor Julien	13d491f577	profiling: lower overhead when disabled Instead of a large (6k+) structure in the Packet, make the profiling storage dynamic. To do this the Packet->profile is now a pointer. Initial support for selective sampling, e.g. only profile every 1000th packet.	12 years ago
Anoop Saldanha	127ef8f903	Use a typdef AppProto <-> uint16_t for representing app layer protocol. Some minor refactoring/cleanup, including renaming functions.	12 years ago
Eric Leblond	34abd818dd	Prefix util-conf function with Config	12 years ago
Eric Leblond	54006de40c	Use new function GetLogDirectory()	12 years ago
Victor Julien	43ba5a677e	DNS: enable mpm/fast_pattern support for dns_query	12 years ago
Victor Julien	ffffe6c10e	profiling: add formatted totals, percents to packet stats	12 years ago
Victor Julien	82769a1b37	profiling: fix missing profile names	13 years ago
Eric Leblond	e176be6fcc	Use unlikely for error treatment. When handling error case on SCMallog, SCCalloc or SCStrdup we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc. This patch has been obtained via coccinelle. The transformation is the following: @istested@ identifier x; statement S1; identifier func =~ "(SCMalloc\|SCStrdup\|SCCalloc)"; @@ x = func(...) ... when != x - if (x == NULL) S1 + if (unlikely(x == NULL)) S1	13 years ago
Eric Leblond	655577cbbc	Add some missing checks of SCMalloc return.	13 years ago
Victor Julien	7babb35aeb	profiling: remove obsolete unit test	13 years ago
Victor Julien	2343ff8950	profiling: fix memory error in case of rule reload.	13 years ago
Victor Julien	ec7e79c748	Rule profiling update - Remove usage of counters api. - Store stats in detect engine thread ctx to remove locking - Support rule reloads	13 years ago
Eric Leblond	7af9fd7735	freebsd: fix warning about redeclaration.	13 years ago
Eric Leblond	0227a87fcb	cleaning: fix warning when building with clang. clang was issuing some warnings related to unused return in function. This patch adds some needed error treatment and ignore the rest of the warnings by adding a cast to void.	13 years ago
Eric Leblond	a0e57f58e5	OpenBSD: introduce SCLocalTime function. This function is a wrapper to localtime_r. It is needed to avoid a compilation warning on OpenBSD. I'm forced to type the function to a non pointer first parameter. If not we will have to use two differents functions in OpenBSD where tv->tv_sec is a long (different from time_t).	13 years ago
Victor Julien	d908e707d7	profiling: add per lock location profiling Add profiling per lock location in the code. Accounts how often a lock is requested, how often it was contended, the max number of ticks spent waiting for it, avg number of ticks waiting for it and the total ticks for that location. Added a new configure flag --enable-profiling-locks to enable this feature.	14 years ago
Victor Julien	979edf0b97	Add way to profile mutex/spin locks per thread module.	14 years ago
Victor Julien	0b9038b971	Add atomics to ticks unittests.	14 years ago
Victor Julien	89f83e714c	Introduce http_server_body keyword. The http_server_body content modifier modifies the previous content to inspect the normalized (dechunked, unzipped) http_server_body. The workings are similar to http_client_body. Additionally, a new pcre flag was introduced "/S". To facilitate this change the signature flags field was changed to be 64 bit.	14 years ago
Victor Julien	af51493da2	Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm.	14 years ago
Anoop Saldanha	4307ea2348	Replace all frees with SCFrees	14 years ago
Anoop Saldanha	13ea299ee0	Replace all mallocs with SCMallocs	14 years ago
Victor Julien	cc4e89fbe1	Profiling: convert all packet profile counters/variables to u64. Improve output for larger numbers.	14 years ago
Eileen Donlon	89599d3b9b	fixed bug 288; corrected config boolean parsing problems	14 years ago
Victor Julien	63f834d9a7	Add profiling to various HTTP buffer MPM calls.	14 years ago
Victor Julien	3693a7a9ee	Profiling: add accounting for several detection phases.	14 years ago
Victor Julien	e8e392fb1f	Profiling: add per packet accounting of how much ticks are spend in protocol detection.	14 years ago
Victor Julien	21f387d2c7	profiling: fix stream ticks miscalculation on stream end pseudo packets.	14 years ago
Victor Julien	fca541f40e	Add per app layer parser profiling Per packet per app layer parser profiling. Example summary output: Per App layer parser stats: App Layer IP ver Proto cnt min max avg -------------------- ------ ----- ------ ------ ---------- ------- ALPROTO_HTTP IPv4 6 163394 126 38560320 42814 ALPROTO_FTP IPv4 6 644 117 26100 2566 ALPROTO_TLS IPv4 6 670 117 7137 799 ALPROTO_SMB IPv4 6 114794 126 225270 957 ALPROTO_DCERPC IPv4 6 5207 126 25596 1266 Also added to the csv out. In the csv out there is a new column "stream (no app)" that removes the app layer parsers from the stream tracking. So raw stream engine performance becomes visible.	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Victor Julien	1bd1a62526	Rename profile macro's and variables to reflect that they are for rule profiling.	14 years ago
William	bca8fbc79e	Add Num, Rev, and Gid columns to rule perf output	14 years ago
Victor Julien	ece8e5444b	Minor profiling fix: don't close stdout.	14 years ago
William	85643fe780	Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them.	14 years ago
Jason Ish	a4d19e4130	Add new profiling sort option, maxticks.	15 years ago
Victor Julien	746ef0d2f8	Remove stray newline char from profiling output.	15 years ago
Jason Ish	9ad1924dba	Track the max ticks for each rule. This is the highest tick count this rule used to evaluate in a single sample.	15 years ago
Pablo Rincon	c1486d7f2e	Fix bug 217 (segv on profiling summary if no rule was specified	15 years ago
Victor Julien	8514132851	Cleanup thresholding code.	15 years ago
Jason Ish	ea4b7cc33b	add profiling to stateful detection engine + other fixups.	15 years ago
Jason Ish	a9ed9a8196	In profiling output, include a % for each rule.	15 years ago
William Metcalf	2eef905c07	GPL and Copyright header updates.	15 years ago
Jason Ish	18e5ac8cde	Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules.	15 years ago

46 Commits (a37a1d9de7b4e5d4d763963e569e69d4b0b8fdf5)