suricata

Commit Graph

Author	SHA1	Message	Date
Victor Julien	d66fa1add1	detect: update detect engine management Update detect engine management to make it easier to reload the detect engine. Core of the new approach is a 'master' ctx, that keeps a list of one or more detect engines. The detect engines will not be passed to any thread directly, but instead will only be accessed through the detect engine thread contexts. As we can replace those atomically, replacing a detect engine becomes easier. Each thread keeps a reference to its detect context. When a detect engine is replaced or removed, it's added to a free list. Once its reference count reaches 0, it is freed.	10 years ago
Victor Julien	f5f148805c	app layer: uint16_t alproto -> AppProto alproto This conversion was missing in a couple of places.	12 years ago
Victor Julien	5cdeadb33d	Use u8 for ipproto In a few places in app layer and unittests u16 was used.	12 years ago
Anoop Saldanha	429c6388f6	App layer API rewritten. The main files in question are: app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch]. Things addressed in this commit: - Brings out a proper separation between protocol detection phase and the parser phase. - The dns app layer now is registered such that we don't use "dnstcp" and "dnsudp" in the rules. A user who previously wrote a rule like this - "alert dnstcp....." or "alert dnsudp....." would now have to use, alert dns (ipproto:tcp;) or alert udp (app-layer-protocol:dns;) or alert ip (ipproto:udp; app-layer-protocol:dns;) The same rules extend to other another such protocol, dcerpc. - The app layer parser api now takes in the ipproto while registering callbacks. - The app inspection/detection engine also takes an ipproto. - All app layer parser functions now take direction as STREAM_TOSERVER or STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the functions. - FlowInitialize() and FlowRecycle() now resets proto to 0. This is needed by unittests, which would try to clean the flow, and that would call the api, AppLayerParserCleanupParserState(), which would try to clean the app state, but the app layer now needs an ipproto to figure out which api to internally call to clean the state, and if the ipproto is 0, it would return without trying to clean the state. - A lot of unittests are now updated where if they are using a flow and they need to use the app layer, we would set a flow ipproto. - The "app-layer" section in the yaml conf has also been updated as well.	12 years ago
Victor Julien	9419b9cbb6	detect: add tostring function for DETECT_SM_LIST_ enum.	12 years ago
Anoop Saldanha	d4d18e3136	Transaction engine redesigned. Improved accuracy, improved performance. Performance improvement noticeable with http heavy traffic and ruleset. A lot of other cosmetic changes carried out as well. Wrappers introduced for a lot of app layer functions. Failing dce unittests disabled. Will be reintroduced in the updated dce engine. Cross transaction matching taken care of. FPs emanating from these matches have now disappeared. Double inspection of transactions taken care of as well.	12 years ago
Anoop Saldanha	b99f9fe890	New app inspection engine introduced. Moved existing inspecting engines to use it.	13 years ago
Anoop Saldanha	2bc7d0792d	update clean up of old detection engine contexts for live rule swap	13 years ago
Anoop Saldanha	ecad4a24fa	live rule support added To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup	13 years ago
Victor Julien	820b0ded82	Add per packet profiling. Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Anoop Saldanha	bc99328ec8	define a new conf paramter detect-engine:inspection-recursion-limit; Defines a recursion limit for content inspection code	15 years ago
William Metcalf	ce01927515	Import of GPLv2 Header 050410	15 years ago
Pablo Rincon	b6a3395c08	Adding unittest helper functions for building generic packets, checking arrays of expected match results, perform generic tests, etc. Look at util-unittest-helper.c and detect-ipproto.c for references	16 years ago
Victor Julien	2d0e9658f8	Speed up per sgh content maxlen calc. Remove mpm ptrs from mpm ctx. Add unittests testing the detection engine internals.	16 years ago
Victor Julien	e4c98c562c	Merge DetectAddressData and DetectAddressGroup	16 years ago
Victor Julien	cfb605aa8a	Put the precooked runmodes in a separate file.	16 years ago
Brian Rectanus	fa5939ca91	64 bit cleanup part2	16 years ago
Victor Julien	5df5b35e90	Put all globals in the detection engine ctx. Add HashList type, a hash that also stores the items ina list to it can be traversed. Many cleanups.	16 years ago
Victor Julien	54ffe2053e	Large detection engine update.	16 years ago
Victor Julien	dc48c58473	Switch to using a detection engine ctx.	16 years ago

20 Commits (b038b6a2f87549474503ff80c2e5586ccd02c92d)