131 Commits (347c0df9c4e821f33938628855e3ceed115fd9d0)

Author	SHA1	Message	Date
Victor Julien	634eb1d35c	app-layer proto detect: optimization ... Don't use FlowGetProtoMapping at runtime, use f->protomap instead. Add safety check to make sure its value is within range, as it's used to index an array. Update unittests to initialized flows (somewhat).	12 years ago
Victor Julien	4f1f395bb5	Coverity 1153935: fix confusing sizeof	12 years ago
Victor Julien	1f00ff6ab3	App-layer proto detect cleanups ... Remove unnecessay inlining. Rename functions with wrong naming scheme. E.g. AllocAppLayer.. instead of AppLayer..Alloc. Use AppProto instead of uint16_t. Convert u16 ipproto cases to u8.	12 years ago
Victor Julien	9952db6d6e	Fix crash in AppLayer Proto Detect ... The App Proto Detect code would use the wrong pattern count to index a results array, leading to SEGVs. Bug #1080.	12 years ago
Victor Julien	0ec375d95a	stream msg: remove structure	12 years ago
Victor Julien	446e68adca	app-layer: only typedef opaque pointers once	12 years ago
Victor Julien	3b8e9ffbe9	app layer: void -> AppLayerProtoDetectThreadCtx ... User AppLayerProtoDetectThreadCtx ptr instead of void.	12 years ago
Victor Julien	8dbf7a0d78	Update tests to use AppLayerParserThreadCtx ptr instead of void. Fix a few bugs uncovered by this.	12 years ago
Victor Julien	fdefb65be4	app-layer: rename AppLayerThreadCtx funcs ... AppLayerParserGetCtxThread -> AppLayerParserThreadCtxAlloc AppLayerParserDestroyCtxThread -> AppLayerParserThreadCtxFree	12 years ago
Victor Julien	f5f148805c	app layer: uint16_t alproto -> AppProto alproto ... This conversion was missing in a couple of places.	12 years ago
Victor Julien	5cdeadb33d	Use u8 for ipproto ... In a few places in app layer and unittests u16 was used.	12 years ago
Victor Julien	30f16ee446	Rename AppLayerProtoDetectCtxThread -> AppLayerProtoDetectThreadCtx	12 years ago
Victor Julien	67053e6ed0	Fix AppLayerProtoDetectPMFreeSignature related valgrind errors	12 years ago
Victor Julien	ff16d6fa53	app proto detect: fix valgrind test warnings ... Only in unittests when debug is enabled would valgrind warn about a print statement.	12 years ago
Victor Julien	a456bd8181	Cleanup and fix scan-build warning ... Add comments and slightly refactor to make function more understandable and fix a scan-build warning too.	12 years ago
Victor Julien	59327e0fd4	Various style fixes	12 years ago
Anoop Saldanha	429c6388f6	App layer API rewritten. The main files in question are: ... app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch]. Things addressed in this commit: - Brings out a proper separation between protocol detection phase and the parser phase. - The dns app layer now is registered such that we don't use "dnstcp" and "dnsudp" in the rules. A user who previously wrote a rule like this - "alert dnstcp....." or "alert dnsudp....." would now have to use, alert dns (ipproto:tcp;) or alert udp (app-layer-protocol:dns;) or alert ip (ipproto:udp; app-layer-protocol:dns;) The same rules extend to other another such protocol, dcerpc. - The app layer parser api now takes in the ipproto while registering callbacks. - The app inspection/detection engine also takes an ipproto. - All app layer parser functions now take direction as STREAM_TOSERVER or STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the functions. - FlowInitialize() and FlowRecycle() now resets proto to 0. This is needed by unittests, which would try to clean the flow, and that would call the api, AppLayerParserCleanupParserState(), which would try to clean the app state, but the app layer now needs an ipproto to figure out which api to internally call to clean the state, and if the ipproto is 0, it would return without trying to clean the state. - A lot of unittests are now updated where if they are using a flow and they need to use the app layer, we would set a flow ipproto. - The "app-layer" section in the yaml conf has also been updated as well.	12 years ago
Anoop Saldanha	a49cbf8a49	Code cleanup. ... Use the MpmAddPattern[CS|CI] wrapper to add patterns to the mpm context. Also use MpmInitCtx() to init the mpm context.	12 years ago
Victor Julien	7074ca373b	proto detection: add limit for one sided sessions ... If a session only has data in one direction, like ftp data sessions, protocol detection will only run in one direction. This led to a situation where reassembly would hold all the segments as proto detection was never flagged as complete. This patch introduces a limit for protocol detection in this case. If the limit is reached, detection will give up.	12 years ago
Anoop Saldanha	36bd444406	Introduce new API to allow case insensitive protocol detection patterns.	12 years ago
Anoop Saldanha	e383cc27cd	Fix a leak in probing parsers. We were freeing just the head of the list, ... instead of all the members.	12 years ago
Anoop Saldanha	16144fe38a	Rename function pointer var to use the FuncPtr typing convention. Resupply "dns" as the alproto name for ALPROTO_DNS.	12 years ago
Anoop Saldanha	b1dffdfbe0	Add app layer protocol packet event detection support.	12 years ago
Anoop Saldanha	64b0939b4a	code cleanup.	12 years ago
Anoop Saldanha	0d7159b525	App layer protocol detection updated and improved. We now use ... confirmation from both directions and set events if there's a mismatch between the 2 directions. FPs from corrupt flows have disappeared with this.	12 years ago
Anoop Saldanha	c044541b1c	Provide convenience macros for setting flow flags on protocol matching by ... PM and PP phase. Replace the areas of the code that would otherwise rely on setting/reading these flags with these macros. Other minor tweaks to some api calls.	12 years ago
Anoop Saldanha	00f546e739	update pmp to return whole set of matches, rather than a single match.	12 years ago
Anoop Saldanha	d9686fae57	Now supports accepting port addresses as strings, like the ones accepted in our rules. As a consequence we now accept port range, and other such combination. Support PP for ports based on ipproto as well.	12 years ago
Eric Leblond	3dbf6c6fee	solaris: fix compilation failure ... This patch fixes a compilation failure on Solaris. Compiler does not support when a function returning void is used in return of an other function returning void.	12 years ago
Eric Leblond	cd3e32ce19	unittests: some functions needs a flow lock. ... In debug validation mode, it is required to call application layer parsing and other functions with a lock on flow. This patch updates the code to do so.	12 years ago
Anoop Saldanha	3c2ddf04c1	Update mpm init ctx to not accept the final cuda_rc_module argument. ... It was a part of our older architecture and is no longer used.	12 years ago
Anoop Saldanha	b787da5643	Remove all cuda related code in the engine except for the cuda api wrappers	12 years ago
Ken Steele	394f99e32c	Use PacketGetfromAlloc() for packet allocation instead of SCMalloc. ... Only changed in one file for testing.	12 years ago
Anoop Saldanha	34a9c047fc	updated to fix unix shutdown sequence ... Should fix crashes occuring from unix mode shutdown/cleanup phase.	13 years ago
Last G	8ae11f73b2	Added parentheses to fix Eclipse static code analysis ... Fixed bug in action priority (REJECT_DST had lowest prio)	13 years ago
Eric Leblond	e176be6fcc	Use unlikely for error treatment. ... When handling error case on SCMallog, SCCalloc or SCStrdup we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc. This patch has been obtained via coccinelle. The transformation is the following: @istested@ identifier x; statement S1; identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)"; @@ x = func(...) ... when != x - if (x == NULL) S1 + if (unlikely(x == NULL)) S1	13 years ago
Anoop Saldanha	109662450d	Add new command line option --list-app-layer-protocols to list supported app layer protocols in sigs	14 years ago
Anoop Saldanha	7511fa67cd	Add BUG_ON to avoid overruning AppLayerDetectDirection map array	14 years ago
Victor Julien	7173256754	Fix compiler warnings in a couple of unittests.	14 years ago
Anoop Saldanha	9a6aef459e	modify all relevant app layer API calls to accomodate passing parser local storage argument	14 years ago
Anoop Saldanha	01a35bb604	introduce app layer local storage api support	14 years ago
Victor Julien	262a7300d7	flow: shrink Flow datatype ... Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address that doesn't have the family in it like the Address structure. Instead, the family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6. Add macro's to check the family, copy the address, etc. Update many unittests to reflect these changes. Introduce unittest helper functions for creating and initializing a flow and freeing it again. On 64 bit this shrinks the flow with 8 bytes.	14 years ago
Victor Julien	06904c9024	App Layer cleanup ... Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate. Various cleanups and dead code removal from the app layer API. Should safe 100+ bytes memory per flow on 64 bit. Updated lots of unittests to reflect these changes.	14 years ago
Anoop Saldanha	ed3b44b3b5	fix parsing content keywords. We are more strict now. All content keywords need to be enclosed in double quotes. Better validation for sid, priority and rev keywords	14 years ago
Anoop Saldanha	d68775d47d	introduce bitmasks instead of alproto_masks for use by the probing parser. Remove all alproto_masks we had previouslys for PP	14 years ago
Anoop Saldanha	d3989e7cee	probing parser updated to always accept u32 buflens. Update all probing parser functions to accomodate this change	14 years ago
Anoop Saldanha	80d80000bb	fix probing parser flag usage during protocol detection	14 years ago
Anoop Saldanha	432c3317d2	app layer probing parser updates	14 years ago
Anoop Saldanha	d68f182ebd	introduce SCPerfSyncCounters/SCPerfSyncCounters macro to synchronize counters	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago