Victor Julien
004b5dde88
Support libhtp's different handling of CONNECT requests.
14 years ago
Victor Julien
117d51c965
Fix a compile warning when debug is enabled.
14 years ago
Victor Julien
1df3304655
Clean up for unittests code: only compile unittest api code when unittests are enabled. Fix unittest code that wasn't wrapped in the proper UNITTESTS ifdefs.
14 years ago
Victor Julien
a138b32533
flow manager: timing change
...
Set default timeout for the flow manager to wake up to 1 second. The 0.4 sec
performed best on a Xeon, but in kvm vm's it was horrible:
32 bit vm: 60% cpu for flowmgr when idle.
64 bit vm: 30% cpu for flowmgr when idle.
With the 1 second timeout both are at 0.3% cpu.
14 years ago
Victor Julien
786148319c
Lower flow manager wake up timer to 0.4 seconds as that performs 2% better in my tests.
14 years ago
Anoop Saldanha
776bf633e3
flow manager code cleanup. Remove unused code + fix indentation. Remove unused vars
14 years ago
Anoop Saldanha
5133098bd6
Accomodate pcap-file mode to signal flow mgr to wakeup when it exceeds a certain time interval. This let's the flow mgr keep in sync with pcap timestamp changes
14 years ago
Anoop Saldanha
9917744707
separate timers for flow mgr thread for normal and emerg mode. Signal flow mgr thread when in emerg mode
14 years ago
Eric Leblond
5a63662766
Flow: use condition system instead of short sleep
...
Short sleep can lead to some really annoying performance issue in
some environnement like virtual systems. This technic was used in
the flow manager. This patch uses an alternate approach based on
a timed condition which is triggered each time a new flow has to
be created. This avoid to run out of flow. A counter is also done
to be able not to run the cleaning code at each new flow.
14 years ago
Victor Julien
34450b9b57
Don't parse layers / ext headers above ipv6 frag header. This is taken care of by defrag.
14 years ago
Victor Julien
938e9b3db0
Fix filestore related segv.
14 years ago
Victor Julien
e6d8d0443c
Unify output functions for alert-debug for IPv4 and IPv6.
14 years ago
Victor Julien
3c7f09d1ea
Add debug output to engine event.
14 years ago
Victor Julien
e6af837b25
Convert StreamTcpSetEvent function into macro. Eases debug.
14 years ago
Victor Julien
58011554b0
Don't consider payload len in ACK value validation check.
14 years ago
Victor Julien
9878eca086
file handling: expand filestore keyword
...
Filestore keyword by default (... filestore; ... ) marks only the file in the
same direction as the rule match for storing. This makes sense when inspecting
individual files (filemagic, filename, etc) but not so much when looking at
suspicious file requests, where the actual file is in the response.
The filestore keyword now takes 2 optional options:
filestore:<direction>,<scope>;
By default the direction is "same as rule match", and scope is "currently
inspected file".
For direction the following values are possible: "request" and "to_server",
"response" and "to_client", "both".
For scope the following values are possible: "tx" for all files in the current
HTTP/1.1 transation, "ssn" and "flow" for all files in the session/flow.
For the above case, where a suspious request should lead to a response file
download, this would work:
alert http ... content:"/suspicious/"; http_uri; filestore:response; ...
14 years ago
Victor Julien
ddfa5c49c6
Stream engine: gap handling
...
Set a stream event for stream gaps.
Add a (disabled by default) signature to the stream-event.rules.
14 years ago
Victor Julien
45d86ff58a
Stream reassembly / app layer: disable gap errors
...
Gap errors on the app layer are now silently handled. No longer printed
to the screen.
14 years ago
Victor Julien
425294f912
stream reassembly: account stream gaps
...
Add counter to the stream reassembly engine to count stream gaps. Stream gaps
are the result of missing packets (usually due to packet loss). This missing
data stops the reassembly for the app layer.
14 years ago
Victor Julien
d8d8fdd9f5
Improve handling of packets when stream is in the fin_wait1 or fin_wait2 state.
14 years ago
Victor Julien
b74c73309b
file handling: improve filestore keyword handling
...
In stateful detection only inspect the file portion of the rule after all
other conditions matched. This to prevent "filestore" from tagging files
for storage during a partial match.
Add a couple of unittests to test the behaviour change.
14 years ago
Victor Julien
4cbe7519fa
Add missing file util code.
14 years ago
Victor Julien
56b96363b8
Fix merge artefact.
14 years ago
Victor Julien
63c9a3ab85
Remove duplicate include.
14 years ago
Victor Julien
042fd850fc
Make sure we check the sgh for no magic and no store once per flow direction.
14 years ago
Victor Julien
f3fbc1a44c
file handling: filemagic matching improvement
...
Magic buffer is a null terminated string. Allow matching on the final
\0 using filemagic:"somevalue|00|"; so we can anchor to the end of the
buffer.
14 years ago
Victor Julien
2ccd35c6e4
Fix code after rebase.
14 years ago
Victor Julien
33848124d1
Fix a multipart body parsing issue.
14 years ago
Victor Julien
96d20098b0
file inspect: stateful inspection split
...
Split stateful detection of the files in a HTTP state between toserver
and toclient inspection.
14 years ago
Victor Julien
d59ca75e46
file extract: split toserver and toclient tracking
...
Split toserver and toclient file tracking for the http state.
14 years ago
Victor Julien
04ea70ccf7
file extract: pruning
...
Add pruning of files in memory so we keep only memory what we really need.
Fix magic logic.
Reset file part of the de_state on receiving another file in the same tx.
14 years ago
Victor Julien
1c934acc85
Don't store fd per file (too many fd's). Enable IPv6 storing. Close file on receiving stream end flag.
14 years ago
Victor Julien
b402d97179
File carving -- enable reponse file extraction
...
- Enable response body tracking
- Enable file extraction for responses
- File store meta file includes magic, close reason.
- Option to force magic lookup for all stored files.
- Fix libmagic calls thead safety.
14 years ago
Victor Julien
66a3cd96a8
Prepare HTTP response body tracking.
14 years ago
Victor Julien
417495e542
file-extraction: remove no longer used files.
14 years ago
Victor Julien
e1022ee5ae
file-extraction: Disconnect file handling from flow and move into the app layer state.
14 years ago
Victor Julien
27645f64c6
Remove unused util-filetype.[ch] from Makefile.am.
14 years ago
Victor Julien
9b62ec65ab
Make sure filemagic works properly regardless of filestore being in use for a flow.
14 years ago
Victor Julien
5945e652d6
Initial implementation of filemagic keyword.
14 years ago
Victor Julien
f4a6f4b293
Add libmagic detection, linking and a basic API.
14 years ago
Victor Julien
23e01d23d3
Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored.
14 years ago
Victor Julien
3e7baa6810
Fix improper error handling in http body chunk function.
14 years ago
Victor Julien
403b2788d6
Add support for extracting PUT files.
14 years ago
Victor Julien
59cda9a358
Fix not using new htp callback when using the bundled htp. Add indication to --build-info. Fix valgrind warning in test and further improve test.
14 years ago
Victor Julien
ef0536794c
Adding comments, some cleanups.
14 years ago
Victor Julien
21acd72adf
Cleanups to the Multipart parsing code. Fixes to negation in filename and fileext.
14 years ago
Victor Julien
70f0d3d2e7
Add negation to filename and fileext, use same syntax as with content.
14 years ago
Victor Julien
32fb9f375d
log-file log-dir option added, meta file created, fixes.
14 years ago
Victor Julien
a6b7a560f1
Fix a bug in the HTTP file closing.
14 years ago
Victor Julien
7e3d537338
Fix setting libhtp personality.
14 years ago
Victor Julien
1eef36b011
Initial checkin of a log-file module, that can write files extracted from flows to disk.
14 years ago
Victor Julien
3c1edf3763
Add a file descriptor to the flow file structure.
14 years ago
Victor Julien
cd618e48df
Allow for 0 (unlimited) HTTP request_body_limit, fix option parsing.
14 years ago
Victor Julien
4723f07254
Improve testing and fix some bugs.
14 years ago
Victor Julien
9d5d46c4bb
Implement flow file storage API, create HTP wrappers for it, use it in HTTP parsing.
14 years ago
Victor Julien
a0ee6ade3e
Improve HTTP multipart parsing, add streaming parsing for files.
14 years ago
Victor Julien
4537f889ef
Handle all strings as raw strings in HTTP content-type and content-disposition header parsing.
14 years ago
System Administrator
222bc6e935
Flow files
14 years ago
Pablo Rincon
6d60b3a747
filename and fileext keywords
14 years ago
Victor Julien
06b1d71032
Small optimizations to IPV4 and TCP header parsing.
14 years ago
Eric Leblond
0256ca2422
af-packet: fix compilation on new systems.
...
Inclusion of if_packet.h was missing when the support of new options
related to packet fanout is present in the file.
14 years ago
Anoop Saldanha
bf24272c28
changes to accomodate master rebase
14 years ago
Anoop Saldanha
997eaf42a8
add thread local storage support for smtp + remove pmq that was init/freed as part of smtp_state alloc to use the thread local data passed by the app layer engine
14 years ago
Anoop Saldanha
9a6aef459e
modify all relevant app layer API calls to accomodate passing parser local storage argument
14 years ago
Anoop Saldanha
d3468d88b0
app layer udp cleanup + update dcerpc udp todo
14 years ago
Anoop Saldanha
01a35bb604
introduce app layer local storage api support
14 years ago
Anoop Saldanha
87599bc78d
minor changes in smtp parser decoder wrt direction check loop + add missing ifdef unittests
14 years ago
Anoop Saldanha
3a856fed12
update detection engine to compare flow alproto with sig_alproto, rather than sm alproto.
14 years ago
Anoop Saldanha
4d38a571cc
smtp reply code mpm phase support added
14 years ago
Anoop Saldanha
4a6908d3e9
fix smtp parser handling fragmented lines + add new unittests to check the same
14 years ago
Anoop Saldanha
2b356dadff
Support for tos keyword added
14 years ago
deltay
211193b0af
Get pidfile from config file if not available in command options
14 years ago
Victor Julien
262a7300d7
flow: shrink Flow datatype
...
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.
Add macro's to check the family, copy the address, etc.
Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.
On 64 bit this shrinks the flow with 8 bytes.
14 years ago
Victor Julien
06904c9024
App Layer cleanup
...
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
14 years ago
Victor Julien
a0b532dc45
stream reassembly: simplify base_seq tracking for protocol detection. Shrinks TcpStream structure.
14 years ago
Victor Julien
7e3c15e54a
stream: improve TCP ssn reuse cleanup.
14 years ago
Victor Julien
9769510ba3
flow: support requeue of flows from closed to new list for TCP ssn reuse.
14 years ago
Anoop Saldanha
4130c5e2b8
if flow has disabled app layer inspection, disable buffering the segments unnecessarily in inline reassembly
14 years ago
Anoop Saldanha
43cbed8c92
enable toclient alproto detection for inline reassembly
14 years ago
Anoop Saldanha
f684b60127
if flow has disabled app layer inspection, disable buffering the segments unnecessarily
14 years ago
Anoop Saldanha
08bd8ec4e2
on failed alproto detection on both sides, only disable app layer inspection. No reassembly disabling for any direction
14 years ago
Victor Julien
c9960473bb
Fix stream reassembly engine rejecting valid packet for reassembly.
14 years ago
Anoop Saldanha
55ed6c2a55
disable session reassembly for either/both the directions, only when we have established failed proto detection in both the directions
14 years ago
Anoop Saldanha
4650bf7170
minor code cleanup. remove commented out code
14 years ago
Anoop Saldanha
de9ad02b59
Remove leftover imap and msn toclient alproto PM contents
14 years ago
Anoop Saldanha
caf26c2618
More updates to FFR code. Handle cases where we actually need to force stream reassembly and just have smsgs to be processsed by detection engine separately
14 years ago
Anoop Saldanha
bc216a3396
fix/updates to app layer proto detection
14 years ago
Anoop Saldanha
78e6a7f713
enable toclient alproto detection. Detection all current alproto toclient PMP patterns
14 years ago
Anoop Saldanha
9c8d404db1
FFR update-fix. Fix check where we decide whether we need to send pseudo pkt or not
14 years ago
Anoop Saldanha
b08b390bcd
fix for bug 375 - update radix test that wrongly uses memset and sizeof
14 years ago
Victor Julien
3d845b6c77
Consider Windows new line chars as well when parsing rule files. Bug #374 .
14 years ago
Eileen Donlon
a92d15ed37
Fixed duplicate signature check
14 years ago
Anoop Saldanha
99baf18c8d
updates to ac-gfbs search. Remove unnecessary casting of pointers
14 years ago
Anoop Saldanha
11e7dda59a
updates to ac-gfbs search. Introduce handling cases where state_count is < 32k
14 years ago
Anoop Saldanha
708c4ad055
updates to ac-gfbs search. Combine output presence with mod goto table
14 years ago
Anoop Saldanha
a4ea7e6197
updates to ac-gfbs search. Combine failure table along with mod goto table for better cache perf
14 years ago
Anoop Saldanha
b69ac9514f
updates to ac-gfbs search. Disable handling < 65k states separately. Now any state count would be given same treatment
14 years ago
Anoop Saldanha
efb4c27b1f
updates to ac-gfbs search. Add new unittests + fix cases where we have 2 patterns that are same but one is CS and other CI + Use SCMemcmp for state < 65k instead of custom memcmp
14 years ago
Anoop Saldanha
0920296aaa
updates to ac-gfbs search. Remove unnecessary casting of pointers
14 years ago
Anoop Saldanha
d149a5e806
updates to ac-gfbs search. Use SCMemcmp instead of the custom pattern searching used
14 years ago
Anoop Saldanha
47f2d6e07b
updates to ac-gfbs search. Optimize pointer de-referencing for pid_pat_list
14 years ago
Anoop Saldanha
991f6d2d83
updates to ac-gfbs search. Optimize pointer de-referencing for frequently used pointers
14 years ago
Anoop Saldanha
ffb925e3b3
indentation fixes for ac-gfbs
14 years ago
Anoop Saldanha
e9eb0e502c
updates to ac-gfbs search. Handle cases where we have a single entry for a state goto transition, just like how we handle for no entry for a particular state
14 years ago
Eric Leblond
9b75de3339
pfring: fix compilation when pfring is desactivated.
14 years ago
Eric Leblond
0ac1cabf2a
autotools: fix problem of pfring configuration.
14 years ago
deltay
d5e254d504
Add pfring bpf filter, require pfring >= 5.1
14 years ago
Eric Leblond
9f73503daa
capability: rework capability assignement
...
THis patch rework the capability code to use a switch
instead of a if. It also "reduces" PF_RING and NFQ capabilities.
14 years ago
Anoop Saldanha
d034b10180
remove debug prints added to ac code
14 years ago
Anoop Saldanha
781e7c776f
fix indentation in ac code
14 years ago
Anoop Saldanha
5c56053a33
Reintroduced optimized support for < 32k states for ac
14 years ago
Victor Julien
fb76561b09
Set version to 1.2dev to reflect we're in the 1.2 branch.
14 years ago
Victor Julien
8cc82c7241
Add -S commandline option that loads a rule file exclusively. Issue #338 .
14 years ago
Victor Julien
c484b7a59e
Bump version to 1.1 (final)
14 years ago
Eric Leblond
62e63e3fe9
af-packet: fix reconnection on netdown error.
...
AFPRead can fail following a NETDOWN error. This patch treat errors
of AFPRead by forcing a reconnection (instead of exiting thread
with error).
14 years ago
Eric Leblond
361bf22121
af-packet: suppress annoying debug message.
...
This message was firing multiple per second when a monitored
interface disappear.
14 years ago
Victor Julien
0fadd93011
Fix an invalid free in bpf code.
14 years ago
Victor Julien
ea53f72f7d
Fix CUDA build.
14 years ago
Eric Leblond
9f7ee03deb
log: read output filter from config file.
...
The output filter was not read from configuration file and thus
not used in this case.
14 years ago
Eric Leblond
866d681ff2
pfring: fix stupid enum usage.
...
pfring code is not using standard notation for the cluster_type enum
and this leads to a horrific code in pfring acquisition code.
14 years ago
Eric Leblond
a6a0d4eae6
pfring: use deinit function.
...
This fixes #368 .
14 years ago
Eric Leblond
a54afe7052
Fix printing of sizeof.
14 years ago
Victor Julien
2d16abcf8b
Minor code cleanups fixing all GCC 4.6 compiler warnings for default, debug and unittests mode.
14 years ago
Eric Leblond
2387c6b0e8
pcap: Fix setting of buffer size from command line.
14 years ago
Victor Julien
1be65e7b68
Fixes for building in Cygwin.
14 years ago
Victor Julien
85033f5afe
Fix windows adapter id being truncated for pcap mode.
14 years ago
Eric Leblond
2bc0be6e65
af-packet: fix compilation problem on windows.
14 years ago
Victor Julien
404868c28b
Get rid of strcasestr call as win32 doesn't have it.
14 years ago
Victor Julien
561630d864
Fix SMTP unittest.
14 years ago
Victor Julien
47abd0ef19
Fix compiler warning.
14 years ago
Anoop Saldanha
0acfcc206c
fix unittests. fix replace unittests that allow alproto keywords with replace
14 years ago
Anoop Saldanha
a0eec3d846
fix detection code that handles cases when we use recursion(from recursive keyword)
14 years ago
Anoop Saldanha
7433d92dd2
undo this commit -
...
commit eff08f93d8
Author: Anoop Saldanha <poonaatsoc@gmail.com>
Date: Thu Nov 3 14:31:24 2011 +0530
update failing unittest to reflect the mpm design update
Fixed a bug in the mpm code that would make all the changes in the commit just undone wrong.
14 years ago
Anoop Saldanha
1b1332fff0
fix mpm bug on running stream mpm for packets not added to stream mpm
14 years ago
Victor Julien
9f0e3f7c85
Bump version to 1.1rc1.
14 years ago
Victor Julien
55da9787a4
Win32 compile fixes.
14 years ago
Victor Julien
d070869c48
Reinstate replace validation check.
14 years ago
Anoop Saldanha
eff08f93d8
update failing unittest to reflect the mpm design update
14 years ago
Victor Julien
af51493da2
Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm.
14 years ago
Anoop Saldanha
539ce13695
fix broken unittests
14 years ago
Anoop Saldanha
17f3f36d38
packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same
14 years ago
Anoop Saldanha
e0c36f7aff
fix dsize sigs handling. We can't use more than 2 dsizes in the same sig
14 years ago
Anoop Saldanha
c7b9d3fecb
Remove broken dsize_sm in SigMatch used by dsize in detection engine
14 years ago
Anoop Saldanha
d3ca65de03
support bdat smtp keyword - bug #347
14 years ago
Victor Julien
91957d70a8
Fix unittest compiler warning.
14 years ago
Victor Julien
b5e17ec1d8
Rewrite SetupLogging to not leak the fd. Thanks to Steve Grubb for advice on this.
14 years ago
Eric Leblond
ba9fb53461
threshold: fix handling of multiple threshold.
...
This patch fixes the unittest and may fix the real work.
14 years ago
Eric Leblond
142fe6e4b6
threshold: fix recently introduced function.
14 years ago
Victor Julien
e0d7f64a14
Fix thresholding code suppressing an alert if no threshold/suppress rules needed to be checked.
14 years ago
Eric Leblond
86f9759427
threshold: fix thresholding on signature with multiple threshold.
...
This patch uses the newly introduced SigGetThresholdTypeIter
function to try all threshold for a signature. This should fix
issue #366 .
14 years ago