aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,799 Commits
8 Branches
163 Tags
184 MiB
main-7.0.x
main-8.0.x
main
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'ee7e689b54'
${ noResults }
Commit Graph

7049 Commits (ee7e689b5423295d17f1560e2a3b1a1491cdf314)

Author SHA1 Message Date
Jason Ish bbaa79b80e DNP3: Application layer decoder. 
Decodes TCP DNP3 and raises some DNP3 decoder alerts.
 9 years ago
Jason Ish da40714cb1 common: define json_boolean when not defined 
Older versions of jansson in current use don't have this
macro defined.
 9 years ago
fooinha f6c0abaae7 eve: check redis reply in non pipeline mode 
We may lose the reply if disconnection happens.
Reconnection is needed.
 9 years ago
Victor Julien 2758f82515 flowvar: cleanups 9 years ago
Jason Ish 9d271e9a71 fast-pattern: fix tls_sni 
Use all 38 arguments in call to SigMatchGetLastSMFromLists

Was preventing fast_pattern from being applied to tls_sni:
https://redmine.openinfosecfoundation.org/issues/1936
 9 years ago
Jason Ish 7d734edca8 dns: use new unittest macros 9 years ago
Jason Ish a8f6fb0f78 dns: support back to back requests without a response 
Address the issue where a DNS response would not be logged when
the traffic is like:
- Request 1
- Request 2
- Response 1
- Response 2
which can happen on dual stack machines where the request for A
and AAAA are sent out at the same time on the same UDP "session".

A "window" is used to set the maximum number of outstanding
responses before considering the olders lost.
 9 years ago
Jason Ish 64cc91a569 tcp dns: unit test for multi-request buffer 9 years ago
Jason Ish 2d4df19401 tcp dns: fix advancement to next request in buffer 
The advancement through the buffer was not taking into account
the size of the length field resulting in the second request
being detected as bad data.
 9 years ago
Victor Julien db1c47cb6e multi-tenant: make less verbose 9 years ago
Victor Julien 51bb1f0d77 multi-tenants: fix minor memleak 9 years ago
Victor Julien 059b25b564 detect: suppress debug message for reloads 9 years ago
Victor Julien 321fb6463e vars: small cleanups 9 years ago
Victor Julien e4b2729399 nfq: support bypass for rebuilt fragment packets 9 years ago
Victor Julien 629fa30345 nfq_set_mask: set mark on root pkt for tunnels 9 years ago
Eric Leblond d8acf3542d source-nfq: document bypass function 9 years ago
Eric Leblond e0000eb58d source-nfq: fix tunnel mark callback algorithm 
In case of a tunnel packet, adding a mark to the root packet will have
for consequence to bypass all the flows that are hosted in this tunnel.
This is not the attended behavior and as initial fix let's simply warn
suricata that bypass for NFQ is not possible for this kind of packets.

This patch also fixes a segfault. The root packet was accessed even if it is
NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
 9 years ago
Victor Julien 397c541c09 detect: fix multi-tenant loaders 9 years ago
Victor Julien 7e54ee7d0e flow-timeout: fix memory errors on flow bypass 
For flow bypass, the flow timeout handling is triggered which may
create up to 3 pseudo packets that hold a reference to the flow.
However, in the bypass case the code signaled to the timeout logic
that the flow can be freed unconditionally by returning 1. This
lead to packets going through the engine with a pointer to a now
freed/recycled flow.

This patch fixes the logic by removing the special bypass case,
which seemed redundant anyway. Effectively reverts 68d9677.

Bug #1928.
 9 years ago
Victor Julien d1d618a668 flow-manager: cleanups and comment improvements 9 years ago
Victor Julien 368d5d931c flow-timeout: don't leak flow reference in error path 9 years ago
Victor Julien e072e70ea6 alert: fix rate_filter issues 
Fix rate_filter issues: if action was modified it wouldn't be logged
in EVE. To address this pass the PacketAlert structure to the threshold
code so it can flag the PacketAlert as modified. Use this in logging.

Update API to use const where possible. Fix a timout issue that this
uncovered.
 9 years ago
Jason Ish dcdf160ab2 conf: cleanup compiler warning (unintialized vars) 9 years ago
Jason Ish 8f56c23468 detect-flow: no_frag and only_frag keyword support 
Support flow:no_frag and flow:only_frag keywords from Snort.
 9 years ago
Jason Ish f81619a13e defrag: set flag on packets reassembled from fragments 
Set the PKT_REBUILT_FRAGMENT on packets that are re-assembled
from fragments.
 9 years ago
Jason Ish 571f56cfcf detect-flow: support flow:not_established 9 years ago
Jason Ish dc762cd44d detect-flow: use new unit test macros 9 years ago
Duarte Silva 6948b2332a file-hashing: Fixed line parsing code 9 years ago
Victor Julien 449c93e062 detect-app-layer-protocol: improve rule validation 
Also add tests for PD-only conditions
 9 years ago
Victor Julien 0ed119068d detect-app-layer-protocol: implement prefilter 
Introduce 'Protocol detection'-only rules. These rules will only be
fully evaluated when the protocol detection completed. To allow
mixing of the app-layer-protocol keyword with other types of matches
the keyword can also inspect the flow's app-protos per packet.

Implement prefilter for the 'PD-only' rules.
 9 years ago
Victor Julien 8094b2b12e detect-app-layer-protocol: convert to pkt match 9 years ago
Victor Julien c28d9d0538 eve: print app_proto_ts/app_proto_tc 9 years ago
Victor Julien dbb3a12b32 logging: return string for ALPROTO_FAILED 9 years ago
Victor Julien 93298e91c7 app-layer counters: count failed protocol detect 9 years ago
Victor Julien 3b98feef01 proto-detect: clean up UDP handling 
Set FAILED instead of using a flow flag. Flag packets in both
sides when detection is done. Detection is only done in one
direction.
 9 years ago
Victor Julien 90bf2b5a32 proto detect: improve error case handling 
Improve flags logic, update tests.
 9 years ago
Victor Julien e955cf3366 detect-app-layer-protocol: improve error handling 
Redo tests.
 9 years ago
Victor Julien 9560e8b5b2 proto-detect: update mismatch handling 
Improve protocol mismatch handling. Preserve both protos. Use otherdir
if already sent to parser, use toclient otherwise.
 9 years ago
Victor Julien 7d7ec78cc3 app-layer-protocol: improve detection 
Add negated matches to match list instead of amatch.

Allow matching on 'failed'.

Introduce per packet flags for proto detection. Flags are used to
only inspect once per direction. Flag packet on PD-failure too.
 9 years ago
Victor Julien ac2cf526f1 proto detect: remove flow data tracking 
The Flow::data_al_so_far was used for tracking data already
parsed when protocol for the current direction wasn't known yet. As
this behaviour has changed the tracking can be removed.
 9 years ago
Victor Julien d7c828bcb0 proto detect: update behavior on partial detection 
When the current direction doesn't get a protocol detection, but the
opposing direction did, previously we would send the current data to
the parser. Then when we'd be invoked again (until the protocol
detection finally failed) we'd get the same data + the new data. To
make sure we'd not send the same data to the parser again, the flow
kept track of how much was already sent to the app-layer using
data_al_so_far.

This patch changes the behaviour. Instead of sending the data for
the current direction right away, we only do this when protocol
detection is complete. This way we won't have to track anything.
 9 years ago
Victor Julien 6022fa44a5 proto detect: TCP cleanup 
Split function into multiple smaller ones.
 9 years ago
Victor Julien 8347aa01fa app-layer: clean up counters registration 9 years ago
Victor Julien b789d2ae3d tls: change 'no-reassemble' option to default off 
This option was broken so there should be no visible change to
actual deployments.
 9 years ago
Thomas Andrejak c17402fdcb prelude: add IPv6 support 9 years ago
Thomas Andrejak dcce225102 prelude: add missing TCP header to additional data 9 years ago
Thomas Andrejak e33060cee0 prelude: coding style, it's better to use macro 9 years ago
Thomas Andrejak b1c1699699 prelude: Add other actions than just ACTION_DROP when packet drop 9 years ago
Thomas Andrejak 4d4a3d0b8f prelude: Add log when failed to create assessment or impact object 9 years ago
Thomas Andrejak 18c9312380 Add macro for TCP and UDP header access 9 years ago
Eric Leblond 4eca40ac34 app-layer-tls: stop detection if no reassembly 
It no-reassembly is asked in TLS conf then we can stop inspection
after handshake and cause bypass to be triggered on the flow.
 9 years ago
Eric Leblond 69e1ff7ba7 stream-tcp: bypass encrypted when both side ready 
Suricata should not completely bypass a flow before both end of it
have reached the stream depth or have reached a certain state.
Justification is that suricata need the ACK to treat the other side
so we can't really decide to cut only one side.
 9 years ago
Nicolas Thill e95e6ccded lua: add an SCPacketTimestamp function 
The SCPacketTimestamp function returns packet timestamps as 2 real
numbers (seconds & microseconds).

Example:

  local sec, usec = SCPacketTimestamp()

Signed-off-by: Nicolas Thill <ntl@p1sec.com>
 9 years ago
Victor Julien f4b165de94 file: register filedata loggers before file 
This fixes the issue that 'stored' remained false even if the file
was stored.

Reported-by: Chris Wakelin
 9 years ago
Victor Julien 43aed70976 detect: during detection sgh is read only so turn into const 9 years ago
Victor Julien 0e31124609 detect: add util func for post-inspect tasks on first sgh 9 years ago
Victor Julien d3fb4de1b5 detect: move file flags update into it's own function 9 years ago
Victor Julien 664f9aa906 flow: use BIT_U32 for flags 9 years ago
Victor Julien c81aaeda7b flow: move file flags into their own variable 
Move FLOW_FILE_* flags into Flow::file_flags. Rename them to
FLOWFILE_* so non updated code will break.
 9 years ago
Jason Ish 3fab684f97 logging: don't log that json is disabled in each logger 
A warning log is already emitted if eve-log is enabled in the
configuration but json support is not built so the logger
registration functions can be silent.
 9 years ago
Jason Ish 0bce4b5534 macOS: thread return value affects newer macOS as well 
ALl OS X/macOS versions since 10.10 return EDEADLK here instead
of EBUSY. Assume they will moving forward as well.
 9 years ago
Victor Julien f867bb61e6 http: fix memory leak in error path 9 years ago
Victor Julien 40af9aad02 streaming: improve error handling 
When memory allocations happened in HTTP body and general file
tracking, malloc/realloc errors (most likely in the form of memcap
reached conditions) could lead to an endless loop in the buffer
grow logic.

This patch implements proper error handling for all Append/Insert
functions for the streaming API, and it explicitly enables compiler
warnings if the results are ignored.
 9 years ago
Victor Julien 879c3d8ad7 detect: fix scan-build 0-size alloc warnings 9 years ago
Jason Ish 09c3e1dd8a pcap-log: cleanup allocations at exit 
Particularly in multi-mode, allocations made for each thread were
not being cleaned.

ASAN reports no leaks now on exit.
 9 years ago
Victor Julien f80ce51ddf unix-socket: don't try to change permissions on BSD 
On BSD using fchmod on a socket is not supported and will result
in EINVAL.
 9 years ago
Victor Julien 96c28b2995 bug 1353: don't cut off last char of unix path 9 years ago
Victor Julien 4a190e07a6 pcre: disable JIT if RWX pages not supported 9 years ago
Victor Julien 46f5f4cff8 util: add facility to check for RWX page support 
Some code won't work well when the OS doesn't allow RWX pages. This
page introduces a check for runtime evaluation of the OS' policy on
this.

Thanks to Shawn Webb from HardenedBSD for suggesting this solution.
 9 years ago
Victor Julien a3a1757472 flow-mgr: fix bypass counter registration 9 years ago
Victor Julien 595c20ddf4 der: fix asan/valgrind errors in time parsing 9 years ago
Victor Julien 7e4df3a1d1 tls-validity: fix memory handling 9 years ago
Mats Klepsland 10d827639e detect-tls-cert-validity: clean up unit tests 
Remove locks, unnecessary function calls and conditional statements.
 9 years ago
Mats Klepsland 1fea52dd8a detect: add keyword tls_cert_valid 
Add keyword to check if TLS certificate is valid.
 9 years ago
Mats Klepsland f7e0083269 detect-cert-validity: fix typos 9 years ago
Mats Klepsland f22c9d9781 detect: add keyword tls_cert_expired 
Add keyword to check if TLS certificate is expired.
 9 years ago
Mats Klepsland 07d2312d96 detect-tls-validity: use flags for modes 
Use flags for modes to support using multiple modes at the same time.
 9 years ago
Giuseppe Longo 3f214b506a file-store: add depth setting 
When a rules match and fired filestore we may want
to increase the stream reassembly depth for this specific.

This add the 'depth' setting in file-store config,
which permits to specify how much data we want to reassemble
into a stream.
 9 years ago
Giuseppe Longo 4751677e24 app-layer: use StreamTcpSetReassemblyDepth 
This calls StreamTcpSetReassemblyDepth to set the stream depth
specified for the protocol.
 9 years ago
Giuseppe Longo 9ab1194f68 modbus: set stream depth 
Some protocol like modbus requires
a infinite stream depth because session
are kept open and we want to analyze everything.

Since we have a stream reassembly depth per stream,
we can also set a stream reassembly depth per proto.
 9 years ago
Giuseppe Longo b160c49e9e app-layer-parser: add stream depth 
This permits to set a stream depth value for each
app-layer.

By default, the stream depth specified for tcp is set,
then it's possible to specify a own value into the app-layer
module with a proper API.
 9 years ago
Eric Leblond a63c6b320e stream: per TcpStream reassembly depth 9 years ago
Victor Julien 960ebb2822 enip: fix scan-build warnings 
detect-cipservice.c:161:29: warning: Assigned value is garbage or undefined
    cipserviced->cipservice = input[0];
                            ^ ~~~~~~~~
detect-cipservice.c:162:27: warning: Assigned value is garbage or undefined
    cipserviced->cipclass = input[1];
                          ^ ~~~~~~~~
detect-cipservice.c:163:31: warning: Assigned value is garbage or undefined
    cipserviced->cipattribute = input[2];
                              ^ ~~~~~~~~
3 warnings generated.
 9 years ago
Victor Julien 80c3aedbfc enip: parsing and tests cleanup 9 years ago
Victor Julien 72b5da4313 enip/cip: improve output & style 
Remove printf, remove \n from SCLogDebug. Add SCLogError for
rule parsing issues.

Fix various style issues
 9 years ago
Victor Julien 6b1c21b115 enip/cip: register inspect engines 9 years ago
kwong a3ffebd835 Adding SCADA EtherNet/IP and CIP protocol support 
Add support for the ENIP/CIP Industrial protocol

This is an app layer implementation which uses the "enip" protocol
and "cip_service" and "enip_command" keywords

Implements AFL entry points
 9 years ago
Victor Julien d9811e58b6 http_header: don't separately inspect trailer yet 
Currently the regular 'Header' inspection code will run each time
after the HTTP progress moved beyond 'headers'. This will include
the trailers if there are any.

Leave the code in place as this model will change in the not too
distant future.
 9 years ago
Victor Julien 358eacf14f http_header: only run trailer mpm if we have trailers 9 years ago
Victor Julien 44022743f2 http: track if request/response have trailers 9 years ago
Victor Julien 798ba010ca prefilter: use array of engines per sgh 
Instead of the linked list of engines setup an array
with the engines. This should provide better locality.

Also shrink the engine structure so that we can fit
2 on a cacheline.

Remove the FreeFunc from the runtime engines. Engines
now have a 'gid' (global id) that can be used to look
up the registered Free function.
 9 years ago
Victor Julien 8321f04ef3 prefilter: clean up setup code 9 years ago
Victor Julien d36c0c15ea detect: reshuffle keyword registration order 
The order of keyword registration currently affects inspect engine
registration order and ultimately the order of inspect engines per
rule. Which in turn affects state keeping.

This patch makes sure the ordering is the same as with older
releases.
 9 years ago
Victor Julien 58ac4027ef detect: clean up inspect engine registration 9 years ago
Victor Julien a24870f29f detect app-layer-event: clean up registration 
Move engine and registration into the keyword file.

Register as 'ALPROTO_UNKNOWN' instead of per alproto. The
registration will only apply it to those rules that have
events set.
 9 years ago
Victor Julien 9e35fa7f41 detect: remove empty app registration table 9 years ago
Victor Julien 8a0bea872c template_buffer: register inspect engine from keyword 9 years ago
Victor Julien 6f253e1ea7 file detect: register inspect engines from keyword 9 years ago
Victor Julien 08d0fe0916 modbus detect: register inspect engine from keyword 9 years ago
Victor Julien 2db094ab7a dns detect: register inspect engine from keyword 9 years ago
Victor Julien c9bb762f64 tls_cert_issuer: register inspect engine from keyword 9 years ago
Victor Julien e28e98bcaa tls_cert_subject: register inspect engine from keyword 9 years ago
Victor Julien a87c196b60 tls_sni: register inspect engine from keyword 9 years ago
Victor Julien 200a4c1593 http_stat_code: register inspect engine from keyword 9 years ago
Victor Julien cd705752db http_stat_msg: register inspect engine from keyword 9 years ago
Victor Julien 20e93ba419 file_data: register inspect engine from keyword 9 years ago
Victor Julien 0496b3f6a5 http_raw_host: register inspect engine from keyword 9 years ago
Victor Julien a00629ab55 http_host: register inspect engine from keyword 9 years ago
Victor Julien edb2936998 http_user_agent: register inspect engine from keyword 9 years ago
Victor Julien fc857c5455 http_raw_uri: register inspect engine from keyword 9 years ago
Victor Julien b1adea6eee http_cookie: register inspect engine from keyword 9 years ago
Victor Julien cd8b1b0b4c http_method: register inspect engine from keyword 9 years ago
Victor Julien b314829614 http_raw_header: register inspect engine from keyword 9 years ago
Victor Julien eb19eb3fe4 http_header: register inspect engine from keyword 9 years ago
Victor Julien 4096f76b1b http_client_body: register inspect engine from keyword 9 years ago
Victor Julien b96c2c5db5 http_uri: register inspect engine from keyword 9 years ago
Victor Julien cc96fedb90 http_response_line: register inspect engine from keyword 9 years ago
Victor Julien 0feeb8d538 http_request_line: register inspect engine from keyword 9 years ago
Victor Julien 5bde86b0e8 detect-engine: new registration call 
Make it more in line with MPM registration.
 9 years ago
Victor Julien 9a0bbd6239 detect mpm: small optimization 9 years ago
Victor Julien ad3c97f470 detect-mpm: cleanup 9 years ago
Victor Julien 5f994756e6 detect-engine: improved inspect engines 
Inspect engines are called per signature per sigmatch list. Most
wrap around DetectEngineContentInspection, but it's more generic.

Until now, the inspect engines were setup in a large per ipproto,
per alproto, per direction table. For stateful inspection each
engine needed a global flag.

This approach had a number of issues:
1. inefficient: each inspection round walked the table and then
   checked if the inspect engine was even needed for the current
   rule.
2. clumsy registration with global flag registration.
3. global flag space was approaching the need for 64 bits
4. duplicate registration for alprotos supporting both TCP and
   TCP (DNS).

This patch introduces a new approach.

First, it does away with the per ipproto engines. This wasn't used.

Second, it adds a per signature list of inspect engine containing
only those engines that actually apply to the rule.

Third, it gets rid of the global flags and replaces it with flags
assigned per rule per engine.
 9 years ago
Victor Julien bac37fc9ae detect state: reorganize flags 
List the common non-buffer specific flags on top.
 9 years ago
Victor Julien f1e3840516 http_response_body: implement keyword with mpm 
Implemented as 'stickybuffer'.
 9 years ago
Victor Julien 4c98b6cef3 http_request_line: implement keyword and mpm 
Implemented as 'stickybuffer'.

Move all logic into the keyword file and remove bad tests that tested
URI instead of request line.
 9 years ago
Victor Julien 960461f4db fast_pattern: register app layer mpms automatically 
Allow for duplicate registrations for the same list. After the first
registration new calls will be ignored.
 9 years ago
Victor Julien 6dd4dff7b2 mpm: remove empty app_mpms table 9 years ago
Victor Julien e68b2214e5 tls: register mpm from keywords 9 years ago
Victor Julien 57ae3c43e5 dns_query: register mpm from keyword 9 years ago
Victor Julien a1a2187a0c http_cookie: register mpm from keyword 9 years ago
Victor Julien 74661449e0 http_raw_host: register mpm from keyword 9 years ago
Victor Julien b5cd4889ae http_host: register mpm from keyword 9 years ago
Victor Julien 91695c81aa http_client_body: register mpm from keyword 9 years ago
Victor Julien 644d4dc61b http_stat_code: register mpm from keyword 9 years ago
Victor Julien cf96db095a http_stat_msg: register mpm from keyword 9 years ago
Victor Julien 43b281a510 file_data: register mpm from keyword 9 years ago
Victor Julien 6d0632a9c6 http_method: register mpm from keyword 9 years ago
Victor Julien e4ea38a8de http_raw_header: register mpm from keyword 9 years ago
Victor Julien 7813a834d0 http_user_agent: register mpm from keyword 9 years ago
Victor Julien 7b98c0073f http_header: register mpm from keyword 9 years ago
Victor Julien 38e018e2d3 http_raw_uri: register mpm from keyword 9 years ago
Victor Julien 7289d12f1b http_uri: register mpm from keyword 9 years ago
Victor Julien 5b2e36a1b0 mpm: add App Layer MPM registery 
Register keywords globally at start up.

Create a map of the registery per detection engine. This we need because
the sgh_mpm_context value is set per detect engine.

Remove APP_MPMS_MAX.
 9 years ago
Victor Julien ae5846b4de detect: simplify content inspection types 
Instead of a type per buffer type, pass just 3 possible types:
packet, stream, state.

The individual types weren't used. State is just there to be
not packet and not stream.
 9 years ago
Victor Julien e1eb481647 prefilter: cleanup and optimization 9 years ago
Victor Julien dba14b676c profiling: more prefilter profiling 9 years ago
Victor Julien 125603871b detect: config opt to enable keyword prefilters 9 years ago
Victor Julien 36f713c8d4 prefilter: in profiling print totals 9 years ago
Victor Julien 2e878c2024 prefilter: alloc CLS aligned memory 9 years ago
Victor Julien 732921922a detect mpm: consider sgh direction when adding rules 9 years ago
Victor Julien 9bb12ccb27 prefilter: move payload engines into separate list 9 years ago
Victor Julien e3b98d5bbf detect-ack: extra match support 9 years ago
Victor Julien a41bf2ae14 detect-seq: extra match support 9 years ago
Victor Julien a1accbbaf0 detect-ttl: extra match support 9 years ago
Victor Julien a270dfa008 detect-id: extra match support 9 years ago
Victor Julien fbb0490c31 detect-dsize: extra match support 9 years ago
Victor Julien 34e3484dad detect-flags: prefilter extra match support 9 years ago
Victor Julien ace8f9f5df detect-flow: prefilter extra match support 9 years ago
Victor Julien e2eb9f8ede prefilter: add 'extra match' logic to packet engines 
Many of the packet engines are very generic. Rules are generally more
limited.

A rule like 'alert tcp any any -> any 888 (flags:S; sid:1;)' would still
be inspected against every SYN packet in most cases (it depends a bit on
rule grouping though).

This extra match logic adds an additional check to these packet engines.
It can add a check based on alproto, source port and dest port. It uses
only one of these 3. Priority order is src port > alproto > dst port.

For the ports only 'single' ports are used at this time.
 9 years ago
Victor Julien 9187c20782 detect mpm: negated setup fix 9 years ago
Victor Julien 5537e25f38 detect-icmp-id: prefilter 9 years ago
Victor Julien fbe7e0aaeb detect-icmp-seq: prefilter 9 years ago
Victor Julien 3a86aeac65 detect-icode: implement as u8 hash prefilter 9 years ago
Victor Julien 6a3917b375 detect-itype: implement as u8 hash prefilter 9 years ago
Victor Julien f5d2166e23 detect-id: implement prefilter 9 years ago
Victor Julien d5e5c11bd1 detect-icode: implement prefilter 9 years ago
Victor Julien 10f8e636d6 detect-itype: implement prefilter 9 years ago
Victor Julien b88c0a56b9 detect-ttl: implement prefilter 9 years ago
Victor Julien 9ce300620e detect-seq: implement prefilter 9 years ago
Victor Julien 822e034753 detect-flow: implement prefilter 9 years ago
Victor Julien 14b0537f95 prefilter: implement basic prefilter priority order 9 years ago
Victor Julien 4104f8c066 detect-fragoffset: implement prefilter 9 years ago
Victor Julien 9195708d58 detect analyzer: give minimal prefilter info 9 years ago
Victor Julien 065d9bceae detect-dsize: enable prefilter support 
Enable prefilter support for the dsize keyword.
 9 years ago
Victor Julien 9ccd0c0f90 prefilter: implement fragbits 9 years ago
Victor Julien 3b4aa06377 prefilter: engine for ack rules 
Rules for the 'ack' keyword are uncommon, but if used inspected
against almost every packet.
 9 years ago
Victor Julien 31ad0a133b prefilter: engine for tcp flags keyword 
If there are many rules for TCP flags these rules would be inspected
against each TCP packet. Even though the flags check is not expensive,
the combined cost of inspecting multiple rules against each and every
packet is high.

This patch implements a prefilter engine for flags. If a rule group
has rules looking for specific flags and engine for that flag or
flags combination is set up. This way those rules are only inspected
if the flag is actually present in the packet.
 9 years ago
Victor Julien 8798bf48b2 profiling: support prefilter engines 9 years ago
Victor Julien ea26ee906f prefilter: intro common engine for u8 matches 9 years ago
Victor Julien 99b9896bd7 prefilter: common funcs for packet header prefilters 9 years ago
Victor Julien f80623fd73 prefilter: show prefilter capability in --list-keywords 9 years ago
Victor Julien 56239690d0 prefilter: implement prefilter keyword 
Introduce prefilter keyword to force a keyword to be used as prefilter.

e.g.
alert tcp any any -> any any (content:"A"; flags:R; prefilter; sid:1;)
alert tcp any any -> any any (content:"A"; flags:R; sid:2;)
alert tcp any any -> any any (content:"A"; dsize:1; prefilter; sid:3;)
alert tcp any any -> any any (content:"A"; dsize:1; sid:4;)

In sid 2 and 4 the content keyword is used in the MPM engine.
In sid 1 and 3 the flags and dsize keywords will be used.
 9 years ago
Victor Julien 85cb749e8b detect cleanup: remove sgh mpm_ctx pointers 9 years ago
Victor Julien 82d3c0b520 sgh: remove unused flags 9 years ago
Victor Julien 08407b6d47 tls: mpm prefilter engines 9 years ago
Victor Julien 7acdc66061 smtp file_data: mpm prefilter engine 9 years ago
Victor Julien 0019a7bd9f http_raw_header: mpm prefilter engine 
Register for both regular headers and trailer.
 9 years ago
Victor Julien cef12ed80f http_server_body / file_data: mpm prefilter engine 9 years ago
Victor Julien 5646dd9ecf http_client_body: mpm prefilter engine 9 years ago
Victor Julien 9b6fd6bb48 http_headers: mpm prefilter engines 
Register for both regular headers and trailers.
 9 years ago
Victor Julien 9cab3ea2cd http_stat_code: mpm prefilter engine 9 years ago
Victor Julien 4d57b2fc63 http_stat_msg: mpm prefilter engine 9 years ago
Victor Julien 86d303e32b http_raw_host: mpm prefilter engine 9 years ago
Victor Julien 5218849213 http_host: mpm prefilter engine 9 years ago
Victor Julien 61c3748fc4 http_user_agent: mpm prefilter engine 9 years ago
Victor Julien a43a69305d http_cookie: mpm prefilter engine 9 years ago
Victor Julien 7a46364e42 http_raw_uri: mpm prefilter engine 9 years ago
Victor Julien 746a169127 dns_query: mpm prefilter engine 9 years ago
Victor Julien 9ff5703c49 packet/stream: mpm prefilter engine 9 years ago
Victor Julien 72f2a78b1f http_method: mpm prefilter engine 9 years ago
Victor Julien b62c4cc359 http_uri: mpm prefilter engine 
Inspect partial request line as well.
 9 years ago
Victor Julien 5bcdbe3922 prefilter: introduce prefilter engines 
Introduce abstraction layer for prefilter engines.
 9 years ago
Victor Julien 3dad824fb2 detect: rename SignatureNonMpmStore 
New name is SignatureNonPrefilterStore to reflect that it's not just
about MPM anymore.
 9 years ago
Victor Julien 17bc0299fe detect: rename non_mpm lists/vars to non_pf 
Rename to non_pf: non prefilter.
 9 years ago
Victor Julien bb0cd0e883 prefilter: rename PatternMatcherQueue datatype 
In preparation of the introduction of more general purpose prefilter
engines, rename PatternMatcherQueue to PrefilterRuleStore. The new
engines will fill this structure a similar way to the current mpm
prefilters.
 9 years ago
Victor Julien 4c0ab681f2 mpm: remove Cleanup API call 
It's unused by all of the implementations.
 9 years ago
Victor Julien 7c47016913 detect-fragoffset: minor cleanup 9 years ago
Victor Julien a41695f29f uricontent: remove left over func decl 9 years ago
Victor Julien ff70e0cca0 mpm tls: remove unused function args 9 years ago
Victor Julien ad3a55d938 mpm dns query: remove unused function args 9 years ago
Victor Julien d647db1775 mpm stat code: remove unused function args 9 years ago
Victor Julien bd03307921 mpm stat msg: remove unused function args 9 years ago
Victor Julien 6d54b70db4 mpm ua: remove unused function args 9 years ago
Victor Julien 704afeb078 mpm cookie: remove unused function args 9 years ago
Victor Julien 4229e603f0 mpm raw host: remove unused function args 9 years ago
Victor Julien 1380853ee8 mpm host: remove unused function args 9 years ago
Victor Julien b40ecb7356 mpm method: remove unused function args 9 years ago
Victor Julien 3d5807ba44 mpm raw uri: remove unused function args 9 years ago
Victor Julien d461c7888a mpm uri: remove unused function args 9 years ago
Victor Julien c4dcb20522 detect-parse: add new func to get last sigmatch 
Add SigMatchGetLastSM which simply returns the very last SM added
to the signature.

Minor cleanups.
 9 years ago
Eric Leblond 3ca663d7ff output-json-flow: display bypass method 
In the case of a bypassed flow we add a 'bypass' key that can
be 'local' or 'capture'. This will allow the user to know if
capture bypass method is failing by looking at the 'bypass' key.
 9 years ago
Giuseppe Longo e6bac998d9 flow: add timeout for local bypass 
This adds a new timeout value for local bypassed state. For user
simplication it is called only `bypassed`. The patch also adds
a emergency value so we can clean bypassed flows a bit faster.
 9 years ago
Eric Leblond 51bfe4960a flow: discard packets belonging to bypassed flows 9 years ago
Eric Leblond 724069626d flow: downgrade to local bypass if we see packets 
If we see packets for a capture bypassed flow after some times, it
means that the capture method is not handling correctly the bypass
so it is better to switch to local bypass method.
 9 years ago
Eric Leblond 4cf887b4f7 flow: update lastts in FlowHandlePacketUpdate 
This allows to make it conditional to the state of packet and
then trigger modified behavior.
 9 years ago
Giuseppe Longo 5b71b5834f filestore: avoid conflict with bypass keyword 
If a packet triggers a rule which contains both
bypass and filestore keywords,
it won't be stored since it's not inspected.

To avoid that, when a rule containing filestore keyword
we make sure that also bypass keyword is present.
 9 years ago
Giuseppe Longo 07564c4e41 detect: add bypass keyword 
This adds a new keyword which permits to call the
bypass callback when a sig is matched.

The callback must be called when the match of the sig
is complete.
 9 years ago
Eric Leblond c19cd12620 flow: bypass encrypted and after stream depth flow 
This patch activates bypass for encrypted flow and for flow
that have reached stream depth on both side.

For encrypted flow , suricata is stopping the inspection so
we can just get it out via bypass. The same logic apply
for flow that have reached the stream depth.

For a basic test of feature, use the following ruleset:

```
table ip filter {
	chain output {
		type filter hook output priority 0; policy accept;
		ct mark 0x1 counter accept
		oif lo counter queue num 0
	}

	chain connmark_save {
		type filter hook output priority 1; policy accept;
		mark 0x1 ct mark set mark counter
		ct mark 0x1 counter
	}
}
```

And use bypass mark and mask of 1 in nfq configuration. Then you
can test the system by scp big file to 127.0.0.1. You can also
use iperf to measure the performance on localhost. It is recommended
to lower the MTU to 1500 to get something more realistic by increasing
the number of packets..
 9 years ago
Giuseppe Longo 177df305d4 stream-tcp: enable bypass setting 
This permits to enable/disable in suricata.yaml
and the bypass function will be called
when stream.depth is reached.
 9 years ago
Giuseppe Longo 97783f8142 nfq: introduce bypass function 9 years ago
Eric Leblond 285b4dd981 decode: implement bypass function 
Call the packet bypass callback if necessary and update the flow
state. In case of failure we switch to local bypassed state and set
capture bypassed state if the callback is successful.
 9 years ago
Eric Leblond 68d9677eea flow: force reassembly for bypassed flows 
As capture method like nfq will cut both side of the flow instantly
we will not get the hack for most data which have been received. So
it is better to force reassembly to be sure to get the timeout of
the entry.
 9 years ago
Eric Leblond 39c8786a8e flow: get bypass info in get used flow function 9 years ago
Eric Leblond 07ef451c2b flow: add pruned bypassed flow counter 9 years ago
Eric Leblond 745dad9809 flow: display info about bypass in log 9 years ago
Eric Leblond e88555caf9 flow: add bypassed states 
This patch adds two new states to the flow:
* local bypass: for suricata only bypass, packets belonging to
a flow in this state will be discard fast
* capture bypass: capture method is handling the bypass and suricata
will discard packets that are currently queued

A bypassed state to flow that will be set on flow when a bypass
decision is taken. In the case of capture bypass this will allow
to remove faster the flow entry from the flow table instead of
waiting for the "established" timeout.
 9 years ago
Giuseppe Longo 616782aa98 packet: add API for bypass 9 years ago
Jason Ish 1f4725fcab detect-tls: make check on fingerprint directional 9 years ago
Jason Ish 44c846f2f8 tls-json: make tls events direction sensitive 
Previously the src/dest ips in TLS events would differ between
IDS and IPS modes. Make the header creation direction sensitive
so they are identical in both modes.
 9 years ago
Mats Klepsland c0f93503b7 util-decode-der-get: fix coverity warning 
*** CID 1373380:  Control flow issues  (DEADCODE)
/src/util-decode-der-get.c: 126 in UtctimeToTime()
120         year = strtol(yy, NULL, 10);
121         if (year >= 50)
122             snprintf(buf, sizeof(buf), "%i%s", 19, utctime);
123         else if (year < 50)
124             snprintf(buf, sizeof(buf), "%i%s", 20, utctime);
125         else
>>>     CID 1373380:  Control flow issues  (DEADCODE)
>>>     Execution cannot reach this statement: "goto error;".
126             goto error;
127
128         time = GentimeToTime(buf);
129         if (time == -1)
130             goto error;
131
 9 years ago
Victor Julien d6f051cdf9 http: removed unused flags 9 years ago
Eric Leblond a194dfbd5b app-layer: tx counter implementation 
This patch adds a transaction counter for application layers
supporting it. Analysis is done after the parsing by the
different application layers.

This result in new data in the stats output, that looks like:
```
    "app-layer": {
      "tx": {
        "dns_udp": 21433,
        "http": 12766,
        "smtp": 0,
        "dns_tcp": 0
      }
    },
```
 9 years ago
Giuseppe Longo 675fa56497 app-layer: add ThreadVars to AppLayerParserParse 
To be able to add a transaction counter we will need a ThreadVars
in the AppLayerParserParse function.
This function is massively used in unittests
and this result in an long commit.
 9 years ago
Giuseppe Longo 5908dd0804 app-layer: add flow counters 
This adds per flow counters for all
supported protocols.

This results in new data in stats output that looks like:
```
    "app-layer": {
      "flow": {
        "http": 9310,
        "ftp": 0,
        "smtp": 0,
        "tls": 71,
        "ssh": 0,
        "imap": 0,
        "msn": 0,
        "smb": 170,
        "dcerpc_udp": 0,
        "dns_udp": 870,
        "dcerpc_tcp": 2,
        "dns_tcp": 0
      },
    },
```
 9 years ago
Eric Leblond 398489e6df stream: fix depth reached detection 
When a segment only partially fit in streaming depth, the stream
depth reached flag was not set resulting in a continuous
inspection of the rest of the session.

By setting the stream depth reached flag when the segment partially
fit we avoid to reenter the code and we don't take anymore a code
path resulting in the flag not to be set.
 9 years ago
Mats Klepsland dc8e0b3cf2 detect: add detect engine for tls validity keywords 
Add detect engine for tls validity keywords (tls_cert_notbefore and
tls_cert_notafter).
 9 years ago
Mats Klepsland d91664d67a detect-dns: move DetectEngineInspectGenericList to detect-engine.c 
Move DetectEngineInspectGenericList from detect-engine-dns.c to
detect-engine.c to enable it to be used other places as well.
 9 years ago
Mats Klepsland cad638697d lua: add lua functions for certificate validity dates 
Add functions TlsGetCertNotBefore and TLSGetCertNotAfter to get notBefore
and notAfter fields from TLS certificate in lua scripts.
 9 years ago
Mats Klepsland 67ea821521 util-lua: add (wrapper) function to push integer to lua scripts 9 years ago
Mats Klepsland ee24949065 log-tls: add notBefore and notAfter fields to extended output 
Add notBefore and NotAfter fields from TLS certificate to extended tls
log output.
 9 years ago
Mats Klepsland 5b230bbce5 output-json-tls: add notBefore and notAfter fields to extended output 
Add notBefore and notAfter fields from TLS certificate to extended JSON
output.
 9 years ago
Mats Klepsland ac4e308140 util-time: add function to create a UTC time string 
Add function CreateUtcIsoTimeString to create a UTC time string.
 9 years ago
Mats Klepsland ea5696812f detect: add tls_cert_notbefore and tls_cert_notafter keywords 
Detection plugin for TLS certificate fields notBefore and notAfter.

Supports equal to, less than, greater than, and range operations
for both keywords. Dates can be represented as either ISO 8601 or
epoch (Unix time).

Examples:
alert tls [...] tls_cert_notafter:1445852105; [...]
alert tls [...] tls_cert_notbefore:<2015-10-22T23:59:59; [...]
alert tls [...] tls_cert_notbefore:>2015-10-22; [...]
alert tls [...] tls_cert_notafter:2000-10-22<>2020-05-15; [...]
 9 years ago
Mats Klepsland c49cb05399 util-time: add function to parse a date string based on patterns 
Add function SCStringPatternToTime to parse a date string based on an
array of pattern strings.
 9 years ago
Mats Klepsland bfd16dc74e app-layer-ssl: add validity dates from certificate 
Parsing of certificate validity dates to get notBefore and notAfter
fields.
 9 years ago
Mats Klepsland 6c1c53b5a1 util-time: add function to convert tm to time_t 
Add function SCMkTimeUtc to convert broken-down time to Unix epoch in UTC.
 9 years ago
Mats Klepsland 03cda74b95 util-decode-der: decode GeneralizedTime 
Decode ASN.1 element type GeneralizedTime in DER-encoded
structures.
 9 years ago
Mats Klepsland b914861692 app-layer-ssl: use new unit test macros 9 years ago
Mats Klepsland 12356d1fca detect-ssl-version: use new unit test macros 9 years ago
Mats Klepsland 1503ac97a6 detect-tls-version: use new unit test macros 9 years ago
Mats Klepsland d9e2cde585 detect-tls-sni: use new unit test macros 9 years ago
Mats Klepsland 8e77d0c312 detect: fix faulty tls_sni unittests 9 years ago
Mats Klepsland 9d23ad9d25 tls: fix faulty unittests 9 years ago
Mats Klepsland b74f3fd978 coverty: fix CID 1361873 9 years ago
Mats Klepsland c36595eb35 tls: set event if input buffer overflows 
Set HANDSHAKE_INVALID_LENGTH event if input buffer overflows while
decoding client_hello/server_hello.
 9 years ago
Mats Klepsland 1f7b813080 app-layer-tls: add name to authors 9 years ago
Mats Klepsland 12da0e8681 tls: add function for decoding client_hello 
Add function TLSDecodeHandshakeHello() to enable using the same code
for decoding both client_hello and server_hello.
 9 years ago
Jason Ish 04da43d65d rule parsing: check for balanced double quotes 
If a rule option value starts with a double quote, ensure it
ends with a double quote, exclusive of white space which gets
trimmed anyways.

Catches errors like 'filemagic:"picture" sid:5555555;' reporting
that a missing semicolon may be the error.
 9 years ago
Victor Julien 48b3cb0492 unittests: fix tests 9 years ago
Victor Julien 6530c3d0d8 unittests: replace SCMutex* calls by FLOWLOCK_* 9 years ago
Victor Julien 682459d640 file: remove dead code 9 years ago
Victor Julien 70c16f50e7 flow-manager: optimize hash walking 
Until now the flow manager would walk the entire flow hash table on an
interval. It would thus touch all flows, leading to a lot of memory
and cache pressure. In scenario's where the number of tracked flows run
into the hundreds on thousands, and the memory used can run into many
hundreds of megabytes or even gigabytes, this would lead to serious
performance degradation.

This patch introduces a new approach. A timestamp per flow bucket
(hash row) is maintained by the flow manager. It holds the timestamp
of the earliest possible timeout of a flow in the list. The hash walk
skips rows with timestamps beyond the current time.

As the timestamp depends on the flows in the hash row's list, and on
the 'state' of each flow in the list, any addition of a flow or
changing of a flow's state invalidates the timestamp. The flow manager
then has to walk the list again to set a new timestamp.

A utility function FlowUpdateState is introduced to change Flow states,
taking care of the bucket timestamp invalidation while at it.

Empty flow buckets use a special value so that we don't have to take
the flow bucket lock to find out the bucket is empty.

This patch also adds more performance counters:

flow_mgr.flows_checked         | Total    | 929
flow_mgr.flows_notimeout       | Total    | 391
flow_mgr.flows_timeout         | Total    | 538
flow_mgr.flows_removed         | Total    | 277
flow_mgr.flows_timeout_inuse   | Total    | 261
flow_mgr.rows_checked          | Total    | 1000000
flow_mgr.rows_skipped          | Total    | 998835
flow_mgr.rows_empty            | Total    | 290
flow_mgr.rows_maxlen           | Total    | 2

flow_mgr.flows_checked: number of flows checked for timeout in the
                        last pass
flow_mgr.flows_notimeout: number of flows out of flow_mgr.flows_checked
                        that didn't time out
flow_mgr.flows_timeout: number of out of flow_mgr.flows_checked that
                        did reach the time out
flow_mgr.flows_removed: number of flows out of flow_mgr.flows_timeout
                        that were really removed
flow_mgr.flows_timeout_inuse: number of flows out of flow_mgr.flows_timeout
                        that were still in use or needed work

flow_mgr.rows_checked: hash table rows checked
flow_mgr.rows_skipped: hash table rows skipped because non of the flows
                        would time out anyway

The counters below are only relating to rows that were not skipped.

flow_mgr.rows_empty:   empty hash rows
flow_mgr.rows_maxlen:  max number of flows per hash row. Best to keep low,
                        so increase hash-size if needed.
flow_mgr.rows_busy:    row skipped because it was locked by another thread
 9 years ago
Victor Julien aee1f0bb99 flow: simplify timeout logic 
Instead of a single big FlowProto array containing timeouts separately
for normal and emergency cases, plus the 'Free' pointer for the
protoctx, split up these arrays.

An array made of FlowProtoTimeout for just the normal timeouts and an
mirror of that for emergency timeouts are used through a pointer that
will be set at init and by swapped by the emergency logic. It's swapped
back when the emergency is over.

The free funcs are moved to their own array.

This simplifies the timeout lookup code and shrinks the data that is
commonly used.
 9 years ago
Victor Julien 96427cf371 flow: remove dead code 9 years ago
Victor Julien da8f3c987b offloading: make disabling offloading configurable 
Add a generic 'capture' section to the YAML:

  # general settings affecting packet capture
  capture:
    # disable NIC offloading. It's restored when Suricata exists.
    # Enabled by default
    #disable-offloading: false
    #
    # disable checksum validation. Same as setting '-k none' on the
    # commandline
    #checksum-validation: none
 9 years ago
Victor Julien 5e3b61cc65 offloading: reduce verbosity to 'perf' 9 years ago
Victor Julien 2b2984dae9 offloading: implement restoring settings for BSD 9 years ago
Victor Julien 499e27de14 offloading: restore settings on exit 9 years ago
Victor Julien 9d48720f9a af-packet: optionally disable offloading 9 years ago
Victor Julien 98092f63b5 offloading: Linux ethtool offloading support 9 years ago
Victor Julien bc370606fc pcap: optionally disable offloading 9 years ago
Victor Julien 723a14c0fe netmap: optionally disable offloading 9 years ago
Victor Julien 2780fba1d1 device: add global flag for disabling offloading 
Add global flag to disable offloading or just warn on it.
 9 years ago
Victor Julien 7004987670 offloading: preparation for disabling offload on BSD 
Add functions for setting IFCAP flags.
 9 years ago
Jason Ish 30c853a304 detect-ssl-state: use new unit test macros 9 years ago
Jason Ish 487cdda93d ssl: issue 1231 - support ssl state negation 
Snort compatible SSL state negation. Adds "," as a state
separator, but keeps "|" for compatibility with existing
Suricata rules.
 9 years ago
Jason Ish afc796a099 ssl: store current state separately from cumulative state 
The ssl_state keyword needs the current state, not the cumulative state
in order be compatible with Snort's implementation.
 9 years ago
Jason Ish 7ce196e3bf detect-pcre: use new unit test macros 9 years ago
Jason Ish 4cdcada397 pcre: fix missing quote in pcre unit test 9 years ago
Victor Julien f7481c4078 file-hashing: restore 'force-md5' 
We don't want to break existing setups.

Do issue a warning that a new option is available.
 9 years ago
Victor Julien 4426f3ff55 file: introduce common flags handling function 9 years ago
Victor Julien 2f5663dfe9 common: introduce BIT_U16 9 years ago
Duarte Silva 53ebe4c538 file-hashing: added configuration options and common parsing code 9 years ago
Duarte Silva 89eb935f73 file-hashing: added support for SHA-256 file hashing 9 years ago
Duarte Silva a6d928e269 file-hashing: added support for SHA-1 file hashing 9 years ago
Duarte Silva 188b382c46 file-hashing: common code added 
Moved and adapted code from detect-filemd5 to util-detect-file-hash,
generalised code to work with SHA-1 and SHA-256 and added necessary
flags and other constants.
 9 years ago
Mats Klepsland f1b550d973 tls: add unit tests for tls_cert_issuer 9 years ago
Mats Klepsland 20b41567d9 tls: add unit tests for tls_cert_subject 9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject 
This keyword is a replacement for tls.subject.
 9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer 
This keyword is a replacement for tls.issuerdn.
 9 years ago
Victor Julien f1117ba2dc iponly: fix unittests 9 years ago
Victor Julien 215d0d54c7 detect: optimize rule address parsing 
Many rules have the same address vars, so instead of parsing them
each time use a hash to store the string and the parsed result.

Rules now reference the stored result in the hash table.
 9 years ago
Tom DeCanio 04faf1a93a util-decode-mime: remove quote from boundary= string. 
remove quote from the end of the boundary= string.  This was throwing off
the mime parser so that it wouldn't always catch mime boundaries causing
things like missed attachments.
 9 years ago
Eric Leblond f2d1e93e65 unix-socket: add auto mode 
When running in live mode, the new default 'auto' value of
unix-command.enabled causes unix-command to be activated. This
will allow users of live capture to benefit from the feature and
result in no side effect for user running in offline capture.
 9 years ago
Eric Leblond f6c3845397 util-time: new function to know if live or offline 9 years ago
Andreas Herz 7d54d8c590 rule-reload: remember pending USR2 signals 
We did ignore additional USR2 signals while a rule-reload was running.
This changes the counter to be incremented with every additional USR2
signal so we don't ignore them anymore but it's still limited to prevent
huge overload or even overflow.
 9 years ago
Jason Ish b454aa46c6 defrag: use frag_pkt_too_large instead of frag_too_large 
The rules were using the wrong decoder event type, which was
only set in the unlikely event of a complete overlap, which
really had nothing to do with being too large.

Remove FRAG_TOO_LARGE as its no longer being used, an overlap
event is already set in the case where this event would be set.
 9 years ago
Victor Julien 00313b2140 decoder-event: BUG_ON on table mismatches 
Abort when the event enum and the name<>event table are not matching.
 9 years ago
Jason Ish 108d37a52d logging: proper failure on memory allocation error 
unwinds all previous logger allocations
 9 years ago
Jason Ish a1de7e6dae flow-vars: remove flow locks 
Code is now entered under flow lock.
 9 years ago
Jason Ish 5a783e6854 flow-bits: remove flow locks 
Code is now entered under flow lock.
 9 years ago
Jason Ish 8865009fca lua: remove flow locking from the lua layer 9 years ago
Jason Ish 688e8dbe7f stream: remove lock from StreamTcpSegmentForEach 
This is only entered from logging functions which are already
called with a locked flow.
 9 years ago
Jason Ish 55f2704a25 logging: remove the packetqueue's from the logging path 
They are not referenced by any loggers, and they probably
shouldn't be either.
 9 years ago
Jason Ish 00b6e628d1 logging: hook into flow worker thread 9 years ago
Jason Ish f8c2c3653b output-streaming: free thread store on deinit 9 years ago
Jason Ish 1b4ba4496c logging: rename registration functions to not have tmm 
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.

Also, move the registration to the output.c which will handle
registration of the loggers.
 9 years ago
Jason Ish 04a44a077d logging: convert pcap log to non-thread module 9 years ago
Jason Ish a093580527 logging: just return if no tx loggers 9 years ago
Jason Ish fc35a78ba1 logging: use a single entry point for all loggers 
Introduces a new thread module, TMM_LOGGER, which is the
root most logger.

Only handles loggers in the packet path, stats and flow
logging are not included.

The loggers are made up of a hierarchy of loggers. At the top we
have the root logger which is the main entry point to
logging. Under the root there exists parent loggers that are the
entry point for specific types of loggers such as packet logger,
transaction loggers, etc. Each parent logger may have 0 or more
loggers that actual handle the job of producing output to something
like a file.
 9 years ago
Jason Ish d2c17ce9a0 logging: remove output priorities: not used 9 years ago
Jason Ish 9489d5b9e3 logging: remove dead code from output-json 
The "parent" json logger was setup like a real logger, but
some of that code was never being called.
 9 years ago
Jason Ish 8b38b9d728 output.[ch]: consistent style 
- Clean up function declaration.
- Consistenly use typedefs for function points.

No functional changes.
 9 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers 
The loggers moved away from a TMM required a new
profiling support.
 9 years ago
Jason Ish 42b8f30272 logging: convert lua output to non-thread module 9 years ago
Jason Ish 7a0737b9a9 logging: convert tls log to non-thread module 9 years ago
Jason Ish 7cb16bc90d logging: convert alert debug log to non-thread module 9 years ago
Jason Ish 7a8e8343e5 logging: convert tcp data logging to non-thread module 9 years ago
Jason Ish 4d8b8ca046 logging: convert tls store logging to non-thread module 9 years ago
Jason Ish 60b6ccc3c4 logging: convert file data logging to non-thread module 9 years ago
Jason Ish f9bb9029c5 logging: convert file logging to non-thread module 9 years ago
Jason Ish 669827ae16 logging: convert unified2 to non-thread module 9 years ago
Jason Ish b580016c80 logging: convert stats loggers to non-thread module 9 years ago
Jason Ish 9475c83713 logging: convert http log to non-thread module 9 years ago
Jason Ish e00dcd52a0 logging: convert alert syslog to non-thread module 9 years ago
Jason Ish 869d2eb701 logging: convert drop output to non-thread module 9 years ago
Jason Ish 5bbb4fd134 logging: convert json template output to non-thread module 9 years ago
Jason Ish b605984f34 tests: setup unit test framework earlier 
Allows tests to be registered early, in support of moving
outputs away from thread modules.
 9 years ago
Jason Ish bac65f09e8 logging: convert json drop output to non-thread module 9 years ago
Jason Ish 38354479b7 logging: convert json smtp output to non-thread module 9 years ago
Jason Ish 3fea12d7b3 logging: convert json ssh output to non-thread module 9 years ago
Jason Ish 01cc508257 logging: convert json netflow output to non-thread module 9 years ago
Jason Ish 983a619ff0 logging: convert json flow output to non-thread module 9 years ago
Jason Ish ad15ac8297 logging: convert json alert output to non-thread module 9 years ago
Jason Ish aaa65f3d16 logging: convert json tls output to non-thread module 9 years ago
Jason Ish 31663f1627 logging: convert prelude output to non-thread module 9 years ago
Jason Ish dedda33f01 logging: convert eve http to non-thread module 9 years ago
Jason Ish 687602c0ca logging: convert eve dns logging to non-thread module 9 years ago
Jason Ish b1200dba54 logging: convert fast log to a non-thread module 9 years ago
Jason Ish 637aa34610 logging: convert dns log to a non-thread module 9 years ago