aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,268 Commits
7 Branches
159 Tags
196 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'f4ce9011d2'
${ noResults }
Commit Graph

484 Commits (f4ce9011d22d2323b1029ca907f4578fe4b99be3)

Author SHA1 Message Date
Victor Julien 18d79c4215 file store: respect flowbits and other keywords 
The filestore keyword until now flagged a file, tx or ssn for storage as soon
as the keyword was inspected. This happens before flowbits and some other
keywords, so files were stored that weren't supposed to.

This patch makes the filestore keyword fill an array in the detect engine
thread ctx. Then if the full signature matches, a post-match filestore
function makes the store final.
 14 years ago
Victor Julien 7adac3048d file-data: create initial keyword registration. 14 years ago
Victor Julien dd9da1a56f Merge all http mpm related signature flags into a single set: SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_HTTP_NEG. 14 years ago
Victor Julien 2650551192 Rename signature init flags to indicate they are init flags. 14 years ago
Victor Julien 291ddd95f2 Detection engine -- mpm 
Each signature is in one mpm ctx at max, but there were 3 separate
id's in use: packet, stream, http. Merged them all into one.

Could shrink the SignatureHeader structure with 8 bytes because of this,
should lead to better caching performance.
 14 years ago
Victor Julien 7db72bce75 Optimize detection engine prefiltering logic. 14 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword. 
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
 14 years ago
Anoop Saldanha b970273163 fix broken unittest 14 years ago
Anoop Saldanha d40fb5b933 Remove unnecessary flow NULL check 14 years ago
Anoop Saldanha cc7db6315c Move setting packet iponly flags from decode section to stream section 14 years ago
Victor Julien 117d51c965 Fix a compile warning when debug is enabled. 14 years ago
Victor Julien 042fd850fc Make sure we check the sgh for no magic and no store once per flow direction. 14 years ago
Victor Julien d59ca75e46 file extract: split toserver and toclient tracking 
Split toserver and toclient file tracking for the http state.
 14 years ago
Victor Julien 04ea70ccf7 file extract: pruning 
Add pruning of files in memory so we keep only memory what we really need.
Fix magic logic.
Reset file part of the de_state on receiving another file in the same tx.
 14 years ago
Victor Julien e1022ee5ae file-extraction: Disconnect file handling from flow and move into the app layer state. 14 years ago
Victor Julien 9b62ec65ab Make sure filemagic works properly regardless of filestore being in use for a flow. 14 years ago
Victor Julien 5945e652d6 Initial implementation of filemagic keyword. 14 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 14 years ago
Pablo Rincon 6d60b3a747 filename and fileext keywords 14 years ago
Anoop Saldanha 9a6aef459e modify all relevant app layer API calls to accomodate passing parser local storage argument 14 years ago
Anoop Saldanha 2b356dadff Support for tos keyword added 14 years ago
Victor Julien 262a7300d7 flow: shrink Flow datatype 
Introduce a separate FlowAddress structure for holding the ipv4 or ipv6 address
that doesn't have the family in it like the Address structure. Instead, the
family is stored in the flow as a flag: FLOW_IPV4 and FLOW_IPV6.

Add macro's to check the family, copy the address, etc.

Update many unittests to reflect these changes. Introduce unittest helper
functions for creating and initializing a flow and freeing it again.

On 64 bit this shrinks the flow with 8 bytes.
 14 years ago
Victor Julien 06904c9024 App Layer cleanup 
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
 14 years ago
Victor Julien 3d845b6c77 Consider Windows new line chars as well when parsing rule files. Bug #374. 14 years ago
Victor Julien 8cc82c7241 Add -S commandline option that loads a rule file exclusively. Issue #338. 14 years ago
Victor Julien 2d16abcf8b Minor code cleanups fixing all GCC 4.6 compiler warnings for default, debug and unittests mode. 14 years ago
Anoop Saldanha 0acfcc206c fix unittests. fix replace unittests that allow alproto keywords with replace 14 years ago
Anoop Saldanha a0eec3d846 fix detection code that handles cases when we use recursion(from recursive keyword) 14 years ago
Anoop Saldanha 7433d92dd2 undo this commit - 
commit eff08f93d8
Author: Anoop Saldanha <poonaatsoc@gmail.com>
Date:   Thu Nov 3 14:31:24 2011 +0530

    update failing unittest to reflect the mpm design update

Fixed a bug in the mpm code that would make all the changes in the commit just undone wrong.
 14 years ago
Anoop Saldanha 1b1332fff0 fix mpm bug on running stream mpm for packets not added to stream mpm 14 years ago
Anoop Saldanha eff08f93d8 update failing unittest to reflect the mpm design update 14 years ago
Victor Julien af51493da2 Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm. 14 years ago
Anoop Saldanha 17f3f36d38 packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same 14 years ago
Anoop Saldanha c7b9d3fecb Remove broken dsize_sm in SigMatch used by dsize in detection engine 14 years ago
Eric Leblond 60a99915c1 doc: create http support group 
This patch create an httplayer group and adds related files to
it. It also fixes some typo in documentation string and format.
 14 years ago
Eric Leblond 6bf15bac31 Fix various packet access. 
The coccinelle based tests have detected invalid uses of access to
Packet data. This patch fixes the detected problems.
 14 years ago
Victor Julien 7ef34b7bcc Exlcude DSIZE LT case from setting the 'need payload' mask bit as it can include 0, which means no payload. 14 years ago
Victor Julien 2e2e80b812 Add packet alert flag to indicate a match happened (partly) in the app layer state. Make unified2 use this flag. 14 years ago
Victor Julien a1f68bf411 Fix detection engine informational message misrepresenting decoder only signatures. 14 years ago
Eric Leblond 3f75b10f79 Suppress useless code. 
This code was making a warning for some time now. This patch kills
it.
 14 years ago
Eric Leblond a85dc9b0e2 Add support for replace keyword. 
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.

One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.

Replace is only activated in IPS mode and apply only to raw match.
 14 years ago
Anoop Saldanha 8028392e9a fix mpm segv. Use sgh flags to check if the sgh has packet or stream mpm set or not 14 years ago
Victor Julien e13181496c ip-only: added support for matching on ports. 14 years ago
Victor Julien 751a77a9be Make sure stream/engine-event signatures are recognized as such. 14 years ago
Victor Julien 63f834d9a7 Add profiling to various HTTP buffer MPM calls. 14 years ago
Victor Julien 3693a7a9ee Profiling: add accounting for several detection phases. 14 years ago
Eric Leblond e3a6d8955e Introduce engine-event keyword 
This patch renames the 'decode-event' keyword to 'engine-event' and
keep it for backword compatibility of rulesets. All *DecodeEvent*
references in the code are replaced by EngineEvent version.
 14 years ago
Eric Leblond 2ac8755382 Rename detect-decode-event to detect-engine-event 
This patch does a simple renaming of detect-decode-event file to
the more global detect-engine-event name.
 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Victor Julien 1bd1a62526 Rename profile macro's and variables to reflect that they are for rule profiling. 14 years ago
Anoop Saldanha 7e5c52c80b add flowbits:set; only sigs to be treated as ip only 14 years ago
Victor Julien 16b41a5eff Use p->proto in detect to determine TCP/UDP/SCTP. 14 years ago
Victor Julien f7f037c1d1 Make sure stateful detection engine inspecting HTTP streams works well for to_client rules as well. 14 years ago
Anoop Saldanha 35f3eafa5e byte extract added to the engine. Detection support added for packet payload, uri and dce detection engines 14 years ago
Victor Julien 73efb4c70f Add a app layer state and stateful detection engine counter that makes sure the stateful inspection is only done when the state changes. 14 years ago
Victor Julien 0768ca9806 Fix SIMD mask checking on 64 bit systems. 14 years ago
Victor Julien 350215966b Fix signature mask bitorder. 14 years ago
Victor Julien aa822c0ac1 Always reset alert cnt and always increment det_ctx->pkts. 14 years ago
Victor Julien 1e0b050a54 Add more mask flags. 14 years ago
Victor Julien 4b52823ab6 Use 64 bit mask on 64-bit systems. 14 years ago
Victor Julien e5b6c0f518 Check 32 masks per run instead of 16 in the SIMD code. 14 years ago
Victor Julien 2dbfdd40af Clean up new SIMD mask checking code, improve non-SIMD checks. 14 years ago
Victor Julien b421019cef Match packet mask against 16 signature masks at once using SIMD instructions for SSE3 and up. 14 years ago
Victor Julien e1d4e16645 Simplify packet decoding macro's. 14 years ago
Victor Julien 75439863ed Shrink PacketAlerts structure so that Packet structure is a lot smaller. Reduce max events per packet from 256 to 15. 14 years ago
Victor Julien 6aa551c558 Small optimizations to IPV4 and TCP header parsing. 14 years ago
Anoop Saldanha 966119b6aa support for http_raw_uri keyword + mpm engine 14 years ago
Victor Julien 864c8718e1 Store matching stream msg (ptr) in packets alert structure so it's available to the output plugins. 14 years ago
Victor Julien 140eb4fde8 Fix decode-event keyword parsing. Fix code that indicates a signature is decode-event only. Add 'pkthdr' protocol as an alias for any/ip to be used by decode-event signatures. 14 years ago
Victor Julien e197f50727 Fix IP-Only unittests failing on Big Endian. 14 years ago
Eric Leblond 9beebf621a Add support for 'nfq_set_mark' keyword 
This patch introduces 'nfq_set_mark' which is new rules option. If a packet
matches a rule using nfq_set_mark in NFQ mode, it is marked with the mark/mask
specified in the option during the verdict.
It is thus possible to trigger different behaviour on the packet inside
Linux/Netfilter.
 14 years ago
Anoop Saldanha c105a739e9 support for ssl_state keyword added 14 years ago
Victor Julien b9fd978253 Fix compiler warnings in two unittests. 14 years ago
Eric Leblond 005dc599a6 detect.c: Fix usage of integer standing for protocol 
This patch fixes direct usage of integer to code protocol value.
 15 years ago
Eric Leblond 2c80f18dc9 detect: Add sctp detection and parsing. 
This patch adds the support of SCTP in signature subsystem.
 15 years ago
Gurvinder Singh 8f8b1212af support for ssl_version keyword 15 years ago
Victor Julien ec5b622553 Inspect all stream msgs at any time when running in stream-inline mode. Skip detection for packets flagged for dropping before detect. 15 years ago
Victor Julien 1d971b53a6 Update all unittests 15 years ago
Victor Julien f606621e8c Fix the pseudo packet having the wrong proto set, causing massive fp's. Flag packets to be part of the established phase of a tcp session, so we won't prematurely inspect the app layer state. 15 years ago
Victor Julien 24f071cabb Make sure http_cookie inspects all HTTP transactions. Clean up error messages. Get rid of unused code and dead comments. 15 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 15 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 15 years ago
Eric Leblond 49adc264bc Don't print message after SCMalloc failure. 
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
 15 years ago
Eric Leblond 1db4aadd16 Supress usage of Packet declaration in tests. 
For convenience, a massive usage of 'Packet p;' declaration has
been done in the tests function. Although this was completely
legal, this is not possible anymore because of the new Packet
allocation structure. This massive patch modifies all suricata
files to use a SCMalloc allocated pointer to Packet instead.

This patch has been done using coccinelle (http://coccinelle.lip6.fr)
which is a semantic patching tool. This ensures that things like call
to SCFree() should have not been forget because the semantic patch
explicitly forces the call to SCFree(p) before each return. With this
patch all unittests are running fine with a small and a big default
packet size.
 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access 
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
 15 years ago
Victor Julien 18b4e3380f Make mpm-algo use the mpm_table that has the actual mpm's registered. Clean up dead code. 15 years ago
Victor Julien 3f47eade6b Fix couple of cases where incorrect handling of keyword parsing errors would lead to access of uninitialized memory. Found by clang. 15 years ago
Victor Julien 3d60e9bfeb Clean up output. 15 years ago
Anoop Saldanha d241e51b32 adapt fast pattern engine analysis to reflect the new changes made to your mpm design 15 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 15 years ago
Victor Julien 0e8e8e3728 Don't stop stream reassembly if protocol detection failed, only stop/prevent app layer parsing. 15 years ago
Victor Julien 39dea56a84 Remove flowbits as a mask prefilter as they are dynamic. Add a dynamic check. 15 years ago
Anoop Saldanha 8c21511c99 don't buffer raw headers. Retrieve them individually from htp_state during mpm stage and content valiadation stage 15 years ago
Victor Julien 55ca988222 Change locking of http_header, http_raw_header and http_client_body so that flow isn't accessed without lock anywhere. 15 years ago
Victor Julien 435d0fb327 Clean up signature flags creating room for merging flags and mpm_flags. Merge flags and mpm_flags. Move new mpm id's into signature header. Get rid of full signature access in signature prefiltering. 15 years ago
Victor Julien 169aa5581a Rename SIG_FLAG_AMATCH flag to SIG_FLAG_STATE_MATCH to better reflects its purpose. 15 years ago
Victor Julien d7b92d9bfe Consolidate several signature flags into one. 15 years ago
Victor Julien 2102a54c26 Cleanup and rearrange detection code slightly. 15 years ago
Anoop Saldanha 25588b6910 comment out hrhd flags that we were using previously. Also remove the de_mpm_ based flags inside detect.h used by uri|hcbd|hhd|hrhd mpms. indentation fix as well 15 years ago
Anoop Saldanha e50503e820 cleanup/remove dead code 15 years ago