aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,250 Commits
7 Branches
155 Tags
195 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '58b595cc21'
${ noResults }
Commit Graph

86 Commits (58b595cc21ccdcad845c86e03ddeb38fb6147a76)

Author SHA1 Message Date
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
William 6730c3ace1 Actually limit recursion and backtracking and stack usage by PCRE. Logic was broken, no example was provided in suricata.yaml even though it could be set from there. 14 years ago
Eric Leblond fb6494abc1 config file: add missing variable example 
The variable threads was not shown in the default configuration file
and this is really needed to optimize the system easily.
 14 years ago
William 85643fe780 Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them. 14 years ago
Anoop Saldanha e4d890e186 modify runmode api to accept conf runmode paramter as a char string, instead of an interger id 14 years ago
Anoop Saldanha 229f7281ea list runmodes. Allow specification of runmode id from cof file. Also allow for command line override 14 years ago
Victor Julien a96eae01fa Disable alert-debuglog that was accidently enabled in a previous commit. 14 years ago
Victor Julien 4f5aad1476 Enforce configurable minimum chunk size in raw stream reassembly. Minor stream cleanups, unittest updates. 14 years ago
Victor Julien bc7e21aee6 Add special sguil mode to log-pcap to support logging into date based directory structure and rotate when the day passes. Also do not log packets beyond stream reassembly depth and encrypted traffic. 14 years ago
Victor Julien f1e3d636d9 Add limit option to pcap-log logging config. 14 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 14 years ago
Victor Julien da423a59d5 Allow users of the alert-syslog to set the identity. 14 years ago
Victor Julien 91f28afef4 Add option to PF_RING to have multiple reader threads. Improve general performance of the PF_RING module. 14 years ago
Victor Julien a3303fcf9d Rename request-body-limit to request_body_limit to remain consistant with other options. Keep old notation around for compatibility. 14 years ago
Gurvinder Singh 7d0781b349 added support to log dropped packet as netfilter logs while in inline mode 14 years ago
Victor Julien 1c7b7a01a6 Add option to set the syslog level for the alerts. Minor cleanups. 14 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 14 years ago
Eric Leblond 0b5e5b8772 affinity: change config format and misc fixes 
This patch fixes some problem with affinity work and modify the
configuration file format.

For example, the detect cup set can be formatted as follow:
     - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        prio:
          low: [ 0 ] # threads on CPU 0 have low prio
          medium: [ "1-2" ] # threads on CPU 1 and 2 have medium prio
          high: [ 3 ] # threads on CPU 3 have high prio
          default: "medium" #default priority is "medium"
 14 years ago
Eric Leblond c74116949c source-nfq: improve nfq option system 
This patch modifies the NFQ option system to avoid implicit
choice. 'nfq.mode' is now a string which can take a value
in the 'accept', 'repeat' and 'route' set.

Signed-off-by: Eric Leblond <eric@regit.org>
 14 years ago
Eric Leblond 1e600c1054 source-nfq: add simulated non-terminal NFQUEUE verdict 
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
    iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.

To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.

NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.

Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
   nfq:
     repeat_mode: (false|true)
     mark: $MARK
     mask: $MASK
Default is false to preserve backward compatibility.

Signed-off-by: Eric Leblond <eric@regit.org>
 14 years ago
Eric Leblond 789d46cc3c Add per-cpu prio handling 
This patch updates affinity setting to add a support for per cpu
priority setting. In exclusive mode a thread is dedicated to a CPU.
This patch adds the ability to set the thread prio for all threads
of a family running on a given CPU.

With this patch we can write
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        medium_prio: [ "1-2" ]
        high_prio: [ 3 ]
With this configuration, detect threads assigned to cpu 0 will
have a low priority. Detect threads on cpus 1 and 2 will have
prio medium...

The previous configuration is equivalent to:
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        high_prio: [ 3 ]
        prio: "medium"
because the prio value is used a default.

Signed-off-by: Eric Leblond <eric@regit.org>
 14 years ago
Eric Leblond 559b5db7df yaml: add config for cpu_affinity 
This patch introduces the configuration part of cpu_affinity
evolution. The idea is to have thread families (receive, stream, ...)
and to be able to specify for each of them:
 - The cpus to use
 - The mode of balancing between the cpus
 - The thread priority to use (value of nice)

Signed-off-by: Eric Leblond <eric@regit.org>
 14 years ago
Victor Julien d00c6172c9 Update stream section of example configuration. 14 years ago
Eric Leblond e802e1ed16 Modify Packet structure and prepare accessor. 
This patch modifies decode.c and decode.h to avoid the usage
by default of a bigger than 65535 bytes array in Packet structure.
The idea is that the packet are mainly under 1514 bytes size and
a bigger size must be supported but should not be the default.

If the packet length is bigger than DFLT_PACKET_SIZE then the
data are stored in a dynamically allocated part of the memory.

To ease the modification of the rest of the code, functions to
access and set the payload/length in a Packet have been introduced.

The default packet size can be set at runtime via the default-packet-size
configuration variable.
 14 years ago
Eric Leblond 8471626916 Fix error message and adds information to config 
This patch fixes a typo in an error message and add some
information to the checksum verification option.
 14 years ago
Gurvinder Singh f4392e1dcc added support for appending the log files 14 years ago
Pierre Chifflier de41612ea1 Add options to choose if we log header and content in Prelude alert module. 
Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
 14 years ago
Victor Julien e66b3e3ee7 Switch mpm-algo in example suricata.yaml to ac 14 years ago
Victor Julien 355f237bfd Fix compiler warnings, cleanup counters config code. 14 years ago
Gurvinder Singh ba18110abd support for stats.log configurable and fixed timezone issue in faslog and debuglog 14 years ago
Martin Beyer b1c577f829 cuda streams support in b2g-cuda MPM 14 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 14 years ago
Victor Julien 0dd07df60d Add reference to suricata.yaml documentation in our redmine wiki. 14 years ago
Victor Julien 6ebe7b7cd3 Change the way the request body limit is enforced. 14 years ago
Anoop Saldanha 778ec0939c make client body buffer limit configurable. Also some minor changes 14 years ago
Jason Ish a4d19e4130 Add new profiling sort option, maxticks. 15 years ago
Anoop Saldanha 5d9a453e0d find an optimal value for detect-engine:inspection-recursion_limit + unittest 15 years ago
Anoop Saldanha 88d94b136d Support for reference.config file 15 years ago
Victor Julien 344ea14695 Change mpm hash_size config setting highest to higher as highest wasn't the... highest. Max was higher. Leaving highest as an alias to higher for backwards compatibility. 15 years ago
Victor Julien 3bd7441ea5 Default to 'single' ctx for ac-gfbs as well. 15 years ago
Anoop Saldanha 62f814a4c3 change default value for detect-engine.sgh_mpm_context to auto 15 years ago
Anoop Saldanha b367c37ae6 suricata.yaml conf update to support single mpm context distribution over multiple sghs + code to parse this conf 15 years ago
Anoop Saldanha a85fa6b792 support for fast_pattern only and fast_pattern:offset,length. Also support the new option for engine-analysis 15 years ago
Pablo Rincon b3a8f0a90f Fix asn1 decoder frame oob mem. Adding max stack frames to suricata.yaml 15 years ago
Victor Julien b4454b6846 Switch to b2gc as default pattern matcher as it uses less memory and is a little faster. 15 years ago
Victor Julien 87f88867f4 Further improve B2gc. Add B2gm. Improve memory layout. 15 years ago
Victor Julien 9dfbab42f8 WIP B2gc 15 years ago
Gurvinder Singh 6a5bc52461 support for several tcp evasion attacks. Thanks to Judy Novak and G2 Inc for reporting them 15 years ago
Victor Julien 875184a4ba Cleanup suricata.yaml. 15 years ago
Gurvinder Singh f0928a4555 support for enforcing the depth until when the reassembly will be performed 15 years ago