Commit Graph

114 Commits (c4b34e6ef73f9915b4da1d52673279ea7537f6f0)

Author SHA1 Message Date
Eric Leblond 279b8b4027 config: Add explanation for checksum-checks options 13 years ago
Eric Leblond 67f791e891 af-packet: add variable to disable offloading detection
This flag adds variable to disable offloading detection. The effect
of the flag is to avoid to transmit auxiliary data at each packet.
This could result in a potential performance gain.
13 years ago
Mike Pomraning b5c11bd9fa Document new "filetype" argument for 'fast', 'http-log', etc. 13 years ago
Victor Julien a397599fbb file extraction: add waldo option to file log module. This will store the last used file_id so extracted files won't get overwritten is Suricata is restarted. 13 years ago
Victor Julien 801f92f705 Disable the drop.log in the default config. 13 years ago
Anoop Saldanha 7c9d1b80fd Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls 13 years ago
Anoop Saldanha 52b37fef3e Update yaml size params to use kb, mb, gb to indicate size, in place of raw bytes 13 years ago
Eileen Donlon dbdf2d888f Enable/disable core dump in config (feature 319) 13 years ago
Victor Julien a556338936 Add magic-file example to suricta.yaml. 13 years ago
Victor Julien 53df3982a1 Update suricata.yaml for file extraction. 13 years ago
Victor Julien b402d97179 File carving -- enable reponse file extraction
- Enable response body tracking
- Enable file extraction for responses
- File store meta file includes magic, close reason.
- Option to force magic lookup for all stored files.
- Fix libmagic calls thead safety.
13 years ago
Victor Julien 64aee5e70c Add file log to default suricata.yaml. 13 years ago
Victor Julien 1eef36b011 Initial checkin of a log-file module, that can write files extracted from flows to disk. 13 years ago
deltay 211193b0af Get pidfile from config file if not available in command options 13 years ago
deltay d5e254d504 Add pfring bpf filter, require pfring >= 5.1 13 years ago
Eric Leblond a5b1de4f0d http log: Add extended option
This patch adds a extended option to log extended HTTP information
when activated.
13 years ago
Eric Leblond 391d813c82 Remove unified1 output module. 13 years ago
Victor Julien 498d1d9287 Update default suricata.yaml to use more sane settings for EXTERNAL_NET and AIM_SERVERS. 13 years ago
Eric Leblond 27f1d88374 Add pcap-info alert format.
This patch adds a new alert format called pcap-info. It aims at
providing an easy to parse one-line per-alert format containing
the packet id in the parsed pcap for each alert. This permit to
add information inside the pcap parser.

This format is made to be used with suriwire which is a plugin for
wireshark. Its target is to enable the display of suricata results
inside wireshark.

This format doesn't use append mode per default because a clean file
is needed to operate with wireshark.

The format is a list of values separated by ':':
  Packet number:GID of matching signature:SID of signature:REV of signature:Flow:To Server:To Client:0:0:Message of signature
The two zero are not yet used values. Candidate for usage is the
part of the packet that matched the signature.
13 years ago
Eric Leblond c3c20dd041 pcap: add new config style 13 years ago
Eric Leblond 78467f1846 Update configuration file to new pfring format. 13 years ago
Anoop Saldanha de635b5866 remove trailing whitespace from conf file 13 years ago
William Metcalf 3b3f5816bf You spin me right round baby, right round like a rotating packet capture right round. Oh, also log file size counters are now uint64_t 13 years ago
Eric Leblond 5bbd8fe910 Add reference to events sig files in suricata YAML config. 13 years ago
Eric Leblond df7dbe36b6 af-packet: Add option to disable promiscuous mode
This patch adds an option to suricata.yaml to be able to disable
the switch of the interface into promiscuous mode.
13 years ago
Eric Leblond 18c6503a08 af-packet: change configuration format for multi interface 13 years ago
Eric Leblond e80b30c082 af-packet: finalize code
This patch handles the end of AF_PACKET socket support work. It
provides conditional compilation, autofp and single runmode.

It also adds a 'defrag' option which is used to activate defrag
support in kernel to avoid rx_hash computation in flow mode to fail
due to fragmentation.

This patch contains some fixes by Anoop Saldanha, and incorporate
change following review by Anoop Saldanha and Victor Julien.

AF_PACKET support is only build if the --enable-af-packet flag is
given to the configure command line. Detection of code availability
is also done: a check of the existence of AF_PACKET in standard
header is done. It seems this variable is Linux specific and it
should be enough to avoid compilation of AF_PACKET support on other
OSes.
Compilation does not depend on up-to-date headers on the system. If
none are present, wemake our own declaration of FANOUT variables. This
will permit compilation of the feature for system where only the kernel
has been updated to a version superior to 3.1.
13 years ago
Eric Leblond c45d898572 af-packet: basic support for AF_PACKET socket
This patch provides basic support for AF_PACKET socket. It is
completed by a subsequent patches prodiding extended features
and bugfixes.
13 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
13 years ago
William 6730c3ace1 Actually limit recursion and backtracking and stack usage by PCRE. Logic was broken, no example was provided in suricata.yaml even though it could be set from there. 13 years ago
Eric Leblond fb6494abc1 config file: add missing variable example
The variable threads was not shown in the default configuration file
and this is really needed to optimize the system easily.
14 years ago
William 85643fe780 Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them. 14 years ago
Anoop Saldanha e4d890e186 modify runmode api to accept conf runmode paramter as a char string, instead of an interger id 14 years ago
Anoop Saldanha 229f7281ea list runmodes. Allow specification of runmode id from cof file. Also allow for command line override 14 years ago
Victor Julien a96eae01fa Disable alert-debuglog that was accidently enabled in a previous commit. 14 years ago
Victor Julien 4f5aad1476 Enforce configurable minimum chunk size in raw stream reassembly. Minor stream cleanups, unittest updates. 14 years ago
Victor Julien bc7e21aee6 Add special sguil mode to log-pcap to support logging into date based directory structure and rotate when the day passes. Also do not log packets beyond stream reassembly depth and encrypted traffic. 14 years ago
Victor Julien f1e3d636d9 Add limit option to pcap-log logging config. 14 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 14 years ago
Victor Julien da423a59d5 Allow users of the alert-syslog to set the identity. 14 years ago
Victor Julien 91f28afef4 Add option to PF_RING to have multiple reader threads. Improve general performance of the PF_RING module. 14 years ago
Victor Julien a3303fcf9d Rename request-body-limit to request_body_limit to remain consistant with other options. Keep old notation around for compatibility. 14 years ago
Gurvinder Singh 7d0781b349 added support to log dropped packet as netfilter logs while in inline mode 14 years ago
Victor Julien 1c7b7a01a6 Add option to set the syslog level for the alerts. Minor cleanups. 14 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 14 years ago
Eric Leblond 0b5e5b8772 affinity: change config format and misc fixes
This patch fixes some problem with affinity work and modify the
configuration file format.

For example, the detect cup set can be formatted as follow:
     - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        prio:
          low: [ 0 ] # threads on CPU 0 have low prio
          medium: [ "1-2" ] # threads on CPU 1 and 2 have medium prio
          high: [ 3 ] # threads on CPU 3 have high prio
          default: "medium" #default priority is "medium"
14 years ago
Eric Leblond c74116949c source-nfq: improve nfq option system
This patch modifies the NFQ option system to avoid implicit
choice. 'nfq.mode' is now a string which can take a value
in the 'accept', 'repeat' and 'route' set.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 1e600c1054 source-nfq: add simulated non-terminal NFQUEUE verdict
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
    iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.

To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.

NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.

Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
   nfq:
     repeat_mode: (false|true)
     mark: $MARK
     mask: $MASK
Default is false to preserve backward compatibility.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 789d46cc3c Add per-cpu prio handling
This patch updates affinity setting to add a support for per cpu
priority setting. In exclusive mode a thread is dedicated to a CPU.
This patch adds the ability to set the thread prio for all threads
of a family running on a given CPU.

With this patch we can write
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        medium_prio: [ "1-2" ]
        high_prio: [ 3 ]
With this configuration, detect threads assigned to cpu 0 will
have a low priority. Detect threads on cpus 1 and 2 will have
prio medium...

The previous configuration is equivalent to:
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        high_prio: [ 3 ]
        prio: "medium"
because the prio value is used a default.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 559b5db7df yaml: add config for cpu_affinity
This patch introduces the configuration part of cpu_affinity
evolution. The idea is to have thread families (receive, stream, ...)
and to be able to specify for each of them:
 - The cpus to use
 - The mode of balancing between the cpus
 - The thread priority to use (value of nice)

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago