Commit Graph

2422 Commits (79c329f81b1afc90a10ac10a5199a9954565925c)
 

Author SHA1 Message Date
Eric Leblond 7425bf5ca6 Rename some decode event structure and macro.
This patch renames DECODER_SET_EVENT, DECODER_ISSET_EVENT and some
other structures to ENGINE equivalent to take into account the fact
the event list is now related to all engines and not only to decoder.
14 years ago
Eric Leblond 5bbd8fe910 Add reference to events sig files in suricata YAML config. 14 years ago
Eric Leblond 552c6731b2 Add signature file for stream events.
This patch adds a rules/stream-events.rules file which contains
alert related to all stream events.
14 years ago
Eric Leblond de65b11c42 decode signature optimisation requires different treatment
Decode signature are using the fact that no proto is set on packet
to increase the matching speed. This is not the case of stream and
other engine events. Thus a difference needs to be made.
14 years ago
Eric Leblond 3f153fb0da Add 'stream-event' keyword.
This patch adds an alias to the 'engine-event' keyword. It is now
possible to access to the stream events via the 'stream-event'
keyword. A simple transformation is done:
    stream-event:reassembly_segment_before_base_seq
is a shortcut for:
    engine-event:stream.reassembly_segment_before_base_seq
14 years ago
Eric Leblond eb0d4e4d8b Add stream events support to 'engine-event' keyword
This patch adds the list of stream events (with associated
keywords) to the list of events that can be treated by 'engine-event'.
14 years ago
Eric Leblond e3a6d8955e Introduce engine-event keyword
This patch renames the 'decode-event' keyword to 'engine-event' and
keep it for backword compatibility of rulesets. All *DecodeEvent*
references in the code are replaced by EngineEvent version.
14 years ago
Eric Leblond 2ac8755382 Rename detect-decode-event to detect-engine-event
This patch does a simple renaming of detect-decode-event file to
the more global detect-engine-event name.
14 years ago
Victor Julien 21f387d2c7 profiling: fix stream ticks miscalculation on stream end pseudo packets. 14 years ago
Eric Leblond ff6365dd33 af-packet: switch to pcktacqloop API.
This patch gets rid of the old API and brings some optimisation
by reordering structure and optimisinf an error test.
14 years ago
Eric Leblond 834c91eece af-packet: add AFP to per packet performance system. 14 years ago
Eric Leblond fb4be6199f af-packet: change option name
This patch changes the option name. af-packet long option is
now used instead of -a to mimic pfring behaviour.

This patch improves the standard parsing of the command line.
Running
 suricata -c suricata.yaml --af-packet
will start a suricata running in AF_PACKET mode listening on all
interfaces defined in the suricata.yaml configuration file. The
traditionnal syntax:
 suricata -c suricata.yaml --af-packet=ppp0
will start a suricata listening on ppp0 only.
14 years ago
Eric Leblond e253da092c device: Add function to build interface list from config
This patch adds a new function which build the list of interfaces to
use by parsing the configuration file. This is using the new format
and thus only af-packet can benefit of this feature.
14 years ago
Eric Leblond df7dbe36b6 af-packet: Add option to disable promiscuous mode
This patch adds an option to suricata.yaml to be able to disable
the switch of the interface into promiscuous mode.
14 years ago
Eric Leblond fbca1a4e6b af-packet: multi interface support
This patch adds multi interface support to AF_PACKET. A structure
is used at thread creation to give all needed information to the
input module. Parsing of the options is done in runmode preparation
through a dedicated function which return the configuration in a
structure usable by thread creation.
14 years ago
Eric Leblond 18c6503a08 af-packet: change configuration format for multi interface 14 years ago
Eric Leblond dc667af1a1 conf: Introduce new function to input configuration.
The input modules are needing a per interface configuration. This
implies some new operations to be able to parse easily te configuration.

The syntax of the configuration file is for example:
af-packet:
  - interface: eth0
    threads: 2
  - interface: eth1
    threads: 3
We need a way to express get a configuration variable for interface[eth0].
This is by using ConfNodeLookupKeyValue() to get the matching node. And
after that value can be fetch by using ConfGetChildValue*() functions.
14 years ago
Eric Leblond e80b30c082 af-packet: finalize code
This patch handles the end of AF_PACKET socket support work. It
provides conditional compilation, autofp and single runmode.

It also adds a 'defrag' option which is used to activate defrag
support in kernel to avoid rx_hash computation in flow mode to fail
due to fragmentation.

This patch contains some fixes by Anoop Saldanha, and incorporate
change following review by Anoop Saldanha and Victor Julien.

AF_PACKET support is only build if the --enable-af-packet flag is
given to the configure command line. Detection of code availability
is also done: a check of the existence of AF_PACKET in standard
header is done. It seems this variable is Linux specific and it
should be enough to avoid compilation of AF_PACKET support on other
OSes.
Compilation does not depend on up-to-date headers on the system. If
none are present, wemake our own declaration of FANOUT variables. This
will permit compilation of the feature for system where only the kernel
has been updated to a version superior to 3.1.
14 years ago
Eric Leblond 871b21892a factorize pcap live device function
They are not specific to pcap and could thus be used in other module.
14 years ago
Eric Leblond c45d898572 af-packet: basic support for AF_PACKET socket
This patch provides basic support for AF_PACKET socket. It is
completed by a subsequent patches prodiding extended features
and bugfixes.
14 years ago
Victor Julien 5c8feb3851 profiling / qa: make plot-csv-large-all.sh much more flexible. 14 years ago
Victor Julien a7182353e7 Remove vim .swp file from repo. 14 years ago
Anoop Saldanha 58b595cc21 fastlog print updates for ipv6. combine the io write 14 years ago
Anoop Saldanha e8f9557664 fastlog print updates. combine the io write 14 years ago
Victor Julien d45e01e129 Add a few more example gnuplot scripts. 14 years ago
Victor Julien fca541f40e Add per app layer parser profiling
Per packet per app layer parser profiling. Example summary output:

Per App layer parser stats:

App Layer              IP ver   Proto   cnt        min      max          avg
--------------------   ------   -----   ------     ------   ----------   -------
ALPROTO_HTTP            IPv4       6    163394        126     38560320     42814
ALPROTO_FTP             IPv4       6       644        117        26100      2566
ALPROTO_TLS             IPv4       6       670        117         7137       799
ALPROTO_SMB             IPv4       6    114794        126       225270       957
ALPROTO_DCERPC          IPv4       6      5207        126        25596      1266

Also added to the csv out.

In the csv out there is a new column "stream (no app)" that removes the
app layer parsers from the stream tracking. So raw stream engine performance
becomes visible.
14 years ago
Victor Julien 0cc9f39200 Move TlsConfig structure out of app-layer-protos.h and rename it to SslConfig. 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Victor Julien 1bd1a62526 Rename profile macro's and variables to reflect that they are for rule profiling. 14 years ago
Eric Leblond 88559901d4 pcap-file: Allocated packet must be free if there's error 14 years ago
Eric Leblond f6628f140d detect: fix regular expression used for check. 14 years ago
Eric Leblond a354034cfc nfq: Fix deinit phase
If receive thread is failling, we need to restart it but the code was
not restarting the queue (this was done in verdict thread).
14 years ago
Eric Leblond eddcedba0a nfq: make thread abort if NFQ verdict fail 14 years ago
Eric Leblond 2ffcef0a8e nfq: Add iterator on nfq_set_verdict
This patch adds retry to nfq_set_verdict in case of error.
14 years ago
Eric Leblond a8b21066df tm-thread: fix documentation string 14 years ago
Eric Leblond a8ae1c42c3 Fix macro about default packet size
Being pessimistic about packet default size has side effect in
some module. Falling back to the sane correct value.
14 years ago
Eric Leblond 685e0e1a63 Rename rule_type_t to ThresholdRuleType. 14 years ago
Eric Leblond 8787e6f6d0 suppress: use DetectAddress instead of DetectAddressHead 14 years ago
Eric Leblond 8ff8ec4f82 Export some DetectAddress related function. 14 years ago
Eric Leblond 7938344e1b threshold: refactoring of parsing code
This patch factorize the regular expression to be ease the parsing
process. It also adds a missing free and factorize exit code.
14 years ago
Eric Leblond 03c185a3ad threshold: add suppress keyword
This patch adds the suppress keyword to the threshold.config file.
The alerts are suppressed but the other elements like flowbits are
maintained.
14 years ago
Eric Leblond 85e8d8e200 Add sanity check to DetectAdressParse.
The function is only used at parsing time, this is not costly to add
a simple sanity check.
14 years ago
Eric Leblond 7168e0aafc threshold: fix trivial typo in parsing. 14 years ago
Eric Leblond a56f8dd6b2 doc: introduce doxygen group "threshold"
This patch introduces a doxygen group to put together the documentation
relative to threshold. Group appear in a separate page and they can have
their own documentation. This is useful when a feature is splitted into
different files.
14 years ago
Victor Julien dc218388e5 Fix flowint keyword pcre_get_substring issue. 14 years ago
Victor Julien 1740c3a7c7 Fix urilen keyword pcre_get_substring issue. 14 years ago
Victor Julien f52b54f63e Fix ssl keyword pcre_get_substring issue. 14 years ago
Eric Leblond 6b9d1012ff Transform inet_ntop call into PrintInet one. 14 years ago
Eric Leblond 2fa07780c2 Introduce PrintInet function
This function has the same signature than inet_ntop() and it
will be used as substitution in the code. For IPv4 this is a simple
wrapper. For IPv6, it display addresses with fixed length.
14 years ago
Victor Julien 7e1d911215 Small optimizations to pkt acq loop code. 14 years ago