The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".
To facilitate this change the signature flags field was changed to be 64 bit.
This patch modifies LiveBuildQueueList name to LiveBuildDeviceList
to have a consistent naming accross function. It also adds a
doxygen comment to add author and description of util-device.c
file.
This patch adds a new alert format called pcap-info. It aims at
providing an easy to parse one-line per-alert format containing
the packet id in the parsed pcap for each alert. This permit to
add information inside the pcap parser.
This format is made to be used with suriwire which is a plugin for
wireshark. Its target is to enable the display of suricata results
inside wireshark.
This format doesn't use append mode per default because a clean file
is needed to operate with wireshark.
The format is a list of values separated by ':':
Packet number:GID of matching signature:SID of signature:REV of signature:Flow:To Server:To Client:0:0:Message of signature
The two zero are not yet used values. Candidate for usage is the
part of the packet that matched the signature.
This patch adds a --pcap option which can be used to select or
an interface if an argument is provided or the interfaces defined
in the configuration file.
This patch convert pfring to pktacqloop and use the new factorisation
function. This also fixes commmand line parsing of pfring which is now
able to work like af-packet:
- 'suricata -c s.yaml --pfring' start suricata with all interfaces in
conf
- 'suricata -c s.yaml --pfring=eth2' start suricata on eth2
This patch changes the option name. af-packet long option is
now used instead of -a to mimic pfring behaviour.
This patch improves the standard parsing of the command line.
Running
suricata -c suricata.yaml --af-packet
will start a suricata running in AF_PACKET mode listening on all
interfaces defined in the suricata.yaml configuration file. The
traditionnal syntax:
suricata -c suricata.yaml --af-packet=ppp0
will start a suricata listening on ppp0 only.
This patch adds multi interface support to AF_PACKET. A structure
is used at thread creation to give all needed information to the
input module. Parsing of the options is done in runmode preparation
through a dedicated function which return the configuration in a
structure usable by thread creation.
This patch handles the end of AF_PACKET socket support work. It
provides conditional compilation, autofp and single runmode.
It also adds a 'defrag' option which is used to activate defrag
support in kernel to avoid rx_hash computation in flow mode to fail
due to fragmentation.
This patch contains some fixes by Anoop Saldanha, and incorporate
change following review by Anoop Saldanha and Victor Julien.
AF_PACKET support is only build if the --enable-af-packet flag is
given to the configure command line. Detection of code availability
is also done: a check of the existence of AF_PACKET in standard
header is done. It seems this variable is Linux specific and it
should be enough to avoid compilation of AF_PACKET support on other
OSes.
Compilation does not depend on up-to-date headers on the system. If
none are present, wemake our own declaration of FANOUT variables. This
will permit compilation of the feature for system where only the kernel
has been updated to a version superior to 3.1.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.
Stats per packet include:
1) total ticks spent
2) ticks spent per individual thread module
3) "threading overhead" which is simply calculated by subtracting (2) of (1).
A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.
Code depends on --enable-profiling just like the rule profiling code.
New yaml parameters:
profiling:
# packet profiling
packets:
# Profiling can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: yes
filename: packet_stats.log
append: yes
# per packet csv output
csv:
# Output can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: no
filename: packet_stats.csv
Example output of summary stats:
IP ver Proto cnt min max avg
------ ----- ------ ------ ---------- -------
IPv4 6 19436 11448 5404365 32993
IPv4 256 4 11511 49968 30575
Per Thread module stats:
Thread Module IP ver Proto cnt min max avg
------------------------ ------ ----- ------ ------ ---------- -------
TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770
TMM_DETECT IPv4 6 19436 1107 137241 1504
TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155
TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138
TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154
TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944
TMM_STREAMTCP IPv4 6 19434 828 1299159 19438
The proto 256 is a counter for handling of pseudo/tunnel packets.
Example output of csv:
pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337
First line of the file contains labels.
2 example gnuplot scripts added to plot the data.
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.
To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.
NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.
Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
nfq:
repeat_mode: (false|true)
mark: $MARK
mask: $MASK
Default is false to preserve backward compatibility.
Signed-off-by: Eric Leblond <eric@regit.org>
This patch adds support for multiple Netfilter queue
in the NFQ run mode. Suricata can now be started on
multiple queue by using a comma separated list of
queue identifier on the command line. The following syntax:
suricata -q 0 -q 1 -c /opt/suricata/etc/suricata.yaml
will start a suricata listening to Netfilter queue 0 and 1.
Signed-off-by: Eric Leblond <eric@regit.org>
Timer in the main loop was of 100 usec. This patch increases it
to 10 ms which should be a reasonnable delay to declare some threads
dead.
Signed-off-by: Eric Leblond <eric@regit.org>
If default-packet-size is not set, it is possible in some case to
guess a correct value.
If PCAP or PF_RING are used we are linked to a "physical" interface.
Thus, it is possible to get information about the link MTU and
hardware header size. This give us the ability to auto discover a
decent default-packet-size.
If suricata is running under a different running-mode, it will
default to 1514.
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
This patch modifies decode.c and decode.h to avoid the usage
by default of a bigger than 65535 bytes array in Packet structure.
The idea is that the packet are mainly under 1514 bytes size and
a bigger size must be supported but should not be the default.
If the packet length is bigger than DFLT_PACKET_SIZE then the
data are stored in a dynamically allocated part of the memory.
To ease the modification of the rest of the code, functions to
access and set the payload/length in a Packet have been introduced.
The default packet size can be set at runtime via the default-packet-size
configuration variable.
Use the --dag <dagname> cmd line option to specify from which DAG card to read pkts
from.
Issue at the moment with pkts being ejected during shutdown -- at the moment we
ignore any packets that are not of link type Ethernet.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:
- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.
This commit fixes bug #124.