In unix socket mode, Suricata was stopping processing pcap files
when a pcap file with an unsupported datalink was treated. This
patch updates error handling to allow Suricata to treat other
pcap files.
This patch adds a 'conf-get' command which get the configuration
value from suricata. Argument of the command is the name of the
variable to fetch.
The command syntax is the following:
{
"command": "conf-get",
"arguments": { "variable":value}
}
Pcap snaplen related modification broke compilation of Suricata for
system having old pcap library. This patch fixes the issue and allow
old pcap library to honour the snaplen value.
As reported in bug #688, htp_config_set_path_decode_u_encoding
function is not included in libhtp header before 0.3.0. Result
is that suricata compilation fail with an external htp library.
The following patch detect the issue and adds the missing
declaration.
Added a napatech section in the yaml configuration.
hba - host buffer allowance
use-all-streams - whether all streams should be used
streams - list of stream numbers to use when use-all-streams is no
The source-napatech.* files were modified to support the host buffer allowance configuration.
The runmode-napatech.c file was modified to support both the host buffer allowance configuration and stream configuration
Signed-off-by: Matt Keeler <mk@npulsetech.com>
This patch introduces 'snaplen' a new YAML variable in the pcap section.
It can be set per-interface to force pcap capture snaplen. If not set
it defaults to interface MTU if MTU can be known via a ioctl call and to
full capture if not.
Main objective of this patch is to use a dynamic snaplen to avoid
to truncate packet at the currently fixed snaplen.
It set snaplen to MTU length if the MTU can be retrieved. If not, it
does not set the snaplen which results in using a 65535 snaplen.
libpcap is trying to use mmaped capture and setup the ring by using buffer_size
as the total memory. It also use "rounded" snaplen as frame size. So if we set
snaplen to MTU when available we are optimal regarding the building of the ring.
This patch fixes an error in pointer arythmetic and add some
comments to increase maintanability of the code. It also
simplify the decoding code as a careful RFC reading indicate
that if we discard packet containing an authentication field,
it is only possible to have a single origin indication field.
Adds support for match-on conditions (src, dst, any, both)
Uses GEOIP_MEMORY_CACHE for performance reasons
Adds support for negation and multiple countries in the same rule
Bug fixes
Changed to take flow direction from rule, if present
Comments addressed. Unit tests added.
This patch introduces a new set of functions to the ConfGetChildValue
family. They permit to look under a default node if looking under
base node as failed. This will be used to access to default parameters
for a data type (for instance, first usage will be interface).
TAILQ_FOREACH macro was not safe for element removal as it was
accessing the next element in case of a free. This patch is inspired
by Linux list handling and provide a new macro TAILQ_FOREACH_SAFE.
This macro is removal safe and only differs by a last argument being
a temporaty pointer to an element.
prelude_string_set_ref don't like when it is called with a NULL
parameter. This patch adds check for NULL value. This is formally
good as there is no use of a NULL description.
Insert pseudo packet under low load conditions to complete rule swap.
This is necessary when we use autofp active packets where most packets
would be sent to the first queue under low load conditions.
This patch should fix the bug #637. Between pcap files, it uses a
new function HostCleanup() to clear tag and threshold on host with
an IP regputation. An other consequence of this modification is
that Host init and shutdown are now init and shutdown unconditionaly.
Treat sigs with negated addresses as non ip-only.
This fix exposes bug #608, which results in 2 failed unittest which
have now been disabled by this commit. Would be reenabled when we
have #608 fix in.
This patch adds two commands to unix-command. 'iface-list' displays
the list of interface which are sniffed by Suricata and 'iface-stat'
display the available statistics for a single interface. For now,
this is the number of packets and the number of invalid checksums.
The affinity setting code was using the old API. This patch updates
to the new API and also adds a call to RunModeInitiaze() which was
missing in Single running mode.
This patch transforms the unix socket into a flexible system to
add commands (triggered by user) and taks (run periodically).
It introduces two functions UnixManagerRegisterCommand and
UnixManagerRegisterBackroundTask to registed commands and tasks.
Other part of Suricata can then declare a new command via a simple
call of the function. In the case of a command the caller is
responsible of building the answer message using Jansson API. The
sending of the message is made by unix manager code.
This patch introduces a unix command socket. JSON formatted messages
can be exchanged between suricata and a program connecting to a
dedicated socket.
The protocol is the following:
* Client connects to the socket
* It sends a version message: { "version": "$VERSION_ID" }
* Server answers with { "return": "OK|NOK" }
If server returns OK, the client is now allowed to send command.
The format of command is the following:
{
"command": "pcap-file",
"arguments": { "filename": "smtp-clean.pcap", "output-dir": "/tmp/out" }
}
The server will try to execute the "command" specified with the
(optional) provided "arguments".
The answer by server is the following:
{
"return": "OK|NOK",
"message": JSON_OBJECT or information string
}
A simple script is provided and is available under scripts/suricatasc. It
is not intended to be enterprise-grade tool but it is more a proof of
concept/example code. The first command line argument of suricatasc is
used to specify the socket to connect to.
Configuration of the feature is made in the YAML under the 'unix-command'
section:
unix-command:
enabled: yes
filename: custom.socket
The path specified in 'filename' is not absolute and is relative to the
state directory.
A new running mode called 'unix-socket' is also added.
When starting in this mode, only a unix socket manager
is started. When it receives a 'pcap-file' command, the manager
start a 'pcap-file' running mode which does not really leave at
the end of file but simply exit. The manager is then able to start
a new running mode with a new file.
To start this mode, Suricata must be started with the --unix-socket
option which has an optional argument which fix the file name of the
socket. The path is not absolute and is relative to the state directory.
THe 'pcap-file' command adds a file to the list of files to treat.
For each pcap file, a pcap file running mode is started and the output
directory is changed to what specified in the command. The running
mode specified in the 'runmode' YAML setting is used to select which
running mode must be use for the pcap file treatment.
This requires modification in suricata.c file where initialisation code
is now conditional to the fact 'unix-socket' mode is not used.
Two other commands exists to get info on the remaining tasks:
* pcap-file-number: return the number of files in the waiting queue
* pcap-file-list: return the list of waiting files
'pcap-file-list' returns a structured object as message. The
structure is the following:
{
'count': 2,
'files': ['file1.pcap', 'file2.pcap']
}
This patch modifies the file store system to have it create the
file store directory if needed. It dos not create the full
directory tree as the parent directory must have already been
created.
This patch update the glafs list to be able to indicate that a
flag is not supported. This information is used by list-keyword to
display information to the user.
The output of the list-keyword is modified to include the url to
the keyword documentation when this is available. All documented
keywords should have their link set.
list-keyword can be used with an optional value:
no option or short: display list of keywords
csv: display a csv output on info an all keywords
all: display a human readable output of keywords info
$KWD: display the info about one keyword.
In list-keywords and list-app-layer mode, suricata now only
displays the messages linked with the feature. This allow users
to redirect the output and easily work on it. For exemple, the
csv output will be easily imported into a spreadsheet.
This patch update the list-keyword command. Without any option,
the previous behavior is conserved. If 'all' is used as option,
suricata print a csv formatted output of keyword information:
name;features;description
If a keyword name is used as argument, suricata print a readable
message:
tls.subject
Features: state inspecting
Description: Match TLS/SSL certificate Subject field
As we don't parse the YAML file when listing of keywords is asked,
suricata make a test on existence of the build-default directory.
So with a non standard (working) install (even a single configure
without option lead to a failure), the keyword listing fails
because the default logging directory does not exist.
It is now possible to use the 'daemon-directory' configuration
variable to specify the working directory of suricata in daemon
mode. This will permit to specify the place for core and other
related files.
This patch creates a pid file per default and use it to avoid to be
able to run two Suricata. Separate pid file have to be provided to
be able to do it.
Removed the Napatech 2GD support
runmode-napatech-3gd.c had an include from runmode-napatech.h which was erroneous and has been removed as well.
Signed-off-by: Matt Keeler <mk@npulsetech.com>
For use with Network Cards from Napatech utilizing the 3GD driver/api.
- Implemented new run modes in runmode-napatech-3gd.*
- Implemented capture/decode threads in source-napatech-3gd.*
- Integrated the new run modes and source into the build infrastructure.
New configure switches
--enabled-napatech-3gd : Turns on the NT 3GD support
--with-napatech-3gd-includes : The directory containing the NT 3GD header files
--with-napatech-3gd-libraries : The directory containing the NT 3GD libraries to link against.
New CLI switch
--napatech-3gd : Uses the Napatech 3GD run mode
Runmodes Supported:
- auto
- autofp
- workers
Notes:
- tested with 1 Gbps sustained traffic (no drops)
Signed-off-by: Matt Keeler <mk@npulsetech.com>
stream.inline YAML configuration variable now support the 'auto' value.
In this case, inline mode is activated for IPS running mode (NFQ and
IPFW) and is deactivated for IDS mode. This patch should fix bug #592.
If no packet arrives to a capture thread, it is possible that the
AFPReadLoop() function goes into an infinite loop. This could cause
suricata to hang at exit on non busy system.
This patch adds a counter to detect when Suricata start looping in
the ring to stop when it reaches this point.
Set event on overlapping data segments that have different data.
Add stream-events option stream-event:reassembly_overlap_different_data and
add an example rule.
Issue 603.
This patch adds a call to close the queue when the acquisition
loop is ending. This way the incoming packets will be accepted
during all the shutdown phase (if the queue-bypass option of
NFQUEUE is used). At the same time the currently processed packets
will be dropped but the time scale are different: suricata will
drop 20 ms of packets and the shutdown can take 0.5 seconds.
Patch based on an idea of Victor Julien.
If a pattern has matched on mpm, don't re-inspect it later, subject to certain
conditions met by the pattern - namely, not negated, right chop, no replacet
attached to it.
The power of libhtp customisation now available to users.
Options available -
path-backslash-separators: yes
path-compress-separators: yes
path-control-char-handling: none
path-convert-utf8: yes
path-decode-separators: yes
path-decode-u-encoding: yes
path-invalid-encoding-handling: preserve_percent
path-invalid-utf8-handling: none
path-nul-encoded-handling: none
path-nul-raw-handling: none
set-path-replacement-char: ?
set-path-unicode-mapping: bestfit
You can use this for your libhtp customisation. Options explained in our
wiki.
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Advanced_libhtp_customization