Commit Graph

471 Commits (33fca4d4db112b75ffa22eb2e6f14f038cbcc1f9)

Author SHA1 Message Date
Victor Julien 60bfade351 eve: implement frame logging
This is mostly to assist development and QA. It produces too much data
for practical use.
3 years ago
Jeff Lucovsky 163f70be9d logging: Stacktrace on signal term setting
This commit adds a configuration setting to enable a stack trace message
if Suricata receives a signal that terminates execution, such as
SIGSEGV, SIGABRT.
3 years ago
Lukas Sismis a7faed1245 dpdk: initial support with workers runmode
Register a new runmode - DPDK. This enables a new flag on Suricata start
(--dpdk).

With the flag given, DPDK runmode is enabled.

Runmode loads the configuration and then initializes EAL.

If successful, it configures the physical NICs according to the configuration
file. After that, worker threads are initialized and then are in continuous
receive loop.
3 years ago
Philippe Antoine 424dcda2c0 http2: enable by default 3 years ago
Victor Julien 7a797631e0 http/range: cleanup and simplification
Simplify locking by using the THashData lock instead of a separate
range lock.

Avoid size_t in function arguments.

Clean up file handling functions.

Implement handling of alloc errors.

Rename yaml entry to byterange

Unify public api naming
3 years ago
Philippe Antoine e82416a415 http/range: reassemble files from different flows with range
adds a container, ie a thread safe hash table whose
key is the filename

keep a tree of unordered ranges, up to a memcap limit

adds HTPFileOpenWithRange to handle like HTPFileOpen
if there is a range : open 2 files, one for the whole reassembled,
and one only for the current range
3 years ago
Jason Ish 54be743c48 prelude: remove the prelude output
It was broken in 6 and that didn't cause much issue. Just remove
it for 7.
4 years ago
Sascha Steinbiss 9aedc7fd1a mqtt: enable in config and remove misleading comment 4 years ago
Victor Julien b08a7b9a66 stream: update memcaps in code to match config 4 years ago
Victor Julien 3c1cc1e345 mqtt: move sub/unsub limits into app-layer config 4 years ago
Sascha Steinbiss 4c0ef73bf2 detect/mqtt: add topic inspection limit
We add a new 'mqtt.(un)subscribe-topic-match-limit' option
to allow a user to specify the maximum number of topics in
a MQTT SUBSCRIBE or UNSUBSCRIBE message to be evaluated
in detection.
4 years ago
Jeff Lucovsky 8867dcf403 config/plugin: Add template for plugins 4 years ago
frank honza ecdf9f6b0b ikev1: rename ikev2 to common ike
Renaming was done with shell commands, git mv for moving the files and content like
find -iname '*.c' | xargs sed -i 's/ikev1/ike/g' respecting the different mixes of upper/lower case.
4 years ago
Philippe Antoine 7500c29300 decode: limits the number of decoded layers
so as to avoid overrecursion leading to stack exhaustion
4 years ago
Victor Julien af13d4de18 detect: set HTTP SWF decompress limits 4 years ago
Philippe Antoine a04b5566a6 http: makes decompression time limit configurable 4 years ago
Phil Young dc5349a30c napatech: Add Deprecation Warning Message for HBA
Added a message that HBA will be deprecated in the future and removed
hba from default conf file.
4 years ago
Jeff Lucovsky 8f009cf9b5 output/json: Update threaded filename example 4 years ago
Shivani Bhardwaj 1286b0a8f1 datasets: parse defaults section from yaml
Datasets can now have a global defaults setting in suricata.yaml. In
case the settings for memcap and hashsize are not find in the yaml or
rule, this shall be the fallback.

Example:

datasets:
  defaults:
    memcap: 100mb
    hashsize: 2048
  ua-seen:
    type: string
    load: datasets.csv
5 years ago
Philippe Antoine 9b5c923327 http: disables lzma by default for HTTP 5 years ago
Victor Julien 57a611b429 decode/geneve: add config to yaml 5 years ago
Ali Jad Khalil 5d955c1836 decode/geneve: Add Geneve decoding functionality
These changes are in response to feature request 3063. Geneve is
very similar to VXLAN, but uses a slightly different encapsulation
scheme.
5 years ago
Jason Ish 5d5eef624b suricata.yaml: mark http2 as experimental
Make it clear that HTTP2 is experimental and disabled by default.
5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Sascha Steinbiss 4e1a41a17d output-json: add MAC address output
This commit adds MAC address output to the EVE-JSON format. We follow the
remarks made in Redmine ticket #962: for packets, log MAC src/dst as a
scalar field in EVE; for flows, log MAC src/dst as lists in EVE. Field names
are different between flow and packet context to avoid type confusion
(src_mac vs. src_macs). Configuration approach and JSON representation is
taken from previous GitHub PR #2700.
5 years ago
Victor Julien 5db1d9b841 eve/yaml: move mqtt down 5 years ago
Sascha Steinbiss c31360070b rust/mqtt: add MQTT parser 5 years ago
Jeff Lucovsky 30ae98f658 output/json: Multi-threaded EVE logging support
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.

Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
5 years ago
Shivani Bhardwaj 9f9670ebdc logging: Add DCERPC logger 5 years ago
Jeff Lucovsky a58fdcd41d suricata.yaml.in: update stream-depth description 5 years ago
Vadym Malakhatko a80f705d4b userguide: add documentation for Hassh usage
1. Rules keywords
2. Json keywords
3. Usage in lua
4. Enabling in configuration file
5 years ago
Jason Ish e26718aea3 drop-log: remove drop log (deprecated)
Remove the old style line based drop log.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2381
5 years ago
Jason Ish 6ce9b2972b rdp: enable by default
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3255
5 years ago
Jason Ish 5a7ba62493 sip: enable by default
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3256
5 years ago
Jason Ish 6850dbc852 suricata.yaml: remove filestore v1 configuration 5 years ago
Victor Julien 1aaf9a80c5 decode/vxlan: minor yaml example clarrification 5 years ago
Victor Julien e97cdb48f3 decode/teredo: implement port support
Implement support for limiting Teredo detection and decoding to specific
UDP ports, with 3544 as the default.

If no ports are specified, the old behaviour of detecting/decoding on any
port is still in place. This can also be forced by specifying 'any' as the
port setting.
5 years ago
Frank Honza 1c8943dedd add RFB parser
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
5 years ago
Jeff Lucovsky 0c5c2173bc config: General typo and grammar cleanup 5 years ago
Jason Ish 76582e34c1 suricata.yaml/dns: removed unused settings
Remove DNS settings global-memcap, state-memcap and request-flood.
These have never been used in the Rust implementation of the DNS
decoder.
5 years ago
Jason Ish d86973b386 unified2: remove deprecated output unified2
Ticket 2385:
https://redmine.openinfosecfoundation.org/issues/2385
5 years ago
Phil Young 1c99536945 napatech: add hardware based bypass support
Napatech hardware bypass support enables Suricata to utilize
capabilities of Napatech SmartNICs to selectively bypass flow-based
traffic.
5 years ago
Philippe Antoine 4a2918e6b5 yaml: clarify comment about dump-all-headers
Logs a warning if the value is unknown
Fixes #2810
5 years ago
Jason Ish 16221c0b33 suricata.yaml/dns: small cleanups, not that default is v2
Note that the eve dns log format is version 2 by default.

Make the value of commented out values their default.

Update the comment on the types to better reflect what it does.
5 years ago
Philippe Antoine 6921608673 http: updates suricata.yaml comments
As well as the userguide documentation about suricata.yaml
5 years ago
Konstantin Klinger 808ea0dba9 app-layer: remove obsolete msn protocol detection 5 years ago
Victor Julien ebecaca7ea eve/anomaly: enable by default
Default config will only enable 'app-layer' type within the anomaly
logger.
5 years ago
Victor Julien 514c7c1a04 yaml: minor improvements 5 years ago
Victor Julien cec8067001 yaml: clean up 'autofp-scheduler' option 5 years ago
Jeff Lucovsky 883cad1a86 logging/anomaly: Clarify anomaly logging
Clarify the description of the anomaly logging types.
5 years ago
Jeff Lucovsky af615baaf7 logging/alert: Expand alert logging description
Clarify the configuration requirements for alerts and http-body logging.
5 years ago
Victor Julien 788c9f8f11 tls/ja3: don't disable; allowing runtime enabling 5 years ago
jason taylor e4156b2f89 config: update lzma size notes to match others
Signed-off-by: jason taylor <jtfas90@gmail.com>
6 years ago
Jason Ish 6c2e9ac27c suricata.yaml: mark drop.log as deprecated 6 years ago
Jason Ish 4e12984ac8 suricata.yaml: mark unified2 as deprecated 6 years ago
Jason Ish d891a8cb79 config: remove all but a stub of file-store v1.
Remove most of the file-store v1 configuration section and mark
it as deprecated. Provide a link where to find the available
options.
6 years ago
Victor Julien be6cdd37f8 stream: remove fix stream.depth references 6 years ago
Philippe Antoine af4f816204 http: sets compression bomb limit 6 years ago
Philippe Antoine c09ad01836 http: disable lzma decompression from configuration 6 years ago
Victor Julien 8765839084 sip: disable output by default 6 years ago
Jason Ish a45a2fa1fc sip: disable by default in 5.0 6 years ago
Giuseppe Longo edc2a583a9 rust/sip: add SIP logger 6 years ago
Giuseppe Longo 2e975a0481 rust/sip: add parser for SIP protocol 6 years ago
Jason Ish 4111272c88 config/anomaly: use enabled key word; cleanups
The anomaly section was commented out, but the types sub object
was not, which then attached the types keyword to the previous
object.

Instead keep "anomaly" enabled in the yaml (not commented out)
and use the "enabled: no" to have it disabled by default.

Additonally reformat the comments to be better viewed in 80
columns.
6 years ago
Jason Ish 61a6eaf330 htp/lzma: set limit from configuration
Also use a default defined in Suricata, not libhtp.
6 years ago
Jeff Lucovsky aaacbf28c2 logging/anomaly: Support configuration filter types 6 years ago
Victor Julien c1b333c96e rdp: disable eve.rdp by default 6 years ago
Jason Ish 664605b5f1 rdp: disable rdp by default for 5.0 6 years ago
Zach Kelly caef8b5b38 protocol parser: rdp
Initial implementation of feature 2314:
1. Add protocol parser for RDP
2. Add transactions for RDP negotiation
3. Add eve logging of transactions
6 years ago
Shivani Bhardwaj 59da7ae302 counters: Add new default for decoder events
Set the new default for decoder events to `decoder.event` instead of the
previously used `decoder`. Remove the corresponding warning for 5.0.
6 years ago
Victor Julien d5009c5d8c doc/stream: briefly explain bypass 6 years ago
Jason Ish 55852d0de3 rules: remove configuration for legacy rule handling
Removes the autoconf, and suricata.yaml sections for using
the legacy style of rule management.
6 years ago
Jeff Lucovsky 2149807bd6 eve/ftp: Transaction support for unmatched requests
Modified transaction logic to create a new transaction with each
request; replies location transactions by using the oldest "open"
(unmatched) transaction or the last transaction if none are open.
6 years ago
Jeff Lucovsky 9b88ecb3c1 suricata.yaml: Add ftp logging option to eve-log 6 years ago
Bill Meeks a291209e47 detect/geoip: migrate to GeoIP2 database format
Issue #2765
6 years ago
Victor Julien 82de6e0659 decoder/vxlan: improvements and cleanups
Implement port config handling. Also check both src port and dest
port for tunnels that only set the destination port to the VXLAN
port. At the point of the check we don't know the packet direction
yet.

Implement as Suricata tunnel similar to Teredo.

Cleanups.
6 years ago
Jason Ish 577c8cb0c0 dns-log: remove from config
dns-log has been removed from the code.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2297
6 years ago
Eric Leblond 4e94c2b8c2 suricata.yaml: fix path to ebpf and xdp doc 6 years ago
Pierre Chifflier 6fc7fc74cb SNMP: add logger 6 years ago
Pierre Chifflier 2df840a8b8 Add SNMP (v1/v2c/v3) application layer 6 years ago
Mats Klepsland a4eaef25d6 eve: add JA3S field to TLS JSON logger
Add JA3S object to TLS JSON logger (extended log).
6 years ago
Eric Leblond 5d76f0897c af-packet: remove rollover reference
This patch removes reference to rollover in the configuration file
and add warnings when it is used.
6 years ago
Jeff Lucovsky cc492c50c8 eve/logging: disable anomaly logging by default
Disable anomaly logging by default. Networks with excessive issues may
experience packet processing degradation.
6 years ago
Jeff Lucovsky a8938f449d logging: Anomaly logging
This changeset adds anomaly logging to suricata for issue 2282.

Anomaly logging is controlled via the `anomaly` section within eve-log.
There is a single option -- `packethdr` -- for including the packet header
in the anomaly.
6 years ago
Jason Ish fc3191dc2d config: enable all things requiring Rust
Instead of only enabling them if Rust is enabled, as Rust is
always enabled now.
6 years ago
Phil Young 05271bfbe5 napatech: simplify integration with Napatech cards
- There is now an option to automatically create streams on the
  correct NUMA node when using cpu affinity.

- When not using cpu affinity the user can specify streams to be
  created in the suricata.yaml file.  It is no longer required to
  use NTPL to create streams before running suricata.

- The legacy usage model of running NTPL to create streams is still
  available. This can be used for legacy configurations and complex
  configurations that cannot be satisfied by the auto-config option.
6 years ago
Victor Julien d00950be81 log/file: use default-log-dir for suricata.log
Default to just suricata.log instead of the full path, so that
in user mode we can log in the user mode location.
6 years ago
Eric Leblond abe2836caf suricata.yaml: fix name of encryption-handling var 6 years ago
Victor Julien d6903e70c1 file-log: remove and add warning
Feature was deprecated and scheduled for removal.

Ticket #2376
6 years ago
Victor Julien 6fcd2db043 tile: remove files 6 years ago
Victor Julien 517b45ea2d netmap: switch to nm_* API
Process multiple packets at nm_dispatch. Use zero copy for workers
recv mode.

Add configure check netmap check for API 11+ and find netmap api version.

Add netmap guide to the userguide.
6 years ago
Maurizio Abba 6c0ec0b2f3 eve/http: add request/response http headers
Add a keyword configuration dump-all-headers, with allowed values
{both, request, response}, dumping all HTTP headers in the eve-log http
object. Each header is a single object in the list request_headers
(response_headers) with the following notation:

{
    "name": <header name>,
    "value": <header value>
}

To avoid forged malicious headers, the header name size is capped at 256
bytes, the header value size at 2048.

By default, dump-all-headers is disabled.
6 years ago
Maurizio Abba 4697351188 smtp: create raw-extraction feature
Add a raw-extraction option for smtp. When enabled, this feature will
store the raw e-mail inside a file, including headers, e-mail content,
attachments (base64 encoded). This content is stored in a normal File *,
allowing for normal file detection.
It'd also allow for all-emails extraction if a rule has
detect-filename:"rawmsg" matcher (and filestore).
Note that this feature is in contrast with decode-mime.

This feature is disabled by default, and will be disabled automatically
if decode-mime is enabled.
6 years ago
Victor Julien 0d86263efd eve.stats: make decoder event prefix configurable 6 years ago
Victor Julien 1dd81f7346 yaml: add missing eve pcap-file comment 6 years ago
Jason Ish 87250da0fc rust/dns: add v1 dns logging
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2704
6 years ago
Travis Green 3539ae3041 Updated link for Prelude SIEM
Updated link for Prelude SIEM to https://www.prelude-siem.org/
6 years ago
Eric Leblond 24806c2102 suricata.yaml: fix path to XDP doc 6 years ago
Victor Julien 1467c30883 pfring: implement 'threads: auto'
If threads is set to auto, first try the CPU count. If that would
fail, fall back to RSS queue count.
6 years ago
Victor Julien 3ba2c9fba7 pfring: multiple receive threads is not experimental 6 years ago