Commit Graph

8762 Commits (0bb07b550c1f00a95e6803aba7cdeb22d637fbc2)

Author SHA1 Message Date
Eric Leblond 5b056c15bf af-packet: fix default in pinned maps name 7 years ago
Eric Leblond eff56acca5 af-packet: be sure to nullify option if not set 7 years ago
Eric Leblond 36c6a62954 util-ebpf: simplify function declarations 7 years ago
Eric Leblond 69630d7a17 util-ebpf: micro optimization 7 years ago
Eric Leblond d21c3a6555 util-ebpf: create flow from bypassed flows 7 years ago
Eric Leblond 04c65a309e flow-hash: new function to get flow from flowkey 7 years ago
Eric Leblond 885fc992de ebpf: make table iterator generic
Also adds a basic skeleton for flow creation loop.
7 years ago
Eric Leblond 880c42f11c af-packet: bypass with init function 7 years ago
Eric Leblond 522e98d830 util-ebpf: fix iteration in flow timeout
We were not setting the key using the correct item in map. Result
was deletion of wrong flow.
7 years ago
Eric Leblond 0c3e1e8579 af-packet: correctly set up hardware offload 7 years ago
Eric Leblond bd28f77a1e af-packet: fix loading of ebpf filter 7 years ago
Eric Leblond 6fdcb127e9 af-packet: fix the start when XDP is pinned 7 years ago
Eric Leblond 7e0ef4cec8 util-ebpf: change return of pinned maps loading
The calling function needs to be able to see when this is a success
and XDP do not need to be reloaded.
7 years ago
Eric Leblond d950a9f272 util-ebpf: conditional flow table loading 7 years ago
Eric Leblond d25e8dbfc7 af-packet: implement pinned-maps-name 7 years ago
Eric Leblond 36838017fe af-packet: fix build when eBPF not built-in 7 years ago
Eric Leblond b1769d5f8f util-ebpf: implement pinned maps loading
Load flow tables at start if asked to.
7 years ago
Eric Leblond 96f1454ebf util-ebpf: only unlink pinned maps in eBPF filter 7 years ago
Eric Leblond 4cf531008e af-packet: conditionaly remove XDP filter
Only remove the XDP filter if we are in XDP mode and not using the
pinned maps.
7 years ago
Eric Leblond 4f48c45727 util-ebpf: conditional pinning of maps
Only pin maps if `pinned-maps` is set in the configuration. This
ensure backward compatibility.
7 years ago
Eric Leblond 73a928fa0b util-ebpf: fix loop on maps
We were missing the last element of the map by working on previous
key instead of current key.
7 years ago
Eric Leblond 3ce69c446b util-ebpf: suppress spaces at end of line 7 years ago
Eric Leblond 954762a429 ebpf: add filter by maps on example filter 7 years ago
Eric Leblond 5d8ac36a49 util-ebpf: pin the maps
By pinning the maps we are creating a file in /sys/fs/bpf that can
be used by external program to access the map. This has multiple
benefits such as handling list from an external program.

The pinned maps could be persistent accross Suricata reload but
this can be complicated in term of handling everything in the life
of Suricata.
7 years ago
Eric Leblond bf4381b17b ebpf: document XDP iface redirect 7 years ago
Eric Leblond a37a7c4a21 ebpf: reduce counter size to allow netronome offload 7 years ago
Eric Leblond 315c29a8e6 ebpf: change the logic to avoid ktime usage
Kernel time is not available (and/or costly) on NIC such as
Netronome so we update the logic to detect dead flows based on a
lack of update of packets counters. This way, the XDP filter will
be usable by network card.

This patch also updates the ebpf code to support per CPU and
regular mapping. Netronome is not supporting it and the structure
is using atomic for counter so the cost of simultaneous update
is really low.

This patch also updates the xdp_filter to be able to select if the
flow table is per CPU on shared. Second option will be used for
hardward offload. To deactivate the per cpu hash, you need to set
USE_PERCPU_HASH to 0.

This patch also adds an new option to af-packet named no-percpu-hash
If this option is set to yes then the Flow bypassed manager thread
will use one CPU instead of the number of cores. By doing that
we are able to handle the case where USE_PERCPU_HASH is unset (so
hardware offload for Netronome).

This patch also remove aligment indications in the eBPF filter. This
was not really needed and it seems it is causing problem with
some recent version of LLVM toolchain.
7 years ago
Eric Leblond 1745408c13 flow-manager: no force reassembly on bypassed flow
When a bypassed flow is created we are forcing the reassembly so
we don't need to do it again when it timeout.
7 years ago
Shivani Bhardwaj 7c7e04487f detect/content: Message for escaping backslash
So far, if the rule loaded had a backslash character ("\") in its
content field, the rule will fail to load but without giving a
descriptive error message. This patch tells the user to escape the
troubling character.

Before
```
9409] 7/6/2019 -- 16:12:22 - (detect-engine-loader.c:184) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Backslash needs escaping msg"; flow:established,to_server; content:"MyBackslash\here"; sid:86; rev:1; )" from file /var/lib/suricata/rules/myrule.rules at line 1
```

After
```
[9409] 7/6/2019 -- 16:12:22 - (detect-content.c:155) <Error> (DetectContentDataParse) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - '\' has to be escaped
[9409] 7/6/2019 -- 16:12:22 - (detect-engine-loader.c:184) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Backslash needs escaping msg"; flow:established,to_server; content:"MyBackslash\here"; sid:86; rev:1; )" from file /var/lib/suricata/rules/myrule.rules at line 1
```

Closes redmine ticket #2626
7 years ago
Victor Julien 7513f6cae2 threads: minor code cleanups 7 years ago
Victor Julien 92d38683ce threads: improve flow timeout loop
Improve thread safety and remove BUG_ON
7 years ago
Victor Julien e6b633cf95 stats: walk tv_root under lock 7 years ago
Victor Julien 9474b84683 stats: more accurate time handling for wakeup thread 7 years ago
Victor Julien ab1268cfea stats: minor code cleanups 7 years ago
Philippe Antoine b5f3e03209 New app layer event for invalid http request line
Handles logs from libhtp even if case of error
7 years ago
Philippe Antoine 4a74e9edcc signature: error for rules with illegal port
Fixes #2080
7 years ago
Philippe Antoine 248f9c66cf warnings: fixes integer sizes in format strings
Fixes #3009
7 years ago
Philippe Antoine a3e38861b9 http: adds unit tests for HTPParseContentRange 7 years ago
Philippe Antoine 90ab0b0ec2 http: fix overflow in HTPParseContentRange 7 years ago
Victor Julien 39cf5b0c8b snmp: fix 'defined but not used' warning
Remove HAVE_RUST guards as well.
8 years ago
Pierre Chifflier 8c61cfdca5 snmp.version: move unittests to tests/ 8 years ago
Pierre Chifflier 58d85b1460 snmp.pdu_type: move unittests to tests/ 8 years ago
Pierre Chifflier ff0ff6344f snmp.community: move unittests to tests/ 8 years ago
Pierre Chifflier 509a54281f detect/snmp: convert snmp.community keyword to v2, and MPM 8 years ago
Pierre Chifflier c67a53773b detect/snmp: rename keywords to snmp.<name>, and make rust mandatory 8 years ago
Pierre Chifflier 9dfec7e734 SNMP: add the "snmp.pdu_type" detection keyword 8 years ago
Pierre Chifflier e1dd19a0eb SNMP: add the "snmp.community" detection keyword 8 years ago
Pierre Chifflier aa608e0ca2 SNMP: add the "snmp.version" detection keyword 8 years ago
Pierre Chifflier ed4823bbe8 SNMP: add missing case for profiling enum 8 years ago
Pierre Chifflier 6fc7fc74cb SNMP: add logger 8 years ago
Pierre Chifflier 2df840a8b8 Add SNMP (v1/v2c/v3) application layer 8 years ago
Victor Julien cdd061ba37 ikev2: clean up c glue code 8 years ago
Victor Julien 3f36a6ce59 tftp: c glue code cleanup 8 years ago
Victor Julien ade2b1e6cf output: get rid of BUG_ONs 8 years ago
Victor Julien 9ea6db3889 pool: don't call Cleanup after failed Init
Stream reassembly memcap is regulated by the Init and Cleanup
callbacks. If Init fails due to memcap reached, Cleanup had no
way of knowing and it would decrease the memcap even if it hadn't
been increased by Init. This could lead to too much memory use and
memcap counter underflow.

This patch fixes the issue by not calling Cleanup in this case. It's
fair to see a failed Init the responsibility of Init.
8 years ago
Victor Julien 597ff3c8e3 pool/thread: clean up tests 8 years ago
Victor Julien 025cc64f55 pool/thread: remove old grow function 8 years ago
Victor Julien 67ba132c10 stream: simplify segment and ssn pool inits 8 years ago
Victor Julien 0b02efdf01 pool/thread: introduce simpler way to grow thread pool 8 years ago
Victor Julien 6e6c1bad7d pool/thread: minor code cleanups 8 years ago
Victor Julien 924982e792 app-layer: minor cleanups 8 years ago
Victor Julien ce71bf1fff capture: check for flow packets on capture timeout
The capture threads can receive packets from the flow manager in their
Threadvars::stream_pq packet queue. This mechanism makes sure the packets
the flow manager injects into the engine are processed by the correct
worker thread.

If the capture thread(s) would not receive packets for a long time, the
Threadvars::stream_pq would not be checked and processed. This could
lead to packet pool depletion in the flow manager. It would also lead
to flows not being timed out/logged until either packets started flowing
again or until the engine was shut down.

The scenario is more likely to happen in a test (e.g. replay) but could
also delay logging on low traffic sensors.
8 years ago
Victor Julien 952cbb563c app-layer: mandatory tx registration checks
All protocols now implement the TX API, so the runtime checks for
whether or not a protocol supports the TX API can be removed.
8 years ago
Victor Julien 8bc1e120ea app-layer: make tests tx aware
Make tests minimally TX-aware so we can asume all parsers use
the tx API.
8 years ago
Victor Julien 3526569b25 ssh: test cleanup
Remove explicit locks. Set ALPROTO_SSH in all tests.
8 years ago
Victor Julien 480f497d05 packetpool: move return stack to pool earlier
If waiting for N packets move the return stack to the main
stack every time we take the return stack lock.

Make sure we consider enough packets when setting the pending pool
flush logic. This should at least make sure to have the 9 packets
the flow manager requires per run.
8 years ago
jason taylor fa0008dbbe detect/content: clean up logging output
Signed-off-by: jason taylor <jtfas90@gmail.com>
8 years ago
Jeff Lucovsky 0960ca0d00 detect/analyzer Add missing HTTP values
This changeset adds recognition of missing HTTP values
- Raw host
- Header names
- Server body
- User agent
8 years ago
Philippe Antoine a370d2a91d signature: fix memory leak in DetectBytejumpSetup 8 years ago
Philippe Antoine 8a339e73d3 http: adds an event for double encoded uri 8 years ago
Philippe Antoine e5879650bb http: fixes double-encode configuration keywords 8 years ago
Mats Klepsland f634da61c9 ja3s-string: move unittests to tests/ 8 years ago
Mats Klepsland 736d35f36e ja3s-hash: move unittests to tests/ 8 years ago
Mats Klepsland 0b489f329c detect: add (mpm) keyword ja3s.string
Match on JA3S string using ja3s.string keyword, e.g:

  alert tls any any -> any any (msg:"ja3s.string test";
      ja3s.string; content:"10-11-12"; sid:1;)
8 years ago
Mats Klepsland 80cee50916 detect: add (mpm) keyword ja3s.hash
Match on JA3S hash using ja3s.hash keyword, e.g:

  alert tls any any -> any any (msg:"ja3s.hash test";
      ja3s.hash; content:"b26c652e0a402a24b5ca2a660e84f9d5"; sid:1;)
8 years ago
Mats Klepsland 03e8e658d7 lua: add Ja3SGetString function
Add Ja3SGetString() to return the content of the JA3S string buffer from
the TLS session.

Example:

  function init (args)
      local needs = {}
      needs["protocol"] = "tls"
      return needs
  end

  function setup (args)
      filename = SCLogPath() .. "/ja3s_string.log"
      file = assert(io.open(filename, "a"))
  end

  function log (args)
      ja3s_string = Ja3SGetString()
      if ja3s_string == nil then
          return
      end

      file:write(ja3s_string .. "\n")
      file:flush()
  end

  function deinit (args)
      file:close()
  end
8 years ago
Mats Klepsland b1d5fe9657 lua: add Ja3SGetHash function
Add Ja3SGetHash() to return the content of the JA3S hash buffer from
the TLS session.

Example:

  function init (args)
      local needs = {}
      needs["protocol"] = "tls"
      return needs
  end

  function setup (args)
      filename = SCLogPath() .. "/ja3s_hash.log"
      file = assert(io.open(filename, "a"))
  end

  function log (args)
      ja3s_hash = Ja3SGetHash()
      if ja3s_hash == nil then
          return
      end

      file:write(ja3s_hash .. "\n")
      file:flush()
  end

  function deinit (args)
      file:close()
  end

In the example above, each JA3S hash is logged to a log file.
8 years ago
Mats Klepsland a4eaef25d6 eve: add JA3S field to TLS JSON logger
Add JA3S object to TLS JSON logger (extended log).
8 years ago
Mats Klepsland a4471987ba app-layer-ssl: generate JA3S fingerprints
Generate JA3S fingerprints based on fields in the ServerHello record.
8 years ago
Alexander Gozman 3a16009966 Bug #2965: fix NFQ arguments parsing 8 years ago
Andreas Herz 8baf64f5e9 af-packet: fix setting block_timeout value through afpconfig 8 years ago
Jeff Lucovsky 462a4e2b5b detect/analyzer: Improve warning message
This changeset modifies the warning printed when a rule
is determined to detect in both directions.
8 years ago
Eric Leblond 5d76f0897c af-packet: remove rollover reference
This patch removes reference to rollover in the configuration file
and add warnings when it is used.
8 years ago
Philippe Antoine 037d50ef06 signature: fix overflow in parsing 8 years ago
Philippe Antoine 3e12066819 http: adds events for each libhtp log
Fixes #997
8 years ago
Mats Klepsland 3c57ac144c detect-ssl-version: move unittests to tests/ 8 years ago
Mats Klepsland 238797cc66 detect-ssl-state: move unittests to tests/ 8 years ago
Mats Klepsland 479e73b98e detect-tls-version: move unittests to tests/ 8 years ago
Mats Klepsland 767bde5e74 detect-tls-cert-validity: move unittests to tests/ 8 years ago
Mats Klepsland a260a57b68 detect-tls-sni: move unittests to tests/ 8 years ago
Mats Klepsland adb4da3975 detect-tls-ja3-string: move unittests to tests/ 8 years ago
Mats Klepsland 74a7b7e3cf detect-tls-ja3-hash: move unittests to tests/ 8 years ago
Mats Klepsland 5d3b94b3e4 detect-tls-cert-subject: move unittests to tests/ 8 years ago
Mats Klepsland 0d728ee4c6 detect-tls-cert-serial: move unittests to tests/ 8 years ago
Mats Klepsland e125e58c97 detect-tls-cert-issuer: move unittests to tests/ 8 years ago
Mats Klepsland 3646234ac5 detect-tls-cert-fingerprint: move unittests to tests/ 8 years ago
Mats Klepsland 12d37b8b2c detect-tls: tidy up unittests
By doing the following:
- removing unnecessary locks
- moving variable declarations
- removing redundant function 'SigCleanSignatures'
8 years ago
Mats Klepsland 15012fc908 ja3: check if JA3 is disabled on one line 8 years ago
Mats Klepsland 285855d928 detect-tls: remove NULL settings from keyword registration 8 years ago
Mats Klepsland 008f08c1b3 detect-tls: declare ssl_state as const in GetData() 8 years ago
Mats Klepsland 0f7f35bd85 detect-tls: check return values of functions on setup
Check the return values of DetectBufferSetActiveList() and
DetectSignatureSetAppProto().
8 years ago
Mats Klepsland 1c04d7cdae detect-tls: remove confusing underscores from variables
Remove confusing underscore prefix from variables in GetData() for
all tls keywords.
8 years ago
Jeff Lucovsky de983fb7c9 app-layer-ftp: Potential memory leak fixed
Ensure that when handling failures during STOR command
processing, that all memory is freed on the error path.
8 years ago
Mats Klepsland ba857e9739 detect: add tls.certs keyword
Add keyword to do "raw" matching on each of the certificates in the
TLS certificate sticky buffer.

Example:
  alert tls any any -> any any (msg:"tls.certs test"; tls.certs; \
          content:"|01 02 03 04|"; sid:1;)
8 years ago
Victor Julien edae50de94 detect/ssh: fix ssh.protoversion memory leak 8 years ago
Victor Julien 567a7c3cef detect/ssh: mark old ssh keywords as deprecated 8 years ago
Victor Julien d623dc4ac0 detect/parse: add flag to indicate keyword is deprecated
Issue warning when it is still used.
8 years ago
Victor Julien b84eba80aa detect/nfs.version: minor cleanups 8 years ago
Victor Julien 2ea11da230 detect/nfs: add nfs.version 8 years ago
Victor Julien 3299f007f8 detect/dcerpc: add dcerpc.iface
Keep dce_iface as an alias.
8 years ago
Victor Julien cdff1d50b7 detect/dcerpc.opnum: minor code cleanups 8 years ago
Victor Julien 6840e5c7df detect/dcerpc: add dcerpc.opnum as new name for dce_opnum 8 years ago
Philippe Antoine b6b7778e2d http: adds event for header repetition 8 years ago
Jason Ish 9d8eb7b5f0 filestore: remove jansson ifdefs
Jansson is now required.
8 years ago
Jason Ish 3dc973d4b1 eve/file: remove rust and jansson ifdefs.
Both Rust and Jansson are required now.
8 years ago
Jason Ish 42c327adc4 filestore: fix leak in contructing json
Use json_array_append_new instead of json_array_append to transfer
ownership of the integer object to jansson so it gets freed.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2961
8 years ago
Victor Julien ddfcf76c57 detect/engine: make DetectAppLayerMpmRegister decprecated 8 years ago
Victor Julien 752bb1c410 detect/dnp3: add dnp3.data with v2 api support
Adds MPM support as well. Add TxDetectFlags support to the parser
to avoid duplicate matches.
8 years ago
magenbluten 09a21627d5 filestore: fix dropping of unwanted files (Issue #2853) 8 years ago
Victor Julien 9132e4032a files: open files with track id only 8 years ago
Victor Julien 3b31bad855 detect/dce_stub_data: add dcerpc.stub_data
Also use v2 API for inspect and mpm registration.
8 years ago
Victor Julien d270a7603a detect/inspect: add flags to inspect buffer 8 years ago
Victor Julien 32fb7d773a detect/content-inspect: turn void arg into Packet
Replace the 'void *data' argument by a 'Packet *p' as this was
the only user left of the data pointer.
8 years ago
Victor Julien b7a7517273 detect/dce_stub_data: minor cleanups 8 years ago
Victor Julien 55db6d6fb4 detect/dcerpc: move endian handling from pointer to flags 8 years ago
Victor Julien b2638f7195 detect/krb5: add krb5.sname and krb5.cname 8 years ago
Victor Julien aefce4d761 detect/nfs: remove HAVE_RUST guards 8 years ago
Victor Julien 15eac12a39 afl: fix compilation 8 years ago
Victor Julien 3ae2edb22a ftp: fix realloc handling to avoid valgrind warning
Bug #2951
8 years ago
Victor Julien 84881bf1b8 detect/file.magic: add sticky buffer
Add sticky buffer to inspect file magic. Includes mpm support.
8 years ago
Victor Julien d78c6ff714 detect/thread: ctx info is allowed to have NULL data 8 years ago
Victor Julien aa52dfab04 detect/smb: clean up keywords 8 years ago
Victor Julien d64fbb71ae detect/file: add file.data, small cleanups 8 years ago
Victor Julien b5d5389438 detect/ssh: minor --list-keywords improvements 8 years ago
Victor Julien f246e319b2 detect/http.header.raw: minor cleanups 8 years ago
Victor Julien a21a7d16bd detect/http.host.raw: minor cleanups 8 years ago
Victor Julien 0e1d47c87b detect/http.method: minor cleanups 8 years ago
Victor Julien bdd8e6152b detect/http.start: modernize name and code 8 years ago
Victor Julien cd2e6511c9 detect/http: cleanup http stat * 8 years ago
Victor Julien 84da0376fb detect/http.host: rename file for consistency 8 years ago
Victor Julien 2b8311beff detect/http.host: fix --list-keywords output 8 years ago
Victor Julien 0e5c987533 detect/http.uri: fix up --list-keywords output 8 years ago
Victor Julien 19163ca2e1 detect/http: request/response line keyword modernization 8 years ago
Victor Julien fb2e4e4453 detect/http.header_names: use v2 api and new name 8 years ago
Philippe Antoine 316a411b6b ssl : SSLProbingParser overflow fix
Found by fuzzing
Fixes ssl detection evasion by packet splitting
8 years ago
Victor Julien 666bb1b6e4 parse/ip: fix potential oob write in ipv4 validation
Found using AFL.
8 years ago
Jason Ish b8ce7f2885 mpls: check buffer length before peeking at next header
Check that we have enough bytes before peaking into the MPLS
packet payload.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2884
8 years ago
Jason Ish 8d7d6a96a5 ethernet: fix next packet size on DCE packet
Missing parans on the DCE length caused the length update
for the next call to DecodeEthernet to be wrong.

Tests added.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2887
8 years ago
Victor Julien 76cc03010a ssh: fix banner overflow issue
Reported-by: Sirko Höer - Code Intelligence
8 years ago
Victor Julien 2b75222250 runmodes: for test runmodes, clean up properly
For conf test and engine analysis, clean up memory correctly.

This helps valgrind tests for leaks.
8 years ago
Jeff Lucovsky 74f436d209 logging: display base64 decoded string for packet
This changeset changes the packet display to be base64, rather than hex.
8 years ago
Jeff Lucovsky 7d28c19f05 logging: Ensure all anomalous events have an event_type
This change ensures that each anomaly is tagged with an
event type to support querying.

Each anomalous event will include `"event_type": "anomaly"`
in the log record.
8 years ago
Jeff Lucovsky 5e222129d5 eve/alert: Remove unused results from PrintRawLineHexBuf
This changeset removes the call to `PrintRawLineHexBuf`. The
return values were never used.
8 years ago
Jeff Lucovsky a8938f449d logging: Anomaly logging
This changeset adds anomaly logging to suricata for issue 2282.

Anomaly logging is controlled via the `anomaly` section within eve-log.
There is a single option -- `packethdr` -- for including the packet header
in the anomaly.
8 years ago
Philippe Antoine a1c6e091ac http: new event for auth unrecognized
activates libhtp auth parsing
Fixes #984
8 years ago
Jeff Lucovsky 7f102d95b6 detect: Modernize TLS keywords
This changeset adds keywords for "tls.<name>" and moves the existing
value of "tls_<name>" to an alias.
8 years ago
Alexander Bluhm 36796de731 init: pledge(2) needs "fattr" during suricata reload.
When killed with SIGHUP, suricata reopens the log files.  If filemode
is set in the config, it needs pledge promise "fattr" to allow the
chmod(2) on OpenBSD.
8 years ago
Giuseppe Longo af9399f2ac detect-http-protocol: use v2 inspect/mpm engines
This updates inspect/mpm engines to v2.
8 years ago
Victor Julien a6a0b0aa4a detect/files: fix file sigs state handling
Make sure all file sig mismatches indicate this in their return
code, not just the ones with filestore enabled. This is needed
to tell the stateful detect engine that it is dealing with a file
sig, so it can make sure these are inspected correctly even if
there are possibly multiple files per tx.
8 years ago
Victor Julien 225cdf996e eve/alert: take vlan from packet, not flow
Flow is not guaranteed to exist.
8 years ago
Eric Leblond 497f35164b detect-filename: avoid multiple inspections of buf
If the filename inspection function is returning nomatch this will
trigger iterative inspections with same content (aka filename) being
inspected. To avoid this we change the return as the buffer inspection
has not to be inspected anymore.
8 years ago
Eric Leblond 63f2032ed6 doc: fix way to build URL 8 years ago
Eric Leblond f48ac1860a detect-lua: implement sticky buffer
This patch implement an option named 'buffer' that can be used in the
init function of a lua signature:

 function init (args)
     local needs = {}
     needs["buffer"] = tostring(true)
     return needs
 end

With this, the lua script will get access to the sticky buffer
content.
8 years ago
Eric Leblond 62a11dd3ed detect-lua: fix DNP3 value 8 years ago
Mark Janssen 600f2ab391 eve/json: always output vlan field as array 8 years ago
Mark Janssen 0cc3c2cc6c eve/flow: add in_iface field
Fixes #2057
8 years ago
Mark Janssen fed9b7a180 eve/flow: add vlan field 8 years ago
Bendik Hagen f558ef2c55 Flow/Stream: set psuedopacket iface/vlan from flow
This fixes redmine bug #2057 by setting pseudopacket iface and vlan from
flow values, solving the problem of missing vlan/iface when psuedopacket
gets logged/alerted on.
8 years ago
Bendik Hagen b7b40393dc Flow: Set flow iface and vlan_idx
Setting flow iface and vlan_idx from packet, making it possible to log
iface and vlan on psuedopackets and in flow-logs.
8 years ago
Bendik Hagen ec0dd0209a Flow: Adding livedev and vlan_idx on flow
Adding livedev and vlan_idx on flow, making it possible to use it for
logging in_iface on flow-logs and fix in_iface on psuedopackets.
8 years ago
Philippe Antoine bef190f767 http: logs content range
Fixes #2485
8 years ago
Philippe Antoine 43e205fc32 smtp: rset command resets bdat chunks length
Fixes #1860
8 years ago
Philippe Antoine ff52bb14b7 ssh : code style consistency
Adds SSH_FLAG_VERSION_PARSED to flags before each return
This way, we are sure SSHParseBanner does not get called again
And proto_version does not get leaked
8 years ago
Jeff Lucovsky 4f33b8c18d decode: Improved FTP active mode handling
This changeset addresses 2 issues:
- 2459
- 2527
and improves handling for FTP active mode over IPv4 and IPv6.

Active mode is triggered when the FTP client conveys the port
that should be used for a data connection (PORT, EPRT).

When this occurs, the FTP state is marked as "active".
8 years ago
Jason Ish 164fb71898 mpls: fix misaligned read
Instead of casting the packet buffer to a uint32, memcpy it to
avoid misaligned read error, as caught by the undefined behavior
detector (ubsan).

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2903
8 years ago
Victor Julien f7d8401c2e eve/smb: minor cleanup now Rust is mandatory 8 years ago
Victor Julien c2cb155ebb rust/smb: rename files and code from RustSMB to SMB 8 years ago
Victor Julien e572324c5a detect/dcerpc: cleanup now Rust is mandatory 8 years ago
Victor Julien 50709144f9 detect/app-layer-event: cleanup test 8 years ago
Victor Julien f30c05e684 smb: remove C implementation
Now that Rust is mandatory it is obsolete.

Ticket: #2849
8 years ago
Jason Ish 67b2692d34 dns: remove as much C DNS code as possible
As some of the C code is still used it can't all be removed.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2850
8 years ago
Jason Ish 78b82ce6a5 dns-log: remove, not supported now that Rust is required
The non-json line based DNS log is not supported with Rust only
builds and has been scheduled for removal in Suricata 5.0.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2297
8 years ago
Phil Young 05271bfbe5 napatech: simplify integration with Napatech cards
- There is now an option to automatically create streams on the
  correct NUMA node when using cpu affinity.

- When not using cpu affinity the user can specify streams to be
  created in the suricata.yaml file.  It is no longer required to
  use NTPL to create streams before running suricata.

- The legacy usage model of running NTPL to create streams is still
  available. This can be used for legacy configurations and complex
  configurations that cannot be satisfied by the auto-config option.
8 years ago
Victor Julien fd9f64d00f byte: suppress errors in byte extraction utils 8 years ago
Victor Julien a496c8be0c detect/bytejump: suppress runtime error messages 8 years ago
Victor Julien 5703ce371e detect/byteextract: suppress runtime error messages 8 years ago
jason taylor 7f63ec185a pfring: update PfringThreadVars_ for gcc 4.x
Signed-off-by: jason taylor <jtfas90@gmail.com>
8 years ago
Emmanuel Roullit 00917a0415 init: use pledge(2) after suricata initialization.
pledge(2) can be used on OpenBSD to restrict suricata possible
operation on the system once initialization is completed.
The process promises to only make use of:
- "stdio" to allow read(2) on IPS rules and write(2) on log file
- "rpath wpath cpath" to allow log rotation
- "unix" to operate the control unix socket and log unix sockets
- "dns" to retrieve DNS from recvfrom(2)/sento(2) in IPFW mode
- "bpf" as suricata uses libpcap, which uses the BIOCGSTATS operation

Signed-off-by: Emmanuel Roullit <emmanuel.roullit@cognitix.de>
8 years ago
Emmanuel Roullit 8b75e69165 log: output file mode in octal on chmod warning
The mode input in chmod is an octal integer. However when the warning is logged,
the file mode is printed in decimal which is confusing.

Signed-off-by: Emmanuel Roullit <emmanuel.roullit@cognitix.de>
8 years ago
Jeff Lucovsky 8e464530ef decode: Change return type of IPv4 and TCP options decode
The return value from the options decoder in TCP and IPv4 is ignored.
This commit changes the return type of the function to `void` and
modifies existing return points to return without a value.

When an error occurs, the packet state is being set to indicate whether
it's valid or not and the existing return value is never used.
8 years ago
Jeff Lucovsky 03b0e4272b parse: Improve unknown protocol parse message
The message associated with unknown protocols during parsing is incomplete.
This commit improves the message readability.
8 years ago
Jeff Lucovsky b10125af07 sticky: Convert ssh_software to new format 8 years ago
Jeff Lucovsky ceba8c89e9 sticky: Convert ssh_proto to new format
This changest converts the 'ssh_proto' sticky buffer
into the v2 framework.
8 years ago
Jeff Lucovsky 80be07a534 detect/http: Use v2 inspect and mpm engines
This changeset updates the http stub detect logic to use the v2 inspect
and npm engines.
8 years ago
Victor Julien aaf502d383 stream: suppress noisy debug info messages 8 years ago
Victor Julien 63569aeb57 pcap: suppress info messages 8 years ago
Victor Julien ee128c7460 detect/disable-detect: suppress info message
The message would be displayed even when level was higher than
info.
8 years ago
Victor Julien d00950be81 log/file: use default-log-dir for suricata.log
Default to just suricata.log instead of the full path, so that
in user mode we can log in the user mode location.
8 years ago
Victor Julien a36482e924 user mode: use CWD as logdir
Introduce util func for handling user mode settings.
8 years ago
Victor Julien c72dd84ade logopenfile: remove duplicate ifdefs 8 years ago
Victor Julien 56e4c348e1 runmodes: improve error messaging 8 years ago
Victor Julien 1a7e93f4a6 startup: log system mode with version 8 years ago
Victor Julien 86ed5815cb instance: set system vs user mode 8 years ago
Victor Julien 968e613037 list-keywords: don't load yaml
Avoids a useless warning if the file is not found.
8 years ago
Jeff Lucovsky 1b1fc9fee2 rule-analyzer: Ensure content counts are accurate
Fix for issue 2605.  Make sure that content is counted,
even if none of the specific content types are matched.
8 years ago
Giuseppe Longo 1c97423adf source-nflog: fix memleaks
This fixes two memleaks found with ASAN.

Direct leak of 96 byte(s) in 1 object(s) allocated from:
   #0 0x7f59cf4a4d28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28)
   #1 0xd7f92f in ReceiveNFLOGThreadInit /home/glongo/suricata/src/source-nflog.c:221
   #2 0xe9c8eb in TmThreadsSlotPktAcqLoop /home/glongo/suricata/src/tm-threads.c:293
   #3 0x7f59cd7aa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)

Indirect leak of 70000 byte(s) in 1 object(s) allocated from:
   #0 0x7f59cf4a4d28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28)
   #1 0xd814ea in ReceiveNFLOGThreadInit /home/glongo/suricata/src/source-nflog.c:324
   #2 0xe9c8eb in TmThreadsSlotPktAcqLoop /home/glongo/suricata/src/tm-threads.c:293
   #3 0x7f59cd7aa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)

SUMMARY: AddressSanitizer: 70096 byte(s) leaked in 2 allocation(s).
8 years ago
Emmanuel Roullit 4b4bb31c30 log: add NULL ptr guard on fclose when reopening.
Signed-off-by: Emmanuel Roullit <emmanuel.roullit@cognitix.de>
8 years ago
Victor Julien 884ca0cd3a detect/ftp: small ftpdata_command cleanups 8 years ago
Victor Julien 834d579f7e detect/uricontent: suggest http.uri as alternative 8 years ago
Victor Julien 6974eb3116 detect/dns: add dns.query for dns_query keyword
Improve error checking.

Part of #2283.
8 years ago
Victor Julien 822a434036 nfs: implement midstream reverse flow support
Register special midstream version of protocol detection that
can indicate the flow is the wrong direction based on the record
properties.
8 years ago
Victor Julien bb78d48c0a app-layer/probing-parser: implement reverse flow
Implement midstream support for the pure probing parsers. These
need to look up the appropriate parsers based on the reverse
tuple.
8 years ago
Victor Julien 34cb49f1e9 flow: add macro to get sp/dp
Account for possibility of a reversed flow.
8 years ago
Victor Julien 37893d8874 alert/unified2: remove useless packed attributes 8 years ago
Victor Julien 9dd200b5c9 decode: remove useless packed attributes 8 years ago
Victor Julien 948a4ccb3b detect/iponly: only run iponly address parsers for iponly rules 8 years ago
Victor Julien 92f08d85aa detect/iponly: improve negation handling in parsing 8 years ago
Victor Julien ba1de99f10 detect/address: clean up 'any' logic 8 years ago
Victor Julien 6a5084ccf9 detect/parse: set the type of signature early
This way we can know much sooner if the rule is ip-only
or not.
8 years ago
Victor Julien 449f169848 detect/parse: reduce scope of main parse func 8 years ago
Victor Julien ac2fb706d4 detect/content: test cleanup 8 years ago
Victor Julien 5626528c73 smb3: support direction check in midstream
As the records contain no indicator, fall back to checking the
flows port number.
8 years ago
Victor Julien 422e4892cc proto-detect: improve midstream support
When Suricata picks up a flow it assumes the first packet is
toserver. In a perfect world without packet loss and where all
sessions neatly start after Suricata itself started, this would be
true. However, in reality we have to account for packet loss and
Suricata starting to get packets for flows already active be for
Suricata is (re)started.

The protocol records on the wire would often be able to tell us more
though. For example in SMB1 and SMB2 records there is a flag that
indicates whether the record is a request or a response. This patch
is enabling the procotol detection engine to utilize this information
to 'reverse' the flow.

There are three ways in which this is supported in this patch:

1. patterns for detection are registered per direction. If the proto
   was not recognized in the traffic direction, and midstream is
   enabled, the pattern set for the opposing direction is also
   evaluated. If that matches, the flow is considered to be in the
   wrong direction and is reversed.

2. probing parsers now have a way to feed back their understanding
   of the flow direction. They are now passed the direction as
   Suricata sees the traffic when calling the probing parsers. The
   parser can then see if its own observation matches that, and
   pass back it's own view to the caller.

3. a new pattern + probing parser set up: probing parsers can now
   be registered with a pattern, so that when the pattern matches
   the probing parser is called as well. The probing parser can
   then provide the protocol detection engine with the direction
   of the traffic.

The process of reversing takes a multi step approach as well:

a. reverse the current packets direction
b. reverse most of the flows direction sensitive flags
c. tag the flow as 'reversed'. This is because the 5 tuple is
   *not* reversed, since it is immutable after the flows creation.

Most of the currently registered parsers benefit already:

- HTTP/SMTP/FTP/TLS patterns are registered per direction already
  so they will benefit from the pattern midstream logic in (1)
  above.

- the Rust based SMB parser uses a mix of pattern + probing parser
  as described in (3) above.

- the NFS detection is purely done by probing parser and is updated
  to consider the direction in that parser.

Other protocols, such as DNS, are still to do.

Ticket: #2572
8 years ago
Danny Browning c0ab45aa6f source-pcap-file: Pcap File Init Failure Handling (#1694)
Better handle case where pcap file receive thread fails to initialize. Allow
initialize to complete, but terminate the thread quickly. Delay exiting
unix socket runmode as late as possible.
8 years ago
Victor Julien 55ab112e77 app-layer/pd: minor code cleanup 8 years ago
Victor Julien 2cf0a34e66 app-layer/pd: set offset and depth in mpm 8 years ago
Victor Julien 7cea1e4181 mpm/hs: track maxdepth 8 years ago
Alexander Gozman 928fe1b859 Bug 2857: NFQ ASAN 'heap-use-after-free' error.
Global NFQ contexts were not freed properly causing
'use-after-free' error. Moving contexts cleanup to a
separate NFQContextsCleanup() and calling it from
GlobalsDestroy(), like it's done for AFPacket, solves
the problem.
8 years ago
Jeff Lucovsky 81c1af0887 detect: implement http {location,server} sticky buffer
This implements inspection of the Server and Location buffer as a
content sticky buffer.
8 years ago
Max Fillinger 081fdc6804 pcap-log: Don't leak memory in LZ4 error paths 8 years ago
Alexander Bluhm 0ea3fa92a8 Avoid use-after-free during pid file cleanup.
In case the pid file is given in the config file, the file name is
stored in volatile memory.  Removal of the pid file happens after
cleanup of config memory.  Create a copy of the name which will be
freed after the pid file has been removed.
8 years ago
Giuseppe Longo c63fea8d0a reputation: remove dead code
This removes some code commented long time ago.
8 years ago
Giuseppe Longo 6b2a5ef21e reputation: declare max value as constant 8 years ago
Victor Julien 2622006faa stream: minor test cleanups 8 years ago
Victor Julien 636fb8d387 mpm: track maxdepth
Track max depth setting per MpmCtx.

To make sure the data structure doesn't increase in size change global
bool to use a flags field.
8 years ago
Victor Julien 55e5d50496 detect/flow: optimize flow check
Flow direction doesn't need explicit checking as the rule groups (sgh)
are already per direction. So if a rule sets only flow:to_server or
flow:to_client, we can avoid adding a sigmatch to the signature.
8 years ago
Victor Julien 119db92c99 detect/tag: minor code cleanup 8 years ago
Victor Julien 6bf35a42f1 detect/mark: use postmatch instead of tag list
Keep the tag list for just tags. Post match list is better so the
keyword also works with pass and noalert rules.
8 years ago
Victor Julien 4dbf600d64 detect/mark: minor code cleanups 8 years ago
Victor Julien bd31e4e332 detect: packet match micro optimization 8 years ago
Victor Julien 7497c633c4 detect/filestore: use postmatch callback 8 years ago
Victor Julien 065c3379ed detect/replace: implement post-match
Implement replace executor as a post match callback so that it
is only considered if there actually is a replace keyword in use.
8 years ago
Victor Julien 00beeef031 eve/flow/netflow: log correct tulpe on reversed flows 8 years ago
Victor Julien 9c241fa32c stream: use flow/packet swap logic for SYN/ACK midstream 8 years ago
Victor Julien 2f1ace64c6 flow: on timeout, consider reverse flows for pseudo packets 8 years ago
Victor Julien 4d6e6b8270 flow: introduce flow swap support 8 years ago
Victor Julien 0225ab3828 common: introduce flags and var swapping macros 8 years ago
Victor Julien d307cfad05 packet: add 'swap' function to reverse direction 8 years ago
Victor Julien d5baac3f85 detect/flow: don't require flow for direction options
Flow isn't directly used for direction checks, so don't require
a flow to be present.
8 years ago
Victor Julien c1dcc26965 detect/parse: improve direction setting on sigs
Only set both directions if no direction has been explicitly set before.
8 years ago