Commit Graph

8832 Commits (c6cda99bcdeffce3fa775f0b2b414529e94bae11)

Author SHA1 Message Date
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
7 years ago
Jeff Lucovsky 488446cf37 logging/anomaly: Track event log progress
This changeset adds a mechanism to track when individual events
are logged. Transactions can be provided more than once; track
events to prevent event re-logging.
7 years ago
Jeff Lucovsky 4a39d7a1ee app-layer: Extend event container with progress 7 years ago
Jeff Lucovsky 07c05f7dd8 logging/anomaly: TX based logging addition 7 years ago
Jeff Lucovsky 36644907f5 app-layer/logging Add TX packet logging support
Add transactional logging support for packet based loggers, like
the anomaly logger.
7 years ago
Jeff Lucovsky f7b934f83f app-layer/logging: protocol parser updates 7 years ago
Jeff Lucovsky 49438569a2 logging: anomaly log updates 7 years ago
Jeff Lucovsky 50e23ba93a app-layer: Initial app layer logging 7 years ago
Victor Julien 4e8d38348f flow: no bypass manager for Windows 7 years ago
Victor Julien 287b87b81b bypass: switch to gettimeofday 7 years ago
Victor Julien d8614a15c6 mingw: fix compile error
Declare _POSIX_C_SOURCE before sys/time.h to avoid:

util-time.c: In function 'SCUtcTime':
util-time.c:222:12: error: implicit declaration of function 'gmtime_r'; did you mean 'gmtime_s'? [-Werror=implicit-function-declaration]
  222 |     return gmtime_r(&timep, result);
      |            ^~~~~~~~
      |            gmtime_s
util-time.c:222:12: warning: returning 'int' from a function with return type 'struct tm *' makes pointer from integer without a cast [-Wint-conversion]
  222 |     return gmtime_r(&timep, result);
      |            ^~~~~~~~~~~~~~~~~~~~~~~~
util-time.c: In function 'SCLocalTime':
util-time.c:305:9: error: implicit declaration of function 'localtime_r'; did you mean 'localtime_s'? [-Werror=implicit-function-declaration]
  305 |         localtime_r(&timep, &cached_local_tm[lru]);
      |         ^~~~~~~~~~~
      |         localtime_s
util-time.c:321:56: warning: comparison between pointer and integer
  321 |         if (localtime_r(&timep, &cached_local_tm[lru]) == NULL)
      |                                                        ^~
cc1.exe: some warnings being treated as errors

Tickets: #2994 #3051
7 years ago
Eric Leblond 6126f105ea util-ebpf: fix creation of flow from pinned maps 7 years ago
Eric Leblond 7df3007066 af-packet: xdp pinned maps need to be read 7 years ago
Eric Leblond 4ba02830b6 flow-bypass: registration of non periodic check
This patch adds the capability to register a set of functions
without providing a periodic check function. This permit to
run a task only at init.
7 years ago
Eric Leblond 3e8fd26ee3 flow-hash: real hash computation for FlowKey 7 years ago
Eric Leblond 0963fea390 util-ebpf: log bypassed flow maps count 7 years ago
Eric Leblond 7e5439bc15 bypass: add counter for local captured bypass
Packets from captured bypassed flows that are received by Suricata
before the capture method start to bypass them can represent an
important part due to various buffer and insertion delay.

This patch adds a two counters to know the number of packets and
bytes in this case.
7 years ago
Eric Leblond de52e78e65 bypass: flow bypass is not ebpf only 7 years ago
Eric Leblond c938dbde27 util-ebpf: early exit if no map 7 years ago
Eric Leblond e5d325a6ca util-ebpf: reindex 7 years ago
Eric Leblond aa989a7445 af-packet: some conditional fields 7 years ago
Eric Leblond abbb066a96 bypass: bytes and pkts counters for local bypass 7 years ago
Eric Leblond 9206b30fe1 af-packet: better accounting and error handling
This patch improves the bypass error handling add adds more counters
to the interface so it is possible to get a view on success and
failure of insertion in the eBPF maps via the `iface-bypassed-stat`
command.
7 years ago
Eric Leblond aeb2bd3aa1 util-ebpf: optimization on flow storage queries 7 years ago
Eric Leblond 288f335aa5 util-ebpf: simplify free function
First key can't be null.
7 years ago
Eric Leblond f4abe2f9c0 util-ebpf: set livedev in flow
This will fix the accounting for pinned maps as the livedev field
of Flow is used to do the accounting of bypass flows.
7 years ago
Eric Leblond 98e7d9d1c0 util-device: introduce bypassed stats sub function 7 years ago
Eric Leblond f29a4b8bee flow-manager: move bypass timeout to a inline func 7 years ago
Eric Leblond 89e8cb50ed util-ebpf: case is not possible so remove warning 7 years ago
Eric Leblond 8c97998cb9 bypass: implement iface-bypassed-stat for callback 7 years ago
Eric Leblond 51ab06256a bypass: account callback method in stats 7 years ago
Eric Leblond f78e5ba1e1 bypass: restore interface counter 7 years ago
Eric Leblond d119845d98 bypass: compress flow keys structure 7 years ago
Eric Leblond 69d2c8eb75 ebpf: get rid of hash in map value 7 years ago
Eric Leblond b07bda7a7b bypass: new callback stragegy
This patch introduces and uses a new bypass strategy
based on a callback. EBPF bypass implementation is
updated to use this new strategy.

Once the flow manager detect that a flow should be timeouted,
it asks the capture method if it has seen packets in the interval.
If it is the case the lastts of the flow is updated and the timeout
is postponed.
7 years ago
Eric Leblond fcae1c18de af-packet: improve error handling for some hw
Some cards seems to return EAGAIN when there is no more place in
the hash table.
7 years ago
Eric Leblond 0bfbb4a889 bypass: fix accounting
The flow bypass stats is computed at every pass so the accounting
needs to be done at each pass. This patch fixes the accounting
in the flow_bypassed counters.
7 years ago
Eric Leblond 44566e5a24 ebpf: only display that file is loaded if we do it 7 years ago
Eric Leblond 5e62ae6d28 af-packet: avoid error flooding when bypass fails 7 years ago
Eric Leblond af6daceeda util-ebpf: more useful error message
At the time of the writing, libbpf output useful error message
on strdout only and errno is not really interesting. So let's
tell user to look at stdout.
7 years ago
Eric Leblond 0f64c25b73 util-ebpf: improve code readability
As pointed by Victor Julien, the pkts_cnt usage was quite confusing
so functions are now returning a bool.
7 years ago
Eric Leblond a8f35cc30e util-ebpf: discard flow if no Flow storage 7 years ago
Eric Leblond efb648aa24 util-ebpf: fix ebpf bypass
Fix endian order in eBPF bypass. It has to be updated after the
bypassed flows handling change.
7 years ago
Eric Leblond f8aa9ee986 bypass: fix wait time at exit
The loop on bypassed flow maps can take a few seconds on heavily
loaded system causing Suricata to not honor a stop before a few
seconds.

This patch adds the code needed to detect the need to exit from
the check loop.
7 years ago
Eric Leblond a277f2eb0c af-packet: fix use after free on config
ASAN did find that afp config was used after free. This was in
fact done in the Flow bypass manager hence this patch.
7 years ago
Eric Leblond fc2f2fa7d3 bypass: allow bypass for packet without flow
For capture method that have their own flow structure (not maintained
by Suricata), it can make sense to bypass a packet even if there is
no Flow in Suricata.

For AF_PACKET it does not make sense as the eBPF map entry will
be destroyed as soon as it will be checked by the flow bypass
manager. Thus we shortcut the bypass function if ever no Flow is
attached to the packet.

This path also removes reference to Flow in the bypass functions
for AF_PACKET. It was not necessary and we possibly could benefit
of it if ever we change the bypass algorithm.
7 years ago
Eric Leblond 6ab1cbcb8e bypass: use flow storage for bypass counter
There is a synchronization issue occuring when a flow is
added to the eBPF bypass maps. The flow can have packets
in the ring buffer that have already passed the eBPF stage.
By consequences, they are not accounted in the eBPF counter
but are accounted by Suricata flow engine.

This was causing counters to be completely wrong. This code
fixes the issue by avoiding the counter change in invalid
case.

To avoid adding 4 64bits integers to the Flow structure for the
bypass accounting, we use instead a FlowStorage. This limits the
memory usage to the size of a pointer.
7 years ago
Eric Leblond 4e6add7faa bypass: generalize iface bypass stats
Introduce functions in util-device.c to be able to manage the
flow bypassed count stats.
7 years ago
Eric Leblond 258e90be76 util-ebpf: change flow accounting logic
Update the flow counters during the life of a bypassed flow
instead of just accounting at the end of it.
7 years ago
Eric Leblond 3026e9a80d util-ebpf: better error handling 7 years ago
Eric Leblond 2ffd3ad2b7 util-ebpf: better error handling of map unlink 7 years ago
Eric Leblond b952b32a26 util-ebpf: rename field 'unlink' to avoid confusion 7 years ago
Eric Leblond edf2db4e30 af-packet: improve warning message 7 years ago
Eric Leblond 567b5ee1bc af-packet: rename option 'no-percpu-hash' 7 years ago
Eric Leblond 0f6b1297a9 af-packet: warn when XDP is not supported 7 years ago
Eric Leblond b1c9b39435 af-packet: remove question from code 7 years ago
Eric Leblond 1992093c88 flow-bypass: rename variables 7 years ago
Eric Leblond d239e0f2d5 flow-hash: doc and code cleaning 7 years ago
Eric Leblond b736344975 flow-bypass: clock_gettime error handling
Only reason clock_gettime could fail is a permission so let's
error and leave the flow bypass manager if it is the case.

Also let's suppress the error message if ever the error appear in
the middle of a run (which is unlikely).
7 years ago
Eric Leblond 4129938c21 util-ebpf: log level fixes and code cleaning 7 years ago
Eric Leblond 140269a6be util-ebpf: init code optimization 7 years ago
Eric Leblond c5e2af0545 util-ebpf: fix error reported by coccinelle check
Some allocation errors were not checked during init phase.
7 years ago
Eric Leblond c1fd0da550 af-packet: add vlan_id in bypass key
Bypassing on vlan was not supported due to the missing key.
7 years ago
Eric Leblond 651a27e4fb ebpf: fix percpu hash handling
An alignement issue was preventing the code to work properly.
We introduce macros taken from Linux source code sample to get
something that should work on the long term.
7 years ago
Eric Leblond 142c69e1ef flow-bypass: increase bypass timeout
This is needed as we did switch from counter maintained in kernel
to internal polling so we need a bigger value.
7 years ago
Eric Leblond b8e184ceb5 flow-bypass: fix timeout of maps bypassed flows
The time is taken from the parameter and is checked against real
flow entries so we need a standard time.
7 years ago
Eric Leblond 07d0bd3a0f util-ebpf: fix IPv6 deletion loop 7 years ago
Eric Leblond b481f290e2 af-packet: fix bypass for IPv6 7 years ago
Eric Leblond 3bd8ba5d00 util-ebpf: add message if key deletion fails 7 years ago
Eric Leblond 269f601f8a util-ebpf: can't delete in place so update algo 7 years ago
Eric Leblond 5b056c15bf af-packet: fix default in pinned maps name 7 years ago
Eric Leblond eff56acca5 af-packet: be sure to nullify option if not set 7 years ago
Eric Leblond 36c6a62954 util-ebpf: simplify function declarations 7 years ago
Eric Leblond 69630d7a17 util-ebpf: micro optimization 7 years ago
Eric Leblond d21c3a6555 util-ebpf: create flow from bypassed flows 7 years ago
Eric Leblond 04c65a309e flow-hash: new function to get flow from flowkey 7 years ago
Eric Leblond 885fc992de ebpf: make table iterator generic
Also adds a basic skeleton for flow creation loop.
7 years ago
Eric Leblond 880c42f11c af-packet: bypass with init function 7 years ago
Eric Leblond 522e98d830 util-ebpf: fix iteration in flow timeout
We were not setting the key using the correct item in map. Result
was deletion of wrong flow.
7 years ago
Eric Leblond 0c3e1e8579 af-packet: correctly set up hardware offload 7 years ago
Eric Leblond bd28f77a1e af-packet: fix loading of ebpf filter 7 years ago
Eric Leblond 6fdcb127e9 af-packet: fix the start when XDP is pinned 7 years ago
Eric Leblond 7e0ef4cec8 util-ebpf: change return of pinned maps loading
The calling function needs to be able to see when this is a success
and XDP do not need to be reloaded.
7 years ago
Eric Leblond d950a9f272 util-ebpf: conditional flow table loading 7 years ago
Eric Leblond d25e8dbfc7 af-packet: implement pinned-maps-name 7 years ago
Eric Leblond 36838017fe af-packet: fix build when eBPF not built-in 7 years ago
Eric Leblond b1769d5f8f util-ebpf: implement pinned maps loading
Load flow tables at start if asked to.
7 years ago
Eric Leblond 96f1454ebf util-ebpf: only unlink pinned maps in eBPF filter 7 years ago
Eric Leblond 4cf531008e af-packet: conditionaly remove XDP filter
Only remove the XDP filter if we are in XDP mode and not using the
pinned maps.
7 years ago
Eric Leblond 4f48c45727 util-ebpf: conditional pinning of maps
Only pin maps if `pinned-maps` is set in the configuration. This
ensure backward compatibility.
7 years ago
Eric Leblond 73a928fa0b util-ebpf: fix loop on maps
We were missing the last element of the map by working on previous
key instead of current key.
7 years ago
Eric Leblond 3ce69c446b util-ebpf: suppress spaces at end of line 7 years ago
Eric Leblond 954762a429 ebpf: add filter by maps on example filter 7 years ago
Eric Leblond 5d8ac36a49 util-ebpf: pin the maps
By pinning the maps we are creating a file in /sys/fs/bpf that can
be used by external program to access the map. This has multiple
benefits such as handling list from an external program.

The pinned maps could be persistent accross Suricata reload but
this can be complicated in term of handling everything in the life
of Suricata.
7 years ago
Eric Leblond bf4381b17b ebpf: document XDP iface redirect 7 years ago
Eric Leblond a37a7c4a21 ebpf: reduce counter size to allow netronome offload 7 years ago
Eric Leblond 315c29a8e6 ebpf: change the logic to avoid ktime usage
Kernel time is not available (and/or costly) on NIC such as
Netronome so we update the logic to detect dead flows based on a
lack of update of packets counters. This way, the XDP filter will
be usable by network card.

This patch also updates the ebpf code to support per CPU and
regular mapping. Netronome is not supporting it and the structure
is using atomic for counter so the cost of simultaneous update
is really low.

This patch also updates the xdp_filter to be able to select if the
flow table is per CPU on shared. Second option will be used for
hardward offload. To deactivate the per cpu hash, you need to set
USE_PERCPU_HASH to 0.

This patch also adds an new option to af-packet named no-percpu-hash
If this option is set to yes then the Flow bypassed manager thread
will use one CPU instead of the number of cores. By doing that
we are able to handle the case where USE_PERCPU_HASH is unset (so
hardware offload for Netronome).

This patch also remove aligment indications in the eBPF filter. This
was not really needed and it seems it is causing problem with
some recent version of LLVM toolchain.
7 years ago
Eric Leblond 1745408c13 flow-manager: no force reassembly on bypassed flow
When a bypassed flow is created we are forcing the reassembly so
we don't need to do it again when it timeout.
7 years ago
Shivani Bhardwaj 7c7e04487f detect/content: Message for escaping backslash
So far, if the rule loaded had a backslash character ("\") in its
content field, the rule will fail to load but without giving a
descriptive error message. This patch tells the user to escape the
troubling character.

Before
```
9409] 7/6/2019 -- 16:12:22 - (detect-engine-loader.c:184) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Backslash needs escaping msg"; flow:established,to_server; content:"MyBackslash\here"; sid:86; rev:1; )" from file /var/lib/suricata/rules/myrule.rules at line 1
```

After
```
[9409] 7/6/2019 -- 16:12:22 - (detect-content.c:155) <Error> (DetectContentDataParse) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - '\' has to be escaped
[9409] 7/6/2019 -- 16:12:22 - (detect-engine-loader.c:184) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Backslash needs escaping msg"; flow:established,to_server; content:"MyBackslash\here"; sid:86; rev:1; )" from file /var/lib/suricata/rules/myrule.rules at line 1
```

Closes redmine ticket #2626
7 years ago
Victor Julien 7513f6cae2 threads: minor code cleanups 7 years ago
Victor Julien 92d38683ce threads: improve flow timeout loop
Improve thread safety and remove BUG_ON
7 years ago
Victor Julien e6b633cf95 stats: walk tv_root under lock 7 years ago
Victor Julien 9474b84683 stats: more accurate time handling for wakeup thread 7 years ago
Victor Julien ab1268cfea stats: minor code cleanups 7 years ago
Philippe Antoine b5f3e03209 New app layer event for invalid http request line
Handles logs from libhtp even if case of error
7 years ago
Philippe Antoine 4a74e9edcc signature: error for rules with illegal port
Fixes #2080
7 years ago
Philippe Antoine 248f9c66cf warnings: fixes integer sizes in format strings
Fixes #3009
7 years ago
Philippe Antoine a3e38861b9 http: adds unit tests for HTPParseContentRange 7 years ago
Philippe Antoine 90ab0b0ec2 http: fix overflow in HTPParseContentRange 7 years ago
Victor Julien 39cf5b0c8b snmp: fix 'defined but not used' warning
Remove HAVE_RUST guards as well.
8 years ago
Pierre Chifflier 8c61cfdca5 snmp.version: move unittests to tests/ 8 years ago
Pierre Chifflier 58d85b1460 snmp.pdu_type: move unittests to tests/ 8 years ago
Pierre Chifflier ff0ff6344f snmp.community: move unittests to tests/ 8 years ago
Pierre Chifflier 509a54281f detect/snmp: convert snmp.community keyword to v2, and MPM 8 years ago
Pierre Chifflier c67a53773b detect/snmp: rename keywords to snmp.<name>, and make rust mandatory 8 years ago
Pierre Chifflier 9dfec7e734 SNMP: add the "snmp.pdu_type" detection keyword 8 years ago
Pierre Chifflier e1dd19a0eb SNMP: add the "snmp.community" detection keyword 8 years ago
Pierre Chifflier aa608e0ca2 SNMP: add the "snmp.version" detection keyword 8 years ago
Pierre Chifflier ed4823bbe8 SNMP: add missing case for profiling enum 8 years ago
Pierre Chifflier 6fc7fc74cb SNMP: add logger 8 years ago
Pierre Chifflier 2df840a8b8 Add SNMP (v1/v2c/v3) application layer 8 years ago
Victor Julien cdd061ba37 ikev2: clean up c glue code 8 years ago
Victor Julien 3f36a6ce59 tftp: c glue code cleanup 8 years ago
Victor Julien ade2b1e6cf output: get rid of BUG_ONs 8 years ago
Victor Julien 9ea6db3889 pool: don't call Cleanup after failed Init
Stream reassembly memcap is regulated by the Init and Cleanup
callbacks. If Init fails due to memcap reached, Cleanup had no
way of knowing and it would decrease the memcap even if it hadn't
been increased by Init. This could lead to too much memory use and
memcap counter underflow.

This patch fixes the issue by not calling Cleanup in this case. It's
fair to see a failed Init the responsibility of Init.
8 years ago
Victor Julien 597ff3c8e3 pool/thread: clean up tests 8 years ago
Victor Julien 025cc64f55 pool/thread: remove old grow function 8 years ago
Victor Julien 67ba132c10 stream: simplify segment and ssn pool inits 8 years ago
Victor Julien 0b02efdf01 pool/thread: introduce simpler way to grow thread pool 8 years ago
Victor Julien 6e6c1bad7d pool/thread: minor code cleanups 8 years ago
Victor Julien 924982e792 app-layer: minor cleanups 8 years ago
Victor Julien ce71bf1fff capture: check for flow packets on capture timeout
The capture threads can receive packets from the flow manager in their
Threadvars::stream_pq packet queue. This mechanism makes sure the packets
the flow manager injects into the engine are processed by the correct
worker thread.

If the capture thread(s) would not receive packets for a long time, the
Threadvars::stream_pq would not be checked and processed. This could
lead to packet pool depletion in the flow manager. It would also lead
to flows not being timed out/logged until either packets started flowing
again or until the engine was shut down.

The scenario is more likely to happen in a test (e.g. replay) but could
also delay logging on low traffic sensors.
8 years ago
Victor Julien 952cbb563c app-layer: mandatory tx registration checks
All protocols now implement the TX API, so the runtime checks for
whether or not a protocol supports the TX API can be removed.
8 years ago
Victor Julien 8bc1e120ea app-layer: make tests tx aware
Make tests minimally TX-aware so we can asume all parsers use
the tx API.
8 years ago
Victor Julien 3526569b25 ssh: test cleanup
Remove explicit locks. Set ALPROTO_SSH in all tests.
8 years ago
Victor Julien 480f497d05 packetpool: move return stack to pool earlier
If waiting for N packets move the return stack to the main
stack every time we take the return stack lock.

Make sure we consider enough packets when setting the pending pool
flush logic. This should at least make sure to have the 9 packets
the flow manager requires per run.
8 years ago
jason taylor fa0008dbbe detect/content: clean up logging output
Signed-off-by: jason taylor <jtfas90@gmail.com>
8 years ago
Jeff Lucovsky 0960ca0d00 detect/analyzer Add missing HTTP values
This changeset adds recognition of missing HTTP values
- Raw host
- Header names
- Server body
- User agent
8 years ago
Philippe Antoine a370d2a91d signature: fix memory leak in DetectBytejumpSetup 8 years ago
Philippe Antoine 8a339e73d3 http: adds an event for double encoded uri 8 years ago
Philippe Antoine e5879650bb http: fixes double-encode configuration keywords 8 years ago
Mats Klepsland f634da61c9 ja3s-string: move unittests to tests/ 8 years ago
Mats Klepsland 736d35f36e ja3s-hash: move unittests to tests/ 8 years ago
Mats Klepsland 0b489f329c detect: add (mpm) keyword ja3s.string
Match on JA3S string using ja3s.string keyword, e.g:

  alert tls any any -> any any (msg:"ja3s.string test";
      ja3s.string; content:"10-11-12"; sid:1;)
8 years ago
Mats Klepsland 80cee50916 detect: add (mpm) keyword ja3s.hash
Match on JA3S hash using ja3s.hash keyword, e.g:

  alert tls any any -> any any (msg:"ja3s.hash test";
      ja3s.hash; content:"b26c652e0a402a24b5ca2a660e84f9d5"; sid:1;)
8 years ago
Mats Klepsland 03e8e658d7 lua: add Ja3SGetString function
Add Ja3SGetString() to return the content of the JA3S string buffer from
the TLS session.

Example:

  function init (args)
      local needs = {}
      needs["protocol"] = "tls"
      return needs
  end

  function setup (args)
      filename = SCLogPath() .. "/ja3s_string.log"
      file = assert(io.open(filename, "a"))
  end

  function log (args)
      ja3s_string = Ja3SGetString()
      if ja3s_string == nil then
          return
      end

      file:write(ja3s_string .. "\n")
      file:flush()
  end

  function deinit (args)
      file:close()
  end
8 years ago
Mats Klepsland b1d5fe9657 lua: add Ja3SGetHash function
Add Ja3SGetHash() to return the content of the JA3S hash buffer from
the TLS session.

Example:

  function init (args)
      local needs = {}
      needs["protocol"] = "tls"
      return needs
  end

  function setup (args)
      filename = SCLogPath() .. "/ja3s_hash.log"
      file = assert(io.open(filename, "a"))
  end

  function log (args)
      ja3s_hash = Ja3SGetHash()
      if ja3s_hash == nil then
          return
      end

      file:write(ja3s_hash .. "\n")
      file:flush()
  end

  function deinit (args)
      file:close()
  end

In the example above, each JA3S hash is logged to a log file.
8 years ago
Mats Klepsland a4eaef25d6 eve: add JA3S field to TLS JSON logger
Add JA3S object to TLS JSON logger (extended log).
8 years ago
Mats Klepsland a4471987ba app-layer-ssl: generate JA3S fingerprints
Generate JA3S fingerprints based on fields in the ServerHello record.
8 years ago
Alexander Gozman 3a16009966 Bug #2965: fix NFQ arguments parsing 8 years ago
Andreas Herz 8baf64f5e9 af-packet: fix setting block_timeout value through afpconfig 8 years ago
Jeff Lucovsky 462a4e2b5b detect/analyzer: Improve warning message
This changeset modifies the warning printed when a rule
is determined to detect in both directions.
8 years ago
Eric Leblond 5d76f0897c af-packet: remove rollover reference
This patch removes reference to rollover in the configuration file
and add warnings when it is used.
8 years ago
Philippe Antoine 037d50ef06 signature: fix overflow in parsing 8 years ago
Philippe Antoine 3e12066819 http: adds events for each libhtp log
Fixes #997
8 years ago
Mats Klepsland 3c57ac144c detect-ssl-version: move unittests to tests/ 8 years ago
Mats Klepsland 238797cc66 detect-ssl-state: move unittests to tests/ 8 years ago
Mats Klepsland 479e73b98e detect-tls-version: move unittests to tests/ 8 years ago
Mats Klepsland 767bde5e74 detect-tls-cert-validity: move unittests to tests/ 8 years ago
Mats Klepsland a260a57b68 detect-tls-sni: move unittests to tests/ 8 years ago
Mats Klepsland adb4da3975 detect-tls-ja3-string: move unittests to tests/ 8 years ago
Mats Klepsland 74a7b7e3cf detect-tls-ja3-hash: move unittests to tests/ 8 years ago
Mats Klepsland 5d3b94b3e4 detect-tls-cert-subject: move unittests to tests/ 8 years ago
Mats Klepsland 0d728ee4c6 detect-tls-cert-serial: move unittests to tests/ 8 years ago
Mats Klepsland e125e58c97 detect-tls-cert-issuer: move unittests to tests/ 8 years ago
Mats Klepsland 3646234ac5 detect-tls-cert-fingerprint: move unittests to tests/ 8 years ago
Mats Klepsland 12d37b8b2c detect-tls: tidy up unittests
By doing the following:
- removing unnecessary locks
- moving variable declarations
- removing redundant function 'SigCleanSignatures'
8 years ago
Mats Klepsland 15012fc908 ja3: check if JA3 is disabled on one line 8 years ago
Mats Klepsland 285855d928 detect-tls: remove NULL settings from keyword registration 8 years ago
Mats Klepsland 008f08c1b3 detect-tls: declare ssl_state as const in GetData() 8 years ago
Mats Klepsland 0f7f35bd85 detect-tls: check return values of functions on setup
Check the return values of DetectBufferSetActiveList() and
DetectSignatureSetAppProto().
8 years ago
Mats Klepsland 1c04d7cdae detect-tls: remove confusing underscores from variables
Remove confusing underscore prefix from variables in GetData() for
all tls keywords.
8 years ago
Jeff Lucovsky de983fb7c9 app-layer-ftp: Potential memory leak fixed
Ensure that when handling failures during STOR command
processing, that all memory is freed on the error path.
8 years ago
Mats Klepsland ba857e9739 detect: add tls.certs keyword
Add keyword to do "raw" matching on each of the certificates in the
TLS certificate sticky buffer.

Example:
  alert tls any any -> any any (msg:"tls.certs test"; tls.certs; \
          content:"|01 02 03 04|"; sid:1;)
8 years ago
Victor Julien edae50de94 detect/ssh: fix ssh.protoversion memory leak 8 years ago
Victor Julien 567a7c3cef detect/ssh: mark old ssh keywords as deprecated 8 years ago
Victor Julien d623dc4ac0 detect/parse: add flag to indicate keyword is deprecated
Issue warning when it is still used.
8 years ago
Victor Julien b84eba80aa detect/nfs.version: minor cleanups 8 years ago
Victor Julien 2ea11da230 detect/nfs: add nfs.version 8 years ago
Victor Julien 3299f007f8 detect/dcerpc: add dcerpc.iface
Keep dce_iface as an alias.
8 years ago
Victor Julien cdff1d50b7 detect/dcerpc.opnum: minor code cleanups 8 years ago
Victor Julien 6840e5c7df detect/dcerpc: add dcerpc.opnum as new name for dce_opnum 8 years ago
Philippe Antoine b6b7778e2d http: adds event for header repetition 8 years ago
Jason Ish 9d8eb7b5f0 filestore: remove jansson ifdefs
Jansson is now required.
8 years ago
Jason Ish 3dc973d4b1 eve/file: remove rust and jansson ifdefs.
Both Rust and Jansson are required now.
8 years ago
Jason Ish 42c327adc4 filestore: fix leak in contructing json
Use json_array_append_new instead of json_array_append to transfer
ownership of the integer object to jansson so it gets freed.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2961
8 years ago
Victor Julien ddfcf76c57 detect/engine: make DetectAppLayerMpmRegister decprecated 8 years ago
Victor Julien 752bb1c410 detect/dnp3: add dnp3.data with v2 api support
Adds MPM support as well. Add TxDetectFlags support to the parser
to avoid duplicate matches.
8 years ago
magenbluten 09a21627d5 filestore: fix dropping of unwanted files (Issue #2853) 8 years ago
Victor Julien 9132e4032a files: open files with track id only 8 years ago
Victor Julien 3b31bad855 detect/dce_stub_data: add dcerpc.stub_data
Also use v2 API for inspect and mpm registration.
8 years ago
Victor Julien d270a7603a detect/inspect: add flags to inspect buffer 8 years ago
Victor Julien 32fb7d773a detect/content-inspect: turn void arg into Packet
Replace the 'void *data' argument by a 'Packet *p' as this was
the only user left of the data pointer.
8 years ago
Victor Julien b7a7517273 detect/dce_stub_data: minor cleanups 8 years ago
Victor Julien 55db6d6fb4 detect/dcerpc: move endian handling from pointer to flags 8 years ago
Victor Julien b2638f7195 detect/krb5: add krb5.sname and krb5.cname 8 years ago
Victor Julien aefce4d761 detect/nfs: remove HAVE_RUST guards 8 years ago
Victor Julien 15eac12a39 afl: fix compilation 8 years ago
Victor Julien 3ae2edb22a ftp: fix realloc handling to avoid valgrind warning
Bug #2951
8 years ago
Victor Julien 84881bf1b8 detect/file.magic: add sticky buffer
Add sticky buffer to inspect file magic. Includes mpm support.
8 years ago
Victor Julien d78c6ff714 detect/thread: ctx info is allowed to have NULL data 8 years ago
Victor Julien aa52dfab04 detect/smb: clean up keywords 8 years ago
Victor Julien d64fbb71ae detect/file: add file.data, small cleanups 8 years ago
Victor Julien b5d5389438 detect/ssh: minor --list-keywords improvements 8 years ago
Victor Julien f246e319b2 detect/http.header.raw: minor cleanups 8 years ago
Victor Julien a21a7d16bd detect/http.host.raw: minor cleanups 8 years ago
Victor Julien 0e1d47c87b detect/http.method: minor cleanups 8 years ago
Victor Julien bdd8e6152b detect/http.start: modernize name and code 8 years ago
Victor Julien cd2e6511c9 detect/http: cleanup http stat * 8 years ago
Victor Julien 84da0376fb detect/http.host: rename file for consistency 8 years ago
Victor Julien 2b8311beff detect/http.host: fix --list-keywords output 8 years ago
Victor Julien 0e5c987533 detect/http.uri: fix up --list-keywords output 8 years ago
Victor Julien 19163ca2e1 detect/http: request/response line keyword modernization 8 years ago
Victor Julien fb2e4e4453 detect/http.header_names: use v2 api and new name 8 years ago
Philippe Antoine 316a411b6b ssl : SSLProbingParser overflow fix
Found by fuzzing
Fixes ssl detection evasion by packet splitting
8 years ago
Victor Julien 666bb1b6e4 parse/ip: fix potential oob write in ipv4 validation
Found using AFL.
8 years ago
Jason Ish b8ce7f2885 mpls: check buffer length before peeking at next header
Check that we have enough bytes before peaking into the MPLS
packet payload.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2884
8 years ago
Jason Ish 8d7d6a96a5 ethernet: fix next packet size on DCE packet
Missing parans on the DCE length caused the length update
for the next call to DecodeEthernet to be wrong.

Tests added.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2887
8 years ago
Victor Julien 76cc03010a ssh: fix banner overflow issue
Reported-by: Sirko Höer - Code Intelligence
8 years ago
Victor Julien 2b75222250 runmodes: for test runmodes, clean up properly
For conf test and engine analysis, clean up memory correctly.

This helps valgrind tests for leaks.
8 years ago
Jeff Lucovsky 74f436d209 logging: display base64 decoded string for packet
This changeset changes the packet display to be base64, rather than hex.
8 years ago
Jeff Lucovsky 7d28c19f05 logging: Ensure all anomalous events have an event_type
This change ensures that each anomaly is tagged with an
event type to support querying.

Each anomalous event will include `"event_type": "anomaly"`
in the log record.
8 years ago
Jeff Lucovsky 5e222129d5 eve/alert: Remove unused results from PrintRawLineHexBuf
This changeset removes the call to `PrintRawLineHexBuf`. The
return values were never used.
8 years ago
Jeff Lucovsky a8938f449d logging: Anomaly logging
This changeset adds anomaly logging to suricata for issue 2282.

Anomaly logging is controlled via the `anomaly` section within eve-log.
There is a single option -- `packethdr` -- for including the packet header
in the anomaly.
8 years ago
Philippe Antoine a1c6e091ac http: new event for auth unrecognized
activates libhtp auth parsing
Fixes #984
8 years ago
Jeff Lucovsky 7f102d95b6 detect: Modernize TLS keywords
This changeset adds keywords for "tls.<name>" and moves the existing
value of "tls_<name>" to an alias.
8 years ago
Alexander Bluhm 36796de731 init: pledge(2) needs "fattr" during suricata reload.
When killed with SIGHUP, suricata reopens the log files.  If filemode
is set in the config, it needs pledge promise "fattr" to allow the
chmod(2) on OpenBSD.
8 years ago
Giuseppe Longo af9399f2ac detect-http-protocol: use v2 inspect/mpm engines
This updates inspect/mpm engines to v2.
8 years ago
Victor Julien a6a0b0aa4a detect/files: fix file sigs state handling
Make sure all file sig mismatches indicate this in their return
code, not just the ones with filestore enabled. This is needed
to tell the stateful detect engine that it is dealing with a file
sig, so it can make sure these are inspected correctly even if
there are possibly multiple files per tx.
8 years ago
Victor Julien 225cdf996e eve/alert: take vlan from packet, not flow
Flow is not guaranteed to exist.
8 years ago
Eric Leblond 497f35164b detect-filename: avoid multiple inspections of buf
If the filename inspection function is returning nomatch this will
trigger iterative inspections with same content (aka filename) being
inspected. To avoid this we change the return as the buffer inspection
has not to be inspected anymore.
8 years ago
Eric Leblond 63f2032ed6 doc: fix way to build URL 8 years ago
Eric Leblond f48ac1860a detect-lua: implement sticky buffer
This patch implement an option named 'buffer' that can be used in the
init function of a lua signature:

 function init (args)
     local needs = {}
     needs["buffer"] = tostring(true)
     return needs
 end

With this, the lua script will get access to the sticky buffer
content.
8 years ago
Eric Leblond 62a11dd3ed detect-lua: fix DNP3 value 8 years ago
Mark Janssen 600f2ab391 eve/json: always output vlan field as array 8 years ago
Mark Janssen 0cc3c2cc6c eve/flow: add in_iface field
Fixes #2057
8 years ago
Mark Janssen fed9b7a180 eve/flow: add vlan field 8 years ago
Bendik Hagen f558ef2c55 Flow/Stream: set psuedopacket iface/vlan from flow
This fixes redmine bug #2057 by setting pseudopacket iface and vlan from
flow values, solving the problem of missing vlan/iface when psuedopacket
gets logged/alerted on.
8 years ago
Bendik Hagen b7b40393dc Flow: Set flow iface and vlan_idx
Setting flow iface and vlan_idx from packet, making it possible to log
iface and vlan on psuedopackets and in flow-logs.
8 years ago
Bendik Hagen ec0dd0209a Flow: Adding livedev and vlan_idx on flow
Adding livedev and vlan_idx on flow, making it possible to use it for
logging in_iface on flow-logs and fix in_iface on psuedopackets.
8 years ago
Philippe Antoine bef190f767 http: logs content range
Fixes #2485
8 years ago
Philippe Antoine 43e205fc32 smtp: rset command resets bdat chunks length
Fixes #1860
8 years ago
Philippe Antoine ff52bb14b7 ssh : code style consistency
Adds SSH_FLAG_VERSION_PARSED to flags before each return
This way, we are sure SSHParseBanner does not get called again
And proto_version does not get leaked
8 years ago
Jeff Lucovsky 4f33b8c18d decode: Improved FTP active mode handling
This changeset addresses 2 issues:
- 2459
- 2527
and improves handling for FTP active mode over IPv4 and IPv6.

Active mode is triggered when the FTP client conveys the port
that should be used for a data connection (PORT, EPRT).

When this occurs, the FTP state is marked as "active".
8 years ago
Jason Ish 164fb71898 mpls: fix misaligned read
Instead of casting the packet buffer to a uint32, memcpy it to
avoid misaligned read error, as caught by the undefined behavior
detector (ubsan).

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2903
8 years ago
Victor Julien f7d8401c2e eve/smb: minor cleanup now Rust is mandatory 8 years ago
Victor Julien c2cb155ebb rust/smb: rename files and code from RustSMB to SMB 8 years ago
Victor Julien e572324c5a detect/dcerpc: cleanup now Rust is mandatory 8 years ago
Victor Julien 50709144f9 detect/app-layer-event: cleanup test 8 years ago
Victor Julien f30c05e684 smb: remove C implementation
Now that Rust is mandatory it is obsolete.

Ticket: #2849
8 years ago