Commit Graph

452 Commits (1e8d4af29afa1dd8e1765f19ab1ceca0d897f031)

Author SHA1 Message Date
Frank Honza 1c8943dedd add RFB parser
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
5 years ago
Victor Julien acef21b759 app-layer: extend AppLayerResult to add convenience 5 years ago
Victor Julien 2f5834cdfa rust: merge parser.rs into applayer.rs
Both were serving the same purpose.
5 years ago
Victor Julien 21e6f1f063 app-layer: document return macros 5 years ago
Victor Julien 5b9b0b7226 nfs: switch to new 'incomplete' logic
Remove buffering code in favor of using incomplete API.
5 years ago
Victor Julien 66598f9ce7 nfs: switch to AppLayerResult 5 years ago
Victor Julien 5cd9cfb5a0 smb: update return type of GAP handling 5 years ago
Victor Julien 4bf87d30e4 smb: convert to return AppLayerResult
Support returning 'incomplete' and remove the buffering
code from the parser.
5 years ago
Victor Julien 674b8dc0fb app-layer: add 'incomplete' return logic
Allow app-layer parsers to indicate how much data they need
before being called again.
5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
5 years ago
Victor Julien 6ae66cb2bb nfs: code cleanups
Use 'if let' to replace simple 'match' statements.

Use explicit returns to easy code review.
5 years ago
Victor Julien a729d266c3 smb: fix rustc 1.42 warnings 5 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust
And port the C DNS tests to Rust.
5 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
5 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust
Remove app-layer-dns-common.c as its no longer needed.
5 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
5 years ago
Jason Ish d809b0959b dns: cleanup: move DnsGetRcode (Lua) to rust
Move the implementation of Lua DnsGetRcode to Rust.
5 years ago
Philippe Antoine 23f796a021 kerberos: fix against packet split in record size 5 years ago
Victor Julien 76dd951523 krb5/tcp: remove notice logging on failed records 5 years ago
Sascha Steinbiss 11912bd715 sip: address trailing space parsing 5 years ago
Victor Julien 74305c0486 rust/nfs: minor code cleanups 5 years ago
Victor Julien 576e92983e rust/rpc: add partial data tests 5 years ago
Pierre Chifflier 442500678b rust: use the streaming version of combinators to fix incomplete reads 5 years ago
Pierre Chifflier df9a4fd635 rust/rdp: use the streaming version of combinators to fix incomplete reads 5 years ago
Pierre Chifflier 2561da89c0 rust/rdp: fix regression introduced during nom 5 upgrade 5 years ago
Pierre Chifflier f3ddd7127c rust: Add types annotation when required
Unfortunately, the transition to nom 5 (and functions instead of macros)
has side-effects, one of them being requiring lots of types annotations
when using a parsing, for ex in a match instruction.
5 years ago
Pierre Chifflier 62e31396ae rust/ftp: upgrade to nom 5
CompleteByteSlice type has been removed, and replaced by combinators
under the nom::character::complete namespace.
5 years ago
Pierre Chifflier 1a505ccd11 rust/rdp: add custom error handling 5 years ago
Pierre Chifflier d1bf34a427 rust/smb: add custom error handling 5 years ago
Pierre Chifflier 1ab8c5763c rust: add SecBlobError custom error type for the Kerberos parser 5 years ago
Pierre Chifflier 030c9a3d86 rust: add take_until_and_consume replacement function 5 years ago
Pierre Chifflier 8664a55ee7 rust/dns: remove unneeded calls to closure! 5 years ago
Pierre Chifflier 5b809f77f1 rust: upgrade all parsers to nom 5 5 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 5 years ago
Victor Julien f68c255f09 nfs: implement post-GAP transaction cleanup
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic described below.

After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs. Due to the
similarity of the NFS and SMB parsers, this issue is fixed for NFS
as well in this patch.

Bug #3424.
Bug #3425.
5 years ago
Victor Julien d41aeccea4 smb: handle file transactions post-GAP
After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs.
5 years ago
Jason Ish 80cafb2979 flow: expose last time as a function
This function returns the individual components
of the timeval in output pointers making it suitable
for use over Rust FFI.
5 years ago
Jason Ish 1e741641a6 rust: make clean fixups
For make clean, only remove gen/ if cbindgen is available.
This prevents make clean from remove gen when the headers
were bundled, but cbindgen is not available to remove them.

Unconditionally remove gen and vendor in maintainerclean.
5 years ago
Jason Ish 53aaa82deb rust: set edition to 2018 5 years ago
Jason Ish e68dfa46a6 rust: cargo fix for Rust 2018 edition 5 years ago
Jason Ish 5fbe020585 rust/cbindgen: Revert Makefile to a more pre-cbindgen state
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.

Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
5 years ago
Danny Browning b573c16dd5 build: cbindgen
Rust headers are now generated using cbindgen. If cbindgen is present, they can
be generated during dist, otherwise they will be available for builds.
5 years ago
Victor Julien ce0ae81d95 rust: fix vendor use on MinGW 5 years ago
Jason Ish 57b683233d rust: Don't use --frozen during build.
If sources are vendored, we get the same effect of using frozen
with a lock file, and the Cargo.lock is generated based
on the vendored sources.

This also removes the need to ship a Cargo.lock.

Fixed out of source builds with vendored sources.
5 years ago
Jason Ish c6f168eb98 rust/Makefile: Don't include Cargo.toml
There is no need to include Cargo.toml in the distribution,
it is always generated from Cargo.toml.in during
./configure.
5 years ago
Jason Ish 5ee8323028 rust: remove unnecessary parentheses (Rust 1.40 fixup)
Rust 1.40 in strict mode will now fail the build on the
presence of unnecessary parentheses.

warning: unnecessary parentheses around type
  --> src/smb/smb2_ioctl.rs:41:12
   |
41 |         -> (&mut SMBTransaction)
   |            ^^^^^^^^^^^^^^^^^^^^^ help: remove these parentheses
   |
   = note: `#[warn(unused_parens)]` on by default
5 years ago
Victor Julien f5b33a070a smb1: allow empty trans records 5 years ago
Victor Julien 40fe29de96 smb1: fix 'event' txs not getting closed
If the only reason we created a request side TX was to set an event,
we would not close it.

This patch always looks up the TX from the response side.
5 years ago
Victor Julien 129cd28058 smb/dcerpc: close request tx sooner 5 years ago
Victor Julien 44ac3e30dd smb: post-GAP handling update
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic.
5 years ago
Victor Julien a7ee2ffbde smb: winreg is a DCERPC facility 5 years ago
Ciprian c9cd7559fd configure: fixing rust/cargo cross compile command
adding --target argument to cargo command line when cross compiling
5 years ago
Jason Ish e2c846d01f snmp: add tx detect flags 5 years ago
Jason Ish fa4b9d37c2 krb5: register tx detect flags
Related ticker #3345:
https://redmine.openinfosecfoundation.org/issues/3345
5 years ago
Jason Ish 8a232be77e rust: define TxDetectFlag struct and binding macros
Define a TxDetectFlag type and macros to generating C
bindings for getting and settings the tx detect
flags.
5 years ago
Jason Ish cde49ec246 rust: add tx detect flags function to registration struct 5 years ago
Victor Julien f9155aa121 files: simplify pruning logic
Since ebcc4db84a the flow worker runs
file pruning after parsing, detection and loging. This means we can
simplify the pruning logic. If a file is in state >= CLOSED, we can
prune it. Detection and outputs will have had a final chance to
process it.

Remove the calls to the pruning code from Rust. They are no longer
needed.
5 years ago
Victor Julien b4318a11e3 rust: remove build system HAVE_RUST guards 5 years ago
Jason Ish a3cdef2b4d rust: run tests with same features as build
Cargo check wasn't being passed --features so could have a different
configuration than the build.
5 years ago
Jason Ish 389272f4c7 rustup: handle rustup for sudo and su
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.

To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
5 years ago
Jason Ish 6e981fd15a rust: fix build when source directory has spaces in it
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2668
5 years ago
Jason Ish f9c9548b74 configure: detect lua integer size
Lua 5.1 and 5.3 use a different integer size. Run a test program
to set the integer size used in the Rust FFI layer to Rust.
5 years ago
Jeff Lucovsky bd691778eb rust/ftp: add parser for active mode port handling 6 years ago
Jason Ish 517ecd68a9 sip: rustfmt
As this is new Rust code, format with rustfmt using default
styling.
6 years ago
Jason Ish a45a2fa1fc sip: disable by default in 5.0 6 years ago
Jason Ish fdbc2fe49c sip rust fixup: remove unused import in tests 6 years ago
Giuseppe Longo e06291922f detect/sip.response_line: add sticky buffer
Matches on response line field in SIP.
6 years ago
Giuseppe Longo 17de4a8023 detect/sip.request_line: add sticky buffer
Matches on request line field in SIP.
6 years ago
Giuseppe Longo 8939ece538 detect/sip.stat_msg: add sticky buffer
Matches on status msg field in SIP.
6 years ago
Giuseppe Longo bd2219cac6 detect/sip.stat_code: add sticky buffer
Matches on status code field in SIP.
6 years ago
Giuseppe Longo 8454122eb2 detect/sip.protocol: add sticky buffer
Matches on protocol field in SIP.
6 years ago
Giuseppe Longo 2661c5b298 detect/sip.uri: add sticky buffer
Matches on uri field in SIP.
6 years ago
Giuseppe Longo 424eead8c0 detect/sip.method: add sticky buffer
Matches on uri field in SIP.
6 years ago
Giuseppe Longo edc2a583a9 rust/sip: add SIP logger 6 years ago
Giuseppe Longo 2e975a0481 rust/sip: add parser for SIP protocol 6 years ago
Jason Ish d79c23baa3 dns/detect: dns.opcode keyword
Add a rule keyword, dns.opcode to match on the opcode flag
found in the DNS request and response headers.

Only exact matches are allowed with negation.

Examples:
  - dns.opcode:4;
  - dns.opcode:!1;
6 years ago
Victor Julien 85ba2e16ba rust/conf: don't print failed conf lookups at info level 6 years ago
Jason Ish 664605b5f1 rdp: disable rdp by default for 5.0 6 years ago
Jason Ish 0f10298990 rdp: address comments in pull request
Pull request:
https://github.com/OISF/suricata/pull/4174

- fix commit: range -> set
- OUTPUT_BUFFER_SIZE -> JSON_OUTPUT_BUFFER_SIZE
- output: check for initdata first
6 years ago
Zach Kelly caef8b5b38 protocol parser: rdp
Initial implementation of feature 2314:
1. Add protocol parser for RDP
2. Add transactions for RDP negotiation
3. Add eve logging of transactions
6 years ago
Jason Ish 5f1d21f247 dns: handle mid stream pickup on response packet
Related Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2146
6 years ago
Jason Ish 42e5065ab8 rust: update to Rust 2018 with cargo fix
Migrate to Rust 2018 edition.

Credit to Danny Browning for first demontrating this:
https://github.com/OISF/suricata/pull/3604/commits
6 years ago
Shivani Bhardwaj 8bebea5d4c rust: Get rid of unneeded macros, fix warnings 6 years ago
Victor Julien 579cc9f02b const: constify decoder, app-layer, detect funcs 6 years ago
Jason Ish 8f66f39b29 rustfmt: use default rustfmt configuration
Remove our config that limited line length to 80 columns.
6 years ago
Shivani Bhardwaj f750e4ca40 configure: Remove enable-rust-debug
Get rid of enable-rust-debug flag and use enable-debug for acheiving the
desired functionality. From now, adding `--enable-debug` to `configure`
shall create an [unoptimitized + debuginfo] target. Rest behavior stays
the same.

Closes redmine ticket #3054
6 years ago
Shivani Bhardwaj 6d39f6fd7d rust: Fix deprecation warnings
Fix the following warnings by compiler,
(1) warning: use of deprecated item 'take_until_s': Please use `take_until` instead
(2) warning: `...` range patterns are deprecated

For the second warning, the builtin lint
"ellipsis_inclusive_range_pattern" has been added which causes the
following warning to show up with rustc 1.24.

warning: unknown lint: `ellipsis_inclusive_range_patterns`
  --> /home/travis/build/OISF/suricata/suricata-5.0.0-dev/rust/src/lib.rs:18:10
   |
18 | #![allow(ellipsis_inclusive_range_patterns)]
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(unknown_lints)] on by default

Since there is no other way to fix this, the above warning shall stay.
We need to take care of modifying this if and when the support for 1.24
as MSRV is dropped.
6 years ago
Shivani Bhardwaj bbfd706e1f rust: fix compiler warning
rustc 1.36 introduced:

error: variable does not need to be mutable
   --> src/dhcp/parser.rs:202:17
    |
202 |             let mut malformed_options = false;
    |                 ----^^^^^^^^^^^^^^^^^
    |                 |
    |                 help: remove this `mut`
    |
note: lint level defined here
   --> src/lib.rs:18:38
    |
18  | #![cfg_attr(feature = "strict", deny(warnings))]
    |                                      ^^^^^^^^
    = note: #[deny(unused_mut)] implied by #[deny(warnings)]

error: aborting due to previous error

error: Could not compile `suricata`.

Ticket #3072.
6 years ago
Pierre Chifflier af7d245a31 rust/snmp: add event when expected/received PDU versions mismatch 6 years ago
Pierre Chifflier 1880f6945c rust/snmp: use generic parsing function, for all SNMP versions
Do no restrict parsing to the version seen in the first packet, but
use a generic function, independent of the version.
6 years ago
Jeff Lucovsky 6911cc01ad rust/snmp: Support get-info-by-id 6 years ago
Jeff Lucovsky 7560b75591 rust/ntp: Support get-info-by-id 6 years ago
Jeff Lucovsky 12c2d18c8b rust/krb: Support get-info-by-id 6 years ago
Jeff Lucovsky fb01641629 rust/ikev2: Support get-info-by-id 6 years ago
Jeff Lucovsky e3ca6b43fc rust/dhcp: Support get-info-by-id 6 years ago
Jeff Lucovsky a5d9d37c34 rust/parser: Extend Rust parser for event-by-id
Extend the Rust parsing infrastructure with the "get event info by id"
calls. This changeset extends the parser structure, the C-based
registration handlers and the template parser.
6 years ago
Jeff Lucovsky 9ccc28baeb rust/smb: Implement get event by id 6 years ago
Jeff Lucovsky 643864a8f5 rust/snmp: fix libc deprecation warnings 6 years ago
Victor Julien 3f6624bf16 rust: remove libc crate dependency
Use std::os::raw instead.
6 years ago