DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.
In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
Fixes issue 2845.
pcap_stats is based on 32bit counters and given a big enough throughput
will overflow them. This was reported by people using Myricom cards which
should only be a happenstance. The problem exists for all pcap-based
interfaces.
Let's use internal 64bit counters that drag along the pcap_stats and
handle pcap_stats wrap-around as we update the 64bit stats "often enough"
before the pcap_stats can wrap around twice.
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.
AppLayerTxConfig will be used by the detection engine to configure
the transaction.
This way the AppLayerTxData is set up from the start. Any type of
processing (logging, detection) will lead to setting up the user
data later on anyway.
Remove other places where it was added.
Replaces all patterns of SCLogError() followed by exit() with
FatalError(). Cocci script to do this:
@@
constant C;
constant char[] msg;
@@
- SCLogError(C,
+ FatalError(SC_ERR_FATAL,
msg);
- exit(EXIT_FAILURE);
Closes redmine ticket 3188.
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* functions.
Partially closes redmine ticket 3053.
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* functions.
Closes redmine ticket 3053.
Match on Hassh using ssh.hassh, ssh.hassh.server, ssh.hassh.string, ssh.hassh.server.string keywords, e.g:
alert ssh any any -> any any (msg:"match SSH hash"; ssh.hassh; content:"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"; sid:1000010;)
alert ssh any any -> any any (msg:"match SSH hash-server"; ssh.hassh.server; content:"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"; sid:1000020;)
alert ssh any any -> any any (msg:"match SSH hash-string"; ssh.hassh.string; content:"none,zlib@openssh.com,zlib"; sid:1000030;)
alert ssh any any -> any any (msg:"match SSH hash-server-string"; ssh.hassh.server.string; content:"umac-64-etm@openssh.com,umac-128-etm@openssh.com,"; sid:1000040;)
When under stress, the packet threads ultimately fall back
to walking the hash table until they find a flow they can
safely evict and reuse. This could lead to all threads
fighting over the FlowBucket locks.
Fix by adding a limit to the number of hash rows that are
checked for a new flow. If the limit is reached, simply fail
to get a flow.
This commit adds an (optional) entry for a validation function. The
validation function, if present, will be used during rule processing.
Its role is to determine if the arguments are compatible with the
transform. E.g., a content string of "this string has whitespace" is not
compatible with the `strip_whitespace` transform.
This changeset fixes a bug on the computation of the buffer
lenght for raw http headers. The bug is due to a mismatch
on the data type of the length (uint8_t vs uint32_t) and it
was causing signature misses.
Using the '--reject-dev <dev>' commandline option. This is a global option
that applies to all 'reject' actions.
If the interface specified is using ethernet, libnet will use the faster
L2 (link) layer. Suricata sets up the ethernet header based on the packet.
When the interface is specified, cache libnet_t ctx for (much) better
performance.
Loading file hash lists uses dirname(3) on the
de_ctx->rule_file which modifies the contents,
removing the last part of the path. So on subsequent
calls the rule_file no longer contains the rule_file,
but instead just the directory name.
Mostly noticed when using "-S" with rule files outside
of the default-rule-path which requires more hunting for
the rule file.
This commit creates a common file output helper function based on the
logic in output-file-info.c:BuildBuildFileInfoRecord
The refactored helper will be used to create "fileinfo" information
during the alert output path.
atoi() and related functions lack a mechanism for reporting errors for
invalid values. Replace them with calls to the appropriate
ByteExtractString* and StringParse* functions.
Partially closes redmine ticket #3053.
JB_SET_TRUE(jb, key), and JB_SET_STRING(string, key, val) are C macros
around jb_set_formatted to set static string and true values as a
(micro) optimization.
This commit adds a new transform -- pcrexform -- that applies a regular
expression to the transformation buffer. If an expression was captured,
that is output to the transformation buffer. Otherwise, the
transformation buffer is unchanged.
This commit adds support for transform-specific options. During Setup,
transforms have the signature string available for options detection.
When a transform detects an option, it should convert the option into an
internal format and supply a pointer to this format as the last argument
to DetectSignatureAddTransform.
Transforms that support options must provide a function in their
Sigmatch table entry. When the transform is freed, a pointer to the
internal format of the option is passed to this function.
- Fix relative_offset keyword option to be relative in regards to the
last content match
- Change relative_offset to int32_t with bounds check to allow the full
range of the packet buffer size (uint16_t)
- Added checks for over/underflows
- Changed the offset type to uint16_t because the offset is applied to
the payload length, which is a uint16_t
- Adjusted test cases to work relative to the content match
- Added test case to verify bounds
Clang's build in travis-ci is actually failing because of this error:
output-json-alert.c:476:40: error: missing field 'state_index' initializer
[-Werror,-Wmissing-field-initializers]
JsonBuilderMark mark = { 0 };
Previously a single handle to the FlowStream (which is used to program
flows to the card) was shared between the threads. This resulted
in contention between the threads where sometimes programming the flow would
silently fail.
This is to prepare for JsonBuilder conversion where we can't
overwrite an already set value. Here we prepare the addresses
to be logged in a struct, overwite with XFF if needed, then
log.
Convert alert Eve logging JsonBuilder. Currently
makes heavy use of JsonBuilder being able to log Jansson's json_t
which is a temporary measure until all protocols loggers can be
converted to JsonBuilder.
New functions that replace Jansson versions with JsonBuilder
variations use "Eve" instead of "JSON".
Update the source/target logging to use the cached address info
instead of fetching it from the constructed json_t object.
This is required for migration to JsonBuilder which does not
have the ability to retrieve already set fields.
Currently the flow logger also logs app_proto information,
but not to the flow object, but instead to the root object
of the log record.
Refactor into 2 separate methods, one for the app_proto
and one for the flow, to make this more clear, as well
as make it easier to refactor for JsonBuilder as JsonBuilder
can only write to the currently open object.
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.
As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
Remove the Jansson specific feature of being able to delete
an object from json_t, in prep for refactors to JsonBuilder.
Instead create a new header for each alert to be logged.
Move the logging of the rule text to where the alert object
is being logged to remove the usage of json_object_get...
Getting previously logged objects will not be possible with
JsonBuilder.
When PCRE `jit` is available, store the JIT stack in the keyword context
instead of on a global id. This ensures proper cleanup and
re-initialization over a rule reload.
File store v1 has been deprecated and was scheduled for removal
by June 2020.
Log an error if a file-store configuration is loaded without
version set to 2.
All the dead code in C after the Rust implementation is hereby removed.
Invalid/migrated tests have also been deleted.
All the function calls in C have been replaced with appropriate calls to
Rust functions. Same has been done for smb/detect.rs as a part of this
migration.
This commit improves how the parser handles the `RSET` command.
Termination of the transaction occurs when the `RSET` ack is seen (reply
code 250).
Bug: #3677
In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar
structures would not be cleaned up because the cleanup depends on the
'header' pointer being set, but the error handling would unset that.
This could mean these structures were already filled with values before
the error was detected. As packets were recycled, the next packet decoding
would use this unclean structure.
To make things worse these structures are part of unions. IPv4/IPv6 and
TCP/ICMPv4/ICMPv6 share the same memory location.
LibFuzzer+UBSAN found this both locally and in Oss-Fuzz:
decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool'
#0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9
#1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13
#2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9
#3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25
#4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
#5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
#6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
#7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5
#8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6
#9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
#10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8)
Bug: #3496
Support reassembling multi-frag certificates. For this the cert queuing
code is changed to queue just the cert, not entire tls record.
Improve message tracking. Better track where a message starts and ends
before passing data around.
Add wrapper macros to check for 'impossible' conditions that are activate
in debug validation mode. This helps fuzzers find input that might trigger
these conditions, if they exist.
To avoid future memcpy issues introduce a wrapper and check the
result of it.
When compiled with --enable-debug-validation the wrapper will abort if
the input is wrong.
In some error conditions, or potentially in case of multiple 'certificate'
records, the extracted subject, issuerdn and serial could be overwritten
without freeing the original memory.
'trec' buffer was not grown properly when it was checked as too small.
After this it wasn't checked again so that copying into the buffer could
overflow it.
This commit checks whether pre-6.x settings for ERSPAN Type I are
present. ERSPAN Type I is no longer enabled/disabled through a
configuration setting -- it's always enabled.
When a setting exists to enable/disable ERSPAN Type I decoding, a
warning message is logged.
Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x
This commit adds a warning value when ERSPAN Type I configuration
settings are detected; specifically, when ERSPAN Type I `enabled` is
specified.
Enabling/disabling ERSPAN Type I decode has been deprecated in 6.x