To some EVE fields and a "suricata" object that contains an array of
keywords. These are the keywords that map directly to this field, or
somehow cover this field.
This is an attempt at tooling to help with EVE and keyword parity.
Related to tickets: #5642, #6463, #4772
Issue 7129
When configured with the existing "ethernet" switch, include the ether
type in the output.
This is most useful with anomaly records indicating unknown ethertypes.
If TCP urgent handling is set to the OOB processing, the number of OOB
bytes is tracked for SEQ offset calculations. If this offset is
non-zero, add the field to the flow record.
Ticket: #7411.
TCP urgent handling is a complex topic due to conflicting RFCs and
implementations.
Until now the URG flag and urgent pointer values were simply ignored,
leading to an effective "inline" processing of urgent data. Many
implementations however, do not default to this behavior.
Many actual implementations use the urgent mechanism to send 1 byte of
data out of band to the application.
Complicating the matter is that the way the urgent logic is handled is
generally configurable both of the OS and the app level. So from the
network it is impossible to know with confidence what the settings are.
This patch adds the following policies:
`stream.reassembly.urgent.policy`:
- drop: drop URG packets before they affect the stream engine
- inline: ignore the urgent pointer and process all data inline
- oob (out of band): treat the last byte as out of band
- gap: skip the last byte, but do no adjust sequence offsets, leading to
gaps in the data
For the `oob` option, tracking of a sequence number offset is required,
as the OOB data does "consume" sequence number space. This is limited to
64k. For this reason, there is a second policy:
`stream.reassembly.urgent.oob-limit-policy`:
- drop: drop URG packets before they affect the stream engine
- inline: ignore the urgent pointer and process all data inline
- gap: skip the last byte, but do no adjust sequence offsets, leading to
gaps in the data
Bug: #7411.
Ticket: 7199
Uses a config parameter detect.guess-applayer-tx to enable
this behavior (off by default)
This feature is requested for use cases with signatures not
using app-layer keywords but still targetting application
layer transactions, such as pass/drop rule combination,
or lua usage.
This overrides the previous behavior of checking if the signature
has a content match, by checking if there is only one live
transaction, in addition to the config parameter being set.
Ticket: 7311
If response_status_number is not a valid poisitive integer,
we should not try to parse it again, and fail again,
but just log the raw string.
Add EVE documentation for QUIC and Pgsql to their respective sections of
the userguide.
Also add a complete EVE reference as an appendix.
Other protocols can be done, but its a manual process to document in the
schema, then add the glue to pull them into the documentation.
The documentation is generated during "make dist", or if it doesn't
exist, "conf.py" will attempt to generate the eve documentation for
building on Readthedocs.
Issue: 4974
1. Use https instead of http everywhere
2. Organize and annotate references by
- Referenced by ET/Open and ET/Pro
- URL resolves and works as intended (to provide supplemental
information regarding a reference value, e.g., bug id, cve
value)
- URL no longer resolves
- URL resolves but doesn't work as intended (to provide
supplemental information)
- Not referenced by ET/Open nor ET/Pro
- URL resolves and works as intended (to provide supplemental
information regarding a reference value, e.g., bug id, cve
value)
- URL no longer resolves
- URL resolves but doesn't work as intended (to provide
supplemental information)
This introduces a new parser registration function for LDAP/UDP, and update
ldap configuration in order to be able to enable/disable a single parser
independently (such as dns).
Also, GAPs are accepted only for TCP parser and not for UDP.
Ticket #7203
V3 style DNS logging fixes the discrepancies between request and
response logging better dns records and alert records.
The main change is that queries and answers are always logged as
arrays, and header fields are not logged in array items.
For alerts this means that answers are now logged as arrays, queries
already were.
DNS records will get this new format as well, but with a configuration
parameter.
Bug: #6281
Feature: 7017
Add DNSRDataOPT struct and DNSRData enum type OPT.
Add OPT parsing function and test function.
Add DNSRData OPT type to lua.rs match.
Log OPT rdata.
Ticket: 3958
- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
enip.product_name, enip.protocol_version, enip.revision,
enip.identity_status, enip.state, enip.serial, enip.product_code,
enip.device_type, enip.vendor_id, enip.capabilities,
enip.cip_attribute, enip.cip_class, enip.cip_instance,
enip.cip_status, enip.cip_extendedstatus
Update the Lua allocated to set a code on memory allocation limit
exceeded errors so an appropriate error message can be logged and a
state incremented.
Fixes the tracking of the allocated size by using the difference
between original size, and new size and toss in some debug
validations.
Distinguish between a generic Lua script error and an error created by a
function being blocked, so each is logged once respective of each other.
Also add a stat that is incremented when a script fails due to a
blocked function.
NOTE: This does not catch calls to functions that are blocked by not
having the library loaded, such as "io.open", as they are blocked by
not even loading the "io" library.
If a rule script crashed, the return value was treated as a no
match. This would make a negation of the rule match and alert.
Instead cleanup and exit early if the rule script crashed and don't
run negation logic.
A stat, detect.lua.errors has been added to count how many times a
script crashes.
Also consolidates the running of the Lua script and return value
handling to a common function.
Bug: #6940
This implements a logger for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.
Ticket #6627
This commit moves the memcap pressure/pressure_max stats from the global
stats namespace into the memcap namespace.
With per-thread stats, they will be within the flow-manager's values.
Issue: 6398
We will register stats counters for all policies, even though for now
Suri only uses one possible configuration policy at a time. The idea is
that this could change in the near future, so we want to have this
ready.
Task #5816
A CanceldRequest can occur after any query request, and is sent over a
new connection, leading to a new flow. It won't take any reply, but, if
processed by the backend, will lead to an ErrorResponse.
Task #6577
* Log vendor client identifier (dhcp option 60) if extended dhcp
logging is turned on. This required the `vendor_client_identifier` to
be added to the json schema. Validation done using an SV Test
* Added `requested_ip` to the json schema as well, since it was
missed. My SV test failed without it.
Feature #4587
Debug facility to get a per packet view into the stream engine's state.
Logs after a packet has been processed in the stream engine, so the view
into the state includes the updates based on the current packet.
Marked as experimental so it can be changed w/o notice.
Bug: #5876.
Looks like this was due to an error in the dcerpc logging where the
interfaces should have been logged to the "interfaces" array that was
already defined.
Issue: 5814
Accept DNS messages with an invalid opcode that are otherwise
valid. Such DNS message will create a parser event.
This is a change of behavior, previously an invalid opcode would cause
the DNS message to not be detected or parsed as DNS.
Issue: #5444
Issue: 2497
This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.
The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.
Pgsql's parameters - for message types like StartupMessage and
ParameterStatus, for instance, don't have a finite, definitive set, as
per their documentation. Our json schema was allow expecting a fixed set
of parameters, though, resulting in SV tests failing if different, valid
parameters appeared.
Bug #5579
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.
Feature #5413.
As an SMB2 async response does not have a tree id, even if
the request has it.
Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.
Ticket: #5508
Also logs if the ticket encryption is weak.
It is different from the encryption used for the rest of the
packet, and this allows to detect kerberoasting attack.
Ticket: #5442
Install classification.config and reference.config to $datadir,
where they can be updated on every upgrade.
This required moving them into a sub-directory for autotools
to do its thing.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3209