Commit Graph

62 Commits (a85340b1ab2aa9490a4d4d937ad0f8808b052dfe)

Author SHA1 Message Date
Jason Ish 6392216f6b base64: use the Rust base64 encode implementation
Replace our internal base64 implementation with a ffi wrapper
around the Rust implementation provided by an external crate.
5 years ago
Jason Ish 6d3dcf27a6 eve: use JsonBuilder for encoding base64
Replaces all usages of Base64Encode just before writing to a
JsonBuilder with jb_set_base64 and jb_append_base64.
5 years ago
Jason Ish 06f58650d6 eve: refactor OutputJsonBuilderBuffer to take context
All callers of OutputJsonBuilderBuffer are now calling it
using fields from an OutputJsonThreadCtx, so just pass
a pointer to the thread context now.
5 years ago
Jason Ish 08eee26d27 eve: convert many loggers to use generate thread context
- mqtt
- dnp3
- smtp
- ike
- dns
- alert
- tls
- anomaly
- drop
- file
- http
- http2
- templates
- dhcp

The idea is to factor out the commom code for setting
up the output file objects, which is repetitive, and
often done wrong when it comes to threading.
5 years ago
Jason Ish 06ba611667 eve cleanup: remove duplicate/redundant code
The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
5 years ago
Eric Leblond 7304389438 eve: only output ja3 and ja3s if present
This will prevent JSON entries like the following that occur
with the dedault configuration (ja3 deactivated and extended
tls ouput activated):

  "tls": {
    "subject": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "issuerdn": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "serial": "00:9C:FC:DA:1D:A4:70:87:5D",
    "fingerprint": "b8:18:2d:cb:c9:f8:1a:66:75:13:18:31:24:e0:92:35:42🆎96:89",
    "version": "TLSv1",
    "notbefore": "2020-05-03T11:07:28",
    "notafter": "2021-05-03T11:07:28",
    "ja3": {},
    "ja3s": {}
  }
6 years ago
Shivani Bhardwaj ea0936199d output: use BASE64_BUFFER_SIZE macro
Base64Encode function requires the maximum length of the output string
as its last parameter. Use the macro BASE64_BUFFER_SIZE to calculate it
correctly.
6 years ago
Jeff Lucovsky ff29345527 output: Remove unused output functions
This commit removes registration, initialization, and de-initialization
functions no longer needed
6 years ago
Jeff Lucovsky 15caf3eea5 log: Remove standalone output registration
Remove standalone output logger registration since eve is
multi-instance.
6 years ago
Jeff Lucovsky 30ae98f658 output/json: Multi-threaded EVE logging support
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.

Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
6 years ago
Victor Julien 7ccfa177de eve/tls: minor cleanups 6 years ago
Jason Ish 037c449b85 tls/eve: convert to jsonbuilder 6 years ago
Jason Ish 99f460aa5a eve/fivetuple: use intermediate address struct (jsonbuilder prep)
Currently alert logging relies on the ability to change existing
values in the json_t structure to overwrite addresses with xff
data. This feature is also used for the "target" logging.

As we can't do this with JsonBuilder, create a new struct to
hold the 5 tuple, with the values swapped as needed, and
overwritten with XFF data if needed. This struct will now
be used to write out the 5 tuple, as well as cache the information
for log fields to be written out later on in the log path.
6 years ago
Victor Julien aeefc82eb9 tls: fix missing extern logic for cert_id tracking 7 years ago
Victor Julien edd2cd626f jansson: remove HAVE_LIBJANSSON guards 7 years ago
Jeff Lucovsky 3d5eccf084 output/json: Refactor output buffer size macro 7 years ago
Mats Klepsland a4eaef25d6 eve: add JA3S field to TLS JSON logger
Add JA3S object to TLS JSON logger (extended log).
7 years ago
Mats Klepsland a4471987ba app-layer-ssl: generate JA3S fingerprints
Generate JA3S fingerprints based on fields in the ServerHello record.
7 years ago
Victor Julien e956b484c5 eve/json: handle common options in central function 8 years ago
Victor Julien df1ec82b55 eve/json: move common settings into it's own struct 8 years ago
Mats Klepsland 814e1624c2 output-json-tls: don't log as "resumed" without ServerHello
Don't log a session as "resumed" if a ServerHello record has not been
seen. This makes sure that incomplete TLS sessions where the ClientHello
contains a session ticket, is not logged as a session resumption.
8 years ago
Mats Klepsland 25fb02da9a output-json-tls: add support for TLSv1.3 8 years ago
Mats Klepsland e0ef578c46 app-layer-ssl: add support for session tickets
Add support for logging a session as 'resumed' when using a non-empty
session ticket extension in the client hello record.
8 years ago
Maurizio Abba bf4398b15d output-json: ensure string is json-encodable
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.

rust modification will encode any non-printable character during its
conversion in to_cstring.
8 years ago
Victor Julien 44daf84321 output/tls: don't pass NULL fingerprint to json logging 8 years ago
Mats Klepsland d9ddae4592 tls: replace variable in header file with TAILQ_EMPTY()
Use TAILQ_EMPTY to check if a certificate exists instead of
setting two variables in the app-layer-ssl header file for
that purpose only.
8 years ago
Mats Klepsland d55e455504 eve: add JA3 fields to TLS JSON logger
Add JA3 object to TLS JSON logger (extended log).
8 years ago
Victor Julien 710c7b821f output/json: update callers to use explicit directions 9 years ago
Victor Julien 44c4008f77 output/json: clean up CreateJSONHeader calls 9 years ago
Jason Ish 88ac0f2b1a eve: tls: respect global metadata config 9 years ago
Jason Ish 00e6cd4ced output: introduce init return type
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.

Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.

Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
9 years ago
Mats Klepsland 74aa65073b output-json-tls: log 'from_proto' field
Log the original application level protocol when protocol have been
changed because of STARTTLS, HTTP CONNECT or similar.
9 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
9 years ago
Ray Ruvinskiy 7539973109 tls: logging for session resumption
We assume session resumption has occurred if the Client Hello message
included a session id, we have not seen the server certificate, but
we have seen a Change Cipher Spec message from the server.

Previously, these transactions were not logged at all because the
server cert was never seen.

Ticket: https://redmine.openinfosecfoundation.org/issues/1969
9 years ago
Mats Klepsland 2b460b8d06 output-json-lua: log certificate serial number 10 years ago
Mats Klepsland bbd32c1419 output-json-tls: code cleanup 10 years ago
Mats Klepsland 115b3138cc output-json-tls: log certificate and chain
Log entire certificate and certificate chain Base64 encoded.
10 years ago
Mats Klepsland 0716199acb output-json-tls: custom tls logging 10 years ago
Jason Ish 3fab684f97 logging: don't log that json is disabled in each logger
A warning log is already emitted if eve-log is enabled in the
configuration but json support is not built so the logger
registration functions can be silent.
10 years ago
Jason Ish 44c846f2f8 tls-json: make tls events direction sensitive
Previously the src/dest ips in TLS events would differ between
IDS and IPS modes. Make the header creation direction sensitive
so they are identical in both modes.
10 years ago
Mats Klepsland 5b230bbce5 output-json-tls: add notBefore and notAfter fields to extended output
Add notBefore and notAfter fields from TLS certificate to extended JSON
output.
10 years ago
Jason Ish 1b4ba4496c logging: rename registration functions to not have tmm
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.

Also, move the registration to the output.c which will handle
registration of the loggers.
10 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers
The loggers moved away from a TMM required a new
profiling support.
10 years ago
Jason Ish aaa65f3d16 logging: convert json tls output to non-thread module 10 years ago
Jason Ish 687602c0ca logging: convert eve dns logging to non-thread module 10 years ago
Jason Ish b23d74ac88 tls-json-log: register module as tls-json-log, not dns-json-log
Fixes issue:
https://redmine.openinfosecfoundation.org/issues/1792
where dns-json-log would not log any data.
10 years ago
Mats Klepsland 47a4daeeb8 output: allow more than one TLS logger 10 years ago
Mats Klepsland e9074bc829 jsontls: use TxLogger 10 years ago
maxtors 06d74b5775 Module specific error code for init ctx error. 10 years ago
maxtors 69863f7b1c Corrected and unified debugmessages for init data errors in *ThreadInit. 10 years ago