Commit Graph

11361 Commits (20df715e64ccab00c6f6778e1453b051aee7be17)

Author SHA1 Message Date
Victor Julien d3675d5197 detect/krb5.cname: update buffer initialization logic 4 years ago
Victor Julien 849f1cf1b8 detect/ike.vendor: update buffer initialization logic 4 years ago
Victor Julien 158e648d87 detect/http2: update buffer initialization logic 4 years ago
Victor Julien 5e783a01fe detect/file.name: update buffer initialization logic 4 years ago
Victor Julien 576bfc6bf0 detect/file.magic: update buffer initialization logic 4 years ago
Victor Julien 50fd691efb detect/file.data: update buffer initialization logic 4 years ago
Victor Julien 9c34e82471 detect/http.uri: update buffer initialization logic 4 years ago
Victor Julien f6fd1b017f detect/dce.stub_data: update buffer initialization logic 4 years ago
Victor Julien e31ead9bc2 detect/dns: update buffer initialization logic 4 years ago
Victor Julien 70125a29f6 detect/buffer: add initialized flag to simplify buffer logic 4 years ago
Jeff Lucovsky c1c67536b6 decode/stat: Add decode counters for unknown/arp
Issue: 5761

This commit adds statistics for ARP and unknown ethertype packets for
diagnostic purposes.
4 years ago
Philippe Antoine e3105a6614 ftp: adds a config option ftp-hash for autofp-scheduler
This allows ftp-data and ftp flows to be processed by the same
thread. Otherwise, there may be a concurrency issue where the
would-be ftp-data flow is first processed, and thus not recognized
as such. And the ftp flow gets processed later and the expectation
coming from it is never found.

To do so, the flow hash gets used as usual, except for flows that
may be either ftp or ftp-data, that is either one port is 21, or
both ports are high ones.

Ticket: #5205
4 years ago
Philippe Antoine c1f615b8d2 src: fix coverity warning about sizeof
CID: 1520601
CID: 1520602

> In this particular case sizeof (char **) happens to be equal to
sizeof (char *), but this is not a portable assumption.
4 years ago
Jason Ish 84d1ed58bb config: check return value of dotted override
Fixes commit fbb0d2b0f4.
4 years ago
Jason Ish 3f4dad8676 ftp: add events for command too long
Issue: 5235
4 years ago
Jason Ish dcfa6a6002 suricata: allow additional include files on command line
Add a new command line option, --include. This will merge additional
configuration files into the configuration specified in the main
suricata.yaml.  It can be provided multiple times and the files will be
included in the order they appear on the command line.

Ticket: 3912
4 years ago
Jason Ish cb1ae92a1b yaml-loader: add test for fully qualified override
Also set the parent node for regular nodes on creation as this is
useful in unit-tests to verify the parent of a node.
4 years ago
Jason Ish fbb0d2b0f4 config: allow fully qualified overrides
Allow configuration parameters to be overrided usually a fully
qualified name such as:

vars.address-groups.HOME_NET: "7.1.2.0/24"

In configuration files (including "include" files).  This allows the
overriding of a specific value deeply nested in the configuration
without having to redefine the complete top-layer object.

Ticket: 4783
4 years ago
Philippe Antoine 9cb0bc3332 util/landlock: check return values for ConfGet
CID 1514671
CID 1514669
4 years ago
Shivani Bhardwaj f941ceae2b decode/udp: fix payload_len calculation
Fix payload_len calculation post removal of the condition that returned
error code if the length to the decode fn did not match the length of
header from the UDP packet.

Bug 5379
4 years ago
Shivani Bhardwaj eebdfe9a3e decode/events: add event type UDP_LEN_INVALID 4 years ago
Lukas Sismis d18e52ed93 decode-udp: Allow shorter UDP packets than the remaining payload length
If the packet is shorter than IP payload length we no longer flag it as an
invalid UDP packet. UDP packet can be therefore shorter than IP payload.
Keyword "udp.hlen_invalid" became outdated as we no longer flag short UDP
packets as invalid.

Redmine ticket: #5693
4 years ago
Jason Ish a0fc00bb48 log-pcap: fix inverse logic error
We shouldn't early initialize when *offline*.  Instead this accidentally
delayed initializing when if an online mode, however its likely not to
have been noticed as delaying initializing in online mode is supported
as well.
4 years ago
Philippe Antoine e07556b961 runmodes: fix memory leak
By using constant for string instead of allocating and leaking it

CID: 1520497
CID: 1520500
4 years ago
Philippe Antoine b281199e9a test: do not output non ascii character
The unit test for content |aa bz| transforms in place the string
str to replace the 2 characters aa by one character 0xaa
Then, when z is not recognized as a valid hexadeicmal character,
the whole modified string is printed out, inclusing the non-ascii
0xaa

Ticket: #5558
4 years ago
Victor Julien 7a47eabf82 streaming: fix possible use after free
Don't use ptr after freeing it. Reported by Coverity Scan.
4 years ago
Victor Julien f41646a012 streaming: remove dead code
As reported by Coverity Scan.
4 years ago
Victor Julien 7b18b9c01c flow: make next_ts unsigned
To silence a coverity warning.
4 years ago
Victor Julien 420351eda2 time: fix various time issues
Found by Coverity Scan.
4 years ago
Shivani Bhardwaj 06caef988c detect/ftp: use AppLayerResult API 4 years ago
Juliana Fajardini d9cd05e828 stream: remove unused retval from fn description 4 years ago
Philippe Antoine e8db6255cb tls: accept validity before 1970
modify TLS certificate decoding of validity timestamps
to support times between 1950 and 2049,
as per RFC 5280

Ticket: #3253
4 years ago
Philippe Antoine 0c0fcc9411 detect: config keyword transaction logic fix
When the keyword config:logging disable,type tx is used,
OutputTxLog checks a flag to skip the transaction without logging
it, but AppLayerParserTransactionsCleanup waits for the
transaction to be marked as logged to clean it.

So, OutputTxLog now marks the tx as logged, so that it can
get cleaned away.

Ticket: #5456
4 years ago
Juliana Fajardini 416f7522e7 stream/tcp: fix typos, update copyright year
Bug #5765
4 years ago
Juliana Fajardini 8e9905e0d8 exceptions: ignore policy if stream.midstream=true
Set the engine to ignore the stream.midstream-policy if stream.midstream
is enabled.

If we had both stream.midstream AND stream.midstream_policy enabled,
this could lead to midstream flows being dropped (or bypassed, or...)
instead of being accepted by the engine, as it was probably meant when
the user enabled midstream flows.

Bug #5765
4 years ago
Juliana Fajardini 0d9289014b exceptions: add master switch config option
This allows all traffic Exception Policies to be set from one
configuration point. All exception policy options are available in IPS
mode. Bypass, pass and auto (disabled) are also available in iDS mode

Exception Policies set up individually will overwrite this setup for the
given traffic exception.

Task #5219
4 years ago
Jason Ish 30a2cfa693 log-pcap: remove early output initializing if offline
Remove early opening of output files if running in an offline mode, as
we don't yet know the timestamp to use.

Prevents the first pcap files being opened with a timestamp of 0,
bringing us back to the same behvaviour of pcap logging in 6.0.

Issue: 5374
4 years ago
Jason Ish 64c069f162 log-pcap: fix typo in multi-mode error message 4 years ago
Jason Ish e4f85ecc2a log-pcap: display mininum limit on error
On fatal error due to limit being less than the allowed minimum,
display the minimum value in bytes.
4 years ago
Lukas Sismis 30d9d8c7f4 util-debug: clang-format required change 4 years ago
Lukas Sismis 03c21bfaa8 runmodes: Determine engine's copy-mode as early as possible
Configuration and behavior of HTP app layer depends on the copy
mode of Suricata engine. Copy mode was set after the app layer setup.
Decision of engine's copy mode operation is now made earlier.

Ticket: #5706
4 years ago
Lukas Sismis 958f94276a runmodes: remove misleading commment 4 years ago
Lukas Sismis e2a5bc7961 dpdk: add support for enabling IPS support in DPDK mode 4 years ago
Lukas Sismis 449943e1a9 util-device: remove unused functions 4 years ago
Lukas Sismis bed16ba44c runmodes: change function prototype of runmode init functions
Commit contains prototype changes of RunModeSetLiveCaptureAutoFp and
RunModeSetLiveCaptureWorkers functions to move the IPS enable logic
out of suricata.c file.
4 years ago
Lukas Sismis ee4f75e4b4 dpdk: port deprecated DPDK macros to the newer forms 4 years ago
Lukas Sismis 79130103c2 dpdk: print debug xstats counters of all DPDK ports on shutdown 4 years ago
Lukas Sismis cb6fa894ef dpdk: add a check for HW checksum validation offload
Ticket: #5553
4 years ago
Victor Julien da1ad843b4 time: -Wstrict-prototypes fix 4 years ago
Jeff Lucovsky 9fbe683642 time: Rework SCTime_t into a struct
Issue: 5718

This commit changes SCTime_t to a struct with members setup as
bitfields.
4 years ago
Jeff Lucovsky 31793aface time: Replace struct timeval with scalar value
Issue: 5718

This commit switches the majority of time handling to a new type --
SCTime_t -- which is a 64 bit container for time:
- 44 bits -- seconds
- 20 bits -- useconds
4 years ago
Jeff Lucovsky 7702ad410e format/clang: Apply clang-format
Issue: 5718
4 years ago
Victor Julien c15d511064 frames: enable only used frames
Enable only frames that are actually used.

Ticket: #4979.
4 years ago
Victor Julien aa376a3b21 detect/frame: improve frame detection
Add a per frame progress tracker.
4 years ago
Victor Julien 169ee11ead output/frame: log frame type stream 4 years ago
Victor Julien d72bc364de output/frame: improve 'complete' logging 4 years ago
Victor Julien 6cbb5306c6 frame: add debug validation check 4 years ago
Victor Julien b43dc5a64a app-layer/frames: use absolute frame offset
Frame offset was already a 64 bit integer, so simplify things by
making it an absolute offset from the start of the stream.
4 years ago
Victor Julien 39d9b3adbe frames: implement generic <alproto>.stream frames
Add a hard coded <alproto>.stream option for all stream data for
a protocol.

Starts at stream offset 0 or at the point of a protocol upgrade
in case of STARTTLS or CONNECT.
4 years ago
Victor Julien f773b714e9 detect/frames: minor cleanup in buffer handling
Don't lookup a buffer twice, even if the lookup should be fast.
4 years ago
Victor Julien d0f1507c83 htp: simplify streaming buffer config
Use a single static config instead of the per profile config.
4 years ago
Victor Julien c79c0ca347 streaming: remove config pointer from struct 4 years ago
Victor Julien 53d9a1f39f streaming: internal switch to sbcfg by argument 4 years ago
Victor Julien ff882cd73f streaming: add sbcfg to StreamingBufferClear 4 years ago
Victor Julien 6e5f35e0a0 streaming: add sbcfg to StreamingBufferFree 4 years ago
Victor Julien 96f29440cb streaming: add sbcfg to StreamingBufferAppendRaw 4 years ago
Victor Julien 30ee9165b5 streaming: add sbcfg to StreamingBufferAppend 4 years ago
Victor Julien 3b5deb4ec7 streaming: add sbcfg to StreamingBufferSlideToOffset 4 years ago
Victor Julien b9540d1073 streaming: add sbcfg to StreamingBufferInsertAt 4 years ago
Victor Julien 058dc02e81 streaming: add sbcfg to StreamingBufferAppendNoTrack 4 years ago
Victor Julien 355f259b8c output/filedata: trunc file in output again 4 years ago
Victor Julien e3e55406a7 files: update API and callers to take stream config
This is to allow not storing the stream buffer config in each file.
4 years ago
Victor Julien f7dbdb7631 output/filedata: don't call file close
Will be reenabled after file API is updated.
4 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 4 years ago
Victor Julien d57510a10f files: remove unused Rust binding for file pruning 4 years ago
Victor Julien f19b40a7f6 streaming: set size and max regions defaults 4 years ago
Victor Julien 229b82721d htp: remove duplicate prototypes 4 years ago
Victor Julien f788d31f22 htp/body: minor optimization 4 years ago
Victor Julien b62a513c47 flow: inline commonly used getters 4 years ago
Victor Julien 76a256a8b1 streaming: remove inefficient buffer grow logic 4 years ago
Victor Julien 467234659d streaming: remove unused slide logic 4 years ago
Victor Julien 0b36bde9be streaming: remove unused config member 4 years ago
Victor Julien 2cfbefb6c6 streaming: make minimum region gap size configurable 4 years ago
Victor Julien 8e9dac99f3 stream/reassembly: make max-regions configurable; set default 4 years ago
Victor Julien f896f03b7b streaming: add max regions config option 4 years ago
Victor Julien 42d3cd2061 stream/list: hack around GAP handling in tests 4 years ago
Victor Julien d2001ef94b stream: improve gap tests 4 years ago
Victor Julien 96dfd65b96 eve: log max regions 4 years ago
Victor Julien 1bb6f44ff0 stream: flow timeout improvement
Check continuous data for app-layer and post gap data as well for
stream inspection.
4 years ago
Victor Julien 1dac2467c5 streaming: implement memory regions
In TCP, large gaps in the data could lead to an extremely poor utilization
of the streaming buffer memory. This was caused by the implementation using
a single continues memory allocation from the "stream offset" to the
current data. If a 100 byte segment was inserted for ISN + 20MiB, we would
allocate 20MiB, even if only 100 bytes were actually used.

This patch addresses the issue by implementing a list of memory regions.
The StreamingBuffer structure holds a static "main" region, which can be
extended in the form of a simple list of regions.

    [ main region ] [ gap ] [ aux region ]
    [ sbb ]                 [ sbb ]

On insert, find the correct region and see if the new data fits. If it
doesn't, see if we can expand the current region, or than we need to add
a new region. If expanding the current region means we overlap or get
too close to the next region, we merge them.

On sliding, we free any regions that slide out of window and consolidate
auxilary regions into main where needed.

Bug: #4580.
4 years ago
Victor Julien 61e47ad6f5 stream: reduce streaming buffer internals use 4 years ago
Philippe Antoine e22b59b8c1 fuzz: check libpcap timestamp consistency
That is microseconds should be positive
4 years ago
Philippe Antoine 1c436fe0ca fuzz: fix use of uninitialized value
packet timestamp is not set when function returns error.
also use C positive modulo for microseconds
4 years ago
Philippe Antoine 1660172a8b ftp: completely resets port_line
In the case port_line is first allocated and port_line_len is set,
Then a second request reaches memcap and frees port_line,
port_line_len should also be reset, because both will get used
by the response parsing.

Ticket: #5701
4 years ago
Victor Julien b79c14f710 flow-worker: prune frames and stream for pseudo packets as well 4 years ago
Victor Julien af211cbfad debug: add pkt_src to detect/stream entry log 4 years ago
Victor Julien ef2ed35311 stream: add debug statements 4 years ago
Victor Julien db28d85d01 stream: consider any uninspected data 4 years ago
Victor Julien 1a35801f96 stream: minor constification 4 years ago
Victor Julien 989e5c7ba2 streaming/sbb: fix debug message 4 years ago