As input, input_len and direction only last for the scope of
one call of AppLayerParserParse, it is not necessary to keep them
in FtpState which lives longer, so we consume less memory.
Set correct direction for PORT mode, where the server connects
to the client.
The direction is not also strictly enforced. No data in the wrong
direction will be accepted to setup the file or to be added to the
file after setup.
This also fixes files getting closed twice.
Adds some general cleanups.
Bug: #3542.
The first segment was not limited to the configured maximum line length
allowing it to be up to 65k. This could result in the next input length
being negative, which while handled properly by the code, did trigger a
debug validation assertion.
The fix is to be consistent and apply the limit to the first segment as
well, which does ensure the input_len could never be less than 0.
Ticket #5281
FTP control commands will be buffered forever until a new line is seen,
this can lead to memory exhaustion in Suricata.
To fix, set an upper bound, 4096 bytes on the size of the command that
is saved in the transaction. The input continues to be parsed to find
the end of the command so the parser can continue to move onto the next
command.
The result is that the command data in the transaction is truncated,
which also shows up in the ftp transaction logs.
This value is configurable with the max-line-length field in the ftp
app-layer.protocols section.
As FTP doesn't have events at this time, add a new fields to eve-log
that specificy if the request, or the response has been truncated.
Ticket #5024
Ticket: 4530
As for HTTP2 and MQTT.
In FTP case, transactions are pipelined, not identified by an id.
So, there are less chances of DOS by quadratic complexity.
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.
This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.
Update all parsers to take advantage of this.
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2
This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
For a done transaction with command PORT,
we expect FTP_STATE_FINISHED
and we got FTP_STATE_PORT_DONE instead
which prevented logging of these transactions
We change the order of the evaluations to get the right result
Protects against evasion by TCP packet splitting
The problem arised if the FTP response is split on multiple packets
The fix is to bufferize the content, until we get a complete line
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.
Introduce two macros:
APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)
Update all parsers to use this.
Ensure that RETR (STOR) have a filename -- otherwise, treat the command
string as malformed.
Added unittests for each command and verified that SEGV's occur without
parser change and no longer occur with the parser change.
If a keyword like filemd5 was being used without a filestore,
or a file output enabled, it would be pruned before detection
had a chance to match.
Consolidate file pruning to the end of the flow worker so files
are available for detection even when a file output is not
enabled.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2490