Avoid looping in transaction output.
Update app-layer API to store the bits in one step
and retrieve the bits in a single step as well.
Update users of the API.
Decoding client certificate overwrites the validity dates from the
server certificate, so we therefore don't decode it, since we don't
do anything with it (right now) anyway.
Fixes Bug #2050
TCP reassembly is now deactivated more frequently and triggering a
bypass on it is resulting in missing some alerts due forgetting
about packet based signature.
So this patch is introducing a dedicated flag that can be set in
the app layer and transmitted in the streaming to trigger bypass.
It is currently used by the SSL app layer to trigger bypass when
the stream becomes encrypted.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
In various scenarios buffers would be checked my MPM more than
once. This was because the buffers would be inspected for a
certain progress value or higher.
For example, for each packet in a file upload, the engine would
not just rerun the 'http client body' MPM on the new data, it
would also rerun the method, uri, headers, cookie, etc MPMs.
This was obviously inefficent, so this patch changes the logic.
The patch only runs the MPM engines when the progress is exactly
the intended progress. If the progress is beyond the desired
value, it is run once. A tracker is added to the app layer API,
where the completed MPMs are tracked.
Implemented for HTTP, TLS and SSH.
We assume session resumption has occurred if the Client Hello message
included a session id, we have not seen the server certificate, but
we have seen a Change Cipher Spec message from the server.
Previously, these transactions were not logged at all because the
server cert was never seen.
Ticket: https://redmine.openinfosecfoundation.org/issues/1969
When registering a probing parser allow to_server and
to_client parsers to be registered. Previously the
probing parser may be called for both directions which
in some cases works OK, but in others can cause
the to_client side to be detected as failed.
Tls packets may contain several records. This increase the number
of allowed records per packet from 30 to 255, and adds a new and
more informative decoder event when this limit is reached.
No extensions are allowed in <TLSv.1.2, so don't trigger SURICATA
TLS handshake invalid length decoder event when no extensions are
specified in CLIENT HELLO.
To be able to add a transaction counter we will need a ThreadVars
in the AppLayerParserParse function.
This function is massively used in unittests
and this result in an long commit.
Change AppLayerParserRegisterGetStateProgressCompletionStatus to
only store one ProgressCompletionStatus callback function for each
alproto, instead of storing one for each ipproto.
This enables us to use AppLayerParserGetStateProgressCompletionStatus
in functions where we do not know the ipproto used.
On bad traffic the parser could allocated memory that was not
intialized. This was later used in the JSON output logging as
a valid null terminated string.
The OpenSSL implementation of RFC 6520 (Heartbeat extension) does not
check the payload length correctly, resulting in a copy of at most 64k
of memory from the server (ref: CVE-2014-0160).
This patch adds support for decoding heartbeat messages (if not
encrypted), and checking several parts (type, length and padding).
When an anomaly is detected, a TLS event is raised.
For AppLayerThreadCtx, AppLayerParserState, AppLayerParserThreadCtx
and AppLayerProtoDetectThreadCtx, use opaque pointers instead of
void pointers.
AppLayerParserState is declared in flow.h as it's part of the Flow
structure.
AppLayerThreadCtx is declared in decode.h, as it's part of the
DecodeThreadVars structure.
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].
Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
"dnsudp" in the rules. A user who previously wrote a rule like this -
"alert dnstcp....." or
"alert dnsudp....."
would now have to use,
alert dns (ipproto:tcp;) or
alert udp (app-layer-protocol:dns;) or
alert ip (ipproto:udp; app-layer-protocol:dns;)
The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0. This is
needed by unittests, which would try to clean the flow, and that would
call the api, AppLayerParserCleanupParserState(), which would try to
clean the app state, but the app layer now needs an ipproto to figure
out which api to internally call to clean the state, and if the ipproto
is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
1. Proto detection
2. Parsers
For app layer protocols.
libhtp has now been moved to the section under app-layer.protocols.http,
but we still provide backward compatibility with older conf files.
The parser wasn't carrying out a bounds check on record length while
in the middle of parsing a handshake. As a result we would step onto the
next record header and consider it a part of the current handshake.
- Contains an unittest to test the issue.
- Disable the duplicate parser unittest registration.
The issue came to light through an irregular ssl record, which was
reported by Sebastian Roschke, via CVE-2013-5919.
Thanks to Sebastian Roschke for reporting this issue.
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
When multiple certificates forming a chain are sent. A pointer to
the start of each certificate is kept. This will allow treatment
on certificates chains.
This patch adds a TLS store option to save certificate in PEM format.
Each time the store action is met, a file and a metafile are created.
Reworked-by: Eric Leblond <eric@regit.org>
Since we now parse the content of the TLS messages, we need to handle
the case multiple messages are shipped in a single TLS record, and
taking care of the multiple levels of fragmentation (message, record,
and TCP).
Additionally, fix a bug where the parser state was not reset after an
empty record.
Change the function to return the number of bytes processed, and fix a bug
where the input buffer was wrong.
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
Decode the SERVER_HELLO message to extract the ciphersuite and compression
chosen by the server.
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
Add a decoder for the SERVER_CERTIFICATE during a TLS handshake, extracts the
certificates and keep the subject name.
Add the tls.subject keyword for substring match in rules (TLS layer).
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
Removal of per flow 'aldata' array. It contained a ptr for each ALPROTO. Instead now we have 2 ptrs in the flow: alparser and alstate.
Various cleanups and dead code removal from the app layer API.
Should safe 100+ bytes memory per flow on 64 bit.
Updated lots of unittests to reflect these changes.
Victor Julien
Eric Leblond
Mats Klepsland
Ray Ruvinskiy
Jason Ish
Giuseppe Longo
Jason Ish
Will Metcalf
Pierre Chifflier
Anoop Saldanha
Last G
Jean-Paul Roliers
Anoop Saldanha
Nikolay Denev
Gurvinder Singh
Pablo Rincon