This patch replaces PacketPseudoPktSetup by a better named
PacketTunnelPktSetup function which is also in charge of doing
the decoding of the tunneled packet.
This allow to clean the code. But it also fixes an issue.
Previously, if the DecodeTunnel function was failling (cause of
an invalid packet mainly), the result was that the original packet
to be considered as a tunnel packet (and not inspected by payload
detection).
In some cases, the decoding is not possible and some really invalid
packet can be created. This is in particular the case of tunnel. In
that case, it is more interesting to forget about the tunneled
packet and only consider the original packet.
DecodeTunnel function is maked as warn_unused_result because it is
meaningful for the decoder to know if the underlying data were not
correct. And in this case, only focus detection on the content.
Fix sigatures not supporting [10.0.0.0/24, !10.1.1.1] notation when
used directly in a rule instead of through a variable.
Add tests for Bugs #815 and #920.
When inspecting HTTP bodies there are several limits involved.
In this patch the reaching of the body limit will trigger body
inspection.
Without this, the body would only be inspected when inspection
limits "request-body-minimal-inspect-size" or
"response-body-minimal-inspect-size" were reached. If the body
limit was smaller than this value, the body would only be
inspected at the end of the tx or stream.
Use the new GetIfaceOffloading function to display a warning message
if pcap capture is used on Linux with GRO or LRO activated. This is
helpful for kernel after 2.6.31 were pcap will use mmaped capture.
TPACKET_V2 is used and this limit the size of the packet resulting
in truncated packets when merged packets are received.
When the PKT_NOPAYLOAD_INSPECTION flag is set, don't apply it to smsgs.
This way we can still inspect the outstanding smsgs.
The PKT_NOPAYLOAD_INSPECTION is set for encrypted traffic, and is combined
with disabling stream reassembly. So we only inspect the smsgs up to the
point of the disable detection point.
When checking the reassembly limit for raw reassembly, consider the
STREAMTCP_STREAM_FLAG_NOREASSEMBLY a trigger immediately. We won't
process any more segments in the reassembly engine anyway.
The meta-field-option allows for setting the hard limit of request
and response fields in HTTP. In requests this applies to the request
line and headers, not the body. In responses, this applies to the
response line and headers, not the body.
Libhtp uses a default limit of 18k. If this is reached an event is
raised.
Ticket 986.
In the SSE 4.2 SCMemcmpLowercase implementation, there would be a
_mm_load_si128 of a 2 byte array. However, _mm_load_si128 loads
16 bytes, causing it to read beyond the var. I don't think this lead
to crashes, as it was a static var, but clangs ASAN complained about
it.
Share memory space for IPV4Vars and (IPV6Vars, IPV6ExtHdrs), since a
packet can only be either IPv4 or IPv6, but not both.
Share memory for TCPVars, UDPVars, ICMPV4Vars and ICMPV6Vars, since a
packet can only be only of these.
Then move other structure members around to remove holes reported by pahole.
This reduces the size of the Packet structure from 2944 bytes (46 cachelines)
down to 1976 (31 cachelines), a 33% reduction.
Strip the 'proxy' parts from the normalized uri as inspected by http_uri,
urilen, pcre /U and others.
In a request line like:
GET http://suricata-ids.org/blah/ HTTP/1.1
the normalized URI will now be:
/blah/
This doesn't affect http_raw_uri. So matching the hostname, etc is still
possible through this keyword.
Additionally, a new per HTTP 'personality' option was added to change
this behavior: "uri-include-all":
uri-include-all: <true|false>
Include all parts of the URI. By default the
'scheme', username/password, hostname and port
are excluded. Setting this option to true adds
all of them to the normalized uri as inspected
by http_uri, urilen, pcre with /U and the other
keywords that inspect the normalized uri.
Note that this does not affect http_raw_uri.
So adding uri-include-all:true to all personalities in the yaml will
restore the old default behavior.
Ticket 1008.
Move the allocation of the mPipe ingress queue from a loop over
the number of workers in the main init function to being done inside
each worker thread. This allows allocating the memory locally on the
worker's CPU without needing to figure out ahead of time where that thread
will be running. This fixes one case of static mapping of workers to CPUs.
Use __thread to hold the queue rather than a global tables of queues.
pcre_study returning NULL is not necessarily an error, from the man page
pcre_study(3):
"If the function returns NULL, either it could not find any additional
information, or there was an error. You can tell the difference by
looking at the error value. It is NULL in first case."
Older libpcre versions would return NULL, causing errors.
Keep a separate checksum for IPV4, since a packet can have both an IPV4
checksum and a TCPV4 checksum, or IPV4 and UDPV4 checksum.
This will allow future sharing of more values.
Use PACKET_RESET_CHECKSUMS() in Unit Tests in place of setting the
individual checksum values.
When http and/or tls logging is disabled, the app layer would still
be flagged as logging. This caused transactions not to be freed until
the end of the flow as the logged tx id would never increment.
This fix postpones the setting of the app layer parser "logger"
flag to the point where we know the logger is enabled.
Eric Leblond
Victor Julien
Giuseppe Longo
Jason Ish
Ken Steele