Register loggers for to server and to client so requests
and responses can be logged independently of each other.
This results in the request log having the actual timestamp of
the request instead of the reply.
When the string representation of a JSON message grew bigger than
64k, the JSON record would just be truncated. This lead to errors
in the parser(s) of the JSON stream.
This patch changes the buffer logic to grow the buffer on demand.
packets. Where rcode isn't "no error" this is displayed in both DNS and
JSON logs.
Note that this changes the current "No such domain" to "NXDOMAIN" in DNS
logs. This could be fixed if desired to maintain compatibility with
anybody crazy enough to parse the DNS log.
When the rcode is not "no error" (for example NXDOMAIN or SERVFAIL) it
is unlikely that there will be answer RRs. Therefore the rname from the
query is used.
Because the rcode applies to a whole answer packet (not individual
queries) it is impossible to determine which query RR caused the error.
Because of this most DNS servers currently reject multiple queries per
packet. Therefore each query RR is output instead with the relevant
error code, likely to be FORMERR if queries > 1.
The TM_FLAG_LOGAPI_TM flag indicates that a module is run by the log
api, not by the 'regular' thread module call functions.
Set flag in all all Log API users' registration code.
Purpose of this flag is in profiling. In profiling output it will be
used to list log api thread modules separately.
This patch adds an event_type key to the generated events. Current
value is one of "dns", "alert, "file", "tls", "http", "drop". It is
then easy to differentiate in log analysis tools the events based on
source inside Suricata.
Without this patch DNS answers for a single query are stored in a
single json event. The result is an array in the object like this one:
{"type":"answer","id":45084,"rrname":"s-static.ak.facebook.com","rrtype":"CNAME","ttl":734},
{"type":"answer","id":45084,"rrname":"s-static.ak.facebook.com.edgekey.net","rrtype":"CNAME","ttl":1710},
This type of output is not well supported in logstash. It is
displayed as it is written above and it is not possible to
query the fields.
I think the reason is that this is not logical if we consider search
query. For example if we search for "rrname" equal "s-static.ak.facebook.com"
we got one entry with two values in it. That's against the logic
of event. Furthermore, if we want to get a complete query, we can
used the id.
This patch splits the answer part in mulitple message. The result
is then accepted by logstash and fields can be queried easily.
This patch updates DNS field name to be in sync with RFC 2629:
https://github.com/adulau/pdns-qof
This will allow to easily use Suricata with other passive DNS tools.
Victor Julien
Tom DeCanio
Jason Ish
maxtors
David Cannings
Ken Steele
Eric Leblond