A bad timestamp would lead to SCLocalTime returning NULL. This case
wasn't checked, leading to a NULL deref.
Reported-by: Kostya Kortchinsky of the Google Security Team
This patch updates the timestamp format used in eve loggin.
It uses a ISO 8601 comptatible string. This allow tools parsing
the output to easily detect adn/or use the timestamp.
In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.
Logstash configuration is simple:
input {
file {
path => [ "/usr/local/var/log/suricata/eve.json" ]
codec => json
type => "suricata-log"
}
}
filter {
if [type] == "suricata-log" {
date {
match => [ "timestamp", "ISO8601" ]
}
}
}
In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:
[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0
Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:
[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata
In both cases the timestamp are correctly imported by
the tools.
Reduced the size of the cached string buffer from 128 to 32, which is
still larger than the largest possible time string, which is 26
characters.
Added a check for the user passing in an output buffer that is smaller
than the cached string. Previously, the code would have copied past
the end of the users buffer.
When converting a time in seconds (64-bit seconds since 1970) to
Month/Day/Year hours minutes, Suricata calls localtime_r(), which
always aquires a lock and then does complex comutation based on the
current time zone. The time zone can be specified in the TZ
environment variable, which is only parsed the first time it is used,
or from a file. The default file is /etc/localtime. The file is
checked each time to see if it might have changed and is reparsed if
it has changed.
The GLIBC library has a lock inside localtime_r(), which limits
parallelism, which is a problem when the rate of generating alerts is
high, since Suricata generates a new ascii time string for each alert
into fast.log.
This change caches the value returned by localtime_t() and then sets
the seconds within the minute based on the cached start-of-minute
time. All of the values return, expect for the seconds, is constant
within the same minute. Switching to a new seconds could change all
the other values, year, month, day, hour. The cache stores the current
and previous minute values.
The same trick is used in CreateTimeString() for generated time
string. The string, up to the minutes, is cached and then copied into
the result string, followed by printing the new seconds into the
result string.
The seconds within a minute are calculated as the difference in
seconds from the start of the current minute.
There were 8 identical copies of CreateTimeString() in 8 files.
Most used SCLocalTime, to replace localtime_r(), but some did not.
Created one copy in util-time.c.
This function is a wrapper to localtime_r. It is needed to avoid
a compilation warning on OpenBSD. I'm forced to type the function
to a non pointer first parameter. If not we will have to use two
differents functions in OpenBSD where tv->tv_sec is a long
(different from time_t).
- make sure before each unittest is run the time is reset
- add functions to set the time to current time and increment the time
- convert alert-unified* Rotate tests to use them
- convert time based counters to use them
- use GetTime instead of gettimeofday for creating the unified* filenames
- Implement "closing" state in flow.
- Add protocol specific timeouts.
- Lots of stream tracking updates, fixing a lot of out of window issues.
- Stream reassembly fixes.
- Implement a new IDS runmode with 4 stream and detect threads.
- Added a BUG_ON macro that aborts the engine if the expression is true.
- Better balance the flow queue handler for traffic that doesn't have flow (like icmp currently).
- Simplify application level protocol in the Tcp Session.
- Add some debugging memory counters.
Victor Julien
Eric Leblond
Ken Steele
William Metcalf
Pablo Rincon
Anoop Saldanha
Brian Rectanus