Put common counters on the first cache line. Please the flow output
pointer last as it's use depends on the flow logging being enabled
and even then it's only called very rarely.
With the previous code the order of the logging modules in the
YAML were determining which module was run first. This was not
wished and a consequences was that the EVE fileinfo module was
not correctly displaying the key 'stored' because it was
depending on a flag set alter by the filestore module.
This patch adds a priority file to the TmModule structure. The
higher the priority is set, the sooner the module is run in the
logging process. The RunModeOutput structure has also been
updated to contain the name of the original TmModule. Thus allowing
to define a priority for a RunModeOutput.
Currently only the filestore has a priority set. The rest of them is
set to the default value of zero.
Modbus parser does not check length to extract/read data (read or write address,
quantity of data, etc.) that should be present.
In case of malformated data (invalid length in header), Modbus parser reads data
over the input data length.
Add check before extracting/reading data from input buffer to avoid head buffer
overflow.
Use proper LogFileCtx and MemBuffer handling so we can have multiple
loggers active at the same time.
Change 'date' field to timestamp, and use ISO notation to make it
the same as the other JSON outputs.
Merge counters so the table contains combined values from counters
from each thread.
Use global counter id's, track them in a hash.
Rename SCPCAElem members
Fix and improve average counters
It uses the new type field in the LogFileCtx instead.
This fixes the problem of not being able to use two eve-json
instance with different logging methods.
The cast of data to AlertJsonThread was not correct as the real
type of the void pointer is a OutputJsonCtx. This was working by
luck because they both have a file_ctx as first element.
Without support for OPT RR from RFC6891 (Extension mechanisms for DNS)
values of RCODE above 15 are not possible. Remove dead code which will
never match.
An IPv6 entry specified before an IPv4 entry on the host-os-policy
table can cause the stream byte array to be access one byte after
the end of the allocated memory at util-radix-tree.c:578.
Remove all strtok uses and replace them by strtok_r.
Do the same for Windows builds. Cygwin builds fine with strtok_r.
Add strtok to banned function list.
Due to an error at initialization, the stream engine would not disable
'raw' reassembly automatically when --disable-detection was used.
This lead to segments not getting cleared from the segment lists.
Instead, intruduce StreamTcpDisableAppLayer to disable app layer
tracking and reassembly. StreamTcpAppLayerIsDisabled can be used
to check it.
Replace all uses of FlowSetSessionNoApplayerInspectionFlag and
the FLOW_NO_APPLAYER_INSPECTION.
When the session/flow was flagged as 'no applayer inspect', which
could happen as a result various reasons, packets would still be
considered by the app layer reassembly.
When ACK'd, they would be removed again. Depending also on the raw
reassembly.
In very long sessions however, this meganism could fail leading to
virtually endlessly growing segment lists.
This patch makes sure that segments that come in on a 'no app layer'
session are tagged properly or even not added at all.
Use a new ssn flag instead of flow flag for no app tracking.
If for a packet we have a TX N that has detect state and a TX N+1 that
has no detect state, but does have 'progress', we have a corner case
in stateful detection.
ContinueDetection inspects TX N, but cannot flag the rule in the
de_state_sig_array as the next (TX N+1) has already started and needs
to be inspected. 'StartDetection' however, is then unaware of the fact
that ContinueDetection already inspected the rule. It uses the per
session 'inspect_id' that is only moved forward at the end of the
detection run.
This patch adds a workaround. It uses the DetectEngineThreadCtx::
de_state_sig_array to store an offset between the 'base' inspect_id
and the inspect_id that StartDetection should use. The data type is
limited, so if the offset would be too big, a search based fall back
is implemented as well.
Fix storing state and bypassing detection. Previously we'd store
on a match only, meaning that StartDetection would rerun often.
Make sure StartDetection only stores if there is something to store.
Previously we could never be calling DetectEngineHHDGetBufferForTX
for TX N and then afterwards for TX N - 1. Due to changes in the
stateful detection code this is now possible.
This patch changes the buffer logic to take the 'inspect_id' as it's
base, instead of the first transaction that we are called with.
When using engine analysis for print fast_pattern stats, print a
short summary at the end containing per buffer:
- smallest fp
- biggest fp
- number of patterns
- avg fp len
If TmThreadsUnregisterThread was called with out of range 'id', a lock
would not be cleared after returning from the function.
** CID 1264421: Missing unlock (LOCK)
/src/tm-threads.c: 2186 in TmThreadsUnregisterThread()
The code was assuming that the dictionnary containing the parameter
of a interface was ordered. But for YAML, the order is not assumed
so in case the configuration is generated we may not be able to
parse correctly the configuration file.
By iterating on child on main node and then iterating on subchild
and doing a match on the name, we are able to find the interface
list. In term of code, this algorithm was obtained by simply
removing the test on the name of the first child.
This checks if the signature's protocol is http
when setup the content keyword.
Also sets the proper flags based by protocol
since the flag SIG_FLAG_TOSERVER has to be set
if the proto is smtp, otherwise SIG_FLAG_TOCLIENT
is it's http.
This commit do a find and replace of the following:
- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*
- HSBD by FILEDATA:
sed -i 's/HSBDMATCH/FILEDATA/g' src/*
Wrap lines longer than 80 characters
Add "static" for unit tests.
Use (void) for () for function arguments.
Add space after "while(" -> "while ("
Remove space after function names.
Put open bracket of function on a new line.
I see two problems:
1) If allocating a newlist fails, the function returns NULL, which then
leaks any existing list elements.
2) The code to add the new value to the list works for the first two, but
for not the third. For example, replist=A, A->next=B, B->next=NULL, then
adding C results in replist=A, A->next=C, C->next=NULL, B is lost.
The fix pushes new values onto the head of the list, which might not be
what is needed, but there are no comments on what the function does, so I
made an assumption.
Allow next_seq updating to recover from cases where last_ack has been
moved beyond it. This can happen if ACK's have been accepted for missing
data that is later retransmitted.
This undoes some of the previous last_ack update changes
packets. Where rcode isn't "no error" this is displayed in both DNS and
JSON logs.
Note that this changes the current "No such domain" to "NXDOMAIN" in DNS
logs. This could be fixed if desired to maintain compatibility with
anybody crazy enough to parse the DNS log.
When the rcode is not "no error" (for example NXDOMAIN or SERVFAIL) it
is unlikely that there will be answer RRs. Therefore the rname from the
query is used.
Because the rcode applies to a whole answer packet (not individual
queries) it is impossible to determine which query RR caused the error.
Because of this most DNS servers currently reject multiple queries per
packet. Therefore each query RR is output instead with the relevant
error code, likely to be FORMERR if queries > 1.
If a boundary was longer than 254 bytes a stack overflow would result
in mime decoding.
Ticket #1449
Reported-by: Kostya Kortchinsky of the Google Security Team
A bad timestamp would lead to SCLocalTime returning NULL. This case
wasn't checked, leading to a NULL deref.
Reported-by: Kostya Kortchinsky of the Google Security Team
Implement LINKTYPE_NULL for pcap live and pcap file.
From: http://www.tcpdump.org/linktypes.html
"BSD loopback encapsulation; the link layer header is a 4-byte field,
in host byte order, containing a PF_ value from socket.h for the
network-layer protocol of the packet.
Note that ``host byte order'' is the byte order of the machine on
which the packets are captured, and the PF_ values are for the OS
of the machine on which the packets are captured; if a live capture
is being done, ``host byte order'' is the byte order of the machine
capturing the packets, and the PF_ values are those of the OS of
the machine capturing the packets, but if a ``savefile'' is being
read, the byte order and PF_ values are not necessarily those of
the machine reading the capture file."
Feature ticket #1445
Coverity:
** CID 1296115: Program hangs (ORDER_REVERSAL)
/src/tm-threads.c: 1670 in TmThreadClearThreadsFamily()
The problem is with the by default unused '%m' output parameter.
To get the thread vars it takes the tv_root_lock. This may already
be locked by the calling thread. Also, it could lead to a case of
wrong lock order between the tv_root_lock and the thread_store_lock.
Very unlikely to happen though.
As the %m param isn't really used (by default) this patch just
disables it.
Don't kill flow manager and recyclers before the rest of the threads. The
packet threads may still have packets from their pools. As the flow threads
would destroy their pools the packets would be lost.
This patch doesn't kill the threads, it just pulls them out of their run
loop and into a wait loop. The packet pools won't be cleared until all
threads are killed.
Wait for flow management threads to close before moving on to the
next steps in the shutdown process.
Don't destroy flow force reassembly packet pool too early. Worker
threads may still want to return packets to it.