aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,272 Commits
8 Branches
165 Tags
224 MiB
main-7.0.x
main-8.0.x
main
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'ee5fd4613b'
${ noResults }
Commit Graph

5774 Commits (ee5fd4613bec43c13b65c7bb792ad6ef0f01ffdb)

Author SHA1 Message Date
Victor Julien e755913b4b mpm: minor fixes and cleanups 10 years ago
Victor Julien cacf425bd3 stream: improve handling of GAPs at stream start 
Detect and handle gaps at the start of the stream, when there may
be no segments in the list (yet).
 10 years ago
Victor Julien 574ef0ad2a stream: RST last_ack update fix 
Only use ACK if ACK flag was set and ACK value is valid.
 10 years ago
Victor Julien 3ca44219dc proto detect: more bypass conditions 
More exceptional cases for protocol detection. In very unbalanced flows,
where just a few bytes are sent toserver and many toclient, proto detect
might not complete in time on the toserver direction. This can lead to
queuing up many segments in the toclient direction.

Another case is that in come cases the stream is flagged as proto detect
done, but the flows proto detect flags are not set. This is now handled
by the ProtoDetectDone() check.
 10 years ago
Victor Julien fa8dc77dcc debug validation: add segment list sanity check 10 years ago
Victor Julien e67188e437 detect: fix issue with smsg and seq wraps 
Due to a broken sequence number check, detect could fail to process
smsgs in case of a sequence wrap. This could lead to excessive use
of smsg's but also of segments, since these aren't cleared until the
smsg containing them is.
 10 years ago
Victor Julien 8ac49d9129 stream: allow next_seq catch up after pkt loss 
If next_seq falls behind last_ack, force update it.
 10 years ago
Victor Julien 596465b76d stream: use reassembly fast path after proto detect 
Use the reassembly fast paths only after protocol detection has completed.
In some corner cases the sending of smaller segments lead to protocol
detection failing.
 10 years ago
Victor Julien 34ed15e182 stream: fix protocol detection issue for GAPs 
If the protocol required TOSERVER data first, but the SSN started with
a GAP, then the TOCLIENT side would get stuck in an expensive path:

1. it would run detection on TOCLIENT
2. it would try to force reassembly for TOSERVER
3. it would reset the detected protocol as TOSERVER failed
4. it would not evict any segment

This had 2 consequences:
1. on long running sessions this could lead to using lots of memory
   on segments, denying other sessions resources
2. wasted cycles on protocol detection and segment list management

This patch introduces a fix. It checks in the (2) stage above, whether
the opposing stream (that we depend on) it is a NOREASSEMBLY state. If
so, it gives up on this side of the session as well.
 10 years ago
Victor Julien 708e80c900 stream: optimize proto detect segment handling 
In case of protocol detection not yet being complete, the segment
list was walked unconditionally to unset the app layer processed
flag. Optimize this to bail on the first segment that doesn't have
the flag set.
 10 years ago
Eric Leblond a286715367 host-storage: document host storage API 10 years ago
Jason Ish 06beca62f5 app-layer: template for application layer content inspection 10 years ago
Jason Ish bcda92134d app-layer: template for application layer tx logger 10 years ago
Jason Ish c1b921265f app-layer: template for application layer parser 10 years ago
Victor Julien cb060f16fd detect plugin: add template 10 years ago
Victor Julien 830dd32417 decoder: add template/example 
Add fictional example / template for a packet decoder. It's not invoked
anywhere.
 10 years ago
Victor Julien 371648a8c6 lua: TLS support 
Support TLS in Lua detection scripts.

function init (args)
    local needs = {}
    needs["tls"] = tostring(true)
    return needs
end

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    if version == nil then
        return 0
    end
    str = string.format("Version %s\nIssuer %s\nSubject %s\nFingerprint %s",
                        version, issuer, subject, fingerprint)
    SCLogInfo(str);
    return 1
end
 10 years ago
Victor Julien 1e84950017 detect-lua: set direction 10 years ago
Victor Julien 8b3afb6986 lua: add direction support 10 years ago
Victor Julien a798cf9518 DNS: refactor tx completion logic 
Use simple bool values to track the transaction state in both directions.

A tx is only created in two cases:
1. full request parsed
2. response parsed (request missing)

This is true even for multi-packet TCP requests.

This leads to the following tx completion logic for the request side:
the presence of a tx implies the request is complete

On the response side, we consider the tx complete when we have seen
the response. If the DNS parser thinks the response was lost, we also
flag the response side as complete.
 10 years ago
Victor Julien 6b2fb3fd5d Fix minor format string issues 10 years ago
Victor Julien ba81c4d290 autotools: cleanup 
Remove most of the CFLAGS updates from configure. Flags are now (mostly)
set in AM_CLFLAGS.

Update all -DBLAH additions to CFLAGS to use AC_DEFINE([BLAH], ...)

Improve Lua vs LuaJIT checking.

Improve the configure output a bit.

Lots of smaller cleanups.
 10 years ago
Antti Tönkyrä 63a47967d7 source-pfring: don't set cluster mode when using ZC and VLAN tracking is disabled 10 years ago
gureedo c31660e5fd netmap: enable zero-copy mode only when copy-mode is specified. 
Perform zero-copy checks only when copy-mode is specified.
 10 years ago
Victor Julien cbf46c44ec transaction inspection: fix limit enforcement 
Make sure we're never wrapping around the size value.
 10 years ago
Antti Tönkyrä 834c36659e Allow colon in SSH version, at least some trojaned PuTTY clients have version like Putty-Local: Timestamp HH:MM:SS 10 years ago
Victor Julien 1235c578b2 xff: support ports and more ipv6 notations 
It's not uncommon to see an header like:
X-Forwarded-For: 1.2.3.4:56789

This patch recognizes this case and ignores the port. It also supports
this for IPv6 if the address has the following notation:
X-Forwarded-For: [12::34]:1234

This patch also adds unittests.
 10 years ago
Victor Julien 979bd35277 detect loader: move to own file 10 years ago
Victor Julien cfeaf42cab detect-loaders: configurable amount of loaders 10 years ago
Victor Julien f871c0e1b8 debug: packet pool init/destroy validation 
Validate packet pool handling:
- pools are initialized before use
- pools are not used after destroy
- pools are not double initialized/destroyed
 10 years ago
Victor Julien e44fd97d72 unittests: use a global packetpool 10 years ago
Victor Julien 99c0a7ad72 multi-detect: improve memory handling of setup code 10 years ago
Victor Julien b7b27684c2 multi-detect: detect loader for unix socket 
Move the tenant load and reload commands to be executed by the detect
loader thread(s).

Limitation: no yaml parsing in parallel. The Conf API is currently not
thread safe, so don't load the tenant config (yaml) in parallel.
 10 years ago
Victor Julien eb09118d64 detect: create loader threads 
To speed up startup with many tenants, tenant loading will be parallelized.
As no tempary threads should be used for these memory allocation heavy
tasks, this patch adds new type of 'command' thread that can be used to
load and reload tenants.

This patch hardcodes the number of loaders to 4. Future work will make it
dynamic.

The loader thread essentially sleeps constantly. When a tasks is sent to
it, it will wake up and execute it.
 10 years ago
Victor Julien 82aa419431 multi-detect: set tenant id on pseudo packets 
Store the tenant id in the flow and use the stored id when setting
up pesudo packets.

For tunnel and defrag packets, get tenant from parent. This will only
pass tenant_id's set at capture time.

For defrag packets, the tenant selector based on vlan id will still
work as the vlan id(s) are stored in the defrag tracker before being
passed on.
 10 years ago
Victor Julien e19c41a807 multi-detect: hash lookup for tenants 
Use hash for storing and looking up det_ctxs.
 10 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code 
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
 10 years ago
Victor Julien 4f77e8967b multi-detect: make threshold prefix aware 
Make threshold loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.

Note: currently per host thresholds are tracked globally and NOT per
tenant.
 10 years ago
Victor Julien a6e3cec9e5 multi-detect: make reference prefix aware 
Make reference loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
 10 years ago
Victor Julien 5fff250749 multi-detect: make classification prefix aware 
Make classification loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
 10 years ago
Victor Julien d8181802d3 multi-detect: add reload-tenant command 
Allow for a tenant to be reloaded. The command is the same as the
register-tenant command, so with a yaml and tenant-id as argument.
However this replaces an existing tenant.
 10 years ago
Victor Julien 8673801ea3 multi-detect: add tenant id to alert json output 
Add a integer field "tenant_id" to the JSON alert output.
 10 years ago
Victor Julien f4c9915066 multi-detect: store tenant id in packet 
Store tenant id in the packet so that the output API's can log it.
 10 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant 
Generate error if tentant to be added is already loaded.
 10 years ago
Victor Julien 4e0683c3f5 multi-detect: cleanup, reuse tenant loading code 
Reuse tenant loading from YAML code for Unix Socket.
 10 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file 
Load tenants and mappings from the suricata.yaml when available.
 10 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler 
Remove a tenant handler from the list and apply it.
 10 years ago
Victor Julien 93f856a1b3 detect: don't error out on no de_ctx 
This can happen on a multi-detect setup with no registered
engines yet.
 10 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml 
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
 10 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered 
Force user to select the method at startup.
 10 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx 
Otherwise counters are only registered after the stats api is
already fixed.
 10 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 10 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler 
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
 10 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants 
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
 10 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry 
Limited to Pcap only currently.
 10 years ago
Victor Julien bc7e9be5c6 unix-socket: allow tenant id with pcap-file 
Register the tenant id that the pcap-file optionally got.
 10 years ago
Victor Julien bea66156b9 pcap-file: set tenant-id if available 
Set the id to each packet's 'pcap_v' structure.
 10 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 10 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch 
At start up we will set this flag based on "multi-detect.enabled".
 10 years ago
Victor Julien 3f12bdd4f8 tenants: apply added/removed tenant 
Apply to the engine.
 10 years ago
Victor Julien def2b58725 detect: initial MT lookup logic 
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
 10 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands 
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
 10 years ago
Victor Julien 3aa58f25ad eve alert: fix stream payload printing 10 years ago
Victor Julien 2ef0ebb24b detect: fix pass transaction handling 
If a flow was 'pass'd, it means that no packet of it will flow be handled
by the detection engine. A side effect of this was that the per flow
inspect_id would never be moved forward. This in turn lead to a situation
where transactions wouldn't be freed.

This patch addresses this case by incrementing the inspect_id anyway for
the pass case.
 10 years ago
Victor Julien 06ee2bc87e detect: set flow noinspect on pass in applayer/stream 
If a pass rule matches in the reassembled stream and/or in the
app-layer state, it means the rest of the flow should not be
inspected.
 11 years ago
Victor Julien bbc9874b81 http: destroy htp_tx_t even if incomplete 11 years ago
Victor Julien 814f0b4094 detect: make http prefilter use disrupt flags 11 years ago
Victor Julien b592f98727 detect: optimize http prefilter handing 11 years ago
Victor Julien 6946e0be55 detect: pass flags to inspect_id update logic 11 years ago
Victor Julien c087708fa9 output-tx: use disrupt flags 11 years ago
Victor Julien 00ef789ffc app-layer: pass full flags around in tx handling 11 years ago
Victor Julien ea571add73 app-layer: disruption flags 
Stream GAPs and stream reassembly depth are tracked per direction. In
many cases they will happen in one direction, but not in the other.

Example:
HTTP requests a generally smaller than responses. So on the response
side we may hit the depth limit, but not on the request side.

The asynchronious 'disruption' has a side effect in the transaction
engine. The 'progress' tracking would never mark such transactions
as complete, and thus some inspection and logging wouldn't happen
until the very last moment: when EOF's are passed around.

Especially in proxy environments with _very_ many transactions in a
single TCP connection, this could lead to serious resource issues. The
EOF handling would suddenly have to handle thousands or more
transactions. These transactions would have been stored for a long time.

This patch introduces the concept of disruption flags. Flags passed to
the tx progress logic that are and indication of disruptions in the
traffic or the traffic handling. The idea is that the progress is
marked as complete on disruption, even if a tx is not complete. This
allows the detection and logging engines to process the tx after which
it can be cleaned up.
 11 years ago
Victor Julien 8125e04b39 detect: clean up flag usage 11 years ago
Victor Julien 6e2c90a83b detect: constify some DetectMpmPrefilter args 11 years ago
Victor Julien c419f33f44 dns: fix state progress handling 11 years ago
Victor Julien b0f5f7ee97 app-layer: fix args to state progress calls 11 years ago
Victor Julien 62fa9f09d4 Sync alversion/appversion types 
The app layer state 'version' field is incremented with each update
to the state. It is used by the detection engine to see if the current
version of the state has already been inspected. Since app layer and
detect always run closely together there is no need for a big number
here. The detect code really only checks for equal/not-equal, so wrap
arounds are not an issue.
 11 years ago
Victor Julien 11d3f5f67a http: harden tx inspection code 11 years ago
Victor Julien be2849044b flow/stream: xfer noinspect flags to pseudo pkts 
Set noinspection flags for payloads and packets on flow and stream
pseudo packets. Without these, the pseudo packets could trigger
inspection even though this was disabled for a flow.
 11 years ago
Victor Julien a0899cbe85 detect: optimize Signature layout 11 years ago
Victor Julien baadcab1b8 detect: default to u32 for SigIntId 11 years ago
Eric Leblond 1cd97713c2 file-json: add file_id to message 
This will allow to get the filename and by consequence the file
after a parsing of the EVEV log file.
 11 years ago
Eric Leblond 8fde842f97 af-packet: implement rollover option 
This patch implements the rollover option in af_packet capture.
This should heavily minimize the packet drops as well as the
maximum bandwidth treated for a single flow.

The option has been deactivated by default but it is activated in
the af_packet default section. This ensure there is no change for
old users using an existing YAML. And new users will benefit from
the change.

This option is available since Linux 3.10. An analysis of af_packet
kernel code shows that setting the flag in all cases should not
cause any trouble for older kernel.
 11 years ago
Eric Leblond dc306f3bad af-packet: implement new load balancing modes 
This patch implements the fanout load balancing modes available
in kernel 4.0. The more interesting is cluster_qm that does the
load balancing based on the RSS queues. So if the network card
is doing a flow based load balancing then a given socket will
receive all packets of a flow indepently of the CPU affinity.
 11 years ago
Eric Leblond 851fcef962 af-packet: sync header with latest features 
Sync the replacement define with the latest Linux code.
This patch also updates the detection part in configure.ac
to do a declaration of all fields if the newest features are
not present.
 11 years ago
Aleksey Katargin caa2438b98 netmap: support SW rings 
Netmap uses SW rings to send and receive packets from OS.
 11 years ago
Aleksey Katargin 823167bde2 netmap: strict check for zero copy mode 
Netmap does not guarantees that mmap'ed regions for different interfaces would be the same.
 11 years ago
Aleksey Katargin 5104b02f32 netmap: fixed autofp mode. 
Previous implementation does not work with this mode.
 11 years ago
Aleksey Katargin 491fca468d netmap: support non-equal count of Rx and Tx rings on interface. 
Netmap does not guarantees that NIC will have equal number of transmit and receive rings.
 11 years ago
Schnaffon 6484299701 Use unlikely for error treatment in alert-prelude.c 
When handling error cases on creation of a new idmef field, we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc.
 11 years ago
Victor Julien 334e8656bf introduce fatal error macro's 
Add 'FatalError' and 'FatalErrorConditonal' that will take the same
args as SCLogError.

FatalError logs the error using SCLogError and then exits with return
code EXIT_FAILURE.

FatalErrorOnInit does the same only during init and with
--init-errors-fatal enabled, otherwise it just calls SCLogWarning. So
then the macro returns to the caller.

Implement this for output setup.
 11 years ago
Victor Julien 4dd53c8f03 lua: dns support 11 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 11 years ago
Victor Julien 110e23964e detect: add AppLayerTxMatch call 11 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 11 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 11 years ago
Victor Julien a1e50b3138 lua: dns extensions 
Add DNS lua calls for getting queries, answers, authorities. Also
rcode and recusion desired flags.
 11 years ago
Victor Julien c46d472921 lua: initial DNS logging support 11 years ago
Eric Leblond 159a6d1cb4 tls-store: avoid log flooding 
In case we can't write in the certs directory, this is possible
we flood the log for each TLS session or even worse each TLS
packet.  So this patch puts a limit in the number of logged
messages related to file creation.
 11 years ago
Eric Leblond cbf5d88447 filestore: use SCFree instead of free 11 years ago
Eric Leblond b77cd22b98 tls-store: backward compatibility 
This patch implements backward compatibility in suricata.yaml
file. In case the new 'tls-store' output is not present in the
YAML we have to use the value defined in 'tls-log'.
 11 years ago
Eric Leblond 4db0a35f25 tls-store: now a separate module 
An design error was made when doing the TLS storage module which
has been made dependant of the TLS logging. At the time there was
only one TLS logging module but there is now two different ones.

By putting the TLS store module in a separate module, we can now
use EVE output and TLS store at the same time.
 11 years ago
Jeff Barber 893fc9660d Support for reconnecting unix domain socket log files 
Issue #1423
 11 years ago
Jason Ish b512580bbe logging: integrate rotation into SCConfLogOpenGeneric. 
Addresses issue 1492, and will make it harder to omit
rotation on new outputs.
 11 years ago
Jason Ish 14981cb2a8 rule vars: strip leading white space before looking up var. 11 years ago
sfd e58cfb6a05 Fix compile bad dereferences 
The src/source-erf-dag.c code was not compiling. It looks like some stats counters were added but not tested as the dereferences are incorrect.
 11 years ago
Zopieux cd038419fd stream_size operator comparison (fix issue #1488) 
`DetectStreamSizeParse` was first checking if mode[0] is '<', which is true for both '<' and '<=', thus '<=' (and resp. '>=') is never matched. This patch does the `strcmp` to '<=' (resp. '>=') within the if block of '<' (resp. '>') to fix #1488.
 11 years ago
Victor Julien 45fc619f79 logging: json output 
Make JSON output optional.

Cleanups.
 11 years ago
Victor Julien 126ecb3ebf logging: fix per output log formats 11 years ago
Victor Julien b51c4e608f logging: optional colors output 
Construct message per output method.
 11 years ago
Victor Julien b13de5bf08 logging: change newline handling 11 years ago
Victor Julien 1927b3000c output: cleanup 11 years ago
Victor Julien b30bdc21b5 logging: cleanup output API 
Make SCLogMessage master of the logging. Reduces complexity
of the SCLog macro's.
 11 years ago
Victor Julien d6fc6e874f log: reorganize SCLogOPIfaceCtx to make it more efficient 11 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 11 years ago
Alexander Gozman cd9cc2559e Issue 1491: fix capabilities for pf_ring mode when running under non-root account 11 years ago
Alexander Gozman d36eba4e5e Fix issues #1493 and #1494 11 years ago
Giuseppe Longo d592d57039 file_data: check for signature alproto and flow 
Currently the following rule can't be loaded:
alert tcp any any -> any 25 (msg:"SMTP file_data test"; flow:to_server,established; file_data; content:"abc";sid:1;)
and produces the error output:
"Can't use file_data with flow:to_server or from_client with http or smtp."

This checks if the alproto is not http in a signature,
so permits to use flow keyword also.

Issue reported by rmkml.
 11 years ago
Victor Julien e583de0582 Minor unittest cleanups 11 years ago
Victor Julien f4f53924bb app-layer: fix coverity warnings 11 years ago
Victor Julien 6c792cb4cc erspan: respect vlan.use-for-tracking setting 11 years ago
Victor Julien b8211e8c04 htp: hide BUG_ON's behind DEBUG_VALIDATION 11 years ago
Victor Julien bd73553027 smtp json: fix potential crash on malloc failure 
** CID 1298888:    (FORWARD_NULL)
/src/output-json-email-common.c: 117 in JsonEmailLogJson()
/src/output-json-email-common.c: 140 in JsonEmailLogJson()
 11 years ago
Victor Julien 1ed8d7b538 detect analizer: fix minor coverity warning 
** CID 1298889:  Integer handling issues  (NEGATIVE_RETURNS)
/src/detect-engine-analyzer.c: 102 in EngineAnalysisFP()
 11 years ago
Victor Julien dd2afd51f7 file_data smtp: fix minor coverity warning 
CID 1298891:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "curr_file" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
 11 years ago
Victor Julien bcff35fd9f smb: fix coverity warning 
** CID 1298892:  Incorrect expression  (UNUSED_VALUE)
/src/app-layer-smb.c: 1486 in SMBProbingParser()
 11 years ago
Victor Julien aa095864d3 pfring runmode: remove set that is never read 11 years ago
Victor Julien 21db5ee691 counters: reduce global usage 11 years ago
Victor Julien a5168d5977 dce_opnum: improve memory handling on parsing error 11 years ago
Victor Julien b3b7625be5 htp: fix test 11 years ago
Victor Julien c0807c3df5 fast log: clean up tests 11 years ago
Victor Julien 29fbcce50d detect hsbd: simplify resize logic 11 years ago
Victor Julien 4e7cb7b863 app-layer: update all protocols to accept NULL+EOF 
Update all non-HTTP protocol parsers to accept a NULL+EOF input.
 11 years ago
Victor Julien cf9ff6adbd app-layer: improve EOF handling 
On receiving TCP end of stream packets (e.g. RST, but also sometimes FIN
packets), in some cases the AppLayer parser would not be notified. This
could happen in IDS mode, but would especially be an issue in IPS mode.

This patch changes the logic of the AppLayer API to handle this. When no
new data is available, and the stream ends, the AppLayer API now gets
called with a NULL/0 input, but with the EOF flag set.

This allows the AppLayer parser to call it's final routines still in the
context of a real packet.
 11 years ago
Victor Julien df79c1019f print: make PrintRawDataFp take a const arg 11 years ago
Victor Julien 7451d33396 stream: update StreamMsg to don't have fixed size 
StreamMsg would have a fixed size buffer. This patch replaces the buffer
by a dynamically allocated buffer.

Preparation of allowing bigger and customizable buffer sizes.
 11 years ago
Victor Julien 3ffa01d150 stream: remove STREAMTCP_STREAM_FLAG_CLOSE_INITIATED logic 11 years ago
Giuseppe Longo 32563d51d1 http: rework UT 
This reworks UT based on
commit 3203555708
 11 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 11 years ago
Victor Julien 3163a57577 smtp file_data: fix wrong free 11 years ago
Victor Julien 9f52bdd1e5 flow timeout: prevent dead locks 
The flow timeout mechanism called both from the flow manager at run time
and at shutdown creates pseudo packets. For this it has it's own packet
pool, which can be depleted if the timeout logic is faster than the packet
processing threads. In this case the flow timeout would enter a wait loop.
The problem however, is that this wait loop would happen while keeping a
flow locked. This could lead to a race condition when the packet thread(s)
are waiting for the lock that the flow manager has.

This patch introduces a new packet pool call 'PacketPoolWaitForN', meant
to make sure that the thread's packet pool has at least N available
packets. The flow timeout paths use this to make sure enough packets are
available *before* grabbing the flow lock. If there aren't enough packets
available yet, the wait happens before the lock as well.

This still means the wait can happen while the flow hash row is locked, so
we do make sure some more packets are available when entering that. But
perhaps in the future we need a more precise logic there as well.
 11 years ago
Victor Julien 423d5fd0de http: improve inline body tracking 
Rewrite the sliding window handling for IPS mode for the server body.

The buffer used will have the following properties:
    left edge: inspected data - window_size
    right edge: the most recent data

Due to handling of compressed bodies, the data can be much bigger than
the configured window size.
 11 years ago
Victor Julien 79d34f4575 http: body pruning update 
Take inspect limits into account. Consider per direction inspect settings.
 11 years ago
Giuseppe Longo 8f1998e35f hsbd: inspect buffer depending on the engine mode 
Currently, data is buffered up to response-body-minimal size
and response-body-inspect-window before being inspected.
With this, in IPS mode, inspect data as it comes in up.

The sliding window concept is used here,
some data chunks are copied into the window (buffer)
then it's inspected.
 11 years ago
Giuseppe Longo a459376d2e app-layer-htp: add http_body_inline setting 11 years ago
Victor Julien 80615f7d43 ips: move counters in common struct and funcs 11 years ago
Victor Julien 813930f027 nfq: add ips stats 11 years ago
Victor Julien 14466a803d decode: create util function for basic counter updates 11 years ago
Jason Ish e3ce29f694 json-stats: log deltas 
If "deltas" is yes, log delta values as the name of the value
suffixed with _delta.
 11 years ago
Victor Julien 3e14aa729a detect-state: remove/hide BUG_ON statements 11 years ago
Victor Julien b7f1e9e370 file extract: add app_proto to logging 11 years ago
Victor Julien 925aac854e alproto: improve AppProtoToString 
Make AppProtoToString compilation fail on missing 'case's.
 11 years ago
Eric Leblond 0bafc71689 app-layer: add modbus to AppProtoToString 
It was missing causing protocol identified as modbus not to be
displayed in netflow events.
 11 years ago
Victor Julien ebb42f831c suppress: add track by_either mode 
So far suppress rules would apply to src or dst addresses of a packet.
This meant that if a ip would need to suppressed both as src and as dst,
2 suppress rules would be needed.

This patch introduces track by_either, which means that the ip(s) in the
suppress rule are tested against both the packets source and dest ip's.
If either of them is on the suppress list, the alert is suppressed.
 11 years ago
Victor Julien e85a44c383 suppress: support ip-lists 
Ticket: 1137

Support supplying a list of IP's to the suppress keyword. Variables from
the address-groups and negation is supported. The same logic (and code) is
used that is also used in parting the IP portions of regular detection
rules.
 11 years ago
Jason Ish 26fc5682ad hostbits: ignore leading and trailing white space 
Ignore leading and trailing space around the name and
direction tokens.
 11 years ago
Jason Ish 7c40c73482 json-stats: reorg threads and totals 
Totals are now placed at the top level instead of under a "Total"
object.

Threads are placed under a "threads" object.
 11 years ago
Jason Ish 1f2caf78c3 json-stats: log uptime in seconds, instead of a string 11 years ago
Jason Ish 0f1dd0d7ea flowbits: strip leading and trailing spaces in name 
Redmine bug 1481. Strip leading and trailing white space. Factor
out parsing from setup while in here.
 11 years ago
Zachary Rasmor 0edf28a4f8 Add Feature #1454. Generic eve-log prefix support. 11 years ago
Victor Julien a083513c49 decode: optimize DecodeThreadVars layout 
Put common counters on the first cache line. Please the flow output
pointer last as it's use depends on the flow logging being enabled
and even then it's only called very rarely.
 11 years ago
Victor Julien fe5a85aea0 decode: add erspan counter 11 years ago
Victor Julien 928957f0a3 decode: add ERSPANv1 decoder 
Only allow v1 to be parsed as thats what is tested.

Take vlan_id from the ERSPAN layer.
 11 years ago
Victor Julien aa6b24f814 decode: clean up tunnel decode logic 
Don't use mix of existing and custom types to indicate the next
layer.
 11 years ago
Victor Julien ef7cd043cc detect: various header cleanups 11 years ago
Victor Julien 5483b800c5 detect: remove struct/union tricks from Signature 11 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 11 years ago
Victor Julien 9fa2f85cc7 http: improve body pruning 
Take inspect window into account.
 11 years ago
Victor Julien 0bbc818b2d http: fix body tracking 
In HTTP body tracking for response bodies, pruning body chunks was broken
as the body parsing code wouldn't update HtpBody::body_parsed.
 11 years ago
Victor Julien 3203555708 http-client-body: create unittest util func 11 years ago
Eric Leblond d837562441 logging: fix modules ordering during logging 
With the previous code the order of the logging modules in the
YAML were determining which module was run first. This was not
wished and a consequences was that the EVE fileinfo module was
not correctly displaying the key 'stored' because it was
depending on a flag set alter by the filestore module.

This patch adds a priority file to the TmModule structure. The
higher the priority is set, the sooner the module is run in the
logging process. The RunModeOutput structure has also been
updated to contain the name of the original TmModule. Thus allowing
to define a priority for a RunModeOutput.

Currently only the filestore has a priority set. The rest of them is
set to the default value of zero.
 11 years ago
Eric Leblond be07620a60 output-lua: sync variable name with yaml 
'script-dir' was used in the code but we had 'scripts-dir' in the
configuration file. This patch fixes it to 'scripts-dir'.
 11 years ago
Jason Ish ae23144b67 --set - handle spaces on either side of '=' 
Discard spaces when provided as part of --set around the '='. For
example, "val=key", "val = key", "val= key" and "val =key" are
all equivalent now.
 11 years ago
Jason Ish d9fe95bc8a conf - function declaration style 
Use consistent style - function return type and declaration on
same line.
 11 years ago
DIALLO David 0a4fd39f9c modbus: fix heap-buffer-overflow in Modbus parser 
Modbus parser does not check length to extract/read data (read or write address,
quantity of data, etc.) that should be present.

In case of malformated data (invalid length in header), Modbus parser reads data
over the input data length.

Add check before extracting/reading data from input buffer to avoid head buffer
overflow.
 11 years ago
Victor Julien 07efec550d counters: use ptr to name instead of copy 
All counters have hardcoded names, so copies are not needed.
 11 years ago
Victor Julien 7e66c70507 counters: don't run if no counters have been registered 11 years ago
Victor Julien cb5aa8f8d5 counters: work around unix-socket init issues 11 years ago
Victor Julien e48153c6b0 counters: make threads cleanup all memory 11 years ago
Victor Julien 81548ae3e8 counters: clean up global context 11 years ago
Victor Julien 84b8829cb4 counters: turn flow.memuse into a global counter 11 years ago
Victor Julien 0a262acdfb counters: make DNS counters globals 11 years ago
Victor Julien ac069c579a counters: make tcp.memuse a global counter 11 years ago
Victor Julien cddbb0f606 http: make http.memuse a global counter 
http.memcap as well.
 11 years ago
Victor Julien f05d0692ef counters: remove references to 'perf' counters 11 years ago
Victor Julien faef92f8da counters: remove last and now unused tm_name reference 11 years ago
Victor Julien 83f27ae2a5 counters: remove old unix socket json logic 11 years ago
Victor Julien 41ead6611a counters: minor internal API cleanups 11 years ago
Victor Julien d2a9ef2680 counters: rename unparsable SCPCAElem to StatsLocalCounter 11 years ago
Victor Julien 4c3ccda72e counters: minor header cleanup 11 years ago
Victor Julien 752f03e7a4 counters: remaining s/SCPerf/Stats/g 11 years ago
Victor Julien 4362d0a6e9 counters: s/SCPerfPrivateContext/StatsPrivateThreadContext/g 11 years ago
Victor Julien 628c3b1bc7 counters: s/SCPerfPublicContext/StatsPublicThreadContext/g 11 years ago
Victor Julien 7e70f136ec counters: various renames and cleanups 11 years ago
Victor Julien 30cce2bd29 counters: s/SCPerfCounterSetUI64/StatsSetUI64/g 11 years ago
Victor Julien 1c0b4ee0ae counters: s/SCPerfCounterIncr/StatsIncr/g 11 years ago
Victor Julien 8992275b0c counters: s/SCPerfCounterAddUI64/StatsAddUI64/g 11 years ago
Victor Julien 60d9eb6790 counters: clean up defines 11 years ago
Victor Julien 1ef786e7cb counters: rename register API calls 
Also remove 'type' parameter which was always the same.
 11 years ago
Victor Julien 3fab736539 log-stats: make global/threads logging configurable 11 years ago
Victor Julien 2c9a2c8327 stats: support per thread stats in json output 
Default is only to output totals. Optionally per thread can be added.

Both can be enabled together.
 11 years ago
Victor Julien 175831331c stats json: replace strndup 
strndup is a banned function.
 11 years ago
Victor Julien 6565c86f96 stats-json: fixes and improvements 
Use proper LogFileCtx and MemBuffer handling so we can have multiple
loggers active at the same time.

Change 'date' field to timestamp, and use ISO notation to make it
the same as the other JSON outputs.
 11 years ago
Tom DeCanio e4e07d0c3b eve-log: stats logging code cleanup. 11 years ago
Tom DeCanio 117eed0385 eve-log: add JSON stats logging 
Support for counters in stats.log in eve output JSON stream.
 11 years ago
Victor Julien 23f17950bc counters: pass per thread stats to output api 
As well as the global (merged) stats.
 11 years ago
Victor Julien de82b6d31e counters: rename widely used pctmi var to sts (stats thread store) 11 years ago
Victor Julien 60c5ad4649 counters: call global counters funcs 11 years ago
Victor Julien 33756abd87 counters: split API init 
Split into early ctx initialization and post-config setup.
 11 years ago
Victor Julien d05eed3735 counters: start using Stats prefix 11 years ago
Victor Julien f300ad253e counters: simplify and speedup counters sync 11 years ago
Victor Julien 0478407833 counters: remove threadvars arg from SCPerfAddToClubbedTMTable 11 years ago
Victor Julien 2346a88db7 counters: remove thread module name from counters API 11 years ago
Victor Julien b5bd3dee13 stream: make tcp.reassembly_memuse counter global 
Fixes bugs #632 and #1178
 11 years ago
Victor Julien 06461e37da counters: global counters registration 11 years ago
Victor Julien 9bbef55c4d Fix harmless typo in IPOnlyCIDRItemNew's SCReturnPtr use 11 years ago
Victor Julien b293a4b7d0 counters: remove unused description 11 years ago
Victor Julien 711cd7b59b counters: merge counters from threads for output 
Merge counters so the table contains combined values from counters
from each thread.

Use global counter id's, track them in a hash.

Rename SCPCAElem members

Fix and improve average counters
 11 years ago
Victor Julien 7da657dc3d counters: remove unused public API calls and make them private 11 years ago
Victor Julien ac6e24c06a counters: make SCPerfSetupPrivate a function 11 years ago
Victor Julien 66635f0741 counters: minor cleanups 11 years ago
Victor Julien 74ab84c194 counters: introduce SCPerfSetupPrivate for thread setup 11 years ago
Victor Julien 799640f906 counters: make threadvars::perf_private_ctx static 
Update SCPerfGetAllCountersArray and add a UT workaround.
 11 years ago
Victor Julien 55cfab89e4 counters: SCPerfGetLocalCounterValue cleanup 
Return u64, update arguments.
 11 years ago
Victor Julien b34c6dc93a counters: remove references to SCPerfCounterAddDouble 
They were all in comments anyway.
 11 years ago
Victor Julien e9b067c1eb counters: make increment call take threadvars 
This hides the implementation from the caller.
 11 years ago
Victor Julien 9a8bff7d96 counters: threadvars s/sc_perf_pca/perf_private_ctx/g 11 years ago
Victor Julien 50bb995458 counters: rename threadvars public counters 11 years ago
Victor Julien 6ffbc3a362 counters: s/SCPerfContext/SCPerfPublicContext/g 11 years ago
Victor Julien 0a5ae1b403 counters: s/SCPerfCounterArray/SCPerfPrivateContext/g 
Goal is to make it's purpose clear.
 11 years ago
Victor Julien 9f584483be counters: minor cleanups 11 years ago
Victor Julien 1e8142c699 logfile: rename ALERT_ types to LOGFILE_TYPE_ 11 years ago
Eric Leblond 4c6a7bea30 output-json: suppress global variable 
It uses the new type field in the LogFileCtx instead.

This fixes the problem of not being able to use two eve-json
instance with different logging methods.
 11 years ago
Eric Leblond 636e3d93c0 log file: add type flag 
It will be used to store if the file is syslog or a real file.
 11 years ago
Eric Leblond 39d667ff56 output-json: fix type of data parameter 
The cast of data to AlertJsonThread was not correct as the real
type of the void pointer is a OutputJsonCtx. This was working by
luck because they both have a file_ctx as first element.
 11 years ago
Alexander Gozman f11e237d77 Feature #1440: support wildcards in rule filenames 11 years ago
David Cannings 4f8f53d080 Fix rcode parsing, as noticed by Coverity. 
Without support for OPT RR from RFC6891 (Extension mechanisms for DNS)
values of RCODE above 15 are not possible.  Remove dead code which will
never match.
 11 years ago
Jason Ish 9fdae82815 conf - process includes even if not at root node. 11 years ago
Jason Ish 56f6e37304 radix-tree - prevent out of bounds array access 
An IPv6 entry specified before an IPv4 entry on the host-os-policy
table can cause the stream byte array to be access one byte after
the end of the allocated memory at util-radix-tree.c:578.
 11 years ago
Jason Ish 3e5b8f48b1 Bug 1281 - Add tests for rule content of lengths > 255. 11 years ago
Jason Ish e2b04635a7 Bug 1281 - Accept rule content with lengths greater than 255. 11 years ago
Victor Julien 0e22e95e47 alert-json: fix stream logging for IPS mode 
Switch direction in IPS mode.
 11 years ago
Victor Julien 5037ea93f3 threads: add untimed control cond call 
The control conditions so far could only do timed waits, not normal
waits.
 11 years ago
Victor Julien c7bc9ae6a8 detect: minor cleanups 11 years ago
Victor Julien bc2b53f10b parsing: s/strtok/strtok_r/g 
Remove all strtok uses and replace them by strtok_r.

Do the same for Windows builds. Cygwin builds fine with strtok_r.

Add strtok to banned function list.
 11 years ago
Victor Julien fb479902e4 threading: explain purpose of threadvars mucond 11 years ago
Victor Julien 478719ee9d flow: don't hold tv_root_lock longer than needed 
Don't hold it longer than needed in shutting down.
 11 years ago
Victor Julien c96805e839 threading: remove unused cmd thread create func 11 years ago
Victor Julien df5e9d44ca unix-manager: convert to thread module 
Sync command thread for unix manager with other managers and make
it a full thread module.
 11 years ago
Victor Julien cc01b5f6b6 reference/classification: call global init for unittests 11 years ago