aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,834 Commits
7 Branches
161 Tags
170 MiB
main-7.0.x
master
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '723e90a174'
${ noResults }
Commit Graph

151 Commits (723e90a174fd6f4df6677eff348fa46c25e36f65)

Author SHA1 Message Date
Eric Leblond 9ca7257279 nfq: suppress unused functions. 14 years ago
Eric Leblond aee2e3ddd6 nfq: Add autofp mode support 14 years ago
Eric Leblond 115c3499d2 nfq: factorize auto mode 14 years ago
Eric Leblond 830ca7c2c8 source-nfq: suppress insecable space. 
This patch supresses an insecable space and fixes an
indentation.
 14 years ago
Eric Leblond bbd04fde30 NFQ: fix race condition at exit. 
A race condition was observed when leaving NFQ. This was caused by
the queue handle being accessed after been nullified. This patch
uses the handle mutex to protect the destruction and adds tests
on nullity to avoid crashed.
 14 years ago
Anoop Saldanha 3f1c4efceb Add new flags var to tm module. TMs can now set flags to identify special properties. Also use these to identify receive TMs 14 years ago
Victor Julien c590bba4a4 Undo tunnel reference counting using atomic operations. Revert to mutex based code. 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Eric Leblond a354034cfc nfq: Fix deinit phase 
If receive thread is failling, we need to restart it but the code was
not restarting the queue (this was done in verdict thread).
 14 years ago
Eric Leblond eddcedba0a nfq: make thread abort if NFQ verdict fail 14 years ago
Eric Leblond 2ffcef0a8e nfq: Add iterator on nfq_set_verdict 
This patch adds retry to nfq_set_verdict in case of error.
 14 years ago
Eric Leblond bc68c108a7 NFQ: use per thread allocated data for recv buffer. 14 years ago
Victor Julien 52eb8d2be0 Convert mutex protected tunnel counters to lockless atomic counters. 14 years ago
Eric Leblond ee6552f25e nfq_set_mark: handle feature in NFQ. 
This patch implements the nfq_set_mark related modification of verdict
handling.
 15 years ago
Eric Leblond 04f2afa81b nfq: fix exit function 
Exit function was trying to close the nfq handler even if it was
null. This was causing a crash.
 15 years ago
Victor Julien 477bc1d050 Set DROP flag on a packet in addition to the REJECT flags. This makes sure we not only send a reject, but also drop the offending packet. Closes #248. 15 years ago
Victor Julien d424ac7c61 Fix nfq lockup due to improper handling of PKT_PSEUDO_STREAM_END packets. 15 years ago
Victor Julien c9f9e3f9a4 Add configure check for signed or unsigned nfq_get_payload, adapt code. 15 years ago
Eric Leblond c74116949c source-nfq: improve nfq option system 
This patch modifies the NFQ option system to avoid implicit
choice. 'nfq.mode' is now a string which can take a value
in the 'accept', 'repeat' and 'route' set.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 94596ff282 source-nfq: add queue redirect support 
This patch adds the support of queue redirect. If 'next_queue'
variable is set, the verdict sent to kernel is modified to contain
the indication of a queue number (equal to 'next_queue') which will
receive the packet after the verdict. This feature can be used to
chain easily tools using NFQUEUE.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond aded7b4fae source-nfq: add detection of already treated packet. 
This patch adds detection of already treated packet. If a packet is
coming with an already set mark, it will be accepted and the processing
of the packet is aborted.
The patch display a message when the problem occurs but the number of
message is limited to a fix counter in a way to avoid flooding the log.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond e399e74fc1 source-nfq: Factorize buffer usage 
A big sized buffer was allocated at each packet parsing. This patch
uses a per-thread variable to have a persistent memory usage.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 1e600c1054 source-nfq: add simulated non-terminal NFQUEUE verdict 
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
    iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.

To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.

NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.

Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
   nfq:
     repeat_mode: (false|true)
     mark: $MARK
     mask: $MASK
Default is false to preserve backward compatibility.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 72ec56ab23 source-nfq: autodetection of queue max length function 
Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 1375e90030 Prepare multi queue support in NFQ 
This patch prepare support for multiqueue in the
source file. The NFQ vars contained in Packet structure
has a new member. It is a reference to the NFQ thread var
it comes from. The behaviour is modified as a single verdict
thread treat packet for all Netfilter queues.

Locking is done in the verdict function to ensure that
simultaneous modifications of counters can not occur.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 96e3852191 source-nfq: add define of SOL_NETLINK 
Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Victor Julien 929ce0bb9b Add a counter to NFQ for modified packets. 15 years ago
Victor Julien 1dca88fe69 Do the actual checksum recalculation and packet replacement on modifing a packet in the stream engine. 15 years ago
Victor Julien acc38c9ebf Make sure we don't try to 'verdict' the fake PKT_PSEUDO_STREAM_END packets. 15 years ago
Eric Leblond 89558ab9a4 RFC: modify error treatment in PacketCopyData 
Hello Victor
This patch modifies error treatment following our discussion on IRC.
It tries to follow the error treatment guideline I've been able to
read in the different files.
I will merge this patch in the original commit if the error treatment
seems ok for you.

BR,
Eric
 15 years ago
Eric Leblond 156b202597 Fix decode part of source-nfq 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access 
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
 15 years ago
Eric Leblond f73c60b4ff nfq: set some options on netlink socket 
This patch modify nfq system to set some options on the netlink socket.
This should improve performances by handling more correctly capacity
overrun.
 15 years ago
Victor Julien 4cacb1e970 Disable adding to unregistered mbit/s counter. 15 years ago
Victor Julien 6519a86ec7 Move packet pool to ringbuffer, update packet pool api and ringbuffer api. Remove memset usage from PACKET_RECYCLE, add proper cleanup macros. 15 years ago
Victor Julien dce323b1f4 Fix SCondWait -> SCCondWait typo. 15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
William Metcalf b629b7c5c1 only show cli opts via help that we have support for 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
Victor Julien bb685751d9 Fix NFQ receive/verdict race condition in cases where the packetpool is empty. 15 years ago
Victor Julien 647b1c0eba Fix NFQ compilation. 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 16 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 16 years ago
Victor Julien 591c898f6a NFQ inline changes 
- split NFQRecvPkt into a Linux and Windows version for readability
- Add many function header comments
- convert tabs to spaces in Windows code.
 16 years ago
Jan Jezek 366671a8ce Added inline mode support on Windows 16 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 16 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Victor Julien 16b6f536a0 Fixup Linux compilation after applying win32 patches. 16 years ago
Gurvinder Singh 5293681860 b86 16 years ago
Victor Julien 2cb2989ad8 Apply configurable max pending packets to nfq and ipfw 16 years ago
Pablo Rincon ad2c136e8f Renaming errors (naming conventions) 16 years ago
Gurvinder Singh cf5266094d bug 66 patch 16 years ago
Victor Julien df4c642c70 Fix weird compile error 16 years ago
Pablo Rincon 51dc773eec Changing the veredict actions to flags to allow simultaneous veredict 16 years ago
Eric Leblond 51be576a30 nfq: modify queue length computation logic 
This patch modifies  max queue length computation logic. The max queue
length was set to MAX_PENDING which is the total number of packet
processed simultaneously in suricata.

This value is correct but this will not permit to take all burst
effects into account (read sudden quantity of packet that arrives
faster than suricata is enable to parse). Furthermore there is a
delaying system when suricata gets overloaded which make necessary
to have packet storable into kernel for some time.

To improve this situation the patch increases the maximum queue
length to NFQ_BURST_FACTOR (4) time the MAX_PENDING packet and
it also increase the nfnetlink buffer size to be able to store
all packets waiting for suricata in the netlink receive buffer.
 16 years ago
Eric Leblond 72d48f6658 nfq: add sanity checking 
This patch adds sanity checking to payload handling. It set length
of packet to zero if an error occurs.
 16 years ago
Eric Leblond 56cccdfa62 nfq: use switch instead of 'else if' 
This patch convert a 'else if' serie to a switch to increase
the readability of the decision related code.

 Please enter the commit message for your changes. Lines starting
 16 years ago
Victor Julien 13e10ccd86 Enable bytes per sec and mbit per sec for nfq as well 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Victor Julien aa653157bf Convert stats printing in nfq to logging api. 16 years ago
Victor Julien e5301effe6 Make nfq module use logging api. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Anoop Saldanha 8beef4a9fc stats upgrade. Added interval counters to the decoder module 16 years ago
Anoop Saldanha ceb7e495ae refactoring perf stats code 16 years ago
Gurvinder Singh e7952b1331 some changes in threading constants 16 years ago
Gurvinder Singh 40b8afdd56 support for thread exit constants 16 years ago
Victor Julien f0d556b9e3 Suppress nfq debug output. 16 years ago
Victor Julien a4fe971897 Fix datalink retrieval for pcap file mode and nfq mode for use in unified2. 16 years ago
Victor Julien 0d0ffb9963 Reorganize header inclusions. 16 years ago
William Metcalf 07bb501aca compile fix for source-nfq.c 16 years ago
Victor Julien a39108843e Small tm module API rename to reflect that Init/Deinit/ExitPrintStats are per thread calls. 16 years ago
Victor Julien e7f8a02484 Improve GRE counting. Actually use the GRE decoder. Register the GRE counter. 16 years ago
Victor Julien 57f71f7e4b Pass the DecodeThreadVars to all Decoder functions properly. Improve the error handling. 16 years ago
Anoop Saldanha 244f5d547a new registration functions for the stats api, with local thread storage for counter ids 16 years ago
Victor Julien e31563f61c Fix nfq compilation. 16 years ago
Brian Rectanus fa5939ca91 64 bit cleanup part2 16 years ago
Anoop Saldanha 8af9f902e8 additional support for type qualifier for the stats api 16 years ago
Victor Julien 175eaeca93 Slightly moved around the NFQ define a bit. 16 years ago
William Metcalf a3510f2025 Made NFQ optional via --enable-nfqueue, --enable-logsigs will now load local.rules in the path other fixes 16 years ago
Victor Julien 51a9e36e10 Remove vips references. Rename to eidps. 16 years ago
Victor Julien 8397413942 Comment updates. 16 years ago
Victor Julien 9c7f5afa79 Large update: pcap support, threading fixes, initial stream tracking, time handling, pool support, runmodes, decoders added, autojunk update. 16 years ago
Victor Julien 657be002d1 Big detection engine update: scan improvements, b2g/b3g updates, bloom fixes, iponly detection implementation, dsize/flow grouping. 16 years ago
Victor Julien 5df5b35e90 Put all globals in the detection engine ctx. Add HashList type, a hash that also stores the items ina list to it can be traversed. Many cleanups. 16 years ago
Victor Julien b2eb954099 Add b3g 3gram BNDM pattern matcher. Fix multi queue nfq initialization. Improve speed of b2g and wumanber. 16 years ago
Victor Julien efb10fc0d6 big update 16 years ago
Victor Julien 4c4862d838 Improve logging, add alert-output module, at module exit stats, add HTTP POST uri capture. 16 years ago
Victor Julien 4b6a8715fd Fix packets getting stuck in NFQ under high load. 16 years ago
Victor Julien edf8650a7e Tunnel update. 16 years ago
Victor Julien 8cda70668c Tunnel fixes. 16 years ago
Victor Julien 28d9415e37 New approach to tunnel decoding. 16 years ago
Victor Julien a454fe4fb7 Small fixes 16 years ago
William Metcalf 0ffa1c2465 updates for configure.in, added reject code, some decode stuff for tcp 16 years ago
Victor Julien 867d493d7f Source NFQ update... less hackish, but still needs work as soon as we know how to do configuration. 16 years ago
Victor Julien 6c1f2071be Add unittest registration to the threading modules api. 16 years ago
William Metcalf 559edc01e3 NFQUE drop support added with ident of 4 :-( 16 years ago