aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,935 Commits
7 Branches
161 Tags
173 MiB
main-7.0.x
master
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c9436a6aef'
${ noResults }
Commit Graph

176 Commits (c9436a6aef1e79161dbaa13d8d3ead952158b4c6)

Author SHA1 Message Date
Anoop Saldanha e68ca2f32f Rewrite forced reassembly v2 using while loop instead of goto 14 years ago
Anoop Saldanha 6cc179fad8 flow mgr code doesn't have to bother on immediately exiting on seeing a suricata_ctl flag set 14 years ago
Anoop Saldanha b09c9751aa Now flow hash section can force reassemble flows as well 14 years ago
Anoop Saldanha 42493ee6b7 rename pseudo packet creation function. Shift the check for forced reassembly necessity on a session/direction to an inline function in the stream api 14 years ago
Anoop Saldanha 6c95526423 Introduce a new wrapper macro that wait loops till the flag(s) in question have been set 14 years ago
Anoop Saldanha b0a588beeb Introduce another solution to solve stream timeout shutdown issue using thread flags. No more MSSyncPts 14 years ago
Anoop Saldanha aef957c6eb cleanup flow code and pseudo packet creation function 14 years ago
Anoop Saldanha b4887943fb packet queue len member is now 32 bit unsigned from the previous 16 bit unsigned. Should take care of the overflow for now 14 years ago
Anoop Saldanha 9256c7bf0a always keep queue locked till we exit flowprune. Should prevent potential threading issues 14 years ago
Anoop Saldanha d4ba869a35 fix - we need to set direction flags for reassembly pseudo packet. Also reset local flags for every flow that is force reassembled in ForQ 14 years ago
Anoop Saldanha edebdee1e5 update flow pruning - v6 14 years ago
Anoop Saldanha 99207c718d Avoid possiblity of potential engine idling from consumption of all packetpool packets - v1 
- Now forced reassembly uses only malloced packets.
 14 years ago
Anoop Saldanha 7d3e501f57 shutdown stream reassembly now avoids looking at flows that have already been processed by flow mgr reassembly 14 years ago
Anoop Saldanha a559bfc165 signal the post pq if possible, whenever pseudo packets are injected into engine flow. Also carry out post pq processing irrespective of packet retrieval from the flow. 14 years ago
Anoop Saldanha fd9bacb02d fix usage of htons to htonl in creation of pseudo packet 14 years ago
Anoop Saldanha 56fba8e275 move flow incr cnt while we actually create the pseudo packet in forced reassembly 14 years ago
Anoop Saldanha 51d2b64902 update flow pruning - v5 14 years ago
Anoop Saldanha c30dbff63d update flow pruning - v4 14 years ago
Anoop Saldanha 3b0142fa46 update flow pruning - v3 14 years ago
Anoop Saldanha 6dcb68abb0 update flow pruning - v2 14 years ago
Anoop Saldanha f197b32a55 update flow pruning - v1 14 years ago
Anoop Saldanha 272c2433ec Cleanup flow.c before further changes 14 years ago
Anoop Saldanha 8363533a02 support for forced stream reassembly for to be pruned flows 14 years ago
Anoop Saldanha 762ac0fe31 update conditional in shutdown forced reassembly to check for flows that required flow reassemly 14 years ago
Anoop Saldanha 9552e6f696 Shutdown flow timeout reassembly now supports ipv6 14 years ago
Anoop Saldanha c365bafbf6 We now inspect timed out streams + streams not processed as yet, at engine shutdown 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Anoop Saldanha ff7284e7b7 Fix code that allows the engine to restart threads that have exited on failure 14 years ago
Victor Julien a5d9c86dd3 Shrink Flow structure with 20 bytes (on 32 bit) and reorder it. Clean up init, recycle, destroy macro's. 15 years ago
Anoop Saldanha a165d45da9 naming changes for runmodes 15 years ago
Eric Leblond 674b0bfae7 flow: Add basic SCTP support 
This patch adds a basic flow support to SCTP. SCTP specificities
like the verification tag are not taken into account.
 15 years ago
Victor Julien 668bd46c1c Add flow prune debug counters (disabled by default). 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access 
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
 15 years ago
Victor Julien 3409513a44 Fix FlowTest* unittests to fail sometimes. 15 years ago
Victor Julien fc248ca7a1 Many small performance updates. 15 years ago
Pablo Rincon 5c43db85ce Drop streams on inline mode when a drop rule match from a reassembled stream and/or app layer inspection 15 years ago
William Metcalf 50eb3cba6a seems to be a race between FlowTestPrune and FLOW_DESTROY in FlowTest0* comment out the later for now 15 years ago
Victor Julien cff0a0bda2 Fix segv conditions caused by broken flow cleanup code. 15 years ago
William Metcalf 0e4235cc94 FLOW_DESTROY added to clean-up UT's that init flow 15 years ago
Victor Julien 2f29b8a724 Improve detection of app layer, making sure we only handle app layer on 'established' packets. Should really fix #166. 15 years ago
Victor Julien ba12f3c109 Applayer to flow fixes and cleanups. 15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
William Metcalf cc76aa4bc6 properly init flows inside of unit-tests caused lock-up when falling back to using mutex locks 15 years ago
Victor Julien c26434fef1 Move flow use cnt to atomic and outside of the flow mutex protection. 15 years ago
Victor Julien 87345e5c60 Switch flow memuse counter to the atomic api. 15 years ago
Victor Julien 3484e2abde Fix flow engine memory handling. 15 years ago
Pablo Rincon b491ef2be0 Removing FlowAllocDirect since it's not needed anymore 15 years ago
Victor Julien 2fd31a1a11 Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory. 15 years ago
Pablo Rincon 29a6fc2f03 Adding some flow improvements and recovery on emergency mode 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine. 
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
Victor Julien 53b388b669 Improve flow hash debugging, switch to csv output. 15 years ago
Victor Julien 548a3b2c93 Improve flow hash debugging functions. Make sure ICMP errors don't create flows. Handle ICMP DEST UNREACH errors in the flow they are sending the error about. 15 years ago
Victor Julien 2dc5405d3a Add debug code for tracking flow hash distribution. Only add ICMP DEST_UNREACH packets to the flow engine. 15 years ago
Victor Julien 0ebf7cbc5e Convert flow bucket lock from mutex to spinlock. Locks should be very short, so spinlocks should be faster. 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 16 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 16 years ago
Victor Julien fc2029ef58 Fix small memuse counter issue in flow subsys, remove emergency printfs 16 years ago
Jason Ish 00974d157b Fix issue 131. 
Flow-timeouts likely don't need to be a sequence, but rather mappings.  We'd only need a sequence if you wanted to list something like "tcp" twice which I don't think makes sense for configuration section.

Also fixup flow.c to not attempt to use the sequence, and put the timeouts into their correct place.
 16 years ago
Victor Julien 958b61ab85 Fix errors in the unittests reported by valgrind's drd tool. Add explanation of a FP. 16 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 16 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 16 years ago
Victor Julien ae2698c9cc Properly cleanup used mutexes and condition vars in the flow subsys. 16 years ago
Victor Julien 5e3f072fa6 Clean up flow mutexes after use. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Gurvinder Singh bf236e4567 better htp memory handling & flow valgrind error fixed 16 years ago
Victor Julien f7f33ec889 Fix the flow manager sleeping for way too long in some situations. 16 years ago
Pablo Rincon 7f250a814a Fixing redeclaration of run_mode 16 years ago
Pablo Rincon 5592189c04 Loading flow settings from config 16 years ago
Victor Julien cae8e06cb9 Properly lock app layer result pool and add some debugging code for memory tracking. 16 years ago
Pablo Rincon c80160b96d More examples of unittest helper functions usage reference 16 years ago
Victor Julien 27a138d862 Suppress some flow messages. 16 years ago
Victor Julien 9ececacda3 Fix packet timestamp handling for encapsulated packets. 16 years ago
Victor Julien 6b36e23e45 Fix not decreasing the flow use_cnt reference counter in some cases from the app layer detection code. This caused some streams to never fully time out and thus clutter up the flow table and session pool. 16 years ago
Victor Julien b7bac14040 Fixup code to compile with -Wall -Werror -Wextra -Wno-unused-parameter compiler options. 16 years ago
Victor Julien 6beee776ca Move rand seed code into util-random 16 years ago
Pablo Rincon 6224c30548 Adding preseending to rands 16 years ago
Victor Julien 10cc9d5b6a Add icmp flow handling. 16 years ago
Victor Julien a8116f65c8 Fix packet flags field not being cleared properly when the packet is being reused. Add some debug statements and cleanup some. 16 years ago
Victor Julien 4824868766 Application layer detection improvements 
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Victor Julien ba7e8012af Add some debugging and simplify locking for app layer slightly. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Gurvinder Singh 3f23f52c52 Fixed FlowTest01 and StreamReassemblyTest30 16 years ago
Anoop Saldanha ceb7e495ae refactoring perf stats code 16 years ago
Gurvinder Singh 48c94bbf03 initial thread code support 16 years ago
Gurvinder Singh 7895748606 add unit test for no stream reassembly 16 years ago
Gurvinder Singh 71da019734 some minor changes 16 years ago
Gurvinder Singh bbb385422d function to set packet flags 16 years ago
Gurvinder Singh cc0b4f7181 initial version to support detection byepass 16 years ago
Victor Julien 91bc83e5c6 More logging API usage changes. 16 years ago
Victor Julien f4acd5a27e First batch of fixes for new debug and logging API usage. 16 years ago
Victor Julien 597d0e9a20 Fix detection of failed thread startup. Cleanup startup output a bit. 16 years ago
Victor Julien b6deadd2b4 Fixed a few missing places for tv->flags access. Changed mutex init for packets slightly. 16 years ago
Anoop Saldanha 3a45b2711c Fixed the Perf API startup issue 16 years ago
Victor Julien b102ea2123 Big update: 
- Implement "closing" state in flow.
- Add protocol specific timeouts.
- Lots of stream tracking updates, fixing a lot of out of window issues.
- Stream reassembly fixes.
- Implement a new IDS runmode with 4 stream and detect threads.
- Added a BUG_ON macro that aborts the engine if the expression is true.
- Better balance the flow queue handler for traffic that doesn't have flow (like icmp currently).
- Simplify application level protocol in the Tcp Session.
- Add some debugging memory counters.
 16 years ago
Victor Julien b4f0e82463 Small flow updates. 16 years ago
Gurvinder Singh 9dcf6e2de2 handle the FLOW_STATE_CLOSED 16 years ago