aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,691 Commits
7 Branches
161 Tags
209 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '79e0299643'
${ noResults }
Commit Graph

150 Commits (79e0299643043eec4031012c48cd15dbbd4df10c)

Author SHA1 Message Date
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Anoop Saldanha ff7284e7b7 Fix code that allows the engine to restart threads that have exited on failure 14 years ago
Victor Julien a5d9c86dd3 Shrink Flow structure with 20 bytes (on 32 bit) and reorder it. Clean up init, recycle, destroy macro's. 14 years ago
Anoop Saldanha a165d45da9 naming changes for runmodes 15 years ago
Eric Leblond 674b0bfae7 flow: Add basic SCTP support 
This patch adds a basic flow support to SCTP. SCTP specificities
like the verification tag are not taken into account.
 15 years ago
Victor Julien 668bd46c1c Add flow prune debug counters (disabled by default). 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access 
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
 15 years ago
Victor Julien 3409513a44 Fix FlowTest* unittests to fail sometimes. 15 years ago
Victor Julien fc248ca7a1 Many small performance updates. 15 years ago
Pablo Rincon 5c43db85ce Drop streams on inline mode when a drop rule match from a reassembled stream and/or app layer inspection 15 years ago
William Metcalf 50eb3cba6a seems to be a race between FlowTestPrune and FLOW_DESTROY in FlowTest0* comment out the later for now 15 years ago
Victor Julien cff0a0bda2 Fix segv conditions caused by broken flow cleanup code. 15 years ago
William Metcalf 0e4235cc94 FLOW_DESTROY added to clean-up UT's that init flow 15 years ago
Victor Julien 2f29b8a724 Improve detection of app layer, making sure we only handle app layer on 'established' packets. Should really fix #166. 15 years ago
Victor Julien ba12f3c109 Applayer to flow fixes and cleanups. 15 years ago
Pablo Rincon 8cc525c939 UDP support at AppLayer message handling 15 years ago
William Metcalf cc76aa4bc6 properly init flows inside of unit-tests caused lock-up when falling back to using mutex locks 15 years ago
Victor Julien c26434fef1 Move flow use cnt to atomic and outside of the flow mutex protection. 15 years ago
Victor Julien 87345e5c60 Switch flow memuse counter to the atomic api. 15 years ago
Victor Julien 3484e2abde Fix flow engine memory handling. 15 years ago
Pablo Rincon b491ef2be0 Removing FlowAllocDirect since it's not needed anymore 15 years ago
Victor Julien 2fd31a1a11 Remove dsize grouping from detection engine grouping reducing memory usage. Store sgh in flow to reduce lookups. Reduce locking in alert handling. Increase default grouping values as we use less memory. 15 years ago
Pablo Rincon 29a6fc2f03 Adding some flow improvements and recovery on emergency mode 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine. 
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
Victor Julien 53b388b669 Improve flow hash debugging, switch to csv output. 15 years ago
Victor Julien 548a3b2c93 Improve flow hash debugging functions. Make sure ICMP errors don't create flows. Handle ICMP DEST UNREACH errors in the flow they are sending the error about. 15 years ago
Victor Julien 2dc5405d3a Add debug code for tracking flow hash distribution. Only add ICMP DEST_UNREACH packets to the flow engine. 15 years ago
Victor Julien 0ebf7cbc5e Convert flow bucket lock from mutex to spinlock. Locks should be very short, so spinlocks should be faster. 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 15 years ago
Victor Julien fc2029ef58 Fix small memuse counter issue in flow subsys, remove emergency printfs 15 years ago
Jason Ish 00974d157b Fix issue 131. 
Flow-timeouts likely don't need to be a sequence, but rather mappings.  We'd only need a sequence if you wanted to list something like "tcp" twice which I don't think makes sense for configuration section.

Also fixup flow.c to not attempt to use the sequence, and put the timeouts into their correct place.
 15 years ago
Victor Julien 958b61ab85 Fix errors in the unittests reported by valgrind's drd tool. Add explanation of a FP. 16 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 16 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 16 years ago
Victor Julien ae2698c9cc Properly cleanup used mutexes and condition vars in the flow subsys. 16 years ago
Victor Julien 5e3f072fa6 Clean up flow mutexes after use. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Gurvinder Singh bf236e4567 better htp memory handling & flow valgrind error fixed 16 years ago
Victor Julien f7f33ec889 Fix the flow manager sleeping for way too long in some situations. 16 years ago
Pablo Rincon 7f250a814a Fixing redeclaration of run_mode 16 years ago
Pablo Rincon 5592189c04 Loading flow settings from config 16 years ago
Victor Julien cae8e06cb9 Properly lock app layer result pool and add some debugging code for memory tracking. 16 years ago
Pablo Rincon c80160b96d More examples of unittest helper functions usage reference 16 years ago
Victor Julien 27a138d862 Suppress some flow messages. 16 years ago
Victor Julien 9ececacda3 Fix packet timestamp handling for encapsulated packets. 16 years ago
Victor Julien 6b36e23e45 Fix not decreasing the flow use_cnt reference counter in some cases from the app layer detection code. This caused some streams to never fully time out and thus clutter up the flow table and session pool. 16 years ago
Victor Julien b7bac14040 Fixup code to compile with -Wall -Werror -Wextra -Wno-unused-parameter compiler options. 16 years ago
Victor Julien 6beee776ca Move rand seed code into util-random 16 years ago
Pablo Rincon 6224c30548 Adding preseending to rands 16 years ago
Victor Julien 10cc9d5b6a Add icmp flow handling. 16 years ago
Victor Julien a8116f65c8 Fix packet flags field not being cleared properly when the packet is being reused. Add some debug statements and cleanup some. 16 years ago
Victor Julien 4824868766 Application layer detection improvements 
- improve locking of application layer handling, making sure that the flow cannot be freed/cleared when the detection engine is still working with it.
- add a check to the app layer detection to make sure that a match function will only inspect an app layer state if it's of the right type.
 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Victor Julien ba7e8012af Add some debugging and simplify locking for app layer slightly. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Gurvinder Singh 3f23f52c52 Fixed FlowTest01 and StreamReassemblyTest30 16 years ago
Anoop Saldanha ceb7e495ae refactoring perf stats code 16 years ago
Gurvinder Singh 48c94bbf03 initial thread code support 16 years ago
Gurvinder Singh 7895748606 add unit test for no stream reassembly 16 years ago
Gurvinder Singh 71da019734 some minor changes 16 years ago
Gurvinder Singh bbb385422d function to set packet flags 16 years ago
Gurvinder Singh cc0b4f7181 initial version to support detection byepass 16 years ago
Victor Julien 91bc83e5c6 More logging API usage changes. 16 years ago
Victor Julien f4acd5a27e First batch of fixes for new debug and logging API usage. 16 years ago
Victor Julien 597d0e9a20 Fix detection of failed thread startup. Cleanup startup output a bit. 16 years ago
Victor Julien b6deadd2b4 Fixed a few missing places for tv->flags access. Changed mutex init for packets slightly. 16 years ago
Anoop Saldanha 3a45b2711c Fixed the Perf API startup issue 16 years ago
Victor Julien b102ea2123 Big update: 
- Implement "closing" state in flow.
- Add protocol specific timeouts.
- Lots of stream tracking updates, fixing a lot of out of window issues.
- Stream reassembly fixes.
- Implement a new IDS runmode with 4 stream and detect threads.
- Added a BUG_ON macro that aborts the engine if the expression is true.
- Better balance the flow queue handler for traffic that doesn't have flow (like icmp currently).
- Simplify application level protocol in the Tcp Session.
- Add some debugging memory counters.
 16 years ago
Victor Julien b4f0e82463 Small flow updates. 16 years ago
Gurvinder Singh 9dcf6e2de2 handle the FLOW_STATE_CLOSED 16 years ago
Gurvinder Singh ced5157dc9 Flow get state protocol specific 16 years ago
Gurvinder Singh ab76253ec6 registering stream timeouts and flow prunning unit tests 16 years ago
Gurvinder Singh a4ad7939d2 proto specific free function 16 years ago
Gurvinder Singh 55cdf8947e some modification in protocol specifc timeouts and free funtion 16 years ago
Gurvinder Singh efcde9f52e Some fixes for timeouts 16 years ago
Gurvinder Singh e3b76dcaad efficient protocol specific timeouts 16 years ago
Gurvinder Singh b46d9dc2b4 part2 and part3.1 of timout task 16 years ago
Gurvinder Singh 59e2232211 setting timeout on first packet in the flow 16 years ago
Gurvinder Singh a31a261737 Another and right way i guess for timeouts 16 years ago
Gurvinder Singh 0e64fa78b4 Protocol Specific Timeouts 16 years ago
Anoop Saldanha f35d9f0437 threading improvements. Replaced the use of slot(2/3) with varslot. Improve error handling in slot functions. Additional helper functions for thread creation 16 years ago
Victor Julien 3636ca9703 Adding a "flow" queue handler. This queue handler passes packets of the same flow to the same queue. Changed the default IDS mode to use this. 
Some output cleanups, shutdown should be cleaner now.
 16 years ago
Victor Julien b26b5aa462 More cleanups. 16 years ago
Victor Julien d67eec918f Adapted flow comments to doxygen style. 16 years ago
Victor Julien 5501212e62 Fix sizeof printf related compiler warnings. 16 years ago
Brian Rectanus fa5939ca91 64 bit cleanup part2 16 years ago
Anoop Saldanha 53c21410b6 improve the threading api for the ids 16 years ago
Victor Julien dbe2a86ffe valgrind drd: fix flow mutex reinitialization in the flow subsystem when called by unittests. 16 years ago
Victor Julien 9854c19a88 Adapt Flow subsys init function to be able to initialize quietly for us in unit tests. Add flow to PPP unit tests. Fixes a floating point exception error. 16 years ago
Victor Julien ffa013b2d8 Implement flow:established and flow:stateless 16 years ago
Victor Julien 51a9e36e10 Remove vips references. Rename to eidps. 16 years ago
Victor Julien 9c7f5afa79 Large update: pcap support, threading fixes, initial stream tracking, time handling, pool support, runmodes, decoders added, autojunk update. 16 years ago
Victor Julien ff4b5a5db7 Add support for flowbits. 16 years ago
Victor Julien 657be002d1 Big detection engine update: scan improvements, b2g/b3g updates, bloom fixes, iponly detection implementation, dsize/flow grouping. 16 years ago
Victor Julien 4c4862d838 Improve logging, add alert-output module, at module exit stats, add HTTP POST uri capture. 16 years ago
Victor Julien bab4b62376 Initial add of the files. 16 years ago