Commit Graph

64 Commits (88a21456e324f589575c4bccfaa63dee0e3b301a)

Author SHA1 Message Date
Victor Julien 19a7e7f395 flow: create a flow lock macro API, implement it for mutex and rwlocks. Mutex remains the default. 14 years ago
Victor Julien 28e15be526 Clean up default output. Use simpler output format for releases. 14 years ago
Victor Julien 2c62b50ed5 Fix 2 compiler warnings. 14 years ago
Mike Pomraning dfec9c0f6a Switch 'fast', 'http-log', 'drop' and 'alert-debug' to SCConfLogOpenGeneric. 14 years ago
Victor Julien e6d8d0443c Unify output functions for alert-debug for IPv4 and IPv6. 14 years ago
Eric Leblond 2073b9db0c debuglog: uses state selection system. 14 years ago
Eric Leblond 1596241687 debuglog: fix segment logging.
StreamSegmentForEach returns the number of segments or < 0 in case
of error. This patch synchronizes debuglog output module with this
behaviour.
14 years ago
Anoop Saldanha 67be07bf15 fix threading issue in debug log. locked mutex isn't freed before returning. fixed 14 years ago
Eric Leblond 2fa837bcec alert-debuglog: Add logging of stream segments.
This patch introduces logging of the stream segments in case of
a signature match on application layer.
14 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Eileen Donlon 89599d3b9b fixed bug 288; corrected config boolean parsing problems 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Eric Leblond 6b9d1012ff Transform inet_ntop call into PrintInet one. 14 years ago
Victor Julien 75439863ed Shrink PacketAlerts structure so that Packet structure is a lot smaller. Reduce max events per packet from 256 to 15. 14 years ago
Victor Julien a5d9c86dd3 Shrink Flow structure with 20 bytes (on 32 bit) and reorder it. Clean up init, recycle, destroy macro's. 14 years ago
Anoop Saldanha fe6e41e3ef Removed FLOW_AL_NO_APPLAYER_INSPECTION. Moved it as FLOW_NO_APPLAYER_INSPECTION in Flow->flags. Turned Flow->flags into uint32_t and removed Flow->alflags 14 years ago
Anoop Saldanha ac5584a863 Removed FLOW_AL_PROTO_DETECT_DONE. Replaced it with FLOW_ALPROTO_DETECT_DONE, stored it in Flow->flags 14 years ago
Victor Julien 864c8718e1 Store matching stream msg (ptr) in packets alert structure so it's available to the output plugins. 15 years ago
Victor Julien 6f58ef13c4 Improve error cleanup in output function. Thanks to iswalker. 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
15 years ago
Victor Julien d48ff8f6aa Extend 'append' option to stats.log as well. Small cleanups. 15 years ago
Gurvinder Singh f4392e1dcc added support for appending the log files 15 years ago
Gurvinder Singh ba18110abd support for stats.log configurable and fixed timezone issue in faslog and debuglog 15 years ago
Victor Julien 39cb1bdbda Fix app layer sigs being recognized as decoder event only or ip only. 15 years ago
Victor Julien d41b5645ef Make sure decoder event rules are inspected even if the packet is invalid and has no addesses or proto. Update fast log and alert debug log to display the alerts. Fixes #179. 15 years ago
Gurvinder Singh 8852b83fa7 flowbits, flowvars, pktvars, flow flags and app layer info added to alert-debug.log 15 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
Victor Julien 6dd5446893 Use proper tcp/udp macro's in alert-debuglog 15 years ago
Victor Julien cc5c78dae9 Add tcp seq info and pcap file packet number to the alert-debuglog. 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 15 years ago
Victor Julien 2dd28ea7fd Use threadsafe time functions. 15 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 15 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 15 years ago
Jason Ish 40f9653c06 Have output plugs use an OutputCtx which is a little more generic than LogFileCtx. The OutputCtx provides a place for module private data to avoi overriding the LogFileCtx. 16 years ago
Victor Julien dd846c9b0e Remove all search code from the pattern matchers, cleanup mpm api, remove unused http code, more cleanups. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Pablo Rincon cef12d30b5 Unified output fixes: alert count per module (not per thread), fix timestamps on pcap mode, write *all* the alerts of a packet, write the log header once also on unified alert 16 years ago
Steve Grubb f6653752c5 memory leak cleanup in alerts
Hello,

I ran the code through an analysis program and found several memory leaks
in the alert code.

*In src/alert-fastlog.c at line 178, aft was not being freed
*In src/alert-debuglog.c at line 205, aftwas not being freed
*In src/alert-unified-log.c at lines 234 and 243, aun was not being freed
*In src/alert-unified-alert.c at lines 219 and 230, aun was not being freed
*In src/alert-unified2-alert.c at line 505, aun was not being freed

The patch below fixes this.

-Steve
16 years ago
Jason Ish 095f2cf6ef Consistency fix.. Xxxlog -> XxxLog. 16 years ago
Jason Ish e204d07717 Have output modules register themselves so run mode configurator becomes aware of them for purposes of being configured from the config file. 16 years ago
Jason Ish 844c444af1 Use the configuration file to setup alert logging (and http logging).
Only setup for the live pcap modes at the moment.
16 years ago
Anoop Saldanha 8189f4d88e Change error log messags to debug ones in the log modules 16 years ago
Victor Julien 9e5f7459c2 Actually use classification msg 16 years ago
Anoop Saldanha 4d430060d2 fix for unclear error messages bug 15 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago