Commit Graph

30 Commits (4749420f325acc5965cd942d78ba9ce28c686eda)

Author SHA1 Message Date
Eric Leblond 786cbb1244 log-pcap: don't limit snaplen. 13 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Victor Julien 829238e49c OpenBSD 5.2 build fixes, Unit test fix. 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien 8cd460dde5 Don't display a warning when log-pcap tries to remove an already removed file. 13 years ago
Eric Leblond a0e57f58e5 OpenBSD: introduce SCLocalTime function.
This function is a wrapper to localtime_r. It is needed to avoid
a compilation warning on OpenBSD. I'm forced to type the function
to a non pointer first parameter. If not we will have to use two
differents functions in OpenBSD where tv->tv_sec is a long
(different from time_t).
13 years ago
Anoop Saldanha 6c5b596ada coverity fixes 13 years ago
Anoop Saldanha f1863370a5 clean log pcap 14 years ago
Anoop Saldanha 6392202872 restructure log pcap to use a different setup, which is resilient to thread failure/restarts 14 years ago
Anoop Saldanha fd21b83f3e don't return TM failure on failing to remove log file 14 years ago
Anoop Saldanha 4bc907414b init every new pf instance in log pcap 14 years ago
Victor Julien 372ab9c433 Another batch of minor fixed for issues found by Coverity. 14 years ago
Victor Julien 11bdf4838f Various improvements to error handling found by Coverity. 14 years ago
Nikolay Denev 6819ec8b54 Remove the underscored "sguil_base_dir" compatibility option. 14 years ago
Victor Julien 5fa195ffb3 Allow log-pcap to use both absolute and relative sguil-base-dir settings when in 'sguil' mode. 14 years ago
Victor Julien 6f0ca120d1 Make sure existing log-pcap and unified2-alert 'limit' settings don't break. 14 years ago
Anoop Saldanha 7c9d1b80fd Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls 14 years ago
Anoop Saldanha e0c13434ef bug 333 - support new Size Parsing API. Update various conf params inside the engine to use this API to parse sizes in the format xxx <-just the no represents bytes, xxxkb <- kilobytes, xxxmb <- megabytes, xxxgb <- gigabytes, where xxx is a \d+ 14 years ago
Victor Julien 1a5931e878 pcap-log: fall back to sguil_base_dir option if 'dir' isn't set. Minor cleanups. 14 years ago
William Metcalf 3b3f5816bf You spin me right round baby, right round like a rotating packet capture right round. Oh, also log file size counters are now uint64_t 14 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Victor Julien 40bf422453 Fix log-pcap compilation on Windows. 14 years ago
Victor Julien 0814e41e50 Ignore tunnel/defrag packets in log-pcap module. 15 years ago
Victor Julien acda69865a Fix potential segv in pcap logging deinit code. 15 years ago
Victor Julien bc7e21aee6 Add special sguil mode to log-pcap to support logging into date based directory structure and rotate when the day passes. Also do not log packets beyond stream reassembly depth and encrypted traffic. 15 years ago
Victor Julien 77505f8873 Allow pcap-log to log outside of default-log-dir by passing a absolute path as filename. 15 years ago
Eric Leblond 9be1f1a31c Use GET_PKT macros. 15 years ago
Victor Julien f90a5cfffd Misc pcap logging cleanups. 15 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 15 years ago