Commit Graph

72 Commits (98c30be2db5ecafd26f29303f95409cc8379b95f)

Author SHA1 Message Date
Victor Julien 1d59324a68 Add missing space to http.log. 13 years ago
Victor Julien b5a3995904 Fix minor memleak in an start up error condition. 13 years ago
Anoop Saldanha 69ed12fd28 Introduce new buffer API that lets you create and manage a buffer. Update http log to use this as well 13 years ago
Victor Julien 4df25ef499 Apply http.log formatting fix by Chris Wakelin. 13 years ago
Anoop Saldanha 090d098114 provide generic macro to buffer data using snprintf 13 years ago
Anoop Saldanha 23b64c5c08 restructure http logging to use fine grained locking 13 years ago
Victor Julien 19a7e7f395 flow: create a flow lock macro API, implement it for mutex and rwlocks. Mutex remains the default. 14 years ago
Victor Julien 9696902b68 Small http.log improvement: bail out early if there is nothing to log. Make output locking more fine grained. 14 years ago
Victor Julien 11bdf4838f Various improvements to error handling found by Coverity. 14 years ago
Victor Julien 7066a79c10 Register HTTP logger at registration, not thread init. 14 years ago
Victor Julien 28e15be526 Clean up default output. Use simpler output format for releases. 14 years ago
Mike Pomraning dfec9c0f6a Switch 'fast', 'http-log', 'drop' and 'alert-debug' to SCConfLogOpenGeneric. 14 years ago
Victor Julien 5395071c11 Make http logging code more robust against cases where the htp state is incomplete (out of memory conditions). 14 years ago
Victor Julien 7eb83314b4 Fix compiler warning and fix using GET_IPV4_DST_ADDR_PTR macro to access IPv6 header. 14 years ago
Eric Leblond 1df183ac38 http log: factorize logging function.
With the introduction of the PrintInet function there was almost
no difference between IPv4 and IPv6 HTTP logging functions. This
patch adds a wrapper that factorizes the code.
14 years ago
Eric Leblond 2a8ffe07ea http log: factorize extended logging
Extended logging is not dependant on IP protocol version.
14 years ago
Eric Leblond a5b1de4f0d http log: Add extended option
This patch adds a extended option to log extended HTTP information
when activated.
14 years ago
Chris Wakelin 8b81063fc2 http log: Add extended information 14 years ago
Anoop Saldanha 4e44073c79 http logging module should log all txs in the list and not just the last complete tx available on EOF 14 years ago
Anoop Saldanha 4307ea2348 Replace all frees with SCFrees 14 years ago
Eileen Donlon 89599d3b9b fixed bug 288; corrected config boolean parsing problems 14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Eric Leblond 6b9d1012ff Transform inet_ntop call into PrintInet one. 14 years ago
Victor Julien d48ff8f6aa Extend 'append' option to stats.log as well. Small cleanups. 15 years ago
Gurvinder Singh f4392e1dcc added support for appending the log files 15 years ago
Victor Julien 3fcfaef9f7 Fix compiler warning in log-httplog.c & change stats.log to log as mm/dd/yyyy as well. 15 years ago
Gurvinder Singh 791d177c7f fixed the timestamp issue in http.log 15 years ago
Victor Julien 412498f4e4 Converts port vars in http logger from uint32_t to Port and update output. 15 years ago
Gurvinder Singh b7ff6537d2 fixed the incorrect port issue in http.log 15 years ago
Victor Julien 6c6e6321fc Fix HTTP HEAD detection code. 15 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 15 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 15 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
15 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 15 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 15 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 15 years ago
Victor Julien 2dd28ea7fd Use threadsafe time functions. 15 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 15 years ago
Gerardo Iglesias Galvan 9f35a24a1f Set threads name. Fix bug #83 15 years ago
Jason Ish 40f9653c06 Have output plugs use an OutputCtx which is a little more generic than LogFileCtx. The OutputCtx provides a place for module private data to avoi overriding the LogFileCtx. 16 years ago
Victor Julien 7a8cd61fdf Cleanups. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Victor Julien f442c1f5ec Make sure log-http checks for a valid p->flow ptr before using it. 16 years ago
Victor Julien d6c53b68bf Fix two separate segv's in the http logging code. 16 years ago
Victor Julien 45b85c063a Fix segv in http log module. 16 years ago
Gurvinder Singh 50f7d0a887 app layer htp logging and better htp request handling. removed recent_in_tx. 16 years ago
Pablo Rincon d0404d8447 Renaming errors with naming conventions 16 years ago
Steve Grubb c95cd2e80a memory leak cleanups in misc places
Hello,

This is all the rest of the memory leaks I found.

*In src/source-pcap-file.c at line 152, ptv is not being freed.
*In src/util-unittest-helper.c at line 152, p was not being freed.
*In src/log-httplog.c at line 195, aft was not being freed
*In src/counters.c at line 51, log_filename was not being freed. At line 1188
pctx is being tested to see if its NULL. However, at 1173 it exits the
function if it were NULL. This test is not needed and should be deleted.
*In src/defrag.c at line 351, tracker was not being freed. At line 390, dc is
being checked for NULL but this was already done at line 384. Probably what
was meant was checking the value of dc->frag_table which was just assigned.

The patch below makes the above described changes.

-Steve
16 years ago
Jason Ish 095f2cf6ef Consistency fix.. Xxxlog -> XxxLog. 16 years ago
Jason Ish e204d07717 Have output modules register themselves so run mode configurator becomes aware of them for purposes of being configured from the config file. 16 years ago