Commit Graph

128 Commits (4e04cd2d1be85cbd41c34ea922302121d05d55c9)

Author SHA1 Message Date
Duarte Silva 4e04cd2d1b Adding XFF support to EVE alert output
- Created app-layer-htp-xff.c and app-layer-htp-xff.h
- Added entries in the Makefile.am
- Added the necessary configuration options to EVE alert section
- Updated Unified2 XFF configuration comments and removed unnecessary whitespace
- Created a generic function to parse the configuration
- Release the flow locks sooner and remove debug logging
- Added XFF support to EVE alert output
11 years ago
Duarte Silva 3a18db13dc Simple code fixes
- Removed unnecessary assignment of the data field
- Removed else condition (same function called for IPv4 and IPV6)
- Fixed constants to be a power of two (used in bitwise operations)
11 years ago
Ken Steele c91b7fd3bc More structures that need to be marked Packed.
These structures are cast from raw packet data, so they should be packed.
The case is:

typedef struct Foo_ {
} Foo;

Foo *f = (Foo *)pkt;
11 years ago
Ken Steele a38d5a0135 Implement thread specific data option when __thread is not available. 11 years ago
Victor Julien 4a104ae315 unified2: fix memory leak at shutdown
Module didn't properly free output context at shutdown. Led to a leak
in Unix Socket mode.
11 years ago
Ken Steele 497575d38e Add option on Tile-Gx for logging for fast.log alerts over PCIe
When running on a TILEncore-Gx PCIe card, setting the filetype of fast.log
to pcie, will open a connection over PCIe to a host application caleld
tile-pcie-logd, that receives the alert strings and writes them to a file
on the host. The file name to open is also passed over the PCIe link.

This allows running Suricata on the TILEncore-Gx PCIe card, but have the
alerts logged to the host system's file system efficiently. The PCIe API that
is used is the Tilera Packet Queue (PQ) API which can access PCIe from User
Space, thus avoiding system calls.

Created util-logopenfile-tile.c and util-logopen-tile.h for the TILE
specific PCIe logging functionality.

Using Write() and Close() function pointers in LogFileCtx, which
default to standard write and close for files and sockets, but are
changed to PCIe write and close functions when a PCIe channel is
openned for logging.

Moved Logging contex out of tm-modules.h into util-logopenfile.h,
where it makes more sense. This required including util-logopenfile.h
into a couple of alert-*.c files, which previously were getting the
definitions from tm-modules.h.

The source and Makefile for tile-pcie-logd are added in contrib/tile-pcie-logd.

By default, the file name for fast.log specified in suricata.yaml is used as
the filename on the host. An optional argument to tile-pcie-logd, --prefix=,
can be added to prepend the supplied file path. For example, is the file
in suricata.yaml is specified as "/var/log/fast.log" and --prefix="/tmp",
then the file will be written to "/tmp/var/log/fast.log".

Check for TILERA_ROOT environment variable before building tile_pcie_logd

Building tile_pcie_logd on x86 requires the Tilera MDE for its PCIe libraries
and API header files. Configure now checs for TILERA_ROOT before enabling
builing tile_pcie_logd in contrib/tile_pcie_logd
12 years ago
Ken Steele 235cd0211a Alert file formatting clean up.
Put { on new line for function declarations. Remove space after function
name.

Add static to unit tests delcaration.
12 years ago
Victor Julien a3b0577a1f output: add TM_FLAG_LOGAPI_TM thread module flag
The TM_FLAG_LOGAPI_TM flag indicates that a module is run by the log
api, not by the 'regular' thread module call functions.

Set flag in all all Log API users' registration code.

Purpose of this flag is in profiling. In profiling output it will be
used to list log api thread modules separately.
12 years ago
Victor Julien 4049c2f74c Packet logging API: convert unified2
Convert unified2 alert to new logging API.
12 years ago
Anoop Saldanha 429c6388f6 App layer API rewritten. The main files in question are:
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].

Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
  parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
  "dnsudp" in the rules.  A user who previously wrote a rule like this -

  "alert dnstcp....." or
  "alert dnsudp....."

  would now have to use,

  alert dns (ipproto:tcp;) or
  alert udp (app-layer-protocol:dns;) or
  alert ip (ipproto:udp; app-layer-protocol:dns;)

  The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
  callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
  STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
  functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0.  This is
  needed by unittests, which would try to clean the flow, and that would
  call the api, AppLayerParserCleanupParserState(), which would try to
  clean the app state, but the app layer now needs an ipproto to figure
  out which api to internally call to clean the state, and if the ipproto
  is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
  they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
12 years ago
Ken Steele f9705377ae Remove pkt variable from Packet structure.
The uint8_t *pkt in the Packet structure always points to the memory
immediately following the Packet structure. It is better to simply
calculate that value every time than store the 8 byte pointer.
12 years ago
Eric Leblond 28c5c68192 error checking: add missing alloc error treatment
The return of some malloc like functions was not treated in some
places of the code.
12 years ago
Victor Julien 49087f21e4 Optimizations to reduce branch misses 12 years ago
Victor Julien 1d18155a16 XFF: use per alert tx id
Use the tx id stored for each alert to find the correct XFF address
to add to the extra-data field.

In overwrite mode we still only grab the first available XFF addr,
as this address is set in the header preceeding the individual alerts.

Issue #904.
12 years ago
Victor Julien c8b71938ff Add a fallback memrchr implementation for those platforms that dont support it. Bug #963. 12 years ago
Victor Julien 02cbbd0b89 unified2: fix tags not being logged. Bug #968 12 years ago
Victor Julien 77b429c402 xff: fix unittest crashes 12 years ago
Victor Julien 05d68ce394 xff: don't do xff check if there are no alerts anyway. 12 years ago
Duarte Silva 7dbb305255 Adds X-Forwarded-For support to the Unified2 output format
- Added the Unified2 file format related constants
 - Added IPv6 support
 - Two modes of operation with a fall-back to "extra-data" mode if
   "overwrite" mode is not applicable
 - Changed the configuration loading code to handle the new
   configuration structure
 - When creating the packet that fakes the one that generated the alert
   the flow direction wasn't taken into account in overwrite mode
 - Fixed BUG_ON condition
12 years ago
Eric Leblond c5bd04f102 unittest: recycle packet before exit
To avoid an issue with flow validation, we need to recycle the packet
before cleaning the flow.
12 years ago
Eric Leblond 34abd818dd Prefix util-conf function with Config 12 years ago
Eric Leblond 54006de40c Use new function GetLogDirectory() 12 years ago
Eric Leblond 3f107fa130 decode: Packet action start with PACKET
Rename all Packet action macro to have them prefixed by PACKET.
12 years ago
Eric Leblond a35c367942 action handling: use macro for test.
Use test macro instead of direct access to action field.

This patch has been obtained by using the following
spatch file:

  @@
  Packet *p;
  expression E;
  @@

  - p->action & E
  + TEST_PACKET_ACTION(p, E)
12 years ago
Ken Steele 9c7b411a5d More PacketGetFromMalloc() to allocate packets. 12 years ago
Victor Julien 4b3166b193 unified2: more udp fixes 12 years ago
Victor Julien b54a19937f unified2: only call stream callback for TCP 12 years ago
Jake Gionet 1ac8938787 Adding support for Feature #667 13 years ago
Victor Julien 72443a0d62 unified2: append open instead of trucate open so that in case we rotate within a second we don't overwrite files. Instead we violate the limit. 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien 2a4992e7a0 inline: fix unified2 alert direction selection 13 years ago
Victor Julien 1906d317ec unified2: minor cleanups 13 years ago
Victor Julien 66856831fa unified2: big rewrite to clean up code that deals with tcp segment logging. 13 years ago
Victor Julien 02e19502c7 unified2: minor cleanups. 13 years ago
Victor Julien ea0d172693 No longer pass StreamMsg to output for alert logging, instead use the same callback code as is used for state alerts. 13 years ago
Victor Julien b8e741de9e Minor optimizations to unified2 and fast.log. 14 years ago
Victor Julien 0a80e362aa Fix some minor clang scan-build warnings. 14 years ago
Victor Julien e237841a8e Fix compilation with profiling enabled. Minor unittest fixes. 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Victor Julien eba3cecc5d Fix unified2 records generated based on reassembled stream data. 14 years ago
Victor Julien 6f0ca120d1 Make sure existing log-pcap and unified2-alert 'limit' settings don't break. 14 years ago
Anoop Saldanha 7c9d1b80fd Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls 14 years ago
Anoop Saldanha e0c13434ef bug 333 - support new Size Parsing API. Update various conf params inside the engine to use this API to parse sizes in the format xxx <-just the no represents bytes, xxxkb <- kilobytes, xxxmb <- megabytes, xxxgb <- gigabytes, where xxx is a \d+ 14 years ago
Victor Julien f47f601f09 Fix unified2 setting the wrong eth_type. 14 years ago
Eric Leblond 9422a36851 unified2: avoid to log RAW packet
If the packet datalink is ethernet, we add a fake ethernet
header to stream logging to avoid that barnyard2 create
different files.
14 years ago
Eric Leblond fc56abfcd0 unified2: log an ethernet header for stream alert.
If packet is a of type ethernet, we log the alert reconstructed
payload as an ethernet packet and not a raw packet. This will avoid
to confuse barnyard2 pcap output.
14 years ago
Victor Julien 39edb23ac4 Support stream.inline mode in unified2 tcp segments logging. 14 years ago
Victor Julien 2e2e80b812 Add packet alert flag to indicate a match happened (partly) in the app layer state. Make unified2 use this flag. 14 years ago
Eric Leblond 128261cb97 unified2: Fix event_id computation
This patch fixes event_id computation which was not incremented
for each alert.
14 years ago
Eric Leblond b3023643ec unified2: fix multiple alerts case 14 years ago