Commit Graph

180 Commits (3dceca70ee7b73c8b4587fa5f4c4c161256e4fff)

Author SHA1 Message Date
Ken Steele c6a8d0ab6b Share Packet checksum values for TCP, UDP, IPv6. ICMPv4 and ICMPv6
Keep a separate checksum for IPV4, since a packet can have both an IPV4
checksum and a TCPV4 checksum, or IPV4 and UDPV4 checksum.

This will allow future sharing of more values.

Use PACKET_RESET_CHECKSUMS() in Unit Tests in place of setting the
individual checksum values.
12 years ago
Eric Leblond eea760de3c decode: fix typo in comment 12 years ago
Victor Julien edeeb7ed44 Store TX id with alerts
When generating an alert and storing it in the packet, store the tx_id
as well. This way the output modules can log the tx_id and access the
proper tx for logging.

Issue #904.
12 years ago
Anoop Saldanha b1dffdfbe0 Add app layer protocol packet event detection support. 12 years ago
Ken Steele 316190c6b9 Add TILE-Gx mPIPE packet processing support.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h

A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline.  It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h

The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.

The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack.  This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.

The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.

The code is not yet optimized for high performance. Performance
improvements will follow shortly.

The code was originally written by Tom Decanio and then further
modified by Tilera.

This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
12 years ago
Victor Julien 16c3487444 Add yaml option to disable vlan ids hashing
In some cases using the vlan id(s) in flow hashing is problematic. Cases
of broken routers have been reported. So this option allows for disabling
the use of vlan id(s) while calculating the flow hash, and in the future
other hashes.

Vlan tracking for flow is enabled by default.
12 years ago
Victor Julien 58ed1f2411 flow: take vlan_id's into account in the flow hash
In VLAN we can have 2 layers of encapsulation. In this patch both
layers are used in the flow hash to distinguish between encapsulated
traffic.
12 years ago
Ken Steele b076a26cdc Replace ReleaseData function on Packet Structure with ReleasePacket.
This commit allows handling Packets allocated by different methods.
The ReleaseData function pointer in the Packet structure is replaced
with ReleasePacket function pointer, which is then always called to
release the memory associated with a Packet.

Currently, the only usage of ReleaseData is in AF Packet. Previously
ReleaseData was only called when it was not NULL. To implement the
same functionality as before in AF Packet, a new function is defined
in AF Packet to first call the AFP specific ReleaseData function and
then releases the Packet structure.

Three new general functions are defined for releasing packets in the
default case:
    1) PacketFree() - To release a packet alloced with SCMalloc()
    2) PacketPoolReturnPacket() - For packets allocated from the Packet Pool.
                                  Calls RECYCLE_PACKET(p)
    3) PacketFreeOrRelease() - Calls PacketFree() or PacketPoolReturnPacket()
                                 based on the PKT_ALLOC flag.

Having these functions removes the need to check the PKT_ALLOC flag
when releasing a packet in most cases, since the ReleasePacket
function encodes how the Packet was allocated. The PKT_ALLOC flag is
still set and is needed when AF Packet releases a packet, since it
replaces the ReleasePacket function pointer with its own function and
then calls PacketFreeOfRelease(), which uses the PKT_ALLOC flag.
12 years ago
Victor Julien f4dcba6de3 In case of fragments, don't consider ports. Bug #847. 12 years ago
Eric Leblond 2f2916d9ec ccccinelle: add formatted comment for flag test 12 years ago
Victor Julien 4a0050b9ea Print pkt src to alert-debug log 12 years ago
Eric Leblond c0c59fbd17 decode: factorize macro code
PACKET_* are now wrapper to the newly introduced PACKET_SET_ACTION
macro.
12 years ago
Eric Leblond 3f107fa130 decode: Packet action start with PACKET
Rename all Packet action macro to have them prefixed by PACKET.
12 years ago
Anoop Saldanha 602c91ed41 Minor cosmetic changes to the cuda code.
Moved a couple of functions to more cuda relevant files;
Re-structured some data types.
12 years ago
Anoop Saldanha 17c763f855 Version 1 of AC Cuda. 12 years ago
Anoop Saldanha b787da5643 Remove all cuda related code in the engine except for the cuda api wrappers 12 years ago
Eric Leblond efaa9a7302 action handling: define and use macros
The action field in Packet structure should not be accessed
directly as the tunneled packet needs to update the root packet
and not the initial packet.

This patch is fixing issue #819 where suricata was not able to
drop fragmented packets in AF_PACKET IPS mode. It also fixes
drop capability for tunneled packets.
12 years ago
Ken Steele 93e7304117 Preserve PKT_ALLOC flag inside PACKET_RECYCLE().
The PKT_ALLOC flag was being cleared by PACKET_RECYCLE(), which could
then result in a packet being pushed back to the Packet ring buffer
incorrectly.
12 years ago
Ken Steele 699d9e01f1 Move memset() out of PACKET_INITIALIZE()
The memset() inside PACKET_INITIALIZE() is redundant in some cases and
it is cleaner to do as part of the memory allocation. This simplifies
changes for integrating Tilera mPIPE support because the size of memory
cleared in that case is different from SIZE_OF_PACKET.

For the cases where Packets are directly allocated and then call
PACKET_INITIALIZE() without memset() first, this patch adds memset() calls.

A further change would use GetPacketFromAlloc() directly.
12 years ago
Victor Julien f5cd7c6a92 decode events: add debug statement 13 years ago
Victor Julien e30b1bfe64 Simple IP reputation implementation 13 years ago
Anoop Saldanha f08497d1e4 Move Flow Reference/Dereferene api from flow-util.h to flow.h.
Remove duplicate FlowDeReference from decode.h
13 years ago
Anoop Saldanha 67981d1c5c Update suricata to use FlowReference/FlowDeReference for the ones left out
from last update.
13 years ago
Victor Julien a01130d2ed packet src: move pkt_src field up in the structure to fix in an existing hole (found with pahole -C Packet_ src/.libs/suricata). 13 years ago
Anoop Saldanha b33986c887 Add a packet src for every packet generated inside suricata. 13 years ago
Eric Leblond 775f379e2b decode: clean release function 13 years ago
Eric Leblond f241312a36 defrag: don't use message for repetitive error
When nothing can be fetch from the pool, this can repeat frequently.
Thus displaying a message in the log will not help. This patch
uses a counter instead of a log message. As this is a sort of memcap
this is conformed to what is done for other issues of the same type.
13 years ago
Eric Leblond 2011a3f87e capture: add data release mechanism
This patch adds a data release mechanism. If the capture module
has a call to indicate that userland has finished with the data,
it is possible to use this system. The data will then be released
when the treatment of the packet is finished.

To do so the Packet structure has been modified:
+    TmEcode (*ReleaseData)(ThreadVars *, struct Packet_ *);
If ReleaseData is null, the function is called when the treatment
of the Packet is finished.
Thus it is sufficient for the capture module to code a function
wrapping the data release mechanism and to assign it to ReleaseData
field.

This patch also includes an implementation of this mechanism for
AF_PACKET.
13 years ago
Eric Leblond e6e339aacf Add counters for IPv4 in IPv6 and IPv6 in IPv6 13 years ago
Eric Leblond f9046d8284 Add teredo counter. 13 years ago
Anoop Saldanha 32183faa82 free flowvar entries in flow after live rule swap. Sync flowbits entries into packet struct to be used by alert debuglog when alert debuglog is enabled 13 years ago
Victor Julien ea0d172693 No longer pass StreamMsg to output for alert logging, instead use the same callback code as is used for state alerts. 13 years ago
Victor Julien 98c30be2db ipv6: improve handling of packets with duplicate (or more) ipv6 extension headers. 13 years ago
Victor Julien 79691f675a defrag: don't increment recursion level for reassembled packets. Fixes defragged packets not seeing the same flow. 13 years ago
Victor Julien 92e7a7d254 ipv6: fix detection engine using the originals IPv6 header's nxt hdr value instead of the upper layer one. 14 years ago
Victor Julien d908e707d7 profiling: add per lock location profiling
Add profiling per lock location in the code. Accounts how often a
lock is requested, how often it was contended, the max number of
ticks spent waiting for it, avg number of ticks waiting for it and
the total ticks for that location.

Added a new configure flag --enable-profiling-locks to enable this
feature.
14 years ago
Victor Julien 979edf0b97 Add way to profile mutex/spin locks per thread module. 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Eric Leblond 34b3f19465 af-packet: Implement zero copy
This patch adds support for zero copy to AF_PACKET running mode.
This requires to use the 'worker' mode which is the only one where
the threading architecture is simple enough to permit this without
heavy modification.
14 years ago
Eric Leblond 3593cb051e decode: add PacketSetData funtion
This patch adds a function which can be used to set the payload
of a packet when a zero copy mode is used.
14 years ago
Eric Leblond c3eaa6cc60 Add per-interface counter for invalid checksum.
This patch adds a per-device counter for invalid checksum as
well as a simple packet counter.
14 years ago
Eric Leblond 551cb3e4c2 decode: introduce checksum mode enum. 14 years ago
Eric Leblond 81bc6f5518 Treat incomplete checksum.
Checksum of local traffic is often offloaded to the network device.
This causes some problems on parsing of this traffic. This patch
introduces a PKT_INCOMPLETE_CHECKSUM flag which can be used to
indicate that the checksum is not computed/correct for good reason.
14 years ago
Eric Leblond 6e7a8f38bf ipfw: Add support for autofp and worker runmode
This patch convert ipfw code to the PcktAcqLoop API and
rework the running mode to use the running mode wrapper
already used by NFQ.
14 years ago
Anoop Saldanha 7c9d1b80fd Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls 14 years ago
Anoop Saldanha 01a35bb604 introduce app layer local storage api support 14 years ago
Eric Leblond 60a99915c1 doc: create http support group
This patch create an httplayer group and adds related files to
it. It also fixes some typo in documentation string and format.
14 years ago
Victor Julien 2e2e80b812 Add packet alert flag to indicate a match happened (partly) in the app layer state. Make unified2 use this flag. 14 years ago
Victor Julien 4c641f0deb Fix compilation with profiling enabled. 14 years ago
Anoop Saldanha 246a4e9fff for shutdown reassembly properly init the reassembly packet using PACKET_RECYCLE 14 years ago