Add TILE-Gx mPIPE packet processing support.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h
A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline. It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h
The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.
The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack. This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.
The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.
The code is not yet optimized for high performance. Performance
improvements will follow shortly.
The code was originally written by Tom Decanio and then further
modified by Tilera.
This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
12 years ago
|
|
|
/* Copyright (C) 2007-2013 Open Information Security Foundation
|
|
|
|
|
*
|
|
|
|
|
* You can copy, redistribute or modify this Program under the terms of
|
|
|
|
|
* the GNU General Public License version 2 as published by the Free
|
|
|
|
|
* Software Foundation.
|
|
|
|
|
*
|
|
|
|
|
* This program is distributed in the hope that it will be useful,
|
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
* GNU General Public License for more details.
|
|
|
|
|
*
|
|
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
|
* version 2 along with this program; if not, write to the Free Software
|
|
|
|
|
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
|
|
|
|
|
* 02110-1301, USA.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/** \file
|
|
|
|
|
*
|
|
|
|
|
* \author Victor Julien <victor@inliniac.net>
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifndef __RUNMODES_H__
|
|
|
|
|
#define __RUNMODES_H__
|
|
|
|
|
|
|
|
|
|
/* Run mode */
|
|
|
|
|
enum {
|
|
|
|
|
RUNMODE_UNKNOWN = 0,
|
|
|
|
|
RUNMODE_PCAP_DEV,
|
|
|
|
|
RUNMODE_PCAP_FILE,
|
|
|
|
|
RUNMODE_PFRING,
|
|
|
|
|
RUNMODE_NFQ,
|
|
|
|
|
RUNMODE_NFLOG,
|
|
|
|
|
RUNMODE_IPFW,
|
|
|
|
|
RUNMODE_ERF_FILE,
|
|
|
|
|
RUNMODE_DAG,
|
|
|
|
|
RUNMODE_AFP_DEV,
|
|
|
|
|
RUNMODE_NETMAP,
|
Add TILE-Gx mPIPE packet processing support.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h
A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline. It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h
The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.
The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack. This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.
The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.
The code is not yet optimized for high performance. Performance
improvements will follow shortly.
The code was originally written by Tom Decanio and then further
modified by Tilera.
This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
12 years ago
|
|
|
RUNMODE_TILERA_MPIPE,
|
|
|
|
|
RUNMODE_UNITTEST,
|
|
|
|
|
RUNMODE_NAPATECH,
|
unix-manager: add unix command socket and associated script
This patch introduces a unix command socket. JSON formatted messages
can be exchanged between suricata and a program connecting to a
dedicated socket.
The protocol is the following:
* Client connects to the socket
* It sends a version message: { "version": "$VERSION_ID" }
* Server answers with { "return": "OK|NOK" }
If server returns OK, the client is now allowed to send command.
The format of command is the following:
{
"command": "pcap-file",
"arguments": { "filename": "smtp-clean.pcap", "output-dir": "/tmp/out" }
}
The server will try to execute the "command" specified with the
(optional) provided "arguments".
The answer by server is the following:
{
"return": "OK|NOK",
"message": JSON_OBJECT or information string
}
A simple script is provided and is available under scripts/suricatasc. It
is not intended to be enterprise-grade tool but it is more a proof of
concept/example code. The first command line argument of suricatasc is
used to specify the socket to connect to.
Configuration of the feature is made in the YAML under the 'unix-command'
section:
unix-command:
enabled: yes
filename: custom.socket
The path specified in 'filename' is not absolute and is relative to the
state directory.
A new running mode called 'unix-socket' is also added.
When starting in this mode, only a unix socket manager
is started. When it receives a 'pcap-file' command, the manager
start a 'pcap-file' running mode which does not really leave at
the end of file but simply exit. The manager is then able to start
a new running mode with a new file.
To start this mode, Suricata must be started with the --unix-socket
option which has an optional argument which fix the file name of the
socket. The path is not absolute and is relative to the state directory.
THe 'pcap-file' command adds a file to the list of files to treat.
For each pcap file, a pcap file running mode is started and the output
directory is changed to what specified in the command. The running
mode specified in the 'runmode' YAML setting is used to select which
running mode must be use for the pcap file treatment.
This requires modification in suricata.c file where initialisation code
is now conditional to the fact 'unix-socket' mode is not used.
Two other commands exists to get info on the remaining tasks:
* pcap-file-number: return the number of files in the waiting queue
* pcap-file-list: return the list of waiting files
'pcap-file-list' returns a structured object as message. The
structure is the following:
{
'count': 2,
'files': ['file1.pcap', 'file2.pcap']
}
14 years ago
|
|
|
RUNMODE_UNIX_SOCKET,
|
|
|
|
|
RUNMODE_USER_MAX, /* Last standard running mode */
|
|
|
|
|
RUNMODE_LIST_KEYWORDS,
|
|
|
|
|
RUNMODE_LIST_APP_LAYERS,
|
|
|
|
|
RUNMODE_LIST_CUDA_CARDS,
|
|
|
|
|
RUNMODE_LIST_RUNMODES,
|
|
|
|
|
RUNMODE_PRINT_VERSION,
|
|
|
|
|
RUNMODE_PRINT_BUILDINFO,
|
|
|
|
|
RUNMODE_PRINT_USAGE,
|
|
|
|
|
RUNMODE_DUMP_CONFIG,
|
|
|
|
|
RUNMODE_CONF_TEST,
|
|
|
|
|
RUNMODE_LIST_UNITTEST,
|
|
|
|
|
RUNMODE_ENGINE_ANALYSIS,
|
|
|
|
|
#ifdef OS_WIN32
|
|
|
|
|
RUNMODE_INSTALL_SERVICE,
|
|
|
|
|
RUNMODE_REMOVE_SERVICE,
|
|
|
|
|
RUNMODE_CHANGE_SERVICE_PARAMS,
|
|
|
|
|
#endif
|
|
|
|
|
RUNMODE_MAX,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
char *RunmodeGetActive(void);
|
|
|
|
|
const char *RunModeGetMainMode(void);
|
|
|
|
|
|
|
|
|
|
void RunModeListRunmodes(void);
|
|
|
|
|
void RunModeDispatch(int, const char *);
|
|
|
|
|
void RunModeRegisterRunModes(void);
|
|
|
|
|
void RunModeRegisterNewRunMode(int, const char *, const char *,
|
|
|
|
|
int (*RunModeFunc)(void));
|
|
|
|
|
void RunModeInitialize(void);
|
|
|
|
|
void RunModeInitializeOutputs(void);
|
|
|
|
|
void SetupOutputs(ThreadVars *);
|
|
|
|
|
void RunModeShutDown(void);
|
|
|
|
|
|
|
|
|
|
/* bool indicating if file logger is enabled */
|
|
|
|
|
int RunModeOutputFileEnabled(void);
|
|
|
|
|
/* bool indicating if filedata logger is enabled */
|
|
|
|
|
int RunModeOutputFiledataEnabled(void);
|
|
|
|
|
|
|
|
|
|
#include "runmode-pcap.h"
|
|
|
|
|
#include "runmode-pcap-file.h"
|
|
|
|
|
#include "runmode-pfring.h"
|
Add TILE-Gx mPIPE packet processing support.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h
A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline. It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h
The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.
The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack. This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.
The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.
The code is not yet optimized for high performance. Performance
improvements will follow shortly.
The code was originally written by Tom Decanio and then further
modified by Tilera.
This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
12 years ago
|
|
|
#include "runmode-tile.h"
|
|
|
|
|
#include "runmode-nfq.h"
|
|
|
|
|
#include "runmode-ipfw.h"
|
|
|
|
|
#include "runmode-erf-file.h"
|
|
|
|
|
#include "runmode-erf-dag.h"
|
|
|
|
|
#include "runmode-napatech.h"
|
|
|
|
|
#include "runmode-af-packet.h"
|
|
|
|
|
#include "runmode-nflog.h"
|
unix-manager: add unix command socket and associated script
This patch introduces a unix command socket. JSON formatted messages
can be exchanged between suricata and a program connecting to a
dedicated socket.
The protocol is the following:
* Client connects to the socket
* It sends a version message: { "version": "$VERSION_ID" }
* Server answers with { "return": "OK|NOK" }
If server returns OK, the client is now allowed to send command.
The format of command is the following:
{
"command": "pcap-file",
"arguments": { "filename": "smtp-clean.pcap", "output-dir": "/tmp/out" }
}
The server will try to execute the "command" specified with the
(optional) provided "arguments".
The answer by server is the following:
{
"return": "OK|NOK",
"message": JSON_OBJECT or information string
}
A simple script is provided and is available under scripts/suricatasc. It
is not intended to be enterprise-grade tool but it is more a proof of
concept/example code. The first command line argument of suricatasc is
used to specify the socket to connect to.
Configuration of the feature is made in the YAML under the 'unix-command'
section:
unix-command:
enabled: yes
filename: custom.socket
The path specified in 'filename' is not absolute and is relative to the
state directory.
A new running mode called 'unix-socket' is also added.
When starting in this mode, only a unix socket manager
is started. When it receives a 'pcap-file' command, the manager
start a 'pcap-file' running mode which does not really leave at
the end of file but simply exit. The manager is then able to start
a new running mode with a new file.
To start this mode, Suricata must be started with the --unix-socket
option which has an optional argument which fix the file name of the
socket. The path is not absolute and is relative to the state directory.
THe 'pcap-file' command adds a file to the list of files to treat.
For each pcap file, a pcap file running mode is started and the output
directory is changed to what specified in the command. The running
mode specified in the 'runmode' YAML setting is used to select which
running mode must be use for the pcap file treatment.
This requires modification in suricata.c file where initialisation code
is now conditional to the fact 'unix-socket' mode is not used.
Two other commands exists to get info on the remaining tasks:
* pcap-file-number: return the number of files in the waiting queue
* pcap-file-list: return the list of waiting files
'pcap-file-list' returns a structured object as message. The
structure is the following:
{
'count': 2,
'files': ['file1.pcap', 'file2.pcap']
}
14 years ago
|
|
|
#include "runmode-unix-socket.h"
|
|
|
|
|
#include "runmode-netmap.h"
|
|
|
|
|
|
|
|
|
|
int threading_set_cpu_affinity;
|
|
|
|
|
extern float threading_detect_ratio;
|
|
|
|
|
|
|
|
|
|
extern int debuglog_enabled;
|
|
|
|
|
|
|
|
|
|
#endif /* __RUNMODES_H__ */
|
