Commit Graph

5300 Commits (a18e2ef4022212c574dae3aca2df0e80502d792a)
 

Author SHA1 Message Date
Victor Julien 7fa2b8769b Update Changelog for 2.1beta1 11 years ago
Victor Julien 2bcff80d32 Update Changelog for 2.0.3 11 years ago
Ken Steele 44aeb9c637 Fix GRE Source Routing Header definition
The Source Routing Header had routing defined as a char* for a field
of variable size. Since that field was not being used in the code, I
removed the pointer and added a comment.
11 years ago
Ken Steele c91b7fd3bc More structures that need to be marked Packed.
These structures are cast from raw packet data, so they should be packed.
The case is:

typedef struct Foo_ {
} Foo;

Foo *f = (Foo *)pkt;
11 years ago
Ken Steele 045966676d Add Packed attribute on Header structures
Structures that are used to cast packet data into fields need to be packed
so that the compiler doesn't add any padding to these fields. This also helps
Tile-Gx to avoid unaligned loads because the compiler will insert code to
handle the possible unaligned load.
11 years ago
Victor Julien da1fe75975 lua: improve configure checks
The base 'lua' library has different names on different OS' and even
Linux distro's. Instead of selecting the proper one, we now just try
all. This way no OS/distro specific knowledge about the name is needed.
11 years ago
Victor Julien f453fb810f alert-json: clean up flags
Make payload/packet logging code also use the flags field in
AlertJsonOutputCtx, instead of in the LogFileCtx.
11 years ago
Giuseppe Longo b188d93630 json-alert: include HTTP info on the alerts
Extends the JSON alert output to include the HTTP data
at the time of the alert.
11 years ago
Giuseppe Longo 288f0b1fb7 json-http: refactoring output code
Splits the output code in two public functions and permits
to call them from the alert function
11 years ago
Duarte Silva db9588a2ce Use extra data structure in json alert output
to store local configuration
11 years ago
Victor Julien 6b0ff0193d stream: detect and filter out bad window updates
Reported in bug 1238 is an issue where stream reassembly can be
disrupted.

A packet that was in-window, but otherwise unexpected set the
window to a really low value, causing the next *expected* packet
to be considered out of window. This lead to missing data in the
stream reassembly.

The packet was unexpected in various ways:
- it would ack unseen traffic
- it's sequence number would not match the expected next_seq
- set a really low window, while not being a proper window update

Detection however, it greatly hampered by the fact that in case of
packet loss, quite similar packets come in. Alerting in this case
is unwanted. Ignoring/skipping packets in this case as well.

The logic used in this patch is as follows. If:

- the packet is not a window update AND
- packet seq > next_seq AND
- packet acq > next_seq (packet acks unseen data) AND
- packet shrinks window more than it's own data size
THEN set event and skip the packet in the stream engine.

So in case of a segment with no data, any window shrinking is rejected.

Bug #1238.
11 years ago
Victor Julien 7cc63918c3 ipv6: fix dst/hop header option parsing
The extension header option parsing used a uint8_t internally. However
much bigger option sizes are valid.
11 years ago
Victor Julien 2b84cd9483 defrag: use 'struct timeval' for timeout tracking
Until now the time out handling in defrag was done using a single
uint32_t that tracked seconds. This lead to corner cases, where
defrag trackers could be timed out a little too early.
11 years ago
Victor Julien 7c05685421 ipv6: set event on unsupported nh
If a next header / protocol is encountered that we can't handle (yet)
set an event. Disabled the rule by default.

    decode-event:ipv6.unknown_next_header;
11 years ago
Victor Julien bbcdb657da ipv6: more robust ipv6 exthdr handling
Skip past Shim6, HIP and Mobility header.

Detect data after 'none' header.
    decode-event:ipv6.data_after_none_header;
11 years ago
Victor Julien 938602c55e ipv6: detect frag header reserved field non-zero
Frag Header length field is reserved, and should be set to 0.

    decode-event:ipv6.fh_non_zero_reserved_field;
11 years ago
Victor Julien 8c19e5ff63 ipv6: make exthdr parsing more robust
Improve data length checks. Detect PadN option with 0 length.
11 years ago
Victor Julien abee95ca4f ipv6: set flag on type 0 routing header
Type 0 Routing headers are deprecated per RFC 5095.

This patch sets an decode event flag that can be matched on through:
    decode-event:ipv6.rh_type_0;
11 years ago
Victor Julien 83b031b4e0 ipv6 defrag: fix unfragmentable exthdr handling
Fix or rather implement handling of unfragmentable exthdrs in ipv6.
The exthdr(s) appearing before the frag header were copied into the
reassembled packet correctly, however the stripping of the frag header
did not work correctly.

Example:
The common case is a frag header directly after the ipv6 header:

[ipv6 header]->[frag header]->[icmpv6 (part1)]
[ipv6 header]->[frag header]->[icmpv6 (part2)]

This would result in:
[ipv6 header]->[icmpv6]

The ipv6 headers 'next header' setting would be updated to point to
whatever the frag header was pointing to.

This would also happen when is this case:

[ipv6 header]->[hop header]->[frag header]->[icmpv6 (part1)]
[ipv6 header]->[hop header]->[frag header]->[icmpv6 (part2)]

The result would be:
[ipv6 header]->[hop header]->[icmpv6]

However, here too the ipv6 header would have been updated to point
to what the frag header pointed at. So it would consider the hop header
as if it was an ICMPv6 header, or whatever the frag header pointed at.

The result is that packets would not be correctly parsed, and thus this
issue can lead to evasion.

This patch implements handling of the unfragmentable part. In the first
segment that is stored in the list for reassembly, this patch detects
unfragmentable headers and updates it to have the last unfragmentable
header point to the layer after the frag header.

Also, the ipv6 headers 'next hdr' is only updated if no unfragmentable
headers are used. If they are used, the original value is correct.

Reported-By: Rafael Schaefer <rschaefer@ernw.de>

Bug #1244.
11 years ago
Victor Julien e66c73abcd packet pool: init pthread key before using it
In the packet pool code, it's critical to initialize the pthread key
before using it. Applies only to the code used if __thread isn't
supported.
11 years ago
Victor Julien a441441636 packet pool: cosmetic cleanups 11 years ago
Eric Leblond 4b9eecc7f6 prscript: update URL
Buildbot server has been moved to another box.
11 years ago
Victor Julien 6de34489b3 magic: disable tests depending on magic version
Some tests depend on specific results by specific magic versions.
Disable these.
11 years ago
Eric Leblond fe82a83e79 suricata: RunUnittests now return void
RunUnittests function is now a terminal function (calling exit
before leaving).
11 years ago
Eric Leblond 0780c07043 unittests: don't register app layer test
Some tests are already registered via the function
AppLayerParserRegisterProtocolUnittests. So we don't need to
egister them during runmode initialization.
11 years ago
Victor Julien a0b421c47e Fix up mistaken style change 11 years ago
Ken Steele 228abb89ac fixup 11 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
11 years ago
Victor Julien de4e2221d8 eve: add tx_id to output for alerts and events
Add tx_id field for correlating alerts and events per tx.
11 years ago
sxhlinux c85674b0a6 Update app-layer-htp.c
When keyword "boundary=" doesn't exist in the http_header, the value of r is 0 and this condition shouldn't return 0 instead 1;
11 years ago
Ken Steele 033ad9e974 Reduce reallocation in AC Tile MPM creation.
Exponentially increase the memory allocated for new states when adding new
states, then at the end resize down to the actually final size so that no space is wasted.
11 years ago
Alexander Gozman a6dbf627b2 Add input interface's name to JSON log 11 years ago
Victor Julien 2bcd48bc12 pcap-log: yaml comment update 11 years ago
Victor Julien 0c1696f84b pcap-log: unify lock handling, fixes Coverity warn
*** CID 1229124:  Data race condition  (MISSING_LOCK)
/src/log-pcap.c: 363 in PcapLog()
357         {
358             return TM_ECODE_OK;
359         }
360
361         PcapLogLock(pl);
362
>>>     CID 1229124:  Data race condition  (MISSING_LOCK)
>>>     Accessing "pl->pkt_cnt" without holding lock "PcapLogData_.plog_lock". Elsewhere, "PcapLogData_.pkt_cnt" is accessed with "PcapLogData_.plog_lock" held 1 out of 2 times (1 of these accesses strongly imply that it is necessary).
363         pl->pkt_cnt++;
364         pl->h->ts.tv_sec = p->ts.tv_sec;
365         pl->h->ts.tv_usec = p->ts.tv_usec;
366         pl->h->caplen = GET_PKT_LEN(p);
367         pl->h->len = GET_PKT_LEN(p);
368         len = sizeof(*pl->h) + GET_PKT_LEN(p);
11 years ago
Ken Steele edb702a7b6 Cleanup Packet Pools when done. 11 years ago
Ken Steele b045fcb032 Fix Packet Stacks for non-TLS Operating Systems
On non-TLS systems, check each time the Thread Local Storage
is requested and if it has not been initialized for this thread, initialize it.
The prevents not initializing the worker threads in autofp run mode.
11 years ago
Eric Leblond 85b96d5032 travis-ci: use make check
This patch replaces unittest run by make check. It also install
coccinelle so we should have more test.
11 years ago
Eric Leblond d7982fae8c coccinelle: fix problem with coccinelle 1.0rc21
coccinelle 1.0rc21 has a problem with regular expression handling.
This result in a Fatal Error when test system detects an coding
error.

This patch fixes the problem by using a simple blob inside
semantic patch instead of using a regular expression to define
the function.

It also fixes add an optimization on matching suppressing a
useless <.. ..> construction.

Fixes have been suggested by Julia Lawall.
11 years ago
Victor Julien 0ac94ef777 flow-recycler: support multiple instances
Use new management API to run the flow recycler.

Make number of threads configurable:

flow:
  memcap: 64mb
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30
  managers: 2
  recyclers: 2

This sets up 2 flow recyclers.
11 years ago
Victor Julien e0841218f0 flow-manager: support multiple instances
Use new management API to run the flow manager.

Support multiple flow managers, where each of them works with it's
own part of the flow hash.

Make number of threads configurable:

flow:
  memcap: 64mb
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30
  managers: 2

This sets up 2 flow managers.

Handle misc tasks only in instance 1: Handle defrag hash timeout
handing, host hash timeout handling and flow spare queue updating
only from the first instance.
11 years ago
Victor Julien 46cee88ef8 threads: add management API
Currently management threads do their own thread setup and handling. This
patch introduces a new way of handling management threads.

Functionality that needs to run as a management thread can now register
itself as a regular 'thread module' (TmModule), where the 'Management'
callback is registered.
11 years ago
Victor Julien f1185d051c flow id: quick and dirty first stab at a flow id
Add a 'flow_id' that is the same for all records produced for packets
belonging to the same flow.

This patch simply takes the flow's memory address.
11 years ago
Victor Julien 9f55ca0057 flow: add flow_end_flags field, add logging
The flow end flags field is filled by the flow manager or the flow
hash (in case of forced timeout of a flow) to record the timeout
conditions in the flow:
- emergency mode
- state
- reason (timed out or forced)

Add logging to the flow logger.
11 years ago
Victor Julien fc6ad56944 flow: move FlowGetFlowState
Move FlowGetFlowState to flow-private.h so that all parts of the flow
engine can use it.
11 years ago
Victor Julien e6ed6731b1 flow log: log TCP state
Log the TCP state at timeout.
11 years ago
Victor Julien 8c231702d9 flow-recycler: speed up flow-recycler shutdown
Thread was killed by the generic TmThreadKillThreads instead of
the FlowKillFlowRecyclerThread. The latter wakes the thread up, so
that shutdown is quite a bit faster.
11 years ago
Victor Julien 6f9a2fcd58 flow: log individual tcp flags
Log the tcp flags.
11 years ago
Victor Julien f4dfaacff3 netflow: log individual tcp flags
Log the tcp flags.
11 years ago
Victor Julien eaf01449e3 json: add tcp flags to json utility function
Turns a flags bitfield into a set of json bools.
11 years ago
Victor Julien db15339f47 netflow-json: initial version
Initial version of netflow module, a flow logger that logs each
direction in a completely separate record (line).
11 years ago