aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,062 Commits
7 Branches
155 Tags
194 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'a809f090d3'
${ noResults }
Commit Graph

8062 Commits (a809f090d3ae2f87d438a6a9ef59f54cd1d45f0d)
 

Author SHA1 Message Date
Victor Julien cf7f819888 state: check progress before calling engine 
Make sure progress of an inspect engine is available.
 8 years ago
Victor Julien 1bbf555318 detect: improve stateful detection 
Now that MPM runs when the TX progress is right, stateful detection
operates differently.

Changes:

1. raw stream inspection is now also an inspect engine

   Since this engine doesn't take the transactions into account, it
   could potentially run multiple times on the same data. To avoid
   this, basic result caching is in place.

2. the engines are sorted by progress, but the 'MPM' engine is first
   even if the progress is higher

   If MPM flags a rule to be inspected, the inspect engine for that
   buffer runs first. If this step fails, the rule is no longer
   evaluated. No state is stored.
 8 years ago
Victor Julien d1b7a83905 detect: change mask logic 
Previously the MPM/Prefilter engines would suggest the same rule
candidates multiple times.

For example, while processing the request body, the http headers
would be inspected by MPM multiple times.

The mask check was one way to quickly decide which rules could be
skipped.

Now that the MPM engines normally return a rule just once, this
mask check no longer makes sense. If the rule meets the ip/port/
direction based conditions, it needs to be evaluated if the MPM
said so. Even if not all conditions are yet true.

WIP disable mask as it no longer makes sense

WIP redo mask match
 8 years ago
Victor Julien a0fad6bb7f mpm: run engines as few times as possible 
In various scenarios buffers would be checked my MPM more than
once. This was because the buffers would be inspected for a
certain progress value or higher.

For example, for each packet in a file upload, the engine would
not just rerun the 'http client body' MPM on the new data, it
would also rerun the method, uri, headers, cookie, etc MPMs.

This was obviously inefficent, so this patch changes the logic.

The patch only runs the MPM engines when the progress is exactly
the intended progress. If the progress is beyond the desired
value, it is run once. A tracker is added to the app layer API,
where the completed MPMs are tracked.

Implemented for HTTP, TLS and SSH.
 8 years ago
Victor Julien d304be5bc3 detect: register progress in inspect engines 
Register required progress so we can stop inspecting as soon
as the progress isn't far enough yet.
 8 years ago
Victor Julien bc1698cfbe detect-state: don't use casts to uint 8 years ago
Victor Julien 53b21e5ee1 http_uri: unittest cleanup 8 years ago
Victor Julien 8d2f3b46e6 http_header: add another trailer test 8 years ago
Victor Julien 8d18be1fdb http_header (trailer) test cleanup 8 years ago
Victor Julien 1c46af477e ssh: fix test 8 years ago
Victor Julien a744d00f45 ssh: fix banner state setting 8 years ago
Victor Julien e3bd5f371d detect: more detailed state profiling 8 years ago
Victor Julien 6d562f3b5e app-layer: set stream-depth after stream init 8 years ago
Victor Julien 358e41b935 detect: clean up stateful detect 8 years ago
Victor Julien 9f4884a132 stream: reduce scope of new ssn func 8 years ago
Victor Julien 5c31f22e09 autotools: add src/tests to extra dist 8 years ago
Victor Julien 5a210984d5 stream: move inline tests 8 years ago
Victor Julien bea2b2c00c stream: list management cleanups 8 years ago
Victor Julien 34f7cb2b55 stream: debug improvements 8 years ago
Victor Julien aba9cd7d02 stream inspection: add debug counters 8 years ago
Victor Julien 2b433fab53 stream: pack config struct 8 years ago
Victor Julien 606f515fe9 stream: enforce gap earlier in app reassembly 8 years ago
Victor Julien 314516ffe2 stream: don't call app reassembly if disable flag set 8 years ago
Victor Julien 89af036336 stream: app-layer micro optimizations 8 years ago
Victor Julien 2f77302eeb stream: raw reassembly explicit disable raw handling 8 years ago
Victor Julien d6d7f65050 stream: mpm inspect micro optimizations 8 years ago
Victor Julien 7bddd0e168 stream: improve --disable-detection GAP handling 8 years ago
Victor Julien 6fefe70196 stream: remove unused StreamTcpGetStreamSize function 8 years ago
Victor Julien 422095668e stream: optimize session pruning 8 years ago
Victor Julien 79389558ac doc: update for stream changes 8 years ago
Victor Julien a995734b3a yaml: sync with new stream engine 8 years ago
Victor Julien ee00a6f2ec stream: validate code 8 years ago
Victor Julien e1aba7d6c2 detect: only do flow dependent cleanup if a flow is present 8 years ago
Victor Julien 61c35d3c39 detect: make SigMatchSignatures void 
None of the callers cared for it's retval, so get rid of it.
 8 years ago
Victor Julien f49150ddb9 detect: turn single detect flag into bool 8 years ago
Victor Julien 6f76cbb870 detect: remove unused detect flag 8 years ago
Victor Julien 04b24cf24e stream: improve needs reassembly code 8 years ago
Victor Julien 55e19bfb89 stream: more aggressive StreamReassembleRawHasDataReady 8 years ago
Victor Julien bf3f3ce6b2 app-layer: change logic of setting 'no reassembly' 
Instead of killing all reassembly instantly do things slightly more
gracefully:
1. disable app-layer reassembly immediately
2. flag raw reassembly not to accept new data

This will allow the current data to be inspected still.

After detect as run the raw reassembly will be fully disabled and
thus all reassembly will be as well.
 8 years ago
Victor Julien de4f4e23a0 stream: new depth / disable raw logic 
Depth reach sets NOREASSEMBLY after detect.

No new raw sets NORAW after detect.
 8 years ago
Victor Julien 7c56c9ada0 stream: allow raw reassembly catch up 
If raw reassembly falls behind, for example because no raw mpm is
active, then we need to sync up to the app progress if that is
available, or to the generic tcp tracking otherwise.
 8 years ago
Victor Julien 89d0267df2 stream: detect stream GAP also during reassembly 8 years ago
Victor Julien 0c1ec17c92 debug-validation: add stream checks 8 years ago
Victor Julien 69519bda48 stream: StreamTcpReassembleRawCheckLimit cleanup 8 years ago
Victor Julien b099008b94 stream: handle no stream scanning case 
Now that detect moves the raw progress forward, it's important
to deal with the case where detect don't consider raw inspection.

If no 'stream' rules are active, disable raw. For this the disable
raw flag is now per stream.
 8 years ago
Victor Julien 0ef46a8fd2 stream: raw content inspection inline mode 
Implement the inline mode for raw content inspection. Packets
are leading, and when a packet's payload has been added to the
stream, the packet is inspected in the context of the stream.

Reassembly will return a buffer with the packet data with older
data in front of it and after it, if available.
 8 years ago
Victor Julien 149e324060 flow/stream: reduce/disable pseudo packet injections 
At flow timeout, we no longer need to first run reassembly in
one dir, then inspection in the other. We can do both in single
packet now.

Disable pseudo packets when receiving stream end packets. Instead
call the app-layer parser in the packet direction for stream end
packets and flow end packets.

These changes in handling of those stream end packets make the
pseudo packets unnecessary.
 8 years ago
Victor Julien 2d223b69cd stream: set 'trigger raw' per direction 8 years ago
Victor Julien 971ab18b95 detect / stream: new 'raw' stream inspection 
Remove the 'StreamMsg' approach from the engine. In this approach the
stream engine would create a list of chunks for inspection by the
detection engine. There were several issues:

1. the messages had a fixed size, so blocks of data bigger than ~4k
   would be cut into multiple messages

2. it lead to lots of data copying and unnecessary memory use

3. the StreamMsgs used a central pool

The Stream engine switched over to the streaming buffer API, which
means that the reassembled data is always available. This made the
StreamMsg approach even clunkier.

The new approach exposes the streaming buffer data to the detection
engine. It has to pay attention to an important issue though: packet
loss. The data may have gaps. The streaming buffer API tracks the
blocks of continuous data.

To access the data for inspection a callback approach is used. The
'StreamReassembleRaw' function is called with a callback and data.
This way it runs the MPM and individual rule inspection code. At
the end of each detection run the stream engine is notified that it
can move forward it's 'progress'.
 8 years ago
Victor Julien 564c0bd2c1 stream: constify StreamTcpReassembleRawCheckLimit 8 years ago