Commit Graph

7971 Commits (bc7c01ecbc816618aa8a4a3be133d16a2728fc07)
 

Author SHA1 Message Date
Victor Julien bc7c01ecbc detect: use BIT_U32 macro for content flags 10 years ago
Victor Julien c65a119cc0 debug: suppress notice message 10 years ago
Victor Julien 276125c1ef cleanup: remove unused ringbuffer code 10 years ago
Victor Julien cda6e0291f cleanup: remove libpcap < 1 support 10 years ago
Victor Julien 119115d3b6 configure: remove CentOS5 pkg-config fix 10 years ago
Victor Julien 0516b5d704 cleanup: from AS_VERSION_COMPARE CentOS5 workaround 10 years ago
Victor Julien d31cb083e9 detect: update tests that mix state/stream inspect 10 years ago
Victor Julien eb5857b68a unittests: add/improve helpers for stream/flow 10 years ago
Victor Julien 15dcac92f2 http_header: enable trailer prefilter engines
Now that the mpm engines run only for the proper 'progress'
value, the trailing headers need their own engine registration.
10 years ago
Victor Julien cf7f819888 state: check progress before calling engine
Make sure progress of an inspect engine is available.
10 years ago
Victor Julien 1bbf555318 detect: improve stateful detection
Now that MPM runs when the TX progress is right, stateful detection
operates differently.

Changes:

1. raw stream inspection is now also an inspect engine

   Since this engine doesn't take the transactions into account, it
   could potentially run multiple times on the same data. To avoid
   this, basic result caching is in place.

2. the engines are sorted by progress, but the 'MPM' engine is first
   even if the progress is higher

   If MPM flags a rule to be inspected, the inspect engine for that
   buffer runs first. If this step fails, the rule is no longer
   evaluated. No state is stored.
10 years ago
Victor Julien d1b7a83905 detect: change mask logic
Previously the MPM/Prefilter engines would suggest the same rule
candidates multiple times.

For example, while processing the request body, the http headers
would be inspected by MPM multiple times.

The mask check was one way to quickly decide which rules could be
skipped.

Now that the MPM engines normally return a rule just once, this
mask check no longer makes sense. If the rule meets the ip/port/
direction based conditions, it needs to be evaluated if the MPM
said so. Even if not all conditions are yet true.

WIP disable mask as it no longer makes sense

WIP redo mask match
10 years ago
Victor Julien a0fad6bb7f mpm: run engines as few times as possible
In various scenarios buffers would be checked my MPM more than
once. This was because the buffers would be inspected for a
certain progress value or higher.

For example, for each packet in a file upload, the engine would
not just rerun the 'http client body' MPM on the new data, it
would also rerun the method, uri, headers, cookie, etc MPMs.

This was obviously inefficent, so this patch changes the logic.

The patch only runs the MPM engines when the progress is exactly
the intended progress. If the progress is beyond the desired
value, it is run once. A tracker is added to the app layer API,
where the completed MPMs are tracked.

Implemented for HTTP, TLS and SSH.
10 years ago
Victor Julien d304be5bc3 detect: register progress in inspect engines
Register required progress so we can stop inspecting as soon
as the progress isn't far enough yet.
10 years ago
Victor Julien bc1698cfbe detect-state: don't use casts to uint 10 years ago
Victor Julien 53b21e5ee1 http_uri: unittest cleanup 10 years ago
Victor Julien 8d2f3b46e6 http_header: add another trailer test 10 years ago
Victor Julien 8d18be1fdb http_header (trailer) test cleanup 10 years ago
Victor Julien 1c46af477e ssh: fix test 10 years ago
Victor Julien a744d00f45 ssh: fix banner state setting 10 years ago
Victor Julien e3bd5f371d detect: more detailed state profiling 10 years ago
Victor Julien 6d562f3b5e app-layer: set stream-depth after stream init 10 years ago
Victor Julien 358e41b935 detect: clean up stateful detect 10 years ago
Victor Julien 9f4884a132 stream: reduce scope of new ssn func 10 years ago
Victor Julien 5c31f22e09 autotools: add src/tests to extra dist 10 years ago
Victor Julien 5a210984d5 stream: move inline tests 10 years ago
Victor Julien bea2b2c00c stream: list management cleanups 10 years ago
Victor Julien 34f7cb2b55 stream: debug improvements 10 years ago
Victor Julien aba9cd7d02 stream inspection: add debug counters 10 years ago
Victor Julien 2b433fab53 stream: pack config struct 10 years ago
Victor Julien 606f515fe9 stream: enforce gap earlier in app reassembly 10 years ago
Victor Julien 314516ffe2 stream: don't call app reassembly if disable flag set 10 years ago
Victor Julien 89af036336 stream: app-layer micro optimizations 10 years ago
Victor Julien 2f77302eeb stream: raw reassembly explicit disable raw handling 10 years ago
Victor Julien d6d7f65050 stream: mpm inspect micro optimizations 10 years ago
Victor Julien 7bddd0e168 stream: improve --disable-detection GAP handling 10 years ago
Victor Julien 6fefe70196 stream: remove unused StreamTcpGetStreamSize function 10 years ago
Victor Julien 422095668e stream: optimize session pruning 10 years ago
Victor Julien 79389558ac doc: update for stream changes 10 years ago
Victor Julien a995734b3a yaml: sync with new stream engine 10 years ago
Victor Julien ee00a6f2ec stream: validate code 10 years ago
Victor Julien e1aba7d6c2 detect: only do flow dependent cleanup if a flow is present 10 years ago
Victor Julien 61c35d3c39 detect: make SigMatchSignatures void
None of the callers cared for it's retval, so get rid of it.
10 years ago
Victor Julien f49150ddb9 detect: turn single detect flag into bool 10 years ago
Victor Julien 6f76cbb870 detect: remove unused detect flag 10 years ago
Victor Julien 04b24cf24e stream: improve needs reassembly code 10 years ago
Victor Julien 55e19bfb89 stream: more aggressive StreamReassembleRawHasDataReady 10 years ago
Victor Julien bf3f3ce6b2 app-layer: change logic of setting 'no reassembly'
Instead of killing all reassembly instantly do things slightly more
gracefully:
1. disable app-layer reassembly immediately
2. flag raw reassembly not to accept new data

This will allow the current data to be inspected still.

After detect as run the raw reassembly will be fully disabled and
thus all reassembly will be as well.
10 years ago
Victor Julien de4f4e23a0 stream: new depth / disable raw logic
Depth reach sets NOREASSEMBLY after detect.

No new raw sets NORAW after detect.
10 years ago
Victor Julien 7c56c9ada0 stream: allow raw reassembly catch up
If raw reassembly falls behind, for example because no raw mpm is
active, then we need to sync up to the app progress if that is
available, or to the generic tcp tracking otherwise.
10 years ago