This patch updates the unix socket protocol. Messages send from
the server and the client have now a '\n' at the end. This allows
both sides to detect easily the end of a command.
As a side effect, this fixes the problem of long answer in
suricatasc. There is now a limit at the arbitrary value of 65536.
Backward compatility is preserved as a client with the older
version of the protocol can still connect to a Suricata with
version 2 of the protocol.
So far, the suricata socket suricata-command.socket has the rights
rw-r----- suricata:user.
When suricata is used with restricted access, an other application
(suricatasc like) that needs to access to the command socket also
with restricted access can not write to the socket since it is not
the owner (e.g suricata within container, with an hardened value
for umask and hardened rights for users).
The socket should be set as rw-rw----. Use chmod instead of fchmod
and set it after the socket creation.
This permits to handle memcap values through
unix socket for:
- stream
- stream-reassembly
- flow
- applayer-proto-http
- defrag
- ippair
- host
It will be possible to show or change a memcap value
for a specified configuration and list all memcap values
available.
The following commands are registered for unix-socket:
- memcap-set
- memcap-show
- memcap-list
Output:
>>> memcap-show flow
Success:
{
"value": "64mb"
}
>>> memcap-set flow 64mb
Success:
"memcap value for 'flow' updated: 67108864"
Command with invalid memcap key:
>>> memcap-set udp 32mb
Error:
"Available config: stream stream-reassembly flow applayer-proto-http defrag ippair host"
Command with an invalid memcap value:
>>> memcap-set http 32mmb
Error:
"error parsing memcap specified, value not changed"
We did had a race condition with running logrotate with multiple
EVE Json files. Consequence was one of the file not being reopen
by suricata that did continue to write to the rotated one.
Trying fix on signal handler did fail so this patch implements
log rotation support by adding a dedicated command to unix socket
to reopen the log files.
Currently, when live reload is executed through
unix-socket, suri prints in the console the following
error message:
"Live rule reload not possible if -s or -S option used at runtime."
Instead, prints "done" in unix socket,
when the live reload is not executed.
This permits to print the invalid rules through
unix socket.
An example output is the following:
>>> show-failed-rules
Success:
[
{
"filename": "/home/eric/git/oisf/benches/tls-store.rules",
"line": 2,
"rule": "alert ts any any -> any 334 (msg:\"Store TLS\"; tls.store; sid:2; rev:1;)"
},
{
"filename": "/home/eric/git/oisf/benches/tls-store.rules",
"line": 3,
"rule": "alert ls any any -> any 334 (msg:\"Store TLS\"; tls.store; sid:3; rev:1;)"
}
]
The dump is limited to 20 entries to avoid to send a too big
message to the client that don't support it by default.
Add a non blocking function to reload rules. It will be useful
for remote system management to avoid to block them waiting the
reload. And as we now have a last-reload command we can get the
status of the current reload.
Remove the DONE state to fix a problem with state not being
changed correctly when multiple reload were done. As DONE was
not really useful, we can remove it.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
UnixManagerThreadInit needs to return a failure code if the socket
fails to initialize to avoid entering the UnixManager loop which
will continuously fail on the call to bind, as no listening
socket was setup.
This can occur when the socket fails to initialize due to a
permissions error and fatal init errors is not on.
add-hostbit adds a named hostbit with an expire time in seconds.
remove-hostbit removes hostbit by name.
add-hostbit, remove-hostbit return success or failure.
list-hostbit returns a json array of hostbits with their name and
expire time:
{
"message": {
"count": 1,
"hostbits":
[{
"expire": 3222,
"name": "firefox-users"
}]
},
"return": "OK"
}
This fixes:
2,595 (1,824 direct, 771 indirect) bytes in 57 blocks are definitely lost in loss record 328 of 332
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x66D0C7B: ??? (in /usr/lib/x86_64-linux-gnu/libjansson.so.4.7.0)
by 0x911A27: UnixManagerListCommand (unix-manager.c:766)
by 0x9108A9: UnixCommandExecute (unix-manager.c:486)
by 0x910D9E: UnixCommandRun (unix-manager.c:545)
by 0x9111B3: UnixMain (unix-manager.c:593)
by 0x913D27: UnixManager (unix-manager.c:961)
by 0x907773: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Instead of sending responses to clients in small chunks, send it in
one big chunk. For this the JSON message is first serialized into
a MemBuffer before sending.
Under high load it is possible that the thread is not yet started
and that we register a command at the same time. As a consequence,
the commands list is not yet initialized and we have a segfault.
This patch moves the initialization in the ThreadInit function to
be sure the commands list is available when needed.
Allow for a tenant to be reloaded. The command is the same as the
register-tenant command, so with a yaml and tenant-id as argument.
However this replaces an existing tenant.
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.
Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
Make available to live mode and unix socket mode.
register-tenant:
Loads a new YAML, does basic validation.
Loads a new detection engine
Loads rules
Add new de_ctx to master store and stores tenant id in the de_ctx so
we can look it up by tenant id later.
unregister-tenant:
Gets the de_ctx, moves it to the freelist
Removes config
Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
Eric Leblond
Pascal Delalande
Victor Julien
Giuseppe Longo
Jason Ish
Zachary Rasmor
Alexander Gozman