Correct permissions for global sieve scripts
Hi there,
I've set up dovecot via global sieve scripts to send mails that a user manually moved to their junk directory to rspamd to learn them as spam (and learn messages as ham if they are moved out of it). I thought I had it all properly set up, but I'm now again seeing log messages like this:
Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: failed to execute to program `rspamd-learn-spam.sh': refer to server log for more information. [2024-09-15 13:01:00] Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: Execution of script /etc/dovecot/sieve/global/learn-spam.sieve failed
The content of learn-spam.sieve is this:
require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamd-learn-spam.sh";
And the content of rspamd-learn-spam.sh is this:
#!/bin/sh exec /usr/bin/rspamc learn_spam
All the necessary files are located in /etc/dovecot/sieve/global and are owned by dovecot:dovecot with rw, r, r permissions. Also, I have compiled the .sieve files with sievec, the resulting .svbin files have the same permissions and ownership. What am I doing wrong? I don't see any further "server log" that will tell me more information.
I'm using dovecot 2.3.19.1 (9b53102964) on Debian 12.7
Best
Richard
On 15-09-2024 14:30, Richard via dovecot wrote:
Hi there,
I've set up dovecot via global sieve scripts to send mails that a user manually moved to their junk directory to rspamd to learn them as spam (and learn messages as ham if they are moved out of it). I thought I had it all properly set up, but I'm now again seeing log messages like this:
Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: failed to execute to program `rspamd-learn-spam.sh': refer to server log for more information. [2024-09-15 13:01:00] Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: Execution of script /etc/dovecot/sieve/global/learn-spam.sieve failed
The content of learn-spam.sieve is this:
require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamd-learn-spam.sh";
And the content of rspamd-learn-spam.sh is this:
#!/bin/sh exec /usr/bin/rspamc learn_spam
Does the rspamc program log any errors? Is actually executed?
Good question. There isn't any entry in journal, the only additional message I get is from dovecot
Sep 15 13:01:00 dovecot[523226]: imap(rrosner)<823661><4+8RXyYi2L9/AAAB>: program exec:/etc/dovecot/sieve/global/rspamd-learn-spam.sh (823662): Terminated with non-zero exit code 1
right before the other error messages. But it's not impossible that rspamc doesn't successfully execute, as rspamd currently has some weird issues where I have to wait for the developer to respond.
On 15.09.24 23:05, Tom Hendrikx via dovecot wrote:
Does the rspamc program log any errors? Is actually executed?
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 15/09/2024 15:30 EEST Richard via dovecot dovecot@dovecot.org wrote:
Hi there,
I've set up dovecot via global sieve scripts to send mails that a user manually moved to their junk directory to rspamd to learn them as spam (and learn messages as ham if they are moved out of it). I thought I had it all properly set up, but I'm now again seeing log messages like this:
Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: failed to execute to program `rspamd-learn-spam.sh': refer to server log for more information. [2024-09-15 13:01:00] Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: Execution of script /etc/dovecot/sieve/global/learn-spam.sieve failed
The content of learn-spam.sieve is this:
require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamd-learn-spam.sh";
And the content of rspamd-learn-spam.sh is this:
#!/bin/sh exec /usr/bin/rspamc learn_spam
All the necessary files are located in /etc/dovecot/sieve/global and are owned by dovecot:dovecot with rw, r, r permissions. Also, I have compiled the .sieve files with sievec, the resulting .svbin files have the same permissions and ownership. What am I doing wrong? I don't see any further "server log" that will tell me more information.
I'm using dovecot 2.3.19.1 (9b53102964) on Debian 12.7
Best
Richard
Did you remember to set +x on the rspamd-learn-spam.sh script?
Does it work if you invoke it by hand?
Aki
I did forget to mention both shell scripts have rwx, rwx, - permissions. manually executing rspamc does result in a "IO read error: unexpected EOF". So yet another issue to add to the list I guess...
Richard
On 16.09.24 09:07, Aki Tuomi wrote:
Did you remember to set +x on the rspamd-learn-spam.sh script?
Does it work if you invoke it by hand?
Aki
Also to double-check, can you compare your setup against https://doc.dovecot.org/2.3/configuration_manual/howto/antispam_with_sieve/#... ?
Aki
On 16/09/2024 12:18 EEST Richard via dovecot dovecot@dovecot.org wrote:
I did forget to mention both shell scripts have rwx, rwx, - permissions. manually executing rspamc does result in a "IO read error: unexpected EOF". So yet another issue to add to the list I guess...
Richard
On 16.09.24 09:07, Aki Tuomi wrote:
Did you remember to set +x on the rspamd-learn-spam.sh script?
Does it work if you invoke it by hand?
Aki
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
This actually helped.
So I'm not entirely sure when my method stopped working, but for all I can tell it used to work. Now this has changed. I've changed rspamd-learn-spam.sh to this:
#!/bin/sh
exec /usr/bin/rspamc -h localhost:11333 -P
Manually testing this now works better, so I'll just have to wait for the spam mail to be missed to test if this actually solves the issue. Can't be that long.
PS: rspamd for my config doesn't use socket files. But the needed port is the same that's written in /etc/rspamd/local.d/worker-normal.inc (and in /etc/rspamd/rspamd.conf in the worker "normal"{} section).
On 16.09.24 11:21, Aki Tuomi wrote:
Also to double-check, can you compare your setup against https://doc.dovecot.org/2.3/configuration_manual/howto/antispam_with_sieve/#... ?
Aki
So, there was actually one piece missing to fully solve this puzzle. For whatever reason, the guide I used back when I set up the scripts put "exec" in front of the actual command. Not sure what it does, but it's garbage in this scenario. As far as I read it somehow replaces bash with whatever you execute. Executing the command from rspamd-learn-spam.sh with the exec infront on a remote machine will terminate the remote session. So just removing that seems to have fixed all error messages (the IO error due to EOF can still happen, but it won't happen every time anymore). So it wasn't a permission issue or any issue with dovecot, just a bad command inside the script.
Thanks all for chiming in.
On 16.09.24 13:12, Richard wrote:
This actually helped.
So I'm not entirely sure when my method stopped working, but for all I can tell it used to work. Now this has changed. I've changed rspamd-learn-spam.sh to this:
#!/bin/sh exec /usr/bin/rspamc -h localhost:11333 -P
learn_spam Manually testing this now works better, so I'll just have to wait for the spam mail to be missed to test if this actually solves the issue. Can't be that long.
PS: rspamd for my config doesn't use socket files. But the needed port is the same that's written in /etc/rspamd/local.d/worker-normal.inc (and in /etc/rspamd/rspamd.conf in the worker "normal"{} section).
On sön, 2024/09/15 at 14:30:19 +0200, Richard via dovecot wrote:
I've set up dovecot via global sieve scripts to send mails that a user manually moved to their junk directory to rspamd to learn them as spam (and learn messages as ham if they are moved out of it). I thought I had it all properly set up, but I'm now again seeing log messages like this:
root@debian:~# dpkg -l | egrep '^ii.*(rspam|dovecot-core)' ii dovecot-core 1:2.3.21.1+dfsg1-1~bpo12+1 amd64 secure POP3/IMAP server - core files ii rspamd 3.4-1 amd64 Rapid spam filtering system
I do the same as you.
Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: failed to execute to program `rspamd-learn-spam.sh': refer to server log for more information. [2024-09-15 13:01:00] Sep 15 13:01:00 dovecot[523226]: imap(username)<823661><4+8RXyYi2L9/AAAB>: Error: sieve: Execution of script /etc/dovecot/sieve/global/learn-spam.sieve failed
The content of learn-spam.sieve is this:
require ["vnd.dovecot.pipe", "copy", "imapsieve"]; pipe :copy "rspamd-learn-spam.sh";
And the content of rspamd-learn-spam.sh is this:
#!/bin/sh exec /usr/bin/rspamc learn_spam
All the necessary files are located in /etc/dovecot/sieve/global and are owned by dovecot:dovecot with rw, r, r permissions. Also, I have compiled the .sieve files with sievec, the resulting .svbin files have the same permissions and ownership. What am I doing wrong? I don't see any further "server log" that will tell me more information.
I'm using dovecot 2.3.19.1 (9b53102964) on Debian 12.7
root@debian:~# ls -l /etc/dovecot/sieve total 32 -rw-r----- 1 vmail dovecot 188 Sep 4 13:41 global-spam.sieve -rw-r----- 1 vmail vmail 330 Sep 4 14:01 global-spam.svbin -rwxr-x--- 2 vmail dovecot 2579 Sep 4 13:44 learn-ham.rspamd.script -rw-r----- 1 vmail dovecot 256 Sep 4 13:42 learn-ham.sieve -rw-r----- 1 vmail dovecot 442 Sep 5 03:55 learn-ham.svbin -rwxr-x--- 2 vmail dovecot 2579 Sep 4 13:44 learn-spam.rspamd.script -rw-r----- 1 vmail dovecot 151 Sep 4 13:43 learn-spam.sieve -rw-r----- 1 vmail dovecot 341 Sep 5 03:56 learn-spam.svbin
The scripts need to be executable.
I have this in the plugin {} section of dovecot.conf:
# This will automatically move spam into Junk/ and when you move a message # into Junk, it will tell rspamd that it is spam for Bayes learning. Moving # false positives out of Junk/ will teach rspamd that it is ham. sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +editheader +imapflags +mboxmetadata +notify +servermetadata +spamtest +spamtestplus +virustest imapsieve_mailbox1_before = file:/etc/dovecot/sieve/learn-spam.sieve imapsieve_mailbox1_causes = COPY APPEND FLAG imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/learn-ham.sieve imapsieve_mailbox2_causes = COPY APPEND FLAG imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_spamtest_status_type = score sieve_spamtest_status_header = X-Spam_score: (-?[[:digit:]]+\.[[:digit:]]) sieve_spamtest_max_value = 6 sieve_before = /etc/dovecot/sieve/global-spam.sieve
The only other thing was to get the password out of /etc/rspamd/worker-controller.inc and put it in /etc/dovecot/rspamd-controller.password (or whatever file your learn-{sp,h}am script points at to get the password). It needed to be pointed at 127.0.0.1:11334 for the socket.
root@debian:~# cat /etc/dovecot/rspamd-controller.conf.sh
# Path to file containing the controller password
# (Or, if it doesn't start with '/' or '.', the password itself.
# But it might leak the password through ps to other users)
RSPAMD_CONTROLLER_PASSWORD=/etc/dovecot/rspamd-controller.password
# passed to rspamc with the -h option (host and port)
RSPAMD_CONTROLLER_SOCKET=127.0.0.1:11334
# if set uses curl instead of rspamc; should start with http: or https:
RSPAMD_CONTROLLER_HOST=
# classifier to learn for (default by rspamc: bayes), e.g. bayes_user
RSPAMD_CLASSIFIER=bayes
From what I remember, it was somewhat fiddly to get this working as I was not intimately familiar with rspamd, nor dovecot or sieve, but this works and it works well. Relatively low incident rate of false positives after a some weeks.
-- Kind regards,
/S
On 16.09.24 09:50, Sirius wrote:
On sön, 2024/09/15 at 14:30:19 +0200, Richard via dovecot wrote:
I've set up dovecot via global sieve scripts to send mails that a user manually moved to their junk directory to rspamd to learn them as spam (and learn messages as ham if they are moved out of it). I thought I had it all properly set up, but I'm now again seeing log messages like this: root@debian:~# dpkg -l | egrep '^ii.*(rspam|dovecot-core)' ii dovecot-core 1:2.3.21.1+dfsg1-1~bpo12+1 amd64 secure POP3/IMAP server - core files ii rspamd 3.4-1 amd64 Rapid spam filtering system
I do the same as you.
Not exactly. I'm on rspamd 3.9.1-1~82f43560f~bookworm. From rspamd's repo.
root@debian:~# ls -l /etc/dovecot/sieve total 32 -rw-r----- 1 vmail dovecot 188 Sep 4 13:41 global-spam.sieve -rw-r----- 1 vmail vmail 330 Sep 4 14:01 global-spam.svbin -rwxr-x--- 2 vmail dovecot 2579 Sep 4 13:44 learn-ham.rspamd.script -rw-r----- 1 vmail dovecot 256 Sep 4 13:42 learn-ham.sieve -rw-r----- 1 vmail dovecot 442 Sep 5 03:55 learn-ham.svbin -rwxr-x--- 2 vmail dovecot 2579 Sep 4 13:44 learn-spam.rspamd.script -rw-r----- 1 vmail dovecot 151 Sep 4 13:43 learn-spam.sieve -rw-r----- 1 vmail dovecot 341 Sep 5 03:56 learn-spam.svbin
The scripts need to be executable.
Yes, I forgot to mention that both shell scripts have 770 permissions.
I have this in the plugin {} section of dovecot.conf:
# This will automatically move spam into Junk/ and when you move a message # into Junk, it will tell rspamd that it is spam for Bayes learning. Moving # false positives out of Junk/ will teach rspamd that it is ham. sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +editheader +imapflags +mboxmetadata +notify +servermetadata +spamtest +spamtestplus +virustest
I only have sieve_global_extensions enabled. As user scripts aren't supposed to be able to access external programs I don't see any benefit configuring that setting
imapsieve_mailbox1_before =file:/etc/dovecot/sieve/learn-spam.sieve imapsieve_mailbox1_causes = COPY APPEND FLAG imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before =file:/etc/dovecot/sieve/learn-ham.sieve imapsieve_mailbox2_causes = COPY APPEND FLAG imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
I only had +vnd.dovecot.pipe in here. Let's the if adding +vnd.dovecot.environment changes anything.
sieve_spamtest_status_type = score sieve_spamtest_status_header = X-Spam_score: (-?[[:digit:]]+\.[[:digit:]]) sieve_spamtest_max_value = 6 sieve_before = /etc/dovecot/sieve/global-spam.sieve
I never configured these. What are their use? I've just set up another sieve script (which seems to be working just fine) to sort out all messages marked as spam into the users junk directory. Is this just to do that?
The only other thing was to get the password out of /etc/rspamd/worker-controller.inc and put it in /etc/dovecot/rspamd-controller.password (or whatever file your learn-{sp,h}am script points at to get the password). It needed to be pointed at 127.0.0.1:11334 for the socket.
What is the password needed for? Because manually executing rspamc doesn't ask for some password either. The worker-controller.inc does point at 127.0.0.1 and ::1, but no port seems to be configured.
root@debian:~# cat /etc/dovecot/rspamd-controller.conf.sh # Path to file containing the controller password # (Or, if it doesn't start with '/' or '.', the password itself. # But it might leak the password through ps to other users) RSPAMD_CONTROLLER_PASSWORD=/etc/dovecot/rspamd-controller.password # passed to rspamc with the -h option (host and port) RSPAMD_CONTROLLER_SOCKET=127.0.0.1:11334 # if set uses curl instead of rspamc; should start with http: or https: RSPAMD_CONTROLLER_HOST= # classifier to learn for (default by rspamc: bayes), e.g.
bayes_userRSPAMD_CLASSIFIER=bayesFrom what I remember, it was somewhat fiddly to get this working as I was not intimately familiar with rspamd, nor dovecot or sieve, but this works and it works well. Relatively low incident rate of false positives after a some weeks.
We used to have the same for quite a while now, without needing any of that. I guess the last rspamd update was just borked that much that it won't work for the time being. Some messages aren't even processed at all...
On mån, 2024/09/16 at 11:41:24 +0200, Richard via dovecot wrote:
On 16.09.24 09:50, Sirius wrote:
On sön, 2024/09/15 at 14:30:19 +0200, Richard via dovecot wrote:
[snip]
I do the same as you.
Not exactly. I'm on rspamd 3.9.1-1~82f43560f~bookworm. From rspamd's repo.
Ah, that may be the source of your problems.
I have this in the plugin {} section of dovecot.conf:
# This will automatically move spam into Junk/ and when you move a message # into Junk, it will tell rspamd that it is spam for Bayes learning. Moving # false positives out of Junk/ will teach rspamd that it is ham. sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +editheader +imapflags +mboxmetadata +notify +servermetadata +spamtest +spamtestplus +virustest
I only have sieve_global_extensions enabled. As user scripts aren't supposed to be able to access external programs I don't see any benefit configuring that setting
I did it in case I want to have stricter personal settings than the rest of my family. The default global sieve cuts spam off at a score of 10. Personally, I may end up lowering that to 8.0-8.5 somewhere.
[snip]
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
I only had +vnd.dovecot.pipe in here. Let's the if adding +vnd.dovecot.environment changes anything.
sieve_spamtest_status_type = score sieve_spamtest_status_header = X-Spam_score: (-?[[:digit:]]+\.[[:digit:]]) sieve_spamtest_max_value = 6 sieve_before = /etc/dovecot/sieve/global-spam.sieve
I never configured these. What are their use? I've just set up another sieve script (which seems to be working just fine) to sort out all messages marked as spam into the users junk directory. Is this just to do that?
This comes from https://github.com/darix/dovecot-sieve-antispam-rspamd/ which is what I ended up adapting as it was not an exact match for my environment.
First two lines does the spam testing based on the score rspamd have assigned the incoming message (hence the last three sieve extensions in the section above).
The only other thing was to get the password out of /etc/rspamd/worker-controller.inc and put it in /etc/dovecot/rspamd-controller.password (or whatever file your learn-{sp,h}am script points at to get the password). It needed to be pointed at 127.0.0.1:11334 for the socket.
What is the password needed for? Because manually executing rspamc doesn't ask for some password either. The worker-controller.inc does point at 127.0.0.1 and ::1, but no port seems to be configured.
When you connect to rspamd controller (port 11334), it is authenticated. Presumably so that not just anyone can report spam/ham and mess up the scoring. If it runs only on loopback and there is no port-forwarding, authentication is perhaps unnecessary, but it makes sense to have it enabled anyway if someone else can log into the system as a regular user.
root@debian:~# cat /etc/dovecot/rspamd-controller.conf.sh # Path to file containing the controller password # (Or, if it doesn't start with '/' or '.', the password itself. # But it might leak the password through ps to other users) RSPAMD_CONTROLLER_PASSWORD=/etc/dovecot/rspamd-controller.password # passed to rspamc with the -h option (host and port) RSPAMD_CONTROLLER_SOCKET=127.0.0.1:11334 # if set uses curl instead of rspamc; should start with http: or https: RSPAMD_CONTROLLER_HOST= # classifier to learn for (default by rspamc: bayes), e.g.
bayes_userRSPAMD_CLASSIFIER=bayesFrom what I remember, it was somewhat fiddly to get this working as I was not intimately familiar with rspamd, nor dovecot or sieve, but this works and it works well. Relatively low incident rate of false positives after a some weeks.
We used to have the same for quite a while now, without needing any of that. I guess the last rspamd update was just borked that much that it won't work for the time being. Some messages aren't even processed at all...
Worth checking how rspamc calls into rspamd (maybe you use a socket in /run or similar) and see if they have defaulted to authentication on for the sockets. Or roll rspamd back to previous version for the time being.
-- Kind regards,
/S
On 16.09.24 13:07, Sirius wrote:
On mån, 2024/09/16 at 11:41:24 +0200, Richard via dovecot wrote:
On 16.09.24 09:50, Sirius wrote:
On sön, 2024/09/15 at 14:30:19 +0200, Richard via dovecot wrote: [snip] I do the same as you. Not exactly. I'm on rspamd 3.9.1-1~82f43560f~bookworm. From rspamd's repo. Ah, that may be the source of your problems.
Parts of my problem, but not in this case actually. My solution is now in the thread with Aki Tuomi. Seems to have been an unlucky combination of outdated rspamc handling and using bash's "exec" where it seems to not belong.
I only have sieve_global_extensions enabled. As user scripts aren't supposed to be able to access external programs I don't see any benefit configuring that setting I did it in case I want to have stricter personal settings than the rest of my family. The default global sieve cuts spam off at a score of 10. Personally, I may end up lowering that to 8.0-8.5 somewhere.
Good to know
[snip]
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environmentI only had +vnd.dovecot.pipe in here. Let's the if adding +vnd.dovecot.environment changes anything.
sieve_spamtest_status_type = score sieve_spamtest_status_header = X-Spam_score: (-?[[:digit:]]+\.[[:digit:]]) sieve_spamtest_max_value = 6 sieve_before = /etc/dovecot/sieve/global-spam.sieveI never configured these. What are their use? I've just set up another sieve script (which seems to be working just fine) to sort out all messages marked as spam into the users junk directory. Is this just to do that? This comes from https://github.com/darix/dovecot-sieve-antispam-rspamd/ which is what I ended up adapting as it was not an exact match for my environment.
First two lines does the spam testing based on the score rspamd have assigned the incoming message (hence the last three sieve extensions in the section above).
Interesting solution. Didn't know dovecot could do that. I have a larger sieve script that will match all messages containing a spam flag in the header - our spam filter is actually a secondary one, because while the primary is a commercial one, it's pretty bad so it misses a lot - and some typical keywords from the subject you won't ever find in a legit mail and move them to junk. Also, I have some logic to help keep ham out of the junk directory if rspamd is falsely categorizing mail as spam from known trustworthy senders, as rspamd's whitelisting is quite flaky.
The only other thing was to get the password out of /etc/rspamd/worker-controller.inc and put it in /etc/dovecot/rspamd-controller.password (or whatever file your learn-{sp,h}am script points at to get the password). It needed to be pointed at 127.0.0.1:11334 for the socket. What is the password needed for? Because manually executing rspamc doesn't ask for some password either. The worker-controller.inc does point at 127.0.0.1 and ::1, but no port seems to be configured. When you connect to rspamd controller (port 11334), it is authenticated. Presumably so that not just anyone can report spam/ham and mess up the scoring. If it runs only on loopback and there is no port-forwarding, authentication is perhaps unnecessary, but it makes sense to have it enabled anyway if someone else can log into the system as a regular user.
Good to know, but I think it may be safer to use worker-normal instead with setting allow_learn to true in its config. That way you don't need the higher privileges the controller has. And you shouldn't need the worker-controller's password. At least as far as I understand https://rspamd.com/doc/workers/normal.html. But I in my trials, sending mail to the port of worker-normal still requires the password of worker-controller. So that's only theoretical. Also, as all of this has no permissions for normal user to access it, it's a bit redundant.
Worth checking how rspamc calls into rspamd (maybe you use a socket in /run or similar) and see if they have defaulted to authentication on for the sockets. Or roll rspamd back to previous version for the time being.
Thankfully that's all resolved now adding the port to use (and currently the password, maybe that can be dropped somehow). So I guess rspamd changed its behavior at some point I didn't notice and deprecated the old ways.
But thanks for the details.
participants (4)
-
Aki Tuomi
-
Richard
-
Sirius
-
Tom Hendrikx