[Dovecot] trying to make cmusieve happen globally
In Dovecot 1.1.11 cmusieve is apparently integrated in the Ubuntu 9.10 package "dovecot-common" since the files are there. I am wanting to right now just do a very basic test of the setup to see how it runs before doing more sophisticated steps that could obscure any errors or bugs (e.g. unit testing).
So I took the following example sieve script from the wiki1 documentation:
require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "spam"; }
and recoded it to skip the test and just always do the fileinto step like so:
require "fileinto"; fileinto "spamtest";
Then I configured my dovecot.conf file (via dovecot-postfix.conf it really points to) by setting:
protocol lda { ... mail_plugins = cmusieve mail_plugin_dir = /usr/lib/dovecot/modules/lda ... }
and:
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve }
I manually compiled /etc/dovecot/sieve-global.d/spam.sieve so there also exists /etc/dovecot/sieve-global.d/spam.sievec in the same directory. Then I restarted both Dovecot and Postfix.
Mail is being delivered, but still only to the same place it always has. The filesystem shows these sieve files are not being accessed at all. I've read documents http://wiki1.dovecot.org/LDA/Sieve and http://wiki1.dovecot.org/LDA/Sieve/CMU to come up with this. But it just doesn't seem to be happening. So I guess it is a good thing I've broken my testing down into smaller units. Anything standout that I misinterpreted about the documentation? If not, I can post the whole config files (in case there is a conflict with something else I have done).
FYI, I'm only trying to do a global sieve, not per-user sieve scripts. I'll tackle per-user stuff at a later date, after the global is working.
-- sHiFt HaPpEnS!
Am 22.12.2010 um 16:45 schrieb Phil Howard:
protocol lda { ... mail_plugins = cmusieve mail_plugin_dir = /usr/lib/dovecot/modules/lda … }
Did you apply magic to the Postfix 'master.cf' as well? Just a hunch because it is not mentioned in your post… For Sieve to work the Dovecot LDA needs to be called by Postfix.
Hi,
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use "sieve_before", which is not documented in the wiki.
Only this way the sievescript really gets executed.
The global path requires a sieve script per home directory, including the global file.
best regards,
Anton
Op 22-12-2010 17:46, Anton Dollmaier schreef:
Hi,
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use "sieve_before", which is not documented in the wiki.
Eh.. no. CMUSieve does not have multiscript support, so sieve_before is not available.
Only this way the sievescript really gets executed.
The global default is not executed when the user has a script of his own, otherwise it is executed.
The global path requires a sieve script per home directory, including the global file.
You are confusing the sieve_global_dir and sieve_global_path settings. The first is used by include, the latter specifies the default script executed when the user has none. Maybe these settings should be named differently, probably naming the latter sieve_default.
Regards,
Stephan
On Wed, Dec 22, 2010 at 13:12, Stephan Bosch stephan@rename-it.nl wrote:
Op 22-12-2010 17:46, Anton Dollmaier schreef:
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use "sieve_before", which is not documented in the wiki.
Eh.. no. CMUSieve does not have multiscript support, so sieve_before is not available.
Only this way the sievescript really gets executed.
The global default is not executed when the user has a script of his own, otherwise it is executed.
That sounds like a reasonable way to do it.
The global path requires a sieve script per home directory, including the global file.
You are confusing the sieve_global_dir and sieve_global_path settings. The first is used by include, the latter specifies the default script executed when the user has none. Maybe these settings should be named differently, probably naming the latter sieve_default.
Yeah, since it behaves as a default, that name would have made sense. That, or a sequence of path names that may or may not have %u to test for an available script, and stop at the first one found.
sieve_global_path should have the file name? Should it be the source or the compiled sieve? Does it need chmod +x ?
-- sHiFt HaPpEnS!
Am 22.12.2010 um 19:17 schrieb Phil Howard:
Yeah, since it behaves as a default, that name would have made sense. That, or a sequence of path names that may or may not have %u to test for an available script, and stop at the first one found.
sieve_global_path should have the file name? Should it be the source or the compiled sieve? Does it need chmod +x ?
May be a good idea to post some snippets of the actual Postfix delivery attempts just to make sure it really uses the external LDA. You can also crank up 'mail_debug = yes' to log more verbosely. IIRC it should show CMU Sieve warnings too (it's way old). Going forward using Pigeonhole would make more sense IMHO as it spits out more useful information. Can you also post the relevant Postfix 'main.cf' and 'master.cf' sections?
Thomas
On Wed, Dec 22, 2010 at 13:27, Thomas Leuxner tlx@leuxner.net wrote:
May be a good idea to post some snippets of the actual Postfix delivery attempts just to make sure it really uses the external LDA. You can also crank up 'mail_debug = yes' to log more verbosely. IIRC it should show CMU Sieve warnings too (it's way old). Going forward using Pigeonhole would make more sense IMHO as it spits out more useful information. Can you also post the relevant Postfix 'main.cf' and 'master.cf' sections?
If it is not using the LDA, I don't know what it is doing. There have been several hundred thousand messages going from Postfix to Dovecot since I installed this a few months ago, all successfully arriving in the appropriate user INBOX as far as I can tell. Note that I am using a gmail account for this mailing list because I set this up way back when I was installing the mail server, and chose to leave this set up just in case things break. Right now, the mail server is working "fine" except for the sieve part behaving as if there is no sieve part. I just started trying to run it today.
part of main.cf is: ================================================================ begin # Use Dovecot SASL for authentication to submit email. smtpd_sasl_type = dovecot smtpd_sasl_path = private/dovecot-auth smtpd_sasl_auth_enable = yes
# Use Dovecot for transport delivery. virtual_transport = dovecot dovecot_destination_recipient_limit = 1 virtual_mailbox_domains = cdb:/var/maildb/domains virtual_mailbox_maps = cdb:/var/maildb/mailbox ================================================================ end
part of master.cf is: ================================================================ begin # # This is used by "virtual_transport = dovecot" in main.cf # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -a ${recipient} -d ${user}@${nexthop} -f ${sender} ================================================================ end
Note that the -a ${recipient} part was added on today, due to this being shown in the http://wiki1.dovecot.org/LDA/Sieve page. I tested that it did not break ordinary delivery. But things have been working (sans sieve) for months without it. I just assumed sieve might need it, or benefit from it.
The filesystem access timestamps are not showing any accesses happening on either sieve file except my own accesses from the command line.
I do see the following as suspicious, that the settings in "protocol lda" do not show up in "dovecot -n" at all. But I don't know if they should or not. Of particular concern was that "main_plugin" was missing. But here (domain name redacted with XXXXXXXXXXXX) is the whole output of "dovecot -n" just in case something is relevant (the "protocol lda" part of dovecot-postfix.conf follows it): ================================================================ begin # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31-22-server x86_64 Ubuntu 9.10 ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 imaps pop3s listen: 172.30.0.28, [fcca::1c], 127.0.0.1, [::1] ssl_cert_file: /etc/ssl/certs/imapx.XXXXXXXXXXXX.crt ssl_key_file: /etc/ssl/private/imapx.XXXXXXXXXXXX.key ssl_parameters_regenerate: 24 ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 verbose_proctitle: yes first_valid_uid: 250 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:/home/mail/%Ld/%Ln/mail mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 768 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@+ username_format: %Ln@%Ld verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: scheme=crypt username_format=%Ln /var/maildb/authdir/%Ld/passwd userdb: driver: passwd-file args: username_format=%Ln /var/maildb/authdir/%Ld/passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: sieve_global_path: /etc/dovecot/sieve-global.d/spam.sieve ================================================================ end
here is part of dovecot-postfix.conf (pointed to by dovecot.conf): ================================================================ begin protocol lda { log_path = /var/log/deliver/error.log info_log_path = /var/log/deliver/info.log
user = vmail
group = vmail
postmaster_address = postmaster
hostname = XXXXXXXXXXXX
mail_plugins = cmusieve
mail_plugin_dir = /usr/lib/dovecot/modules/lda
quota_full_tempfail = yes
# Format to use for logging mail deliveries. You can use variables:
# %$ - Delivery status message (e.g. "saved to INBOX")
# %m - Message-ID
# %s - Subject
# %f - From address
deliver_log_format = msgid=%m: %$
#sendmail_path = /usr/lib/sendmail
rejection_reason = Your message to <%t> was automatically rejected:%n%r
# UNIX socket path to master authentication server to find users.
#auth_socket_path = /var/run/dovecot/auth-master} ================================================================ end
-- sHiFt HaPpEnS!
On Wed, Dec 22, 2010 at 11:46, Anton Dollmaier antondollmaier@aditsystems.de wrote:
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use "sieve_before", which is not documented in the wiki.
Changed.
Only this way the sievescript really gets executed.
The global path requires a sieve script per home directory, including the global file.
I changed "sieve_global_path" to "sieve_before", restarted both Dovecot and Postfix (though if "deliver" reads the config file each time, I guess I should not need to restart for this stuff), but there's still no effect. Mail is delivered, but still just to the regular inbox.
Do I need to have per-user sieves set up, too, just to get it to work? Are there any log messages that I should expect to see when it is working?
-- sHiFt HaPpEnS!
Op 22-12-2010 19:12, Phil Howard schreef:
On Wed, Dec 22, 2010 at 11:46, Anton Dollmaier antondollmaier@aditsystems.de wrote:
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve Use "sieve_before", which is not documented in the wiki. Changed.
Don't. Check my other e-mail.
Mail is delivered, but still just to the regular inbox.
The main question is: did you read and apply the following:
http://wiki.dovecot.org/LDA/Postfix
Do I need to have per-user sieves set up, too, just to get it to work?
No.
Regards,
Stephan.
On Wed, Dec 22, 2010 at 13:28, Stephan Bosch stephan@rename-it.nl wrote:
Op 22-12-2010 19:12, Phil Howard schreef:
On Wed, Dec 22, 2010 at 11:46, Anton Dollmaier antondollmaier@aditsystems.de wrote:
plugin { sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use "sieve_before", which is not documented in the wiki.
Changed.
Don't. Check my other e-mail.
I backed that out. It is now back to "sieve_global_path".
Mail is delivered, but still just to the regular inbox.
The main question is: did you read and apply the following:
You mean did I set up Postfix to run Dovecot deliver as the virtual transport? Yes. It has been doing its thing for several months and hundreds of thousands of emails. I did add the -a option as described in the http://wiki1.dovecot.org/LDA/Sieve document.
Parts of some files, and "dovecot -n" output, in a previous message.
-- sHiFt HaPpEnS!
On Wed, Dec 22, 2010 at 13:28, Stephan Bosch stephan@rename-it.nl wrote:
Op 22-12-2010 19:12, Phil Howard schreef:
Do I need to have per-user sieves set up, too, just to get it to work?
No.
I did some debugging. It looks like I do need to have that set up: ================================================================ begin deliver(phil@ham.org): 2010-12-22 15:10:03 Error: Per-user script path is unknown. See http://wiki.dovecot.org/LDA/Sieve#location ================================================================ end
However, this message itself is broken, because that page (redirects to wiki1, now) does not have a label called "location". Or is it that the page is broken by not having a label that works for code already distributed. Whatever.
So I changed it to this, adding the setup for per-user scripts: ================================================================ begin plugin { sieve_global_path = /home/sieve-scripts/DEFAULT.sieve sieve = /home/sieve-scripts/%u } ================================================================ end
With this /home/sieve-scripts/DEFAULT.sieve compiled: ================================================================ begin require "fileinto"; fileinto "INBOX.spamtest"; ================================================================ end
And now I get this: ================================================================ begin deliver(phil@ham.org): 2010-12-22 15:32:42 Info: Loading modules from directory: /usr/lib/dovecot/modules/lda deliver(phil@ham.org): 2010-12-22 15:32:42 Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so deliver(phil@ham.org): 2010-12-22 15:32:42 Info: maildir: data=/home/mail/ham.org/phil/mail deliver(phil@ham.org): 2010-12-22 15:32:42 Info: maildir++: root=/home/mail/ham.org/phil/mail, index=, control=, inbox=/home/mail/ham.org/phil/mail deliver(phil@ham.org): 2010-12-22 15:32:42 Info: cmusieve: /home/sieve-scripts/phil@ham.org doesn't exist deliver(phil@ham.org): 2010-12-22 15:32:42 Info: cmusieve: Using sieve path: /home/sieve-scripts/DEFAULT.sieve deliver(phil@ham.org): 2010-12-22 15:32:42 Info: cmusieve: Executing script /home/sieve-scripts/DEFAULT.sievec deliver(phil@ham.org): 2010-12-22 15:32:42 Info: msgid=1293049962.4148.20.camel@lorentz: saved mail to INBOX.spamtest ================================================================ end
Success!
-- sHiFt HaPpEnS!
participants (4)
-
Anton Dollmaier
-
Phil Howard
-
Stephan Bosch
-
Thomas Leuxner