Hello,
Following up on old thread:
http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html
I have the exact same issue with 2.2.10:
ii dovecot-antispam 2.0+20130822-2 amd64
Dovecot plugins for training spam filters
ii dovecot-core 1:2.2.10-1 amd64
secure POP3/IMAP server - core files
ii dovecot-imapd 1:2.2.10-1 amd64
secure POP3/IMAP server - IMAP daemon
ii dovecot-ldap 1:2.2.10-1 amd64
secure POP3/IMAP server - LDAP support
ii dovecot-lmtpd 1:2.2.10-1 amd64
secure POP3/IMAP server - LMTP server
ii dovecot-managesieved 1:2.2.10-1 amd64
secure POP3/IMAP server - ManageSieve server
ii dovecot-mysql 1:2.2.10-1 amd64
secure POP3/IMAP server - MySQL support
ii dovecot-pop3d 1:2.2.10-1 amd64
secure POP3/IMAP server - POP3 daemon
ii dovecot-sieve 1:2.2.10-1 amd64
secure POP3/IMAP server - Sieve filters support
As even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers.
The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated.
I have tried to add additional sieve related options to the replicator:
service replicator { process_min_avail = 1 }
plugin { mail_replica = tcp:1.2.3.4 replication_full_sync_interval = 1 hours
#Added (makes no difference) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve_after sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 1h #Added (makes no difference) }
service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } }
It did not help. If there is one way I would like to get this work without upgrading dovecot.
Thank you.
Op 19-7-2016 om 11:30 schreef Luescher Claude:
Hello,
Following up on old thread:
http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html
I have the exact same issue with 2.2.10:
It did not help. If there is one way I would like to get this work without upgrading dovecot.
Many bugs were fixed since this version. That is likely Pigeonhole 0.4.2, which is very old (26-09-2013). The synchronization didn't work flawlessly until at least version 0.4.4, which goes with Dovecot 2.2.15.
Regards,
Stephan.
Hello,
Are you sure about that I would need so recent version? That is ok to have some bugfixes but you are talking about the latest cutting edge debian jessie backports package. Even the mainline debian jessie only have 2.2.13 in it and my system is a complex wheezy installation, the last thing I want to do is to move all my components into jessie just because of this or do you know 2.2.15 deb packages built for wheezy? My installation is a multi-node cluster already I would have to update all the nodes but if it would be only to install a couple of new debs on them I would not hesitate trying a newer dovecot version.
Thank you.
On Tue, 19 Jul 2016 11:54:27 +0200, Stephan Bosch wrote:
Op 19-7-2016 om 11:30 schreef Luescher Claude:
Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html [1] I have the exact same issue with 2.2.10: It did not help. If there is one way I would like to get this work without upgrading dovecot.
Many bugs were fixed since this version. That is likely Pigeonhole 0.4.2, which is very old (26-09-2013). The synchronization didn't work flawlessly until at least version 0.4.4, which goes with Dovecot 2.2.15.
Regards,
Stephan.
Links:
[1] http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html
On 2016.07.19. 15:03, Luescher Claude wrote:
Hello,
Are you sure about that I would need so recent version? That is ok to have some bugfixes but you are talking about the latest cutting edge debian jessie backports package. Even the mainline debian jessie only have 2.2.13 in it and my system is a complex wheezy installation, the last thing I want to do is to move all my components into jessie just because of this or do you know 2.2.15 deb packages built for wheezy? My installation is a multi-node cluster already I would have to update all the nodes but if it would be only to install a couple of new debs on them I would not hesitate trying a newer dovecot version.
Thank you.
If you have a huge multi node system, then you need to have testing system also. It's easy to compile Dovecot by yourself and make deb packages, if needed.
-- KSB
I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot:
/usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766)
/usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql
and dovecot-2.2-pigeonhole-0.4.15.
It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. Otherwise I will have to look for other solutions like lsync, rsync to sync them between the nodes.
Here is my complete doveconf -n output:
# 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_cache_size = 1 M auth_mechanisms = plain login cram-md5 default_vsz_limit = 512 M dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 7777 login_greeting = mail.company.com is ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = " quota notify replication" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader namespace cyruscompat1 { alias_for = hidden = yes list = no location = prefix = INBOX/ separator = / type = private } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = spool2dir antispam_spam = Spam;SPAM antispam_spool2dir_notspam = /home/vmail/spam_training/good/new/%%020lu-%u-%%05lu antispam_spool2dir_spam = /home/vmail/spam_training/bad/new/%%020lu-%u-%%05lu antispam_trash_pattern_ignorecase = trash;Deleted *;éléments supprimés;gelöschte *;&AMk-l&AOk-ments supprim&AOk-s mail_replica = tcp:1.2.3.4 quota = dict:User quota::proxy::quota quota_grace = 10M quota_rule = *:storage=100M quota_rule2 = Trash:storage=+10%% quota_rule3 = Deleted Messages:storage=+10%% quota_status_nouser = DUNNO quota_status_success = DUNNO quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u replication_full_sync_interval = 1 hours sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20 } pop3c_host = pop3.e2advance.net postmaster_address = postmaster@company.com protocols = imap pop3 lmtp imap lmtp sieve pop3 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = $default_internal_user } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 7777 } } service imap-login { process_limit = 256 process_min_avail = 5 } service lmtp { unix_listener /var/spool/postfix/public/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10020 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = </etc/ssl/private/sub.class1.server.ca.pem ssl_cert = </etc/ssl/private/star.company.com.crt ssl_key = </etc/ssl/private/star.company.com.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota notify replication sieve" } protocol lda { mail_plugins = " quota notify replication sieve" } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 mail_plugins = " quota notify replication imap_quota antispam" } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_no_flag_updates = yes pop3_save_uidl = yes pop3_uidl_format = %v.%u } protocol doveadm { mail_plugins = " quota notify replication" }
On Tue, 19 Jul 2016 15:27:54 +0300, KSB wrote:
On 2016.07.19. 15:03, Luescher Claude wrote:
Hello, Are you sure about that I would need so recent version? That is ok to have some bugfixes but you are talking about the latest cutting edge debian jessie backports package. Even the mainline debian jessie only have 2.2.13 in it and my system is a complex wheezy installation, the last thing I want to do is to move all my components into jessie just because of this or do you know 2.2.15 deb packages built for wheezy? My installation is a multi-node cluster already I would have to update all the nodes but if it would be only to install a couple of new debs on them I would not hesitate trying a newer dovecot version. Thank you.
If you have a huge multi node system, then you need to have testing system also. It's easy to compile Dovecot by yourself and make deb packages, if needed.
-- KSB
On 2016-07-20 14:51, Luescher Claude wrote:
I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot:
/usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766)
/usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql
and dovecot-2.2-pigeonhole-0.4.15.
It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere.
I did report a year ago [1] that I did observe mails being synced but no sieves. Although I did report at the end that replication started again, I do have to report today that it stopped again shortly afterwards (even on current version of dovecot and pigeonhole). The only way to get sieves synced in my case is: removing the corresponding files at one server, update the sieve files at the other server [2]. Only then, sieves become synced. On very rare occasions syncing happens without applying the first step just mentioned. This issue looks to me a very subtle one I am unable to reproduce reliably, sorry.
HTH, Michael
[1] http://dovecot.org/pipermail/dovecot/2015-May/100839.html [2] I can live with that workaround because I am the only user having sieves activated.
Op 20-7-2016 om 14:51 schreef Luescher Claude:
I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot:
/usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766)
/usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql
and dovecot-2.2-pigeonhole-0.4.15.
It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. Otherwise I will have to look for other solutions like lsync, rsync to sync them between the nodes.
Is the doveadm sieve plugin properly installed at both sides of the replication? On my system (Debian Jessie), it is located at /usr/lib/dovecot/modules/doveadm. The file name is lib10_doveadm_sieve_plugin.so.
When mail_debug is enabled, the replication should log all kinds of doveadm-sieve debug messages.
Any errors in the log?
Regards,
Stephan.
Hello,
Thanks for the advice. I have looked for the libs and here is the difference:
Dovecot production env 2.2.10:
/usr/lib/dovecot/modules/doveadm rw-r--r-- 1 root root 18560 Jan 9 2014 lib10_doveadm_acl_plugin.so -rw-r--r-- 1 root root 14256 Jan 9 2014 lib10_doveadm_expire_plugin.so -rw-r--r-- 1 root root 10232 Jan 9 2014 lib10_doveadm_quota_plugin.so -rw-r--r-- 1 root root 51352 Jan 9 2014 lib10_doveadm_sieve_plugin.so -rw-r--r-- 1 root root 14344 Jan 9 2014 lib20_doveadm_fts_plugin.so
Dovecot new test env:
/usr/dovecot/lib/dovecot/doveadm -rw-r--r-- 1 root root 140574 Jul 20 12:53 lib10_doveadm_acl_plugin.a -rwxr-xr-x 1 root root 1066 Jul 20 12:53 lib10_doveadm_acl_plugin.la -rwxr-xr-x 1 root root 98910 Jul 20 12:53 lib10_doveadm_acl_plugin.so -rw-r--r-- 1 root root 56108 Jul 20 12:53 lib10_doveadm_expire_plugin.a -rwxr-xr-x 1 root root 1087 Jul 20 12:53 lib10_doveadm_expire_plugin.la -rwxr-xr-x 1 root root 43879 Jul 20 12:53 lib10_doveadm_expire_plugin.so -rw-r--r-- 1 root root 97212 Jul 20 12:53 lib10_doveadm_quota_plugin.a -rwxr-xr-x 1 root root 1080 Jul 20 12:53 lib10_doveadm_quota_plugin.la -rwxr-xr-x 1 root root 63227 Jul 20 12:53 lib10_doveadm_quota_plugin.so -rw-r--r-- 1 root root 338560 Jul 20 12:53 lib10_doveadm_sieve_plugin.a -rwxr-xr-x 1 root root 1286 Jul 20 12:53 lib10_doveadm_sieve_plugin.la -rwxr-xr-x 1 root root 177604 Jul 20 12:53 lib10_doveadm_sieve_plugin.so -rw-r--r-- 1 root root 139748 Jul 20 12:53 lib20_doveadm_fts_plugin.a -rwxr-xr-x 1 root root 1066 Jul 20 12:53 lib20_doveadm_fts_plugin.la -rwxr-xr-x 1 root root 94615 Jul 20 12:53 lib20_doveadm_fts_plugin.so
Also I have set the mail_log to debug as you have suggested and I see the following for users in the logs (with my "old" version):
Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com):
Debug: doveadm-sieve: Iterating Sieve mailbox attributes
Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com):
Debug: sieve: Pigeonhole version 0.4.2 initializing
Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com):
Debug: sieve: include: sieve_global_dir is not set; it is currently not
possible to include :global' scripts. Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/user1@company.com//.dovecot.sieve Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/user1@company.com//sieve Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com): Debug: sieve-storage: using permissions from /home/vmail/company.com/user1@company.com//sieve: mode=0700 gid=-1 Jul 21 11:52:27 server1 dovecot: dsync-server(user1@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include :global' scripts.
Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug:
sieve-storage: using active sieve script path:
/home/vmail/company.com/user2@company.com//.dovecot.sieve
Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug:
sieve-storage: using sieve script storage directory:
/home/vmail/company.com/user2@company.com//sieve
Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug:
sieve-storage: using permissions from
/home/vmail/company.com/user2@company.com//sieve: mode=0700 gid=-1
Jul 21 15:17:21 server1 dovecot: dsync-local(user2@company.com): Debug:
sieve-storage: relative path to sieve storage in active link: sieve/
Jul 21 15:17:21 server1 dovecot: dsync-local(user3@company.com): Debug:
doveadm-sieve: Iterating Sieve mailbox attributes
Jul 21 15:17:21 server1 dovecot: dsync-local(user3@company.com): Debug:
sieve: Pigeonhole version 0.4.2 initializing
Jul 21 15:17:21 server1 dovecot: dsync-local(user3@company.com): Debug:
sieve: include: sieve_global_dir is not set; it is currently not
possible to include `:global' scripts.
Jul 21 15:17:21 server1 dovecot: dsync-local(user3@company.com): Debug:
sieve-storage: using active sieve script path:
/home/vmail/company.com/user3@company.com//.dovecot.sieve
First problem I notice is this /home/vmail/company.com/user1@company.com//.dovecot.sieve missing. Is this supposed to be a file or a directory? The second sieve_global_dir, I don't need global rules. The third is this minus gid=-1 value. The user/group/rights on that directory is not different from the mail dirs (where the replication works):
ls -al /home/vmail/company.com/user2@company.com//sieve drwx--S--- 2 vmail vmail 4096 Jul 21 15:17 tmp
The important directory for me is the siev_after because roundcube vacation plugin puts the sieves in there for example:
-rw------- 1 vmail vmail 413 Jul 21 15:23 01_vacation.sieve -rw------- 1 vmail vmail 124 Jul 21 15:23 01_vacation.svbin -rw-r--r-- 1 vmail vmail 193 Feb 26 2014 antispam.sieve -rw-r--r-- 1 vmail vmail 272 Apr 13 2015 antispam.svbin
Gets created after setting out of office message but will not get replicated to the other node.
As I have showed in my earlier doveconf output is defined:
sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20
And the sieve scripts work locally per server they are just not replicating as they should be.
On Wed, 20 Jul 2016 16:31:55 +0200, Stephan Bosch wrote:
Op 20-7-2016 om 14:51 schreef Luescher Claude:
I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot: /usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766) /usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql and dovecot-2.2-pigeonhole-0.4.15. It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. Otherwise I will have to look for other solutions like lsync, rsync to sync them between the nodes.
Is the doveadm sieve plugin properly installed at both sides of the replication? On my system (Debian Jessie), it is located at /usr/lib/dovecot/modules/doveadm. The file name is lib10_doveadm_sieve_plugin.so.
When mail_debug is enabled, the replication should log all kinds of doveadm-sieve debug messages.
Any errors in the log?
Regards,
Stephan.
Op 21-7-2016 om 15:50 schreef Luescher Claude:
Hello,
Thanks for the advice. I have looked for the libs and here is the difference:
[...]
First problem I notice is this /home/vmail/company.com/user1@company.com//.dovecot.sieve missing. Is this supposed to be a file or a directory? The second sieve_global_dir, I don't need global rules. The third is this minus gid=-1 value. The user/group/rights on that directory is not different from the mail dirs (where the replication works):
ls -al /home/vmail/company.com/user2@company.com//sieve drwx--S--- 2 vmail vmail 4096 Jul 21 15:17 tmp
The important directory for me is the siev_after because roundcube vacation plugin puts the sieves in there for example:
-rw------- 1 vmail vmail 413 Jul 21 15:23 01_vacation.sieve -rw------- 1 vmail vmail 124 Jul 21 15:23 01_vacation.svbin -rw-r--r-- 1 vmail vmail 193 Feb 26 2014 antispam.sieve -rw-r--r-- 1 vmail vmail 272 Apr 13 2015 antispam.svbin
Gets created after setting out of office message but will not get replicated to the other node.
As I have showed in my earlier doveconf output is defined:
sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20
And the sieve scripts work locally per server they are just not replicating as they should be.
There is your problem. The sieve_after/sieve_before settings usually point to administrator-controlled global scripts. Therefore, these are not replicated by the doveadm_sieve plugin. Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated.
Regards,
Stephan.
Hello,
Ok so I have set everything to ~/sieve dir on the 2 nodes:
doveconf -n | grep siev
managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader sieve = ~/.dovecot.sieve sieve_after = ~/sieve sieve_before = ~/sieve sieve_dir = ~/sieve sieve_extensions = +editheader sieve_max_redirects = 20 protocols = " imap lmtp sieve pop3" service managesieve-login { inet_listener sieve { mail_plugins = " quota notify replication sieve" mail_plugins = " quota notify replication sieve"
This way after I add a brand new user the sieve dir will be replicated but nothing after that. I have added a new user on node2 then enabled out of office message, the sieve files were created under the:
/home/vmail/company.com/user5@company.com/sieve
-rw------- 1 vmail vmail 1011 Jul 22 09:49 01_vacation.sieve -rw------- 1 vmail vmail 644 Jul 22 09:49 01_vacation.svbin
directory but were never replicated not even after restarting both dove nodes.
Node2
Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com):
Debug: doveadm-sieve: Iterating Sieve mailbox attributes
Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com):
Debug: sieve: Pigeonhole version 0.4.2 initializing
Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com):
Debug: sieve: include: sieve_global_dir is not set; it is currently not
possible to include :global' scripts. Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/user5@company.com//.dovecot.sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/user5@company.com//sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com): Debug: sieve-storage: permission lookup failed from /home/vmail/company.com/user5@company.com//sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com): Debug: sieve-storage: created storage directory /home/vmail/company.com/user5@company.com//sieve/tmp Jul 22 09:47:54 mailnode2 dovecot: dsync-local(user5@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include :global' scripts.
Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com):
Debug: sieve-storage: using active sieve script path:
/home/vmail/company.com/user5@company.com//.dovecot.sieve
Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com):
Debug: sieve-storage: using sieve script storage directory:
/home/vmail/company.com/user5@company.com//sieve
Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com):
Debug: sieve-storage: using permissions from
/home/vmail/company.com/user5@company.com//sieve: mode=0700 gid=-1
Jul 22 09:47:54 mailnode2 dovecot: dsync-server(user5@company.com):
Debug: sieve-storage: relative path to sieve storage in active link:
sieve/
Node1
Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com):
Debug: doveadm-sieve: Iterating Sieve mailbox attributes
Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com):
Debug: sieve: Pigeonhole version 0.4.2 initializing
Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com):
Debug: sieve: include: sieve_global_dir is not set; it is currently not
possible to include :global' scripts. Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/user5@company.com//.dovecot.sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/user5@company.com//sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com): Debug: sieve-storage: permission lookup failed from /home/vmail/company.com/user5@company.com//sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com): Debug: sieve-storage: created storage directory /home/vmail/company.com/user5@company.com//sieve/tmp Jul 22 09:50:16 mailnode1 dovecot: dsync-server(user5@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include :global' scripts.
Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com):
Debug: sieve-storage: using active sieve script path:
/home/vmail/company.com/user5@company.com//.dovecot.sieve
Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com):
Debug: sieve-storage: using sieve script storage directory:
/home/vmail/company.com/user5@company.com//sieve
Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com):
Debug: sieve-storage: using permissions from
/home/vmail/company.com/user5@company.com//sieve: mode=0700 gid=-1
Jul 22 09:50:16 mailnode1 dovecot: dsync-local(user5@company.com):
Debug: sieve-storage: relative path to sieve storage in active link:
sieve/
On Thu, 21 Jul 2016 17:19:25 +0200, Stephan Bosch wrote:
Op 21-7-2016 om 15:50 schreef Luescher Claude:
Hello, Thanks for the advice. I have looked for the libs and here is the difference:
[...]
First problem I notice is this /home/vmail/company.com/user1@company.com [1]//.dovecot.sieve missing. Is this supposed to be a file or a directory? The second sieve_global_dir, I don't need global rules. The third is this minus gid=-1 value. The user/group/rights on that directory is not different from the mail dirs (where the replication works): ls -al /home/vmail/company.com/user2@company.com [2]//sieve drwx--S--- 2 vmail vmail 4096 Jul 21 15:17 tmp The important directory for me is the siev_after because roundcube vacation plugin puts the sieves in there for example: -rw------- 1 vmail vmail 413 Jul 21 15:23 01_vacation.sieve -rw------- 1 vmail vmail 124 Jul 21 15:23 01_vacation.svbin -rw-r--r-- 1 vmail vmail 193 Feb 26 2014 antispam.sieve -rw-r--r-- 1 vmail vmail 272 Apr 13 2015 antispam.svbin Gets created after setting out of office message but will not get replicated to the other node. As I have showed in my earlier doveconf output is defined: sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20 And the sieve scripts work locally per server they are just not replicating as they should be.
There is your problem. The sieve_after/sieve_before settings usually point to administrator-controlled global scripts. Therefore, these are not replicated by the doveadm_sieve plugin. Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated.
Regards,
Stephan.
Links:
[1] mailto:/home/vmail/company.com/user1@company.com [2] mailto:/home/vmail/company.com/user2@company.com
Stephan Bosch <stephan@rename-it.nl> wrote:
Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated.
I do have those files in those directories:
KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
-rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
-rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve
KB> ls -al default
-rw-r--r-- 1 root wheel 5909 Jul 21 19:39 default
KB> doveadm sieve put -u mike -a default < defaultI do see traffic arriving at MW when listening on the defined port for synchronisation (tcpdump), but the new default file at KB doesn't become synchronized:
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
-rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieveIf I do remove that file at MW and repeat "doveadm sieve put" at KB, I do see synchronization succeed:
MW> rm /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
ls: /var/mail/.homedirs/mike/.sieves/default.sieve: No such file or directory
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
KB> doveadm sieve put -u mike -a default < default
KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
-rw------- 1 vmail dovecot uarch 5909 Jul 21 19:40 /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve
lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
-rw------- 1 vmail dovecot uarch 5909 Jan 1 1970 /var/mail/.homedirs/mike/.sieves/default.sieveThe date of the file puzzles me, though.
JFTR: This is a recent FreeBSD and ZFS. JFTR: The logfiles do show the very same messages as Claud has shown in his last mail.
Regards, Michael
Hello,
So basically this sieve replication support is still in some broken alfa stage and one should not build production system on top of it.
I have found this older thread: http://www.dovecot.org/list/dovecot/2013-February/088640.html
We came up with a few options for syncing:
rsync/unison ~/sieve every x seconds
move sieve dirs to separatate partition and sync with gluster or drbd
patch roundcube managesieve sieve plugin to upload to both managesieve servers (only roundcube can access managesieve in our setup)
store sieve scripts in mysql and patch roundcube managesieve plugin to directly access the db instead of using managesieve (we tried this, but apparently sieve scripts read from db via dovecot dict can't have more than one line? show stopper due to roundcube sieve plugin
needing comments to parse the scripts)
The database solution would be perfect since the nodes are using a db master-master cluster but at the time of that post this was not working. My original 2.2.10 dovecot is around that time so I would have to upgrade for that again and at the end it would might not work because as he said roundcube using multi-line sieve scripts.
The problem with rsync is that rsync from where? If I rsync from machine A -> B and let's say the user already have sieves on both nodes and changes his holiday sieve on node B then the file will be overwritten. I will give unison a try otherwise I have to put the sieves on a shared filesystem like OCFS2 which imo is not reliable for production either. When nodes lose connectivity that can end in kernel crash, autoreboot on 1 node or both.
On Sat, 23 Jul 2016 18:17:58 +0200, Michael Grimm wrote:
Stephan Bosch wrote:
Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated.
I do have those files in those directories:
KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve
KB> ls -al default -rw-r--r-- 1 root wheel 5909 Jul 21 19:39 default
KB> doveadm sieve put -u mike -a default < default
I do see traffic arriving at MW when listening on the defined port for synchronisation (tcpdump), but the new default file at KB doesn't become synchronized:
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw------- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve
If I do remove that file at MW and repeat "doveadm sieve put" at KB, I do see synchronization succeed:
MW> rm /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve ls: /var/mail/.homedirs/mike/.sieves/default.sieve: No such file or directory lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve
KB> doveadm sieve put -u mike -a default < default
KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw------- 1 vmail dovecot uarch 5909 Jul 21 19:40 /var/mail/.homedirs/mike/.sieves/default.sieve
MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx------ 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw------- 1 vmail dovecot uarch 5909 Jan 1 1970 /var/mail/.homedirs/mike/.sieves/default.sieve
The date of the file puzzles me, though.
JFTR: This is a recent FreeBSD and ZFS. JFTR: The logfiles do show the very same messages as Claud has shown in his last mail.
Regards, Michael
Links:
[1] mailto:stephan@rename-it.nl
Op 7/19/2016 om 11:30 AM schreef Luescher Claude:
Hello,
Following up on old thread:
http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html
I have the exact same issue with 2.2.10:
ii dovecot-antispam 2.0+20130822-2
amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1
amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1
amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1
amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1
amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1
amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1
amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1
amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1
amd64 secure POP3/IMAP server - Sieve filters supportAs even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers.
The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated.
The following bugs were fixed recently:
https://github.com/dovecot/core/commit/b4adb461ce12bf578d2d70806b205cf3cbf1a... https://github.com/dovecot/core/commit/27ccbb0f36e07141785db94557afb63a2aa9e...
I wonder whether this also applies to your problem.
Regards,
Stephan.
participants (4)
-
KSB
-
Luescher Claude
-
Michael Grimm
-
Stephan Bosch