<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<div>Hi all,<br></div><div><br></div><div>I'm trying to replace a shared NFS (rock solid, btw) with a replicated two-node setup. However I've been <br></div><div>struggling with an issue connected in some way with the replication between the two servers (master-master).<br></div><div><br></div><div>The issue is once a folder (say "aaa") is created one one account, it cannot be renamed or deleted. If deleted, it will instantly reappear (when replication is running) or when renamed to "bbb", the operation will fail with an error like the one below:<br></div><div><br></div><div>Jan 15 18:21:22 ms1 dovecot: dsync-local(<a href="mailto:test2@mydomain.ext" rel="noopener noreferrer" target="_blank">test2@mydomain.ext</a>)<JtJJARLPAWBPJAAAxdOQ8Q>: Error: rename(/var/vmail-local/<a href="http://mydomain.ext/test2/idx/.bbb" rel="noopener noreferrer" target="_blank">mydomain.ext/test2/idx/.bbb</a>, /var/vmail-local/<a href="http://mydomain.ext/test2/idx/.aaa" rel="noopener noreferrer" target="_blank">mydomain.ext/test2/idx/.aaa</a>) failed: Directory not empty <br></div><div>Jan 15 18:21:22 ms1 dovecot: dsync-local(<a href="mailto:test2@mydomain.ext" rel="noopener noreferrer" target="_blank">test2@mydomain.ext</a>)<JtJJARLPAWBPJAAAxdOQ8Q>: Error: rename(/var/vmail-local/<a href="http://mydomain.ext/test2/idx/.bbb" rel="noopener noreferrer" target="_blank">mydomain.ext/test2/idx/.bbb</a>, /var/vmail-local/<a href="http://mydomain.ext/test2/idx/.aaa" rel="noopener noreferrer" target="_blank">mydomain.ext/test2/idx/.aaa</a>) failed: Directory not empty<br></div><div><br></div><div>I can see that the index directory contains both the .aaa and .bbb index directories with dovecot.log files inside.</div><div><br></div><div>If I stop the one of the nodes all works nicely and I can create, delete and rename folders at will. <br></div><div><br></div><div>My setup is as below.<br></div><div><br></div><div>dovecot --version -> 2.3.4.1 (f79e8e7e4)<br></div><div><div>the indexes are in a separate directory on /var/vmail-local/ (performance reasons).<br></div><div>the mail spool is mounted on /var/vmail/<br></div><div>One node has local disks for everything. It runs with mmap_disable=no, mail_fsync=optimized, mail_nfs_storage=no, mail_nfs_index=no<br></div><div>The other node has an NFS based /var/vmail/ and local disks for indexes. On this node I'm using the following settings:<br></div><div>mmap_disable=yes, mail_fsync=always, mail_nfs_storage=yes, mail_nfs_index=no<br></div><div>The nfs server is v4 </div><div><br></div><div>The two servers are "orchestrated" by a two-node director cluster.<br></div><div><br></div><div>I'm sure I'm missing or misconfigured something.</div><div><br></div></div><div>The doveconf -n config is, part from the NFS quick, the same on both nodes as follows: <br></div><div>--------<br></div><div># 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf <br></div><div># Pigeonhole version 0.5.4 ()<br></div><div>doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (2012)<br></div><div>doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1515)<br></div><div># OS: Linux 5.4.78-2-pve x86_64 Debian 10.7<br></div><div># Hostname: dovecot1.mydomain.ext<br></div><div>auth_master_user_separator = +<br></div><div>auth_mechanisms = plain login<br></div><div>default_process_limit = 500<br></div><div>disable_plaintext_auth = no<br></div><div>doveadm_password = # hidden, use -P to show it<br></div><div>doveadm_port = 12727<br></div><div>imap_capability = +SPECIAL-USE XLIST<br></div><div>listen = *,[::]<br></div><div>lmtp_rcpt_check_quota = yes<br></div><div>log_timestamp = "%Y-%m-%d %H:%M:%S "<br></div><div>login_trusted_networks = 172.16.20.0/24 172.16.30.0/24<br></div><div>mail_fsync = always<br></div><div>mail_max_userip_connections = 100<br></div><div>mail_nfs_storage = yes<br></div><div>mail_plugins = quota notify replication lazy_expunge acl<br></div><div>mail_privileged_group = vmail<br></div><div>mmap_disable = yes<br></div><div>namespace inbox {<br></div><div> inbox = yes<br></div><div> location =<br></div><div> mailbox Drafts {<br></div><div> special_use = \Drafts<br></div><div> }<br></div><div> mailbox EXPUNGED {<br></div><div> autoexpunge = 1 days<br></div><div> autoexpunge_max_mails = 20<br></div><div> }<br></div><div> mailbox Junk {<br></div><div> special_use = \Junk<br></div><div> }<br></div><div> mailbox Sent {<br></div><div> special_use = \Sent<br></div><div> }<br></div><div> mailbox "Sent Messages" {<br></div><div> special_use = \Sent<br></div><div> }<br></div><div> mailbox Trash {<br></div><div> special_use = \Trash<br></div><div> }<br></div><div> prefix =<br></div><div> separator = .<br></div><div>}<br></div><div>passdb {<br></div><div> args = /etc/dovecot/dovecot-sql.conf<br></div><div> driver = sql<br></div><div>}<br></div><div>passdb {<br></div><div> args = /etc/dovecot/master-users<br></div><div> driver = passwd-file<br></div><div> master = yes<br></div><div> result_success = continue<br></div><div>}<br></div><div>passdb {<br></div><div> driver = pam<br></div><div>}<br></div><div>plugin {<br></div><div> acl = vfile:/etc/dovecot/dovecot.acl<br></div><div> lazy_expunge = EXPUNGED<br></div><div> mail_replica = tcp:172.16.30.100:12727<br></div><div> quota = dict:user::file:/var/vmail/%d/%n/.quotausage<br></div><div> quota_rule = EXPUNGED:ignore<br></div><div> quota_status_nouser = DUNNO<br></div><div> quota_status_overquota = 552 5.2.2 Mailbox is full<br></div><div> quota_status_success = DUNNO<br></div><div> sieve = /var/vmail/%d/%n/.sieve<br></div><div> sieve_after = /var/vmail/%d/%n/.ispconfig.sieve<br></div><div> sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve<br></div><div> sieve_max_actions = 100<br></div><div> sieve_max_redirects = 25<br></div><div> sieve_max_script_size = 2M<br></div><div>}<br></div><div>protocols = imap pop3 lmtp<br></div><div>service aggregator {<br></div><div> fifo_listener replication-notify-fifo {<br></div><div> group = root<br></div><div> mode = 0660<br></div><div> user = vmail<br></div><div> }<br></div><div> unix_listener replication-notify {<br></div><div> group = root<br></div><div> mode = 0660<br></div><div> user = vmail<br></div><div> }<br></div><div>}<br></div><div>service auth {<br></div><div> unix_listener /var/spool/postfix/private/auth {<br></div><div> group = postfix<br></div><div> mode = 0660<br></div><div> user = postfix<br></div><div> }<br></div><div> unix_listener auth-userdb {<br></div><div> group = vmail<br></div><div> mode = 0600<br></div><div> user = vmail<br></div><div> }<br></div><div> user = root<br></div><div>}<br></div><div>service doveadm {<br></div><div> inet_listener {<br></div><div> port = 12727<br></div><div> }<br></div><div>}<br></div><div>service imap-login {<br></div><div> client_limit = 1000<br></div><div> process_limit = 512<br></div><div>}<br></div><div>service lmtp {<br></div><div> inet_listener lmtp {<br></div><div> port = 24<br></div><div> }<br></div><div> unix_listener /var/spool/postfix/private/dovecot-lmtp {<br></div><div> group = postfix<br></div><div> mode = 0600<br></div><div> user = postfix<br></div><div> }<br></div><div>}<br></div><div>service quota-status {<br></div><div> client_limit = 1<br></div><div> executable = quota-status -p postfix<br></div><div> unix_listener /var/spool/postfix/private/quota-status {<br></div><div> group = postfix<br></div><div> mode = 0660<br></div><div> user = postfix<br></div><div> }<br></div><div>}<br></div><div>service replicator {<br></div><div> process_min_avail = 1<br></div><div> unix_listener replicator-doveadm {<br></div><div> mode = 0600<br></div><div> user = vmail<br></div><div> }<br></div><div>}<br></div><div>service stats {<br></div><div> unix_listener stats-reader {<br></div><div> group = vmail<br></div><div> mode = 0660<br></div><div> user = vmail<br></div><div> }<br></div><div> unix_listener stats-writer {<br></div><div> group = vmail<br></div><div> mode = 0660<br></div><div> user = vmail<br></div><div> }<br></div><div>}<br></div><div>ssl = required<br></div><div>ssl_cert = </etc/letsencrypt/live/<a href="http://mail.mydomain.ext/fullchain.pem" rel="noopener noreferrer" target="_blank">mail.mydomain.ext/fullchain.pem</a><br></div><div>ssl_dh = # hidden, use -P to show it<br></div><div>ssl_key = # hidden, use -P to show it<br></div><div>userdb {<br></div><div> driver = prefetch<br></div><div>}<br></div><div>userdb {<br></div><div> args = /etc/dovecot/dovecot-sql.conf<br></div><div> driver = sql<br></div><div>}<br></div><div>userdb {<br></div><div> driver = passwd<br></div><div>}<br></div><div>protocol imap {<br></div><div> auth_verbose = yes<br></div><div> mail_plugins = quota imap_quota notify replication lazy_expunge acl<br></div><div>}<br></div><div>protocol pop3 {<br></div><div> auth_verbose = yes<br></div><div> mail_plugins = quota notify replication lazy_expunge acl<br></div><div> pop3_uidl_format = %08Xu%08Xv<br></div><div>}<br></div><div>protocol lda {<br></div><div> mail_plugins = sieve quota notify replication lazy_expunge acl<br></div><div> postmaster_address = <a href="mailto:postmaster@dovecot1.mydomain.ext" rel="noopener noreferrer" target="_blank">postmaster@dovecot1.mydomain.ext</a><br></div><div>}<br></div><div>protocol lmtp {<br></div><div> mail_plugins = quota sieve notify replication lazy_expunge acl<br></div><div> postmaster_address = <a href="mailto:postmaster@dovecot1.mydomain.ext" rel="noopener noreferrer" target="_blank">postmaster@dovecot1.mydomain.ext</a><br></div><div>}<br></div><div>--------------<br></div><div><br></div><div>My sql config is this:<br></div><div>---<br></div><div>password_query = SELECT email as user, password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, '/', IF(maildir_format='maildir','Maildir',maildir_format)) as userdb_mail, uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM mail_user WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id = '20' AND EXISTS (SELECT domain_id FROM mail_domain WHERE domain = '%d' AND active = 'y' AND server_id = 20) <br></div><div><br></div><div>user_query = SELECT email as user, maildir as home, CONCAT( maildir_format, ':', maildir, '/', IF(maildir_format='maildir','Maildir',maildir_format),':INDEX=/var/vmail-local/%d/%n/idx') as mail, uid, gid, CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id = '20'<br></div><div><br></div><div>---</div><div><br></div><div>Best regards,</div><div>Dave<br></div> </body>
</html>