On the target, I enabled the replication service without mail_replica and:
doveadm replicator status 'dummy-c-1*' username priority fast sync full sync success sync failed dummy-c-1 none - - - -
It only knows of dummy-c-1, no trace of his evil twin dummy-c-1@univ-nantes.fr.
On the target, I do have the same number of files
find . -type f |wc -l 8705
which is around half of what the quota is reporting (plus de index files, etc):
~# doveadm -D quota get -u dummy-c-1 Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_replication_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(dummy-c-1)<24051><>: Debug: auth USER input: dummy-c-1 home=/vmail/d/u/dummy-c-1/ quota_rule=*:backend=1000000000S doveadm(dummy-c-1)<24051><>: Debug: Added userdb setting: plugin/quota_rule=*:backend=1000000000S doveadm(dummy-c-1): Debug: Effective uid=5000, gid=5000, home=/vmail/d/u/dummy-c-1/ doveadm(dummy-c-1): Debug: Quota root: name=Quota Utilisateur backend=maildir args= doveadm(dummy-c-1): Debug: Quota rule: root=Quota Utilisateur mailbox=* bytes=1000000000 messages=0 doveadm(dummy-c-1): Debug: Quota rule: root=Quota Utilisateur mailbox=INBOX.Trash bytes=+104857600 messages=0 doveadm(dummy-c-1): Debug: Quota grace: root=Quota Utilisateur bytes=100000000 (10%) doveadm(dummy-c-1): Debug: replication: No mail_replica setting - replication disabled doveadm(dummy-c-1): Debug: Namespace : type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/vmail/d/u/dummy-c-1/ doveadm(dummy-c-1): Debug: maildir++: root=/vmail/d/u/dummy-c-1, index=, indexpvt=, control=, inbox=/vmail/d/u/dummy-c-1, alt= doveadm(dummy-c-1): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none doveadm(dummy-c-1): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= doveadm(dummy-c-1): Debug: quota: quota_over_flag check: quota_over_script unset - skipping Quota name Type Value Limit % Quota Utilisateur STORAGE 1126751 976563 115 Quota Utilisateur MESSAGE 16686 - 0
The debug mode isn't much of any help here.
Arnaud
On 08/11/2021 11:30, Aki Tuomi wrote:
Recalculation won't fix replication mistakes.
Did you ensure on the *target* server that it has only dummy-c-1 in replication and that the file count on the target server matches source server?
You might also get something useful out of
doveadm -D quota recalc|get -u userAki
On 08/11/2021 12:22 Arnaud Abélard <arnaud.abelard@univ-nantes.fr> wrote:
On 08/11/2021 10:39, Aki Tuomi wrote:
Try doveadm replicator remove dummy-c-1@univ-nantes.fr
I did so on the source server, checked indeed that dummy-c-1@univ-nantes.fr wasn't showing up in the doveadm replicator status list and it was indeed the case, so far so good. On the destination server, dummy-c-1's mailbox was still showing a 115% quota so I tried to make dovecot recalculate the quota but it did not change anything. I ended up deleting the user's mailbox and forced a replication and, to my surprise, the new mailbox still has a 115% quota.
Arnaud
Aki
On 08/11/2021 11:32 Arnaud Abélard <arnaud.abelard@univ-nantes.fr> wrote:
Hello Aki,
Thanks for the tip. I changed postfix configuration so it will not use the domain part anymore and added "auth_username_format = %Ln" just in case (and reloaded dovecot). So I shouldn't have any more new user@domain users but for all the existing users, doveadm replicator status still show user@domain clone and on the new server mails are still counted twice.
Our userdb and passdb don't have the @domain defined anywhere so I don't think it's coming from there.
Is there a local cache that still has a list of those user@domain usernames that I could reset or something?
Thanks,
Arnaud
On 08/11/2021 09:48, Aki Tuomi wrote:
This sounds like you are not normalizing usernames properly.
Either use
auth_username_format = %Ln
as global setting, or return
userattribute in both userdb and passdb lookups.Aki
On 07/11/2021 20:31 Arnaud Abélard <arnaud.abelard@univ-nantes.fr> wrote:
Hello again,
I've found out that some mailboxes are actually duplicated. Doveadm replicator status on the production server gives this:
~# doveadm replicator status 'dummy-c-1*' username priority fast sync full sync success sync failed dummy-c-1 none 01:13:19 01:13:19 01:13:19 - dummy-c-1@univ-nantes.fr none 00:15:28 00:15:28 00:15:28 -
That'd explain why mails are counted twice when replicated on the new server but where does this come from since I don't have this quota problem on the production server?
From the logs, it seems that postfix uses username@univ-nantes.fr whencalling dovecot's LDA so I suppose that the non-'@univ-nantes.fr' mailboxes are the erroneous ones.
The users aren't duplicated in our ldap user backend and aren't using the @univ-nantes.fr part.
~# doveadm user 'dummy-c-1*' dummy-c-1
Any ideas?
Thanks,
Arnaud
On 05/11/2021 16:21, Arnaud Abélard wrote: > Hello, > > We are very long time happy dovecot users (since 2008 at least). We have > around 90k mailboxes and since we had to move away from our NAS storage > to a ceph storage I jumped on the opportunity to enable compression with > the zlib plugin and dovecot's replication mecanism. We are using > debian's dovecot 2.2.27 packages on production and our new server is > running dovecot's own ce-2.3.17 packages. > > On the production server everything works fine but on the new server, > replicated mailboxes' quota is all wrong: > > on production: > # doveadm quota get -u dummy-c-1 > Quota name Type Value Limit % > Quota Utilisateur STORAGE 660026 976563 67 > Quota Utilisateur MESSAGE 8651 - 0 > > on new server: > doveadm quota get -u dummy-c-1 > Quota name Type Value Limit % > Quota Utilisateur STORAGE 1125251 976563 115 > Quota Utilisateur MESSAGE 16646 - 0 > > If I add all the S flag from the filenames n both servers I get exactly > the same usage, which is coherent with the quota on the production server: > > # find . -type f | grep 'S=' | awk -F'S=' '{print $2}' | awk -F',' > '{print $1}' | awk -F':' '{print $1}' | paste -sd+ | bc -l > 675865938 > > And I have exactly the same amountof mails on the two server, the > replication works as expected, no unwanted duplication of mails occurs. > > Of course, I've tried to ask dovecot to recalculate quotas with doveadm > quota recalc -u <username>, but it doesn't fix the problem. > > What am I missing? > > Thanks, > > Arnaud > > PS: Here is my doveconf -n output: > > # 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.17 (054dddfa) > # OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11 > # Hostname: gromel-test > auth_cache_size = 10 k > auth_verbose = yes > disable_plaintext_auth = no > doveadm_password = # hidden, use -P to show it > hostname = gromel1.univ-nantes.prive > lda_mailbox_autosubscribe = yes > listen = * > log_timestamp = "%Y-%m-%d %H:%M:%S " > login_trusted_networks = (...) > mail_gid = 5000 > mail_location = maildir:%h > mail_plugins = quota zlib notify replication > mail_privileged_group = vmail > mail_uid = 5000 > maildir_stat_dirs = yes > maildir_very_dirty_syncs = yes > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character subaddress comparator-i;ascii-numeric relational regex > imap4flags copy include variables body enotify environment mailbox date > index ihave duplicate mime foreverypart extracttext > namespace { > inbox = yes > location = > prefix = INBOX. > separator = . > type = private > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > plugin { > quota = maildir:Quota Utilisateur > quota_exceeded_message = Cet utilisateur a dépassé son quota, votre > message n'a pu lui être livré. > quota_full_tempfail = yes > quota_rule = *:storage=1000M > quota_rule2 = INBOX.Trash:storage=+100M > sieve = ~/dovecot.sieve > sieve_dir = ~/sieve > sieve_extensions = -vacation > sieve_global_dir = /var/lib/dovecot/sieve/global/ > sieve_max_redirects = 1 > zlib_save = gz > zlib_save_level = 6 > } > postmaster_address = postmaster@<snip> > protocols = imap pop3 sieve > replication_max_conns = 50 > service auth { > client_limit = 49452 > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service dict { > unix_listener dict { > mode = 0600 > user = vmail > } > } > service doveadm { > inet_listener { > port = 12345 > } > } > service imap-login { > process_min_avail = 8 > service_count = 0 > user = mail > } > service imap { > executable = imap > process_limit = 16384 > vsz_limit = 2 G > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > process_min_avail = 8 > service_count = 0 > user = mail > vsz_limit = 2 G > } > service managesieve { > drop_priv_before_exec = yes > process_limit = 16384 > } > service pop3-login { > process_min_avail = 8 > service_count = 0 > user = mail > } > service pop3 { > drop_priv_before_exec = yes > process_limit = 16384 > } > ssl = no > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > verbose_proctitle = yes > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = quota zlib notify replication imap_quota zlib > } > protocol pop3 { > mail_plugins = quota zlib notify replication > } > protocol sieve { > mail_max_userip_connections = 10 > } > protocol lda { > mail_plugins = quota zlib notify replication sieve zlib > } > > > >
-- Arnaud Abélard Responsable pôle Système et Stockage Service Infrastructures DSIN Université de Nantes
-- Arnaud Abélard Responsable pôle Système et Stockage Service Infrastructures DSIN Université de Nantes
-- Arnaud Abélard Responsable pôle Système et Stockage Service Infrastructures DSIN Université de Nantes