altmove reverse doesn't work

Wed Apr 14 17:55:42 EEST 2021

Same here.

doveadm altmove -r is broken, needs to be fixed. We want to recompres our ALT mdbox storage from LZMA to ZSTD and we can not do it because doveadm altmove -r does not work.

Regards

De: dovecot <dovecot-bounces at dovecot.org> En nombre de Zdenek Zámecník
Enviado el: viernes, 9 de abril de 2021 15:12
Para: Aki Tuomi <aki.tuomi at open-xchange.com>; Dovecot Mailing List <dovecot at dovecot.org>
Asunto: Re: altmove reverse doesn't work

I already trued doveadm purge but with no luck. Also debug parameter doesn't show any interesting output as you can see below. It shows that it's moving about 70000 messages but in fact it doesn't do anything. If I repeat the command the output is still same. I just found that a few other people already explained same problem, for example here: https://dovecot.org/pipermail/dovecot/2021-February/121329.html

Is there any chance to get it fixed in upstream?

Apr 09 14:58:00 Debug: Loading modules from directory: /usr/lib/dovecot/modules

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so

Apr 09 14:58:00 Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so

Apr 09 14:58:00 Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)

Apr 09 14:58:00 Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so

Apr 09 14:58:00 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: auth-master: userdb lookup(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Started userdb lookup

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Connecting

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (pid=14462,uid=0): Client connected (fd=8)

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: auth-master: userdb lookup(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): auth USER input: myuser at mydomain.yyy<mailto:myuser at mydomain.yyy> quota_rule=*:bytes=20GB

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: auth-master: userdb lookup(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Finished userdb lookup (username=myuser at mydomain.yyy<mailto:username=myuser at mydomain.yyy> quota_rule=*:bytes=20GB)

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>)<27721><>: Debug: Added userdb setting: plugin/quota_rule=*:bytes=20GB

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Effective uid=2000, gid=2000, home=/var/vmail/mydomain.yyy.com/myuser

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota root: name=User quota backend=dict args=:proxy::quota

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota rule: root=User quota mailbox=* bytes=21474836480 messages=0

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota rule: root=User quota mailbox=Trash ignored

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota rule: root=User quota mailbox=Junk ignored

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota warning: bytes=17179869184 (80%) messages=0 reverse=no command=quota-warning 90 myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota warning: bytes=18253611008 (85%) messages=0 reverse=no command=quota-warning 95 myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota warning: bytes=20401094656 (95%) messages=0 reverse=no command=quota-warning 105 myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Quota grace: root=User quota bytes=2147483648 (10%)

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: quota-dict: user=myuser at mydomain.yyy<mailto:user=myuser at mydomain.yyy>, uri=proxy::quota, noenforcing=0

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:ALT=/var/vmail-archive/mydomain.yyy.com/myuser/mdbox

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: fs: root=/var/vmail/mydomain.yyy.com/myuser/mdbox, index=, indexpvt=, control=, inbox=, alt=/var/vmail-archive/mydomain.yyy.com/myuser/mdbox

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: fts: No fts setting - plugin disabled

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: initializing backend with data: vfile

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: acl username = myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: owner = 1

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl vfile: Global ACLs disabled

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Namespace : type=shared, prefix=shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=mdbox:/var/vmail/%d/%n/mdbox:ALT=/var/vmail-archive/%d/%n/mdbox

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt=

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: fts: No fts setting - plugin disabled

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: initializing backend with data: vfile

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: acl username = myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl: owner = 0

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl vfile: Global ACLs disabled

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: quota: quota_over_flag check: quota_over_script unset - skipping

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl vfile: file /var/vmail/mydomain.yyy.com/myuser/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-acl not found

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl vfile: file /var/vmail/mydomain.yyy.com/myuser/mdbox/mailboxes/dovecot-acl not found

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: acl vfile: file /var/vmail/mydomain.yyy.com/myuser/mdbox/mailboxes/Sent Items/dbox-Mails/dovecot-acl not found

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Mailbox Sent Items: Mailbox opened because: altmove

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=Sent Items uid=4279

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: Namespace : Using permissions from /var/vmail/mydomain.yyy.com/myuser/mdbox: mode=0770 gid=default

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=Sent Items uid=4280

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=Sent Items uid=4281

Apr 09 14:58:00 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=Sent Items uid=4282

...

Apr 09 14:58:01 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=INBOX uid=165052

Apr 09 14:58:01 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: altmove: box=INBOX uid=165053

Apr 09 14:58:31 doveadm(myuser at mydomain.yyy<mailto:myuser at mydomain.yyy>): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb (pid=14462,uid=0): Disconnected: Connection closed (fd=8)

On 4/9/21 2:40 PM, Aki Tuomi wrote:

On 09/04/2021 15:34 Zdeněk Zámečník <diego at dixy.cz><mailto:diego at dixy.cz> wrote:

I am having troubles with moving emails from alternative storage to primary storage. If I call command like:

doveadm altmove -r -u myuser at mydomain.yyy<mailto:myuser at mydomain.yyy> ALL

it is doing something (it took almost a minute at first time) but there are still emails in alternative storage. There is not reported any error. Below is my config:

Try running

doveadm -D altmove -r -u myuser at mydomain.yyy<mailto:myuser at mydomain.yyy> ALL

maybe it sheds some light?

Also, try running doveadm purge after altmove.

Aki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20210414/5a8ce926/attachment-0001.html>