"doveadm altmove -r" not working ?
Hello, I have some trouble using "doveadm altmove -r". Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect. We are using Dovecot 2.3.4 (Debian Buster package). mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox doveadm altmove -u myuser since 90d -> some message files are created under /slow/my/homedir/mdbox/storage doveadm altmove -r -u myuser all -> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage. Is there some known issues with doveadm altmove in this version ? Or am I missing something ? I can share more config details if needed. -- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr
Hi, Your problem might be similar to mine a long time ago. IMO behavior of "doveadm altmove -r" command with mdbox format is different to sdbox format: - With sdbox format, when you move back a message to primary storage (aka default storage), only the message and the file containing it are affected. There is nothing impacted. - With mdbox format, a file in storage contains multiple messages. When a message is moved back to primary storage, other messages in the same file storage will go with it. The whole file is moved as well. My workaround for this case is: In the mdbox_altmove_add_files function, after defining action for the message which is marked as MOVE_FROM_ALT, we will continue checking other messages in the file storage. If they are not listed in altmoves (they stay in alternate storage), their action will be MDBOX_MSG_ACTION_MOVE_TO_ALT. That means they will be moved 2 times: to primary storage as usual then to alternate storage again to ensure finally they are in alternate storage. - Pros: - Simple - Do not impact other parts - Do not require to update other processes to perform MOVE_FROM_ALT messages - Cons: - Reduce performance when number of messages is big - Increase file storage counter - If you don't mind building Dovecot from source, give my patch a look. If you don't mind building Dovecot from source, give my patch a look. On Tue, 21 Mar 2023 at 05:26, Benoit Branciard < Benoit.Branciard@univ-paris1.fr> wrote:
Hello,
I have some trouble using "doveadm altmove -r".
Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect.
We are using Dovecot 2.3.4 (Debian Buster package).
mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox
doveadm altmove -u myuser since 90d
-> some message files are created under /slow/my/homedir/mdbox/storage
doveadm altmove -r -u myuser all
-> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage.
Is there some known issues with doveadm altmove in this version ? Or am I missing something ?
I can share more config details if needed.
-- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr
Hi, Sorry, just read your email again and I realize your problem is different to mine. - Yours is that everything should be moved back to the default storage but nothing - Mine is that some of them should be moved back but everything Does "-Dv" give you any hint? On Tue, 21 Mar 2023 at 08:45, Duc Anh Do <doducanh2710@gmail.com> wrote:
Hi,
Your problem might be similar to mine a long time ago. IMO behavior of "doveadm altmove -r" command with mdbox format is different to sdbox format:
- With sdbox format, when you move back a message to primary storage (aka default storage), only the message and the file containing it are affected. There is nothing impacted. - With mdbox format, a file in storage contains multiple messages. When a message is moved back to primary storage, other messages in the same file storage will go with it. The whole file is moved as well.
My workaround for this case is: In the mdbox_altmove_add_files function, after defining action for the message which is marked as MOVE_FROM_ALT, we will continue checking other messages in the file storage. If they are not listed in altmoves (they stay in alternate storage), their action will be MDBOX_MSG_ACTION_MOVE_TO_ALT. That means they will be moved 2 times: to primary storage as usual then to alternate storage again to ensure finally they are in alternate storage.
- Pros: - Simple - Do not impact other parts - Do not require to update other processes to perform MOVE_FROM_ALT messages - Cons: - Reduce performance when number of messages is big - Increase file storage counter - If you don't mind building Dovecot from source, give my patch a look.
If you don't mind building Dovecot from source, give my patch a look.
On Tue, 21 Mar 2023 at 05:26, Benoit Branciard < Benoit.Branciard@univ-paris1.fr> wrote:
Hello,
I have some trouble using "doveadm altmove -r".
Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect.
We are using Dovecot 2.3.4 (Debian Buster package).
mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@ @_:ALT=/slow%h/mdbox
doveadm altmove -u myuser since 90d
-> some message files are created under /slow/my/homedir/mdbox/storage
doveadm altmove -r -u myuser all
-> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage.
Is there some known issues with doveadm altmove in this version ? Or am I missing something ?
I can share more config details if needed.
-- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr
On 21/03/2023 00:26 EET Benoit Branciard <benoit.branciard@univ-paris1.fr> wrote:
Hello,
I have some trouble using "doveadm altmove -r".
Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect.
We are using Dovecot 2.3.4 (Debian Buster package).
mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox
doveadm altmove -u myuser since 90d
-> some message files are created under /slow/my/homedir/mdbox/storage
doveadm altmove -r -u myuser all
-> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage.
Is there some known issues with doveadm altmove in this version ? Or am I missing something ?
I can share more config details if needed.
-- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr
Can you try doveadm -D altmove -r -u myuser all and post logs? Aki
Here is the log of a test "doveadm -D altmove -r -u user all" on our setup. But I doubt it contains anything useful besides mailbox and config details. The post from Ewald Dieterich <ewald@mailbox.org> seems on the right track, although I didn't tested the patch. By the way, the "doveadm altmove -r" does not exactly has no effect: it renumbers and equalize (compress) the mail files in alt storage. But the messages that were here stay here. Le 21/03/2023 à 07:35, Aki Tuomi a écrit :
On 21/03/2023 00:26 EET Benoit Branciard <benoit.branciard@univ-paris1.fr> wrote:
Hello,
I have some trouble using "doveadm altmove -r".
Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect.
We are using Dovecot 2.3.4 (Debian Buster package).
mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox
Can you try
doveadm -D altmove -r -u myuser all
and post logs?
Aki
-- Benoit BRANCIARD Université Paris 1 Panthéon-Sorbonne - DSIUN-SIS B405 - Centre PMF - 90 rue de Tolbiac - 75013 Paris Tel. 01 44 07 89 68 http://dsiun.univ-paris1.fr
I had the same problem and solved it by reverting this commit: https://github.com/dovecot/core/commit/2795f6183049a8a4cc489869b3e866dc20a8a... Here's my original mail to this mailing list: https://dovecot.org/pipermail/dovecot/2021-February/121329.html On 20.03.23 23:26, Benoit Branciard wrote:
Hello,
I have some trouble using "doveadm altmove -r".
Regular "doveadm altmove" is OK, selected mails were moved to alternate storage as expected. But I can't manage to get them back to original location, "doveadm altmove -r" has no effect.
We are using Dovecot 2.3.4 (Debian Buster package).
mail_location = mdbox:~/mdbox:DIRNAME=_@@_dbox-Mails_@@_:ALT=/slow%h/mdbox
doveadm altmove -u myuser since 90d
-> some message files are created under /slow/my/homedir/mdbox/storage
doveadm altmove -r -u myuser all
-> message files are still sitting under /slow/my/homedir/mdbox/storage, none were created into /my/homedir/mdbox/storage.
Is there some known issues with doveadm altmove in this version ? Or am I missing something ?
I can share more config details if needed.
participants (4)
-
Aki Tuomi
-
Benoit Branciard
-
Duc Anh Do
-
Ewald Dieterich