Hi,
I am not an professional system admin, but I am running a small debian mail server for many years. It is time to migrate from mbox to Maildir. There is a nice and clean description for doing this without service downtime at https://wiki.dovecot.org/Tools/Doveadm/Sync I have to set per-user config for mailbox type using userdb extra fields.
However, I miss some information and I can not find useful info by google.
The main question: dovecot will know that for some users the mailbox format is still mbox, but others already have Maildir. I do not see any hint that I should setup postfix/procmail for handling the delivery per-user basis. But I think I have to set this, too.
I have not learned much about userdb and it seems that dovecot was written for and used by experts. I mean that the docs/wiki is not enough for me: from the wiki page of userdb it will be for me difficult to setup one. Can you suggest some docs for dummies?
Thanks for your help, Tamas
dovecot --version 2.2.27 (c0f36b0) postfix 3.1.8 procmail v3.23pre 2001/09/13
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
This email has been checked for viruses by AVG. https://www.avg.com
I am also migration away from mbox to in my case mdbox. I am not fully aware of something that allows you to do this without downtime.
To try to do as much as possible online, I have created an archive namespace and am moving the larger >5GB mbox files with old emails to that. So when the time is there to do convert active mailboxes, I have less to convert. Maybe this is helpful.
-----Original Message----- From: Tamas Hegedus [mailto:tamas@hegelab.org] Sent: 29 January 2019 17:39 To: dovecot@dovecot.org Subject: mbox 2 Maildir
Hi,
I am not an professional system admin, but I am running a small debian mail server for many years. It is time to migrate from mbox to Maildir. There is a nice and clean description for doing this without service downtime at https://wiki.dovecot.org/Tools/Doveadm/Sync I have to set per-user config for mailbox type using userdb extra fields.
However, I miss some information and I can not find useful info by google.
The main question: dovecot will know that for some users the mailbox format is still mbox, but others already have Maildir. I do not see any hint that I should setup postfix/procmail for handling the delivery per-user basis. But I think I have to set this, too.
I have not learned much about userdb and it seems that dovecot was written for and used by experts. I mean that the docs/wiki is not enough for me: from the wiki page of userdb it will be for me difficult to setup one. Can you suggest some docs for dummies?
Thanks for your help, Tamas
dovecot --version 2.2.27 (c0f36b0) postfix 3.1.8 procmail v3.23pre 2001/09/13
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
This email has been checked for viruses by AVG. https://www.avg.com
W/o downtime: see the wiki page below, the "Converting" section. Briefly for my case something like this should work and generally seems simple (no syntax checking yet, pseudo-code like):
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush -- doveadm kick (to force clients of USER to use the new Maildir) -- doveadm sync -R mbox:~/mails (rerun to sync the very last few mails, which arrived during the editing and reloading the userdb config)
- when done for all users, I set maildir:~/Maildir for mail location and remove the userdb file definition
The problem of postfix/procmail: procmail; https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP
- I found this, but I do not prefer edit procmail config this way http://dmc1961.id.au/howto_17101101.php
- I think that dovecot/lmpt is a better solution: -- dovecot takes care of the delivery and it will also know where to deliver during the mixed situation -- on the long term: told to have better performance compared to
On 2019. 01. 29. 19:49, Marc Roos wrote:
I am also migration away from mbox to in my case mdbox. I am not fully aware of something that allows you to do this without downtime.
To try to do as much as possible online, I have created an archive namespace and am moving the larger >5GB mbox files with old emails to that. So when the time is there to do convert active mailboxes, I have less to convert. Maybe this is helpful.
-----Original Message----- From: Tamas Hegedus [mailto:tamas@hegelab.org] Sent: 29 January 2019 17:39 To: dovecot@dovecot.org Subject: mbox 2 Maildir
Hi,
I am not an professional system admin, but I am running a small debian mail server for many years. It is time to migrate from mbox to Maildir. There is a nice and clean description for doing this without service downtime at https://wiki.dovecot.org/Tools/Doveadm/Sync I have to set per-user config for mailbox type using userdb extra fields.
However, I miss some information and I can not find useful info by google.
The main question: dovecot will know that for some users the mailbox format is still mbox, but others already have Maildir. I do not see any hint that I should setup postfix/procmail for handling the delivery per-user basis. But I think I have to set this, too.
I have not learned much about userdb and it seems that dovecot was written for and used by experts. I mean that the docs/wiki is not enough for me: from the wiki page of userdb it will be for me difficult to setup one. Can you suggest some docs for dummies?
Thanks for your help, Tamas
dovecot --version 2.2.27 (c0f36b0) postfix 3.1.8 procmail v3.23pre 2001/09/13
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
This email has been checked for viruses by AVG. https://www.avg.com
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
Hi,
Tamas Hegedus, 29.01.19:
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush
IMHO there is no need to create or change special userdb entries. See https://wiki.dovecot.org/MailLocation: "By default the mail_location setting is empty, which means that Dovecot attempts to locate automatically where your mails are."
Regards, Christian
-- No signature available.
Tamas Hegedus, 29.01.19:
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time [1] -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; [2] the conditionally optional: doveadm auth cache flush
More or less how I would plan a migration, but to be fanatically safe and avoiding race conditions where a user could be manipulating their mailbox or new having mail delivered, I would put some service mutex between [1] and [2].
By that I mean at [1]
a) tempfail any incoming mail for that user; and
b) kick the user off IMAP/POP and make sure they can't log back
in during the final synching. (Add them to separate deny userdb?)
After the final sync [2], undo these steps.
Joseph Tam <jtam.home@gmail.com>
Hi,
I am experimenting with the migration. I unset mail_location, thus it is empty and dovecot performs automatic mailbox detection. Therefore I do not need per-user set of mail location. I login as a testuser and do: doveadm sync maildir:~/Maildir
It seems for me that in this situation the new Maildir becomes the active mail delivery location at the time of the creation(?) or completing the sync(?).
So if I want to rerun to avoid loosing new messages: doveadm sync maildir:~/Maildir I got an error that I can not sync to itself.
I guess this would not be a problem and would be rather simply. However, if I issue a revers sync to be sure not loosing new messages: doveadm sync -R mail/ doveadm(hegedus): Fatal: execvp(mail/) failed: Permission denied dsync-local(hegedus): Error: read(remote) failed: EOF (version not received)
(I tried to set a+x on my old mbox 'mail' directory and its contents)
I would appreciate any suggestion. Thanks, Tamas
On 01/30/2019 03:32 PM, Christian Schmidt wrote:
Hi,
Tamas Hegedus, 29.01.19:
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush
IMHO there is no need to create or change special userdb entries. See https://wiki.dovecot.org/MailLocation: "By default the mail_location setting is empty, which means that Dovecot attempts to locate automatically where your mails are."
Regards, Christian
-- Tamas Hegedus, PhD Senior Research Fellow Department of Biophysics and Radiation Biology Semmelweis University | phone: (36) 1-459 1500/60233 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
Sorry. I had wrong syntax. Missing "mbox:" It is OK with this: doveadm sync -RD mbox:~/mail/
On 02/05/2019 10:48 AM, Tamas Hegedus wrote:
Hi,
I am experimenting with the migration. I unset mail_location, thus it is empty and dovecot performs automatic mailbox detection. Therefore I do not need per-user set of mail location. I login as a testuser and do: doveadm sync maildir:~/Maildir
It seems for me that in this situation the new Maildir becomes the active mail delivery location at the time of the creation(?) or completing the sync(?).
So if I want to rerun to avoid loosing new messages: doveadm sync maildir:~/Maildir I got an error that I can not sync to itself.
I guess this would not be a problem and would be rather simply. However, if I issue a revers sync to be sure not loosing new messages: doveadm sync -R mail/ doveadm(hegedus): Fatal: execvp(mail/) failed: Permission denied dsync-local(hegedus): Error: read(remote) failed: EOF (version not received)
(I tried to set a+x on my old mbox 'mail' directory and its contents)
I would appreciate any suggestion. Thanks, Tamas
On 01/30/2019 03:32 PM, Christian Schmidt wrote:
Hi,
Tamas Hegedus, 29.01.19:
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush
IMHO there is no need to create or change special userdb entries. See https://wiki.dovecot.org/MailLocation: "By default the mail_location setting is empty, which means that Dovecot attempts to locate automatically where your mails are."
Regards, Christian
-- Tamas Hegedus, PhD Senior Research Fellow Department of Biophysics and Radiation Biology Semmelweis University | phone: (36) 1-459 1500/60233 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
Hi,
I have some strange things and at the end I am not sure if the migration is (will be) OK. I have a testuser for testing the mbox 2 Maildir migration. It did not received new email during the following steps:
sudo -u testuser doveadm sync maildir:/home/testuser/Maildir dsync(testuser): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID 4ca2fa0b9bb536564a470000e1afe915 (maybe it was just deleted?)
doveadm kick testuser
#sudo -u testuser doveadm sync maildir:/home/testuser/Maildir ##dsync(testuser): Error: Both source and destination mail_location points to same directory: /home/testuser/Maildir
sudo -u testuser doveadm sync -R mbox:/home/testuser/mail dsync(testuser): Error: Can't sync mailbox Public folders: Mailbox isn't selectable
If I check the folders and mails in Thunderbird, everything seems OK ('Public folders' seems to be synced, too). However, du -hsc ./* 320M ./mail 157M ./Maildir
I did not find anything that seem to be useful in order to get at a conclusion.
Thanks for your help and suggestions, Tamas
On 2/5/19 10:48 AM, Tamas Hegedus wrote:
Hi,
I am experimenting with the migration. I unset mail_location, thus it is empty and dovecot performs automatic mailbox detection. Therefore I do not need per-user set of mail location. I login as a testuser and do: doveadm sync maildir:~/Maildir
It seems for me that in this situation the new Maildir becomes the active mail delivery location at the time of the creation(?) or completing the sync(?).
So if I want to rerun to avoid loosing new messages: doveadm sync maildir:~/Maildir I got an error that I can not sync to itself.
I guess this would not be a problem and would be rather simply. However, if I issue a revers sync to be sure not loosing new messages: doveadm sync -R mail/ doveadm(hegedus): Fatal: execvp(mail/) failed: Permission denied dsync-local(hegedus): Error: read(remote) failed: EOF (version not received)
(I tried to set a+x on my old mbox 'mail' directory and its contents)
I would appreciate any suggestion. Thanks, Tamas
On 01/30/2019 03:32 PM, Christian Schmidt wrote:
Hi,
Tamas Hegedus, 29.01.19:
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush
IMHO there is no need to create or change special userdb entries. See https://wiki.dovecot.org/MailLocation: "By default the mail_location setting is empty, which means that Dovecot attempts to locate automatically where your mails are."
Regards, Christian
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
Dear Marc, Thanks for pointing this small but very important detail - it simplifies the process so much! Tamas
On 01/29/2019 09:22 PM, Tamas Hegedus wrote:
W/o downtime: see the wiki page below, the "Converting" section. Briefly for my case something like this should work and generally seems simple (no syntax checking yet, pseudo-code like):
- Configuration uses mail_location = mbox:~/mails
- setup per-user mail location and do for each user individually in a serial manner: -- doveadm sync maildir:~/Maildir; mbox is synced to Maildir, long running time -- doveadm sync maildir:~/Maildir; rerun to do it for new messages (fast) -- add USER to userdb and set extra field to maildir:~/Maildir users not present in the userdb should default to dovcot default; conditionally optional: doveadm auth cache flush -- doveadm kick (to force clients of USER to use the new Maildir) -- doveadm sync -R mbox:~/mails (rerun to sync the very last few mails, which arrived during the editing and reloading the userdb config)
- when done for all users, I set maildir:~/Maildir for mail location and remove the userdb file definition
The problem of postfix/procmail: procmail; https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP
- I found this, but I do not prefer edit procmail config this way http://dmc1961.id.au/howto_17101101.php
- I think that dovecot/lmpt is a better solution: -- dovecot takes care of the delivery and it will also know where to deliver during the mixed situation -- on the long term: told to have better performance compared to
On 2019. 01. 29. 19:49, Marc Roos wrote:
I am also migration away from mbox to in my case mdbox. I am not fully aware of something that allows you to do this without downtime.
To try to do as much as possible online, I have created an archive namespace and am moving the larger >5GB mbox files with old emails to that. So when the time is there to do convert active mailboxes, I have less to convert. Maybe this is helpful.
-----Original Message----- From: Tamas Hegedus [mailto:tamas@hegelab.org] Sent: 29 January 2019 17:39 To: dovecot@dovecot.org Subject: mbox 2 Maildir
Hi,
I am not an professional system admin, but I am running a small debian mail server for many years. It is time to migrate from mbox to Maildir. There is a nice and clean description for doing this without service downtime at https://wiki.dovecot.org/Tools/Doveadm/Sync I have to set per-user config for mailbox type using userdb extra fields.
However, I miss some information and I can not find useful info by google.
The main question: dovecot will know that for some users the mailbox format is still mbox, but others already have Maildir. I do not see any hint that I should setup postfix/procmail for handling the delivery per-user basis. But I think I have to set this, too.
I have not learned much about userdb and it seems that dovecot was written for and used by experts. I mean that the docs/wiki is not enough for me: from the wiki page of userdb it will be for me difficult to setup one. Can you suggest some docs for dummies?
Thanks for your help, Tamas
dovecot --version 2.2.27 (c0f36b0) postfix 3.1.8 procmail v3.23pre 2001/09/13
-- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
This email has been checked for viruses by AVG. https://www.avg.com
-- Tamas Hegedus, PhD Senior Research Fellow Department of Biophysics and Radiation Biology Semmelweis University | phone: (36) 1-459 1500/60233 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org
participants (4)
-
Christian Schmidt
-
Joseph Tam
-
Marc Roos
-
Tamas Hegedus