[Dovecot] converting from mbox to maildir ?
Hello
We are planning to convert our mailhub ( freebsd 7.4 ) from mbox format to maildir format.
I've read the documentation and performed some tests on another machine it is a bit long ...
I would like some feedback from guys who did this operation and need some advice on what to convert first ?
- first convert INBOX then convert IMAP folders ?
- first convert IMAP folders then convert INBOX ?
the machine use real users thru openldap ( pam_ldap + nss_ldap )
another problem is disk space. The users's email data takes about 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on the raid array of the server.
My idea is to use one of our NFS netapp filer during the convertion to throw the result of the convertion on an NFS mounted directory.
Anyone did this before ? If yes I would be greatly interrested by their experience
Thank you
On 30.1.2012, at 10.37, Frank Bonnet wrote:
We are planning to convert our mailhub ( freebsd 7.4 ) from mbox format to maildir format.
I'd do it one user at a time with dsync, but unfortunately:
the machine use real users thru openldap ( pam_ldap + nss_ldap )
nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp.
another problem is disk space. The users's email data takes about 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on the raid array of the server.
With one-user-at-a-time conversion this wouldn't be an issue.
Le 02/01/2012 09:39 PM, Timo Sirainen a écrit :
On 30.1.2012, at 10.37, Frank Bonnet wrote:
We are planning to convert our mailhub ( freebsd 7.4 ) from mbox format to maildir format. I'd do it one user at a time with dsync, but unfortunately:
the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp.
another problem is disk space. The users's email data takes about 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue.
so is there a massive conversion method ?
thanks
On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote:
Le 02/01/2012 09:39 PM, Timo Sirainen a écrit :
On 30.1.2012, at 10.37, Frank Bonnet wrote:
We are planning to convert our mailhub ( freebsd 7.4 ) from mbox format to maildir format. I'd do it one user at a time with dsync, but unfortunately:
the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp.
another problem is disk space. The users's email data takes about 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue.
so is there a massive conversion method ?
dsync works for that too:
- dsync backup all users somewhere
- wait close to the time you can shut down dovecot
- dsync backup all users the second time
- shutdown dovecot and MTA
- dsync backup once more (hopefully this will be relatively quick)
The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space..
Le 02/02/2012 10:20 AM, Timo Sirainen a écrit :
On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote:
Le 02/01/2012 09:39 PM, Timo Sirainen a écrit :
On 30.1.2012, at 10.37, Frank Bonnet wrote:
We are planning to convert our mailhub ( freebsd 7.4 ) from mbox format to maildir format. I'd do it one user at a time with dsync, but unfortunately:
the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp.
another problem is disk space. The users's email data takes about 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue.
so is there a massive conversion method ? dsync works for that too:
- dsync backup all users somewhere
- wait close to the time you can shut down dovecot
- dsync backup all users the second time
- shutdown dovecot and MTA
- dsync backup once more (hopefully this will be relatively quick)
The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space..
Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ?
see below the output of doveconf -n
thanks
mail# doveconf -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.4-RELEASE amd64 auth_verbose_passwords = plain auth_worker_max_count = 100 debug_log_path = /var/log/dovecot/debug.log default_client_limit = 6144 default_process_limit = 1024 default_vsz_limit = 1 G disable_plaintext_auth = no info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/dovecot.log mail_location = mbox:%h/:INDEX=/raid/cache/dovecot/index/%u:INBOX=/raid/var/mail/%u:CONTROL=%h/ mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_lock_timeout = 3 mins mbox_write_locks = fcntl mmap_disable = yes passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster@esiee.fr protocols = imap pop3 sendmail_path = /usr/local/sbin/sendmail service auth { client_limit = 7169 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 service_count = 1 vsz_limit = 3 G } service imap { process_limit = 2048 vsz_limit = 3 G } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/xxx.pem ssl_cert =
On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote:
dsync works for that too:
- dsync backup all users somewhere
- wait close to the time you can shut down dovecot
- dsync backup all users the second time
- shutdown dovecot and MTA
- dsync backup once more (hopefully this will be relatively quick)
The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space..
Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ?
No need for any modifications. As long as you don't have multiple Dovecots in different servers accessing the same mails via NFS, there's no trouble.
mmap_disable = yes
Looks like you also have this already enabled. This is recommended (not required) with NFS, but if you're not normally using NFS at all you shouldn't enable this.
Le 02/02/2012 11:10 AM, Timo Sirainen a écrit :
On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote:
dsync works for that too:
- dsync backup all users somewhere
- wait close to the time you can shut down dovecot
- dsync backup all users the second time
- shutdown dovecot and MTA
- dsync backup once more (hopefully this will be relatively quick)
The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space..
Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ? No need for any modifications. As long as you don't have multiple Dovecots in different servers accessing the same mails via NFS, there's no trouble.
mmap_disable = yes Looks like you also have this already enabled. This is recommended (not required) with NFS, but if you're not normally using NFS at all you shouldn't enable this.
OK thank you
participants (2)
-
Frank Bonnet
-
Timo Sirainen