Re: [Dovecot] integration of old mails into new dovecot Maildir
Thank you for your prompt reply and timely help. I will try to be very explanative in describing my situation. I will begin first with answering the questions you asked;
backend do you use, maybe Maildir?how are the messages storred on the old server, which storage
Yes, Maildir. Dovecot version is 2.1.7.
Virtual users are authenticated by mysql backend. Login is plain login.
the storage backend on the new server?
Maildir. Dovecot Version 2.2.9. Mysql virtual usersauthentication. Plain login
do you care about the message UIDs?
I Would like to preserve the UIDs as much as possible. However, the most important thing is for the users to have access to their old emails.
- How many users do you have (approx.) and can you access the user's mailboxes via IMAP, e.g. using a master user or do you know their password or can you switch to a known password during migration of one particular user?
I have about 50 total users. I can access their mailboxes via imap but I don't have their password handy. Mysql database with the userdata and passwords was imported into the new new server so, the users can get authenticated into the new server with the old server password.
Did you've changed namespaces?
No. I did not change namespaces.
- From your first post I figure that the basic mail server functionality of the new server is working, most importantly login, message delivery and retrieval.
Yes, the new server is functional and all the users are able to log in, view delivered messages and retrieve the messages into their mail clients. However, they cannot view any old messages or retrieve any old messages.
Here is the dovecot.conf of the old mail server;
protocols = imap pop3
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_location = maildir:/var/vmail/%d/%n
mail_privileged_group = mail
first_valid_uid = 6060
ssl_cert=
ssl_key =< /etc/ssl/private/dovecot.pem
#ssl_ca =
ssl = yes
#ssl_cert =
#ssl_key =
protocol imap {
mail_plugins = autocreate quota imap_quota
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
mail_plugins = quota
}
protocol lda {
mail_plugins = quota sieve
quota_full_tempfail = yes
rejection_reason = Your message to <%t> was automatically rejected:%n%r%n%nThe Subject of your message was: %s
auth_socket_path = /var/run/dovecot/auth-master
log_path = /var/log/dovecot-deliver.log
postmaster_address = postmaster@sbm1.motherindia.com
# sieve_path = /var/sieve/global.sieve
}
auth_mechanisms = plain login cram-md5
auth_debug = no
auth_debug_passwords = yes
auth_verbose = no
auth_verbose_passwords = no
disable_plaintext_auth = no
instance_name = dovecot
login_greeting = Dovecot ready.
mail_debug = no
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
-----Original Message----- From: Steffen Kaiser [mailto:skdovecot@smail.inf.fh-brs.de] Sent: Monday, April 14, 2014 2:18 AM To: K. C. Mohan Cc: dovecot@dovecot.org Subject: Re: [Dovecot] integration of old mails into new dovecot Maildir
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Sun, 13 Apr 2014, K. C. Mohan wrote:
Please, do not top post.
Thank you Steffen, for addressing my problem of migrating old mails
into new mail server. I went through the article you pointed to but
could not succueed. I will appreciate if you could guide me to step by
step process to migrate the mails. My old mail server used 2.x dovecot version.
you post very little information:
how are the messages storred on the old server, which storage backend do you use, maybe Maildir?
the storage backend on the new server?
do you care about the message UIDs?
How many users do you have (approx.) and can you access the user's mailboxes via IMAP, e.g. using a master user or do you know their password or can you switch to a known password during migration of one particular user?
Did you've changed namespaces?
- From your first post I figure that the basic mail server functionality of the new server is working, most importantly login, message delivery and retrieval.
I have attached both the old and new servers configuration files as .txt attachments.
-----Original Message----- From: Steffen Kaiser
Sent: Thursday, April 10, 2014 6:14 AM
To: K. C. Mohan - Personal s
Cc: mailto:dovecot@dovecot.org dovecot@dovecot.org
Subject: Re: [Dovecot] integration of old mails into new dovecot
Maildir
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Tue, 8 Apr 2014, K. C. Mohan - Personal wrote:
I just installed dovecot 2.2.9 on to a new server and want to
transfer all the old mails into the email accounts I imported from
the previous server into MySQL database. I am not able to view the
subfolders and or old mails that I copied into their respective
maildir folders. I was very much appreciate if anyone could let me
know how to import old mails into the new server. All I see, is the
new mails that have come into the account since I started dovecot service in the new server.
Check out http://wiki2.dovecot.org/Migration http://wiki2.dovecot.org/Migration . Read esp. about
"preserving messages' UIDs".
If the old server uses Maildir, too, and you don't care about the
UIDs, copy the files over and you are done.
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBU0t9r3D1/YhP6VMHAQL21wgAx1GgDwQDWwDMPxbMQ+1vR3i7dsyUk5jm
PNmRigoV8qylbn2AuZ1YMxGN2wEZfSl2zi/j2CtSEUX+d9LOf+lvI9V8zXkvSHT6
jYEHwuIQACjPUxWmh7x8XelvQvZHB9PEH56z8eJhkgyVxpae8Rbb1EHurvmxckef
O0fdmCIglXJdJJ+B9WZ7JZVsB3M6PFFsllizS17+Hus6O2NeVaszLa7o7bk5XInD
m+Co4e4sCVphVlssHRCIRa8Lj6nDC+D6g3RjCR+n0rVREXRbvEcSQPerEAqrZhf8
HB46x3V3n6C2fEu7RNWLB0+DzglHLBI5p9ZqAjJRMr3D9Xy9L5SKlQ==
=IH25
-----END PGP SIGNATURE-----
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com http://www.avast.com
This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
Yes, the new server is functional and all the users are able to log in, view delivered messages and retrieve the messages into their mail clients. However, they cannot view any old messages or retrieve any old messages.
If I wanted to do this, I'd probably use something along these lines:
- get IMAP access to your users' mailboxes. No need to know their passwords, you can set up master user for that, see wiki:
http://wiki2.dovecot.org/Authentication/MasterUsers
(this needs to be set up on the old server)
- run this command on new server. (Let's say your master user is called "admin" and you set auth_master_user_separator = * )
doveadm -o plugin/sieve_dir=
-o imapc_user=user@example.org*admin
-o imapc_password=adminspassword
-o imapc_host=oldbox.example.org
-o imapc_ssl=starttls
-o imapc_ssl_verify=yes
-o ssl_client_ca_dir=/etc/ssl/
-o mail_fsync=never
import -u user@example.org imapc: "" mailbox inbox
(Rinse and repeat for every user, the only thing that changes between runs is username in imapc_user=xxx and -u xxx)
This command uses Dovecot's builtin IMAP client to connect to your old server, downloads messages and stores them into specified user's mailbox on new server. New mail should be preserved in the process, so should messages' UIDs.
It's pretty much the same as described here - http://wiki2.dovecot.org/Migration/Dsync - you just use import instead of dsync (from my experience dsync doesn't do what you need in this scenario.)
The -o plugin/sieve_dir= line prevents Dovecot from trying to import Sieve scripts (which doesn't work for me, they're not writable in my setup and doveadm segfaults.) If you don't use sieve plugin or you can import Sieve scripts this way, you can omit that.
Rest of parameters is pretty straightforward, see man doveadm-import for explanations. You can use "ALL" instead of "mailbox inbox" to import messages from all IMAP folders (add -s parameter in that case)
And of course - this is pretty much untested (tried on one mailbox), so use on your own risk.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 14 Apr 2014, K. C. Mohan - Personal wrote:
how are the messages storred on the old server, which storageYes, Maildir. Dovecot version is 2.1.7.
the storage backend on the new server?
Maildir. Dovecot Version 2.2.9. Mysql virtual usersauthentication. Plain login
As both are Maildir from Dovecot, you can copy all the Maildir hierarchie 1:1 to the new server. Dovecot upgrades them automatically most of the time. I did so from Dovecot v1.x to v2.2 with almost no problem.
To keep the UIDs for POP3, ensure the same:
pop3_uidl_format = %08Xu%08Xv
is used on the new server.
If you already got new messages on the new server, you can think about:
a) migration, e.g. as Jiri pointed out
b) merge both Maildirs, but remove all "new" control files dovecot*, keep only the old ones. That way the new messages will be added to the old store by getting new UIDs.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBU0zj8XD1/YhP6VMHAQKaXAgAqAXaBTDNvDWh3Yl56UrA5InEJ+E5UfiX l9qwWTDrjneGxN6CHNmA8+VRACcZDa9Ha2oEnbfjE4ZosHK29cGzr846bwGKHv0T Jmo2BMs/yz0psSXXf4p/46rXckQaK5HAvJTozdoRHK84OGjbTfjI2QCE/xdnrWXb vH/wsFw5W7/hi+sSAXsFBmGkzm4Opig3V+aSQFSswlKgzUAB2H3nHlz18XObNPTK GGjB2YV6OM0x8F87E/wgapxv4jHd48kArvDWabv9FygDQziVSggKFeQnrjgGuVZM wlL7m1YgC1x0fGXPnYpO0gg5o0wD9X6nCStmeKKigmGwhRTvAgWc/w== =QKGI -----END PGP SIGNATURE-----
participants (3)
-
Jiri Bourek
-
K. C. Mohan - Personal
-
Steffen Kaiser