On 2012-06-24 11:57 AM, Reinhard Vicinus <r.vicinus@metaways.de> wrote:

...

i try to migrate mails from a non dovecot imap server to a dovecot imap server with doveadm backup as described there:

What version of dovecot (doveconf -n output_? dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = xxx instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.10.10.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes

On 25/06/12 13:39, Charles Marcus wrote: passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve }

On Wed, 2012-06-27 at 15:10 +0200, Reinhard Vicinus wrote:

Hi,

if i delete the home directory and all content below an existing account user@example.org. Then run:

/usr/bin/doveadm quota recalc -u user@example.org

Are you sure quota recalc makes a difference here? What if you simply run doveadm twice?

and afterwards:

/usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -D -v backup -R -f -u user@example.org imapc:

dsync(user@example.org): Error: Mailbox INBOX changed its GUID (c92f64f79f0d1ed01e6d5b314f04886c -> 54c23c119d04eb4f005100004f99b03d) dsync(user@example.org): Error: msg iteration failed: Couldn't open mailbox c92f64f79f0d1ed01e6d5b314f04886c

Bug/"feature" .. you could try if running with "imapc:/tmp/imapc-username" instead of "imapc:" helps.

On 28/06/12 09:03, Reinhard Vicinus wrote:

...

...

and afterwards:

/usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -D -v backup -R -f -u user@example.org imapc:

dsync(user@example.org): Error: Mailbox INBOX changed its GUID (c92f64f79f0d1ed01e6d5b314f04886c -> 54c23c119d04eb4f005100004f99b03d) dsync(user@example.org): Error: msg iteration failed: Couldn't open mailbox c92f64f79f0d1ed01e6d5b314f04886c Bug/"feature" .. you could try if running with "imapc:/tmp/imapc-username" instead of "imapc:" helps. This works also without problems. So thanks for your help because this solves my problem. Let me know if i should test something more.

Sorry, I either made a mistake in my test setup or i can't reproduce it, but adding imapc:/tmp/imapc-username instead of imapc: doesn't help. I have circumvented my problem by changing the quota values directly in the database in my migration process.

But there is the following difference between using imapc:/tmp/imapc-username and plain imapc: if i backup a single, on both servers empty mailbox with different guids from the non dovecot imap server to the dovecot imap server, then plain imapc: throws some errors but works, imapc:/tmp/imapc-username throws more errors and only deletes the mailbox on the destination.

Test setup is as follow:

Both accounts don't contain a mailbox Test1: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox status -u user@example.org all Test1 <empty reply> /usr/bin/doveadm mailbox status -u user@example.org all Test1 <empty reply>

Create Mailbox Test1 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox create -u user@example.org Test1 <empty reply>

Create Mailbox Test1 on the dovecot server: doveadm mailbox create -u user@example.org Test1 <empty reply>

List the status of mailbox Test1 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox status -u user@example.org all Test1 Test1 messages=0 recent=0 uidnext=0 uidvalidity=87991 unseen=0 highestmodseq=0 vsize=0 guid=0f6e69ad71659995677b43f8a8312025

List the status of mailbox Test1 on the dovecot server: /usr/bin/doveadm mailbox status -u user@example.org Test1 Test1 messages=0 recent=0 uidnext=1 uidvalidity=1340879819 unseen=0 highestmodseq=1 vsize=0 guid=a8076214cb33ec4f396700004f99b03d

Start Backup with imapc:/tmp/user: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 backup -R -f -u user@example.org -m Test1 imapc:/tmp/user dsync(user@example.org): Error: Failed to sync mailbox Test1: Mailbox doesn't exist: Test1 dsync(user@example.org): Error: msg iteration failed: Couldn't open mailbox 0f6e69ad71659995677b43f8a8312025 dsync(user@example.org): Error: Failed to sync mailbox Test1: Mailbox doesn't exist: Test1 dsync(user@example.org): Error: Trying to open a non-listed mailbox with guid=a8076214cb33ec4f396700004f99b03d dsync(user@example.org): Error: msg iteration failed: Couldn't open mailbox a8076214cb33ec4f396700004f99b03d dsync(user@example.org): Error: Trying to open a non-listed mailbox with guid=a8076214cb33ec4f396700004f99b03d

List the status of mailbox Test1 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox status -u user@example.org all Test1 Test1 messages=0 recent=0 uidnext=0 uidvalidity=87991 unseen=0 highestmodseq=0 vsize=0 guid=0f6e69ad71659995677b43f8a8312025

List the status of mailbox Test1 on the dovecot server: /usr/bin/doveadm mailbox status -u user@example.org all Test1 <empty reply>

result: the mailbox Test1 on the dovecot server got deleted.

with plain imapc: copying works but there are also still error messages:

Create Mailbox Test2 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox create -u user@example.org Test2 <empty reply>

Create Mailbox Test2 on the dovecot server: doveadm mailbox create -u user@example.org Test2 <empty reply>

List the status of mailbox Test2 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox status -u user@example.org all Test2 Test2 messages=0 recent=0 uidnext=0 uidvalidity=87993 unseen=0 highestmodseq=0 vsize=0 guid=c0fd4ba8bd514c5c43ab9a897c8c014e

List the status of mailbox Test2 on the dovecot server: /usr/bin/doveadm mailbox status -u user@example.org Test2 Test2 messages=0 recent=0 uidnext=1 uidvalidity=1340879820 unseen=0 highestmodseq=1 vsize=0 guid=a19eee292435ec4f676a00004f99b03d

Start Backup with imapc: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 backup -R -f -u user@example.org -m Test2 imapc: dsync(user@example.org): Error: Trying to open a non-listed mailbox with guid=a19eee292435ec4f676a00004f99b03d dsync(user@example.org): Error: msg iteration failed: Couldn't open mailbox a19eee292435ec4f676a00004f99b03d dsync(user@example.org): Error: Trying to open a non-listed mailbox with guid=a19eee292435ec4f676a00004f99b03d

List the status of mailbox Test2 on the imapc server: /usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mail=imapc: mailbox status -u user@example.org all Test2 Test2 messages=0 recent=0 uidnext=0 uidvalidity=87993 unseen=0 highestmodseq=0 vsize=0 guid=c0fd4ba8bd514c5c43ab9a897c8c014e

List the status of mailbox Test2 on the dovecot server: /usr/bin/doveadm mailbox status -u user@example.org Test2 Test2 messages=0 recent=0 uidnext=1 uidvalidity=87993 unseen=0 highestmodseq=1 vsize=0 guid=c0fd4ba8bd514c5c43ab9a897c8c014e

If instead of a normal Mailbox the special mailbox INBOX is used there are still more errors and both ways don't work. I think because backup isn't able to delete the mailbox INBOX on the destination site.

So i'll make sure that when i migrate an account the mail destination is really empty.

On 25/06/12 18:46, Timo Sirainen wrote:

On 24.6.2012, at 18.57, Reinhard Vicinus wrote:

...

i try to migrate mails from a non dovecot imap server to a dovecot imap server with doveadm backup as described there:

http://wiki2.dovecot.org/Migration/Dsync

i first tried (local-mailbox port 18143 is the non dovecot imap server):

/usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mailbox_list_index=no -v -D backup -R -f -u user@example.org -m Sent imapc:

and got the following error:

dsync(user@example.org): Fatal: dsync backup: Looks like you're trying to run backup in wrong direction. Source is empty and destination is not. Strange. -R is supposed to make it copy from imapc to mdbox.. Have you tried if Dovecot can see mails at all from the remote server? Try doveadm -o mail=imapc: -o ... fetch instead of doveadm backup command. You're right it was an error in my setup that caused this problem. After fixing that problem it now works as expected. The only thing I don't get working is running it via the doveadm-server socket with:

/usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mailbox_list_index=no -v -D backup -S /var/run/dovecot-director/doveadm-server -R -u user@example.org imapc:

In the logfile on the server there is the following error message:

Jun 25 20:01:26 10.129.3.200 dovecot: dsync(user@example.org): Error: user user@example.org: Initialization failed: Initializing mail storage from mail_location setting failed: imapc: missing imapc_host Jun 25 20:01:26 10.129.3.200 dovecot: dsync(user@example.org): Fatal: User init failed

So I think that all the -o configurations aren't transfered via the doveadm-server socket.

...

As the dovecot imap account is newly created and therefore empty it seams to try to backup from the dovecot imap server to the non dovecot imap server. So i tried instead:

/usr/bin/doveadm -o imapc_user=user@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mailbox_list_index=no -v -D backup -f -u user@example.org -m Sent imapc: Now this is copying from mdbox to imapc, which is also why you're getting the crash:

...

Sometimes (every other time?) i got the following segmentation fault:

bt #0 0x00007f15e2c9ed74 in strcasecmp () from /lib/libc.so.6 #1 0x00007f15e327eaff in imapc_save_callback (reply=0x7fff56096a70, context=<value optimized out>) at imapc-save.c:168 Note how it's saving a mail to imapc. But still, that's a bug, fixed: http://hg.dovecot.org/dovecot-2.1/rev/20703dbd1168

...

dsync(user@example.org): Warning: Destination mailbox Sent has been modified, need to recreate it before we can continue syncing I think this is also because it's going to wrong direction. Yes, the problem there was that it was the wrong direction.