[Dovecot] dsync Migration Couldn't keep all uids dove 2.1.16
Hi Timo, using
http://wiki2.dovecot.org/Migration/Dsync
for migration
i am getting this in periods
dsync(sys4-test-2@example.de): Info: INBOX: Couldn't keep all uids dsync(sys4-test-2@example.de): Warning: Mailbox changes caused a desync. You may want to run dsync again. dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 15) dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 10) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316)
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 22.05.2013 10:32, schrieb Robert Schetterer:
Hi Timo, using
http://wiki2.dovecot.org/Migration/Dsync
for migration
i am getting this in periods
dsync(sys4-test-2@example.de): Info: INBOX: Couldn't keep all uids dsync(sys4-test-2@example.de): Warning: Mailbox changes caused a desync. You may want to run dsync again. dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 15) dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 10) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316)
Best Regards MfG Robert Schetterer
sorry for noise , i did more testing, guess the general problem is in missing read
Problems
POP3 message order (when it's different from IMAP message order) iscurrently preserved only when destination is Maildir
for small test procedure works with maildir
anyone testet allready , if do a migration to maildir, and afterwards convert mailboxes to mdbox avoiding redownload for pop3 is working ( i.e for pop3 user that left mail on the server )?, my guess its not working by design of mdbox
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 22.05.2013 13:36, schrieb Robert Schetterer:
Am 22.05.2013 10:32, schrieb Robert Schetterer:
Hi Timo, using
http://wiki2.dovecot.org/Migration/Dsync
for migration
i am getting this in periods
dsync(sys4-test-2@example.de): Info: INBOX: Couldn't keep all uids dsync(sys4-test-2@example.de): Warning: Mailbox changes caused a desync. You may want to run dsync again. dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 15) dsync(sys4-test-2@example.de): Warning: I/O leak: 0x756547164490 (line 1229, fd 10) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316) dsync(sys4-test-2@example.de): Warning: Timeout leak: 0x7565471647a0 (line 1316)
Best Regards MfG Robert Schetterer
sorry for noise , i did more testing, guess the general problem is in missing read
Problems
POP3 message order (when it's different from IMAP message order) iscurrently preserved only when destination is Maildir
for small test procedure works with maildir
anyone testet allready , if do a migration to maildir, and afterwards convert mailboxes to mdbox avoiding redownload for pop3 is working ( i.e for pop3 user that left mail on the server )?, my guess its not working by design of mdbox
Best Regards MfG Robert Schetterer
Hi Timo,
more tests showed up, that expunged mails arent synced anyway whatever mailbox format is used
perhaps that problem ?
If source POP3 server merges multiple IMAP mailboxes into one POP3 INBOX, the migration won't be transparent. If source IMAP and POP3 servers return messages somehow differently, pop3-migration plugin might not be able to match the messages
how test this on source server i dont see something special there using telnet and/or Thunderbird
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 22.5.2013, at 15.56, Robert Schetterer rs@sys4.de wrote:
Am 22.05.2013 13:36, schrieb Robert Schetterer:
Am 22.05.2013 10:32, schrieb Robert Schetterer:
Hi Timo, using
http://wiki2.dovecot.org/Migration/Dsync
for migration
i am getting this in periods
dsync(sys4-test-2@example.de): Info: INBOX: Couldn't keep all uids
Did the destination server maybe already get some new mails?
dsync(sys4-test-2@example.de): Warning: Mailbox changes caused a desync. You may want to run dsync again.
Does this help?
sorry for noise , i did more testing, guess the general problem is in missing read
Problems
POP3 message order (when it's different from IMAP message order) is currently preserved only when destination is Maildir
This doesn't break dsync in any way, it just means that POP3 clients will see the UIDL list in different order then they used to, which will probably cause them to redownload mails. I already implemented the ordering support for dbox, but the ordering then also needs to be added to dovecot.index.cache, and that's causing some problems, so I haven't committed the change yet..
more tests showed up, that expunged mails arent synced anyway whatever mailbox format is used
perhaps that problem ?
The source is IMAP+POP3 server? Yeah, could be that it's not handling expunges when doing dsync the second time. v2.2 should be handling them better I think, although I haven't really tested much yet.
If source POP3 server merges multiple IMAP mailboxes into one POP3 INBOX, the migration won't be transparent. If source IMAP and POP3 servers return messages somehow differently, pop3-migration plugin might not be able to match the messages
how test this on source server i dont see something special there using telnet and/or Thunderbird
That is about if the same messages exist in POP3 session and IMAP INBOX. Just checking if their counts are equivalent is pretty much enough alone. In any case even if that happened, it doesn't cause your dsync problems.
Am 22.05.2013 19:03, schrieb Timo Sirainen:
On 22.5.2013, at 15.56, Robert Schetterer rs@sys4.de wrote:
Am 22.05.2013 13:36, schrieb Robert Schetterer:
Am 22.05.2013 10:32, schrieb Robert Schetterer:
Hi Timo, using
http://wiki2.dovecot.org/Migration/Dsync
for migration
i am getting this in periods
dsync(sys4-test-2@example.de): Info: INBOX: Couldn't keep all uids
Did the destination server maybe already get some new mails?
dsync(sys4-test-2@example.de): Warning: Mailbox changes caused a desync. You may want to run dsync again.
Does this help?
no, run again dsync , does not fix the expunge problem
sorry for noise , i did more testing, guess the general problem is in missing read
Problems
POP3 message order (when it's different from IMAP message order) is currently preserved only when destination is Maildir
This doesn't break dsync in any way, it just means that POP3 clients will see the UIDL list in different order then they used to, which will probably cause them to redownload mails. I already implemented the ordering support for dbox, but the ordering then also needs to be added to dovecot.index.cache, and that's causing some problems, so I haven't committed the change yet..
more tests showed up, that expunged mails arent synced anyway whatever mailbox format is used
perhaps that problem ?
The source is IMAP+POP3 server? Yeah, could be that it's not handling expunges when doing dsync the second time. v2.2 should be handling them better I think, although I haven't really tested much yet.
Hi Timo, it looks like a problem with dsync migration in 2.1.16, with "that" source server, a friend reported dsync migration form cyrus to sdbox had worked fine with 2.1.15
If source POP3 server merges multiple IMAP mailboxes into one POP3 INBOX, the migration won't be transparent. If source IMAP and POP3 servers return messages somehow differently, pop3-migration plugin might not be able to match the messages
how test this on source server i dont see something special there using telnet and/or Thunderbird
That is about if the same messages exist in POP3 session and IMAP INBOX. Just checking if their counts are equivalent is pretty much enough alone. In any case even if that happened, it doesn't cause your dsync problems.
i think i wont invest more time in this, dsync has/will much changed code in 2.2.x , so i may retest again with 2.2.x sometime, meanwhile for this case i might use imapsync, but thx for repeat anyway
however in case i would discover any more big news with dsnyc migration 2.1.16 , i will report
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
participants (2)
-
Robert Schetterer
-
Timo Sirainen