[Dovecot] More Migration Questions

[Richard Hobbs]

Hello,

OK, I've found the problem...

If i migrate all email from one server to another, preserving msg UIDs,
then the msg UIDs are ignored by the POP3 client if it has it's
hostname/IP changed in order to point to the new server.

So... mail client points to 192.168.1.1. mail is all downloaded but left
on server. migration to new server occurs, preserving msg UIDs. mail
client IP adjusted to point to 192.168.1.2. All mail downloaded again,
duplicates now exist.

The way to get around this is to either set your mail client to hostname
in the first place, and then make the change in DNS, or to remove all
msgs from the server before migration, so the msg UIDs are irrelevant.

It's also annoying that you cannot fix the problem before migration
because if i tell people to change from the old mail server's IP to the
old mail server's hostname, then again - the mail client will download
all their mail again and duplicate it!

Anyway... i'm glad i've found the problem, even if it does make the
migration a little annoying! :-)

Regards,
Richard.


Richard Hobbs wrote:
> Hello,
> 
> We've also tried setting the size of the msgs to be the same, just in
> case that made a difference. Here is some output from our old mail
> server during the test:
> 
> ======================================================================
> uidl
> +OK uidl command accepted.
> 1 a=="!(3N"!&D'"!87D"!
> 2 _fg!!)%>!!Ic>!!-@'"!
> 3 Yh]!!Zlk"!QfA"!Lj1!!
> .
> list
> +OK 3 visible messages (42852 octets)
> 1 28359
> 2 2918
> 3 11575
> .
> ======================================================================
> 
> Here are the same commands against the new server during our test:
> 
> ======================================================================
> uidl
> +OK
> 1 a=="!(3N"!&D'"!87D"!
> 2 _fg!!)%>!!Ic>!!-@'"!
> 3 Yh]!!Zlk"!QfA"!Lj1!!
> .
> list
> +OK 3 messages:
> 1 28359
> 2 2918
> 3 11575
> .
> ======================================================================
> 
> As far as i can tell, they are identical.
> 
> If i setup the old server in my mail client, it downloads all 3 msgs. If
> i then mark them all as "read" and change the hostname of the mail
> server to point to the new server, it then downloads the same 3 msgs so
> i end up with duplicates in my inbox, despite them having the same UIDLs!
> 
> The only difference i can see is that the old mail server has the X-UIDL
> header towards the end of the headers, and the new server puts the
> X-UIDL header right at the top above all other headers.
> 
> Does anyone know what on earth is going on here?
> 
> Thanks again!
> 
> Richard.
> 
> 
> Richard Hobbs wrote:
>> Hello,
>>
>> Thanks again - your reply will help greatly.
>>
>> I have a rather more annoying problem at the moment, best described by
>> my conversation in the dovecot IRC channel...
>>
>> i've telnetted to 110 on old server
>>
>> the "uidl" command shows 3 UIDLs
>>
>> uidl command also shows 3 UIDLs from new server...
>>
>> trouble is, despite seeing the same 3 UIDLs on the old mail server *and*
>> the new mail server, when issuing the "uidl" command through port 110
>> after logging in, i just setup the account in my mail client against the
>> old server, downloaded 3 msgs, changed the config for the new server,
>> and it downloaded the same msgs again!
>>
>> ok, on the old server, in the received email, towards the bottom of the
>> headers, i see this:
>>
>> X-UIDL: a=="!(3N"!&D'"!87D"!
>>
>> on the new server, in the duplicate received email, right at the top,
>> before any other headers, i see this:
>>
>> X-UIDL: a=="!(3N"!&D'"!87D"!
>>
>> whaton earth is going on here...
>>
>> both now have the same X-UIDL, but are being downloaded separately!
>>
>> So... does anyone have any ideas?
>>
>> Thanks again!
>>
>> Richard.
>>
>>
>> Timo Sirainen wrote:
>>> On Fri, 2009-08-14 at 15:39 +0000, Richard Hobbs wrote:
>>>>> That's better actually, you don't then have to add those ~/mail/
>>>>> compabilitity namespaces.
>>>> Well, i haven't added any ~/mail/ compatibility namespaces knowingly...
>>>> i guess they might be in the default config (my version has been built
>>>> for debian), but how would i check?
>>>>
>>>> Basically, the problem is that some users have their folders in
>>>> "~/mail/" and others have their folders in "~/".
>>> This is kind of what I meant by the compatibility namespaces. If some
>>> users have ~/mail/ and whatelse, you'll probably want to add such
>>> namespaces to Dovecot. See
>>> http://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_UW-IMAP
>>>
>>>> I want to convert both, and also and up with a standard location in the
>>>> maildir setup. I guess the best way to do this is to just copy
>>>> everyone's mailboxes from "~/" into "~/mail/" before i run the
>>>> conversion, right? 
>>> That should work, as long as you can find all the mailboxes (or are all
>>> files in ~/ mailboxes?)
>>>
>>>> As long as people don't object to their folders
>>>> potentially changing places within their IMAP client, everything will be
>>>> OK, right?
>>> With the compatibility namespace it should be pretty transparent to
>>> clients. Also you'll probably want to set separator=/ in all the
>>> namespaces.
>>>
> 

-- 
Richard Hobbs (IT Specialist)
Toshiba Research Europe Ltd. - Cambridge Research Laboratory
Email: richard.hobbs at crl.toshiba.co.uk
Web: http://www.toshiba-europe.com/research/
Tel: +44 1223 436999        Mobile: +44 7811 803377
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3306 bytes
Desc: S/MIME Cryptographic Signature
Url : http://dovecot.org/pipermail/dovecot/attachments/20090817/c2d06f33/attachment.bin