Hello,
OK, I've found the problem...
If i migrate all email from one server to another, preserving msg UIDs, then the msg UIDs are ignored by the POP3 client if it has it's hostname/IP changed in order to point to the new server.
So... mail client points to 192.168.1.1. mail is all downloaded but left on server. migration to new server occurs, preserving msg UIDs. mail client IP adjusted to point to 192.168.1.2. All mail downloaded again, duplicates now exist.
The way to get around this is to either set your mail client to hostname in the first place, and then make the change in DNS, or to remove all msgs from the server before migration, so the msg UIDs are irrelevant.
It's also annoying that you cannot fix the problem before migration because if i tell people to change from the old mail server's IP to the old mail server's hostname, then again - the mail client will download all their mail again and duplicate it!
Anyway... i'm glad i've found the problem, even if it does make the migration a little annoying! :-)
Regards, Richard.
Richard Hobbs wrote:
Hello,
We've also tried setting the size of the msgs to be the same, just in case that made a difference. Here is some output from our old mail server during the test:
====================================================================== uidl +OK uidl command accepted. 1 a=="!(3N"!&D'"!87D"! 2 _fg!!)%>!!Ic>!!-@'"! 3 Yh]!!Zlk"!QfA"!Lj1!! . list +OK 3 visible messages (42852 octets) 1 28359 2 2918 3 11575 .
Here are the same commands against the new server during our test:
====================================================================== uidl +OK 1 a=="!(3N"!&D'"!87D"! 2 _fg!!)%>!!Ic>!!-@'"! 3 Yh]!!Zlk"!QfA"!Lj1!! . list +OK 3 messages: 1 28359 2 2918 3 11575 .
As far as i can tell, they are identical.
If i setup the old server in my mail client, it downloads all 3 msgs. If i then mark them all as "read" and change the hostname of the mail server to point to the new server, it then downloads the same 3 msgs so i end up with duplicates in my inbox, despite them having the same UIDLs!
The only difference i can see is that the old mail server has the X-UIDL header towards the end of the headers, and the new server puts the X-UIDL header right at the top above all other headers.
Does anyone know what on earth is going on here?
Thanks again!
Richard.
Richard Hobbs wrote:
Hello,
Thanks again - your reply will help greatly.
I have a rather more annoying problem at the moment, best described by my conversation in the dovecot IRC channel...
i've telnetted to 110 on old server
the "uidl" command shows 3 UIDLs
uidl command also shows 3 UIDLs from new server...
trouble is, despite seeing the same 3 UIDLs on the old mail server *and* the new mail server, when issuing the "uidl" command through port 110 after logging in, i just setup the account in my mail client against the old server, downloaded 3 msgs, changed the config for the new server, and it downloaded the same msgs again!
ok, on the old server, in the received email, towards the bottom of the headers, i see this:
X-UIDL: a=="!(3N"!&D'"!87D"!
on the new server, in the duplicate received email, right at the top, before any other headers, i see this:
X-UIDL: a=="!(3N"!&D'"!87D"!
whaton earth is going on here...
both now have the same X-UIDL, but are being downloaded separately!
So... does anyone have any ideas?
Thanks again!
Richard.
Timo Sirainen wrote:
On Fri, 2009-08-14 at 15:39 +0000, Richard Hobbs wrote:
That's better actually, you don't then have to add those ~/mail/ compabilitity namespaces. Well, i haven't added any ~/mail/ compatibility namespaces knowingly... i guess they might be in the default config (my version has been built for debian), but how would i check?
Basically, the problem is that some users have their folders in "~/mail/" and others have their folders in "~/". This is kind of what I meant by the compatibility namespaces. If some users have ~/mail/ and whatelse, you'll probably want to add such namespaces to Dovecot. See http://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_UW-IMAP
I want to convert both, and also and up with a standard location in the maildir setup. I guess the best way to do this is to just copy everyone's mailboxes from "~/" into "~/mail/" before i run the conversion, right? That should work, as long as you can find all the mailboxes (or are all files in ~/ mailboxes?)
As long as people don't object to their folders potentially changing places within their IMAP client, everything will be OK, right? With the compatibility namespace it should be pretty transparent to clients. Also you'll probably want to set separator=/ in all the namespaces.
-- Richard Hobbs (IT Specialist) Toshiba Research Europe Ltd. - Cambridge Research Laboratory Email: richard.hobbs@crl.toshiba.co.uk Web: http://www.toshiba-europe.com/research/ Tel: +44 1223 436999 Mobile: +44 7811 803377