[Dovecot] Experience moving mailboxes from Dovecot 0.99.14 to Dovecot 1.07 => Improvement possible
Hi,
This small mail to share my observation about a recent move of mailboxes between two servers and ask about explanation and/or improvement about UIDL in dovecot.
SV1 : Dovecot 0.99.14 / Red Hat Fedore Core 4 SV2 : Dovecot 1.07 / CentOS 5.2 Mailboxes in /var/spool/mail on the twoo servers.
Test will be done with outlook express with option "leave message on server" checked.
Goal is simply to move users mailboxes from SV1 to SV2 without re-downloading all messages.
Try 1:
- Stop dovecot on SV2
- Clear all dovecot indexes on SV2
- Rsync of my mailbox
- Start dovecot on SV2
- Update pop setting in outlook and send/receive Result : => starting to download 3000 mails...
After some debug, I discovered that UIDL where not of the same format => put pop3_uidl_format = %v.%u in dovecot.conf of SV2.
Try 2: Same operations as Try1 Result : => starting to download 3000 mails...
UIDL's where of the same format but values where not corresponding... Teleting on SV1 and asking UIDL, last value is xxxxxxxxxx.85878. On SV2, xxxxxxxxxx.85879 was the FIRST value of the list. Conclusion for the moment, Dovecot has a problem with the detection of existing UID of the moved mailbox.
Comparaison of first header found from the two mailboxes show ... differences !
SV1 give the following : (...) X-UID: 70570 Content-Length: 1561 X-IMAPbase: xxxxxxxxxx 85845 $MDNSent X-Keywords: (...)
SV2 give the following (after first POP attempt) : (...) Content-Length: 1561 X-IMAPbase: xxxxxxxxxx 0000089204 $MDNSent X-Keywords: X-UID: 85846 (...)
Observation : X-UID: 85846 is not at the same place in the header.
Try 3:
- Stop dovecot on SV2
- Clear all dovecot indexes on SV2
- Rsync of my mailbox
- Edit mailbox and move X-UID header after the X-Keywords header
- Start dovecot on SV2
- Update pop setting in outlook and send/receive
Result : => OK.
Is that normal that dovecot is impacted by the position of the headers ? Maybe some improvement possible ?
Maybe theyre is another method to make my migration ?
Any observation or comment is welcome...
-- Benoît Desloges Network Engineer
On 8/6/2008, Benoît DESLOGES (benoit.desloges@gmail.com) wrote:
Goal is simply to move users mailboxes from SV1 to SV2 without re-downloading all messages.
If you're going to go through all of this trouble, you really should go ahead and update to latest version - now at 1.1.2...
rpms for centos available on atrpms.net
--
Best regards,
Charles
Quoting Charles Marcus CMarcus@Media-Brokers.com:
rpms for centos available on atrpms.net
Sadly not for Centos 3.x, only for Centos 4/5... :(
Anyone know about Dovecot 1.1.x rpms for Centos/RHEL 3.x?
-- Eric Rostetter The Department of Physics The University of Texas at Austin
Go Longhorns!
On 8/6/2008, Eric Rostetter (rostetter@mail.utexas.edu) wrote:
Anyone know about Dovecot 1.1.x rpms for Centos/RHEL 3.x?
I'd be more interested in upgrading the server to a reasonably recent version of the distro...
<OT> This is one huge reason why I like gentoo so much. As long as I update it regularly, I never have to worry about a massive update that breaks everything. </OT>
--
Best regards,
Charles
Quoting Charles Marcus CMarcus@Media-Brokers.com:
On 8/6/2008, Eric Rostetter (rostetter@mail.utexas.edu) wrote:
Anyone know about Dovecot 1.1.x rpms for Centos/RHEL 3.x?
I'd be more interested in upgrading the server to a reasonably
recent version of the distro...
Unfortunately, it isn't a redundant setup, so an upgrade is downtime.
I've thought about doing an on-line (e.g., yum) upgrade from 3 to 4, but I'm not sure 4 would qualify as "reasonably recent" and it would still require a reboot, but this is an option and would get me the new dovecot rpms at least...
Since there is no good way to do an on-line upgrade from CentOS/RHEL 3 to CentOS/RHEL 5, that isn't really an option at this time (too much downtime).
I've also had machines that were hardware frozen at older OS versions... Though that is not the case in this instance (was for my print server I had to recently deal with).
This is one huge reason why I like gentoo so much.
It has nothing to do with gentoo, IMHO.
As long as I update it regularly, I never have to worry about a
massive update that breaks everything.
Same can be said for most distros, but I can't afford the downtime of the constant upgrades which mean constant reboots... That is why people pick a "enterprise" solution like RHEL/CentOS, so they can have better uptime (with support) than a non-enterprise systems... I regularly have machines with 2 or 3 years of uptime before I need to reboot them for an upgrade (they are behind firewalls, in case you wonder how I get along on such old kernels).
Obviously, RHEL/CentOS 3.x will end of life, and I'll need to upgrade eventually because of that, but the more I can put it off, then better... But sometimes you just need to bite the bullet, and that day may be close at hand for this server...
Or, I can just roll my own RHEL/CentOS 3 rpm package also... :) Which is less work than an OS upgrade at least...
Best regards,
Charles
-- Eric Rostetter The Department of Physics The University of Texas at Austin
Go Longhorns!
Hi,
it's always interesting to observe and discuss the different update strategies (although not entirely on-topic)...
On Wed, 06 Aug 2008 11:25:59 -0500, Eric Rostetter rostetter@mail.utexas.edu wrote:
Quoting Charles Marcus CMarcus@Media-Brokers.com:
On 8/6/2008, Eric Rostetter (rostetter@mail.utexas.edu) wrote:
Anyone know about Dovecot 1.1.x rpms for Centos/RHEL 3.x?
I'd be more interested in upgrading the server to a reasonably
recent version of the distro...Unfortunately, it isn't a redundant setup, so an upgrade is downtime.
I've thought about doing an on-line (e.g., yum) upgrade from 3 to 4, but I'm not sure 4 would qualify as "reasonably recent" and it would still require a reboot, but this is an option and would get me the new dovecot rpms at least...
Since there is no good way to do an on-line upgrade from CentOS/RHEL 3 to CentOS/RHEL 5, that isn't really an option at this time (too much downtime).
How can such an important system be a non-redundant setup? Hardware breakage (or a cracker, see below) would cause minutes or probably even hours of downtime...
I've also had machines that were hardware frozen at older OS versions... Though that is not the case in this instance (was for my print server I had to recently deal with).
This is one huge reason why I like gentoo so much.
It has nothing to do with gentoo, IMHO.
It has in that way, that there are no releases, no big jumps with lots of breakage and config file syntax changes... But I definitely wouldn't say Gentoo is a good distribution for systems that need to be highly available. (I'm using Gentoo myself on desktops and servers, but none of them do run really critical stuff).
As long as I update it regularly, I never have to worry about a
massive update that breaks everything.Same can be said for most distros, but I can't afford the downtime of the constant upgrades which mean constant reboots... That is why people pick a "enterprise" solution like RHEL/CentOS, so they can have better uptime (with support) than a non-enterprise systems...
"Enterprise system" - surely sounds professional and all ;) But not rebooting (during scheduled maintenance on a time of week/day where the least clients will be affected) for a new kernel that fixes a critical security issue definitely does not. IMHO.
I regularly have machines with 2 or 3 years of uptime before I need to reboot them for an upgrade (they are behind firewalls, in case you wonder how I get along on such old kernels).
Maybe you should upgrade your security knowledge along with your kernels ;) Many (if not most) attacks come from the inside (e.g. via trojans/viruses/rootkits on client (laptop) computers). Thus, the concept of something being "secure because behind firewalls" is at least partly obsolete.
Obviously, RHEL/CentOS 3.x will end of life, and I'll need to upgrade eventually because of that, but the more I can put it off, then better... But sometimes you just need to bite the bullet, and that day may be close at hand for this server...
Build it with redundancy this time. At least software-wise (for example using virtualisation), so that you have a test system on which you can "simulate" a pending update before you roll it out on the production system.
Patrick.
-- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005
Since there is no good way to do an on-line upgrade from CentOS/RHEL 3 to CentOS/RHEL 5, that isn't really an option at this time (too much downtime).
..
This is one huge reason why I like gentoo so much.
It has nothing to do with gentoo, IMHO.
It has in that way, that there are no releases, no big jumps with lots of breakage and config file syntax changes... But I definitely wouldn't say Gentoo is a good distribution for systems that need to be highly available. (I'm using Gentoo myself on desktops and servers, but none of them do run really critical stuff).
I agree - I use gentoo on all my main servers after having painted myself into a corner like you with Redhat 7.1 some years back.
The machine which replaced that Redhat 7.1 machine (you do the maths) is running nicely here still and hasn't had an "upgrade" since - it just gets updated with key packages from time to time
My next major epiphany has been virtualisation. The machine mentioned above if end of life now (1Ghz machine) and the replacement which is coming online uses vservers with near every major service in it's own vserver instance. This is just fantastic
If you switch to virtualisation on your next upgrade you will find youself in this same situation, but this time you can boot up a copy of the main machine - upgrade it, then within seconds kill the old machine and start the new machine up as the live master (I keep the data separate to the virtual machine instance so I can boot up two instances both playing with the same data)
It's fantastic for web hosting also - got a new app which you don't trust - couple of mins later you can spin up a new server and stick it all on that!
Vservers are nice low weight virtualisation and work well for me. I started with a smallish stage4 hardened gentoo tar file (ie roughly you zip up a working server) and then I customised that in several major ways, eg one with apache, another with nginx, another clean. Then I can use any of those base installations as a starting point for a new server. I keep all the vservers as similar as possible for ease of installation and basically I test upgrading one, then the others use the compiled packages and update in around 10 mins for the whole machine.
It's very cool also to get a machine into a state where you think a reboot is the quickest fix and realise that this takes about 3-4 seconds only with a quickly "vserver mail1 restart" !! Wahey!
Ed W
Hi,
On Wed, Aug 06, 2008 at 10:55:08AM -0500, Eric Rostetter wrote:
Quoting Charles Marcus CMarcus@Media-Brokers.com:
rpms for centos available on atrpms.net Sadly not for Centos 3.x, only for Centos 4/5... :( Anyone know about Dovecot 1.1.x rpms for Centos/RHEL 3.x?
You could try to rebuild from ATrpms' src.rpm, but to spare some trouble this is what I had with 1.1.rc4 4 months ago:
checking for auth_userokay... no checking for krb5-config... YES configure: error: Can't build with GSSAPI support: v1.2 library not supported
Maybe one could patch the specfile/package up to support RHEL3, and if you want to you could maintain this at ATrpms.
Axel.Thimm at ATrpms.net
On Aug 6, 2008, at 9:28 AM, Benoît DESLOGES wrote:
Try 3:
- Stop dovecot on SV2
- Clear all dovecot indexes on SV2
- Rsync of my mailbox
- Edit mailbox and move X-UID header after the X-Keywords header
The important change was that X-UID: came after X-IMAPbase: header.
- Start dovecot on SV2
- Update pop setting in outlook and send/receive
Result : => OK.
Is that normal that dovecot is impacted by the position of the
headers ? Maybe some improvement possible ?
I did think about previously if it should work like this, but it
seemed like a lot more extra work to support this kind of a situation
properly and I thought it shouldn't happen normally anyway. I guess
v0.99.x then wrote them in wrong order..
participants (7)
-
Axel Thimm
-
Benoît DESLOGES
-
Charles Marcus
-
Ed W
-
Eric Rostetter
-
Patrick Nagel
-
Timo Sirainen