[Dovecot] Staged migration from mbox to maildir
So much changes in this migration that the ideal way to do it would be to begin with a few users or a department, then migrate the users affinity group by affinity group: first an institute or so, then the faculty, then the staff, then the students, moving to bigger and bigger groupings as the bugs work out of the migration and the move becomes more assured.
We use sendmail and procmail. There's no problem there, as the ~./procmailrc can be changed to over-ride the mbox default until all groups are done and it become the default. The problem comes with IMAP. While dovecot can tell if a folder is mbox or maildir, it has to be pointed to the right place (by namespace definitions in the client, IIRC), and the default of putting the inbox under ~/mail is one I'd like to embrace for various reason...but given that that means moved inbox folders and *that* means either making a global change (there goes staged migration) OR changing the namespace definitions on each PC. I can get to the early few and change the namespaces definition, but there doesn't appear to any equivalent (enlighten me, if I'm missing something) to ~/.procmailrc for imap, so that I don't have to get on the client machine.
Is this correct or am I (hopefully) wrong and there *is* a way to change things on the server that allows for staged migration? Oh, I would so like to be wrong! IMAP should have an rc file.............
"Eppur si muove." (But Still it moves) Galileo, leaving the Inquisition, after buckling under the threat of torture and excommunication and recanting from his proof that the heavens do not revolve around the earth -- Stewart Dean, Unix System Admin, Henderson Computer Center, Bard College, Annandale, New York 12504 sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
Hi,
I'm working with a an IMAP client for a S60 (Nokia) phone and we are having a small problem (not in Dovecot!) but somewhere deep in our own system which has to do with certificates that are self signed.
Somehow in some circumstance if you accept a self-signed certificate as an exception then the client will send a strange command to the imap-login which it doesn't recognize. We are quite sure this is a problem in our own system and not with Dovecot
Since we have no access to the certificate (SSL/TLS) handling code we are a bit at loss here and have to "proof" to "the other" guys in Finland that it's there fault :-)
The type of errors that show up in Dovecot in these circumstances are (with the real username and IP address removed)
imap-login: Disconnected (no auth attempts): rip=some.ip.address user_name=192.168.0.2, TLS handshaking: SSL_accept() failed: error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpectedmessage
Is there some more debugging we could enable to see exactly the type of wrong command the SSL/certificate handling are send in the handshake procedure ?
(We have all the debug and/or the auth_* flags in dovecot.conf enabled already)
Any idea?
Johan
Words by Johan Persson [Thu, Mar 19, 2009 at 12:37:25AM +0100]:
Hi,
I'm working with a an IMAP client for a S60 (Nokia) phone and we are having a small problem (not in Dovecot!) but somewhere deep in our own system which has to do with certificates that are self signed.
Hmm, this must be thread highjacking month or something.
-- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc
"One man’s theology is another man’s belly laugh." -- Robert A. Heinlein
On Thu, 2009-03-19 at 00:37 +0100, Johan Persson wrote:
I'm working with a an IMAP client for a S60 (Nokia) phone and we are having a small problem (not in Dovecot!) but somewhere deep in our own system which has to do with certificates that are self signed.
Somehow in some circumstance if you accept a self-signed certificate as an exception then the client will send a strange command to the imap-login which it doesn't recognize. We are quite sure this is a problem in our own system and not with Dovecot
So it's not easily reproducible?
Since we have no access to the certificate (SSL/TLS) handling code we are a bit at loss here and have to "proof" to "the other" guys in Finland that it's there fault :-)
You mean a bug in S60 libraries?
imap-login: Disconnected (no auth attempts): rip=some.ip.address user_name=192.168.0.2, TLS handshaking: SSL_accept() failed: error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpectedmessage .. Is there some more debugging we could enable to see exactly the type of wrong command the SSL/certificate handling are send in the handshake procedure ?
(We have all the debug and/or the auth_* flags in dovecot.conf enabled already)
verbose_ssl=yes makes Dovecot log all errors/warnings that OpenSSL can tell (AFAIK). Perhaps you could use this:
Hi,
t's not easily reproducible?
Yes, this is 100% reproducible if you use the "Accept certificate permanently" when the client receives the warning that the certificate on the server is not trusted.
The strange thing is that if you instead use "Accept certificate only this time" then it works
Since we have no access to the certificate (SSL/TLS) handling code we are a bit at loss here and have to "proof" to "the other" guys in Finland thatc it's there fault :-)
You mean a bug in S60 libraries? Yep. Since it seems that the server receives some erronous messages
verbose_ssl=yes makes Dovecot log all errors/warnings that OpenSSL can Yes, this is already enabled
http://crypto.stanford.edu/~eujin/sslsniffer/index.html Will have a look at this.
Thanks! Johan
Johan,
As far as cert negotiation happens on very early stages of protocol just write as small program with as many debugging as you want.
-Dmitry
Johan Persson wrote:
Hi,
t's not easily reproducible?
Yes, this is 100% reproducible if you use the "Accept certificate permanently" when the client receives the warning that the certificate on the server is not trusted.
The strange thing is that if you instead use "Accept certificate only this time" then it works
Since we have no access to the certificate (SSL/TLS) handling code we are a bit at loss here and have to "proof" to "the other" guys in Finland thatc it's there fault :-)
You mean a bug in S60 libraries? Yep. Since it seems that the server receives some erronous messages
verbose_ssl=yes makes Dovecot log all errors/warnings that OpenSSL can Yes, this is already enabled
http://crypto.stanford.edu/~eujin/sslsniffer/index.html Will have a look at this.
Thanks! Johan
-- Dmitry Samersoff dms@samersoff.net, http://devnull.samersoff.net
- There will come soft rains ...
Words by Stewart Dean [Wed, Mar 18, 2009 at 02:10:53PM -0400]:
So much changes in this migration that the ideal way to do it would be to begin with a few users or a department, then migrate the users affinity group by affinity group: first an institute or so, then the faculty, then the staff, then the students, moving to bigger and bigger groupings as the bugs work out of the migration and the move becomes more assured.
Tell me about it, we have to plan the migration of 6 million accounts (many tens of TB) from Maildir to dbox in the near time :)
-- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc
"One man’s theology is another man’s belly laugh." -- Robert A. Heinlein
- Stewart Dean sdean@bard.edu wrote:
So much changes in this migration that the ideal way to do it would be to begin with a few users or a department, then migrate the users affinity group by affinity group: first an institute or so, then the faculty, then the staff, then the students, moving to bigger and bigger groupings as the bugs work out of the migration and the move becomes more assured.
We use sendmail and procmail. There's no problem there, as the ~./procmailrc can be changed to over-ride the mbox default until all groups are done and it become the default. The problem comes with IMAP. While dovecot can tell if a folder is mbox or maildir, it has to be pointed to the right place (by namespace definitions in the client, IIRC), and the default of putting the inbox under ~/mail is one I'd like to embrace for various reason...but given that that means moved inbox folders and *that* means either making a global change (there goes staged migration) OR changing the namespace definitions on each PC. I can get to the early few and change the namespaces definition, but there doesn't appear to any equivalent (enlighten me, if I'm missing something) to ~/.procmailrc for imap, so that I don't have to get on the client machine.
I don't know whether i fully understand what you are trying to achieve, but dovecot can work with a per user mail_location (passed via userdb) [1] that might help in your situation. Furthermore you can get _very_ flexible in determining the mail location (or even doing a lot of other things) by using a wrapper script to mail_executable [2].
Sebastian
[1] http://wiki.dovecot.org/MailLocation [2] http://wiki.dovecot.org/PostLoginScripting
On Mar 18, 2009, at 2:10 PM, Stewart Dean wrote:
The problem comes with IMAP. While dovecot can tell if a folder is
mbox or maildir, it has to be pointed to the right place (by
namespace definitions in the client, IIRC), and the default of
putting the inbox under ~/mail is one I'd like to embrace for
various reason...but given that that means moved inbox folders and
*that* means either making a global change (there goes staged
migration) OR changing the namespace definitions on each PC. I can
get to the early few and change the namespaces definition, but there
doesn't appear to any equivalent (enlighten me, if I'm missing
something) to ~/.procmailrc for imap, so that I don't have to get on
the client machine.
By namespace definitions you probably mean the IMAP clients' "imap
folder prefix" or whatever they happen to call them in different
clients? You can create namespaces for backwards compatibility, you
probably want similar namespaces to what is described here under UW-
IMAP examples: http://wiki.dovecot.org/Namespaces
You could do that kind of a change already with mbox users to make
sure everything works. Then have your users use one of the two
mail_locations (by returning "mail" extra field from userdb):
- mail_location = mbox:~/mail
- mail_location = maildir:~/Maildir
participants (6)
-
Dmitry Samersoff
-
Johan Persson
-
Jose Celestino
-
Sebastian Kayser
-
Stewart Dean
-
Timo Sirainen