Hello,
just to soothe my paranoia from many years of qpopper usage, which enforces solitary access to a POP mailbox. Only one login per user is possible at the same time and if a session terminates w/o qpopper being properly notified it will remain locked for some time, 30 minutes by default.
With dovecot (0.99.10.4) one can login multiple times (using POP, with IMAP of course this comes as no surprise :) and thus I presume this bit from the configuration:
# Should we create dotlock file even when we want only a read-lock? Setting # this to yes hurts the performance when the mailbox is accessed simultaneously # by multiple processes, but it's needed for reliable reading if no other # locking methods are available.
#mbox_read_dotlock = no
means that if the only other reading accesses to a (mbox style in this case here) mailbox are via other dovecot processes everything will be fine?
Regards,
Christian Balzer
Christian Balzer Network/Systems Engineer NOC chibi@gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/
- Christian Balzer chibi@gol.com (20040518 21:10):
presume this bit from the configuration:
# Should we create dotlock file even when we want only a read-lock? Setting # this to yes hurts the performance when the mailbox is accessed simultaneously # by multiple processes, but it's needed for reliable reading if no other # locking methods are available.
#mbox_read_dotlock = no
means that if the only other reading accesses to a (mbox style in this case here) mailbox are via other dovecot processes everything will be fine?
Yes, but you should be aware there are other processes than Dovecot which can access the mailbox, moreover in a read-write fashion, such as your MDA (procmail, sendmail, maildrop, whatever).
You had better teach them to also use dotlock files to be safe.
Olivier wrote: [...]
means that if the only other reading accesses to a (mbox style in this case here) mailbox are via other dovecot processes everything will be fine?
Yes, but you should be aware there are other processes than Dovecot which can access the mailbox, moreover in a read-write fashion, such as your MDA (procmail, sendmail, maildrop, whatever).
Exim. ^^
You had better teach them to also use dotlock files to be safe.
Sure, it of course does that, it being a write access. Just like one dovecot session deleting a mail from the mailbox would need exclusive access to the thing for that time (and hopefully not utterly confuse the mail clients behind those sessions in the process). I just tested this with our web-mail (Ilohamail) and a reload of the inbox nicely reflects the harsh reality of the mail deleted by the other session.
Regards,
Christian Balzer
Christian Balzer Network/Systems Engineer NOC chibi@gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/
On Tue, 2004-05-18 at 15:10, Christian Balzer wrote:
# Should we create dotlock file even when we want only a read-lock? Setting # this to yes hurts the performance when the mailbox is accessed simultaneously # by multiple processes, but it's needed for reliable reading if no other # locking methods are available.
#mbox_read_dotlock = no
means that if the only other reading accesses to a (mbox style in this case here) mailbox are via other dovecot processes everything will be fine?
This setting only means that we use dotlock also for read-locking, not just write-locking mbox. Normally it uses just fcntl/flock. Hmm. I think I should change the description a bit.
So, currently Dovecot doesn't try to lock mbox for the whole POP3 session. That may create problems if two sessions actually try to access the mailbox concurrently. If message gets expunged by another process, Dovecot replies with -ERR to requests to fetch the message. That might confuse some POP3 clients, or cause them to send errors to user.
I don't think normally anyone even tries concurrent POP3 access? Anyway, I have thought before that Dovecot/pop3 should lock the mbox for the whole time, just like all others POP3 servers do (and RFC says too).. I'll add in TODO.
Timo wrote:
So, currently Dovecot doesn't try to lock mbox for the whole POP3 session. That may create problems if two sessions actually try to access the mailbox concurrently. If message gets expunged by another process, Dovecot replies with -ERR to requests to fetch the message. That might confuse some POP3 clients, or cause them to send errors to user.
That is (for me at least) acceptable as long as there is no danger of actual mailbox corruption (a message gets deleted while another client is actually downloading it). Is this the case in .99.10.4?
Again, just expunging something get some things confused, but not lethally so, as tested with a number of clients here. ^^
I don't think normally anyone even tries concurrent POP3 access?
Unfortunately the harsh reality of ISP life tells us otherwise, at least with substandard clients on slow links. Typical current scenario: User with Outlook (Express) on an ISDN or dialup link gets a huge mail. The default timeout for that client is 1 minute, after that it just gives up, but w/o our end actually noticing it. The result are the previously mentioned 30 minutes of POP-lock until qpopper gives up itself or the user tears down the PPP session which gets the message across to qpopper, too. Now during the server side imposed lock the user will be told that another POP session is active while that isn't true from their point of view of course. With the current dovecot they can try again immediately, of course failing again on the large email but the consistent error given might lead them to the right course of action: Get better software, a better link or at least configure your current setup correctly. ;)
Anyway, I have thought before that Dovecot/pop3 should lock the mbox for the whole time, just like all others POP3 servers do (and RFC says too).. I'll add in TODO.
If the RFC requires it, there will be no argument from me, though more granular locks which prevent really bad things from happening are sufficient for me and as detailed above might be more responsive.
Also you are saying mbox up there, would a maildir storage accessed via POP3 really be immune to these possibly confusing effects for clients?
Regards,
Christian Balzer
Christian Balzer Network/Systems Engineer NOC chibi@gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/
On Sat, 2004-05-22 at 05:30, Christian Balzer wrote:
So, currently Dovecot doesn't try to lock mbox for the whole POP3 session. That may create problems if two sessions actually try to access the mailbox concurrently. If message gets expunged by another process, Dovecot replies with -ERR to requests to fetch the message. That might confuse some POP3 clients, or cause them to send errors to user.
That is (for me at least) acceptable as long as there is no danger of actual mailbox corruption (a message gets deleted while another client is actually downloading it). Is this the case in .99.10.4?
There's no possibility for corruption, as long as all software accessing mbox uses compatible locking. So if you don't have mbox_read_dotlock set, make sure everyone uses also fcntl (or flock) locking.
Anyway, I have thought before that Dovecot/pop3 should lock the mbox for the whole time, just like all others POP3 servers do (and RFC says too).. I'll add in TODO.
If the RFC requires it, there will be no argument from me, though more granular locks which prevent really bad things from happening are sufficient for me and as detailed above might be more responsive.
Also you are saying mbox up there, would a maildir storage accessed via POP3 really be immune to these possibly confusing effects for clients?
Maildir can't be locked and so it has the same problem that message might get lost. That's pretty much the reason why I didn't bother to make pop3 lock mboxes either. Both mbox and pop3 are of smaller priority to me than maildir+imap, and making some mbox+pop3-specific locking kludge there wasn't very attractive.
Timo wrote (hmm, given the usual programmers life style I'd say VERY late at nigth :) :
There's no possibility for corruption, as long as all software accessing mbox uses compatible locking. So if you don't have mbox_read_dotlock set, make sure everyone uses also fcntl (or flock) locking.
Oh, my worries are not about the only other bit accessing the boxes, which is exim and that uses fcntl. My question was about this, not at all theoretical scenario: User has a client at home to auto-download (but leave mails on server) and is using a slow link, so those message downloads take a quite measurable time. Now he logs into webmail (which of course also ultimately uses dovecot to access the mailbox) and deletes a message which is currently in transit to his home machine. Will the result be: a) The "read" of that message has triggered an internal dovecot lock and the "write" of the delete will have to wait until this is released. b) The delete happens immediately, but thanks to buffering the read and delivery of the message is always successfully finished. c) The delete of the message happens immediately, the state of the message in transit is indeterminable and it might be truncated.
Maildir can't be locked and so it has the same problem that message might get lost. That's pretty much the reason why I didn't bother to make pop3 lock mboxes either. Both mbox and pop3 are of smaller priority to me than maildir+imap, and making some mbox+pop3-specific locking kludge there wasn't very attractive.
Totally understood and agreed with. The solution would of course be to totally lock out any other logins (imap or pop) while a pop3 session is active, like qpopper does. Alas that would have the aforementioned unpleasant side effects. So as long as the answer to the above question is a) or b) I see no reason to add either mbox or session locking.
Regards,
Christian Balzer
Christian Balzer Network/Systems Engineer NOC chibi@gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/
On 22.5.2004, at 06:46, Christian Balzer wrote:
Timo wrote (hmm, given the usual programmers life style I'd say VERY late at nigth :) :
Just went to bed, now just the last few things.. :)
User has a client at home to auto-download (but leave mails on server) and is using a slow link, so those message downloads take a quite measurable time. Now he logs into webmail (which of course also ultimately uses dovecot to access the mailbox) and deletes a message which is currently in transit to his home machine. Will the result be: a) The "read" of that message has triggered an internal dovecot lock and the "write" of the delete will have to wait until this is released.
Dovecot read-locks the mailbox while it's reading a mail, so this one happens.
b) The delete happens immediately, but thanks to buffering the read and delivery of the message is always successfully finished.
Something like this would happen with maildir.
c) The delete of the message happens immediately, the state of the message in transit is indeterminable and it might be truncated.
And this would happen if you didn't set the locking correctly :)
Timo Sirainen wrote:
... currently Dovecot doesn't try to lock mbox for the whole POP3 session. That may create problems if two sessions actually try to access the mailbox concurrently. If message gets expunged by another process, Dovecot replies with -ERR to requests to fetch the message. That might confuse some POP3 clients, or cause them to send errors to user.
Is there currently any way to enforce mbox r/w locking for the entire pop3 session in Dovecot? We're considering migrating from qpopper and need a pop3 server that enforces locking while a session is active. Thanks, Ken A
I don't think normally anyone even tries concurrent POP3 access? Anyway, I have thought before that Dovecot/pop3 should lock the mbox for the whole time, just like all others POP3 servers do (and RFC says too).. I'll add in TODO.
On Mon, 2006-04-03 at 12:55 -0700, Ken A wrote:
Timo Sirainen wrote:
... currently Dovecot doesn't try to lock mbox for the whole POP3 session. That may create problems if two sessions actually try to access the mailbox concurrently. If message gets expunged by another process, Dovecot replies with -ERR to requests to fetch the message. That might confuse some POP3 clients, or cause them to send errors to user.
Is there currently any way to enforce mbox r/w locking for the entire pop3 session in Dovecot? We're considering migrating from qpopper and need a pop3 server that enforces locking while a session is active.
There's now pop3_lock_session setting in CVS. I'll try to release beta6 also sometimes soon.
participants (5)
-
Christian Balzer
-
Ken A
-
Olivier Tharan
-
Timo Sirainen
-
Tomi Hakala