[Dovecot] POP3 locking
I'm trying to understand what dovecot's behavior will be in a given scenario to see if it will work as a (hopefully better) replacement for the home-grown pop3 server we currently use.
User A is on a slow dialup connection, and uses, oh, let's say Outlook, since that's where most of our problems seem to spring from. User A uses outlook to download his mail over pop3. Half way through downloading a particularly large message, Outlook decides that the whole process isn't happening fast enough, so it opens another connection to the server and tries to download (potentially) the same message again.
Now, our current server establishes a lock on the mailbox, and when that second connection happens it simply responds that there's no messages to collect... we used to have it terminate the connection, but this brings up confusing error messages on User A's client, and they call up and complain -- not an ideal situation.
I've got a dovecot installation on my testing server, and I think I've set everything up correctly; it authenticates from my LDAP database, finds the home directory, lists the mail, all that... I believe somewhere it there it uses fcntl to establish an advisory lock on the mailbox (oh, we're using Maildir, by the way).
I'm not sure where, though, or how long the lock persists for, or what its actual purpose is, because my tests show me that I can log the same users in with two concurrent sessions and get the same message twice. It's probably a fault in my understanding of how locking in user sessions of a pop3 server should work, but that doesn't sound right to me.
Can someone explain if what I'm seeing is correct, or what problem it may be symptomatic of? Does dovecot even do concurrent-user-session locking? So far I've enjoyed playing with dovecot -- far more than the current server we use. I'd like to use it as our primary pop3 server, but preventing users from having concurrent sessions and getting the same message more than once, and keeping them from seeing errors because of their client trying to open multiple sessions is important.
Oh, and I'm user dovecot 1.0.beta3, since I don't really have the option of upgrading the OS on my mail servers any time soon.
Adrian
Adrian Overbury wrote:
I'm trying to understand what dovecot's behavior will be in a given scenario to see if it will work as a (hopefully better) replacement for the home-grown pop3 server we currently use.
User A is on a slow dialup connection, and uses, oh, let's say Outlook, since that's where most of our problems seem to spring from.
User A uses outlook to download his mail over pop3. Half way through downloading a particularly large message, Outlook decides that the whole process isn't happening fast enough, so it opens another connection to the server and tries to download (potentially) the same message again.
I can't say I have ever seen this happen with Outlook to be honest?
Usually when this happens it means the user has added their one account
in Outlook more than once? Which version do you see this happen in?
Dovecot can be made to do exclusive locks, or the default is to share the mailbox. I don't recall the config option, but check a recent conf file
Oh, and I'm user dovecot 1.0.beta3, since I don't really have the option of upgrading the OS on my mail servers any time soon.
Now that's just silly talk. Upgrade to a more recent version anyway
If you post your OS type then I'm sure someone will help with packages - otherwise compile from source
Ed W
On Wed, 30 Jul 2008, Adrian Overbury wrote:
Now, our current server establishes a lock on the mailbox, and when that
second connection happens it simply responds that there's no messages to
collect... we used to have it terminate the connection, but this brings
up confusing error messages on User A's client, and they call up and
complain -- not an ideal situation.
Why do you do that? Is "-ERR unable to lock maildrop" in response to PASS or APOP not an option? One computer (laptop) accessing the same time as a PC accesses the box will purge its copy of the UIDL and refetch ALL messages next time...
I'm not sure where, though, or how long the lock persists for, or what
its actual purpose is, because my tests show me that I can log the same
users in with two concurrent sessions and get the same message twice.
It's probably a fault in my understanding of how locking in user
sessions of a pop3 server should work, but that doesn't sound right to me.
Well... according to RFC-1939, the session should be locked when entering TRANSACTION state. Purpose is to avoid two processes stomping over each other in the later UPDATE state.
Can someone explain if what I'm seeing is correct, or what problem it
may be symptomatic of? Does dovecot even do concurrent-user-session
locking? So far I've enjoyed playing with dovecot -- far more than the
current server we use. I'd like to use it as our primary pop3 server,
but preventing users from having concurrent sessions and getting the
same message more than once, and keeping them from seeing errors because
of their client trying to open multiple sessions is important.
Same here - I just did a trivial test (Dovecot 1.1 from Mercurial repo, two POP3+SSL sessions at the same time, off of Maildir): log in both, delete a message in one and QUIT (so it passes through UPDATE state), LIST in the remaining session still lists the deleted message, but retrieving it results in an error. Probably confusing for clients ;-)
HOWEVER, if the same message is downloaded twice, that's a client bug. It can use UIDL and avoid scheduling the same message twice for fetching.
Outlook is a piece of sh.. WRT mail capabilities, with Outlook 2003 still not fixing MIME + quoting bugs that Microsoft is aware of since the mid-1990s.
I have some hope that I'll be able to migrate to Kontact (KMail + organizer + stuff) next year. Certainly not my favourite but given there's no recent Windows version of Evolution...
Oh, and I'm user dovecot 1.0.beta3, since I don't really have the option
of upgrading the OS on my mail servers any time soon.
I'd stay away from such a version, but use a recent 1.0.X or 1.1.X - if someone is looking after your server regularly (once or twice a week or so), compiling it yourself shouldn't be an issue.
HTH
-- Matthias Andree
On Wed, 30 Jul 2008, Matthias Andree wrote:
Well... according to RFC-1939, the session should be locked when entering TRANSACTION state. Purpose is to avoid two processes stomping over each other in the later UPDATE state.
setting pop3_lock_session=yes works as expected (ca. 1.1.2 from hg):
+OK Dovecot ready. user login +OK pass password +OK Logged in. list -ERR [IN-USE] Couldn't open INBOX: Timeout while waiting for lock (closes connection).
HTH
Matthias Andree wrote on Wed, 30 Jul 2008 09:59:33 +0200:
setting pop3_lock_session=yes works as expected (ca. 1.1.2 from hg):
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
Kai
-- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
The problem is if you have an MUA/MDA retrieving lots of messages over a potentially slow connection, the mailbox can remain locked for a very long time for a read-only operation.
The problem that requires doing it is basically a client problem. Namely, some clients may get confused if in the middle of a series of RETR commands, they get a mailbox locked reply. Thus, the option exists to accomodate potentially broken clients and has potentially negative effects so I think the default is appropriate.
-- Mark Sapiro mark at msapiro net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jul 31, 2008, at 8:10 PM, Mark Sapiro wrote:
On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
The problem is if you have an MUA/MDA retrieving lots of messages over a potentially slow connection, the mailbox can remain locked for a very long time for a read-only operation.
Or before when dialups were more common, if the modem disconnected
you'd have to wait several minutes until the previous connection timed
out.
The problem that requires doing it is basically a client problem. Namely, some clients may get confused if in the middle of a series of RETR commands, they get a mailbox locked reply.
This should never happen with Dovecot. But what could happen is that
the message got expunged and the RETR would fail because of that.
So, the final determination here is that newer versions of Dovecot than the one I'm currently running will successfully lock up a mailbox and limit access to it to a single session while that session exists (up to a finite timeout, which I think is two minutes), right? Okay, then. Has anyone built packages for Ubuntu Dapper Drake from the latest sources, then? I'd appreciate being able to get my hands on thoses.
Adrian
Timo Sirainen wrote:
On Jul 31, 2008, at 8:10 PM, Mark Sapiro wrote:
On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
The problem is if you have an MUA/MDA retrieving lots of messages over a potentially slow connection, the mailbox can remain locked for a very long time for a read-only operation.
Or before when dialups were more common, if the modem disconnected you'd have to wait several minutes until the previous connection timed out.
The problem that requires doing it is basically a client problem. Namely, some clients may get confused if in the middle of a series of RETR commands, they get a mailbox locked reply.
This should never happen with Dovecot. But what could happen is that the message got expunged and the RETR would fail because of that.
Adrian Overbury wrote:
So, the final determination here is that newer versions of Dovecot than the one I'm currently running will successfully lock up a mailbox and limit access to it to a single session while that session exists (up to a finite timeout, which I think is two minutes), right? Okay, then. Has anyone built packages for Ubuntu Dapper Drake from the latest sources, then? I'd appreciate being able to get my hands on thoses.
Adrian
... I can't believe I actually typed 'thoses'...
Adrian Overbury wrote:
Okay, then. Has anyone built packages for Ubuntu Dapper Drake from the latest sources, then? I'd appreciate being able to get my hands on thoses.
What's wrong with the Ubunutu ones? Two seconds on google gets me: http://packages.ubuntu.com/hardy/dovecot-imapd
(I might be missing the point because I'm a gentoo user)
Ed W
on 7-31-2008 11:46 PM Ed W spake the following:
Adrian Overbury wrote:
Okay, then. Has anyone built packages for Ubuntu Dapper Drake from the latest sources, then? I'd appreciate being able to get my hands on thoses.
What's wrong with the Ubunutu ones? Two seconds on google gets me: http://packages.ubuntu.com/hardy/dovecot-imapd
(I might be missing the point because I'm a gentoo user)
Ed W
Poster did ask about latest, not a few versions back. I guess 2 seconds doesn't buy as much as it used to! ;-D
-- MailScanner is like deodorant... You hope everybody uses it, and you notice quickly if they don't!!!!
On 8/1/2008, Adrian Overbury (adrian@inomial.com) wrote:
Has anyone built packages for Ubuntu Dapper Drake from the latest sources, then? I'd appreciate being able to get my hands on thoses.
It isn't in backports? I couldn't find a web searchable version of ubuntu backports to search...
--
Best regards,
Charles
Timo Sirainen wrote:
On Jul 31, 2008, at 8:10 PM, Mark Sapiro wrote:
On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
The problem is if you have an MUA/MDA retrieving lots of messages over a potentially slow connection, the mailbox can remain locked for a very long time for a read-only operation.
Or before when dialups were more common, if the modem disconnected you'd have to wait several minutes until the previous connection timed out.
Still very common in rural USA. We see occasional pop3 locks that last hours while someone on a 26.4Kbps connect downloads mail. rfc1939 says it should be locked. I do understand why it's not default in dovecot though, since Dovecot is primarily an IMAP server, and the idea of locking a mailbox for hours is probably a bit repulsive to the developer. :-)
Ken
The problem that requires doing it is basically a client problem. Namely, some clients may get confused if in the middle of a series of RETR commands, they get a mailbox locked reply.
This should never happen with Dovecot. But what could happen is that the message got expunged and the RETR would fail because of that.
-- Ken Anderson Pacific.Net
On Thu, 31 Jul 2008, Timo Sirainen wrote:
On Jul 31, 2008, at 8:10 PM, Mark Sapiro wrote:
On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl wrote:
but it seems to be off by default. From my dovecot.conf (1.07):
#pop3_lock_session = no
I think it should be on by default.
The problem is if you have an MUA/MDA retrieving lots of messages over a potentially slow connection, the mailbox can remain locked for a very long time for a read-only operation.
Or before when dialups were more common, if the modem disconnected you'd have to wait several minutes until the previous connection timed out.
Which would hint to signal propagation issues on the dialin server. Modems detect quite quickly if the link is broken and the system can then issue SIGHUP and clean up locks...
-- Matthias Andree
participants (9)
-
Adrian Overbury
-
Charles Marcus
-
Ed W
-
Kai Schaetzl
-
Ken A
-
Mark Sapiro
-
Matthias Andree
-
Scott Silva
-
Timo Sirainen