[Dovecot] Maildir "locking"
Hello,
I'm looking for the alternative of qmail's chmod -t (sticky bit on the maildir root) for Dovecot. What I'm trying to achieve with this lock:
- Dovecot lmtp should give back a temporary error (so the email will be deferred and re-delivered later)
- all other Dovecot daemons (pop, imap) should work as usual, but should not alter maildir contents (they can modify their own files, like indexes, logs etc)
What is the best way to do this? If there is no such thing currently, would it be hard to implement the sticky bit checking on the root?
Thanks,
On Wed, 2011-09-14 at 13:32 +0200, Attila Nagy wrote:
Hello,
I'm looking for the alternative of qmail's chmod -t (sticky bit on the maildir root) for Dovecot. What I'm trying to achieve with this lock:
- Dovecot lmtp should give back a temporary error (so the email will be deferred and re-delivered later)
- all other Dovecot daemons (pop, imap) should work as usual, but should not alter maildir contents (they can modify their own files, like indexes, logs etc)
What is the best way to do this? If there is no such thing currently, would it be hard to implement the sticky bit checking on the root?
dovecot-uidlist.lock basically does this. Dovecot comes with maildirlock utility to properly create it. How long would your locks be? They are assumed stale after 2 minutes if you don't update the mtime. Readers will block and if they're still locked after 2 minutes they'll abort (if mtime has been changed). There's also mail_max_lock_timeout setting that changes this wait (you could e.g. lower it only with lmtp).
On 09/15/11 10:19, Timo Sirainen wrote:
On Wed, 2011-09-14 at 13:32 +0200, Attila Nagy wrote:
Hello,
I'm looking for the alternative of qmail's chmod -t (sticky bit on the maildir root) for Dovecot. What I'm trying to achieve with this lock:
- Dovecot lmtp should give back a temporary error (so the email will be deferred and re-delivered later)
- all other Dovecot daemons (pop, imap) should work as usual, but should not alter maildir contents (they can modify their own files, like indexes, logs etc)
What is the best way to do this? If there is no such thing currently, would it be hard to implement the sticky bit checking on the root? dovecot-uidlist.lock basically does this. Dovecot comes with maildirlock utility to properly create it. How long would your locks be? They are assumed stale after 2 minutes if you don't update the mtime. Readers will block and if they're still locked after 2 minutes they'll abort (if mtime has been changed). There's also mail_max_lock_timeout setting that changes this wait (you could e.g. lower it only with lmtp).
Well, basically "forever" in the sense that I would like to move the mailbox to a different machine, so if lmtp waits for the lock to disappear and that happens when the mailbox is deleted, and it will do the delivery, it's a bad thing. Before Dovecot, we've had the following process of mailbox moving:
- set the sticky bit on the maildir, so qmail won't deliver into it (will give back 4XX)
- start to sync/copy the mailbox to the other machine
- if it's over, remove the directory on the source machine
So what I'm looking for is a lock method, which makes the mailbox read only, so every modification should "soft" fail (no 500 errors on lmtp). What would be the best for this (moving mailboxes between machines)? BTW, the process can be time consuming, even tens of minutes (lots of mails).
On Thu, 2011-09-15 at 10:25 +0200, Attila Nagy wrote:
What is the best way to do this? If there is no such thing currently, would it be hard to implement the sticky bit checking on the root? dovecot-uidlist.lock basically does this. Dovecot comes with maildirlock utility to properly create it. How long would your locks be? They are assumed stale after 2 minutes if you don't update the mtime. Readers will block and if they're still locked after 2 minutes they'll abort (if mtime has been changed). There's also mail_max_lock_timeout setting that changes this wait (you could e.g. lower it only with lmtp).
Well, basically "forever" in the sense that I would like to move the mailbox to a different machine,
So you have a proxy that decides what backend server the connections are redirected to? How about you do it completely without locking with dsync? Moving between servers works basically the same as converting a mailbox format, with the difference of "changing mail_location" you "change backend server". http://wiki2.dovecot.org/Tools/Dsync#example_converting
On 09/15/11 10:39, Timo Sirainen wrote:
On Thu, 2011-09-15 at 10:25 +0200, Attila Nagy wrote:
What is the best way to do this? If there is no such thing currently, would it be hard to implement the sticky bit checking on the root? dovecot-uidlist.lock basically does this. Dovecot comes with maildirlock utility to properly create it. How long would your locks be? They are assumed stale after 2 minutes if you don't update the mtime. Readers will block and if they're still locked after 2 minutes they'll abort (if mtime has been changed). There's also mail_max_lock_timeout setting that changes this wait (you could e.g. lower it only with lmtp).
Well, basically "forever" in the sense that I would like to move the mailbox to a different machine, So you have a proxy that decides what backend server the connections are redirected to? How about you do it completely without locking with dsync? Moving between servers works basically the same as converting a mailbox format, with the difference of "changing mail_location" you "change backend server". http://wiki2.dovecot.org/Tools/Dsync#example_converting
Yes, there is a proxy in front of the servers. Is dsync usable with 3rd party maildir programs? (not only Dovecot uses these mailboxes)
On Thu, 2011-09-15 at 11:37 +0200, Attila Nagy wrote:
So you have a proxy that decides what backend server the connections are redirected to? How about you do it completely without locking with dsync? Moving between servers works basically the same as converting a mailbox format, with the difference of "changing mail_location" you "change backend server". http://wiki2.dovecot.org/Tools/Dsync#example_converting
Yes, there is a proxy in front of the servers. Is dsync usable with 3rd party maildir programs? (not only Dovecot uses these mailboxes)
The problems with 3rd party maildir programs come if during the move they:
Expunge last message(s) from mailbox. (dsync can't know if it should add or expunge them, so it plays it safe and adds them back)
Delete a mailbox. (dsync can't know if it should add or delete it, so again it just adds it back.)
Remove subscriptions. (again pretty much the same reason.)
It's probably quite unlikely that they do any of this during the move. You could even reduce the window by doing:
- dsync backup
- dsync backup
- switch to new server
- kill all existing connections
- dsync mirror
The 3-5 steps probably take only a few seconds. The "dsync backup" then guarantees that the destination server will look exactly like the source server. ("dsync mirror" is used in step 5, because between steps 3-4 either server can get changes.)
On 09/15/11 11:43, Timo Sirainen wrote:
So you have a proxy that decides what backend server the connections are redirected to? How about you do it completely without locking with dsync? Moving between servers works basically the same as converting a mailbox format, with the difference of "changing mail_location" you "change backend server". http://wiki2.dovecot.org/Tools/Dsync#example_converting
Yes, there is a proxy in front of the servers. Is dsync usable with 3rd party maildir programs? (not only Dovecot uses these mailboxes) The problems with 3rd party maildir programs come if during the move
On Thu, 2011-09-15 at 11:37 +0200, Attila Nagy wrote: they:
Expunge last message(s) from mailbox. (dsync can't know if it should add or expunge them, so it plays it safe and adds them back)
Delete a mailbox. (dsync can't know if it should add or delete it, so again it just adds it back.)
Remove subscriptions. (again pretty much the same reason.)
It's probably quite unlikely that they do any of this during the move. You could even reduce the window by doing:
- dsync backup
- dsync backup
- switch to new server
- kill all existing connections
- dsync mirror
The 3-5 steps probably take only a few seconds. The "dsync backup" then guarantees that the destination server will look exactly like the source server. ("dsync mirror" is used in step 5, because between steps 3-4 either server can get changes.)
OK, thanks for the info, I will try it out.
participants (2)
-
Attila Nagy
-
Timo Sirainen