Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
With mdbox and maildir format I could reliably crash my mail storage just by delivering mails to the both dovecots via LMTP to the same user. In maildir dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox.
So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check?
I think it would be really nice if dovecot could use glusterfs for realy scalable postbox servers.
Mit freundlichen Grüßen,
Michael Schwartzkopff
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 1/13/15, Michael Schwartzkopff ms@sys4.de wrote:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
With mdbox and maildir format I could reliably crash my mail storage just by
delivering mails to the both dovecots via LMTP to the same user. In maildir
dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox.
So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check?
Interesting, we use NFSv3 dovecot LDA with maildir, we have at present two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we added a few more over Christmas, so I think about 32 pop3 servers, but with only 4 imap servers incl webmail (IMAP is not heavily used here due to government spy laws) talking to NAS storage server backend, *we do not use director* at all and has never been an issue. Director IIRC solves the problem of IMAP inconsistency, but we never see advantage when we tested, no doubt it solves some fancy setup problem, but since director can not help with pop3, it was not worth the hassle. never had any problems with webmail either, load balancers seem to look after it well
We did see lot of people with issues with LMTP, but we never saw these with lda.
I think it would be really nice if dovecot could use glusterfs for realy scalable postbox servers.
There was some nasty problems with that with cyrus, so much so, most uni's here who used that moved to a NAS backend and no one has whinged since :->
Am Dienstag, 13. Januar 2015, 21:40:34 schrieb Nick Edwards:
On 1/13/15, Michael Schwartzkopff ms@sys4.de wrote:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
With mdbox and maildir format I could reliably crash my mail storage just by
delivering mails to the both dovecots via LMTP to the same user. In maildir
dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox.
So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check?
Interesting, we use NFSv3 dovecot LDA with maildir, we have at present two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we added a few more over Christmas, so I think about 32 pop3 servers, but with only 4 imap servers incl webmail (IMAP is not heavily used here due to government spy laws) talking to NAS storage server backend, *we do not use director* at all and has never been an issue. Director IIRC solves the problem of IMAP inconsistency, but we never see advantage when we tested, no doubt it solves some fancy setup problem, but since director can not help with pop3, it was not worth the hassle. never had any problems with webmail either, load balancers seem to look after it well
Yes. NFS has its own locking. I wanted to use plain glusterfs client without the detour of NFS. Thanks for your hint.
Mit freundlichen Grüßen,
Michael Schwartzkopff
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 1/13/15, 6:02 AM, Michael Schwartzkopff wrote:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
With mdbox and maildir format I could reliably crash my mail storage just by
delivering mails to the both dovecots via LMTP to the same user. In maildir
dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox.
So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check? Interesting, we use NFSv3 dovecot LDA with maildir, we have at present two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we added a few more over Christmas, so I think about 32 pop3 servers, but with only 4 imap servers incl webmail (IMAP is not heavily used here due to government spy laws) talking to NAS storage server backend, *we do not use director* at all and has never been an issue. Director IIRC solves the problem of IMAP inconsistency, but we never see advantage when we tested, no doubt it solves some fancy setup
On 1/13/15, Michael Schwartzkopff ms@sys4.de wrote: problem, but since director can not help with pop3, it was not worth the hassle. never had any problems with webmail either, load balancers seem to look after it well Yes. NFS has its own locking. I wanted to use plain glusterfs client without
Am Dienstag, 13. Januar 2015, 21:40:34 schrieb Nick Edwards: the detour of NFS. Thanks for your hint.
Mit freundlichen Grüßen,
Michael Schwartzkopff
The last time we experimented with Glusterfs (two years ago) the native client was actually not able to maintain consistency as well as the NFS for a reason that I cannot remember anymore. We used maildir, and when using NFS we were able to deliver about a hundred thousand emails per hour and do a couple hundred thousand IMAP and POP3 retrievals per hour against a modest four node Gluster cluster with four Dovecot/Postfix servers (running in vmware).
- Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
Did you try sdbox, too?
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am Dienstag, 13. Januar 2015, 13:30:15 schrieb Marc Schiffbauer:
- Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
Did you try sdbox, too?
not yet. Will try as soon as I find some time.
Mit freundlichen Grüßen,
Michael Schwartzkopff
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am Dienstag, 13. Januar 2015, 13:30:15 schrieb Marc Schiffbauer:
- Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr:
Hi,
I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files.
With the help of the available documentation about NFS and fcntl locks I managed to find out the following:
With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless.
Did you try sdbox, too?
sdbox also gives an error: lmtp(30999, user1@example.net): Error: sdbox: /srv/mail/user1@example.net/mailboxes/INBOX/dbox-Mails/u.69 already exists, rebuilding index
Mit freundlichen Grüßen,
Michael Schwartzkopff
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
participants (4)
-
List
-
Marc Schiffbauer
-
Michael Schwartzkopff
-
Nick Edwards