[Dovecot] Hints for a NFS-Setup
Hi everyone,
as I have often trouble with OCFS2 I want to switch to NFS but I'm not sure how to rebuild my cluster with regard to locking and indexing problems.
By now my I have a four server configuration (there are another 2 servers for outgoing mail but they can be ignored):
MTA(MX10) --(lmtp/socket)--> local dovecot --> iSCSI-LUN with OCFS2 MTA(MX10) --(lmtp/socket)--> local dovecot --> iSCSI-LUN with OCFS2 IMAP-User <--(imap)--> IMAP-Server1 (local dovecot) <--> iSCSI-LUN/OCFS2 IMAP-User <--(imap)--> IMAP-Server2 (local dovecot) <--> iSCSI-LUN/OCFS2
As far as I understood I will get poor performance if I'd just switch from OCFS2 to NFS (while keeping this configuration) with 4 hosts accessing the NFS-share and the index files on it and it is recommended to assign users to a specific host (http://wiki2.dovecot.org/NFS).
I'm uncertain what's the meaning of "user" in this context. Is it an IMAP-User or every incoming mail?
An IMAP-User assigned to a specific IMAP-Server is ok for me and I could store and profit from local index files. However, I want my incoming mailservers to be equally receiving mails. Both should accept mails for every mailbox but in this case I won't have local indexes.
I would appreciate any hints.
Patrick
Hi everyone,
as I have often trouble with OCFS2 I want to switch to NFS but I'm not sure how to rebuild my cluster with regard to locking and indexing problems.
By now my I have a four server configuration (there are another 2 servers for outgoing mail but they can be ignored):
MTA(MX10) --(lmtp/socket)--> local dovecot --> iSCSI-LUN with OCFS2 MTA(MX10) --(lmtp/socket)--> local dovecot --> iSCSI-LUN with OCFS2 IMAP-User <--(imap)--> IMAP-Server1 (local dovecot) <--> iSCSI-LUN/OCFS2 IMAP-User <--(imap)--> IMAP-Server2 (local dovecot) <--> iSCSI-LUN/OCFS2
As far as I understood I will get poor performance if I'd just switch from OCFS2 to NFS (while keeping this configuration) with 4 hosts accessing the NFS-share and the index files on it and it is recommended to assign users to a specific host (http://wiki2.dovecot.org/NFS).
I'm uncertain what's the meaning of "user" in this context. Is it an IMAP-User or every incoming mail?
An IMAP-User assigned to a specific IMAP-Server is ok for me and I could store and profit from local index files. However, I want my incoming mailservers to be equally receiving mails. Both should accept mails for every mailbox but in this case I won't have local indexes.
I would appreciate any hints.
Patrick If you've got a load balancer, it should be fairly easy to do simple IP stickiness, with a long enough timeout, most IMAP and POP3 users will stay on the same server.. I'm sure there is some load balancing software
Il 30/03/2012 13.31, Patrick Westenberg ha scritto: that's also L7 aware, and could direct by username (though you'd probably have to have the LB terminate the SSL, not the server behind it).
SMTP wouldn't have to be balanced in the same way, you could just use round robin in that case..
I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
On 30.3.2012, at 15.38, Nick Warr wrote:
As far as I understood I will get poor performance if I'd just switch from OCFS2 to NFS (while keeping this configuration) with 4 hosts accessing the NFS-share and the index files on it and it is recommended to assign users to a specific host (http://wiki2.dovecot.org/NFS).
I'm uncertain what's the meaning of "user" in this context. Is it an IMAP-User or every incoming mail?
IMAP, POP3, LDA so everything.
If you've got a load balancer, it should be fairly easy to do simple IP stickiness, with a long enough timeout, most IMAP and POP3 users will stay on the same server.. I'm sure there is some load balancing software that's also L7 aware, and could direct by username (though you'd probably have to have the LB terminate the SSL, not the server behind it).
IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA.
I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
Anything else except Dovecot director will cause corruption with NFS. Several really large sites already use director in production.
On 03/30/12 07:35, Timo Sirainen wrote:
IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA.
Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail.
-K
On 18.4.2012, at 1.29, Kelsey Cummings wrote:
On 03/30/12 07:35, Timo Sirainen wrote:
IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA.
Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail.
Right. If indexes or dovecot-uidlist isn't updated then it doesn't matter.
Nick Warr schrieb:
I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
Yes, with director it should be something like that:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS
MTA --(lmtp)--/ \--(lmtp)--> backend2 --/
IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS
IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/
So what happens if user1@example.tld receives a mail?
The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally?
Then, user1@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too?
Patrick Westenberg schrieb:
Nick Warr schrieb:
I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
Yes, with director it should be something like that:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/So what happens if user1@example.tld receives a mail?
The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally?
Then, user1@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too?
Timo? This post is from March, right before you introduced the stones :D
On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote:
Nick Warr schrieb:
I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
Yes, with director it should be something like that:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/
The director above must be the same one for both IMAP and MTA.
So what happens if user1@example.tld receives a mail?
The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally?
Then, user1@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too?
Yes. Director process actually doesn't know anything about mail protocols, it simply does username -> backend mapping. The regular Dovecot IMAP/POP3/LMTP/doveadm proxying code handles the rest.
Timo Sirainen schrieb:
On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/The director above must be the same one for both IMAP and MTA.
I know and it is meant to be the same in this drawing.
However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability?
Patrick
On 4.4.2012, at 14.38, Patrick Westenberg wrote:
Timo Sirainen schrieb:
On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/The director above must be the same one for both IMAP and MTA.
I know and it is meant to be the same in this drawing.
However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability?
Another director. They're meant to connect to each others and do LB/HA.
Timo Sirainen schrieb:
Another director. They're meant to connect to each others and do LB/HA.
But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down?
Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
On 4.4.2012, at 15.02, Patrick Westenberg wrote:
Timo Sirainen schrieb:
Another director. They're meant to connect to each others and do LB/HA.
But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down?
Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
I don't know if Postfix supports that. Typically people use a load balancer (cluster).
On Wed, 4 Apr 2012 15:06:29 +0300 Timo Sirainen articulated:
On 4.4.2012, at 15.02, Patrick Westenberg wrote:
Timo Sirainen schrieb:
Another director. They're meant to connect to each others and do LB/HA.
But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down?
Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
I don't know if Postfix supports that. Typically people use a load balancer (cluster).
Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action.
-- Jerry ♔
Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.
On 4.4.2012, at 15.29, Jerry wrote:
Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
I don't know if Postfix supports that. Typically people use a load balancer (cluster).
Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action.
I have a guess though, or alternatively a suggestion if it already doesn't work like that:
lmtp:tcp:lmtp.example.com:24
where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places.
Quoting Timo Sirainen tss@iki.fi:
On 4.4.2012, at 15.29, Jerry wrote:
Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
I don't know if Postfix supports that. Typically people use a load balancer (cluster).
Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action.
I have a guess though, or alternatively a suggestion if it already
doesn't work like that:lmtp:tcp:lmtp.example.com:24
where lmtp.example.com expands to your two IP addresses. DNS server
handles load balancing by returning IPs in round robin and Postfix
handles fallbacking to the second IP if the first one doesn't work.
Dovecot has similar behavior in several places.
Personally I use MX dns entries, lowest mx entry is localhost, then others.
Timo Sirainen schrieb:
where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places.
As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn.
On 4.4.2012, at 16.04, Patrick Westenberg wrote:
Timo Sirainen schrieb:
where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places.
As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn.
MX lookups are intended for SMTP, I very much doubt LMTP uses them?
Quoting Timo Sirainen tss@iki.fi:
On 4.4.2012, at 16.04, Patrick Westenberg wrote:
Timo Sirainen schrieb:
where lmtp.example.com expands to your two IP addresses. DNS
server handles load balancing by returning IPs in round robin and
Postfix handles fallbacking to the second IP if the first one
doesn't work. Dovecot has similar behavior in several places.As far as I understand, lmtp.example.com with out records would
result in a MX-lookup. If postfix behaves for lmtp like it does for
smtp, it will try all listed MX (my directors) in turn.MX lookups are intended for SMTP, I very much doubt LMTP uses them?
Yes, but postfix lmtp is the same binary as smtp, at least currently.
On 4.4.2012, at 16.57, Patrick Domack wrote:
Quoting Timo Sirainen tss@iki.fi:
On 4.4.2012, at 16.04, Patrick Westenberg wrote:
Timo Sirainen schrieb:
where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places.
As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn.
MX lookups are intended for SMTP, I very much doubt LMTP uses them?
Yes, but postfix lmtp is the same binary as smtp, at least currently.
That's about how to handle incoming lmtp vs. smtp connections. The outgoing smtp vs. lmtp connections are always handled by the same binary anyway.
On Wed, Apr 04, 2012 at 02:02:02PM +0200, Patrick Westenberg wrote:
Timo Sirainen schrieb:
Another director. They're meant to connect to each others and do LB/HA.
But what about my MTAs? How can I tell my two postfix servers that there
are two directors and it should/can use the other one if the first is down?
We use hardware loadbalancers in front of the directors. If a director goes down, the hardware loadbalancers sees that. We also use the poolmon script to tell the directors if a certain imap server is missing.
Cor
Hi again,
I tried to setup a test invironemnt like this:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS
MTA --(lmtp)--/ \--(lmtp)--> backend2 --/
IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS
IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/
but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA?
Is anyone using a similar setup and could post his configuration for the different hosts?
Happy Easter! Patrick
On 6.4.2012, at 16.58, Patrick Westenberg wrote:
Hi again,
I tried to setup a test invironemnt like this:
MTA --(lmtp)--\ /--(lmtp)--> backend1 --
-- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/IMAP-User --> frontend1 --\ /--(imap)--> backend1 --
-- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA?
The cleanest way to run director is to have 2 or more servers running only director itself. If you want to have less servers, it's also possible to place a Dovecot director configuration to any other servers as well, but that's conceptually more complex.
For MTA you'd simply tell its LMTP client to connect to director servers, which could be one of:
a) Load balancer's IP address
b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one.
participants (7)
-
Cor Bosman
-
Jerry
-
Kelsey Cummings
-
Nick Warr
-
Patrick Domack
-
Patrick Westenberg
-
Timo Sirainen