[Dovecot] Dovecot + OCFS2
Hello, We are running Red Hat Enterprise Linux AS release 4 (Nahant Update 4) with OCFS2 for our mail servers. There are three nodes in the cluster with shared storage and OCFS2 filesystem on it. The filesystem is used for mail storage(using Maildir layout) and is accessed by smtpd, pop3 and imap processes(DOVECOT). The system works fine but when there are mailing lists the load increases dramatically. It only occurs when there are mailing lists. We have tested it in lab with good results and now in production we are having some load problems caused by dovecot´s deliver process.
In lab we had sent lots of mails but always to a few accounts. It was OK. Now there are sent less mails but there are more accounts. The load increases until it reaches about 100 and only restart fixes the problem. The umounting of the OCFS2 filesystem is not possible even when all process accessing it are killed. Any suggestions? TIA
Info:
Three nodes. OS: Red Hat Enterprise Linux AS release 4 (Nahant Update 4) Mailbox format: Maildir SMTP daemon: Dovecot deliver + Sendmail OCFS2 version: 1.2.5-6 Kernel Version: 2.6.9-42.0.10.ELsmp Pop3 and imap daemon: Dovecot 1.0.3
--
Jon Blázquez Diez jblazquez@unav.es Servicios Informáticos https://www.unav.es/SI/ Universidad de Navarra https://www.unav.es
Jon Blazquez wrote:
The system works fine but when there are mailing lists the load increases dramatically.
Any suggestions?
First thing that I would do is to reduce number of concurrent deliver processes. I have no clue how this is done with Sendmail but with Postfix one could adjust deliver process "maxproc" value in master.cf.
Tomi
Thank you for your response, With sendmail we have achieved to limit the load :
define(confMAX_DAEMON_CHILDREN', 15')
define(confCONNECTION_RATE_THROTTLE', 2')dnl
define(confQUEUE_LA', 5')dnl
define(confREFUSE_LA', 4')dnl
With this configuration the LOAD only reaches 20 ;-) but it’s not the solution. The cluster is formed by 3 nodes. The load balancer sends mails to a node, when it's full (LOAD 4-5) it denies receiving more mails so the load balancer sends them to the following node in the cluster and so on... In the first node the load continues increasing until it reaches roughly 20, then it falls down.
This is a temporary solution but we don’t know where exactly the problem is: dovecot’s deliver, ocfs2…. Any help would be appreciated,
Jon
Tomi Hakala escribió:
Jon Blazquez wrote:
The system works fine but when there are mailing lists the load increases dramatically.
Any suggestions?
First thing that I would do is to reduce number of concurrent deliver processes. I have no clue how this is done with Sendmail but with Postfix one could adjust deliver process "maxproc" value in master.cf.
Tomi
--
Jon Blázquez Diez jblazquez@unav.es Servicios Informáticos https://www.unav.es/SI/ Universidad de Navarra https://www.unav.es
Hi,
The load reaches 20, which is not that far from your MAX_DAEMON_CHILDREN directive in sendmail. You should try to find why the delivery processes are so slow to accomplish. Maybe it's a I/O backend problem. (I have experienced the same kind of problems with drbd writing, and slow hard disks)
For example, during a mailling delivery, check the wait (wa) state of your cpu with the command 'vmstat 1'. A high value reflect the cpu time spent in waiting for I/O.
Hope this helps, sam
Jon Blazquez a écrit :
Thank you for your response, With sendmail we have achieved to limit the load :
define(
confMAX_DAEMON_CHILDREN',15') define(confCONNECTION_RATE_THROTTLE',2')dnl define(confQUEUE_LA',5')dnl define(confREFUSE_LA',4')dnlWith this configuration the LOAD only reaches 20 ;-) but it’s not the solution. The cluster is formed by 3 nodes. The load balancer sends mails to a node, when it's full (LOAD 4-5) it denies receiving more mails so the load balancer sends them to the following node in the cluster and so on... In the first node the load continues increasing until it reaches roughly 20, then it falls down.
This is a temporary solution but we don’t know where exactly the problem is: dovecot’s deliver, ocfs2…. Any help would be appreciated,
Jon
Tomi Hakala escribió:
Jon Blazquez wrote:
The system works fine but when there are mailing lists the load increases dramatically.
Any suggestions?
First thing that I would do is to reduce number of concurrent deliver processes. I have no clue how this is done with Sendmail but with Postfix one could adjust deliver process "maxproc" value in master.cf.
Tomi
On Tue, 2007-10-02 at 17:46 +0200, Jon Blazquez wrote:
Hello, We are running Red Hat Enterprise Linux AS release 4 (Nahant Update 4) with OCFS2 for our mail servers. There are three nodes in the cluster with shared storage and OCFS2 filesystem on it. The filesystem is used for mail storage(using Maildir layout) and is accessed by smtpd, pop3 and imap processes(DOVECOT). The system works fine but when there are mailing lists the load increases dramatically.
So you mean everything works ok as long as messages are saved into only a few mailboxes, but once there comes a burst of messages to lots of different mailboxes the load increases?
Is it just one node doing the writes or all 3 at the same time? If all 3, does it work better if only 1 is doing it?
How long are the deliver processes running? If they're there long enough to be straced, check what syscalls are taking the longest.
participants (4)
-
Jon Blazquez
-
Samuel HAMEAU
-
Timo Sirainen
-
Tomi Hakala