[Dovecot] Performance problems while running doveadm purge
I have a rather large and active mdbox (28 GB, 3M mess, 1200 deliveries/day). I usually have no problems working with those mails, and there is some batch processing going on (via doveadm).
Every few weeks I try my luck running doveadm purge, and this a) crunches about 5GB (to be expected), b) takes rather long (ok), c) leads to long stretches of blocked mdbox, which is the problem.
I always turn off the batch processing, but deliveries take so long that they sometimes give up after timeout (they will come back later), and the IMAP clients get blocked for long minutes. Sometimes even the purge run aborts because of 3-minute-timeouts.
What could be going on here (2.1.10)?
Jost
| Jost.Krieger+sig@ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 |
On 12/7/2012 8:46 AM, Jost Krieger wrote:
I have a rather large and active mdbox (28 GB, 3M mess, 1200 deliveries/day). I usually have no problems working with those mails, and there is some batch processing going on (via doveadm).
Every few weeks I try my luck running doveadm purge, and this a) crunches about 5GB (to be expected), b) takes rather long (ok), c) leads to long stretches of blocked mdbox, which is the problem.
I always turn off the batch processing, but deliveries take so long that they sometimes give up after timeout (they will come back later), and the IMAP clients get blocked for long minutes. Sometimes even the purge run aborts because of 3-minute-timeouts.
What could be going on here (2.1.10)?
As always "dovecot -n" output would be helpful as well as log entries showing timeouts or errors. Knowing your mdbox_rotate_size setting would be helpful.
Also, are you purging one mailbox at a time, or all of them simultaneously. Have you looked at iostat? It may simply be that you're running out of random seek bandwidth on your storage device. I.e. the heads simply aren't fast enough to keep up with the block requests. Also, which OS and filesystem are you using? Again, "dovecot -n" would have already answered many of these questions.
-- Stan
On 7.12.2012, at 16.46, Jost Krieger wrote:
I have a rather large and active mdbox (28 GB, 3M mess, 1200 deliveries/day). I usually have no problems working with those mails, and there is some batch processing going on (via doveadm).
Every few weeks I try my luck running doveadm purge, and this a) crunches about 5GB (to be expected), b) takes rather long (ok), c) leads to long stretches of blocked mdbox, which is the problem.
I always turn off the batch processing, but deliveries take so long that they sometimes give up after timeout (they will come back later), and the IMAP clients get blocked for long minutes. Sometimes even the purge run aborts because of 3-minute-timeouts.
What could be going on here (2.1.10)?
The purging locks the mdbox only while it's writing to indexes, which should be a very short duration. The mdbox isn't locked while the m.* files are being read/written. What 3 minute timeout does purge complain about (exact error message)? Maybe get a strace output of the purge with "strace -tt -o log doveadm purge -u user@domain" and grep for SETLK in them to see how long the locks are being held.
participants (3)
-
Jost Krieger
-
Stan Hoeppner
-
Timo Sirainen