[Dovecot] performance problems with {s,m}dbox
using rc3 and with the latest pigeonhole, I had problems with dovecot (deliver) keeping up with high volumes of E-mail. I tried sdbox, mdbox, and cyrdir all with similar (failed) result. using the default Maildir I had no problems. I did not try mbox. The problems seemed to happen when there were multiple deliver's running to the same folder I had defined in my sieve rules, either it would timeout and fail, or timeout and get delivered to my inbox, crash, or eventually halt because the queue would reach 20,000 messages which is the maximum with my postfix configuration.
Everything was fine when I was using Maildir, no errors and no sieve errors/timeouts. perhaps it is more of an issue with pigeonhole than sieve, but thought I would post my experience using dbox, since it sounded like it was a winner based on performance.
Jul 29 06:59:12 db1 dovecot: lda(XXXX): Error: Timeout while waiting for lock for transaction log file /home/XXXX/Maildir.mdbox/storage/dovecot.map .index.log Jul 29 06:59:12 db1 dovecot: lda(XXXX): Error: sieve: msgid=< 20100729135337.D7BBA3A03B3@db1.XXXXXXX.COM>: failed to store into mailbox 'XXXX': Internal error occurred. Refer to server log for more information. [2010-07-29 06:56:12] Jul 29 06:59:27 db1 dovecot: lda(XXXX): Error: sieve: execution of script /home/XXXX/.dovecot.sieve failed, but implicit keep was successful
Jul 29 07:13:53 db1 dovecot: lda(XXXX): Error: Timeout while waiting for lock for transaction log file /home/XXXX/Maildir.mdbox/storage/dovecot.map .index.log Jul 29 07:13:53 db1 dovecot: lda(XXXX): Error: sieve: msgid=< 20100729140329.9CBD73A0334@db1.XXXXXxxxxXXXX.com>: failed to store into mailbox 'XXXX': Internal error occurred. Refer to server log for more information. [2010-07-29 07:10:53] Jul 29 07:14:18 db1 dovecot: lda(XXXX): Error: sieve: execution of script /home/XXXX/.dovecot.sieve failed, but implicit keep was successful Jul 29 07:14:18 db1 dovecot: lda(XXXX): Panic: file dbox-file.c: line 76 (dbox_file_free): assertion failed: (file->refcount == 0) Jul 29 07:14:18 db1 dovecot: lda(XXXX): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x39bba) [0x7f40bf715bba] -> /usr/lib/dovecot/libdo vecot.so.0(i_syslog_fatal_handler+0x2b) [0x7f40bf715c4b] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f40bf6f31a6] -> /usr/lib/dovecot/libdov ecot-storage.so.0(+0x794ac) [0x7f40bfbd74ac] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_files_free+0x40) [0x7f40bfbd30b0] -> /usr/lib/doveco t/libdovecot-storage.so.0(+0x77f89) [0x7f40bfbd5f89] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_unref+0x7d) [0x7f40bfb87d4d] -> /usr/ lib/dovecot/libdovecot-storage.so.0(+0x23bf9) [0x7f40bfb81bf9] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_namespaces_deinit+0x23) [0x7f40bfb8 1cd3] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x2bd3d) [0x7f40bfb89d3d] -> /usr/libexec/dovecot/deliver(main+0x688) [0x402ea8] -> /lib/libc.so .6(__libc_start_main+0xfd) [0x7f40bef6bc4d] -> /usr/libexec/dovecot/deliver() [0x4025f9]
On Fri, Jul 30, 2010 at 7:12 PM, Jake Johnson jakej1978@gmail.com wrote:
perhaps it is more of an issue with pigeonhole than sieve, but thought I would post my experience using dbox, since it sounded like it was a winner based on performance.
... i meant to say it may be more of an issue with pigeonhole than dovecot (not sieve, doh!)
On 31.7.2010, at 3.12, Jake Johnson wrote:
using rc3 and with the latest pigeonhole, I had problems with dovecot (deliver) keeping up with high volumes of E-mail. I tried sdbox, mdbox, and cyrdir all with similar (failed) result.
What OS and filesystem?
Jul 29 06:59:12 db1 dovecot: lda(XXXX): Error: Timeout while waiting for lock for transaction log file /home/XXXX/Maildir.mdbox/storage/dovecot.map .index.log
This happens only after the file has been locked for 3 minutes. Pretty much the only reason why that could happen is if there was another process rebuilding the mdbox storage because it was detected to be corrupted. There should be errors logged about it too..
Jul 29 07:14:18 db1 dovecot: lda(XXXX): Panic: file dbox-file.c: line 76 (dbox_file_free): assertion failed: (file->refcount == 0)
This is a bug in any case.. I'll see if I can figure out why it happened.
Did sdbox log about any errors? Or cydir? Especially cydir is a really really simple mailbox format and there is no reason why it could be slower than maildir or do any kind of a long locking.
participants (2)
-
Jake Johnson
-
Timo Sirainen