[Dovecot] performance problems with {s,m}dbox

[Jake Johnson]

using rc3 and with the latest pigeonhole, I had problems with dovecot
(deliver) keeping up with high volumes of E-mail. I tried sdbox, mdbox, and
cyrdir all with similar (failed) result. using the
default Maildir I had no problems. I did not try mbox. The problems seemed
to happen when there were multiple deliver's running to the same folder I
had defined in my sieve rules, either it would timeout and fail, or timeout
and get delivered to my inbox, crash,  or eventually halt because the queue
would reach 20,000 messages which is the maximum with my postfix
configuration.

Everything was fine when I was using Maildir, no errors and no sieve
errors/timeouts. perhaps it is more of an issue with pigeonhole than sieve,
but thought I would post my experience using dbox, since it sounded like it
was a winner based on performance.


Jul 29 06:59:12 db1 dovecot: lda(XXXX): Error: Timeout while waiting for
lock for transaction log file /home/XXXX/Maildir.mdbox/storage/dovecot.map
.index.log
Jul 29 06:59:12 db1 dovecot: lda(XXXX): Error: sieve: msgid=<
20100729135337.D7BBA3A03B3 at db1.XXXXXXX.COM>: failed to store into mailbox
'XXXX': Internal error occurred. Refer to server log for more information.
[2010-07-29 06:56:12]
Jul 29 06:59:27 db1 dovecot: lda(XXXX): Error: sieve: execution of script
/home/XXXX/.dovecot.sieve failed, but implicit keep was successful

Jul 29 07:13:53 db1 dovecot: lda(XXXX): Error: Timeout while waiting for
lock for transaction log file /home/XXXX/Maildir.mdbox/storage/dovecot.map
.index.log
Jul 29 07:13:53 db1 dovecot: lda(XXXX): Error: sieve: msgid=<
20100729140329.9CBD73A0334 at db1.XXXXXxxxxXXXX.com>: failed to store into
mailbox 'XXXX': Internal error occurred. Refer to server log for more
information. [2010-07-29 07:10:53]
Jul 29 07:14:18 db1 dovecot: lda(XXXX): Error: sieve: execution of script
/home/XXXX/.dovecot.sieve failed, but implicit keep was successful
Jul 29 07:14:18 db1 dovecot: lda(XXXX): Panic: file dbox-file.c: line 76
(dbox_file_free): assertion failed: (file->refcount == 0)
Jul 29 07:14:18 db1 dovecot: lda(XXXX): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x39bba) [0x7f40bf715bba] ->
/usr/lib/dovecot/libdo
vecot.so.0(i_syslog_fatal_handler+0x2b) [0x7f40bf715c4b] ->
/usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f40bf6f31a6] ->
/usr/lib/dovecot/libdov
ecot-storage.so.0(+0x794ac) [0x7f40bfbd74ac] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mdbox_files_free+0x40)
[0x7f40bfbd30b0] -> /usr/lib/doveco
t/libdovecot-storage.so.0(+0x77f89) [0x7f40bfbd5f89] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_unref+0x7d)
[0x7f40bfb87d4d] -> /usr/
lib/dovecot/libdovecot-storage.so.0(+0x23bf9) [0x7f40bfb81bf9] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_namespaces_deinit+0x23)
[0x7f40bfb8
1cd3] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x2bd3d) [0x7f40bfb89d3d]
-> /usr/libexec/dovecot/deliver(main+0x688) [0x402ea8] -> /lib/libc.so
.6(__libc_start_main+0xfd) [0x7f40bef6bc4d] ->
/usr/libexec/dovecot/deliver() [0x4025f9]