Hello,
I have unconventional load profile for my email server and I would like to ask if someone can suggest optimizations in my case.
I have only one mailbox on the server for collecting emails from variety of external mail servers. I use Postfix for collecting emails and Dovecot to provide IMAP for downloading emails from this mailbox. Currently Postfix configured to use LMTP for transferring emails to Dovecot. Dovecot mail storage format is mdbox.
Everything was in order (no incoming and active queues in Postfix) when daily incoming email count was up to 1 million emails.
But now I have to process up to 2 million incoming emails and I faced bottleneck of 10 emails per second for saving emails from Postfix active queue to mdbox using LMTP. Currently Postfix receives about 20 emails per second, incoming and active queues in Postfix are growing and I have to find the way to speed up transferring process from Postfix to mdbox.
This setup is running on the virtual machine with 8 CPU and 8 GB RAM.
CPU loaded only by dovecot/lmtp process but not more than 20% per core.
I/O wait is 2-3%. Disk load is about 200 wMB/sec and 300 rKB/sec (and this sounds quite a bit for our storage capacity) .
Postfix has destination concurrency limit 20 for default and virtual transport (I’m using virtual transport due to virtual domains configuration).
Strace showed up that dovecot/lmtp processes have some delays only on waiting for write lock on truncation log file dovecot.index.log. I have temporary moved all index files including truncation log to tmpfs, but this did not help.
The only thing that helped me to increase speed up to 20 emails per second is removing almost all emails from the mailbox (with empty Inbox I got even 30 emails per second). But I have to keep all incoming emails in the mailbox at least for one week so this way is unacceptable.
So I would like to ask for suggestions, because I can’t find the root cause of this bottleneck.
Dovecot version 2.2.31
Postfix version 2.10.1
------------------------------------------------Dovecot conf ------------------------------------------------
mail_location = mdbox:~/mdbox
mdbox_rotate_interval = 1 days
mdbox_rotate_size = 200MB
mail_plugins = " zlib”
plugin {
zlib_save = gz
zlib_save_level = 6
}
userdb {
args = /etc/dovecot/passwd
default_fields = uid=vmail gid=vmail home=/opt/vmail/%u
driver = passwd-file
}
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
service lmtp {
process_min_avail = 8
service_count = 0
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
vsz_limit = 0
}
------------------------------------------------------------------------------------------------
--------------------------------------Postfix conf-------------------------------------------
virtual_alias_maps = hash:/etc/postfix/virtual
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_transport = lmtp:unix:private/dovecot-lmtp
default_destination_concurrency_limit = 20
virtual_destination_concurrency_limit = 20
------------------------------------------------------------------------------------------------