Hello Timo,
I've successfully setup xz compression for my Dovecot installation using the version 2.2.12 from Debian unstable. Together with the activation of the "Single Instance Attachment Storage", this reduced the mail storage utilization for 20 mail accounts by 74%.
*My question:* (For this test just one user was logged in. OS = Debian Wheezy, AMD64) After conversion of my mail storage, IMAP read by Thunderbird worked perfectly. But IMAP write and LMTP write was not possible.* *xz compression seems to need a higher vsz_limit than its default value of 256M without compression. Before the conversion, 256M was enough with the plugins "**acl fts fts_solr" and " imap_zlib". 512M was still not enough for xz compression, 1024M works OK but I'm not sure since this is a relative large increase. My new setting: /etc/dovecot/conf.d/10-master.conf: service imap { vsz_limit=1024M } The absolute minimum to successfully send an empty mail is 710MB (700MB still produces out of memory)! _*Is this normal behaviour? What is a recommended value for "service imap {vsz_limit = }"?*_ I recommend to add this "_vsz_limit"_-increase to the wiki (http://wiki2.dovecot.org/Plugins/Zlib) to make others life easier. Error log below.
- **My config & how I converted:*
/etc/dovecot/conf.d/10-mail.conf: mail_plugins = *zlib *acl fts fts_solr
/etc/dovecot/conf.d/90-plugin.conf: # Enable these only if you want compression while saving: zlib_save_level = 9 # 1..9 zlib_save = xz
For "Single Instance Attachment Storage": /etc/dovecot/conf.d/10-mail.conf: # Directory root where to store mail attachments. Disabled, if empty. mail_attachment_dir = /srv/vmail/mail_attachments
For compressed IMAP access ( to reduce the bandwidth usage of IMAP ) (has nothing to do with storage space): /etc/dovecot/conf.d/20-imap.conf: protocol imap { mail_plugins = $mail_plugins *imap_zlib* }
Previous storage format was standard mdbox Format. To convert a single user, I used dsync from the command line: dsync -u user@domain.de mirror mdbox:/srv/converted_vmail/user@domain.de/
After conversion (~2 hours per account), I moved the original mail folder of the user to a backup location and replaced it by the converted folders from "/srv/converted_vmail/user@domain.de/". (Pay attention that the directory structure matches!, I had to adjust ist.)
*My Problem:* *After conversion IMAP read by Thunderbird worked perfectly**. But IMAP write and **LMTP write was not possible**.** **/var/log/dovecot.log showed that vsz_limit**was to restrictive:*
Log: Apr 03 14:13:14 imap(user@domain.de): Fatal: lzma: Out of memory Apr 03 14:13:14 imap(user@domain.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x674f0) [0x7f8538fd54f0] -> /usr/lib/dovecot/libdovecot.so.0(+0x675ce) [0x7f8538fd55ce] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f8538f90678] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(o_stream_create_lzma+0xeb) [0x7f8538393ecb] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x2c14) [0x7f8538390c14] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x43) [0x7f85392bdc23] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539788824] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539788a6e] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f853979390c] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f85397928ff] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f85397929b5] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539792cd5] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539793065] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7f8538fe55c7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f8538fe6357] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8538fe5168] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8538f95723] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539787324] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f8538be6b45] -> dovecot/imap user@domain.de 192.168.1.42 append [0x7f8539787490] Apr 03 14:13:14 imap-login: Debug: SSL alert: close notify [192.168.1.42] Apr 03 14:13:14 imap(user@domain.de): Fatal: master: service(imap): child 23995 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)
==============================================