Re: [Dovecot] zlib config questions
You only need to add the zlib plugin to mail_plugins once.
As far as I know, there's no indication in the logs that mail is being compressed, and the filename isn't modified to indicate that it's compressed, but, of course, the files are zlib data on disk.
One gotcha if you're using maildir is, if you're planning to compress old mail, make sure that the S= and W= sizes in the filenames are correct, otherwise dovecot errors and closes the client session, and it can't repair the filename on it's own.
On Dec 6, 2013, at 12:51 AM, dovecot-request@dovecot.org wrote:
Message: 6 Date: Thu, 5 Dec 2013 13:55:40 -0800 From: Terry Barnum terry@dop.com To: Dovecot Mailing List dovecot@dovecot.org Subject: [Dovecot] zlib config questions Message-ID: EF7F46D0-DE03-4B11-85ED-D669DE78D931@dop.com Content-Type: text/plain; charset=us-ascii
After nearly running out of space I swapped in larger disks and then saw the recent threads about zlib compression. Unfortunately I'm still confused after reading http://wiki2.dovecot.org/Plugins/Zlib.
In order to compress new email being stored do I only need to change
10-mail.conf to this: mail_plugins = $mail_plugins zlib
and 90-plugin.conf to this: plugin { zlib_save_level = 6 zlib_save = gz ...
Or do I need to instead (or also?) add it to the list of plugins in 20-imap.conf?
A simple 'sudo doveadm reload' to enable?
Once enabled, is there an indication in the logs that compression is taking place?
Do new mail files have a suffix like .Z to indicate they've been compressed?
Any gotchas to be aware of?
I'm running macports dovecot 2.2.5.
Thanks for any help.
-Terry
Terry Barnum digital OutPost http://www.dop.com
On 2013-12-06 10:55 AM, Richard Platel rplatel@tucows.com wrote:
One gotcha if you're using maildir is, if you're planning to compress old mail, make sure that the S= and W= sizes in the filenames are correct,
And how do you make sure of this? Is there a script? Or do you just pray?
--
Best regards,
*/Charles/*
Am 06.12.2013 17:01, schrieb Charles Marcus:
On 2013-12-06 10:55 AM, Richard Platel rplatel@tucows.com wrote:
One gotcha if you're using maildir is, if you're planning to compress old mail, make sure that the S= and W= sizes in the filenames are correct,
And how do you make sure of this? Is there a script? Or do you just pray?
this is only if you manual convert the whole store i.e with dsync to get more free space , if you simple enable the plugin you have not be afraid cause only new mail will may get compressed but however zlib and maildir had bugs in some dove releases or version updates, see list archive how people fixed it
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On Dec 6, 2013, at 8:23 AM, Robert Schetterer rs@sys4.de wrote:
Am 06.12.2013 17:01, schrieb Charles Marcus:
On 2013-12-06 10:55 AM, Richard Platel rplatel@tucows.com wrote:
One gotcha if you're using maildir is, if you're planning to compress old mail, make sure that the S= and W= sizes in the filenames are correct,
And how do you make sure of this? Is there a script? Or do you just pray?
this is only if you manual convert the whole store i.e with dsync to get more free space , if you simple enable the plugin you have not be afraid cause only new mail will may get compressed but however zlib and maildir had bugs in some dove releases or version updates, see list archive how people fixed it
Thank you Richard, Charles and Robert for your insight and suggestions.
I'm curious how many people here use the zlib plugin. Is it common or is drive space cheap enough that it's not worth it to run the risk of potential problems?
I am using Maildir and did read the threads from last year of problems with the manually compressed mail S and W values being incorrect as well as other filename problems. Have these been fixed in dovecot >= v2.2.5?
Once you enable zlib, does this mean you must use it from that point on? Once email is compressed, if you disable the plugin does dovecot lose the ability to access those compressed emails?
Thanks, -Terry
Terry Barnum digital OutPost http://www.dop.com
Am 06.12.2013 20:01, schrieb Terry Barnum:
Once you enable zlib, does this mean you must use it from that point on? Once email is compressed, if you disable the plugin does dovecot lose the ability to access those compressed emails?
you may disable the plugin anytime, but you then have to decompress all mails, which should easy work with dsync
http://wiki2.dovecot.org/Tools/Dsync
specially look at -o options
sorry german only
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 06.12.2013 20:18, schrieb Robert Schetterer:
Am 06.12.2013 20:01, schrieb Terry Barnum:
Once you enable zlib, does this mean you must use it from that point on? Once email is compressed, if you disable the plugin does dovecot lose the ability to access those compressed emails?
you may disable the plugin anytime, but you then have to decompress all mails, which should easy work with dsync
http://wiki2.dovecot.org/Tools/Dsync
specially look at -o options
forgot this
http://sys4.de/de/blog/2013/07/20/fallstricke-mit-acl-und-quota-bei-backup-m...
sorry german only
Best Regards MfG Robert Schetterer
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Terry Barnum terry@dop.com wrote:
I'm curious how many people here use the zlib plugin. Is it common or is drive space cheap enough that it's not worth it to run the risk of potential problems?
HDD space may be cheap, but is also slow, therefor IOPS are very expensive.
I compress the mail using zlib, because even a very slight compression rate as 3 reduces the IOPS by more than 50% in my measurements and in my environment. As long as you don't plan to use a Raspberry Pi as your Dovecot server, todays CPUs are easily able to cope.
And, please anybody correct me if I am wrong, if you use imap_zlib then the IMAP connection is compressed as well which may allow Dovecot to skip the decompressing state completely and just deliver the compressed stream directly. (Althought I don't know anymore where I read this and may be totally wrong and bogus. Please confirm with anyone knowing the code. Thanks.)
Also reducing the needed space is a welcome bonus for me.
Grüße, Sven,.
-- Sigmentation fault. Core dumped.
On 6.12.2013, at 21.28, Sven Hartge sven@svenhartge.de wrote:
And, please anybody correct me if I am wrong, if you use imap_zlib then the IMAP connection is compressed as well which may allow Dovecot to skip the decompressing state completely and just deliver the compressed stream directly. (Althought I don't know anymore where I read this and may be totally wrong and bogus. Please confirm with anyone knowing the code. Thanks.)
No, that’s not done. I don’t think it would be a good idea either.
participants (6)
-
Charles Marcus
-
Richard Platel
-
Robert Schetterer
-
Sven Hartge
-
Terry Barnum
-
Timo Sirainen