[Dovecot] 1.2.15 can't get zlib working
Following these instructions: http://wiki.dovecot.org/Plugins/Zlib
I enabled zlib and restarted Dovecot. I made a backup copy of a 68MB mbox file and moved it to a non mail directory. The mbox file is an existing TBird IMAP folder in my UNIX user mail directory. It is an archive of a defunct mailing list. It was a regular IMAP folder prior to attempting this. I was able to access all emails in the folder. It worked fine. I gzipped the original mbox file down to 14MB and removed the write flag with chmod. I launched TBird and received the following error in Activity Manager when accessing this IMAP folder:
The current operation on '1-Spam-l' did not succeed. The mail server for account stan@hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l
I deleted the Dovecot index files and the TBird 1-Spam-l.msf file which didn't help. Curiously, there are no entries in either dovecot.err or dovecot.log, giving not even the slightest hint as to where I should troubleshoot.
Kinda sucks when one follows instructions exactly, simple instructions at that, and it doesn't work, and generates zero log errors. A bit frustrating...
Any ideas?
/$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz
# 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_plugins: fts fts_squat zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster@hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10
-- Stan
On 22.11.2011, at 19.00, Stan Hoeppner wrote:
The current operation on '1-Spam-l' did not succeed. The mail server for account stan@hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l .. /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz
The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?)
On 11/22/2011 12:30 PM, Timo Sirainen wrote:
On 22.11.2011, at 19.00, Stan Hoeppner wrote:
The current operation on '1-Spam-l' did not succeed. The mail server for account stan@hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l .. /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz
The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?)
Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe.
This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only"
Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes:
2011-11-22 14:59:23 IMAP(stan): Error: file_dotlock_open(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log) failed: Permission denied (euid=1000(stan) egid=1000(stan) access(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log, 4) failed: No such file or directory)
Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only.
It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages. As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs.
Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though....
Once again:
# 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_process_size: 320 mail_plugins: zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster@hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10
Thanks.
-- Stan
On 23.11.2011, at 0.25, Stan Hoeppner wrote:
On 11/22/2011 12:30 PM, Timo Sirainen wrote:
On 22.11.2011, at 19.00, Stan Hoeppner wrote:
The current operation on '1-Spam-l' did not succeed. The mail server for account stan@hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l .. /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz
The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?)
Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe.
I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :)
This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only"
Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes:
I've fixed this in some version. I guess in v2.0.
Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only.
Well, I don't really care that much about v1.x anymore.
It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages.
Yeah, it's uncompressing the entire file until it finds the message you're opening.
As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs.
Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though....
mail_plugins: zlib
Doesn't look like fts, fts_squat is enabled?
On 11/22/2011 5:07 PM, Timo Sirainen wrote:
On 23.11.2011, at 0.25, Stan Hoeppner wrote:
On 11/22/2011 12:30 PM, Timo Sirainen wrote:
On 22.11.2011, at 19.00, Stan Hoeppner wrote:
The current operation on '1-Spam-l' did not succeed. The mail server for account stan@hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l .. /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz
The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?)
Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe.
I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :)
Out of curiosity, what (or who) prompted the development of the compressed mbox feature? Or was it that you wanted to do it for maildir, and then figured you should for mbox as well? The implementation seems to work ok. The instructions just seem a bit...thin. :)
This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only"
Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes:
I've fixed this in some version. I guess in v2.0.
Yeah, I'm waiting for Debian to get a backport of 2.0.x. For many reasons the only thing I'm comfortable installing from source is the Linux kernel.
Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only.
Well, I don't really care that much about v1.x anymore.
Understandable.
It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages.
Yeah, it's uncompressing the entire file until it finds the message you're opening.
It's pretty damn fast at it. I haven't seen anything more than a couple of seconds lag while randomly accessing mail all over the folder. The original gzip of the file took >45 seconds.
As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs.
Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though....
mail_plugins: zlib
Doesn't look like fts, fts_squat is enabled?
Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins had to be declared in a single line directive. I did this as part of my troubleshooting when zlib wasn't working, thinking putting it on it's own line may help--not.
-- Stan
On Tue, 2011-11-22 at 18:56 -0600, Stan Hoeppner wrote:
Out of curiosity, what (or who) prompted the development of the compressed mbox feature? Or was it that you wanted to do it for maildir, and then figured you should for mbox as well? The implementation seems to work ok. The instructions just seem a bit...thin. :)
I think it was one of my friends saying he can't start using Dovecot until it supports compressed mboxes.
mail_plugins: zlib
Doesn't look like fts, fts_squat is enabled?
Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins had to be declared in a single line directive. I did this as part of my troubleshooting when zlib wasn't working, thinking putting it on it's own line may help--not.
With v2.0 you can do:
mail_plugins = $mail_plugins zlib mail_plugins = $mail_plugins fts fts_squat
participants (2)
-
Stan Hoeppner
-
Timo Sirainen